44 lines
1.3 KiB
Markdown
44 lines
1.3 KiB
Markdown
# ai.noris.de Endpoint Reference (Session 2026-06-18)
|
||
|
||
## Endpoint Details
|
||
|
||
- **URL**: `https://ai.noris.de/v1`
|
||
- **Type**: OpenAI-compatible Chat Completions
|
||
- **Auth**: Bearer token via `Authorization` header
|
||
- **API Key**: `sk-bf-8779d73c-6a51-49d6-a22f-00a0290ca6a7`
|
||
- **Models endpoint**: `GET /v1/models` → returns 4 models
|
||
|
||
## Discovered Models (2026-06-18)
|
||
|
||
| Model ID | Size | Notes |
|
||
|----------|------|-------|
|
||
| `vllm/gemma-4-31b-it` | 31B | Google Gemma 4 |
|
||
| `vllm/gpt-oss-120b` | 120B | GPT-OSS, largest model |
|
||
| `vllm/moonshotai/kimi-k2.6` | ~12B | Kimi K2.6, strongest generalist |
|
||
| `vllm/qwen3.6-27b-nvfp4` | 27B | Qwen 3.6 with NVFP4 quantization |
|
||
|
||
All models served via vLLM.
|
||
|
||
## Benchmark Results (Single Request, Synthetic 512/128)
|
||
|
||
### gemma-4-31b-it
|
||
- **TTFT**: 361.7 ms
|
||
- **Throughput**: 13.78 tok/s
|
||
- **ISL**: 512, **OSL**: 128
|
||
|
||
### gpt-oss-120b
|
||
- Slightly higher latency expected due to size
|
||
|
||
### moonshotai/kimi-k2.6
|
||
- Generally strongest quality
|
||
|
||
### qwen3.6-27b-nvfp4
|
||
- NVFP4 quantization is aggressive – monitor quality vs. speed trade-off
|
||
|
||
## Endpoint Compatibility Notes
|
||
|
||
- **Prometheus metrics**: `/v1/metrics` returns HTML (web UI), not Prometheus exposition format
|
||
- `--no-server-metrics` recommended when benchmarking
|
||
- **GPU telemetry**: No DCGM endpoints exposed (localhost:9400/9401 unreachable)
|
||
- `--no-gpu-telemetry` recommended
|