# AIPerf 0.10.0+ Kurzreferenz

## Installation

```bash
pip install aiperf
# Extras: [mlflow] [otel] [wandb]
```

**NumPy compatibility warning:** On older x86_64 VMs without X86_V2 CPU features, NumPy 2.x (pulled by AIPerf) crashes. Pre-fix with:
```bash
pip install "numpy<2.0" --force-reinstall --no-cache-dir
pip install aiperf
```

## CLI Quickstart

```bash
aiperf profile \
  --config config.yaml \
  --api-key "$API_KEY" \
  --concurrency 10 \
  --request-count 50 \
  --artifact-dir results/
```

## Key CLI Flags (0.10.0+)

| Flag | YAML Override | Description |
|------|---------------|-------------|
| `--config FILE` | - | Load Schema 2.0 YAML config |
| `--api-key KEY` | `endpoint.api_key` | Auth key (overrides YAML) |
| `--concurrency N` | phase.sessions | Concurrent requests/sessions |
| `--request-rate N` | - | Requests/sec (alt. to concurrency) |
| `--request-count N` | phase.requests | Total requests |
| `--duration N` | phase.duration | Time-based run (seconds) |
| `--streaming` | `endpoint.streaming` | Enable streaming (TTFT/ITL) |
| `--ui simple` | `ui` | Progress bars (vs. `dashboard` TUI) |
| `--ui none` | `ui` | Headless |
| `--artifact-dir DIR` | `output_dir` | Results directory |
| `--no-gpu-telemetry` | - | Skip DCGM/GPU metrics |
| `--no-server-metrics` | - | Skip Prometheus scraping |
| `--tokenizer builtin` | `tokenizer.name` | tiktoken (no HF download) |
| `--random-seed N` | `random_seed` | Reproducibility |

## Schema 2.0 YAML Config

```yaml
schema_version: "2.0"
benchmark:
  models:
    items:
      - name: "vllm/gemma-4-31b-it"
    strategy: "round_robin"
  endpoint:
    urls:
      - "https://ai.noris.de/v1"
    type: "chat"
    api_key: "${API_KEY}"
    streaming: true
    headers:
      Authorization: "Bearer ${API_KEY}"
  datasets:
    - type: "public"        # "public" = HF download; "synthetic" = local gen
      name: "main"
      dataset: "sharegpt"
      sampling: "shuffle"
  phases:
    - type: "concurrency"
      name: "profiling"
      requests: 50
      sessions: 1
  tokenizer:
    name: "builtin"
```

Run from YAML:
```bash
aiperf profile --config config.yaml
```

## Outputs
- **CSV** (`profile_export_aiperf.csv`): Per-request metrics
- **JSON** (`profile_export_aiperf.json`): Aggregated stats (avg, p50, p90, p99)
- **Log** (`logs/aiperf.log`): Detailed debug
- **Plots**: `aiperf plot <artifact-dir>`

## Dataset Types Quick Reference

| Type | Speed | Source | Use |
|------|-------|--------|-----|
| `public` (ShareGPT) | Slow first-run (~2-5 min) | HuggingFace download | Realistic workload |
| `synthetic` (fixed ISL/OSL) | Instant | Local generation | Capacity testing, reproducibility |

## Orchestration Pattern

YAML base + CLI sweeps:
```bash
for conc in 1 5 10 25 50 100; do
  aiperf profile --config capacity.yaml --concurrency "$conc" \
    --request-count 100 --artifact-dir "results/conc${conc}"
done
```

## Validation Errors & Quick Fixes

| Error | Fix |
|-------|-----|
| `1 validation error for AIPerfConfig` | Wrap YAML in `schema_version: "2.0"` / `benchmark:` envelope |
| `Phase 'profiling': at least one of 'requests', 'duration', or 'sessions' must be specified` | Add `requests: N` or `duration: N` to phase |
| `NumPy: baseline optimizations (X86_V2) not supported` | `pip install "numpy<2.0" --force-reinstall` |