hermes-skills/mlops/llm-inference-benchmarking/references/aiperf-kurzreferenz.md

AIPerf 0.10.0+ Kurzreferenz

Installation

pip install aiperf
# Extras: [mlflow] [otel] [wandb]

NumPy compatibility warning: On older x86_64 VMs without X86_V2 CPU features, NumPy 2.x (pulled by AIPerf) crashes. Pre-fix with:

pip install "numpy<2.0" --force-reinstall --no-cache-dir
pip install aiperf

CLI Quickstart

aiperf profile \
  --config config.yaml \
  --api-key "$API_KEY" \
  --concurrency 10 \
  --request-count 50 \
  --artifact-dir results/

Key CLI Flags (0.10.0+)

Flag	YAML Override	Description
--config FILE	-	Load Schema 2.0 YAML config
--api-key KEY	endpoint.api_key	Auth key (overrides YAML)
--concurrency N	phase.sessions	Concurrent requests/sessions
--request-rate N	-	Requests/sec (alt. to concurrency)
--request-count N	phase.requests	Total requests
--duration N	phase.duration	Time-based run (seconds)
--streaming	endpoint.streaming	Enable streaming (TTFT/ITL)
--ui simple	ui	Progress bars (vs. dashboard TUI)
--ui none	ui	Headless
--artifact-dir DIR	output_dir	Results directory
--no-gpu-telemetry	-	Skip DCGM/GPU metrics
--no-server-metrics	-	Skip Prometheus scraping
--tokenizer builtin	tokenizer.name	tiktoken (no HF download)
--random-seed N	random_seed	Reproducibility

Schema 2.0 YAML Config

schema_version: "2.0"
benchmark:
  models:
    items:
      - name: "vllm/gemma-4-31b-it"
    strategy: "round_robin"
  endpoint:
    urls:
      - "https://ai.noris.de/v1"
    type: "chat"
    api_key: "${API_KEY}"
    streaming: true
    headers:
      Authorization: "Bearer ${API_KEY}"
  datasets:
    - type: "public"        # "public" = HF download; "synthetic" = local gen
      name: "main"
      dataset: "sharegpt"
      sampling: "shuffle"
  phases:
    - type: "concurrency"
      name: "profiling"
      requests: 50
      sessions: 1
  tokenizer:
    name: "builtin"

Run from YAML:

aiperf profile --config config.yaml

Outputs

• CSV (profile_export_aiperf.csv): Per-request metrics
• JSON (profile_export_aiperf.json): Aggregated stats (avg, p50, p90, p99)
• Log (logs/aiperf.log): Detailed debug
• Plots: aiperf plot <artifact-dir>

Dataset Types Quick Reference

Type	Speed	Source	Use
public (ShareGPT)	Slow first-run (~2-5 min)	HuggingFace download	Realistic workload
synthetic (fixed ISL/OSL)	Instant	Local generation	Capacity testing, reproducibility

Orchestration Pattern

YAML base + CLI sweeps:

for conc in 1 5 10 25 50 100; do
  aiperf profile --config capacity.yaml --concurrency "$conc" \
    --request-count 100 --artifact-dir "results/conc${conc}"
done

Validation Errors & Quick Fixes

Error	Fix
1 validation error for AIPerfConfig	Wrap YAML in schema_version: "2.0" / benchmark: envelope
Phase 'profiling': at least one of 'requests', 'duration', or 'sessions' must be specified	Add requests: N or duration: N to phase
NumPy: baseline optimizations (X86_V2) not supported	pip install "numpy<2.0" --force-reinstall