# AIPerf Background-Process & Orchestrator Pitfalls

Session artifact: debugging why a full 12-run benchmark suite silently failed with `aiperf: command not found` despite successful manual runs.

## The Problem: `aiperf: command not found` in Background

The background process (started via `bash -lic`) did not inherit the venv-activated PATH. All 12 runs produced:
```
./scripts/run-suite.sh: line 90: aiperf: command not found
```

**Root cause:** `aiperf` was installed into a Python venv, but the background shell did not activate the venv before running the orchestrator.

## The Fix: Absolute Path to venv Binary

In the orchestrator script, define an absolute path variable:
```bash
AIPERF="$BASE_DIR/venv/bin/aiperf"
if [ ! -x "$AIPERF" ]; then
    echo "ERROR: AIPerf not found at $AIPERF"
    exit 1
fi
```

Then call `"$AIPERF" profile ...` everywhere instead of plain `aiperf profile`.

## ShareGPT Pre-Download Strategy

`type: "public"` datasets download ShareGPT from HuggingFace on first use (~2–5 minutes). To avoid this delay during production benchmark runs, perform a pre-download:

```bash
# Pre-download ShareGPT into AIPerf's local cache
venv/bin/aiperf profile \
    --config configs/<any-model>/sales.yaml \
    --concurrency 1 --request-count 1 \
    --no-gpu-telemetry --no-server-metrics \
    --artifact-dir /tmp/predownload
```

Subsequent sales-scenario runs will use the cached dataset instantly.

## AIPerf 0.10.0 CLI Flag Reference

Critical flags used in this session:

| Flag | What it does | Status |
|------|-------------|--------|
| `--ui simple` | Minimal UI mode (no TUI) | ✅ Works |
| `--ui dashboard` | Full TUI dashboard | ✅ Works |
| `--artifact-dir PATH` | Output directory for CSV/JSON/logs | ✅ Works |
| `--no-gpu-telemetry` | Skip DCGM/GPU metrics | ✅ Works |
| `--no-server-metrics` | Skip Prometheus scraping | ✅ Works |
| `--request-count N` | Total requests (overrides YAML `requests`) | ✅ Works |
| `--concurrency N` | Concurrent sessions | ✅ Works |
| `--simple` | ❌ NOT a flag; use `--ui simple` | ❌ Rejected |
| `--output PATH` | ❌ NOT a flag; use `--artifact-dir` | ❌ Rejected |
| `--output-artifact-dir PATH` | Alias for `--artifact-dir` | ✅ Works |
| `--api-key KEY` | Overrides YAML `api_key` | ✅ Works |

## The `api_key` + `requests`/`duration`/`sessions` Interaction

Sequence of errors encountered:

1. `api_key must be a valid string` — this appeared when `phases[].requests` was missing.
2. With `requests: 50` and `sessions: 1` added to phases → config validated.
3. **Lesson:** The `api_key` error was misleading. The real issue was always a missing `requests`/`duration`/`sessions` in the phase definition.

Pattern that works:
```yaml
phases:
  - type: "concurrency"
    name: "profiling"
    requests: 50          # REQUIRED
    sessions: 1           # optional, default 1
```

## Complete Working Config (Single File)

```yaml
schema_version: "2.0"
benchmark:
  models:
    items:
      - name: "vllm/gemma-4-31b-it"
    strategy: "round_robin"
  endpoint:
    urls:
      - "https://ai.noris.de/v1"
    type: "chat"
    api_key: "${NORIS_API_KEY}"
    streaming: true
    headers:
      Authorization: "Bearer ${NORIS_API_KEY}"
  datasets:
    - type: "synthetic"
      name: "main"
      entries: 100
      isl:
        type: "fixed"
        value: 512
      osl:
        type: "fixed"
        value: 128
  phases:
    - type: "concurrency"
      name: "profiling"
      requests: 50
      sessions: 1
  tokenizer:
    name: "builtin"
```