# OpenAI-Compatible Multi-Model API Wrapper for ComfyUI

FastAPI proxy that exposes `POST /v1/images/generations` (OpenAI format) and
routes to **multiple** ComfyUI workflows depending on the requested `model`.

Supported models: `ideogram4`, `flux2`, `flux1-dev`, `flux1-schnell`.

## When to use

- You need one endpoint that can switch between Ideogram 4 and FLUX
  families without reconfiguring the frontend.
- Your chat UI only speaks OpenAI `/v1/images/generations` but you want
  to use ComfyUI's native pipelines (including non-standard nodes like
  `Ideogram4Scheduler`, `ModelSamplingFlux`, `DualCLIPLoader`).
- You run ComfyUI inside a Proxmox LXC/VM and want a systemd-managed
  adapter service.

## Architecture

```
Frontend (OpenWebUI, LibreChat, etc.)
   POST /v1/images/generations  {model, prompt, size, ...}
         │
         ▼
   FastAPI Adapter (this wrapper)
   ─ model switch ──► _build_ideogram4_workflow()
                     _build_flux2_workflow()
                     _build_flux1_workflow()
         │
         ▼
   ComfyUI REST API  POST /prompt
         │
         ▼
   Poll /history/{prompt_id} → fetch /view
```

## Adapter Code (Multi-Model)

The full source is provided as a **template** in this skill:
`templates/comfyui-openai-adapter.py`

Key design choices:

- **Model routing** — `req.model` selects the workflow builder.
- **Size validation / fallback** — Ideogram4 only accepts specific
  resolutions. FLUX is more permissive. Invalid sizes silently fall back to the model's default.
- **Loader node per model family:**
  - Ideogram4 → `UNETLoader` (safetensors)
  - FLUX (GGUF) → `UnetLoaderGGUF` (from ComfyUI-GGUF custom node)
  - Ideogram4 → single `CLIPLoader` with `type: "ideogram4"`
  - FLUX → `DualCLIPLoader` with `type: "flux"` (clip_l + t5xxl)
- **Quality → steps mapping:** `low` (8), `medium` (20), `high` (30), `ultra` (50).
  FLUX1-schnell hard-capped to 8 steps.
- **Ideogram4 minimum steps:** do not go below ~12 steps — the model
  produces near-blank gray blocks at ≤8 steps. Clamp or warn when
  the user explicitly requests fewer.
- **Ideogram4 asymmetric CFG wiring (CRITICAL):** The adapter's `_build_ideogram4_workflow()`
  MUST use `CFGOverride` + `DualModelGuider` + a separate unconditional UNet
  (`ideogram4_unconditional_fp8_scaled`) + `ConditioningZeroOut`. Using `BasicGuider`
  does NOT throw an error but silently degrades image quality. The `DualModelGuider`
  input for the unconditional model is `model_negative` (NOT `model_1`).
  See `references/ideogram4.md` for the full blueprint.
- **Gray-block detection:** If Ideogram4 output has PIL variance < 15 across all
  channels, the generation produced a blank/gray image (often at `quality=low` + `style=photorealistic`).
  Flag these as failures and retry with higher steps or different style.
- **ROCm environment** — if ComfyUI runs on AMD ROCm inside an LXC,
  the *systemd service* for ComfyUI must set
  `Environment=HSA_OVERRIDE_GFX_VERSION=11.0.0` (or whatever matches
  the GPU). The adapter itself does not need ROCm env vars.
- **Timeout for long generations** — set the adapter's HTTP poll loop
  to at least 1800 s (30 min) for high-resolution workloads.
  Edit `for i in range(720)` (30 min at 5 s intervals) and set
  `httpx.AsyncClient(timeout=2400)` on the initial `/prompt` POST.
- **Synchronous (blocking) by design** — the adapter does NOT implement
  async jobs. The HTTP response blocks until ComfyUI finishes. For
  multi-hour jobs this means the client must keep the connection open.
  If a true async job system is needed, build `POST → job_id` +
  `GET /v1/jobs/{id}` separately.

## Running it

### Option A: Python venv (quick test)

```bash
python3 -m venv /opt/ideogram4-api/venv
source /opt/ideogram4-api/venv/bin/activate
pip install fastapi uvicorn httpx pydantic
export COMFYUI_HOST=localhost
export COMFYUI_PORT=8188
python3 comfyui-openai-adapter.py
```

### Option B: Docker on dedicated host (persistent)

```dockerfile
FROM python:3.12-slim
WORKDIR /app
RUN pip install --no-cache-dir fastapi uvicorn httpx pydantic
COPY comfyui-openai-adapter.py ./main.py
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
```

> The adapter and ComfyUI should be on the same LAN or host. Latency matters because the adapter polls ComfyUI every 5 s for up to 60 min.

### Option C: systemd service (LXC/VM — recommended for Proxmox)

**Prerequisite:** Kill any previously manual/nohup-started ComfyUI or adapter processes *before* enabling systemd units. Old processes hold ports 8188 and 8000 and cause `address already in use`.

```bash
# Inside LXC container (e.g. CT 204)
ss -tlnp | grep -E '8188|8000'
fuser -k 8188/tcp 2>/dev/null
fuser -k 8000/tcp 2>/dev/null
```

**`comfyui.service`** (note the ROCm env var for gfx1150):

```ini
[Unit]
Description=ComfyUI
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/ComfyUI
ExecStart=/usr/bin/python3 main.py --listen 0.0.0.0 --port 8188
Restart=always
RestartSec=10
Environment=HIP_VISIBLE_DEVICES=0
Environment=HSA_OVERRIDE_GFX_VERSION=11.0.0
Environment=PATH=/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin
Environment=LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/torch/lib:/opt/rocm/lib

[Install]
WantedBy=multi-user.target
```

**`ideogram4-api.service`** (rename to `comfyui-api.service` if generic):

```ini
[Unit]
Description=ComfyUI OpenAI API Wrapper
After=network.target comfyui.service
Requires=comfyui.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/ideogram4-api
ExecStart=/opt/ideogram4-api/venv/bin/python /opt/ideogram4-api/main.py
Restart=always
RestartSec=5
Environment=PATH=/opt/ideogram4-api/venv/bin:/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin
Environment=COMFYUI_HOST=localhost
Environment=COMFYUI_PORT=8188

[Install]
WantedBy=multi-user.target
```

> **Filename note:** The adapter source may be saved as `main.py`, `app.py`, or `adapter.py`. Check `ls /opt/ideogram4-api/` and match `ExecStart=` exactly.

Enable & start:
```bash
systemctl daemon-reload
systemctl enable --now comfyui.service
sleep 15
systemctl enable --now ideogram4-api.service
```

## Extended OpenAI-compatible parameters

Beyond the standard `model`, `prompt`, `n`, `size`, `response_format`,
the adapter accepts these ComfyUI extensions:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `seed` | int | random | Fixed seed. Random if omitted. |
| `steps` | int | from `quality` | Denoising steps (1–100). Overrides `quality` if set. |
| `cfg` | float | model-specific | Guidance scale. Ideogram4=1.0, FLUX=3.5, others=7.5 |
| `quality` | string | `medium` | Maps to steps: `low`(8), `medium`(20), `high`(30), `ultra`(50). |
| `negative_prompt` | string | `""` | Negative prompt text. |
| `style` | string | `""` | Style modifier string. |
| `control_strength` | float | — | Reserved for future ControlNet integration. |

Example request body:
```json
{
  "model": "flux2",
  "prompt": "a cyberpunk alley at night, neon signs",
  "n": 1,
  "size": "1024x1024",
  "quality": "high",
  "seed": 42,
  "cfg": 3.5,
  "negative_prompt": "blurry, low quality"
}
```

## Frontend configuration (OpenWebUI example)

1. Admin → **Settings** → **Images**
2. **Image Generation Engine:** `Open AI`
3. **OpenAI API Base URL:** `http://<adapter-host>:8000/v1`
4. **OpenAI API Key:** any dummy string (adapter does not validate)
5. **Model:** `ideogram4` or `flux2` or `flux1-dev`

## Pitfalls

| Issue | Cause | Fix |
|---|---|---|
| `TimeoutError` after 300 s | High resolution or GPU under load | Clamp `size`, increase timeout, reduce `steps` |
| `500` from ComfyUI + `value_not_in_list` | `size` not in model's allowed resolutions | Check allowed sizes; use defaults |
| `[Errno 98] address already in use` | Old manual `nohup` process still running | `fuser -k 8000/tcp` / `fuser -k 8188/tcp`, then restart systemd |
| `can't open file '/opt/ideogram4-api/app.py'` | `ExecStart` mismatches actual filename | Check `ls /opt/ideogram4-api/` and align `ExecStart` |
| `UNETLoader` node error for `.gguf` model | Using `UNETLoader` for a GGUF file | Switch to `UnetLoaderGGUF` |
| `RuntimeError` during text-encode (ROCm) | Missing `HSA_OVERRIDE_GFX_VERSION` | Add to `comfyui.service` `Environment=` line |
| Ideogram4 black/corrupted output | Corrupt `.safetensors` or wrong VAE | Verify file integrity; use `flux2-vae.safetensors` |
| FLUX output dimensions wrong | Used `EmptyLatentImage` instead of `EmptyFlux2LatentImage` | Ideogram4 uses `EmptyFlux2LatentImage`; FLUX.1 uses `EmptyLatentImage` |
| `DualModelGuider.execute() got unexpected keyword argument 'model_1'` | Wrong workflow key name | Use `model_negative`, NOT `model_1` |
| systemd `comfyui.service` keeps failing with `exit-code` | Port 8188 held by old manual ComfyUI | `pkill -f "main.py.*8188"`, then `systemctl restart` |

## Model-specific node quick-reference

### Ideogram 4 local (with asymmetric CFG — correct wiring)

```json
{
  "1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"},
  "2": {"inputs": {"unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default"},
        "class_type": "UNETLoader"},
  "3": {"inputs": {"clip_name": "qwen3vl_8b_fp8_scaled.safetensors", "type": "ideogram4"},
        "class_type": "CLIPLoader"},
  "4": {"inputs": {"text": "...", "clip": ["3", 0]}, "class_type": "CLIPTextEncode"},
  "5": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
  "6": {"inputs": {"noise_seed": 42}, "class_type": "RandomNoise"},
  "7": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"},
  "8": {"inputs": {"steps": 20, "width": 1024, "height": 1024, "mu": 0.5, "std": 1.75},
        "class_type": "Ideogram4Scheduler"},
  "15": {"inputs": {"unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default"},
         "class_type": "UNETLoader"},
  "16": {"inputs": {"conditioning": ["4", 0]}, "class_type": "ConditioningZeroOut"},
  "17": {"inputs": {"cfg": 3.0, "start_percent": 0.7, "end_percent": 1.0, "model": ["2", 0]},
         "class_type": "CFGOverride"},
  "9": {"inputs": {"cfg": 7.0, "model": ["17", 0], "positive": ["4", 0],
         "model_negative": ["15", 0], "negative": ["16", 0]},
        "class_type": "DualModelGuider"},
  "10": {"inputs": {"noise": ["6", 0], "guider": ["9", 0], "sampler": ["7", 0],
          "sigmas": ["8", 0], "latent_image": ["5", 0]}, "class_type": "SamplerCustomAdvanced"},
  "11": {"inputs": {"samples": ["10", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"},
  "12": {"inputs": {"filename_prefix": "ideogram4_api", "images": ["11", 0]}, "class_type": "SaveImage"}
}
```
Allowed sizes: `1024x1024`, `1024x1536`, `1536x1024`, `832x1216`, `1216x832`, `896x1152`, `1152x896`.

### FLUX.2 / FLUX.1
```json
{
  "1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"},
  "2": {"inputs": {"unet_name": "flux1-dev-Q4_K_S.gguf"}, "class_type": "UnetLoaderGGUF"},
  "3": {"inputs": {"model": ["2", 0], "max_shift": 1.15, "base_shift": 0.5,
          "width": 1024, "height": 1024}, "class_type": "ModelSamplingFlux"},
  "4": {"inputs": {"clip_name1": "clip_l.safetensors", "clip_name2": "t5xxl_fp8_e4m3fn.safetensors",
          "type": "flux", "device": "default"}, "class_type": "DualCLIPLoader"},
  "5": {"inputs": {"text": "...", "clip": ["4", 0]}, "class_type": "CLIPTextEncode"},
  "6": {"inputs": {"conditioning": ["5", 0], "guidance": 3.5}, "class_type": "FluxGuidance"},
  "7": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyLatentImage"},
  "8": {"inputs": {"noise_seed": 42}, "class_type": "RandomNoise"},
  "9": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"},
  "10": {"inputs": {"model": ["3", 0], "steps": 20, "denoise": 1.0, "scheduler": "simple",
           "sampler": ["9", 0]}, "class_type": "BasicScheduler"},
  "11": {"inputs": {"model": ["3", 0], "conditioning": ["6", 0]}, "class_type": "BasicGuider"},
  "12": {"inputs": {"noise": ["8", 0], "guider": ["11", 0], "sampler": ["9", 0],
           "sigmas": ["10", 0], "latent_image": ["7", 0]}, "class_type": "SamplerCustomAdvanced"},
  "13": {"inputs": {"samples": ["12", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"},
  "14": {"inputs": {"filename_prefix": "flux2_api", "images": ["13", 0]}, "class_type": "SaveImage"}
}
```
Allowed sizes: all common resolutions up to 1536×1536. FLUX schnell defaults to 4–8 steps; dev to 20–50.
