# OpenAI-Compatible Multi-Model API Wrapper for ComfyUI FastAPI proxy that exposes `POST /v1/images/generations` (OpenAI format) and routes to **multiple** ComfyUI workflows depending on the requested `model`. Supported models: `ideogram4`, `flux2`, `flux1-dev`, `flux1-schnell`. ## When to use - You need one endpoint that can switch between Ideogram 4 and FLUX families without reconfiguring the frontend. - Your chat UI only speaks OpenAI `/v1/images/generations` but you want to use ComfyUI's native pipelines (including non-standard nodes like `Ideogram4Scheduler`, `ModelSamplingFlux`, `DualCLIPLoader`). - You run ComfyUI inside a Proxmox LXC/VM and want a systemd-managed adapter service. ## Architecture ``` Frontend (OpenWebUI, LibreChat, etc.) POST /v1/images/generations {model, prompt, size, ...} │ ▼ FastAPI Adapter (this wrapper) ─ model switch ──► _build_ideogram4_workflow() _build_flux2_workflow() _build_flux1_workflow() │ ▼ ComfyUI REST API POST /prompt │ ▼ Poll /history/{prompt_id} → fetch /view ``` ## Adapter Code (Multi-Model) The full source is provided as a **template** in this skill: `templates/comfyui-openai-adapter.py` Key design choices: - **Model routing** — `req.model` selects the workflow builder. - **Size validation / fallback** — Ideogram4 only accepts specific resolutions. FLUX is more permissive. Invalid sizes silently fall back to the model's default. - **Loader node per model family:** - Ideogram4 → `UNETLoader` (safetensors) - FLUX (GGUF) → `UnetLoaderGGUF` (from ComfyUI-GGUF custom node) - Ideogram4 → single `CLIPLoader` with `type: "ideogram4"` - FLUX → `DualCLIPLoader` with `type: "flux"` (clip_l + t5xxl) - **Quality → steps mapping:** `low` (8), `medium` (20), `high` (30), `ultra` (50). FLUX1-schnell hard-capped to 8 steps. - **Ideogram4 minimum steps:** do not go below ~12 steps — the model produces near-blank gray blocks at ≤8 steps. Clamp or warn when the user explicitly requests fewer. - **Ideogram4 asymmetric CFG wiring (CRITICAL):** The adapter's `_build_ideogram4_workflow()` MUST use `CFGOverride` + `DualModelGuider` + a separate unconditional UNet (`ideogram4_unconditional_fp8_scaled`) + `ConditioningZeroOut`. Using `BasicGuider` does NOT throw an error but silently degrades image quality. The `DualModelGuider` input for the unconditional model is `model_negative` (NOT `model_1`). See `references/ideogram4.md` for the full blueprint. - **Gray-block detection:** If Ideogram4 output has PIL variance < 15 across all channels, the generation produced a blank/gray image (often at `quality=low` + `style=photorealistic`). Flag these as failures and retry with higher steps or different style. - **ROCm environment** — if ComfyUI runs on AMD ROCm inside an LXC, the *systemd service* for ComfyUI must set `Environment=HSA_OVERRIDE_GFX_VERSION=11.0.0` (or whatever matches the GPU). The adapter itself does not need ROCm env vars. - **Timeout for long generations** — set the adapter's HTTP poll loop to at least 1800 s (30 min) for high-resolution workloads. Edit `for i in range(720)` (30 min at 5 s intervals) and set `httpx.AsyncClient(timeout=2400)` on the initial `/prompt` POST. - **Synchronous (blocking) by design** — the adapter does NOT implement async jobs. The HTTP response blocks until ComfyUI finishes. For multi-hour jobs this means the client must keep the connection open. If a true async job system is needed, build `POST → job_id` + `GET /v1/jobs/{id}` separately. ## Running it ### Option A: Python venv (quick test) ```bash python3 -m venv /opt/ideogram4-api/venv source /opt/ideogram4-api/venv/bin/activate pip install fastapi uvicorn httpx pydantic export COMFYUI_HOST=localhost export COMFYUI_PORT=8188 python3 comfyui-openai-adapter.py ``` ### Option B: Docker on dedicated host (persistent) ```dockerfile FROM python:3.12-slim WORKDIR /app RUN pip install --no-cache-dir fastapi uvicorn httpx pydantic COPY comfyui-openai-adapter.py ./main.py EXPOSE 8000 CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"] ``` > The adapter and ComfyUI should be on the same LAN or host. Latency matters because the adapter polls ComfyUI every 5 s for up to 60 min. ### Option C: systemd service (LXC/VM — recommended for Proxmox) **Prerequisite:** Kill any previously manual/nohup-started ComfyUI or adapter processes *before* enabling systemd units. Old processes hold ports 8188 and 8000 and cause `address already in use`. ```bash # Inside LXC container (e.g. CT 204) ss -tlnp | grep -E '8188|8000' fuser -k 8188/tcp 2>/dev/null fuser -k 8000/tcp 2>/dev/null ``` **`comfyui.service`** (note the ROCm env var for gfx1150): ```ini [Unit] Description=ComfyUI After=network.target [Service] Type=simple User=root WorkingDirectory=/opt/ComfyUI ExecStart=/usr/bin/python3 main.py --listen 0.0.0.0 --port 8188 Restart=always RestartSec=10 Environment=HIP_VISIBLE_DEVICES=0 Environment=HSA_OVERRIDE_GFX_VERSION=11.0.0 Environment=PATH=/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin Environment=LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/torch/lib:/opt/rocm/lib [Install] WantedBy=multi-user.target ``` **`ideogram4-api.service`** (rename to `comfyui-api.service` if generic): ```ini [Unit] Description=ComfyUI OpenAI API Wrapper After=network.target comfyui.service Requires=comfyui.service [Service] Type=simple User=root WorkingDirectory=/opt/ideogram4-api ExecStart=/opt/ideogram4-api/venv/bin/python /opt/ideogram4-api/main.py Restart=always RestartSec=5 Environment=PATH=/opt/ideogram4-api/venv/bin:/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin Environment=COMFYUI_HOST=localhost Environment=COMFYUI_PORT=8188 [Install] WantedBy=multi-user.target ``` > **Filename note:** The adapter source may be saved as `main.py`, `app.py`, or `adapter.py`. Check `ls /opt/ideogram4-api/` and match `ExecStart=` exactly. Enable & start: ```bash systemctl daemon-reload systemctl enable --now comfyui.service sleep 15 systemctl enable --now ideogram4-api.service ``` ## Extended OpenAI-compatible parameters Beyond the standard `model`, `prompt`, `n`, `size`, `response_format`, the adapter accepts these ComfyUI extensions: | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `seed` | int | random | Fixed seed. Random if omitted. | | `steps` | int | from `quality` | Denoising steps (1–100). Overrides `quality` if set. | | `cfg` | float | model-specific | Guidance scale. Ideogram4=1.0, FLUX=3.5, others=7.5 | | `quality` | string | `medium` | Maps to steps: `low`(8), `medium`(20), `high`(30), `ultra`(50). | | `negative_prompt` | string | `""` | Negative prompt text. | | `style` | string | `""` | Style modifier string. | | `control_strength` | float | — | Reserved for future ControlNet integration. | Example request body: ```json { "model": "flux2", "prompt": "a cyberpunk alley at night, neon signs", "n": 1, "size": "1024x1024", "quality": "high", "seed": 42, "cfg": 3.5, "negative_prompt": "blurry, low quality" } ``` ## Frontend configuration (OpenWebUI example) 1. Admin → **Settings** → **Images** 2. **Image Generation Engine:** `Open AI` 3. **OpenAI API Base URL:** `http://:8000/v1` 4. **OpenAI API Key:** any dummy string (adapter does not validate) 5. **Model:** `ideogram4` or `flux2` or `flux1-dev` ## Pitfalls | Issue | Cause | Fix | |---|---|---| | `TimeoutError` after 300 s | High resolution or GPU under load | Clamp `size`, increase timeout, reduce `steps` | | `500` from ComfyUI + `value_not_in_list` | `size` not in model's allowed resolutions | Check allowed sizes; use defaults | | `[Errno 98] address already in use` | Old manual `nohup` process still running | `fuser -k 8000/tcp` / `fuser -k 8188/tcp`, then restart systemd | | `can't open file '/opt/ideogram4-api/app.py'` | `ExecStart` mismatches actual filename | Check `ls /opt/ideogram4-api/` and align `ExecStart` | | `UNETLoader` node error for `.gguf` model | Using `UNETLoader` for a GGUF file | Switch to `UnetLoaderGGUF` | | `RuntimeError` during text-encode (ROCm) | Missing `HSA_OVERRIDE_GFX_VERSION` | Add to `comfyui.service` `Environment=` line | | Ideogram4 black/corrupted output | Corrupt `.safetensors` or wrong VAE | Verify file integrity; use `flux2-vae.safetensors` | | FLUX output dimensions wrong | Used `EmptyLatentImage` instead of `EmptyFlux2LatentImage` | Ideogram4 uses `EmptyFlux2LatentImage`; FLUX.1 uses `EmptyLatentImage` | | `DualModelGuider.execute() got unexpected keyword argument 'model_1'` | Wrong workflow key name | Use `model_negative`, NOT `model_1` | | systemd `comfyui.service` keeps failing with `exit-code` | Port 8188 held by old manual ComfyUI | `pkill -f "main.py.*8188"`, then `systemctl restart` | ## Model-specific node quick-reference ### Ideogram 4 local (with asymmetric CFG — correct wiring) ```json { "1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"}, "2": {"inputs": {"unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"}, "3": {"inputs": {"clip_name": "qwen3vl_8b_fp8_scaled.safetensors", "type": "ideogram4"}, "class_type": "CLIPLoader"}, "4": {"inputs": {"text": "...", "clip": ["3", 0]}, "class_type": "CLIPTextEncode"}, "5": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"}, "6": {"inputs": {"noise_seed": 42}, "class_type": "RandomNoise"}, "7": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"}, "8": {"inputs": {"steps": 20, "width": 1024, "height": 1024, "mu": 0.5, "std": 1.75}, "class_type": "Ideogram4Scheduler"}, "15": {"inputs": {"unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"}, "16": {"inputs": {"conditioning": ["4", 0]}, "class_type": "ConditioningZeroOut"}, "17": {"inputs": {"cfg": 3.0, "start_percent": 0.7, "end_percent": 1.0, "model": ["2", 0]}, "class_type": "CFGOverride"}, "9": {"inputs": {"cfg": 7.0, "model": ["17", 0], "positive": ["4", 0], "model_negative": ["15", 0], "negative": ["16", 0]}, "class_type": "DualModelGuider"}, "10": {"inputs": {"noise": ["6", 0], "guider": ["9", 0], "sampler": ["7", 0], "sigmas": ["8", 0], "latent_image": ["5", 0]}, "class_type": "SamplerCustomAdvanced"}, "11": {"inputs": {"samples": ["10", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"}, "12": {"inputs": {"filename_prefix": "ideogram4_api", "images": ["11", 0]}, "class_type": "SaveImage"} } ``` Allowed sizes: `1024x1024`, `1024x1536`, `1536x1024`, `832x1216`, `1216x832`, `896x1152`, `1152x896`. ### FLUX.2 / FLUX.1 ```json { "1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"}, "2": {"inputs": {"unet_name": "flux1-dev-Q4_K_S.gguf"}, "class_type": "UnetLoaderGGUF"}, "3": {"inputs": {"model": ["2", 0], "max_shift": 1.15, "base_shift": 0.5, "width": 1024, "height": 1024}, "class_type": "ModelSamplingFlux"}, "4": {"inputs": {"clip_name1": "clip_l.safetensors", "clip_name2": "t5xxl_fp8_e4m3fn.safetensors", "type": "flux", "device": "default"}, "class_type": "DualCLIPLoader"}, "5": {"inputs": {"text": "...", "clip": ["4", 0]}, "class_type": "CLIPTextEncode"}, "6": {"inputs": {"conditioning": ["5", 0], "guidance": 3.5}, "class_type": "FluxGuidance"}, "7": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyLatentImage"}, "8": {"inputs": {"noise_seed": 42}, "class_type": "RandomNoise"}, "9": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"}, "10": {"inputs": {"model": ["3", 0], "steps": 20, "denoise": 1.0, "scheduler": "simple", "sampler": ["9", 0]}, "class_type": "BasicScheduler"}, "11": {"inputs": {"model": ["3", 0], "conditioning": ["6", 0]}, "class_type": "BasicGuider"}, "12": {"inputs": {"noise": ["8", 0], "guider": ["11", 0], "sampler": ["9", 0], "sigmas": ["10", 0], "latent_image": ["7", 0]}, "class_type": "SamplerCustomAdvanced"}, "13": {"inputs": {"samples": ["12", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"}, "14": {"inputs": {"filename_prefix": "flux2_api", "images": ["13", 0]}, "class_type": "SaveImage"} } ``` Allowed sizes: all common resolutions up to 1536×1536. FLUX schnell defaults to 4–8 steps; dev to 20–50.