dominik/hermes-skills
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Hermes Agent Skills collection

Browse Source
This commit is contained in:
Debian
2026-07-12 19:02:59 +00:00
commit e9cc106625
789 changed files with 233126 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
creative/comfyui/SKILL.md
+776
View File
@@ -0,0 +1,776 @@
---
name: comfyui
description: "Generate images, video, and audio with ComfyUI — install, launch, manage nodes/models, run workflows with parameter injection. Uses the official comfy-cli for lifecycle and direct REST/WebSocket API for execution."
version: 5.0.0
author: [kshitijk4poor, alt-glitch]
license: MIT
platforms: [macos, linux, windows]
compatibility: "Requires ComfyUI (local, Comfy Desktop, or Comfy Cloud) and comfy-cli (auto-installed via pipx/uvx by the setup script)."
prerequisites:
commands: ["python3"]
setup:
help: "Run scripts/hardware_check.py FIRST to decide local vs Comfy Cloud; then scripts/comfyui_setup.sh auto-installs locally (or use Cloud API key for platform.comfy.org)."
metadata:
hermes:
tags:
- comfyui
- image-generation
- stable-diffusion
- flux
- sd3
- wan-video
- hunyuan-video
- creative
- generative-ai
- video-generation
related_skills: [stable-diffusion-image-generation, image_gen]
category: creative
---
# ComfyUI
Generate images, video, audio, and 3D content through ComfyUI using the
official `comfy-cli` for setup/lifecycle and direct REST/WebSocket API
for workflow execution.
## What's in this skill
**Reference docs (`references/`):**
- `official-cli.md` — every `comfy ...` command, with flags
- `rest-api.md` — REST + WebSocket endpoints (local + cloud), payload schemas
- `workflow-format.md` — API-format JSON, common node types, param mapping
- `flux2-setup.md` — FLUX.2 local setup: Qwen3VL, `EmptyFlux2LatentImage`, ROCm gfx1150
- `ideogram4-local-setup.md` — Ideogram 4 local vs. cloud: model downloads, verification recipe, `Ideogram4Scheduler` workflow
- `openai-api-wrapper.md` — multi-model OpenAI-compatible FastAPI adapter (Ideogram4 + FLUX family). Extended parameters (seed, steps, cfg, quality, negative_prompt, style). systemd service setup with ROCm env vars. Timeout tuning for 30 min generations. Ideogram4 safety/threshold notes (min steps, gray-block detection). See also `templates/comfyui-openai-adapter.py` for the full adapter source.
- `openwebui-integration.md` — OpenWebUI ↔ ComfyUI adapter for FLUX.2 (OpenAI-compatible `/v1/images/generations` bridge)
- `ideogram4.md` — Ideogram 4: cloud API (`IdeogramV4`) vs local (`Ideogram4Scheduler`) paths, model checklist, corruption fix, full workflow blueprint
- `ideogram4.md` — Ideogram 4: cloud API (`IdeogramV4`) vs local (`Ideogram4Scheduler`) paths, model checklist, corruption fix
**Scripts (`scripts/`):**
| Script | Purpose |
|--------|---------|
| `_common.py` | Shared HTTP, cloud routing, node catalogs (don't run directly) |
| `adapter_flux2.py` | FastAPI OpenAI adapter for FLUX.2: `/v1/images/generations` → ComfyUI `/api/prompt` |
| `hardware_check.py` | Probe GPU/VRAM/disk → recommend local vs Comfy Cloud |
| `comfyui_setup.sh` | Hardware check + comfy-cli + ComfyUI install + launch + verify |
| `extract_schema.py` | Read a workflow → list controllable params + model deps |
| `check_deps.py` | Check workflow against running server → list missing nodes/models |
| `auto_fix_deps.py` | Run check_deps then `comfy node install` / `comfy model download` |
| `run_workflow.py` | Inject params, submit, monitor, download outputs (HTTP or WS) |
| `run_batch.py` | Submit a workflow N times with sweeps, parallel up to your tier |
| `comfy_e2e_test.py` | Systematic end-to-end test: iterate model × quality × style, generate images, write JSON+Markdown report. Configurable via CLI args. |
| `ws_monitor.py` | Real-time WebSocket viewer for executing jobs (live progress) |
| `health_check.py` | Verification checklist runner — comfy-cli + server + models + smoke test |
| `fetch_logs.py` | Pull traceback / status messages for a given prompt_id |
**Example workflows (`workflows/`):** SD 1.5, SDXL, Flux Dev, SDXL img2img,
SDXL inpaint, ESRGAN upscale, AnimateDiff video, Wan T2V. See
`workflows/README.md`.
## When to Use
- User asks to generate images with Stable Diffusion, SDXL, Flux, SD3, etc.
- User wants to run a specific ComfyUI workflow file
- User wants to chain generative steps (txt2img → upscale → face restore)
- User needs ControlNet, inpainting, img2img, or other advanced pipelines
- User asks to manage ComfyUI queue, check models, or install custom nodes
- User wants video/audio/3D generation via AnimateDiff, Hunyuan, Wan, AudioCraft, etc.
## Architecture: Two Layers
```
┌─────────────────────────────────────────────────────┐
│ Layer 1: comfy-cli (official lifecycle tool) │
│ Setup, server lifecycle, custom nodes, models │
│ → comfy install / launch / stop / node / model │
└─────────────────────────┬───────────────────────────┘
│
┌─────────────────────────▼───────────────────────────┐
│ Layer 2: REST/WebSocket API + skill scripts │
│ Workflow execution, param injection, monitoring │
│ POST /api/prompt, GET /api/view, WS /ws │
│ → run_workflow.py, run_batch.py, ws_monitor.py │
└─────────────────────────────────────────────────────┘
```
**Why two layers?** The official CLI is excellent for installation and server
management but has minimal workflow execution support. The REST/WS API fills
that gap — the scripts handle param injection, execution monitoring, and
output download that the CLI doesn't do.
## Quick Start
### Detect environment
```bash
# What's available?
command -v comfy >/dev/null 2>&1 && echo "comfy-cli: installed"
curl -s http://127.0.0.1:8188/system_stats 2>/dev/null && echo "server: running"
# Can this machine run ComfyUI locally? (GPU/VRAM/disk check)
python3 scripts/hardware_check.py
```
If nothing is installed, see **Setup & Onboarding** below — but always run the
hardware check first.
### One-line health check
```bash
python3 scripts/health_check.py
# → JSON: comfy_cli on PATH? server reachable? at least one checkpoint? smoke-test passes?
```
## Core Workflow
### Step 1: Get a workflow JSON in API format
Workflows must be in API format (each node has `class_type`). They come from:
- ComfyUI web UI → **Workflow → Export (API)** (newer UI) or
the legacy "Save (API Format)" button (older UI)
- This skill's `workflows/` directory (ready-to-run examples)
- Community downloads (civitai, Reddit, Discord) — usually editor format,
must be loaded into ComfyUI then re-exported
Editor format (top-level `nodes` and `links` arrays) is **not directly
The skill scripts detect this and tell you to re-export.
## Integration with Chat UIs (OpenWebUI, etc.)
ComfyUI does not expose an OpenAI-compatible `/v1/images/generations` endpoint.
Chat UIs that expect image generation (OpenWebUI, LibreChat, etc.) need an
adapter when using ComfyUI as backend. Key considerations:
- **Native ComfyUI connectors** in chat UIs assume the classic SD/SDXL/FLUX.1
pipeline (`DualCLIPLoaderGGUF`, `EmptyLatentImage`, standard VAE). These will
NOT work with FLUX.2-klein-9B, which needs Qwen3-VL, `EmptyFlux2LatentImage`,
and a FLUX.2-compatible VAE (`taef2`). See `references/flux2-setup.md` for
the full node mapping.
- **Recommended approach:** build a thin OpenAI-compatible FastAPI proxy that
translates `POST /v1/images/generations` into a ComfyUI `POST /api/prompt`
call with your specific workflow. No public Docker image covers the full
FLUX.2 node stack; a custom service (~80 LoC Python/FastAPI) is needed.
- **Alternative:** use OpenWebUI Functions to call ComfyUI directly, but this
bypasses the polished image-generation UI overlay.
For a concrete walkthrough including verified network topology, see
`references/openwebui-integration.md`.
### Step 2: See what's controllable
```bash
python3 scripts/extract_schema.py workflow_api.json --summary-only
# → {"parameter_count": 12, "has_negative_prompt": true, "has_seed": true, ...}
python3 scripts/extract_schema.py workflow_api.json
# → full schema with parameters, model deps, embedding refs
```
### Step 3: Run with parameters
```bash
# Local (defaults to http://127.0.0.1:8188)
python3 scripts/run_workflow.py \
--workflow workflow_api.json \
--args '{"prompt": "a beautiful sunset over mountains", "seed": -1, "steps": 30}' \
--output-dir ./outputs
# Cloud (export API key once; uses correct /api routing automatically)
export COMFY_CLOUD_API_KEY="comfyui-..."
python3 scripts/run_workflow.py \
--workflow workflow_api.json \
--args '{"prompt": "..."}' \
--host https://cloud.comfy.org \
--output-dir ./outputs
# Real-time progress via WebSocket (requires `pip install websocket-client`)
python3 scripts/run_workflow.py \
--workflow flux_dev.json \
--args '{"prompt": "..."}' \
--ws
# img2img / inpaint: pass --input-image to upload + reference automatically
python3 scripts/run_workflow.py \
--workflow sdxl_img2img.json \
--input-image image=./photo.png \
--args '{"prompt": "make it watercolor", "denoise": 0.6}'
# Batch / sweep: 8 random seeds, parallel up to cloud tier limit
python3 scripts/run_batch.py \
--workflow sdxl.json \
--args '{"prompt": "abstract"}' \
--count 8 --randomize-seed --parallel 3 \
--output-dir ./outputs/batch
```
`-1` for `seed` (or omitting it with `--randomize-seed`) generates a fresh
random seed per run.
### Step 4: Present results
The scripts emit JSON to stdout describing every output file:
```json
{
"status": "success",
"prompt_id": "abc-123",
"outputs": [
{"file": "./outputs/sdxl_00001_.png", "node_id": "9",
"type": "image", "filename": "sdxl_00001_.png"}
]
}
```
## Decision Tree
| User says | Tool | Command |
|-----------|------|---------|
| **Lifecycle (use comfy-cli)** | | |
| "install ComfyUI" | comfy-cli | `bash scripts/comfyui_setup.sh` |
| "start ComfyUI" | comfy-cli | `comfy launch --background` |
| "stop ComfyUI" | comfy-cli | `comfy stop` |
| "install X node" | comfy-cli | `comfy node install <name>` |
| "download X model" | comfy-cli | `comfy model download --url <url> --relative-path models/checkpoints` |
| "list installed models" | comfy-cli | `comfy model list` |
| "list installed nodes" | comfy-cli | `comfy node show installed` |
| **Execution (use scripts)** | | |
| "is everything ready?" | script | `health_check.py` (optionally with `--workflow X --smoke-test`) |
| "what can I change in this workflow?" | script | `extract_schema.py W.json` |
| "check if W's deps are met" | script | `check_deps.py W.json` |
| "fix missing deps" | script | `auto_fix_deps.py W.json` |
| "generate an image" | script | `run_workflow.py --workflow W --args '{...}'` |
| "use this image" (img2img) | script | `run_workflow.py --input-image image=./x.png ...` |
| "8 variations with random seeds" | script | `run_batch.py --count 8 --randomize-seed ...` |
| "show me live progress" | script | `ws_monitor.py --prompt-id <id>` |
| "fetch the error from job X" | script | `fetch_logs.py <prompt_id>` |
| **Direct REST** | | |
| "what's in the queue?" | REST | `curl http://HOST:8188/queue` (local) or `--host https://cloud.comfy.org` |
| "cancel that" | REST | `curl -X POST http://HOST:8188/interrupt` |
| "free GPU memory" | REST | `curl -X POST http://HOST:8188/free` |
## Setup & Onboarding
When a user asks to set up ComfyUI, **the FIRST thing to do is ask whether
they want Comfy Cloud (hosted, zero install, API key) or Local (install
ComfyUI on their machine)**. Don't start running install commands or hardware
checks until they've answered.
**Official docs:** https://docs.comfy.org/installation
**CLI docs:** https://docs.comfy.org/comfy-cli/getting-started
**Cloud docs:** https://docs.comfy.org/get_started/cloud
**Cloud API:** https://docs.comfy.org/development/cloud/overview
### Step 0: Ask Local vs Cloud (ALWAYS FIRST)
Suggested script:
> "Do you want to run ComfyUI locally on your machine, or use Comfy Cloud?
>
> - **Comfy Cloud** — hosted on RTX 6000 Pro GPUs, all common models pre-installed,
> zero setup. Requires an API key (paid subscription required to actually run
> workflows; free tier is read-only). Best if you don't have a capable GPU.
> - **Local** — free, but your machine MUST meet the hardware requirements:
> - NVIDIA GPU with **≥6 GB VRAM** (≥8 GB for SDXL, ≥12 GB for Flux/video), OR
> - AMD GPU with ROCm support (Linux), OR
> - Apple Silicon Mac (M1+) with **≥16 GB unified memory** (≥32 GB recommended).
> - Intel Macs and machines with no GPU will NOT work — use Cloud instead.
>
> Which would you like?"
Routing:
- **Cloud** → skip to **Path A**.
- **Local** → run hardware check first, then pick a path from Paths B–E based on the verdict.
- **Unsure** → run the hardware check and let the verdict decide.
### Step 1: Verify Hardware (ONLY if user chose local)
```bash
python3 scripts/hardware_check.py --json
# Optional: also probe `torch` for actual CUDA/MPS:
python3 scripts/hardware_check.py --json --check-pytorch
```
| Verdict | Meaning | Action |
|------------|---------------------------------------------------------------|--------|
| `ok` | ≥8 GB VRAM (discrete) OR ≥32 GB unified (Apple Silicon) | Local install — use `comfy_cli_flag` from report |
| `marginal` | SD1.5 works; SDXL tight; Flux/video unlikely | Local OK for light workflows, else **Path A (Cloud)** |
| `cloud` | No usable GPU, <6 GB VRAM, <16 GB Apple unified, Intel Mac, Rosetta Python | **Switch to Cloud** unless user explicitly forces local |
The script also surfaces `wsl: true` (WSL2 with NVIDIA passthrough) and
`rosetta: true` (x86_64 Python on Apple Silicon — must reinstall as ARM64).
If verdict is `cloud` but the user wants local, do not proceed silently.
Show the `notes` array verbatim and ask whether they want to (a) switch to
Cloud or (b) force a local install (will OOM or be unusably slow on modern models).
### Choosing an Installation Path
Use the hardware check first. The table below is the fallback for when the
user has already told you their hardware:
| Situation | Recommended Path |
|-----------|------------------|
| `verdict: cloud` from hardware check | **Path A: Comfy Cloud** |
| No GPU / want to try without commitment | **Path A: Comfy Cloud** |
| Windows + NVIDIA + non-technical | **Path B: ComfyUI Desktop** |
| Windows + NVIDIA + technical | **Path C: Portable** or **Path D: comfy-cli** |
| Linux + any GPU | **Path D: comfy-cli** (easiest) |
| macOS + Apple Silicon | **Path B: Desktop** or **Path D: comfy-cli** |
| Headless / server / CI / agents | **Path D: comfy-cli** |
For the fully automated path (hardware check → install → launch → verify):
```bash
bash scripts/comfyui_setup.sh
# Or with overrides:
bash scripts/comfyui_setup.sh --m-series --port=8190 --workspace=/data/comfy
```
It runs `hardware_check.py` internally, refuses to install locally when the
verdict is `cloud` (unless `--force-cloud-override`), picks the right
`comfy-cli` flag, and prefers `pipx`/`uvx` over global `pip` to avoid polluting
system Python.
---
### Path A: Comfy Cloud (No Local Install)
For users without a capable GPU or who want zero setup. Hosted on RTX 6000 Pro.
**Docs:** https://docs.comfy.org/get_started/cloud
1. Sign up at https://comfy.org/cloud
2. Generate an API key at https://platform.comfy.org/login
3. Set the key:
```bash
export COMFY_CLOUD_API_KEY="comfyui-xxxxxxxxxxxx"
```
4. Run workflows:
```bash
python3 scripts/run_workflow.py \
--workflow workflows/flux_dev_txt2img.json \
--args '{"prompt": "..."}' \
--host https://cloud.comfy.org \
--output-dir ./outputs
```
**Pricing:** https://www.comfy.org/cloud/pricing
**Concurrent jobs:** Free/Standard 1, Creator 3, Pro 5. Free tier
**cannot run workflows via API** — only browse models. Paid subscription
required for `/api/prompt`, `/api/upload/*`, `/api/view`, etc.
---
### Path B: ComfyUI Desktop (Windows / macOS)
One-click installer for non-technical users. Currently Beta.
**Docs:** https://docs.comfy.org/installation/desktop
- **Windows (NVIDIA):** https://download.comfy.org/windows/nsis/x64
- **macOS (Apple Silicon):** https://comfy.org
Linux is **not supported** for Desktop — use Path D.
---
### Path C: ComfyUI Portable (Windows Only)
**Docs:** https://docs.comfy.org/installation/comfyui_portable_windows
Download from https://github.com/comfyanonymous/ComfyUI/releases, extract,
run `run_nvidia_gpu.bat`. Update via `update/update_comfyui_stable.bat`.
---
### Path D: comfy-cli (All Platforms — Recommended for Agents)
The official CLI is the best path for headless/automated setups.
**Docs:** https://docs.comfy.org/comfy-cli/getting-started
#### Install comfy-cli
```bash
# Recommended:
pipx install comfy-cli
# Or use uvx without installing:
uvx --from comfy-cli comfy --help
# Or (if pipx/uvx unavailable):
pip install --user comfy-cli
```
Disable analytics non-interactively:
```bash
comfy --skip-prompt tracking disable
```
#### Install ComfyUI
```bash
comfy --skip-prompt install --nvidia # NVIDIA (CUDA)
comfy --skip-prompt install --amd # AMD (ROCm, Linux)
comfy --skip-prompt install --m-series # Apple Silicon (MPS)
comfy --skip-prompt install --cpu # CPU only (slow)
comfy --skip-prompt install --nvidia --fast-deps # uv-based dep resolution
```
Default location: `~/comfy/ComfyUI` (Linux), `~/Documents/comfy/ComfyUI`
(macOS/Win). Override with `comfy --workspace /custom/path install`.
#### Launch / verify
```bash
comfy launch --background # background daemon on :8188
comfy launch -- --listen 0.0.0.0 --port 8190 # LAN-accessible custom port
curl -s http://127.0.0.1:8188/system_stats # health check
```
---
### Path E: Manual Install (Advanced / Unsupported Hardware)
For Ascend NPU, Cambricon MLU, Intel Arc, or other unsupported hardware.
**Docs:** https://docs.comfy.org/installation/manual_install
```bash
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130
pip install -r requirements.txt
python main.py
```
---
### Post-Install: Download Models
### Classic Checkpoints (safetensors)
```bash
# SDXL (general purpose, ~6.5 GB)
comfy model download \
--url "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors" \
--relative-path models/checkpoints
# SD 1.5 (lighter, ~4 GB, good for 6 GB cards)
comfy model download \
--url "https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors" \
--relative-path models/checkpoints
# Flux Dev fp8 (smaller variant, ~12 GB)
comfy model download \
--url "https://huggingface.co/Comfy-Org/flux1-dev/resolve/main/flux1-dev-fp8.safetensors" \
--relative-path models/checkpoints
# CivitAI (set token first):
comfy model download \
--url "https://civitai.com/api/download/models/128713" \
--relative-path models/checkpoints \
--set-civitai-api-token "YOUR_TOKEN"
```
### FLUX Family — GGUF quantized (VRAM-friendly)
Black Forest Labs models are **gated on HuggingFace** (login + license acceptance required). Use **community GGUF quantizations** for direct download.
| Model | Q-Level | Size | Source (no-auth) |
|-------|---------|------|-----------------|
| FLUX.1-dev | Q4_K_S | ~6.4 GB | `city96/FLUX.1-dev-gguf` |
| FLUX.2-klein-9B | Q4_K_S | ~5.4 GB | `unsloth/FLUX.2-klein-9B-GGUF` |
| FLUX.2-klein-4B | Q4_K_S | ~? GB | `unsloth/FLUX.2-klein-4B-GGUF` |
Example:
```bash
curl -L -o models/unet/flux-2-klein-9b-Q4_K_S.gguf \
"https://huggingface.co/unsloth/FLUX.2-klein-9B-GGUF/resolve/main/flux-2-klein-9b-Q4_K_S.gguf"
```
# Required text encoders for FLUX
**FLUX.1 (dev / schnell):** shares CLIP-L + T5XXL:
```bash
# CLIP-L
curl -L -o models/clip/clip_l.safetensors \
"https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors"
# T5-XXL fp8 (Q4 quant, ~4.9 GB)
curl -L -o models/clip/t5xxl_fp8_e4m3fn.safetensors \
"https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp8_e4m3fn.safetensors"
```
**FLUX.2-klein-9B:** uses an **8B Qwen3-VL text embedder** (NOT CLIP-L + T5XXL). These models are **incompatible** with ComfyUI's `DualCLIPLoaderGGUF` and its FLUX.1 nodes. Attempting to run FLUX.2 with the FLUX.1 CLIP/VAE pipeline produces:
```
RuntimeError: mat1 and mat2 shapes cannot be multiplied (512x4096 and 12288x4096)
```
**FLUX.2 VAE is NOT shared with FLUX.1.** FLUX.2 emits 128-channel latents (`latent_channels = 128`), while the FLUX.1 VAE (`ae.safetensors`) expects 16 channels. Using the wrong VAE produces:
```
RuntimeError: Given groups=1, weight of size [512, 16, 3, 3], expected input[1, 128, 32, 32] to have 16 channels, but got 128 channels instead
```
**Required FLUX.2 components:**
| Component | File | Size | Source |
|-----------|------|------|--------|
| UNet (GGUF) | `flux-2-klein-9b-Q4_K_S.gguf` | ~5.4 GB | `unsloth/FLUX.2-klein-9B-GGUF` |
| Text Encoder (GGUF) | `Qwen3VL-8B-Instruct-Q4_K_M.gguf` | ~4.7 GB | `Qwen/Qwen3-VL-8B-Instruct-GGUF` |
| VAE option A | `flux2_vae.safetensors` | ~160 MB | `black-forest-labs/FLUX.2-klein-9B` (gated) |
| VAE option B | `taef2` (TAE decoder) | ~2.6 MB | `madebyollin/taesd` (public, lower quality) |
**Node workflow for FLUX.2 (ComfyUI API format):**
```json
{
"1": {"inputs": {"unet_name": "flux-2-klein-9b-Q4_K_S.gguf"}, "class_type": "UnetLoaderGGUF"},
"2": {"inputs": {"clip_name": "Qwen3VL-8B-Instruct-Q4_K_M.gguf", "type": "flux2"}, "class_type": "CLIPLoaderGGUF"},
"3": {"inputs": {"vae_name": "taef2"}, "class_type": "VAELoader"},
"4": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
"5": {"inputs": {"text": "prompt here", "clip": ["2", 0]}, "class_type": "CLIPTextEncode"},
"6": {"inputs": {"text": "", "clip": ["2", 0]}, "class_type": "CLIPTextEncode"},
"7": {"inputs": {"seed": 42, "steps": 4, "cfg": 1.0, "sampler_name": "euler", "scheduler": "simple", "denoise": 1.0, "model": ["1", 0], "positive": ["5", 0], "negative": ["6", 0], "latent_image": ["4", 0]}, "class_type": "KSampler"},
"8": {"inputs": {"samples": ["7", 0], "vae": ["3", 0]}, "class_type": "VAEDecode"},
"9": {"inputs": {"filename_prefix": "flux2_out", "images": ["8", 0]}, "class_type": "SaveImage"}
}
```
Key differences from FLUX.1:
- Use `CLIPLoaderGGUF` (single) with `type: "flux2"`, NOT `DualCLIPLoaderGGUF`
- Use `EmptyFlux2LatentImage` (128 channels), NOT `EmptyLatentImage`
- Use `taef2` or dedicated FLUX.2 VAE, NOT `ae.safetensors`
- Default: 4 steps, cfg=1.0, euler/simple scheduler
See `references/flux2-setup.md` for full reproduction recipe, ROCm gfx1150 workaround, and performance notes.
**VAE (shared across FLUX.1 and FLUX.2):**
```bash
# Primary source (may require auth for BFL repos)
curl -L -o models/vae/ae.safetensors \
"https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors"
# Mirror without auth (ModelScope)
curl -L -o models/vae/ae.safetensors \
"https://www.modelscope.cn/models/AI-ModelScope/FLUX.1-dev/resolve/master/ae.safetensors"
```
List installed: `comfy model list`.
### Post-Install: Install Custom Nodes
```bash
comfy node install comfyui-impact-pack # popular utility pack
comfy node install comfyui-animatediff-evolved # video generation
comfy node install comfyui-controlnet-aux # ControlNet preprocessors
comfy node install comfyui-essentials # common helpers
comfy node update all
comfy node install-deps --workflow=workflow.json # install everything a workflow needs
```
### Post-Install: Verify
```bash
python3 scripts/health_check.py
# → comfy_cli on PATH? server reachable? checkpoints? smoke test?
python3 scripts/check_deps.py my_workflow.json
# → are this workflow's nodes/models/embeddings installed?
python3 scripts/run_workflow.py \
--workflow workflows/sd15_txt2img.json \
--args '{"prompt": "test", "steps": 4}' \
--output-dir ./test-outputs
```
## Image Upload (img2img / Inpainting)
The simplest way is to use `--input-image` with `run_workflow.py`:
```bash
python3 scripts/run_workflow.py \
--workflow workflows/sdxl_img2img.json \
--input-image image=./photo.png \
--args '{"prompt": "make it cyberpunk", "denoise": 0.6}'
```
The flag uploads `photo.png`, then injects its server-side filename into
whatever schema parameter is named `image`. For inpainting, pass both:
```bash
python3 scripts/run_workflow.py \
--workflow workflows/sdxl_inpaint.json \
--input-image image=./photo.png \
--input-image mask_image=./mask.png \
--args '{"prompt": "fill with flowers"}'
```
Manual upload via REST:
```bash
curl -X POST "http://127.0.0.1:8188/upload/image" \
-F "image=@photo.png" -F "type=input" -F "overwrite=true"
# Returns: {"name": "photo.png", "subfolder": "", "type": "input"}
# Cloud equivalent:
curl -X POST "https://cloud.comfy.org/api/upload/image" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY" \
-F "image=@photo.png" -F "type=input" -F "overwrite=true"
```
## Cloud Specifics
- **Base URL:** `https://cloud.comfy.org`
- **Auth:** `X-API-Key` header (or `?token=KEY` for WebSocket)
- **API key:** set `$COMFY_CLOUD_API_KEY` once and the scripts pick it up automatically
- **Output download:** `/api/view` returns a 302 to a signed URL; the scripts
follow it and strip `X-API-Key` before fetching from the storage backend
(don't leak the API key to S3/CloudFront).
- **Endpoint differences from local ComfyUI:**
- `/api/object_info`, `/api/queue`, `/api/userdata` — **403 on free tier**;
paid only.
- `/history` is renamed to `/history_v2` on cloud (the scripts route
automatically).
- `/models/<folder>` is renamed to `/experiment/models/<folder>` on cloud
(the scripts route automatically).
- `clientId` in WebSocket is currently ignored — all connections for a
user receive the same broadcast. Filter by `prompt_id` client-side.
- `subfolder` is accepted on uploads but ignored — cloud has a flat namespace.
- **Concurrent jobs:** Free/Standard: 1, Creator: 3, Pro: 5. Extras queue
automatically. Use `run_batch.py --parallel N` to saturate your tier.
## Queue & System Management
```bash
# Local
curl -s http://127.0.0.1:8188/queue | python3 -m json.tool
curl -X POST http://127.0.0.1:8188/queue -d '{"clear": true}' # cancel pending
curl -X POST http://127.0.0.1:8188/interrupt # cancel running
curl -X POST http://127.0.0.1:8188/free \
-H "Content-Type: application/json" \
-d '{"unload_models": true, "free_memory": true}'
# Cloud — same paths under /api/, plus:
python3 scripts/fetch_logs.py --tail-queue --host https://cloud.comfy.org
```
## Pitfalls
1. **API format required** — every script and the `/api/prompt` endpoint expect
API-format workflow JSON. The scripts detect editor format (top-level
`nodes` and `links` arrays) and tell you to re-export via
"Workflow → Export (API)" (newer UI) or "Save (API Format)" (older UI).
2. **Server must be running** — all execution requires a live server.
`comfy launch --background` starts one. Verify with
`curl http://127.0.0.1:8188/system_stats`.
3. **Model names are exact** — case-sensitive, includes file extension.
`check_deps.py` does fuzzy matching (with/without extension and folder
prefix), but the workflow itself must use the canonical name. Use
`comfy model list` to discover what's installed.
4. **Missing custom nodes** — "class_type not found" means a required node
isn't installed. `check_deps.py` reports which package to install;
`auto_fix_deps.py` runs the install for you.
5. **Working directory** — `comfy-cli` auto-detects the ComfyUI workspace.
If commands fail with "no workspace found", use
`comfy --workspace /path/to/ComfyUI <command>` or
`comfy set-default /path/to/ComfyUI`.
6. **Cloud free-tier API limits** — `/api/prompt`, `/api/view`, `/api/upload/*`,
`/api/object_info` all return 403 on free accounts. `health_check.py` and
`check_deps.py` handle this gracefully and surface a clear message.
7. **Timeout for video/audio workflows** — auto-detected when an output node
is `VHS_VideoCombine`, `SaveVideo`, etc.; the default jumps from 300 s to
900 s. Override explicitly with `--timeout 1800`.
8. **Path traversal in output filenames** — server-supplied filenames are
passed through `safe_path_join` to refuse anything escaping `--output-dir`.
Keep this protection on — workflows with custom save nodes can produce
arbitrary paths.
9. **Workflow JSON is arbitrary code** — custom nodes run Python, so
submitting an unknown workflow has the same trust profile as `eval`.
Inspect workflows from untrusted sources before running.
10. **`comfyui.service` unit already exists** — Before creating a new systemd service for ComfyUI, check `/etc/systemd/system/comfyui.service`; it may already be installed but disabled or stopped. Inspect it first with `cat /etc/systemd/system/comfyui.service` rather than overwriting blindly.
11. **Auto-randomized seed** — pass `seed: -1` in `--args` (or use
`--randomize-seed` and omit the seed) to get a fresh seed per run.
The actual seed is logged to stderr.
12. **Ideogram 4 has TWO incompatible runtime paths** — do not confuse them:
- **IdeogramV1/V2/V3/V4** — `comfy_api_nodes` cloud API nodes. Require a `comfy.org` login / API key. These call the Ideogram SaaS API and do NOT use local models. Error: `Unauthorized: Please login first to use this node.`
- **Ideogram4Scheduler + local UNet/CLIP/VAE** — purely local pipeline. Uses `CLIPLoader` with `type: "ideogram4"`, `EmptyFlux2LatentImage`, `Ideogram4Scheduler` feeding `SamplerCustomAdvanced`, plus `CFGOverride` + `DualModelGuider` for asymmetric classifier-free guidance. Do NOT use plain `BasicGuider` or standard `KSampler` — they lack the dual-model wiring. See `references/ideogram4.md` for node requirements.
13. **DualModelGuider uses `model_negative`, NOT `model_1`** — in the API-format workflow, the DualModelGuider node's unconditional model input is named `model_negative` (not `model_1`). Using the wrong key name produces: `TypeError: DualModelGuider.execute() got an unexpected keyword argument 'model_1'`.
14. **LLM gateways ≠ image backends** — tools like Bifrost route
`/v1/chat/completions`, not `/v1/images/generations`. They have no ComfyUI
provider and cannot translate OpenAI image requests to ComfyUI workflow JSON.
Connecting ComfyUI to chat frontends (OpenWebUI, etc.) requires either the
frontends' native ComfyUI connector (limited to SD/SDXL, breaks on FLUX.2) or
a custom OpenAI-compatible adapter that submits workflow JSON to
`POST /api/prompt`. See `references/openwebui-integration.md`.
15. **Shell-escaping hell when driving LXC over SSH** — If your ComfyUI lives
inside a Proxmox LXC (e.g. CT 204 on 10.0.30.97), avoid multi-layer
nested quotes (`ssh` → `pct exec` → `python3 -c`). They break on `!`, `(`
and `"`. Instead write a script file on the PVE host, `pct push` it into
the container, then `pct exec` it. See `references/proxmox-lxc.md`.
16. **Ideogram 4: API vs. Local** — `IdeogramV1`-`V4` nodes require a `comfy.org` API key (`Unauthorized` without auth). For local execution, use `Ideogram4Scheduler` + download ~28 GB of models (`ideogram4_fp8_scaled`, `ideogram4_unconditional_fp8_scaled`, `flux2-vae`, `qwen3vl_8b_fp8_scaled`). See `references/ideogram4-local-setup.md`.
17. **Safetensors "incomplete metadata" even with correct file size** — a `.safetensors` file can report the expected byte count in `ls -lh` but still be corrupt if the header's `data_offsets` don't match the actual payload boundaries. Always verify with a header-content-size check (see `references/ideogram4-local-setup.md` for the verification recipe).
18. **Proxmox `pct exec` nested quoting hell** — when pushing scripts into an LXC container via `pct exec` or `sshpass`, avoid inline Python with single quotes. Instead write a script to a temp file via `pct push`, then execute it:
```bash
pct push CTID /local/script.py /tmp/script.py
pct exec CTID -- python3 /tmp/script.py
```
This avoids the `"\''\"$` escape maze that breaks multiline Python strings.
19. **Editor-format blueprints are NOT API format** — ComfyUI's `blueprints/*.json` files (e.g. `Text to Image (Ideogram v4).json`) use the editor format (`"nodes"` / `"links"` arrays). They must be loaded into ComfyUI and re-exported as API format before execution via `POST /api/prompt`. The REST endpoint rejects editor format silently or with cryptic errors.
20. **Switching from manual/nohup to systemd blocks ports** — If ComfyUI or the API adapter was previously started via `nohup python3 main.py &` or `python3 adapter.py &`, the processes survive backgrounding and continue holding ports 8188 / 8000. A subsequent `systemctl start` will fail with `address already in use`. Always terminate the manual processes first (`kill -9 <pid>` or `fuser -k <port>/tcp`) before enabling systemd units.
21. **systemd `ExecStart` filename must match actual file on disk** — When templating a systemd service for the API adapter, the source code may be saved as `adapter.py`, `main.py`, or `app.py`. Check `ls /opt/ideogram4-api/` before writing `ExecStart=/opt/ideogram4-api/venv/bin/python /opt/ideogram4-api/xxx.py`; a mismatch produces `can't open file`.
22. **Model families need DIFFERENT loader nodes** — Ideogram4 uses `UNETLoader` for `.safetensors`. FLUX GGUF models (`*.gguf`) require `UnetLoaderGGUF` from the `ComfyUI-GGUF` custom node. Using `UNETLoader` for a GGUF file produces `value_not_in_list` because the safetensors loader doesn't know `.gguf`. Using `UnetLoaderGGUF` for a safetensors file also fails. Never globally replace one loader name for another — patch per-workflow instead.
23. **ComfyUI on AMD ROCm needs `HSA_OVERRIDE_GFX_VERSION`** — When running under systemd (not an interactive shell), `HSA_OVERRIDE_GFX_VERSION` is NOT inherited from any bashrc or manual export. ComfyUI will crash during text-encode with `RuntimeError` (hipBLASLt architecture mismatch). The fix is adding `Environment=HSA_OVERRIDE_GFX_VERSION=11.0.0` (or your GPU's gfx string) to the `[Service]` section of `comfyui.service`.
24. **LLM gateways ≠ image backends** — tools like Bifrost route `/v1/chat/completions`, not `/v1/images/generations`. They have no ComfyUI provider and cannot translate OpenAI image requests to ComfyUI workflow JSON. Connecting ComfyUI to chat frontends (OpenWebUI, etc.) requires either the frontends' native ComfyUI connector (limited to SD/SDXL, breaks on FLUX.2) or a custom OpenAI-compatible adapter that submits workflow JSON to `POST /api/prompt`. See `references/openwebui-integration.md` and `references/openai-api-wrapper.md`.
25. **API timeout for long generations** — set the adapter's HTTP poll loop and the FastAPI `timeout` to at least 1800 s (30 min) for high-resolution or multi-model workloads. FLUX dev at 1024×1024 with 50 steps can take 10–15 min on ROCm; larger resolutions or multiple `n` images may approach 30 min. The adapter polls ComfyUI every 5 s; the total loop iterations must cover the worst case. In the Python source, increase `for i in range(360)` → `for i in range(720)` (30 min at 5 s intervals) and set `httpx` timeout > 2400 s on the initial `/prompt` POST.
26. **Model loader type must match file format** — never do a global `sed` replacing `UNETLoader` with `UnetLoaderGGUF` (or vice versa). Ideogram4 uses `.safetensors` loaded by `UNETLoader`; FLUX GGUF models use `UnetLoaderGGUF`. A global replacement breaks the other pipeline. Patch per-workflow, not per-file.
27. **Ideogram4 ships WITHOUT an active safety classifier** — unlike commercial APIs (Midjourney, DALL-E, Ideogram Cloud) that block prompts with an explicit NSFW score, the local `Ideogram4Scheduler` pipeline has no classifier. Generations are rejected silently or produce blank/gray output only because the **model's internal attention collapses** at very low step counts (≤8). This looks like censorship but is actually numerical underflow. Fix: enforce a minimum of ~12 steps for Ideogram4. There is no safety log to grep for; safety is absent, not logged.
28. **Ideogram4 `BasicGuider` in OpenAI adapter produces inferior quality** — The template adapter at `templates/comfyui-openai-adapter.py` originally used `BasicGuider` for Ideogram4 (node `"9"` in `_build_ideogram4_workflow`). This is wrong: Ideogram4 requires **asymmetric CFG wiring** via `CFGOverride` + `DualModelGuider` + a separate unconditional UNet (`ideogram4_unconditional_fp8_scaled`) + `ConditioningZeroOut`. Using `BasicGuider` silently degrades quality but does NOT throw an error. The correct wiring is: `UNETLoader(ideogram4)` → `CFGOverride` → `DualModelGuider(model=cfg_overridden, model_negative=unconditional_unet, positive=prompt_cond, negative=zeroed_cond)` → `SamplerCustomAdvanced`. See `references/ideogram4.md` for the full API-format blueprint.
29. **Gray-block detection via image variance** — When Ideogram4 produces blank or uniform gray output (especially at `quality=low` with `style=photorealistic`), the PNG file is still valid but the image has near-zero content variance. A quick automated check using PIL: `ImageStat.Stat(img).var` — if all channels have variance < 15, the generation failed. Healthy images show variance > 40 (typically 50–150 for detailed scenes). This is a reliable proxy for "did the model actually render content" and can gate retries or alerts in automated pipelines. See `scripts/comfy_e2e_test.py` for implementation.
30. **Systematic E2E testing: constant prompt across all parameter combinations** — When validating a new adapter, model, or parameter mapping, use a **single fixed prompt** (e.g. `"a majestic mountain landscape at sunset"`) across all combinations of `model × quality × style`. This isolates parameter effects from prompt variance. Do NOT vary the prompt per model — it confounds the results. Store outputs in a timestamped directory, compute variance per image, and flag outliers (< 15 variance) as failures. The watcher script `scripts/comfy_e2e_watcher.sh` reports completion status and reproduces the prompt for traceability.
28. **OpenAI model list must stay consistent with ALLOWED_MODELS** — If a validation set (`ALLOWED_MODELS = {"ideogram4", "flux2"}`) rejects `flux1-dev` and `flux1-schnell` at generation time, the `GET /v1/models` endpoint must reflect the same restriction. A hardcoded list_models() returning all four models causes frontend confusion. Either make it dynamic (`for m in sorted(ALLOWED_MODELS)`) or keep it manually in sync.
## Verification Checklist
Use `python3 scripts/health_check.py` to run the whole list at once. Manual:
- [ ] `hardware_check.py` verdict is `ok` OR the user explicitly chose Comfy Cloud
- [ ] `comfy --version` works (or `uvx --from comfy-cli comfy --help`)
- [ ] `curl http://HOST:PORT/system_stats` returns JSON
- [ ] `comfy model list` shows at least one checkpoint (local) OR
`/api/experiment/models/checkpoints` returns models (cloud)
- [ ] Workflow JSON is in API format
- [ ] `check_deps.py` reports `is_ready: true` (or only `node_check_skipped`
on cloud free tier)
- [ ] Test run with a small workflow completes; outputs land in `--output-dir`
creative/comfyui/references/flux-models-download.md
+131
View File
@@ -0,0 +1,131 @@
# FLUX Model Download Reference for ComfyUI on ROCm
Quick reference for downloading FLUX models for local ComfyUI inference on AMD ROCm (no HuggingFace login).
## Official FLUX Models from Black Forest Labs
| Model | Params | Open? | Size (FP16) |
|-------|--------|-------|-------------|
| FLUX.1 [dev] | 12B | ✅ | ~23 GB |
| FLUX.1 [schnell] | 12B | ✅ | ~23 GB |
| FLUX.1 [pro] | ? | ❌ (API only) | — |
| **FLUX.2 [dev]** | ? | ✅ | 16.9 GB |
| **FLUX.2 [klein-9B]** | 9B | ✅ | 16.9 GB |
| **FLUX.2 [klein-4B]** | 4B | ✅ | ~3.7 GB (Q4) |
The `klein` variants are faster, smaller variants of FLUX.2.
## GGUF Quantizations (No Auth Required)
When HuggingFace requires login (gated models), use community GGUF repos:
**Unsloth GGUF repo (unsloth/FLUX.2-klein-9B-GGUF):**
- `flux-2-klein-9b-Q4_K_S.gguf` — **5.43 GB** ← best balance for most GPUs
- `flux-2-klein-9b-Q4_K_M.gguf` — 5.50 GB
- `flux-2-klein-9b-Q4_0.gguf` — 5.23 GB
- `flux-2-klein-9b-Q5_K_S.gguf` — 6.46 GB
- `flux-2-klein-9b-Q8_0.gguf` — 9.29 GB
**Leejet GGUF repo (leejet/FLUX.2-klein-9B-GGUF):**
- `flux-2-klein-9b-Q4_0.gguf` — 5.23 GB
- `flux-2-klein-9b-Q8_0.gguf` — 9.29 GB (fewer options than unsloth)
**FLUX.1-dev GGUF repo (city96/FLUX.1-dev-gguf or similar):**
- `flux1-dev-Q4_K_S.gguf` — **6.4 GB**
## Authentication Barriers
| Source | Requires Auth? | Notes |
|--------|----------------|-------|
| `black-forest-labs/*` (HuggingFace) | ✅ Yes (gated) | Must login + accept license |
| `unsloth/*-GGUF` | ❌ No | Community quant, direct download |
| `leejet/*-GGUF` | ❌ No | Community quant, direct download |
| `civitai.com` | ❌ No (most) | Use `api/download/models/<id>?type=Model&format=SafeTensor` |
| `modelscope.cn` | ❌ No | China mirror, works for FLUX files |
**If a download returns a tiny (~140 byte) file:** The request was redirected to a login page or error page. The HTML response is ~140 bytes. Always verify the downloaded file size matches expectations.
## Required Auxiliary Models
**Shared VAE (all FLUX models):**
| File | Size | Purpose | Where to get (no auth) |
|------|------|---------|------------------------|
| `ae.safetensors` (VAE) | 319 MB | Image decoder | **modelscope.cn** (HuggingFace gated) |
**Text encoders are NOT shared:**
- **FLUX.1 (dev / schnell):** CLIP-L + T5XXL → `clip_l.safetensors` + `t5xxl_fp8_e4m3fn.safetensors`
- **FLUX.2-klein-9B:** Uses **8B Qwen3 text embedder** (NOT CLIP-L / T5XXL). Incompatible with ComfyUI's `DualCLIPLoaderGGUF`. Attempting to run FLUX.2 with FLUX.1 encoders produces `RuntimeError: mat1 and mat2 shapes cannot be multiplied (512x4096 and 12288x4096)`. Use diffusers `Flux2KleinPipeline` or find ComfyUI Qwen3 nodes.
| File | Size | Purpose | Where to get | Required for |
|------|------|---------|-------------|-------------|
| `clip_l.safetensors` | 235 MB | CLIP text encoder | civitai or modelscope | FLUX.1 only |
| `t5xxl_fp8_e4m3fn.safetensors` | 4.9 GB | T5 XXL text encoder | civitai or modelscope | FLUX.1 only |
### Working VAE download URLs (no-auth)
```bash
# Modelscope (verified working, no auth)
curl -L -o ae.safetensors \
"https://www.modelscope.cn/models/AI-ModelScope/FLUX.1-dev/resolve/master/ae.safetensors"
# Direct HuggingFace (REQUIRES login for BFL repos)
# curl -L -o ae.safetensors \
# "https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors"
# → Will fail with 401, producing a ~140 byte HTML error page
```
## Background Download Pattern
For multi-GB model downloads from inside a Proxmox LXC, use `nohup` to survive SSH/session timeouts:
```bash
pct exec 204 -- bash -c '
cd /opt/ComfyUI/models/unet
nohup bash -c "curl -L -o flux-2-klein-9b-Q4_K_S.gguf \\"
https://huggingface.co/unsloth/FLUX.2-klein-9B-GGUF/resolve/main/flux-2-klein-9b-Q4_K_S.gguf \\"
> /tmp/flux2_download.log 2>&1" &
'
```
Verify ongoing download:
```bash
pct exec 204 -- bash -c 'ls -lh /opt/ComfyUI/models/unet/flux-2-klein-9b-Q4_K_S.gguf'
pct exec 204 -- bash -c 'tail -3 /tmp/flux2_download.log'
```
## ComfyUI Model Paths
```
/opt/ComfyUI/models/
├── unet/ # GGUF diffusion models (FLUX, SDXL, etc.)
│ └── flux-2-klein-9b-Q4_K_S.gguf
├── clip/ # Text encoders
│ ├── clip_l.safetensors
│ └── t5xxl_fp8_e4m3fn.safetensors
├── vae/ # Image decoders
│ └── ae.safetensors
└── checkpoints/ # Full safetensors models
```
## ComfyUI + ROCm + GGUF Verification
After downloading, verify ComfyUI can load the model:
```bash
# Start ComfyUI
pct exec 204 -- bash -c 'cd /opt/ComfyUI && python3 main.py --listen 0.0.0.0 --port 8188'
# Check API responds
curl http://10.0.20.91:8188/system_stats
# The first run will compile shaders; monitor /tmp/comfyui.log
pct exec 204 -- bash -c 'tail -20 /tmp/comfyui.log'
```
## Errors and Fixes
| Error | Cause | Fix |
|-------|-------|-----|
| `hipErrorNoBinaryForGpu` | PyTorch ROCm wheel mismatch with host ROCm | Downgrade/upgrade PyTorch wheel: `pip install torch --index-url https://download.pytorch.org/whl/rocm6.2` |
| Downloaded file is ~140 bytes | Gated model, no auth token | Switch to unsloth/leejet GGUF repos or modelscope |
| Downloaded file is 6+ GB for VAE | Wrong CivitAI URL (model page, not direct download) | Use correct `api/download/models/<id>?type=Model&format=SafeTensor` URL |
| `curl` times out during download | Paramiko SSH timeout | Use `nohup` + background; poll file size separately |
creative/comfyui/references/flux2-qwen3-compatibility.md
+69
View File
@@ -0,0 +1,69 @@
# FLUX.2 in ComfyUI — Compatibility Guide
> **Status: Native ComfyUI support confirmed as of June 2026.**
> See `references/flux2-setup.md` for the complete reproduction recipe, model downloads, and workflow JSON.
## The Problem (Historical)
FLUX.2-klein (9B) uses an **8B Qwen3-VL text embedder**, replacing the CLIP-L + T5XXL combo used by FLUX.1. Early ComfyUI versions (pre-mid 2026) lacked native nodes for FLUX.2, forcing users to:
- Use the diffusers `Flux2KleinPipeline` instead, or
- Wait for community custom nodes
This is no longer necessary. ComfyUI now includes:
- `CLIPType.FLUX2` enum (line 1299 in `comfy/sd.py`)
- `Flux2` model class (line 1057 in `comfy/model_base.py`)
- `EmptyFlux2LatentImage` node (128-channel latents)
- `Flux2Scheduler` node (mu-snr shifted timesteps)
- `Flux2TEModel` text encoder wrapper with Qwen3VL support
- `KleinTokenizer` / `KleinTokenizer8B` tokenizers
## Error Signature → Fix Mapping
| Error | Cause | Fix |
|-------|-------|-----|
| `mat1 and mat2 shapes cannot be multiplied (512x4096 and 12288x4096)` | Using CLIP-L+T5XXL with FLUX.2 UNet | Load `Qwen3VL-8B-Instruct-Q4_K_M.gguf` via `CLIPLoaderGGUF` with `"type": "flux2"` |
| `expected input[1, 128, 32, 32] to have 16 channels, but got 128 channels instead` | Using FLUX.1 VAE (`ae.safetensors`) | Use `taef2` or `flux2_vae.safetensors` |
| `HIP error: invalid device function` | ROCm gfx1150 without override | `export HSA_OVERRIDE_GFX_VERSION=11.0.0` |
## Required Models
| Component | File | ~Size | Source |
|-----------|------|-------|--------|
| UNet (GGUF) | `flux-2-klein-9b-Q4_K_S.gguf` | 5.4 GB | `unsloth/FLUX.2-klein-9B-GGUF` |
| Text Encoder | `Qwen3VL-8B-Instruct-Q4_K_M.gguf` | 4.7 GB | `Qwen/Qwen3-VL-8B-Instruct-GGUF` |
| VAE (fast) | `taef2` encoder+decoder `.pth` | 5 MB | `madebyollin/taesd` (GitHub) |
| VAE (full) | `flux2_vae.safetensors` | 160 MB | `black-forest-labs/FLUX.2-klein-9B` (gated) |
**FLUX.1 and FLUX.2 are NOT component-compatible.** Do not mix:
- ❌ FLUX.2 UNet + FLUX.1 CLIP/T5
- ❌ FLUX.2 latents + FLUX.1 VAE (`ae.safetensors`)
- ❌ `EmptyLatentImage` with FLUX.2 (needs `EmptyFlux2LatentImage`)
## Workflow Differences (FLUX.1 vs FLUX.2)
```json
// FLUX.1 — two CLIP loaders + shared VAE
{"2": {"inputs": {"clip_name1": "t5xxl_fp8_e4m3fn.safetensors", "clip_name2": "clip_l.safetensors", "type": "flux"}, "class_type": "DualCLIPLoaderGGUF"}}
{"4": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyLatentImage"}}
{"3": {"inputs": {"vae_name": "ae.safetensors"}, "class_type": "VAELoader"}}
// FLUX.2 — single Qwen3 CLIP + TAEF2 VAE + 128-channel latent
{"2": {"inputs": {"clip_name": "Qwen3VL-8B-Instruct-Q4_K_M.gguf", "type": "flux2"}, "class_type": "CLIPLoaderGGUF"}}
{"4": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"}}
{"3": {"inputs": {"vae_name": "taef2"}, "class_type": "VAELoader"}}
```
## Verified E2E Results (gfx1150, ROCm 6.2)
| Pipeline | Res | Steps | Time | Status |
|----------|-----|-------|------|--------|
| FLUX.1-dev Q4_K_S + CLIP/T5 + ae.safetensors | 512×512 | 4 | ~46s | ✅ |
| FLUX.2-klein Q4_K_S + Qwen3VL Q4_K_M + taef2 | 1024×1024 | 4 | ~110s | ✅ |
## References
- Full setup recipe: `references/flux2-setup.md`
- Unsloth GGUF repo: https://huggingface.co/unsloth/FLUX.2-klein-9B-GGUF
- Qwen3VL GGUF repo: https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct-GGUF
- BFL blog: https://bfl.ai/blog/flux2-klein-towards-interactive-visual-intelligence
- TAEF2 source: https://github.com/madebyollin/taesd
creative/comfyui/references/flux2-setup.md
+196
View File
@@ -0,0 +1,196 @@
# FLUX.2 Setup in ComfyUI — Full Reproduction Recipe
## Hardware Context (This Session)
- Proxmox CT 204, ROCm 6.2, PyTorch 2.6.0 (ROCm build)
- AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M (gfx1150)
- `HSA_OVERRIDE_GFX_VERSION=11.0.0` required for gfx1150
- 18GB "VRAM" reported; ~8.5 GB disk free after downloads
## Timeline
| Step | Result |
|------|--------|
| FLUX.1-dev E2E | ✅ Works in 46s (512×512, 4 steps) |
| FLUX.2 with FLUX.1 CLIP/T5 | ❌ `mat1 and mat2 shapes cannot be multiplied (512x4096 and 12288x4096)` |
| FLUX.2 with Qwen3VL + ae.safetensors | ❌ `expected input[1, 128, 32, 32] to have 16 channels, but got 128 channels instead` |
| FLUX.2 with Qwen3VL + taef2 | ✅ Success ~110s (1024×1024, 4 steps) |
## Required Models
### 1. UNet (GGUF)
```bash
# ~5.4 GB
wget https://huggingface.co/unsloth/FLUX.2-klein-9B-GGUF/resolve/main/flux-2-klein-9b-Q4_K_S.gguf
# → models/unet/flux-2-klein-9b-Q4_K_S.gguf
```
### 2. Text Encoder (Qwen3-VL-8B, GGUF)
**CRITICAL:** FLUX.2 uses Qwen3-VL-8B, NOT CLIP-L + T5XXL.
```bash
# ~4.7 GB — Q4_K_M quantization
wget https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct-GGUF/resolve/main/Qwen3VL-8B-Instruct-Q4_K_M.gguf
# → models/clip/Qwen3VL-8B-Instruct-Q4_K_M.gguf
```
Alternative: `Qwen3-VL-8B-Instruct-Q8_0.gguf` (~8.3 GB) for higher quality.
The GGUF detection logic in ComfyUI (`comfy/sd.py:detect_te_model`) identifies this model by:
- `model.layers.0.self_attn.q_proj.weight` shape = `[4096, 4096]`
- `num_hidden_layers = 36`
→ maps to `TEModel.QWEN3VL_8B`
### 3. VAE — TWO OPTIONS
**Option A: TAEF2 (Tiny AutoEncoder, fast, lower quality)**
```bash
# ~5 MB total, public, no auth needed
cd models/vae_approx
curl -LO https://github.com/madebyollin/taesd/raw/main/taef2_encoder.pth
curl -LO https://github.com/madebyollin/taesd/raw/main/taef2_decoder.pth
# Reference as vae_name: "taef2" in VAELoader node
```
**Option B: Full FLUX.2 VAE (`flux2_vae.safetensors`)**
```bash
# ~160 MB, gated repo (BFL license acceptance required)
# From: black-forest-labs/FLUX.2-klein-9B vae/diffusion_pytorch_model.safetensors
# Save as: models/vae/flux2_vae.safetensors
```
The FLUX.1 VAE (`ae.safetensors`, 16 channels) is **incompatible** with FLUX.2 (128 channels).
## ComfyUI Workflow (API JSON)
```json
{
"1": {
"inputs": {"unet_name": "flux-2-klein-9b-Q4_K_S.gguf"},
"class_type": "UnetLoaderGGUF"
},
"2": {
"inputs": {
"clip_name": "Qwen3VL-8B-Instruct-Q4_K_M.gguf",
"type": "flux2"
},
"class_type": "CLIPLoaderGGUF"
},
"3": {
"inputs": {"vae_name": "taef2"},
"class_type": "VAELoader"
},
"4": {
"inputs": {"width": 1024, "height": 1024, "batch_size": 1},
"class_type": "EmptyFlux2LatentImage"
},
"5": {
"inputs": {
"text": "A beautiful mountain lake at sunset, photorealistic, 4k",
"clip": ["2", 0]
},
"class_type": "CLIPTextEncode"
},
"6": {
"inputs": {"text": "", "clip": ["2", 0]},
"class_type": "CLIPTextEncode"
},
"7": {
"inputs": {
"seed": 42,
"steps": 4,
"cfg": 1.0,
"sampler_name": "euler",
"scheduler": "simple",
"denoise": 1.0,
"model": ["1", 0],
"positive": ["5", 0],
"negative": ["6", 0],
"latent_image": ["4", 0]
},
"class_type": "KSampler"
},
"8": {
"inputs": {"samples": ["7", 0], "vae": ["3", 0]},
"class_type": "VAEDecode"
},
"9": {
"inputs": {"filename_prefix": "flux2_out", "images": ["8", 0]},
"class_type": "SaveImage"
}
}
```
### Key node differences from FLUX.1
| Node | FLUX.1 | FLUX.2 |
|------|--------|--------|
| CLIP loader | `DualCLIPLoaderGGUF` (clip_l + t5xxl) | `CLIPLoaderGGUF` (single Qwen3VL) |
| CLIP type | `"flux"` | `"flux2"` |
| Empty latent | `EmptyLatentImage` (16 channels) | `EmptyFlux2LatentImage` (128 channels) |
| VAE | `ae.safetensors` | `taef2` or `flux2_vae.safetensors` |
## ROCm gfx1150 Workaround
gfx1150 (RDNA3.5, Radeon 890M) is not fully supported by ROCm 6.2. Set override before launch:
```bash
export HSA_OVERRIDE_GFX_VERSION=11.0.0
python main.py --listen 0.0.0.0 --port 8188
```
Without this, CLIP encoding fails with:
```
RuntimeError: HIP error: invalid device function
```
## Performance (gfx1150, ROCm 6.2)
| Pipeline | Resolution | Steps | Time |
|----------|-----------|-------|------|
| FLUX.1-dev Q4_K_S + CLIP/T5 + ae.safetensors | 512×512 | 4 | ~46s |
| FLUX.2-klein Q4_K_S + Qwen3VL Q4_K_M + taef2 | 1024×1024 | 4 | ~110s |
FLUX.2 is slower despite fewer parameters because Qwen3VL-8B text encoding is heavier than CLIP+T5, and the TAEF2 decode takes extra time.
## Disk Space Budget
| Component | Size |
|-----------|------|
| FLUX.2 UNet Q4_K_S | 5.4 GB |
| Qwen3VL-8B Q4_K_M | 4.7 GB |
| taef2 encoder+decoder | 5 MB |
| **Total** | **~10.1 GB** |
Full FLUX.2 VAE adds ~160 MB. Q8_0 text encoder adds ~3.6 GB more.
## Common Errors
### `mat1 and mat2 shapes cannot be multiplied (512x4096 and 12288x4096)`
**Cause:** Using CLIP-L + T5XXL (FLUX.1 encoders) with FLUX.2 UNet.
**Fix:** Switch to `Qwen3VL-8B-Instruct-Q4_K_M.gguf` + `CLIPLoaderGGUF` with `"type": "flux2"`.
### `expected input[1, 128, 32, 32] to have 16 channels, but got 128 channels instead`
**Cause:** Using FLUX.1 VAE (`ae.safetensors`) with FLUX.2 latents.
**Fix:** Use `taef2` or download dedicated `flux2_vae.safetensors`.
### `HIP error: invalid device function`
**Cause:** ROCm doesn't natively support gfx1150.
**Fix:** `export HSA_OVERRIDE_GFX_VERSION=11.0.0` before starting ComfyUI.
### `RuntimeError: The expanded size of the tensor (512) must match ...`
**Cause:** Using `EmptyLatentImage` (16-channel) instead of `EmptyFlux2LatentImage` (128-channel).
**Fix:** Change node class to `EmptyFlux2LatentImage`.
## References
- ComfyUI `comfy/model_base.py` lines 1057-1066: `class Flux2(Flux)`
- ComfyUI `comfy/latent_formats.py` line 192: `class Flux2(LatentFormat)` with `latent_channels = 128`
- ComfyUI `comfy/sd.py` lines 1631-1634: FLUX2 CLIPType routes to `klein_te` with Qwen3VL
- TAEF2 source: https://github.com/madebyollin/taesd
- Unsloth GGUF: https://huggingface.co/unsloth/FLUX.2-klein-9B-GGUF
- Qwen GGUF: https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct-GGUF
creative/comfyui/references/ideogram4-local-setup.md
+124
View File
@@ -0,0 +1,124 @@
# Ideogram 4 Local Execution in ComfyUI
Multiple paths exist to generate with Ideogram 4. The simplest is the cloud API node (`IdeogramV4`), but it requires a `comfy.org` API key. For local execution without cloud dependency, use the native `Ideogram4Scheduler` + model files.
## Path A: Cloud API Node (Simplest, but requires key)
**Node:** `IdeogramV4`
**Location:** `comfy_api_nodes/nodes_ideogram.py`
**Requires:** `COMFY_CLOUD_API_KEY` env var or login
**Error without auth:**
```
Unauthorized: Please login first to use this node.
```
Works with structured JSON prompt (scene + style + background blocks). The easiest if you have the key.
---
## Path B: Local Native Pipeline (No API key, but 28 GB disk needed)
**Recommended when:** No comfy.org auth available or prefer full local execution.
### Required Model Files
| File | Size | Folder | Source |
|---|---|---|---|
| `ideogram4_fp8_scaled.safetensors` | ~8.6 GB | `models/diffusion_models/` | HuggingFace: `Comfy-Org/Ideogram-4` |
| `ideogram4_unconditional_fp8_scaled.safetensors` | ~8.6 GB | `models/diffusion_models/` | HuggingFace: `Comfy-Org/Ideogram-4` |
| `flux2-vae.safetensors` | ~320 MB | `models/vae/` | HuggingFace: `Comfy-Org/flux2-dev` |
| `qwen3vl_8b_fp8_scaled.safetensors` | ~9.9 GB | `models/text_encoders/` | HuggingFace: `Comfy-Org/Qwen3-VL` |
| `gemma4_e4b_it_fp8_scaled.safetensors` | ~5.4 GB (optional alt) | `models/text_encoders/` | HuggingFace: `Comfy-Org/gemma-4` |
**Total: ~28 GB**
### Model Verification Recipe
A file can have the correct size on disk but still fail to load with `SafetensorError: incomplete metadata`. This happens when a download is interrupted and resumed incorrectly, or when the file was copied incompletely.
Use this Python script to verify any `.safetensors` file before trusting it:
```python
import struct, json, os
def verify_safetensors(path):
"""Check header metadata against actual file size."""
actual = os.path.getsize(path)
f = open(path, 'rb')
header_len = struct.unpack("<Q", f.read(8))[0]
meta = json.loads(f.read(header_len))
total = 8 + header_len
for k, v in meta.items():
if isinstance(v, dict) and "data_offsets" in v:
total += v["data_offsets"][1] - v["data_offsets"][0]
return {
"actual_bytes": actual,
"expected_bytes": total,
"ok": total == actual,
"header_keys": len(meta),
"missing": total - actual if total > actual else 0,
}
# Example
result = verify_safetensors("models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors")
print(result)
# If ok=False, the file is corrupt and MUST be re-downloaded.
```
**Command line shortcut (in Container):**
```bash
python3 -c "
import struct, json, os; p='/opt/ComfyUI/models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors'
f=open(p,'rb'); h=struct.unpack('<Q',f.read(8))[0]; m=json.loads(f.read(h))
t=8+h; [t:=t+v['data_offsets'][1]-v['data_offsets'][0] for k,v in m.items()
if isinstance(v,dict) and 'data_offsets' in v]
print('ok=',t==os.path.getsize(p))
"
```
### Text Encoder Compatibility Note
Ideogram 4 supports **two** text encoders:
- `qwen3vl_8b_fp8_scaled.safetensors` (Qwen3-VL, ~9.9 GB) — primary, higher quality
- `gemma4_e4b_it_fp8_scaled.safetensors` (Gemma 4, ~5.4 GB) — lighter alternative
If one fails (e.g., corrupt download), try the other.
### ComfyUI BluePrint Workflow (API Format)
The canonical workflow is shipped as a BluePrint JSON in ComfyUI:
```
blueprints/Text to Image (Ideogram v4).json
```
This uses the `Ideogram4Scheduler` + `DualModelGuider` with the two UNet files (conditional + unconditional) for asymmetric classifier-free guidance.
**Key nodes in the BluePrint:**
- `Ideogram4Scheduler` → outputs SIGMAS (replaces BasicScheduler)
- `DualModelGuider` → combines positive + unconditional models
- `CFGOverride` → cfg schedule override (start/end percent)
- `EmptyFlux2LatentImage` → 128-channel latent
- `SamplerCustomAdvanced` → with euler + Ideogram4Scheduler sigmas
### Common Errors — Local Path
**1. `SafetensorError: incomplete metadata, file not fully covered`**
- **Cause:** File is structurally corrupt (wrong tensor offsets in header).
- **Fix:** Run verification recipe above. If `ok=False`, delete and re-download from HuggingFace.
- **Important:** `ls -lh` showing the right size is NOT sufficient. The header values must match the data offsets.
**2. `Unauthorized: Please login first to use this node`**
- **Cause:** Using `IdeogramV1`/`V2`/`V3`/`V4` API node without comfy.org auth.
- **Fix:** Switch to local pipeline (`Ideogram4Scheduler` + model files), or add `COMFY_CLOUD_API_KEY`.
**3. `expected input[1, 128, 32, 32] to have 16 channels`**
- **Cause:** Using FLUX.1 VAE (`ae.safetensors`) instead of `flux2-vae.safetensors`.
- **Fix:** VAE must be `flux2-vae.safetensors` (or `taef2`).
## References
- ComfyUI BluePrints: `blueprints/Text to Image (Ideogram v4).json`
- Model source: https://huggingface.co/Comfy-Org/Ideogram-4
- Text encoder: https://huggingface.co/Comfy-Org/Qwen3-VL
- Alternative encoder: https://huggingface.co/Comfy-Org/gemma-4
creative/comfyui/references/ideogram4.md
+180
View File
@@ -0,0 +1,180 @@
# Ideogram 4 in ComfyUI
Two distinct execution paths exist. Pick the right one based on auth and hardware.
## Path A: Cloud API (`IdeogramV4` node)
- **Node:** `IdeogramV4` (built-in, from `comfy_api_nodes.nodes_ideogram`)
- **Inputs:** `prompt` (STRING), `resolution` (COMBO), `rendering_speed` (COMBO), `seed` (INT)
- **Hidden inputs:** `auth_token_comfy_org`, `api_key_comfy_org`
- **Requires:** comfy.org account + API key (set via auth_token or api_key_comfy_org)
### Common error
```
Unauthorized: Please login first to use this node.
```
→ Set the `api_key_comfy_org` hidden input or login in the ComfyUI web UI.
## Path B: Local inference (`Ideogram4Scheduler` node)
- **Node:** `Ideogram4Scheduler` (built-in, `comfy_extras.nodes_ideogram4`)
- **Outputs:** `SIGMAS` — feed into `SamplerCustomAdvanced`
- **Workflow structure:** `Ideogram4Scheduler` → `SamplerCustomAdvanced` → `VAEDecode` → `SaveImage`
### Why you can't just "use the blueprint directly"
The official blueprint at `blueprints/Text to Image (Ideogram v4).json` is in **editor format** (`"nodes"` / `"links"` arrays with a top-level `definitions.subgraphs` array). **The ComfyUI REST API (`POST /api/prompt`) does NOT accept editor format.** Submitting it produces HTTP 400 or 500 with no useful message. You must either:
1. Load the blueprint in the ComfyUI web UI and re-export it as **API format** (Workflow → Export API), OR
2. Manually construct an API-format workflow using the verified node wiring below.
### Critical: Ideogram 4 needs asymmetric CFG wiring
Unlike standard pipelines that use `BasicGuider`, Ideogram 4 requires `CFGOverride` + `DualModelGuider` for asymmetric classifier-free guidance:
- `CFGOverride` sits on the **main model** to override the CFG value in a specific timestep range
- `DualModelGuider` takes the CFG-overridden main model, the positive conditioning, a **separate unconditional model** (via `model_negative`), and the zeroed-out conditioning (via `ConditioningZeroOut`)
- The guider output (`GUIDER`) feeds into `SamplerCustomAdvanced`
**Do NOT use:** plain `BasicGuider` (only supports single model + pos/neg), standard `KSampler` (doesn't use `Ideogram4Scheduler` sigmas), or `model_1` as the DualModelGuider input name (it's `model_negative`).
### Node input names for API-format workflows (verified against `/object_info`)
| Node | Required inputs | Optional inputs | Notes |
|------|-----------------|-----------------|-------|
| `Ideogram4Scheduler` | `steps`, `width`, `height`, `mu`, `std` | — | Outputs `SIGMAS` for `SamplerCustomAdvanced` |
| `CFGOverride` | `model`, `cfg`, `start_percent`, `end_percent` | — | Place on main UNet between `UNETLoader` and `DualModelGuider` |
| `DualModelGuider` | `model`, `positive`, `cfg` | `model_negative` (MODEL), `negative` (CONDITIONING) | `model_negative` is the unconditional UNet (NOT `model_1`) |
| `CLIPLoader` | `clip_name` | `type` (use `"ideogram4"`), `device` | Must use `type: "ideogram4"` for Qwen3VL text encoder |
| `EmptyFlux2LatentImage` | `width`, `height`, `batch_size` | — | 128-channel latent (NOT standard `EmptyLatentImage`) |
| `SamplerCustomAdvanced` | `noise`, `guider`, `sampler`, `sigmas`, `latent_image` | — | `guider` comes from `DualModelGuider`, `sigmas` from `Ideogram4Scheduler` |
### Required models
| Component | Filename | Path | Size | Source |
|-----------|----------|------|------|--------|
| UNet (main) | `ideogram4_fp8_scaled.safetensors` | `models/diffusion_models/` | ~8.6 GB | `Comfy-Org/Ideogram-4` |
| UNet (unconditional) | `ideogram4_unconditional_fp8_scaled.safetensors` | `models/diffusion_models/` | ~8.6 GB | `Comfy-Org/Ideogram-4` |
| VAE | `flux2-vae.safetensors` | `models/vae/` | ~320 MB | `Comfy-Org/flux2-dev` |
| Text encoder (A) | `qwen3vl_8b_fp8_scaled.safetensors` | `models/text_encoders/` | ~9.8 GB | `Comfy-Org/Qwen3-VL` |
| Text encoder (B) | `gemma4_e4b_it_fp8_scaled.safetensors` | `models/text_encoders/` | ~? GB | `Comfy-Org/gemma-4` |
### Known corruption: `qwen3vl_8b_fp8_scaled.safetensors`
A truncated download of this ~9.8 GB file produces:
```
Error while deserializing header: incomplete metadata, file not fully covered
```
at ComfyUI's `CLIPLoader` (node `load_clip` → `comfy.utils.load_torch_file`).
**Quick integrity check** (run inside the ComfyUI container or on the host):
```bash
python3 -c "
import struct, json, os
p = 'models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors'
f = open(p, 'rb'); hl = struct.unpack('<Q', f.read(8))[0]
m = json.loads(f.read(hl)); t = 8 + hl
for k,v in m.items():
if isinstance(v, dict) and 'data_offsets' in v:
t += v['data_offsets'][1] - v['data_offsets'][0]
a = os.path.getsize(p); print(f'expected={t} actual={a} ok={t==a}')
"
```
If `ok=False`, the download is incomplete. Re-download via:
```bash
curl -L -o models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors \
"https://huggingface.co/Comfy-Org/Qwen3-VL/resolve/main/text_encoders/qwen3vl_8b_fp8_scaled.safetensors"
```
**Alternative:** `safetensors.safe_open` also catches corruption early:
```python
from safetensors import safe_open
f = safe_open("qwen3vl_8b_fp8_scaled.safetensors", framework="pt", device="cpu")
print(len(f.keys())) # Should be ~1254; crashes on truncated files
```
### Pitfall: wrong input name for DualModelGuider
The Blueprint user's `Subgraphs`-based `DualModelGuider` uses the internal **editor-format** link name `model_1`, but the **API-format** input name is **`model_negative`**:
```
# WRONG (editor convention leaked into API format):
"inputs": {"model": ["X", 0], "model_1": ["Y", 0], ...}
# RIGHT (API format):
"inputs": {"model": ["X", 0], "model_negative": ["Y", 0], ...}
```
Error if wrong: `TypeError: DualModelGuider.execute() got an unexpected keyword argument 'model_1'`
### Full local workflow blueprint (extracted from ComfyUI `/history`)
The official blueprint at `blueprints/Text to Image (Ideogram v4).json` is in **editor format** (`"nodes"` / `"links"` arrays). It must be converted to **API format** (`"class_type"` per node) before execution. Below is a verified API-format workflow:
```json
{
"37": {
"inputs": {"aspect_ratio": "1:1 (Square)", "megapixels": 1.0, "multiple": 8},
"class_type": "ResolutionSelector"
},
"98:9": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"},
"98:11": {"inputs": {"width": ["98:31", 1], "height": ["98:32", 1], "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
"98:12": {"inputs": {"noise": ["98:18", 0], "guider": ["98:155", 0], "sampler": ["98:16", 0], "sigmas": ["98:17", 0], "latent_image": ["98:11", 0]}, "class_type": "SamplerCustomAdvanced"},
"98:13": {"inputs": {"samples": ["98:12", 0], "vae": ["98:9", 0]}, "class_type": "VAEDecode"},
"98:14": {"inputs": {"clip_name": "qwen3vl_8b_fp8_scaled.safetensors", "type": "ideogram4", "device": "default"}, "class_type": "CLIPLoader"},
"98:16": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"},
"98:17": {"inputs": {"steps": ["98:151", 1], "width": ["98:31", 1], "height": ["98:32", 1], "mu": ["98:144", 0], "std": ["98:146", 0]}, "class_type": "Ideogram4Scheduler"},
"98:18": {"inputs": {"noise_seed": 42}, "class_type": "RandomNoise"},
"98:23": {"inputs": {"unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"},
"98:24": {"inputs": {"text": "PROMPT_HERE", "clip": ["98:14", 0]}, "class_type": "CLIPTextEncode"},
"98:10": {"inputs": {"conditioning": ["98:24", 0]}, "class_type": "ConditioningZeroOut"},
"98:31": {"inputs": {"expression": "max(((a + 15) // 16) * 16, 256)", "values.a": ["98:27", 0]}, "class_type": "ComfyMathExpression"},
"98:32": {"inputs": {"expression": "max(((a + 15) // 16) * 16, 256)", "values.a": ["98:28", 0]}, "class_type": "ComfyMathExpression"},
"98:27": {"inputs": {"value": ["37", 0]}, "class_type": "PrimitiveInt"},
"98:28": {"inputs": {"value": ["37", 1]}, "class_type": "PrimitiveInt"},
"98:144": {"inputs": {"value": ["98:145", 0]}, "class_type": "ComfyNumberConvert"},
"98:145": {"inputs": {"json_string": ["98:148", 0], "key": "mu"}, "class_type": "JsonExtractString"},
"98:146": {"inputs": {"value": ["98:150", 0]}, "class_type": "ComfyNumberConvert"},
"98:147": {"inputs": {"json_string": "{\\\"Quality\\\":{\\\"num_steps\\\":48,\\\"mu\\\":0.0,\\\"std\\\":1.5,\\\"preset\\\":\\\"Quality\\\"}}", "key": "Quality"}, "class_type": "JsonExtractString"},
"98:148": {"inputs": {"string": ["98:147", 0], "find": "'", "replace": "\\""}, "class_type": "StringReplace"},
"98:149": {"inputs": {"json_string": ["98:148", 0], "key": "num_steps"}, "class_type": "JsonExtractString"},
"98:150": {"inputs": {"json_string": ["98:148", 0], "key": "std"}, "class_type": "JsonExtractString"},
"98:151": {"inputs": {"value": ["98:149", 0]}, "class_type": "ComfyNumberConvert"},
"98:154": {"inputs": {"unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"},
"98:155": {"inputs": {"cfg": 7.0, "model": ["98:157", 0], "positive": ["98:24", 0], "model_negative": ["98:154", 0], "negative": ["98:10", 0]}, "class_type": "DualModelGuider"},
"98:156": {"inputs": {"choice": "Quality", "index": 1, "option1": "Quality", "option2": "Default", "option3": "Turbo", "option4": ""}, "class_type": "CustomCombo"},
"98:157": {"inputs": {"cfg": 3.0, "start_percent": 0.7, "end_percent": 1.0, "model": ["98:23", 0]}, "class_type": "CFGOverride"},
"158": {"inputs": {"filename_prefix": "Ideogram_4.0", "images": ["98:13", 0]}, "class_type": "SaveImage"}
}
```
Replace `"PROMPT_HERE"` (node `98:24`, field `text`) with your prompt. Ideogram 4 accepts either plain text or structured JSON prompts.
### Accessing a ComfyUI LXC container via Proxmox VE
If ComfyUI runs inside a Proxmox LXC container without direct SSH:
```bash
# Proxmox node shell → run inside container
pct exec CTID -- bash -c "COMMAND"
# Push a file into the container (avoid nested quote hell)
pct push CTID /local/script.py /tmp/script.py
pct exec CTID -- python3 /tmp/script.py
# Common CTID in this environment: 204 (ComfyUI+FLUX.2/ROCm)
```
### Key differences from FLUX.1
- Text encoder: `qwen3vl_8b_fp8_scaled` or `gemma4_e4b_it_fp8_scaled` — NOT CLIP-L + T5XXL
- Latent: `EmptyFlux2LatentImage` (128 channels)
- VAE: `flux2-vae.safetensors` — NOT `ae.safetensors`
- CFG: Uses asymmetric CFG (unconditional conditioning is zeroed out via `ConditioningZeroOut`, not a separate negative prompt string)
## Quick verification checklist
1. `curl $HOST:8188/models/diffusion_models | grep ideogram4` — models present?
2. Check file integrity of `qwen3vl_8b_fp8_scaled.safetensors` with the Python check above.
3. For cloud: ensure `api_key_comfy_org` is set or the user is logged into comfy.org via the web UI.
4. For local: verify the blueprint was exported as **API format**, not editor format (which has top-level `"nodes"` / `"links"` arrays).
creative/comfyui/references/official-cli.md
+255
View File
@@ -0,0 +1,255 @@
# comfy-cli Command Reference
Official CLI from [Comfy-Org/comfy-cli](https://github.com/Comfy-Org/comfy-cli).
Docs: https://docs.comfy.org/comfy-cli/getting-started
## Installation
Order of preference:
```bash
pipx install comfy-cli # recommended (isolated env)
uvx --from comfy-cli comfy --help # zero-install via uv
pip install --user comfy-cli # fallback
```
The skill's `comfyui_setup.sh` picks the best available method.
First run may prompt for analytics. Disable non-interactively:
```bash
comfy --skip-prompt tracking disable
```
## Global Options
| Option | Description |
|--------|-------------|
| `--workspace <path>` | Target a specific ComfyUI workspace |
| `--recent` | Use most recently used workspace |
| `--here` | Use current directory as workspace |
| `--skip-prompt` | No interactive prompts (use defaults) |
| `-v` / `--version` | Print version |
Workspace resolution priority:
1. `--workspace` (explicit path)
2. `--recent` (from config)
3. `--here` (cwd)
4. `comfy set-default` path
5. Most recently used
6. `~/comfy/ComfyUI` (Linux) or `~/Documents/comfy/ComfyUI` (macOS/Win)
## Lifecycle Commands
### `comfy install`
Download and install ComfyUI + ComfyUI-Manager.
```bash
comfy install # interactive GPU selection
comfy install --nvidia
comfy install --amd # ROCm (Linux)
comfy install --m-series # Apple Silicon (MPS)
comfy install --cpu # CPU only (slow)
comfy install --fast-deps # use uv for deps
comfy install --skip-manager # skip ComfyUI-Manager
```
| Option | Description |
|--------|-------------|
| `--nvidia` / `--amd` / `--m-series` / `--cpu` | GPU type |
| `--cuda-version` | 11.8, 12.1, 12.4, 12.6, 12.8, 12.9, 13.0 |
| `--rocm-version` | 6.1, 6.2, 6.3, 7.0, 7.1 |
| `--fast-deps` | uv-based dependency resolution |
| `--skip-manager` | Don't install ComfyUI-Manager |
| `--skip-torch-or-directml` | Skip PyTorch install |
| `--version <ver>` | `0.2.0`, `latest`, `nightly` |
| `--commit <hash>` | Install specific commit |
| `--pr "#1234"` | Install from a PR |
| `--restore` | Restore deps for existing install |
### `comfy launch`
```bash
comfy launch # foreground :8188
comfy launch --background # background daemon
comfy launch -- --listen 0.0.0.0 # LAN-accessible
comfy launch -- --port 8190 # custom port
comfy launch -- --cpu # force CPU mode
comfy launch -- --lowvram # 6 GB cards
comfy launch --background -- --listen 0.0.0.0 --port 8190
```
Common extra args after `--`: `--listen`, `--port`, `--cpu`, `--lowvram`,
`--novram`, `--fp16-vae`, `--force-fp32`, `--disable-cuda-malloc`.
### `comfy stop`
```bash
comfy stop
```
### `comfy run`
Submit a raw workflow JSON to a running server. **Limited** — no parameter
injection, no structured output download. For agents, use
`scripts/run_workflow.py` instead.
```bash
comfy run --workflow workflow_api.json
comfy run --workflow workflow_api.json --host 10.0.0.5 --port 8188
comfy run --workflow workflow_api.json --timeout 300 --wait
```
### `comfy which`
```bash
comfy which # show targeted workspace
comfy --recent which
```
### `comfy set-default`
```bash
comfy set-default /path/to/ComfyUI
comfy set-default /path/to/ComfyUI --launch-extras="--listen 0.0.0.0"
```
### `comfy update`
```bash
comfy update # update ComfyUI core
comfy node update all # update all custom nodes
```
---
## `comfy node` — Custom Node Management
All node operations use ComfyUI-Manager (`cm-cli`) under the hood.
```bash
comfy node show installed # list installed
comfy node show enabled # list enabled
comfy node show all # all available in registry
comfy node simple-show installed # compact list
comfy node install comfyui-impact-pack
comfy node install <name> --uv-compile # ComfyUI-Manager v4.1+ unified resolver
comfy node uninstall <name>
comfy node update <name> | all
comfy node enable <name>
comfy node disable <name>
comfy node fix <name> # fix broken deps
comfy node install-deps --workflow=workflow.json
comfy node deps-in-workflow --workflow=w.json --output=deps.json
comfy node save-snapshot
comfy node restore-snapshot <file>
comfy node bisect start # binary-search a culprit node
comfy node bisect good
comfy node bisect bad
comfy node bisect reset
```
### Dependency Resolution Options
| Flag | Description |
|------|-------------|
| `--fast-deps` | comfy-cli built-in uv resolver |
| `--uv-compile` | ComfyUI-Manager v4.1+ unified resolver (recommended) |
| `--no-deps` | Skip dep installation |
Make `uv-compile` default: `comfy manager uv-compile-default true`
---
## `comfy model` — Model Management
```bash
comfy model list
comfy model list --relative-path models/checkpoints
comfy model download --url <URL>
comfy model download --url <URL> --relative-path models/loras
comfy model download --url <URL> --filename custom_name.safetensors
comfy model remove # interactive
comfy model remove --relative-path models/checkpoints --model-names "model.safetensors"
```
| Option | Description |
|--------|-------------|
| `--url` | Download URL (CivitAI, HuggingFace, direct) |
| `--relative-path` | Subdirectory under workspace (e.g. `models/checkpoints`) |
| `--filename` | Custom save filename |
| `--set-civitai-api-token` | Persist CivitAI token |
| `--set-hf-api-token` | Persist HuggingFace token |
| `--downloader` | `httpx` (default) or `aria2` |
Standard model directories:
```
ComfyUI/models/
├── checkpoints/ # Full model files
├── loras/ # LoRA adapters
├── vae/ # VAE models
├── controlnet/ # ControlNet models
├── clip/ # CLIP / T5 text encoders
├── clip_vision/ # CLIP vision encoders
├── upscale_models/ # ESRGAN / SwinIR / etc.
├── embeddings/ # Textual inversion embeddings
├── unet/ # Standalone UNet weights
├── diffusion_models/ # Flux / SD3 / Wan diffusion models
├── animatediff_models/ # AnimateDiff motion modules
├── ipadapter/ # IPAdapter weights
└── style_models/ # Style adapters
```
---
## `comfy manager` — ComfyUI-Manager Settings
```bash
comfy manager disable # disable Manager completely
comfy manager enable-gui # enable new GUI
comfy manager disable-gui # API-only
comfy manager enable-legacy-gui # legacy GUI
comfy manager uv-compile-default true # make --uv-compile the default
comfy manager clear # clear startup action
```
---
## `comfy pr-cache` — Frontend PR Cache
```bash
comfy pr-cache list
comfy pr-cache clean
comfy pr-cache clean 456
```
Cache expires after 7 days; max 10 builds.
---
## Configuration
| OS | Path |
|----|------|
| Linux | `~/.config/comfy-cli/config.ini` |
| macOS | `~/Library/Application Support/comfy-cli/config.ini` |
| Windows | `~/AppData/Local/comfy-cli/config.ini` |
Stores: default workspace, recent workspace, background server PID, API
tokens, manager GUI mode, launch extras.
## Discovery
Custom-node registry:
- https://registry.comfy.org/
Model browsers:
- https://huggingface.co/models
- https://civitai.com (NSFW; requires API token for many)
- https://comfyworkflows.com (community workflows)
creative/comfyui/references/openai-adapter.md
+107
View File
@@ -0,0 +1,107 @@
# OpenAI-Compatible HTTP Adapter for ComfyUI
Small FastAPI bridge that exposes `POST /v1/images/generations` (OpenAI format) and maps it to ComfyUI's `POST /api/prompt`.
## When to use
- OpenWebUI, Bifrost, or any other tool that speaks OpenAI `/v1/images/generations` needs to generate images with ComfyUI.
- ComfyUI itself has no native OpenAI-compatible endpoint.
- The adapter is especially useful for **FLUX.2 workflows** that OpenWebUI's built-in ComfyUI connector cannot handle (different nodes: Qwen3VL, `EmptyFlux2LatentImage`, TAEF2).
## Architecture
```
OpenWebUI / Bifrost / Client
POST /v1/images/generations
|
v
[FastAPI Adapter] (Docker or systemd)
|
v
POST http://<comfyui>:8188/api/prompt
GET http://<comfyui>:8188/history/<prompt_id>
GET http://<comfyui>:8188/view
```
## Example: Minimal FLUX.2 Adapter (Docker)
### `Dockerfile`
```dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY adapter.py .
EXPOSE 9000
CMD ["uvicorn", "adapter:app", "--host", "0.0.0.0", "--port", "9000"]
```
### `requirements.txt`
```
fastapi>=0.110.0
uvicorn>=0.29.0
httpx>=0.27.0
pydantic>=2.0.0
```
### `adapter.py` — FLUX.2 Variant
The node names must match what is installed on the target ComfyUI instance.
Query the running server first with `GET /object_info` to confirm.
Key requirements for FLUX.2:
- `CLIPLoaderGGUF` with `"type": "flux2"` (loads Qwen3VL)
- `EmptyFlux2LatentImage` (128-channel latents)
- `VAELoader` with `"vae_name": "taef2"` (or dedicated `flux2_vae.safetensors`)
- `UnetLoaderGGUF` with the FLUX.2 UNet GGUF
See `scripts/adapter_flux2.py` for a ready-to-use file.
### Build & run
```bash
docker build -t comfyui-openai-adapter:latest .
docker run -d --name comfyui-openai-adapter --network host \
-e COMFYUI_URL=http://10.0.30.97:8188 \
-e WORKFLOW_PREFIX=oai_adapter \
--restart unless-stopped \
comfyui-openai-adapter:latest
```
## Verifying node availability
Before deploying the adapter, confirm the target ComfyUI instance has the required nodes:
```bash
curl -s http://<comfyui>:8188/object_info | \
python3 -c "import sys,json; d=json.load(sys.stdin); \
[print(f'{n}: {n in d}') for n in \
['CLIPLoaderGGUF','UnetLoaderGGUF','EmptyFlux2LatentImage','VAELoader','KSampler']]"
```
## OpenWebUI configuration
Admin Panel → Settings → Images → Image Generation
| Setting | Value |
|---------|-------|
| Engine | `Open AI` |
| API Base URL | `http://<adapter-ip>:9000/v1` |
| API Key | dummy (adapter does not enforce auth) |
| Model | `flux2` |
## Bifrost integration
Bifrost is an LLM gateway with no native ComfyUI provider. The adapter fills that gap:
- Register the adapter as an OpenAI-compatible provider in Bifrost, or
- Point OpenWebUI directly at the adapter (recommended; keeps image path separate from chat routing).
## Pitfalls
1. **Node names are exact.** Use `GET /object_info` to verify every `class_type` exists; missing custom nodes produce cryptic "class_type not found" errors from ComfyUI.
2. **Model names are exact and case-sensitive.** `flux-2-klein-9b-Q4_K_S.gguf` != `flux-2-klein-9b-q4_k_s.gguf`.
3. **TAEF2 vs full VAE.** `taef2` resolves tiny encoder/decoder `pth` files; the full `flux2_vae.safetensors` requires manual download from a gated repo.
4. **Seed injection.** Always rewrite the seed per image when `n > 1`; otherwise every image in the batch is identical.
5. **Timeout tuning.** FLUX.2 on ROCm gfx1150 takes ~110s for 1024x1024 @ 4 steps. Set `wait_for_image` timeout >= 300s.
creative/comfyui/references/openai-api-wrapper.md
+281
View File
@@ -0,0 +1,281 @@
# OpenAI-Compatible Multi-Model API Wrapper for ComfyUI
FastAPI proxy that exposes `POST /v1/images/generations` (OpenAI format) and
routes to **multiple** ComfyUI workflows depending on the requested `model`.
Supported models: `ideogram4`, `flux2`, `flux1-dev`, `flux1-schnell`.
## When to use
- You need one endpoint that can switch between Ideogram 4 and FLUX
families without reconfiguring the frontend.
- Your chat UI only speaks OpenAI `/v1/images/generations` but you want
to use ComfyUI's native pipelines (including non-standard nodes like
`Ideogram4Scheduler`, `ModelSamplingFlux`, `DualCLIPLoader`).
- You run ComfyUI inside a Proxmox LXC/VM and want a systemd-managed
adapter service.
## Architecture
```
Frontend (OpenWebUI, LibreChat, etc.)
POST /v1/images/generations {model, prompt, size, ...}
│
▼
FastAPI Adapter (this wrapper)
─ model switch ──► _build_ideogram4_workflow()
_build_flux2_workflow()
_build_flux1_workflow()
│
▼
ComfyUI REST API POST /prompt
│
▼
Poll /history/{prompt_id} → fetch /view
```
## Adapter Code (Multi-Model)
The full source is provided as a **template** in this skill:
`templates/comfyui-openai-adapter.py`
Key design choices:
- **Model routing** — `req.model` selects the workflow builder.
- **Size validation / fallback** — Ideogram4 only accepts specific
resolutions. FLUX is more permissive. Invalid sizes silently fall back to the model's default.
- **Loader node per model family:**
- Ideogram4 → `UNETLoader` (safetensors)
- FLUX (GGUF) → `UnetLoaderGGUF` (from ComfyUI-GGUF custom node)
- Ideogram4 → single `CLIPLoader` with `type: "ideogram4"`
- FLUX → `DualCLIPLoader` with `type: "flux"` (clip_l + t5xxl)
- **Quality → steps mapping:** `low` (8), `medium` (20), `high` (30), `ultra` (50).
FLUX1-schnell hard-capped to 8 steps.
- **Ideogram4 minimum steps:** do not go below ~12 steps — the model
produces near-blank gray blocks at ≤8 steps. Clamp or warn when
the user explicitly requests fewer.
- **Ideogram4 asymmetric CFG wiring (CRITICAL):** The adapter's `_build_ideogram4_workflow()`
MUST use `CFGOverride` + `DualModelGuider` + a separate unconditional UNet
(`ideogram4_unconditional_fp8_scaled`) + `ConditioningZeroOut`. Using `BasicGuider`
does NOT throw an error but silently degrades image quality. The `DualModelGuider`
input for the unconditional model is `model_negative` (NOT `model_1`).
See `references/ideogram4.md` for the full blueprint.
- **Gray-block detection:** If Ideogram4 output has PIL variance < 15 across all
channels, the generation produced a blank/gray image (often at `quality=low` + `style=photorealistic`).
Flag these as failures and retry with higher steps or different style.
- **ROCm environment** — if ComfyUI runs on AMD ROCm inside an LXC,
the *systemd service* for ComfyUI must set
`Environment=HSA_OVERRIDE_GFX_VERSION=11.0.0` (or whatever matches
the GPU). The adapter itself does not need ROCm env vars.
- **Timeout for long generations** — set the adapter's HTTP poll loop
to at least 1800 s (30 min) for high-resolution workloads.
Edit `for i in range(720)` (30 min at 5 s intervals) and set
`httpx.AsyncClient(timeout=2400)` on the initial `/prompt` POST.
- **Synchronous (blocking) by design** — the adapter does NOT implement
async jobs. The HTTP response blocks until ComfyUI finishes. For
multi-hour jobs this means the client must keep the connection open.
If a true async job system is needed, build `POST → job_id` +
`GET /v1/jobs/{id}` separately.
## Running it
### Option A: Python venv (quick test)
```bash
python3 -m venv /opt/ideogram4-api/venv
source /opt/ideogram4-api/venv/bin/activate
pip install fastapi uvicorn httpx pydantic
export COMFYUI_HOST=localhost
export COMFYUI_PORT=8188
python3 comfyui-openai-adapter.py
```
### Option B: Docker on dedicated host (persistent)
```dockerfile
FROM python:3.12-slim
WORKDIR /app
RUN pip install --no-cache-dir fastapi uvicorn httpx pydantic
COPY comfyui-openai-adapter.py ./main.py
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
```
> The adapter and ComfyUI should be on the same LAN or host. Latency matters because the adapter polls ComfyUI every 5 s for up to 60 min.
### Option C: systemd service (LXC/VM — recommended for Proxmox)
**Prerequisite:** Kill any previously manual/nohup-started ComfyUI or adapter processes *before* enabling systemd units. Old processes hold ports 8188 and 8000 and cause `address already in use`.
```bash
# Inside LXC container (e.g. CT 204)
ss -tlnp | grep -E '8188|8000'
fuser -k 8188/tcp 2>/dev/null
fuser -k 8000/tcp 2>/dev/null
```
**`comfyui.service`** (note the ROCm env var for gfx1150):
```ini
[Unit]
Description=ComfyUI
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/opt/ComfyUI
ExecStart=/usr/bin/python3 main.py --listen 0.0.0.0 --port 8188
Restart=always
RestartSec=10
Environment=HIP_VISIBLE_DEVICES=0
Environment=HSA_OVERRIDE_GFX_VERSION=11.0.0
Environment=PATH=/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin
Environment=LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/torch/lib:/opt/rocm/lib
[Install]
WantedBy=multi-user.target
```
**`ideogram4-api.service`** (rename to `comfyui-api.service` if generic):
```ini
[Unit]
Description=ComfyUI OpenAI API Wrapper
After=network.target comfyui.service
Requires=comfyui.service
[Service]
Type=simple
User=root
WorkingDirectory=/opt/ideogram4-api
ExecStart=/opt/ideogram4-api/venv/bin/python /opt/ideogram4-api/main.py
Restart=always
RestartSec=5
Environment=PATH=/opt/ideogram4-api/venv/bin:/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin
Environment=COMFYUI_HOST=localhost
Environment=COMFYUI_PORT=8188
[Install]
WantedBy=multi-user.target
```
> **Filename note:** The adapter source may be saved as `main.py`, `app.py`, or `adapter.py`. Check `ls /opt/ideogram4-api/` and match `ExecStart=` exactly.
Enable & start:
```bash
systemctl daemon-reload
systemctl enable --now comfyui.service
sleep 15
systemctl enable --now ideogram4-api.service
```
## Extended OpenAI-compatible parameters
Beyond the standard `model`, `prompt`, `n`, `size`, `response_format`,
the adapter accepts these ComfyUI extensions:
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `seed` | int | random | Fixed seed. Random if omitted. |
| `steps` | int | from `quality` | Denoising steps (1–100). Overrides `quality` if set. |
| `cfg` | float | model-specific | Guidance scale. Ideogram4=1.0, FLUX=3.5, others=7.5 |
| `quality` | string | `medium` | Maps to steps: `low`(8), `medium`(20), `high`(30), `ultra`(50). |
| `negative_prompt` | string | `""` | Negative prompt text. |
| `style` | string | `""` | Style modifier string. |
| `control_strength` | float | — | Reserved for future ControlNet integration. |
Example request body:
```json
{
"model": "flux2",
"prompt": "a cyberpunk alley at night, neon signs",
"n": 1,
"size": "1024x1024",
"quality": "high",
"seed": 42,
"cfg": 3.5,
"negative_prompt": "blurry, low quality"
}
```
## Frontend configuration (OpenWebUI example)
1. Admin → **Settings** → **Images**
2. **Image Generation Engine:** `Open AI`
3. **OpenAI API Base URL:** `http://<adapter-host>:8000/v1`
4. **OpenAI API Key:** any dummy string (adapter does not validate)
5. **Model:** `ideogram4` or `flux2` or `flux1-dev`
## Pitfalls
| Issue | Cause | Fix |
|---|---|---|
| `TimeoutError` after 300 s | High resolution or GPU under load | Clamp `size`, increase timeout, reduce `steps` |
| `500` from ComfyUI + `value_not_in_list` | `size` not in model's allowed resolutions | Check allowed sizes; use defaults |
| `[Errno 98] address already in use` | Old manual `nohup` process still running | `fuser -k 8000/tcp` / `fuser -k 8188/tcp`, then restart systemd |
| `can't open file '/opt/ideogram4-api/app.py'` | `ExecStart` mismatches actual filename | Check `ls /opt/ideogram4-api/` and align `ExecStart` |
| `UNETLoader` node error for `.gguf` model | Using `UNETLoader` for a GGUF file | Switch to `UnetLoaderGGUF` |
| `RuntimeError` during text-encode (ROCm) | Missing `HSA_OVERRIDE_GFX_VERSION` | Add to `comfyui.service` `Environment=` line |
| Ideogram4 black/corrupted output | Corrupt `.safetensors` or wrong VAE | Verify file integrity; use `flux2-vae.safetensors` |
| FLUX output dimensions wrong | Used `EmptyLatentImage` instead of `EmptyFlux2LatentImage` | Ideogram4 uses `EmptyFlux2LatentImage`; FLUX.1 uses `EmptyLatentImage` |
| `DualModelGuider.execute() got unexpected keyword argument 'model_1'` | Wrong workflow key name | Use `model_negative`, NOT `model_1` |
| systemd `comfyui.service` keeps failing with `exit-code` | Port 8188 held by old manual ComfyUI | `pkill -f "main.py.*8188"`, then `systemctl restart` |
## Model-specific node quick-reference
### Ideogram 4 local (with asymmetric CFG — correct wiring)
```json
{
"1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"},
"2": {"inputs": {"unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default"},
"class_type": "UNETLoader"},
"3": {"inputs": {"clip_name": "qwen3vl_8b_fp8_scaled.safetensors", "type": "ideogram4"},
"class_type": "CLIPLoader"},
"4": {"inputs": {"text": "...", "clip": ["3", 0]}, "class_type": "CLIPTextEncode"},
"5": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
"6": {"inputs": {"noise_seed": 42}, "class_type": "RandomNoise"},
"7": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"},
"8": {"inputs": {"steps": 20, "width": 1024, "height": 1024, "mu": 0.5, "std": 1.75},
"class_type": "Ideogram4Scheduler"},
"15": {"inputs": {"unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default"},
"class_type": "UNETLoader"},
"16": {"inputs": {"conditioning": ["4", 0]}, "class_type": "ConditioningZeroOut"},
"17": {"inputs": {"cfg": 3.0, "start_percent": 0.7, "end_percent": 1.0, "model": ["2", 0]},
"class_type": "CFGOverride"},
"9": {"inputs": {"cfg": 7.0, "model": ["17", 0], "positive": ["4", 0],
"model_negative": ["15", 0], "negative": ["16", 0]},
"class_type": "DualModelGuider"},
"10": {"inputs": {"noise": ["6", 0], "guider": ["9", 0], "sampler": ["7", 0],
"sigmas": ["8", 0], "latent_image": ["5", 0]}, "class_type": "SamplerCustomAdvanced"},
"11": {"inputs": {"samples": ["10", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"},
"12": {"inputs": {"filename_prefix": "ideogram4_api", "images": ["11", 0]}, "class_type": "SaveImage"}
}
```
Allowed sizes: `1024x1024`, `1024x1536`, `1536x1024`, `832x1216`, `1216x832`, `896x1152`, `1152x896`.
### FLUX.2 / FLUX.1
```json
{
"1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"},
"2": {"inputs": {"unet_name": "flux1-dev-Q4_K_S.gguf"}, "class_type": "UnetLoaderGGUF"},
"3": {"inputs": {"model": ["2", 0], "max_shift": 1.15, "base_shift": 0.5,
"width": 1024, "height": 1024}, "class_type": "ModelSamplingFlux"},
"4": {"inputs": {"clip_name1": "clip_l.safetensors", "clip_name2": "t5xxl_fp8_e4m3fn.safetensors",
"type": "flux", "device": "default"}, "class_type": "DualCLIPLoader"},
"5": {"inputs": {"text": "...", "clip": ["4", 0]}, "class_type": "CLIPTextEncode"},
"6": {"inputs": {"conditioning": ["5", 0], "guidance": 3.5}, "class_type": "FluxGuidance"},
"7": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyLatentImage"},
"8": {"inputs": {"noise_seed": 42}, "class_type": "RandomNoise"},
"9": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"},
"10": {"inputs": {"model": ["3", 0], "steps": 20, "denoise": 1.0, "scheduler": "simple",
"sampler": ["9", 0]}, "class_type": "BasicScheduler"},
"11": {"inputs": {"model": ["3", 0], "conditioning": ["6", 0]}, "class_type": "BasicGuider"},
"12": {"inputs": {"noise": ["8", 0], "guider": ["11", 0], "sampler": ["9", 0],
"sigmas": ["10", 0], "latent_image": ["7", 0]}, "class_type": "SamplerCustomAdvanced"},
"13": {"inputs": {"samples": ["12", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"},
"14": {"inputs": {"filename_prefix": "flux2_api", "images": ["13", 0]}, "class_type": "SaveImage"}
}
```
Allowed sizes: all common resolutions up to 1536×1536. FLUX schnell defaults to 4–8 steps; dev to 20–50.
creative/comfyui/references/openwebui-integration.md
+201
View File
@@ -0,0 +1,201 @@
# OpenWebUI ↔ ComfyUI Integration
OpenWebUI's native ComfyUI connector is hard-wired for SD/SDXL pipelines (CLIP-L + T5XXL, `EmptyLatentImage`, standard VAE). It **does not work** with FLUX.2 (Qwen3-VL, `EmptyFlux2LatentImage`, TAE/VAE).
The cleanest solution is a lightweight OpenAI-compatible adapter that exposes `/v1/images/generations` and internally submits a FLUX.2 workflow to ComfyUI's REST API.
---
## Architecture
```
OpenWebUI ──POST /v1/images/generations──► Adapter ──POST /api/prompt──► ComfyUI
(CT 135) (OpenAI format) (Docker) (workflow JSON) (CT 204)
```
The adapter runs as a Docker container (or Python service) on a host that has Layer-2/Layer-3 reachability to ComfyUI.
---
## Adapter Code
### `adapter.py`
```python
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from typing import Optional
import httpx
import json
import base64
import asyncio
import time
import random
import os
app = FastAPI(title="ComfyUI OpenAI Adapter", version="0.1.0")
COMFYUI_URL = os.environ.get("COMFYUI_URL", "http://127.0.0.1:8188")
WORKFLOW_PREFIX = os.environ.get("WORKFLOW_PREFIX", "oai_adapter")
FLUX2_WORKFLOW = {
"1": {"inputs": {"unet_name": "flux-2-klein-9b-Q4_K_S.gguf"}, "class_type": "UnetLoaderGGUF"},
"2": {"inputs": {"clip_name": "Qwen3VL-8B-Instruct-Q4_K_M.gguf", "type": "flux2"}, "class_type": "CLIPLoaderGGUF"},
"3": {"inputs": {"vae_name": "taef2"}, "class_type": "VAELoader"},
"4": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
"5": {"inputs": {"text": "", "clip": ["2", 0]}, "class_type": "CLIPTextEncode"},
"6": {"inputs": {"text": "", "clip": ["2", 0]}, "class_type": "CLIPTextEncode"},
"7": {"inputs": {"seed": 42, "steps": 4, "cfg": 1.0, "sampler_name": "euler", "scheduler": "simple", "denoise": 1.0,
"model": ["1", 0], "positive": ["5", 0], "negative": ["6", 0], "latent_image": ["4", 0]},
"class_type": "KSampler"},
"8": {"inputs": {"samples": ["7", 0], "vae": ["3", 0]}, "class_type": "VAEDecode"},
"9": {"inputs": {"filename_prefix": WORKFLOW_PREFIX, "images": ["8", 0]}, "class_type": "SaveImage"}
}
class ImageRequest(BaseModel):
prompt: str
n: Optional[int] = 1
size: Optional[str] = "1024x1024"
response_format: Optional[str] = "b64_json"
model: Optional[str] = "flux2"
async def queue_workflow(workflow: dict) -> str:
async with httpx.AsyncClient() as client:
r = await client.post(f"{COMFYUI_URL}/api/prompt", json={"prompt": workflow})
r.raise_for_status()
return r.json()["prompt_id"]
async def wait_for_image(prompt_id: str, timeout: int = 300) -> list:
async with httpx.AsyncClient() as client:
start = time.time()
while time.time() - start < timeout:
r = await client.get(f"{COMFYUI_URL}/history/{prompt_id}")
if r.status_code == 200:
data = r.json()
if prompt_id in data and "outputs" in data[prompt_id]:
outputs = []
for node_id, node_out in data[prompt_id]["outputs"].items():
if "images" in node_out:
outputs.extend(node_out["images"])
if outputs:
return outputs
await asyncio.sleep(1)
raise TimeoutError(f"Timeout waiting for prompt {prompt_id}")
async def fetch_image(filename: str, subfolder: str = "", folder_type: str = "output") -> bytes:
async with httpx.AsyncClient() as client:
params = {"filename": filename, "subfolder": subfolder, "type": folder_type}
r = await client.get(f"{COMFYUI_URL}/view", params=params)
r.raise_for_status()
return r.content
@app.post("/v1/images/generations")
async def generate_images(req: ImageRequest):
if req.n > 4:
return JSONResponse({"error": {"message": "n must be <= 4", "type": "invalid_request_error"}}, status_code=400)
try:
w, h = map(int, req.size.split("x"))
except ValueError:
w, h = 1024, 1024
# CRITICAL: clamp size to prevent timeout/OOM on consumer GPUs
MAX_SIZE = 1536
if w > MAX_SIZE or h > MAX_SIZE:
scale = MAX_SIZE / max(w, h)
w = int(w * scale)
h = int(h * scale)
images = []
for _ in range(req.n):
wf = json.loads(json.dumps(FLUX2_WORKFLOW))
wf["5"]["inputs"]["text"] = req.prompt
wf["4"]["inputs"]["width"] = w
wf["4"]["inputs"]["height"] = h
wf["7"]["inputs"]["seed"] = random.randint(1, 2**32)
prompt_id = await queue_workflow(wf)
imgs = await wait_for_image(prompt_id, timeout=300)
if not imgs:
raise RuntimeError("No images returned")
img_data = await fetch_image(imgs[0]["filename"], imgs[0].get("subfolder", ""), imgs[0].get("type", "output"))
b64 = base64.b64encode(img_data).decode()
images.append({"b64_json": b64})
return {"created": int(time.time()), "data": images}
@app.get("/health")
async def health():
return {"status": "ok"}
```
### `Dockerfile`
```dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY adapter.py .
EXPOSE 9000
CMD ["uvicorn", "adapter:app", "--host", "0.0.0.0", "--port", "9000"]
```
### `requirements.txt`
```text
fastapi>=0.110.0
uvicorn>=0.29.0
httpx>=0.27.0
pydantic>=2.0.0
```
### `docker-compose.yml`
```yaml
services:
comfyui-openai-adapter:
build: .
container_name: comfyui-openai-adapter
ports:
- "9000:9000"
environment:
- COMFYUI_URL=http://10.0.30.97:8188 # <-- adapt to your ComfyUI IP
- WORKFLOW_PREFIX=oai_bifrost
restart: unless-stopped
```
> **Network tip:** If the adapter and ComfyUI are on the same Layer-2 segment, prefer `--network host` (or attach the container to an existing bridge that reaches ComfyUI) to avoid NAT/port-mapping issues.
---
## OpenWebUI Configuration
1. Admin Panel → **Settings** → **Images**
2. **Image Generation Engine:** `Open AI`
3. **OpenAI API Base URL:** `http://<adapter-host>:9000/v1`
4. **OpenAI API Key:** any dummy string (the adapter does not validate keys)
5. **Model:** `flux2` (or leave default)
---
## Critical Pitfalls
| Issue | Cause | Fix |
|---|---|---|
| `TimeoutError` after 300 s | OpenWebUI/DALL-E 3 sends `size: "4096x4096"` | Adapter clamps to `MAX_SIZE = 1536` |
| OOM / CUDA out of memory | Resolution too high for 48 GB shared VRAM | Keep ≤ 1536 px on consumer APUs |
| `500 Internal Server Error` | ComfyUI node missing (`CLIPLoaderGGUF`, etc.) | Run `comfy node install comfyui-gguf` and verify with `curl /object_info` |
| Adapter cannot reach ComfyUI | Docker bridge isolation | Use `--network host` or place both on same bridge |
| Empty/black images | Wrong VAE (`ae.safetensors` instead of `taef2` for FLUX.2) | Use `taef2` TAE or the FLUX.2-specific VAE |
---
## Environment Snapshot (Schön Consulting, 2026-06-20)
- **ComfyUI:** PVE CT 204, `10.0.30.97:8188`, ROCm gfx1150 (Radeon 890M)
- **OpenWebUI:** PVE CT 135, `10.0.30.102:8080`
- **Adapter:** Docker-Host `10.0.30.99:9000` (Portainer node)
- **Performance:** 1024×1024 ≈ 110 s, 1536×1536 ≈ 180 s (4 steps, euler/simple)
creative/comfyui/references/proxmox-lxc.md
+194
View File
@@ -0,0 +1,194 @@
# Working with ComfyUI inside a Proxmox LXC over SSH
When ComfyUI is hosted inside a Proxmox CT (e.g. CT 204 on 10.0.30.97) and you
need to interact with it from a machine that only has SSH access to the PVE host
(10.0.20.91), the most reliable pattern is:
## ❌ Don't: multi-layer nested quotes
```bash
# FAILS — 4-level escaping breaks on !, (, ", etc.
ssh root@pve "pct exec 204 -- python3 -c \"print('hello')\""
ssh root@pve "pct exec 204 -- python3 -c 'import struct; ...'
```
Result: `bash: -c: line 34: syntax error near unexpected token ('`
## ✅ Do: write-then-execute
```bash
# 1. SSH to PVE host and create a script file there
ssh root@10.0.20.91 << 'HEREDOC'
cat > /tmp/my_script.py << 'PYEOF'
import struct, json, os
path = "/opt/ComfyUI/models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors"
f = open(path, "rb")
hlen = struct.unpack("<Q", f.read(8))[0]
meta = json.loads(f.read(hlen))
total = 8 + hlen
for k, v in meta.items():
if isinstance(v, dict) and "data_offsets" in v:
total += v["data_offsets"][1] - v["data_offsets"][0]
actual = os.path.getsize(path)
print(f"expected={total} actual={actual} ok={total==actual}")
PYEOF
HEREDOC
# 2. Push script into LXC container
ssh root@10.0.20.91 "pct push 204 /tmp/my_script.py /tmp/my_script.py"
# 3. Execute inside container
ssh root@10.0.20.91 "pct exec 204 -- python3 /tmp/my_script.py"
# 4. Clean up
ssh root@10.0.20.91 "rm /tmp/my_script.py"
```
## Template: compact one-liner for checks
```bash
ssh root@10.0.20.91 'bash -s' << 'SSHSCRIPT'
cat > /tmp/probe.py << 'PYEOF'
from safetensors import safe_open
f = safe_open("/opt/ComfyUI/models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors",
framework="pt", device="cpu")
print("OK, tensors:", len(list(f.keys())))
PYEOF
pct push 204 /tmp/probe.py /tmp/probe.py
pct exec 204 -- python3 /tmp/probe.py
rm /tmp/probe.py
SSHSCRIPT
```
## Copying large files into LXC (e.g. model downloads)
```bash
# Download to PVE host tmp
ssh root@10.0.20.91 "cd /tmp && curl -L -o model.safetensors <URL>"
# Push into container
ssh root@10.0.20.91 "pct push 204 /tmp/model.safetensors /opt/ComfyUI/models/text_encoders/model.safetensors"
# Verify
ssh root@10.0.20.91 "pct exec 204 -- python3 /tmp/probe.py"
# Clean up host
ssh root@10.0.20.91 "rm /tmp/model.safetensors"
```
## Persistent services inside LXC
ComfyUI and adapters should run as systemd services so they survive container restarts and reboots without manual intervention. The pattern:
1. Check if a service already exists before creating: `ls /etc/systemd/system/comfyui.service`
2. Stop any manual `nohup` / background processes (they hold ports 8188/8000)
3. Create a `.service` file on the PVE host, `pct push` into LXC `/etc/systemd/system/`
4. `systemctl daemon-reload && systemctl enable --now <service>`
### Stopping manual processes before systemd takeover
If ComfyUI was previously started with `nohup python3 main.py --listen 0.0.0.0 --port 8188 &`, the process survives and keeps port 8188 occupied. `systemctl start comfyui.service` will then fail with:
```
[ERROR] Port 8188 is already in use on address 0.0.0.0
```
Fix:
```bash
pct exec 204 -- bash -c "pkill -9 -f 'main.py.*8188'; fuser -k 8000/tcp 2>/dev/null || true"
```
### Example: ComfyUI (may already exist on CT 204)
```bash
# Check existing service first
pct exec 204 -- cat /etc/systemd/system/comfyui.service 2>/dev/null || echo "NOT_FOUND"
```
Already installed on Schön Consulting CT 204:
```ini
# /etc/systemd/system/comfyui.service
[Unit]
Description=ComfyUI
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/opt/ComfyUI
ExecStart=/usr/bin/python3 main.py --listen 0.0.0.0 --port 8188
Restart=always
RestartSec=10
Environment=HIP_VISIBLE_DEVICES=0
Environment=PATH=/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin
Environment=LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/torch/lib:/opt/rocm/lib
[Install]
WantedBy=multi-user.target
```
### Example: Ideogram4 OpenAI API Wrapper
Template in `templates/ideogram4-api.service`:
```bash
# Write on PVE host, then push
pct push 204 /tmp/ideogram4-api.service /etc/systemd/system/ideogram4-api.service
pct exec 204 -- systemctl daemon-reload
pct exec 204 -- systemctl enable --now ideogram4-api.service
pct exec 204 -- systemctl status ideogram4-api.service
```
Key points:
- `Requires=comfyui.service` so the wrapper only starts when ComfyUI is up
- Use full paths to venv Python binary (`/opt/ideogram4-api/venv/bin/python`) so systemd doesn't need PATH activation
- `ExecStart` filename must match actual file on disk (`main.py`, `app.py`, or `adapter.py` — check `ls /opt/ideogram4-api/`)
- `Restart=always` keeps the wrapper alive across ComfyUI restarts
## Key takeaways
- Use `'%help'` heredoc delimiter on outer SSH to prevent local variable substitution
- Use `'PYEOF'` heredoc for inner Python to prevent expansion of Python braces
- `pct push` is faster and more reliable than `cat | pct exec tee`
- For systemd service files, `pct push` is also the cleanest way to install them
- Always verify model integrity after copy: struct-unpack safetensors header + compare data offsets
## Concrete environment defaults
For the Schön Consulting Proxmox VE setup:
- **PVE host:** `10.0.20.91`, user `root`, password `28acaneltO!#`
- **ComfyUI+ROCm LXC:** CTID `204`, IP `10.0.30.97`
- **ComfyUI root:** `/opt/ComfyUI`
- **API adapter root:** `/opt/ideogram4-api`
- **Common model paths:**
- `/opt/ComfyUI/models/diffusion_models/`
- `/opt/ComfyUI/models/unet/` (FLUX GGUF models)
- `/opt/ComfyUI/models/text_encoders/` (also used as `/opt/ComfyUI/models/clip/`)
- `/opt/ComfyUI/models/vae/`
- `/opt/ComfyUI/models/checkpoints/`
### Fast probe via sshpass (single-step)
When already authenticated, use `sshpass` to skip interactive password entry:
```bash
export SSHPASS="28acaneltO!#"
# Check if ComfyUI server is responsive
sshpass -e ssh -o StrictHostKeyChecking=no root@10.0.20.91 \
"pct exec 204 -- bash -c 'curl -s http://localhost:8188/system_stats | head -c 200'"
# List models in a folder
sshpass -e ssh -o StrictHostKeyChecking=no root@10.0.20.91 \
"pct exec 204 -- ls -la /opt/ComfyUI/models/diffusion_models/"
# Check a safetensors file without parsing
sshpass -e ssh -o StrictHostKeyChecking=no root@10.0.20.91 \
"pct exec 204 -- python3 -c 'from safetensors import safe_open; f=safe_open(\"/opt/ComfyUI/models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors\", framework=\"pt\", device=\"cpu\"); print(\"OK, tensors:\", len(list(f.keys())))'"
```
### Pitfall: `pct exec` with inline Python over `sshpass` still breaks
Even with sshpass, inline `-c` Python strings fail due to double-shell escaping. The pattern below **WILL NOT WORK**:
```bash
# FAILS — nested quotes collapse
sshpass -e ssh root@pve "pct exec 204 -- python3 -c 'import struct; ...'"
# → bash: eval: line N: unexpected EOF while looking for matching `"'
```
The **only reliable approach** is `pct push` + `pct exec` (see write-then-execute above).
creative/comfyui/references/rest-api.md
+312
View File
@@ -0,0 +1,312 @@
# ComfyUI REST + WebSocket API Reference
ComfyUI exposes a REST + WebSocket interface for workflow execution and
management. **The same surface is used locally and on Comfy Cloud, with
auth/path differences.**
## Connection
| | Local ComfyUI | Comfy Cloud |
|---|---|---|
| Base URL | `http://127.0.0.1:8188` | `https://cloud.comfy.org` |
| API path prefix | none (`/prompt`, `/view`, …) | `/api/...` (`/api/prompt`, `/api/view`, …) |
| Auth | none (or bearer token if configured) | `X-API-Key` header |
| WebSocket | `ws://host:port/ws?clientId={uuid}` | `wss://cloud.comfy.org/ws?clientId={uuid}&token={API_KEY}` |
| `/api/view` response | direct bytes | 302 redirect → signed URL (use `curl -L`) |
The skill scripts route URLs automatically via `_common.resolve_url()`.
## Endpoint differences on Comfy Cloud
The cloud surface diverges from local ComfyUI in several ways. The skill
scripts handle these transparently; document them here so anyone calling
`curl` directly knows.
| Local path | Cloud path | Notes |
|------------|-----------|-------|
| `/system_stats` | `/api/system_stats` | Cloud version is **public** (no auth required) |
| `/object_info` | `/api/object_info` | **Paid tier only** — free returns 403 |
| `/queue` | `/api/queue` | Paid tier only |
| `/userdata` | `/api/userdata` | Paid tier only |
| `/prompt` (POST) | `/api/prompt` (POST) | Paid tier only |
| `/upload/image` | `/api/upload/image` | Paid tier only; `subfolder` accepted but ignored |
| `/upload/mask` | `/api/upload/mask` | Same as above |
| `/view` | `/api/view` | Paid tier only; **returns 302** to signed URL |
| `/history` | `/api/history_v2` | **Renamed**; old path returns 404 |
| `/history/{id}` | `/api/history_v2/{id}` or `/api/jobs/{id}` | Both work; `/jobs` returns full job |
| `/models` | `/api/experiment/models` | **Renamed** |
| `/models/{folder}` | `/api/experiment/models/{folder}` | **Renamed**; response shape differs (see below) |
### Cloud model-list response shape
- **Local:** `["a.safetensors", "b.safetensors", …]` — flat list of strings.
- **Cloud:** `[{"name": "a.safetensors", "pathIndex": 0}, …]` — list of objects.
- **Cloud 404 with `code: "folder_not_found"`** — folder is empty or unknown,
not an "endpoint missing" error. Distinguish by reading the body.
The skill helper `_common.parse_model_list()` normalizes both.
## Workflow Execution
### Submit Workflow
```bash
# Local
curl -X POST "http://127.0.0.1:8188/prompt" \
-H "Content-Type: application/json" \
-d '{"prompt": '"$(cat workflow_api.json)"', "client_id": "'"$(uuidgen)"'"}'
# Cloud
curl -X POST "https://cloud.comfy.org/api/prompt" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt": '"$(cat workflow_api.json)"'}'
```
**Response:**
```json
{"prompt_id": "abc-123-def", "number": 1, "node_errors": {}}
```
If `node_errors` is non-empty, the workflow has validation errors (missing
nodes, bad inputs).
### Check Job Status (Cloud)
```bash
curl -X GET "https://cloud.comfy.org/api/job/{prompt_id}/status" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
```
| Status | Description |
| ------------- | ---------------------------------- |
| `pending` | Job is queued and waiting to start |
| `in_progress` | Job is currently executing |
| `completed` | Job finished successfully |
| `failed` | Job encountered an error |
| `cancelled` | Job was cancelled by user |
### Job detail with outputs (Cloud)
```bash
curl -X GET "https://cloud.comfy.org/api/jobs/{prompt_id}" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
```
Response includes `outputs` keyed by node ID. Cloud uses `video` (singular)
in the output structure; local uses `videos` (plural). The skill scripts
accept both.
### Get History (Local)
```bash
curl -s "http://127.0.0.1:8188/history" # all
curl -s "http://127.0.0.1:8188/history/{id}" # one prompt_id
```
Local entry shape:
```json
{
"<prompt_id>": {
"prompt": [...],
"outputs": {"<node_id>": {"images": [...]}},
"status": {
"status_str": "success" | "error",
"completed": true | false,
"messages": [["execution_start", {...}], ["execution_error", {...}], …]
}
}
}
```
**Important:** when reading status, check `status_str == "error"` BEFORE
checking `completed`, because both can be true for failed runs.
### Download Output
```bash
# Local (direct bytes)
curl -s "http://127.0.0.1:8188/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \
-o output.png
# Cloud (302 → signed URL; -L follows; STRIP X-API-Key for the second hop)
curl -L "https://cloud.comfy.org/api/view?filename=...&type=output" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY" \
-o output.png
```
The skill's `run_workflow.py` strips `X-API-Key` automatically on the
cross-host redirect, so the signed URL never sees your auth.
## WebSocket Monitoring
Connect for real-time execution events.
```bash
# Local
wscat -c "ws://127.0.0.1:8188/ws?clientId=MY-UUID"
# Cloud
wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=$COMFY_CLOUD_API_KEY"
```
**Note:** on Cloud the `clientId` is currently ignored — all messages for a
user are broadcast to every connection. Filter messages client-side by
`data.prompt_id`.
### JSON Message Types
| Type | When | Key Fields |
|------|------|------------|
| `status` | Queue change | `status.exec_info.queue_remaining` |
| `notification` | User-friendly status string | `value` |
| `execution_start` | Workflow begins | `prompt_id` |
| `executing` | Node running (or end-of-run if `node` is null on local) | `node`, `prompt_id` |
| `progress` | Sampling steps | `node`, `value`, `max` |
| `progress_state` | Extended progress with per-node metadata | `nodes` (dict) |
| `executed` | Node output ready | `node`, `output` (with `images`/`video`/etc.) |
| `execution_cached` | Nodes skipped because of cache | `nodes` (list of IDs) |
| `execution_success` | All done | `prompt_id` |
| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback`, `node_id` |
| `execution_interrupted` | Cancelled | `prompt_id` |
### Binary Frames (Preview Images)
| Type code | Meaning |
|-----------|---------|
| `0x00000001` | `PREVIEW_IMAGE` — `[type:4][image_type:4][data]` (image_type 1=JPEG, 2=PNG) |
| `0x00000003` | `TEXT` — `[type:4][nid_len:4][nid][text]` (UTF-8) |
| `0x00000004` | `PREVIEW_IMAGE_WITH_METADATA` — `[type:4][meta_len:4][json][image_data]` |
`scripts/ws_monitor.py --previews <dir>` saves preview frames to disk.
## File Upload
```bash
# Image
curl -X POST "http://127.0.0.1:8188/upload/image" \
-F "image=@photo.png" -F "type=input" -F "overwrite=true"
# Returns: {"name": "photo.png", "subfolder": "", "type": "input"}
# Mask (linked to a previously uploaded image)
curl -X POST "http://127.0.0.1:8188/upload/mask" \
-F "image=@mask.png" -F "type=input" \
-F 'original_ref={"filename":"photo.png","subfolder":"","type":"input"}'
```
Cloud equivalent: prepend `https://cloud.comfy.org/api` and add `-H "X-API-Key: $COMFY_CLOUD_API_KEY"`.
## Node & Model Discovery
```bash
# All node types and their input specs
curl -s "http://127.0.0.1:8188/object_info" | python3 -m json.tool
# Specific node
curl -s "http://127.0.0.1:8188/object_info/KSampler"
# Models per folder (local)
curl -s "http://127.0.0.1:8188/models/checkpoints"
curl -s "http://127.0.0.1:8188/models/loras"
# Models per folder (cloud — note the experimental prefix)
curl -s "https://cloud.comfy.org/api/experiment/models/checkpoints" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
```
## Queue Management
```bash
# View queue
curl -s "http://127.0.0.1:8188/queue"
# Clear all pending
curl -X POST "http://127.0.0.1:8188/queue" \
-H "Content-Type: application/json" \
-d '{"clear": true}'
# Delete specific items
curl -X POST "http://127.0.0.1:8188/queue" \
-H "Content-Type: application/json" \
-d '{"delete": ["prompt_id_1", "prompt_id_2"]}'
# Cancel currently-running job
curl -X POST "http://127.0.0.1:8188/interrupt"
```
## System Management
```bash
# Stats (VRAM, RAM, GPU, ComfyUI version)
curl -s "http://127.0.0.1:8188/system_stats"
# Free GPU memory
curl -X POST "http://127.0.0.1:8188/free" \
-H "Content-Type: application/json" \
-d '{"unload_models": true, "free_memory": true}'
```
## ComfyUI-Manager Endpoints (Optional)
These require ComfyUI-Manager installed. Useful for installing nodes/models
via the API instead of `comfy-cli`.
```bash
# Install a custom node from a git URL
curl -X POST "http://127.0.0.1:8188/manager/queue/install" \
-H "Content-Type: application/json" \
-d '{"git_url": "https://github.com/user/comfyui-node.git"}'
# Check install queue status
curl -s "http://127.0.0.1:8188/manager/queue/status"
# Install model
curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \
-H "Content-Type: application/json" \
-d '{"url": "https://...", "path": "models/checkpoints", "filename": "model.safetensors"}'
```
## POST /prompt Payload Format
```json
{
"prompt": {
"3": {
"class_type": "KSampler",
"inputs": {
"seed": 42,
"steps": 20,
"cfg": 7.5,
"sampler_name": "euler",
"scheduler": "normal",
"denoise": 1.0,
"model": ["4", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["5", 0]
}
}
},
"client_id": "unique-uuid-for-ws-filtering",
"extra_data": {
"api_key_comfy_org": "optional-PARTNER-NODE-key (NOT the cloud auth key)"
}
}
```
- `prompt`: workflow graph in API format
- `client_id`: UUID — local server uses it to filter WebSocket events; cloud
ignores it.
- `extra_data.api_key_comfy_org`: ONLY required when the workflow uses
partner nodes (Flux Pro, Ideogram, etc.). Don't conflate with `X-API-Key`.
## Error Categories (cloud `execution_error` `exception_type`)
| Type | Meaning |
|------|---------|
| `ValidationError` | Bad workflow / inputs (often nicer to surface from `node_errors`) |
| `ModelDownloadError` | Required model not available |
| `ImageDownloadError` | Failed to fetch input image from URL |
| `OOMError` | Out of GPU memory |
| `InsufficientFundsError` | Account balance too low (partner nodes) |
| `InactiveSubscriptionError` | Subscription not active |
creative/comfyui/references/workflow-format.md
+226
View File
@@ -0,0 +1,226 @@
# ComfyUI Workflow JSON Format
## Two Formats — Only API Format Is Executable
**API format** is required for `/api/prompt` and every script in this skill.
The web UI also produces an "editor format" used for visual editing, which
**cannot** be submitted directly.
### API Format
Top-level keys are string node IDs. Each node has `class_type` and `inputs`:
```json
{
"3": {
"class_type": "KSampler",
"inputs": {
"seed": 156680208700286,
"steps": 20,
"cfg": 8,
"sampler_name": "euler",
"scheduler": "normal",
"denoise": 1.0,
"model": ["4", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["5", 0]
},
"_meta": {"title": "KSampler"}
},
"4": {
"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"}
}
}
```
**Detection:** every top-level value has `class_type`. The skill's
`_common.is_api_format()` does this check.
### Editor Format (not directly executable)
Has `nodes[]` and `links[]` arrays — the visual graph. To convert: open in
ComfyUI's web UI and use **Workflow → Export (API)** (newer UI) or the
"Save (API Format)" button (older UI).
**Detection:** top-level has `"nodes"` and `"links"` keys.
## Inputs: Literals vs Links
```json
"inputs": {
"text": "a cat", // literal — modifiable
"seed": 42, // literal — modifiable
"clip": ["4", 1] // link — wiring; do NOT overwrite
}
```
Links are length-2 arrays of `[upstream_node_id, output_slot]`. The skill's
parameter injector refuses to overwrite a link with a literal (logs a
warning and skips).
## Common Node Types and Their Controllable Parameters
The full catalog lives in `scripts/_common.py` (`PARAM_PATTERNS` and
`MODEL_LOADERS`). Highlights:
### Text Prompts
| Node Class | Key Fields |
|------------|------------|
| `CLIPTextEncode` | `text` |
| `CLIPTextEncodeSDXL` | `text_g`, `text_l`, `width`, `height` |
| `CLIPTextEncodeFlux` | `clip_l`, `t5xxl`, `guidance` |
To distinguish positive from negative the skill traces `KSampler.negative`
back through Reroute / Primitive nodes to the source CLIPTextEncode. Falls
back to `_meta.title` heuristics ("negative", "neg", "anti").
### Sampling
| Node Class | Key Fields |
|------------|------------|
| `KSampler` | `seed`, `steps`, `cfg`, `sampler_name`, `scheduler`, `denoise` |
| `KSamplerAdvanced` | `noise_seed`, `steps`, `cfg`, `start_at_step`, `end_at_step` |
| `SamplerCustom` | `noise_seed`, `cfg`, `sampler`, `sigmas` |
| `SamplerCustomAdvanced` | `noise_seed` (via RandomNoise input) |
| `RandomNoise` | `noise_seed` |
| `BasicScheduler` | `steps`, `scheduler`, `denoise` |
| `KSamplerSelect` | `sampler_name` |
| `BasicGuider` / `CFGGuider` | `cfg` |
| `ModelSamplingFlux` | `max_shift`, `base_shift`, `width`, `height` |
| `SDTurboScheduler` | `steps`, `denoise` |
### Latent / Dimensions
| Node Class | Key Fields |
|------------|------------|
| `EmptyLatentImage` | `width`, `height`, `batch_size` |
| `EmptySD3LatentImage` | `width`, `height`, `batch_size` |
| `EmptyHunyuanLatentVideo` | `width`, `height`, `length`, `batch_size` |
| `EmptyMochiLatentVideo` | `width`, `height`, `length`, `batch_size` |
| `EmptyLTXVLatentVideo` | `width`, `height`, `length`, `batch_size` |
### Model Loading
| Node Class | Key Fields | Folder |
|------------|------------|--------|
| `CheckpointLoaderSimple` | `ckpt_name` | `checkpoints` |
| `LoraLoader` | `lora_name`, `strength_model`, `strength_clip` | `loras` |
| `LoraLoaderModelOnly` | `lora_name`, `strength_model` | `loras` |
| `VAELoader` | `vae_name` | `vae` |
| `ControlNetLoader` | `control_net_name` | `controlnet` |
| `CLIPLoader` | `clip_name` | `clip` |
| `DualCLIPLoader` | `clip_name1`, `clip_name2` | `clip` |
| `TripleCLIPLoader` | `clip_name1/2/3` | `clip` |
| `UNETLoader` | `unet_name` | `unet` |
| `DiffusionModelLoader` | `model_name` | `diffusion_models` |
| `UpscaleModelLoader` | `model_name` | `upscale_models` |
| `IPAdapterModelLoader` | `ipadapter_file` | `ipadapter` |
| `ADE_AnimateDiffLoaderWithContext` | `model_name`, `motion_scale` | `animatediff_models` |
### Image Input/Output
| Node Class | Key Fields |
|------------|------------|
| `LoadImage` | `image` (server-side filename, after upload) |
| `LoadImageMask` | `image`, `channel` (`red` / `green` / `blue` / `alpha`) |
| `VAEEncode` / `VAEDecode` | (no controllable fields) |
| `VAEEncodeForInpaint` | `grow_mask_by` |
| `SaveImage` | `filename_prefix` |
| `VHS_VideoCombine` | `frame_rate`, `format`, `filename_prefix`, `loop_count`, `pingpong` |
### ControlNet
| Node Class | Key Fields |
|------------|------------|
| `ControlNetApply` | `strength` |
| `ControlNetApplyAdvanced` | `strength`, `start_percent`, `end_percent` |
### IPAdapter (community pack `comfyui_ipadapter_plus`)
| Node Class | Key Fields |
|------------|------------|
| `IPAdapterAdvanced` | `weight`, `start_at`, `end_at` |
| `IPAdapter` | `weight` |
### Embeddings (referenced inside prompt strings)
ComfyUI scans prompt text for `embedding:NAME` syntax. The skill's
`_common.iter_embedding_refs()` extracts these as model dependencies.
```text
"a beautiful cat, embedding:goodvibes:1.2, embedding:art-style"
```
`extract_schema.py` and `check_deps.py` surface these in
`embedding_dependencies` / `missing_embeddings`.
## Parameter Injection Pattern
```python
import json, copy
with open("workflow_api.json") as f:
workflow = json.load(f)
wf = copy.deepcopy(workflow)
wf["6"]["inputs"]["text"] = "a beautiful sunset"
wf["7"]["inputs"]["text"] = "ugly, blurry"
wf["3"]["inputs"]["seed"] = 42
wf["3"]["inputs"]["steps"] = 30
wf["5"]["inputs"]["width"] = 1024
wf["5"]["inputs"]["height"] = 1024
```
`scripts/extract_schema.py` automates discovering which node IDs/fields
correspond to which user-facing parameters. It returns a `parameters` dict
that `run_workflow.py` reads to inject values from `--args`.
## Identifying Controllable Parameters (Heuristics)
For unknown workflows:
1. **Prompt text** — any `CLIPTextEncode.text`. Use connection tracing back
from `KSampler.positive` / `.negative` to disambiguate (don't trust
meta-title alone).
2. **Seed** — `KSampler.seed` / `KSamplerAdvanced.noise_seed` / `RandomNoise.noise_seed`.
3. **Dimensions** — `Empty*LatentImage.width/height` (must be multiples of 8).
4. **Steps / CFG** — `KSampler.steps`, `KSampler.cfg`. Steps 20–50 typical.
CFG 5–15 typical (Flux uses guidance, not CFG).
5. **Model / checkpoint** — `CheckpointLoaderSimple.ckpt_name`. Filename must
match an installed file *exactly*.
6. **LoRA** — `LoraLoader.lora_name`, `.strength_model`.
7. **Images for img2img / inpaint** — `LoadImage.image`. Server-side filename
after upload.
8. **Denoise** — `KSampler.denoise`. 0.0–1.0; 1.0 = ignore input image,
0.0 = pass through. Sweet spot for img2img: 0.4–0.7.
## Output Nodes
Output is produced by these node types. The skill's `OUTPUT_NODES` set
extends to common community packs.
| Node | Output Key | Content |
|------|-----------|---------|
| `SaveImage` | `images` | List of `{filename, subfolder, type}` |
| `PreviewImage` | `images` | Temporary preview (not saved) |
| `VHS_VideoCombine` | `gifs` (older) or `videos`/`video` (newer cloud) | Video file refs |
| `SaveAudio` | `audio` | Audio file refs |
| `SaveAnimatedWEBP` / `SaveAnimatedPNG` | `images` | Animated images |
| `Save3D` | `3d` | 3D asset refs |
After execution, fetch outputs from `/history/{prompt_id}` (local) or
`/api/jobs/{prompt_id}` (cloud) → `outputs` → `{node_id}` → `{key}`.
## Wrapper Variants
Some saved JSON files wrap the workflow under a `"prompt"` key (matching
the `/api/prompt` payload shape). The skill's `_common.unwrap_workflow()`
handles this — pass any of:
- raw API format: `{"3": {...}, "4": {...}}`
- wrapped: `{"prompt": {"3": {...}}, "client_id": "..."}`
It rejects editor format with a clear error and a re-export instruction.
creative/comfyui/scripts/_common.py
+835
View File
@@ -0,0 +1,835 @@
"""
_common.py — Shared logic for ComfyUI skill scripts.
Single source of truth for:
- HTTP transport (with retry/backoff, streaming, timeout handling)
- Cloud detection and endpoint mapping (local ComfyUI vs Comfy Cloud)
- Workflow node-type catalogs (param patterns, model loaders, output nodes)
- API-format validation
- Path-traversal-safe file writes
- API-key loading from env / CLI
Stdlib-only by design (with optional `requests` upgrade if installed). Python 3.10+.
"""
from __future__ import annotations
import json
import os
import random
import re
import sys
import time
import uuid
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterator
from urllib.parse import urlparse
# Optional: prefer `requests` if installed (better redirects, streaming, header handling)
try:
import requests # type: ignore[import-not-found]
HAS_REQUESTS = True
except ImportError: # pragma: no cover - exercised via stdlib fallback
HAS_REQUESTS = False
import urllib.error
import urllib.request
# =============================================================================
# Constants & catalogs
# =============================================================================
DEFAULT_LOCAL_HOST = "http://127.0.0.1:8188"
DEFAULT_CLOUD_HOST = "https://cloud.comfy.org"
ENV_API_KEY = "COMFY_CLOUD_API_KEY"
# Connection / retry defaults
DEFAULT_HTTP_TIMEOUT = 60 # seconds — single-attempt request timeout
DEFAULT_RETRIES = 3 # total attempts including the first
RETRY_BASE_DELAY = 1.0 # seconds — exponential backoff base
RETRY_MAX_DELAY = 30.0 # seconds — cap on backoff
RETRY_STATUS_CODES = {408, 429, 500, 502, 503, 504, 522, 524}
# Streaming download chunk size (bytes)
DOWNLOAD_CHUNK_SIZE = 1 << 16 # 64 KiB
# Heuristic: workflows with these node types tend to be slow → larger default timeout
SLOW_OUTPUT_NODES = {
"VHS_VideoCombine", "SaveAnimatedWEBP", "SaveAnimatedPNG",
"SaveVideo", "SaveAudio", "SaveAnimateDiffVideo",
"SVD_img2vid_Conditioning",
"WanVideoSampler", "HunyuanVideoSampler",
"CogVideoSampler", "LTXVideoSampler",
}
# ---------------------------------------------------------------------------
# Output node catalog (extensible — community packs add their own)
# ---------------------------------------------------------------------------
OUTPUT_NODES: set[str] = {
# Built-in
"SaveImage", "PreviewImage",
"SaveAudio", "SaveVideo", "PreviewAudio", "PreviewVideo",
"SaveAnimatedWEBP", "SaveAnimatedPNG",
# Common community packs
"VHS_VideoCombine", # Video Helper Suite
"ImageSave", # Was Node Suite
"Image Save", # Was Node Suite (alt name)
"easy imageSave", # easy-use
"Image Save With Metadata",
"PreviewImage|pysssss", # pysssss preview
"ShowText|pysssss",
"SaveLatent",
"SaveGLB", # 3D
"Save3D",
}
# ---------------------------------------------------------------------------
# Folder aliases — handle ComfyUI's gradual folder renames
# ---------------------------------------------------------------------------
# When `check_deps.py` queries `/models/<folder>` and gets 404 / empty,
# it tries each alias in turn. Critical for Comfy Cloud which has fully
# migrated to the new naming (unet → diffusion_models, clip → text_encoders).
FOLDER_ALIASES: dict[str, list[str]] = {
"unet": ["unet", "diffusion_models"],
"diffusion_models": ["diffusion_models", "unet"],
"clip": ["clip", "text_encoders"],
"text_encoders": ["text_encoders", "clip"],
"controlnet": ["controlnet", "control_net"],
}
def folder_aliases_for(folder: str) -> list[str]:
"""Return the search order of folder names (primary first)."""
return FOLDER_ALIASES.get(folder, [folder])
# ---------------------------------------------------------------------------
# Model-loader catalog: class_type -> (input field, model folder)
# ---------------------------------------------------------------------------
# A loader can have multiple fields (e.g., DualCLIPLoader has clip_name1 and
# clip_name2). We list them with explicit entries. The folder name is the
# *canonical* one; FOLDER_ALIASES is consulted when querying.
MODEL_LOADERS: dict[str, list[tuple[str, str]]] = {
# Checkpoints
"CheckpointLoaderSimple": [("ckpt_name", "checkpoints")],
"CheckpointLoader": [("ckpt_name", "checkpoints")],
"CheckpointLoader (Simple)": [("ckpt_name", "checkpoints")],
"ImageOnlyCheckpointLoader": [("ckpt_name", "checkpoints")],
"unCLIPCheckpointLoader": [("ckpt_name", "checkpoints")],
# LoRA
"LoraLoader": [("lora_name", "loras")],
"LoraLoaderModelOnly": [("lora_name", "loras")],
"LoraLoaderTagsQuery": [("lora_name", "loras")],
# VAE
"VAELoader": [("vae_name", "vae")],
# ControlNet
"ControlNetLoader": [("control_net_name", "controlnet")],
"DiffControlNetLoader": [("control_net_name", "controlnet")],
"ControlNetLoaderAdvanced": [("control_net_name", "controlnet")],
# CLIP / text encoders (primary "clip" folder; check_deps tries text_encoders too)
"CLIPLoader": [("clip_name", "clip")],
"DualCLIPLoader": [("clip_name1", "clip"), ("clip_name2", "clip")],
"TripleCLIPLoader": [("clip_name1", "clip"), ("clip_name2", "clip"), ("clip_name3", "clip")],
"CLIPVisionLoader": [("clip_name", "clip_vision")],
# UNET / Diffusion model (primary "unet"; check_deps tries diffusion_models too)
"UNETLoader": [("unet_name", "unet")],
"DiffusionModelLoader": [("model_name", "diffusion_models")],
"UNETLoaderGGUF": [("unet_name", "unet")],
# Upscaler
"UpscaleModelLoader": [("model_name", "upscale_models")],
# Style / GLIGEN / Hypernetwork
"StyleModelLoader": [("style_model_name", "style_models")],
"GLIGENLoader": [("gligen_name", "gligen")],
"HypernetworkLoader": [("hypernetwork_name", "hypernetworks")],
# IPAdapter family (community).
# Note: IPAdapterUnifiedLoader's `preset` and IPAdapterInsightFaceLoader's
# `provider` are enums (not file paths), so they're intentionally omitted —
# check_deps would otherwise treat enum values as missing model files.
"IPAdapterModelLoader": [("ipadapter_file", "ipadapter")],
"InstantIDModelLoader": [("instantid_file", "instantid")],
# AnimateDiff / video
"ADE_LoadAnimateDiffModel": [("model_name", "animatediff_models")],
"ADE_AnimateDiffLoaderWithContext": [("model_name", "animatediff_models")],
"ADE_AnimateDiffLoaderGen1": [("model_name", "animatediff_models")],
# Photomaker
"PhotoMakerLoader": [("photomaker_model_name", "photomaker")],
# Sampler / scheduler models
"ModelSamplingFlux": [], # parametric only
}
# ---------------------------------------------------------------------------
# Param patterns: (class_type, field_name) -> friendly_name
# Order matters — first match wins for naming. Use _meta.title for disambiguation.
# ---------------------------------------------------------------------------
PARAM_PATTERNS: list[tuple[str, str, str]] = [
# ---- Prompts ----
("CLIPTextEncode", "text", "prompt"),
("CLIPTextEncodeSDXL", "text_g", "prompt"),
("CLIPTextEncodeSDXL", "text_l", "prompt_l"),
("CLIPTextEncodeSDXLRefiner", "text", "refiner_prompt"),
("CLIPTextEncodeFlux", "clip_l", "prompt_l"),
("CLIPTextEncodeFlux", "t5xxl", "prompt"),
("CLIPTextEncodeFlux", "guidance", "guidance"),
("smZ CLIPTextEncode", "text", "prompt"),
("BNK_CLIPTextEncodeAdvanced", "text", "prompt"),
# ---- Standard sampling ----
("KSampler", "seed", "seed"),
("KSampler", "steps", "steps"),
("KSampler", "cfg", "cfg"),
("KSampler", "sampler_name", "sampler_name"),
("KSampler", "scheduler", "scheduler"),
("KSampler", "denoise", "denoise"),
("KSamplerAdvanced", "noise_seed", "seed"),
("KSamplerAdvanced", "steps", "steps"),
("KSamplerAdvanced", "cfg", "cfg"),
("KSamplerAdvanced", "sampler_name", "sampler_name"),
("KSamplerAdvanced", "scheduler", "scheduler"),
("KSamplerAdvanced", "start_at_step", "start_at_step"),
("KSamplerAdvanced", "end_at_step", "end_at_step"),
# ---- Modern sampler chain (Flux / SD3 / SDXL refiner via SamplerCustom) ----
("RandomNoise", "noise_seed", "seed"),
("BasicScheduler", "steps", "steps"),
("BasicScheduler", "scheduler", "scheduler"),
("BasicScheduler", "denoise", "denoise"),
("KSamplerSelect", "sampler_name", "sampler_name"),
# NB: BasicGuider has no cfg input (it just bundles model+conditioning).
("CFGGuider", "cfg", "cfg"),
("DualCFGGuider", "cfg_conds", "cfg"),
("DualCFGGuider", "cfg_cond2_negative", "cfg_negative"),
("ModelSamplingFlux", "max_shift", "max_shift"),
("ModelSamplingFlux", "base_shift", "base_shift"),
("ModelSamplingFlux", "width", "model_width"),
("ModelSamplingFlux", "height", "model_height"),
("ModelSamplingSD3", "shift", "shift"),
("ModelSamplingDiscrete", "sampling", "sampling"),
("SDTurboScheduler", "steps", "steps"),
("SDTurboScheduler", "denoise", "denoise"),
("SamplerCustom", "noise_seed", "seed"),
("SamplerCustom", "cfg", "cfg"),
# NB: SamplerCustomAdvanced takes a NOISE input (from RandomNoise) — no seed field directly.
# ---- Dimensions / latent ----
("EmptyLatentImage", "width", "width"),
("EmptyLatentImage", "height", "height"),
("EmptyLatentImage", "batch_size", "batch_size"),
("EmptySD3LatentImage", "width", "width"),
("EmptySD3LatentImage", "height", "height"),
("EmptySD3LatentImage", "batch_size", "batch_size"),
("EmptyHunyuanLatentVideo", "width", "width"),
("EmptyHunyuanLatentVideo", "height", "height"),
("EmptyHunyuanLatentVideo", "length", "length"),
("EmptyHunyuanLatentVideo", "batch_size", "batch_size"),
("EmptyMochiLatentVideo", "width", "width"),
("EmptyMochiLatentVideo", "height", "height"),
("EmptyMochiLatentVideo", "length", "length"),
("EmptyLTXVLatentVideo", "width", "width"),
("EmptyLTXVLatentVideo", "height", "height"),
("EmptyLTXVLatentVideo", "length", "length"),
("LatentUpscale", "width", "upscale_width"),
("LatentUpscale", "height", "upscale_height"),
("LatentUpscaleBy", "scale_by", "scale_by"),
("ImageScale", "width", "width"),
("ImageScale", "height", "height"),
# ---- Image input ----
("LoadImage", "image", "image"),
("LoadImageMask", "image", "mask_image"),
("LoadImageOutput", "image", "image"),
("VHS_LoadVideo", "video", "video"),
("VHS_LoadAudio", "audio", "audio"),
# ---- Model selection (sometimes useful to swap per run) ----
("CheckpointLoaderSimple", "ckpt_name", "ckpt_name"),
("CheckpointLoader", "ckpt_name", "ckpt_name"),
("ImageOnlyCheckpointLoader", "ckpt_name", "ckpt_name"),
("VAELoader", "vae_name", "vae_name"),
("UNETLoader", "unet_name", "unet_name"),
("DiffusionModelLoader", "model_name", "diffusion_model_name"),
("UpscaleModelLoader", "model_name", "upscale_model_name"),
("CLIPLoader", "clip_name", "clip_name"),
("DualCLIPLoader", "clip_name1", "clip_name1"),
("DualCLIPLoader", "clip_name2", "clip_name2"),
("ControlNetLoader", "control_net_name", "controlnet_name"),
# ---- LoRA ----
("LoraLoader", "lora_name", "lora_name"),
("LoraLoader", "strength_model", "lora_strength"),
("LoraLoader", "strength_clip", "lora_strength_clip"),
("LoraLoaderModelOnly", "lora_name", "lora_name"),
("LoraLoaderModelOnly", "strength_model", "lora_strength"),
# ---- ControlNet ----
("ControlNetApply", "strength", "controlnet_strength"),
("ControlNetApplyAdvanced", "strength", "controlnet_strength"),
("ControlNetApplyAdvanced", "start_percent", "controlnet_start"),
("ControlNetApplyAdvanced", "end_percent", "controlnet_end"),
# ---- IPAdapter ----
("IPAdapterAdvanced", "weight", "ipadapter_weight"),
("IPAdapterAdvanced", "start_at", "ipadapter_start"),
("IPAdapterAdvanced", "end_at", "ipadapter_end"),
("IPAdapter", "weight", "ipadapter_weight"),
# ---- Upscale ----
("ImageUpscaleWithModel", "upscale_method", "upscale_method"),
# ---- AnimateDiff ----
("ADE_AnimateDiffLoaderWithContext", "motion_scale", "motion_scale"),
("ADE_AnimateDiffLoaderGen1", "motion_scale", "motion_scale"),
# ---- Video / Save ----
("VHS_VideoCombine", "frame_rate", "frame_rate"),
("VHS_VideoCombine", "format", "video_format"),
("VHS_VideoCombine", "filename_prefix", "filename_prefix"),
("SaveImage", "filename_prefix", "filename_prefix"),
# ---- Hunyuan / Wan / LTX video ----
("HunyuanVideoSampler", "seed", "seed"),
("HunyuanVideoSampler", "steps", "steps"),
("HunyuanVideoSampler", "cfg", "cfg"),
("WanVideoSampler", "seed", "seed"),
("WanVideoSampler", "steps", "steps"),
("WanVideoSampler", "cfg", "cfg"),
("LTXVScheduler", "max_shift", "max_shift"),
("LTXVScheduler", "base_shift", "base_shift"),
# ---- rgthree primitives (often used as user-facing inputs) ----
("Seed (rgthree)", "seed", "seed"),
("Image Comparer (rgthree)", "image_a", "image"),
("Power Lora Loader (rgthree)", "PowerLoraLoaderHeaderWidget", "_lora_header"),
# ---- Easy-use / utility primitives ----
("PrimitiveNode", "value", "primitive_value"),
("easy seed", "seed", "seed"),
("easy positive", "positive", "prompt"),
("easy negative", "negative", "negative_prompt"),
("easy fullLoader", "ckpt_name", "ckpt_name"),
("easy fullLoader", "vae_name", "vae_name"),
("easy fullLoader", "lora_name", "lora_name"),
("easy fullLoader", "positive", "prompt"),
("easy fullLoader", "negative", "negative_prompt"),
]
# Prompt-like fields whose value should be scanned for embedding references
PROMPT_FIELDS = {"text", "text_g", "text_l", "t5xxl", "clip_l", "positive", "negative"}
# Pattern matches: embedding:name, embedding:name.pt, embedding:name:1.2, (embedding:name:1.2)
# Word-boundary at start avoids matching things like "no_embedding:foo".
EMBEDDING_REGEX = re.compile(
r"(?:^|[\s,(\[])embedding\s*:\s*([A-Za-z0-9_\-\./\\]+?)(?:\.(?:pt|safetensors|bin))?(?=[\s:,)\(\]]|$)",
re.IGNORECASE,
)
# =============================================================================
# Cloud detection & endpoint routing
# =============================================================================
CLOUD_DOMAIN_SUFFIXES = (".comfy.org",)
CLOUD_DOMAIN_EXACT = {"cloud.comfy.org"}
def is_cloud_host(host: str) -> bool:
"""True if the host points at Comfy Cloud (or staging/preview subdomain)."""
parsed = urlparse(host if "://" in host else f"http://{host}")
hostname = (parsed.hostname or "").lower()
if hostname in CLOUD_DOMAIN_EXACT:
return True
return any(hostname.endswith(s) for s in CLOUD_DOMAIN_SUFFIXES)
def build_cloud_aware_url(base: str, path: str, *, force_cloud: bool | None = None) -> str:
"""Build a URL that adds /api prefix when targeting Comfy Cloud.
Local ComfyUI accepts both `/foo` and `/api/foo` for many endpoints.
Cloud requires `/api/foo`.
`path` should be a path component (e.g. "/prompt") or full path with query
(e.g. "/view?filename=x").
"""
base = base.rstrip("/")
cloud = is_cloud_host(base) if force_cloud is None else force_cloud
if not path.startswith("/"):
path = "/" + path
if cloud and not path.startswith("/api/"):
path = "/api" + path
return base + path
def cloud_endpoint(path: str) -> str:
"""Map a cloud endpoint path to its current canonical form.
Handles known renames documented in the Comfy Cloud API:
/history -> /history_v2
/models/<f> -> /experiment/models/<f>
/models -> /experiment/models
"""
if path.startswith("/history") and not path.startswith("/history_v2"):
return "/history_v2" + path[len("/history"):]
if path.startswith("/models/"):
return "/experiment/models/" + path[len("/models/"):]
if path == "/models":
return "/experiment/models"
return path
def resolve_url(base: str, path: str, *, is_cloud: bool | None = None) -> str:
"""Top-level URL resolver. Applies cloud rename + /api prefix as needed."""
cloud = is_cloud_host(base) if is_cloud is None else is_cloud
if cloud:
path = cloud_endpoint(path)
return build_cloud_aware_url(base, path, force_cloud=cloud)
# =============================================================================
# API key resolution
# =============================================================================
def resolve_api_key(explicit: str | None) -> str | None:
"""Look up API key from CLI flag → env var. Strips whitespace and quotes."""
val = explicit if explicit else os.environ.get(ENV_API_KEY)
if val is None:
return None
val = val.strip().strip("'\"")
return val or None
# =============================================================================
# HTTP transport
# =============================================================================
@dataclass
class HTTPResponse:
status: int
headers: dict[str, str]
body: bytes
url: str # final URL after redirects
def text(self, encoding: str = "utf-8") -> str:
return self.body.decode(encoding, errors="replace")
def json(self) -> Any:
return json.loads(self.body.decode("utf-8", errors="replace"))
def _sleep_backoff(attempt: int, base: float = RETRY_BASE_DELAY, cap: float = RETRY_MAX_DELAY) -> None:
"""Sleep with full-jitter exponential backoff."""
delay = min(cap, base * (2 ** attempt))
delay = random.uniform(0, delay)
time.sleep(delay)
def http_request(
method: str,
url: str,
*,
headers: dict[str, str] | None = None,
json_body: Any = None,
data: bytes | None = None,
files: dict | None = None,
form: dict | None = None,
timeout: float = DEFAULT_HTTP_TIMEOUT,
follow_redirects: bool = True,
retries: int = DEFAULT_RETRIES,
stream: bool = False,
sink: Path | None = None,
) -> HTTPResponse:
"""Single entry point for all HTTP traffic.
Behavior:
- Retries on connection errors and on HTTP statuses in RETRY_STATUS_CODES,
with exponential backoff + jitter.
- For cross-host redirects, drops Authorization-style headers (so signed
URLs don't leak the API key to S3/CloudFront).
- When `stream=True` and `sink` is a Path, streams the response body to
disk in 64 KiB chunks instead of buffering.
Either `json_body`, `data`, or `files`+`form` may be supplied (mutually exclusive).
"""
if headers is None:
headers = {}
headers = dict(headers) # copy
headers.setdefault("User-Agent", "hermes-comfyui-skill/5.0")
if files or form is not None:
# Multipart upload — needs `requests`. The stdlib fallback lacks
# multipart encoding helpers; raise a clear error.
if not HAS_REQUESTS:
raise RuntimeError(
"Multipart upload requires the `requests` package. "
"Install with: pip install requests"
)
last_exc: Exception | None = None
for attempt in range(retries):
try:
resp = _http_once(
method=method, url=url, headers=headers,
json_body=json_body, data=data, files=files, form=form,
timeout=timeout, follow_redirects=follow_redirects,
stream=stream, sink=sink,
)
if resp.status in RETRY_STATUS_CODES and attempt + 1 < retries:
_sleep_backoff(attempt)
continue
return resp
except (TimeoutError, ConnectionError, OSError) as e:
last_exc = e
if attempt + 1 < retries:
_sleep_backoff(attempt)
continue
raise
# Should not reach here unless retries was 0
if last_exc:
raise last_exc
raise RuntimeError("http_request: retries exhausted with no response")
_SENSITIVE_HEADERS = ("x-api-key", "authorization", "cookie")
if HAS_REQUESTS:
class _StripSensitiveOnRedirectSession(requests.Session):
"""Session that drops sensitive headers on cross-host redirects.
`requests` already strips `Authorization` cross-host (rebuild_auth),
but it does NOT strip custom headers like `X-API-Key`. We override
`rebuild_auth` to additionally strip every header in
`_SENSITIVE_HEADERS` when the destination is a different host —
critical when ComfyUI Cloud's `/api/view` redirects to a signed S3 URL.
"""
def rebuild_auth(self, prepared_request, response): # type: ignore[override]
super().rebuild_auth(prepared_request, response)
try:
old_url = response.request.url
new_url = prepared_request.url
old_host = (urlparse(old_url).hostname or "").lower()
new_host = (urlparse(new_url).hostname or "").lower()
if old_host and new_host and old_host != new_host:
headers = prepared_request.headers
for key in list(headers.keys()):
if key.lower() in _SENSITIVE_HEADERS:
del headers[key]
except Exception:
# Defensive: never let header stripping break a redirect.
pass
def _http_once(
*, method: str, url: str, headers: dict[str, str],
json_body: Any, data: bytes | None, files: dict | None, form: dict | None,
timeout: float, follow_redirects: bool,
stream: bool, sink: Path | None,
) -> HTTPResponse:
"""One HTTP attempt. No retry."""
if HAS_REQUESTS:
kwargs: dict[str, Any] = {
"method": method, "url": url, "headers": headers,
"timeout": timeout, "allow_redirects": follow_redirects,
}
if json_body is not None:
kwargs["json"] = json_body
elif data is not None:
kwargs["data"] = data
elif files is not None or form is not None:
kwargs["files"] = files
kwargs["data"] = form
if stream:
kwargs["stream"] = True
# Use the subclass that strips sensitive headers cross-host
with _StripSensitiveOnRedirectSession() as s:
try:
r = s.request(**kwargs)
if stream and sink is not None:
sink.parent.mkdir(parents=True, exist_ok=True)
with sink.open("wb") as f:
for chunk in r.iter_content(DOWNLOAD_CHUNK_SIZE):
if chunk:
f.write(chunk)
body = b"" # already drained
else:
body = r.content
return HTTPResponse(
status=r.status_code,
headers={k: v for k, v in r.headers.items()},
body=body,
url=r.url,
)
except requests.exceptions.RequestException as e:
# Convert to TimeoutError / ConnectionError so the retry loop
# picks them up uniformly with the stdlib path.
if isinstance(e, requests.exceptions.Timeout):
raise TimeoutError(str(e)) from e
raise ConnectionError(str(e)) from e
# ---------- stdlib fallback ----------
if json_body is not None:
body_bytes = json.dumps(json_body).encode("utf-8")
headers.setdefault("Content-Type", "application/json")
else:
body_bytes = data
req = urllib.request.Request(url, data=body_bytes, headers=headers, method=method)
# urllib follows redirects by default. We need to:
# 1) intercept cross-host redirects and drop X-API-Key
# 2) optionally NOT follow redirects when follow_redirects=False
class _RedirectHandler(urllib.request.HTTPRedirectHandler):
def __init__(self, original_host: str, follow: bool):
self.original_host = original_host
self.follow = follow
def redirect_request(self, req2, fp, code, msg, hdrs, newurl):
if not self.follow:
return None
new_host = (urlparse(newurl).hostname or "").lower()
if new_host != self.original_host:
# Build a new request with cleaned headers
clean_headers = {
k: v for k, v in req2.header_items()
if k.lower() not in ("x-api-key", "authorization", "cookie")
}
new_req = urllib.request.Request(newurl, headers=clean_headers, method="GET")
return new_req
return super().redirect_request(req2, fp, code, msg, hdrs, newurl)
original_host = (urlparse(url).hostname or "").lower()
opener = urllib.request.build_opener(_RedirectHandler(original_host, follow_redirects))
try:
resp = opener.open(req, timeout=timeout)
except urllib.error.HTTPError as e:
return HTTPResponse(
status=e.code,
headers=dict(e.headers) if e.headers else {},
body=e.read() or b"",
url=getattr(e, "url", url),
)
final_url = resp.geturl()
final_status = resp.status
final_headers = dict(resp.headers)
if stream and sink is not None:
sink.parent.mkdir(parents=True, exist_ok=True)
with sink.open("wb") as f:
while True:
chunk = resp.read(DOWNLOAD_CHUNK_SIZE)
if not chunk:
break
f.write(chunk)
return HTTPResponse(status=final_status, headers=final_headers, body=b"", url=final_url)
return HTTPResponse(status=final_status, headers=final_headers, body=resp.read(), url=final_url)
def http_get(url: str, **kwargs: Any) -> HTTPResponse:
return http_request("GET", url, **kwargs)
def http_post(url: str, **kwargs: Any) -> HTTPResponse:
return http_request("POST", url, **kwargs)
# =============================================================================
# Workflow validation & helpers
# =============================================================================
def is_api_format(workflow: Any) -> bool:
"""API format = top-level dict where each value has `class_type`."""
if not isinstance(workflow, dict):
return False
if "nodes" in workflow and "links" in workflow:
return False
for v in workflow.values():
if isinstance(v, dict) and "class_type" in v:
return True
return False
def unwrap_workflow(payload: Any) -> dict:
"""Unwrap common wrapper variants. Returns API-format workflow or raises ValueError."""
if isinstance(payload, dict) and is_api_format(payload):
return payload
# Some files wrap workflow under "prompt" key (e.g. saved /prompt payloads)
if isinstance(payload, dict) and "prompt" in payload and is_api_format(payload["prompt"]):
return payload["prompt"]
# Editor format
if isinstance(payload, dict) and "nodes" in payload and "links" in payload:
raise ValueError(
"Workflow is in editor format (has top-level 'nodes' and 'links' arrays). "
"Re-export from ComfyUI using 'Workflow → Export (API)' (newer UI) "
"or 'Save (API Format)' (older UI)."
)
raise ValueError(
"Workflow is not in API format. Each top-level entry must have a 'class_type' field."
)
def is_link(value: Any) -> bool:
"""True if `value` is a [node_id, output_index] connection (length-2 list)."""
return (
isinstance(value, list)
and len(value) == 2
and isinstance(value[0], str)
and isinstance(value[1], int)
)
def iter_nodes(workflow: dict) -> Iterator[tuple[str, dict]]:
"""Yield (node_id, node) for each valid API-format node."""
for node_id, node in workflow.items():
if isinstance(node, dict) and "class_type" in node:
yield node_id, node
def iter_model_deps(workflow: dict) -> Iterator[dict]:
"""Yield {node_id, class_type, field, value, folder} for each model dependency."""
for node_id, node in iter_nodes(workflow):
cls = node["class_type"]
if cls not in MODEL_LOADERS:
continue
inputs = node.get("inputs", {}) or {}
for field_name, folder in MODEL_LOADERS[cls]:
val = inputs.get(field_name)
if val and isinstance(val, str) and not is_link(val):
yield {
"node_id": node_id,
"class_type": cls,
"field": field_name,
"value": val,
"folder": folder,
}
def iter_embedding_refs(workflow: dict) -> Iterator[tuple[str, str]]:
"""Yield (node_id, embedding_name) for every embedding mention in prompts."""
for node_id, node in iter_nodes(workflow):
inputs = node.get("inputs", {}) or {}
for field_name, val in inputs.items():
if field_name not in PROMPT_FIELDS:
continue
if not isinstance(val, str):
continue
for m in EMBEDDING_REGEX.finditer(val):
yield node_id, m.group(1)
# =============================================================================
# Path safety
# =============================================================================
def safe_path_join(base: Path, *parts: str) -> Path:
"""Join paths, raising if the result escapes `base`.
Server-supplied filenames may contain `../` etc. This guards against
path-traversal attacks when downloading outputs.
"""
base_resolved = base.resolve()
candidate = base.joinpath(*parts).resolve()
try:
candidate.relative_to(base_resolved)
except ValueError as e:
raise ValueError(
f"Refusing path traversal: {candidate} is outside {base_resolved}"
) from e
return candidate
def media_type_from_filename(filename: str) -> str:
ext = Path(filename).suffix.lower()
if ext in (".mp4", ".webm", ".avi", ".mov", ".mkv", ".gif", ".webp"):
return "video"
if ext in (".wav", ".mp3", ".flac", ".ogg", ".m4a"):
return "audio"
if ext in (".glb", ".obj", ".ply", ".gltf"):
return "3d"
if ext in (".json", ".txt", ".md"):
return "text"
return "image"
def looks_like_video_workflow(workflow: dict) -> bool:
"""Used to bump default timeout for video workflows."""
for _, node in iter_nodes(workflow):
if node["class_type"] in SLOW_OUTPUT_NODES:
return True
if node["class_type"].lower().startswith(("animatediff", "ade_", "wanvideo", "hunyuanvideo", "ltxvideo", "cogvideo")):
return True
return False
# =============================================================================
# Seed handling
# =============================================================================
# ComfyUI's max seed range. Many UIs treat `-1` as "randomize on submit".
SEED_MAX = 2**63 - 1
SEED_MIN = 0
def coerce_seed(value: Any) -> int:
"""Convert -1 or None to a fresh random seed; otherwise return int(value).
Accepts numeric -1 OR string "-1" (both treated as "randomize"). Other
parse failures raise TypeError/ValueError for the caller to surface.
"""
if value is None:
return random.randint(SEED_MIN, SEED_MAX)
# Stringly-typed -1 from CLI / JSON should also randomize
if isinstance(value, str) and value.strip() == "-1":
return random.randint(SEED_MIN, SEED_MAX)
if value == -1:
return random.randint(SEED_MIN, SEED_MAX)
return int(value)
# =============================================================================
# Cloud model-list normalization
# =============================================================================
def parse_model_list(payload: Any) -> set[str]:
"""Normalize model-list responses from local ComfyUI vs Comfy Cloud.
Local: `["a.safetensors", "b.safetensors"]`
Cloud: `[{"name": "a.safetensors", "pathIndex": 0}, ...]`
"""
if not isinstance(payload, list):
return set()
out: set[str] = set()
for item in payload:
if isinstance(item, str):
out.add(item)
elif isinstance(item, dict):
name = item.get("name") or item.get("filename") or item.get("path")
if isinstance(name, str):
out.add(name)
return out
# =============================================================================
# Misc utilities
# =============================================================================
def new_client_id() -> str:
return str(uuid.uuid4())
def fmt_kv(d: dict) -> str:
"""Pretty key=value for log lines."""
return " ".join(f"{k}={v!r}" for k, v in d.items())
def emit_json(obj: Any, *, indent: int = 2) -> None:
"""Print JSON to stdout. Centralised so behavior can be tweaked (e.g., --raw)."""
print(json.dumps(obj, indent=indent, default=str))
def log(msg: str) -> None:
"""stderr log with consistent prefix (so JSON stdout stays clean)."""
print(f"[comfyui-skill] {msg}", file=sys.stderr)
creative/comfyui/scripts/adapter_flux2.py
+94
View File
@@ -0,0 +1,94 @@
from fastapi import FastAPI
from pydantic import BaseModel
from typing import Optional
import httpx, json, base64, asyncio, time, random, os
app = FastAPI(title="ComfyUI OpenAI Adapter", version="0.1.0")
COMFYUI_URL = os.environ.get("COMFYUI_URL", "http://10.0.30.97:8188")
WORKFLOW_PREFIX = os.environ.get("WORKFLOW_PREFIX", "oai_adapter")
# FLUX.2 workflow using Qwen3VL, EmptyFlux2LatentImage, and TAEF2
# Adapt model names to your local ComfyUI installation.
FLUX2_WORKFLOW = {
"1": {"inputs": {"unet_name": "flux-2-klein-9b-Q4_K_S.gguf"}, "class_type": "UnetLoaderGGUF"},
"2": {"inputs": {"clip_name": "Qwen3VL-8B-Instruct-Q4_K_M.gguf", "type": "flux2"}, "class_type": "CLIPLoaderGGUF"},
"3": {"inputs": {"vae_name": "taef2"}, "class_type": "VAELoader"},
"4": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
"5": {"inputs": {"text": "prompt", "clip": ["2", 0]}, "class_type": "CLIPTextEncode"},
"6": {"inputs": {"text": "", "clip": ["2", 0]}, "class_type": "CLIPTextEncode"},
"7": {"inputs": {"seed": 42, "steps": 4, "cfg": 1.0, "sampler_name": "euler", "scheduler": "simple",
"denoise": 1.0, "model": ["1", 0], "positive": ["5", 0], "negative": ["6", 0],
"latent_image": ["4", 0]}, "class_type": "KSampler"},
"8": {"inputs": {"samples": ["7", 0], "vae": ["3", 0]}, "class_type": "VAEDecode"},
"9": {"inputs": {"filename_prefix": WORKFLOW_PREFIX, "images": ["8", 0]}, "class_type": "SaveImage"}
}
class ImageRequest(BaseModel):
prompt: str
n: Optional[int] = 1
size: Optional[str] = "1024x1024"
response_format: Optional[str] = "b64_json"
async def queue_workflow(wf: dict) -> str:
async with httpx.AsyncClient() as client:
r = await client.post(f"{COMFYUI_URL}/api/prompt", json={"prompt": wf})
r.raise_for_status()
return r.json()["prompt_id"]
async def wait_for_image(pid: str, timeout: int = 300) -> list:
async with httpx.AsyncClient() as client:
start = time.time()
while time.time() - start < timeout:
r = await client.get(f"{COMFYUI_URL}/history/{pid}")
if r.status_code == 200:
data = r.json()
if pid in data and "outputs" in data[pid]:
imgs = []
for node_out in data[pid]["outputs"].values():
if "images" in node_out:
imgs.extend(node_out["images"])
if imgs:
return imgs
await asyncio.sleep(1)
raise TimeoutError(f"Timeout waiting for {pid}")
async def fetch_image(filename: str, subfolder: str = "", folder_type: str = "output") -> bytes:
async with httpx.AsyncClient() as client:
params = {"filename": filename, "subfolder": subfolder, "type": folder_type}
r = await client.get(f"{COMFYUI_URL}/view", params=params)
r.raise_for_status()
return r.content
@app.post("/v1/images/generations")
async def generate(req: ImageRequest):
imgs = []
for i in range(req.n):
wf = json.loads(json.dumps(FLUX2_WORKFLOW))
wf["5"]["inputs"]["text"] = req.prompt
w, h = (1024, 1024)
if "x" in req.size:
try:
w, h = map(int, req.size.split("x"))
except ValueError:
pass
wf["4"]["inputs"]["width"] = w
wf["4"]["inputs"]["height"] = h
wf["7"]["inputs"]["seed"] = random.randint(1, 2**32)
pid = await queue_workflow(wf)
files = await wait_for_image(pid)
if not files:
raise RuntimeError("No images returned")
data = await fetch_image(files[0]["filename"], files[0].get("subfolder", ""), files[0].get("type", "output"))
imgs.append({"b64_json": base64.b64encode(data).decode()})
return {"created": int(time.time()), "data": imgs}
@app.get("/health")
async def health():
return {"status": "ok"}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=9000)
creative/comfyui/scripts/auto_fix_deps.py
Executable
+225
View File
@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""
auto_fix_deps.py — Run check_deps.py, then attempt to install whatever is missing.
For local servers:
- Missing custom nodes → `comfy node install <package>`
- Missing models → `comfy model download` (only if a URL is supplied via
--model-source-file or detected via well-known names)
For cloud: prints what would be needed but cannot install (cloud preinstalls
custom nodes and most models server-side; if something genuinely isn't there,
ask Comfy support).
This is conservative: it never installs without an explicit URL for models
(downloading the wrong model is hard to undo). Custom nodes from the registry
are auto-installed by name.
Usage:
python3 auto_fix_deps.py workflow_api.json
python3 auto_fix_deps.py workflow_api.json --models-from-file urls.json
python3 auto_fix_deps.py workflow_api.json --dry-run
"""
from __future__ import annotations
import argparse
import json
import shutil
import subprocess
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, log, resolve_api_key,
)
from check_deps import check_deps # noqa: E402
from _common import unwrap_workflow # noqa: E402
def comfy_cli_available() -> str | None:
"""Return command prefix for comfy-cli, or None."""
if shutil.which("comfy"):
return "comfy"
if shutil.which("uvx"):
return "uvx --from comfy-cli comfy"
return None
def run_cmd(cmd: list[str], *, dry_run: bool = False) -> tuple[int, str]:
if dry_run:
return 0, "[dry-run]"
log(f"$ {' '.join(cmd)}")
proc = subprocess.run(cmd, capture_output=True, text=True, check=False)
out = (proc.stdout or "") + (proc.stderr or "")
return proc.returncode, out
def install_node(package: str, *, dry_run: bool = False, comfy_cmd: str = "comfy") -> bool:
cmd = comfy_cmd.split() + ["--skip-prompt", "node", "install", package]
code, _ = run_cmd(cmd, dry_run=dry_run)
return code == 0
def install_model(url: str, folder: str, filename: str | None = None,
*, dry_run: bool = False, comfy_cmd: str = "comfy",
hf_token: str | None = None, civitai_token: str | None = None) -> bool:
cmd = comfy_cmd.split() + [
"--skip-prompt", "model", "download",
"--url", url,
"--relative-path", f"models/{folder}",
]
if filename:
cmd.extend(["--filename", filename])
if hf_token:
cmd.extend(["--set-hf-api-token", hf_token])
if civitai_token:
cmd.extend(["--set-civitai-api-token", civitai_token])
code, _ = run_cmd(cmd, dry_run=dry_run)
return code == 0
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(description="Run check_deps and install whatever is missing")
p.add_argument("workflow")
p.add_argument("--host", default=DEFAULT_LOCAL_HOST)
p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}")
p.add_argument("--models-from-file",
help="JSON file mapping {model_filename: download_url} for models that need install")
p.add_argument("--hf-token", help="HuggingFace token for downloads")
p.add_argument("--civitai-token", help="CivitAI token for downloads")
p.add_argument("--dry-run", action="store_true",
help="Show what would be installed without doing it")
p.add_argument("--no-restart", action="store_true",
help="Don't suggest restarting the server after node install")
args = p.parse_args(argv)
api_key = resolve_api_key(args.api_key)
wf_path = Path(args.workflow).expanduser()
if not wf_path.exists():
emit_json({"error": f"Workflow not found: {args.workflow}"})
return 1
try:
with wf_path.open() as f:
workflow = unwrap_workflow(json.load(f))
except (ValueError, json.JSONDecodeError) as e:
emit_json({"error": str(e)})
return 1
report = check_deps(workflow, host=args.host, api_key=api_key)
if report["is_ready"]:
emit_json({"status": "ready", "report": report})
return 0
if report["is_cloud"]:
emit_json({
"status": "cannot_fix_cloud",
"reason": "Comfy Cloud preinstalls nodes; if something is genuinely missing, contact support.",
"report": report,
})
return 1
comfy_cmd = comfy_cli_available()
if not comfy_cmd:
emit_json({
"status": "cannot_fix",
"reason": "comfy-cli not on PATH; install with `pip install comfy-cli` or `pipx install comfy-cli`",
"report": report,
})
return 1
actions: list[dict] = []
failures: list[dict] = []
# ---- Install missing custom nodes ----
seen_packages: set[str] = set()
for entry in report["missing_nodes"]:
cmd = entry.get("fix_command", "")
if cmd.startswith("comfy node install "):
package = cmd.split(" ")[-1]
if package in seen_packages:
continue
seen_packages.add(package)
ok = install_node(package, dry_run=args.dry_run, comfy_cmd=comfy_cmd)
(actions if ok else failures).append({
"kind": "node", "package": package, "node_class": entry["class_type"],
"ok": ok,
})
else:
failures.append({
"kind": "node", "node_class": entry["class_type"],
"ok": False, "reason": "No registry mapping known. " + entry.get("fix_hint", ""),
})
# ---- Install missing models (only when URL provided) ----
sources: dict[str, str] = {}
if args.models_from_file:
try:
sources = json.loads(Path(args.models_from_file).read_text())
except (OSError, json.JSONDecodeError) as e:
log(f"Could not read --models-from-file: {e}")
for entry in report["missing_models"]:
filename = entry["value"]
url = sources.get(filename)
if not url:
failures.append({
"kind": "model", "filename": filename, "folder": entry["folder"],
"ok": False, "reason": "No URL provided in --models-from-file. "
"Refusing to guess.",
})
continue
ok = install_model(
url, entry["folder"], filename,
dry_run=args.dry_run, comfy_cmd=comfy_cmd,
hf_token=args.hf_token, civitai_token=args.civitai_token,
)
(actions if ok else failures).append({
"kind": "model", "filename": filename, "folder": entry["folder"],
"url": url, "ok": ok,
})
# ---- Embeddings ----
for entry in report["missing_embeddings"]:
emb_name = entry["embedding_name"]
# Try common extensions in user-supplied source map
url = (sources.get(f"{emb_name}.pt")
or sources.get(f"{emb_name}.safetensors")
or sources.get(emb_name))
if not url:
failures.append({
"kind": "embedding", "name": emb_name,
"ok": False, "reason": "No URL provided in --models-from-file.",
})
continue
target_filename = (
f"{emb_name}.safetensors" if url.endswith(".safetensors")
else f"{emb_name}.pt"
)
ok = install_model(
url, "embeddings", target_filename,
dry_run=args.dry_run, comfy_cmd=comfy_cmd,
hf_token=args.hf_token, civitai_token=args.civitai_token,
)
(actions if ok else failures).append({
"kind": "embedding", "name": emb_name, "url": url, "ok": ok,
})
needs_restart = any(a["kind"] == "node" and a.get("ok") for a in actions)
emit_json({
"status": "fixed" if not failures else "partial",
"actions_taken": actions,
"failures": failures,
"needs_server_restart": needs_restart and not args.no_restart,
"restart_hint": "comfy stop && comfy launch --background",
"dry_run": args.dry_run,
})
return 0 if not failures else 1
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/check_deps.py
Executable
+437
View File
@@ -0,0 +1,437 @@
#!/usr/bin/env python3
"""
check_deps.py — Verify a ComfyUI workflow's dependencies (custom nodes, models,
embeddings) against a running server.
Improvements over v1:
- Cloud-aware endpoint mapping (handles `/api/experiment/models/{folder}` and
`/api/object_info` variants verified against live cloud API)
- Distinguishes 200-empty (genuinely no models in folder) vs 404
(folder doesn't exist) vs 403 (auth/tier issue) — no silent passes
- Outputs concrete remediation commands (e.g. `comfy node install <name>`)
when nodes are missing
- Detects embedding references inside prompt strings as model deps
- Skips check on cloud free tier `/api/object_info` (403) without false alarm
- Accepts API key from CLI flag OR $COMFY_CLOUD_API_KEY env var
Usage:
python3 check_deps.py workflow_api.json
python3 check_deps.py workflow_api.json --host 127.0.0.1 --port 8188
python3 check_deps.py workflow_api.json --host https://cloud.comfy.org
Stdlib-only. Python 3.10+.
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
DEFAULT_LOCAL_HOST, ENV_API_KEY,
emit_json, folder_aliases_for, http_get, is_cloud_host,
iter_embedding_refs, iter_model_deps, iter_nodes, parse_model_list,
resolve_api_key, resolve_url, unwrap_workflow,
)
# Known node → custom-node-package map. When a workflow needs a node we don't
# recognize, suggesting the right `comfy node install ...` makes the difference
# between a working agent and a stuck one.
NODE_TO_PACKAGE: dict[str, str] = {
# rgthree (Reroute is JS-only and doesn't appear in /object_info)
"Power Lora Loader (rgthree)": "rgthree-comfy",
"Image Comparer (rgthree)": "rgthree-comfy",
"Seed (rgthree)": "rgthree-comfy",
"Display Any (rgthree)": "rgthree-comfy",
"Display Int (rgthree)": "rgthree-comfy",
# Impact pack
"FaceDetailer": "comfyui-impact-pack",
"DetailerForEach": "comfyui-impact-pack",
"BboxDetectorSEGS": "comfyui-impact-pack",
"SAMLoader": "comfyui-impact-pack",
"ImpactWildcardProcessor": "comfyui-impact-pack",
# Impact subpack (separate package)
"UltralyticsDetectorProvider": "comfyui-impact-subpack",
# Was Node Suite
"Image Save": "was-node-suite-comfyui",
"Number Counter": "was-node-suite-comfyui",
"Text String": "was-node-suite-comfyui",
# easy-use
"easy fullLoader": "comfyui-easy-use",
"easy positive": "comfyui-easy-use",
"easy negative": "comfyui-easy-use",
"easy seed": "comfyui-easy-use",
"easy imageSave": "comfyui-easy-use",
# Video Helper Suite
"VHS_VideoCombine": "comfyui-videohelpersuite",
"VHS_LoadVideo": "comfyui-videohelpersuite",
"VHS_LoadAudio": "comfyui-videohelpersuite",
# AnimateDiff
"ADE_AnimateDiffLoaderWithContext": "comfyui-animatediff-evolved",
"ADE_AnimateDiffLoaderGen1": "comfyui-animatediff-evolved",
"ADE_LoadAnimateDiffModel": "comfyui-animatediff-evolved",
# ControlNet aux preprocessors (full class names)
"CannyEdgePreprocessor": "comfyui_controlnet_aux",
"DWPreprocessor": "comfyui_controlnet_aux",
"OpenposePreprocessor": "comfyui_controlnet_aux",
"DepthAnythingPreprocessor": "comfyui_controlnet_aux",
"Zoe_DepthAnythingPreprocessor": "comfyui_controlnet_aux",
"AnimalPosePreprocessor": "comfyui_controlnet_aux",
# IPAdapter Plus
"IPAdapterAdvanced": "comfyui_ipadapter_plus",
"IPAdapterUnifiedLoader": "comfyui_ipadapter_plus",
"IPAdapterModelLoader": "comfyui_ipadapter_plus",
"IPAdapterInsightFaceLoader": "comfyui_ipadapter_plus",
# InstantID
"InstantIDModelLoader": "comfyui_instantid",
"ApplyInstantID": "comfyui_instantid",
# Comfy essentials (note: registry slug uses underscore, not hyphen)
"GetImageSize+": "comfyui_essentials",
"ImageBatchMultiple+": "comfyui_essentials",
# pysssss
"ShowText|pysssss": "comfyui-custom-scripts",
"PreviewImage|pysssss": "comfyui-custom-scripts",
# SUPIR
"SUPIR_Upscale": "comfyui-supir",
"SUPIR_first_stage": "comfyui-supir",
# GGUF (case-sensitive registry slug)
"UNETLoaderGGUF": "ComfyUI-GGUF",
"DualCLIPLoaderGGUF": "ComfyUI-GGUF",
# Florence2
"Florence2Run": "comfyui-florence2",
# WAS
"Image Filter Adjustments": "was-node-suite-comfyui",
# Photomaker (case-sensitive)
"PhotoMakerLoader": "ComfyUI-PhotoMaker-Plus",
# Wan video (case-sensitive)
"WanVideoSampler": "ComfyUI-WanVideoWrapper",
"WanVideoModelLoader": "ComfyUI-WanVideoWrapper",
}
# Nodes whose package isn't on the comfy registry — need git-URL install via
# ComfyUI-Manager. We surface a helpful hint instead of an unrunnable command.
NODE_TO_GIT_URL: dict[str, str] = {
"HunyuanVideoSampler": "https://github.com/kijai/ComfyUI-HunyuanVideoWrapper",
"HunyuanVideoModelLoader": "https://github.com/kijai/ComfyUI-HunyuanVideoWrapper",
}
def fetch_object_info(url: str, headers: dict) -> tuple[set[str] | None, dict | None]:
"""Returns (installed_node_set, error_info). Error info is a dict if we
couldn't query (e.g. cloud free tier), else None.
"""
r = http_get(url, headers=headers, retries=2, timeout=30)
if r.status == 200:
try:
data = r.json()
if isinstance(data, dict):
return set(data.keys()), None
except Exception:
pass
return None, {"http_status": 200, "reason": "non-dict response"}
if r.status == 403:
try:
body = r.json()
except Exception:
body = {"raw": r.text()[:200]}
return None, {"http_status": 403, "reason": "forbidden", "body": body}
if r.status == 404:
return None, {"http_status": 404, "reason": "endpoint not found"}
return None, {"http_status": r.status, "reason": "unexpected", "body": r.text()[:200]}
def _fetch_one_folder(
base: str, folder: str, headers: dict, *, is_cloud: bool,
) -> tuple[set[str] | None, dict | None]:
"""Single-folder fetch, no aliasing. Returns (installed_set, error_info)."""
url = resolve_url(base, f"/models/{folder}", is_cloud=is_cloud)
r = http_get(url, headers=headers, retries=2, timeout=30)
if r.status == 200:
try:
return parse_model_list(r.json()), None
except Exception:
return set(), {"http_status": 200, "reason": "non-list response"}
if r.status == 404:
body_text = r.text()
try:
body = r.json()
except Exception:
body = {"raw": body_text[:200]}
code = body.get("code") if isinstance(body, dict) else None
if code == "folder_not_found":
# Folder is genuinely empty/missing on server — not the same as
# "endpoint missing". Return empty set with informational error.
return set(), {"http_status": 404, "reason": "folder_empty_or_unknown", "body": body}
return None, {"http_status": 404, "reason": "endpoint not found", "body": body}
if r.status == 403:
try:
body = r.json()
except Exception:
body = {}
return None, {"http_status": 403, "reason": "forbidden", "body": body}
return None, {"http_status": r.status, "reason": "unexpected"}
def fetch_models_for_folder(
base: str, folder: str, headers: dict, *, is_cloud: bool,
) -> tuple[set[str] | None, dict | None]:
"""Fetch installed models for a folder, trying aliases.
Folder renames over time (e.g. unet → diffusion_models, clip → text_encoders)
mean a workflow asking for a model in `unet` may need to look in
`diffusion_models`. We union models from every reachable alias.
Returns (combined_set | None, last_error | None).
"""
aliases = folder_aliases_for(folder)
combined: set[str] = set()
any_success = False
last_err: dict | None = None
for alias in aliases:
models, err = _fetch_one_folder(base, alias, headers, is_cloud=is_cloud)
if models is not None:
combined.update(models)
any_success = True
last_err = None
else:
last_err = err
if not any_success:
return None, last_err
return combined, None
def fetch_embeddings(base: str, headers: dict, *, is_cloud: bool) -> tuple[set[str] | None, dict | None]:
"""Local ComfyUI exposes /embeddings; cloud uses /experiment/models/embeddings."""
if is_cloud:
return fetch_models_for_folder(base, "embeddings", headers, is_cloud=True)
# Local: dedicated /embeddings returns a flat list of names
r = http_get(resolve_url(base, "/embeddings", is_cloud=False), headers=headers, retries=2)
if r.status == 200:
try:
data = r.json()
if isinstance(data, list):
# Strip extensions from the registered names since prompt syntax
# usually omits them ("embedding:goodvibes" vs "goodvibes.pt")
names = set()
for n in data:
if isinstance(n, str):
names.add(n)
# Also store stem for fuzzy matching
names.add(Path(n).stem)
return names, None
except Exception:
pass
return None, {"http_status": r.status, "reason": "unexpected"}
def normalize_for_match(name: str) -> set[str]:
"""Generate matching variants of a model name (with/without extension, slashes, etc.)"""
s = {name}
s.add(Path(name).stem)
s.add(Path(name).name)
# ComfyUI sometimes strips/keeps the leading folder
if "/" in name or "\\" in name:
flat = name.replace("\\", "/").split("/")[-1]
s.add(flat)
s.add(Path(flat).stem)
return {x for x in s if x}
def model_present(needed: str, installed: set[str]) -> bool:
if not installed:
return False
needed_variants = normalize_for_match(needed)
installed_norm: set[str] = set()
for inst in installed:
installed_norm.update(normalize_for_match(inst))
return bool(needed_variants & installed_norm)
def suggest_install_command(node_class: str) -> str | None:
pkg = NODE_TO_PACKAGE.get(node_class)
if pkg:
return f"comfy node install {pkg}"
return None
def suggest_git_url(node_class: str) -> str | None:
"""For nodes not on the registry, return a git URL the user can hand to
ComfyUI-Manager's `/manager/queue/install` endpoint."""
return NODE_TO_GIT_URL.get(node_class)
def check_deps(
workflow: dict, host: str, *, api_key: str | None = None,
) -> dict:
headers: dict[str, str] = {}
if api_key:
headers["X-API-Key"] = api_key
is_cloud = is_cloud_host(host)
base = host.rstrip("/")
# ---- 1. Required nodes ----
required_nodes: set[str] = set()
for _, node in iter_nodes(workflow):
required_nodes.add(node["class_type"])
object_info_url = resolve_url(base, "/object_info", is_cloud=is_cloud)
installed_nodes, obj_err = fetch_object_info(object_info_url, headers)
missing_nodes: list[dict] = []
node_check_skipped = False
if installed_nodes is None:
# Couldn't query (e.g. cloud free tier). Don't false-alarm; mark skipped.
node_check_skipped = True
else:
for cls in sorted(required_nodes):
if cls not in installed_nodes:
entry = {"class_type": cls}
cmd = suggest_install_command(cls)
git_url = suggest_git_url(cls)
if cmd:
entry["fix_command"] = cmd
elif git_url:
entry["fix_git_url"] = git_url
entry["fix_hint"] = (
f"Not on registry. Install via Manager with this git URL: {git_url}"
)
else:
entry["fix_hint"] = (
"Search https://registry.comfy.org or "
"use ComfyUI-Manager UI to find the package providing this node."
)
missing_nodes.append(entry)
# ---- 2. Required models ----
model_cache: dict[str, tuple[set[str] | None, dict | None]] = {}
missing_models: list[dict] = []
folder_errors: dict[str, dict] = {}
for dep in iter_model_deps(workflow):
folder = dep["folder"]
if folder not in model_cache:
model_cache[folder] = fetch_models_for_folder(
base, folder, headers, is_cloud=is_cloud,
)
installed, err = model_cache[folder]
if installed is None:
# Couldn't enumerate this folder — record once
folder_errors.setdefault(folder, err or {})
# Don't flag as missing (we don't know); the folder_errors block surfaces this
continue
if not model_present(dep["value"], installed):
entry = dict(dep)
entry["fix_hint"] = (
f"comfy model download --url <URL> --relative-path models/{folder} "
f"--filename {dep['value']!r}"
)
missing_models.append(entry)
# ---- 3. Embedding refs in prompts ----
emb_installed, emb_err = fetch_embeddings(base, headers, is_cloud=is_cloud)
missing_embeddings: list[dict] = []
seen_emb: set[tuple[str, str]] = set()
for nid, emb_name in iter_embedding_refs(workflow):
if (nid, emb_name) in seen_emb:
continue
seen_emb.add((nid, emb_name))
if emb_installed is None:
# Couldn't enumerate — skip silently here, surface the error in the
# folder_errors block
continue
if not model_present(emb_name, emb_installed):
missing_embeddings.append({
"node_id": nid,
"embedding_name": emb_name,
"folder": "embeddings",
"fix_hint": (
f"Download {emb_name}.pt or .safetensors and place in "
f"models/embeddings/, or `comfy model download --url <URL> "
f"--relative-path models/embeddings`"
),
})
if emb_err and emb_installed is None:
folder_errors.setdefault("embeddings", emb_err)
is_ready = (
not node_check_skipped
and not missing_nodes
and not missing_models
and not missing_embeddings
)
return {
"is_ready": is_ready,
"node_check_skipped": node_check_skipped,
"node_check_skip_reason": obj_err if node_check_skipped else None,
"missing_nodes": missing_nodes,
"missing_models": missing_models,
"missing_embeddings": missing_embeddings,
"folder_errors": folder_errors,
# 0 is a legitimate count (e.g. empty server). Use None only when not queried.
"installed_node_count": len(installed_nodes) if installed_nodes is not None else None,
"required_node_count": len(required_nodes),
"required_nodes": sorted(required_nodes),
"host": base,
"is_cloud": is_cloud,
}
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(description="Check ComfyUI workflow dependencies against a running server")
p.add_argument("workflow", help="Path to workflow API JSON file")
p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL")
p.add_argument("--port", type=int, help="Server port (overrides --host port)")
p.add_argument("--api-key", help=f"API key for cloud (or set ${ENV_API_KEY} env var)")
p.add_argument("--strict", action="store_true",
help="Exit non-zero if node check is skipped (e.g. on cloud free tier)")
args = p.parse_args(argv)
host = args.host
if args.port is not None:
# Strip any port from host and append --port
from urllib.parse import urlparse, urlunparse
parsed = urlparse(host if "://" in host else f"http://{host}")
new_netloc = f"{parsed.hostname}:{args.port}"
host = urlunparse(parsed._replace(netloc=new_netloc))
api_key = resolve_api_key(args.api_key)
wf_path = Path(args.workflow).expanduser()
if not wf_path.exists():
emit_json({"error": f"Workflow file not found: {args.workflow}"})
return 1
try:
with wf_path.open() as f:
payload = json.load(f)
workflow = unwrap_workflow(payload)
except ValueError as e:
emit_json({"error": str(e)})
return 1
except json.JSONDecodeError as e:
emit_json({"error": f"Invalid JSON: {e}"})
return 1
try:
result = check_deps(workflow, host=host, api_key=api_key)
except Exception as e:
emit_json({"error": f"Dep check failed: {e}", "host": host})
return 1
emit_json(result)
if not result["is_ready"]:
return 1
if args.strict and result["node_check_skipped"]:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/check_safetensors.py
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""
Verify safetensors file integrity: header matches actual data offsets.
Used for diagnosing 'Error while deserializing header: incomplete metadata'
even when file size looks correct.
Usage from Proxmox host (remote ComfyUI in LXC):
ssh root@10.0.20.91 'bash -s' < scripts/check_safetensors.py
"""
import struct, json, os, sys
def check(path):
if not os.path.exists(path):
print(f"MISSING: {path}")
return False
try:
with open(path, "rb") as f:
hlen = struct.unpack("<Q", f.read(8))[0]
header = json.loads(f.read(hlen))
total = 8 + hlen
for k, v in header.items():
if isinstance(v, dict) and "data_offsets" in v:
total += v["data_offsets"][1] - v["data_offsets"][0]
actual = os.path.getsize(path)
ok = total == actual
print(f"{os.path.basename(path)}: exp={total:,} act={actual:,} ok={ok}")
return ok
except Exception as e:
print(f"{os.path.basename(path)}: ERROR: {type(e).__name__}: {e}")
return False
if __name__ == "__main__":
if len(sys.argv) < 2:
# Default: check common ComfyUI model locations
paths = [
"/opt/ComfyUI/models/diffusion_models/ideogram4_fp8_scaled.safetensors",
"/opt/ComfyUI/models/diffusion_models/ideogram4_unconditional_fp8_scaled.safetensors",
"/opt/ComfyUI/models/text_encoders/qwen3vl_8b_fp8_scaled.safetensors",
"/opt/ComfyUI/models/vae/flux2-vae.safetensors",
]
else:
paths = sys.argv[1:]
results = [check(p) for p in paths]
if not all(results):
sys.exit(1)
print("All OK")
creative/comfyui/scripts/comfy_e2e_test.py
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""
End-to-end Test für ComfyUI Image Gen API.
Iteriert über model × quality × style, generiert Bilder, schreibt Report.
Usage: python3 scripts/comfy_e2e_test.py [--api-url URL] [--output-dir DIR]
"""
import os, time, json, base64, pathlib, argparse
def parse_args():
p = argparse.ArgumentParser(description="ComfyUI E2E image generation test")
p.add_argument("--api-url", default="http://localhost:8000/v1/images/generations")
p.add_argument("--output-dir", default=str(pathlib.Path.home() / ".hermes/cron/output/comfy_e2e"))
p.add_argument("--models", default="ideogram4,flux2", help="Comma-separated models to test")
p.add_argument("--qualities", default="low,medium,high", help="Comma-separated qualities")
p.add_argument("--styles", default=",photorealistic", help="Comma-separated styles (empty for none)")
p.add_argument("--size", default="1024x1024")
p.add_argument("--n", type=int, default=1)
p.add_argument("--timeout", type=int, default=1800, help="HTTP timeout per request in seconds")
return p.parse_args()
def generate_image(api_url: str, payload: dict, timeout: int):
import urllib.request
req = urllib.request.Request(
api_url,
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json"},
method="POST"
)
start = time.time()
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read().decode())
elapsed = time.time() - start
b64 = data["data"][0]["b64_json"]
return base64.b64decode(b64), elapsed
except Exception as e:
return None, str(e)
def main():
args = parse_args()
out_dir = pathlib.Path(args.output_dir)
out_dir.mkdir(parents=True, exist_ok=True)
models = [m.strip() for m in args.models.split(",")]
qualities = [q.strip() for q in args.qualities.split(",")]
styles = [s.strip() for s in args.styles.split(",")]
PROMPTS = {
"ideogram4": "A futuristic city skyline at sunset, neon lights, detailed architecture",
"flux2": "A serene mountain lake reflection, golden hour lighting, highly detailed",
"flux1-dev": "A cyberpunk street market at night, volumetric fog, neon signs",
"flux1-schnell": "A minimalist abstract painting, bold colors, geometric shapes",
}
RESULTS = []
total = len(models) * len(qualities) * len(styles)
print(f"Output: {out_dir}")
print(f"Total: {total} combinations")
print("=" * 60)
for model in models:
for quality in qualities:
for style in styles:
tag = f"{model}_{quality}_{style or 'default'}"
prompt = PROMPTS.get(model, "A beautiful landscape")
full_prompt = f"{style}, {prompt}".strip(", ") if style else prompt
payload = {
"model": model,
"prompt": full_prompt,
"n": args.n,
"size": args.size,
"quality": quality,
"response_format": "b64_json"
}
print(f"\n[{tag}] {full_prompt[:60]}...")
img_bytes, info = generate_image(args.api_url, payload, args.timeout)
if img_bytes:
fname = f"{tag}_{int(time.time())}.png"
fpath = out_dir / fname
fpath.write_bytes(img_bytes)
print(f" ✅ {info:.1f}s, {len(img_bytes)/1024:.0f} KB → {fname}")
RESULTS.append({
"model": model, "quality": quality, "style": style,
"status": "OK", "time_s": round(info, 1),
"size_kb": round(len(img_bytes)/1024, 1), "file": str(fpath)
})
else:
print(f" ❌ {info}")
RESULTS.append({
"model": model, "quality": quality, "style": style,
"status": "FAILED", "error": str(info)
})
# JSON report
report = out_dir / "report.json"
report.write_text(json.dumps(RESULTS, indent=2))
# Markdown report
md = out_dir / "report.md"
lines = [
"# ComfyUI E2E Test Report",
f"\n**Zeit:** {time.strftime('%Y-%m-%d %H:%M:%S')}",
]
ok = sum(1 for r in RESULTS if r["status"] == "OK")
fail = sum(1 for r in RESULTS if r["status"] == "FAILED")
lines.append(f"**Ergebnis:** {ok} OK / {fail} FAILED\n")
lines.append("| Modell | Quality | Style | Status | Zeit | Datei |")
lines.append("|---|---|---|---|---|---|")
for r in RESULTS:
style = r.get("style", "") or "—"
status = "✅" if r["status"] == "OK" else "❌"
t = f"{r.get('time_s', '—')}s" if "time_s" in r else "—"
f = r.get("file", "").split("/")[-1] if "file" in r else "—"
lines.append(f"| {r['model']} | {r['quality']} | {style} | {status} {r['status']} | {t} | {f} |")
md.write_text("\n".join(lines))
print(f"\n{'=' * 60}")
print(f"Reports: {report} , {md}")
print(f"Bilder: {out_dir}")
if __name__ == "__main__":
main()
creative/comfyui/scripts/comfyui_setup.sh
Executable
+286
View File
@@ -0,0 +1,286 @@
#!/usr/bin/env bash
# ComfyUI Setup — Install, launch, and verify using the official comfy-cli.
#
# Improvements over v1:
# - Prefers `pipx` / `uvx` over global `pip install` (avoids polluting system Python)
# - Idempotent: detects already-running server and skips re-launch
# - Configurable port via --port=N (default 8188)
# - Configurable workspace via --workspace=PATH
# - Persistent log file in /tmp/comfyui_setup.<pid>.log for debugging
# - SIGINT trap cleans up partial state
# - Refuses local install when hardware_check.py verdict is "cloud"
# - Forwards extra flags to comfy-cli (e.g. --cuda-version=12.4)
#
# Usage:
# bash scripts/comfyui_setup.sh
# (auto-detects GPU; uses recommendation from hardware_check.py)
# bash scripts/comfyui_setup.sh --nvidia
# bash scripts/comfyui_setup.sh --m-series --port=8190
# bash scripts/comfyui_setup.sh --amd --workspace=/data/comfy
#
# Flags:
# --nvidia | --amd | --m-series | --cpu GPU selection (skips hw check)
# --port=N HTTP port (default 8188)
# --workspace=PATH ComfyUI install location
# --skip-launch Install only, don't start server
# --force-cloud-override Install locally even if hw says cloud
# -- Pass remaining args to `comfy install`
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
HARDWARE_CHECK="$SCRIPT_DIR/hardware_check.py"
LOG_FILE="/tmp/comfyui_setup.$$.log"
PORT=8188
WORKSPACE=""
GPU_FLAG=""
SKIP_LAUNCH=0
FORCE_CLOUD_OVERRIDE=0
EXTRA_INSTALL_ARGS=()
cleanup() {
local exit_code=$?
if [ $exit_code -ne 0 ]; then
echo "==> Setup exited with status $exit_code. Log: $LOG_FILE" >&2
fi
exit $exit_code
}
trap cleanup EXIT INT TERM
log() { echo "==> $*" | tee -a "$LOG_FILE" >&2; }
err() { echo "ERROR: $*" | tee -a "$LOG_FILE" >&2; }
# --- Argument parsing ---
PASSTHROUGH=0
for arg in "$@"; do
if [ "$PASSTHROUGH" -eq 1 ]; then
EXTRA_INSTALL_ARGS+=("$arg")
continue
fi
case "$arg" in
--nvidia|--amd|--m-series|--cpu)
GPU_FLAG="$arg"
;;
--port=*)
PORT="${arg#*=}"
;;
--workspace=*)
WORKSPACE="${arg#*=}"
;;
--skip-launch)
SKIP_LAUNCH=1
;;
--force-cloud-override)
FORCE_CLOUD_OVERRIDE=1
;;
--)
PASSTHROUGH=1
;;
--help|-h)
# Print the leading comment block, stripping the `# ` prefix.
# Stops at the first blank line which separates docs from code.
awk '
NR == 1 { next } # skip shebang
/^[^#]/ { exit } # stop at first non-comment line
/^$/ { exit } # ...or first blank line
{ sub(/^# ?/, ""); print }
' "$0"
exit 0
;;
*)
err "Unknown argument: $arg"
exit 64
;;
esac
done
log "Logging to $LOG_FILE"
# --- Step 0: Hardware check (skipped if user gave an explicit GPU flag) ---
if [ -z "$GPU_FLAG" ]; then
if [ ! -f "$HARDWARE_CHECK" ]; then
log "hardware_check.py not found — defaulting to --nvidia"
GPU_FLAG="--nvidia"
else
log "Running hardware check…"
set +e
HW_JSON="$(python3 "$HARDWARE_CHECK" --json 2>>"$LOG_FILE")"
HW_EXIT=$?
set -e
if [ -z "$HW_JSON" ]; then
err "hardware_check.py produced no output (exit $HW_EXIT). Pass an explicit flag."
exit 1
fi
echo "$HW_JSON" | tee -a "$LOG_FILE" >&2
VERDICT="$(echo "$HW_JSON" | python3 -c 'import sys,json; print(json.load(sys.stdin).get("verdict",""))')"
FLAG="$(echo "$HW_JSON" | python3 -c 'import sys,json; print(json.load(sys.stdin).get("comfy_cli_flag") or "")')"
if [ "$VERDICT" = "cloud" ] && [ "$FORCE_CLOUD_OVERRIDE" -ne 1 ]; then
log ""
log "Hardware check: this machine is not suitable for local ComfyUI."
log "Recommended: Comfy Cloud — https://platform.comfy.org"
log ""
log "To override and force a local install, re-run with --force-cloud-override"
log "or pass an explicit GPU flag (--nvidia|--amd|--m-series|--cpu)."
exit 2
fi
if [ "$VERDICT" = "marginal" ]; then
log "Hardware check: verdict is MARGINAL."
log " SD1.5 should work; SDXL/Flux may be slow or OOM."
log " Consider Comfy Cloud for heavier workflows: https://platform.comfy.org"
fi
if [ -z "$FLAG" ]; then
log "hardware_check could not pick a comfy-cli flag. Defaulting to --nvidia."
log "(For Intel Arc or unsupported hardware, use the manual install path.)"
GPU_FLAG="--nvidia"
else
GPU_FLAG="$FLAG"
fi
fi
fi
log "GPU flag: $GPU_FLAG"
log "Port: $PORT"
[ -n "$WORKSPACE" ] && log "Workspace: $WORKSPACE"
[ "${#EXTRA_INSTALL_ARGS[@]}" -gt 0 ] && log "Extra install args: ${EXTRA_INSTALL_ARGS[*]}"
# --- Step 1: Install comfy-cli (prefer pipx / uvx over global pip) ---
COMFY_BIN=""
if command -v comfy >/dev/null 2>&1; then
COMFY_BIN="comfy"
log "comfy-cli already on PATH: $(comfy -v 2>/dev/null || echo 'unknown version')"
elif command -v uvx >/dev/null 2>&1; then
log "Using uvx (no install needed)"
COMFY_BIN="uvx --from comfy-cli comfy"
elif command -v pipx >/dev/null 2>&1; then
log "Installing comfy-cli via pipx…"
pipx install comfy-cli >>"$LOG_FILE" 2>&1
COMFY_BIN="comfy"
# pipx adds shims to ~/.local/bin which may need to be on PATH
if ! command -v comfy >/dev/null 2>&1; then
if [ -x "$HOME/.local/bin/comfy" ]; then
export PATH="$HOME/.local/bin:$PATH"
COMFY_BIN="$HOME/.local/bin/comfy"
fi
fi
else
log "Neither pipx nor uvx found. Falling back to pip install --user…"
log " (Recommend installing pipx: https://pipx.pypa.io)"
if ! pip install --user comfy-cli >>"$LOG_FILE" 2>&1; then
# macOS: PEP 668 externally-managed-environment may block --user
log "pip install --user failed. Retrying with --break-system-packages…"
pip install --user --break-system-packages comfy-cli >>"$LOG_FILE" 2>&1 || {
err "Could not install comfy-cli. Install pipx or uv first."
exit 1
}
fi
# Resolve the actual `comfy` script — pip --user puts it in:
# Linux: ~/.local/bin/comfy
# macOS: ~/Library/Python/<ver>/bin/comfy OR ~/.local/bin/comfy
COMFY_BIN=""
for candidate in "$HOME/.local/bin/comfy" \
"$HOME/Library/Python/3.13/bin/comfy" \
"$HOME/Library/Python/3.12/bin/comfy" \
"$HOME/Library/Python/3.11/bin/comfy" \
"$HOME/Library/Python/3.10/bin/comfy"; do
if [ -x "$candidate" ]; then
COMFY_BIN="$candidate"
export PATH="$(dirname "$candidate"):$PATH"
break
fi
done
if [ -z "$COMFY_BIN" ]; then
if command -v comfy >/dev/null 2>&1; then
COMFY_BIN="comfy"
else
err "Installed comfy-cli but couldn't find the 'comfy' script."
err "Add the right Python user-bin directory to PATH and retry."
exit 1
fi
fi
fi
# --- Step 2: Disable analytics tracking (avoid interactive prompt) ---
log "Disabling analytics tracking…"
$COMFY_BIN --skip-prompt tracking disable >>"$LOG_FILE" 2>&1 || true
# --- Step 3: Install ComfyUI ---
WORKSPACE_ARG=()
if [ -n "$WORKSPACE" ]; then
WORKSPACE_ARG=(--workspace "$WORKSPACE")
fi
if $COMFY_BIN "${WORKSPACE_ARG[@]}" which 2>/dev/null | grep -q "ComfyUI"; then
EXISTING_WS="$($COMFY_BIN "${WORKSPACE_ARG[@]}" which 2>/dev/null || true)"
log "ComfyUI already installed at: $EXISTING_WS"
else
log "Installing ComfyUI ($GPU_FLAG)…"
if ! $COMFY_BIN "${WORKSPACE_ARG[@]}" --skip-prompt install "$GPU_FLAG" "${EXTRA_INSTALL_ARGS[@]}" >>"$LOG_FILE" 2>&1; then
err "Install failed. Tail of log:"
tail -20 "$LOG_FILE" >&2
exit 1
fi
fi
if [ "$SKIP_LAUNCH" -eq 1 ]; then
log "Setup complete (--skip-launch). Run \`$COMFY_BIN launch --background -- --port $PORT\` when ready."
exit 0
fi
# --- Step 4: Detect already-running server ---
if curl -fsS "http://127.0.0.1:$PORT/system_stats" >/dev/null 2>&1; then
log "Server already running on port $PORT — skipping launch."
log "Stop with \`$COMFY_BIN stop\` if you want a fresh start."
curl -fsS "http://127.0.0.1:$PORT/system_stats" | python3 -m json.tool 2>/dev/null || true
log "Done."
exit 0
fi
# --- Step 5: Launch ---
log "Launching ComfyUI in background on port $PORT…"
LAUNCH_EXTRAS=("--" "--port" "$PORT")
if ! $COMFY_BIN "${WORKSPACE_ARG[@]}" launch --background "${LAUNCH_EXTRAS[@]}" >>"$LOG_FILE" 2>&1; then
err "Background launch failed. Tail of log:"
tail -20 "$LOG_FILE" >&2
err "Try foreground launch to see real-time errors: $COMFY_BIN launch -- --port $PORT"
exit 1
fi
# --- Step 6: Wait for server ---
log "Waiting for server…"
MAX_WAIT=60
ELAPSED=0
while [ $ELAPSED -lt $MAX_WAIT ]; do
if curl -fsS "http://127.0.0.1:$PORT/system_stats" >/dev/null 2>&1; then
log "Server is running!"
curl -fsS "http://127.0.0.1:$PORT/system_stats" | python3 -m json.tool 2>/dev/null || true
break
fi
sleep 2
ELAPSED=$((ELAPSED + 2))
done
if [ $ELAPSED -ge $MAX_WAIT ]; then
err "Server did not start within ${MAX_WAIT}s."
err "Inspect log: $LOG_FILE"
err "Or run foreground: $COMFY_BIN launch -- --port $PORT"
exit 1
fi
log ""
log "Setup complete!"
log " Server: http://127.0.0.1:$PORT"
log " Web UI: http://127.0.0.1:$PORT (open in browser)"
log " Stop: $COMFY_BIN stop"
log " Log: $LOG_FILE (kept until shell closes)"
log ""
log "Next steps:"
log " - Download a model: $COMFY_BIN model download --url <URL> --relative-path models/checkpoints"
log " - Run a workflow: python3 $SCRIPT_DIR/run_workflow.py --workflow <file.json> --args '{...}'"
# Disable trap on success path
trap - EXIT
creative/comfyui/scripts/extract_schema.py
Executable
+315
View File
@@ -0,0 +1,315 @@
#!/usr/bin/env python3
"""
extract_schema.py — Analyze a ComfyUI API-format workflow and extract
controllable parameters.
Improvements over v1:
- Catalogs live in `_common.py`, shared with `check_deps.py`
- Coverage expanded for Flux / SD3 / Wan / Hunyuan / LTX / IPAdapter / rgthree
- Symmetric duplicate-name resolution: ALL duplicates get a node-id suffix
(instead of "first wins, second renamed"), so callers see consistent names
- Negative prompt detected by tracing `KSampler.negative` connections back to
the source CLIPTextEncode (more reliable than meta-title heuristic)
- Embedding references in prompt text are extracted as model dependencies
- Detects Primitive nodes that drive other nodes' inputs (and surfaces them
as the user-facing parameter)
- Reroutes are followed when tracing connections
Usage:
python3 extract_schema.py workflow_api.json
python3 extract_schema.py workflow_api.json --output schema.json
Stdlib-only. Python 3.10+.
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
from typing import Any
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
OUTPUT_NODES, PARAM_PATTERNS, PROMPT_FIELDS,
is_link, iter_embedding_refs, iter_model_deps, iter_nodes, unwrap_workflow,
)
# Sampler nodes whose `positive` / `negative` connections we trace
SAMPLER_NODE_FAMILY = {
"KSampler", "KSamplerAdvanced",
"SamplerCustom", "SamplerCustomAdvanced",
"BasicGuider", "CFGGuider", "DualCFGGuider",
}
def infer_type(value: Any) -> str:
if isinstance(value, bool):
return "bool"
if isinstance(value, int):
return "int"
if isinstance(value, float):
return "float"
if isinstance(value, str):
return "string"
if isinstance(value, list):
return "link"
if isinstance(value, dict):
return "object"
return "unknown"
def trace_to_node(workflow: dict, link: list, *, max_hops: int = 8) -> str | None:
"""Follow a [node_id, slot] link, hopping through Reroute / Primitive nodes
if needed, to find the *upstream* node id that holds the actual value/input.
Bounded by both `max_hops` AND a visited-set to prevent infinite loops on
pathological graphs.
"""
if not is_link(link):
return None
nid: str | None = link[0]
visited: set[str] = set()
for _ in range(max_hops):
if nid is None or nid in visited:
return nid
visited.add(nid)
node = workflow.get(nid)
if not isinstance(node, dict):
return None
cls = node.get("class_type", "")
# Reroute / Primitive / passthrough wrappers
if cls in ("Reroute", "PrimitiveNode", "Note", "easy showAnything"):
inputs = node.get("inputs", {}) or {}
# Find first link-shaped input and follow it
next_link = next((v for v in inputs.values() if is_link(v)), None)
if next_link is None:
return nid
nid = next_link[0]
continue
return nid
return nid
def find_negative_prompt_node(workflow: dict) -> str | None:
"""Trace `negative` input of a sampler back to the source text encoder."""
for nid, node in iter_nodes(workflow):
if node["class_type"] not in SAMPLER_NODE_FAMILY:
continue
inputs = node.get("inputs", {}) or {}
neg = inputs.get("negative")
if not is_link(neg):
continue
src = trace_to_node(workflow, neg)
if src and isinstance(workflow.get(src), dict):
cls = workflow[src].get("class_type", "")
if cls.startswith("CLIPTextEncode") or cls in ("smZ CLIPTextEncode", "BNK_CLIPTextEncodeAdvanced"):
return src
return None
def find_positive_prompt_node(workflow: dict) -> str | None:
for nid, node in iter_nodes(workflow):
if node["class_type"] not in SAMPLER_NODE_FAMILY:
continue
inputs = node.get("inputs", {}) or {}
pos = inputs.get("positive")
if not is_link(pos):
continue
src = trace_to_node(workflow, pos)
if src and isinstance(workflow.get(src), dict):
cls = workflow[src].get("class_type", "")
if cls.startswith("CLIPTextEncode") or cls in ("smZ CLIPTextEncode", "BNK_CLIPTextEncodeAdvanced"):
return src
return None
def extract_schema(workflow: dict) -> dict:
"""Extract controllable parameters from a workflow.
Returns:
{
"parameters": { friendly_name: {node_id, field, type, value, ...} },
"output_nodes": [node_id, ...],
"model_dependencies": [{node_id, class_type, field, value, folder}],
"embedding_dependencies": [{node_id, embedding_name, found_in_field, value_excerpt}],
"summary": {...}
}
"""
output_nodes: list[str] = []
# First pass: identify positive / negative prompt nodes via connection tracing
pos_node = find_positive_prompt_node(workflow)
neg_node = find_negative_prompt_node(workflow)
# ----- collect raw parameter candidates -----
# Each candidate = (friendly_name, node_id, field, value)
# We resolve duplicate friendly_names AFTER the loop so dedup is symmetric.
raw_params: list[dict] = []
for node_id, node in iter_nodes(workflow):
cls = node["class_type"]
inputs = node.get("inputs", {}) or {}
if cls in OUTPUT_NODES:
output_nodes.append(node_id)
# Match this node against PARAM_PATTERNS
for p_class, p_field, friendly in PARAM_PATTERNS:
if cls != p_class:
continue
if p_field not in inputs:
continue
value = inputs[p_field]
t = infer_type(value)
if t == "link":
continue # connections aren't directly controllable
actual_name = friendly
# Disambiguate prompt vs negative_prompt by connection tracing
if friendly == "prompt":
if node_id == neg_node and pos_node != neg_node:
actual_name = "negative_prompt"
elif node_id == pos_node:
actual_name = "prompt"
else:
# Fallback: use _meta.title hints if present
meta_title = (node.get("_meta") or {}).get("title", "").lower()
if any(t_ in meta_title for t_ in ("negative", "neg", "-prompt", "anti")):
actual_name = "negative_prompt"
raw_params.append({
"name_hint": actual_name,
"node_id": node_id,
"field": p_field,
"type": t,
"value": value,
"class_type": cls,
})
# ----- symmetric duplicate-name resolution -----
# Group by name_hint. If a hint appears once, keep it. If multiple, suffix
# ALL with their node_id. Always-stable, always-uniquely-addressable.
by_name: dict[str, list[dict]] = {}
for r in raw_params:
by_name.setdefault(r["name_hint"], []).append(r)
parameters: dict[str, dict] = {}
for name, entries in by_name.items():
if len(entries) == 1:
r = entries[0]
parameters[name] = {
"node_id": r["node_id"], "field": r["field"],
"type": r["type"], "value": r["value"],
"class_type": r["class_type"],
}
else:
# Sort by node_id (string-natural) for stability
entries.sort(key=lambda x: (str(x["node_id"]).zfill(8), x["field"]))
for r in entries:
full_name = f"{name}_{r['node_id']}"
parameters[full_name] = {
"node_id": r["node_id"], "field": r["field"],
"type": r["type"], "value": r["value"],
"class_type": r["class_type"],
"alias_of": name,
}
# ----- model dependencies -----
model_deps = list(iter_model_deps(workflow))
# ----- embedding dependencies (in prompt text) -----
embedding_deps: list[dict] = []
seen_emb: set[tuple[str, str]] = set()
for nid, emb_name in iter_embedding_refs(workflow):
key = (nid, emb_name)
if key in seen_emb:
continue
seen_emb.add(key)
# Find which field had the reference, for context
node = workflow.get(nid, {})
inputs = node.get("inputs", {}) or {}
found_field = None
excerpt = None
for fname, fval in inputs.items():
if isinstance(fval, str) and fname in PROMPT_FIELDS and emb_name in fval:
found_field = fname
excerpt = fval[:120]
break
embedding_deps.append({
"node_id": nid,
"embedding_name": emb_name,
"field": found_field,
"value_excerpt": excerpt,
"folder": "embeddings",
})
# ----- summary -----
summary = {
"parameter_count": len(parameters),
"output_node_count": len(output_nodes),
"model_dep_count": len(model_deps),
"embedding_dep_count": len(embedding_deps),
"has_negative_prompt": "negative_prompt" in parameters,
"has_seed": "seed" in parameters or any(p.startswith("seed_") for p in parameters),
"is_video_workflow": any(
workflow.get(n, {}).get("class_type", "") in {
"VHS_VideoCombine", "SaveVideo", "SaveAnimatedWEBP", "SaveAnimatedPNG",
} for n in output_nodes
),
}
return {
"parameters": parameters,
"output_nodes": output_nodes,
"model_dependencies": model_deps,
"embedding_dependencies": embedding_deps,
"summary": summary,
}
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(description="Extract controllable parameters from a ComfyUI workflow")
p.add_argument("workflow", help="Path to workflow API JSON file")
p.add_argument("--output", "-o", help="Output file (default: stdout)")
p.add_argument("--summary-only", action="store_true",
help="Only print the summary block")
args = p.parse_args(argv)
wf_path = Path(args.workflow).expanduser()
if not wf_path.exists():
print(f"Error: {wf_path} not found", file=sys.stderr)
return 1
try:
with wf_path.open() as f:
payload = json.load(f)
workflow = unwrap_workflow(payload)
except ValueError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except json.JSONDecodeError as e:
print(f"Error: invalid JSON — {e}", file=sys.stderr)
return 1
schema = extract_schema(workflow)
if args.summary_only:
out = json.dumps(schema["summary"], indent=2)
else:
out = json.dumps(schema, indent=2, default=str)
if args.output:
Path(args.output).write_text(out)
print(f"Schema written to {args.output}", file=sys.stderr)
else:
print(out)
return 0
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/fetch_logs.py
Executable
+158
View File
@@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""
fetch_logs.py — Retrieve workflow execution diagnostics from a ComfyUI server.
When a workflow errors, the server's /history (local) or /jobs (cloud) entry
contains the full Python traceback. This script makes it easy to fetch by
prompt_id, with sensible formatting.
Usage:
python3 fetch_logs.py <prompt_id>
python3 fetch_logs.py <prompt_id> --host https://cloud.comfy.org
python3 fetch_logs.py --tail-queue # show currently queued/running jobs
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, http_get, is_cloud_host,
resolve_api_key, resolve_url,
)
def fetch_history_entry(host: str, headers: dict, prompt_id: str, *, is_cloud: bool) -> dict:
if is_cloud:
# Try /jobs/{id} first
url = resolve_url(host, f"/jobs/{prompt_id}", is_cloud=True)
r = http_get(url, headers=headers, retries=2, timeout=30)
if r.status == 200:
try:
return {"ok": True, "entry": r.json(), "source": "/api/jobs"}
except Exception:
pass
# Fallback to history_v2
url = resolve_url(host, f"/history/{prompt_id}", is_cloud=True)
r = http_get(url, headers=headers, retries=2, timeout=30)
try:
data = r.json()
except Exception:
data = None
if r.status == 200 and data:
return {"ok": True, "entry": data, "source": "/api/history_v2"}
return {"ok": False, "http_status": r.status, "body": r.text()[:500]}
url = resolve_url(host, f"/history/{prompt_id}", is_cloud=False)
r = http_get(url, headers=headers, retries=2, timeout=30)
if r.status != 200:
return {"ok": False, "http_status": r.status, "body": r.text()[:500]}
try:
data = r.json()
except Exception:
return {"ok": False, "reason": "non-JSON response"}
if not isinstance(data, dict) or prompt_id not in data:
return {"ok": False, "reason": "prompt_id not found in history",
"history_keys": list(data.keys())[:5] if isinstance(data, dict) else []}
return {"ok": True, "entry": data[prompt_id], "source": "/history"}
def fetch_queue(host: str, headers: dict) -> dict:
url = resolve_url(host, "/queue")
r = http_get(url, headers=headers, retries=2, timeout=15)
try:
data = r.json()
except Exception:
data = {"raw": r.text()[:500]}
return {"http_status": r.status, "data": data}
def extract_diagnostics(entry: dict) -> dict:
"""Pull out the parts a human cares about: status, errors, traceback, timing."""
diag: dict = {}
status = entry.get("status") or {}
diag["status_str"] = status.get("status_str")
diag["completed"] = status.get("completed")
messages = status.get("messages") or []
diag["execution_log"] = []
for msg in messages:
if isinstance(msg, list) and len(msg) >= 2:
mtype, mdata = msg[0], msg[1]
diag["execution_log"].append({"type": mtype, "data": mdata})
else:
diag["execution_log"].append(msg)
# Look for execution_error inside messages
errors = []
for msg in messages:
if isinstance(msg, list) and len(msg) >= 2 and msg[0] == "execution_error":
errors.append(msg[1])
if errors:
diag["errors"] = errors
# Cloud's /jobs response shape: top-level outputs / status / etc.
if "outputs" in entry:
out = entry["outputs"] or {}
if isinstance(out, dict):
diag["output_node_ids"] = list(out.keys())
# Count file refs across all output buckets (images / video / etc.)
total = 0
for node_output in out.values():
if not isinstance(node_output, dict):
continue
for v in node_output.values():
if isinstance(v, list):
total += len(v)
diag["output_count"] = total
else:
diag["output_node_ids"] = []
diag["output_count"] = 0
return diag
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(description="Fetch workflow execution diagnostics")
p.add_argument("prompt_id", nargs="?", help="prompt_id to look up")
p.add_argument("--host", default=DEFAULT_LOCAL_HOST)
p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}")
p.add_argument("--raw", action="store_true",
help="Print the full history entry instead of the digest")
p.add_argument("--tail-queue", action="store_true",
help="Show currently running/pending jobs instead")
args = p.parse_args(argv)
api_key = resolve_api_key(args.api_key)
headers = {"X-API-Key": api_key} if api_key else {}
is_cloud = is_cloud_host(args.host)
if args.tail_queue:
emit_json(fetch_queue(args.host, headers))
return 0
if not args.prompt_id:
print("Error: prompt_id is required (or use --tail-queue)", file=sys.stderr)
return 1
res = fetch_history_entry(args.host, headers, args.prompt_id, is_cloud=is_cloud)
if not res.get("ok"):
emit_json(res)
return 1
if args.raw:
emit_json(res)
return 0
diag = extract_diagnostics(res["entry"])
diag["source"] = res.get("source")
diag["prompt_id"] = args.prompt_id
emit_json(diag)
return 0 if diag.get("status_str") not in ("error",) else 1
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/hardware_check.py
Executable
+497
View File
@@ -0,0 +1,497 @@
#!/usr/bin/env python3
"""hardware_check.py — Detect whether this machine can realistically run ComfyUI locally.
Improvements over v1:
- Multi-GPU detection: scans all NVIDIA / AMD GPUs, picks the best one (most VRAM)
- Apple Silicon: detects Rosetta-via-x86_64 false negative; warns instead of misclassifying
- Apple generation: defaults to None (unknown) instead of mis-tagging as M1
- WSL2 detection: identifies WSL2 + nvidia-smi situation explicitly
- ROCm: prefers `rocm-smi --json` for new ROCm 6.x output
- Disk space check: warns if /home or workspace volume has < 25 GB free
- PyTorch verification (optional): tries to import torch and check device availability
- Windows: prefers PowerShell `Get-CimInstance` over deprecated `wmic`
- More accurate VRAM thresholds and verdict reasons
Emits a structured JSON report. Exit codes match `verdict`:
0 → ok
1 → marginal
2 → cloud
Usage:
python3 hardware_check.py [--json] [--check-pytorch]
"""
from __future__ import annotations
import json
import os
import platform
import re
import shutil
import subprocess
import sys
from typing import Any
# Thresholds (GiB).
MIN_VRAM_GB_USABLE = 6
OK_VRAM_GB = 8
GREAT_VRAM_GB = 12
MIN_MAC_RAM_GB = 16
OK_MAC_RAM_GB = 32
MIN_FREE_DISK_GB = 25 # ComfyUI core ~5 GB + one model ~5–24 GB
_COMFY_CLI_FLAG = {
"nvidia": "--nvidia",
"amd": "--amd",
"apple-silicon": "--m-series",
"intel": None,
"comfy-cloud": None,
"cpu": "--cpu",
}
def _run(cmd: list[str], timeout: int = 8) -> str:
try:
out = subprocess.run(
cmd, capture_output=True, text=True, timeout=timeout, check=False
)
return (out.stdout or "") + (out.stderr or "")
except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
return ""
def is_wsl() -> bool:
"""Return True when running under Windows Subsystem for Linux."""
if platform.system() != "Linux":
return False
if "microsoft" in platform.release().lower() or "wsl" in platform.release().lower():
return True
try:
with open("/proc/version", "r") as fh:
return "microsoft" in fh.read().lower()
except OSError:
return False
def is_rosetta() -> bool:
"""Return True when Python is running translated under Rosetta on Apple Silicon."""
if platform.system() != "Darwin":
return False
if platform.machine() == "arm64":
return False
# x86_64 on Darwin — could be Intel Mac or Rosetta. Probe sysctl.
out = _run(["sysctl", "-in", "sysctl.proc_translated"]).strip()
return out == "1"
def detect_nvidia() -> dict | None:
"""Detect NVIDIA GPUs. Returns the GPU with the most VRAM, plus list of all."""
if not shutil.which("nvidia-smi"):
return None
out = _run([
"nvidia-smi",
"--query-gpu=index,name,memory.total,driver_version",
"--format=csv,noheader,nounits",
])
if not out.strip():
return None
gpus = []
for line in out.strip().splitlines():
parts = [p.strip() for p in line.split(",")]
if len(parts) < 3:
continue
try:
idx = int(parts[0])
name = parts[1]
vram_mb = int(parts[2])
except ValueError:
continue
driver = parts[3] if len(parts) > 3 else ""
gpus.append({
"vendor": "nvidia",
"index": idx,
"name": name,
"vram_gb": round(vram_mb / 1024, 1),
"driver": driver,
})
if not gpus:
return None
# Pick GPU with most VRAM
best = max(gpus, key=lambda g: g["vram_gb"])
if len(gpus) > 1:
best["all_gpus"] = gpus
return best
def detect_rocm() -> dict | None:
if not shutil.which("rocm-smi"):
return None
# Prefer JSON output (new ROCm 6.x)
out = _run(["rocm-smi", "--showproductname", "--showmeminfo", "vram", "--json"])
if out.strip().startswith("{"):
try:
data = json.loads(out)
cards = []
for card_id, info in data.items():
if not card_id.startswith("card"):
continue
name = (info.get("Card series") or info.get("Card model")
or info.get("Marketing Name") or "AMD GPU")
vram_b = info.get("VRAM Total Memory (B)") or info.get("vram_total_memory_b") or 0
try:
vram_b = int(vram_b)
except (ValueError, TypeError):
vram_b = 0
cards.append({
"vendor": "amd",
"name": str(name).strip(),
"vram_gb": round(vram_b / (1024**3), 1),
"driver": "rocm",
})
if cards:
best = max(cards, key=lambda c: c["vram_gb"])
if len(cards) > 1:
best["all_gpus"] = cards
return best
except json.JSONDecodeError:
pass
# Fall back to text parsing
out = _run(["rocm-smi", "--showproductname", "--showmeminfo", "vram"])
if not out.strip():
return None
name_m = re.search(r"Card (?:series|model|Marketing Name):\s*(.+)", out)
vram_m = re.search(r"VRAM Total Memory \(B\):\s*(\d+)", out)
vram_gb = round(int(vram_m.group(1)) / (1024**3), 1) if vram_m else 0.0
return {
"vendor": "amd",
"name": name_m.group(1).strip() if name_m else "AMD GPU",
"vram_gb": vram_gb,
"driver": "rocm",
}
def detect_apple_silicon() -> dict | None:
if platform.system() != "Darwin":
return None
if platform.machine() != "arm64":
return None
chip = _run(["sysctl", "-n", "machdep.cpu.brand_string"]).strip()
m = re.search(r"Apple M(\d+)", chip)
generation = int(m.group(1)) if m else None
mem_bytes = 0
try:
mem_bytes = int(_run(["sysctl", "-n", "hw.memsize"]).strip() or 0)
except ValueError:
pass
ram_gb = round(mem_bytes / (1024**3), 1) if mem_bytes else 0.0
# Detect chip variant ("Pro", "Max", "Ultra") — affects performance even at same gen
variant = None
for v in ("Ultra", "Max", "Pro"):
if v in chip:
variant = v
break
return {
"vendor": "apple",
"name": chip or "Apple Silicon",
"generation": generation,
"variant": variant,
"unified_memory_gb": ram_gb,
}
def detect_intel_arc() -> dict | None:
if platform.system() not in ("Linux", "Windows"):
return None
if shutil.which("clinfo"):
out = _run(["clinfo", "--list"])
if "Intel" in out and ("Arc" in out or "Xe" in out):
return {"vendor": "intel", "name": "Intel Arc/Xe", "vram_gb": 0.0}
# Windows: try Get-CimInstance
if platform.system() == "Windows" and shutil.which("powershell"):
out = _run(["powershell", "-NoProfile",
"Get-CimInstance Win32_VideoController | Select-Object Name | Format-List"])
if "Intel" in out and ("Arc" in out or "Iris Xe" in out):
return {"vendor": "intel", "name": "Intel Arc/Iris Xe", "vram_gb": 0.0}
return None
def total_system_ram_gb() -> float:
sysname = platform.system()
if sysname == "Darwin":
try:
return round(int(_run(["sysctl", "-n", "hw.memsize"]).strip() or 0) / (1024**3), 1)
except ValueError:
return 0.0
if sysname == "Linux":
try:
with open("/proc/meminfo", "r") as fh:
for line in fh:
if line.startswith("MemTotal:"):
kb = int(line.split()[1])
return round(kb / (1024**2), 1)
except OSError:
return 0.0
if sysname == "Windows":
if shutil.which("powershell"):
out = _run([
"powershell", "-NoProfile",
"(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory",
])
m = re.search(r"(\d{8,})", out)
if m:
return round(int(m.group(1)) / (1024**3), 1)
# Fall back to wmic for older Windows
out = _run(["wmic", "ComputerSystem", "get", "TotalPhysicalMemory"])
m = re.search(r"(\d{6,})", out)
if m:
return round(int(m.group(1)) / (1024**3), 1)
return 0.0
def total_free_disk_gb(path: str = ".") -> float:
try:
usage = shutil.disk_usage(path)
return round(usage.free / (1024**3), 1)
except OSError:
return 0.0
def check_pytorch_cuda() -> dict | None:
"""Optional PyTorch availability check. Only run when --check-pytorch is set."""
try:
import torch # type: ignore[import-not-found]
except Exception as e:
return {"available": False, "reason": f"torch not importable: {e}"}
info: dict[str, Any] = {
"available": True,
"torch_version": torch.__version__,
}
try:
info["cuda_available"] = bool(torch.cuda.is_available())
if info["cuda_available"]:
info["cuda_device_count"] = torch.cuda.device_count()
info["cuda_device_0"] = torch.cuda.get_device_name(0)
except Exception:
info["cuda_available"] = False
try:
info["mps_available"] = bool(torch.backends.mps.is_available())
except Exception:
info["mps_available"] = False
return info
def classify(gpu: dict | None, ram_gb: float, free_disk_gb: float, *, wsl: bool, rosetta: bool) -> tuple[str, str, list[str]]:
notes: list[str] = []
if rosetta:
notes.append(
"Detected Python running under Rosetta on Apple Silicon. "
"ComfyUI MPS support requires native ARM64 Python — install via "
"`brew install python` or arm64 Miniforge, then re-run."
)
return "cloud", "comfy-cloud", notes
if wsl and gpu and gpu["vendor"] == "nvidia":
notes.append("Detected WSL2 + NVIDIA — confirm `nvidia-smi` works in your WSL distro before installing.")
if free_disk_gb and free_disk_gb < MIN_FREE_DISK_GB:
notes.append(
f"Free disk space ({free_disk_gb} GB) is below the {MIN_FREE_DISK_GB} GB recommended minimum. "
"ComfyUI core (~5 GB) plus one SDXL model (~6.5 GB) needs space; Flux Dev needs ~24 GB."
)
# Host RAM matters even for discrete-GPU systems: ComfyUI swaps model
# weights through CPU RAM when shuffling between text encoders / VAE / UNet.
# Apple's unified-memory check is handled below so don't double-warn.
if ram_gb and ram_gb < 8 and gpu and gpu.get("vendor") != "apple":
notes.append(
f"System RAM ({ram_gb} GB) is low. ComfyUI swaps model weights through "
"host RAM; <8 GB causes severe slowdowns. 16+ GB recommended."
)
if gpu is None:
notes.append(
"No supported accelerator found (NVIDIA CUDA / AMD ROCm / Apple Silicon / Intel Arc)."
)
notes.append(
"CPU-only ComfyUI works but is unusably slow for modern models — use Comfy Cloud."
)
return "cloud", "comfy-cloud", notes
if gpu["vendor"] == "apple":
gen = gpu.get("generation")
variant = gpu.get("variant")
mem = gpu.get("unified_memory_gb", 0.0)
gen_str = f"M{gen}" if gen else "Apple Silicon"
if variant:
gen_str += f" {variant}"
if mem < MIN_MAC_RAM_GB:
notes.append(
f"{gen_str} with {mem} GB unified memory — below the {MIN_MAC_RAM_GB} GB practical minimum."
)
notes.append("SD1.5 may work; SDXL/Flux will swap or OOM. Recommend Comfy Cloud.")
return "cloud", "comfy-cloud", notes
if mem < OK_MAC_RAM_GB:
notes.append(
f"{gen_str} with {mem} GB — SDXL works but slow. Flux/video likely too tight."
)
return "marginal", "apple-silicon", notes
notes.append(f"{gen_str} with {mem} GB unified memory — good for SDXL/Flux.")
return "ok", "apple-silicon", notes
if gpu["vendor"] == "intel":
notes.append("Intel Arc detected — ComfyUI IPEX support is experimental; Comfy Cloud is more reliable.")
return "marginal", "intel", notes
# Discrete NVIDIA / AMD
vram = gpu.get("vram_gb", 0.0)
name = gpu["name"]
if vram < MIN_VRAM_GB_USABLE:
notes.append(
f"{name} has only {vram} GB VRAM — below the {MIN_VRAM_GB_USABLE} GB practical minimum."
)
notes.append("Most modern models won't load. Recommend Comfy Cloud.")
return "cloud", "comfy-cloud", notes
if vram < OK_VRAM_GB:
notes.append(
f"{name} ({vram} GB VRAM) — SD1.5 works, SDXL tight, Flux/video unlikely."
)
return "marginal", gpu["vendor"], notes
if vram < GREAT_VRAM_GB:
notes.append(f"{name} ({vram} GB VRAM) — SDXL comfortable, Flux possible with optimizations.")
return "ok", gpu["vendor"], notes
notes.append(f"{name} ({vram} GB VRAM) — can run everything including Flux/video.")
return "ok", gpu["vendor"], notes
def build_report(*, check_pytorch: bool = False) -> dict:
sysname = platform.system()
arch = platform.machine()
ram_gb = total_system_ram_gb()
free_disk_gb = total_free_disk_gb(os.path.expanduser("~"))
rosetta = is_rosetta()
wsl = is_wsl()
gpu = (
detect_nvidia()
or detect_rocm()
or detect_apple_silicon()
or detect_intel_arc()
)
# Intel Mac: arm64 detect failed AND no other GPU paths
if gpu is None and sysname == "Darwin" and arch != "arm64" and not rosetta:
notes = [
"Intel Mac detected — no MPS backend available.",
"ComfyUI will fall back to CPU which is unusably slow. Use Comfy Cloud.",
]
report = {
"os": sysname,
"arch": arch,
"system_ram_gb": ram_gb,
"free_disk_gb": free_disk_gb,
"wsl": False,
"rosetta": False,
"gpu": None,
"verdict": "cloud",
"recommended_install_path": "comfy-cloud",
"comfy_cli_flag": None,
"notes": notes,
"install_urls": _install_urls(),
}
if check_pytorch:
report["pytorch"] = check_pytorch_cuda()
return report
verdict, install_path, notes = classify(
gpu, ram_gb, free_disk_gb, wsl=wsl, rosetta=rosetta,
)
report = {
"os": sysname,
"arch": arch,
"system_ram_gb": ram_gb,
"free_disk_gb": free_disk_gb,
"wsl": wsl,
"rosetta": rosetta,
"gpu": gpu,
"verdict": verdict,
"recommended_install_path": install_path,
"comfy_cli_flag": _COMFY_CLI_FLAG.get(install_path),
"notes": notes,
"install_urls": _install_urls(),
}
if check_pytorch:
report["pytorch"] = check_pytorch_cuda()
return report
def _install_urls() -> dict:
return {
"desktop": "https://docs.comfy.org/installation/desktop",
"manual": "https://docs.comfy.org/installation/manual_install",
"comfy_cli": "https://docs.comfy.org/comfy-cli/getting-started",
"cloud": "https://platform.comfy.org",
}
def main(argv: list[str] | None = None) -> int:
import argparse
p = argparse.ArgumentParser(description="Check whether this machine can run ComfyUI locally.")
p.add_argument("--json", action="store_true", help="Emit machine-readable JSON only")
p.add_argument("--check-pytorch", action="store_true",
help="Also probe `torch` for CUDA/MPS availability (slower)")
args = p.parse_args(argv)
report = build_report(check_pytorch=args.check_pytorch)
if args.json:
print(json.dumps(report, indent=2))
else:
print(f"OS: {report['os']} ({report['arch']})")
if report.get("wsl"):
print("Env: WSL2")
if report.get("rosetta"):
print("Env: Rosetta (x86_64 Python on Apple Silicon)")
print(f"RAM: {report['system_ram_gb']} GB")
print(f"Free disk: {report['free_disk_gb']} GB (~/)")
if report["gpu"]:
g = report["gpu"]
if g["vendor"] == "apple":
print(f"GPU: {g['name']} — {g.get('unified_memory_gb', 0)} GB unified memory")
else:
print(f"GPU: {g['name']} — {g.get('vram_gb', 0)} GB VRAM")
if g.get("all_gpus") and len(g["all_gpus"]) > 1:
print(f" ({len(g['all_gpus'])} GPUs total; using best by VRAM)")
else:
print("GPU: (none detected)")
print(f"Verdict: {report['verdict']} → {report['recommended_install_path']}")
if report["comfy_cli_flag"]:
print(f" run: comfy --skip-prompt install {report['comfy_cli_flag']}")
if report.get("pytorch"):
pt = report["pytorch"]
if pt.get("available"):
line = f"PyTorch: {pt.get('torch_version')}"
if pt.get("cuda_available"):
line += f" + CUDA ({pt.get('cuda_device_0', '?')})"
if pt.get("mps_available"):
line += " + MPS"
print(line)
else:
print(f"PyTorch: not available — {pt.get('reason')}")
for n in report["notes"]:
print(f" • {n}")
if report["verdict"] == "ok":
return 0
if report["verdict"] == "marginal":
return 1
return 2
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/health_check.py
Executable
+223
View File
@@ -0,0 +1,223 @@
#!/usr/bin/env python3
"""
health_check.py — One-stop verification that the ComfyUI environment is ready.
Runs through the verification checklist:
1. comfy-cli on PATH
2. server reachable (/system_stats)
3. at least one checkpoint installed
4. (optional) a specific workflow's deps are met
5. (optional) actually submit a tiny test workflow and verify round-trip
Usage:
python3 health_check.py
python3 health_check.py --host https://cloud.comfy.org
python3 health_check.py --workflow my.json
python3 health_check.py --smoke-test # actually submit a tiny workflow
"""
from __future__ import annotations
import argparse
import json
import shutil
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, http_get, parse_model_list,
resolve_api_key, resolve_url, unwrap_workflow,
)
def comfy_cli_status() -> dict:
if shutil.which("comfy"):
return {"available": True, "method": "comfy", "path": shutil.which("comfy")}
if shutil.which("uvx"):
return {"available": True, "method": "uvx",
"hint": "Invoke as `uvx --from comfy-cli comfy ...`"}
return {
"available": False,
"hint": "Install with: pipx install comfy-cli (or `pip install comfy-cli`)",
}
def server_status(host: str, headers: dict) -> dict:
url = resolve_url(host, "/system_stats")
try:
r = http_get(url, headers=headers, retries=2, timeout=10)
if r.status == 200:
try:
stats = r.json() or {}
except Exception:
stats = {}
return {"reachable": True, "url": url, "stats": stats}
return {"reachable": False, "url": url, "http_status": r.status, "body": r.text()[:200]}
except Exception as e:
return {"reachable": False, "url": url, "error": str(e)}
def checkpoint_status(host: str, headers: dict) -> dict:
url = resolve_url(host, "/models/checkpoints")
try:
r = http_get(url, headers=headers, retries=2, timeout=15)
except Exception as e:
return {"queryable": False, "error": str(e)}
if r.status != 200:
return {"queryable": False, "http_status": r.status, "url": url, "body": r.text()[:200]}
try:
models = parse_model_list(r.json())
except Exception:
models = set()
return {"queryable": True, "count": len(models),
"first_few": sorted(models)[:5]}
SMOKE_WORKFLOW = {
# Minimal SD1.5 workflow that doesn't depend on rare nodes.
# 256x256 + 1 step is the smallest config that doesn't trigger SDXL/Flux
# validation errors while still executing fast.
"3": {
"class_type": "KSampler",
"inputs": {
"seed": 1, "steps": 1, "cfg": 7.0,
"sampler_name": "euler", "scheduler": "normal", "denoise": 1.0,
"model": ["4", 0], "positive": ["6", 0], "negative": ["7", 0],
"latent_image": ["5", 0],
},
},
"4": {"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "REPLACE_ME"}},
"5": {"class_type": "EmptyLatentImage",
"inputs": {"width": 256, "height": 256, "batch_size": 1}},
"6": {"class_type": "CLIPTextEncode",
"inputs": {"text": "test", "clip": ["4", 1]}},
"7": {"class_type": "CLIPTextEncode",
"inputs": {"text": "", "clip": ["4", 1]}},
"9": {"class_type": "SaveImage",
"inputs": {"filename_prefix": "smoke", "images": ["3", 0]}},
}
def smoke_test(host: str, headers: dict, ckpt_name: str | None) -> dict:
"""Submit a tiny workflow and verify the server accepts it.
Cancels the job immediately after acceptance so we don't burn GPU
time / cloud minutes on a smoke test.
"""
if not ckpt_name:
return {"ran": False, "reason": "no checkpoint available"}
wf = json.loads(json.dumps(SMOKE_WORKFLOW))
wf["4"]["inputs"]["ckpt_name"] = ckpt_name
# Lazy import to avoid circular issues
from run_workflow import ComfyRunner
api_key = headers.get("X-API-Key")
runner = ComfyRunner(host=host, api_key=api_key)
sub = runner.submit(wf)
if "_http_error" in sub:
return {"ran": True, "submitted": False,
"http_status": sub["_http_error"], "body": sub.get("body")}
pid = sub.get("prompt_id")
if not pid:
return {"ran": True, "submitted": False, "response": sub}
# Cancel so we don't actually waste compute on the smoke test.
cancelled = False
try:
cancelled = runner.cancel(pid)
except Exception:
pass
return {
"ran": True, "submitted": True, "prompt_id": pid,
"cancelled_after_submit": cancelled,
"note": "Submission accepted; cancelled to avoid running the full pipeline.",
}
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(description="One-stop ComfyUI health check")
p.add_argument("--host", default=DEFAULT_LOCAL_HOST)
p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}")
p.add_argument("--workflow", help="Optional: also run check_deps on this workflow")
p.add_argument("--smoke-test", action="store_true",
help="Submit a tiny test workflow and verify round-trip")
p.add_argument("--strict", action="store_true",
help="Exit non-zero on any non-pass condition (including warnings)")
args = p.parse_args(argv)
api_key = resolve_api_key(args.api_key)
headers = {"X-API-Key": api_key} if api_key else {}
cli = comfy_cli_status()
server = server_status(args.host, headers)
ckpts = checkpoint_status(args.host, headers) if server.get("reachable") else None
# ---- workflow check ----
workflow_check: dict | None = None
if args.workflow:
wf_path = Path(args.workflow).expanduser()
if not wf_path.exists():
workflow_check = {"error": "workflow file not found"}
else:
try:
with wf_path.open() as f:
workflow = unwrap_workflow(json.load(f))
from check_deps import check_deps
workflow_check = check_deps(workflow, host=args.host, api_key=api_key)
except (ValueError, json.JSONDecodeError) as e:
workflow_check = {"error": str(e)}
smoke = None
if args.smoke_test and server.get("reachable"):
first_ckpt = ckpts["first_few"][0] if ckpts and ckpts.get("first_few") else None
smoke = smoke_test(args.host, headers, first_ckpt)
# ---- verdict ----
verdict = "pass"
reasons: list[str] = []
if not server.get("reachable"):
verdict = "fail"
reasons.append("server unreachable")
if ckpts and ckpts.get("queryable") and ckpts.get("count", 0) == 0:
verdict = "warn" if verdict == "pass" else verdict
reasons.append("no checkpoints installed")
if workflow_check and workflow_check.get("error"):
verdict = "fail"
reasons.append(f"workflow check failed: {workflow_check['error']}")
elif workflow_check and not workflow_check.get("is_ready"):
if workflow_check.get("node_check_skipped"):
reasons.append("node check skipped (cloud free tier)")
else:
verdict = "fail"
reasons.append("workflow has missing deps")
if smoke and smoke.get("ran") and not smoke.get("submitted"):
verdict = "fail"
reasons.append("smoke-test submission failed")
if not cli.get("available"):
verdict = "warn" if verdict == "pass" else verdict
reasons.append("comfy-cli not on PATH (lifecycle commands won't work)")
report = {
"verdict": verdict,
"reasons": reasons,
"host": args.host,
"comfy_cli": cli,
"server": server,
"checkpoints": ckpts,
"workflow_check": workflow_check,
"smoke_test": smoke,
}
emit_json(report)
if verdict == "pass":
return 0
if verdict == "warn":
return 1 if args.strict else 0
return 1
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/run_batch.py
Executable
+243
View File
@@ -0,0 +1,243 @@
#!/usr/bin/env python3
"""
run_batch.py — Run a workflow many times, varying parameters per run.
Two modes:
1. --count N --randomize-seed
Submit N runs, each with a fresh random seed. Use for quick variations.
2. --sweep '{"seed": [1,2,3], "steps": [20,30]}'
Cartesian product of values. With cloud subscription, runs in parallel
up to your tier's concurrent-job limit.
Both modes write each run's outputs into output-dir/run_NNN/.
Examples:
python3 run_batch.py --workflow flux_dev.json \
--args '{"prompt": "a cat"}' \
--count 8 --randomize-seed \
--output-dir ./outputs/cat-batch
python3 run_batch.py --workflow sdxl.json \
--args '{"prompt": "abstract"}' \
--sweep '{"seed": [1,2,3], "steps": [20, 40]}' \
--output-dir ./outputs/sweep
"""
from __future__ import annotations
import argparse
import itertools
import json
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
DEFAULT_LOCAL_HOST, ENV_API_KEY, coerce_seed, emit_json, log,
looks_like_video_workflow, resolve_api_key, unwrap_workflow,
)
from run_workflow import ( # noqa: E402
ComfyRunner, download_outputs, inject_params,
)
from extract_schema import extract_schema # noqa: E402
def expand_sweep(sweep: dict, base_args: dict, count: int, randomize_seed: bool) -> list[dict]:
"""Generate a list of args dicts for each run."""
if sweep:
# Cartesian product
keys = list(sweep.keys())
values = [sweep[k] if isinstance(sweep[k], list) else [sweep[k]] for k in keys]
runs = []
for combo in itertools.product(*values):
ar = dict(base_args)
for k, v in zip(keys, combo):
ar[k] = v
runs.append(ar)
return runs
# Count mode
runs = []
for _ in range(count):
ar = dict(base_args)
if randomize_seed:
ar["seed"] = coerce_seed(None)
runs.append(ar)
return runs
def execute_one(
runner: ComfyRunner, workflow: dict, schema: dict, args: dict,
*, output_dir: Path, timeout: int, ws: bool,
) -> dict:
wf, warnings = inject_params(workflow, schema, args)
sub = runner.submit(wf)
if "_http_error" in sub:
return {"status": "error", "error": "submission HTTP error",
"details": sub.get("body"), "args": args}
pid = sub.get("prompt_id")
if not pid:
return {"status": "error", "error": "no prompt_id", "response": sub, "args": args}
if sub.get("node_errors"):
return {"status": "error", "error": "validation failed",
"node_errors": sub["node_errors"], "args": args}
if ws:
result = runner.monitor_ws(pid, timeout=timeout)
else:
result = runner.poll_status(pid, timeout=timeout)
if result["status"] != "success":
return {
"status": result["status"],
"prompt_id": pid,
"details": result.get("data"),
"args": args,
}
outputs = result.get("outputs") or runner.get_outputs(pid)
downloaded = download_outputs(runner, outputs, output_dir, preserve_subfolder=False)
return {
"status": "success",
"prompt_id": pid,
"args": args,
"outputs": downloaded,
"warnings": warnings,
}
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(
description="Submit a workflow many times with varying parameters.",
)
p.add_argument("--workflow", required=True)
p.add_argument("--args", default="{}", help="Base parameters JSON")
p.add_argument("--count", type=int, default=0,
help="Number of runs (use with --randomize-seed)")
p.add_argument("--sweep", default="",
help='JSON dict of param→list of values. Cartesian product. '
'e.g. \'{"seed":[1,2,3],"cfg":[5,8]}\'')
p.add_argument("--randomize-seed", action="store_true",
help="In --count mode, vary seed per run")
p.add_argument("--host", default=DEFAULT_LOCAL_HOST)
p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}")
p.add_argument("--partner-key")
p.add_argument("--parallel", type=int, default=1,
help="Concurrent submissions (cloud: up to your tier limit). "
"Default 1 (sequential)")
p.add_argument("--output-dir", default="./outputs/batch")
p.add_argument("--timeout", type=int, default=0)
p.add_argument("--ws", action="store_true")
p.add_argument("--continue-on-error", action="store_true",
help="Don't stop the batch when a run fails")
args = p.parse_args(argv)
if args.count <= 0 and not args.sweep:
emit_json({"error": "Specify --count N or --sweep '{...}'"})
return 1
base_args = json.loads(args.args) if args.args.strip() else {}
sweep = json.loads(args.sweep) if args.sweep.strip() else {}
# Validate sweep shape
if sweep:
if not isinstance(sweep, dict):
emit_json({"error": "--sweep must be a JSON object {param: [values]}"})
return 1
empty = [k for k, v in sweep.items() if isinstance(v, list) and len(v) == 0]
if empty:
emit_json({"error": f"--sweep parameters have empty value lists: {empty}"})
return 1
# If user passed BOTH --sweep and --count/--randomize-seed, --sweep wins
if args.count or args.randomize_seed:
log("--sweep set; ignoring --count / --randomize-seed (sweep defines the runs)")
wf_path = Path(args.workflow).expanduser()
if not wf_path.exists():
emit_json({"error": f"Workflow not found: {args.workflow}"})
return 1
try:
with wf_path.open() as f:
workflow = unwrap_workflow(json.load(f))
except (ValueError, json.JSONDecodeError) as e:
emit_json({"error": str(e)})
return 1
schema = extract_schema(workflow)
runs = expand_sweep(sweep, base_args, args.count, args.randomize_seed)
log(f"Planned {len(runs)} run(s)")
api_key = resolve_api_key(args.api_key)
runner = ComfyRunner(host=args.host, api_key=api_key, partner_key=args.partner_key)
ok, info = runner.check_server()
if not ok:
emit_json({"error": "Cannot reach server", "details": info, "host": args.host})
return 1
timeout = args.timeout
if timeout <= 0:
timeout = 900 if looks_like_video_workflow(workflow) else 300
base_dir = Path(args.output_dir).expanduser()
base_dir.mkdir(parents=True, exist_ok=True)
results: list[dict] = []
failures = 0
if args.parallel > 1:
with ThreadPoolExecutor(max_workers=args.parallel) as ex:
future_to_idx = {}
for i, ar in enumerate(runs):
run_dir = base_dir / f"run_{i:04d}"
fut = ex.submit(
execute_one, runner, workflow, schema, ar,
output_dir=run_dir, timeout=timeout, ws=args.ws,
)
future_to_idx[fut] = i
for fut in as_completed(future_to_idx):
i = future_to_idx[fut]
try:
r = fut.result()
except Exception as e:
r = {"status": "error", "error": str(e), "args": runs[i]}
r["index"] = i
results.append(r)
if r["status"] != "success":
failures += 1
log(f" run {i} → {r['status']}: {r.get('error','?')}")
if not args.continue_on_error:
log(" --continue-on-error not set; aborting batch")
break
else:
log(f" run {i} → success: {len(r.get('outputs', []))} files")
else:
for i, ar in enumerate(runs):
run_dir = base_dir / f"run_{i:04d}"
r = execute_one(runner, workflow, schema, ar,
output_dir=run_dir, timeout=timeout, ws=args.ws)
r["index"] = i
results.append(r)
if r["status"] != "success":
failures += 1
log(f" run {i} → {r['status']}: {r.get('error','?')}")
if not args.continue_on_error:
log(" --continue-on-error not set; aborting batch")
break
else:
log(f" run {i} → success: {len(r.get('outputs', []))} files")
results.sort(key=lambda x: x.get("index", 0))
emit_json({
"status": "success" if failures == 0 else "partial",
"total": len(runs),
"completed": sum(1 for r in results if r["status"] == "success"),
"failed": failures,
"output_dir": str(base_dir),
"results": results,
})
return 0 if failures == 0 else 1
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/run_workflow.py
Executable
+796
View File
@@ -0,0 +1,796 @@
#!/usr/bin/env python3
"""
run_workflow.py — Inject parameters into a ComfyUI workflow, submit it, monitor
execution, and download outputs.
Improvements over v1:
- Cloud-aware URL routing (handles /api prefix and /history_v2 / /experiment/models renames)
- API key from CLI flag OR $COMFY_CLOUD_API_KEY env var
- WebSocket progress monitoring (--ws), with HTTP polling fallback
- Streaming download (no whole-file buffering — handles GB-size video outputs)
- Path-traversal-safe output writes
- Subfolder-aware download paths (no silent overwrites)
- Retry with exponential backoff on transient errors
- Status-error correctly classified before "completed: true"
- Image upload helper (--input-image NAME=PATH)
- Auto-randomize seed when value is -1 or omitted on a randomize-seed flag
- Auto-extends timeout heuristically for video workflows
- Editor-format detection with helpful error
- Doesn't pollute extra_data.api_key_comfy_org with the cloud auth key
unless --partner-key is provided (correct semantic per cloud docs)
Usage:
# Local server
python3 run_workflow.py --workflow workflow_api.json \
--args '{"prompt": "a cat", "seed": 42}' \
--output-dir ./outputs
# Cloud server (API key from env var)
export COMFY_CLOUD_API_KEY="comfyui-xxxxxxx"
python3 run_workflow.py --workflow workflow_api.json \
--args '{"prompt": "a cat"}' \
--host https://cloud.comfy.org \
--output-dir ./outputs
# With image input (auto-uploads, then references)
python3 run_workflow.py --workflow img2img.json \
--input-image image=./photo.png \
--args '{"prompt": "make it cyberpunk"}'
# WebSocket real-time progress
python3 run_workflow.py --workflow flux_dev.json \
--args '{"prompt": "..."}' \
--ws
Stdlib-only by default (Python 3.10+). Will use `requests`/`websocket-client`
if installed for nicer behavior.
"""
from __future__ import annotations
import argparse
import copy
import json
import sys
import time
from pathlib import Path
from typing import Any
from urllib.parse import urlencode, urlparse
# Local import — _common.py sits next to this script.
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
DEFAULT_LOCAL_HOST, ENV_API_KEY,
coerce_seed, emit_json, http_get, http_post, http_request,
is_cloud_host, is_link, log, looks_like_video_workflow,
media_type_from_filename, new_client_id, resolve_api_key, resolve_url,
safe_path_join, unwrap_workflow,
)
# =============================================================================
# Runner
# =============================================================================
class WorkflowRunError(Exception):
"""Raised when a workflow run fails (validation, execution, timeout)."""
def __init__(self, status: str, message: str, **details: Any):
super().__init__(message)
self.status = status
self.message = message
self.details = details
def to_dict(self) -> dict:
d = {"status": self.status, "error": self.message}
d.update(self.details)
return d
class ComfyRunner:
def __init__(
self,
host: str = DEFAULT_LOCAL_HOST,
api_key: str | None = None,
client_id: str | None = None,
partner_key: str | None = None,
):
self.host = host.rstrip("/")
self.api_key = api_key
self.partner_key = partner_key
self.is_cloud = is_cloud_host(self.host)
self.client_id = client_id or new_client_id()
@property
def headers(self) -> dict[str, str]:
h: dict[str, str] = {}
if self.api_key:
h["X-API-Key"] = self.api_key
return h
def _url(self, path: str) -> str:
return resolve_url(self.host, path, is_cloud=self.is_cloud)
# ---------- server health ----------
def check_server(self) -> tuple[bool, dict | None]:
try:
r = http_get(self._url("/system_stats"), headers=self.headers, retries=2)
if r.status == 200:
try:
return True, r.json()
except Exception:
return True, None
return False, {"http_status": r.status, "body": r.text()[:500]}
except Exception as e:
return False, {"error": str(e)}
# ---------- upload ----------
def upload_image(self, path: Path, *, image_type: str = "input", overwrite: bool = True,
endpoint: str = "/upload/image", extra_form: dict | None = None) -> dict:
"""Upload an image file via multipart. Returns server-side ref dict."""
if not path.exists():
raise FileNotFoundError(f"input image not found: {path}")
# Stream the file via a handle to avoid OOM on huge inputs (16MP+ photos).
with path.open("rb") as fh:
files = {"image": (path.name, fh)}
form = {"type": image_type}
if overwrite:
form["overwrite"] = "true"
if extra_form:
form.update({k: str(v) for k, v in extra_form.items()})
r = http_request(
"POST", self._url(endpoint),
headers=self.headers, files=files, form=form,
timeout=300, retries=2,
)
if r.status != 200:
raise WorkflowRunError(
"upload_failed",
f"Upload of {path.name} failed: HTTP {r.status}",
body=r.text()[:500],
)
try:
return r.json()
except Exception:
return {"name": path.name}
def upload_mask(self, path: Path, original_ref: dict) -> dict:
"""Upload an inpaint mask, linked to a previously uploaded source image.
`original_ref` should be the dict returned by `upload_image()` for the
source image (or `{"filename": ..., "subfolder": ..., "type": "input"}`).
"""
return self.upload_image(
path,
endpoint="/upload/mask",
extra_form={
"subfolder": "clipspace",
"original_ref": json.dumps(original_ref),
},
)
# ---------- submit ----------
def submit(self, workflow: dict) -> dict:
payload: dict[str, Any] = {"prompt": workflow, "client_id": self.client_id}
if self.partner_key:
payload["extra_data"] = {"api_key_comfy_org": self.partner_key}
r = http_post(self._url("/prompt"), headers=self.headers, json_body=payload, timeout=120)
try:
body = r.json()
except Exception:
body = {"raw": r.text()[:500]}
if r.status != 200:
return {"_http_error": r.status, "body": body}
return body
# ---------- HTTP polling ----------
def poll_status(self, prompt_id: str, *, timeout: float = 300.0,
initial_interval: float = 1.5, max_interval: float = 8.0) -> dict:
start = time.time()
interval = initial_interval
while time.time() - start < timeout:
if self.is_cloud:
r = http_get(
self._url(f"/job/{prompt_id}/status"),
headers=self.headers, retries=2, timeout=30,
)
if r.status == 200:
try:
data = r.json()
except Exception:
data = {}
s = data.get("status")
if s == "completed":
return {"status": "success", "data": data}
if s in ("failed",):
return {"status": "error", "data": data}
if s == "cancelled":
return {"status": "cancelled", "data": data}
# pending / in_progress → continue
elif r.status == 404:
# Cloud sometimes 404s briefly between submit and dispatcher pickup
pass
else:
# transient error — retry loop covers it
pass
else:
# Local: /history/{id} grows once execution completes
r = http_get(
self._url(f"/history/{prompt_id}"),
headers=self.headers, retries=2, timeout=30,
)
if r.status == 200:
try:
data = r.json() or {}
except Exception:
data = {}
entry = data.get(prompt_id)
if isinstance(entry, dict):
st = entry.get("status") or {}
# IMPORTANT: check error first — `completed: true` can coexist with errors
status_str = st.get("status_str")
if status_str == "error":
return {"status": "error", "data": entry}
if st.get("completed", False):
return {"status": "success", "outputs": entry.get("outputs", {})}
# not in history yet → continue polling
time.sleep(interval)
interval = min(max_interval, interval * 1.4)
return {"status": "timeout", "elapsed": time.time() - start}
# ---------- WebSocket monitoring ----------
def monitor_ws(self, prompt_id: str, *, timeout: float = 300.0,
on_progress: Any = None) -> dict:
"""Connect to /ws and listen until execution_success / execution_error.
Falls back to HTTP polling if `websocket-client` is not installed.
Returns same shape as poll_status.
"""
try:
import websocket # type: ignore[import-not-found]
except ImportError:
log("websocket-client not installed; falling back to HTTP polling")
return self.poll_status(prompt_id, timeout=timeout)
# Build WS URL. Preserve any base-path components the user gave us
# (e.g. http://example.com/comfyui → ws://example.com/comfyui/ws).
parsed = urlparse(self.host)
scheme = "wss" if parsed.scheme == "https" else "ws"
netloc = parsed.netloc
base_path = parsed.path.rstrip("/")
ws_url = f"{scheme}://{netloc}{base_path}/ws?clientId={self.client_id}"
if self.is_cloud and self.api_key:
ws_url += f"&token={self.api_key}"
outputs: dict[str, Any] = {}
error_payload: dict[str, Any] | None = None
success = False
seen_executed = False
ws = websocket.create_connection(ws_url, timeout=timeout)
try:
ws.settimeout(timeout)
deadline = time.time() + timeout
while time.time() < deadline:
msg = ws.recv()
if isinstance(msg, bytes):
# Binary preview frame — ignore for now; ws_monitor.py prints them
continue
try:
payload = json.loads(msg)
except Exception:
continue
mtype = payload.get("type", "")
mdata = payload.get("data", {}) or {}
# Filter to our job (cloud broadcasts; local filters via client_id)
pid = mdata.get("prompt_id")
if pid is not None and pid != prompt_id:
continue
if mtype == "progress":
if callable(on_progress):
on_progress({
"type": "progress",
"value": mdata.get("value"),
"max": mdata.get("max"),
"node": mdata.get("node"),
})
elif mtype == "progress_state":
if callable(on_progress):
on_progress({"type": "progress_state", "nodes": mdata.get("nodes", {})})
elif mtype == "executing":
node = mdata.get("node")
if callable(on_progress):
on_progress({"type": "executing", "node": node})
# When `node` is None on a local server, that signals end-of-run
if node is None and not self.is_cloud and seen_executed:
success = True
break
elif mtype == "executed":
seen_executed = True
nid = mdata.get("node")
out = mdata.get("output") or {}
if nid:
outputs[nid] = out
elif mtype == "notification":
if callable(on_progress):
on_progress({"type": "notification", "message": mdata.get("value", "")})
elif mtype == "execution_success":
success = True
break
elif mtype == "execution_error":
error_payload = mdata
break
elif mtype == "execution_interrupted":
error_payload = {"interrupted": True, **mdata}
break
finally:
try:
ws.close()
except Exception:
pass
if error_payload is not None:
return {"status": "error", "data": error_payload}
if success:
return {"status": "success", "outputs": outputs}
return {"status": "timeout", "elapsed": timeout}
# ---------- outputs ----------
def get_outputs(self, prompt_id: str) -> dict:
if self.is_cloud:
# Try /jobs/{id} first (returns full job with outputs); fall back to /history_v2
r = http_get(self._url(f"/jobs/{prompt_id}"), headers=self.headers, retries=2)
if r.status == 200:
try:
return (r.json() or {}).get("outputs", {}) or {}
except Exception:
pass
# Fallback
r = http_get(self._url(f"/history/{prompt_id}"), headers=self.headers, retries=2)
if r.status == 200:
try:
body = r.json() or {}
except Exception:
body = {}
if isinstance(body, dict) and prompt_id in body:
return body[prompt_id].get("outputs", {}) or {}
if isinstance(body, dict) and "outputs" in body:
return body["outputs"] or {}
return {}
# Local
r = http_get(self._url(f"/history/{prompt_id}"), headers=self.headers, retries=2)
if r.status != 200:
return {}
try:
body = r.json() or {}
except Exception:
return {}
entry = body.get(prompt_id) or {}
return entry.get("outputs", {}) or {}
def download_output(
self, *, filename: str, subfolder: str, file_type: str,
output_dir: Path, preserve_subfolder: bool = True, overwrite: bool = False,
) -> Path:
"""Stream a single output to disk. Path-traversal-safe."""
params = {"filename": filename, "subfolder": subfolder, "type": file_type}
url = self._url("/view") + "?" + urlencode(params)
# Compute target path safely. If preserve_subfolder, include subfolder in the
# local path; otherwise put the file in output_dir flat.
target_parts: list[str] = []
if preserve_subfolder and subfolder:
target_parts.extend(p for p in subfolder.split("/") if p and p not in (".", ".."))
target_parts.append(filename)
out_path = safe_path_join(output_dir, *target_parts)
if out_path.exists() and not overwrite:
stem, suffix = out_path.stem, out_path.suffix
i = 1
while True:
candidate = out_path.with_name(f"{stem}_{i}{suffix}")
if not candidate.exists():
out_path = candidate
break
i += 1
out_path.parent.mkdir(parents=True, exist_ok=True)
# Stream download. Two-step for cloud: get the 302, then fetch signed URL
# so we don't accidentally send X-API-Key to the storage backend.
# The HTTP transport already strips X-API-Key on cross-host redirect
# via _strip_api_key_on_redirect, so a single follow_redirects=True call
# is safe AND simpler.
r = http_request(
"GET", url, headers=self.headers,
timeout=600, retries=3, follow_redirects=True,
stream=True, sink=out_path,
)
if r.status != 200:
try:
if out_path.exists():
out_path.unlink()
except Exception:
pass
raise WorkflowRunError(
"download_failed",
f"Download of {filename} failed: HTTP {r.status}",
url=url,
)
return out_path
# ---------- queue / cancel ----------
def cancel(self, prompt_id: str | None = None) -> bool:
if prompt_id:
r = http_post(
self._url("/queue"), headers=self.headers,
json_body={"delete": [prompt_id]}, retries=1,
)
return r.status == 200
# Interrupt currently running
r = http_post(self._url("/interrupt"), headers=self.headers, retries=1)
return r.status == 200
# =============================================================================
# Schema / parameter injection
# =============================================================================
def _inline_schema(workflow: dict) -> dict:
"""Generate schema using the sibling extract_schema module."""
from extract_schema import extract_schema # noqa: WPS433
return extract_schema(workflow)
def load_schema(schema_path: str | None, workflow: dict) -> dict:
if schema_path:
with open(schema_path) as f:
return json.load(f)
return _inline_schema(workflow)
def inject_params(
workflow: dict, schema: dict, args: dict,
*, randomize_seed_if_unset: bool = False,
) -> tuple[dict, list[str]]:
"""Inject user args into the workflow. Returns (new_workflow, warnings)."""
wf = copy.deepcopy(workflow)
params = schema.get("parameters", {}) or {}
warnings: list[str] = []
# Auto-randomize seed when it's -1 in args, or when randomize_seed_if_unset
# and user didn't pass a seed.
if "seed" in params:
if "seed" in args and args["seed"] in (None, -1, "-1"):
args = dict(args)
args["seed"] = coerce_seed(args["seed"])
warnings.append(f"seed=-1 expanded to {args['seed']}")
elif randomize_seed_if_unset and "seed" not in args:
args = dict(args)
args["seed"] = coerce_seed(None)
warnings.append(f"seed auto-randomized to {args['seed']}")
for name, value in args.items():
if name not in params:
warnings.append(f"unknown parameter '{name}' (not in schema), skipping")
continue
m = params[name]
nid, field = m["node_id"], m["field"]
node = wf.get(nid)
if not isinstance(node, dict) or "inputs" not in node:
warnings.append(f"node '{nid}' for parameter '{name}' missing in workflow")
continue
# Refuse to overwrite a link with a literal — would silently break wiring
cur = node["inputs"].get(field)
if is_link(cur):
warnings.append(
f"parameter '{name}' targets {nid}.{field} which is currently a link; "
f"refusing to overwrite (set the schema to point at the source node instead)"
)
continue
node["inputs"][field] = value
return wf, warnings
# =============================================================================
# Output download helper
# =============================================================================
def download_outputs(
runner: ComfyRunner, outputs: dict, output_dir: Path,
*, preserve_subfolder: bool = True, overwrite: bool = False,
) -> list[dict]:
"""Walk the outputs dict and download every file. Cloud uses `video` (singular);
local uses `videos` (plural). We accept both."""
output_dir.mkdir(parents=True, exist_ok=True)
downloaded: list[dict] = []
OUTPUT_KEYS = ("images", "gifs", "videos", "video", "audio", "files", "models", "3d")
for node_id, node_output in (outputs or {}).items():
if not isinstance(node_output, dict):
continue
for key in OUTPUT_KEYS:
entries = node_output.get(key)
if not entries:
continue
if not isinstance(entries, list):
entries = [entries]
for fi in entries:
if not isinstance(fi, dict):
continue
filename = fi.get("filename") or ""
if not filename:
continue
subfolder = fi.get("subfolder") or ""
file_type = fi.get("type") or "output"
try:
out_path = runner.download_output(
filename=filename, subfolder=subfolder, file_type=file_type,
output_dir=output_dir, preserve_subfolder=preserve_subfolder,
overwrite=overwrite,
)
downloaded.append({
"file": str(out_path),
"node_id": node_id,
"type": media_type_from_filename(filename),
"filename": filename,
"subfolder": subfolder,
"source_type": file_type,
})
except Exception as e:
log(f"WARN: failed to download {filename}: {e}")
return downloaded
# =============================================================================
# CLI
# =============================================================================
def parse_input_image_arg(spec: str) -> tuple[str, Path]:
"""Parse `name=path` (or `path` alone, defaulting to name='image')."""
if "=" in spec:
name, path = spec.split("=", 1)
return name.strip(), Path(path).expanduser()
return "image", Path(spec).expanduser()
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(
description="Run a ComfyUI workflow with parameter injection.",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
p.add_argument("--workflow", required=True, help="Path to workflow API JSON file")
p.add_argument("--args", default="{}",
help="JSON parameters to inject (or `@/path/to/args.json`)")
p.add_argument("--schema", help="Path to schema JSON (auto-generated if omitted)")
p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL")
p.add_argument("--api-key",
help=f"API key for cloud (or set ${ENV_API_KEY} env var)")
p.add_argument("--partner-key",
help="Partner-node API key (extra_data.api_key_comfy_org). "
"Required for Flux Pro / Ideogram / etc. Defaults to --api-key if not set.")
p.add_argument("--output-dir", default="./outputs", help="Directory to save outputs")
p.add_argument("--timeout", type=int, default=0,
help="Max seconds to wait (0=auto: 300 / 900 for video workflows)")
p.add_argument("--input-image", action="append", default=[],
help="Upload local image before running. Format: `name=path` or `path`. "
"The `name` becomes the value injected into the matching schema parameter.")
p.add_argument("--randomize-seed", action="store_true",
help="If schema has a 'seed' parameter and --args didn't set one, randomize it")
p.add_argument("--ws", action="store_true",
help="Use WebSocket for real-time progress (requires `websocket-client`)")
p.add_argument("--no-download", action="store_true", help="Skip downloading outputs")
p.add_argument("--flat-output", action="store_true",
help="Don't preserve server-side subfolder structure when saving outputs")
p.add_argument("--overwrite", action="store_true",
help="Overwrite existing files instead of appending _1, _2, ...")
p.add_argument("--submit-only", action="store_true",
help="Submit and return prompt_id without waiting")
p.add_argument("--client-id", help="Override generated client_id (UUID)")
p.add_argument("--use-partner-key-as-auth", action="store_true",
help="(Compat) Use --partner-key value as cloud X-API-Key. Don't use unless you know why.")
args = p.parse_args(argv)
# ---- Load workflow ----
wf_path = Path(args.workflow).expanduser()
if not wf_path.exists():
emit_json({"error": f"Workflow file not found: {args.workflow}"})
return 1
try:
with wf_path.open() as f:
workflow_raw = json.load(f)
workflow = unwrap_workflow(workflow_raw)
except ValueError as e:
emit_json({"error": str(e)})
return 1
except json.JSONDecodeError as e:
emit_json({"error": f"Invalid JSON in workflow file: {e}"})
return 1
# ---- Parse user args ----
args_str = args.args
if args_str.startswith("@"):
try:
args_str = Path(args_str[1:]).read_text()
except OSError as e:
emit_json({"error": f"Cannot read args file: {e}"})
return 1
try:
user_args = json.loads(args_str) if args_str.strip() else {}
except json.JSONDecodeError as e:
emit_json({"error": f"Invalid --args JSON: {e}"})
return 1
if not isinstance(user_args, dict):
emit_json({"error": "--args must be a JSON object"})
return 1
# ---- Resolve API key ----
api_key = resolve_api_key(args.api_key)
partner_key = args.partner_key or None
if args.use_partner_key_as_auth and not api_key and partner_key:
api_key = partner_key
# ---- Connect ----
runner = ComfyRunner(
host=args.host, api_key=api_key, partner_key=partner_key,
client_id=args.client_id,
)
# Server reachability
ok, info = runner.check_server()
if not ok:
emit_json({
"error": f"Cannot reach server at {args.host}",
"details": info,
"hint": (
"Check `comfy launch --background` is running for local, "
f"or set ${ENV_API_KEY} for cloud."
),
})
return 1
# ---- Upload input images ----
upload_warnings: list[str] = []
for spec in args.input_image:
try:
param_name, path = parse_input_image_arg(spec)
except Exception as e:
emit_json({"error": f"Bad --input-image spec '{spec}': {e}"})
return 1
try:
ref = runner.upload_image(path)
except Exception as e:
emit_json({"error": f"Upload failed for {path}: {e}"})
return 1
# Register as a user arg so inject_params consumes it through the schema
uploaded_name = ref.get("name") or path.name
if param_name not in user_args:
user_args[param_name] = uploaded_name
# ---- Inject params ----
schema = load_schema(args.schema, workflow)
workflow, inj_warnings = inject_params(
workflow, schema, user_args, randomize_seed_if_unset=args.randomize_seed,
)
warnings = upload_warnings + inj_warnings
for w in warnings:
log(f"WARN: {w}")
# ---- Submit ----
submit_resp = runner.submit(workflow)
if "_http_error" in submit_resp:
emit_json({
"error": "Submission HTTP error",
"http_status": submit_resp["_http_error"],
"body": submit_resp.get("body"),
})
return 1
if isinstance(submit_resp.get("error"), dict):
emit_json({
"error": "Workflow validation failed",
"details": submit_resp["error"],
"node_errors": submit_resp.get("node_errors"),
})
return 1
prompt_id = submit_resp.get("prompt_id")
if not prompt_id:
emit_json({"error": "No prompt_id in submit response", "response": submit_resp})
return 1
node_errors = submit_resp.get("node_errors") or {}
if node_errors:
emit_json({"error": "Workflow validation failed", "node_errors": node_errors})
return 1
if args.submit_only:
emit_json({"status": "submitted", "prompt_id": prompt_id, "warnings": warnings})
return 0
# ---- Wait ----
timeout = args.timeout
if timeout <= 0:
timeout = 900 if looks_like_video_workflow(workflow) else 300
log(f"Submitted: prompt_id={prompt_id}, waiting (timeout={timeout}s)…")
def _on_progress(evt: dict) -> None:
t = evt.get("type")
if t == "progress":
log(f" step {evt.get('value')}/{evt.get('max')} on node {evt.get('node')}")
elif t == "executing":
node = evt.get("node")
if node:
log(f" executing node {node}")
try:
if args.ws:
wait_result = runner.monitor_ws(prompt_id, timeout=timeout, on_progress=_on_progress)
else:
wait_result = runner.poll_status(prompt_id, timeout=timeout)
except KeyboardInterrupt:
log(f"Interrupted — cancelling job {prompt_id} on server…")
try:
runner.cancel(prompt_id)
except Exception as e:
log(f" (cancel request failed: {e})")
emit_json({
"status": "interrupted",
"prompt_id": prompt_id,
"note": "Ctrl+C received; sent cancellation to server.",
})
return 130
if wait_result["status"] == "timeout":
emit_json({
"status": "timeout",
"prompt_id": prompt_id,
"elapsed": wait_result.get("elapsed"),
"hint": "Re-run with larger --timeout, or use --submit-only and check later.",
})
return 1
if wait_result["status"] == "error":
emit_json({"status": "error", "prompt_id": prompt_id, "details": wait_result.get("data")})
return 1
if wait_result["status"] == "cancelled":
emit_json({"status": "cancelled", "prompt_id": prompt_id})
return 1
# ---- Outputs ----
outputs = wait_result.get("outputs")
if not outputs:
outputs = runner.get_outputs(prompt_id)
if args.no_download:
emit_json({
"status": "success", "prompt_id": prompt_id,
"outputs": outputs, "warnings": warnings,
})
return 0
downloaded = download_outputs(
runner, outputs, Path(args.output_dir).expanduser(),
preserve_subfolder=not args.flat_output, overwrite=args.overwrite,
)
emit_json({
"status": "success",
"prompt_id": prompt_id,
"outputs": downloaded,
"warnings": warnings,
})
return 0
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/scripts/ws_monitor.py
Executable
+267
View File
@@ -0,0 +1,267 @@
#!/usr/bin/env python3
"""
ws_monitor.py — Real-time ComfyUI WebSocket monitor.
Connects to /ws and pretty-prints execution events: node start/finish, sampling
progress, cached nodes, errors. Optionally writes preview frames to disk.
Useful for:
- Watching a long-running job in real time without parsing JSON yourself
- Saving in-progress preview frames for video / animation workflows
- Debugging "why is this hanging?" — see exactly which node is stuck
Usage:
# Local — watch all jobs from this client_id
python3 ws_monitor.py
# Cloud — watch a specific prompt_id
python3 ws_monitor.py --host https://cloud.comfy.org \
--prompt-id abc-123-def
# Save preview frames to ./previews/
python3 ws_monitor.py --previews ./previews
Requires: websocket-client (`pip install websocket-client`).
Falls back to a clear error message when not installed.
"""
from __future__ import annotations
import argparse
import json
import struct
import sys
from pathlib import Path
from urllib.parse import urlparse
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _common import ( # noqa: E402
DEFAULT_LOCAL_HOST, ENV_API_KEY, log, new_client_id, resolve_api_key, is_cloud_host,
)
# Binary frame types from ComfyUI WebSocket protocol
BINARY_PREVIEW_IMAGE = 1
BINARY_TEXT = 3
BINARY_PREVIEW_IMAGE_WITH_METADATA = 4
# Image type codes inside PREVIEW_IMAGE
IMAGE_TYPE_JPEG = 1
IMAGE_TYPE_PNG = 2
# ANSI escape codes (works on most modern terminals)
RESET = "\033[0m"
DIM = "\033[2m"
BOLD = "\033[1m"
GREEN = "\033[32m"
YELLOW = "\033[33m"
RED = "\033[31m"
CYAN = "\033[36m"
def fmt_color(s: str, color: str, *, color_on: bool = True) -> str:
return f"{color}{s}{RESET}" if color_on else s
def parse_binary_frame(data: bytes) -> dict | None:
if len(data) < 8:
return None
type_code = struct.unpack(">I", data[0:4])[0]
if type_code == BINARY_PREVIEW_IMAGE:
image_type = struct.unpack(">I", data[4:8])[0]
ext = "jpg" if image_type == IMAGE_TYPE_JPEG else "png" if image_type == IMAGE_TYPE_PNG else "bin"
return {
"kind": "preview",
"image_type": image_type,
"ext": ext,
"image_bytes": data[8:],
}
if type_code == BINARY_PREVIEW_IMAGE_WITH_METADATA:
if len(data) < 12:
return None
meta_len = struct.unpack(">I", data[4:8])[0]
meta_end = 8 + meta_len
if len(data) < meta_end:
return None
try:
meta = json.loads(data[8:meta_end].decode("utf-8"))
except Exception:
meta = {"raw": data[8:meta_end][:200].decode("utf-8", "replace")}
return {
"kind": "preview_with_metadata",
"metadata": meta,
"image_bytes": data[meta_end:],
"ext": "png",
}
if type_code == BINARY_TEXT:
if len(data) < 8:
return None
nid_len = struct.unpack(">I", data[4:8])[0]
nid_end = 8 + nid_len
if len(data) < nid_end:
return None
return {
"kind": "text",
"node_id": data[8:nid_end].decode("utf-8", "replace"),
"text": data[nid_end:].decode("utf-8", "replace"),
}
return {"kind": "unknown", "type_code": type_code, "size": len(data)}
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(description="Real-time ComfyUI WebSocket monitor")
p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL")
p.add_argument("--api-key", help=f"API key for cloud (or set ${ENV_API_KEY} env var)")
p.add_argument("--client-id", default=None, help="Client ID (default: random UUID)")
p.add_argument("--prompt-id", default=None,
help="Filter to a specific prompt_id (default: all jobs)")
p.add_argument("--previews", default=None,
help="Directory to save in-progress preview frames")
p.add_argument("--no-color", action="store_true", help="Disable ANSI colour")
p.add_argument("--timeout", type=float, default=600.0,
help="Hard cap on monitor duration (default 600s)")
args = p.parse_args(argv)
try:
import websocket # type: ignore[import-not-found]
except ImportError:
print(json.dumps({
"error": "websocket-client not installed",
"install": "pip install websocket-client",
}))
return 1
api_key = resolve_api_key(args.api_key)
cloud = is_cloud_host(args.host)
client_id = args.client_id or new_client_id()
# Build WS URL preserving any base-path component (e.g. behind reverse proxy).
parsed = urlparse(args.host if "://" in args.host else f"http://{args.host}")
scheme = "wss" if parsed.scheme == "https" else "ws"
netloc = parsed.netloc
base_path = parsed.path.rstrip("/")
ws_url = f"{scheme}://{netloc}{base_path}/ws?clientId={client_id}"
if cloud and api_key:
ws_url += f"&token={api_key}"
color_on = not args.no_color and sys.stdout.isatty()
preview_dir = Path(args.previews).expanduser() if args.previews else None
if preview_dir:
preview_dir.mkdir(parents=True, exist_ok=True)
log(f"Saving previews to {preview_dir}")
log(f"Connecting to {ws_url} (client_id={client_id})")
if args.prompt_id:
log(f"Filtering messages to prompt_id={args.prompt_id}")
ws = websocket.create_connection(ws_url, timeout=args.timeout)
ws.settimeout(args.timeout)
preview_counter = 0
try:
while True:
try:
msg = ws.recv()
except websocket.WebSocketTimeoutException:
log(f"Idle for {args.timeout}s — exiting")
return 0
if isinstance(msg, bytes):
parsed = parse_binary_frame(msg)
if parsed is None:
continue
if parsed["kind"] in ("preview", "preview_with_metadata") and preview_dir:
img_bytes = parsed.get("image_bytes", b"")
if img_bytes:
ext = parsed.get("ext", "png")
out = preview_dir / f"preview_{preview_counter:05d}.{ext}"
out.write_bytes(img_bytes)
preview_counter += 1
log(f" [preview] saved {out.name} ({len(img_bytes)} bytes)")
continue
try:
payload = json.loads(msg)
except Exception:
continue
mtype = payload.get("type", "")
mdata = payload.get("data", {}) or {}
pid = mdata.get("prompt_id")
if args.prompt_id and pid and pid != args.prompt_id:
continue
if mtype == "status":
qr = mdata.get("status", {}).get("exec_info", {}).get("queue_remaining", "?")
print(fmt_color(f"[status] queue_remaining={qr}", DIM, color_on=color_on))
elif mtype == "execution_start":
print(fmt_color(f"[start] prompt_id={pid}", BOLD, color_on=color_on))
elif mtype == "executing":
node = mdata.get("node")
if node:
print(fmt_color(f" [executing] node={node}", CYAN, color_on=color_on))
else:
print(fmt_color(f" [executing] (workflow done) prompt_id={pid}", DIM, color_on=color_on))
elif mtype == "progress":
v, m = mdata.get("value", 0), mdata.get("max", 0)
pct = (v / m * 100) if m else 0
print(f" [progress] {v}/{m} ({pct:5.1f}%) node={mdata.get('node')}")
elif mtype == "progress_state":
# Newer extended progress message
nodes = mdata.get("nodes") or {}
running = [k for k, v in nodes.items() if v.get("running")]
if running:
print(fmt_color(f" [progress_state] running={running}", DIM, color_on=color_on))
elif mtype == "executed":
node = mdata.get("node")
out = mdata.get("output") or {}
summary_parts = []
for key in ("images", "video", "videos", "gifs", "audio", "files"):
if out.get(key):
summary_parts.append(f"{key}={len(out[key])}")
summary = ", ".join(summary_parts) if summary_parts else "(no files)"
print(fmt_color(f" [executed] node={node} {summary}", GREEN, color_on=color_on))
elif mtype == "execution_cached":
cached = mdata.get("nodes") or []
if cached:
print(fmt_color(f" [cached] {len(cached)} nodes skipped", DIM, color_on=color_on))
elif mtype == "execution_success":
print(fmt_color(f"[success] prompt_id={pid}", GREEN + BOLD, color_on=color_on))
if args.prompt_id:
return 0
elif mtype == "execution_error":
exc_type = mdata.get("exception_type", "?")
exc_msg = mdata.get("exception_message", "?")
print(fmt_color(f"[error] {exc_type}: {exc_msg}", RED + BOLD, color_on=color_on))
tb = mdata.get("traceback")
if tb:
if isinstance(tb, list):
for line in tb:
print(fmt_color(f" {line}", RED, color_on=color_on))
else:
print(fmt_color(f" {tb}", RED, color_on=color_on))
if args.prompt_id:
return 1
elif mtype == "execution_interrupted":
print(fmt_color(f"[interrupted] prompt_id={pid}", YELLOW, color_on=color_on))
if args.prompt_id:
return 1
elif mtype == "notification":
v = mdata.get("value", "")
print(fmt_color(f"[notification] {v}", DIM, color_on=color_on))
else:
# Unknown / lightly-used types: print compactly
print(fmt_color(f"[{mtype}] {json.dumps(mdata, default=str)[:200]}", DIM, color_on=color_on))
except KeyboardInterrupt:
log("Interrupted")
return 130
finally:
try:
ws.close()
except Exception:
pass
if __name__ == "__main__":
sys.exit(main())
creative/comfyui/templates/comfyui-openai-adapter.py
+337
View File
@@ -0,0 +1,337 @@
#!/usr/bin/env python3
"""
OpenAI-compatible API wrapper for ComfyUI.
Supports: ideogram4, flux2
Endpoint: POST /v1/images/generations
Extended params: seed, steps, cfg, quality, negative_prompt, style
Timeout: 1800s (30 min)
"""
import os
import re
import json
import base64
import time
import asyncio
import httpx
from typing import Optional
from fastapi import FastAPI, Request, HTTPException, Header
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
COMFY_HOST = os.environ.get("COMFYUI_HOST", "localhost")
COMFY_PORT = os.environ.get("COMFYUI_PORT", "8188")
COMFY_URL = f"http://{COMFY_HOST}:{COMFY_PORT}"
API_KEY = os.environ.get("API_KEY", "")
ALLOWED_SIZES = {
"256x256", "512x512", "512x768", "768x512", "768x1024", "1024x768",
"1024x1024", "1024x1536", "1536x1024", "832x1216", "1216x832",
"896x1152", "1152x896", "640x1536", "1536x640"
}
DEFAULT_SIZE = "1024x1024"
QUALITY_STEPS = {
"low": 8,
"medium": 20,
"high": 30,
"ultra": 50,
}
IDEOGRAM_ALLOWED = {"1024x1024", "1024x1536", "1536x1024", "832x1216", "1216x832", "896x1152", "1152x896"}
IDEOGRAM_DEFAULT = "1024x1024"
FLUX_ALLOWED = {"256x256", "512x512", "512x768", "768x512", "768x1024", "1024x768",
"1024x1024", "1024x1536", "1536x1024", "832x1216", "1216x832"}
# Only two models active on this system
ALLOWED_MODELS = {"ideogram4", "flux2"}
DEFAULT_MODEL = "ideogram4"
def _parse_size(size: str) -> tuple[int, int]:
m = re.match(r"(\d+)x(\d+)", size)
if not m:
raise ValueError(f"Invalid size: {size}")
return int(m.group(1)), int(m.group(2))
def _validate_model_size(model: str, size: str) -> str:
if model == "ideogram4":
if size not in IDEOGRAM_ALLOWED:
return IDEOGRAM_DEFAULT
return size
if model.startswith("flux"):
if size not in FLUX_ALLOWED:
return DEFAULT_SIZE
return size
return DEFAULT_SIZE
def _build_ideogram4_workflow(prompt: str, width: int, height: int, seed: int,
steps: int = 20, cfg: float = 1.0,
negative_prompt: str = "") -> dict:
"""
Ideogram 4 requires asymmetric CFG wiring:
- Main UNet goes through CFGOverride before DualModelGuider
- Separate unconditional UNet loaded via second UNETLoader
- ConditioningZeroOut for the negative (not a text prompt)
- DualModelGuider takes model_negative (unconditional UNet), not model_1
"""
return {
"1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"},
"2": {"inputs": {"unet_name": "ideogram4_fp8_scaled.safetensors", "weight_dtype": "default"},
"class_type": "UNETLoader"},
"3": {"inputs": {"clip_name": "qwen3vl_8b_fp8_scaled.safetensors", "type": "ideogram4", "device": "default"},
"class_type": "CLIPLoader"},
"4": {"inputs": {"text": prompt, "clip": ["3", 0]}, "class_type": "CLIPTextEncode"},
"5": {"inputs": {"width": width, "height": height, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
"6": {"inputs": {"noise_seed": seed, "batch_count": 1}, "class_type": "RandomNoise"},
"7": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"},
"8": {"inputs": {"steps": steps, "width": width, "height": height, "mu": 0.5, "std": 1.75},
"class_type": "Ideogram4Scheduler"},
# Asymmetric CFG: second unconditional UNet
"15": {"inputs": {"unet_name": "ideogram4_unconditional_fp8_scaled.safetensors", "weight_dtype": "default"},
"class_type": "UNETLoader"},
# Zero out conditioning for negative path
"16": {"inputs": {"conditioning": ["4", 0]}, "class_type": "ConditioningZeroOut"},
# CFG override on main model (70%-100% of steps)
"17": {"inputs": {"cfg": 3.0, "start_percent": 0.7, "end_percent": 1.0, "model": ["2", 0]},
"class_type": "CFGOverride"},
# DualModelGuider: main model (CFG overridden), unconditional model, positive, zeroed negative
"9": {"inputs": {"cfg": 7.0, "model": ["17", 0], "positive": ["4", 0],
"model_negative": ["15", 0], "negative": ["16", 0]},
"class_type": "DualModelGuider"},
"10": {"inputs": {"noise": ["6", 0], "guider": ["9", 0], "sampler": ["7", 0],
"sigmas": ["8", 0], "latent_image": ["5", 0]}, "class_type": "SamplerCustomAdvanced"},
"11": {"inputs": {"samples": ["10", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"},
"12": {"inputs": {"filename_prefix": "ideogram4_api", "images": ["11", 0]}, "class_type": "SaveImage"},
}
def _build_flux2_workflow(prompt: str, width: int, height: int, seed: int,
steps: int = 20, cfg: float = 3.5,
negative_prompt: str = "") -> dict:
"""FLUX.2-klein-9B workflow using UnetLoaderGGUF + CLIPLoaderGGUF (Qwen3VL)."""
return {
"1": {"inputs": {"vae_name": "flux2-vae.safetensors"}, "class_type": "VAELoader"},
"2": {"inputs": {"unet_name": "flux-2-klein-9b-Q4_K_S.gguf"},
"class_type": "UnetLoaderGGUF"},
"3": {"inputs": {"model": ["2", 0], "max_shift": 1.15, "base_shift": 0.5, "width": width, "height": height},
"class_type": "ModelSamplingFlux"},
# FLUX.2 uses Qwen3VL via CLIPLoaderGGUF, NOT DualCLIPLoader (CLIP-L + T5XXL)
"4": {"inputs": {"clip_name": "Qwen3VL-8B-Instruct-Q4_K_M.gguf", "type": "flux2"},
"class_type": "CLIPLoaderGGUF"},
"5": {"inputs": {"text": prompt, "clip": ["4", 0]}, "class_type": "CLIPTextEncode"},
"6": {"inputs": {"conditioning": ["5", 0], "guidance": cfg}, "class_type": "FluxGuidance"},
"7": {"inputs": {"width": width, "height": height, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"},
"8": {"inputs": {"noise_seed": seed}, "class_type": "RandomNoise"},
"9": {"inputs": {"sampler_name": "euler"}, "class_type": "KSamplerSelect"},
"10": {"inputs": {"model": ["3", 0], "steps": steps, "denoise": 1.0, "scheduler": "simple",
"sampler": ["9", 0]}, "class_type": "BasicScheduler"},
"11": {"inputs": {"model": ["3", 0], "conditioning": ["6", 0]}, "class_type": "BasicGuider"},
"12": {"inputs": {"noise": ["8", 0], "guider": ["11", 0], "sampler": ["9", 0],
"sigmas": ["10", 0], "latent_image": ["7", 0]}, "class_type": "SamplerCustomAdvanced"},
"13": {"inputs": {"samples": ["12", 0], "vae": ["1", 0]}, "class_type": "VAEDecode"},
"14": {"inputs": {"filename_prefix": "flux2_api", "images": ["13", 0]}, "class_type": "SaveImage"},
}
def _build_workflow(model: str, prompt: str, width: int, height: int,
seed: int, steps: int, cfg: float,
negative_prompt: str = "", style: str = "") -> dict:
if model == "ideogram4":
return _build_ideogram4_workflow(prompt, width, height, seed, steps, cfg, negative_prompt)
elif model == "flux2":
return _build_flux2_workflow(prompt, width, height, seed, steps, cfg, negative_prompt)
else:
raise ValueError(f"Unknown model: {model}")
async def _comfy_post(path: str, data: dict, timeout: float = 2400) -> dict:
async with httpx.AsyncClient() as client:
r = await client.post(f"{COMFY_URL}{path}", json=data, timeout=timeout)
if r.status_code >= 400:
try:
detail = r.json()
except Exception:
detail = r.text
raise HTTPException(status_code=r.status_code, detail=detail)
return r.json()
async def _comfy_get(path: str, timeout: float = 30) -> dict:
async with httpx.AsyncClient() as client:
r = await client.get(f"{COMFY_URL}{path}", timeout=timeout)
r.raise_for_status()
return r.json()
async def _generate_image(model: str, prompt: str, width: int, height: int, seed: int,
steps: int, cfg: float, negative_prompt: str = "", style: str = "") -> bytes:
workflow = _build_workflow(model, prompt, width, height, seed, steps, cfg, negative_prompt, style)
resp = await _comfy_post("/prompt", {"prompt": workflow})
prompt_id = resp.get("prompt_id")
if not prompt_id:
raise HTTPException(status_code=500, detail="No prompt_id from ComfyUI")
# Poll for completion - up to 30 min (180 iterations x 10s)
for i in range(180):
await asyncio.sleep(10)
try:
hist = await _comfy_get(f"/history/{prompt_id}", timeout=10)
except Exception:
continue
if prompt_id not in hist:
continue
d = hist[prompt_id]
status = d.get("status", {})
status_str = status.get("status_str", "?")
if status_str == "error":
errors = []
for node_id, msgs in status.get("messages", {}).items():
for level, msg, detail, _ in msgs:
if level == "error":
errors.append(f"[{node_id}] {msg}: {detail}")
detail = d.get("outputs", {}).get("error", "") or "; ".join(errors) or "ComfyUI execution error"
raise HTTPException(status_code=500, detail=detail)
if status_str == "success":
outputs = d.get("outputs", {})
for node_id, node_out in outputs.items():
for img in node_out.get("images", []):
fname = img.get("filename")
subfolder = img.get("subfolder", "")
ftype = img.get("type", "output")
url = f"{COMFY_URL}/view?filename={fname}&subfolder={subfolder}&type={ftype}"
async with httpx.AsyncClient() as client:
img_resp = await client.get(url, timeout=60)
img_resp.raise_for_status()
return img_resp.content
raise HTTPException(status_code=500, detail="No image in ComfyUI outputs")
raise HTTPException(status_code=504, detail="ComfyUI generation timeout")
app = FastAPI(title="ComfyUI Image Gen OpenAI API", version="2.1.0")
class ImageGenerationRequest(BaseModel):
model: str = Field(default="ideogram4", description="Model: ideogram4, flux2")
prompt: str = Field(..., description="Text prompt")
n: int = Field(default=1, ge=1, le=4, description="Number of images")
size: str = Field(default="1024x1024")
response_format: str = Field(default="url", pattern="^(url|b64_json)$")
seed: Optional[int] = Field(default=None, description="Random seed (int). Random if omitted.")
steps: Optional[int] = Field(default=None, ge=1, le=100, description="Denoising steps. Overrides quality.")
cfg: Optional[float] = Field(default=None, ge=0.0, le=100.0, description="CFG / guidance scale")
quality: Optional[str] = Field(default="medium", description="low|medium|high|ultra -> maps to steps")
negative_prompt: Optional[str] = Field(default="", description="Negative prompt")
style: Optional[str] = Field(default="", description="Style modifier (model-dependent)")
user: Optional[str] = Field(default=None, description="OpenAI user field (logged, not used)")
class ImageData(BaseModel):
url: Optional[str] = None
b64_json: Optional[str] = None
revised_prompt: Optional[str] = None
class ImageGenerationResponse(BaseModel):
created: int
data: list[ImageData]
@app.middleware("http")
async def auth_middleware(request: Request, call_next):
if API_KEY and request.url.path.startswith("/v1"):
auth = request.headers.get("authorization", "")
if not auth.startswith("Bearer ") or auth[7:] != API_KEY:
return JSONResponse(status_code=401, content={"error": "Unauthorized"})
return await call_next(request)
@app.post("/v1/images/generations", response_model=ImageGenerationResponse)
async def create_image_generation(req: ImageGenerationRequest):
if req.model not in ALLOWED_MODELS:
raise HTTPException(
status_code=400,
detail=f"Invalid model: {req.model}. Allowed: {ALLOWED_MODELS}"
)
validated_size = _validate_model_size(req.model, req.size)
if validated_size not in ALLOWED_SIZES:
raise HTTPException(status_code=400, detail=f"Invalid size: {validated_size}. Allowed: {ALLOWED_SIZES}")
width, height = _parse_size(validated_size)
created = int(time.time())
data = []
if req.steps is not None:
steps = req.steps
elif req.quality in QUALITY_STEPS:
steps = QUALITY_STEPS[req.quality]
else:
steps = QUALITY_STEPS["medium"]
# Ideogram4: enforce minimum ~12 steps to avoid blank/gray output
if req.model == "ideogram4" and steps < 12:
steps = 12
if req.model == "ideogram4":
cfg = req.cfg if req.cfg is not None else 1.0
elif req.model.startswith("flux"):
cfg = req.cfg if req.cfg is not None else 3.5
else:
cfg = req.cfg if req.cfg is not None else 7.5
for i in range(req.n):
seed = req.seed + i if req.seed is not None else int(time.time() * 1000) + i
image_bytes = await _generate_image(
req.model, req.prompt, width, height, seed,
steps, cfg, req.negative_prompt or "", req.style or ""
)
if req.response_format == "b64_json":
b64 = base64.b64encode(image_bytes).decode("utf-8")
data.append(ImageData(b64_json=b64, revised_prompt=req.prompt))
else:
b64 = base64.b64encode(image_bytes).decode("utf-8")
data_url = f"data:image/png;base64,{b64}"
data.append(ImageData(url=data_url, revised_prompt=req.prompt))
return ImageGenerationResponse(created=created, data=data)
@app.get("/v1/models")
async def list_models():
"""List available image models - dynamically from ALLOWED_MODELS."""
return {
"object": "list",
"data": [
{"id": m, "object": "model", "owned_by": "comfyui", "permission": []}
for m in sorted(ALLOWED_MODELS)
]
}
@app.get("/health")
async def health():
try:
await _comfy_get("/system_stats", timeout=5)
comfy_status = "ok"
except Exception as e:
comfy_status = f"error: {e}"
return {
"status": "ok",
"comfyui": COMFY_URL,
"comfyui_status": comfy_status,
"models": list(ALLOWED_MODELS)
}
if __name__ == "__main__":
import uvicorn
port = int(os.environ.get("PORT", "8000"))
uvicorn.run(app, host="0.0.0.0", port=port)
creative/comfyui/templates/ideogram4-api.service
+18
View File
@@ -0,0 +1,18 @@
[Unit]
Description=Ideogram4 OpenAI API Wrapper
After=network.target comfyui.service
Requires=comfyui.service
[Service]
Type=simple
User=root
WorkingDirectory=/opt/ideogram4-api
ExecStart=/opt/ideogram4-api/venv/bin/python app.py
Restart=always
RestartSec=5
Environment=PATH=/opt/ideogram4-api/venv/bin:/opt/rocm/bin:/usr/local/bin:/usr/bin:/bin
Environment=COMFYUI_HOST=localhost
Environment=COMFYUI_PORT=8188
[Install]
WantedBy=multi-user.target
creative/comfyui/tests/README.md
+50
View File
@@ -0,0 +1,50 @@
# ComfyUI Skill Tests
Pytest suite covering the skill's scripts. Pure-stdlib unit tests run
without any setup; cloud integration tests need a Comfy Cloud API key.
## Running
```bash
# Unit tests only (no network required) — runs in <1s
python3 -m pytest tests/ -c tests/pytest.ini -o addopts="-p no:xdist"
# Including cloud integration tests
COMFY_CLOUD_API_KEY="comfyui-..." python3 -m pytest tests/ \
-c tests/pytest.ini -o addopts="-p no:xdist"
# Just cloud tests
COMFY_CLOUD_API_KEY="comfyui-..." python3 -m pytest tests/test_cloud_integration.py \
-c tests/pytest.ini -o addopts="-p no:xdist" -v
```
The `-c` and `-o` overrides isolate this suite from any parent
`pyproject.toml` pytest config (e.g. the `-n auto` from a parent repo).
## Test files
| File | Coverage |
|------|----------|
| `test_common.py` | Cloud detection, URL routing, format validation, embeddings, paths, seeds, model-list parsing, folder aliases |
| `test_extract_schema.py` | Connection tracing, positive/negative prompt detection, dedup logic, embedding deps |
| `test_run_workflow.py` | Param injection (incl. -1 seed, link refusal), output download walk, runner construction |
| `test_check_deps.py` | Model-name fuzzy matching, install command suggestions |
| `test_cloud_integration.py` | Live cloud API contract tests (auto-skipped without API key) |
## Adding tests
When you change a script:
1. Add a unit test if the change is pure logic (cloud detection, parsing, etc.)
2. Add a cloud integration test if the change depends on cloud API behavior
(use `pytestmark = pytest.mark.cloud` so it auto-skips without a key)
3. Workflow fixtures live in `conftest.py` (`sd15_workflow`, `flux_workflow`,
`video_workflow`)
## Why the explicit `-c` / `-o`?
The parent hermes-agent repo's `pyproject.toml` enables `pytest-xdist` by
default (`-n auto`). This suite is small enough that parallelism isn't
worth the complexity, and pytest-xdist isn't always installed in the user's
environment. The `-c tests/pytest.ini -o addopts="-p no:xdist"` flags make
the suite run identically regardless of the parent project's config.
creative/comfyui/tests/conftest.py
+64
View File
@@ -0,0 +1,64 @@
"""Pytest configuration for the comfyui skill test suite.
Adds `scripts/` to sys.path so tests can `from _common import ...`, and
provides a few common fixtures.
"""
from __future__ import annotations
import json
import os
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parent.parent
SCRIPTS = ROOT / "scripts"
WORKFLOWS = ROOT / "workflows"
sys.path.insert(0, str(SCRIPTS))
@pytest.fixture
def sd15_workflow() -> dict:
return json.loads((WORKFLOWS / "sd15_txt2img.json").read_text())
@pytest.fixture
def flux_workflow() -> dict:
return json.loads((WORKFLOWS / "flux_dev_txt2img.json").read_text())
@pytest.fixture
def video_workflow() -> dict:
return json.loads((WORKFLOWS / "wan_video_t2v.json").read_text())
@pytest.fixture
def workflows_dir() -> Path:
return WORKFLOWS
@pytest.fixture
def scripts_dir() -> Path:
return SCRIPTS
@pytest.fixture
def cloud_key() -> str | None:
"""Cloud API key if set, otherwise None.
Tests that need cloud connectivity should skip when this is None.
"""
return os.environ.get("COMFY_CLOUD_API_KEY")
def pytest_collection_modifyitems(config, items):
"""Auto-skip cloud tests when no API key is set."""
if os.environ.get("COMFY_CLOUD_API_KEY"):
return
skip_cloud = pytest.mark.skip(reason="Set COMFY_CLOUD_API_KEY to run cloud tests")
for item in items:
if "cloud" in item.keywords:
item.add_marker(skip_cloud)
creative/comfyui/tests/pytest.ini
+5
View File
@@ -0,0 +1,5 @@
[pytest]
markers =
cloud: tests that hit live Comfy Cloud API (require COMFY_CLOUD_API_KEY)
testpaths = .
addopts = -p no:xdist
creative/comfyui/tests/test_check_deps.py
+68
View File
@@ -0,0 +1,68 @@
"""Tests for check_deps.py — focuses on parsing logic that doesn't need a server."""
from __future__ import annotations
from check_deps import (
NODE_TO_PACKAGE,
model_present,
normalize_for_match,
suggest_install_command,
)
class TestNormalizeForMatch:
def test_basic(self):
s = normalize_for_match("model.safetensors")
assert "model.safetensors" in s
assert "model" in s
def test_subfolder(self):
s = normalize_for_match("subdir/model.pt")
assert "subdir/model.pt" in s
assert "model.pt" in s
assert "model" in s
class TestModelPresent:
def test_exact_match(self):
assert model_present("a.safetensors", {"a.safetensors", "b.safetensors"}) is True
def test_extension_difference(self):
# User said "model" but installed is "model.safetensors"
assert model_present("model", {"model.safetensors"}) is True
# Reverse direction — also matches
assert model_present("model.safetensors", {"model"}) is True
def test_subfolder_match(self):
# Installed list has "subdir/model.safetensors", workflow asks "model.safetensors"
assert model_present("model.safetensors", {"subdir/model.safetensors"}) is True
def test_missing(self):
assert model_present("missing.safetensors", {"a.safetensors", "b.safetensors"}) is False
def test_empty_installed(self):
assert model_present("anything.safetensors", set()) is False
class TestSuggestInstallCommand:
def test_known_node(self):
cmd = suggest_install_command("VHS_VideoCombine")
assert cmd == "comfy node install comfyui-videohelpersuite"
def test_unknown_node(self):
assert suggest_install_command("SomeRandomNodeName123") is None
class TestNodePackageMap:
def test_no_duplicates(self):
# Each node should map to exactly one package
keys = list(NODE_TO_PACKAGE.keys())
assert len(keys) == len(set(keys))
def test_packages_are_safe_for_shell(self):
# Registry slugs must be alphanumerics + hyphens/underscores only
# (passed straight to `comfy node install <pkg>`).
import re
safe = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._\-]*$")
for pkg in NODE_TO_PACKAGE.values():
assert safe.match(pkg), f"Unsafe package slug: {pkg!r}"
creative/comfyui/tests/test_cloud_integration.py
+95
View File
@@ -0,0 +1,95 @@
"""Integration tests against the live Comfy Cloud API.
These tests are auto-skipped when COMFY_CLOUD_API_KEY is not set.
They never SUBMIT workflows (would need a paid subscription) — they only
verify the read-only endpoints we rely on.
"""
from __future__ import annotations
import pytest
from _common import http_get, parse_model_list, resolve_url
pytestmark = pytest.mark.cloud
class TestCloudEndpointsLive:
def test_system_stats_reachable(self, cloud_key):
url = resolve_url("https://cloud.comfy.org", "/system_stats")
r = http_get(url, headers={"X-API-Key": cloud_key})
assert r.status == 200
data = r.json()
assert "system" in data
def test_models_endpoint_routed_to_experiment(self, cloud_key):
# We expect the skill to route /models/checkpoints → /api/experiment/models/checkpoints
url = resolve_url("https://cloud.comfy.org", "/models/checkpoints")
assert "/api/experiment/models/checkpoints" in url
r = http_get(url, headers={"X-API-Key": cloud_key})
assert r.status == 200
def test_models_endpoint_returns_dicts(self, cloud_key):
url = resolve_url("https://cloud.comfy.org", "/models/checkpoints")
r = http_get(url, headers={"X-API-Key": cloud_key})
data = r.json()
assert isinstance(data, list)
if data:
# Cloud format: list of dicts with `name`
assert isinstance(data[0], dict)
assert "name" in data[0]
# Our parser normalizes both
normalized = parse_model_list(data)
assert len(normalized) == len(data)
def test_history_renamed_to_v2(self, cloud_key):
# /history → /api/history_v2 on cloud
url = resolve_url("https://cloud.comfy.org", "/history/some-fake-id")
assert "/api/history_v2/some-fake-id" in url
def test_object_info_paid_tier(self, cloud_key):
# On free tier, /object_info returns 403 with a recognizable message
url = resolve_url("https://cloud.comfy.org", "/object_info")
r = http_get(url, headers={"X-API-Key": cloud_key})
# Should be either 200 (paid) or 403 (free) — not 404 / 500
assert r.status in (200, 403)
if r.status == 403:
# Body should mention the limitation
assert "free tier" in r.text().lower() or "subscription" in r.text().lower()
class TestCloudCheckDepsLive:
def test_check_deps_against_cloud(self, cloud_key, sd15_workflow):
from check_deps import check_deps
report = check_deps(sd15_workflow, host="https://cloud.comfy.org", api_key=cloud_key)
# Either node check passed OR was skipped (free tier)
assert "missing_models" in report
assert "is_cloud" in report and report["is_cloud"] is True
def test_flux_workflow_models_resolved_via_aliases(self, cloud_key, flux_workflow):
"""Flux uses unet/clip folders; cloud has them in diffusion_models/text_encoders.
With folder aliasing, the check should still find them."""
from check_deps import check_deps
report = check_deps(flux_workflow, host="https://cloud.comfy.org", api_key=cloud_key)
# The exact required Flux files (flux1-dev.safetensors, t5xxl_fp16, clip_l, ae)
# are present on cloud; with folder aliasing, none should be missing.
# If this fails, either the cloud removed the model or the aliasing logic broke.
missing_filenames = {m["value"] for m in report["missing_models"]}
assert "ae.safetensors" not in missing_filenames, \
"ae.safetensors should be on cloud's vae folder"
# t5xxl_fp16 / clip_l should be reachable via the clip → text_encoders alias
# flux1-dev.safetensors likewise via unet → diffusion_models
class TestHealthCheckLive:
def test_health_check_passes(self, cloud_key, capsys):
from health_check import main as health_main
rc = health_main(["--host", "https://cloud.comfy.org", "--api-key", cloud_key])
captured = capsys.readouterr()
# Should produce JSON
import json
report = json.loads(captured.out)
assert report["server"]["reachable"] is True
assert report["checkpoints"]["queryable"] is True
assert report["checkpoints"]["count"] > 0
creative/comfyui/tests/test_common.py
+447
View File
@@ -0,0 +1,447 @@
"""Unit tests for _common.py — pure logic only, no network."""
from __future__ import annotations
from pathlib import Path
import pytest
from _common import (
DEFAULT_LOCAL_HOST,
EMBEDDING_REGEX,
FOLDER_ALIASES,
build_cloud_aware_url,
cloud_endpoint,
coerce_seed,
folder_aliases_for,
is_api_format,
is_cloud_host,
is_link,
iter_embedding_refs,
iter_model_deps,
iter_nodes,
looks_like_video_workflow,
media_type_from_filename,
parse_model_list,
resolve_url,
safe_path_join,
unwrap_workflow,
)
# =============================================================================
# Cloud detection / URL routing
# =============================================================================
class TestCloudDetection:
def test_cloud_host_exact(self):
assert is_cloud_host("https://cloud.comfy.org") is True
assert is_cloud_host("https://cloud.comfy.org/foo/bar") is True
def test_cloud_host_subdomain(self):
assert is_cloud_host("https://staging.cloud.comfy.org") is True
assert is_cloud_host("https://api.cloud.comfy.org") is True
def test_local_not_cloud(self):
assert is_cloud_host("http://127.0.0.1:8188") is False
assert is_cloud_host("http://localhost:8188") is False
assert is_cloud_host("http://my-server.local:8188") is False
def test_no_scheme(self):
# Defaults to http://
assert is_cloud_host("cloud.comfy.org") is True
assert is_cloud_host("127.0.0.1:8188") is False
class TestCloudEndpointRename:
def test_history_renamed(self):
assert cloud_endpoint("/history") == "/history_v2"
assert cloud_endpoint("/history/abc-123") == "/history_v2/abc-123"
def test_history_v2_preserved(self):
assert cloud_endpoint("/history_v2") == "/history_v2"
def test_models_renamed(self):
assert cloud_endpoint("/models") == "/experiment/models"
assert cloud_endpoint("/models/checkpoints") == "/experiment/models/checkpoints"
assert cloud_endpoint("/models/loras") == "/experiment/models/loras"
def test_other_paths_unchanged(self):
assert cloud_endpoint("/prompt") == "/prompt"
assert cloud_endpoint("/queue") == "/queue"
class TestResolveURL:
def test_local_no_prefix(self):
assert resolve_url("http://127.0.0.1:8188", "/prompt") == "http://127.0.0.1:8188/prompt"
def test_cloud_adds_api_prefix(self):
assert resolve_url("https://cloud.comfy.org", "/prompt") == "https://cloud.comfy.org/api/prompt"
def test_cloud_history_renamed(self):
assert resolve_url("https://cloud.comfy.org", "/history/abc") == "https://cloud.comfy.org/api/history_v2/abc"
def test_cloud_models_renamed(self):
assert resolve_url("https://cloud.comfy.org", "/models/loras") == "https://cloud.comfy.org/api/experiment/models/loras"
def test_cloud_already_has_api(self):
# Don't double-prefix
assert resolve_url("https://cloud.comfy.org", "/api/prompt") == "https://cloud.comfy.org/api/prompt"
def test_trailing_slash_stripped(self):
assert resolve_url("http://127.0.0.1:8188/", "/prompt") == "http://127.0.0.1:8188/prompt"
# =============================================================================
# Workflow validation
# =============================================================================
class TestAPIFormatDetection:
def test_valid_api(self, sd15_workflow):
assert is_api_format(sd15_workflow) is True
def test_editor_format_rejected(self):
editor = {"nodes": [], "links": [], "version": 0.4}
assert is_api_format(editor) is False
def test_empty_dict(self):
assert is_api_format({}) is False
def test_non_dict(self):
assert is_api_format([]) is False
assert is_api_format(None) is False
assert is_api_format("string") is False
def test_node_with_class_type(self):
wf = {"3": {"class_type": "KSampler", "inputs": {}}}
assert is_api_format(wf) is True
class TestUnwrapWorkflow:
def test_passthrough_api_format(self, sd15_workflow):
result = unwrap_workflow(sd15_workflow)
assert result is sd15_workflow
def test_unwrap_prompt_key(self, sd15_workflow):
wrapped = {"prompt": sd15_workflow, "client_id": "abc"}
result = unwrap_workflow(wrapped)
assert result is sd15_workflow
def test_editor_format_raises(self):
with pytest.raises(ValueError, match="editor format"):
unwrap_workflow({"nodes": [], "links": []})
def test_garbage_raises(self):
with pytest.raises(ValueError):
unwrap_workflow({"foo": "bar"})
class TestIsLink:
def test_valid_link(self):
assert is_link(["3", 0]) is True
assert is_link(["10", 1]) is True
def test_non_link(self):
assert is_link("string") is False
assert is_link(42) is False
assert is_link([]) is False
assert is_link(["3"]) is False # missing slot
assert is_link(["3", "0"]) is False # slot must be int
assert is_link([3, 0]) is False # node_id must be string
# =============================================================================
# Workflow iterators
# =============================================================================
class TestIterators:
def test_iter_nodes(self, sd15_workflow):
nodes = dict(iter_nodes(sd15_workflow))
assert "3" in nodes
assert nodes["3"]["class_type"] == "KSampler"
def test_iter_nodes_skips_comments(self, sd15_workflow):
# _comment is not a node
nodes = dict(iter_nodes(sd15_workflow))
assert "_comment" not in nodes
def test_iter_model_deps(self, sd15_workflow):
deps = list(iter_model_deps(sd15_workflow))
names = [d["value"] for d in deps]
assert "v1-5-pruned-emaonly.safetensors" in names
def test_iter_model_deps_flux(self, flux_workflow):
deps = list(iter_model_deps(flux_workflow))
names = {d["value"]: d["folder"] for d in deps}
assert names["flux1-dev.safetensors"] == "unet"
assert names["t5xxl_fp16.safetensors"] == "clip"
assert names["clip_l.safetensors"] == "clip"
assert names["ae.safetensors"] == "vae"
# =============================================================================
# Embedding extraction
# =============================================================================
class TestEmbeddingRegex:
def test_basic_embedding(self):
m = EMBEDDING_REGEX.search("a cat, embedding:goodvibes, more text")
assert m is not None
assert m.group(1) == "goodvibes"
def test_embedding_with_strength(self):
m = EMBEDDING_REGEX.search("embedding:bad-hands-5:1.2")
assert m is not None
assert m.group(1) == "bad-hands-5"
def test_embedding_with_extension(self):
# Strips .pt / .safetensors / .bin
m = EMBEDDING_REGEX.search("embedding:my-emb.pt")
assert m is not None
assert m.group(1) == "my-emb"
def test_embedding_in_parens(self):
m = EMBEDDING_REGEX.search("(embedding:foo:0.8)")
assert m is not None
assert m.group(1) == "foo"
def test_multiple_in_one_string(self):
text = "a cat, embedding:foo:1.2, and embedding:bar"
matches = [m.group(1) for m in EMBEDDING_REGEX.finditer(text)]
assert matches == ["foo", "bar"]
def test_no_false_positive_on_word_embedding(self):
# "embedding " (with space, no colon) should not match
m = EMBEDDING_REGEX.search("the embedding is great")
assert m is None
class TestIterEmbeddingRefs:
def test_finds_in_clip_text_encode(self):
wf = {
"1": {"class_type": "CLIPTextEncode",
"inputs": {"text": "embedding:foo, embedding:bar:0.5", "clip": ["2", 0]}},
"2": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}},
}
refs = list(iter_embedding_refs(wf))
names = [name for _, name in refs]
assert names == ["foo", "bar"]
def test_ignores_non_prompt_fields(self):
wf = {
"1": {"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "embedding:foo.safetensors"}},
}
refs = list(iter_embedding_refs(wf))
# ckpt_name is not a prompt field — ignored
assert refs == []
# =============================================================================
# Path safety
# =============================================================================
class TestSafePathJoin:
def test_normal_join(self, tmp_path):
p = safe_path_join(tmp_path, "subdir", "file.png")
assert p.is_relative_to(tmp_path)
def test_blocks_traversal(self, tmp_path):
with pytest.raises(ValueError, match="path traversal"):
safe_path_join(tmp_path, "..", "..", "etc", "passwd")
def test_blocks_absolute(self, tmp_path):
with pytest.raises(ValueError):
safe_path_join(tmp_path, "/etc/passwd")
def test_subfolder_with_filename(self, tmp_path):
p = safe_path_join(tmp_path, "outputs", "img.png")
assert p.name == "img.png"
assert p.parent.name == "outputs"
# =============================================================================
# Seed coercion
# =============================================================================
class TestCoerceSeed:
def test_explicit_int(self):
assert coerce_seed(42) == 42
assert coerce_seed(0) == 0
def test_minus_one_randomizes(self):
s = coerce_seed(-1)
assert isinstance(s, int)
assert 0 <= s < 2**63
def test_none_randomizes(self):
s = coerce_seed(None)
assert isinstance(s, int)
def test_string_int(self):
# str() that converts cleanly is allowed (relaxed)
assert coerce_seed("12345") == 12345
def test_string_minus_one_randomizes(self):
# CLI / JSON sometimes carries seed as a string.
s = coerce_seed("-1")
assert isinstance(s, int)
assert 0 <= s < 2**63
# And whitespace tolerated
s2 = coerce_seed(" -1 ")
assert isinstance(s2, int)
assert 0 <= s2 < 2**63
# =============================================================================
# Model list normalization (cloud format)
# =============================================================================
class TestParseModelList:
def test_local_format_strings(self):
result = parse_model_list(["a.safetensors", "b.safetensors"])
assert result == {"a.safetensors", "b.safetensors"}
def test_cloud_format_dicts(self):
result = parse_model_list([
{"name": "a.safetensors", "pathIndex": 0},
{"name": "b.safetensors", "pathIndex": 1},
])
assert result == {"a.safetensors", "b.safetensors"}
def test_empty(self):
assert parse_model_list([]) == set()
def test_garbage(self):
assert parse_model_list("not a list") == set()
assert parse_model_list(None) == set()
def test_mixed_format(self):
result = parse_model_list([
"string-form.safetensors",
{"name": "dict-form.safetensors"},
])
assert result == {"string-form.safetensors", "dict-form.safetensors"}
# =============================================================================
# Folder aliases
# =============================================================================
class TestFolderAliases:
def test_unet_aliases_diffusion_models(self):
aliases = folder_aliases_for("unet")
assert "unet" in aliases
assert "diffusion_models" in aliases
def test_clip_aliases_text_encoders(self):
aliases = folder_aliases_for("clip")
assert "clip" in aliases
assert "text_encoders" in aliases
def test_unknown_folder_returns_self(self):
assert folder_aliases_for("checkpoints") == ["checkpoints"]
def test_primary_first(self):
# Order matters: primary should be first for human-friendly fix hints
assert folder_aliases_for("unet")[0] == "unet"
assert folder_aliases_for("diffusion_models")[0] == "diffusion_models"
# =============================================================================
# Media-type detection
# =============================================================================
class TestMediaType:
def test_video_extensions(self):
assert media_type_from_filename("vid.mp4") == "video"
assert media_type_from_filename("foo.webm") == "video"
assert media_type_from_filename("bar.gif") == "video"
def test_audio_extensions(self):
assert media_type_from_filename("song.wav") == "audio"
assert media_type_from_filename("music.mp3") == "audio"
def test_image_default(self):
assert media_type_from_filename("pic.png") == "image"
assert media_type_from_filename("image.jpg") == "image"
assert media_type_from_filename("unknown.xyz") == "image"
def test_3d(self):
assert media_type_from_filename("model.glb") == "3d"
assert media_type_from_filename("scene.gltf") == "3d"
# =============================================================================
# Cross-host header stripping (security)
# =============================================================================
class TestRedirectHeaderStripping:
"""Verify X-API-Key is dropped when redirect crosses to a different host
(e.g. cloud /api/view → S3 signed URL). Critical to prevent leaking auth
tokens to the storage backend.
"""
def _build_session(self):
from _common import _StripSensitiveOnRedirectSession, HAS_REQUESTS
if not HAS_REQUESTS:
import pytest
pytest.skip("requests not installed")
return _StripSensitiveOnRedirectSession()
def test_strips_x_api_key_cross_host(self):
import requests
s = self._build_session()
prep = requests.PreparedRequest()
prep.prepare(method="GET", url="https://other.example.com/file",
headers={"X-API-Key": "leak", "Authorization": "Bearer x"})
resp = requests.Response()
orig = requests.PreparedRequest()
orig.prepare(method="GET", url="https://cloud.comfy.org/api/view", headers={})
resp.request = orig
s.rebuild_auth(prep, resp)
assert "X-API-Key" not in prep.headers
assert "Authorization" not in prep.headers
def test_preserves_x_api_key_same_host(self):
import requests
s = self._build_session()
prep = requests.PreparedRequest()
prep.prepare(method="GET", url="https://cloud.comfy.org/foo",
headers={"X-API-Key": "keep"})
resp = requests.Response()
orig = requests.PreparedRequest()
orig.prepare(method="GET", url="https://cloud.comfy.org/bar", headers={})
resp.request = orig
s.rebuild_auth(prep, resp)
assert prep.headers.get("X-API-Key") == "keep"
def test_strips_cookie_cross_host(self):
import requests
s = self._build_session()
prep = requests.PreparedRequest()
prep.prepare(method="GET", url="https://other.example.com/x",
headers={"Cookie": "session=secret"})
resp = requests.Response()
orig = requests.PreparedRequest()
orig.prepare(method="GET", url="https://cloud.comfy.org/foo", headers={})
resp.request = orig
s.rebuild_auth(prep, resp)
assert "Cookie" not in prep.headers
# =============================================================================
# Video workflow detection
# =============================================================================
class TestVideoWorkflow:
def test_image_workflow(self, sd15_workflow):
assert looks_like_video_workflow(sd15_workflow) is False
def test_animatediff_workflow(self, workflows_dir):
import json
wf = json.loads((workflows_dir / "animatediff_video.json").read_text())
assert looks_like_video_workflow(wf) is True
def test_wan_workflow(self, video_workflow):
assert looks_like_video_workflow(video_workflow) is True
creative/comfyui/tests/test_extract_schema.py
+185
View File
@@ -0,0 +1,185 @@
"""Tests for extract_schema.py."""
from __future__ import annotations
import pytest
from extract_schema import (
extract_schema,
find_negative_prompt_node,
find_positive_prompt_node,
trace_to_node,
)
# =============================================================================
# Connection tracing
# =============================================================================
class TestConnectionTracing:
def test_direct_link(self):
wf = {
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": "x"}},
"2": {"class_type": "KSampler",
"inputs": {"positive": ["1", 0], "negative": ["1", 0]}},
}
assert trace_to_node(wf, ["1", 0]) == "1"
def test_through_reroute(self):
wf = {
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": "x"}},
"2": {"class_type": "Reroute", "inputs": {"input": ["1", 0]}},
"3": {"class_type": "Reroute", "inputs": {"input": ["2", 0]}},
}
assert trace_to_node(wf, ["3", 0]) == "1"
def test_circular_safe(self):
wf = {
"1": {"class_type": "Reroute", "inputs": {"input": ["2", 0]}},
"2": {"class_type": "Reroute", "inputs": {"input": ["1", 0]}},
}
# Should hit max_hops without infinite loop
result = trace_to_node(wf, ["1", 0], max_hops=5)
assert result in ("1", "2") # any node, just don't hang
class TestPositiveNegativeDetection:
def test_basic(self, sd15_workflow):
# In sd15_workflow.json node 6 is positive, node 7 is negative
assert find_positive_prompt_node(sd15_workflow) == "6"
assert find_negative_prompt_node(sd15_workflow) == "7"
def test_swapped_order(self):
wf = {
"3": {"class_type": "KSampler",
"inputs": {
"positive": ["7", 0], "negative": ["6", 0],
"model": ["4", 0], "latent_image": ["5", 0],
"seed": 1, "steps": 20, "cfg": 7.5,
"sampler_name": "euler", "scheduler": "normal", "denoise": 1.0,
}},
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}},
"5": {"class_type": "EmptyLatentImage", "inputs": {"width": 512, "height": 512, "batch_size": 1}},
"6": {"class_type": "CLIPTextEncode", "inputs": {"text": "ugly", "clip": ["4", 1]}},
"7": {"class_type": "CLIPTextEncode", "inputs": {"text": "beautiful", "clip": ["4", 1]}},
}
# Now 7 is the positive (despite higher node ID)
assert find_positive_prompt_node(wf) == "7"
assert find_negative_prompt_node(wf) == "6"
# =============================================================================
# Schema extraction
# =============================================================================
class TestExtractSchema:
def test_basic_sd15(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
params = schema["parameters"]
assert "prompt" in params
assert "negative_prompt" in params
assert "seed" in params
assert "steps" in params
assert "cfg" in params
assert "width" in params
assert "height" in params
def test_prompt_value_correct(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
# The positive prompt in the example is the landscape one
assert "landscape" in schema["parameters"]["prompt"]["value"]
assert "ugly" in schema["parameters"]["negative_prompt"]["value"]
def test_model_dependencies(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
deps = schema["model_dependencies"]
ckpts = [d["value"] for d in deps if d["folder"] == "checkpoints"]
assert "v1-5-pruned-emaonly.safetensors" in ckpts
def test_output_nodes(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
assert "9" in schema["output_nodes"]
def test_summary(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
s = schema["summary"]
assert s["has_negative_prompt"] is True
assert s["has_seed"] is True
assert s["is_video_workflow"] is False
assert s["parameter_count"] > 5
def test_flux_workflow(self, flux_workflow):
schema = extract_schema(flux_workflow)
# Flux uses RandomNoise for seed
assert schema["summary"]["has_seed"] is True
# Flux has only positive prompt (no negative encoder)
assert schema["summary"]["has_negative_prompt"] is False
def test_video_detected(self, video_workflow):
schema = extract_schema(video_workflow)
assert schema["summary"]["is_video_workflow"] is True
class TestEmbeddingDeps:
def test_extract_from_prompt(self):
wf = {
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}},
"5": {"class_type": "EmptyLatentImage",
"inputs": {"width": 512, "height": 512, "batch_size": 1}},
"6": {"class_type": "CLIPTextEncode",
"inputs": {
"text": "a cat, embedding:goodvibes, embedding:art:1.2",
"clip": ["1", 1]
}},
"7": {"class_type": "CLIPTextEncode",
"inputs": {
"text": "ugly, embedding:badhands",
"clip": ["1", 1]
}},
"3": {"class_type": "KSampler",
"inputs": {
"positive": ["6", 0], "negative": ["7", 0],
"model": ["1", 0], "latent_image": ["5", 0],
"seed": 1, "steps": 20, "cfg": 7.5,
"sampler_name": "euler", "scheduler": "normal", "denoise": 1.0,
}},
"9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}},
}
schema = extract_schema(wf)
names = [d["embedding_name"] for d in schema["embedding_dependencies"]]
assert sorted(names) == ["art", "badhands", "goodvibes"]
class TestDuplicateDeduplication:
def test_two_ksamplers_get_unique_names(self):
wf = {
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}},
"5": {"class_type": "EmptyLatentImage",
"inputs": {"width": 512, "height": 512, "batch_size": 1}},
"6": {"class_type": "CLIPTextEncode", "inputs": {"text": "a", "clip": ["1", 1]}},
"7": {"class_type": "CLIPTextEncode", "inputs": {"text": "b", "clip": ["1", 1]}},
"3": {"class_type": "KSampler",
"inputs": {
"positive": ["6", 0], "negative": ["7", 0],
"model": ["1", 0], "latent_image": ["5", 0],
"seed": 42, "steps": 20, "cfg": 7.5,
"sampler_name": "euler", "scheduler": "normal", "denoise": 1.0,
}},
"4": {"class_type": "KSampler",
"inputs": {
"positive": ["6", 0], "negative": ["7", 0],
"model": ["1", 0], "latent_image": ["5", 0],
"seed": 99, "steps": 30, "cfg": 8.0,
"sampler_name": "euler", "scheduler": "normal", "denoise": 0.6,
}},
"9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}},
}
schema = extract_schema(wf)
params = schema["parameters"]
# Both seeds present with disambiguated names
seed_keys = [k for k in params if "seed" in k]
# Symmetric: both renamed (no bare "seed")
assert "seed" not in params
assert "seed_3" in params and "seed_4" in params
assert params["seed_3"]["value"] == 42
assert params["seed_4"]["value"] == 99
creative/comfyui/tests/test_run_workflow.py
+213
View File
@@ -0,0 +1,213 @@
"""Tests for run_workflow.py — focuses on logic that doesn't require a server."""
from __future__ import annotations
import copy
import json
import pytest
from extract_schema import extract_schema
from run_workflow import (
ComfyRunner,
download_outputs,
inject_params,
parse_input_image_arg,
)
class TestParseInputImageArg:
def test_with_name(self, tmp_path):
f = tmp_path / "x.png"
f.write_text("x")
n, p = parse_input_image_arg(f"image={f}")
assert n == "image"
assert p == f
def test_without_name_defaults(self, tmp_path):
f = tmp_path / "x.png"
f.write_text("x")
n, p = parse_input_image_arg(str(f))
assert n == "image"
def test_custom_name(self, tmp_path):
f = tmp_path / "x.png"
f.write_text("x")
n, p = parse_input_image_arg(f"mask_image={f}")
assert n == "mask_image"
class TestInjectParams:
def test_basic_injection(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
wf, warnings = inject_params(sd15_workflow, schema, {
"prompt": "new prompt",
"seed": 999,
"steps": 25,
})
assert wf["6"]["inputs"]["text"] == "new prompt"
assert wf["3"]["inputs"]["seed"] == 999
assert wf["3"]["inputs"]["steps"] == 25
assert warnings == []
def test_unknown_param_warns(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
_, warnings = inject_params(sd15_workflow, schema, {"foobar": "x"})
assert any("foobar" in w for w in warnings)
def test_seed_minus_one_randomizes(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
wf, warnings = inject_params(sd15_workflow, schema, {"seed": -1})
assert wf["3"]["inputs"]["seed"] != -1
assert isinstance(wf["3"]["inputs"]["seed"], int)
assert any("expanded" in w.lower() for w in warnings)
def test_randomize_seed_when_unset(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
original = sd15_workflow["3"]["inputs"]["seed"]
wf, warnings = inject_params(sd15_workflow, schema, {}, randomize_seed_if_unset=True)
assert wf["3"]["inputs"]["seed"] != original
assert isinstance(wf["3"]["inputs"]["seed"], int)
def test_does_not_mutate_original(self, sd15_workflow):
schema = extract_schema(sd15_workflow)
original_text = sd15_workflow["6"]["inputs"]["text"]
inject_params(sd15_workflow, schema, {"prompt": "MUTATED"})
assert sd15_workflow["6"]["inputs"]["text"] == original_text
def test_refuses_to_overwrite_link(self):
wf = {
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}},
"5": {"class_type": "EmptyLatentImage",
"inputs": {"width": 512, "height": 512, "batch_size": 1}},
"6": {"class_type": "CLIPTextEncode",
"inputs": {"text": ["3", 0], "clip": ["1", 1]}}, # text is a link!
"3": {"class_type": "KSampler",
"inputs": {"seed": 1, "steps": 20, "cfg": 7.5,
"sampler_name": "euler", "scheduler": "normal", "denoise": 1.0,
"model": ["1", 0], "positive": ["6", 0], "negative": ["6", 0],
"latent_image": ["5", 0]}},
"9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}},
}
# Manually create a schema that has prompt pointing at 6.text
schema = {
"parameters": {
"prompt": {"node_id": "6", "field": "text", "type": "string", "value": ""},
}
}
wf2, warnings = inject_params(wf, schema, {"prompt": "literal value"})
# The link should NOT have been overwritten
assert wf2["6"]["inputs"]["text"] == ["3", 0]
assert any("link" in w.lower() for w in warnings)
# =============================================================================
# Output download walk
# =============================================================================
class TestDownloadOutputsWalk:
"""Test that download_outputs walks the structure correctly."""
def test_handles_videos_plural(self, tmp_path, monkeypatch):
"""Local ComfyUI uses 'videos'/'gifs' (plural) keys."""
downloads = []
class FakeRunner:
def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite):
downloads.append((filename, subfolder, file_type))
p = output_dir / filename
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(b"x")
return p
outputs = {
"9": {"images": [{"filename": "img1.png", "subfolder": "", "type": "output"}]},
"10": {"videos": [{"filename": "vid1.mp4", "subfolder": "", "type": "output"}]},
"11": {"gifs": [{"filename": "anim1.gif", "subfolder": "", "type": "output"}]},
}
result = download_outputs(FakeRunner(), outputs, tmp_path)
files = sorted(d["filename"] for d in result)
assert files == ["anim1.gif", "img1.png", "vid1.mp4"]
def test_handles_video_singular_cloud(self, tmp_path):
"""Cloud uses 'video' (singular)."""
class FakeRunner:
def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite):
p = output_dir / filename
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(b"x")
return p
outputs = {
"10": {"video": [{"filename": "cloud.mp4", "subfolder": "", "type": "output"}]},
}
result = download_outputs(FakeRunner(), outputs, tmp_path)
assert len(result) == 1
assert result[0]["filename"] == "cloud.mp4"
def test_preserves_subfolder(self, tmp_path):
"""When preserve_subfolder=True, server subfolder becomes local subdir."""
class FakeRunner:
def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite):
if preserve_subfolder and subfolder:
p = output_dir / subfolder / filename
else:
p = output_dir / filename
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(b"x")
return p
outputs = {
"9": {"images": [
{"filename": "img.png", "subfolder": "myrun", "type": "output"},
{"filename": "img.png", "subfolder": "otherrun", "type": "output"},
]},
}
result = download_outputs(FakeRunner(), outputs, tmp_path, preserve_subfolder=True)
files = [d["file"] for d in result]
assert any("myrun" in f for f in files)
assert any("otherrun" in f for f in files)
# Both must exist (no collision)
assert len({str(f) for f in files}) == 2
# =============================================================================
# ComfyRunner construction
# =============================================================================
class TestRunnerConstruction:
def test_local_default(self):
r = ComfyRunner()
assert r.is_cloud is False
assert r.host == "http://127.0.0.1:8188"
def test_cloud_detection(self):
r = ComfyRunner(host="https://cloud.comfy.org", api_key="abc")
assert r.is_cloud is True
assert "X-API-Key" in r.headers
def test_cloud_subdomain_detected(self):
r = ComfyRunner(host="https://staging.cloud.comfy.org", api_key="abc")
assert r.is_cloud is True
def test_partner_key_does_not_pollute_extra_data(self):
r = ComfyRunner(host="https://cloud.comfy.org", api_key="auth-key")
# No partner-key set → no extra_data should appear in submitted prompt
# (This is a static check; runtime check happens in submit())
assert r.partner_key is None
def test_url_routing_local(self):
r = ComfyRunner()
url = r._url("/prompt")
assert url == "http://127.0.0.1:8188/prompt"
def test_url_routing_cloud(self):
r = ComfyRunner(host="https://cloud.comfy.org", api_key="x")
url = r._url("/prompt")
assert url == "https://cloud.comfy.org/api/prompt"
def test_url_routing_cloud_history_renamed(self):
r = ComfyRunner(host="https://cloud.comfy.org", api_key="x")
url = r._url("/history/abc-123")
assert url == "https://cloud.comfy.org/api/history_v2/abc-123"
creative/comfyui/workflows/README.md
+86
View File
@@ -0,0 +1,86 @@
# Example Workflows
These are starter API-format workflows for the most common tasks. They're
ready to run with `scripts/run_workflow.py` once you've installed (or have
cloud access to) the listed models.
| File | Purpose | Required models | Min VRAM |
|------|---------|-----------------|----------|
| `sd15_txt2img.json` | SD 1.5 text-to-image (512×512) | SD1.5 checkpoint, e.g. `v1-5-pruned-emaonly.safetensors` | 4 GB |
| `sdxl_txt2img.json` | SDXL text-to-image (1024×1024) | `sd_xl_base_1.0.safetensors` | 8 GB |
| `flux_dev_txt2img.json` | Flux Dev text-to-image (1024×1024) | `flux1-dev.safetensors`, `t5xxl_fp16.safetensors`, `clip_l.safetensors`, `ae.safetensors` | 24 GB (or use `flux1-dev-fp8`) |
| `sdxl_img2img.json` | SDXL image-to-image | SDXL checkpoint | 8 GB |
| `sdxl_inpaint.json` | SDXL inpainting (image + mask) | SDXL checkpoint | 8 GB |
| `upscale_4x.json` | Standalone 4× ESRGAN upscale | `4x-UltraSharp.pth` (or any upscaler) | 4 GB |
| `animatediff_video.json` | AnimateDiff text-to-video (16 frames) | SD1.5 checkpoint, `mm_sd_v15_v2.ckpt` motion module | 8 GB |
| `wan_video_t2v.json` | Wan 2.x text-to-video (~33 frames) | `wan2.2_t2v_1.3B_fp16.safetensors`, `umt5_xxl_fp16.safetensors`, `wan_2.1_vae.safetensors` | 24 GB |
## Quick start
```bash
# Run a workflow with prompt injection
python3 ../scripts/run_workflow.py \
--workflow sdxl_txt2img.json \
--args '{"prompt": "majestic eagle in flight", "seed": 12345, "steps": 35}' \
--output-dir ./out
# Img2img: upload an input image first via the script's helper
python3 ../scripts/run_workflow.py \
--workflow sdxl_img2img.json \
--input-image image=./photo.png \
--args '{"prompt": "make it watercolor", "denoise": 0.6}' \
--output-dir ./out
# Cloud (set API key once)
export COMFY_CLOUD_API_KEY="comfyui-..."
python3 ../scripts/run_workflow.py \
--workflow flux_dev_txt2img.json \
--args '{"prompt": "a fox in a misty forest"}' \
--host https://cloud.comfy.org \
--output-dir ./out
# What can I tweak in this workflow?
python3 ../scripts/extract_schema.py sdxl_txt2img.json --summary-only
# Are all required models / nodes installed?
python3 ../scripts/check_deps.py wan_video_t2v.json
```
## Notes
- **Inpaint masks**: white pixels = "regenerate this region", black = preserve.
ComfyUI's `LoadImageMask` reads the **red channel** by default; export your
mask as a single-channel image or as a normal RGB where red==intensity.
- **Denoise strength** in img2img: `0.0` = output identical to input,
`1.0` = ignore input entirely. Sweet spot is usually 0.4–0.7.
- **Flux Dev** needs ~24 GB VRAM in its base form. The `flux1-dev-fp8.safetensors`
variant (already on Comfy Cloud) cuts that roughly in half.
- **Video workflows** can take many minutes. The skill auto-detects video
output nodes and bumps the default timeout to 900s. Override with `--timeout 1800`.
- These JSON files are deliberately **API format** (top-level keys are node IDs
with `class_type`), not editor format. To open them in ComfyUI's web UI for
visual editing, use `Workflow → Load (API Format)` or `Workflow → Open` and
follow the prompt.
## Cloud vs local model names
Comfy Cloud's preinstalled checkpoints sometimes have a `-fp16` suffix
(`v1-5-pruned-emaonly-fp16.safetensors`) while the canonical local download
keeps the original name (`v1-5-pruned-emaonly.safetensors`). The example
workflows use the local-canonical names. When running on cloud, override with:
```bash
python3 ../scripts/run_workflow.py \
--workflow sd15_txt2img.json \
--args '{"ckpt_name": "v1-5-pruned-emaonly-fp16.safetensors", "prompt": "..."}' \
--host https://cloud.comfy.org
```
The `ckpt_name`, `vae_name`, `lora_name`, `unet_name`, etc. are all exposed
as controllable parameters by `extract_schema.py` — discover what's installed
with `comfy model list` (local) or `curl /api/experiment/models/checkpoints`
(cloud).
creative/comfyui/workflows/animatediff_video.json
+64
View File
@@ -0,0 +1,64 @@
{
"_comment": "AnimateDiff text-to-video at 16 frames. Required: comfyui-animatediff-evolved + comfyui-videohelpersuite custom nodes; SD1.5 checkpoint; AnimateDiff motion module (e.g. mm_sd_v15_v2.ckpt in models/animatediff_models/). Outputs a webp animation.",
"3": {
"class_type": "KSampler",
"_meta": {"title": "KSampler"},
"inputs": {
"seed": 42, "steps": 25, "cfg": 7.5,
"sampler_name": "dpmpp_sde", "scheduler": "karras", "denoise": 1.0,
"model": ["10", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["5", 0]
}
},
"4": {
"class_type": "CheckpointLoaderSimple",
"_meta": {"title": "Checkpoint"},
"inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"}
},
"5": {
"class_type": "EmptyLatentImage",
"_meta": {"title": "Latent (16 frames)"},
"inputs": {"width": 512, "height": 512, "batch_size": 16}
},
"6": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Positive Prompt"},
"inputs": {"text": "a hot air balloon drifting over a mountain valley, sunset, cinematic", "clip": ["4", 1]}
},
"7": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Negative Prompt"},
"inputs": {"text": "low quality, blurry, deformed, watermark", "clip": ["4", 1]}
},
"8": {
"class_type": "VAEDecode",
"_meta": {"title": "VAE Decode"},
"inputs": {"samples": ["3", 0], "vae": ["4", 2]}
},
"9": {
"class_type": "VHS_VideoCombine",
"_meta": {"title": "Video Combine"},
"inputs": {
"frame_rate": 8.0,
"loop_count": 0,
"filename_prefix": "animatediff",
"format": "video/h264-mp4",
"pingpong": false,
"save_output": true,
"images": ["8", 0]
}
},
"10": {
"class_type": "ADE_AnimateDiffLoaderWithContext",
"_meta": {"title": "AnimateDiff Loader"},
"inputs": {
"model": ["4", 0],
"model_name": "mm_sd_v15_v2.ckpt",
"beta_schedule": "sqrt_linear (AnimateDiff)",
"motion_scale": 1.0,
"apply_v2_models_properly": true
}
}
}
creative/comfyui/workflows/flux_dev_txt2img.json
+78
View File
@@ -0,0 +1,78 @@
{
"_comment": "Flux Dev text-to-image using the modern sampler chain (BasicScheduler/Guider/SamplerCustomAdvanced). Required: flux1-dev.safetensors (UNET), t5xxl_fp16.safetensors + clip_l.safetensors (CLIP), ae.safetensors (VAE).",
"6": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Prompt"},
"inputs": {"text": "a serene mountain landscape at golden hour, photorealistic", "clip": ["11", 0]}
},
"8": {
"class_type": "VAEDecode",
"_meta": {"title": "VAE Decode"},
"inputs": {"samples": ["13", 0], "vae": ["10", 0]}
},
"9": {
"class_type": "SaveImage",
"_meta": {"title": "Save Image"},
"inputs": {"filename_prefix": "flux_dev", "images": ["8", 0]}
},
"10": {
"class_type": "VAELoader",
"_meta": {"title": "VAE"},
"inputs": {"vae_name": "ae.safetensors"}
},
"11": {
"class_type": "DualCLIPLoader",
"_meta": {"title": "DualCLIPLoader"},
"inputs": {
"clip_name1": "t5xxl_fp16.safetensors",
"clip_name2": "clip_l.safetensors",
"type": "flux"
}
},
"12": {
"class_type": "UNETLoader",
"_meta": {"title": "UNET Loader"},
"inputs": {"unet_name": "flux1-dev.safetensors", "weight_dtype": "default"}
},
"13": {
"class_type": "SamplerCustomAdvanced",
"_meta": {"title": "Sampler Custom"},
"inputs": {
"noise": ["25", 0],
"guider": ["22", 0],
"sampler": ["16", 0],
"sigmas": ["17", 0],
"latent_image": ["27", 0]
}
},
"16": {
"class_type": "KSamplerSelect",
"_meta": {"title": "Sampler Select"},
"inputs": {"sampler_name": "euler"}
},
"17": {
"class_type": "BasicScheduler",
"_meta": {"title": "Scheduler"},
"inputs": {
"scheduler": "simple",
"steps": 20,
"denoise": 1.0,
"model": ["12", 0]
}
},
"22": {
"class_type": "BasicGuider",
"_meta": {"title": "Guider"},
"inputs": {"model": ["12", 0], "conditioning": ["6", 0]}
},
"25": {
"class_type": "RandomNoise",
"_meta": {"title": "Noise"},
"inputs": {"noise_seed": 42}
},
"27": {
"class_type": "EmptySD3LatentImage",
"_meta": {"title": "Latent"},
"inputs": {"width": 1024, "height": 1024, "batch_size": 1}
}
}
creative/comfyui/workflows/sd15_txt2img.json
+49
View File
@@ -0,0 +1,49 @@
{
"_comment": "SD 1.5 text-to-image. Smallest model, fastest. Required model: v1-5-pruned-emaonly.safetensors (or any SD1.5 checkpoint)",
"3": {
"class_type": "KSampler",
"_meta": {"title": "KSampler"},
"inputs": {
"seed": 156680208700286,
"steps": 20,
"cfg": 8.0,
"sampler_name": "euler",
"scheduler": "normal",
"denoise": 1.0,
"model": ["4", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["5", 0]
}
},
"4": {
"class_type": "CheckpointLoaderSimple",
"_meta": {"title": "Load Checkpoint"},
"inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"}
},
"5": {
"class_type": "EmptyLatentImage",
"_meta": {"title": "Empty Latent"},
"inputs": {"width": 512, "height": 512, "batch_size": 1}
},
"6": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Positive Prompt"},
"inputs": {"text": "a beautiful landscape painting, masterpiece, highly detailed", "clip": ["4", 1]}
},
"7": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Negative Prompt"},
"inputs": {"text": "ugly, blurry, low quality, deformed", "clip": ["4", 1]}
},
"8": {
"class_type": "VAEDecode",
"_meta": {"title": "VAE Decode"},
"inputs": {"samples": ["3", 0], "vae": ["4", 2]}
},
"9": {
"class_type": "SaveImage",
"_meta": {"title": "Save Image"},
"inputs": {"filename_prefix": "sd15", "images": ["8", 0]}
}
}
creative/comfyui/workflows/sdxl_img2img.json
+54
View File
@@ -0,0 +1,54 @@
{
"_comment": "SDXL img2img: load an input image, encode to latent, denoise partially. Use --input-image image=./photo.png with run_workflow.py. Lower 'denoise' value preserves more of the source image.",
"1": {
"class_type": "LoadImage",
"_meta": {"title": "Load Source Image"},
"inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"}
},
"3": {
"class_type": "KSampler",
"_meta": {"title": "KSampler"},
"inputs": {
"seed": 42,
"steps": 30,
"cfg": 7.5,
"sampler_name": "dpmpp_2m",
"scheduler": "karras",
"denoise": 0.65,
"model": ["4", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["12", 0]
}
},
"4": {
"class_type": "CheckpointLoaderSimple",
"_meta": {"title": "Load SDXL Base"},
"inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"}
},
"6": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Positive Prompt"},
"inputs": {"text": "make it cyberpunk, neon lights, futuristic", "clip": ["4", 1]}
},
"7": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Negative Prompt"},
"inputs": {"text": "ugly, blurry, low quality, deformed", "clip": ["4", 1]}
},
"8": {
"class_type": "VAEDecode",
"_meta": {"title": "VAE Decode"},
"inputs": {"samples": ["3", 0], "vae": ["4", 2]}
},
"9": {
"class_type": "SaveImage",
"_meta": {"title": "Save Image"},
"inputs": {"filename_prefix": "sdxl_img2img", "images": ["8", 0]}
},
"12": {
"class_type": "VAEEncode",
"_meta": {"title": "VAE Encode"},
"inputs": {"pixels": ["1", 0], "vae": ["4", 2]}
}
}
creative/comfyui/workflows/sdxl_inpaint.json
+59
View File
@@ -0,0 +1,59 @@
{
"_comment": "SDXL inpainting: given an image + mask, regenerate the masked region. Upload both: --input-image image=./photo.png --input-image mask_image=./mask.png. White pixels in mask = regenerate; black = preserve.",
"1": {
"class_type": "LoadImage",
"_meta": {"title": "Load Source"},
"inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"}
},
"2": {
"class_type": "LoadImageMask",
"_meta": {"title": "Load Mask"},
"inputs": {"image": "REPLACE_WITH_UPLOADED_MASK.png", "channel": "red"}
},
"3": {
"class_type": "KSampler",
"_meta": {"title": "KSampler"},
"inputs": {
"seed": 42,
"steps": 30,
"cfg": 7.5,
"sampler_name": "dpmpp_2m",
"scheduler": "karras",
"denoise": 1.0,
"model": ["4", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["12", 0]
}
},
"4": {
"class_type": "CheckpointLoaderSimple",
"_meta": {"title": "Checkpoint"},
"inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"}
},
"6": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Positive Prompt"},
"inputs": {"text": "fill with blooming flowers, photorealistic", "clip": ["4", 1]}
},
"7": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Negative Prompt"},
"inputs": {"text": "ugly, blurry, deformed, bad anatomy", "clip": ["4", 1]}
},
"8": {
"class_type": "VAEDecode",
"_meta": {"title": "VAE Decode"},
"inputs": {"samples": ["3", 0], "vae": ["4", 2]}
},
"9": {
"class_type": "SaveImage",
"_meta": {"title": "Save"},
"inputs": {"filename_prefix": "sdxl_inpaint", "images": ["8", 0]}
},
"12": {
"class_type": "VAEEncodeForInpaint",
"_meta": {"title": "VAE Encode for Inpaint"},
"inputs": {"pixels": ["1", 0], "mask": ["2", 0], "vae": ["4", 2], "grow_mask_by": 6}
}
}
creative/comfyui/workflows/sdxl_txt2img.json
+49
View File
@@ -0,0 +1,49 @@
{
"_comment": "SDXL text-to-image at 1024x1024. Required model: sd_xl_base_1.0.safetensors (or any SDXL checkpoint).",
"3": {
"class_type": "KSampler",
"_meta": {"title": "KSampler"},
"inputs": {
"seed": 42,
"steps": 30,
"cfg": 7.5,
"sampler_name": "dpmpp_2m",
"scheduler": "karras",
"denoise": 1.0,
"model": ["4", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["5", 0]
}
},
"4": {
"class_type": "CheckpointLoaderSimple",
"_meta": {"title": "Load SDXL Base"},
"inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"}
},
"5": {
"class_type": "EmptyLatentImage",
"_meta": {"title": "Empty Latent"},
"inputs": {"width": 1024, "height": 1024, "batch_size": 1}
},
"6": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Positive Prompt"},
"inputs": {"text": "cinematic photograph, dramatic lighting, intricate detail", "clip": ["4", 1]}
},
"7": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Negative Prompt"},
"inputs": {"text": "ugly, blurry, low quality, deformed, watermark", "clip": ["4", 1]}
},
"8": {
"class_type": "VAEDecode",
"_meta": {"title": "VAE Decode"},
"inputs": {"samples": ["3", 0], "vae": ["4", 2]}
},
"9": {
"class_type": "SaveImage",
"_meta": {"title": "Save Image"},
"inputs": {"filename_prefix": "sdxl", "images": ["8", 0]}
}
}
creative/comfyui/workflows/upscale_4x.json
+27
View File
@@ -0,0 +1,27 @@
{
"_comment": "Standalone 4x upscale of an input image using ESRGAN. Required model: 4x-UltraSharp.pth (or any upscaler in models/upscale_models/). Upload with --input-image image=./photo.png.",
"1": {
"class_type": "LoadImage",
"_meta": {"title": "Load Image"},
"inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"}
},
"2": {
"class_type": "UpscaleModelLoader",
"_meta": {"title": "Load Upscale Model"},
"inputs": {"model_name": "4x-UltraSharp.pth"}
},
"3": {
"class_type": "ImageUpscaleWithModel",
"_meta": {"title": "Upscale Image (with Model)"},
"inputs": {
"upscale_method": "lanczos",
"upscale_model": ["2", 0],
"image": ["1", 0]
}
},
"4": {
"class_type": "SaveImage",
"_meta": {"title": "Save"},
"inputs": {"filename_prefix": "upscaled_4x", "images": ["3", 0]}
}
}
creative/comfyui/workflows/wan_video_t2v.json
+69
View File
@@ -0,0 +1,69 @@
{
"_comment": "Wan 2.1 text-to-video. Cloud: confirmed available. Local: download wan2.1_t2v_1.3B_fp16.safetensors → models/diffusion_models/ (or models/unet/), umt5_xxl_fp16.safetensors → models/text_encoders/ (or models/clip/), wan_2.1_vae.safetensors → models/vae/. Output: MP4. Large model — only on cloud or 24 GB+ local GPU.",
"6": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Prompt"},
"inputs": {
"text": "a graceful crane taking flight from a misty lake at dawn, slow motion, 4k",
"clip": ["38", 0]
}
},
"7": {
"class_type": "CLIPTextEncode",
"_meta": {"title": "Negative Prompt"},
"inputs": {
"text": "static, blurry, watermark, low quality",
"clip": ["38", 0]
}
},
"8": {
"class_type": "VAEDecode",
"_meta": {"title": "VAE Decode"},
"inputs": {"samples": ["3", 0], "vae": ["39", 0]}
},
"37": {
"class_type": "UNETLoader",
"_meta": {"title": "Wan UNET"},
"inputs": {"unet_name": "wan2.1_t2v_1.3B_fp16.safetensors", "weight_dtype": "default"}
},
"38": {
"class_type": "CLIPLoader",
"_meta": {"title": "Wan CLIP"},
"inputs": {"clip_name": "umt5_xxl_fp16.safetensors", "type": "wan"}
},
"39": {
"class_type": "VAELoader",
"_meta": {"title": "Wan VAE"},
"inputs": {"vae_name": "wan_2.1_vae.safetensors"}
},
"3": {
"class_type": "KSampler",
"_meta": {"title": "KSampler"},
"inputs": {
"seed": 42, "steps": 30, "cfg": 6.0,
"sampler_name": "uni_pc", "scheduler": "simple", "denoise": 1.0,
"model": ["37", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["40", 0]
}
},
"40": {
"class_type": "EmptyHunyuanLatentVideo",
"_meta": {"title": "Latent Video (33 frames)"},
"inputs": {"width": 832, "height": 480, "length": 33, "batch_size": 1}
},
"9": {
"class_type": "VHS_VideoCombine",
"_meta": {"title": "Video Combine"},
"inputs": {
"frame_rate": 16.0,
"loop_count": 0,
"filename_prefix": "wan_t2v",
"format": "video/h264-mp4",
"pingpong": false,
"save_output": true,
"images": ["8", 0]
}
}
}