70 lines
3.5 KiB
Markdown
70 lines
3.5 KiB
Markdown
# FLUX.2 in ComfyUI — Compatibility Guide
|
||
|
||
> **Status: Native ComfyUI support confirmed as of June 2026.**
|
||
> See `references/flux2-setup.md` for the complete reproduction recipe, model downloads, and workflow JSON.
|
||
|
||
## The Problem (Historical)
|
||
|
||
FLUX.2-klein (9B) uses an **8B Qwen3-VL text embedder**, replacing the CLIP-L + T5XXL combo used by FLUX.1. Early ComfyUI versions (pre-mid 2026) lacked native nodes for FLUX.2, forcing users to:
|
||
- Use the diffusers `Flux2KleinPipeline` instead, or
|
||
- Wait for community custom nodes
|
||
|
||
This is no longer necessary. ComfyUI now includes:
|
||
- `CLIPType.FLUX2` enum (line 1299 in `comfy/sd.py`)
|
||
- `Flux2` model class (line 1057 in `comfy/model_base.py`)
|
||
- `EmptyFlux2LatentImage` node (128-channel latents)
|
||
- `Flux2Scheduler` node (mu-snr shifted timesteps)
|
||
- `Flux2TEModel` text encoder wrapper with Qwen3VL support
|
||
- `KleinTokenizer` / `KleinTokenizer8B` tokenizers
|
||
|
||
## Error Signature → Fix Mapping
|
||
|
||
| Error | Cause | Fix |
|
||
|-------|-------|-----|
|
||
| `mat1 and mat2 shapes cannot be multiplied (512x4096 and 12288x4096)` | Using CLIP-L+T5XXL with FLUX.2 UNet | Load `Qwen3VL-8B-Instruct-Q4_K_M.gguf` via `CLIPLoaderGGUF` with `"type": "flux2"` |
|
||
| `expected input[1, 128, 32, 32] to have 16 channels, but got 128 channels instead` | Using FLUX.1 VAE (`ae.safetensors`) | Use `taef2` or `flux2_vae.safetensors` |
|
||
| `HIP error: invalid device function` | ROCm gfx1150 without override | `export HSA_OVERRIDE_GFX_VERSION=11.0.0` |
|
||
|
||
## Required Models
|
||
|
||
| Component | File | ~Size | Source |
|
||
|-----------|------|-------|--------|
|
||
| UNet (GGUF) | `flux-2-klein-9b-Q4_K_S.gguf` | 5.4 GB | `unsloth/FLUX.2-klein-9B-GGUF` |
|
||
| Text Encoder | `Qwen3VL-8B-Instruct-Q4_K_M.gguf` | 4.7 GB | `Qwen/Qwen3-VL-8B-Instruct-GGUF` |
|
||
| VAE (fast) | `taef2` encoder+decoder `.pth` | 5 MB | `madebyollin/taesd` (GitHub) |
|
||
| VAE (full) | `flux2_vae.safetensors` | 160 MB | `black-forest-labs/FLUX.2-klein-9B` (gated) |
|
||
|
||
**FLUX.1 and FLUX.2 are NOT component-compatible.** Do not mix:
|
||
- ❌ FLUX.2 UNet + FLUX.1 CLIP/T5
|
||
- ❌ FLUX.2 latents + FLUX.1 VAE (`ae.safetensors`)
|
||
- ❌ `EmptyLatentImage` with FLUX.2 (needs `EmptyFlux2LatentImage`)
|
||
|
||
## Workflow Differences (FLUX.1 vs FLUX.2)
|
||
|
||
```json
|
||
// FLUX.1 — two CLIP loaders + shared VAE
|
||
{"2": {"inputs": {"clip_name1": "t5xxl_fp8_e4m3fn.safetensors", "clip_name2": "clip_l.safetensors", "type": "flux"}, "class_type": "DualCLIPLoaderGGUF"}}
|
||
{"4": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyLatentImage"}}
|
||
{"3": {"inputs": {"vae_name": "ae.safetensors"}, "class_type": "VAELoader"}}
|
||
|
||
// FLUX.2 — single Qwen3 CLIP + TAEF2 VAE + 128-channel latent
|
||
{"2": {"inputs": {"clip_name": "Qwen3VL-8B-Instruct-Q4_K_M.gguf", "type": "flux2"}, "class_type": "CLIPLoaderGGUF"}}
|
||
{"4": {"inputs": {"width": 1024, "height": 1024, "batch_size": 1}, "class_type": "EmptyFlux2LatentImage"}}
|
||
{"3": {"inputs": {"vae_name": "taef2"}, "class_type": "VAELoader"}}
|
||
```
|
||
|
||
## Verified E2E Results (gfx1150, ROCm 6.2)
|
||
|
||
| Pipeline | Res | Steps | Time | Status |
|
||
|----------|-----|-------|------|--------|
|
||
| FLUX.1-dev Q4_K_S + CLIP/T5 + ae.safetensors | 512×512 | 4 | ~46s | ✅ |
|
||
| FLUX.2-klein Q4_K_S + Qwen3VL Q4_K_M + taef2 | 1024×1024 | 4 | ~110s | ✅ |
|
||
|
||
## References
|
||
|
||
- Full setup recipe: `references/flux2-setup.md`
|
||
- Unsloth GGUF repo: https://huggingface.co/unsloth/FLUX.2-klein-9B-GGUF
|
||
- Qwen3VL GGUF repo: https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct-GGUF
|
||
- BFL blog: https://bfl.ai/blog/flux2-klein-towards-interactive-visual-intelligence
|
||
- TAEF2 source: https://github.com/madebyollin/taesd
|