132 lines
5.4 KiB
Markdown
132 lines
5.4 KiB
Markdown
# FLUX Model Download Reference for ComfyUI on ROCm
|
|
|
|
Quick reference for downloading FLUX models for local ComfyUI inference on AMD ROCm (no HuggingFace login).
|
|
|
|
## Official FLUX Models from Black Forest Labs
|
|
|
|
| Model | Params | Open? | Size (FP16) |
|
|
|-------|--------|-------|-------------|
|
|
| FLUX.1 [dev] | 12B | ✅ | ~23 GB |
|
|
| FLUX.1 [schnell] | 12B | ✅ | ~23 GB |
|
|
| FLUX.1 [pro] | ? | ❌ (API only) | — |
|
|
| **FLUX.2 [dev]** | ? | ✅ | 16.9 GB |
|
|
| **FLUX.2 [klein-9B]** | 9B | ✅ | 16.9 GB |
|
|
| **FLUX.2 [klein-4B]** | 4B | ✅ | ~3.7 GB (Q4) |
|
|
|
|
The `klein` variants are faster, smaller variants of FLUX.2.
|
|
|
|
## GGUF Quantizations (No Auth Required)
|
|
|
|
When HuggingFace requires login (gated models), use community GGUF repos:
|
|
|
|
**Unsloth GGUF repo (unsloth/FLUX.2-klein-9B-GGUF):**
|
|
- `flux-2-klein-9b-Q4_K_S.gguf` — **5.43 GB** ← best balance for most GPUs
|
|
- `flux-2-klein-9b-Q4_K_M.gguf` — 5.50 GB
|
|
- `flux-2-klein-9b-Q4_0.gguf` — 5.23 GB
|
|
- `flux-2-klein-9b-Q5_K_S.gguf` — 6.46 GB
|
|
- `flux-2-klein-9b-Q8_0.gguf` — 9.29 GB
|
|
|
|
**Leejet GGUF repo (leejet/FLUX.2-klein-9B-GGUF):**
|
|
- `flux-2-klein-9b-Q4_0.gguf` — 5.23 GB
|
|
- `flux-2-klein-9b-Q8_0.gguf` — 9.29 GB (fewer options than unsloth)
|
|
|
|
**FLUX.1-dev GGUF repo (city96/FLUX.1-dev-gguf or similar):**
|
|
- `flux1-dev-Q4_K_S.gguf` — **6.4 GB**
|
|
|
|
## Authentication Barriers
|
|
|
|
| Source | Requires Auth? | Notes |
|
|
|--------|----------------|-------|
|
|
| `black-forest-labs/*` (HuggingFace) | ✅ Yes (gated) | Must login + accept license |
|
|
| `unsloth/*-GGUF` | ❌ No | Community quant, direct download |
|
|
| `leejet/*-GGUF` | ❌ No | Community quant, direct download |
|
|
| `civitai.com` | ❌ No (most) | Use `api/download/models/<id>?type=Model&format=SafeTensor` |
|
|
| `modelscope.cn` | ❌ No | China mirror, works for FLUX files |
|
|
|
|
**If a download returns a tiny (~140 byte) file:** The request was redirected to a login page or error page. The HTML response is ~140 bytes. Always verify the downloaded file size matches expectations.
|
|
|
|
## Required Auxiliary Models
|
|
|
|
**Shared VAE (all FLUX models):**
|
|
| File | Size | Purpose | Where to get (no auth) |
|
|
|------|------|---------|------------------------|
|
|
| `ae.safetensors` (VAE) | 319 MB | Image decoder | **modelscope.cn** (HuggingFace gated) |
|
|
|
|
**Text encoders are NOT shared:**
|
|
- **FLUX.1 (dev / schnell):** CLIP-L + T5XXL → `clip_l.safetensors` + `t5xxl_fp8_e4m3fn.safetensors`
|
|
- **FLUX.2-klein-9B:** Uses **8B Qwen3 text embedder** (NOT CLIP-L / T5XXL). Incompatible with ComfyUI's `DualCLIPLoaderGGUF`. Attempting to run FLUX.2 with FLUX.1 encoders produces `RuntimeError: mat1 and mat2 shapes cannot be multiplied (512x4096 and 12288x4096)`. Use diffusers `Flux2KleinPipeline` or find ComfyUI Qwen3 nodes.
|
|
|
|
| File | Size | Purpose | Where to get | Required for |
|
|
|------|------|---------|-------------|-------------|
|
|
| `clip_l.safetensors` | 235 MB | CLIP text encoder | civitai or modelscope | FLUX.1 only |
|
|
| `t5xxl_fp8_e4m3fn.safetensors` | 4.9 GB | T5 XXL text encoder | civitai or modelscope | FLUX.1 only |
|
|
### Working VAE download URLs (no-auth)
|
|
|
|
```bash
|
|
# Modelscope (verified working, no auth)
|
|
curl -L -o ae.safetensors \
|
|
"https://www.modelscope.cn/models/AI-ModelScope/FLUX.1-dev/resolve/master/ae.safetensors"
|
|
|
|
# Direct HuggingFace (REQUIRES login for BFL repos)
|
|
# curl -L -o ae.safetensors \
|
|
# "https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors"
|
|
# → Will fail with 401, producing a ~140 byte HTML error page
|
|
```
|
|
|
|
## Background Download Pattern
|
|
|
|
For multi-GB model downloads from inside a Proxmox LXC, use `nohup` to survive SSH/session timeouts:
|
|
|
|
```bash
|
|
pct exec 204 -- bash -c '
|
|
cd /opt/ComfyUI/models/unet
|
|
nohup bash -c "curl -L -o flux-2-klein-9b-Q4_K_S.gguf \\"
|
|
https://huggingface.co/unsloth/FLUX.2-klein-9B-GGUF/resolve/main/flux-2-klein-9b-Q4_K_S.gguf \\"
|
|
> /tmp/flux2_download.log 2>&1" &
|
|
'
|
|
```
|
|
|
|
Verify ongoing download:
|
|
```bash
|
|
pct exec 204 -- bash -c 'ls -lh /opt/ComfyUI/models/unet/flux-2-klein-9b-Q4_K_S.gguf'
|
|
pct exec 204 -- bash -c 'tail -3 /tmp/flux2_download.log'
|
|
```
|
|
|
|
## ComfyUI Model Paths
|
|
|
|
```
|
|
/opt/ComfyUI/models/
|
|
├── unet/ # GGUF diffusion models (FLUX, SDXL, etc.)
|
|
│ └── flux-2-klein-9b-Q4_K_S.gguf
|
|
├── clip/ # Text encoders
|
|
│ ├── clip_l.safetensors
|
|
│ └── t5xxl_fp8_e4m3fn.safetensors
|
|
├── vae/ # Image decoders
|
|
│ └── ae.safetensors
|
|
└── checkpoints/ # Full safetensors models
|
|
```
|
|
|
|
## ComfyUI + ROCm + GGUF Verification
|
|
|
|
After downloading, verify ComfyUI can load the model:
|
|
|
|
```bash
|
|
# Start ComfyUI
|
|
pct exec 204 -- bash -c 'cd /opt/ComfyUI && python3 main.py --listen 0.0.0.0 --port 8188'
|
|
|
|
# Check API responds
|
|
curl http://10.0.20.91:8188/system_stats
|
|
|
|
# The first run will compile shaders; monitor /tmp/comfyui.log
|
|
pct exec 204 -- bash -c 'tail -20 /tmp/comfyui.log'
|
|
```
|
|
|
|
## Errors and Fixes
|
|
|
|
| Error | Cause | Fix |
|
|
|-------|-------|-----|
|
|
| `hipErrorNoBinaryForGpu` | PyTorch ROCm wheel mismatch with host ROCm | Downgrade/upgrade PyTorch wheel: `pip install torch --index-url https://download.pytorch.org/whl/rocm6.2` |
|
|
| Downloaded file is ~140 bytes | Gated model, no auth token | Switch to unsloth/leejet GGUF repos or modelscope |
|
|
| Downloaded file is 6+ GB for VAE | Wrong CivitAI URL (model page, not direct download) | Use correct `api/download/models/<id>?type=Model&format=SafeTensor` URL |
|
|
| `curl` times out during download | Paramiko SSH timeout | Use `nohup` + background; poll file size separately |
|