Files

T

Dominik Schön bbbe4f8985 feat: patterns/ directory + skill-impact tracker (WikiSkill-inspired)

- New patterns/ directory with 7 initial failure-mode patterns (PAT-001..007)
- skill-impact.md audit trail for skill modifications
- _template.md for future pattern creation
- index.md updated with Patterns section
- log.md entry for this change
- Inspired by arXiv:2608.27454 (WikiSkill)

2026-08-30 11:16:19 +00:00

2.3 KiB

Raw Blame History

pattern_id, title, category, severity, status, first_observed, last_updated, related_systems, related_solution_docs, related_skills

pattern_id

title

VFIO GPU passthrough fails — missing softdep + incomplete DRM blacklist

Symptom

GPU passthrough to a VM fails intermittently or consistently. Symptoms include:

/dev/dri/renderD128 missing in the guest
amdgpu driver not loading in guest
Kernel BUG in host dmesg
GPU device visible in lspci but not bound to vfio-pci

Root Cause

Two intertwined issues:

Module loading race condition: Without softdep amdgpu pre: vfio-pci, the amdgpu driver races to bind the GPU before vfio-pci can claim it. Normal bind time ~2.4s; racing bind time ~302s (or never succeeds).
Incomplete DRM blacklist: If drm and drm_kms_helper are not blacklisted, the kernel's DRM subsystem grabs the GPU before vfio-pci, preventing passthrough.

Mitigation

Apply to the PVE host's modprobe config:

# /etc/modprobe.d/blacklist-drm.conf
blacklist drm
blacklist drm_kms_helper

# /etc/modprobe.d/amdgpu-vfio.conf
softdep amdgpu pre: vfio-pci

Then rebuild initramfs and reboot:

update-initramfs -u -k all
reboot

Also ensure rombar=1 in the VM config for AMD GPUs (PCI ID 1002:xxxx):

# /etc/pve/qemu-server/<VMID>.conf
hostpci0: 0000:XX:YY.Z,pcie=1,rombar=1,x-vga=1

Prevention

Always configure softdep + DRM blacklist BEFORE attempting GPU passthrough on a new node
Add to provisioning scripts (Ansible/Tofu) for any node with AMD GPU intended for passthrough
ROCm 7.x: gfx1150 supported, gfx1036 NOT supported — verify GPU compute ISA before assignment

Evidence

Observed on ms-a2-1 (race condition, Jul 2026) and worker-05/VM 102 (kernel BUG, rombar)
Solution docs: 2026-07-24-vfio-pci-module-loading-race-condition.md, 2026-07-21-amd-gpu-passthrough-rombar.md
MEMORY.md entry: "GPU:2 active—Immich(CT111/n5pro gfx1150)+Frigate(CT120/px7). ROCm7.14=gfx1150 supported,gfx1036 NOT."

2.3 KiB Raw Blame History