Dominik Schön
bbbe4f8985
- New patterns/ directory with 7 initial failure-mode patterns (PAT-001..007) - skill-impact.md audit trail for skill modifications - _template.md for future pattern creation - index.md updated with Patterns section - log.md entry for this change - Inspired by arXiv:2608.27454 (WikiSkill)
72 lines
2.3 KiB
Markdown
72 lines
2.3 KiB
Markdown
---
|
|
pattern_id: PAT-004
|
|
title: "VFIO GPU passthrough fails — missing softdep + incomplete DRM blacklist"
|
|
category: infrastructure
|
|
severity: high
|
|
status: active
|
|
first_observed: 2026-07
|
|
last_updated: 2026-08-30
|
|
related_systems: [proxmox-cluster, rke2-kubernetes]
|
|
related_solution_docs:
|
|
- docs/solutions/bug-fixes/2026-07-24-vfio-pci-module-loading-race-condition.md
|
|
- docs/solutions/bug-fixes/2026-07-21-amd-gpu-passthrough-rombar.md
|
|
related_skills: []
|
|
---
|
|
|
|
# VFIO GPU passthrough fails — missing softdep + incomplete DRM blacklist
|
|
|
|
## Symptom
|
|
|
|
GPU passthrough to a VM fails intermittently or consistently. Symptoms include:
|
|
- `/dev/dri/renderD128` missing in the guest
|
|
- `amdgpu` driver not loading in guest
|
|
- Kernel BUG in host dmesg
|
|
- GPU device visible in `lspci` but not bound to `vfio-pci`
|
|
|
|
## Root Cause
|
|
|
|
Two intertwined issues:
|
|
|
|
1. **Module loading race condition**: Without `softdep amdgpu pre: vfio-pci`, the amdgpu
|
|
driver races to bind the GPU before vfio-pci can claim it. Normal bind time ~2.4s;
|
|
racing bind time ~302s (or never succeeds).
|
|
|
|
2. **Incomplete DRM blacklist**: If `drm` and `drm_kms_helper` are not blacklisted, the
|
|
kernel's DRM subsystem grabs the GPU before vfio-pci, preventing passthrough.
|
|
|
|
## Mitigation
|
|
|
|
Apply to the PVE host's modprobe config:
|
|
```bash
|
|
# /etc/modprobe.d/blacklist-drm.conf
|
|
blacklist drm
|
|
blacklist drm_kms_helper
|
|
|
|
# /etc/modprobe.d/amdgpu-vfio.conf
|
|
softdep amdgpu pre: vfio-pci
|
|
```
|
|
Then rebuild initramfs and reboot:
|
|
```bash
|
|
update-initramfs -u -k all
|
|
reboot
|
|
```
|
|
|
|
Also ensure `rombar=1` in the VM config for AMD GPUs (PCI ID 1002:xxxx):
|
|
```
|
|
# /etc/pve/qemu-server/<VMID>.conf
|
|
hostpci0: 0000:XX:YY.Z,pcie=1,rombar=1,x-vga=1
|
|
```
|
|
|
|
## Prevention
|
|
|
|
- Always configure `softdep` + DRM blacklist BEFORE attempting GPU passthrough on a new node
|
|
- Add to provisioning scripts (Ansible/Tofu) for any node with AMD GPU intended for passthrough
|
|
- ROCm 7.x: gfx1150 supported, gfx1036 NOT supported — verify GPU compute ISA before assignment
|
|
|
|
## Evidence
|
|
|
|
- Observed on ms-a2-1 (race condition, Jul 2026) and worker-05/VM 102 (kernel BUG, rombar)
|
|
- Solution docs: `2026-07-24-vfio-pci-module-loading-race-condition.md`,
|
|
`2026-07-21-amd-gpu-passthrough-rombar.md`
|
|
- MEMORY.md entry: "GPU:2 active—Immich(CT111/n5pro gfx1150)+Frigate(CT120/px7). ROCm7.14=gfx1150 supported,gfx1036 NOT."
|