# 10.0.30.100 Decommission — Session Detail (2026-06-30)

## Host Info
- **Host**: 10.0.30.100 (Ubuntu, hostname `ubuntu`)
- **User**: dominik (password auth initially, SSH key deployed)
- **Docker**: 28.1.1, ZFS backend (`pool01_n2_redundant/docker` dataset, 2.7TB)
- **SSH Key**: `~/.ssh/hermes_pve_agent` deployed to dominik's authorized_keys

## SUDO_ASKPASS Setup
- Password: provided in chat (do not persist)
- Askpass script deployed to `/tmp/askpass.sh` on remote host
- Pattern: `SUDO_ASKPASS=/tmp/askpass.sh sudo -A COMMAND`

## Services Removed (10 initial services)

### Containers
None were running — only images and volumes remained. All 10 services had been stopped previously.

### Images Removed (~28 GB)
| Service | Images | Approx Size |
|---------|--------|-------------|
| Frigate | 0.16.0-beta2, beta4, stable | ~8.5 GB |
| Immich | server + ML + ML-openvino | ~3.0 GB |
| Paperless | 6 versions (2.9.0–2.17.1) | ~8.3 GB |
| Scrypted | latest | 2.2 GB |
| Homegear | stable | 1.5 GB |
| ioBroker | latest | 1.3 GB |
| Homebridge | ubuntu-no-avahi | 0.9 GB |
| openHAB | 3.4.5 | 0.7 GB |
| Node-RED | latest | 0.6 GB |
| Jenkins | jekyll-docker | 0.5 GB |

### Volumes Removed (20 volumes)
frigate_config, frigate_storage, frigate_postgres-data, frigate_double-take, homebridge, homegear-data-etc, homegear-data-lib, homegear-data-log, immich_model-cache, immich_pgdata, iobrokerdata, jenkins_home, nodered, openhab4_openhab4_addons, openhab4_openhab4_conf, openhab4_openhab4_userdata, paperless_data, paperless_media, paperless_redisdata, scrypted

### Bind Mounts Cleaned
- `/pool01_n2_redundant/HomeAssistant/frigate` — deleted (clips, recordings)
- `/pool01_n2_redundant/homes/dominik/.homebridge` — deleted (config, accessories)
- `/pool01_n2_redundant/Frigate/` — emptied (ZFS dataset left, "dataset is busy")

## GitLab + Runner Removal (additional)

### Containers Removed
- `development-gitlab-1` (GitLab CE 15.8.1)
- `gitlab-runner` (gitlab-runner:alpine)
- `development-register-runner-1` (registration container, already exited)
- `development-dind-1` (Docker-in-Docker, **was causing restart-loop**)

### Images Removed
- gitlab/gitlab-ce:15.8.1-ce.0
- gitlab/gitlab-runner:alpine
- docker:20-dind
- docker:dind

### Volumes Removed
- gitlab_config, gitlab_data, gitlab_logs
- development_gitlab-runner
- 2× runner cache volumes (runner-xtxmd-6p-project-11-concurrent-0-cache-*)

## InfluxDB + Home Assistant Volume Removal

### Volumes Deleted (~273 GB freed)
| Volume | Size | Content | Data Range |
|--------|------|---------|------------|
| `506f355d...` (anonymous) | 270 GB | InfluxDB engine (data + wal) | Oct 2022 – Aug 2025 |
| `homeassistant_home_assistant_config` | 2.7 GB | HA config (duplicate, HA runs on 10.0.30.10) | — |

## Remaining Volumes on Host (analysis done, not yet acted on)

### Large Volumes
| Volume | Size | Content | Data Range |
|--------|------|---------|------------|
| seafile_data | 246 GB | Seafile (logs, nginx, seafile-conf) | — |
| seafile_custom_data | 226 GB | Seafile storage (blocks, commits, fs) | Mar 2015 – May 2025 |
| registry_lib | 1.4 GB | Docker Registry | — |

### Medium Volumes
| Volume | Size | Content |
|--------|------|---------|
| embyconfig | 860 MB | Emby Media Server |
| trivy_cache | 433 MB | Trivy Security Scanner |
| jellyfin_jellyfin_config | 314 MB | Jellyfin |
| matomo | 134 MB | Matomo Analytics |
| monitoring_prometheus_data | 146 MB | Prometheus |
| monitoring_grafana_db | 61 MB | Grafana |
| portainer_data | 63 MB | Portainer |
| pgdata | 46 MB | PostgreSQL |
| ghost-imkerei | 43 MB | Ghost Blog |
| openproject_postgresql_data | 25 MB | OpenProject |

### Small Volumes (<10 MB)
grafana, vicare, handbrake_config, openproject_assets, spoolman, proxy_conf, traefik_acme, telegraf, sabnzbd, bitmonero, ethereum, bienen_proxy, ghost_proxy, xenia_*, ~60 anonymous hash volumes (mostly 7 KB each, one 943 KB)

## Docker Daemon Instability

### Root Cause
`development-dind-1` container had `restart: always` and crashed on boot, creating a restart-loop. This made the Docker daemon extremely slow/unresponsive:
- `docker images` hung or returned empty (false verification!)
- `docker rmi` timed out
- `docker volume rm` timed out
- `docker ps` worked but slowly

### Resolution
1. `kill -9 $(pgrep dockerd)` — force kill daemon
2. `systemctl start docker` — restart
3. Wait 30-60s for ZFS backend to initialize
4. Quickly `docker rm -f development-dind-1` before it restarts
5. After removal, daemon became stable

### False Verification Incident
Initial verification showed `docker images | grep -i frigate` returning empty → concluded "all images deleted." After daemon restart, images were visible again. Lesson: always verify daemon stability before trusting `docker images` output.

## Remaining Containers After Cleanup
- portainer (Up)
- Traefik (Up)
- spoolman-spoolman-1 (Exited)
- website-imkerei-bienen-proxy-1 (Up)
- website-imkerei-proxy-1 (Up)
- website-imkerei-website-1 (Up)

## Model ID Drift (noris.ai)
- Old model ID: `vllm/gemma-4-31b-it-dynamo` (no longer available)
- New model ID: `vllm/gemma-4-31b-it` (verified via `/v1/models` endpoint)
- Also available: `vllm/prod/gemma-4-31b-it`
- Fixed in: `recipe_scraper_v8_intl.py` line 108
- Updated: cronjob prompt, memory entry