hermes-skills/devops/docker-host-administration/references/10.0.30.100-decommission.md

10.0.30.100 Decommission — Session Detail (2026-06-30)

Host Info

• Host: 10.0.30.100 (Ubuntu, hostname ubuntu)
• User: dominik (password auth initially, SSH key deployed)
• Docker: 28.1.1, ZFS backend (pool01_n2_redundant/docker dataset, 2.7TB)
• SSH Key: ~/.ssh/hermes_pve_agent deployed to dominik's authorized_keys

SUDO_ASKPASS Setup

• Password: provided in chat (do not persist)
• Askpass script deployed to /tmp/askpass.sh on remote host
• Pattern: SUDO_ASKPASS=/tmp/askpass.sh sudo -A COMMAND

Services Removed (10 initial services)

Containers

None were running — only images and volumes remained. All 10 services had been stopped previously.

Images Removed (~28 GB)

Service	Images	Approx Size
Frigate	0.16.0-beta2, beta4, stable	~8.5 GB
Immich	server + ML + ML-openvino	~3.0 GB
Paperless	6 versions (2.9.0–2.17.1)	~8.3 GB
Scrypted	latest	2.2 GB
Homegear	stable	1.5 GB
ioBroker	latest	1.3 GB
Homebridge	ubuntu-no-avahi	0.9 GB
openHAB	3.4.5	0.7 GB
Node-RED	latest	0.6 GB
Jenkins	jekyll-docker	0.5 GB

Volumes Removed (20 volumes)

frigate_config, frigate_storage, frigate_postgres-data, frigate_double-take, homebridge, homegear-data-etc, homegear-data-lib, homegear-data-log, immich_model-cache, immich_pgdata, iobrokerdata, jenkins_home, nodered, openhab4_openhab4_addons, openhab4_openhab4_conf, openhab4_openhab4_userdata, paperless_data, paperless_media, paperless_redisdata, scrypted

Bind Mounts Cleaned

• /pool01_n2_redundant/HomeAssistant/frigate — deleted (clips, recordings)
• /pool01_n2_redundant/homes/dominik/.homebridge — deleted (config, accessories)
• /pool01_n2_redundant/Frigate/ — emptied (ZFS dataset left, "dataset is busy")

GitLab + Runner Removal (additional)

Containers Removed

• development-gitlab-1 (GitLab CE 15.8.1)
• gitlab-runner (gitlab-runner:alpine)
• development-register-runner-1 (registration container, already exited)
• development-dind-1 (Docker-in-Docker, was causing restart-loop)

Images Removed

• gitlab/gitlab-ce:15.8.1-ce.0
• gitlab/gitlab-runner:alpine
• docker:20-dind
• docker:dind

Volumes Removed

• gitlab_config, gitlab_data, gitlab_logs
• development_gitlab-runner
• 2× runner cache volumes (runner-xtxmd-6p-project-11-concurrent-0-cache-*)

InfluxDB + Home Assistant Volume Removal

Volumes Deleted (~273 GB freed)

Volume	Size	Content	Data Range
506f355d... (anonymous)	270 GB	InfluxDB engine (data + wal)	Oct 2022 – Aug 2025
homeassistant_home_assistant_config	2.7 GB	HA config (duplicate, HA runs on 10.0.30.10)	—

Remaining Volumes on Host (analysis done, not yet acted on)

Large Volumes

Volume	Size	Content	Data Range
seafile_data	246 GB	Seafile (logs, nginx, seafile-conf)	—
seafile_custom_data	226 GB	Seafile storage (blocks, commits, fs)	Mar 2015 – May 2025
registry_lib	1.4 GB	Docker Registry	—

Medium Volumes

Volume	Size	Content
embyconfig	860 MB	Emby Media Server
trivy_cache	433 MB	Trivy Security Scanner
jellyfin_jellyfin_config	314 MB	Jellyfin
matomo	134 MB	Matomo Analytics
monitoring_prometheus_data	146 MB	Prometheus
monitoring_grafana_db	61 MB	Grafana
portainer_data	63 MB	Portainer
pgdata	46 MB	PostgreSQL
ghost-imkerei	43 MB	Ghost Blog
openproject_postgresql_data	25 MB	OpenProject

Small Volumes (<10 MB)

grafana, vicare, handbrake_config, openproject_assets, spoolman, proxy_conf, traefik_acme, telegraf, sabnzbd, bitmonero, ethereum, bienen_proxy, ghost_proxy, xenia_*, ~60 anonymous hash volumes (mostly 7 KB each, one 943 KB)

Docker Daemon Instability

Root Cause

development-dind-1 container had restart: always and crashed on boot, creating a restart-loop. This made the Docker daemon extremely slow/unresponsive:

• docker images hung or returned empty (false verification!)
• docker rmi timed out
• docker volume rm timed out
• docker ps worked but slowly

Resolution

1. kill -9 $(pgrep dockerd) — force kill daemon
2. systemctl start docker — restart
3. Wait 30-60s for ZFS backend to initialize
4. Quickly docker rm -f development-dind-1 before it restarts
5. After removal, daemon became stable

False Verification Incident

Initial verification showed docker images | grep -i frigate returning empty → concluded "all images deleted." After daemon restart, images were visible again. Lesson: always verify daemon stability before trusting docker images output.

Remaining Containers After Cleanup

• portainer (Up)
• Traefik (Up)
• spoolman-spoolman-1 (Exited)
• website-imkerei-bienen-proxy-1 (Up)
• website-imkerei-proxy-1 (Up)
• website-imkerei-website-1 (Up)

Model ID Drift (noris.ai)

• Old model ID: vllm/gemma-4-31b-it-dynamo (no longer available)
• New model ID: vllm/gemma-4-31b-it (verified via /v1/models endpoint)
• Also available: vllm/prod/gemma-4-31b-it
• Fixed in: recipe_scraper_v8_intl.py line 108
• Updated: cronjob prompt, memory entry