Initial commit: Hermes Agent Skills collection

2026-07-12 19:02:59 +00:00
commit e9cc106625
789 changed files with 233126 additions and 0 deletions
@@ -0,0 +1,218 @@
+# Dead Integration Cleanup — Bulk Entity & Config Entry Removal
+
+Session: 2026-07-12. Removed 3 dead integrations (HAME Energy, Spoolman, Kia UVO) totaling 194 entities from a HA 2026.7.1 instance with 2,282 entities (28.5% unavailable).
+
+## Decision Framework: When to Remove vs Disable
+
+| Situation | Action |
+|-----------|--------|
+| Integration has a config entry AND no MQTT discovery | `DELETE /api/config/config_entries/entry/{ENTRY_ID}` — entities auto-remove |
+| Integration is MQTT-discovered (shares MQTT broker config entry) | Cannot delete broker entry — remove entities individually via WebSocket + clear MQTT discovery topics |
+| Integration in `setup_error` state, unrecoverable (e.g. cloud API deprecated) | Delete config entry via REST API |
+| Integration in `setup_retry` but device may recover | Disable, don't delete — `config_entries/disable` via WebSocket |
+| Feature-Not-Supported unavailable entities (vacuum sub-entities) | Leave alone — normal behavior, not a defect |
+
+## Step-by-Step Procedure
+
+### Phase 1: Baseline Measurement
+
+Always measure before making changes:
+
+```python
+import json, urllib.request
+from collections import Counter
+
+TOKEN = open("/tmp/ha_token.txt").read().strip()
+HA_URL = "http://10.0.30.10:8123"
+
+req = urllib.request.Request(f"{HA_URL}/api/states",
+    headers={"Authorization": f"Bearer {TOKEN}"})
+states = json.loads(urllib.request.urlopen(req, timeout=10).read())
+
+total = len(states)
+unavail = [e for e in states if e['state'] == 'unavailable']
+print(f"Total: {total}, Unavailable: {len(unavail)} ({len(unavail)/total*100:.1f}%)")
+
+# Group unavailable by prefix for targeting
+groups = Counter()
+for e in unavail:
+    parts = e['entity_id'].replace('.', '_').split('_')
+    groups['_'.join(parts[:3])] += 1
+for g, c in groups.most_common(20):
+    print(f"  {g}: {c}")
+```
+
+### Phase 2: Identify Config Entry IDs
+
+```python
+entries = json.loads(urllib.request.urlopen(urllib.request.Request(
+    f"{HA_URL}/api/config/config_entries/entry",
+    headers={"Authorization": f"Bearer {TOKEN}"})).read())
+
+# Find target entries
+for e in entries:
+    dom = e.get('domain', '')
+    if dom in ['target_domain1', 'target_domain2']:
+        print(f"{e['entry_id']} | {dom} | {e.get('title')} | {e.get('state')}")
+```
+
+### Phase 3: Delete Config Entries (REST API)
+
+Only for integrations with their OWN config entry (not MQTT-shared):
+
+```python
+def api(path, method="GET", data=None):
+    headers = {"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"}
+    req = urllib.request.Request(f"{HA_URL}{path}", headers=headers, method=method,
+                                data=data.encode() if data else None)
+    return urllib.request.urlopen(req, timeout=10).read()
+
+# Delete — returns {\"require_restart\": false} on success
+result = api(f"/api/config/config_entries/entry/{entry_id}", method="DELETE")
+```
+
+### Phase 4: Remove MQTT-Discovered Entities (WebSocket)
+
+MQTT-discovered entities share the broker's `config_entry_id`. You CANNOT delete the broker entry — it serves all MQTT devices. Remove entities individually:
+
+```python
+import asyncio, json, websockets
+
+async def remove_entities(entity_ids):
+    ws = await websockets.connect("ws://10.0.30.10:8123/api/websocket",
+                                   max_size=20*1024*1024)
+    # Auth
+    await ws.recv()  # auth_required
+    await ws.send(json.dumps({"type": "auth", "access_token": TOKEN}))
+    await ws.recv()  # auth_ok
+
+    # Get entity registry
+    await ws.send(json.dumps({"id": 1, "type": "config/entity_registry/list"}))
+    entities = json.loads(await ws.recv()).get("result", [])
+
+    # Filter targets
+    targets = [e for e in entities
+               if any(t in e.get("entity_id", "").lower()
+                      for t in ["hame", "spoolman"])]
+
+    # Remove each
+    removed = 0
+    for i, entity in enumerate(targets):
+        await ws.send(json.dumps({
+            "id": 2 + i,
+            "type": "config/entity_registry/remove",
+            "entity_id": entity["entity_id"]
+        }))
+        resp = json.loads(await ws.recv())
+        if resp.get("success"):
+            removed += 1
+
+    print(f"Removed: {removed}/{len(targets)}")
+    await ws.close()
+
+asyncio.run(remove_entities([]))
+```
+
+**Key point:** `config/entity_registry/remove` IS supported via WebSocket (unlike many other registry commands). It returns `{\"success\": true}` on success.
+
+### Phase 5: Clear MQTT Discovery Topics
+
+Without this step, MQTT discovery topics will recreate the entities on next HA restart.
+
+```python
+def publish_mqtt(topic, payload="", retain=True):
+    data = json.dumps({"topic": topic, "payload": payload,
+                       "retain": retain}).encode()
+    req = urllib.request.Request(
+        f"{HA_URL}/api/services/mqtt/publish",
+        headers={"Authorization": f"Bearer {TOKEN}",
+                 "Content-Type": "application/json"},
+        data=data, method="POST")
+    urllib.request.urlopen(req, timeout=10)
+
+# Get device identifiers from device registry
+# identifiers field: [["mqtt", "hame_energy_device-mac-address"]]
+# Discovery topic pattern: homeassistant/<domain>/<device_id>/config
+
+domains = ["binary_sensor", "sensor", "switch", "number",
+           "select", "text", "button", "update"]
+
+for dev_id in device_identifiers:
+    for domain in domains:
+        topic = f"homeassistant/{domain}/{dev_id}/config"
+        publish_mqtt(topic)  # Empty retained payload clears the topic
+```
+
+**Pitfall:** The REST API `mqtt/publish` returns `[]` (empty array) on success, not a confirmation object. Check HTTP 200, not response body content.
+
+### Phase 6: Stop Source HA Addons
+
+**Critical and easy to miss:** After removing MQTT-discovered entities, the HA addon that bridges them to MQTT may still be running and will re-create the entities. Check and stop source addons:
+
+```bash
+# List all HA addons (via qm guest exec on VM 106)
+ssh -i ~/.ssh/id_ed25519_proxmox root@10.0.20.10 \
+  "qm guest exec 106 -- sh -c 'ha addons'"
+
+# Stop a specific addon
+ssh -i ~/.ssh/id_ed25519_proxmox root@10.0.20.10 \
+  "qm guest exec 106 -- sh -c 'ha addons stop <slug>'"
+```
+
+Known addon-to-integration mappings:
+- `1376549a_hm2mqtt` → HAME Energy entities (must stop after HAME cleanup)
+- `ccab4aaf_frigate` → Conflicts with CT-based Frigate (stop if CT Frigate is primary)
+
+### Phase 7: Verify
+
+Re-run the Phase 1 measurement script. Compare before/after.
+
+## Worked Example: 2026-07-12 Cleanup
+
+| Integration | Type | Entities | Method |
+|-------------|------|----------|--------|
+| HAME Energy | MQTT-discovered | 81 entities, 3 devices | WebSocket entity_registry/remove + 37 MQTT topics cleared + hm2mqtt addon stopped |
+| Spoolman | Own config entry | 117 entities (2 in registry) | REST DELETE config entry + WebSocket entity removal |
+| Kia UVO | Own config entry | ~5 entities | REST DELETE config entry (setup_error state) |
+
+### Results
+
+| Metric | Before | After | Delta |
+|--------|--------|-------|-------|
+| Total entities | 2,282 | 2,088 | −194 |
+| Unavailable | 650 (28.5%) | 456 (21.8%) | −194 |
+| Unknown | 197 (8.6%) | 198 (9.5%) | +1 |
+
+### After Frigate Recovery (same session)
+
+| Metric | After Cleanup | After Frigate | Delta |
+|--------|--------------|---------------|-------|
+| Total entities | 2,088 | 2,088 | 0 |
+| Unavailable | 456 (21.8%) | 373 (17.9%) | −83 |
+| Frigate cameras | 0 online | 2 recording | +2 |
+| Frigate sensors | 18 dead | 93 available | +93 |
+
+### Remaining Unavailable After Cleanup
+
+| Group | Count | Category | Action |
+|-------|-------|----------|--------|
+| x40_master + staubsauger_oben | ~196 | Feature-Not-Supported | Leave (normal) |
+| Double Take | 13 | Defunct (no service exists) | Disappear on HA restart |
+| TRADFRI Schlafzimmer | 16 | Stale IP / VLAN | Re-add via Config Flow |
+| Weather Station | 5 | Battery? | Check sensors |
+
+## Pitfalls Encountered
+
+1. **MQTT-discovered entities have the MQTT broker's `config_entry_id`, not their own.** Deleting the broker entry would destroy ALL MQTT devices. Always check `domain` in the config entry — if it's `mqtt`, the entities are MQTT-discovered and must be removed individually.
+
+2. **`config/device_registry/remove` may silently fail.** The WebSocket command `config/device_registry/remove_config_entry_from_device` and `config/device_registry/remove` do not reliably succeed for MQTT-discovered devices. Focus on entity removal + MQTT topic clearing — devices with no remaining entities are cleaned up by HA automatically over time.
+
+3. **Entity count in registry ≠ entity count in states.** The entity registry may contain more entries than `/api/states` shows (e.g., 81 HAME in registry vs 77 in states — some were already disabled/removed but kept in registry).
+
+4. **`require_restart: false` from config entry deletion is misleading.** While the config entry is removed immediately, orphaned entities may linger in `/api/states` until HA's periodic cleanup runs. They disappear from the entity registry instantly but may show as `unavailable` in states for a short period.
+
+5. **Deriving MQTT discovery topics from device registry identifiers requires guessing the topic format.** The `identifiers` field in the device registry (e.g., `[\"mqtt\", \"hame_energy_device-mac-address\"]`) gives the device identifier, but the actual discovery topic may use different separators or casing. Clear multiple variations to be thorough.
+
+6. **MQTT-bridging HA addons must be stopped, not just entities removed.** After cleaning up HAME Energy MQTT-discovered entities, the `hm2mqtt` HA addon (`1376549a_hm2mqtt`) was still running and would re-discover them on next MQTT cycle. Always check for and stop the source addon: `qm guest exec 106 -- sh -c 'ha addons stop <slug>'`. Similarly, a conflicting Frigate HA addon (`ccab4aaf_frigate`) was in `error` state alongside the CT-based Frigate — stop it to avoid port/MQTT conflicts.
+
+7. **HA addon inventory is accessible via `ha addons` CLI through qm guest exec.** The HA Supervisor REST API requires a supervisor token that differs from the normal HA long-lived token. Instead, use: `ssh -i ~/.ssh/id_ed25519_proxmox root@10.0.20.10 "qm guest exec 106 -- sh -c 'ha addons'"` — returns YAML list of all addons with slug, name, state, and version.