219 lines
10 KiB
Markdown
219 lines
10 KiB
Markdown
# Dead Integration Cleanup — Bulk Entity & Config Entry Removal
|
||
|
||
Session: 2026-07-12. Removed 3 dead integrations (HAME Energy, Spoolman, Kia UVO) totaling 194 entities from a HA 2026.7.1 instance with 2,282 entities (28.5% unavailable).
|
||
|
||
## Decision Framework: When to Remove vs Disable
|
||
|
||
| Situation | Action |
|
||
|-----------|--------|
|
||
| Integration has a config entry AND no MQTT discovery | `DELETE /api/config/config_entries/entry/{ENTRY_ID}` — entities auto-remove |
|
||
| Integration is MQTT-discovered (shares MQTT broker config entry) | Cannot delete broker entry — remove entities individually via WebSocket + clear MQTT discovery topics |
|
||
| Integration in `setup_error` state, unrecoverable (e.g. cloud API deprecated) | Delete config entry via REST API |
|
||
| Integration in `setup_retry` but device may recover | Disable, don't delete — `config_entries/disable` via WebSocket |
|
||
| Feature-Not-Supported unavailable entities (vacuum sub-entities) | Leave alone — normal behavior, not a defect |
|
||
|
||
## Step-by-Step Procedure
|
||
|
||
### Phase 1: Baseline Measurement
|
||
|
||
Always measure before making changes:
|
||
|
||
```python
|
||
import json, urllib.request
|
||
from collections import Counter
|
||
|
||
TOKEN = open("/tmp/ha_token.txt").read().strip()
|
||
HA_URL = "http://10.0.30.10:8123"
|
||
|
||
req = urllib.request.Request(f"{HA_URL}/api/states",
|
||
headers={"Authorization": f"Bearer {TOKEN}"})
|
||
states = json.loads(urllib.request.urlopen(req, timeout=10).read())
|
||
|
||
total = len(states)
|
||
unavail = [e for e in states if e['state'] == 'unavailable']
|
||
print(f"Total: {total}, Unavailable: {len(unavail)} ({len(unavail)/total*100:.1f}%)")
|
||
|
||
# Group unavailable by prefix for targeting
|
||
groups = Counter()
|
||
for e in unavail:
|
||
parts = e['entity_id'].replace('.', '_').split('_')
|
||
groups['_'.join(parts[:3])] += 1
|
||
for g, c in groups.most_common(20):
|
||
print(f" {g}: {c}")
|
||
```
|
||
|
||
### Phase 2: Identify Config Entry IDs
|
||
|
||
```python
|
||
entries = json.loads(urllib.request.urlopen(urllib.request.Request(
|
||
f"{HA_URL}/api/config/config_entries/entry",
|
||
headers={"Authorization": f"Bearer {TOKEN}"})).read())
|
||
|
||
# Find target entries
|
||
for e in entries:
|
||
dom = e.get('domain', '')
|
||
if dom in ['target_domain1', 'target_domain2']:
|
||
print(f"{e['entry_id']} | {dom} | {e.get('title')} | {e.get('state')}")
|
||
```
|
||
|
||
### Phase 3: Delete Config Entries (REST API)
|
||
|
||
Only for integrations with their OWN config entry (not MQTT-shared):
|
||
|
||
```python
|
||
def api(path, method="GET", data=None):
|
||
headers = {"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"}
|
||
req = urllib.request.Request(f"{HA_URL}{path}", headers=headers, method=method,
|
||
data=data.encode() if data else None)
|
||
return urllib.request.urlopen(req, timeout=10).read()
|
||
|
||
# Delete — returns {\"require_restart\": false} on success
|
||
result = api(f"/api/config/config_entries/entry/{entry_id}", method="DELETE")
|
||
```
|
||
|
||
### Phase 4: Remove MQTT-Discovered Entities (WebSocket)
|
||
|
||
MQTT-discovered entities share the broker's `config_entry_id`. You CANNOT delete the broker entry — it serves all MQTT devices. Remove entities individually:
|
||
|
||
```python
|
||
import asyncio, json, websockets
|
||
|
||
async def remove_entities(entity_ids):
|
||
ws = await websockets.connect("ws://10.0.30.10:8123/api/websocket",
|
||
max_size=20*1024*1024)
|
||
# Auth
|
||
await ws.recv() # auth_required
|
||
await ws.send(json.dumps({"type": "auth", "access_token": TOKEN}))
|
||
await ws.recv() # auth_ok
|
||
|
||
# Get entity registry
|
||
await ws.send(json.dumps({"id": 1, "type": "config/entity_registry/list"}))
|
||
entities = json.loads(await ws.recv()).get("result", [])
|
||
|
||
# Filter targets
|
||
targets = [e for e in entities
|
||
if any(t in e.get("entity_id", "").lower()
|
||
for t in ["hame", "spoolman"])]
|
||
|
||
# Remove each
|
||
removed = 0
|
||
for i, entity in enumerate(targets):
|
||
await ws.send(json.dumps({
|
||
"id": 2 + i,
|
||
"type": "config/entity_registry/remove",
|
||
"entity_id": entity["entity_id"]
|
||
}))
|
||
resp = json.loads(await ws.recv())
|
||
if resp.get("success"):
|
||
removed += 1
|
||
|
||
print(f"Removed: {removed}/{len(targets)}")
|
||
await ws.close()
|
||
|
||
asyncio.run(remove_entities([]))
|
||
```
|
||
|
||
**Key point:** `config/entity_registry/remove` IS supported via WebSocket (unlike many other registry commands). It returns `{\"success\": true}` on success.
|
||
|
||
### Phase 5: Clear MQTT Discovery Topics
|
||
|
||
Without this step, MQTT discovery topics will recreate the entities on next HA restart.
|
||
|
||
```python
|
||
def publish_mqtt(topic, payload="", retain=True):
|
||
data = json.dumps({"topic": topic, "payload": payload,
|
||
"retain": retain}).encode()
|
||
req = urllib.request.Request(
|
||
f"{HA_URL}/api/services/mqtt/publish",
|
||
headers={"Authorization": f"Bearer {TOKEN}",
|
||
"Content-Type": "application/json"},
|
||
data=data, method="POST")
|
||
urllib.request.urlopen(req, timeout=10)
|
||
|
||
# Get device identifiers from device registry
|
||
# identifiers field: [["mqtt", "hame_energy_device-mac-address"]]
|
||
# Discovery topic pattern: homeassistant/<domain>/<device_id>/config
|
||
|
||
domains = ["binary_sensor", "sensor", "switch", "number",
|
||
"select", "text", "button", "update"]
|
||
|
||
for dev_id in device_identifiers:
|
||
for domain in domains:
|
||
topic = f"homeassistant/{domain}/{dev_id}/config"
|
||
publish_mqtt(topic) # Empty retained payload clears the topic
|
||
```
|
||
|
||
**Pitfall:** The REST API `mqtt/publish` returns `[]` (empty array) on success, not a confirmation object. Check HTTP 200, not response body content.
|
||
|
||
### Phase 6: Stop Source HA Addons
|
||
|
||
**Critical and easy to miss:** After removing MQTT-discovered entities, the HA addon that bridges them to MQTT may still be running and will re-create the entities. Check and stop source addons:
|
||
|
||
```bash
|
||
# List all HA addons (via qm guest exec on VM 106)
|
||
ssh -i ~/.ssh/id_ed25519_proxmox root@10.0.20.10 \
|
||
"qm guest exec 106 -- sh -c 'ha addons'"
|
||
|
||
# Stop a specific addon
|
||
ssh -i ~/.ssh/id_ed25519_proxmox root@10.0.20.10 \
|
||
"qm guest exec 106 -- sh -c 'ha addons stop <slug>'"
|
||
```
|
||
|
||
Known addon-to-integration mappings:
|
||
- `1376549a_hm2mqtt` → HAME Energy entities (must stop after HAME cleanup)
|
||
- `ccab4aaf_frigate` → Conflicts with CT-based Frigate (stop if CT Frigate is primary)
|
||
|
||
### Phase 7: Verify
|
||
|
||
Re-run the Phase 1 measurement script. Compare before/after.
|
||
|
||
## Worked Example: 2026-07-12 Cleanup
|
||
|
||
| Integration | Type | Entities | Method |
|
||
|-------------|------|----------|--------|
|
||
| HAME Energy | MQTT-discovered | 81 entities, 3 devices | WebSocket entity_registry/remove + 37 MQTT topics cleared + hm2mqtt addon stopped |
|
||
| Spoolman | Own config entry | 117 entities (2 in registry) | REST DELETE config entry + WebSocket entity removal |
|
||
| Kia UVO | Own config entry | ~5 entities | REST DELETE config entry (setup_error state) |
|
||
|
||
### Results
|
||
|
||
| Metric | Before | After | Delta |
|
||
|--------|--------|-------|-------|
|
||
| Total entities | 2,282 | 2,088 | −194 |
|
||
| Unavailable | 650 (28.5%) | 456 (21.8%) | −194 |
|
||
| Unknown | 197 (8.6%) | 198 (9.5%) | +1 |
|
||
|
||
### After Frigate Recovery (same session)
|
||
|
||
| Metric | After Cleanup | After Frigate | Delta |
|
||
|--------|--------------|---------------|-------|
|
||
| Total entities | 2,088 | 2,088 | 0 |
|
||
| Unavailable | 456 (21.8%) | 373 (17.9%) | −83 |
|
||
| Frigate cameras | 0 online | 2 recording | +2 |
|
||
| Frigate sensors | 18 dead | 93 available | +93 |
|
||
|
||
### Remaining Unavailable After Cleanup
|
||
|
||
| Group | Count | Category | Action |
|
||
|-------|-------|----------|--------|
|
||
| x40_master + staubsauger_oben | ~196 | Feature-Not-Supported | Leave (normal) |
|
||
| Double Take | 13 | Defunct (no service exists) | Disappear on HA restart |
|
||
| TRADFRI Schlafzimmer | 16 | Stale IP / VLAN | Re-add via Config Flow |
|
||
| Weather Station | 5 | Battery? | Check sensors |
|
||
|
||
## Pitfalls Encountered
|
||
|
||
1. **MQTT-discovered entities have the MQTT broker's `config_entry_id`, not their own.** Deleting the broker entry would destroy ALL MQTT devices. Always check `domain` in the config entry — if it's `mqtt`, the entities are MQTT-discovered and must be removed individually.
|
||
|
||
2. **`config/device_registry/remove` may silently fail.** The WebSocket command `config/device_registry/remove_config_entry_from_device` and `config/device_registry/remove` do not reliably succeed for MQTT-discovered devices. Focus on entity removal + MQTT topic clearing — devices with no remaining entities are cleaned up by HA automatically over time.
|
||
|
||
3. **Entity count in registry ≠ entity count in states.** The entity registry may contain more entries than `/api/states` shows (e.g., 81 HAME in registry vs 77 in states — some were already disabled/removed but kept in registry).
|
||
|
||
4. **`require_restart: false` from config entry deletion is misleading.** While the config entry is removed immediately, orphaned entities may linger in `/api/states` until HA's periodic cleanup runs. They disappear from the entity registry instantly but may show as `unavailable` in states for a short period.
|
||
|
||
5. **Deriving MQTT discovery topics from device registry identifiers requires guessing the topic format.** The `identifiers` field in the device registry (e.g., `[\"mqtt\", \"hame_energy_device-mac-address\"]`) gives the device identifier, but the actual discovery topic may use different separators or casing. Clear multiple variations to be thorough.
|
||
|
||
6. **MQTT-bridging HA addons must be stopped, not just entities removed.** After cleaning up HAME Energy MQTT-discovered entities, the `hm2mqtt` HA addon (`1376549a_hm2mqtt`) was still running and would re-discover them on next MQTT cycle. Always check for and stop the source addon: `qm guest exec 106 -- sh -c 'ha addons stop <slug>'`. Similarly, a conflicting Frigate HA addon (`ccab4aaf_frigate`) was in `error` state alongside the CT-based Frigate — stop it to avoid port/MQTT conflicts.
|
||
|
||
7. **HA addon inventory is accessible via `ha addons` CLI through qm guest exec.** The HA Supervisor REST API requires a supervisor token that differs from the normal HA long-lived token. Instead, use: `ssh -i ~/.ssh/id_ed25519_proxmox root@10.0.20.10 "qm guest exec 106 -- sh -c 'ha addons'"` — returns YAML list of all addons with slug, name, state, and version.
|