# HA Recorder Silent Connection Drop

## Symptom

HA shows no historical data (History/Logbook empty or stale), but HA itself
runs normally — entities update, automations fire, no error logs.

## Root Cause

HA Recorder loses its MySQL/Galera DB connection silently. Unlike the
exhausted-retries-at-boot case, no `ERROR (Recorder)` entries appear in
the HA log. The recorder thread is alive but disconnected — it never
reconnects on its own.

Observed triggers:
- MaxScale `FLUSH HOSTS` during Galera maintenance
- Galera node restart (brief connection interruption)
- MaxScale VM failover (VIP moves to standby)

## Diagnosis Procedure

### 1. Check HA logs for recorder errors

```bash
ssh -i ~/.ssh/id_ed25519_proxmox -p 22222 root@10.0.30.10 \
  "docker logs homeassistant 2>&1 | grep -i recorder | tail -20"
```

If EMPTY — this is the silent-drop pattern (not the exhausted-retries case,
which floods the log with `ERROR (Recorder)` messages).

### 2. Check MaxScale for active HA sessions

```bash
ssh -i ~/.ssh/id_ed25519_proxmox debian@10.0.30.82 "maxctrl list sessions"
```

If `ha_recorder` is NOT listed among active sessions, the recorder
connection is dead. Only `recipe_app` or other apps may be connected.

### 3. Check `recorder_runs` table on Galera

```bash
ssh -i ~/.ssh/id_ed25519_proxmox debian@10.0.30.73 \
  "sudo mariadb homeassistant -e 'SELECT run_id, start, \`end\` FROM recorder_runs ORDER BY run_id DESC LIMIT 3\G'"
```

An open run (`end = NULL`) with an old `start` timestamp confirms the
recorder started but never closed the run (connection dropped mid-run).

### 4. Check latest state timestamp

```bash
ssh -i ~/.ssh/id_ed25519_proxmox debian@10.0.30.73 \
  "sudo mariadb homeassistant -e 'SELECT FROM_UNIXTIME(MAX(last_updated_ts)) AS latest_state FROM states;'"
```

Compare with current time. If `latest_state` is hours/days behind, recorder
stopped writing at that point.

### 5. Check `states` table size (information_schema — fast, no COUNT(*))

```bash
ssh -i ~/.ssh/id_ed25519_proxmox debian@10.0.30.73 \
  "sudo mariadb homeassistant -e 'SELECT table_name, table_rows, ROUND(data_length/1024/1024) AS data_mb FROM information_schema.tables WHERE table_schema=\"homeassistant\";'"
```

**⚠️ Do NOT run `SELECT COUNT(*) FROM states`** — with 62M+ rows and 6.6 GB,
this query hangs for 30+ seconds. Use `information_schema.table_rows`
instead (instant, approximate).

## Fix

```bash
# Restart HA core — recorder reconnects and resumes writing
ssh -i ~/.ssh/id_ed25519_proxmox -p 22222 root@10.0.30.10 "ha core restart"
```

Wait 2-3 minutes for HA to fully start (large DB schema check takes time).

## Verification

```bash
# Check latest state is now current
ssh -i ~/.ssh/id_ed25519_proxmox debian@10.0.30.73 \
  "sudo mariadb homeassistant -e 'SELECT FROM_UNIXTIME(MAX(last_updated_ts)) AS latest_state FROM states;'"
# Should show a timestamp within the last few seconds

# Check MaxScale now has ha_recorder session
ssh -i ~/.ssh/id_ed25519_proxmox debian@10.0.30.82 "maxctrl list sessions"
```

## Data Loss

The gap between the silent disconnect and the restart is permanently lost.
No state/event data was recorded during that period. InfluxDB (if configured
for HA) may still have the data since it's a separate integration with its
own connection.

## Session Reference

- **2026-07-08**: Recorder silently dropped after MaxScale `FLUSH HOSTS`
  during Galera maintenance. Latest state was 2026-07-04 12:48 (4 days gap).
  62.7M states, 6.6 GB on Galera. `ha core restart` fixed immediately.
  `recorder_runs` run_id=106 open since 2026-07-01, `end=NULL`.