122 lines
4.7 KiB
Markdown
122 lines
4.7 KiB
Markdown
# Retrospective Session-Log Analysis
|
|
|
|
Technique for applying compound-learning to past session logs — mining completed work for
|
|
learnings that weren't captured at the time.
|
|
|
|
## When to Use
|
|
|
|
- User asks to "analyze recent sessions" or "apply compound-learning to session logs"
|
|
- Periodic knowledge audit (e.g., weekly review of what was solved but not documented)
|
|
- After a burst of activity where multiple sessions completed without individual compound-learning passes
|
|
|
|
## Process
|
|
|
|
### Step 1: Enumerate Recent Sessions
|
|
|
|
Query the Hermes `state.db` directly (SQLite is in `~/.hermes/state.db`):
|
|
|
|
```sql
|
|
SELECT id, title, source, started_at, ended_at, message_count, tool_call_count,
|
|
estimated_cost_usd
|
|
FROM sessions
|
|
WHERE archived = 0
|
|
ORDER BY started_at DESC
|
|
LIMIT 10;
|
|
```
|
|
|
|
Column `id` is the session ID (TEXT). `started_at` is a Unix REAL timestamp.
|
|
|
|
### Step 2: Extract Key Content Per Session
|
|
|
|
For each session, extract:
|
|
- **First user message** — the task/goal
|
|
- **Last assistant message** — the resolution/outcome
|
|
- **Tool usage counts** — `SELECT tool_name, COUNT(*) FROM messages WHERE session_id = ? GROUP BY tool_name`
|
|
- **User messages** (skip system preambles like `[IMPORTANT:` and `[CONTEXT COMPACTION`)
|
|
|
|
```sql
|
|
SELECT id, role, content
|
|
FROM messages
|
|
WHERE session_id = ? AND content IS NOT NULL AND content != '' AND length(content) > 50
|
|
ORDER BY id ASC;
|
|
```
|
|
|
|
**Note:** Some assistant messages have empty `content` but populated `tool_calls` (JSON). If
|
|
content-based extraction yields thin results, check `tool_calls` column for the actual work.
|
|
|
|
### Step 3: Classify Each Session
|
|
|
|
Apply compound-learning Phase 2 classification:
|
|
|
|
| Signal | Type | Worth a solution doc? |
|
|
|---|---|---|
|
|
| Bug fixed, root cause found | `bug-fix` | Yes, if non-trivial |
|
|
| Architecture decision made | `architecture-pattern` | Yes |
|
|
| Tool/library chosen or configured | `tooling-decision` | Yes |
|
|
| Multi-step workflow executed | `workflow` | Yes, if reusable |
|
|
| Third-party API/device integrated | `integration` | Yes |
|
|
| Price search, research, Q&A | — | Usually no (one-off) |
|
|
| Cron job ran successfully | — | Only if new failure mode discovered |
|
|
|
|
**Skip sessions that are:** routine cron runs with no errors, pure research/Q&A, or
|
|
sessions that ended inconclusively without a resolution.
|
|
|
|
### Step 4: Write Solution Docs
|
|
|
|
Follow compound-learning Phases 3-6 for each qualifying session:
|
|
1. Overlap check: `search_files("keyword", path="docs/solutions/")`
|
|
2. Write to `docs/solutions/{type}/{YYYY-MM-DD}-{slug}.md` with YAML frontmatter
|
|
3. Cross-reference related docs (e.g., HA token expiry → HA SSH addon integration)
|
|
4. Include the session ID in the References section for traceability
|
|
|
|
### Step 5: Hindsight Sync (Phase 4.5)
|
|
|
|
**Check daemon health FIRST:**
|
|
```bash
|
|
curl -s http://127.0.0.1:9177/health
|
|
```
|
|
|
|
If healthy: batch `hindsight_retain` for all solution docs.
|
|
If down: skip Hindsight sync — `docs/solutions/` files are sufficient alone.
|
|
|
|
### Step 6: Self-Improvement Routing (Phase 7)
|
|
|
|
Review all solution docs collectively for systemic gaps:
|
|
- Does a skill need a patch? (e.g., "always try SSH addon first for HA")
|
|
- Does MEMORY.md need a durable fact? (check for overflow first!)
|
|
- Is there a recurring failure pattern across multiple sessions?
|
|
|
|
Present SI suggestions as a batch table to the user.
|
|
|
|
## Pitfalls
|
|
|
|
- **Don't write a solution doc for every session** — routine cron runs and one-off research don't qualify
|
|
- **Include session IDs in References** — enables future `session_search(session_id=...)` deep-dives
|
|
- **Cross-reference related solutions** — e.g., "HA token expiry" and "HA SSH addon integration" should link to each other
|
|
- **MEMORY.md overflow** — batch SI suggestions may collectively exceed the 2,200 char limit. Check current usage before proposing multiple memory additions.
|
|
- **Empty content fields** — some assistant messages store work in `tool_calls` JSON, not `content`. If extraction looks thin, query `tool_calls` as well.
|
|
|
|
## SQLite Schema Reference
|
|
|
|
```sql
|
|
-- sessions table
|
|
id TEXT PRIMARY KEY -- session ID
|
|
source TEXT -- 'telegram', 'cron', etc.
|
|
started_at REAL -- Unix timestamp
|
|
ended_at REAL -- Unix timestamp (nullable)
|
|
message_count INTEGER
|
|
tool_call_count INTEGER
|
|
estimated_cost_usd REAL
|
|
title TEXT -- session title (nullable)
|
|
archived INTEGER DEFAULT 0
|
|
|
|
-- messages table
|
|
id INTEGER PRIMARY KEY
|
|
session_id TEXT -- FK to sessions.id
|
|
role TEXT -- 'user', 'assistant', 'tool'
|
|
content TEXT -- message text (may be empty for tool-call-only turns)
|
|
tool_calls TEXT -- JSON array of tool calls
|
|
tool_name TEXT -- tool name (for role='tool' rows)
|
|
timestamp REAL
|
|
```
|