Initial commit: Hermes Agent Skills collection
This commit is contained in:
Debian
@@ -0,0 +1,121 @@
|
||||
# Retrospective Session-Log Analysis
|
||||
|
||||
Technique for applying compound-learning to past session logs — mining completed work for
|
||||
learnings that weren't captured at the time.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to "analyze recent sessions" or "apply compound-learning to session logs"
|
||||
- Periodic knowledge audit (e.g., weekly review of what was solved but not documented)
|
||||
- After a burst of activity where multiple sessions completed without individual compound-learning passes
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Enumerate Recent Sessions
|
||||
|
||||
Query the Hermes `state.db` directly (SQLite is in `~/.hermes/state.db`):
|
||||
|
||||
```sql
|
||||
SELECT id, title, source, started_at, ended_at, message_count, tool_call_count,
|
||||
estimated_cost_usd
|
||||
FROM sessions
|
||||
WHERE archived = 0
|
||||
ORDER BY started_at DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
Column `id` is the session ID (TEXT). `started_at` is a Unix REAL timestamp.
|
||||
|
||||
### Step 2: Extract Key Content Per Session
|
||||
|
||||
For each session, extract:
|
||||
- **First user message** — the task/goal
|
||||
- **Last assistant message** — the resolution/outcome
|
||||
- **Tool usage counts** — `SELECT tool_name, COUNT(*) FROM messages WHERE session_id = ? GROUP BY tool_name`
|
||||
- **User messages** (skip system preambles like `[IMPORTANT:` and `[CONTEXT COMPACTION`)
|
||||
|
||||
```sql
|
||||
SELECT id, role, content
|
||||
FROM messages
|
||||
WHERE session_id = ? AND content IS NOT NULL AND content != '' AND length(content) > 50
|
||||
ORDER BY id ASC;
|
||||
```
|
||||
|
||||
**Note:** Some assistant messages have empty `content` but populated `tool_calls` (JSON). If
|
||||
content-based extraction yields thin results, check `tool_calls` column for the actual work.
|
||||
|
||||
### Step 3: Classify Each Session
|
||||
|
||||
Apply compound-learning Phase 2 classification:
|
||||
|
||||
| Signal | Type | Worth a solution doc? |
|
||||
|---|---|---|
|
||||
| Bug fixed, root cause found | `bug-fix` | Yes, if non-trivial |
|
||||
| Architecture decision made | `architecture-pattern` | Yes |
|
||||
| Tool/library chosen or configured | `tooling-decision` | Yes |
|
||||
| Multi-step workflow executed | `workflow` | Yes, if reusable |
|
||||
| Third-party API/device integrated | `integration` | Yes |
|
||||
| Price search, research, Q&A | — | Usually no (one-off) |
|
||||
| Cron job ran successfully | — | Only if new failure mode discovered |
|
||||
|
||||
**Skip sessions that are:** routine cron runs with no errors, pure research/Q&A, or
|
||||
sessions that ended inconclusively without a resolution.
|
||||
|
||||
### Step 4: Write Solution Docs
|
||||
|
||||
Follow compound-learning Phases 3-6 for each qualifying session:
|
||||
1. Overlap check: `search_files("keyword", path="docs/solutions/")`
|
||||
2. Write to `docs/solutions/{type}/{YYYY-MM-DD}-{slug}.md` with YAML frontmatter
|
||||
3. Cross-reference related docs (e.g., HA token expiry → HA SSH addon integration)
|
||||
4. Include the session ID in the References section for traceability
|
||||
|
||||
### Step 5: Hindsight Sync (Phase 4.5)
|
||||
|
||||
**Check daemon health FIRST:**
|
||||
```bash
|
||||
curl -s http://127.0.0.1:9177/health
|
||||
```
|
||||
|
||||
If healthy: batch `hindsight_retain` for all solution docs.
|
||||
If down: skip Hindsight sync — `docs/solutions/` files are sufficient alone.
|
||||
|
||||
### Step 6: Self-Improvement Routing (Phase 7)
|
||||
|
||||
Review all solution docs collectively for systemic gaps:
|
||||
- Does a skill need a patch? (e.g., "always try SSH addon first for HA")
|
||||
- Does MEMORY.md need a durable fact? (check for overflow first!)
|
||||
- Is there a recurring failure pattern across multiple sessions?
|
||||
|
||||
Present SI suggestions as a batch table to the user.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Don't write a solution doc for every session** — routine cron runs and one-off research don't qualify
|
||||
- **Include session IDs in References** — enables future `session_search(session_id=...)` deep-dives
|
||||
- **Cross-reference related solutions** — e.g., "HA token expiry" and "HA SSH addon integration" should link to each other
|
||||
- **MEMORY.md overflow** — batch SI suggestions may collectively exceed the 2,200 char limit. Check current usage before proposing multiple memory additions.
|
||||
- **Empty content fields** — some assistant messages store work in `tool_calls` JSON, not `content`. If extraction looks thin, query `tool_calls` as well.
|
||||
|
||||
## SQLite Schema Reference
|
||||
|
||||
```sql
|
||||
-- sessions table
|
||||
id TEXT PRIMARY KEY -- session ID
|
||||
source TEXT -- 'telegram', 'cron', etc.
|
||||
started_at REAL -- Unix timestamp
|
||||
ended_at REAL -- Unix timestamp (nullable)
|
||||
message_count INTEGER
|
||||
tool_call_count INTEGER
|
||||
estimated_cost_usd REAL
|
||||
title TEXT -- session title (nullable)
|
||||
archived INTEGER DEFAULT 0
|
||||
|
||||
-- messages table
|
||||
id INTEGER PRIMARY KEY
|
||||
session_id TEXT -- FK to sessions.id
|
||||
role TEXT -- 'user', 'assistant', 'tool'
|
||||
content TEXT -- message text (may be empty for tool-call-only turns)
|
||||
tool_calls TEXT -- JSON array of tool calls
|
||||
tool_name TEXT -- tool name (for role='tool' rows)
|
||||
timestamp REAL
|
||||
```
|
||||
@@ -0,0 +1,115 @@
|
||||
# Self-Improvement Loop Architecture
|
||||
|
||||
The closed-loop system connecting compound-learning, Hindsight, and skill/memory self-improvement.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PROBLEM SOLVED │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ compound-learning │
|
||||
│ ├─ Phase 1-3: Identify, classify, overlap-check │
|
||||
│ ├─ Phase 4: Write solution doc → docs/solutions/{type}/ │
|
||||
│ ├─ Phase 4.5: Hindsight sync (hindsight_retain) │
|
||||
│ │ └─ Summary + tags + source path → PostgreSQL │
|
||||
│ ├─ Phase 5-6: CONCEPTS.md, discoverability │
|
||||
│ └─ Phase 7: Self-Improvement Routing │
|
||||
│ ├─ Skill gap? → skill_manage(patch) [ASK USER] │
|
||||
│ ├─ Memory gap? → memory(add) │
|
||||
│ ├─ Instruction file gap? → patch(file) [ASK USER] │
|
||||
│ └─ Track: hindsight_retain(tags=["self-improvement"]) │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ GROUNDING SCANS (next session) │
|
||||
│ ├─ brainstorming Step 1.5: search_files + hindsight_recall │
|
||||
│ ├─ plan Step 1.5: search_files + hindsight_recall │
|
||||
│ └─ writing-plans Step 1.5: search_files + hindsight_recall │
|
||||
│ └─ Semantic search finds solutions even when keywords │
|
||||
│ don't match (e.g. "Galera bootstrap" → "SST Recovery")│
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ META-MEMORY AUTOMATION (cronjobs) │
|
||||
│ │
|
||||
│ Nightly Dream (0 3 * * *) │
|
||||
│ ├─ Contradiction detection │
|
||||
│ ├─ Blocked/stale TODOs │
|
||||
│ ├─ Outdated facts │
|
||||
│ └─ Self-improvement: learnings vs. existing skills │
|
||||
│ │
|
||||
│ Weekly Digest (0 8 * * 0) │
|
||||
│ ├─ Recurring patterns │
|
||||
│ ├─ Infra changes │
|
||||
│ ├─ Open TODOs │
|
||||
│ ├─ Recommendations │
|
||||
│ ├─ Solution patterns (compound-learning) │
|
||||
│ └─ Self-improvement audit (open vs. acted-on) │
|
||||
│ │
|
||||
│ Monthly Cleanup (0 3 1 * *) │
|
||||
│ └─ Dedup + decay report │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Three-Tier Memory Strategy
|
||||
|
||||
| Layer | Scope | Capacity | Retrieval |
|
||||
|---|---|---|---|
|
||||
| MEMORY.md | Hot facts, TODOs, quick refs | ~2,200 chars | Injected every turn (always visible) |
|
||||
| Hindsight (PostgreSQL) | Unlimited detail, history, context | Unbounded | Semantic search (auto-recall) |
|
||||
| `docs/solutions/` | Structured problem-solution docs | Unbounded (files) | `search_files` (keyword) + `hindsight_recall` (semantic via Phase 4.5) |
|
||||
|
||||
**Routing rule:** One fact, one layer. Don't duplicate.
|
||||
- Short pointers → MEMORY.md
|
||||
- Full detail → Hindsight
|
||||
- Structured solutions (root cause + fix + lesson + prevention) → `docs/solutions/` + Hindsight summary
|
||||
- Recurring procedures → Skills
|
||||
- Credentials → 1Password
|
||||
|
||||
## Phase 4.5: Hindsight Sync
|
||||
|
||||
After writing a solution doc, sync a compact summary to Hindsight:
|
||||
|
||||
```python
|
||||
hindsight_retain(
|
||||
content=f"SOLUTION [{problem_type}]: {problem_summary}. "
|
||||
f"Root cause: {root_cause}. Fix: {fix}. "
|
||||
f"Lesson: {lesson}. Tags: {tags}. "
|
||||
f"Source: docs/solutions/{type}/{date}-{slug}.md",
|
||||
tags=["solution"] + tags
|
||||
)
|
||||
```
|
||||
|
||||
Graceful degradation: if Hindsight daemon is down (`curl -s http://127.0.0.1:9177/health` fails),
|
||||
the file in `docs/solutions/` is sufficient on its own.
|
||||
|
||||
## Phase 7: Self-Improvement Routing
|
||||
|
||||
Decision tree for when a learning reveals a system gap:
|
||||
|
||||
| Learning implies... | Action | Tool | Mode |
|
||||
|---|---|---|---|
|
||||
| Skill has wrong/missing/outdated step | Patch skill | `skill_manage(action='patch')` | Ask user |
|
||||
| No skill for recurring workflow | Propose new skill | `skill_manage(action='create')` | Ask user |
|
||||
| Durable environment fact | Update MEMORY.md | `memory` tool | Just do it |
|
||||
| Tool quirk/workaround | Update MEMORY.md | `memory` tool | Just do it |
|
||||
| Convention change (AGENTS.md/CLAUDE.md) | Patch instruction file | `patch` tool | Ask user |
|
||||
|
||||
Tracking: every SI suggestion is stored via `hindsight_retain(tags=["self-improvement"])` so the
|
||||
weekly digest can audit what was acted on vs. what's still open.
|
||||
|
||||
## Files Modified (Integration Points)
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `compound-learning/SKILL.md` | Phase 4.5 (Hindsight sync), Phase 7 (SI routing), updated integrations + frontmatter |
|
||||
| `brainstorming/SKILL.md` | Step 1.5 Grounding Scan: + `hindsight_recall` |
|
||||
| `plan/SKILL.md` | Step 1.5 Grounding Scan: + `hindsight_recall` |
|
||||
| `writing-plans/SKILL.md` | Step 1.5 Grounding Scan: + `hindsight_recall` |
|
||||
| `optimize-loops/SKILL.md` | Phase 5: note about Phase 4.5 Hindsight sync |
|
||||
| `hindsight/SKILL.md` | Three-tier strategy, SI loop documentation, cronjob descriptions updated |
|
||||
| `hindsight/scripts/hindsight_nightly_dream.py` | Check #4: self-improvement suggestions |
|
||||
| `hindsight/scripts/hindsight_weekly_digest.py` | Section #5: solution patterns, #6: SI audit |
|
||||
Reference in New Issue
Block a user