217 lines
6.5 KiB
Markdown
217 lines
6.5 KiB
Markdown
---
|
|
name: memory-sync
|
|
description: "Set up and maintain a markdown-based personal memory system with qmd semantic search and Git/Gitea sync. Based on Karpathy's LLM Wiki pattern. Covers wiki structure, qmd search setup, git sync, and maintenance protocol."
|
|
version: 2.0.0
|
|
category: research
|
|
tags: [wiki, memory, qmd, gitea, git-sync, knowledge-base, markdown, semantic-search]
|
|
---
|
|
|
|
# Memory Sync
|
|
|
|
Set up and maintain a markdown-based personal memory system using qmd for semantic search and Git/Gitea for version control.
|
|
|
|
This umbrella skill covers:
|
|
- **Wiki setup and maintenance** — directory structure, git sync, session log protocol
|
|
- **qmd semantic search** — BM25, vector embeddings, LLM reranking for the wiki
|
|
- **MCP server integration** (optional) — HTTP transport for LLM tool access
|
|
|
|
## Quick Setup
|
|
|
|
### 1. Install qmd (Query Markup Documents)
|
|
|
|
```bash
|
|
# Check for Node.js
|
|
node --version 2>/dev/null || echo "Not installed"
|
|
|
|
# If not installed, download Node.js for x86_64:
|
|
curl -fsSL https://nodejs.org/dist/v20.11.0/node-v20.11.0-linux-x64.tar.xz -o /tmp/node.tar.xz
|
|
tar -xJf /tmp/node.tar.xz -C /tmp/
|
|
export PATH="/tmp/node-v20.11.0-linux-x64/bin:$PATH"
|
|
|
|
# Install qmd
|
|
npm install -g @tobilu/qmd
|
|
```
|
|
|
|
### 2. Create Memory Directory
|
|
|
|
```bash
|
|
mkdir -p ~/.hermes/memory
|
|
cd ~/.hermes/memory
|
|
|
|
# Add as qmd collection
|
|
qmd collection add ~/.hermes/memory --name memory
|
|
|
|
# Add context for better search results
|
|
qmd context add qmd://memory "Personal knowledge base using Karpathy LLM Wiki pattern"
|
|
|
|
# Generate embeddings (downloads model on first run, ~250MB)
|
|
qmd embed
|
|
```
|
|
|
|
### 3. Set Up Git Sync
|
|
|
|
```bash
|
|
cd ~/.hermes/memory
|
|
git init
|
|
git config user.name "Your Name"
|
|
git config user.email "your@email.com"
|
|
|
|
# Add remote (Gitea HTTPS with token auth)
|
|
git remote add origin "https://USER:TOKEN@git.familie-schoen.com/USER/memory.git"
|
|
|
|
# Push to main branch
|
|
git branch -m master main 2>/dev/null
|
|
git push -u origin main
|
|
```
|
|
|
|
### 4. Create Initial Structure
|
|
|
|
```
|
|
~hermes/memory/
|
|
├── index.md # Content catalog
|
|
├── log.md # Chronological log (append-only)
|
|
├── Entities/
|
|
│ └── *.md # Entity pages (person, system, tool)
|
|
├── Concepts/
|
|
│ └── *.md # Concept pages (architecture, workflow)
|
|
├── Projects/ # Empty — future project pages
|
|
├── Templates/ # Empty — future templates
|
|
└── memory-entry-NNN.md # Session log entries (numbered)
|
|
```
|
|
|
|
The index.md links to subdirectory pages using relative paths: `[[Entities/Infrastructure.md]]`.
|
|
|
|
## Maintenance Protocol (CRITICAL)
|
|
|
|
**The memory must be kept current at ALL times. This is not optional.**
|
|
|
|
### Start of Session
|
|
1. Read `log.md` and check `git status` to see what was last synced
|
|
2. Read `index.md` to understand current state
|
|
3. If memory is stale, correct it before proceeding with any task
|
|
4. Use `hindsight_recall` or direct file reads to verify facts before answering
|
|
|
|
### End of Session
|
|
1. Append to `log.md` with today's changes
|
|
2. Create `memory-entry-NNN.md` (next available number) summarizing the session
|
|
3. Update `index.md` if new entities/concepts were created
|
|
4. Commit and push:
|
|
```bash
|
|
cd ~/.hermes/memory && git add . && git commit -m "Update: brief description" && git push
|
|
```
|
|
5. Verify push succeeded before ending the session
|
|
|
|
### Never
|
|
- Do NOT use `hindsight_retain` for memory files — the daemon is not available and will silently fail
|
|
- Do NOT commit without pushing — unsynced memory is stale memory
|
|
- Do NOT let memory files go stale between sessions
|
|
|
|
## Sync Workflow
|
|
|
|
### Manual Sync
|
|
```bash
|
|
cd ~/.hermes/memory
|
|
git add .
|
|
git commit -m "Description" && git push 2>/dev/null || echo "No changes"
|
|
qmd embed 2>/dev/null # Re-index new files
|
|
```
|
|
|
|
### Automatic Sync (cron — script-only, no agent)
|
|
For purely git-based sync operations, use `--no-agent` to skip the LLM entirely (saves time/cost):
|
|
```bash
|
|
hermes cron create \
|
|
--name "memory-sync-daily" \
|
|
--schedule "0 22 * * *" \
|
|
--script 'cd ~/.hermes/memory && git add . && git commit -m "Auto-sync: $(date +%%Y-%%m-%%d)" && git push origin main' \
|
|
--no-agent
|
|
```
|
|
|
|
## Memory Entry Format
|
|
|
|
Use numbered entries for session logs:
|
|
```markdown
|
|
# Memory Entry 001 - System Initialization
|
|
|
|
## Date
|
|
2026-04-28
|
|
|
|
## Summary
|
|
Brief description of what was done.
|
|
|
|
## Details
|
|
- [x] Task 1
|
|
- [x] Task 2
|
|
- [ ] Task 3 (pending)
|
|
|
|
## Status
|
|
- Completed items
|
|
- Pending items
|
|
|
|
## Notes
|
|
Additional context or questions.
|
|
```
|
|
|
|
## Pitfalls
|
|
|
|
- **qmd embed may not index new files** — if files were added after collection creation, recreate the collection: `qmd collection rm memory && qmd collection add ~/.hermes/memory --name memory && qmd embed`
|
|
- **First qmd embed downloads a model** — ~250MB, may take minutes
|
|
- **Token scope** — Gitea tokens need `write:repository` scope for push access
|
|
- **Branch naming** — Gitea repos default to `main`, local git may create `master` — rename with `git branch -m master main`
|
|
- **HTTPS vs SSH** — When SSH is blocked, use HTTPS with token in URL: `https://USER:TOKEN@git.server.com/USER/repo.git`
|
|
- **hindsight_retain silently fails** — the daemon is not available; use direct file edits + git instead
|
|
- **Memory goes stale between sessions** — always start by reading log.md and checking git status
|
|
|
|
## When to Use
|
|
|
|
- Setting up a new memory/knowledge system
|
|
- Adding semantic search to a markdown wiki
|
|
- Syncing memory files to Gitea/GitHub
|
|
- Creating structured memory entries after sessions
|
|
- Maintaining up-to-date knowledge base across sessions
|
|
|
|
## Search Commands
|
|
|
|
```bash
|
|
# Fast keyword search (BM25)
|
|
qmd search "authentication"
|
|
|
|
# Semantic search (embedding-based)
|
|
qmd vsearch "how to deploy"
|
|
|
|
# Hybrid + LLM reranking (best quality)
|
|
qmd query "quarterly planning process"
|
|
|
|
# Get a specific document
|
|
qmd get "meetings/2024-01-15.md"
|
|
|
|
# Search within a specific collection
|
|
qmd search "API" -c wiki
|
|
|
|
# JSON output for agents
|
|
qmd search "authentication" --json -n 10
|
|
```
|
|
|
|
## MCP Server (Optional)
|
|
|
|
```bash
|
|
# Start MCP server (HTTP transport)
|
|
qmd mcp --http
|
|
|
|
# Start as daemon
|
|
qmd mcp --http --daemon
|
|
|
|
# Stop
|
|
qmd mcp stop
|
|
|
|
# Check status
|
|
qmd status
|
|
```
|
|
|
|
Configure MCP clients to connect to `http://localhost:8181/mcp`.
|
|
|
|
## Pitfalls
|
|
|
|
- **qmd embed may not index new files** — if files were added after collection creation, recreate: `qmd collection rm wiki && qmd collection add $WIKI --name wiki && qmd embed`
|
|
- **First qmd embed downloads model** — ~250MB, may take minutes
|
|
- **Node.js version** — qmd requires Node.js 20.17.0+ for some deps
|
|
- **Memory usage** — ~1GB RAM when model is loaded |