Initial commit: Hermes Agent Skills collection
This commit is contained in:
Debian
@@ -0,0 +1,202 @@
|
||||
# Seafile & Immich Backup Strategy Analysis
|
||||
|
||||
## When to Use
|
||||
|
||||
When planning offsite/backup strategy for Seafile (558 GB) or Immich
|
||||
on bandwidth-limited remote PBS, and deciding what to back up vs.
|
||||
what to leave on Ceph-only redundancy.
|
||||
|
||||
## Seafile Backup Strategies (6 Options)
|
||||
|
||||
### Context
|
||||
|
||||
- Seafile Pro 13.0.19 MC Docker edition on CT111
|
||||
- File blocks: `/opt/seafile-data/seafile/seafile-data/storage/blocks/` (intrinsically deduplicated, content-addressable)
|
||||
- Config: `/opt/seafile/.env`, `seafile.conf`, `seahub_settings.py`
|
||||
- DB: MySQL (`seafile_db`, `ccnet_db`, `seahub_db`) — currently in `seafile-mysql` container, migration to Galera planned
|
||||
- Total size: ~558 GB (mostly file blocks)
|
||||
- Ceph redundancy: blocks stored on Ceph EC4+1 (5 HDD OSDs, 1-failure tolerance)
|
||||
|
||||
### Option Matrix
|
||||
|
||||
| # | Strategy | Offsite Volume | Pros | Cons |
|
||||
|---|----------|---------------|------|------|
|
||||
| 1 | **PBS Full (current)** | 558 GB init, then delta | Simple, everything included | Huge for offsite, slow restore |
|
||||
| 2 | **Config+DB only** | <1 GB | Fast daily offsite, trivial restore | Blocks only in Ceph — total Ceph loss = all files gone |
|
||||
| 3 | **SeafGC + PBS exclude blocks** | ~2 GB | Clean, regularly garbage-collected | Blocks only in Ceph, needs GC discipline |
|
||||
| 4 | **Seafile Native Backup** | Variable | Official way, incremental | Needs second Seafile server (not S3) |
|
||||
| 5 | **RBD Snapshot + PBS** | 558 GB init, delta like #1 | Point-in-time consistent | Ceph pool must support snapshots |
|
||||
| 6 | **Blocks on S3 (Pro feature)** | Rootfs ~2 GB offsite, blocks on S3 | Clean separation, Ceph relieved | Migration effort, S3 costs |
|
||||
|
||||
### Recommendation
|
||||
|
||||
- **Short-term**: Option 2 (Config+DB offsite) + regular `seaf-gc.sh`. Blocks
|
||||
are redundant in Ceph EC4+1. For DR against total Ceph loss: Option 4 as
|
||||
second pillar.
|
||||
- **Long-term**: Option 6 (S3 storage backend) — moves blocks out of Ceph
|
||||
entirely, making PBS offsite trivial (~2 GB) and Ceph only hosts config+DB.
|
||||
|
||||
## Seafile Pro S3 Storage Backend
|
||||
|
||||
Seafile Pro supports S3 as **primary storage backend** (not backup target).
|
||||
All file blocks are stored directly on S3 instead of local disk/Ceph.
|
||||
|
||||
### Configuration (in `seafile.conf`)
|
||||
|
||||
```ini
|
||||
[storage_backend]
|
||||
name = s3
|
||||
bucket = seafile-data
|
||||
key_id = XXX
|
||||
key = XXX
|
||||
region = eu-central-1
|
||||
endpoint = https://s3.example.com
|
||||
```
|
||||
|
||||
### Key Distinctions
|
||||
|
||||
- **Primary storage** — replaces local/Ceph block storage entirely
|
||||
- **NOT a backup target** — S3 is where live data lives, not a backup copy
|
||||
- **Migration required** — existing blocks must be migrated to S3
|
||||
- **Ceph relief** — once on S3, Ceph only holds config+DB (~2 GB), making
|
||||
PBS offsite backup trivial and fast
|
||||
|
||||
### When to Choose S3 Backend
|
||||
|
||||
- Ceph capacity is constrained (OSDs near full)
|
||||
- Offsite backup of 558 GB blocks is impractical (bandwidth-limited)
|
||||
- S3 storage is cheaper than maintaining Ceph OSDs
|
||||
- Want clean separation: config+DB on Ceph (small, fast), blocks on S3 (large, cheap)
|
||||
|
||||
## Seafile Native Backup (`seaf-backup-cmd`)
|
||||
|
||||
### How It Works
|
||||
|
||||
- Uses Seafile's internal RPC API to synchronize libraries to a **second
|
||||
Seafile server**
|
||||
- Incremental — only changed commits/blocks are transferred
|
||||
- The second server must be a full Seafile server (same version)
|
||||
- Commands: `seaf-backup-cmd status`, `seaf-backup-cmd sync <repo-id>`
|
||||
|
||||
### Limitations
|
||||
|
||||
- **Needs a second Seafile server** — not a simple file/rsync target
|
||||
- **Not S3-compatible** — target must be Seafile server with storage backend
|
||||
- **Setup complexity** — must configure backup server in `seafile.conf`
|
||||
- **Per-library sync** — no global "backup everything" by default
|
||||
|
||||
### Location
|
||||
|
||||
```bash
|
||||
# In the seafile container:
|
||||
docker exec seafile /opt/seafile/seafile-server-latest/seahub/scripts/seaf-backup-cmd.sh
|
||||
# Error without proper SEAFILE_DATA_DIR env — needs server running
|
||||
```
|
||||
|
||||
### When to Choose
|
||||
|
||||
- Need true DR replica (second running Seafile instance)
|
||||
- Have infrastructure for a second server
|
||||
- Want incremental, deduplicated replication (not full copies)
|
||||
|
||||
## Immich Backup Considerations
|
||||
|
||||
### Architecture
|
||||
|
||||
- CT115 (immich): stopped, 20 GB rootfs — upload directory location unclear
|
||||
- CT117 (immich-postgresql): running, 914 MB DB on proxmox6
|
||||
- Photos/videos are NOT deduplicated (unlike Seafile blocks)
|
||||
|
||||
### Strategy Options
|
||||
|
||||
| # | Strategy | Offsite Volume | Notes |
|
||||
|---|----------|---------------|-------|
|
||||
| 1 | PBS Full both CTs | ~21 GB | Simple but may grow |
|
||||
| 2 | DB dump + upload-dir rsync | DB ~1 GB + uploads | Selective |
|
||||
| 3 | DB only offsite | <1 GB | Photos only in Ceph |
|
||||
| 4 | Uploads on Ceph, DB on Galera | DB offsite <1 GB | Clean separation |
|
||||
|
||||
### Key Question
|
||||
|
||||
Where are the actual photo/video uploads stored? CT115 is stopped —
|
||||
uploads may be on a mount point, external volume, or within the
|
||||
PostgreSQL CT. Must identify before choosing a strategy.
|
||||
|
||||
## Bandwidth-Aware Offsite Backup Prioritization
|
||||
|
||||
When adding services to a bandwidth-limited offsite PBS:
|
||||
|
||||
### Tier 1: Small + Critical Infrastructure (add first)
|
||||
|
||||
- Traefik (~3 GB) — reverse proxy config, all routes
|
||||
- Gitea (~1.2 GB) — code repositories
|
||||
- Authelia, DNS — tiny but critical
|
||||
- Paperless (~16 GB) — documents (medium size, high value)
|
||||
|
||||
### Tier 2: Medium + Important
|
||||
|
||||
- Litellm (~3.4 GB) — API config
|
||||
- Seafile config+DB (~2 GB) — without blocks
|
||||
|
||||
### Tier 3: Evaluate Separately
|
||||
|
||||
- Seafile blocks (~558 GB) — consider S3 backend or native backup
|
||||
- Immich uploads — identify storage location first
|
||||
|
||||
### Skip
|
||||
|
||||
- Frigate (~40 GB) — rewritable video, low value
|
||||
- High-churn data that's easily regenerated
|
||||
|
||||
## PBS Offsite Job Configuration
|
||||
|
||||
### Adding CTs to Offsite Job
|
||||
|
||||
```bash
|
||||
# Edit /etc/pve/jobs.cfg from any PVE node (pmxcfs replicates)
|
||||
# Current: vmid 106
|
||||
# Target: vmid 106,108,104,99999
|
||||
sed -i '/offsite-ha-daily/,/^$/{s/vmid 106/vmid 106,108,104,99999/}' /etc/pve/jobs.cfg
|
||||
```
|
||||
|
||||
### Initial Full Backup (Manual)
|
||||
|
||||
Must run `vzdump` on the node where each CT actually runs (see
|
||||
`proxmox-ve-administration` → `references/pbs-lxc-setup-2026-07.md`
|
||||
pitfall #17 for the silent-skip issue):
|
||||
|
||||
```bash
|
||||
# Determine node for each CT first
|
||||
pvesh get /cluster/resources --type vm --output-format json | python3 -c "
|
||||
import json,sys
|
||||
for v in json.load(sys.stdin):
|
||||
if v.get('vmid') in [108,99999,104]:
|
||||
print(f\"CT{v['vmid']:5d} node={v['node']}\")"
|
||||
|
||||
# Run on correct nodes
|
||||
ssh root@10.0.20.60 "vzdump 108 --storage noris_offsite --mode snapshot --compress zstd"
|
||||
ssh root@10.0.20.70 "vzdump 99999 --storage noris_offsite --mode snapshot --compress zstd"
|
||||
ssh root@10.0.20.91 "vzdump 104 --storage noris_offsite --mode snapshot --compress zstd"
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# Check backups arrived at PBS (may need content cache refresh)
|
||||
pvesh get /nodes/proxmox1/storage/noris_offsite/content --output-format json | \
|
||||
python3 -c "
|
||||
import json,sys,datetime
|
||||
for b in sorted(json.load(sys.stdin), key=lambda x: x.get('ctime',0), reverse=True)[:10]:
|
||||
print(f\"{b['volid']:55s} {b.get('size',0)//1024//1024:>8} MB {datetime.datetime.fromtimestamp(b.get('ctime',0)).strftime('%Y-%m-%d %H:%M')}\")"
|
||||
```
|
||||
|
||||
## Session Context (2026-07-07)
|
||||
|
||||
- Offsite job expanded: `vmid 106` → `vmid 106,108,104,99999`
|
||||
- Initial full backups triggered manually on correct nodes
|
||||
- CT111 (Seafile) backup showed 0 MB on noris_s3 — empty/failed backup
|
||||
(previous valid: 558 GB on 2026-07-06)
|
||||
- Frigate excluded from all backups per user decision
|
||||
- Seafile backup strategy discussion: user exploring alternatives to
|
||||
558 GB full backup, considering S3 storage backend (Pro feature)
|
||||
- Immich: CT115 stopped, upload location unidentified
|
||||
Reference in New Issue
Block a user