43 lines
1.4 KiB
Markdown
43 lines
1.4 KiB
Markdown
---
|
|
title: Ceph Cluster
|
|
category: systems
|
|
tags: [ceph, storage, rbd, ec-pool, osd]
|
|
created: "2026-07-24"
|
|
modified: "2026-07-24"
|
|
---
|
|
|
|
# Ceph Cluster
|
|
|
|
## Übersicht
|
|
- **14 OSDs** (HDD + NVMe/SSD混合)
|
|
- **Health:** HEALTH_WARN (BlueFS spillover osd.8, slow ops, nearfull)
|
|
- Läuft auf Proxmox PVE Hosts (integriert via pveceph)
|
|
|
|
## Pools
|
|
| Pool | Type | Usage |
|
|
|------|------|-------|
|
|
| ceph-flash | replicated | K8s CSI (fast storage) |
|
|
| ceph-hdd | replicated | Bulk storage |
|
|
| media_ec | EC k=4+m=1 | Media (photos, backups) |
|
|
|
|
## Bekannte Probleme
|
|
- **osd.0 NVMe** — 92% Wear, Critical Warning → Austausch planen
|
|
- **osd.2/5** — 93-94% full → entlasten oder neue OSDs
|
|
- **EC Pool k=4+m=1** — mit 5 OSDs kein Rebalance Headroom. Siehe Solution Doc: `docs/solutions/architecture/2026-07-12-ceph-ec-pool-no-rebalance-headroom.md`
|
|
- **BlueFS Spillover** osd.8 — `ceph-bluestore-tool bluefs-bdev-expand --path /var/lib/ceph/osd/ceph-8`
|
|
|
|
## CRUSH Topology
|
|
- Device Classes: hdd, ssd, nvme
|
|
- EC Pools benötigen Device-Class-Separation (mixed disk sizes → Anti-Pattern: artificially low weights)
|
|
|
|
## RBD Management
|
|
- Proxmox RBD Double-Mount Deadlock Pitfall: Niemals `pct mount` und `pct exec` gleichzeitig auf demselben Container
|
|
- Siehe Solution Doc: `docs/solutions/bug-fixes/2026-07-23-proxmox-rbd-double-mount-deadlock.md`
|
|
|
|
## Related Skills
|
|
- `ceph-cluster-administration` (devops)
|
|
|
|
## Related
|
|
- [[systems/proxmox-cluster]]
|
|
- [[systems/rke2-kubernetes]] (Ceph CSI)
|