hermes-skills/devops/rke2-cluster-administration/references/traefik-lxc-to-k8s-migration-2026-07.md

# Traefik LXC → K8s Migration (2026-07-14)

## CT99999 Inventory (Source)

- **Location:** proxmox7, VLAN 60, IP `10.0.60.10/24`
- **Traefik version:** 3.4.1
- **Static config:** `/etc/traefik/traefik.yaml`
- **Dynamic config:** `/etc/traefik/conf.d/` (file provider, watch=true)
- **TLS:** Let's Encrypt via TLS challenge, `acme.json` at `/etc/traefik/ssl/acme.json`
- **Entry Points:** web (:80 → redirect to :443), websecure (:443), traefik (:8080 dashboard)

## Services on CT99999

| Router | Host | Backend | Port | Notes |
|--------|------|---------|------|-------|
| grafana | grafana.familie-schoen.com | 10.0.30.141 | 3000 | CT141 monitoring |
| librechat | chat.familie-schoen.com, chat.schoen.eu | 10.0.30.102 | 8080 | LibreChat VM |
| seafile_old | seafile.familie-schoen.com | 10.0.30.100 | 8081 | Seafile old |
| imkerei | imkerei.familie-schoen.com | 10.0.30.100 | 80 | Catchall forwarder |
| immich | immich.familie-schoen.com | 10.0.30.99 | 2283 | Immich |
| paperless | dokumente.familie-schoen.com | 10.0.30.101 | 8000 | CT104 paperless |
| homeassistant | homeassistant.familie-schoen.com | 10.0.30.10 | 8123 | VM106 HA |
| seafile | cloud.familie-schoen.com | 10.0.30.99 | 80 | Seafile |
| seadoc | cloud.familie-schoen.com + /sdoc-server | 10.0.30.99 | 8888 | SeaDoc, stripPrefix middleware |
| authelia | auth.familie-schoen.com | 192.168.100.11 | 9091 | CT112 authelia |
| authelia-admin | auth-admin.familie-schoen.com | 192.168.100.11 | 9092 | CT112 authelia admin |
| gitea | git.familie-schoen.com | 10.0.30.105 | 3000 | CT108 (being replaced by K8s) |
| catchall-tcp | HostSNI(`*`) passthrough | 10.0.30.100 | 80 | TCP catchall |

## K8s Traefik (Target)

- **Type:** DaemonSet `rke2-traefik` in `kube-system`
- **Ports:** hostPort 80 + 443 on all 6 nodes (10.0.30.51-53, .61-63)
- **IngressClass:** `traefik`
- **Existing Ingresses:** argocd-server (argocd ns), gitea (gitea ns)
- **Needs:** Let's Encrypt certResolver (not configured by default)

## HA Failover VIP via Cilium L2 (Implemented 2026-07-14)

### Architecture

Instead of pointing the firewall NAT at a single K8s node (SPOF), a
`LoadBalancer` Service requests a VIP from the existing Cilium LB IPAM
pool. Cilium announces the VIP via ARP on one node; auto-failover to
another node on failure (~1-2s).

```
Firewall NAT → 10.0.30.203 (Cilium VIP)
                    ↓ L2 Announcement (ARP)
              [aktiver K8s Node — Traefik DaemonSet]
              Automatisches Failover bei Node-Ausfall (~1-2s)
```

### Components Deployed

| Component | File | Purpose |
|-----------|------|---------|
| `Service` (LoadBalancer) | `clusters/main/proxy/traefik-lb-service.yaml` | VIP `.203`, selects Traefik pods |
| `Namespace` | `clusters/main/proxy/namespace.yaml` | Shared proxy namespace |
| ArgoCD App | `clusters/main/apps/proxy.yaml` | GitOps for proxy namespace |

### traefik-lb-service.yaml

```yaml
apiVersion: v1
kind: Service
metadata:
  name: traefik-lb
  namespace: kube-system
spec:
  type: LoadBalancer
  loadBalancerIP: 10.0.30.203
  selector:
    app.kubernetes.io/instance: rke2-traefik-kube-system
    app.kubernetes.io/name: rke2-traefik
  ports:
    - name: web
      port: 80
      targetPort: web
    - name: websecure
      port: 443
      targetPort: websecure
```

### Why Cilium L2 (not MetalLB/keepalived)

- Cilium L2 LB IPAM already running (3 services use it: .200, .201, .202)
- No additional software needed
- Automatic failover — no manual VIP management
- Pool: `10.0.30.200–250` (CiliumLoadBalancerIPPool `default-pool`)
- L2 policy: `default-l2-policy` announces LB IPs via ARP on `^eth[0-9]+`

### Verification

After deploying, tested from mgmt-runner (.124):
- `10.0.30.203:80/443` → 404 (no default route — correct)
- `git.schoen.codes` via VIP → Gitea 1.26.1 ✅
- `argocd` via VIP → ArgoCD UI ✅

### Firewall Cutover

Change NAT target: `10.0.60.10` → `10.0.30.203` (one rule, instant cutover).
Rollback: revert NAT to `10.0.60.10`. CT99999 stays running throughout.

## Pattern for External Backends (not in K8s)

Use `ExternalName` Service + `IngressRoute` CRD:

```yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: grafana
  namespace: proxy
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`grafana.familie-schoen.com`)
      kind: Rule
      services:
        - name: grafana-external
          port: 3000
  tls:
    certResolver: letsencrypt
---
apiVersion: v1
kind: Service
metadata:
  name: grafana-external
  namespace: proxy
spec:
  type: ExternalName
  externalName: 10.0.30.141
```

## SeaDoc StripPath Middleware

```yaml
middlewares:
  seadoc-strippath:
    stripPrefix:
      prefixes:
        - /sdoc-server
```

## Accessing CT99999

```bash
# CT99999 is on proxmox7 (10.0.20.70)
ssh -i ~/.ssh/id_ed25519_proxmox root@10.0.20.70 "pct exec 99999 -- ..."
```

## Remaining Steps

1. Add Let's Encrypt certResolver via HelmChartConfig for `rke2-traefik`
2. Create IngressRoute + ExternalName Service for each of the 11 external backends
3. Handle catchall-TCP (make specific, not HostSNI(`*`) — see pitfall in SKILL.md)
4. Cutover firewall NAT: `10.0.60.10` → `10.0.30.203`
5. Verify each service, then decommission CT99999 after 1 week

## Reference

Full migration plan: `docs/plans/2026-07-14-traefik-ct99999-k8s-migration.md` in iac-homelab repo.