- New: smart-home/home-assistant-dashboard-conventions (Mushroom cards, view tabs, no Bubble Cards) - Updated: rke2, ceph, galera, proxmox, brainstorming, compound-learning, 1password-cli, smart-home-automation skills - New references: ceph-cluster-administration, docker-volume-forensics, ceph-crush-weight, ceph-ec-mixed-size
217 lines
6.4 KiB
Markdown
217 lines
6.4 KiB
Markdown
# Gitea Helm Chart v12 Deployment — Session 2026-07-14
|
|
|
|
## Context
|
|
Deploying Gitea on K8s via ArgoCD using the official `gitea-charts/gitea`
|
|
chart v12.6.0 (app v1.26.1). DB = external MariaDB Galera via MaxScale
|
|
VIP 10.0.30.70:3306. Secrets via 1Password ESO.
|
|
|
|
## Issue 1: DB Password Not Reaching Init Container
|
|
|
|
### Symptom
|
|
```
|
|
[F] Failed to initialize ORM engine: Error 1045 (28000):
|
|
Access denied for user 'gitea'@'10.0.30.62' (using password: NO)
|
|
```
|
|
|
|
### Root Cause
|
|
The `configure-gitea` init container runs `gitea migrate` which reads
|
|
`app.ini`. The chart generates `app.ini` from `gitea.config` values —
|
|
but `PASSWD` was not set there. Env vars `GITEA__database__PASSWD`
|
|
override app.ini at runtime, but only if they reach the init container.
|
|
|
|
### Failed Attempts
|
|
1. `gitea.env` — not a valid chart value path; silently ignored
|
|
2. `gitea.database.existingSecret` — not a valid chart value; ignored
|
|
3. `gitea.config.database.PASSWD: __placeholder__` — literal string
|
|
reaches DB, not a secret reference
|
|
|
|
### Solution
|
|
`gitea.additionalConfigFromEnvs` — templated into ALL containers
|
|
(init-directories, init-app-ini, configure-gitea, main):
|
|
```yaml
|
|
gitea:
|
|
additionalConfigFromEnvs:
|
|
- name: GITEA__database__PASSWD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: gitea-db
|
|
key: password
|
|
```
|
|
|
|
### Verification
|
|
After fix, init-app-ini log showed `database + PASSWD` in config,
|
|
and configure-gitea logged `PING DATABASE mysql` followed by
|
|
successful schema migration.
|
|
|
|
## Issue 2: Valkey Auto-Deployed
|
|
|
|
### Symptom
|
|
3 `gitea-valkey-cluster-*` pods appeared despite `redis-cluster.enabled: false`.
|
|
|
|
### Root Cause
|
|
Chart v12 renamed `redis-cluster` → `valkey-cluster`. The old key
|
|
no longer controls the cache subsystem.
|
|
|
|
### Fix
|
|
```yaml
|
|
valkey-cluster:
|
|
enabled: false
|
|
```
|
|
ArgoCD pruned the Valkey StatefulSet after the updated values synced.
|
|
|
|
## Issue 3: RWO PVC Blocks Rolling Update
|
|
|
|
### Symptom
|
|
New pod stuck in `Init:0/3` with:
|
|
```
|
|
Multi-Attach error for volume "pvc-..."
|
|
Volume is already used by pod(s) gitea-OLD_POD
|
|
```
|
|
|
|
### Root Cause
|
|
`ReadWriteOnce` PVC + single replica + ArgoCD selfHeal creates new
|
|
pod before old one terminates. Both compete for the same volume.
|
|
|
|
### Fix
|
|
```bash
|
|
kubectl scale deploy -n gitea gitea --replicas=0
|
|
sleep 15
|
|
kubectl scale deploy -n gitea gitea --replicas=1
|
|
```
|
|
|
|
## Issue 4: Galera Schema Migration Slowness
|
|
|
|
### Symptom
|
|
`configure-gitea` init container stayed at `Init:2/3` for 5+ minutes.
|
|
|
|
### Diagnosis
|
|
- 104 tables created by Gitea 1.26.1
|
|
- Galera replicates each CREATE TABLE + index synchronously to 3 nodes
|
|
- `SHOW PROCESSLIST` on Galera showed `creating table` / `Committing alter table`
|
|
- Total time: ~8 minutes from first table to pod Running
|
|
|
|
### Key Insight
|
|
This is normal for Galera — do NOT CrashLoop the pod or delete it.
|
|
Monitor progress via `SELECT COUNT(*) FROM information_schema.tables
|
|
WHERE table_schema='gitea'`.
|
|
|
|
## ArgoCD App Architecture Used
|
|
|
|
Two separate ArgoCD Applications:
|
|
1. `gitea-config` (sync-wave 1): Git source → `clusters/main/gitea/`
|
|
(namespace.yaml + external-secrets.yaml)
|
|
2. `gitea` (sync-wave 2): Helm source → `https://dl.gitea.com/charts/`
|
|
chart `gitea` v12.6.0
|
|
|
|
The root App-of-Apps (`clusters/main/apps/`) auto-discovers both.
|
|
|
|
## Final Working Values (Key Sections)
|
|
|
|
```yaml
|
|
gitea:
|
|
config:
|
|
database:
|
|
DB_TYPE: mysql
|
|
HOST: 10.0.30.70:3306
|
|
NAME: gitea
|
|
USER: gitea
|
|
additionalConfigFromEnvs:
|
|
- name: GITEA__database__PASSWD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: gitea-db
|
|
key: password
|
|
existingSecret:
|
|
name: gitea-security
|
|
keys:
|
|
internal_token: internal_token
|
|
jwt_secret: jwt_secret
|
|
secret_key: secret_key
|
|
lfs_jwt_secret: lfs_jwt_secret
|
|
|
|
mysql:
|
|
enabled: false
|
|
postgresql:
|
|
enabled: false
|
|
postgresql-ha:
|
|
enabled: false
|
|
redis-cluster:
|
|
enabled: false
|
|
valkey-cluster:
|
|
enabled: false
|
|
|
|
persistence:
|
|
enabled: true
|
|
storageClass: ceph-hdd-replica
|
|
size: 10Gi
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
```
|
|
|
|
## Repo Migration via Gitea Migration API (Task 7)
|
|
|
|
### Prerequisites
|
|
- API tokens on BOTH Gitea instances (source CT108 + dest K8s)
|
|
- Orgs and users created on destination before migrating
|
|
- `migrations.ALLOWED_HOST_LIST` configured (see above)
|
|
|
|
### Token Generation
|
|
|
|
```bash
|
|
# CT108 (source, LXC — Gitea runs as user 'gitea'):
|
|
ssh root@10.0.30.105 "su - gitea -s /bin/bash -c \
|
|
'gitea admin user generate-access-token -u dominik -t migrator --raw \
|
|
--config /etc/gitea/app.ini'"
|
|
|
|
# K8s Gitea (destination):
|
|
kubectl exec -n gitea deploy/gitea -- \
|
|
gitea admin user generate-access-token -u dominik -t migrator --raw \
|
|
--config /data/gitea/conf/app.ini
|
|
```
|
|
|
|
### Migration Script Pattern
|
|
|
|
```python
|
|
for repo in repos:
|
|
migrate_data = {
|
|
"clone_addr": f"{CT108_URL}/{owner}/{name}.git",
|
|
"repo_owner": owner, "repo_name": name,
|
|
"mirror": False, "wiki": True, "issues": True,
|
|
"labels": True, "milestones": True,
|
|
"pull_requests": True, "releases": True,
|
|
"service": "gitea", "auth_token": CT108_TOKEN,
|
|
}
|
|
resp = k8s_api("POST", "/api/v1/repos/migrate", migrate_data)
|
|
```
|
|
|
|
API calls go through `kubectl exec` on the K8s Gitea pod (no
|
|
LoadBalancer/Ingress needed during migration):
|
|
```bash
|
|
kubectl exec -n gitea deploy/gitea -- curl -s -X POST \
|
|
-H "Authorization: token TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{...}' http://localhost:3000/api/v1/repos/migrate
|
|
```
|
|
|
|
### Results (23 repos)
|
|
- 18/23 migrated on first attempt ✅
|
|
- 1/23 skipped (already existed from test) ⏭️
|
|
- 4/23 failed with PR import errors → retried with `pull_requests: false` → all ✅
|
|
- Final: 23/23 repos migrated with issues + wikis
|
|
|
|
### Failed Repos (PR Import Bug)
|
|
- `dominik/hermes-skills` — `initRepository: getRepositoryByID`
|
|
- `pm-infra/homelab-board` — `error while listing pull requests`
|
|
- `d.schoen/packer-proxmox` — `initRepository: getRepositoryByID`
|
|
- `codecamp-n/qr-reader-webapp` — `error while listing pull requests`
|
|
|
|
Fix: `DELETE /api/v1/repos/{owner}/{name}` (cleanup), then retry
|
|
migration with `pull_requests: false`.
|
|
|
|
## Commits
|
|
- `48a26e6` — Initial scaffold (ArgoCD apps + namespace + ES + helm values)
|
|
- `68eda13` — Fix: GITEA__database__PASSWD env + disable valkey-cluster
|
|
- `effb49b` — Fix: use additionalConfigFromEnvs (correct chart mechanism)
|
|
- `40da393` — Fix: allow migration from CT108 host (wrong setting name)
|
|
- `68adfab` — Fix: correct migrations setting names (ALLOWED_HOST_LIST + ALLOW_LOCALNETWORKS)
|