railiance-apps/workplans/RAILIANCE-WP-0015-cnpg-backup-scheduledbackup-coverage.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated
RAILIANCE-WP-0015	workplan	CNPG backup ScheduledBackup coverage — drive cnpg-backup-status to healthy	financials	railiance-apps	ready	codex	railiance	2026-07-14	2026-07-14

CNPG backup ScheduledBackup coverage

Parent / origin: descoped out of RAIL-HO-WP-0005 (Forgejo Production Migration, finished 2026-07-14) T04/T09. The migration is complete; this is the residual DB-backup operational hardening, not a migration blocker.

Goal

Drive make cnpg-backup-status (railiance-apps) from degraded → healthy: scheduled, unattended, encrypted, off-cluster backups with a passing restore drill for all four production CNPG clusters — apps-pg, gitea-db, net-kingdom-pg, state-hub-db.

Current state (2026-07-14)

• cnpg-backup-status: degraded — 4 clusters lack ScheduledBackup coverage (all show backup spec = null, ScheduledBackup = none; clusters otherwise healthy, 1 instance each).
• Decided approach (RAIL-HO-WP-0005 Key Decision 5, Option A, 2026-07-09): daily age-encrypted pg_dump/forgejo dump → Nextcloud WebDAV, 14 daily + 4 weekly retention, RPO 24h / RTO 4h. A one-off encrypted forgejo-db dump was produced + restore-drilled (RAILIANCE-WP-0013, finished 2026-07-11).
• Tooling already present: tools/cnpg-backup-status.sh, manifests/cnpg-backup-readiness.yaml (commented template), make apps-pg-backup-dry-run, platform forgejo-backup, docs/app-data-backup-restore-handoff.md.

Blocking gate (human / operator)

The offsite credential lane CCR-2026-0004 (railiance-backup-offsite-lane) is status: active, decision: approved, but resolvable: false (readiness: applied-pending-verify). No backup Secret exists in the databases namespace. Provisioning NC_WEBDAV_TOKEN / NC_WEBDAV_URL / AGE_PRIVATE_KEY into OpenBao (making warden access railiance-backup-offsite-lane --fetch … resolve) is an operator/OpenBao action — out of scope for agent credential handling per credential-routing. Everything downstream is gated on it.

Task: Provision CCR-2026-0004 offsite credentials (operator)

id: RAILIANCE-WP-0015-T01
status: wait
priority: high
needs_human: true

Operator provisions NC_WEBDAV_TOKEN, NC_WEBDAV_URL, AGE_PRIVATE_KEY per CCR-2026-0004 so the lane becomes resolvable. Done when: warden access railiance-backup-offsite-lane --fetch NC_WEBDAV_TOKEN succeeds and the CCR flips to resolvable: true.

Task: Create offsite backup Secret in databases namespace

id: RAILIANCE-WP-0015-T02
status: wait
priority: high

Once T01 resolves, materialize the databases namespace Secret from the lane (no plaintext in Git — sourced via OpenBao/operator path). Done when: the Secret exists and pg_dump/WebDAV upload authenticates.

Task: Wire scheduled backup for all four CNPG clusters

id: RAILIANCE-WP-0015-T03
status: wait
priority: high

Implement the decided Option A lane as an unattended schedule for apps-pg, gitea-db, net-kingdom-pg, state-hub-db: age-encrypted logical dump → Nextcloud WebDAV, 14 daily + 4 weekly retention. Uncomment/parameterize manifests/cnpg-backup-readiness.yaml (or CronJob equivalent). Done when: each cluster has a running scheduled backup producing encrypted off-cluster artifacts without manual intervention.

Task: Reconcile cnpg-backup-status health definition with Option A

id: RAILIANCE-WP-0015-T04
status: wait
priority: medium

cnpg-backup-status.sh currently reports degraded whenever a CNPG-native ScheduledBackup resource is absent, but the decided approach is logical-dump CronJobs, not barman ScheduledBackup. Update the health check so a passing Option-A schedule reports healthy (or adopt barman ObjectStore if chosen instead). Done when: make cnpg-backup-status reflects the real, decided posture.

Task: Restore drill + RPO/RTO evidence, mark healthy

id: RAILIANCE-WP-0015-T05
status: wait
priority: high

Run an isolated-namespace restore drill from a scheduled artifact for at least one cluster; record RPO 24h / RTO 4h evidence. Done when: make cnpg-backup-status → healthy and restore evidence is recorded.