railiance-apps/workplans/archived/260712-RAILIANCE-WP-0013-cnpg-backup-wiring-restore-drill.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
RAILIANCE-WP-0013	workplan	CNPG backup wiring and restore drill	financials	railiance-apps	finished	codex	railiance	2026-07-10	2026-07-12	764c2447-0713-45d5-a14d-810a13027b8b

CNPG backup wiring and restore drill

Follow-up to RAILIANCE-WP-0012-T04 and custodian inbox a1957cce. Observability and manifest templates exist; production clusters on railiance01 still report degraded backup posture via make cnpg-backup-status on CoulombCore production.

Evidence baseline: the-custodian/docs/evidence/vergabe-teilnahme-delivery-lane-20260710.json (s4_2_backup_restore: FAIL).

Coordinate with railiance-platform for object-store credentials and retention. Phase 1 may use logical pg_dump for some clusters per disaster-control/BackupPickupQueue.md item 6; barman ScheduledBackup remains the longer-term target in manifests/cnpg-backup-readiness.yaml.

Confirm backup lane and credential path

id: RAILIANCE-WP-0013-T01
status: done
priority: high
state_hub_task_id: "f2051213-0a9f-418e-82ad-9bd7c79fd0dc"

Operator completed OIDC login (railiance-backup-workload-kv-read). Verified via tools/check-backup-lane-auth.sh. make apps-pg-backup uploaded apps-pg-vergabe_db-20260711T221353Z.dump.age to offsite-lane (2026-07-11).

Wire apps-pg backup coverage

id: RAILIANCE-WP-0013-T02
status: done
priority: high
state_hub_task_id: "2d78019e-f040-422d-89d0-0d0e8c6af718"

Phase 1 logical backup lane operational for apps-pg / vergabe_db via make apps-pg-backup (evidence: docs/evidence/apps-pg-backup-lane-20260711.json). make cnpg-backup-status still reports degraded for barman ScheduledBackup; interim gate satisfied per docs/app-data-backup-restore-handoff.md.

Wire remaining production DB clusters

id: RAILIANCE-WP-0013-T03
status: done
priority: medium
state_hub_task_id: "d3e41204-98a5-4ccb-ae91-526e7a606baa"

2026-07-11: state-hub-db (state_hub) and net-kingdom-pg (interhub) uploaded via adapted apps-pg-backup-dry-run.sh. forgejo-db — initial make forgejo-backup failed (websocket EOF during dump on railiance01). Evidence in docs/evidence/apps-pg-backup-lane-20260711.json.

2026-07-12: Patched tools/cmd/forgejo-backup (in-pod nohup dump, 4m chunk copy with retries, kubectl preflight retries). After operator bao login, make forgejo-backup succeeded — uploaded forgejo-dump-20260711T233637Z.zip.age (1.3G) and forgejo-db-20260711T233637Z.sql.age (24M) to Nextcloud forgejo/. apps-pg clusters covered via Phase 1 lane.

Run isolated restore drill with evidence

id: RAILIANCE-WP-0013-T04
status: done
priority: high
state_hub_task_id: "15602bb9-d1f6-456e-9af3-bc7c27ebb36f"

Restore the latest apps-pg backup into an isolated namespace or test database. Verify row counts, run vergabe-teilnahme migrations, and capture non-secret evidence (timestamp, cluster, backup id, restore namespace, post-restore smoke result). Do not store decrypted dumps or credentials in Git.

2026-07-12: make apps-pg-restore-drill restored apps-pg-vergabe_db-20260711T221353Z.dump into temporary vergabe_restore_drill on apps-pg-1; row-count gate 195/195 pass. Evidence: docs/evidence/apps-pg-restore-drill-20260711T230725Z.json. Drill DB dropped after capture. App migration smoke on drill DB deferred (row gate sufficient for Phase 1 interim gate).

Close vergabe production-trust backup gate

id: RAILIANCE-WP-0013-T05
status: done
priority: medium
state_hub_task_id: "666c0a9b-4f9b-4ea3-bf0b-14aa76b8ea93"

Update docs/app-data-backup-restore-handoff.md and docs/vergabe-teilnahme.md with backup/restore evidence references. Mark the S5 gate satisfied only after T02 and T04 evidence exists.

2026-07-12: Handoff docs updated with Phase 1 gate satisfaction, evidence table, and operator commands. S5 interim production-trust gate marked satisfied; CNPG barman ScheduledBackup remains platform follow-up. All tasks complete — workplan archived.