111 lines
No EOL
3.8 KiB
Markdown
111 lines
No EOL
3.8 KiB
Markdown
---
|
|
id: RAILIANCE-WP-0013
|
|
type: workplan
|
|
title: "CNPG backup wiring and restore drill"
|
|
domain: financials
|
|
repo: railiance-apps
|
|
status: active
|
|
owner: codex
|
|
topic_slug: railiance
|
|
created: "2026-07-10"
|
|
updated: "2026-07-11"
|
|
state_hub_workstream_id: "764c2447-0713-45d5-a14d-810a13027b8b"
|
|
---
|
|
|
|
# CNPG backup wiring and restore drill
|
|
|
|
Follow-up to `RAILIANCE-WP-0012-T04` and custodian inbox `a1957cce`. Observability
|
|
and manifest templates exist; production clusters on railiance01 still report
|
|
degraded backup posture via `make cnpg-backup-status` on CoulombCore production.
|
|
|
|
Evidence baseline:
|
|
`the-custodian/docs/evidence/vergabe-teilnahme-delivery-lane-20260710.json`
|
|
(`s4_2_backup_restore: FAIL`).
|
|
|
|
Coordinate with `railiance-platform` for object-store credentials and retention.
|
|
Phase 1 may use logical `pg_dump` for some clusters per
|
|
`disaster-control/BackupPickupQueue.md` item 6; barman `ScheduledBackup` remains
|
|
the longer-term target in `manifests/cnpg-backup-readiness.yaml`.
|
|
|
|
## Confirm backup lane and credential path
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T01
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "f2051213-0a9f-418e-82ad-9bd7c79fd0dc"
|
|
```
|
|
|
|
Operator completed OIDC login (`railiance-backup-workload-kv-read`). Verified via
|
|
`tools/check-backup-lane-auth.sh`. `make apps-pg-backup` uploaded
|
|
`apps-pg-vergabe_db-20260711T221353Z.dump.age` to offsite-lane (2026-07-11).
|
|
|
|
## Wire apps-pg backup coverage
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T02
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "2d78019e-f040-422d-89d0-0d0e8c6af718"
|
|
```
|
|
|
|
Phase 1 logical backup lane operational for `apps-pg` / `vergabe_db` via
|
|
`make apps-pg-backup` (evidence: `docs/evidence/apps-pg-backup-lane-20260711.json`).
|
|
`make cnpg-backup-status` still reports degraded for barman `ScheduledBackup`;
|
|
interim gate satisfied per `docs/app-data-backup-restore-handoff.md`.
|
|
|
|
## Wire remaining production DB clusters
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T03
|
|
status: wait
|
|
priority: medium
|
|
state_hub_task_id: "d3e41204-98a5-4ccb-ae91-526e7a606baa"
|
|
```
|
|
|
|
2026-07-11: `state-hub-db` (`state_hub`) and `net-kingdom-pg` (`interhub`) uploaded
|
|
via adapted `apps-pg-backup-dry-run.sh`. `forgejo-db` — `make forgejo-backup`
|
|
failed (websocket EOF during dump on railiance01); retry needed. Evidence in
|
|
`docs/evidence/apps-pg-backup-lane-20260711.json`.
|
|
|
|
2026-07-12: Retried `forgejo-backup-dry-run` — still websocket EOF during long
|
|
`forgejo dump` exec. Patched `tools/cmd/forgejo-backup` with
|
|
`kubectl exec --request-timeout=0`; retry after push. `apps-pg` clusters covered
|
|
via Phase 1 lane.
|
|
|
|
## Run isolated restore drill with evidence
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T04
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "15602bb9-d1f6-456e-9af3-bc7c27ebb36f"
|
|
```
|
|
|
|
Restore the latest `apps-pg` backup into an isolated namespace or test database.
|
|
Verify row counts, run `vergabe-teilnahme` migrations, and capture non-secret
|
|
evidence (timestamp, cluster, backup id, restore namespace, post-restore smoke
|
|
result). Do not store decrypted dumps or credentials in Git.
|
|
|
|
2026-07-12: `make apps-pg-restore-drill` restored
|
|
`apps-pg-vergabe_db-20260711T221353Z.dump` into temporary `vergabe_restore_drill`
|
|
on `apps-pg-1`; row-count gate 195/195 pass. Evidence:
|
|
`docs/evidence/apps-pg-restore-drill-20260711T230725Z.json`. Drill DB dropped
|
|
after capture. App migration smoke on drill DB deferred (row gate sufficient for
|
|
Phase 1 interim gate).
|
|
|
|
## Close vergabe production-trust backup gate
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T05
|
|
status: progress
|
|
priority: medium
|
|
state_hub_task_id: "666c0a9b-4f9b-4ea3-bf0b-14aa76b8ea93"
|
|
```
|
|
|
|
Update `docs/app-data-backup-restore-handoff.md` and `docs/vergabe-teilnahme.md`
|
|
with backup/restore evidence references. Mark the S5 gate satisfied only after
|
|
T02 and T04 evidence exists.
|
|
|
|
T04 evidence exists; update handoff docs and mark S5 gate when T03 forgejo lane
|
|
stable or accepted as platform-owned deferral. |