tegwick
a689270f18
Document OpenBao/warden paths for the offsite backup lane and flag the sealed Vault blocker. Point production Makefile targets at CoulombCore, auto-discover CNPG clusters in backup status, add apps-pg pg_dump dry-run tooling, and record Core Hub Helm cutover findings (live gitea image, adopt required).
106 lines
No EOL
3.3 KiB
Markdown
106 lines
No EOL
3.3 KiB
Markdown
---
|
|
id: RAILIANCE-WP-0013
|
|
type: workplan
|
|
title: "CNPG backup wiring and restore drill"
|
|
domain: financials
|
|
repo: railiance-apps
|
|
status: active
|
|
owner: codex
|
|
topic_slug: railiance
|
|
created: "2026-07-10"
|
|
updated: "2026-07-10"
|
|
state_hub_workstream_id: "764c2447-0713-45d5-a14d-810a13027b8b"
|
|
---
|
|
|
|
# CNPG backup wiring and restore drill
|
|
|
|
Follow-up to `RAILIANCE-WP-0012-T04` and custodian inbox `a1957cce`. Observability
|
|
and manifest templates exist; production clusters on railiance01 still report
|
|
degraded backup posture via `make cnpg-backup-status` on CoulombCore production.
|
|
|
|
Evidence baseline:
|
|
`the-custodian/docs/evidence/vergabe-teilnahme-delivery-lane-20260710.json`
|
|
(`s4_2_backup_restore: FAIL`).
|
|
|
|
Coordinate with `railiance-platform` for object-store credentials and retention.
|
|
Phase 1 may use logical `pg_dump` for some clusters per
|
|
`disaster-control/BackupPickupQueue.md` item 6; barman `ScheduledBackup` remains
|
|
the longer-term target in `manifests/cnpg-backup-readiness.yaml`.
|
|
|
|
## Confirm backup lane and credential path
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T01
|
|
status: progress
|
|
priority: high
|
|
needs_human: true
|
|
intervention_note: "OpenBao sealed at bao.coulomb.social — operator unseal + OIDC login (railiance-backup-workload-kv-read) required"
|
|
state_hub_task_id: "f2051213-0a9f-418e-82ad-9bd7c79fd0dc"
|
|
```
|
|
|
|
Credential routing in `docs/credential-routing-railiance-apps.md`. `warden access`
|
|
reaches the platform broker; KV fetch fails while OpenBao is sealed. Phase 1 lane:
|
|
`platform/workloads/railiance/backup/offsite-lane`. CoulombCore clusters:
|
|
`apps-pg`, `gitea-db`, `net-kingdom-pg`, `state-hub-db`. `make apps-pg-backup-dry-run`
|
|
succeeded (pg_dump + age encrypt; upload blocked on offsite-lane).
|
|
|
|
## Wire apps-pg backup coverage
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T02
|
|
status: wait
|
|
priority: high
|
|
state_hub_task_id: "2d78019e-f040-422d-89d0-0d0e8c6af718"
|
|
```
|
|
|
|
Enable backup coverage for `apps-pg` first because `vergabe_db` is
|
|
production-facing. Apply platform-approved manifests or Makefile-driven operator
|
|
steps. `make cnpg-backup-status` must show a non-null backup spec or active
|
|
`ScheduledBackup` for `apps-pg`.
|
|
|
|
Blocked on: `RAILIANCE-WP-0013-T01`.
|
|
|
|
## Wire remaining production DB clusters
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T03
|
|
status: wait
|
|
priority: medium
|
|
state_hub_task_id: "d3e41204-98a5-4ccb-ae91-526e7a606baa"
|
|
```
|
|
|
|
Extend the same backup lane to `forgejo-db`, `net-kingdom-pg`, and
|
|
`state-hub-db`. Re-run `make cnpg-backup-status` until all four clusters pass.
|
|
|
|
Blocked on: `RAILIANCE-WP-0013-T02`.
|
|
|
|
## Run isolated restore drill with evidence
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T04
|
|
status: wait
|
|
priority: high
|
|
state_hub_task_id: "15602bb9-d1f6-456e-9af3-bc7c27ebb36f"
|
|
```
|
|
|
|
Restore the latest `apps-pg` backup into an isolated namespace or test database.
|
|
Verify row counts, run `vergabe-teilnahme` migrations, and capture non-secret
|
|
evidence (timestamp, cluster, backup id, restore namespace, post-restore smoke
|
|
result). Do not store decrypted dumps or credentials in Git.
|
|
|
|
Blocked on: `RAILIANCE-WP-0013-T02`.
|
|
|
|
## Close vergabe production-trust backup gate
|
|
|
|
```task
|
|
id: RAILIANCE-WP-0013-T05
|
|
status: wait
|
|
priority: medium
|
|
state_hub_task_id: "666c0a9b-4f9b-4ea3-bf0b-14aa76b8ea93"
|
|
```
|
|
|
|
Update `docs/app-data-backup-restore-handoff.md` and `docs/vergabe-teilnahme.md`
|
|
with backup/restore evidence references. Mark the S5 gate satisfied only after
|
|
T02 and T04 evidence exists.
|
|
|
|
Blocked on: `RAILIANCE-WP-0013-T04`. |