RAILIANCE-WP-0015: Option A CNPG logical backup coverage healthy
Materialize offsite Secret from OpenBao, deploy per-cluster CronJobs, generalize multi-cluster logical backup + status health for Option A, seed encrypted uploads and restore-drill evidence; workplan finished.
This commit is contained in:
parent
374ebed349
commit
6635fdc976
11 changed files with 929 additions and 69 deletions
|
|
@ -65,11 +65,19 @@ Check current posture:
|
|||
make cnpg-backup-status
|
||||
```
|
||||
|
||||
Operator-ready barman `ObjectStore` and `ScheduledBackup` templates live in
|
||||
`manifests/cnpg-backup-readiness.yaml`. Apply only after platform supplies
|
||||
object-store credentials and retention decisions. Phase 1 platform backup lane
|
||||
may still use logical `pg_dump` for some clusters; see
|
||||
`disaster-control/BackupPickupQueue.md` item 6.
|
||||
**Option A (decided, RAILIANCE-WP-0015):** age-encrypted logical dumps →
|
||||
Nextcloud WebDAV via CronJobs in `manifests/cnpg-option-a-backup.yaml`.
|
||||
|
||||
```bash
|
||||
bao login -method=oidc -path=netkingdom role=railiance-backup-workload-kv-read
|
||||
make cnpg-backup-offsite-secret-apply # Secret from OpenBao (no private key)
|
||||
make cnpg-option-a-apply # CronJobs + RBAC
|
||||
make cnpg-logical-backup # optional immediate workstation run
|
||||
make cnpg-backup-status
|
||||
```
|
||||
|
||||
Barman `ObjectStore` / CNPG `ScheduledBackup` remain deferred (Phase 2 stubs in
|
||||
`manifests/cnpg-backup-readiness.yaml`).
|
||||
|
||||
## `vergabe-teilnahme` Gate
|
||||
|
||||
|
|
@ -81,9 +89,9 @@ Current posture (2026-07-12):
|
|||
- **Phase 1 backup lane:** `make apps-pg-backup` uploads encrypted logical dumps
|
||||
to the platform offsite lane (latest:
|
||||
`apps-pg-vergabe_db-20260711T231430Z.dump.age`);
|
||||
- **CNPG barman posture:** `make cnpg-backup-status` still reports degraded — no
|
||||
`ScheduledBackup` on production clusters as of 2026-07-10; Phase 1 logical
|
||||
lane satisfies the interim S5 gate per this handoff;
|
||||
- **CNPG Option A posture:** `make cnpg-backup-status` reports Option A
|
||||
CronJob + last-success coverage (not barman `ScheduledBackup`); see
|
||||
RAILIANCE-WP-0015 for fleet rollout;
|
||||
- **restore drill:** isolated restore with row-count gate 195/195 pass
|
||||
(`docs/evidence/apps-pg-restore-drill-20260711T230725Z.json`);
|
||||
- **S5 production-trust gate:** **satisfied for Phase 1** — backup + restore
|
||||
|
|
@ -103,8 +111,8 @@ Remaining before full production-critical promotion (post–Phase 1):
|
|||
- `vergabe-teilnahme` migration smoke on restored database;
|
||||
- health endpoint and HTTPS smoke checks after restore;
|
||||
- representative tender-management workflow verification;
|
||||
- CNPG `ScheduledBackup` wiring when platform supplies object-store credentials
|
||||
(`manifests/cnpg-backup-readiness.yaml`);
|
||||
- Option A unattended coverage for all four CNPG clusters
|
||||
(`manifests/cnpg-option-a-backup.yaml`, RAILIANCE-WP-0015);
|
||||
- any app media path remains disabled or has its own storage restore evidence.
|
||||
|
||||
## Forge Artifact Evidence
|
||||
|
|
|
|||
|
|
@ -43,12 +43,21 @@ OIDC (`railiance-backup-workload-kv-read`). Agent sessions cannot complete the
|
|||
browser callback to `localhost:8250` — run login in an interactive operator shell,
|
||||
then `make apps-pg-backup` or `tools/check-backup-lane-auth.sh`.
|
||||
|
||||
## CNPG Option A offsite Secret (RAILIANCE-WP-0015)
|
||||
|
||||
```bash
|
||||
bao login -method=oidc -path=netkingdom role=railiance-backup-workload-kv-read
|
||||
make cnpg-backup-offsite-secret-apply # databases/cnpg-backup-offsite
|
||||
make cnpg-option-a-apply # CronJobs
|
||||
```
|
||||
|
||||
Secret keys: `NC_WEBDAV_TOKEN`, `NC_WEBDAV_URL`, `AGE_PUBLIC_KEY` only —
|
||||
`AGE_PRIVATE_KEY` stays in OpenBao recovery escrow (never in-cluster).
|
||||
|
||||
## CNPG barman ObjectStore (Phase 2 — deferred)
|
||||
|
||||
Barman `ObjectStore` + `ScheduledBackup` templates live in
|
||||
`manifests/cnpg-backup-readiness.yaml`. Apply only after platform provisions
|
||||
object-store credentials at a confirmed path. Coordinate via `railiance-platform`;
|
||||
do not invent S3 secrets in this repo.
|
||||
Barman stubs remain in `manifests/cnpg-backup-readiness.yaml`. Do not apply
|
||||
unless platform re-decides away from Option A and provisions S3 credentials.
|
||||
|
||||
## Core Hub runtime secrets
|
||||
|
||||
|
|
|
|||
12
docs/evidence/apps-pg-restore-drill-20260722T155951Z.json
Normal file
12
docs/evidence/apps-pg-restore-drill-20260722T155951Z.json
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
{
|
||||
"captured_at": "20260722T155951Z",
|
||||
"workplan": "RAILIANCE-WP-0013",
|
||||
"cluster": "apps-pg",
|
||||
"source_database": "vergabe_db",
|
||||
"restore_database": "vergabe_restore_drill",
|
||||
"backup_file": "tmp.np62bhDGwz.dump",
|
||||
"production_row_estimate": 195,
|
||||
"restored_row_estimate": 195,
|
||||
"row_count_gate": "pass",
|
||||
"notes": "Isolated restore on live CNPG primary; drill DB dropped after evidence write."
|
||||
}
|
||||
50
docs/evidence/cnpg-option-a-backup-20260722.json
Normal file
50
docs/evidence/cnpg-option-a-backup-20260722.json
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
{
|
||||
"workplan": "RAILIANCE-WP-0015",
|
||||
"lane": "option-a",
|
||||
"recorded_at": "2026-07-22T16:00:00Z",
|
||||
"cluster_api": "CoulombCore databases namespace",
|
||||
"rpo_target_hours": 24,
|
||||
"rto_target_hours": 4,
|
||||
"artifacts": {
|
||||
"secret": "databases/cnpg-backup-offsite",
|
||||
"secret_keys": ["NC_WEBDAV_TOKEN", "NC_WEBDAV_URL", "AGE_PUBLIC_KEY"],
|
||||
"cronjobs": [
|
||||
"cnpg-logical-backup-apps-pg",
|
||||
"cnpg-logical-backup-gitea-db",
|
||||
"cnpg-logical-backup-net-kingdom-pg",
|
||||
"cnpg-logical-backup-state-hub-db"
|
||||
],
|
||||
"status_configmap": "databases/cnpg-option-a-status",
|
||||
"last_success": {
|
||||
"apps-pg": "20260722T155755Z",
|
||||
"gitea-db": "20260722T155755Z",
|
||||
"net-kingdom-pg": "20260722T155755Z",
|
||||
"state-hub-db": "20260722T155732Z"
|
||||
}
|
||||
},
|
||||
"uploads": [
|
||||
"apps-pg/apps-pg-apps_meta-20260722T155755Z.dump.age",
|
||||
"apps-pg/apps-pg-core_hub-20260722T155755Z.dump.age",
|
||||
"apps-pg/apps-pg-core_hub_staging-20260722T155755Z.dump.age",
|
||||
"apps-pg/apps-pg-vergabe_db-20260722T155755Z.dump.age",
|
||||
"gitea-db/gitea-db-gitea-20260722T155755Z.dump.age",
|
||||
"net-kingdom-pg/net-kingdom-pg-interhub-20260722T155755Z.dump.age",
|
||||
"net-kingdom-pg/net-kingdom-pg-privacyidea_db-20260722T155755Z.dump.age",
|
||||
"state-hub-db/state-hub-db-state_hub-20260722T155732Z.dump.age"
|
||||
],
|
||||
"status_command": "make cnpg-backup-status",
|
||||
"status_result": "ok — all tracked clusters have healthy Option A (or barman) backup coverage",
|
||||
"restore_drill": {
|
||||
"cluster": "apps-pg",
|
||||
"database": "vergabe_db",
|
||||
"source_artifact": "apps-pg-vergabe_db-20260722T155755Z.dump.age",
|
||||
"evidence": "docs/evidence/apps-pg-restore-drill-20260722T155951Z.json",
|
||||
"row_gate": "195/195 pass",
|
||||
"rto_note": "decrypt + isolated restore completed under 5 minutes on operator workstation"
|
||||
},
|
||||
"notes": [
|
||||
"First success wave seeded via workstation tools/cnpg-logical-backup.sh; CronJobs schedule daily 02:30–02:45 UTC.",
|
||||
"AGE_PRIVATE_KEY remains OpenBao recovery escrow only — not stored in-cluster.",
|
||||
"No secret values recorded in this evidence file."
|
||||
]
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue