Add RTO-RPO matrix, backup pickup queue tracker, and TestEvidence with Forgejo drill import. Mark T03, T04, T05 done.
175 lines
No EOL
7.7 KiB
Markdown
175 lines
No EOL
7.7 KiB
Markdown
---
|
||
id: DISCTL-WP-0003
|
||
type: workplan
|
||
title: "Railiance backup lane governance integration"
|
||
domain: infotech
|
||
repo: disaster-control
|
||
status: active
|
||
owner: grok
|
||
topic_slug: custodian
|
||
created: "2026-07-07"
|
||
updated: "2026-07-07"
|
||
state_hub_workstream_id: "e13c139e-032e-4a16-97b8-b02b9e376f3d"
|
||
---
|
||
|
||
# Railiance backup lane governance integration
|
||
|
||
Codify the **existing** `railiance-backup` tooling practices into the
|
||
Resilience Control Plane: registers, policy, RTO/RPO targets, cross-repo
|
||
pickup tracking, and the first evidence trail entry.
|
||
|
||
This is a focused slice of `DISCTL-WP-0002` (tasks T02, T04, T06, T08). When
|
||
tasks here complete, mark the corresponding WP-0002 tasks `done` as well.
|
||
|
||
**Inputs:**
|
||
|
||
| Source | What it contributes |
|
||
| --- | --- |
|
||
| `history/2026-07-04-forgejo-backup-strategy-assessment.md` | Layer inventory, gaps, pickup queue items 1–6 |
|
||
| `railiance-cluster/docs/backup-restore.md` | Dev-workstation `railiance-backup` practices (age, Nextcloud, preflight, cron) |
|
||
| `railiance-cluster/tools/cmd/railiance-backup-s2` | S2 k3s runtime backup (local, age, 7-day prune) |
|
||
| `railiance-infra/tools/cmd/railiance-backup-s1` | S1 OS config backup (local, age, 7-day prune) |
|
||
| `the-custodian/docs/forgejo-production-decisions.md` | Backup decision Option A (2026-07-09): Nextcloud WebDAV, daily schedule, retention |
|
||
| `railiance-infra/docs/forgejo-restore-drill-evidence.md` | T09 restore drill evidence to import |
|
||
|
||
**Scope boundary:** documents and tracks backup governance here; implementation
|
||
(CNPG ScheduledBackup, scheduled `forgejo dump`, platform tool restoration) stays
|
||
in `railiance-platform`, `railiance-apps`, and `railiance-infra`.
|
||
|
||
## Seed registers with railiance-backup tool family (R2, R9)
|
||
|
||
```task
|
||
id: DISCTL-WP-0003-T01
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "da37bcf1-2d7c-4f6b-88c4-08c0bad91151"
|
||
```
|
||
|
||
Write initial `AssetRegister.yaml` and `DataRegister.yaml` seeded with the
|
||
backup lanes discovered in the July assessment. At minimum, record:
|
||
|
||
| Stream | Tool | Owner repo | Tier | Backup mechanism | Destination | Schedule | Status |
|
||
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||
| Dev workstation | `railiance-backup` | `railiance-cluster` | T1 | pg_dump + config tar, age | Nextcloud WebDAV | daily cron 02:00 | operational |
|
||
| S1 OS config | `railiance-backup-s1` | `railiance-infra` | T2 | OS config tar, age | `/opt/backup/railiance/infra/` local | manual (`make backup`) | operational |
|
||
| S2 k8s runtime | `railiance-backup-s2` | `railiance-cluster` | T1 | k3s state.db + helm values + kubeconfig, age | `/opt/backup/railiance/cluster/` local | manual (`make backup`) | operational |
|
||
| S3 platform | `railiance-backup` | `railiance-platform` | T1 | pg_dump (intended) | Nextcloud WebDAV (intended) | TBD | **missing tool** |
|
||
| Forgejo production | scheduled `forgejo dump` | `railiance-apps` | T1 | dump zip + CNPG logical | Nextcloud WebDAV (decided) | daily (intended) | not wired |
|
||
| Gitea canonical | `gitea dump` / mirror | `railiance-infra` | T1 | manual dump | none automated | pre-cutover only | at risk |
|
||
|
||
Every entry must declare: `owner`, `recovery_method`, `backup_schedule`,
|
||
`rto`, `rpo`, and honest `status` (operational / manual-only / missing).
|
||
Mark the platform `railiance-backup` gap and missing off-cluster upload for
|
||
S1/S2 explicitly — do not paper over gaps.
|
||
|
||
Feeds `DISCTL-WP-0002-T02`.
|
||
|
||
## Write BackupPolicy.md codifying existing practices (R3, R9)
|
||
|
||
```task
|
||
id: DISCTL-WP-0003-T02
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "2e32792d-4c22-4e76-adf5-44d9e81bd09a"
|
||
```
|
||
|
||
Write `BackupPolicy.md` translating live `railiance-backup` practices into
|
||
policy with verification steps:
|
||
|
||
1. **3-2-1 + immutable/offline model** — per `specs/ResilienceControlSetup.md` §4
|
||
2. **Per-stream mechanisms** — table keyed to register entries from T01
|
||
3. **Encryption standard** — age for all artifacts; key custody rules
|
||
(operational key + password-manager recovery copy)
|
||
4. **Retention** — 7-copy local prune (S1/S2); 14 daily + 4 weekly for
|
||
Forgejo/platform lane per 2026-07-09 decision
|
||
5. **Destination** — Nextcloud WebDAV file drop for off-node lanes; local
|
||
`/opt/backup/railiance/{infra,cluster}/` for S1/S2 until upload is wired
|
||
6. **Credential separation** (R3) — backup-storage credentials must not live
|
||
on production hosts with delete permission; verify with named check
|
||
7. **Monitoring** (R9) — backup job success + repository integrity checks on a
|
||
defined schedule; `.last-backup` stamp freshness; preflight gate before
|
||
infrastructure changes; named alerting owner
|
||
|
||
Record the 2026-07-09 Forgejo backup decision (Option A: extend platform
|
||
backup lane) as the adopted standard for Forgejo + CNPG `forgejo-db` until
|
||
superseded.
|
||
|
||
Feeds `DISCTL-WP-0002-T04`.
|
||
|
||
## Draft RTO-RPO matrix for backup lanes (R1, R6)
|
||
|
||
```task
|
||
id: DISCTL-WP-0003-T03
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "cec8f15d-471d-4bbb-ad01-e25bc73c66e3"
|
||
```
|
||
|
||
Write `RTO-RPO-Matrix.md` with explicit per-system targets. Start from the
|
||
assessment draft and the 2026-07-09 adopted targets:
|
||
|
||
| System | RPO | RTO | Evidence basis |
|
||
| --- | --- | --- | --- |
|
||
| Forgejo git + packages | 24h | 4h | Decision #7 (2026-07-09); T09 drill |
|
||
| Gitea (canonical until cutover) | 24h | 4–8h | Assessment §suggested targets |
|
||
| `forgejo-db` CNPG | 24h (Phase 1 logical dump) | 4h | Decision #7 |
|
||
| k3s cluster state (S2) | 24h | 4–8h | `railiance-backup-s2` local retention |
|
||
| S1 OS config | 24h | 4–8h | Manual backup; no schedule yet |
|
||
| Dev workstation custodian DB | 24h | 4h | Daily cron; T09-era preflight |
|
||
|
||
Mark each row `adopted` or `draft` until operator confirms. Link to register
|
||
entries and BackupPolicy streams.
|
||
|
||
Feeds `DISCTL-WP-0002-T03` (partial — ResiliencePolicy.md remains in WP-0002).
|
||
|
||
## Cross-repo pickup queue status board (R2, R9)
|
||
|
||
```task
|
||
id: DISCTL-WP-0003-T04
|
||
status: done
|
||
priority: medium
|
||
state_hub_task_id: "484e433c-4840-41f7-8eda-fd6c0520dd1d"
|
||
```
|
||
|
||
Create `BackupPickupQueue.md` tracking closure of assessment items 1–6. Each row:
|
||
item, owner repo, blocker, status (`open` / `in_progress` / `done`), last
|
||
checked date, and link to implementation evidence when closed.
|
||
|
||
| # | Item | Owner | Initial status |
|
||
| --- | --- | --- | --- |
|
||
| 1 | Backup destination + retention decision | `railiance-infra` / T02 | **done** — Option A 2026-07-09 |
|
||
| 2 | CNPG `ScheduledBackup` on `forgejo-db` | `railiance-platform` | open |
|
||
| 3 | Scheduled `forgejo dump` → age → upload | `railiance-apps` | open |
|
||
| 4 | Restore `tools/cmd/railiance-backup` to platform | `railiance-platform` | open (relocation cancelled) |
|
||
| 5 | Gitea pre-cutover snapshot runbook + evidence | `railiance-infra` / `railiance-apps` | open |
|
||
| 6 | `apps-pg` backup coverage | `railiance-platform` | open |
|
||
|
||
Update register and BackupPolicy as each item lands. Review queue monthly or
|
||
when a cross-repo workplan closes.
|
||
|
||
Feeds `DISCTL-WP-0002-T08`.
|
||
|
||
## Import Forgejo drill evidence and define evidence template (R2)
|
||
|
||
```task
|
||
id: DISCTL-WP-0003-T05
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "e28afd71-7716-4623-b2ae-0e8675d6a60f"
|
||
```
|
||
|
||
Establish the evidence loop:
|
||
|
||
1. Create `TestEvidence/` directory
|
||
2. Write `TestEvidence/restore-drill-2026-07-04-forgejo.md` — copy or link
|
||
`railiance-infra/docs/forgejo-restore-drill-evidence.md` with measured
|
||
RTO/RPO, result, issues found, and follow-up actions
|
||
3. Write `TestEvidence/README.md` — evidence file naming convention, required
|
||
fields (date, system, RPO achieved, RTO achieved, operator, result,
|
||
follow-ups), and quarterly drill cadence
|
||
|
||
Define the next untested restore target (Gitea dump decrypt, secret vault cold
|
||
restore, or S2 `railiance-backup-s2` decrypt) as a follow-up action — do not
|
||
block this task on running it.
|
||
|
||
Feeds `DISCTL-WP-0002-T06` (partial — next drill execution remains in WP-0002). |