Complete DISCTL-WP-0003-T03 through T05

Add RTO-RPO matrix, backup pickup queue tracker, and TestEvidence
with Forgejo drill import. Mark T03, T04, T05 done.

TestEvidence/README.md

@@ -0,0 +1,54 @@
+# Test Evidence
+
+Dated restore-drill reports proving RTO/RPO targets in `RTO-RPO-Matrix.md` are
+actually met — not just documented.
+
+## Naming convention
+
+```text
+TestEvidence/restore-drill-YYYY-MM-DD-<system>.md
+```
+
+Examples: `restore-drill-2026-07-04-forgejo.md`
+
+## Required fields
+
+Every evidence file must include:
+
+| Field | Description |
+| --- | --- |
+| Date | Drill execution date (UTC) |
+| System | What was restored (register ID or system name) |
+| Backup artifact | Filename, size, timestamp, source (manual/automated) |
+| Target environment | Isolated namespace, staging host, clean VM, etc. |
+| Operator | Who ran the drill |
+| Result | `success` / `partial` / `failed` |
+| RPO achieved | Actual recovery point (how stale was the backup?) |
+| RTO achieved | Wall-clock time from start to validated service |
+| Checks performed | API calls, smoke tests, representative workflow |
+| Issues found | Anything that blocked or degraded restore |
+| Follow-up actions | Workplan items, pickup queue refs, register updates |
+| Canonical evidence | Link to owner-repo evidence if drill ran elsewhere |
+
+## Drill cadence
+
+| Tier | Minimum cadence | Owner |
+| --- | --- | --- |
+| T1 irreplaceable source of truth | Quarterly | Bernd |
+| T0 identity / trust | Quarterly (cold decrypt included) | Bernd |
+| T2 runtime platforms | Semi-annual | Bernd |
+
+Quarterly Forgejo drill gate: re-run `forgejo-restore-drill.sh` from an
+**automated** Nextcloud artifact once daily backups are wired (promotion gate
+in `BackupPolicy.md` §7).
+
+## Next untested targets
+
+Priority order after Forgejo manual drill (2026-07-04):
+
+1. **S2 k8s runtime** — decrypt `railiance-backup-s2` artifact on clean target;
+   validate Helm values redeploy
+2. **Dev workstation cold restore** — decrypt Nextcloud artifact without using
+   production key on same machine (password-manager key only)
+3. **Gitea dump** — manual `gitea dump` restore to staging before next migration wave
+4. **Secret vault cold decrypt** — SOPS/age recovery material (R5 mitigation)

TestEvidence/restore-drill-2026-07-04-forgejo.md

@@ -0,0 +1,96 @@
+# Restore Drill: Forgejo production
+
+**Date:** 2026-07-04  
+**System:** `backup-forgejo-production` (Forgejo git + packages + `forgejo-db`)  
+**Operator:** Bernd (via grok session, `railiance-infra`)  
+**Result:** success  
+**Workplan:** RAIL-HO-WP-0005-T09
+
+`no_secret_material_recorded: true`
+
+## Summary
+
+Proved that a production `forgejo dump` can be restored into an isolated
+namespace and serve repository metadata without touching production Forgejo or
+Gitea. This validates **recovery mechanics** — not operational backup coverage.
+
+## Backup artifact
+
+| Field | Value |
+| --- | --- |
+| Method | `forgejo dump` from production pod (manual) |
+| Production pod | `forgejo-gitea-64c5b57684-ph9vt` (namespace `forgejo`) |
+| Archive path | `/tmp/forgejo-drill/forgejo-drill-backup.zip` (workstation) |
+| Archive size | 12,284,847 bytes (~11.7 MiB) |
+| Archive timestamp | 2026-07-04 11:20 +0200 |
+| Contents | `repos/`, `data/`, `forgejo-db.sql`, `app.ini` |
+| Repos in dump | `forgejo-actions-probe`, `glas-harness`, `key-cape` (`coulomb/`) |
+| Automated / offsite | **No** — workstation `/tmp` only; not from Nextcloud |
+
+## Restore target
+
+| Field | Value |
+| --- | --- |
+| Namespace | `forgejo-restore-drill` |
+| Database | CNPG `forgejo-db-restore` (isolated, 1 instance) |
+| App data PVC | `forgejo-restore-data` (`local-path`, 10Gi) |
+| Helm release | `forgejo-restore` (`gitea-charts/gitea` 12.5.0) |
+| Orchestration | `railiance-infra/tools/forgejo-restore-drill.sh` |
+
+Restore steps (Forgejo 11.0.3 — no `forgejo restore` CLI):
+
+1. Unzip dump into import pod staging area.
+2. Copy `repos/` → `/data/git/gitea-repositories/`.
+3. Copy `data/` → `/data/` (packages, attachments, avatars).
+4. Import `forgejo-db.sql` via `psql` into `forgejo-db-restore`.
+5. Deploy isolated Helm release bound to restored PVC + restore DB host.
+
+## Validation checks
+
+Port-forward: `svc/forgejo-restore-gitea-http` → `127.0.0.1:13000`
+
+| Check | Result |
+| --- | --- |
+| `GET /` health | HTTP 200 |
+| `GET /api/v1/repos/coulomb/glas-harness` | `full_name=coulomb/glas-harness`, `default_branch=main` |
+| `GET /api/v1/repos/coulomb/key-cape` | `full_name=coulomb/key-cape`, `default_branch=main` |
+| `GET /api/v1/orgs/coulomb/repos` | 3 repos visible |
+
+Script exit marker: `restore-drill-complete`
+
+## RPO / RTO measured
+
+| Metric | Target (adopted) | Achieved |
+| --- | --- | --- |
+| RPO | 24h | Point-in-time of manual dump — **no scheduled backup** |
+| RTO | 4h | ~3–5 min (isolated namespace on railiance01) |
+| Production impact | None | Read-only dump; separate namespace |
+
+Posture: RTO target met. RPO target **not operational** until daily automated
+backups run (see `BackupPickupQueue.md` items 3–4).
+
+## Issues found
+
+- No scheduled backups; CNPG `Backup` CRs not configured.
+- Dump stored on workstation only — no encryption-at-rest on approved offsite target.
+- SQL import not idempotent — re-runs need `DRILL_CLEAN=1`.
+
+## Follow-up actions
+
+| Action | Owner | Queue / workplan |
+| --- | --- | --- |
+| Wire daily `forgejo dump` + Nextcloud upload | `railiance-platform` | Pickup #3 |
+| Restore platform `railiance-backup` tool | `railiance-platform` | Pickup #4 |
+| Re-drill from Nextcloud artifact | `railiance-infra` | Promotion gate (`BackupPolicy.md` §7) |
+| 7 consecutive daily backup successes | `railiance-platform` | Promotion gate |
+
+## Canonical evidence
+
+Full drill record (owner repo):
+
+`railiance-infra/docs/forgejo-restore-drill-evidence.md`
+
+Supporting artifacts:
+
+- `railiance-infra/tools/forgejo-restore-drill.sh`
+- `railiance-infra/infra/forgejo-restore-drill/`