tegwick
b59b9a565c
Add RTO-RPO matrix, backup pickup queue tracker, and TestEvidence with Forgejo drill import. Mark T03, T04, T05 done.
96 lines
No EOL
3.5 KiB
Markdown
96 lines
No EOL
3.5 KiB
Markdown
# Restore Drill: Forgejo production
|
||
|
||
**Date:** 2026-07-04
|
||
**System:** `backup-forgejo-production` (Forgejo git + packages + `forgejo-db`)
|
||
**Operator:** Bernd (via grok session, `railiance-infra`)
|
||
**Result:** success
|
||
**Workplan:** RAIL-HO-WP-0005-T09
|
||
|
||
`no_secret_material_recorded: true`
|
||
|
||
## Summary
|
||
|
||
Proved that a production `forgejo dump` can be restored into an isolated
|
||
namespace and serve repository metadata without touching production Forgejo or
|
||
Gitea. This validates **recovery mechanics** — not operational backup coverage.
|
||
|
||
## Backup artifact
|
||
|
||
| Field | Value |
|
||
| --- | --- |
|
||
| Method | `forgejo dump` from production pod (manual) |
|
||
| Production pod | `forgejo-gitea-64c5b57684-ph9vt` (namespace `forgejo`) |
|
||
| Archive path | `/tmp/forgejo-drill/forgejo-drill-backup.zip` (workstation) |
|
||
| Archive size | 12,284,847 bytes (~11.7 MiB) |
|
||
| Archive timestamp | 2026-07-04 11:20 +0200 |
|
||
| Contents | `repos/`, `data/`, `forgejo-db.sql`, `app.ini` |
|
||
| Repos in dump | `forgejo-actions-probe`, `glas-harness`, `key-cape` (`coulomb/`) |
|
||
| Automated / offsite | **No** — workstation `/tmp` only; not from Nextcloud |
|
||
|
||
## Restore target
|
||
|
||
| Field | Value |
|
||
| --- | --- |
|
||
| Namespace | `forgejo-restore-drill` |
|
||
| Database | CNPG `forgejo-db-restore` (isolated, 1 instance) |
|
||
| App data PVC | `forgejo-restore-data` (`local-path`, 10Gi) |
|
||
| Helm release | `forgejo-restore` (`gitea-charts/gitea` 12.5.0) |
|
||
| Orchestration | `railiance-infra/tools/forgejo-restore-drill.sh` |
|
||
|
||
Restore steps (Forgejo 11.0.3 — no `forgejo restore` CLI):
|
||
|
||
1. Unzip dump into import pod staging area.
|
||
2. Copy `repos/` → `/data/git/gitea-repositories/`.
|
||
3. Copy `data/` → `/data/` (packages, attachments, avatars).
|
||
4. Import `forgejo-db.sql` via `psql` into `forgejo-db-restore`.
|
||
5. Deploy isolated Helm release bound to restored PVC + restore DB host.
|
||
|
||
## Validation checks
|
||
|
||
Port-forward: `svc/forgejo-restore-gitea-http` → `127.0.0.1:13000`
|
||
|
||
| Check | Result |
|
||
| --- | --- |
|
||
| `GET /` health | HTTP 200 |
|
||
| `GET /api/v1/repos/coulomb/glas-harness` | `full_name=coulomb/glas-harness`, `default_branch=main` |
|
||
| `GET /api/v1/repos/coulomb/key-cape` | `full_name=coulomb/key-cape`, `default_branch=main` |
|
||
| `GET /api/v1/orgs/coulomb/repos` | 3 repos visible |
|
||
|
||
Script exit marker: `restore-drill-complete`
|
||
|
||
## RPO / RTO measured
|
||
|
||
| Metric | Target (adopted) | Achieved |
|
||
| --- | --- | --- |
|
||
| RPO | 24h | Point-in-time of manual dump — **no scheduled backup** |
|
||
| RTO | 4h | ~3–5 min (isolated namespace on railiance01) |
|
||
| Production impact | None | Read-only dump; separate namespace |
|
||
|
||
Posture: RTO target met. RPO target **not operational** until daily automated
|
||
backups run (see `BackupPickupQueue.md` items 3–4).
|
||
|
||
## Issues found
|
||
|
||
- No scheduled backups; CNPG `Backup` CRs not configured.
|
||
- Dump stored on workstation only — no encryption-at-rest on approved offsite target.
|
||
- SQL import not idempotent — re-runs need `DRILL_CLEAN=1`.
|
||
|
||
## Follow-up actions
|
||
|
||
| Action | Owner | Queue / workplan |
|
||
| --- | --- | --- |
|
||
| Wire daily `forgejo dump` + Nextcloud upload | `railiance-platform` | Pickup #3 |
|
||
| Restore platform `railiance-backup` tool | `railiance-platform` | Pickup #4 |
|
||
| Re-drill from Nextcloud artifact | `railiance-infra` | Promotion gate (`BackupPolicy.md` §7) |
|
||
| 7 consecutive daily backup successes | `railiance-platform` | Promotion gate |
|
||
|
||
## Canonical evidence
|
||
|
||
Full drill record (owner repo):
|
||
|
||
`railiance-infra/docs/forgejo-restore-drill-evidence.md`
|
||
|
||
Supporting artifacts:
|
||
|
||
- `railiance-infra/tools/forgejo-restore-drill.sh`
|
||
- `railiance-infra/infra/forgejo-restore-drill/` |