96 lines
3.5 KiB
Markdown
96 lines
3.5 KiB
Markdown
|
# Restore Drill: Forgejo production
|
|||
|
|
|
|||
|
|
**Date:** 2026-07-04
|
|||
|
|
**System:** `backup-forgejo-production` (Forgejo git + packages + `forgejo-db`)
|
|||
|
|
**Operator:** Bernd (via grok session, `railiance-infra`)
|
|||
|
|
**Result:** success
|
|||
|
|
**Workplan:** RAIL-HO-WP-0005-T09
|
|||
|
|
|
|||
|
|
`no_secret_material_recorded: true`
|
|||
|
|
|
|||
|
|
## Summary
|
|||
|
|
|
|||
|
|
Proved that a production `forgejo dump` can be restored into an isolated
|
|||
|
|
namespace and serve repository metadata without touching production Forgejo or
|
|||
|
|
Gitea. This validates **recovery mechanics** — not operational backup coverage.
|
|||
|
|
|
|||
|
|
## Backup artifact
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
| --- | --- |
|
|||
|
|
| Method | `forgejo dump` from production pod (manual) |
|
|||
|
|
| Production pod | `forgejo-gitea-64c5b57684-ph9vt` (namespace `forgejo`) |
|
|||
|
|
| Archive path | `/tmp/forgejo-drill/forgejo-drill-backup.zip` (workstation) |
|
|||
|
|
| Archive size | 12,284,847 bytes (~11.7 MiB) |
|
|||
|
|
| Archive timestamp | 2026-07-04 11:20 +0200 |
|
|||
|
|
| Contents | `repos/`, `data/`, `forgejo-db.sql`, `app.ini` |
|
|||
|
|
| Repos in dump | `forgejo-actions-probe`, `glas-harness`, `key-cape` (`coulomb/`) |
|
|||
|
|
| Automated / offsite | **No** — workstation `/tmp` only; not from Nextcloud |
|
|||
|
|
|
|||
|
|
## Restore target
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
| --- | --- |
|
|||
|
|
| Namespace | `forgejo-restore-drill` |
|
|||
|
|
| Database | CNPG `forgejo-db-restore` (isolated, 1 instance) |
|
|||
|
|
| App data PVC | `forgejo-restore-data` (`local-path`, 10Gi) |
|
|||
|
|
| Helm release | `forgejo-restore` (`gitea-charts/gitea` 12.5.0) |
|
|||
|
|
| Orchestration | `railiance-infra/tools/forgejo-restore-drill.sh` |
|
|||
|
|
|
|||
|
|
Restore steps (Forgejo 11.0.3 — no `forgejo restore` CLI):
|
|||
|
|
|
|||
|
|
1. Unzip dump into import pod staging area.
|
|||
|
|
2. Copy `repos/` → `/data/git/gitea-repositories/`.
|
|||
|
|
3. Copy `data/` → `/data/` (packages, attachments, avatars).
|
|||
|
|
4. Import `forgejo-db.sql` via `psql` into `forgejo-db-restore`.
|
|||
|
|
5. Deploy isolated Helm release bound to restored PVC + restore DB host.
|
|||
|
|
|
|||
|
|
## Validation checks
|
|||
|
|
|
|||
|
|
Port-forward: `svc/forgejo-restore-gitea-http` → `127.0.0.1:13000`
|
|||
|
|
|
|||
|
|
| Check | Result |
|
|||
|
|
| --- | --- |
|
|||
|
|
| `GET /` health | HTTP 200 |
|
|||
|
|
| `GET /api/v1/repos/coulomb/glas-harness` | `full_name=coulomb/glas-harness`, `default_branch=main` |
|
|||
|
|
| `GET /api/v1/repos/coulomb/key-cape` | `full_name=coulomb/key-cape`, `default_branch=main` |
|
|||
|
|
| `GET /api/v1/orgs/coulomb/repos` | 3 repos visible |
|
|||
|
|
|
|||
|
|
Script exit marker: `restore-drill-complete`
|
|||
|
|
|
|||
|
|
## RPO / RTO measured
|
|||
|
|
|
|||
|
|
| Metric | Target (adopted) | Achieved |
|
|||
|
|
| --- | --- | --- |
|
|||
|
|
| RPO | 24h | Point-in-time of manual dump — **no scheduled backup** |
|
|||
|
|
| RTO | 4h | ~3–5 min (isolated namespace on railiance01) |
|
|||
|
|
| Production impact | None | Read-only dump; separate namespace |
|
|||
|
|
|
|||
|
|
Posture: RTO target met. RPO target **not operational** until daily automated
|
|||
|
|
backups run (see `BackupPickupQueue.md` items 3–4).
|
|||
|
|
|
|||
|
|
## Issues found
|
|||
|
|
|
|||
|
|
- No scheduled backups; CNPG `Backup` CRs not configured.
|
|||
|
|
- Dump stored on workstation only — no encryption-at-rest on approved offsite target.
|
|||
|
|
- SQL import not idempotent — re-runs need `DRILL_CLEAN=1`.
|
|||
|
|
|
|||
|
|
## Follow-up actions
|
|||
|
|
|
|||
|
|
| Action | Owner | Queue / workplan |
|
|||
|
|
| --- | --- | --- |
|
|||
|
|
| Wire daily `forgejo dump` + Nextcloud upload | `railiance-platform` | Pickup #3 |
|
|||
|
|
| Restore platform `railiance-backup` tool | `railiance-platform` | Pickup #4 |
|
|||
|
|
| Re-drill from Nextcloud artifact | `railiance-infra` | Promotion gate (`BackupPolicy.md` §7) |
|
|||
|
|
| 7 consecutive daily backup successes | `railiance-platform` | Promotion gate |
|
|||
|
|
|
|||
|
|
## Canonical evidence
|
|||
|
|
|
|||
|
|
Full drill record (owner repo):
|
|||
|
|
|
|||
|
|
`railiance-infra/docs/forgejo-restore-drill-evidence.md`
|
|||
|
|
|
|||
|
|
Supporting artifacts:
|
|||
|
|
|
|||
|
|
- `railiance-infra/tools/forgejo-restore-drill.sh`
|
|||
|
|
- `railiance-infra/infra/forgejo-restore-drill/`
|