d05c3000c5
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a06ecb-456a-71c2-b41e-0755d336e883
106 lines
4.6 KiB
Markdown
106 lines
4.6 KiB
Markdown
# Forgejo backup (railiance01)
|
|
|
|
Workplan: `RAIL-HO-WP-0005` T04/T09 · Secondary copy: Nextcloud + age
|
|
|
|
Scaleway is the platform primary backup provider. This helper currently writes
|
|
Forgejo archives only to Nextcloud. Live inspection on 2026-09-06 found no
|
|
Barman destination on forgejo-db and no reviewed Scaleway blob/archive path.
|
|
Treat Forgejo primary coverage as a gap; primary service selection alone does
|
|
not prove each asset has migrated.
|
|
|
|
## What is backed up
|
|
|
|
| Artifact | Source | Format |
|
|
| --- | --- | --- |
|
|
| Blob state | `forgejo dump` in production pod | zip → age |
|
|
| PostgreSQL | `pg_dump --no-owner --no-acl` from CNPG `forgejo-db` | SQL → age |
|
|
|
|
Covers git repos, packages (OCI/npm/generic), attachments, LFS, avatars, and DB metadata.
|
|
|
|
## Operator commands
|
|
|
|
```bash
|
|
cd ~/railiance-platform
|
|
make forgejo-backup-dry-run # local encrypt only, no upload
|
|
make forgejo-backup # encrypt + upload to Nextcloud forgejo/
|
|
make forgejo-backup-status # last success + 7-day gate hint
|
|
```
|
|
|
|
Requires: `kubectl`, `age`, Python 3, and curl or the Python upload backend.
|
|
Set `KUBECONFIG` to the currently verified railiance01 access path.
|
|
|
|
Preflight: `forgejo-db` pod must be Ready (`make forgejo-db-status`). The validated 2026-09-05 full archive is 5,351,717,533 bytes before age encryption.
|
|
Allow for substantial transfer time and check the account quota before upload.
|
|
This size supersedes the historical ~670 MiB estimate.
|
|
|
|
Uploads require the governed `CCR-2026-0004` OpenBao offsite lane or explicitly
|
|
provided `RAILIANCE_BACKUP_NC_TOKEN` / `RAILIANCE_BACKUP_NC_WEBDAV_URL` inputs.
|
|
There is no built-in credential fallback. Missing credentials stop execution
|
|
before any cluster dump. Local encryption dry-runs skip upload authentication.
|
|
Provider rotation and replacement upload/restore proof are tracked separately
|
|
in `RPF-WP-0029`; removing the source default does not prove revocation.
|
|
The concrete owner procedure is `docs/backup-credential-recovery.md`. Upload
|
|
credentials and credential-bearing URLs are passed to curl through stdin,
|
|
and backend errors never print those values. Redirects are refused.
|
|
|
|
Decrypt: `~/.config/age/railiance-backup.key` (same key as other Railiance backups).
|
|
|
|
## Nextcloud layout
|
|
|
|
Since 2026-09-05, uploads go to user `Backup`, folder `/railiance-backups`,
|
|
through a create-only share delivered from OpenBao. The account has **10 GiB**
|
|
total capacity; the retention target below is subject to available space.
|
|
Operator credentials stay in `operators/nextcloud/backup`; workload jobs receive
|
|
only the derived upload grant. See [account and recovery procedure](backup-credential-recovery.md).
|
|
|
|
```
|
|
forgejo/forgejo-dump-<timestamp>.zip.age
|
|
forgejo/forgejo-db-<timestamp>.sql.age
|
|
forgejo/forgejo-dump-weekly-<timestamp>.zip.age # Sundays only
|
|
forgejo/forgejo-db-weekly-<timestamp>.sql.age
|
|
```
|
|
|
|
Retention target: **14 daily + 4 weekly** on Nextcloud (operator may prune old
|
|
objects in the WebDAV folder; local cache keeps 7 per type).
|
|
|
|
## Cron (workstation)
|
|
|
|
```cron
|
|
# Daily 02:15 UTC — Forgejo backup (RPO 24h)
|
|
15 2 * * * cd $HOME/railiance-platform && make forgejo-backup >>$HOME/.cache/railiance/backups/forgejo/cron.log 2>&1
|
|
```
|
|
|
|
## Promotion gate (tier-3 cutover)
|
|
|
|
Do not promote further production repos until:
|
|
|
|
1. `make forgejo-backup` succeeds **7 consecutive days** (check `success-log`).
|
|
2. One restore drill uses a Nextcloud artifact (not `/tmp/forgejo-drill/`).
|
|
|
|
## Restore
|
|
|
|
See `railiance-infra`:
|
|
|
|
- `tools/forgejo-restore-drill.sh`
|
|
- `docs/forgejo-restore-drill-evidence.md`
|
|
|
|
Download and decrypt a dump from Nextcloud, set `BACKUP_LOCAL` to the zip path,
|
|
then run the drill script.
|
|
|
|
## Archive-integrity incident — 2026-09-05
|
|
|
|
The September 4 encrypted cache sample decrypted but failed ZIP validation.
|
|
The backup command now waits for the exact dump process to succeed, checks
|
|
transfer size/hash and validates ZIP CRCs before encryption/upload. A fresh
|
|
5.35 GB archive passed these checks. This does not certify the other historical
|
|
backups or establish a completed offsite application restore. See WP-0029 and
|
|
its dated evidence; never report a nonempty file as a successful backup.
|
|
|
|
|
|
Activity-core currently receives the validated entry point through the immutable
|
|
`backup-verified-0220ca56520c` bundle, applied by
|
|
`scripts/deploy_verified_backup_bundle.py`. The wrapper mounts over the existing
|
|
backup entry point; helpers/libraries live at `/opt/railiance-backup-verified`.
|
|
Reapply the source-owned delivery helper after reviewed backup changes and verify
|
|
its hash/rollout receipt. Roll back through the previous worker ReplicaSet; do
|
|
not edit another machine's host checkout or remove referenced ConfigMaps.
|