railiance-apps/docs/app-data-backup-restore-handoff.md

App Data Backup And Restore Handoff

This document defines the S5 app release boundary for data durability. It does not create backup jobs, authorize a live restore drill, or move platform backup ownership into railiance-apps.

Current App Data

The live invited pilot uses vergabe_demo_company on databases/apps-pg, with runtime Secret reference vergabe-demo-company/vergabe-demo-env. It has separate Bound local-path claims for media (5Gi) and issue state (1Gi). See the current binding. Historical vergabe_db and July backup receipts below do not establish this new company's recovery.

RAPPS-WP-0014-T03 remains waiting: the September 25 rehearsal proved database restore and restart, with empty file stores and no off-host upload. A populated, coherent three-store recovery point and restored workflow are still required. The CNPG/storage backup mechanisms remain owned by railiance-platform.

Ownership Matrix

Concern	S5 app repo owns	Upstream owner
App database request	App name, namespace, database name, role name, intended use, and production-readiness need	railiance-platform reviews and provisions the role/database
Runtime DB Secret use	Secret name in the app namespace and URL-encoded DSN rebuild helper	railiance-platform owns platform credential source and future secret delivery
Database backup job	Readiness gate and consumer evidence requirement	railiance-platform owns CNPG backup and restore implementation
App restore verification	App-specific post-restore checks, migrations, login/smoke path, and rollback note	railiance-platform restores the backing database
Forge images/packages	Artifact identity and consumer evidence cited by app runbooks	railiance-forge owns registry/package restore evidence
App media/blob data	PVC declaration and app-level restore checks if enabled	railiance-platform owns storage backup mechanism once media is production-critical

Production Readiness Gate

Before an app release treats data as production-critical, the app runbook should record:

• data class: disposable, externally reproducible, or production-critical;
• owning platform workplan or doc for the backup mechanism;
• latest non-secret backup evidence reference;
• latest restore-drill evidence reference, ideally from an isolated environment;
• app-specific post-restore checks, such as migrations, health endpoint, login/admin path, and representative business workflow;
• rollback or disable path if restore fails;
• assertion that no secret material was copied into Git, logs, screenshots, or State Hub notes.

If this gate is missing, the app can still be used for smoke, development, or migration validation, but promotion beyond that should create or link a railiance-platform workplan.

Historical production cluster inventory (2026-07-10)

Custodian delivery-lane snapshot (the-custodian/docs/evidence/vergabe-teilnahme-delivery-lane-20260710.json) confirmed all four production CNPG clusters on railiance01 have spec.backup=null and no ScheduledBackup resources:

Cluster	Consumer impact
apps-pg	vergabe_db and future S5 app databases
forgejo-db	Forgejo metadata and package registry state
net-kingdom-pg	Net Kingdom relational data
state-hub-db	State Hub API persistence

Check current posture:

make cnpg-backup-status

Option A (decided, RAILIANCE-WP-0015): age-encrypted logical dumps → Nextcloud WebDAV via CronJobs in manifests/cnpg-option-a-backup.yaml.

bao login -method=oidc -path=netkingdom role=railiance-backup-workload-kv-read
make cnpg-backup-offsite-secret-apply   # Secret from OpenBao (no private key)
make cnpg-option-a-apply                # schedule CM + RBAC + suspended CronJobs
make cnpg-logical-backup                # immediate / daily workstation run
make cnpg-backup-status

Historical workstation schedule (superseded): RAPPS-WP-0002 records the July 22 move to in-cluster CronJobs on CoulombCore and activity-core on Railiance01. This old workstation example is not a current pilot backup receipt:

# Daily 02:30 UTC — Option A multi-cluster logical backup (RPO 24h)
30 2 * * * cd $HOME/railiance-apps && make cnpg-logical-backup >>$HOME/.cache/railiance/backups/cnpg/cron.log 2>&1

Barman ObjectStore / CNPG ScheduledBackup remain deferred (Phase 2 stubs in manifests/cnpg-backup-readiness.yaml).

Historical vergabe_db gate

Current posture (2026-07-12):

• database: vergabe_db on databases/apps-pg;
• app role Secret: vergabe-app-credentials;
• env Secret: vergabe-teilnahme-env;
• Phase 1 backup lane: make apps-pg-backup uploads encrypted logical dumps to the platform offsite lane (latest: apps-pg-vergabe_db-20260711T231430Z.dump.age);
• CNPG Option A posture: make cnpg-backup-status reports Option A CronJob + last-success coverage (not barman ScheduledBackup); see RAILIANCE-WP-0015 for fleet rollout;
• restore drill: isolated restore with row-count gate 195/195 pass (docs/evidence/apps-pg-restore-drill-20260711T230725Z.json);
• S5 production-trust gate: satisfied for Phase 1 — backup + restore evidence recorded in docs/evidence/apps-pg-backup-lane-20260711.json; app migration smoke on drill DB deferred (row gate sufficient for interim gate).

Evidence index (RAILIANCE-WP-0013):

Check	Evidence
Backup lane auth + upload	docs/evidence/apps-pg-backup-lane-20260711.json
Isolated restore drill	docs/evidence/apps-pg-restore-drill-20260711T230725Z.json
Operator commands	make apps-pg-backup, make apps-pg-restore-drill

Remaining before full production-critical promotion (post–Phase 1):

• vergabe-teilnahme migration smoke on restored database;
• health endpoint and HTTPS smoke checks after restore;
• representative tender-management workflow verification;
• Option A unattended coverage for all four CNPG clusters (manifests/cnpg-option-a-backup.yaml, RAILIANCE-WP-0015);
• any app media path remains disabled or has its own storage restore evidence.

Forge Artifact Evidence

S5 runbooks may cite forge-owned package and blob restore evidence, but must not own Gitea package backup procedures or registry credentials. Use /home/worsch/railiance-forge/docs/backup-restore-secret-handoff.md for the forge artifact boundary.

For app releases, cite:

• image repository, tag, and digest when available;
• source commit and package version;
• forge publish job or evidence reference;
• package/blob restore drill evidence when the artifact is production-critical;
• namespace-local pull Secret or approved workload secret path, without token values.

Filing Upstream Gaps

When the missing durability item is not local to S5:

1. Keep the S5 task focused on the app release impact.
2. Create or link the platform/forge workplan that owns the missing mechanism.
3. Mark the S5 task wait and its workplan blocked when the app release cannot safely continue without that upstream evidence.
4. Record the State Hub workstream/task id in the app runbook or workplan.
5. Revisit the S5 promotion gate after upstream evidence exists.