--- id: RPF-WP-0029 type: workplan title: "Remove backup credential default and verify governed replacement" domain: financials repo: railiance-platform status: blocked owner: codex created: "2026-09-05" updated: "2026-09-06" state_hub_workstream_id: "bb326ebb-a313-549e-b35f-1bf17e1c58fd" --- # Remove backup credential default and verify governed replacement Source: State Hub message `ee702ac9-9118-4b9b-963a-01943052b65a`. Reviewed against current repository state on 2026-09-05. Repository implementation is complete; live closure remains pending. ## Remove source fallback and verify fail-closed behavior ```task id: RPF-WP-0029-T01 status: done priority: high state_hub_task_id: "4b5aefdb-a746-54f9-ba29-ebb840e7848d" ``` Removed the literal upload credential default from tools/cmd/forgejo-backup. Missing governed input now fails before cluster operations with a value-free diagnostic; encryption dry-runs skip upload authentication. Offline tests prove missing-input denial, explicit input, and mocked OpenBao resolution. 2026-09-05 continuation: removed credentials and credential-bearing URLs from curl argv; curl config travels through stdin with curlrc disabled. Both upload backends suppress credential-bearing diagnostics, require HTTPS and reject redirects/non-success status. Added transport containment and failure tests. ## Invalidate predecessor and prove replacement recovery ```task id: RPF-WP-0029-T02 status: wait priority: high state_hub_task_id: "b3f3402f-890b-5781-9b3e-1c9c0d28cea8" ``` Provider-side invalidation and replacement custody need the attended provider owner and CCR-2026-0004 lifecycle procedure. Record only non-secret invalidation, encrypted upload and restore receipts. No provider authority or replacement receipt was available; source removal alone does not close the reported exposure. Never record the predecessor value, fingerprint, length, or shape. Prepared owner execution procedure: `docs/backup-credential-recovery.md`. The dedicated Backup account cutover is complete under T03. Awaiting owner authority for invalidating the old Bernd-owned share. Full offsite application recovery passed on September 6 (evidence below). Activity-core is also a consumer of this upload lane. Preserve AGE_PRIVATE_KEY and historical exposure evidence; upload-token rotation cannot clear recovery-key taint. T03 proves encrypted fixture transport and decryption; September 6 evidence also proves full application recovery. Historical predecessor invalidation remains open. ## Portfolio review — 2026-09-05 INTENT binding: dependable backup custody and recovery. Retain T02 until the provider owner invalidates the predecessor and supplies replacement upload and restore receipts through CCR-2026-0004. Source removal is already complete. S1 backup scheduling belongs to RAIL-HO-WP-0012; forge backup orchestration and artifact retention belong to railiance-forge. RPF-WP-0036-T06 will obtain an accepted compatibility handoff, but this exposure obligation stays visible here until its evidence is accepted. No rotation was executed in this review. ## Move future backups to the dedicated Backup account ```task id: RPF-WP-0029-T03 status: done priority: high state_hub_task_id: "f85b1b4e-9a20-56e2-9e7d-d1d6f1c77bd9" ``` User explicitly selected Nextcloud user `Backup` with 10 GB quota and operator credentials at UI `secrets/operators/nextcloud/backup`. Live resolution is KVv2 `operators/nextcloud/backup`, fields BACKUP_USERNAME/BACKUP_PASSWORD. Native WebDAV login verified; actual quota is 10737418240 bytes (10 GiB). Keep the account password in operator custody, create a Backup-owned create-only share for workload delivery, and preserve the existing age escrow and retained data. Prove encrypted upload/download/decryption and workload delivery. No automatic pruning or personal-account revocation is inferred from this account change. The historical predecessor invalidation obligation in T02 remains separate. Completed 2026-09-05: Backup-owned share permissions 4; upload HTTP 201, owner download/decryption passed, runtime GET/DELETE denied with HTTP 405. Workload KV CAS 2→3 preserved age escrow. ExternalSecret delivery and all three activity-core deployments verified ready with new credentials loaded. All test fixtures removed. Account quota is enforced by Nextcloud; no automatic pruning. Evidence: `docs/evidence/RPF-WP-0029-backup-account-2026-09-05.json`. ## Real recovery attempt — 2026-09-05 The September 4 encrypted backup passed new-account upload/download/decryption but failed ZIP integrity, so no application restore was claimed. Fixed the capture path to require exact producer success, transferred-byte identity and ZIP CRC validation before encryption. A fresh 5,351,717,533-byte production archive passed all three gates; it is encrypted in private local staging. 200 tests passed. The isolated restore implementation now requires a hash-bound receipt for fetched bytes and uses disposable local credentials/network. A later attended OpenBao login failed before command handoff, with revocation not confirmed; the read-only predecessor probe did not execute. Do not reuse that attempt or infer provider invalidation. T02 waits for a fresh attended login to upload/fetch the validated archive and execute isolated recovery, plus the old Bernd-owned share's invalidation receipt or protected owner path. The failed candidate's newly created offsite copy also needs exact owner cleanup. History: `history/2026-09-05-wp0029-real-offsite-recovery.md`. Worker follow-through: its mounted source was still old. Deployed the immutable `backup-verified-0220ca56520c` ConfigMap via the scoped platform helper, rolled actcore-worker and verified all four source hashes. Future worker entry-point invocations use the integrity checks without editing the host checkout. Fresh encrypted archive: 5,353,024,293 bytes; 142 repository HEAD entries. Local drill plaintext and producer temporary files were removed. Owner login and the two remaining acceptance results above are still required. ## Primary/secondary boundary correction — 2026-09-06 User reaffirmed Scaleway as the primary backup provider. Nextcloud remains the independent secondary lane. An actual isolated apps-pg recovery from Scaleway passed in 42.64 seconds, including expected databases and consumer connection limits; production stayed Ready and scratch resources were deleted. This is primary database recovery evidence, not Forgejo or Nextcloud recovery proof. The live primary covers apps-pg/platform-pg/platform-pg-2; forgejo-db has no Barman destination and the Forgejo full-archive uploader still targets Nextcloud. Do not conflate this coverage gap with the old-share incident or silently move archives into a database-owned prefix. WP-0029's secondary acceptance gates remain explicit. Source/platform assurance records now name the correct primary. ## Follow-up — 2026-09-06 Repaired the missing WSL browser-launcher path while retaining Warden containment. Fresh attended login reached the owner command, the full verified 5.35 GB archive uploaded to Backup (201), downloaded (200), matched ciphertext and decrypted archive hashes, and Warden exited 0 after session cleanup. Evidence: `docs/evidence/RPF-WP-0029-secondary-transfer-2026-09-06.json`. Application recovery and predecessor invalidation remain separate gates. The earlier primary gap observation is superseded for forgejo-db: native Scaleway base backup/WAL and isolated database recovery now pass under RPF-WP-0038-T02/T03. Forgejo's primary full-archive delivery remains T04. The Backup account's 10 GiB quota holds only about two current full archives; growth and other consumers require a bounded retention policy, not automatic deletion of retained backups. Replacement recovery PASSED: isolated Forgejo healthy, 142 repositories, six users, two public Git clones plus fsck, and all 2,040 package blob digests verified. Disposable resources removed. Evidence: `docs/evidence/RPF-WP-0029-secondary-restore-2026-09-06.json`. T02 remains `wait` solely for the Bernd-owned predecessor invalidation/custody receipt; do not repeat the completed replacement restore as an open gate. Rejected drill-copy cleanup completed with conditional DELETE 204; attended session exited 0. Temporary plaintext removed; good encrypted backups retained.