railiance-platform/workplans/RPF-WP-0053-coulomb-paired-backup-admission.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
RPF-WP-0053	workplan	Admit Coulomb paired archives and independent email-ledger recovery	financials	railiance-platform	proposed	codex	coulomb-social	2026-10-07	2026-10-08	cb55e474-c7a5-5c6f-bd72-ce65f853e3e3

Consumer request from CSOC-WP-0008-T02. The app now has real paired quiesced database/content capture and isolated restore evidence; local temporary copies and database-only CNPG jobs do not provide its durable recovery contract. This is a proposed platform admission record, not an owner acceptance or live approval. Existing mechanism ownership stays with railiance-platform.

Reviewable consumer implementation: coulomb-social/scripts/paired-backup.py, scripts/snapshot-content-and-db.py and deploy/backup/.service/.timer. Consumer runbook: coulomb-social/docs/release/paired-backup-operations-2026-10-07.md. No credential was fetched, no timer enabled, no live upload/overwrite performed.

T01 — Review and accept the paired backup execution identity

id: RPF-WP-0053-T01
status: progress
priority: high
state_hub_task_id: "ceb58abf-77ef-5d75-a65b-fc67bfd9aa54"

Choose the platform execution owner: the prepared node systemd units or the existing bounded activity-core execution lane. Review single-writer quiescence, lock, termination/resume behavior, exact cluster UID, app digest and data scope. Node systemd candidate uses root/k3s admin, guarded to coulomb-social and coulomb_social_db; that authority is not a new credential grant and needs explicit owner/founder admission before unattended operation. Prefer a scoped platform identity where its execution contract supports coherent DB/PVC capture. Do not mark the warden offsite workload identity resolved without a declaration.

T02 — Admit offsite prefix, retention and caller custody

id: RPF-WP-0053-T02
status: wait
priority: high
needs_human: true
intervention_note: "Platform must admit the exact Scaleway primary archive caller/prefix, reconcile its current custody front door and retention/capacity, and deliver bounded credentials before activation."
state_hub_task_id: "b06f1a44-1a55-588c-9c6c-3c1e290544df"

The original Nextcloud overwrite-slot proposal is withdrawn; its current workload uploader is create-only and secondary. Reuse the platform primary contract: s3://railiance-platform-pg-backup/platform-pg/application-archives/coulomb-social/, Standard Multi-AZ nl-ams. Exact review package is docs/credential-lane-designs/coulomb-paired-backup-primary.md.

scripts/primary_application_archive.py implements conditional immutable multipart completion, own unfinished-upload abort, version-pinned GET and ciphertext hash verification. Independent retrieve needs only a retained public receipt, no original archive/node. requirements/backup-primary.lock pins seven packages with hashes in a dedicated backup-venv; SDK conditional-completion shape is verified. Synthetic provider tests cover uncertain completion, replay without new upload, corrupted bytes, changed versions, caller scope and node-loss retrieval. No live provider rights or custody are inferred from these tests.

CCR-2026-0012 identifies existing primary custody coordinates and its approved rapp-postgres/databases delivery, not a new consumer. Its historical catalog id backup-object-storage is unknown to the current local warden CLI. Owner must reconcile the front door, choose/admit appropriately prefix-scoped IAM authority and deliver AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY to root-owned mode 0600 /etc/coulomb-social-backup/primary.env (parent mode 0700). Metadata-only node check found that file absent. No Secret fetch/copy or namespace expansion occurred. No bootstrap key or age identity belongs in the scheduled worker.

Separate non-mutating application_archive_retention_plan.py proposes 14 daily/4 weekly distinct generations and protects newest plus the exact last application- recovered version/hash. A missing/changed proof refuses; foreign/unverified points are ignored. This is a review, not a deletion executor or activated lifecycle. Bind local/provider headroom, independently retained metadata receipts, protected recovery and compatibility with the existing 30-day bucket lifecycle before activation. No retained-object/version-delete rights belong in the publisher.

T03 — Activate and verify retrieval and application recovery

id: RPF-WP-0053-T03
status: wait
priority: high
needs_human: true
intervention_note: "Needs T01/T02 owner admission and founder go for the exact first quiescence/upload and schedule; independent offsite retrieval and recovery proof remain required."
state_hub_task_id: "871c14da-2940-5be6-8ce8-953973422064"

Founder ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED, plus explicit primary archive creation/version-read authority. Proposed UTC daily03:15 (+up to 5min jitter), RPO target24h, hourly stale threshold 26h, provisional RTO target60min for an admitted attended restore. These are proposal targets, not measured promises. First capture/upload is bounded and app resumes before encryption/upload. Retrieve using the independently retained public receipt, compare encrypted hash, decrypt through recovery escrow with consumer recover-paired-backup.py (strict three-file packet/hash and capture/image binding), then run restore-paired-snapshot.py and compare row/sequence/file hashes, actual rendering and denial. Record receipt paths, counts/timing only; preserve the source recovery point on any failed check. No production data restore is part of backup schedule activation.

T04 — Bind monitoring and recurring recovery custody

id: RPF-WP-0053-T04
status: todo
priority: high
state_hub_task_id: "687fd36d-98a9-5c10-833e-b6bda51fa07b"

Connect non-secret freshness/provider receipts and failed units to the established evidence owner (RPF-WP-0036-T03/T04), prove reception and failure response; local journald failure alone is not monitored acceptance. Name the recovery custodian and approve a monthly isolated drill with offsite readback. Verify key recovery survives node/workstation loss. Record release-triggered backup helper digest refresh and admitted generation/capacity/retention policy. CSOC-WP-0008-T02 stays progress until operating evidence and service contract are accepted.

Implementation evidence — 2026-10-07

New primary/retention 30 cases and existing backup tier/archive lifecycle 23 cases pass (53 total); no live provider/custody interaction. Consumer full make check passes 150 tests/mandatory browser journeys and Ruff; real synthetic age recovery roundtrip passes and all four systemd units verify. T01 is progress for the prepared execution package; owner identity acceptance remains open. T02/T03 remain wait for custody/admission and exact activation; no timer installed/enabled and no productive ciphertext uploaded. Email-connect's separate transport profile is not ledger capture/recovery coverage; CSOC-WP-0010-T04/T05 own that residual.

T05 — Admit the independent email-connect ledger archive caller

id: RPF-WP-0053-T05
status: progress
priority: high
needs_human: true
intervention_note: "email-connect primary.env is delivered from the accepted reader. Retention, capacity, independent receipts, escrow and the first capture remain open. Coulomb node delivery was not requested."
state_hub_task_id: "3f744ce6-28e4-5496-8d21-7231271a7bc8"

Consumer EMAIL-WP-0007 implements default-preview quiesced SQLite backup, stable age sealing, verified primary publication and attended isolated recovery. Its email-connect profile is separate from Coulomb/Barman: platform-pg/application-archives/email-connect/. Review packet: email-connect/docs/ledger-backup-review-2026-10-08.md. No SMTP or ingest credential belongs to the backup worker. Candidate delivery: root-owned mode 0600 /etc/email-connect-backup/primary.env (parent mode 0700), only owner-delivered AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY. Metadata-only node check: file absent. Do not copy the databases backup Secret or Coulomb caller, widen ESO admission, or give publisher retained-object/version-delete or age escrow authority.

Bind appropriately prefix-scoped provider authority and current custody front door; existing CCR-2026-0012 is a coordinate, not this consumer's grant. Use the reviewed hash-locked platform backup runtime. Admit independent receipt/key/image recovery and ledger growth/headroom, local/provider generation retention and compatibility with existing 30-day lifecycle. Provisional daily 04:15 UTC, RPO 24h, stale 26h, RTO 60min need owner acceptance; they are not measured promises.

The SQLite snapshot preserves state only at capture. A later provider submission or challenge consume/cancel can be lost by restoring an older point. Production incident recovery must fence caller/outbox/challenge traffic and reconcile the post-capture window with user-engine/identity/provider owners before release; never auto-enable a recovered receiver or resend an unknown outcome. Exact first pause/upload/schedule, actual offsite/recovery proof and failure reception remain EMAIL-WP-0007-T04/CSOC-WP-0010-T04/T05 obligations. Current receiver image and SMTP/sender/caller admission remain unchanged by source preparation.

T06 — Prepare atomic pair custody and its admitted front door

id: RPF-WP-0053-T06
status: done
priority: high
state_hub_task_id: "ce950b3c-1ca6-5b01-bf2a-d958cafaed25"

2026-10-08 delivery preparation exposed a concrete custody gap: Warden's generic paste-once desk performs a single-field KV PUT, replacing the whole record. Two runs at one path discard the first field. Prepare a platform-owned pair-aware custody procedure and reviewed writer identity/front door for the two new records, initial CAS=0, both AWS fields in one atomic write, no secret argv/output/files, bounded auth/self-revocation and metadata-only KV-version evidence. Do not use the read-only delivery role, the structure-only administration lane, a generic agent read grant, a retired ops-mason builder grant or two single-field desk PUTs. Design/implementation can progress independently; actual provider key provisioning needs its exact owner-reviewed attended custody act. This is a live handoff from WARDEN-WP-0043 and the credential-delivery preparation, not a hidden prose residual.

2026-10-08 — Credential delivery preparation

Draft CCR-2026-0036 and CCR-2026-0037 now bind separate application KV paths, five-minute/four-use native readers, exact namespaces/ServiceAccount/audience and root-owned private delivery files. Both remain draft/unconfirmed, with no reviews or native admission evidence. Their read policies match the existing generator; validator errors and delegated-applier scope violations are empty, with expected unconfirmed-binding warnings. Proposed ServiceAccounts disable automount and have no new RBAC grants. No existing credential, Secret or cluster object was read or changed.

scripts/application_backup_credentials.py previews without effects and performs only admitted, source-hash/KV-version-pinned node reads; token revoked before atomic no-overwrite private publication, readback equality, metadata-only receipts. It refuses source/scope/version drift, dangerous files, injection, unconfirmed revoke and a different existing file. A separate YAML-to-public-JSON exporter preserves canonical source updates while keeping the node helper dependency-free. Thirty synthetic delivery cases, four export cases and thirty primary/retention cases pass; none establish live authorization. Review: docs/credential-lane-designs/application-backup-delivery-2026-10-08.md.

WARDEN-WP-0043 fixes administration/provisioning intent collisions with surveys; installed Warden was rebuilt, phase-memory retained, current catalog hash 5e6b217aa1f6 verified. Surveys now point to ops-mason-survey per MASON-IN-0004. Structure requests require attended OIDC; pair provisioning is only a generic platform pointer until T06 supplies the correct custody contract. T01/T02/T03/T05 remain open; no credentials delivered, productive capture/upload or timer enabled.

2026-10-08 — Pair writer preview

scripts/application_backup_pair_custody.py previews both draft CCRs. The intended writer policy is backup-pair-writer and The founder admitted this writer on 2026-10-08, using the words "bucket-pair-writer". The policy name remains backup-pair-writer (openbao/policies/backup-pair-writer.hcl). The installed policy matches that file, and WRITER_ADMITTED is true. Both backup-primary KV paths were absent. Version 2 is current on each path. Both pairs replaced the placeholder and differ from each other. The values were not read out. The bucket policy now keeps those three statements and adds the coulomb-social and email-connect publisher statements. Each new key aborted its own multipart upload and was denied the sibling prefix, the apps-pg prefix, and object delete. No completed object was stored. The bootstrap key cannot list IAM applications (403). The live bucket policy still names only application ee80333e-5bab-4a04-85a9-5bdb46b7b3c1 on platform-pg/*. No publisher key was created. T05 stays wait for the two console-created applications and their CAS=0 pairs. An admitted writer, covered only by a synthetic transport, must present that one policy, TTL at most 5 minutes and at most 2 uses, prove metadata absence, send both AWS fields in one cas: 0 body, and revoke. Receipts contain no values. The delivery reader and a one-field desk remain refused. No IAM application, provider key, OpenBao version or node file was created. Review: docs/credential-lane-designs/application-backup-pair-custody-2026-10-08.md. T05 stays wait for owner admission of the writer, prefix and escrow. T06 stays progress until that attended act.

2026-10-08 — Reader acceptance and email delivery

Founder acceptance is recorded on CCR-2026-0036 and CCR-2026-0037 with both owner reviews, source ref founder-acceptance-2026-10-08, and confirmed Kubernetes bindings. Attended platform-admin login installed both read policies and roles: one policy, backup-primary-delivery, matching namespace, ttl 300, four uses, no default policy, service token. Both ServiceAccounts exist with automount disabled. Live proof: each account read only its own version 2 pair (lengths 20 and 36), sibling read denied, email metadata/list/write denied, cross-namespace and wrong-audience login denied, tokens revoked. Email delivery admission is true. Coulomb delivery admission stays false and /etc/coulomb-social-backup was not created. email-connect primary.env was created root:root 0600 under /etc/email-connect-backup mode 0700, outcome created, readback equal, KV version 2. No capture, upload, escrow or timer.