railiance-platform/workplans/RPF-WP-0053-coulomb-paired-backup-admission.md

---
id: RPF-WP-0053
type: workplan
title: "Admit Coulomb paired archives and independent email-ledger recovery"
domain: financials
repo: railiance-platform
status: proposed
owner: codex
topic_slug: coulomb-social
created: "2026-10-07"
updated: "2026-10-08"
state_hub_workstream_id: "cb55e474-c7a5-5c6f-bd72-ce65f853e3e3"
---

Consumer request from CSOC-WP-0008-T02. The app now has real paired
quiesced database/content capture and isolated restore evidence; local temporary
copies and database-only CNPG jobs do not provide its durable recovery contract.
This is a proposed platform admission record, not an owner acceptance or live
approval. Existing mechanism ownership stays with railiance-platform.

Reviewable consumer implementation: coulomb-social/scripts/paired-backup.py,
scripts/snapshot-content-and-db.py and deploy/backup/*.service/*.timer.
Consumer runbook: coulomb-social/docs/release/paired-backup-operations-2026-10-07.md.
No credential was fetched, no timer enabled, no live upload/overwrite performed.

## T01 — Review and accept the paired backup execution identity

```task
id: RPF-WP-0053-T01
status: progress
priority: high
state_hub_task_id: "ceb58abf-77ef-5d75-a65b-fc67bfd9aa54"
```

Choose the platform execution owner: the prepared node systemd units or the
existing bounded activity-core execution lane. Review single-writer quiescence,
lock, termination/resume behavior, exact cluster UID, app digest and data scope.
Node systemd candidate uses root/k3s admin, guarded to coulomb-social and
coulomb_social_db; that authority is not a new credential grant and needs
explicit owner/founder admission before unattended operation. Prefer a scoped
platform identity where its execution contract supports coherent DB/PVC capture.
Do not mark the warden offsite workload identity resolved without a declaration.

## T02 — Admit offsite prefix, retention and caller custody

```task
id: RPF-WP-0053-T02
status: wait
priority: high
needs_human: true
intervention_note: "Platform must admit the exact Scaleway primary archive caller/prefix, reconcile its current custody front door and retention/capacity, and deliver bounded credentials before activation."
state_hub_task_id: "b06f1a44-1a55-588c-9c6c-3c1e290544df"
```

The original Nextcloud overwrite-slot proposal is withdrawn; its current workload
uploader is create-only and secondary. Reuse the platform primary contract:
s3://railiance-platform-pg-backup/platform-pg/application-archives/coulomb-social/,
Standard Multi-AZ nl-ams. Exact review package is
docs/credential-lane-designs/coulomb-paired-backup-primary.md.

scripts/primary_application_archive.py implements conditional immutable multipart
completion, own unfinished-upload abort, version-pinned GET and ciphertext hash
verification. Independent retrieve needs only a retained public receipt, no
original archive/node. requirements/backup-primary.lock pins seven packages with
hashes in a dedicated backup-venv; SDK conditional-completion shape is verified.
Synthetic provider tests cover uncertain completion, replay without new upload,
corrupted bytes, changed versions, caller scope and node-loss retrieval. No live
provider rights or custody are inferred from these tests.

CCR-2026-0012 identifies existing primary custody coordinates and its approved
rapp-postgres/databases delivery, not a new consumer. Its historical catalog id
backup-object-storage is unknown to the current local warden CLI. Owner must
reconcile the front door, choose/admit appropriately prefix-scoped IAM authority
and deliver AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY to root-owned mode 0600
/etc/coulomb-social-backup/primary.env (parent mode 0700). Metadata-only node check found
that file absent. No Secret fetch/copy or namespace expansion occurred.
No bootstrap key or age identity belongs in the scheduled worker.

Separate non-mutating application_archive_retention_plan.py proposes 14 daily/4
weekly distinct generations and protects newest plus the exact last application-
recovered version/hash. A missing/changed proof refuses; foreign/unverified points
are ignored. This is a review, not a deletion executor or activated lifecycle.
Bind local/provider headroom, independently retained metadata receipts, protected
recovery and compatibility with the existing 30-day bucket lifecycle before
activation. No retained-object/version-delete rights belong in the publisher.

## T03 — Activate and verify retrieval and application recovery

```task
id: RPF-WP-0053-T03
status: wait
priority: high
needs_human: true
intervention_note: "Needs T01/T02 owner admission and founder go for the exact first quiescence/upload and schedule; independent offsite retrieval and recovery proof remain required."
state_hub_task_id: "871c14da-2940-5be6-8ce8-953973422064"
```

Founder ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED, plus
explicit primary archive creation/version-read authority. Proposed UTC daily03:15
(+up to 5min jitter), RPO target24h, hourly stale threshold 26h, provisional
RTO target60min for an admitted attended restore. These are proposal targets,
not measured promises. First capture/upload is bounded and app resumes before
encryption/upload. Retrieve using the independently retained public receipt,
compare encrypted hash, decrypt through recovery
escrow with consumer recover-paired-backup.py (strict three-file packet/hash and
capture/image binding), then run restore-paired-snapshot.py and compare
row/sequence/file hashes, actual rendering and denial. Record receipt paths,
counts/timing only; preserve the source recovery point on any failed check.
No production data restore is part of backup schedule activation.

## T04 — Bind monitoring and recurring recovery custody

```task
id: RPF-WP-0053-T04
status: todo
priority: high
state_hub_task_id: "687fd36d-98a9-5c10-833e-b6bda51fa07b"
```

Connect non-secret freshness/provider receipts and failed units to the
established evidence owner (RPF-WP-0036-T03/T04), prove reception and failure
response; local journald failure alone is not monitored acceptance. Name the
recovery custodian and approve a monthly isolated drill with offsite readback.
Verify key recovery survives node/workstation loss. Record release-triggered
backup helper digest refresh and admitted generation/capacity/retention policy.
CSOC-WP-0008-T02 stays progress until operating evidence and service contract are accepted.

## Implementation evidence — 2026-10-07

New primary/retention 30 cases and existing backup tier/archive lifecycle 23 cases
pass (53 total); no live provider/custody interaction. Consumer full make check
passes 150 tests/mandatory browser journeys and Ruff; real synthetic age recovery
roundtrip passes and all four systemd units verify. T01 is progress for the
prepared execution package; owner identity acceptance remains open. T02/T03 remain
wait for custody/admission and exact activation; no timer installed/enabled and
no productive ciphertext uploaded. Email-connect's separate transport profile
is not ledger capture/recovery coverage; CSOC-WP-0010-T04/T05 own that residual.

## T05 — Admit the independent email-connect ledger archive caller

```task
id: RPF-WP-0053-T05
status: progress
priority: high
needs_human: true
intervention_note: "email-connect primary.env is delivered from the accepted reader. Retention, capacity, independent receipts, escrow and the first capture remain open. Coulomb node delivery was not requested."
state_hub_task_id: "3f744ce6-28e4-5496-8d21-7231271a7bc8"
```

Consumer EMAIL-WP-0007 implements default-preview quiesced SQLite backup, stable
age sealing, verified primary publication and attended isolated recovery. Its
email-connect profile is separate from Coulomb/Barman:
platform-pg/application-archives/email-connect/. Review packet:
email-connect/docs/ledger-backup-review-2026-10-08.md. No SMTP or ingest credential
belongs to the backup worker. Candidate delivery: root-owned mode 0600
/etc/email-connect-backup/primary.env (parent mode 0700), only owner-delivered
AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY. Metadata-only node check: file absent.
Do not copy the databases backup Secret or Coulomb caller, widen ESO admission,
or give publisher retained-object/version-delete or age escrow authority.

Bind appropriately prefix-scoped provider authority and current custody front
door; existing CCR-2026-0012 is a coordinate, not this consumer's grant. Use the
reviewed hash-locked platform backup runtime. Admit independent receipt/key/image
recovery and ledger growth/headroom, local/provider generation retention and
compatibility with existing 30-day lifecycle. Provisional daily 04:15 UTC, RPO 24h,
stale 26h, RTO 60min need owner acceptance; they are not measured promises.

The SQLite snapshot preserves state only at capture. A later provider submission
or challenge consume/cancel can be lost by restoring an older point. Production
incident recovery must fence caller/outbox/challenge traffic and reconcile the
post-capture window with user-engine/identity/provider owners before release;
never auto-enable a recovered receiver or resend an unknown outcome. Exact
first pause/upload/schedule, actual offsite/recovery proof and failure reception
remain EMAIL-WP-0007-T04/CSOC-WP-0010-T04/T05 obligations. Current receiver image
and SMTP/sender/caller admission remain unchanged by source preparation.


## T06 — Prepare atomic pair custody and its admitted front door

```task
id: RPF-WP-0053-T06
status: done
priority: high
state_hub_task_id: "ce950b3c-1ca6-5b01-bf2a-d958cafaed25"
```

2026-10-08 delivery preparation exposed a concrete custody gap: Warden's generic
paste-once desk performs a single-field KV PUT, replacing the whole record. Two
runs at one path discard the first field. Prepare a platform-owned pair-aware
custody procedure and reviewed writer identity/front door for the two new records,
initial CAS=0, both AWS fields in one atomic write, no secret argv/output/files,
bounded auth/self-revocation and metadata-only KV-version evidence. Do not use
the read-only delivery role, the structure-only administration lane, a generic
agent read grant, a retired ops-mason builder grant or two single-field desk PUTs.
Design/implementation can progress independently; actual provider key provisioning
needs its exact owner-reviewed attended custody act. This is a live handoff from
WARDEN-WP-0043 and the credential-delivery preparation, not a hidden prose residual.

## 2026-10-08 — Credential delivery preparation

Draft CCR-2026-0036 and CCR-2026-0037 now bind separate application KV paths,
five-minute/four-use native readers, exact namespaces/ServiceAccount/audience and
root-owned private delivery files. Both remain draft/unconfirmed, with no reviews
or native admission evidence. Their read policies match the existing generator;
validator errors and delegated-applier scope violations are empty, with expected
unconfirmed-binding warnings. Proposed ServiceAccounts disable automount and have
no new RBAC grants. No existing credential, Secret or cluster object was read or
changed.

scripts/application_backup_credentials.py previews without effects and performs
only admitted, source-hash/KV-version-pinned node reads; token revoked before atomic
no-overwrite private publication, readback equality, metadata-only receipts. It
refuses source/scope/version drift, dangerous files, injection, unconfirmed revoke
and a different existing file. A separate YAML-to-public-JSON exporter preserves
canonical source updates while keeping the node helper dependency-free. Thirty
synthetic delivery cases, four export cases and thirty primary/retention cases
pass; none establish live authorization. Review:
docs/credential-lane-designs/application-backup-delivery-2026-10-08.md.

WARDEN-WP-0043 fixes administration/provisioning intent collisions with surveys;
installed Warden was rebuilt, phase-memory retained, current catalog hash
5e6b217aa1f6 verified. Surveys now point to ops-mason-survey per MASON-IN-0004.
Structure requests require attended OIDC; pair provisioning is only a generic
platform pointer until T06 supplies the correct custody contract. T01/T02/T03/T05
remain open; no credentials delivered, productive capture/upload or timer enabled.

## 2026-10-08 — Pair writer preview

`scripts/application_backup_pair_custody.py` previews both draft CCRs. The
intended writer policy is `backup-pair-writer` and The founder admitted this writer on 2026-10-08, using the words
"bucket-pair-writer". The policy name remains `backup-pair-writer`
(`openbao/policies/backup-pair-writer.hcl`). The installed policy matches
that file, and `WRITER_ADMITTED` is true. Both backup-primary KV paths were absent. Version 2 is current on each path. Both pairs replaced the placeholder and
differ from each other. The values were not read out. The bucket policy now keeps those three statements and adds the coulomb-social
and email-connect publisher statements. Each new key aborted its own multipart
upload and was denied the sibling prefix, the apps-pg prefix, and object
delete. No completed object was stored. The bootstrap key cannot list
IAM applications (403). The live bucket
policy still names only application ee80333e-5bab-4a04-85a9-5bdb46b7b3c1 on
`platform-pg/*`. No publisher key was created. T05 stays wait for the two
console-created applications and their CAS=0 pairs. An admitted writer, covered
only by a synthetic transport, must present that one policy, TTL at most 5
minutes and at most 2 uses, prove metadata absence, send both AWS fields in one
`cas: 0` body, and revoke. Receipts contain no values. The delivery reader and
a one-field desk remain refused. No IAM application, provider key, OpenBao
version or node file was created. Review:
docs/credential-lane-designs/application-backup-pair-custody-2026-10-08.md.
T05 stays wait for owner admission of the writer, prefix and escrow. T06 stays
progress until that attended act.

## 2026-10-08 — Reader acceptance and email delivery

Founder acceptance is recorded on CCR-2026-0036 and CCR-2026-0037 with both
owner reviews, source ref founder-acceptance-2026-10-08, and confirmed
Kubernetes bindings. Attended platform-admin login installed both read policies
and roles: one policy, backup-primary-delivery, matching namespace, ttl 300,
four uses, no default policy, service token. Both ServiceAccounts exist with
automount disabled. Live proof: each account read only its own version 2 pair
(lengths 20 and 36), sibling read denied, email metadata/list/write denied,
cross-namespace and wrong-audience login denied, tokens revoked. Email delivery
admission is true. Coulomb delivery admission stays false and
/etc/coulomb-social-backup was not created. email-connect primary.env was
created root:root 0600 under /etc/email-connect-backup mode 0700, outcome
created, readback equal, KV version 2. No capture, upload, escrow or timer.