coulomb/railiance-platform
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Prepare immutable application archive transfer and retention admission
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
Details
CI Smoke / container-smoke (push) Successful in 2s
Details

Browse source 
Assistant: codex
Assistant-Model: gpt-6.1-sol
Assistant-Session: 01a11538-05ea-7b63-b3c7-a2df5f38898d
This commit is contained in:
codex 2026-10-07 23:23:33 +02:00
parent 59f973c2c5
commit a8fc1a3438
9 changed files with 973 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

75
docs/credential-lane-designs/coulomb-paired-backup-primary.md Normal file
View file

@ -0,0 +1,75 @@
# Coulomb paired-backup primary custody review — 2026-10-07
Pending admission under RPF-WP-0053-T01/T02; this is a concrete consumer request,
not an approved credential grant. No existing Secret, policy or grant is changed.
The revised consumer uses Scaleway primary storage, separate from native Barman
objects. The earlier Nextcloud overwrite-slot proposal is withdrawn: the current
Nextcloud workload uploader is create-only and remains the independent secondary.
| Coordinate | Requested contract |
| --- | --- |
| Consumer | Node-side coulomb-social-paired-backup.service; proposed root executor guarded by current cluster UID/app digest, subject to T01 review |
| Cluster | railiance01; kube-system UID a553c742-0115-43d4-99a4-a5ca56fe0786 |
| Destination | s3://railiance-platform-pg-backup/platform-pg/application-archives/coulomb-social/ |
| Provider | Scaleway Standard Multi-AZ, nl-ams; https://s3.nl-ams.scw.cloud |
| Source scope | coulomb_social_db on apps-pg plus the coulomb-social-content PVC, one quiesced capture |
| Publisher rights | Multipart creation/parts/completion, abort its own unfinished uploads, HEAD and version-pinned GET within the exact archive prefix; prefix-constrained listing if required for HEAD missing-object semantics |
| Publisher exclusions | No retained-object/version deletion, native Barman writes, other application prefixes, bucket/IAM configuration, escrow identity or bootstrap provider key |
| Worker delivery | Owner writes /etc/coulomb-social-backup/primary.env, root:root0600, parent0700; AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY only |
| Recovery custody | Independent read authority plus the age identity; public object key/version/hash/size receipt retained outside the node/workstation |
| Retention | Separate owner; proposed 14 daily/4 weekly recovery generations, deduplicated and protecting newest plus last successfully recovered point; not activated |
| Runtime | Dedicated /opt/railiance-platform/backup-venv, hash-locked requirements/backup-primary.lock; SDK conditional-completion field verified offline |
Existing CCR-2026-0012 identifies the custody owner and source
platform/workloads/railiance/backup/object-storage fields ACCESS_KEY/SECRET_KEY.
It authorizes the existing databases/platform-pg-backup-s3 delivery, not this
consumer. Do not copy that Secret, broaden its ESO namespace conditions, reuse the
bootstrap key or manufacture a successor grant. Owner must determine whether a
separate scoped IAM application/key is needed: Scaleway policies attach to the
application, so merely minting another key under a broad application does not
establish prefix isolation. A worker intentionally has no delete authority even
if an existing provider identity has wider capabilities.
Routing check: warden route show backup-object-storage returns unknown on this
workstation, despite that historical catalog id in CCR-2026-0012. Broad route
lookup returns the Nextcloud lane and generic OpenBao pointer. No secret retrieval
was attempted. Owner must reconcile the current primary front door and delivery;
record its review/grant reference here before execution. Node metadata-only check
on 2026-10-07: primary.env absent. Source presence is not custody admission.
Before first publication, prove bucket versioning and conditional create behavior
under this caller, missing-object HEAD behavior, denied foreign-prefix access and
retained-object deletion denial. Metadata/code tests do not prove those live
rights. The worker does not contact a provider during default preview. Its check
requires the SDK model and delivered fields, without printing values or contacting
Scaleway. Live transfer needs --execute; it never reads Kubernetes Secrets.
The platform documents a 30-day current/noncurrent lifecycle; exact prefix coverage
must be checked before accepting a 14/4 calendar contract. Four weekly selections,
protected recovery points and monthly drills can outlive an age-based expiration.
Do not activate an incompatible lifecycle or promise protected points survive it.
Owner must bind capacity/headroom, local encrypted-copy retention and independent
receipt retention before enabling an unattended timer. No live deletion/lifecycle
change is included in this request.
First run: approved current-image capture, app resumes, age sealing, immutable
conditional completion, version-pinned GET and ciphertext SHA-256 check. Receipt
explicitly says application_restore_verified=false. Recovery on a different
workspace uses the retained receipt with primary_application_archive.py retrieve,
then consumer recover-paired-backup.py and restore-paired-snapshot.py. Retrieve does
only version-pinned GET and needs no original archive. Decryption belongs to the
attended custodian, never the scheduled publisher. Complete rendering and denied
access checks and record full elapsed time before calling this recoverable service.
Provider contract reference: [Scaleway conditional writes](https://www.scaleway.com/en/docs/object-storage/api-cli/using-conditional-writes/).
If-None-Match is applied at multipart completion. The lock pins boto3 1.40.18 and
botocore 1.40.76; the actual botocore service model exposes IfNoneMatch. Synthetic
provider tests cover lost completion response, abort of own pending upload,
changed versions, corrupted readback, scope/hash binding, and node-loss retrieval.
Email-connect has a separate archive profile in the platform helper. That is a
transport capability only: this request neither captures its SQLite ledger nor
admits an email worker. EMAIL-WP-0006 deployment evidence is complete; ongoing
ledger recovery stays tracked by CSOC-WP-0010-T04/T05 and requires a separate
producer/custody/rehearsal contract.

27
requirements/README.md Normal file
View file

@ -0,0 +1,27 @@
# Primary application archive runtime
Use a dedicated platform-managed backup virtual environment. The app image does
not need boto3. backup-primary.in pins the boto3/botocore pair needed for conditional
multipart completion; backup-primary.lock also freezes all transitive dependencies
and accepted package hashes. Reviewed pair: boto3 1.40.18 / botocore 1.40.76.
After admission, prepare the runtime before a capture window:
```bash
uv venv /opt/railiance-platform/backup-venv
uv pip sync --python /opt/railiance-platform/backup-venv/bin/python \
--require-hashes requirements/backup-primary.lock
```
These are installation instructions, not an instruction to alter a running
workload or enable a schedule. Do not resolve dependencies during a scheduled
backup. Node/tool/actor admission remains RPF-WP-0053. Deploy these requirements
with the reviewed scripts and record installed hashes. Credentials are delivered
separately by the owner; no values belong in this directory.
An offline check with synthetic environment fields on 2026-10-07 proved the exact
installed botocore model exposes CompleteMultipartUpload.IfNoneMatch, without
provider contact. A presence check does not prove actual custody or provider
permissions. Requirements were compiled in /tmp; the comments preserve that
command's provenance. Regenerate from this repository's requirements/*.in with
uv pip compile --generate-hashes and review the new lock before installation.

2
requirements/backup-primary.in Normal file
View file

@ -0,0 +1,2 @@
boto3==1.40.18
botocore==1.40.76

35
requirements/backup-primary.lock Normal file
View file

@ -0,0 +1,35 @@
# This file was autogenerated by uv via the following command:
# uv pip compile --generate-hashes --output-file /tmp/backup-primary.lock /tmp/backup-primary.in
boto3==1.40.18 \
--hash=sha256:64301d39adecc154e3e595eaf0d4f28998ef0a5551f1d033aeac51a9e1a688e5 \
--hash=sha256:daa776ba1251a7458c9d6c7627873d0c2460c8e8272d35759065580e9193700a
# via -r /tmp/backup-primary.in
botocore==1.40.76 \
--hash=sha256:2b16024d68b29b973005adfb5039adfe9099ebe772d40a90ca89f2e165c495dc \
--hash=sha256:fe425d386e48ac64c81cbb4a7181688d813df2e2b4c78b95ebe833c9e868c6f4
# via
# -r /tmp/backup-primary.in
# boto3
# s3transfer
jmespath==1.1.0 \
--hash=sha256:472c87d80f36026ae83c6ddd0f1d05d4e510134ed462851fd5f754c8c3cbb88d \
--hash=sha256:a5663118de4908c91729bea0acadca56526eb2698e83de10cd116ae0f4e97c64
# via
# boto3
# botocore
python-dateutil==2.9.0.post0 \
--hash=sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3 \
--hash=sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427
# via botocore
s3transfer==0.13.1 \
--hash=sha256:a981aa7429be23fe6dfc13e80e4020057cbab622b08c0315288758d67cabc724 \
--hash=sha256:c3fdba22ba1bd367922f27ec8032d6a1cf5f10c934fb5d68cf60fd5a23d936cf
# via boto3
six==1.17.0 \
--hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \
--hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81
# via python-dateutil
urllib3==2.8.0 \
--hash=sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3 \
--hash=sha256:63bf2ead4c879426ebf22ef2a781eeb4aa3b4ae798a0435506f8687fd5bb9b63
# via botocore

114
scripts/application_archive_retention_plan.py Normal file
View file

@ -0,0 +1,114 @@
#!/usr/bin/env python3
"""Review primary archive generations; no provider or local deletion capability."""
from __future__ import annotations
import argparse
import json
import re
from datetime import UTC, datetime
from pathlib import Path
from primary_application_archive import PROFILES, require_version, validate_key
def identity(receipt):
return {
"object_key": receipt["object_key"],
"version_id": receipt["version_id"],
"ciphertext_sha256": receipt["ciphertext_sha256"],
"ciphertext_bytes": receipt["ciphertext_bytes"],
}
def plan(application, inventory, recovery, now=None):
now = now or datetime.now(UTC)
if (
recovery.get("status") != "application_restored_verified"
or recovery.get("application") != application
or recovery.get("application_restore_verified") is not True
):
raise ValueError("attended_application_recovery_proof_required")
protected = recovery.get("object_key", "")
validate_key(application, protected)
require_version(recovery.get("version_id"))
items, ignored = {}, []
for receipt in inventory:
key = receipt.get("object_key", "")
if (
receipt.get("application") != application
or receipt.get("status") != "offsite_fetched_verified"
or receipt.get("remote_retrieval_verified") is not True
or not isinstance(key, str)
or not key.startswith(PROFILES[application])
):
ignored.append(key)
continue
validate_key(application, key)
require_version(receipt.get("version_id"))
captured = datetime.fromisoformat(receipt["captured_at"].replace("Z", "+00:00"))
if (
captured.tzinfo is None
or captured > now
or key in items
or not re.fullmatch(r"[a-f0-9]{64}", receipt.get("ciphertext_sha256", ""))
or not isinstance(receipt.get("ciphertext_bytes"), int)
or receipt["ciphertext_bytes"] <= 0
):
raise ValueError("unverified_archive_inventory")
items[key] = (captured.astimezone(UTC), receipt)
if (
protected not in items
or items[protected][1]["version_id"] != recovery["version_id"]
or items[protected][1]["ciphertext_sha256"] != recovery.get("ciphertext_sha256")
):
raise ValueError("protected_recovery_point_missing_or_changed")
ordered = sorted(items, key=lambda key: (items[key][0], key), reverse=True)
daily, weekly = {}, {}
for key in ordered:
stamp = items[key][0]
daily.setdefault(stamp.date(), key)
weekly.setdefault(stamp.isocalendar()[:2], key)
keep = {protected, ordered[0]} | set(list(daily.values())[:14]) | set(list(weekly.values())[:4])
return {
"schema": "platform.application-retention-review.v1",
"status": "reviewed",
"application": application,
"daily_generations": 14,
"weekly_generations": 4,
"protected": identity(items[protected][1]),
"keep": [identity(items[key][1]) for key in ordered if key in keep],
"expiration_candidates": [identity(items[key][1]) for key in ordered if key not in keep],
"ignored": ignored,
"provider_mutations": False,
"local_mutations": False,
"activation_gate": "Separate owner must admit lifecycle/capacity, revalidate protected "
"recovery and exact versions before any expiration.",
}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--application", choices=PROFILES, required=True)
parser.add_argument("--inventory", type=Path, required=True)
parser.add_argument("--recovery-receipt", type=Path, required=True)
args = parser.parse_args()
print(
json.dumps(
plan(
args.application,
json.loads(args.inventory.read_text()),
json.loads(args.recovery_receipt.read_text()),
),
indent=2,
)
)
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except Exception as error:
print(json.dumps({"status": "failed", "failure_type": type(error).__name__}))
raise SystemExit(1)

375
scripts/primary_application_archive.py Normal file
View file

@ -0,0 +1,375 @@
#!/usr/bin/env python3
"""Immutable application archives through the platform's primary storage.
Credentials are owner-delivered environment fields, never read from another
workload's Secret. CLI transfer previews by default. Output is redacted.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import secrets
import stat
import tempfile
from datetime import UTC, datetime
from pathlib import Path
ENDPOINT = "https://s3.nl-ams.scw.cloud"
BUCKET = "railiance-platform-pg-backup"
PROFILES = {
"coulomb-social": "platform-pg/application-archives/coulomb-social/",
"email-connect": "platform-pg/application-archives/email-connect/",
}
PART_SIZE = 64 * 1024**2
MAX_BYTES = 20 * 1024**3
def private_directory(path):
info = path.lstat()
if not stat.S_ISDIR(info.st_mode) or info.st_uid != os.geteuid() or info.st_mode & 0o077:
raise ValueError("private_owner_directory_required")
def regular_file(path):
info = path.lstat()
if not stat.S_ISREG(info.st_mode) or info.st_uid != os.geteuid() or info.st_mode & 0o077:
raise ValueError("private_owner_file_required")
def digest(path):
with path.open("rb") as source:
return hashlib.file_digest(source, "sha256").hexdigest()
def checkpoint(path, receipt):
fd, name = tempfile.mkstemp(prefix="archive-receipt-", dir=path.parent)
with os.fdopen(fd, "w") as output:
json.dump(receipt, output, indent=2)
output.flush()
os.fsync(output.fileno())
Path(name).replace(path)
def key_for(application, captured_at):
stamp = datetime.fromisoformat(captured_at.replace("Z", "+00:00"))
if stamp.tzinfo is None:
raise ValueError("capture_timezone_required")
stamp = stamp.astimezone(UTC)
return (
PROFILES[application]
+ stamp.strftime("%Y/%m/%d/%Y%m%dT%H%M%SZ-")
+ secrets.token_hex(12)
+ ".tar.age"
)
def validate_key(application, key):
suffix = key.removeprefix(PROFILES[application])
if not key.startswith(PROFILES[application]) or not re.fullmatch(
r"\d{4}/\d{2}/\d{2}/\d{8}T\d{6}Z-[a-f0-9]{24}\.tar\.age", suffix
):
raise ValueError("archive_key_outside_contract")
def missing(error):
response = getattr(error, "response", {})
return response.get("ResponseMetadata", {}).get("HTTPStatusCode") == 404
def require_version(value):
if (
not isinstance(value, str)
or not value
or value == "null"
or any(c in value for c in "\r\n")
):
raise ValueError("versioned_archive_required")
return value
def existing(client, key, expected, size):
try:
head = client.head_object(Bucket=BUCKET, Key=key)
except Exception as error:
if missing(error):
return None
raise
if head.get("ContentLength") != size or head.get("Metadata", {}).get("sha256") != expected:
raise ValueError("existing_archive_binding_mismatch")
return require_version(head.get("VersionId"))
def transfer(client, application, source, output, receipt, save=lambda: None):
regular_file(source)
private_directory(output.parent)
size = source.stat().st_size
if not 0 < size <= MAX_BYTES or output.exists() or output.is_symlink():
raise ValueError("invalid_archive_source_or_output")
with source.open("rb") as stream:
if stream.read(22) != b"age-encryption.org/v1\n":
raise ValueError("encrypted_archive_required")
expected = digest(source)
key = receipt["object_key"]
validate_key(application, key)
if receipt.get("ciphertext_sha256", expected) != expected:
raise ValueError("archive_receipt_binding_mismatch")
receipt.update(
application=application,
destination="s3://" + BUCKET + "/" + key,
ciphertext_sha256=expected,
ciphertext_bytes=size,
remote_retrieval_verified=False,
stage="existing_object_check",
)
save()
version = existing(client, key, expected, size)
if receipt.get("version_id") and receipt["version_id"] != version:
raise ValueError("existing_archive_version_changed")
upload_id = None
if version is None:
try:
receipt["stage"] = "multipart_upload"
save()
upload_id = client.create_multipart_upload(
Bucket=BUCKET,
Key=key,
ContentType="application/octet-stream",
Metadata={"sha256": expected, "application": application},
)["UploadId"]
parts = []
with source.open("rb") as stream:
while block := stream.read(PART_SIZE):
number = len(parts) + 1
result = client.upload_part(
Bucket=BUCKET, Key=key, UploadId=upload_id, PartNumber=number, Body=block
)
parts.append({"PartNumber": number, "ETag": result["ETag"]})
completed = client.complete_multipart_upload(
Bucket=BUCKET,
Key=key,
UploadId=upload_id,
MultipartUpload={"Parts": parts},
IfNoneMatch="*",
)
upload_id = None
version = require_version(completed.get("VersionId"))
receipt["multipart_completed"] = True
finally:
if upload_id:
try:
client.abort_multipart_upload(Bucket=BUCKET, Key=key, UploadId=upload_id)
receipt["multipart_aborted"] = True
except Exception:
receipt["multipart_aborted"] = False
save()
else:
receipt["reused_verified_binding"] = True
receipt.update(stage="version_pinned_readback", version_id=version)
save()
download(client, key, version, size, expected, output)
receipt.update(
status="offsite_fetched_verified",
stage="transfer_verified",
remote_retrieval_verified=True,
immutable_create=True,
)
save()
return receipt
def download(client, key, version, size, expected, output):
private_directory(output.parent)
if output.exists() or output.is_symlink() or not 0 < size <= MAX_BYTES:
raise ValueError("fresh_bounded_readback_required")
response = client.get_object(Bucket=BUCKET, Key=key, VersionId=version)
if response["ContentLength"] != size or response.get("VersionId") != version:
response["Body"].close()
raise ValueError("readback_version_or_length_mismatch")
read_bytes = 0
with response["Body"] as body, output.open("xb") as target:
output.chmod(0o600)
while block := body.read(8 * 1024**2):
read_bytes += len(block)
if read_bytes > size:
raise ValueError("readback_exceeds_expected_size")
target.write(block)
if read_bytes != size or digest(output) != expected:
raise ValueError("readback_digest_mismatch")
def retrieve(client, application, output, receipt):
"""Node-loss recovery needs only the independently retained public receipt."""
key = receipt.get("object_key", "")
validate_key(application, key)
version = require_version(receipt.get("version_id"))
expected = receipt.get("ciphertext_sha256", "")
size = receipt.get("ciphertext_bytes", 0)
if (
receipt.get("schema") != "platform.application-primary-archive.v1"
or receipt.get("application") != application
or receipt.get("status") != "offsite_fetched_verified"
or receipt.get("remote_retrieval_verified") is not True
or receipt.get("destination") != "s3://" + BUCKET + "/" + key
or not re.fullmatch(r"[a-f0-9]{64}", expected)
or not isinstance(size, int)
or isinstance(size, bool)
or not 0 < size <= MAX_BYTES
):
raise ValueError("verified_recovery_receipt_required")
download(client, key, version, size, expected, output)
return {
"status": "retrieved_verified",
"application": application,
"object_key": key,
"version_id": version,
"ciphertext_sha256": expected,
"ciphertext_bytes": size,
"application_restore_verified": False,
"provider_mutations": False,
}
def runtime_client():
import boto3
from botocore.config import Config
key = os.environ.get("AWS_ACCESS_KEY_ID", "").strip()
secret = os.environ.get("AWS_SECRET_ACCESS_KEY", "").strip()
if not key or not secret:
raise ValueError("owner_delivered_primary_custody_required")
return boto3.client(
"s3",
endpoint_url=ENDPOINT,
region_name="nl-ams",
aws_access_key_id=key,
aws_secret_access_key=secret,
config=Config(
signature_version="s3v4",
connect_timeout=15,
read_timeout=180,
retries={"max_attempts": 3},
s3={"addressing_style": "path"},
request_checksum_calculation="when_required",
response_checksum_validation="when_required",
),
)
def check_runtime():
from botocore.session import get_session
model = get_session().get_service_model("s3").operation_model("CompleteMultipartUpload")
if "IfNoneMatch" not in model.input_shape.members:
raise ValueError("conditional_multipart_sdk_required")
if not all(
os.environ.get(name, "").strip() for name in ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"]
):
raise ValueError("owner_delivered_primary_custody_required")
return {
"status": "configured",
"credential_values_printed": False,
"provider_contacted": False,
"conditional_multipart_sdk": True,
}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("action", choices=["check", "transfer", "retrieve"])
parser.add_argument("--application", choices=PROFILES, required=True)
parser.add_argument("--source", type=Path)
parser.add_argument("--output", type=Path)
parser.add_argument("--receipt", type=Path)
parser.add_argument("--captured-at")
parser.add_argument("--execute", action="store_true")
args = parser.parse_args()
if args.action == "check":
print(json.dumps(check_runtime()))
return 0
if not args.execute:
print(
json.dumps(
{
"status": "reviewed",
"provider_contacted": False,
"bucket": BUCKET,
"prefix": PROFILES[args.application],
"immutable_create": True,
}
)
)
return 0
if args.action == "retrieve":
if not args.output or not args.receipt:
raise ValueError("retrieval_arguments_required")
regular_file(args.receipt)
receipt = json.loads(args.receipt.read_text())
print(json.dumps(retrieve(runtime_client(), args.application, args.output, receipt)))
return 0
if not all([args.source, args.output, args.receipt, args.captured_at]):
raise ValueError("transfer_arguments_required")
regular_file(args.source)
private_directory(args.output.parent)
private_directory(args.receipt.parent)
if args.output.is_symlink() or args.receipt.is_symlink():
raise ValueError("linked_transfer_path_refused")
if args.receipt.exists():
regular_file(args.receipt)
receipt = json.loads(args.receipt.read_text())
if (
receipt.get("schema") != "platform.application-primary-archive.v1"
or receipt.get("application") != args.application
or receipt.get("captured_at") != args.captured_at
):
raise ValueError("transfer_receipt_scope_mismatch")
else:
receipt = {
"schema": "platform.application-primary-archive.v1",
"status": "pending",
"application": args.application,
"captured_at": args.captured_at,
"object_key": key_for(args.application, args.captured_at),
}
validate_key(args.application, receipt["object_key"])
if args.output.exists():
regular_file(args.output)
if (
receipt.get("status") == "offsite_fetched_verified"
and receipt.get("remote_retrieval_verified")
and receipt.get("immutable_create")
and require_version(receipt.get("version_id"))
and args.source.stat().st_size == receipt.get("ciphertext_bytes")
and digest(args.output) == receipt.get("ciphertext_sha256")
and digest(args.source) == receipt.get("ciphertext_sha256")
):
print(json.dumps(receipt))
return 0
raise ValueError("unverified_existing_readback_requires_review")
try:
checkpoint(args.receipt, receipt)
transfer(
runtime_client(),
args.application,
args.source,
args.output,
receipt,
lambda: checkpoint(args.receipt, receipt),
)
except Exception as error:
receipt.update(status="failed", failure_type=type(error).__name__)
checkpoint(args.receipt, receipt)
raise
print(json.dumps(receipt))
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except Exception as error:
print(json.dumps({"status": "failed", "failure_type": type(error).__name__}))
raise SystemExit(1)

90
tests/test_application_archive_retention_plan.py Normal file
View file

@ -0,0 +1,90 @@
"""Retention proposals preserve known recovered points and never own expiration."""
import importlib.util
import sys
from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest
@pytest.fixture
def planner(monkeypatch):
scripts = Path(__file__).resolve().parents[1] / "scripts"
monkeypatch.syspath_prepend(str(scripts))
spec = importlib.util.spec_from_file_location(
"application_archive_retention_plan", scripts / "application_archive_retention_plan.py"
)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
sys.modules.pop("primary_application_archive", None)
return module
def inventory(planner):
now = datetime(2027, 1, 12, 3, 15, tzinfo=UTC)
result = []
for day in range(70):
stamp = now - timedelta(days=day)
result.append(
{
"application": "coulomb-social",
"status": "offsite_fetched_verified",
"remote_retrieval_verified": True,
"captured_at": stamp.isoformat(),
"object_key": planner.PROFILES["coulomb-social"]
+ stamp.strftime("%Y/%m/%d/%Y%m%dT%H%M%SZ-")
+ f"{day:024x}"
+ ".tar.age",
"version_id": "version-" + str(day),
"ciphertext_sha256": "a" * 64,
"ciphertext_bytes": 5000,
}
)
recovery = dict(
result[-1], status="application_restored_verified", application_restore_verified=True
)
return now, result, recovery
def test_protects_old_recovered_point_and_deduplicates_daily_weekly(planner):
now, records, recovery = inventory(planner)
records.append(dict(records[0], application="email-connect", object_key="foreign"))
records.append(dict(records[0], status="failed", object_key="failed"))
result = planner.plan("coulomb-social", records, recovery, now)
keys = [item["object_key"] for item in result["keep"]]
assert len(keys) == len(set(keys)) <= 19
assert records[0]["object_key"] in keys and recovery["object_key"] in keys
assert all(item["object_key"] in keys for item in records[:14])
assert result["ignored"] == ["foreign", "failed"]
assert result["expiration_candidates"]
assert not result["provider_mutations"] and not result["local_mutations"]
assert result["protected"]["version_id"] == recovery["version_id"]
@pytest.mark.parametrize("change", ["missing", "version", "hash", "unproved"])
def test_refuses_missing_changed_or_unproved_application_recovery(planner, change):
now, records, recovery = inventory(planner)
if change == "missing":
records.pop()
elif change == "version":
recovery["version_id"] = "different-version"
elif change == "hash":
recovery["ciphertext_sha256"] = "b" * 64
else:
recovery["application_restore_verified"] = False
with pytest.raises(ValueError):
planner.plan("coulomb-social", records, recovery, now)
@pytest.mark.parametrize("change", ["duplicate", "future", "unversioned"])
def test_refuses_ambiguous_inventory(planner, change):
now, records, recovery = inventory(planner)
if change == "duplicate":
records.append(dict(records[0]))
elif change == "future":
records[0]["captured_at"] = (now + timedelta(days=1)).isoformat()
else:
records[0]["version_id"] = "null"
with pytest.raises(ValueError):
planner.plan("coulomb-social", records, recovery, now)

204
tests/test_primary_application_archive.py Normal file
View file

@ -0,0 +1,204 @@
"""Immutable archive transfer with a synthetic provider; no live credentials."""
import hashlib
import importlib.util
import io
from pathlib import Path
import pytest
@pytest.fixture
def archive():
spec = importlib.util.spec_from_file_location(
"primary_application_archive",
Path(__file__).resolve().parents[1] / "scripts/primary_application_archive.py",
)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class NotFound(Exception):
response = {"ResponseMetadata": {"HTTPStatusCode": 404}}
class Provider:
def __init__(self, data, failure=None, exists=False):
self.data, self.failure, self.exists = data, failure, exists
self.calls = []
self.version = "synthetic-version-1"
def head_object(self, **kwargs):
self.calls.append(("head", kwargs))
if self.failure == "permission":
raise PermissionError("synthetic provider failure")
if not self.exists:
raise NotFound()
return {
"ContentLength": len(self.data),
"VersionId": self.version,
"Metadata": {"sha256": hashlib.sha256(self.data).hexdigest()},
}
def create_multipart_upload(self, **kwargs):
self.calls.append(("start", kwargs))
return {"UploadId": "own-unfinished-upload"}
def upload_part(self, **kwargs):
self.calls.append(("part", kwargs))
if self.failure == "part":
raise TimeoutError("synthetic timeout")
assert kwargs["Body"] == self.data
return {"ETag": "synthetic-part-etag"}
def complete_multipart_upload(self, **kwargs):
self.calls.append(("complete", kwargs))
assert kwargs["IfNoneMatch"] == "*"
self.exists = True
if self.failure == "lost-response":
raise TimeoutError("synthetic lost acknowledgement")
return {"VersionId": None if self.failure == "unversioned" else self.version}
def abort_multipart_upload(self, **kwargs):
self.calls.append(("abort", kwargs))
assert kwargs["UploadId"] == "own-unfinished-upload"
def get_object(self, **kwargs):
self.calls.append(("get", kwargs))
assert kwargs["VersionId"] == self.version
data = b"X" * len(self.data) if self.failure == "corrupt" else self.data
if self.failure == "oversize":
data += b"extra"
return {
"ContentLength": len(self.data),
"VersionId": "other-version" if self.failure == "wrong-version" else self.version,
"Body": io.BytesIO(data),
}
def fixture(archive, tmp_path, failure=None, exists=False):
tmp_path.chmod(0o700)
source = tmp_path / "packet.tar.age"
source.write_bytes(b"age-encryption.org/v1\nsynthetic-encrypted-pair")
source.chmod(0o600)
receipt = {"object_key": archive.key_for("coulomb-social", "2026-10-07T20:00:00Z")}
return source, tmp_path / "fetched.age", receipt, Provider(source.read_bytes(), failure, exists)
def test_conditional_create_and_version_pinned_readback(archive, tmp_path):
source, output, receipt, client = fixture(archive, tmp_path)
saved = []
result = archive.transfer(
client, "coulomb-social", source, output, receipt, lambda: saved.append(dict(receipt))
)
assert result["status"] == "offsite_fetched_verified"
assert result["remote_retrieval_verified"] and result["immutable_create"]
assert result["version_id"] == client.version
assert output.read_bytes() == source.read_bytes()
assert output.stat().st_mode & 0o777 == 0o600
assert [call[0] for call in client.calls] == ["head", "start", "part", "complete", "get"]
assert not saved[0]["remote_retrieval_verified"]
assert saved[-1]["remote_retrieval_verified"]
@pytest.mark.parametrize(
"failure",
["part", "lost-response", "unversioned", "corrupt", "oversize", "wrong-version", "permission"],
)
def test_failure_never_declares_retrieval_or_deletes_retained_objects(archive, tmp_path, failure):
source, output, receipt, client = fixture(archive, tmp_path, failure)
with pytest.raises((ValueError, TimeoutError, PermissionError)):
archive.transfer(client, "coulomb-social", source, output, receipt)
assert not receipt["remote_retrieval_verified"]
assert not any(name == "delete" for name, _ in client.calls)
if failure in {"part", "lost-response"}:
assert len([call for call in client.calls if call[0] == "abort"]) == 1
else:
assert not any(name == "abort" for name, _ in client.calls)
def test_uncertain_completion_retries_readback_without_another_upload(archive, tmp_path):
source, output, receipt, client = fixture(archive, tmp_path, "lost-response")
with pytest.raises(TimeoutError):
archive.transfer(client, "coulomb-social", source, output, receipt)
key = receipt["object_key"]
client.failure = None
archive.transfer(client, "coulomb-social", source, output, receipt)
assert receipt["object_key"] == key and receipt["reused_verified_binding"]
assert len([call for call in client.calls if call[0] == "start"]) == 1
assert receipt["remote_retrieval_verified"]
def test_existing_key_must_bind_bytes_and_previous_version(archive, tmp_path):
source, output, receipt, client = fixture(archive, tmp_path, exists=True)
client.data = b"other ciphertext"
with pytest.raises(ValueError, match="existing_archive_binding_mismatch"):
archive.transfer(client, "coulomb-social", source, output, receipt)
assert [name for name, _ in client.calls] == ["head"]
client.data = source.read_bytes()
receipt["version_id"] = "previous-version"
with pytest.raises(ValueError, match="existing_archive_version_changed"):
archive.transfer(client, "coulomb-social", source, output, receipt)
assert not output.exists()
@pytest.mark.parametrize(
"invalid", ["plaintext", "scope", "receipt-hash", "linked-source", "linked-output"]
)
def test_invalid_packet_refuses_before_provider_contact(archive, tmp_path, invalid):
source, output, receipt, client = fixture(archive, tmp_path)
if invalid == "plaintext":
source.write_bytes(b"private unencrypted data")
elif invalid == "scope":
receipt["object_key"] = archive.key_for("email-connect", "2026-10-07T20:00:00Z")
elif invalid == "receipt-hash":
receipt["ciphertext_sha256"] = "bad"
elif invalid == "linked-source":
target = tmp_path / "real.age"
source.rename(target)
source.symlink_to(target)
else:
output.symlink_to(tmp_path / "absent")
with pytest.raises(ValueError):
archive.transfer(client, "coulomb-social", source, output, receipt)
assert client.calls == []
def test_retrieve_after_source_loss_uses_saved_version_without_provider_writes(archive, tmp_path):
source, output, receipt, client = fixture(archive, tmp_path)
receipt.update(
schema="platform.application-primary-archive.v1", captured_at="2026-10-07T20:00:00Z"
)
archive.transfer(client, "coulomb-social", source, output, receipt)
source.unlink()
output.unlink()
client.calls.clear()
result = archive.retrieve(client, "coulomb-social", output, receipt)
assert result["status"] == "retrieved_verified"
assert not result["provider_mutations"] and not result["application_restore_verified"]
assert output.read_bytes() == client.data
assert [name for name, _ in client.calls] == ["get"]
@pytest.mark.parametrize(
"field,value",
[
("remote_retrieval_verified", False),
("destination", "s3://another-bucket/key"),
("ciphertext_sha256", "bad"),
("version_id", "null"),
("ciphertext_bytes", -1),
("application", "email-connect"),
],
)
def test_retrieval_rejects_unbound_receipt_before_contact(archive, tmp_path, field, value):
source, output, receipt, client = fixture(archive, tmp_path)
receipt.update(schema="platform.application-primary-archive.v1")
archive.transfer(client, "coulomb-social", source, output, receipt)
output.unlink()
client.calls.clear()
receipt[field] = value
with pytest.raises(ValueError):
archive.retrieve(client, "coulomb-social", output, receipt)
assert client.calls == []

68
workplans/RPF-WP-0053-coulomb-paired-backup-admission.md
View file

@ -27,7 +27,7 @@ No credential was fetched, no timer enabled, no live upload/overwrite performed.
```task
id: RPF-WP-0053-T01
status: todo
status: progress
priority: high
state_hub_task_id: "ceb58abf-77ef-5d75-a65b-fc67bfd9aa54"
```
@ -48,19 +48,41 @@ id: RPF-WP-0053-T02
status: wait
priority: high
needs_human: true
intervention_note: "Platform must admit the backup workload and exact Nextcloud prefix/slot overwrite contract, and deliver bounded caller custody before live activation."
intervention_note: "Platform must admit the exact Scaleway primary archive caller/prefix, reconcile its current custody front door and retention/capacity, and deliver bounded credentials before activation."
state_hub_task_id: "b06f1a44-1a55-588c-9c6c-3c1e290544df"
```
Reuse governed railiance-backup-offsite-lane, OpenBao path
platform/workloads/railiance/backup/offsite-lane. Admit independent prefix
coulomb-social and 14 daily + 4 weekly encrypted fixed slots; prove directory
existence, PUT replacement and recoverable retrieval. Prepared node consumer
expects root-owned mode-0600 /etc/coulomb-social-backup/offsite.env with
RAILIANCE_BACKUP_NC_TOKEN and RAILIANCE_BACKUP_NC_WEBDAV_URL delivered by the
custody owner; public age recipient is inherited from the platform helper.
Do not put AGE_PRIVATE_KEY in the scheduled worker. Decide delivery/rotation
using the owner contract, rather than copying another workload's Secret.
The original Nextcloud overwrite-slot proposal is withdrawn; its current workload
uploader is create-only and secondary. Reuse the platform primary contract:
s3://railiance-platform-pg-backup/platform-pg/application-archives/coulomb-social/,
Standard Multi-AZ nl-ams. Exact review package is
docs/credential-lane-designs/coulomb-paired-backup-primary.md.
scripts/primary_application_archive.py implements conditional immutable multipart
completion, own unfinished-upload abort, version-pinned GET and ciphertext hash
verification. Independent retrieve needs only a retained public receipt, no
original archive/node. requirements/backup-primary.lock pins seven packages with
hashes in a dedicated backup-venv; SDK conditional-completion shape is verified.
Synthetic provider tests cover uncertain completion, replay without new upload,
corrupted bytes, changed versions, caller scope and node-loss retrieval. No live
provider rights or custody are inferred from these tests.
CCR-2026-0012 identifies existing primary custody coordinates and its approved
rapp-postgres/databases delivery, not a new consumer. Its historical catalog id
backup-object-storage is unknown to the current local warden CLI. Owner must
reconcile the front door, choose/admit appropriately prefix-scoped IAM authority
and deliver AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY to root-owned mode 0600
/etc/coulomb-social-backup/primary.env (parent mode 0700). Metadata-only node check found
that file absent. No Secret fetch/copy or namespace expansion occurred.
No bootstrap key or age identity belongs in the scheduled worker.
Separate non-mutating application_archive_retention_plan.py proposes 14 daily/4
weekly distinct generations and protects newest plus the exact last application-
recovered version/hash. A missing/changed proof refuses; foreign/unverified points
are ignored. This is a review, not a deletion executor or activated lifecycle.
Bind local/provider headroom, independently retained metadata receipts, protected
recovery and compatibility with the existing 30-day bucket lifecycle before
activation. No retained-object/version-delete rights belong in the publisher.
## T03 — Activate and verify retrieval and application recovery
@ -74,13 +96,14 @@ state_hub_task_id: "871c14da-2940-5be6-8ce8-953973422064"
```
Founder ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED, plus
explicit offsite prefix write/overwrite authority. Proposed UTC daily03:15
explicit primary archive creation/version-read authority. Proposed UTC daily03:15
(+up to5min jitter), RPO target24h, hourly stale threshold26h, provisional
RTO target60min for an admitted attended restore. These are proposal targets,
not measured promises. First capture/upload is bounded and app resumes before
encryption/upload. Retrieve ciphertext independently, compare encrypted hash,
decrypt through recovery escrow into a private isolated workspace, unpack only
database.dump/content.tar.gz/receipt.json, run app paired restore and compare
encryption/upload. Retrieve using the independently retained public receipt,
compare encrypted hash, decrypt through recovery
escrow with consumer recover-paired-backup.py (strict three-file packet/hash and
capture/image binding), then run restore-paired-snapshot.py and compare
row/sequence/file hashes, actual rendering and denial. Record receipt paths,
counts/timing only; preserve the source recovery point on any failed check.
No production data restore is part of backup schedule activation.
@ -99,5 +122,16 @@ established evidence owner (RPF-WP-0036-T03/T04), prove reception and failure
response; local journald failure alone is not monitored acceptance. Name the
recovery custodian and approve a monthly isolated drill with offsite readback.
Verify key recovery survives node/workstation loss. Record release-triggered
backup helper digest refresh and retention-slot policy. CSOC-WP-0008-T02 stays
progress until operating evidence and service contract are accepted.
backup helper digest refresh and admitted generation/capacity/retention policy.
CSOC-WP-0008-T02 stays progress until operating evidence and service contract are accepted.
## Implementation evidence — 2026-10-07
New primary/retention 30 cases and existing backup tier/archive lifecycle 23 cases
pass (53 total); no live provider/custody interaction. Consumer full make check
passes 150 tests/mandatory browser journeys and Ruff; real synthetic age recovery
roundtrip passes and all four systemd units verify. T01 is progress for the
prepared execution package; owner identity acceptance remains open. T02/T03 remain
wait for custody/admission and exact activation; no timer installed/enabled and
no productive ciphertext uploaded. Email-connect's separate transport profile
is not ledger capture/recovery coverage; CSOC-WP-0010-T04/T05 own that residual.