RPF-WP-0015-T03: E2 proves a fresh write (accepted then duplicate); honest retention note
Driver baseline reports ready for accepted or duplicate and its cleanup owns no server-side object (railiance-platform round-8 review). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 680842@bnt-lap001 Assistant-Session: 54eb9e8e-7d6b-4f0c-b7a6-a6db62075883
This commit is contained in:
parent
6fc2dae86a
commit
748fe6c7db
5 changed files with 47 additions and 33 deletions
|
|
@ -2,7 +2,7 @@
|
|||
# Custodian Brief — railiance-platform
|
||||
|
||||
**Domain:** financials
|
||||
**Last synced:** 2026-10-14 16:36 UTC
|
||||
**Last synced:** 2026-10-14 21:52 UTC
|
||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||
|
||||
## Active Workstreams
|
||||
|
|
@ -14,6 +14,13 @@ Progress: 3/4 done | workplan_id: `f4640325-e89c-591d-b58e-ec6b087900ac`
|
|||
- ! T03 — Define the coordinated railiance01 reboot exercise `ba729d18`
|
||||
*(wait: Single-node outage exercise needs operator access and a declared restart order coordinated with railiance-infra, railiance-cluster, audit-core and the OpenBao package owner.)*
|
||||
|
||||
### Let the hub-writer caller prove its own reach and pick the workload role in the OpenBao UI
|
||||
Progress: 2/4 done | workplan_id: `04f7c4aa-0a24-5d61-994d-2487ca6bfb91`
|
||||
|
||||
**Open tasks:**
|
||||
- ! Apply CCR-2026-0045 `d3236de5`
|
||||
- ! Deploy the overlay `94cdfd1c`
|
||||
|
||||
### Coordinate KeyCape live Secret exposure recovery
|
||||
Progress: 4/6 done | workplan_id: `b2c25a01-4a80-55c1-90cf-8538000f7e0e`
|
||||
|
||||
|
|
|
|||
|
|
@ -216,11 +216,13 @@ State Hub.
|
|||
one separately approved synthetic E2. The E2 runs through the reviewed sender lane in
|
||||
its own attended login: `make audit-t02-attended-window` with
|
||||
`AUDIT_T02_MODE=e2` and an approval receipt whose `task_id` is
|
||||
`RAILIANCE-WP-0024-T03`. It projects one synthetic sender, sends one event
|
||||
(`baseline`, must be accepted), resends it (`expect-recovered`, must be
|
||||
`duplicate`: accepted exactly once), cleans up, and touches no lease. The
|
||||
harness refuses unless the replay is a duplicate, cleanup removes that
|
||||
fixture, and the pod UID and restart count are unchanged. Confirm no chain
|
||||
`RAILIANCE-WP-0024-T03`. It projects one synthetic sender and sends one event
|
||||
twice through the driver's `expect-recovered` phase: the first answer must be a
|
||||
fresh `accepted` (HTTP 200/202), the second `duplicate` for the same fixture
|
||||
(accepted exactly once); it touches no lease. The harness refuses unless both
|
||||
hold and the pod UID and restart count are unchanged. The driver owns no
|
||||
server-side object; the synthetic event stays permanently in the append-only
|
||||
chain under its trial tenant, and only the sender identity is removed. Confirm no chain
|
||||
break, the prior marker remains readable, and the new event is accepted
|
||||
exactly once.
|
||||
Window `rpf-wp-0015-t03-window-20261014-01` does not run this E2. Founder
|
||||
|
|
|
|||
|
|
@ -145,8 +145,8 @@
|
|||
"docs/audit-core-database-lease-approval.example.json": "87e57e3290074508c6517938acf8c7fc1a75d977b8ff630f247a4ff5d2ee45c6",
|
||||
"docs/audit-core-database-lease-recovery.md": "8dfd3fb59c551df0346d2569674227bb2f4e763c3bf058453035591073e20794",
|
||||
"docs/openbao-reboot-snapshot-receipt.example.json": "b95903a3e1aacb607eecc3f0440f08e62480d09da47bd7ab96c3eb8814f2eaee",
|
||||
"docs/railiance01-coordinated-reboot.md": "789f687ff62fe1dde63d33aafd6af18dc791e38984cafdc61f7bce19ebfa0c1f",
|
||||
"scripts/audit-core-database-lease-recovery.py": "8477c280b0584a711d4de8e303caf2dbc8d50b657a32900318b75f33569690b6",
|
||||
"docs/railiance01-coordinated-reboot.md": "975bd3167cd1c3da152fafc8bcf882fb15dade92a89f60e82495cf34201fc81b",
|
||||
"scripts/audit-core-database-lease-recovery.py": "332405cd6fce36d3df5588e7622065e3655eea0e4256766eb07c743349f29b17",
|
||||
"scripts/audit-core-recovery-preflight.py": "d870d7763fabb9d01f72c157f82419257fd1bf9c2705e1661a98022b1caa29ba",
|
||||
"scripts/audit_core_synthetic_sender_lane.py": "8a9c40cfdfd5cfe276bbf3ab79ecdf8e5f18cb85a2811423f54fa3808a4c57a4",
|
||||
"scripts/t02-attended-window.py": "c4531d2daf6b2658cf8e94c9761d9cfe6333cca646b1d1692103945144fd775d",
|
||||
|
|
|
|||
|
|
@ -403,18 +403,20 @@ def synthetic_e2(args: argparse.Namespace) -> dict[str, Any]:
|
|||
before_pod = pod_state(remote)
|
||||
if PREFLIGHT.endpoint_status(remote, "/readyz") != 200:
|
||||
raise ProcedureError("audit-core is not ready for the synthetic E2")
|
||||
accepted = driver.run("baseline")
|
||||
if accepted.get("status") != "ready":
|
||||
raise ProcedureError("synthetic E2 event was not accepted")
|
||||
# The driver's baseline phase reports "ready" for accepted *or* duplicate, so it
|
||||
# cannot prove a write. Send twice: the first answer must be a fresh accept,
|
||||
# the second a duplicate of the same fixture (accepted exactly once).
|
||||
first = driver.run("expect-recovered")
|
||||
if first.get("status") != "accepted" or first.get("http_status") not in (200, 202):
|
||||
raise ProcedureError("synthetic E2 first send was not a fresh accept (fixture already present?)")
|
||||
replay = driver.run("expect-recovered")
|
||||
if replay.get("status") != "duplicate" or replay.get("fixture_id") != accepted.get("fixture_id"):
|
||||
if replay.get("status") != "duplicate" or replay.get("fixture_id") != first.get("fixture_id"):
|
||||
raise ProcedureError("synthetic E2 replay was not recognised as a duplicate")
|
||||
# The driver owns no server-side object; its cleanup only confirms local state.
|
||||
# The synthetic event stays in the append-only chain under the trial tenant.
|
||||
load_cleanup = driver.run("cleanup")
|
||||
if (
|
||||
load_cleanup.get("status") != "clean"
|
||||
or load_cleanup.get("fixture_id") != accepted.get("fixture_id")
|
||||
):
|
||||
raise ProcedureError("synthetic E2 cleanup did not confirm exact fixture removal")
|
||||
if load_cleanup.get("status") != "clean" or load_cleanup.get("fixture_id") != first.get("fixture_id"):
|
||||
raise ProcedureError("synthetic E2 driver cleanup did not report clean for this fixture")
|
||||
after_pod = pod_state(remote)
|
||||
if after_pod["uid"] != before_pod["uid"] or after_pod["restart_count"] != before_pod["restart_count"]:
|
||||
raise ProcedureError("audit-core pod identity or restart count changed during the synthetic E2")
|
||||
|
|
@ -425,11 +427,14 @@ def synthetic_e2(args: argparse.Namespace) -> dict[str, Any]:
|
|||
"approval_id": approval["approval_id"],
|
||||
"completed_at": datetime.now(UTC).replace(microsecond=0).isoformat().replace("+00:00", "Z"),
|
||||
"synthetic_event_accepted": True,
|
||||
"first_send_status": first["status"],
|
||||
"first_send_http_status": first["http_status"],
|
||||
"synthetic_event_retained": "append-only chain, trial tenant (driver owns no server-side object)",
|
||||
"replay_status": replay["status"],
|
||||
"accepted_exactly_once": True,
|
||||
"same_pod_uid": after_pod["uid"] == before_pod["uid"],
|
||||
"restart_count_unchanged": after_pod["restart_count"] == before_pod["restart_count"],
|
||||
"load_cleanup_status": load_cleanup.get("status"),
|
||||
"driver_cleanup_status": load_cleanup.get("status"),
|
||||
"synthetic_load_contract": approval["synthetic_load"]["contract_id"],
|
||||
"synthetic_load_driver_revision": approval["synthetic_load"]["driver_revision"],
|
||||
"secret_values_observed": False,
|
||||
|
|
|
|||
|
|
@ -113,32 +113,32 @@ class DatabaseLeaseRecoveryTests(unittest.TestCase):
|
|||
path, now, require_open_window=True, task_id=module.E2_TASK_ID)["task_id"])
|
||||
|
||||
class Driver:
|
||||
def __init__(self, replay):
|
||||
self.replay = replay
|
||||
def __init__(self, *sends):
|
||||
self.sends = list(sends)
|
||||
def run(self, phase):
|
||||
return {"baseline": {"status": "ready", "fixture_id": "f"},
|
||||
"expect-recovered": {"status": self.replay, "fixture_id": "f"},
|
||||
"cleanup": {"status": "clean", "fixture_id": "f"}}[phase]
|
||||
if phase == "expect-recovered":
|
||||
return {"status": self.sends.pop(0), "http_status": 202, "fixture_id": "f"}
|
||||
return {"status": "clean", "fixture_id": "f"}
|
||||
|
||||
args = argparse.Namespace(approval=path, confirm=module.E2_CONFIRM, remote="r", load_driver=Path("/d"))
|
||||
pods = iter([
|
||||
{"uid": "u", "restart_count": 0}, {"uid": "u", "restart_count": 0},
|
||||
{"uid": "u", "restart_count": 0},
|
||||
{"uid": "u", "restart_count": 0}, {"uid": "changed", "restart_count": 0},
|
||||
])
|
||||
same = {"uid": "u", "restart_count": 0}
|
||||
pods = iter([same, same, same, same, same, {"uid": "changed", "restart_count": 0}])
|
||||
with mock.patch.object(module, "load_approval", wraps=lambda *a, **k: {
|
||||
**approved_receipt(now), "task_id": module.E2_TASK_ID}), \
|
||||
mock.patch.object(module.PREFLIGHT, "Remote"), \
|
||||
mock.patch.object(module.PREFLIGHT, "endpoint_status", return_value=200), \
|
||||
mock.patch.object(module, "pod_state", side_effect=lambda r: next(pods)):
|
||||
with mock.patch.object(module, "LoadDriver", return_value=Driver("duplicate")):
|
||||
with mock.patch.object(module, "LoadDriver", return_value=Driver("accepted", "duplicate")):
|
||||
result = module.synthetic_e2(args)
|
||||
self.assertTrue(result["accepted_exactly_once"] and result["same_pod_uid"])
|
||||
self.assertEqual("accepted", result["first_send_status"])
|
||||
self.assertEqual("post-reboot-synthetic-e2", result["mode"])
|
||||
with mock.patch.object(module, "LoadDriver", return_value=Driver("accepted")):
|
||||
with self.assertRaises(module.ProcedureError): # second accept = not exactly once
|
||||
module.synthetic_e2(args)
|
||||
with mock.patch.object(module, "LoadDriver", return_value=Driver("duplicate")):
|
||||
for sends in (("accepted", "accepted"), ("duplicate", "duplicate")):
|
||||
# a second accept is not exactly once; a duplicate first send wrote nothing now
|
||||
with mock.patch.object(module, "LoadDriver", return_value=Driver(*sends)):
|
||||
with self.assertRaises(module.ProcedureError):
|
||||
module.synthetic_e2(args)
|
||||
with mock.patch.object(module, "LoadDriver", return_value=Driver("accepted", "duplicate")):
|
||||
with self.assertRaises(module.ProcedureError): # a restarted pod is not this proof
|
||||
module.synthetic_e2(args)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue