RPF-WP-0015-T02: select the lease issued for the mounted refresh; leave predecessors

ESO refresh (15m) and role TTL (15m, max 1h) overlap, so the first window
saw two live handles and aborted before revocation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 680842@bnt-lap001
Assistant-Session: 54eb9e8e-7d6b-4f0c-b7a6-a6db62075883
This commit is contained in:
codex 2026-10-14 18:19:38 +02:00
parent c8d45d585f
commit 044df6fe5b
4 changed files with 73 additions and 24 deletions

View file

@ -11,7 +11,7 @@ pod. It does not authorize a live revocation.
| Boundary | Exact scope | Responsible owner | | Boundary | Exact scope | Responsible owner |
| --- | --- | --- | | --- | --- | --- |
| Consumer | Deployment `audit-core/audit-core`; Secret `audit-core/audit-core-database`; ExternalSecret `audit-core/audit-core-database` | `audit-core` | | Consumer | Deployment `audit-core/audit-core`; Secret `audit-core/audit-core-database`; ExternalSecret `audit-core/audit-core-database` | `audit-core` |
| Lease | `database/creds/audit-core-runtime`; exactly one live handle whose issue time coheres with the current ExternalSecret refresh | `railiance-platform` custody operator | | Lease | `database/creds/audit-core-runtime`; the one handle issued for the current ExternalSecret refresh (older predecessors are left alone) | `railiance-platform` custody operator |
| Database | CNPG `databases/platform-pg`, database `audit_core`, runtime group `audit_core_app` | `rapp-postgres` | | Database | CNPG `databases/platform-pg`, database `audit_core`, runtime group `audit_core_app` | `rapp-postgres` |
| Load | One separately approved synthetic sender/tenant contract; no production event or identity | `audit-core` + load owner | | Load | One separately approved synthetic sender/tenant contract; no production event or identity | `audit-core` + load owner |
| Coordination and evidence | `RAILIANCE-WP-0024-T02` | `railiance-platform` | | Coordination and evidence | `RAILIANCE-WP-0024-T02` | `railiance-platform` |
@ -181,10 +181,15 @@ bodies and clear all transient JSON from memory or mode-`0600` temporary files.
3. Inside the attended process, list only lease handles below 3. Inside the attended process, list only lease handles below
`sys/leases/lookup/database/creds/audit-core-runtime`. OpenBao's lease lookup `sys/leases/lookup/database/creds/audit-core-runtime`. OpenBao's lease lookup
returns issue/expiry/TTL metadata but not the leased username, so exact returns issue/expiry/TTL metadata but not the leased username, so exact
selection is deliberately stricter: there must be exactly one live handle selection is deliberately strict: at most 8 handles may exist under that
under that exact runtime prefix, and its issue/expiry interval must contain exact runtime prefix; exactly one must have been issued within 5 seconds of
the current database Secret refresh time. Abort for database-owner the current database Secret refresh (the mounted credential), with at least
disambiguation if zero or multiple handles exist. Retain only a one-way five minutes of TTL left; every other handle must be an older predecessor.
ESO refreshes every 15 minutes while the role TTL is 15 minutes (max 1 hour),
so a predecessor can legitimately still be live; it is never revoked. Abort
if no handle, several handles, or a newer unmatched handle match the refresh.
(2026-10-14: the first attended window aborted here under the earlier
"exactly one handle" rule with two live handles.) Retain only a one-way
handle fingerprint in evidence and clear the lookup response immediately. handle fingerprint in evidence and clear the lookup response immediately.
4. Recheck that the Secret `resourceVersion`, pod UID, and selected handle have 4. Recheck that the Secret `resourceVersion`, pod UID, and selected handle have
not changed. Revoke only that handle. Do not revoke the role prefix and do not changed. Revoke only that handle. Do not revoke the role prefix and do

View file

@ -142,10 +142,10 @@
}, },
"artifact_sha256": { "artifact_sha256": {
"docs/audit-core-database-lease-approval.example.json": "87e57e3290074508c6517938acf8c7fc1a75d977b8ff630f247a4ff5d2ee45c6", "docs/audit-core-database-lease-approval.example.json": "87e57e3290074508c6517938acf8c7fc1a75d977b8ff630f247a4ff5d2ee45c6",
"docs/audit-core-database-lease-recovery.md": "ae9e6b0d6cf60514cfabbd1fd0a9818d33dbbac9291a94539a4e049492c48f21", "docs/audit-core-database-lease-recovery.md": "8dfd3fb59c551df0346d2569674227bb2f4e763c3bf058453035591073e20794",
"docs/openbao-reboot-snapshot-receipt.example.json": "b95903a3e1aacb607eecc3f0440f08e62480d09da47bd7ab96c3eb8814f2eaee", "docs/openbao-reboot-snapshot-receipt.example.json": "b95903a3e1aacb607eecc3f0440f08e62480d09da47bd7ab96c3eb8814f2eaee",
"docs/railiance01-coordinated-reboot.md": "6fd6301f808449ad6519764d994b05aee41a96be879b917ee1d18d8a65d5f6fd", "docs/railiance01-coordinated-reboot.md": "6fd6301f808449ad6519764d994b05aee41a96be879b917ee1d18d8a65d5f6fd",
"scripts/audit-core-database-lease-recovery.py": "3066718d06ce29b98fa8ee513bd967e0f48c47db2d66b5083c591f5f1c2bddd6", "scripts/audit-core-database-lease-recovery.py": "121ade87315606b11dc7366c11a0484dc73bdbeef3d5c59c2cb75a2de3292bc8",
"scripts/audit-core-recovery-preflight.py": "05e89ae9d0b9a636b67d9a8d0c203ac746c97208e32061cb23f1ca785eda8e5c", "scripts/audit-core-recovery-preflight.py": "05e89ae9d0b9a636b67d9a8d0c203ac746c97208e32061cb23f1ca785eda8e5c",
"scripts/audit_core_synthetic_sender_lane.py": "608145f43b5adea110c4c73a2f567370fcfb603ef73ca8230775de9bf7687fa6", "scripts/audit_core_synthetic_sender_lane.py": "608145f43b5adea110c4c73a2f567370fcfb603ef73ca8230775de9bf7687fa6",
"scripts/t02-attended-window.py": "1faca370d14de9a55a3f04e55f6046105b5d3a606a4d79dadbe72c495ab75077" "scripts/t02-attended-window.py": "1faca370d14de9a55a3f04e55f6046105b5d3a606a4d79dadbe72c495ab75077"

View file

@ -160,21 +160,7 @@ class Authority:
def bao(self, args: list[str], *, label: str) -> str: def bao(self, args: list[str], *, label: str) -> str:
return safe_run(["bao", *args], label=label, env=self.env).stdout.strip() return safe_run(["bao", *args], label=label, env=self.env).stdout.strip()
def single_runtime_lease(self) -> dict[str, Any]: def _lookup(self, lease_id: str) -> dict[str, Any]:
raw = self.bao(
["list", "-format=json", f"sys/leases/lookup/{LEASE_PREFIX}"],
label="list exact runtime lease handles",
)
try:
handles = json.loads(raw)
except json.JSONDecodeError as exc:
raise ProcedureError("runtime lease list is invalid JSON") from exc
if not isinstance(handles, list) or len(handles) != 1:
count = len(handles) if isinstance(handles, list) else "unknown"
raise ProcedureError(
f"exact runtime prefix must contain one live handle (observed {count})"
)
lease_id = f"{LEASE_PREFIX}/{handles[0]}"
lookup_raw = self.bao( lookup_raw = self.bao(
["write", "-format=json", "sys/leases/lookup", f"lease_id={lease_id}"], ["write", "-format=json", "sys/leases/lookup", f"lease_id={lease_id}"],
label="lookup exact runtime lease metadata", label="lookup exact runtime lease metadata",
@ -194,6 +180,28 @@ class Authority:
"ttl": ttl, "ttl": ttl,
} }
def runtime_lease_for(self, secret: dict[str, Any]) -> dict[str, Any]:
"""The one handle issued for the mounted Secret's refresh.
ESO refreshes every 15m while the role's TTL is 15m (max 1h), so the
predecessor of the mounted credential can still be live. Select only the
handle coherent with the Secret refresh; every other handle must be an
older predecessor, which is never revoked.
"""
raw = self.bao(
["list", "-format=json", f"sys/leases/lookup/{LEASE_PREFIX}"],
label="list exact runtime lease handles",
)
try:
handles = json.loads(raw)
except json.JSONDecodeError as exc:
raise ProcedureError("runtime lease list is invalid JSON") from exc
if not isinstance(handles, list) or not 1 <= len(handles) <= MAX_RUNTIME_HANDLES:
count = len(handles) if isinstance(handles, list) else "unknown"
raise ProcedureError(f"exact runtime prefix has an unexpected handle count ({count})")
leases = [self._lookup(f"{LEASE_PREFIX}/{handle}") for handle in handles]
return select_runtime_lease(leases, secret)
def revoke(self, lease_id: str) -> None: def revoke(self, lease_id: str) -> None:
self.bao(["lease", "revoke", lease_id], label="revoke exact runtime lease") self.bao(["lease", "revoke", lease_id], label="revoke exact runtime lease")
@ -307,6 +315,24 @@ def pod_state(remote: Any) -> dict[str, Any]:
} }
MAX_RUNTIME_HANDLES = 8
def select_runtime_lease(leases: list[dict[str, Any]], secret: dict[str, Any]) -> dict[str, Any]:
coherent = [
lease for lease in leases
if abs((lease["issue_time"] - secret["refresh_time"]).total_seconds()) <= 5
]
if len(coherent) != 1:
raise ProcedureError(
f"exactly one runtime lease must match the mounted Secret refresh (observed {len(coherent)})"
)
selected = coherent[0]
if any(lease["issue_time"] >= selected["issue_time"] for lease in leases if lease is not selected):
raise ProcedureError("a runtime lease newer than the mounted credential exists")
return {**selected, "predecessor_handles": len(leases) - 1}
def assert_lease_matches_refresh(lease: dict[str, Any], secret: dict[str, Any]) -> None: def assert_lease_matches_refresh(lease: dict[str, Any], secret: dict[str, Any]) -> None:
delta = abs((lease["issue_time"] - secret["refresh_time"]).total_seconds()) delta = abs((lease["issue_time"] - secret["refresh_time"]).total_seconds())
if ( if (
@ -380,7 +406,7 @@ def exercise(args: argparse.Namespace) -> dict[str, Any]:
attest_chain_state(remote) attest_chain_state(remote)
before_pod = pod_state(remote) before_pod = pod_state(remote)
before_secret = secret_state(remote) before_secret = secret_state(remote)
lease = authority.single_runtime_lease() lease = authority.runtime_lease_for(before_secret)
assert_lease_matches_refresh(lease, before_secret) assert_lease_matches_refresh(lease, before_secret)
baseline_load = driver.run("baseline") baseline_load = driver.run("baseline")
if baseline_load.get("status") != "ready": if baseline_load.get("status") != "ready":
@ -395,7 +421,7 @@ def exercise(args: argparse.Namespace) -> dict[str, Any]:
# Close the race with ESO's ordinary refresh before the destructive step. # Close the race with ESO's ordinary refresh before the destructive step.
if secret_state(remote) != before_secret or pod_state(remote) != before_pod: if secret_state(remote) != before_secret or pod_state(remote) != before_pod:
raise ProcedureError("baseline changed before revocation") raise ProcedureError("baseline changed before revocation")
current = authority.single_runtime_lease() current = authority.runtime_lease_for(before_secret)
if current["fingerprint"] != lease["fingerprint"]: if current["fingerprint"] != lease["fingerprint"]:
raise ProcedureError("runtime lease changed before revocation") raise ProcedureError("runtime lease changed before revocation")
authority.revoke(lease["id"]) authority.revoke(lease["id"])

View file

@ -81,6 +81,24 @@ class DatabaseLeaseRecoveryTests(unittest.TestCase):
with self.assertRaises(module.ProcedureError): with self.assertRaises(module.ProcedureError):
module.load_approval(path, now, require_open_window=True) module.load_approval(path, now, require_open_window=True)
def test_select_runtime_lease_ignores_older_predecessors_only(self) -> None:
refresh = datetime(2026, 10, 14, 16, 14, 7, tzinfo=UTC)
def lease(name: str, issued: datetime) -> dict:
return {"id": name, "fingerprint": name, "issue_time": issued,
"expire_time": issued + timedelta(minutes=15), "ttl": 600}
current = lease("current", refresh + timedelta(seconds=1))
older = lease("older", refresh - timedelta(minutes=15))
secret = {"refresh_time": refresh}
selected = module.select_runtime_lease([older, current], secret)
self.assertEqual("current", selected["id"])
self.assertEqual(1, selected["predecessor_handles"])
with self.assertRaises(module.ProcedureError): # nothing matches the mounted refresh
module.select_runtime_lease([older], secret)
with self.assertRaises(module.ProcedureError): # a newer unmatched handle exists
module.select_runtime_lease([current, lease("newer", refresh + timedelta(minutes=1))], secret)
with self.assertRaises(module.ProcedureError): # two handles match the refresh
module.select_runtime_lease([current, lease("twin", refresh)], secret)
def test_approval_requires_exact_owner_and_window_contract(self) -> None: def test_approval_requires_exact_owner_and_window_contract(self) -> None:
now = datetime(2026, 8, 22, 12, 0, tzinfo=UTC) now = datetime(2026, 8, 22, 12, 0, tzinfo=UTC)
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory: