fix(execution): retain scoped cleanup after uncertain outcomes
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a140c4-354a-7823-8f0d-8f717397f786
This commit is contained in:
codex
parent
3b6e647aa0
commit
59d1441cf8
7 changed files with 462 additions and 4 deletions
66
docs/task-execution-cleanup.md
Normal file
66
docs/task-execution-cleanup.md
Normal file
|
|
@ -0,0 +1,66 @@
|
|||
# Durable cleanup obligations for the metadata execution pilot
|
||||
|
||||
RPF-WP-0035-T12 source preparation, 2026-10-16. This source is not an installed
|
||||
mediator, native execution entitlement, founder approval or owner-native proof.
|
||||
|
||||
Platform's existing admission journal made `unknown` and `failed` terminal and
|
||||
blocked all further calls. Its single business-state cleanup transition could not
|
||||
express a timed-out write with confirmed token revocation, or a verified write
|
||||
with unconfirmed cleanup. A future mediator using that interface could suppress
|
||||
cleanup precisely when needed. The journal now retains the original business
|
||||
outcome and records cleanup in a separate durable table.
|
||||
|
||||
At fresh reservation the trusted owner supplies at most one commitment each for
|
||||
`task-token-revoke`, `identity-token-revoke`, and `grant-policy-delete`. Each is a
|
||||
kind and exact native-call digest; no token/accessor, payload, path override or
|
||||
shell command is accepted. Obligations and admission are inserted in one SQLite
|
||||
transaction with FULL synchronous commits and directory fsync. Existing journal
|
||||
files gain the separate table; old admissions gain no obligations or authority.
|
||||
Empty cleanup evidence cannot produce a confirmed result or legacy success state.
|
||||
|
||||
Only the same live exclusive invocation that transitioned this binding through
|
||||
`consume-confirmed` can record cleanup intent. Neither a caller string nor journal
|
||||
observation establishes consume authority: the trusted mediator must independently
|
||||
validate the real served approval/PDP/consume before that transition. Cleanup can
|
||||
proceed when the business call is pending or state is unknown/failed. Starting it
|
||||
permanently closes business dispatch for that admission. An exact obligation moves
|
||||
`owed -> pending -> confirmed|unconfirmed`; no retry transition exists. A crash
|
||||
leaves pending, and reopening never restores admission. Pending business-call
|
||||
uncertainty remains even if every cleanup succeeds.
|
||||
|
||||
`task_execution_cleanup.run_cleanup` is the fixed caller of those journal methods.
|
||||
It is intended for the owner mediator's `finally` block inside the original
|
||||
`Journal.locked()` context. It dispatches only the three named owner cleanup
|
||||
methods, passing their admitted digest for independent native-call comparison.
|
||||
Ordinary exceptions/malformed evidence become closed unconfirmed reason evidence
|
||||
without logging or hashing raw backend text. Remaining independent obligations
|
||||
continue. A process death or BaseException leaves pending intent and requires
|
||||
owner reconciliation. A journal write failure prevents the corresponding effect.
|
||||
The component has no HTTP client, CLI command execution or native credential opener.
|
||||
|
||||
The owner adapter must retain the exact handles in private memory, implement
|
||||
bounded native calls, compare each reconstructed cleanup call against the admitted
|
||||
commitment, and return confirmed only after the appropriate native revoke/absence
|
||||
verification. A timeout, policy deletion alone, or eventual TTL expiry cannot prove
|
||||
issued-token revocation. Cleanup has the original separate obligation rather than
|
||||
the expired business deadline; this component does not extend any business grant.
|
||||
No automatic postcrash recovery is implemented: owner-private recovery requires
|
||||
fresh scoped authority and must never retry a business effect.
|
||||
|
||||
The current Compiler Plan does not yet contain these cleanup commitments. A
|
||||
production mediator must bind them into the exact reviewed grant/full canonical
|
||||
request/owner plan before reservation. The journal's digest parameter does not
|
||||
verify that correspondence. It must also integrate authenticated initiating actor,
|
||||
served claim/PDP and independently verified atomic consume, exact source/body and
|
||||
baseline/readback validation, ephemeral owner-held token materialization, per-call
|
||||
trusted-time/use enforcement, and native audit/recovery. There is no production
|
||||
executor in this repository to wire to this driver today; existing production
|
||||
readiness remains refused. Approval CAS alone cannot provide exactly-once target
|
||||
effects across process/network failure. T12 and SECRETS-WP-0013 remain open.
|
||||
|
||||
Validation uses actual SQLite/process locks and abrupt `os._exit` before and after
|
||||
a separately fsynced synthetic fixture effect. Reopened processes cannot resume
|
||||
cleanup or business work; a four-process race admits one driver. Other tests cover
|
||||
pending business calls, revoke timeout without suppressing later cleanup, changed
|
||||
commitments and bindings, malformed confirmations and empty-evidence refusal.
|
||||
These are disposable local fixtures, not OpenBao or production acceptance.
|
||||
|
|
@ -94,7 +94,9 @@ cleanup use; this component records limits, it does not enforce native expiry/us
|
|||
or provide a hard elapsed timer. Trusted owner runtime must bound parser/import/
|
||||
filesystem/compiler work by remaining action budget before any actual call.
|
||||
Token custody/revoke, temporary object removal, owner-private recovery after unknown
|
||||
and truthful native readback remain gates. General lifecycle/apply/exec/default
|
||||
and truthful native readback remain gates. The 2026-10-16 source cleanup journal
|
||||
and fixed driver are documented in task-execution-cleanup.md; they preserve unknown
|
||||
outcomes but are not yet bound into this compiler or a production mediator. General lifecycle/apply/exec/default
|
||||
migration and SecretsT06/RPF35T02/T12 cannot close from this source component.
|
||||
|
||||
Validation:45 compiler tests and 15 journal tests pass, covering real owner K8s/OIDC body generation,
|
||||
|
|
|
|||
72
scripts/task_execution_cleanup.py
Normal file
72
scripts/task_execution_cleanup.py
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
"""Fixed cleanup driver for a trusted owner mediator; no native transport exists here.
|
||||
|
||||
Call from the mediator's finally block while its original Journal.locked context
|
||||
is still live. The owner adapter retains credential/handle custody and reconstructs
|
||||
and compares the native call with the expected digest before effect. Its three
|
||||
methods take no caller-selected path, body, shell command or token. This source
|
||||
component neither validates consume nor creates a native execution entitlement.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import hashlib
|
||||
import re
|
||||
from typing import Protocol
|
||||
|
||||
from task_execution_journal import Binding, Cleanup, Journal, Refused
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CleanupResult:
|
||||
confirmed: bool
|
||||
evidence_digest: str
|
||||
|
||||
def validate(self):
|
||||
if type(self.confirmed) is not bool or type(self.evidence_digest) is not str or not re.fullmatch(r"[0-9a-f]{64}", self.evidence_digest):
|
||||
raise Refused("invalid owner cleanup evidence")
|
||||
|
||||
|
||||
class OwnerCleanup(Protocol):
|
||||
# Confirm requires actual native revoke/absence evidence; a timeout is never
|
||||
# proof of absence. Implementations must use bounded calls and discard raw
|
||||
# backend failure output. Protected handles never leave the owner adapter.
|
||||
def revoke_task_token(self, expected_call_digest: str) -> CleanupResult: ...
|
||||
def revoke_identity_token(self, expected_call_digest: str) -> CleanupResult: ...
|
||||
def delete_grant_policy(self, expected_call_digest: str) -> CleanupResult: ...
|
||||
|
||||
|
||||
def run_cleanup(journal: Journal, binding: Binding, owner: OwnerCleanup):
|
||||
"""Attempt each fixed obligation once, preserving business/unknown outcomes.
|
||||
|
||||
Ordinary adapter failures record unconfirmed evidence and do not suppress the
|
||||
remaining independent cleanup. BaseException/process death leaves durable
|
||||
pending intent; neither this driver nor a reopened journal retries it.
|
||||
Journal failures propagate and prohibit dispatch without durable intent.
|
||||
"""
|
||||
obligations = journal.cleanup_status(binding)['obligations']
|
||||
if not obligations:
|
||||
raise Refused("explicit owner cleanup obligations required")
|
||||
if any(item['outcome'] != 'owed' for item in obligations):
|
||||
raise Refused("cleanup already attempted; owner reconciliation required")
|
||||
for item in obligations:
|
||||
obligation = Cleanup(item['kind'], item['call_digest'])
|
||||
journal.before_cleanup(binding, item['ordinal'], obligation)
|
||||
try:
|
||||
if obligation.kind == 'task-token-revoke':
|
||||
result = owner.revoke_task_token(obligation.call_digest)
|
||||
elif obligation.kind == 'identity-token-revoke':
|
||||
result = owner.revoke_identity_token(obligation.call_digest)
|
||||
elif obligation.kind == 'grant-policy-delete':
|
||||
result = owner.delete_grant_policy(obligation.call_digest)
|
||||
else:
|
||||
raise Refused("unsupported cleanup obligation")
|
||||
if type(result) is not CleanupResult:
|
||||
raise Refused("typed owner cleanup evidence required")
|
||||
result.validate()
|
||||
except Exception:
|
||||
# Closed metadata only. Never hash/log a backend exception, response,
|
||||
# token, accessor or secret-bearing stdout/stderr.
|
||||
result = CleanupResult(False, hashlib.sha256(b'owner-cleanup-unconfirmed/v1').hexdigest())
|
||||
journal.after_cleanup(binding, item['ordinal'], obligation,
|
||||
confirmed=result.confirmed, evidence_digest=result.evidence_digest)
|
||||
return journal.cleanup_status(binding)
|
||||
|
|
@ -57,6 +57,26 @@ _NEXT = {
|
|||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Cleanup:
|
||||
"""Owner-derived exact call commitment, never a handle or authority token.
|
||||
|
||||
The owner mediator must include these commitments in its approved plan before
|
||||
reservation, then compare the actual native cleanup call immediately before
|
||||
dispatch. This journal does not authenticate that plan or native adapter.
|
||||
"""
|
||||
kind: str
|
||||
call_digest: str
|
||||
|
||||
def validate(self):
|
||||
if type(self.kind) is not str or self.kind not in {
|
||||
"task-token-revoke", "identity-token-revoke", "grant-policy-delete"
|
||||
}:
|
||||
raise Refused("unsupported cleanup obligation")
|
||||
if type(self.call_digest) is not str or not re.fullmatch(r"[0-9a-f]{64}", self.call_digest):
|
||||
raise Refused("invalid cleanup call digest")
|
||||
|
||||
|
||||
class Journal:
|
||||
def __init__(self, directory: Path):
|
||||
# Require an existing owner-private directory: deployment creates it.
|
||||
|
|
@ -82,8 +102,10 @@ class Journal:
|
|||
self.db.execute("PRAGMA journal_mode=DELETE")
|
||||
self.db.execute("CREATE TABLE IF NOT EXISTS operations (approval TEXT PRIMARY KEY, request TEXT NOT NULL, operation TEXT UNIQUE NOT NULL, plan TEXT NOT NULL, state TEXT NOT NULL, evidence TEXT)")
|
||||
self.db.execute("CREATE TABLE IF NOT EXISTS calls (operation TEXT NOT NULL, ordinal INTEGER NOT NULL, call_digest TEXT NOT NULL, outcome TEXT NOT NULL, evidence TEXT, PRIMARY KEY(operation,ordinal))")
|
||||
self.db.execute("CREATE TABLE IF NOT EXISTS cleanup (operation TEXT NOT NULL, ordinal INTEGER NOT NULL, kind TEXT NOT NULL, call_digest TEXT NOT NULL, outcome TEXT NOT NULL, evidence TEXT, PRIMARY KEY(operation,ordinal), UNIQUE(operation,kind))")
|
||||
self._held = False
|
||||
self._admission = None
|
||||
self._consumed_admission = None
|
||||
self._sync_directory()
|
||||
|
||||
@staticmethod
|
||||
|
|
@ -123,11 +145,20 @@ class Journal:
|
|||
yield self
|
||||
finally:
|
||||
self._admission = None
|
||||
self._consumed_admission = None
|
||||
self._held = False
|
||||
fcntl.flock(self._lock_fd, fcntl.LOCK_UN)
|
||||
|
||||
def reserve(self, binding: Binding):
|
||||
def reserve(self, binding: Binding, cleanup: tuple[Cleanup, ...] = ()):
|
||||
binding.validate()
|
||||
if type(cleanup) is not tuple or len(cleanup) > 3:
|
||||
raise Refused("closed cleanup plan required")
|
||||
for item in cleanup:
|
||||
if type(item) is not Cleanup:
|
||||
raise Refused("typed cleanup obligation required")
|
||||
item.validate()
|
||||
if len({item.kind for item in cleanup}) != len(cleanup):
|
||||
raise Refused("duplicate cleanup obligation")
|
||||
if not self._held or self._admission is not None:
|
||||
raise Refused("reservation requires fresh exclusive invocation")
|
||||
try:
|
||||
|
|
@ -135,6 +166,9 @@ class Journal:
|
|||
self.db.execute("INSERT INTO operations VALUES (?,?,?,?,?,NULL)",
|
||||
(binding.approval_id, binding.request_digest,
|
||||
binding.operation_id, binding.plan_digest, "reserved"))
|
||||
self.db.executemany("INSERT INTO cleanup VALUES (?,?,?,?,'owed',NULL)",
|
||||
[(binding.operation_id, i, item.kind, item.call_digest)
|
||||
for i, item in enumerate(cleanup)])
|
||||
self.db.execute("COMMIT")
|
||||
self._sync_directory()
|
||||
except sqlite3.IntegrityError:
|
||||
|
|
@ -158,6 +192,13 @@ class Journal:
|
|||
try:
|
||||
self.db.execute("BEGIN IMMEDIATE")
|
||||
pending=self.db.execute("SELECT COUNT(*) FROM calls WHERE operation=? AND outcome='pending'",(binding.operation_id,)).fetchone()[0]
|
||||
started = self.db.execute("SELECT COUNT(*) FROM cleanup WHERE operation=? AND outcome!='owed'", (binding.operation_id,)).fetchone()[0]
|
||||
if started and target not in {'unknown', 'cleanup-confirmed'}:
|
||||
raise Refused("cleanup has closed business execution")
|
||||
if target == 'cleanup-confirmed':
|
||||
total, owed = self.db.execute("SELECT COUNT(*),COALESCE(SUM(outcome!='confirmed'),0) FROM cleanup WHERE operation=?", (binding.operation_id,)).fetchone()
|
||||
if not total or owed:
|
||||
raise Refused("explicit cleanup obligations are unconfirmed")
|
||||
if pending and target != 'unknown':
|
||||
raise Refused("unconfirmed external call requires unknown")
|
||||
changed = self.db.execute(
|
||||
|
|
@ -168,6 +209,8 @@ class Journal:
|
|||
raise Refused("journal state changed")
|
||||
self.db.execute("COMMIT")
|
||||
self._sync_directory()
|
||||
if target == "consume-confirmed":
|
||||
self._consumed_admission = binding
|
||||
except BaseException:
|
||||
if self.db.in_transaction:
|
||||
self.db.execute("ROLLBACK")
|
||||
|
|
@ -182,6 +225,7 @@ class Journal:
|
|||
raise Refused("journal binding mismatch")
|
||||
# Readback cannot create a live admission, even for an interrupted state.
|
||||
pending = self.db.execute("SELECT COUNT(*) FROM calls WHERE operation=? AND outcome='pending'", (binding.operation_id,)).fetchone()[0]
|
||||
pending += self.db.execute("SELECT COUNT(*) FROM cleanup WHERE operation=? AND outcome='pending'", (binding.operation_id,)).fetchone()[0]
|
||||
return {"state": row[3], "evidence_digest": row[4], "dispatch_allowed": False,
|
||||
"external_outcome_unknown": bool(pending)}
|
||||
|
||||
|
|
@ -201,7 +245,8 @@ class Journal:
|
|||
self.db.execute("BEGIN IMMEDIATE")
|
||||
count,pending = self.db.execute("SELECT COUNT(*), COALESCE(SUM(outcome='pending'),0) FROM calls WHERE operation=?",(binding.operation_id,)).fetchone()
|
||||
state=self.db.execute("SELECT state FROM operations WHERE operation=?",(binding.operation_id,)).fetchone()[0]
|
||||
if ordinal != count or pending or state in {'failed','unknown','cleanup-confirmed'}:
|
||||
cleanup_started = self.db.execute("SELECT COUNT(*) FROM cleanup WHERE operation=? AND outcome!='owed'", (binding.operation_id,)).fetchone()[0]
|
||||
if cleanup_started or ordinal != count or pending or state in {'failed','unknown','cleanup-confirmed'}:
|
||||
raise Refused("call is replayed, uncertain or terminal")
|
||||
self.db.execute("INSERT INTO calls VALUES (?,?,?,'pending',NULL)",(binding.operation_id,ordinal,call_digest))
|
||||
self.db.execute("COMMIT")
|
||||
|
|
@ -229,6 +274,61 @@ class Journal:
|
|||
if self.db.in_transaction: self.db.execute("ROLLBACK")
|
||||
raise
|
||||
|
||||
def cleanup_status(self, binding: Binding):
|
||||
# Observation never restores live execution or cleanup authority.
|
||||
if self.observe(binding) is None:
|
||||
raise Refused("operation is not admitted")
|
||||
rows = self.db.execute("SELECT ordinal,kind,call_digest,outcome,evidence FROM cleanup WHERE operation=? ORDER BY ordinal", (binding.operation_id,)).fetchall()
|
||||
return {"confirmed": bool(rows) and all(row[3] == "confirmed" for row in rows),
|
||||
"obligations": [{"ordinal": row[0], "kind": row[1], "call_digest": row[2],
|
||||
"outcome": row[3], "evidence_digest": row[4]} for row in rows]}
|
||||
|
||||
def _cleanup_admission(self, binding, ordinal, obligation):
|
||||
if not self._held or self._admission != binding or self._consumed_admission != binding:
|
||||
raise Refused("cleanup requires same live consumed admission")
|
||||
if type(ordinal) is not int or not 0 <= ordinal < 3 or type(obligation) is not Cleanup:
|
||||
raise Refused("exact cleanup obligation required")
|
||||
obligation.validate()
|
||||
|
||||
def before_cleanup(self, binding: Binding, ordinal: int, obligation: Cleanup):
|
||||
"""Commit intent for one pre-admitted cleanup call, even after unknown.
|
||||
|
||||
Never resumes an interrupted invocation. Independent owner recovery after
|
||||
a crash still requires the owner's protected recovery channel/approval.
|
||||
Cleanup has no business deadline: its authority is the original separate
|
||||
explicit obligation, and cannot authorize a read, repair or business write.
|
||||
"""
|
||||
self._cleanup_admission(binding, ordinal, obligation)
|
||||
try:
|
||||
self.db.execute("BEGIN IMMEDIATE")
|
||||
changed = self.db.execute("UPDATE cleanup SET outcome='pending' WHERE operation=? AND ordinal=? AND kind=? AND call_digest=? AND outcome='owed'",
|
||||
(binding.operation_id, ordinal, obligation.kind, obligation.call_digest)).rowcount
|
||||
if changed != 1:
|
||||
raise Refused("cleanup is unadmitted, replayed or uncertain")
|
||||
self.db.execute("COMMIT")
|
||||
self._sync_directory()
|
||||
except BaseException:
|
||||
if self.db.in_transaction: self.db.execute("ROLLBACK")
|
||||
raise
|
||||
|
||||
def after_cleanup(self, binding: Binding, ordinal: int, obligation: Cleanup,
|
||||
*, confirmed: bool, evidence_digest: str):
|
||||
self._cleanup_admission(binding, ordinal, obligation)
|
||||
if type(confirmed) is not bool or type(evidence_digest) is not str or not re.fullmatch(r"[0-9a-f]{64}", evidence_digest):
|
||||
raise Refused("closed cleanup evidence required")
|
||||
try:
|
||||
self.db.execute("BEGIN IMMEDIATE")
|
||||
changed = self.db.execute("UPDATE cleanup SET outcome=?, evidence=? WHERE operation=? AND ordinal=? AND kind=? AND call_digest=? AND outcome='pending'",
|
||||
('confirmed' if confirmed else 'unconfirmed', evidence_digest,
|
||||
binding.operation_id, ordinal, obligation.kind, obligation.call_digest)).rowcount
|
||||
if changed != 1:
|
||||
raise Refused("cleanup confirmation mismatch")
|
||||
self.db.execute("COMMIT")
|
||||
self._sync_directory()
|
||||
except BaseException:
|
||||
if self.db.in_transaction: self.db.execute("ROLLBACK")
|
||||
raise
|
||||
|
||||
def close(self):
|
||||
if hasattr(self, "db"):
|
||||
self.db.close()
|
||||
|
|
|
|||
199
tests/test_task_execution_cleanup.py
Normal file
199
tests/test_task_execution_cleanup.py
Normal file
|
|
@ -0,0 +1,199 @@
|
|||
"""Disposable local SQLite/owner-adapter fixtures. No OpenBao or credential access."""
|
||||
import multiprocessing
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import uuid
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'scripts'))
|
||||
from task_execution_journal import Binding, Cleanup, Journal, Refused
|
||||
from task_execution_cleanup import CleanupResult, run_cleanup
|
||||
|
||||
OBLIGATIONS = (Cleanup('task-token-revoke', '1'*64),
|
||||
Cleanup('identity-token-revoke', '2'*64),
|
||||
Cleanup('grant-policy-delete', '3'*64))
|
||||
|
||||
|
||||
class OwnerFixture:
|
||||
def __init__(self, fail=None): self.calls = []; self.fail = fail
|
||||
def call(self, kind, digest):
|
||||
self.calls.append((kind, digest))
|
||||
if kind == self.fail: raise TimeoutError('SYNTHETIC-PRIVATE-BACKEND-OUTPUT')
|
||||
return CleanupResult(True, 'e'*64)
|
||||
def revoke_task_token(self, digest): return self.call('task-token-revoke', digest)
|
||||
def revoke_identity_token(self, digest): return self.call('identity-token-revoke', digest)
|
||||
def delete_grant_policy(self, digest): return self.call('grant-policy-delete', digest)
|
||||
|
||||
|
||||
def crash_worker(directory, binding, after_effect):
|
||||
journal = Journal(Path(directory))
|
||||
with journal.locked():
|
||||
journal.reserve(binding, OBLIGATIONS)
|
||||
journal.transition(binding, 'reserved', 'consume-confirmed')
|
||||
journal.before_cleanup(binding, 0, OBLIGATIONS[0])
|
||||
if after_effect:
|
||||
# A real durable synthetic effect, separate from the admission DB.
|
||||
fd = os.open(Path(directory)/'synthetic-effect', os.O_CREAT | os.O_WRONLY | os.O_APPEND, 0o600)
|
||||
os.write(fd, b'one fixture revocation\n'); os.fsync(fd); os.close(fd)
|
||||
os._exit(23)
|
||||
|
||||
|
||||
def cleanup_racer(directory, binding, gate, results):
|
||||
journal = Journal(Path(directory))
|
||||
gate.wait()
|
||||
try:
|
||||
with journal.locked():
|
||||
journal.reserve(binding, OBLIGATIONS)
|
||||
journal.transition(binding, 'reserved', 'consume-confirmed')
|
||||
run_cleanup(journal, binding, OwnerFixture())
|
||||
results.put('admitted')
|
||||
except Refused: results.put('refused')
|
||||
finally: journal.close()
|
||||
|
||||
|
||||
class Tests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory(); self.addCleanup(self.tmp.cleanup)
|
||||
self.path = Path(self.tmp.name)
|
||||
self.binding = Binding('approval-cleanup', 'sha256:'+'a'*64, str(uuid.uuid4()), 'b'*64)
|
||||
self.journal = Journal(self.path); self.addCleanup(self.journal.close)
|
||||
|
||||
def consumed(self):
|
||||
self.journal.reserve(self.binding, OBLIGATIONS)
|
||||
self.journal.transition(self.binding, 'reserved', 'consume-confirmed')
|
||||
|
||||
def test_timeout_business_call_keeps_unknown_while_all_cleanup_runs(self):
|
||||
with self.journal.locked():
|
||||
self.consumed()
|
||||
self.journal.before_call(self.binding, 0, 'd'*64)
|
||||
self.journal.transition(self.binding, 'consume-confirmed', 'unknown')
|
||||
owner = OwnerFixture()
|
||||
status = run_cleanup(self.journal, self.binding, owner)
|
||||
self.assertTrue(status['confirmed'])
|
||||
self.assertEqual(owner.calls, [(c.kind,c.call_digest) for c in OBLIGATIONS])
|
||||
self.assertEqual(self.journal.observe(self.binding)['state'], 'unknown')
|
||||
self.assertTrue(self.journal.observe(self.binding)['external_outcome_unknown'])
|
||||
with self.assertRaises(Refused): self.journal.before_call(self.binding, 1, 'f'*64)
|
||||
|
||||
def test_failed_business_outcome_preserved(self):
|
||||
with self.journal.locked():
|
||||
self.consumed(); self.journal.transition(self.binding,'consume-confirmed','failed')
|
||||
self.assertTrue(run_cleanup(self.journal,self.binding,OwnerFixture())['confirmed'])
|
||||
self.assertEqual(self.journal.observe(self.binding)['state'], 'failed')
|
||||
|
||||
def test_one_revoke_failure_never_passes_but_independent_cleanup_continues(self):
|
||||
with self.journal.locked():
|
||||
self.consumed(); owner = OwnerFixture('task-token-revoke')
|
||||
result = run_cleanup(self.journal,self.binding,owner)
|
||||
self.assertFalse(result['confirmed']); self.assertEqual(len(owner.calls),3)
|
||||
self.assertEqual([r['outcome'] for r in result['obligations']], ['unconfirmed','confirmed','confirmed'])
|
||||
self.assertNotIn('SYNTHETIC-PRIVATE', str(result))
|
||||
with self.assertRaises(Refused): run_cleanup(self.journal,self.binding,owner)
|
||||
self.assertEqual(len(owner.calls),3)
|
||||
self.assertNotIn(b'SYNTHETIC-PRIVATE', (self.path/'journal.sqlite').read_bytes())
|
||||
|
||||
def test_cleanup_closes_business_before_native_attempt(self):
|
||||
with self.journal.locked():
|
||||
self.consumed(); self.journal.before_cleanup(self.binding,0,OBLIGATIONS[0])
|
||||
with self.assertRaises(Refused): self.journal.before_call(self.binding,0,'d'*64)
|
||||
with self.assertRaises(Refused): self.journal.transition(self.binding,'consume-confirmed','authenticating')
|
||||
|
||||
def test_no_preconsume_cleanup_even_when_unknown(self):
|
||||
with self.journal.locked():
|
||||
self.journal.reserve(self.binding,OBLIGATIONS)
|
||||
self.journal.transition(self.binding,'reserved','unknown')
|
||||
owner = OwnerFixture()
|
||||
with self.assertRaises(Refused): run_cleanup(self.journal,self.binding,owner)
|
||||
self.assertEqual(owner.calls,[])
|
||||
|
||||
def test_changed_kind_digest_ordinal_or_binding_never_dispatches(self):
|
||||
with self.journal.locked():
|
||||
self.consumed()
|
||||
for ordinal,item in [(False,OBLIGATIONS[0]),(3,OBLIGATIONS[0]),(0,Cleanup('task-token-revoke','9'*64)),(0,Cleanup('kv-delete','1'*64)),(0,{'kind':'task-token-revoke'})]:
|
||||
with self.subTest(ordinal=ordinal,item=item):
|
||||
with self.assertRaises(Refused): self.journal.before_cleanup(self.binding,ordinal,item)
|
||||
changed = Binding('another',self.binding.request_digest,self.binding.operation_id,self.binding.plan_digest)
|
||||
with self.assertRaises(Refused): self.journal.before_cleanup(changed,0,OBLIGATIONS[0])
|
||||
self.assertTrue(all(x['outcome']=='owed' for x in self.journal.cleanup_status(self.binding)['obligations']))
|
||||
|
||||
def test_cleanup_plan_must_be_closed_before_reservation(self):
|
||||
for plan in [list(OBLIGATIONS), (OBLIGATIONS[0],OBLIGATIONS[0]), (Cleanup('kv-delete','1'*64),), (Cleanup('task-token-revoke',False),)]:
|
||||
with self.journal.locked():
|
||||
with self.assertRaises(Refused): self.journal.reserve(self.binding,plan)
|
||||
self.assertIsNone(self.journal.observe(self.binding))
|
||||
|
||||
def test_crash_before_and_after_real_synthetic_effect_does_not_resume(self):
|
||||
context = multiprocessing.get_context('fork')
|
||||
for after_effect in (False, True):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
worker = context.Process(target=crash_worker,args=(directory,self.binding,after_effect))
|
||||
worker.start(); worker.join(10)
|
||||
if worker.is_alive(): worker.terminate(); worker.join(); self.fail('crash fixture exceeded bound')
|
||||
self.assertEqual(worker.exitcode,23)
|
||||
reopened = Journal(Path(directory))
|
||||
try:
|
||||
self.assertEqual(reopened.cleanup_status(self.binding)['obligations'][0]['outcome'],'pending')
|
||||
self.assertTrue(reopened.observe(self.binding)['external_outcome_unknown'])
|
||||
owner = OwnerFixture()
|
||||
with reopened.locked():
|
||||
with self.assertRaises(Refused): reopened.reserve(self.binding,OBLIGATIONS)
|
||||
with self.assertRaises(Refused): run_cleanup(reopened,self.binding,owner)
|
||||
with self.assertRaises(Refused): reopened.before_cleanup(self.binding,1,OBLIGATIONS[1])
|
||||
self.assertEqual(owner.calls,[])
|
||||
effect = Path(directory)/'synthetic-effect'
|
||||
self.assertEqual(effect.read_bytes() if effect.exists() else b'',b'one fixture revocation\n' if after_effect else b'')
|
||||
self.assertFalse(reopened.cleanup_status(self.binding)['confirmed'])
|
||||
finally: reopened.close()
|
||||
|
||||
def test_actual_process_race_admits_one_cleanup_driver(self):
|
||||
context = multiprocessing.get_context('fork'); gate = context.Event(); results = context.Queue()
|
||||
workers = [context.Process(target=cleanup_racer,args=(str(self.path),self.binding,gate,results)) for _ in range(4)]
|
||||
for worker in workers: worker.start()
|
||||
gate.set()
|
||||
for worker in workers:
|
||||
worker.join(10)
|
||||
if worker.is_alive(): worker.terminate(); worker.join(); self.fail('race exceeded bound')
|
||||
self.assertEqual(worker.exitcode,0)
|
||||
self.assertEqual(sorted(results.get(timeout=2) for _ in workers),['admitted','refused','refused','refused'])
|
||||
self.assertTrue(self.journal.cleanup_status(self.binding)['confirmed'])
|
||||
|
||||
def test_empty_or_malformed_native_result_never_proves_cleanup(self):
|
||||
with self.journal.locked():
|
||||
self.journal.reserve(self.binding)
|
||||
self.journal.transition(self.binding,'reserved','consume-confirmed')
|
||||
with self.assertRaises(Refused): run_cleanup(self.journal,self.binding,OwnerFixture())
|
||||
self.assertFalse(self.journal.cleanup_status(self.binding)['confirmed'])
|
||||
|
||||
def test_bad_native_confirmation_still_attempts_remaining_cleanup(self):
|
||||
class BadOwner(OwnerFixture):
|
||||
def revoke_task_token(self, digest):
|
||||
self.calls.append(('task-token-revoke',digest))
|
||||
return CleanupResult('true','e'*64)
|
||||
with self.journal.locked():
|
||||
self.consumed(); owner=BadOwner()
|
||||
self.assertFalse(run_cleanup(self.journal,self.binding,owner)['confirmed'])
|
||||
self.assertEqual(len(owner.calls),3)
|
||||
|
||||
def test_confirmation_exact_types_and_no_duplicate_receipts(self):
|
||||
with self.journal.locked():
|
||||
self.consumed(); self.journal.before_cleanup(self.binding,0,OBLIGATIONS[0])
|
||||
for confirmed,digest in [('true','e'*64),(1,'e'*64),(True,None),(True,'E'*64)]:
|
||||
with self.assertRaises(Refused): self.journal.after_cleanup(self.binding,0,OBLIGATIONS[0],confirmed=confirmed,evidence_digest=digest)
|
||||
self.journal.after_cleanup(self.binding,0,OBLIGATIONS[0],confirmed=True,evidence_digest='e'*64)
|
||||
with self.assertRaises(Refused): self.journal.after_cleanup(self.binding,0,OBLIGATIONS[0],confirmed=True,evidence_digest='e'*64)
|
||||
with self.assertRaises(Refused): self.journal.before_cleanup(self.binding,0,OBLIGATIONS[0])
|
||||
|
||||
def test_cleanup_state_is_independent_of_legacy_business_transition(self):
|
||||
with self.journal.locked():
|
||||
self.consumed()
|
||||
states=['consume-confirmed','authenticating','grant-issuing','grant-issued','applying','readback','verified']
|
||||
for before,after in zip(states,states[1:]): self.journal.transition(self.binding,before,after)
|
||||
with self.assertRaises(Refused): self.journal.transition(self.binding,'verified','cleanup-confirmed','e'*64)
|
||||
run_cleanup(self.journal,self.binding,OwnerFixture())
|
||||
self.journal.transition(self.binding,'verified','cleanup-confirmed','e'*64)
|
||||
self.assertEqual(self.journal.observe(self.binding)['state'],'cleanup-confirmed')
|
||||
|
||||
|
||||
if __name__ == '__main__': unittest.main()
|
||||
|
|
@ -12,6 +12,7 @@ module = importlib.util.module_from_spec(spec)
|
|||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
Binding, Journal, Refused = module.Binding, module.Journal, module.Refused
|
||||
Cleanup = module.Cleanup
|
||||
|
||||
|
||||
def reserve_worker(directory, binding, gate, results):
|
||||
|
|
@ -74,8 +75,12 @@ class Tests(unittest.TestCase):
|
|||
def test_full_path_cleanup_and_terminal_replay(self):
|
||||
states=['reserved','consume-confirmed','authenticating','grant-issuing','grant-issued','applying','readback','verified','cleanup-confirmed']
|
||||
with self.journal.locked():
|
||||
self.journal.reserve(self.binding)
|
||||
cleanup = Cleanup('task-token-revoke','d'*64)
|
||||
self.journal.reserve(self.binding,(cleanup,))
|
||||
for before,after in zip(states,states[1:]):
|
||||
if after == 'cleanup-confirmed':
|
||||
self.journal.before_cleanup(self.binding,0,cleanup)
|
||||
self.journal.after_cleanup(self.binding,0,cleanup,confirmed=True,evidence_digest='c'*64)
|
||||
self.journal.transition(self.binding,before,after,'c'*64)
|
||||
with self.assertRaises(Refused): self.journal.transition(self.binding,'cleanup-confirmed','applying')
|
||||
self.assertEqual(self.journal.observe(self.binding),{'state':'cleanup-confirmed','evidence_digest':'c'*64,'dispatch_allowed':False,'external_outcome_unknown':False})
|
||||
|
|
|
|||
|
|
@ -1225,3 +1225,17 @@ this change needs its own integration review. T12 retains all existing full
|
|||
contract, identity/consume, dispatch, custody and native acceptance requirements.
|
||||
RPF-IN-0012 remains open for the independent contract/wrapped-delivery return;
|
||||
this code correction does not authorize either operation.
|
||||
|
||||
2026-10-16 T12 source preparation (isolated owner checkout; not owner delivery):
|
||||
added durable precommitted cleanup obligations and separate outcome evidence to
|
||||
the existing Journal, plus a fixed three-method owner cleanup driver. Same live
|
||||
postconsume admission can clean after a business timeout/failed/unknown outcome;
|
||||
cleanup closes business dispatch and cannot replay or restore admission after a
|
||||
crash. Empty obligations no longer produce cleanup-confirmed. Real SQLite
|
||||
multiprocess races and abrupt process death before/after a separately durable
|
||||
synthetic effect verify refusal on reopen. Current compiler/journal/cleanup suite:
|
||||
73 passed, local source/fixture only. Details: docs/task-execution-cleanup.md.
|
||||
No native adapter or production mediator exists; full grant/request/cleanup binding,
|
||||
authenticated consume/identity, ephemeral token issuance, deadline/use checks,
|
||||
native effect/revoke/absence, audit and protected recovery remain prerequisites.
|
||||
T12 remains progress. No CCR38 default/entitlement or live grant was changed.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue