railiance-platform/docs/task-execution-native-fixture.md

Executable native metadata/grant/cleanup preparation

RPF-WP-0035-T12, 2026-10-16, second source package after the cleanup journal.

The fixed NativeFixtureAdapter actually dispatches to disposable loopback OpenBao. It requires a live owner-spawned OpenBao dev-process handle with the exact loopback port and dev/no-store-token arguments, and refuses every non-loopback origin. Production is deliberately not implemented. This is a guard against accidental misrouting inside the trusted fixture runtime, not a Python security sandbox. No estate credentials, grants, approvals or policy packages are read or changed. The tests start in-memory dev servers with synthetic root/session credentials, synthetic owner admission, grant/state/source digests and a synthetic postconsume journal transition. This proves native primitives, not authenticated admission.

The owner Compiler now deterministically includes the operation-specific temporary policy name and cleanup call commitments in each serialized Plan. Commitments include identity/task credential slots, exact native revoke/verify or delete/verify method/path pairs, and the narrow no-issue/no-create skip branches. The full execution descriptor can be bound into that Plan only by byte equality with an independently installed owner descriptor, with operation/actor/CCR/catalogue/six operations correspondence. Compiler revalidation preserves both descriptor and cleanup content. The descriptor's SHA is preparation provenance; no opaque digest or local file becomes approval or verified canonical request correspondence.

The adapter needs the exact compiler/Plan/binding and separately installed owner fixture inputs. All handles stay in owner memory with hidden representations. It requires the same live consumed journal admission and exact grant-issuing state. It refuses an existing per-operation grant policy, creates only the two compiler metadata endpoints plus revoke-self, verifies HCL, mints one parented service token with only this policy, no default/identity policies, nonrenewable TTL <=120 seconds and twelve uses, then verifies native token bounds through the owner. Six ordered metadata calls use independently recompiled source/method/path/body. A task use is reserved for revoke; native token limits also enforce the ceiling. Business calls check wall and monotonic deadlines before effect, with a five-second cleanup reserve. No retry/remint, caller HCL/JSON/path, mount/AppRole/KV/SecretID or shell API exists. Wave 5 below adds exact sequential snapshot acceptance; native baseline CAS remains absent.

Every business/grant request records durable intent and closed status evidence. Native cleanup has its own durable per-request intents beneath the original committed obligation; accessor verification cannot erase an uncertain revoke. The native adapter compares the exact committed cleanup sequence before request. Self-revoke success must be followed by owner lookup-accessor's specific invalid accessor response; generic HTTP400/403, timeout and policy deletion do not prove revocation. The two explicitly accepted error encodings cover OpenBao's plain and wrapped single-invalid-accessor forms. Native error text is never logged or hashed. Cleanup bypasses expired business deadlines under the retained explicit obligation, never source-driven business repair. Independent cleanup continues after failure; retirement never overwrites the original unknown/failed business outcome.

Current actual native tests cover policy/body and K8s-role application/readback, owner-parented TTL/use policy bounds, native sibling/KV/token/mount/SecretID capability denial, both token accessor-invalid proofs, policy absence, deadline-expired cleanup, source drift, no preconsume materialization, changed descriptor/cleanup commitments, nonloopback refusal, actual write followed by simulated lost response, and actual policy retirement with failed task revocation. The lost response is fault injection after a successful real native write; it does not pretend to be an actual network partition. The failed-revoke fixture independently confirms the still-existing native token, then performs explicit synthetic owner recovery. No blind retry occurs.

Minimal remaining decisions and integration

1. Define/admit task-execution-grant in the owner CCR validator. Bind the exact cleanup commitments, temporary policy/token bodies and source revisions into its immutable projection, referenced by the existing descriptor grant SHA. The descriptor currently has no cleanup field; no engine schema extension is needed if this owner grant binding is made explicit. The immutable projection/excluded append-only fields are not implemented in the current owner validator. This is the concrete owner schema/admission decision, not a need for a new service.
2. Have the existing authoritative engines supply authenticated initiating principal, exact canonical request + served claim/PDP/package and independent signed CAS consume verification to the owner mediator before its journal transition. The native fixture's local transition supplies none of that evidence.
3. Install exact origin/cluster/source/grant/baseline/readback inputs under the attended owner runtime. The current binder compares independently installed descriptor bytes, but does not authenticate source-bundle paths, grant digest or compute immutable CCR projection. The six-operation plan still comes from pinned actual owner source. Production origin authentication and deadline bounds against served approval/PDP remain separate.
4. Finish expected-state normalization and conflict handling for native K8s/OIDC defaults; add per-policy/role approved baseline comparison and exact readback verification. The fixture checks known fields, not total production equality. OpenBao metadata PUT has no native atomic CAS; the owner must choose exclusive admitted target custody or an honest conflict/recovery protocol.
5. Wire the fixed adapter to that mediator with bounded runtime isolation, owner-private uncertain-issuance recovery, external audit outbox and exact founder-scoped attended trial. No production caller or protected recovery channel exists here. The existing production readiness gate remains refused.

No umbrella broker, general executor, new credential catalogue or standing service admin entitlement is introduced. Approval CAS prevents approval replay; it cannot make uncertain native effects exactly-once. RPF T12 and Secrets WP13 remain open.

Exact disposable metadata acceptance (2026-10-16, wave 5)

The owner-installed FixtureAdmission now carries canonical JSON before/after snapshot bytes. Their SHA-256 values must equal the existing descriptor's expected_before and expected_after, and both snapshots must name exactly the two compiled metadata paths. Explicit JSON null means absence. Only the actual OpenBao empty-errors 404 is accepted as absence; denied/malformed reads fail. These inputs contain metadata, never KV values or private token handles.

Policy comparison retains name, HCL, version and cas_required. Only the modification timestamp is excluded, after requiring an aware timestamp and the closed observed policy field set. Role comparison retains every returned field, including aliases, token defaults and unexpected additions, using canonical JSON bytes (therefore false cannot equal zero). The disposable Kubernetes fixture pins the complete OpenBao 2.5.5 response shape. Another plugin/version/default requires an independently reviewed owner snapshot; nothing infers new accepted defaults.

Each baseline gates its corresponding write. Policy readback occurs before role baseline/write/readback. A role conflict can therefore leave the policy applied; that operation remains unknown. This sequence has no native CAS, aggregate baseline gate, concurrent-writer exclusion, or simultaneous final-state proof. After all six calls match the committed snapshots, the same-live adapter records readback and verified with a value-free evidence digest. The mediator reports native_apply verified; cleanup-confirmed requires all separate cleanup receipts. Readback success with unconfirmed cleanup cannot reach cleanup-confirmed.

Reopening SQLite cannot dispatch, reconsume, restore token/accessor handles or retry cleanup. Native effects can survive process loss. Cleanup handles remain memory-only: no independently authorized recovery mechanism is implemented here. This closes disposable sequential apply acceptance only, not TSA03 or native production recovery/admission.

Independent cleanup recovery source successor

task_execution_recovery.py adds separately installed fixture-only owner custody and fresh recovery permits. Optional recovery_custody on NativeFixtureAdapter persists the original binding/cleanup commitments, exact operation policy, backend port and identity accessor before any grant effect; it commits mint intent before token creation and task accessor immediately after a valid issued response. Bearer credentials are never persisted. The custody directory is separate from the public journal, owner-only 0700/0600, bounded at 64KiB, no-follow, with atomic replacement and fsync. It is real process-independent persistence for synthetic fixtures. It does not isolate same-UID hostile execution, encrypt at rest, select production storage or establish a production secret policy. Those are explicit native custody/deployment gates, not supplied by Python types or file modes.

The independent owner reads its private custody revision and installs a fresh bounded permit, current owner credential and retained live disposable Popen. Permit revocation is durably owner-recorded. The recovery runner has no mint, consume, policy-apply or business method. It dispatches only fixed exact-accessor revoke/readback and exact task-exec-<operation> policy read/delete/readback. A differently recreated policy is never deleted. Backend admission remains the same loopback-only owner-spawned dev process; no estate backend can be selected. Accessor-based recovery is an independently admitted alternate cleanup projection, not a reinterpretation of the original revoke-self call digest. Production needs exact owner-reviewed recovery scope; original business grant does not imply it.

Fresh recovery holds the journal process lock but never sets business admission or consumed-admission fields. Separate recovery attempt/call/result tables retain original cleanup statuses before reconciliation, write native intent before calls, and preserve failed/pending outcomes. Proven cleanup alone updates the exact original cleanup obligation to confirmed. Business state and original pending call uncertainty remain unchanged. Reusing the original approval, continuing metadata calls and reusing a recovery permit refuse. Failure needs another independent owner permit; there is no automatic retry. Expired/revoked permits or changed private custody revision refuse before calls.

A token-create response lost before accessor custody remains unresolved token cleanup debt: policy deletion/identity revocation can be independently confirmed, but cannot imply task-token revocation. Neither expiry guessed from wall clock nor policy absence fabricates token absence. A production successor must admit an independent exact ambiguous-issuance reconciliation mechanism or prove bounded expiry from owner backend evidence. No such new grant is implemented here.

test_task_execution_recovery.py forks a real executor and terminates it without finally cleanup after issue, business effect/lost response and revoke/lost response. The independent parent, holding the backend and separately installed custody, recovers specific obligations and verifies actual token/policy absence. Additional cases cover lost-mint response, revoked credential, permit revocation/expiry, revision drift, namespace policy-name reuse and private-file/binding refusals. This is source fixture proof only. Native custody, isolation, exact recovery-role permissions, retention/encryption, clock admission, audit delivery and human scope approval remain missing. No generic broker, production reader, grant or CLI is added.

Recovery R2 exact absence and authenticated capsule refinement

R1 overclaimed policy absence for arbitrary 404 bodies. R2 requires exactly {"errors":[]} for both initial GET and post-DELETE GET. Other 404 bodies, permission denials, extra data, malformed/duplicate-key/nonfinite JSON and failed readback cannot confirm policy cleanup. A successful DELETE status alone cannot confirm absence. Recovery native policy comparison admits precisely one optional terminal LF, matching the retained OpenBao HCL source/readback convention; spaces, tabs, CR, extra terminal LF and any other byte change are refused. This is a narrow declared normalization, not full whitespace or semantic equivalence.

Private fixture capsules now carry HMAC-SHA256 authentication under an exactly 32-byte integrity key supplied independently by the recovery owner. The key is never stored in the capsule or public journal, and is not a bearer credential. The fixture owner retains this key independently across executor death; a fresh owner custody object must supply the same key. Wrong-key or damaged accessor/HCL bytes refuse before admission. This is integrity protection, not encryption or same-UID isolation; production key custody, rotation/retention and recovery actor selection remain explicitly unadmitted.

After authentication the reader enforces exact top-level and nested binding/ cleanup key sets and types, validated Binding/Cleanup formats and the ordered source-derived cleanup_specs operation digests. Duplicate JSON keys, nonfinite numbers, malformed phase/revision combinations and type-coercible values refuse. The reader bounds bytes, rejects indirect/nonprivate files and detects stat/size changes during its single read. A valid older prepared capsule still cannot prove a task token absent when durable journal state/calls reached issuance: that remains unknown cleanup debt. Authentication alone is not anti-rollback. No business or consume admission is restored and no production scope is granted.