diff --git a/docs/task-execution-native-fixture.md b/docs/task-execution-native-fixture.md new file mode 100644 index 0000000..7096f1a --- /dev/null +++ b/docs/task-execution-native-fixture.md @@ -0,0 +1,91 @@ +# Executable native metadata/grant/cleanup preparation + +RPF-WP-0035-T12, 2026-10-16, second source package after the cleanup journal. + +The fixed `NativeFixtureAdapter` actually dispatches to disposable loopback OpenBao. +It requires a live owner-spawned OpenBao dev-process handle with the exact loopback +port and dev/no-store-token arguments, and refuses every non-loopback origin. +Production is deliberately not implemented. This is a guard against accidental +misrouting inside the trusted fixture runtime, not a Python security sandbox. +No estate credentials, grants, approvals or policy packages are read or changed. +The tests start in-memory dev servers with synthetic root/session credentials, +synthetic owner admission, grant/state/source digests and a synthetic postconsume +journal transition. This proves native primitives, not authenticated admission. + +The owner Compiler now deterministically includes the operation-specific temporary +policy name and cleanup call commitments in each serialized Plan. Commitments +include identity/task credential slots, exact native revoke/verify or delete/verify +method/path pairs, and the narrow no-issue/no-create skip branches. The full +execution descriptor can be bound into that Plan only by byte equality with an +independently installed owner descriptor, with operation/actor/CCR/catalogue/six +operations correspondence. Compiler revalidation preserves both descriptor and +cleanup content. The descriptor's SHA is preparation provenance; no opaque digest +or local file becomes approval or verified canonical request correspondence. + +The adapter needs the exact compiler/Plan/binding and separately installed owner +fixture inputs. All handles stay in owner memory with hidden representations. It +requires the same live consumed journal admission and exact grant-issuing state. +It refuses an existing per-operation grant policy, creates only the two compiler +metadata endpoints plus revoke-self, verifies HCL, mints one parented service token +with only this policy, no default/identity policies, nonrenewable TTL <=120 seconds +and twelve uses, then verifies native token bounds through the owner. Six ordered +metadata calls use independently recompiled source/method/path/body. A task use is +reserved for revoke; native token limits also enforce the ceiling. Business calls +check wall and monotonic deadlines before effect, with a five-second cleanup reserve. +No retry/remint, caller HCL/JSON/path, mount/AppRole/KV/SecretID or shell API exists. +The adapter cannot claim complete baseline CAS or normalized readback acceptance. + +Every business/grant request records durable intent and closed status evidence. +Native cleanup has its own durable per-request intents beneath the original +committed obligation; accessor verification cannot erase an uncertain revoke. +The native adapter compares the exact committed cleanup sequence before request. +Self-revoke success must be followed by owner lookup-accessor's specific invalid +accessor response; generic HTTP400/403, timeout and policy deletion do not prove +revocation. The two explicitly accepted error encodings cover OpenBao's plain and +wrapped single-invalid-accessor forms. Native error text is never logged or hashed. +Cleanup bypasses expired business deadlines under the retained explicit obligation, +never source-driven business repair. Independent cleanup continues after failure; +retirement never overwrites the original unknown/failed business outcome. + +Current actual native tests cover policy/body and K8s-role application/readback, +owner-parented TTL/use policy bounds, native sibling/KV/token/mount/SecretID capability +denial, both token accessor-invalid proofs, policy absence, deadline-expired cleanup, +source drift, no preconsume materialization, changed descriptor/cleanup commitments, +nonloopback refusal, actual write followed by simulated lost response, and actual +policy retirement with failed task revocation. The lost response is fault injection +after a successful real native write; it does not pretend to be an actual network +partition. The failed-revoke fixture independently confirms the still-existing +native token, then performs explicit synthetic owner recovery. No blind retry occurs. + +## Minimal remaining decisions and integration + +1. Define/admit `task-execution-grant` in the owner CCR validator. Bind the exact + cleanup commitments, temporary policy/token bodies and source revisions into its + immutable projection, referenced by the existing descriptor grant SHA. The + descriptor currently has no cleanup field; no engine schema extension is needed + if this owner grant binding is made explicit. The immutable projection/excluded + append-only fields are not implemented in the current owner validator. This is + the concrete owner schema/admission decision, not a need for a new service. +2. Have the existing authoritative engines supply authenticated initiating principal, + exact canonical request + served claim/PDP/package and independent signed CAS + consume verification to the owner mediator before its journal transition. + The native fixture's local transition supplies none of that evidence. +3. Install exact origin/cluster/source/grant/baseline/readback inputs under the + attended owner runtime. The current binder compares independently installed + descriptor bytes, but does not authenticate source-bundle paths, grant digest or + compute immutable CCR projection. The six-operation plan still comes from pinned + actual owner source. Production origin authentication and deadline bounds against + served approval/PDP remain separate. +4. Finish expected-state normalization and conflict handling for native K8s/OIDC + defaults; add per-policy/role approved baseline comparison and exact readback + verification. The fixture checks known fields, not total production equality. + OpenBao metadata PUT has no native atomic CAS; the owner must choose exclusive + admitted target custody or an honest conflict/recovery protocol. +5. Wire the fixed adapter to that mediator with bounded runtime isolation, + owner-private uncertain-issuance recovery, external audit outbox and exact + founder-scoped attended trial. No production caller or protected recovery channel + exists here. The existing production readiness gate remains refused. + +No umbrella broker, general executor, new credential catalogue or standing service +admin entitlement is introduced. Approval CAS prevents approval replay; it cannot +make uncertain native effects exactly-once. RPF T12 and Secrets WP13 remain open. diff --git a/scripts/task_execution_journal.py b/scripts/task_execution_journal.py index 7c54ac8..a16291b 100644 --- a/scripts/task_execution_journal.py +++ b/scripts/task_execution_journal.py @@ -103,6 +103,7 @@ class Journal: self.db.execute("CREATE TABLE IF NOT EXISTS operations (approval TEXT PRIMARY KEY, request TEXT NOT NULL, operation TEXT UNIQUE NOT NULL, plan TEXT NOT NULL, state TEXT NOT NULL, evidence TEXT)") self.db.execute("CREATE TABLE IF NOT EXISTS calls (operation TEXT NOT NULL, ordinal INTEGER NOT NULL, call_digest TEXT NOT NULL, outcome TEXT NOT NULL, evidence TEXT, PRIMARY KEY(operation,ordinal))") self.db.execute("CREATE TABLE IF NOT EXISTS cleanup (operation TEXT NOT NULL, ordinal INTEGER NOT NULL, kind TEXT NOT NULL, call_digest TEXT NOT NULL, outcome TEXT NOT NULL, evidence TEXT, PRIMARY KEY(operation,ordinal), UNIQUE(operation,kind))") + self.db.execute("CREATE TABLE IF NOT EXISTS cleanup_calls (operation TEXT NOT NULL, obligation INTEGER NOT NULL, step INTEGER NOT NULL, call_digest TEXT NOT NULL, outcome TEXT NOT NULL, evidence TEXT, PRIMARY KEY(operation,obligation,step))") self._held = False self._admission = None self._consumed_admission = None @@ -226,6 +227,7 @@ class Journal: # Readback cannot create a live admission, even for an interrupted state. pending = self.db.execute("SELECT COUNT(*) FROM calls WHERE operation=? AND outcome='pending'", (binding.operation_id,)).fetchone()[0] pending += self.db.execute("SELECT COUNT(*) FROM cleanup WHERE operation=? AND outcome='pending'", (binding.operation_id,)).fetchone()[0] + pending += self.db.execute("SELECT COUNT(*) FROM cleanup_calls WHERE operation=? AND outcome='pending'", (binding.operation_id,)).fetchone()[0] return {"state": row[3], "evidence_digest": row[4], "dispatch_allowed": False, "external_outcome_unknown": bool(pending)} @@ -318,6 +320,8 @@ class Journal: raise Refused("closed cleanup evidence required") try: self.db.execute("BEGIN IMMEDIATE") + if confirmed and self.db.execute("SELECT COUNT(*) FROM cleanup_calls WHERE operation=? AND obligation=? AND outcome='pending'", (binding.operation_id,ordinal)).fetchone()[0]: + raise Refused("pending cleanup native call cannot confirm") changed = self.db.execute("UPDATE cleanup SET outcome=?, evidence=? WHERE operation=? AND ordinal=? AND kind=? AND call_digest=? AND outcome='pending'", ('confirmed' if confirmed else 'unconfirmed', evidence_digest, binding.operation_id, ordinal, obligation.kind, obligation.call_digest)).rowcount @@ -329,6 +333,41 @@ class Journal: if self.db.in_transaction: self.db.execute("ROLLBACK") raise + def before_cleanup_call(self, binding, ordinal, obligation, step, call_digest): + """Journal each native revoke/delete AND its independent verification.""" + self._cleanup_admission(binding,ordinal,obligation) + if type(step) is not int or not 0<=step<2 or type(call_digest) is not str or not re.fullmatch(r"[0-9a-f]{64}",call_digest): + raise Refused("exact bounded cleanup native call required") + try: + self.db.execute("BEGIN IMMEDIATE") + parent=self.db.execute("SELECT kind,call_digest,outcome FROM cleanup WHERE operation=? AND ordinal=?",(binding.operation_id,ordinal)).fetchone() + count,pending=self.db.execute("SELECT COUNT(*),COALESCE(SUM(outcome='pending'),0) FROM cleanup_calls WHERE operation=? AND obligation=?",(binding.operation_id,ordinal)).fetchone() + if parent!=(obligation.kind,obligation.call_digest,'pending') or step!=count or pending: + raise Refused("cleanup call unadmitted, replayed or uncertain") + self.db.execute("INSERT INTO cleanup_calls VALUES (?,?,?,?,'pending',NULL)",(binding.operation_id,ordinal,step,call_digest)) + self.db.execute("COMMIT");self._sync_directory() + except BaseException: + if self.db.in_transaction:self.db.execute("ROLLBACK") + raise + + def after_cleanup_call(self,binding,ordinal,obligation,step,call_digest,evidence_digest): + self._cleanup_admission(binding,ordinal,obligation) + if (type(step) is not int or not 0<=step<2 or type(call_digest) is not str or not re.fullmatch(r"[0-9a-f]{64}",call_digest) + or type(evidence_digest) is not str or not re.fullmatch(r"[0-9a-f]{64}",evidence_digest)): + raise Refused("exact cleanup native evidence required") + try: + self.db.execute("BEGIN IMMEDIATE") + parent=self.db.execute("SELECT kind,call_digest,outcome FROM cleanup WHERE operation=? AND ordinal=?",(binding.operation_id,ordinal)).fetchone() + if parent!=(obligation.kind,obligation.call_digest,'pending'): + raise Refused("cleanup parent is no longer pending") + changed=self.db.execute("UPDATE cleanup_calls SET outcome='complete',evidence=? WHERE operation=? AND obligation=? AND step=? AND call_digest=? AND outcome='pending'", + (evidence_digest,binding.operation_id,ordinal,step,call_digest)).rowcount + if changed!=1:raise Refused("cleanup native confirmation mismatch") + self.db.execute("COMMIT");self._sync_directory() + except BaseException: + if self.db.in_transaction:self.db.execute("ROLLBACK") + raise + def close(self): if hasattr(self, "db"): self.db.close() diff --git a/scripts/task_execution_native_fixture.py b/scripts/task_execution_native_fixture.py new file mode 100644 index 0000000..7a4d247 --- /dev/null +++ b/scripts/task_execution_native_fixture.py @@ -0,0 +1,229 @@ +"""Fixed native metadata/grant/cleanup adapter for disposable loopback OpenBao. + +This adapter deliberately admits only a disposable loopback fixture. It supplies +real native primitive coverage while production grant schema, identity/consume and +origin/source admission are unfinished. It cannot open an estate backend. Owner +credentials/handles stay in memory; no subprocess, redirects or ambient proxy. +Python privacy is not credential isolation; production needs the owner runtime. +""" +from __future__ import annotations +from dataclasses import dataclass, field +from datetime import datetime, timezone +import hashlib +import http.client +import json +import subprocess +import time +from urllib.parse import urlsplit + +from task_execution_cleanup import CleanupResult +from task_execution_journal import Binding, Cleanup, Journal, Refused +from task_execution_plan import Compiler, Plan, cleanup_specs, encoded, content_hash + + +@dataclass(frozen=True) +class PrivateSession: + token: str = field(repr=False) + accessor: str = field(repr=False) + + +@dataclass(frozen=True) +class FixtureAdmission: + """Independently installed owner fixture inputs, not a request schema.""" + descriptor: bytes + owner_token: str = field(repr=False) + identity_session: PrivateSession = field(repr=False) + server_process: subprocess.Popen = field(repr=False) + + +class NativeFixtureAdapter: + def __init__(self, compiler: Compiler, plan: Plan, journal: Journal, + binding: Binding, admission: FixtureAdmission): + if type(admission) is not FixtureAdmission or type(admission.descriptor) is not bytes: + raise Refused('independent fixture admission required') + self.compiler,self.plan,self.journal,self.binding=compiler,plan,journal,binding + self.admission=admission + self.meta=plan.metadata() + descriptor=json.loads(admission.descriptor) + fresh=compiler.bind_execution(compiler.compile(self.meta['request']),descriptor,admission.descriptor) + if fresh.serialized != plan.serialized or binding.plan_digest != plan.content_digest: + raise Refused('owner bound plan mismatch') + if binding.operation_id != descriptor['operation_id'] or binding.request_digest != self.meta['request']['request_digest']: + raise Refused('owner operation mismatch') + origin=urlsplit(descriptor['backend_origin']) + if origin.scheme!='http' or origin.hostname!='127.0.0.1' or not origin.port or origin.username or origin.password or origin.path or origin.query or origin.fragment: + raise Refused('disposable loopback backend only') + self.host,self.port=origin.hostname,origin.port + process=admission.server_process + if (type(process) is not subprocess.Popen or process.poll() is not None + or type(process.args) is not list or '-dev' not in process.args + or '-dev-no-store-token' not in process.args + or '-dev-listen-address=127.0.0.1:'+str(self.port) not in process.args): + raise Refused('live owner-spawned disposable dev process required') + budget=descriptor['backend_budget'] + if set(budget)!={'ttl_seconds','uses'} or type(budget['ttl_seconds']) is not int or not 30<=budget['ttl_seconds']<=120 or type(budget['uses']) is not int or budget['uses']!=12: + raise Refused('fixed native task budget required') + self.ttl=budget['ttl_seconds']; self.uses=budget['uses']; self.used=0 + self.deadline=datetime.fromisoformat(descriptor['operation_deadline'].replace('Z','+00:00')) + if self.deadline.tzinfo is None: raise Refused('aware operation deadline required') + remaining=(self.deadline-datetime.now(timezone.utc)).total_seconds() + self.monotonic_deadline=time.monotonic()+remaining + self.policy_name='task-exec-'+binding.operation_id + calls=[compiler.before_call(plan,i) for i in range(6)] + paths=sorted({call.path for call in calls}) + self.hcl=''.join('path '+json.dumps(path)+' {\n capabilities = ["read", "create", "update"]\n}\n' for path in paths) + self.hcl+='path "auth/token/revoke-self" {\n capabilities = ["update"]\n}\n' + self.cleanup=tuple(Cleanup(item['kind'],item['call_digest']) for item in cleanup_specs(binding.operation_id)) + if self.meta['cleanup'] != cleanup_specs(binding.operation_id): raise Refused('cleanup plan mismatch') + self.task=None; self._mint_attempted=False; self._policy_attempted=False; self._ordinal=0; self._next_metadata=0 + self._cleanup_steps={item.kind:0 for item in self.cleanup} + + def _live(self): + if not self.journal._held or self.journal._admission!=self.binding or self.journal._consumed_admission!=self.binding: + raise Refused('same live consumed owner admission required') + + def _fresh(self): + self._live() + if min((self.deadline-datetime.now(timezone.utc)).total_seconds(),self.monotonic_deadline-time.monotonic())<=5: + raise Refused('business deadline exhausted') + self.compiler.before_call(self.plan,0) + + def _http(self, method, path, token, body=b''): + # Fixed caller-generated paths only; http.client has no proxy or redirect + # following. Never surface backend body/header or transport exception text. + if self.admission.server_process.poll() is not None: + raise Refused('disposable backend exited; outcome unconfirmed') + conn=http.client.HTTPConnection(self.host,self.port,timeout=2) + try: + conn.request(method,'/v1/'+path,body=body or None, + headers={'X-Vault-Token':token,'Content-Type':'application/json'}) + response=conn.getresponse(); raw=response.read(1024*1024+1) + if len(raw)>1024*1024: raise Refused('oversize native response') + data=json.loads(raw) if raw else {} + if type(data) is not dict: raise Refused('invalid native response') + return response.status,data + except Exception: + raise Refused('native transport or response unconfirmed') from None + finally: conn.close() + + def _business(self, method, path, token, payload=b'', task=False): + self._fresh() + if task: + if self.used>=self.uses-1: raise Refused('task cleanup use reserved') + self.used+=1 + evidence_payload = encoded({'token':'owner-retained-task-token'}) if path=='auth/token/lookup' else payload + digest=content_hash(encoded({'method':method,'path':path,'payload_sha256':'sha256:'+content_hash(evidence_payload),'credential_slot':'task' if task else 'owner'})) + ordinal=self._ordinal + self.journal.before_call(self.binding,ordinal,digest);self._ordinal+=1 + try: status,data=self._http(method,path,token,payload) + except Exception: + state=self.journal.observe(self.binding)['state'] + if state!='unknown': self.journal.transition(self.binding,state,'unknown') + raise + # Native responses may contain tokens/accessors; evidence is closed status + # metadata rather than raw bytes or a guessable secret-value fingerprint. + evidence=content_hash(encoded({'call_digest':digest,'http_status':status})) + self.journal.after_call(self.binding,ordinal,digest,evidence) + return status,data + + @staticmethod + def _ok(status,data,allowed): + if status not in allowed or data.get('errors'): + raise Refused('native request refused or unverified') + + def materialize_grant(self): + self._fresh() + if self.journal.observe(self.binding)['state']!='grant-issuing' or self._policy_attempted: + raise Refused('fresh grant issuing state required') + if min((self.deadline-datetime.now(timezone.utc)).total_seconds(),self.monotonic_deadline-time.monotonic())=len(spec['calls']) or spec['calls'][call_index]!={'method':method,'path':path}: + raise Refused('cleanup native method/target differs from commitment') + # Commit shape/slot only: private accessor/token bytes never enter hashes + # or durable evidence. Their exact identity is owner-retained custody. + digest=content_hash(encoded({'obligation_digest':obligation.call_digest,'step':step, + 'method':method,'path':path})) + self.journal.before_cleanup_call(self.binding,ordinal,obligation,step,digest) + self._cleanup_steps[kind]+=1 + status,data=self._http(method,path,token,body) + evidence=content_hash(encoded({'call_digest':digest,'http_status':status})) + self.journal.after_cleanup_call(self.binding,ordinal,obligation,step,digest,evidence) + return status,data + + def _revoke(self, kind, digest, session, issue_uncertain=False): + self._cleanup_admitted(kind,digest) + if session is None: + return CleanupResult(not issue_uncertain,content_hash(encoded({'kind':kind,'issue_uncertain':issue_uncertain}))) + status,data=self._cleanup_http(kind,'POST','auth/token/revoke-self',session.token) + self._ok(status,data,{200,204}) + status,data=self._cleanup_http(kind,'POST','auth/token/lookup-accessor',self.admission.owner_token,encoded({'accessor':session.accessor})) + confirmed=status==400 and data.get('errors') in (['invalid accessor'],['1 error occurred:\n\t* invalid accessor\n\n']) + return CleanupResult(confirmed,content_hash(encoded({'kind':kind,'revocation_confirmed':confirmed}))) + + def revoke_task_token(self, digest): + return self._revoke('task-token-revoke',digest,self.task,self._mint_attempted and self.task is None) + + def revoke_identity_token(self, digest): + return self._revoke('identity-token-revoke',digest,self.admission.identity_session) + + def delete_grant_policy(self, digest): + self._cleanup_admitted('grant-policy-delete',digest) + path='sys/policies/acl/'+self.policy_name + if self._policy_attempted: + status,data=self._cleanup_http('grant-policy-delete','DELETE',path,self.admission.owner_token);self._ok(status,data,{200,204}) + status,data=self._cleanup_http('grant-policy-delete','GET',path,self.admission.owner_token) + confirmed=status==404 and not data.get('data') + return CleanupResult(confirmed,content_hash(encoded({'kind':'grant-policy-delete','absence_confirmed':confirmed}))) diff --git a/scripts/task_execution_plan.py b/scripts/task_execution_plan.py index d8cb3e5..f3234fc 100644 --- a/scripts/task_execution_plan.py +++ b/scripts/task_execution_plan.py @@ -77,6 +77,26 @@ class Call: return content_hash(encoded({'ordinal':self.ordinal,'kind':self.kind,'method':self.method, 'path':self.path,'payload_sha256':self.payload_sha256})) + +def cleanup_specs(operation_id): + """Deterministic owner cleanup plan; native handles stay in owner memory.""" + try: need(str(UUID(operation_id)) == operation_id, 'operation_id_invalid') + except (ValueError, AttributeError, TypeError): raise Refused('operation_id_invalid') from None + policy_name = 'task-exec-' + operation_id + specs = [] + for kind, slot, calls in [ + ('task-token-revoke', 'task', [('POST','auth/token/revoke-self'), ('POST','auth/token/lookup-accessor')]), + ('identity-token-revoke', 'identity', [('POST','auth/token/revoke-self'), ('POST','auth/token/lookup-accessor')]), + ('grant-policy-delete', 'owner', [('DELETE','sys/policies/acl/'+policy_name), ('GET','sys/policies/acl/'+policy_name)]), + ]: + value = {'kind':kind, 'credential_slot':slot, 'operation_id':operation_id, + 'calls':[{'method':method,'path':path} for method,path in calls]} + if kind == 'task-token-revoke': value['confirm_not_created_if_no_issue_attempt'] = True + if kind == 'grant-policy-delete': value['skip_delete_if_no_create_attempt'] = True + value['call_digest'] = content_hash(encoded(value)) + specs.append(value) + return specs + class Compiler: def __init__(self, config: OwnerConfig): need(type(config) is OwnerConfig,'owner_configuration_required') @@ -232,14 +252,47 @@ class Compiler: return Plan(encoded({'schema':'platform-owner-typed-plan/v1','action':'execute-ccr','operation':'metadata-apply-ccr', 'request':request,'catalog_target':expected['catalog_target'], 'source_pins':dict(self.pins),'task_token_max_ttl':120,'task_token_max_uses':12, - 'cleanup_uses_reserved':1})) + 'cleanup_uses_reserved':1,'cleanup':cleanup_specs(request['operation_id'])})) + + def bind_execution(self, plan, descriptor, owner_descriptor): + """Bind exact engine descriptor to owner plan; no authorization is created. + + owner_descriptor must be installed independently by the owner, never copied + from the request. The engine must separately validate its closed schema and + full canonical request digest/claim/PDP/consume correspondence. + """ + need(type(plan) is Plan and type(descriptor) is dict and type(owner_descriptor) is bytes, + 'independent_owner_descriptor_required') + meta=plan.metadata() + need(self.compile(meta['request']).serialized == plan.serialized, 'immutable_plan_changed') + need(encoded(descriptor) == owner_descriptor, 'owner_descriptor_mismatch') + ccr,_,_=self._derive() + expected = {'contract_version':'SECRETS-EXEC-20261011-v1', + 'operation_id':meta['request']['operation_id'], + 'operation':'metadata-apply-ccr','execution_actor':'service:secrets-engine', + 'ccr_id':ccr['id'],'catalog_sha256':meta['request']['catalog_sha256'], + 'operations':meta['request']['operations']} + need(all(descriptor.get(k)==v for k,v in expected.items()),'execution_descriptor_plan_mismatch') + bound = dict(meta, execution_descriptor=descriptor, + execution_descriptor_sha256=tagged(owner_descriptor)) + # The full descriptor's independently admitted grant immutable digest must + # cover these cleanup commitments. No grant schema exists here yet, so this + # source binding cannot itself admit a grant/native production execution. + return Plan(encoded(bound)) def before_call(self,plan,ordinal): need(type(plan) is Plan and type(ordinal) is int and 0<=ordinal<6,'fixed_plan_ordinal_required') - try: meta=plan.metadata();fresh=self.compile(meta['request']) + try: + meta=plan.metadata();fresh=self.compile(meta['request']) + if 'execution_descriptor' in meta: + fresh=self.bind_execution(fresh,meta['execution_descriptor'],encoded(meta['execution_descriptor'])) except Refused: raise except BaseException: raise Refused('invalid_plan') from None need(fresh.serialized==plan.serialized,'immutable_plan_changed') # Immediately re-derive owner payload bytes, and reconfirm immutable snapshot after derivation. - _,_,calls=self._derive();need(self.compile(meta['request']).serialized==plan.serialized,'immutable_plan_changed') + _,_,calls=self._derive() + final=self.compile(meta['request']) + if 'execution_descriptor' in meta: + final=self.bind_execution(final,meta['execution_descriptor'],encoded(meta['execution_descriptor'])) + need(final.serialized==plan.serialized,'immutable_plan_changed') return calls[ordinal] diff --git a/tests/test_task_execution_cleanup_calls.py b/tests/test_task_execution_cleanup_calls.py new file mode 100644 index 0000000..814ab29 --- /dev/null +++ b/tests/test_task_execution_cleanup_calls.py @@ -0,0 +1,57 @@ +"""Real SQLite subcall durability for native cleanup/verification pairs.""" +import multiprocessing +import os +from pathlib import Path +import sys +import uuid + +import pytest +sys.path.insert(0,str(Path(__file__).resolve().parents[1]/'scripts')) +from task_execution_journal import Binding,Cleanup,Journal,Refused + + +def child_crash(directory,binding,obligation): + journal=Journal(Path(directory)) + with journal.locked(): + journal.reserve(binding,(obligation,)) + journal.transition(binding,'reserved','consume-confirmed') + journal.before_cleanup(binding,0,obligation) + journal.before_cleanup_call(binding,0,obligation,0,'c'*64) + os._exit(29) + + +def test_subcall_crash_persists_and_never_restores_admission(tmp_path): + binding=Binding('fixture-approval','sha256:'+'a'*64,str(uuid.uuid4()),'b'*64) + obligation=Cleanup('task-token-revoke','d'*64) + os.chmod(tmp_path,0o700) + context=multiprocessing.get_context('fork') + process=context.Process(target=child_crash,args=(str(tmp_path),binding,obligation)) + process.start();process.join(10) + if process.is_alive():process.terminate();process.join();pytest.fail('crash fixture exceeded bound') + assert process.exitcode==29 + journal=Journal(tmp_path) + try: + assert journal.observe(binding)['external_outcome_unknown'] + assert not journal.cleanup_status(binding)['confirmed'] + with journal.locked(): + with pytest.raises(Refused):journal.before_cleanup_call(binding,0,obligation,1,'c'*64) + finally:journal.close() + + +def test_pending_native_subcall_never_confirms_parent_and_survives_unconfirmed(tmp_path): + os.chmod(tmp_path,0o700);journal=Journal(tmp_path) + binding=Binding('fixture-approval','sha256:'+'a'*64,str(uuid.uuid4()),'b'*64) + obligation=Cleanup('task-token-revoke','d'*64) + try: + with journal.locked(): + journal.reserve(binding,(obligation,));journal.transition(binding,'reserved','consume-confirmed') + journal.before_cleanup(binding,0,obligation) + journal.before_cleanup_call(binding,0,obligation,0,'c'*64) + with pytest.raises(Refused):journal.after_cleanup(binding,0,obligation,confirmed=True,evidence_digest='e'*64) + with pytest.raises(Refused):journal.before_cleanup_call(binding,0,obligation,1,'c'*64) + for step in [False,0.0,'0',-1,2]: + with pytest.raises(Refused):journal.after_cleanup_call(binding,0,obligation,step,'c'*64,'e'*64) + journal.after_cleanup(binding,0,obligation,confirmed=False,evidence_digest='e'*64) + assert journal.observe(binding)['external_outcome_unknown'] + with pytest.raises(Refused):journal.after_cleanup_call(binding,0,obligation,0,'c'*64,'e'*64) + finally:journal.close() diff --git a/tests/test_task_execution_journal.py b/tests/test_task_execution_journal.py index b1a76fc..654c024 100644 --- a/tests/test_task_execution_journal.py +++ b/tests/test_task_execution_journal.py @@ -7,10 +7,8 @@ import unittest import uuid from pathlib import Path -spec = importlib.util.spec_from_file_location('task_execution_journal', Path(__file__).resolve().parents[1] / 'scripts/task_execution_journal.py') -module = importlib.util.module_from_spec(spec) -sys.modules[spec.name] = module -spec.loader.exec_module(module) +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'scripts')) +import task_execution_journal as module Binding, Journal, Refused = module.Binding, module.Journal, module.Refused Cleanup = module.Cleanup diff --git a/tests/test_task_execution_native_fixture.py b/tests/test_task_execution_native_fixture.py new file mode 100644 index 0000000..382b236 --- /dev/null +++ b/tests/test_task_execution_native_fixture.py @@ -0,0 +1,232 @@ +"""Actual disposable OpenBao effects; all identities/approvals are synthetic. + +These tests do not contact estate OpenBao or claim production owner acceptance. +""" +import hashlib +import http.client +import json +import shutil +import socket +import subprocess +import time +from datetime import datetime,timedelta,timezone +from pathlib import Path +import sys + +import pytest + +sys.path.insert(0,str(Path(__file__).resolve().parents[1]/'scripts')) +from test_task_execution_plan import compiler,request,m +from task_execution_journal import Binding,Cleanup,Journal,Refused +from task_execution_cleanup import run_cleanup +from task_execution_native_fixture import FixtureAdmission,NativeFixtureAdapter,PrivateSession + + +@pytest.fixture +def bao_dev(): + executable=shutil.which('bao') + if not executable: pytest.skip('OpenBao executable unavailable') + with socket.socket() as sock: + sock.bind(('127.0.0.1',0));port=sock.getsockname()[1] + root='synthetic-disposable-fixture-root' + process=subprocess.Popen([executable,'server','-dev','-dev-no-store-token', + '-dev-root-token-id='+root,'-dev-listen-address=127.0.0.1:'+str(port)], + stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL) + def call(method,path,body=None,token=root): + conn=http.client.HTTPConnection('127.0.0.1',port,timeout=2) + try: + conn.request(method,'/v1/'+path,body=json.dumps(body) if body is not None else None, + headers={'X-Vault-Token':token,'Content-Type':'application/json'}) + response=conn.getresponse();raw=response.read() + return response.status,json.loads(raw) if raw else {} + finally: conn.close() + for _ in range(100): + if process.poll() is not None: pytest.fail('disposable backend exited') + try: + if call('GET','sys/health')[0]==200: break + except OSError: pass + time.sleep(0.05) + else: + process.kill();process.wait();pytest.fail('disposable backend failed to start') + try: + yield 'http://127.0.0.1:'+str(port),root,call,process + finally: + process.terminate() + try: process.wait(timeout=5) + except subprocess.TimeoutExpired: process.kill();process.wait() + + +@pytest.fixture +def owner(compiler,bao_dev,tmp_path): + origin,root,call,process=bao_dev + assert call('POST','sys/auth/kubernetes',{'type':'kubernetes'})[0] in {200,204} + hcl='path "auth/token/revoke-self" { capabilities = ["update"] }' + assert call('PUT','sys/policies/acl/fixture-self-only',{'policy':hcl})[0] in {200,204} + status,result=call('POST','auth/token/create',{'policies':['fixture-self-only'],'no_default_policy':True,'renewable':False,'ttl':'300s','num_uses':8}) + assert status==200 + identity=PrivateSession(result['auth']['client_token'],result['auth']['accessor']) + req=request(compiler);plan=compiler.compile(req) + descriptor={'contract_version':'SECRETS-EXEC-20261011-v1','operation_id':req['operation_id'], + 'operation':'metadata-apply-ccr','execution_actor':'service:secrets-engine', + 'ccr_id':compiler._derive()[0]['id'],'ccr_immutable_sha256':req['ccr_sha256'], + 'execution_grant_ccr_id':'FIXTURE-GRANT-NOT-PRODUCTION','grant_immutable_sha256':'sha256:'+'f'*64, + 'catalog_sha256':req['catalog_sha256'], + 'source_bundle':{role:{'path':'/fixture/'+role,'sha256':'sha256:'+'d'*64} for role in ['engine_dispatcher','platform_mediator','typed_ccr','policy_auth']}, + 'backend_origin':origin,'cluster_uid':'disposable-fixture-only', + 'operations':req['operations'],'operation_deadline':(datetime.now(timezone.utc)+timedelta(seconds=240)).isoformat(), + 'backend_budget':{'ttl_seconds':120,'uses':12},'expected_before':'sha256:'+'0'*64,'expected_after':'sha256:'+'1'*64} + # Exercise the actual consumer closed descriptor parser as well; source pins, + # grant/expected-state digests remain deliberately synthetic owner fixtures. + from secrets_engine.execution_contract import ExecutionContract + descriptor=ExecutionContract.parse(descriptor).as_dict() + installed=m.encoded(descriptor) + plan=compiler.bind_execution(plan,descriptor,installed) + binding=Binding('SYNTHETIC-APPROVAL-NOT-PRODUCTION',req['request_digest'],req['operation_id'],plan.content_digest) + state=tmp_path/'private-journal';state.mkdir(mode=0o700);journal=Journal(state) + admission=FixtureAdmission(installed,root,identity,process) + native=NativeFixtureAdapter(compiler,plan,journal,binding,admission) + try: yield native,journal,binding,call,compiler,descriptor + finally: journal.close() + + +def admit(native,journal,binding): + # Synthetic postconsume transition is fixture setup, not served approval proof. + journal.reserve(binding,native.cleanup) + journal.transition(binding,'reserved','consume-confirmed') + journal.transition(binding,'consume-confirmed','authenticating') + journal.transition(binding,'authenticating','grant-issuing') + + +def test_native_fixed_grant_metadata_denials_and_confirmed_cleanup(owner): + native,journal,binding,call,compiler,descriptor=owner + with journal.locked(): + admit(native,journal,binding) + info=native.materialize_grant() + assert info['issued'] and info['ttl_ceiling_seconds']==120 and info['use_ceiling']==12 + journal.transition(binding,'grant-issued','applying') + baseline=native.metadata_call(0);assert baseline[0]==404 + assert native.metadata_call(1)[0] in {200,204} + policy=native.metadata_call(2);assert policy[0]==200 + assert policy[1]['data']['policy'].rstrip()==json.loads(compiler.before_call(native.plan,1).payload)['policy'].rstrip() + assert native.metadata_call(3)[0]==404 + assert native.metadata_call(4)[0] in {200,204} + role=native.metadata_call(5);assert role[0]==200 + # Actual native role metadata correspondence for fixture's known K8s body. + expected=json.loads(compiler.before_call(native.plan,4).payload) + assert role[1]['data']['bound_service_account_names']==expected['bound_service_account_names'] + assert role[1]['data']['bound_service_account_namespaces']==expected['bound_service_account_namespaces'] + assert role[1]['data']['token_policies']==expected['policies'].split(',') + # Root probes native capabilities without spending the limited task token. + for path in ['sys/policies/acl/sibling','auth/kubernetes/role/sibling','secret/data/sibling','auth/token/create','sys/mounts/secret','auth/approle/role/sibling/secret-id']: + status,data=call('POST','sys/capabilities',{'token':native.task.token,'paths':[path]}) + assert status==200 and data['capabilities']==['deny'] + task_accessor=native.task.accessor;identity_accessor=native.admission.identity_session.accessor + result=run_cleanup(journal,binding,native) + # Assert native absence before summary so fixture failures identify protocol semantics. + for accessor in [task_accessor,identity_accessor]: + status,data=call('POST','auth/token/lookup-accessor',{'accessor':accessor}) + assert status==400 and data.get('errors') in (['invalid accessor'],['1 error occurred:\n\t* invalid accessor\n\n']) + assert result['confirmed'] + assert call('GET','sys/policies/acl/'+native.policy_name)[0]==404 + assert journal.observe(binding)['state']=='applying' # native success is not full owner verification + with pytest.raises(Refused): native.metadata_call(6) + with pytest.raises(Refused): run_cleanup(journal,binding,native) + + +def test_no_consume_no_native_grant_and_changed_digest_no_cleanup(owner): + native,journal,binding,call,*_=owner + with journal.locked(): + journal.reserve(binding,native.cleanup) + with pytest.raises(Refused):native.materialize_grant() + assert call('GET','sys/policies/acl/'+native.policy_name)[0]==404 + with pytest.raises(Refused):native.delete_grant_policy('0'*64) + + +def test_native_cleanup_after_deadline_and_pending_business_uncertainty(owner): + native,journal,binding,call,*_=owner + with journal.locked(): + admit(native,journal,binding);native.materialize_grant() + journal.transition(binding,'grant-issued','applying') + journal.before_call(binding,native._ordinal,'a'*64) + journal.transition(binding,'applying','unknown') + native.monotonic_deadline=time.monotonic()-1 + with pytest.raises(Refused):native.metadata_call(0) + assert run_cleanup(journal,binding,native)['confirmed'] + assert journal.observe(binding)['state']=='unknown' + assert journal.observe(binding)['external_outcome_unknown'] + + +def test_whole_descriptor_and_cleanup_commitment_are_bound(owner): + native,journal,binding,call,compiler,descriptor=owner + base=compiler.compile(native.meta['request']) + changed=dict(descriptor,cluster_uid='different-cluster') + with pytest.raises(m.Refused):compiler.bind_execution(base,changed,native.admission.descriptor) + data=native.plan.metadata();data['cleanup'][0]['call_digest']='9'*64 + with pytest.raises(m.Refused):compiler.before_call(m.Plan(m.encoded(data)),0) + with pytest.raises(Refused):NativeFixtureAdapter(compiler,m.Plan(m.encoded(data)),journal,binding,native.admission) + + +def test_source_drift_after_admission_prevents_mint_but_allows_native_cleanup(owner): + native,journal,binding,call,compiler,*_=owner + with journal.locked(): + admit(native,journal,binding) + compiler.ccr.write_bytes(compiler.ccr.read_bytes()+b'\n# changed owner source\n') + with pytest.raises(m.Refused):native.materialize_grant() + assert call('GET','sys/policies/acl/'+native.policy_name)[0]==404 + assert run_cleanup(journal,binding,native)['confirmed'] + + +def test_actual_native_write_then_lost_response_stays_unknown_and_cleans(owner): + native,journal,binding,call,compiler,*_=owner + with journal.locked(): + admit(native,journal,binding);native.materialize_grant() + journal.transition(binding,'grant-issued','applying') + assert native.metadata_call(0)[0]==404 + original=native._http + target=compiler.before_call(native.plan,1).path + def lost_response(method,path,token,body=b''): + result=original(method,path,token,body) + if method=='PUT' and path==target: + raise Refused('synthetic lost response after real native write') + return result + native._http=lost_response + with pytest.raises(Refused):native.metadata_call(1) + assert call('GET',target)[0]==200 # actual side effect despite missing confirmation + assert journal.observe(binding)['state']=='unknown' + assert journal.observe(binding)['external_outcome_unknown'] + assert run_cleanup(journal,binding,native)['confirmed'] + assert journal.observe(binding)['state']=='unknown' + with pytest.raises(Refused):native.metadata_call(1) + assert call('GET',target)[0]==200 # cleanup never rolls back/reapplies business state + + +def test_actual_native_policy_retirement_does_not_prove_failed_token_revoke(owner): + native,journal,binding,call,*_=owner + with journal.locked(): + admit(native,journal,binding);native.materialize_grant() + original=native._http + def refuse_task_revoke(method,path,token,body=b''): + if method=='POST' and path=='auth/token/revoke-self' and token==native.task.token: + return 503,{'errors':['synthetic cleanup transport failure']} + return original(method,path,token,body) + native._http=refuse_task_revoke + result=run_cleanup(journal,binding,native) + assert not result['confirmed'] + assert [item['outcome'] for item in result['obligations']]==['unconfirmed','confirmed','confirmed'] + assert call('GET','sys/policies/acl/'+native.policy_name)[0]==404 + # Actual owner lookup demonstrates token still exists despite deleted ACL. + status,data=call('POST','auth/token/lookup-accessor',{'accessor':native.task.accessor}) + assert status==200 and data['data']['policies']==[native.policy_name] + with pytest.raises(Refused):run_cleanup(journal,binding,native) + # Explicit synthetic owner recovery, not adapter retry/resumption. + assert call('POST','auth/token/revoke-accessor',{'accessor':native.task.accessor})[0] in {200,204} + + +def test_nonloopback_estate_origin_is_refused_before_native_transport(owner): + native,journal,binding,call,compiler,descriptor=owner + changed=dict(descriptor,backend_origin='https://estate.invalid:8200') + installed=m.encoded(changed) + plan=compiler.bind_execution(compiler.compile(native.meta['request']),changed,installed) + changed_binding=Binding(binding.approval_id,binding.request_digest,binding.operation_id,plan.content_digest) + admission=FixtureAdmission(installed,native.admission.owner_token,native.admission.identity_session,native.admission.server_process) + with pytest.raises(Refused):NativeFixtureAdapter(compiler,plan,journal,changed_binding,admission) diff --git a/tests/test_task_execution_plan.py b/tests/test_task_execution_plan.py index 95e4e62..bb04079 100644 --- a/tests/test_task_execution_plan.py +++ b/tests/test_task_execution_plan.py @@ -2,8 +2,8 @@ import importlib.util,json,hashlib,shutil,sys,uuid,os from pathlib import Path import pytest,yaml SOURCE=Path(__file__).resolve().parents[1]/'scripts/task_execution_plan.py' -spec=importlib.util.spec_from_file_location('task_execution_plan',SOURCE) -m=importlib.util.module_from_spec(spec);sys.modules[spec.name]=m;spec.loader.exec_module(m) +sys.path.insert(0,str(SOURCE.parent)) +import task_execution_plan as m PLATFORM=Path(os.environ.get('PLATFORM_PLAN_TEST_SOURCE_ROOT',str(SOURCE.parents[1]))) SECRETS=Path(os.environ.get('SECRETS_ENGINE_SOURCE_ROOT',str(PLATFORM.parent/'secrets-engine'))) @pytest.fixture diff --git a/workplans/RPF-WP-0035-credential-lane-implementation.md b/workplans/RPF-WP-0035-credential-lane-implementation.md index 269ad08..0fd7ccc 100644 --- a/workplans/RPF-WP-0035-credential-lane-implementation.md +++ b/workplans/RPF-WP-0035-credential-lane-implementation.md @@ -1239,3 +1239,15 @@ No native adapter or production mediator exists; full grant/request/cleanup bind authenticated consume/identity, ephemeral token issuance, deadline/use checks, native effect/revoke/absence, audit and protected recovery remain prerequisites. T12 remains progress. No CCR38 default/entitlement or live grant was changed. + +2026-10-16 T12 second source package: Compiler Plans now bind deterministic cleanup +commitments and can bind the complete independently installed engine execution +descriptor. A fixed disposable-loopback native adapter actually materializes exact +per-operation metadata ACL/task token, executes six source-derived metadata calls, +and verifies self-revocation/accessor invalidation and policy retirement. Each +cleanup request has its own durable intent. Native fault tests cover lost response +after real write and policy deletion while task revoke fails. All initiating +identity/approval/consume/grant/state/source admission in this fixture is synthetic; +no production authority or complete native owner acceptance is claimed. Exact +remaining owner schema/admission and mediator work: docs/task-execution-native-fixture.md. +T12 stays progress; no live grant, credential/default change or owner sync performed.