diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index bf8e3d8..3a03053 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -371,7 +371,7 @@ | task | NK-WP-0057-T03 | todo | — | workplans/NK-WP-0057-decision-routing-by-responsibility.md | | task | NK-WP-0057-T04 | todo | — | workplans/NK-WP-0057-decision-routing-by-responsibility.md | | task | NK-WP-0058-T01 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md | -| task | NK-WP-0058-T02 | progress | — | workplans/NK-WP-0058-bounded-execution-operability.md | +| task | NK-WP-0058-T02 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md | | task | NK-WP-0058-T03 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md | | task | NK-WP-0058-T04 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md | | task | NK-WP-0058-T05 | todo | — | workplans/NK-WP-0058-bounded-execution-operability.md | diff --git a/docs/bounded-execution-contract-table.md b/docs/bounded-execution-contract-table.md index ec4e75d..96cf6b0 100644 --- a/docs/bounded-execution-contract-table.md +++ b/docs/bounded-execution-contract-table.md @@ -509,3 +509,25 @@ The agent's default context can read NetworkPolicies in `sso`, `approval-engine` `15df0e3`, `.forgejo/workflows/packaging.yaml`). The run fetches its own tarball with the run's automatic repository-scoped token, clones net-kingdom and railiance-platform anonymously at `main`, and fails on any skip. That covers E8 (ceiling), E9 (egress) and R2 (config, exit codes), on push and nightly. + +### Coverage by row (2026-11-06) + +| Row | Contract test | Runs in | +| --- | --- | --- | +| E1 | `tests/test_executor_lanes.py`: `executor-lanes.py contract()` compares role, KV path, fields, auth shape and the policy file text with CCR-2026-0058/0059 | net-kingdom contract CI | +| E2, E3 | `tools/contract-tests/tests/test_exercises.py::test_t19`: KeyCape binary from the pinned image, approval-engine source | net-kingdom contract CI | +| E4 | `bounded-executor/tests/test_approval_request.py` against informed-decision's intake code | net-kingdom contract CI | +| E5, C6, C7, O1 | `tools/bounded-execution-readiness/tests/test_ready_owner_sources.py`, with both 2026-11-04 states replayed from history | net-kingdom contract CI | +| E6 | `test_exercises.py::test_t15`: audit-core ingestion and sender scope | net-kingdom contract CI | +| E7 | `tools/contract-tests/tests/test_lldap_tool.py`: one LLDAP digest for the executor tool and the server | net-kingdom contract CI | +| E8, R2 | rapp `tests/test_packaging.py`: ceiling, config and exit codes against net-kingdom | rapp packaging CI | +| E9 | `tools/contract-tests/tests/test_ingress.py`: each owner's ingress admits the executor; rapp tests the egress | both | +| R1 | the executor's `source` preflight check in the pinned image (`make preflight`) | before each window | +| C1 | `tests/test_executor_lanes.py` runs railiance-platform's real helper modules | net-kingdom contract CI | +| C2, C3 | the readiness check: `case_type_bytes_match_broker`, `case_materialized` | net-kingdom contract CI | +| C4, C5 | `tools/contract-tests/tests/test_custody_names.py` | net-kingdom contract CI | +| O2–O4, F1, F2 | `tools/bounded-execution-run/tests/test_bx.py` (lane child silent, window always closed); conventions in T07 | net-kingdom contract CI | + +Not covered from source: OpenBao's ingress (rapp-openbao is private) and live drift from source, such as the +approval-engine policy that was live but not in its file. The readiness preflight Job catches unreachable endpoints +before a window, and T07 and O1 cover the drift. diff --git a/tools/contract-tests/tests/test_custody_names.py b/tools/contract-tests/tests/test_custody_names.py new file mode 100644 index 0000000..fe1dc9c --- /dev/null +++ b/tools/contract-tests/tests/test_custody_names.py @@ -0,0 +1,86 @@ +"""Rows C4 and C5: the custody names agree across net-kingdom, railiance-platform and audit-core (NK-WP-0058-T02). + +C4, KeyCape client secret: railiance-platform's ESO template and rendered manifest, CCR-2026-0061's KV path, and +net-kingdom's keycape-client.py naming and the KeyCape Deployment's env binding. +C5, audit-core sender bearer: CCR-2026-0063's KV path, fields and delivery, the platform's ExternalSecret, and the +names audit-core's seeder uses (read as text; the seeder is not imported). +The rapp Job's mount of sso/bounded-executor-tokens is private and checked in the rapp's own CI. +""" +import importlib.util +import re +from pathlib import Path + +import pytest +import yaml + +REPO = Path(__file__).resolve().parents[3] +PLATFORM = REPO.parent / 'railiance-platform' +AUDIT_CORE = REPO.parent / 'audit-core' +ESO = PLATFORM / 'manifests/nk-bind-executor-eso.yaml' +CCRS = PLATFORM / 'credential-change-requests' + + +def need(*paths): + for path in paths: + if not path.exists(): + pytest.skip(f'owner checkout missing: {path.parent.name}/{path.name}') + + +def ccr(prefix): + return yaml.safe_load(next(CCRS.glob(prefix + '-*.yaml')).read_text()) + + +def external_secrets(): + docs = [d for d in yaml.safe_load_all(ESO.read_text()) if d] + stores = {d['metadata']['name']: d for d in docs if d['kind'] == 'ClusterSecretStore'} + return {d['spec']['target']['name']: (d, stores[d['spec']['secretStoreRef']['name']]) + for d in docs if d['kind'] == 'ExternalSecret'} + + +def kv_path(store, remote_key): + return store['spec']['provider']['vault']['path'].strip('/') + '/' + remote_key + + +def keycape_client(): + spec = importlib.util.spec_from_file_location('kc_client_names', REPO / 'sso-mfa/k8s/keycape/keycape-client.py') + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def test_c4_keycape_client_secret_names(): + need(ESO, PLATFORM / 'manifests/templates/keycape-issuer-eso.yaml') + kc = keycape_client() + case = yaml.safe_load((REPO / 'security-cases/nk-bind-executor-approval.yaml').read_text())['spec'] + cid = case['parameters']['client_id'] + secret_name, env = kc.env_secret_name(cid), kc.env_name(cid) + template = (PLATFORM / 'manifests/templates/keycape-issuer-eso.yaml').read_text() + assert 'name: keycape-{client_id}-client' in template, 'template naming convention changed' + assert kc.env_secret_name('{client_id}') == 'keycape-{client_id}-client' + es, store = external_secrets()[secret_name] + (item,) = es['spec']['data'] + custody = ccr('CCR-2026-0061')['openbao'] + assert item['secretKey'] == 'client-secret' + assert kv_path(store, item['remoteRef']['key']) == custody['kv_path'] == f'platform/workloads/keycape/{cid}' + assert [item['remoteRef']['property']] == custody['fields'] == ['CLIENT_SECRET'] + deployment = next(d for d in yaml.safe_load_all((REPO / 'sso-mfa/k8s/keycape/deployment.yaml').read_text()) + if d and d['kind'] == 'Deployment' and d['metadata']['name'] == 'keycape') + bound = {e['name']: e.get('valueFrom', {}).get('secretKeyRef') for c in deployment['spec']['template']['spec'] + ['containers'] for e in c.get('env', [])} + assert bound.get(env) == {'name': secret_name, 'key': 'client-secret'}, 'KeyCape env binding differs' + + +def test_c5_audit_sender_custody_names(): + seeder = AUDIT_CORE / 'scripts/seed-nk-bounded-executor-sender.py' + need(ESO, seeder) + constants = dict(re.findall(r'^([A-Z_]+) = "([^"]*)"', seeder.read_text(), re.M)) + custody = ccr('CCR-2026-0063') + es, store = external_secrets()[constants['SSO_SECRET']] + (item,) = es['spec']['data'] + assert es['metadata']['namespace'] == custody['delivery']['namespace'] == 'sso' + assert custody['openbao']['kv_path'] == constants['CUSTODY_KV'] == kv_path(store, item['remoteRef']['key']) + assert item['remoteRef']['property'] in custody['openbao']['fields'] == ['AUDIT_TOKEN', 'CUSTODY_REQUEST'] + assert item['secretKey'] == constants['SSO_KEY'] == 'audit-core' + target = custody['delivery']['target'] + assert f"sso/{constants['SSO_SECRET']}" in target and constants['SSO_KEY'] in target + assert constants['CCR_ID'] == custody['id'] diff --git a/tools/contract-tests/tests/test_ingress.py b/tools/contract-tests/tests/test_ingress.py new file mode 100644 index 0000000..e34bcb4 --- /dev/null +++ b/tools/contract-tests/tests/test_ingress.py @@ -0,0 +1,103 @@ +"""Row E9: every owner's ingress admits the executor pod (NK-WP-0058-T02), read from the owners' source. + +Kubernetes semantics, for the subset these policies use: a pod that no Ingress policy selects is open; once any +policy selects it, traffic must match an ingress rule of one of them (peer selectors and port). matchExpressions and +ipBlock peers are not used by these policies; a policy that uses them fails the test instead of being guessed. +Not covered here: OpenBao (rapp-openbao is private) and the executor's egress (the rapp's own CI, row E9 there). +""" +from pathlib import Path + +import pytest +import yaml + +REPO = Path(__file__).resolve().parents[3] +SIBLINGS = REPO.parent +EXECUTOR = {'namespace': 'sso', 'labels': {'app.kubernetes.io/name': 'bounded-executor'}} + +SSO = 'sso-mfa/k8s/network-policies/netpol-sso.yaml' +# (owner, policy files relative to the owner, namespace, target pod labels, port), from the executor's config +TARGETS = [ + ('approval-engine', ['deploy/networkpolicies.yaml'], 'approval-engine', + {'app.kubernetes.io/name': 'approval-engine'}, 8080), + ('audit-core', ['deploy/networkpolicies.yaml'], 'audit-core', {'app.kubernetes.io/name': 'audit-core'}, 8080), + ('informed-decision', ['deploy/lifecycle/*.yaml'], 'informed-decision', + {'app.kubernetes.io/name': 'informed-decision', 'app.kubernetes.io/component': 'review'}, 8081), +] + [('net-kingdom', [SSO], 'sso', {'app.kubernetes.io/name': name}, port) + for name, port in (('lldap', 17170), ('keycape', 8080), ('authelia', 9091))] + + +def selects(selector, labels): + if set(selector) - {'matchLabels'}: + raise AssertionError(f'unsupported selector {sorted(selector)}: extend the evaluator, do not guess') + return all(labels.get(k) == v for k, v in (selector.get('matchLabels') or {}).items()) + + +def peer_admits(peer, policy_ns, source): + if set(peer) - {'namespaceSelector', 'podSelector'}: + raise AssertionError(f'unsupported peer {sorted(peer)}') + if 'namespaceSelector' in peer: + ns_ok = selects(peer['namespaceSelector'], {'kubernetes.io/metadata.name': source['namespace']}) + else: + ns_ok = source['namespace'] == policy_ns + return ns_ok and selects(peer.get('podSelector') or {}, source['labels']) + + +def port_admits(ports, port): + return not ports or any(p.get('port') == port and p.get('protocol', 'TCP') == 'TCP' for p in ports) + + +def admitted(policies, namespace, labels, port, source=EXECUTOR): + applying = [p for p in policies if (p['metadata'].get('namespace') or namespace) == namespace + and 'Ingress' in p['spec'].get('policyTypes', ['Ingress']) + and selects(p['spec'].get('podSelector') or {}, labels)] + if not applying: + return 'open' + for policy in applying: + for rule in policy['spec'].get('ingress') or []: + peers = rule.get('from') + if (not peers or any(peer_admits(peer, namespace, source) for peer in peers)) \ + and port_admits(rule.get('ports'), port): + return 'admitted:' + policy['metadata']['name'] + return 'denied' + + +def policies_of(owner, patterns): + root = REPO if owner == 'net-kingdom' else SIBLINGS / owner + if not root.is_dir(): + pytest.skip(f'owner checkout missing: {owner}') + out = [] + for pattern in patterns: + for path in sorted(root.glob(pattern)): + out += [d for d in yaml.safe_load_all(path.read_text()) if d and d.get('kind') == 'NetworkPolicy'] + return out + + +@pytest.mark.parametrize('owner, files, namespace, labels, port', TARGETS, + ids=[f"{t[0]}:{t[3]['app.kubernetes.io/name']}:{t[4]}" for t in TARGETS]) +def test_owner_ingress_admits_the_executor(owner, files, namespace, labels, port): + result = admitted(policies_of(owner, files), namespace, labels, port) + assert result != 'denied', f'{owner} ingress does not admit sso/bounded-executor on {port}' + + +def policy(name, namespace, target, peers=None, ports=None): + rule = {**({'from': peers} if peers is not None else {}), **({'ports': ports} if ports else {})} + return {'kind': 'NetworkPolicy', 'metadata': {'name': name, 'namespace': namespace}, + 'spec': {'podSelector': {'matchLabels': target}, 'policyTypes': ['Ingress'], 'ingress': [rule]}} + + +def test_the_evaluator_on_synthetic_policies(): + target, ns = {'app.kubernetes.io/name': 'svc'}, 'svc' + executor_peer = {'namespaceSelector': {'matchLabels': {'kubernetes.io/metadata.name': 'sso'}}, + 'podSelector': {'matchLabels': {'app.kubernetes.io/name': 'bounded-executor'}}} + other_peer = {'namespaceSelector': {'matchLabels': {'kubernetes.io/metadata.name': 'other'}}} + assert admitted([], ns, target, 8080) == 'open' + assert admitted([policy('a', ns, target, [executor_peer], [{'port': 8080}])], ns, target, 8080) == 'admitted:a' + assert admitted([policy('a', ns, target, [executor_peer], [{'port': 9090}])], ns, target, 8080) == 'denied' + assert admitted([policy('a', ns, target, [other_peer])], ns, target, 8080) == 'denied' + assert admitted([policy('a', ns, {'app.kubernetes.io/name': 'else'}, [other_peer])], ns, target, 8080) == 'open' + deny_all = {'kind': 'NetworkPolicy', 'metadata': {'name': 'deny', 'namespace': ns}, + 'spec': {'podSelector': {}, 'policyTypes': ['Ingress']}} + assert admitted([deny_all], ns, target, 8080) == 'denied' + same_ns_peer = {'podSelector': {'matchLabels': {'app.kubernetes.io/name': 'bounded-executor'}}} + assert admitted([deny_all, policy('b', 'sso', target, [same_ns_peer])], 'sso', target, 8080) == 'admitted:b' + assert admitted([deny_all, policy('b', ns, target, [same_ns_peer])], ns, target, 8080) == 'denied' diff --git a/tools/contract-tests/tests/test_lldap_tool.py b/tools/contract-tests/tests/test_lldap_tool.py new file mode 100644 index 0000000..4fa435b --- /dev/null +++ b/tools/contract-tests/tests/test_lldap_tool.py @@ -0,0 +1,21 @@ +"""Row E7: the executor's lldap_set_password comes from the LLDAP image the sso server runs (NK-WP-0058-T02). + +The executor image copies /app/lldap_set_password from a digest-pinned LLDAP image (bounded-executor/Containerfile). +The tool speaks LLDAP's OPAQUE protocol, so it must be the server's own version: the two digests must be equal. +""" +import re +from pathlib import Path + +import yaml + +REPO = Path(__file__).resolve().parents[3] +DIGEST = r'docker\.io/lldap/lldap@(sha256:[0-9a-f]{64})' + + +def test_executor_tool_matches_the_lldap_server(): + tool = re.findall(DIGEST, (REPO / 'bounded-executor/Containerfile').read_text()) + deployment = next(d for d in yaml.safe_load_all((REPO / 'sso-mfa/k8s/lldap/deployment.yaml').read_text()) + if d and d.get('kind') == 'Deployment') + server = [m for c in deployment['spec']['template']['spec']['containers'] for m in re.findall(DIGEST, c['image'])] + assert len(tool) == 1 and len(server) == 1, (tool, server) + assert tool == server, 'the executor image would copy lldap_set_password from another LLDAP version' diff --git a/workplans/NK-WP-0058-bounded-execution-operability.md b/workplans/NK-WP-0058-bounded-execution-operability.md index eeb7097..7811f25 100644 --- a/workplans/NK-WP-0058-bounded-execution-operability.md +++ b/workplans/NK-WP-0058-bounded-execution-operability.md @@ -246,7 +246,7 @@ controller-per-object table, a founder-step table and the status table for T05. ```task id: NK-WP-0058-T02 -status: progress +status: done priority: high assignee: claude-code depends_on: [NK-WP-0058-T01, NK-WP-0058-T09, NK-WP-0058-T10] @@ -275,6 +275,44 @@ role name, a response code or a memo field. has a test that would have caught it. Code in net-kingdom that duplicated a contract by hand is removed. For duplicates in other repos, a removal request is sent to their owners and recorded. +**Result, 2026-11-06.** + +1. **Owner source in CI.** + - `contracts/owner-pins.json` pins six public owners. + - `tools/contract-tests/fetch_owners.py` makes anonymous partial clones of them: pinned on push, `main` + nightly. + - Workflow `bounded-execution-contracts.yaml` runs `tools/contract-tests/run.sh`, the one suite list (also + `make contract-test`). Any skip fails it. + - pytest and the crypto chain are pinned by hashes checked against PyPI. + - The private rapp tests itself in its own CI (rapp `15df0e3`), using the run's automatic token. +2. **Exercises folded in.** T15 runs against audit-core's ingestion. T19 runs against approval-engine's source, + with the KeyCape binary extracted from the pinned image by verified digests (`oci_extract.py`, no docker). +3. **Per-row tests.** + - ingress for E9, with a small NetworkPolicy evaluator and synthetic cases; + - custody names for C4 and C5; + - the LLDAP tool and server digest for E7; + - readiness against real owner source, replaying both 2026-11-04 states from informed-decision history (E5, + C6, C7). + +The coverage by row is in the contract table. 95 tests pass in a CI-like layout. + +- **The 2026-11-04/05 failures and their tests:** + - the approver routing: the readiness replay; + - `setup:URLError`: the ingress test, plus the preflight Job before a window; + - the window left open overnight: `bx` closes in `finally`, tested; + - the opaque `LaneError`: named causes, tested. + + The ESO apply and admission-guard and the kubectl-wait failures are T07 conventions. The warden token variable is + the O2 owner request. +- **Hand duplicates:** `pilot-run.sh` and the hard-coded informed-decision test path are removed. `LANES` keeps + copies of the CCR values, but `contract()` checks them against the CCRs on every run and in CI. The stale Secret + name in railiance-platform was sent as a request (`f6198285`). +- **Owner agreement:** questions were sent to six owners (`4892658f`, `3a546559`, `aa44a2b3`, `7cf8d61f`, + `a2105e34`, `ee1e5bb5`). Until they answer, the pinned-plus-nightly mode stands. +- **Not covered:** OpenBao's ingress (rapp-openbao is private), and live drift from source, which the preflight + Job and T07 address. +- **Unverified here:** the first Forgejo runs of both workflows. The Actions API needs a login. + ### Readiness check before any window opens ```task