NK-WP-0058-T02 done: per-row contract tests (E7, E9, C4, C5) and the coverage table
Ingress: each owner's NetworkPolicies admit the executor pod (small evaluator, synthetic cases). Custody names: KeyCape client secret (C4) and audit sender (C5) agree across net-kingdom, railiance-platform and audit-core. E7: one LLDAP digest for the executor tool and the server. 95 tests pass in a CI-like layout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 943911@bnt-lap001 Assistant-Session: fc322940-1c8e-4bed-ab39-86cec99f23a0
This commit is contained in:
parent
79f14ce50a
commit
80d280a340
6 changed files with 272 additions and 2 deletions
|
|
@ -371,7 +371,7 @@
|
|||
| task | NK-WP-0057-T03 | todo | — | workplans/NK-WP-0057-decision-routing-by-responsibility.md |
|
||||
| task | NK-WP-0057-T04 | todo | — | workplans/NK-WP-0057-decision-routing-by-responsibility.md |
|
||||
| task | NK-WP-0058-T01 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md |
|
||||
| task | NK-WP-0058-T02 | progress | — | workplans/NK-WP-0058-bounded-execution-operability.md |
|
||||
| task | NK-WP-0058-T02 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md |
|
||||
| task | NK-WP-0058-T03 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md |
|
||||
| task | NK-WP-0058-T04 | done | — | workplans/NK-WP-0058-bounded-execution-operability.md |
|
||||
| task | NK-WP-0058-T05 | todo | — | workplans/NK-WP-0058-bounded-execution-operability.md |
|
||||
|
|
|
|||
|
|
@ -509,3 +509,25 @@ The agent's default context can read NetworkPolicies in `sso`, `approval-engine`
|
|||
`15df0e3`, `.forgejo/workflows/packaging.yaml`). The run fetches its own tarball with the run's automatic
|
||||
repository-scoped token, clones net-kingdom and railiance-platform anonymously at `main`, and fails on any skip.
|
||||
That covers E8 (ceiling), E9 (egress) and R2 (config, exit codes), on push and nightly.
|
||||
|
||||
### Coverage by row (2026-11-06)
|
||||
|
||||
| Row | Contract test | Runs in |
|
||||
| --- | --- | --- |
|
||||
| E1 | `tests/test_executor_lanes.py`: `executor-lanes.py contract()` compares role, KV path, fields, auth shape and the policy file text with CCR-2026-0058/0059 | net-kingdom contract CI |
|
||||
| E2, E3 | `tools/contract-tests/tests/test_exercises.py::test_t19`: KeyCape binary from the pinned image, approval-engine source | net-kingdom contract CI |
|
||||
| E4 | `bounded-executor/tests/test_approval_request.py` against informed-decision's intake code | net-kingdom contract CI |
|
||||
| E5, C6, C7, O1 | `tools/bounded-execution-readiness/tests/test_ready_owner_sources.py`, with both 2026-11-04 states replayed from history | net-kingdom contract CI |
|
||||
| E6 | `test_exercises.py::test_t15`: audit-core ingestion and sender scope | net-kingdom contract CI |
|
||||
| E7 | `tools/contract-tests/tests/test_lldap_tool.py`: one LLDAP digest for the executor tool and the server | net-kingdom contract CI |
|
||||
| E8, R2 | rapp `tests/test_packaging.py`: ceiling, config and exit codes against net-kingdom | rapp packaging CI |
|
||||
| E9 | `tools/contract-tests/tests/test_ingress.py`: each owner's ingress admits the executor; rapp tests the egress | both |
|
||||
| R1 | the executor's `source` preflight check in the pinned image (`make preflight`) | before each window |
|
||||
| C1 | `tests/test_executor_lanes.py` runs railiance-platform's real helper modules | net-kingdom contract CI |
|
||||
| C2, C3 | the readiness check: `case_type_bytes_match_broker`, `case_materialized` | net-kingdom contract CI |
|
||||
| C4, C5 | `tools/contract-tests/tests/test_custody_names.py` | net-kingdom contract CI |
|
||||
| O2–O4, F1, F2 | `tools/bounded-execution-run/tests/test_bx.py` (lane child silent, window always closed); conventions in T07 | net-kingdom contract CI |
|
||||
|
||||
Not covered from source: OpenBao's ingress (rapp-openbao is private) and live drift from source, such as the
|
||||
approval-engine policy that was live but not in its file. The readiness preflight Job catches unreachable endpoints
|
||||
before a window, and T07 and O1 cover the drift.
|
||||
|
|
|
|||
86
tools/contract-tests/tests/test_custody_names.py
Normal file
86
tools/contract-tests/tests/test_custody_names.py
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
"""Rows C4 and C5: the custody names agree across net-kingdom, railiance-platform and audit-core (NK-WP-0058-T02).
|
||||
|
||||
C4, KeyCape client secret: railiance-platform's ESO template and rendered manifest, CCR-2026-0061's KV path, and
|
||||
net-kingdom's keycape-client.py naming and the KeyCape Deployment's env binding.
|
||||
C5, audit-core sender bearer: CCR-2026-0063's KV path, fields and delivery, the platform's ExternalSecret, and the
|
||||
names audit-core's seeder uses (read as text; the seeder is not imported).
|
||||
The rapp Job's mount of sso/bounded-executor-tokens is private and checked in the rapp's own CI.
|
||||
"""
|
||||
import importlib.util
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
REPO = Path(__file__).resolve().parents[3]
|
||||
PLATFORM = REPO.parent / 'railiance-platform'
|
||||
AUDIT_CORE = REPO.parent / 'audit-core'
|
||||
ESO = PLATFORM / 'manifests/nk-bind-executor-eso.yaml'
|
||||
CCRS = PLATFORM / 'credential-change-requests'
|
||||
|
||||
|
||||
def need(*paths):
|
||||
for path in paths:
|
||||
if not path.exists():
|
||||
pytest.skip(f'owner checkout missing: {path.parent.name}/{path.name}')
|
||||
|
||||
|
||||
def ccr(prefix):
|
||||
return yaml.safe_load(next(CCRS.glob(prefix + '-*.yaml')).read_text())
|
||||
|
||||
|
||||
def external_secrets():
|
||||
docs = [d for d in yaml.safe_load_all(ESO.read_text()) if d]
|
||||
stores = {d['metadata']['name']: d for d in docs if d['kind'] == 'ClusterSecretStore'}
|
||||
return {d['spec']['target']['name']: (d, stores[d['spec']['secretStoreRef']['name']])
|
||||
for d in docs if d['kind'] == 'ExternalSecret'}
|
||||
|
||||
|
||||
def kv_path(store, remote_key):
|
||||
return store['spec']['provider']['vault']['path'].strip('/') + '/' + remote_key
|
||||
|
||||
|
||||
def keycape_client():
|
||||
spec = importlib.util.spec_from_file_location('kc_client_names', REPO / 'sso-mfa/k8s/keycape/keycape-client.py')
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def test_c4_keycape_client_secret_names():
|
||||
need(ESO, PLATFORM / 'manifests/templates/keycape-issuer-eso.yaml')
|
||||
kc = keycape_client()
|
||||
case = yaml.safe_load((REPO / 'security-cases/nk-bind-executor-approval.yaml').read_text())['spec']
|
||||
cid = case['parameters']['client_id']
|
||||
secret_name, env = kc.env_secret_name(cid), kc.env_name(cid)
|
||||
template = (PLATFORM / 'manifests/templates/keycape-issuer-eso.yaml').read_text()
|
||||
assert 'name: keycape-{client_id}-client' in template, 'template naming convention changed'
|
||||
assert kc.env_secret_name('{client_id}') == 'keycape-{client_id}-client'
|
||||
es, store = external_secrets()[secret_name]
|
||||
(item,) = es['spec']['data']
|
||||
custody = ccr('CCR-2026-0061')['openbao']
|
||||
assert item['secretKey'] == 'client-secret'
|
||||
assert kv_path(store, item['remoteRef']['key']) == custody['kv_path'] == f'platform/workloads/keycape/{cid}'
|
||||
assert [item['remoteRef']['property']] == custody['fields'] == ['CLIENT_SECRET']
|
||||
deployment = next(d for d in yaml.safe_load_all((REPO / 'sso-mfa/k8s/keycape/deployment.yaml').read_text())
|
||||
if d and d['kind'] == 'Deployment' and d['metadata']['name'] == 'keycape')
|
||||
bound = {e['name']: e.get('valueFrom', {}).get('secretKeyRef') for c in deployment['spec']['template']['spec']
|
||||
['containers'] for e in c.get('env', [])}
|
||||
assert bound.get(env) == {'name': secret_name, 'key': 'client-secret'}, 'KeyCape env binding differs'
|
||||
|
||||
|
||||
def test_c5_audit_sender_custody_names():
|
||||
seeder = AUDIT_CORE / 'scripts/seed-nk-bounded-executor-sender.py'
|
||||
need(ESO, seeder)
|
||||
constants = dict(re.findall(r'^([A-Z_]+) = "([^"]*)"', seeder.read_text(), re.M))
|
||||
custody = ccr('CCR-2026-0063')
|
||||
es, store = external_secrets()[constants['SSO_SECRET']]
|
||||
(item,) = es['spec']['data']
|
||||
assert es['metadata']['namespace'] == custody['delivery']['namespace'] == 'sso'
|
||||
assert custody['openbao']['kv_path'] == constants['CUSTODY_KV'] == kv_path(store, item['remoteRef']['key'])
|
||||
assert item['remoteRef']['property'] in custody['openbao']['fields'] == ['AUDIT_TOKEN', 'CUSTODY_REQUEST']
|
||||
assert item['secretKey'] == constants['SSO_KEY'] == 'audit-core'
|
||||
target = custody['delivery']['target']
|
||||
assert f"sso/{constants['SSO_SECRET']}" in target and constants['SSO_KEY'] in target
|
||||
assert constants['CCR_ID'] == custody['id']
|
||||
103
tools/contract-tests/tests/test_ingress.py
Normal file
103
tools/contract-tests/tests/test_ingress.py
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
"""Row E9: every owner's ingress admits the executor pod (NK-WP-0058-T02), read from the owners' source.
|
||||
|
||||
Kubernetes semantics, for the subset these policies use: a pod that no Ingress policy selects is open; once any
|
||||
policy selects it, traffic must match an ingress rule of one of them (peer selectors and port). matchExpressions and
|
||||
ipBlock peers are not used by these policies; a policy that uses them fails the test instead of being guessed.
|
||||
Not covered here: OpenBao (rapp-openbao is private) and the executor's egress (the rapp's own CI, row E9 there).
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
REPO = Path(__file__).resolve().parents[3]
|
||||
SIBLINGS = REPO.parent
|
||||
EXECUTOR = {'namespace': 'sso', 'labels': {'app.kubernetes.io/name': 'bounded-executor'}}
|
||||
|
||||
SSO = 'sso-mfa/k8s/network-policies/netpol-sso.yaml'
|
||||
# (owner, policy files relative to the owner, namespace, target pod labels, port), from the executor's config
|
||||
TARGETS = [
|
||||
('approval-engine', ['deploy/networkpolicies.yaml'], 'approval-engine',
|
||||
{'app.kubernetes.io/name': 'approval-engine'}, 8080),
|
||||
('audit-core', ['deploy/networkpolicies.yaml'], 'audit-core', {'app.kubernetes.io/name': 'audit-core'}, 8080),
|
||||
('informed-decision', ['deploy/lifecycle/*.yaml'], 'informed-decision',
|
||||
{'app.kubernetes.io/name': 'informed-decision', 'app.kubernetes.io/component': 'review'}, 8081),
|
||||
] + [('net-kingdom', [SSO], 'sso', {'app.kubernetes.io/name': name}, port)
|
||||
for name, port in (('lldap', 17170), ('keycape', 8080), ('authelia', 9091))]
|
||||
|
||||
|
||||
def selects(selector, labels):
|
||||
if set(selector) - {'matchLabels'}:
|
||||
raise AssertionError(f'unsupported selector {sorted(selector)}: extend the evaluator, do not guess')
|
||||
return all(labels.get(k) == v for k, v in (selector.get('matchLabels') or {}).items())
|
||||
|
||||
|
||||
def peer_admits(peer, policy_ns, source):
|
||||
if set(peer) - {'namespaceSelector', 'podSelector'}:
|
||||
raise AssertionError(f'unsupported peer {sorted(peer)}')
|
||||
if 'namespaceSelector' in peer:
|
||||
ns_ok = selects(peer['namespaceSelector'], {'kubernetes.io/metadata.name': source['namespace']})
|
||||
else:
|
||||
ns_ok = source['namespace'] == policy_ns
|
||||
return ns_ok and selects(peer.get('podSelector') or {}, source['labels'])
|
||||
|
||||
|
||||
def port_admits(ports, port):
|
||||
return not ports or any(p.get('port') == port and p.get('protocol', 'TCP') == 'TCP' for p in ports)
|
||||
|
||||
|
||||
def admitted(policies, namespace, labels, port, source=EXECUTOR):
|
||||
applying = [p for p in policies if (p['metadata'].get('namespace') or namespace) == namespace
|
||||
and 'Ingress' in p['spec'].get('policyTypes', ['Ingress'])
|
||||
and selects(p['spec'].get('podSelector') or {}, labels)]
|
||||
if not applying:
|
||||
return 'open'
|
||||
for policy in applying:
|
||||
for rule in policy['spec'].get('ingress') or []:
|
||||
peers = rule.get('from')
|
||||
if (not peers or any(peer_admits(peer, namespace, source) for peer in peers)) \
|
||||
and port_admits(rule.get('ports'), port):
|
||||
return 'admitted:' + policy['metadata']['name']
|
||||
return 'denied'
|
||||
|
||||
|
||||
def policies_of(owner, patterns):
|
||||
root = REPO if owner == 'net-kingdom' else SIBLINGS / owner
|
||||
if not root.is_dir():
|
||||
pytest.skip(f'owner checkout missing: {owner}')
|
||||
out = []
|
||||
for pattern in patterns:
|
||||
for path in sorted(root.glob(pattern)):
|
||||
out += [d for d in yaml.safe_load_all(path.read_text()) if d and d.get('kind') == 'NetworkPolicy']
|
||||
return out
|
||||
|
||||
|
||||
@pytest.mark.parametrize('owner, files, namespace, labels, port', TARGETS,
|
||||
ids=[f"{t[0]}:{t[3]['app.kubernetes.io/name']}:{t[4]}" for t in TARGETS])
|
||||
def test_owner_ingress_admits_the_executor(owner, files, namespace, labels, port):
|
||||
result = admitted(policies_of(owner, files), namespace, labels, port)
|
||||
assert result != 'denied', f'{owner} ingress does not admit sso/bounded-executor on {port}'
|
||||
|
||||
|
||||
def policy(name, namespace, target, peers=None, ports=None):
|
||||
rule = {**({'from': peers} if peers is not None else {}), **({'ports': ports} if ports else {})}
|
||||
return {'kind': 'NetworkPolicy', 'metadata': {'name': name, 'namespace': namespace},
|
||||
'spec': {'podSelector': {'matchLabels': target}, 'policyTypes': ['Ingress'], 'ingress': [rule]}}
|
||||
|
||||
|
||||
def test_the_evaluator_on_synthetic_policies():
|
||||
target, ns = {'app.kubernetes.io/name': 'svc'}, 'svc'
|
||||
executor_peer = {'namespaceSelector': {'matchLabels': {'kubernetes.io/metadata.name': 'sso'}},
|
||||
'podSelector': {'matchLabels': {'app.kubernetes.io/name': 'bounded-executor'}}}
|
||||
other_peer = {'namespaceSelector': {'matchLabels': {'kubernetes.io/metadata.name': 'other'}}}
|
||||
assert admitted([], ns, target, 8080) == 'open'
|
||||
assert admitted([policy('a', ns, target, [executor_peer], [{'port': 8080}])], ns, target, 8080) == 'admitted:a'
|
||||
assert admitted([policy('a', ns, target, [executor_peer], [{'port': 9090}])], ns, target, 8080) == 'denied'
|
||||
assert admitted([policy('a', ns, target, [other_peer])], ns, target, 8080) == 'denied'
|
||||
assert admitted([policy('a', ns, {'app.kubernetes.io/name': 'else'}, [other_peer])], ns, target, 8080) == 'open'
|
||||
deny_all = {'kind': 'NetworkPolicy', 'metadata': {'name': 'deny', 'namespace': ns},
|
||||
'spec': {'podSelector': {}, 'policyTypes': ['Ingress']}}
|
||||
assert admitted([deny_all], ns, target, 8080) == 'denied'
|
||||
same_ns_peer = {'podSelector': {'matchLabels': {'app.kubernetes.io/name': 'bounded-executor'}}}
|
||||
assert admitted([deny_all, policy('b', 'sso', target, [same_ns_peer])], 'sso', target, 8080) == 'admitted:b'
|
||||
assert admitted([deny_all, policy('b', ns, target, [same_ns_peer])], ns, target, 8080) == 'denied'
|
||||
21
tools/contract-tests/tests/test_lldap_tool.py
Normal file
21
tools/contract-tests/tests/test_lldap_tool.py
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
"""Row E7: the executor's lldap_set_password comes from the LLDAP image the sso server runs (NK-WP-0058-T02).
|
||||
|
||||
The executor image copies /app/lldap_set_password from a digest-pinned LLDAP image (bounded-executor/Containerfile).
|
||||
The tool speaks LLDAP's OPAQUE protocol, so it must be the server's own version: the two digests must be equal.
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
REPO = Path(__file__).resolve().parents[3]
|
||||
DIGEST = r'docker\.io/lldap/lldap@(sha256:[0-9a-f]{64})'
|
||||
|
||||
|
||||
def test_executor_tool_matches_the_lldap_server():
|
||||
tool = re.findall(DIGEST, (REPO / 'bounded-executor/Containerfile').read_text())
|
||||
deployment = next(d for d in yaml.safe_load_all((REPO / 'sso-mfa/k8s/lldap/deployment.yaml').read_text())
|
||||
if d and d.get('kind') == 'Deployment')
|
||||
server = [m for c in deployment['spec']['template']['spec']['containers'] for m in re.findall(DIGEST, c['image'])]
|
||||
assert len(tool) == 1 and len(server) == 1, (tool, server)
|
||||
assert tool == server, 'the executor image would copy lldap_set_password from another LLDAP version'
|
||||
|
|
@ -246,7 +246,7 @@ controller-per-object table, a founder-step table and the status table for T05.
|
|||
|
||||
```task
|
||||
id: NK-WP-0058-T02
|
||||
status: progress
|
||||
status: done
|
||||
priority: high
|
||||
assignee: claude-code
|
||||
depends_on: [NK-WP-0058-T01, NK-WP-0058-T09, NK-WP-0058-T10]
|
||||
|
|
@ -275,6 +275,44 @@ role name, a response code or a memo field.
|
|||
has a test that would have caught it. Code in net-kingdom that duplicated a contract by hand is removed. For
|
||||
duplicates in other repos, a removal request is sent to their owners and recorded.
|
||||
|
||||
**Result, 2026-11-06.**
|
||||
|
||||
1. **Owner source in CI.**
|
||||
- `contracts/owner-pins.json` pins six public owners.
|
||||
- `tools/contract-tests/fetch_owners.py` makes anonymous partial clones of them: pinned on push, `main`
|
||||
nightly.
|
||||
- Workflow `bounded-execution-contracts.yaml` runs `tools/contract-tests/run.sh`, the one suite list (also
|
||||
`make contract-test`). Any skip fails it.
|
||||
- pytest and the crypto chain are pinned by hashes checked against PyPI.
|
||||
- The private rapp tests itself in its own CI (rapp `15df0e3`), using the run's automatic token.
|
||||
2. **Exercises folded in.** T15 runs against audit-core's ingestion. T19 runs against approval-engine's source,
|
||||
with the KeyCape binary extracted from the pinned image by verified digests (`oci_extract.py`, no docker).
|
||||
3. **Per-row tests.**
|
||||
- ingress for E9, with a small NetworkPolicy evaluator and synthetic cases;
|
||||
- custody names for C4 and C5;
|
||||
- the LLDAP tool and server digest for E7;
|
||||
- readiness against real owner source, replaying both 2026-11-04 states from informed-decision history (E5,
|
||||
C6, C7).
|
||||
|
||||
The coverage by row is in the contract table. 95 tests pass in a CI-like layout.
|
||||
|
||||
- **The 2026-11-04/05 failures and their tests:**
|
||||
- the approver routing: the readiness replay;
|
||||
- `setup:URLError`: the ingress test, plus the preflight Job before a window;
|
||||
- the window left open overnight: `bx` closes in `finally`, tested;
|
||||
- the opaque `LaneError`: named causes, tested.
|
||||
|
||||
The ESO apply and admission-guard and the kubectl-wait failures are T07 conventions. The warden token variable is
|
||||
the O2 owner request.
|
||||
- **Hand duplicates:** `pilot-run.sh` and the hard-coded informed-decision test path are removed. `LANES` keeps
|
||||
copies of the CCR values, but `contract()` checks them against the CCRs on every run and in CI. The stale Secret
|
||||
name in railiance-platform was sent as a request (`f6198285`).
|
||||
- **Owner agreement:** questions were sent to six owners (`4892658f`, `3a546559`, `aa44a2b3`, `7cf8d61f`,
|
||||
`a2105e34`, `ee1e5bb5`). Until they answer, the pinned-plus-nightly mode stands.
|
||||
- **Not covered:** OpenBao's ingress (rapp-openbao is private), and live drift from source, which the preflight
|
||||
Job and T07 address.
|
||||
- **Unverified here:** the first Forgejo runs of both workflows. The Actions API needs a login.
|
||||
|
||||
### Readiness check before any window opens
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue