net-kingdom/workplans/NK-WP-0058-bounded-execution-operability.md
tegwick e1561c09ee
All checks were successful
CI Smoke / host-smoke (push) Successful in 1s
CI Smoke / container-smoke (push) Successful in 3s
NK-WP-0058-T10 done: contract table part 3 (Argo vs manual applies, warden lanes, founder steps, agent-readable status); C3 corrected
Only informed-decision lifecycle, the Secret admission guard and shared SecretStores are Argo-managed;
two platform-admin session patterns; ~45 founder commands with five hand-carried values; the agent
reads only NetworkPolicies. C3: the canon defines materialization as structure, so the order is right.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 943911@bnt-lap001
Assistant-Session: fc322940-1c8e-4bed-ab39-86cec99f23a0
2026-11-05 23:10:56 +01:00

427 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
id: NK-WP-0058
type: workplan
title: "Bounded execution operability: clear boundaries, lean interfaces, transparent diagnostics"
domain: infotech
repo: net-kingdom
status: active
flavor: implementation
owner: claude
topic_slug: netkingdom
created: "2026-11-05"
updated: "2026-11-05"
related:
- NK-WP-0056
- NK-WP-0057
state_hub_workstream_id: "30b26339-60cd-510b-97cb-93eaf304a2c3"
---
# NK-WP-0058 — Bounded execution operability
Intake from the founder after NK-WP-0056 finished (2026-11-05). The pilot worked: one admission, one approval, then
an unattended rotation in 30 s. Getting there was the problem:
- about 50 founder copy-paste rounds;
- one wasted window, because the approver was configured differently in three repos;
- an opaque executor error that took four founder-run diagnostic pods to locate;
- runbook assumptions that failed live (admission policy on Secrets, Argo-managed ConfigMap, warden environment,
`kubectl wait`);
- a write role left open overnight;
- workers in two repos building competing scripts and using different names.
The review behind this workplan is the session of 2026-11-05. Its findings are in the NK-WP-0056 evidence
(`docs/evidence/2026-11-04-nk56-t06-pilot/`, `docs/evidence/2026-11-05-nk56-t06-pilot/`).
## Boundary conditions (apply to every task)
Founder direction, 2026-11-05. A task is not done unless its result meets all four:
1. **Clear responsibility boundaries between repos.** Every interface has one owner. Every artifact has one source of
truth, which is named, and no repo silently mirrors another's configuration. Where a responsibility is unclear, the
task records the question for NK-WP-0057 (decision routing) or the owning repo, instead of deciding it locally.
2. **Better interfaces between repos and with the human operator.** Cross-repo contracts are written down, versioned
and checked: names, paths, roles, payloads and exit codes. Founder steps are few, short and stateful. Each one
prints a single value-free result and is safe to repeat.
3. **Transparency first, so diagnosis is easy.**
- every failure names where it happened and why, by phase, endpoint or check, never a bare exception type;
- status is readable by the agent identity without secrets;
- every live step leaves a value-free receipt.
4. **Minimal code, carried by fully automated interface tests.**
- Prefer deleting, merging or reusing an owner's tool over adding one.
- Every cross-repo interface used by the bounded-execution path gets automated contract tests, against stubs or
the owner's real source where the service can't run in CI.
- Contract tests replace manual rehearsal wherever they can.
- Tests run in CI; a change to the code an interface depends on, in either repo, fails a test.
## Review, 2026-11-05
Reviewed against the repository state on 2026-11-05 and set to `ready`. Changes from the proposal:
- The overnight write-role check was split out of T06 into T11 and made urgent: it depends on OpenBao audit
retention.
- The inventory (T01) was split into three parts: T01, T09 and T10. They fill one table, which is also the input to
NK-WP-0057-T01, so that no second inventory is started.
- T02 now starts with an owner agreement on how a contract change reaches net-kingdom CI.
- T03 names the owners of the executor mode: net-kingdom owns the source and the image, and `rapp-bounded-executor`
packages and deploys it (NK-WP-0056, founder decision `6d1adce9`).
- T05's decision is created after T01, once the list of statuses the agent could not read is known.
- T07 no longer waits for the inventory.
Suggested order: T11 first, then T01, T09 and T10, then T03. T02 and T04 come before the next live window. T05, T06
and T07 can run in parallel with these, and T08 comes last.
## Tasks
### Confirm the overnight write-role window left no trace
```task
id: NK-WP-0058-T11
status: done
priority: high
assignee: claude-code
state_hub_task_id: "e2d02456-b072-5d42-a25b-964dc14de5ef"
```
This is the open check from 2026-11-05. It must not wait for the rest of the workplan, because the evidence it needs
ages out with OpenBao audit retention.
- Confirm that CCR-2026-0058's write role is absent now.
- Confirm from OpenBao's audit that no login used the role while it was open overnight on 2026-11-04/05.
The agent prepares read-only, value-free queries for both checks. Where the agent identity lacks audit access, the
founder runs them and reports one result line for each.
**Done when:** a dated, value-free receipt records both results under the NK-WP-0056 pilot evidence. Any login that
was found is routed as a security finding to railiance-platform.
**Result, 2026-11-05: clean.** Evidence is in `docs/evidence/2026-11-05-nk58-t11-write-role-check/`, linked from the
NK-WP-0056 pilot README.
- From 23:33:29Z (the last attended login plus the 15-minute token lifetime) to the close at 08:26:56Z, there was no
login with the write role, no request with its policy, and no failed Kubernetes login.
- All five logins in the log are the documented attended runs and diagnostics, plus the pilot run.
- The role is absent at the end of the log (21:33Z). This was derived from the audit log's create and delete
entries, because the direct `window-close` read failed for lack of a platform-admin login.
- No finding was routed.
Tool: `tools/service-binds/write-role-audit.py`, with tests in `tests/test_write_role_audit.py`.
Inputs for other tasks:
- **T06:** the audit file is not rotated (2 GiB PVC, history since 2026-08-03), and one read takes about 10 minutes
through `kubectl exec`. A recurring check should use audit forwarding (NK-WP-0055-T11), not this stream.
- **T03:** the failed close reported only `LaneError`, without its cause.
### Interface and responsibility inventory, part 1: executor and its services
```task
id: NK-WP-0058-T01
status: done
priority: high
assignee: claude-code
state_hub_task_id: "fe618717-0c28-51c5-8edb-e5d2f4ba2252"
```
Create the single contract table in `docs/`. Each row is one interface the bounded-execution path crosses and records:
- **owner and source of truth;**
- **consumer;**
- **the exact contract:** names, paths, payloads, exit codes;
- **how it is tested today** (none, stub, real source, live only);
- **the 2026-11-04/05 failures it caused.**
Part 1 covers the executor's own interfaces:
- executor ↔ approval-engine, KeyCape, audit-core, informed-decision, OpenBao and LLDAP;
- rapp (`rapp-bounded-executor`) ↔ net-kingdom: image, pins, Job template and RBAC.
Mark duplicated or unowned pieces, for example the approver configured in three places. A row whose owner is
unclear is marked "needs an owner decision" and becomes an input to NK-WP-0057-T01, which does not inventory it a
second time.
**Done when:** the table exists with sources for every part 1 row, and each row is marked keep, merge, delete or
needs an owner decision.
**Result, 2026-11-05.** The table is `docs/bounded-execution-contract-table.md`, with 11 rows (E1–E9, R1, R2) and
details per row.
- **keep:** E2, E3, E4, E6, E7 and E9.
- **merge:**
- E1: lane names and policy text are copied from the CCRs;
- E5: the approver is defined four times;
- E8: the RBAC is copied into the rapp;
- R1: no record links the commit, the digest and the admission pins;
- R2: the config and exit codes are copied into the rapp.
- **owner decisions for NK-WP-0057:**
- E5: approver and routing;
- E2: KeyCape client registration, since there is no KeyCape rapp;
- E7: the runtime owner of LLDAP, Authelia and KeyCape in `sso`;
- E3: approval-engine enforcing `required_assurance`.
- **Framing fact for T02:** net-kingdom CI tests net-kingdom alone, so every sibling-dependent test skips there.
rapp-bounded-executor has no CI, and the T15 and T19 exercises are manual.
- **Inputs for other tasks:**
- T03: `executor-lanes.py` lane errors without a cause, and `approval_entry` without the failed field;
- T07: a hard-coded path in `test_approval_request.py:216`.
### Interface and responsibility inventory, part 2: approval, case and custody chain
```task
id: NK-WP-0058-T09
status: done
priority: high
assignee: claude-code
depends_on: [NK-WP-0058-T01]
state_hub_task_id: "b10d162b-b66c-5274-ac81-adeae1574389"
```
Add rows to the T01 table, in the same format, for:
- railiance-platform CCRs and broker ↔ net-kingdom cases;
- audit-core seed ↔ railiance-platform custody;
- informed-decision routing ↔ access-engine policy.
Mark the duplicated pieces, for example the two seeders and the two Secret names.
**Done when:** every part 2 row has sources and a keep, merge, delete or owner-decision mark.
**Result, 2026-11-05.** Part 2 of `docs/bounded-execution-contract-table.md` has seven rows (C1–C7).
- **The two seeders and the two Secret names** were already settled on 2026-11-04: audit-core's script is the
single seeder, and the Secret is `sso/bounded-executor-tokens:audit-core`. One stale copy remains in
railiance-platform `docs/credential-lane-designs/bounded-executor.md`, which is a **delete** request.
- **keep:**
- C2: acknowledged `type.yaml` bytes are hash-pinned by the broker, and net-kingdom has no guard test;
- C3.
- **merge:**
- C1: `executor-lanes.py` imports three railiance-platform internal helpers;
- C4: the KeyCape secret naming convention is written twice;
- C5: the audit custody names appear in four repos;
- C6: the executor's submitter profile is in informed-decision, access-engine and the rapp.
- **owner decisions:**
- C1: a supported platform interface;
- C3, small: who materializes (the canon names ops-mason; railiance-platform did it). An earlier draft called
the status order wrong; the canon confirms it;
- C6: which repo is the source of the profile;
- C7: routing and visibility. access-engine's `list` admits only the routing recipient, so the routing table
decides who can approve. This goes to NK-WP-0057 with E5.
- **For T07:** audit-core's seeder hard-codes a home path and the cluster UID.
### Interface and responsibility inventory, part 3: operations and the founder
```task
id: NK-WP-0058-T10
status: done
priority: high
assignee: claude-code
depends_on: [NK-WP-0058-T01]
state_hub_task_id: "e2515bd5-f4bd-5480-ad86-21e5044b468d"
```
Add rows to the T01 table, in the same format, for:
- Argo ↔ manual applies, including which controller manages each object the runbooks write;
- warden lanes ↔ child scripts, including the environment they pass on;
- founder ↔ each runbook step, with the authority each step needs and the value-free result it prints.
**Done when:** every part 3 row has sources and a mark, and the table lists every status the agent had to ask the
founder for on 2026-11-04/05 (the input for T05).
**Result, 2026-11-05.** Part 3 of `docs/bounded-execution-contract-table.md` has six rows (O1–O4, F1, F2), a
controller-per-object table, a founder-step table and the status table for T05.
- **Argo (O1):** only informed-decision `deploy/lifecycle`, the Secret admission guard and the shared
SecretStores are Argo-managed, all with pinned revisions and manual sync. Everything else is applied by hand.
Three surprises came from this split. Which objects Argo owns is an owner decision for railiance-platform.
- **Sessions (O2, O3):** warden lanes for platform and audit-core steps, but a shell `bao login` for net-kingdom's
window commands. Merge on warden lanes. Owner request to ops-warden: strip every token variable.
- **Founder (F1, F2):** about 45 commands across three runbooks, with five kinds of values carried by hand. The
first approval came from the identity that happened to be signed in. These are inputs for T04.
- **For T05:** the agent can read only NetworkPolicies today. The proposed grant and the founder-only rows are in
the status table.
- **For T07:** four tools hard-code kubeconfigs.
- **Correction:** C3 in part 2 was corrected the same day. The canon defines materialization as structure only, so
committing `materialized` before the mint is right.
### Contract tests for the cross-repo interfaces
```task
id: NK-WP-0058-T02
status: todo
priority: high
assignee: claude-code
depends_on: [NK-WP-0058-T01, NK-WP-0058-T09, NK-WP-0058-T10]
state_hub_task_id: "a33eeee6-caef-57d4-b067-26231a19a165"
```
First, agree with each owner how a change to their contract reaches net-kingdom CI. Two ways are possible:
- the owner publishes a versioned contract file that net-kingdom pins;
- net-kingdom CI tests against a pinned checkout of the owner's source and is told about new versions.
Record the agreement in the owner's workplan. Where no agreement is reached, the test runs against the pinned
version, and the table records that drift is caught only when the pin moves.
Then, for every interface marked keep or merge, add an automated contract test in CI:
- against the owner's real source where it runs locally (approval-engine, audit-core, informed-decision intake,
the KeyCape image);
- otherwise against a stub that is generated from, or checked against, the owner's contract.
Fold the existing exercises (T19 approval identity, T15 audit sender) into this suite instead of keeping them as
separate scripts. A change in either repo that breaks a contract fails CI in the consumer, such as a Secret name, a
role name, a response code or a memo field.
**Done when:** the suite runs in CI for every interface in the table, and each interface that failed on 2026-11-04/05
has a test that would have caught it. Code in net-kingdom that duplicated a contract by hand is removed. For
duplicates in other repos, a removal request is sent to their owners and recorded.
### Readiness check before any window opens
```task
id: NK-WP-0058-T03
status: todo
priority: high
assignee: claude-code
depends_on: [NK-WP-0058-T01, NK-WP-0058-T09]
state_hub_task_id: "0e403b18-cf7b-5899-bd86-661fc7284014"
```
Owners: net-kingdom owns the executor source, its tests and its image, so the `--preflight-only` mode is written
here. `rapp-bounded-executor` owns the Job packaging, so running the mode as a Job is a change to the rapp. The
readiness checks call the owners' validators and do not copy them.
One command that verifies the chain while nothing is at stake, so a window opens only when everything is green:
- the admission is valid and its pins match the pinned image;
- the admission's approver equals informed-decision's recipient and the approval principal;
- the CCR approvals and bindings are complete;
- the case status, custody delivery and endpoint reachability;
- the write role is absent.
Add an executor `--preflight-only` mode that runs setup and preflight without consuming. It replaces hand-written
diagnostic pods.
The check reuses the owners' validators. Where a check needs authority the agent lacks, it names the single
read-only founder command that supplies it.
**Done when:** the readiness check is tested on stubs, runs with the agent identity wherever possible, and
`--preflight-only` reports every failure it finds by phase and endpoint.
### Founder interface: few, short, stateful steps
```task
id: NK-WP-0058-T04
status: todo
priority: high
assignee: claude-code
depends_on: [NK-WP-0058-T03]
state_hub_task_id: "9b70a844-edb2-50b3-a06c-be57d3823b3c"
```
Replace the copy-paste runbooks for a run with a small command set: `ready`, `request`, `run` (which opens the window
and always closes it), `status` and `close`. Each command:
- keeps its inputs in a state file, not in shell variables;
- numbers receipts itself;
- is idempotent;
- has no line over 120 characters in its instructions;
- prints one value-free result line.
Build on `pilot-run.sh`, and delete the runbook prose these commands replace. Record the identity hygiene a step
needs: a browser profile per identity, and when a fresh OTP login is due.
**Done when:** a full pilot run is at most five short founder commands plus the approval in informed-decision. The
commands are tested on stubs, and T06's runbook shrinks accordingly.
### Agent status visibility without secrets
```task
id: NK-WP-0058-T05
status: todo
priority: medium
assignee: claude-code
needs_human: true
intervention_note: "Founder approves a read-only status grant for the agent identity (owner: railiance-cluster / railiance-platform)."
depends_on: [NK-WP-0058-T10]
state_hub_task_id: "30f8c62e-5707-5395-9d86-178cd650fb59"
```
The State Hub decision for this gate is created once T10 has listed the statuses the agent could not read. Its id
then goes into this task block (C-39).
Propose a read-only grant for the agent's Kubernetes identity in the namespaces the path uses. The grant covers:
- the status and conditions of Jobs, pods, ExternalSecrets and Deployments;
- pod logs of executor Jobs, which are value-free by contract.
It never covers Secrets, kubeconfigs or exec. Route the grant through its owner as a reviewed change. Record what
stays founder-only and why.
**Done when:** the grant is decided, and if approved applied and verified with `can-i`. Every status the agent had
to ask the founder for on 2026-11-04/05 is then readable, or explicitly founder-only.
### Write-role window closed by the platform
```task
id: NK-WP-0058-T06
status: todo
priority: medium
assignee: claude-code
depends_on: [NK-WP-0058-T09]
state_hub_task_id: "6a4c7413-0675-5a06-b471-5f849ce0c721"
```
Make the end of CCR-2026-0058's window not depend on a person remembering. Two candidates, to be chosen with
railiance-platform (the owner):
- a scheduled close at `not_after`;
- an alert or finding whenever the write role exists outside an open admission window. The latter could reuse
audit-core's stream findings or posture feedback.
The open check from 2026-11-05 on the overnight window is T11.
**Done when:** an unclosed window is closed or flagged without human memory, and this is tested on a stub.
### Runbook and tooling conventions
```task
id: NK-WP-0058-T07
status: todo
priority: medium
assignee: claude-code
state_hub_task_id: "a99424ed-ee8b-5db4-9fa7-2ae5fc32c39c"
```
This task does not wait for the inventory, because the conventions below are already known. Findings from T10 are
added when they arrive.
Turn the 2026-11-04/05 surprises into checked conventions:
- server-side apply only;
- check the managing controller before writing apply commands;
- poll status instead of `kubectl wait` with several conditions;
- no hard-coded kubeconfigs in owner tools;
- warden strips every token variable (ops-warden owner request);
- receipts are named by the tool, not by the operator.
Prefer a lint or a test over prose, and remove the hard-coded and duplicated bits it finds.
**Done when:** the conventions are enforced by a test or lint in CI for the net-kingdom and rapp runbooks and tools,
and the owner requests for other repos are sent and recorded.
### Multi-repo worker protocol
```task
id: NK-WP-0058-T08
status: todo
priority: low
assignee: claude-code
depends_on: [NK-WP-0058-T02]
state_hub_task_id: "dd2b72c8-7225-5b89-8936-d44c99f95872"
```
Before worker agents start on a cross-repo change, they get one contract excerpt: the names, paths and roles they
touch, and the single owning script per step. Each worker's result is then checked against that excerpt with the
T02 tests. This prevents the competing seeders and mismatched names seen on 2026-11-04.
**Done when:** the protocol is in the agent instructions, and its first use is checked by the contract tests.