railiance-platform/docs/credential-lane-designs/nk-bind-executor-go-live.md
codex 47f5d37e7c
All checks were successful
CCR intake gate / gate (push) Successful in 11s
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
nk-bind-executor go-live: the case of record is net-kingdom security-cases/nk-bind-executor-approval.yaml
The type's example stays an example; accepted cases live in security-cases/ (as nk-synthetic-journey).
The broker takes the case by path and hash, so only the runbook path changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 523928@bnt-lap001
Assistant-Session: 6c737afb-9c14-4152-9f02-477214b9c350
2026-11-04 02:04:26 +01:00

17 KiB

Bounded executor credential go-live (NK-WP-0056-T06; RPF-WP-0069-T05/T09)

Founder runbook for the platform side of the bounded executor's two credentials. Source only: nothing here has been run. Every command is founder-run and attended, from /home/worsch/railiance-platform at the reviewed commit. No command prints a credential value.

Request What it grants Approvers
CCR-2026-0060 KeyCape's ESO reader of platform/workloads/keycape/nk-bind-executor-approval (CLIENT_SECRET), role external-secrets-keycape-nk-bind-executor-approval platform-owner
CCR-2026-0061 The executor's own reader of the same path, role keycape-nk-bind-executor-approval-bounded-executor-workload-kv-read, bound to sso/nk-bind-executor, audience openbao, 15m platform-owner, net-kingdom-owner
CCR-2026-0063 Audit-core sender bearer custody platform/workloads/net-kingdom/bounded-executor/audit-sender, ESO reader role external-secrets-nk-bind-executor-audit platform-owner, audit-core-owner, net-kingdom-owner
CCR-2026-0058 Window-scoped svc-authelia write role (already approved; created per admission window by net-kingdom tools/service-binds/executor-lanes.py, not here) done

The KeyCape client is SecurityCase nk-bind-executor-approval of type oidc-service-client 0.2 (acknowledged 2026-11-01). Its client secret is minted straight into OpenBao by the credential broker, delivered to KeyCape by ESO, and read natively by the executor through CCR-2026-0061's role.

Tools

  • scripts/credential-change.py approve | confirm-binding | applier-apply: approvals and the delegated metadata applier (policies and roles). Since RPF-WP-0069-T07, approve records each approval in turn; for workload-kv-read no hand-recorded second approval is needed.
  • scripts/credential.py generate --grant keycape/client-secret-generate: the broker mint (scripts/keycape_client_secret_generate.py, admits oidc-service-client@0.2 since RPF-WP-0069-T08). Silent, CAS 0, create-only, metadata-only verification, mode-0600 receipt.
  • scripts/nk_bind_executor_custody.py (new): plan (offline), fence (coding-agent deny for both custody paths, append-only on the live boundary) and verify (metadata, capabilities, login results and in-memory hashes only; never reads the KeyCape client secret). Silent; results in the receipt. Tests: tests/test_nk_bind_executor_custody.py.
  • audit-core scripts/attended-seed-nk-bounded-executor-sender.sh: mints the audit bearer CAS 0 into the CCR-2026-0063 path, merges the nk-bounded-executor registry row with compare-and-set, force-syncs audit-core-senders, restarts the receiver and proves the bearer (audit-core docs/bounded-executor-receipts.md, "Founder runbook").
  • manifests/nk-bind-executor-eso.yaml: two ClusterSecretStores (restricted to sso, audience openbao) and two ExternalSecrets: sso/keycape-nk-bind-executor-approval-client (key client-secret, rendered from manifests/templates/keycape-issuer-eso.yaml) and sso/bounded-executor-tokens (key audit-core, the Secret the rapp run Job mounts).

Gates owned elsewhere (before the steps that need them)

  • G1, before step 5 (mint). The founder accepts case nk-bind-executor-approval (owner and founder), and net-kingdom commits security-cases/nk-bind-executor-approval.yaml with status: materialized and acceptance: {at, record, by: [{role: owner}, {role: founder}]}. The broker refuses anything else (case_not_materialized, case_source_validation_failed); a synthetic copy with exactly that shape passed the broker's case check on 2026-11-04.
  • G2, before step 4 (audit seed). audit-core founder runbook steps 1 and 2: the ingress audit-core-nk-bounded-executor-ingress is applied and the live scope ConfigMap carries nk-bounded-executor.

Order relative to KeyCape

KeyCape resolves a confidential client's secret at start and fails closed when the binding is missing. Its Deployment runs one replica with strategy Recreate. So:

  1. platform steps 1 to 7 below finish first, and step 7 reports sso/keycape-nk-bind-executor-approval-client Ready=True;
  2. only then does net-kingdom patch Deployment sso/keycape with env KEYCAPE_NK_BIND_EXECUTOR_APPROVAL_CLIENT_SECRET from Secret keycape-nk-bind-executor-approval-client key client-secret (bind-env), and register client nk-bind-executor-approval (keycape-client.py apply), in one rollout, bind-env first;
  3. then the executor's request phase can exchange tokens.

If the ExternalSecret is not Ready when KeyCape restarts, the new pod cannot start and every client loses sign-in.

Step 0: session

cd /home/worsch/railiance-platform && git pull --ff-only
bridge up openbao-ui-railiance01
mkdir -p -m 700 /home/worsch/.local/state/railiance/receipts
python3 scripts/nk_bind_executor_custody.py plan
python3 -m pytest -q tests/test_nk_bind_executor_custody.py tests/test_keycape_client_secret_generate.py

Expected: the plan lists the three CCRs with the roles, stores and ExternalSecrets above; tests pass.

Step 1: approvals and binding confirmations

cd /home/worsch/railiance-platform
python3 scripts/credential-change.py approve CCR-2026-0060 --reviewer platform-owner --comment "Approved by the founder 2026-11-04: KeyCape ESO reader for SecurityCase nk-bind-executor-approval (oidc-service-client 0.2), NK-WP-0056-T06."
python3 scripts/credential-change.py confirm-binding CCR-2026-0060 --reviewer platform-owner --comment "Binding confirmed: external-secrets/external-secrets, audience openbao, as CCR-2026-0033/0041."
python3 scripts/credential-change.py approve CCR-2026-0061 --reviewer platform-owner --comment "Approved by the founder 2026-11-04: executor reader of the nk-bind-executor-approval client secret, NK-WP-0056-T06."
python3 scripts/credential-change.py approve CCR-2026-0061 --reviewer net-kingdom-owner --comment "Approved by the founder 2026-11-04 for NK-WP-0056-T06."
python3 scripts/credential-change.py confirm-binding CCR-2026-0061 --reviewer platform-owner --comment "Binding confirmed: ServiceAccount nk-bind-executor in namespace sso only (uid bd15569c-8a85-4625-b001-d9f799cb857b), audience openbao, from rapp-bounded-executor."
python3 scripts/credential-change.py approve CCR-2026-0063 --reviewer platform-owner --comment "Approved by the founder 2026-11-04: bounded-executor audit sender custody, ESO to sso/bounded-executor-tokens, NK-WP-0056-T15/T06."
python3 scripts/credential-change.py approve CCR-2026-0063 --reviewer audit-core-owner --comment "Approved by the founder 2026-11-04: sender nk-bounded-executor as registered in AUDIT-WP-0018-T02."
python3 scripts/credential-change.py approve CCR-2026-0063 --reviewer net-kingdom-owner --comment "Approved by the founder 2026-11-04: ESO as drafted (NetKingdom answer recorded 2026-11-04)."
python3 scripts/credential-change.py confirm-binding CCR-2026-0063 --reviewer platform-owner --comment "Binding confirmed: external-secrets/external-secrets, audience openbao; store restricted to namespace sso."
for c in CCR-2026-0060 CCR-2026-0061 CCR-2026-0063; do python3 scripts/credential-change.py validate $c; python3 scripts/credential-change.py applier-dry-run $c | tail -3; done
git add credential-change-requests/CCR-2026-0060-*.yaml credential-change-requests/CCR-2026-0061-*.yaml credential-change-requests/CCR-2026-0063-*.yaml
git commit -m "CCR-2026-0060/0061/0063: founder approvals and binding confirmations (NK-WP-0056-T06)"
git push origin main

Expected: [OK] with no "bound claim is not confirmed" warning; each dry-run ends with (matches). The CCRs must be committed before step 2: nk_bind_executor_custody.py refuses uncommitted CCRs.

Step 2: coding-agent fence (before any role exists)

cd /home/worsch/railiance-platform
BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200 warden access openbao-platform-admin-login --exec -- python3 /home/worsch/railiance-platform/scripts/nk_bind_executor_custody.py fence --receipt /home/worsch/.local/state/railiance/receipts/nk-bind-executor-fence-r1.json
python3 -c 'import json; r=json.load(open("/home/worsch/.local/state/railiance/receipts/nk-bind-executor-fence-r1.json")); print(r["status"], r.get("error"), r.get("fence_lines_added"))'

Expected: fenced None 2 (the audit-sender lines may already be live; then fewer, never a rewrite of existing lines). A rerun needs a new receipt name (-r2.json).

Step 3: policies and roles (delegated applier)

cd /home/worsch/railiance-platform
BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200 warden access openbao-platform-admin-login --exec -- python3 /home/worsch/railiance-platform/scripts/credential-change.py applier-apply CCR-2026-0060 --confirm "DELEGATED APPLY CCR-2026-0060" --actor founder --record-state-hub --quiet
BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200 warden access openbao-platform-admin-login --exec -- python3 /home/worsch/railiance-platform/scripts/credential-change.py applier-apply CCR-2026-0061 --confirm "DELEGATED APPLY CCR-2026-0061" --actor founder --record-state-hub --quiet
BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200 warden access openbao-platform-admin-login --exec -- python3 /home/worsch/railiance-platform/scripts/credential-change.py applier-apply CCR-2026-0063 --confirm "DELEGATED APPLY CCR-2026-0063" --actor founder --record-state-hub --quiet
git add credential-change-requests/CCR-2026-0060-*.yaml credential-change-requests/CCR-2026-0061-*.yaml credential-change-requests/CCR-2026-0063-*.yaml
git commit -m "CCR-2026-0060/0061/0063: delegated metadata apply (NK-WP-0056-T06)"
git push origin main

Expected: each CCR moves to applied with a delegated_metadata_apply evidence entry; the applier reads back the policy and role and refuses on drift.

Step 4: audit sender seed (audit-core's founder runbook, step 3; needs G2)

BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200 warden access openbao-platform-admin-login --exec -- /home/worsch/audit-core/scripts/attended-seed-nk-bounded-executor-sender.sh /home/worsch/audit-core/docs/evidence/nk-bounded-executor-sender-seed-r1.json
python3 -c 'import json; r=json.load(open("/home/worsch/audit-core/docs/evidence/nk-bounded-executor-sender-seed-r1.json")); print(r["status"], r.get("error"), r.get("receiver_probe"), r.get("registry_secret_hash_match"), r["credential_values_emitted"])'

Expected: seeded_pending_sso_delivery None {"read": 403, "own_tenant": 200, "other_tenant": 403, "other_source": 403} True False. The seed refuses unless CCR-2026-0063 carries all three approvals and the fence from step 2 is live.

Step 5: mint the KeyCape client secret into custody (needs G1)

The caller is the founder's own OIDC login to the broker issuer role (net-kingdom-admins + aal2, CCR-2026-0043), not the platform-admin envelope.

cd /home/worsch/railiance-platform
export BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200
unset BAO_TOKEN VAULT_TOKEN
CASE=/home/worsch/net-kingdom/security-cases/nk-bind-executor-approval.yaml
grep -q '^  status: materialized$' "$CASE" && echo case-materialized
REF="nk-bind-executor-approval@sha256:$(sha256sum "$CASE" | cut -d' ' -f1)"
bao login -no-print -method=oidc -path=netkingdom role=credential-broker-keycape-client-secret-generate
python3 scripts/credential.py generate --grant keycape/client-secret-generate --client-id nk-bind-executor-approval --case-file "$CASE" --case-ref "$REF" --dry-run --receipt /home/worsch/.local/state/railiance/receipts/nk-bind-executor-approval-mint-dry.json
python3 scripts/credential.py generate --grant keycape/client-secret-generate --client-id nk-bind-executor-approval --case-file "$CASE" --case-ref "$REF" --record-state-hub --receipt /home/worsch/.local/state/railiance/receipts/nk-bind-executor-approval-mint-1.json
bao token revoke -self
python3 -c 'import json; [print(r["outcome"], r["exit_code"], r.get("kv_version"), r.get("grant_token_revoked"), r["credential_values_emitted"]) for r in (json.load(open("/home/worsch/.local/state/railiance/receipts/nk-bind-executor-approval-mint-"+n+".json")) for n in ("dry","1"))]'

Expected: case-materialized; then dry_run_would_generate 0 None True False and generated 0 1 True False. Exit 3 (already_present) means an entry exists and was not touched; stop and reconcile, never rerun with another tool. No value is printed or returned.

Step 6: ESO delivery to sso

cd /home/worsch/railiance-platform
KUBECONFIG=$HOME/.kube/config-railiance01 kubectl apply --dry-run=server -f manifests/nk-bind-executor-eso.yaml
KUBECONFIG=$HOME/.kube/config-railiance01 kubectl apply -f manifests/nk-bind-executor-eso.yaml
KUBECONFIG=$HOME/.kube/config-railiance01 kubectl wait --for=condition=Ready clustersecretstore/openbao-keycape-nk-bind-executor-approval-issuer clustersecretstore/openbao-nk-bind-executor-audit --timeout=120s
KUBECONFIG=$HOME/.kube/config-railiance01 kubectl -n sso wait --for=condition=Ready externalsecret/keycape-nk-bind-executor-approval-client externalsecret/bounded-executor-tokens --timeout=180s

Expected: four objects created; both stores and both ExternalSecrets condition met. These are status reads only.

Step 7: verify (metadata, capabilities, logins and hashes only)

cd /home/worsch/railiance-platform
KUBECONFIG=$HOME/.kube/config-railiance01 BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200 warden access openbao-platform-admin-login --exec -- python3 /home/worsch/railiance-platform/scripts/nk_bind_executor_custody.py verify --kubeconfig /home/worsch/.kube/config-railiance01 --receipt /home/worsch/.local/state/railiance/receipts/nk-bind-executor-verify-r1.json
python3 -c 'import json; r=json.load(open("/home/worsch/.local/state/railiance/receipts/nk-bind-executor-verify-r1.json")); print(r["status"], r.get("error"), r.get("phase"), r.get("keycape_client_secret_kv_version"), r.get("audit_sender_kv_version"), r.get("audit_copies_agree_by_hash"), r.get("keycape_client_secret_value_read"))'
BAO_ADDR=http://127.0.0.1:18200 VAULT_ADDR=http://127.0.0.1:18200 warden access openbao-platform-admin-login --exec -- /home/worsch/audit-core/scripts/attended-seed-nk-bounded-executor-sender.sh /home/worsch/audit-core/docs/evidence/nk-bounded-executor-sender-seed-r2.json

Expected: custody_verified_pending_keycape_and_executor None done 1 1 True False. The verify proves:

  • policies and roles equal the CCR-generated bodies; the coding-agent fence beats each lane policy; a default-only token cannot read either path;
  • custody metadata: both paths at a live version 1;
  • sso/nk-bind-executor logs in to its role with exactly one policy and a lease of at most 15m, reads only its path (sibling KeyCape clients, the audit path, the svc-authelia bind, metadata and auth/token/create denied); the default ServiceAccount, a wrong audience and the two ESO roles are refused; kubectl auth can-i get secrets -n sso as the executor answers no;
  • both stores and ExternalSecrets are Ready with the exact role, audience and one field; the delivered Secrets hold exactly one key each;
  • the audit bearer in custody, in sso/bounded-executor-tokens and in the registry row agree by SHA-256 in memory; the registry row equals the declared scope.

The KeyCape client secret is never read: its delivery is proven by ExternalSecret status and the executor's own first token exchange (type verification service-token-claims, net-kingdom).

The audit-core rerun (-r2) writes nothing; its sso hash comparison looks for the older Secret name nk-bind-executor-audit and is skipped until audit-core aligns it to bounded-executor-tokens/audit-core (requested 2026-11-04). Step 7's hash check covers it either way.

Then (not platform)

  • net-kingdom: bind-env and register-client in one KeyCape rollout (see "Order relative to KeyCape"), then service-token-claims and excess-scope-refused from the executor.
  • net-kingdom: the window-scoped CCR-2026-0058 role per admission (executor-lanes.py), the founder's admission, then the pilot run (NK-WP-0056-T06).
  • railiance-platform: record the receipts as CCR evidence (credential-change.py record-evidence) and move the three CCRs to verified after the first executor run.

Rollback

  • Executor reader: delete role keycape-nk-bind-executor-approval-bounded-executor-workload-kv-read, then its policy.
  • KeyCape delivery: only after net-kingdom has removed the client and the env binding, delete ExternalSecret sso/keycape-nk-bind-executor-approval-client, the store, role external-secrets-keycape-nk-bind-executor-approval and its policy; then the custody metadata.
  • Audit sender: stop the executor, remove only its registry row with CAS and reload audit-core, then delete sso/bounded-executor-tokens, its store, role and policy; keep the KV versions.
  • The coding-agent fence lines stay.