net-kingdom/sso-mfa/k8s/bounded-executor/T06-PILOT-RUNBOOK.md

# T06 runbook: the live pilot rotation under one admission (founder-run)

NK-WP-0056-T06. The executor rotates `svc-authelia`'s LLDAP bind. Your only part *in the run* is the
admission: you approve it at aal2 in informed-decision. Everything before that (sections 1 to 4) is one-time
go-live of the standing paths. Section 6 is the run.

Each step prints a value-free result. Paste it back, and claude-code verifies it read-only and records it.

**Commands:**

- Admin steps use `K="kubectl --kubeconfig $HOME/.kube/config-railiance01"`.
- OpenBao steps need `bao login -method=oidc -path=netkingdom role=platform-admin` (MFA) first.
- Always run `git pull --ff-only` in a repo before using it.

| # | What | Where | Changes |
| --- | --- | --- | --- |
| 1 | Owner ingress (T16) | approval-engine, audit-core, rapp | two NetworkPolicies; read-only netcheck |
| 2 | Accept the case | State Hub decision | `security-cases/nk-bind-executor-approval.yaml` becomes `accepted` |
| 3 | Custody and audit sender | railiance-platform, audit-core | approve CCR-2026-0060, -0061, -0063; fence, policies, roles; audit seed; mint; ESO |
| 4 | KeyCape client | net-kingdom | bind the env var and register the client in one KeyCape restart |
| 5 | Admission | net-kingdom (claude-code) | `admissions/BEA-2026-0002.yaml` for the window you choose |
| 6 | The run | rapp, net-kingdom | request, approve at aal2, open the window, run, close the window, verify |

## 1. Owner ingress (T16)

Apply each owner's policy object on its own. Live approval-engine also has `informed-decision-review-ingress`, which
is not in its file. Applying the whole file would therefore be a broader change than this step.

```bash
K="kubectl --kubeconfig $HOME/.kube/config-railiance01"
cd ~/approval-engine && git pull --ff-only
python3 -c "import yaml,sys; yaml.safe_dump([d for d in yaml.safe_load_all(open('deploy/networkpolicies.yaml')) if d and d['metadata']['name']=='approval-engine-bounded-executor'][0], sys.stdout)" > /tmp/ae-bx.yaml
$K apply --dry-run=server -f /tmp/ae-bx.yaml && $K apply -f /tmp/ae-bx.yaml
```

audit-core: run steps 1 and 2 of audit-core `docs/bounded-executor-receipts.md`, "Founder runbook —
nk-bounded-executor sender". Step 1 applies `audit-core-nk-bounded-executor-ingress` the same way. Step 2 is a
read-only preflight, which also checks that the live scope ConfigMap carries `nk-bounded-executor`. This is
railiance-platform's gate G2 before the audit seed.

Then prove the path from an executor-labelled pod. It holds no token and makes no login:

```bash
cd ~/rapp-bounded-executor && git pull --ff-only
make netcheck KUBECTL="$K"
```

Expected, on the last lines: `"approval_engine_health": 200`, `"audit_core_health": 200` and
`"informed_decision_tcp": "ok"`, alongside the existing OpenBao, KeyCape and Authelia columns.

## 2. Accept the service-client case

Answer State Hub decision `59d570ef` ("Accept SecurityCase nk-bind-executor-approval"). The case type requires
owner and founder acceptance; you are both. claude-code then commits the case as `accepted`, with the acceptance
record.

## 3. Custody and audit sender (railiance-platform, audit-core)

Follow railiance-platform `docs/credential-lane-designs/nk-bind-executor-go-live.md`, steps 0 to 7, in order:

- **Steps 0 to 3:** session; approve CCR-2026-0060, -0061 and -0063 and confirm their bindings; the coding-agent
  fence; policies and roles.
- **Then tell claude-code.** It commits the case as `materialized`: its OpenBao structure now exists. The broker
  mints only for a materialized, accepted case (gate G1). The case's hash changes with that commit, and step 5
  derives `REF` from the committed file. So `git pull --ff-only` in net-kingdom before step 5.
- **Step 4:** the audit seed, which is step 3 of audit-core's founder runbook. It needs gate G2 from section 1.
- **Step 5:** mint the KeyCape client secret through the broker. No value is printed.
- **Step 6:** ESO delivery to `sso`: KeyCape's `keycape-nk-bind-executor-approval-client`, and the executor's
  `bounded-executor-tokens` key `audit-core`.
- **Step 7:** `verify`, which checks metadata, capabilities, logins and hashes only.

Section 4 needs step 7's `Ready=True` for `keycape-nk-bind-executor-approval-client` immediately before it runs.

## 4. KeyCape: bind and register in one restart

KeyCape runs one replica with strategy Recreate. Changing its env is itself a restart, and registering a confidential
client whose secret is not bound makes the next start fail closed. So: pause the rollout, bind, register, then resume.
That is one restart, with both changes (key-cape R4). Run it in a quiet minute: in-flight sign-ins are dropped.

```bash
cd ~/net-kingdom && git pull --ff-only
K="kubectl --kubeconfig $HOME/.kube/config-railiance01"
$K -n sso get externalsecret keycape-nk-bind-executor-approval-client -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'; echo   # True

C=$(mktemp); python3 sso-mfa/k8s/keycape/keycape-client.py client-from-case security-cases/nk-bind-executor-approval.yaml > "$C"
B=$HOME/.local/share/key-cape/verified-bin/e2b5fa1     # check-config binary from the deployed image (T09 runbook)
test -x "$B/keycape" || { mkdir -p "$B"; cid=$(docker create forgejo.coulomb.social/coulomb/key-cape@sha256:9e082dca78f65a4af3e92d3de3705b99e49ca9b298d9562ad0d37511c4306f8d); docker cp "$cid:/keycape" "$B/keycape"; docker rm "$cid"; }

# keycape-client.py uses its built-in kubeconfig through 127.0.0.1:16444, exactly as in the T09 runbook;
# it pins the cluster identity and refuses another cluster.
$K -n sso rollout pause deployment/keycape
$K -n sso patch deployment keycape --type=json -p '[
  {"op":"test","path":"/spec/template/spec/containers/0/name","value":"keycape"},
  {"op":"add","path":"/spec/template/spec/containers/0/env/-","value":{"name":"KEYCAPE_NK_BIND_EXECUTOR_APPROVAL_CLIENT_SECRET","valueFrom":{"secretKeyRef":{"name":"keycape-nk-bind-executor-approval-client","key":"client-secret"}}}}]'
python3 sso-mfa/k8s/keycape/keycape-client.py inspect --client-file "$C"
#   -> "deployment_env_bound": true, "change_needed": true; note before.uid and before.resource_version
```

Then put the noted values into the apply line, in place of `UID` and `RV`:

```bash
python3 sso-mfa/k8s/keycape/keycape-client.py apply --client-file "$C" \
  --case-acceptance "security-cases/nk-bind-executor-approval.yaml (accepted, hub decision 59d570ef)" \
  --check-config "$B/keycape" --expected-uid UID --expected-resource-version RV
$K -n sso rollout resume deployment/keycape
$K -n sso rollout status deployment/keycape --timeout=180s
```

If `apply` refuses, nothing in the registry changed. Undo the env patch before resuming: run
`$K -n sso rollout undo deployment/keycape`, then `$K -n sso rollout resume deployment/keycape`, and paste the
refusal.

claude-code then runs the type's verification. The executor's request Job (section 6) performs the token exchange
per scope. The excess-scope refusal, the custody denials and the RBAC can-i denial are read-only checks.

## 5. Admission (claude-code prepares, you check)

claude-code runs, for the window you name (at most 2 h):

```bash
python3 bounded-executor/prepare_admission.py --id BEA-2026-0002 --commit 9bd7bf0 \
  --not-before 2026-11-05T09:00:00Z --hours 2 --out admissions/BEA-2026-0002.yaml
```

It commits the file. The pins are the files inside the pinned image (`sha256:2926d7d9…`, built from `9bd7bf0`).
Read it: the plan, the checks, the revert steps, the budget and the window are what you will approve.

## 6. The run

```bash
K="kubectl --kubeconfig $HOME/.kube/config-railiance01"
cd ~/rapp-bounded-executor && git pull --ff-only && cd ~/net-kingdom && git pull --ff-only
A=~/net-kingdom/admissions/BEA-2026-0002.yaml
CFG=$(mktemp --suffix=.json)
python3 -c "import json; c=json.load(open('$HOME/rapp-bounded-executor/examples/config.pilot.json')); c['trust_satisfied']=['runtime_secret_trust','runtime_identity_trust']; json.dump(c, open('$CFG','w'), indent=2)"
#   trust_satisfied is your attestation, recorded in the run's result: OpenBao custody and KeyCape identity are healthy
```

**6a. Request.** The executor files the approval as `service:nk-bind-executor` and presents it in informed-decision:

```bash
cd ~/rapp-bounded-executor && make request ADMISSION=$A CONFIG=$CFG KUBECTL="$K"
#   -> {"outcome": "presented", "approval_id": "...", ...}; note the approval_id
```

**6b. Approve.** Open the memo in informed-decision and approve it. You must be signed in at aal2 (OTP in this login).

**6c. Open the window.** This creates CCR-2026-0058's write role for this admission only:

```bash
cd ~/net-kingdom && bao login -method=oidc -path=netkingdom role=platform-admin
W=$(sha256sum ~/railiance-platform/credential-change-requests/CCR-2026-0058-*.yaml | cut -d' ' -f1)
J=$(sha256sum ~/railiance-platform/credential-change-requests/CCR-2026-0059-*.yaml | cut -d' ' -f1)
E=docs/evidence/$(date +%F)-nk56-t06-pilot; mkdir -p "$E"
python3 tools/service-binds/executor-lanes.py window-open --admission $A --write-ccr-sha256 "$W" \
  --journey-ccr-sha256 "$J" --receipt "$E/01-window-open.json"            # {"status": "window_opened", ...}
```

**6d. Run.** Put the approval id from 6a in place of `APPROVAL`:

```bash
cd ~/rapp-bounded-executor && make run ADMISSION=$A APPROVAL_ID=APPROVAL CONFIG=$CFG KUBECTL="$K"
make run-status ADMISSION_ID=BEA-2026-0002 KUBECTL="$K"
$K -n sso logs job/bounded-executor-bea-2026-0002 > ~/net-kingdom/$E/02-result.json
```

The last line of the log is the value-free result:

| Exit | Result | Meaning |
| --- | --- | --- |
| 0 | `completed` | rotated, delivered, restarted, journey passed |
| 3 | `reverted` | a check failed, and the previous value is back, checked |
| 4 | `refused` | nothing was spent |
| 5 | `stopped`, `unknown` or `revert_failed` | the alert names the step |
| 6 | consumed but not in custody | — |

**Kill switch**, at any moment of the run:

```bash
$K -n sso delete job bounded-executor-bea-2026-0002
```

The executor finishes the step in flight, then reverts what it changed (grace period = the budget). Before consume
it refuses and spends nothing.

**6e. Close the window** right after the result, or at window end at the latest:

```bash
cd ~/net-kingdom && python3 tools/service-binds/executor-lanes.py window-close --admission $A \
  --receipt "$E/03-window-close.json"                                      # {"status": "window_closed", ...}
```

**6f. Tell claude-code.** It does the rest read-only:

- it confirms the run's started event and receipt in audit-core (operator read by `correlation_id`);
- it validates the pair against receipt schema v0.1;
- it runs the posture-feedback report over the receipts (T07);
- it records the evidence and closes T06.