the-custodian/workplans/CUST-WP-0073-agent-credential-separation.md

---
id: CUST-WP-0073
type: workplan
title: "Separate agent credentials and establish supervised privileged execution"
domain: infotech
repo: the-custodian
status: active
owner: the-custodian
topic_slug: custodian
flavor: implementation
created: "2026-09-24"
updated: "2026-09-28"
related:
  - KEY-WP-0033
  - RPF-WP-0044
origin: incident
origin_ref: key-cape/docs/operations.md#before-any-live-change
state_hub_workstream_id: "a98a9f34-83b4-5c8c-8107-e05f7806d50d"
---

# Separate agent credentials and establish supervised privileged execution

## Why

On 2026-09-23 a key-cape agent session meant to read only Secret metadata.
The go-template failed on an absent field, and kubectl printed
`sso/keycape-config` in full as debugging output: the signing key, the LLDAP
bind password and the Authelia client secret. Three more Secrets in `sso`/`mfa`
were probably printed as well.

The root cause is the credentials, not the command:

- Workstation `kubectl` and `ssh railiance01` both authenticate as
  `system:admin` in `system:masters`, which bypasses authorization. No RBAC
  rule can restrict it.
- k3s runs with `--write-kubeconfig-mode=644`, so `/etc/rancher/k3s/k3s.yaml`
  is world-readable, and anything running as `tegwick` is cluster-admin.
- The paths that leak a value are open-ended: template error dumps,
  `-o yaml`/`describe`, `helm get`, `kubectl logs`, `exec … env`, ConfigMaps
  holding credentials, last-applied annotations, and the kubeconfig file itself.
  A command denylist chases them one by one, and it only binds the harness that
  enforces it.

**Goal:** keep privileged credentials outside the agent's direct reach, and
mediate privileged actions according to the identified agent's autonomy mode.
Agents start supervised; proven agents may later receive scoped autopilot
permission with a cost budget and risk limit in EUR. Neither mode implies
unrestricted admin credentials. Founder decision, September 28:
`docs/agent-autonomy-decision.md`.

**Scope:** builder mode, founder decision 2026-09-24. Rotating the exposed
Secrets is deferred, not dropped (T05). This plan removes the problem class.

## Decide the identity model

```task
id: CUST-WP-0073-T01
status: done
priority: high
state_hub_task_id: "4781bb99-020f-59f6-be13-e3b8777d3fd8"
```

**Done 2026-09-28 — policy decision.** The founder chooses autonomy as a
characteristic of the agent: supervised-mode initially, promotion only on
successful proposals without adaptation/refinement, and autopilot bounded by
cost and risk limits in EUR. Decision: `docs/agent-autonomy-decision.md`.

The supervised starting identity is outside `system:masters` and has a reviewed
observation allowlist. Exact privileged actions go through supervisor approval
or supervisor execution, with unchanged-acceptance and verified unchanged-success
recorded separately. Admin credentials remain in the privileged execution path,
outside the agent's direct reach. Later autopilot removes per-action approval
only for an explicit grant within scope, cost and risk constraints; it does not
hand out unrestricted sudo or an admin kubeconfig.

Built-in `view` plus Deployment/ConfigMap writes cannot establish that boundary:
workload writes can extract credentials, and ConfigMaps/pod specs/logs can contain
values. Agent-visible results must be sanitized. T02 selects and proves the
actual account/profile/execution path. This decision resolves the identity and
autonomy policy, not the implementation, numeric promotion thresholds, euro
limits or authorization of a live cutover. Existing human-only lanes still apply.

## Build and hand out the agent identity

```task
id: CUST-WP-0073-T02
status: progress
priority: high
state_hub_task_id: "4b88b0b7-dc7e-5612-aa5d-1a08f0530c35"
```

Owner: railiance-platform (RBAC), railiance-enablement (k3s install),
railiance-infra (host principal mapping), ops-warden (certificate issuance).

- Bind the agent's stable identity and `supervised-mode` to its existing
  instance/assignment record and a restricted runtime profile. Name its
  supervisor and privileged execution path. No raw admin credentials in the
  agent process/account; no unrestricted sudo, socket or GitOps bypass.
- Create the ServiceAccount or client certificate, the ClusterRole and the
  binding in git, applied by the owner's documented path.
- Set `write-kubeconfig-mode` to `600` in the k3s install config. Attended admin
  use becomes `sudo k3s kubectl`.
- Workstation: `~/.kube/config` points at the agent kubeconfig. The admin
  kubeconfig moves out of the agent's reach (another account, or short-lived
  issuance through an attended login).
- Proof: as the agent identity, `kubectl auth can-i get secrets -A` and
  `can-i create pods/exec -A` both answer `no`, and `kubectl auth whoami`
  shows no `system:masters`. Record the output as evidence. Also verify that
  an exact supervisor-approved action can execute through the selected privileged
  path and return sanitized outcome evidence, while an unapproved/revised action
  cannot use that approval. Test from the actual agent account, including denial
  of the old admin SSH/sudo/credential paths.
- Autopilot remains disabled without a scoped promotion decision, concrete EUR
  cost/risk limits and verified enforcement. Implementing a general promotion or
  risk-scoring service is not required for this supervised credential boundary.

## Reject last-applied annotations on Secrets

```task
id: CUST-WP-0073-T03
status: done
priority: medium
needs_human: false
state_hub_task_id: "5abbfcae-eb65-5b62-a7fa-f4f698f95312"
```

Owner: railiance-platform.

**Done 2026-09-28.** Explicit metadata fixes landed in all 31 affected ESO
declarations. The guard is active and Synced/Healthy; all nine native admission
checks and 39/39 fresh ESO refreshes under Deny pass. All 257 active-namespace
Secrets were annotation-free. After explicit founder approval, the one orphan
drill Secret was deleted with its UID precondition; absence verified and the
3 GiB PVC unchanged. Full evidence: `docs/evidence/2026-09-28-secret-annotation-rollout.md`.

- Add a `ValidatingAdmissionPolicy` (v1.35 is available) with its binding. It
  rejects any Secret carrying `kubectl.kubernetes.io/last-applied-configuration`.
- Strip the annotation from existing Secrets first, cluster-wide, using the
  output-suppressing maintenance helper, or the policy blocks their next update. Known:
  `sso/keycape-config`, `sso/authelia-secrets`, `sso/lldap-secrets`,
  `mfa/privacyidea-config`.
- Proof: a client-side `kubectl apply` of a test Secret in a scratch namespace
  is refused.

## Fleet guidance and harness guards

```task
id: CUST-WP-0073-T04
status: progress
priority: medium
state_hub_task_id: "90725511-4e31-549f-b567-47feff1a9ca4"
```

- Orientation doc §6 documents the template-error dump and requires the safe
  maintenance helper; standalone Secret templates/jsonpath are forbidden. The
  September 28 standing notice supersedes the September 24 inline-check exception.
  Add the verified agent identity/execution path once T02 lands.
- Claude Code guard, **done on the workstation 2026-09-24**:
  `~/.claude/hooks/guard-secret-reads.py` (PreToolUse on Bash). It denies Secret
  reads, `helm get`, `config view --raw` and kubeconfig reads, and asks before
  exec/attach/cp/debug and `--raw`. The `kubectl * get secret *` allow rule and
  the expired 2026-07-07 temporary elevation are removed. Known false positive:
  any command naming both kubectl and "secret", including an `echo`. That is
  acceptable for a stopgap.
- Offer the same guard to the Codex and Grok harnesses, or record that they have
  none. Until T02 lands, those agents are protected by instructions only.
- OpenBao/Vault secret-reading permissions also belong behind the privileged
  boundary; preapproved command prefixes do not implement supervision.
- Document the agent-specific mode and supervisor. Reference exact proposal and
  execution receipts; track unchanged acceptance separately from verified
  unchanged success, including refinements, rejections and interventions. Reuse
  existing records and receipts per `docs/agent-autonomy-decision.md`; no new
  dashboard, supervisor service or automatic promotion machinery.

## Rotate what was exposed

```task
id: CUST-WP-0073-T05
status: wait
priority: low
state_hub_task_id: "1b41b532-38a2-5f39-b387-28737934811a"
```

Deferred by founder decision on 2026-09-24 (builder mode). Rotate
`sso/keycape-config` (signing key, LLDAP bind password, Authelia client secret),
`sso/authelia-secrets`, `sso/lldap-secrets` and `mfa/privacyidea-config`
together, with `replace` rather than `apply`, in one attended window.

Reopen on the first of:

- the first production workload or customer data passing through KeyCape
- any sign the 2026-09-23 transcript left the workstation beyond the model
  provider
- a planned key rotation

The passage of time alone reopens nothing.

## Current evidence and handoff — September 28

The founder requests minimal additional tasks, workplans and functionality.
All remaining work stays in T02, T04 and the original deferred T05. No new
workplan, task, controller or supervisor service was introduced.

T01 is done: autonomy is per agent and scope, supervised initially, with later
promotion bounded by EUR cost and risk limits. The decision and descriptive
per-agent record do not establish runtime enforcement or grant autopilot.

T02 remains in progress. The existing sand-boxer `profile.bwrap-local` passed a
synthetic process-isolation proof: admin homes, privileged environment variables,
Kubernetes/container sockets absent; only loopback networking; observation and
proposal paths work; wrong consumer rejected; workspace destroyed. Receipt:
`docs/evidence/2026-09-28-supervised-sandbox-proof.json`. It was not an interactive
agent migration. The selected GLAS local-profile acceptance remains blocked;
existing coordination receipt: `docs/evidence/2026-09-28-supervised-runtime-coordination.json`.
Actual account/profile admission and denial of old admin SSH/sudo/credential
paths are still required. The inspected legacy account has unrestricted sudo
and k3s kubeconfig mode 644; changing its default kubeconfig alone is insufficient.

T03 is done. All 31 affected ESO declarations have explicit target metadata in
23 files across 12 owning repositories, committed and published. The guard is
active and Synced/Healthy at platform source `7daf7e9`, pinned by `db51ec8`.
Nine native admission checks and 39/39 fresh ESO refreshes under Deny passed.
The founder-approved UID-bound deletion removed only the orphan drill Secret;
the 3 GiB PVC was unchanged. The final complete scan checked 257 Secrets with
zero forbidden annotations and zero orphan exceptions. The earlier failed
rollout and rollback remain historical evidence in
`docs/evidence/2026-09-28-secret-annotation-rollout.md`; they are not the live state.

T04's local guidance and reporting are implemented. AGENTS.md now loads the
supervision decision and per-agent record at startup and describes recording
original proposals before approval and verified outcomes afterwards. The
September 28 standing notice withdraws the unsafe inline presence check:
51e9eace-06d2-46d6-921a-4b92440df68f. Receipt: `docs/evidence/2026-09-28-secret-guidance-notice.json`.
Codex/Grok have no equivalent read-denial hook established by this work. The
remaining T04 step is to document the actual verified T02 execution path.
The original rollout remains an unscored failed proposal with refinement and
recovery; successful remediation does not become unchanged-success credit.
There is no eligible acceptance-rate sample or autopilot grant yet.

T05 remains the founder's trigger-based rotation deferral, not cancelled or done.
Neither workplan completion nor live credential separation is claimed.