--- id: CUST-WP-0073 type: workplan title: "Separate agent credentials and establish supervised privileged execution" domain: infotech repo: the-custodian status: active owner: the-custodian topic_slug: custodian flavor: implementation created: "2026-09-24" updated: "2026-09-28" related: - KEY-WP-0033 - RPF-WP-0044 origin: incident origin_ref: key-cape/docs/operations.md#before-any-live-change state_hub_workstream_id: "a98a9f34-83b4-5c8c-8107-e05f7806d50d" --- # Separate agent credentials and establish supervised privileged execution ## Why On 2026-09-23 a key-cape agent session meant to read only Secret metadata. The go-template failed on an absent field, and kubectl printed `sso/keycape-config` in full as debugging output: the signing key, the LLDAP bind password and the Authelia client secret. Three more Secrets in `sso`/`mfa` were probably printed as well. The root cause is the credentials, not the command: - Workstation `kubectl` and `ssh railiance01` both authenticate as `system:admin` in `system:masters`, which bypasses authorization. No RBAC rule can restrict it. - k3s runs with `--write-kubeconfig-mode=644`, so `/etc/rancher/k3s/k3s.yaml` is world-readable, and anything running as `tegwick` is cluster-admin. - The paths that leak a value are open-ended: template error dumps, `-o yaml`/`describe`, `helm get`, `kubectl logs`, `exec … env`, ConfigMaps holding credentials, last-applied annotations, and the kubeconfig file itself. A command denylist chases them one by one, and it only binds the harness that enforces it. **Goal:** keep privileged credentials outside the agent's direct reach, and mediate privileged actions according to the identified agent's autonomy mode. Agents start supervised; proven agents may later receive scoped autopilot permission with a cost budget and risk limit in EUR. Neither mode implies unrestricted admin credentials. Founder decision, September 28: `docs/agent-autonomy-decision.md`. **Scope:** builder mode, founder decision 2026-09-24. Rotating the exposed Secrets is deferred, not dropped (T05). This plan removes the problem class. ## Decide the identity model ```task id: CUST-WP-0073-T01 status: done priority: high state_hub_task_id: "4781bb99-020f-59f6-be13-e3b8777d3fd8" ``` **Done 2026-09-28 — policy decision.** The founder chooses autonomy as a characteristic of the agent: supervised-mode initially, promotion only on successful proposals without adaptation/refinement, and autopilot bounded by cost and risk limits in EUR. Decision: `docs/agent-autonomy-decision.md`. The supervised starting identity is outside `system:masters` and has a reviewed observation allowlist. Exact privileged actions go through supervisor approval or supervisor execution, with unchanged-acceptance and verified unchanged-success recorded separately. Admin credentials remain in the privileged execution path, outside the agent's direct reach. Later autopilot removes per-action approval only for an explicit grant within scope, cost and risk constraints; it does not hand out unrestricted sudo or an admin kubeconfig. Built-in `view` plus Deployment/ConfigMap writes cannot establish that boundary: workload writes can extract credentials, and ConfigMaps/pod specs/logs can contain values. Agent-visible results must be sanitized. T02 selects and proves the actual account/profile/execution path. This decision resolves the identity and autonomy policy, not the implementation, numeric promotion thresholds, euro limits or authorization of a live cutover. Existing human-only lanes still apply. ## Build and hand out the agent identity ```task id: CUST-WP-0073-T02 status: progress priority: high state_hub_task_id: "4b88b0b7-dc7e-5612-aa5d-1a08f0530c35" ``` Owner: railiance-platform (RBAC), railiance-enablement (k3s install), railiance-infra (host principal mapping), ops-warden (certificate issuance). - Bind the agent's stable identity and `supervised-mode` to its existing instance/assignment record and a restricted runtime profile. Name its supervisor and privileged execution path. No raw admin credentials in the agent process/account; no unrestricted sudo, socket or GitOps bypass. - Create the ServiceAccount or client certificate, the ClusterRole and the binding in git, applied by the owner's documented path. - Set `write-kubeconfig-mode` to `600` in the k3s install config. Attended admin use becomes `sudo k3s kubectl`. - Workstation: `~/.kube/config` points at the agent kubeconfig. The admin kubeconfig moves out of the agent's reach (another account, or short-lived issuance through an attended login). - Proof: as the agent identity, `kubectl auth can-i get secrets -A` and `can-i create pods/exec -A` both answer `no`, and `kubectl auth whoami` shows no `system:masters`. Record the output as evidence. Also verify that an exact supervisor-approved action can execute through the selected privileged path and return sanitized outcome evidence, while an unapproved/revised action cannot use that approval. Test from the actual agent account, including denial of the old admin SSH/sudo/credential paths. - Autopilot remains disabled without a scoped promotion decision, concrete EUR cost/risk limits and verified enforcement. Implementing a general promotion or risk-scoring service is not required for this supervised credential boundary. ## Reject last-applied annotations on Secrets ```task id: CUST-WP-0073-T03 status: done priority: medium needs_human: false state_hub_task_id: "5abbfcae-eb65-5b62-a7fa-f4f698f95312" ``` Owner: railiance-platform. **Done 2026-09-28.** Explicit metadata fixes landed in all 31 affected ESO declarations. The guard is active and Synced/Healthy; all nine native admission checks and 39/39 fresh ESO refreshes under Deny pass. All 257 active-namespace Secrets were annotation-free. After explicit founder approval, the one orphan drill Secret was deleted with its UID precondition; absence verified and the 3 GiB PVC unchanged. Full evidence: `docs/evidence/2026-09-28-secret-annotation-rollout.md`. - Add a `ValidatingAdmissionPolicy` (v1.35 is available) with its binding. It rejects any Secret carrying `kubectl.kubernetes.io/last-applied-configuration`. - Strip the annotation from existing Secrets first, cluster-wide, using the output-suppressing maintenance helper, or the policy blocks their next update. Known: `sso/keycape-config`, `sso/authelia-secrets`, `sso/lldap-secrets`, `mfa/privacyidea-config`. - Proof: a client-side `kubectl apply` of a test Secret in a scratch namespace is refused. ## Fleet guidance and harness guards ```task id: CUST-WP-0073-T04 status: progress priority: medium state_hub_task_id: "90725511-4e31-549f-b567-47feff1a9ca4" ``` - Orientation doc §6 documents the template-error dump and requires the safe maintenance helper; standalone Secret templates/jsonpath are forbidden. The September 28 standing notice supersedes the September 24 inline-check exception. Add the verified agent identity/execution path once T02 lands. - Claude Code guard, **done on the workstation 2026-09-24**: `~/.claude/hooks/guard-secret-reads.py` (PreToolUse on Bash). It denies Secret reads, `helm get`, `config view --raw` and kubeconfig reads, and asks before exec/attach/cp/debug and `--raw`. The `kubectl * get secret *` allow rule and the expired 2026-07-07 temporary elevation are removed. Known false positive: any command naming both kubectl and "secret", including an `echo`. That is acceptable for a stopgap. - Offer the same guard to the Codex and Grok harnesses, or record that they have none. Until T02 lands, those agents are protected by instructions only. - OpenBao/Vault secret-reading permissions also belong behind the privileged boundary; preapproved command prefixes do not implement supervision. - Document the agent-specific mode and supervisor. Reference exact proposal and execution receipts; track unchanged acceptance separately from verified unchanged success, including refinements, rejections and interventions. Reuse existing records and receipts per `docs/agent-autonomy-decision.md`; no new dashboard, supervisor service or automatic promotion machinery. ## Rotate what was exposed ```task id: CUST-WP-0073-T05 status: wait priority: low state_hub_task_id: "1b41b532-38a2-5f39-b387-28737934811a" ``` Deferred by founder decision on 2026-09-24 (builder mode). Rotate `sso/keycape-config` (signing key, LLDAP bind password, Authelia client secret), `sso/authelia-secrets`, `sso/lldap-secrets` and `mfa/privacyidea-config` together, with `replace` rather than `apply`, in one attended window. Reopen on the first of: - the first production workload or customer data passing through KeyCape - any sign the 2026-09-23 transcript left the workstation beyond the model provider - a planned key rotation The passage of time alone reopens nothing. ## Current evidence and handoff — September 28 The founder requests minimal additional tasks, workplans and functionality. All remaining work stays in T02, T04 and the original deferred T05. No new workplan, task, controller or supervisor service was introduced. T01 is done: autonomy is per agent and scope, supervised initially, with later promotion bounded by EUR cost and risk limits. The decision and descriptive per-agent record do not establish runtime enforcement or grant autopilot. T02 remains in progress. The existing sand-boxer `profile.bwrap-local` passed a synthetic process-isolation proof: admin homes, privileged environment variables, Kubernetes/container sockets absent; only loopback networking; observation and proposal paths work; wrong consumer rejected; workspace destroyed. Receipt: `docs/evidence/2026-09-28-supervised-sandbox-proof.json`. It was not an interactive agent migration. The selected GLAS local-profile acceptance remains blocked; existing coordination receipt: `docs/evidence/2026-09-28-supervised-runtime-coordination.json`. Actual account/profile admission and denial of old admin SSH/sudo/credential paths are still required. The inspected legacy account has unrestricted sudo and k3s kubeconfig mode 644; changing its default kubeconfig alone is insufficient. T03 is done. All 31 affected ESO declarations have explicit target metadata in 23 files across 12 owning repositories, committed and published. The guard is active and Synced/Healthy at platform source `7daf7e9`, pinned by `db51ec8`. Nine native admission checks and 39/39 fresh ESO refreshes under Deny passed. The founder-approved UID-bound deletion removed only the orphan drill Secret; the 3 GiB PVC was unchanged. The final complete scan checked 257 Secrets with zero forbidden annotations and zero orphan exceptions. The earlier failed rollout and rollback remain historical evidence in `docs/evidence/2026-09-28-secret-annotation-rollout.md`; they are not the live state. T04's local guidance and reporting are implemented. AGENTS.md now loads the supervision decision and per-agent record at startup and describes recording original proposals before approval and verified outcomes afterwards. The September 28 standing notice withdraws the unsafe inline presence check: 51e9eace-06d2-46d6-921a-4b92440df68f. Receipt: `docs/evidence/2026-09-28-secret-guidance-notice.json`. Codex/Grok have no equivalent read-denial hook established by this work. The remaining T04 step is to document the actual verified T02 execution path. The original rollout remains an unscored failed proposal with refinement and recovery; successful remediation does not become unchanged-success credit. There is no eligible acceptance-rate sample or autopilot grant yet. T05 remains the founder's trigger-based rotation deferral, not cancelled or done. Neither workplan completion nor live credential separation is claimed.