Advance supervised agent records and close verified Secret annotation guard
This commit is contained in:
codex
parent
b2f6713721
commit
db91818e84
44 changed files with 6868 additions and 54 deletions
|
|
@ -9,7 +9,7 @@ flavor: planning
|
|||
owner: the-custodian
|
||||
topic_slug: custodian
|
||||
created: "2026-09-11"
|
||||
updated: "2026-09-14"
|
||||
updated: "2026-09-28"
|
||||
related: [STATE-WP-0091, RCLUSTER-WP-0014, RESOURCE-WP-0003, RAPP-TELEMETRY-WP-0001, RAIL-FAB-WP-0028, RAPPS-WP-0014, VERGABE-WP-0019, HFACT-WP-0001]
|
||||
state_hub_workstream_id: "2249bddb-7524-5add-bd5c-c4163a6ca0f3"
|
||||
---
|
||||
|
|
@ -18,7 +18,7 @@ state_hub_workstream_id: "2249bddb-7524-5add-bd5c-c4163a6ca0f3"
|
|||
|
||||
User instruction, 2026-09-11: persist and register this work for later follow-up,
|
||||
then continue the invited Vergabe pilot at an explicitly accepted 60m CPU
|
||||
request. This ready workplan is not a prerequisite to deploying that prototype.
|
||||
request. This workplan is not a prerequisite to deploying that prototype.
|
||||
It is not an assertion that 60m or the inherited 100m is a measured requirement.
|
||||
|
||||
The objective is an explainable, repeatable allocation process across Railiance
|
||||
|
|
@ -94,11 +94,10 @@ updated reef-railiance-k3s owner evidence.
|
|||
|
||||
```task
|
||||
id: CUST-WP-0071-T02
|
||||
status: wait
|
||||
status: progress
|
||||
priority: high
|
||||
assignee: the-custodian
|
||||
depends_on: [CUST-WP-0071-T01]
|
||||
blocking_reason: "Await reliable measurements and the current invited-pilot deployment or an equivalent isolated fixture."
|
||||
state_hub_task_id: "82192370-2fd7-5363-88d2-3c67889d3d68"
|
||||
```
|
||||
|
||||
|
|
@ -112,8 +111,10 @@ database demand and request volume. Distinguish container CPU from incremental
|
|||
database/shared-service demand. No benchmark writes to existing customer data.
|
||||
|
||||
Record workload sizes, concurrency, hardware/image/workers, duration, coverage
|
||||
and limitations so results are reproducible. Agree response-time/error targets
|
||||
with the product owner before claiming adequacy. Recommend request/limit and
|
||||
and limitations so results are reproducible. Founder acceptance target, 2026-09-28: with two simultaneous users, p95 of
|
||||
ordinary operations must be at most 2 seconds and there must be no failed
|
||||
operations. Report document transfer time separately. This resolves the target
|
||||
choice; obtain representative evidence before claiming adequacy. Recommend request/limit and
|
||||
memory values with a stated margin and revisit trigger; label an incomplete
|
||||
pilot sample provisional. A successful smoke test alone is not sizing proof.
|
||||
|
||||
|
|
@ -121,11 +122,10 @@ pilot sample provisional. A successful smoke test alone is not sizing proof.
|
|||
|
||||
```task
|
||||
id: CUST-WP-0071-T03
|
||||
status: wait
|
||||
status: progress
|
||||
priority: high
|
||||
assignee: the-custodian
|
||||
depends_on: [CUST-WP-0071-T01, CUST-WP-0071-T02]
|
||||
blocking_reason: "Await reconciled demand evidence and a measured pilot recommendation."
|
||||
state_hub_task_id: "6dc67558-eb1e-5bb6-a667-986f884dd495"
|
||||
```
|
||||
|
||||
|
|
@ -221,3 +221,40 @@ updated the mounted credential, and KeyCape recovered. This is immediate recover
|
|||
not a fleet sizing conclusion. T01 must include recurring maintenance-job demand
|
||||
and reliable scheduling headroom, not only resident pod allocations. Evidence:
|
||||
informed-decision/docs/evidence/2026-09-14-keycape-renewal-capacity-recovery.json.
|
||||
|
||||
## September 28 bounded completion review
|
||||
|
||||
The founder asks to finish with minimal additional tasks, workplans and
|
||||
functionality. Keep all remaining work in T02–T05; do not spawn a monitoring
|
||||
service, benchmark framework or replacement coordination plan.
|
||||
|
||||
Evidence: `docs/evidence/2026-09-28-sizing-review.md`, allocation reconcile,
|
||||
retained cluster observation, source revisions and exact seven-day PromQL
|
||||
responses alongside it. T01 refreshed: 3420m requested / 4000m, no pending
|
||||
requests, 580m reservation residual; instantaneous node CPU was 3963m, so this
|
||||
is not spare processing capacity. No unsupported pod accounting features were
|
||||
present in this snapshot. Namespace ownership refreshed.
|
||||
|
||||
T02 is now in progress: the exact deployed pilot and seven days of measurements
|
||||
are recorded. CPU p95 0.52m, sampled peak 20.10m, memory peak 191.14Mi; retain
|
||||
60m/256Mi provisionally. Representative two-user activity, response/error
|
||||
acceptance and incremental database attribution remain unproven. Use existing
|
||||
RAPPS-WP-0014-T03 fixture/acceptance work; do not duplicate its recovery scope.
|
||||
|
||||
T03 is now in progress: retain current pilot/Knative allocations, investigate
|
||||
Forgejo/runner demand (namespace CPU p95 1454m), and account for twelve
|
||||
zero-request workloads. RAIL-KNATIVE-WP-0002 and RAIL-EN-WP-0002 are finished;
|
||||
their declaration/deployment work must not be repeated. No new resource change
|
||||
is proposed from the incomplete sample. T04 can verify a justified keep decision;
|
||||
it must not create an unnecessary resize. Its useful-operation acceptance stays.
|
||||
|
||||
T05 still requires a durable activity-core schedule, retained report, owner
|
||||
receipt and missed-run recovery proof. Monday 08:00 Europe/Berlin remains the
|
||||
proposed cadence. None is represented as installed by this session. Existing
|
||||
T02–T05 retain the remaining evidence and execution, with no new work records.
|
||||
|
||||
September 28 follow-up: the founder selected the two-user p95 ≤ 2 seconds,
|
||||
zero-failed-operations target (document transfer excluded from that latency
|
||||
threshold). The current seven-day telemetry remains provisional until the
|
||||
representative workflow runs. This is an acceptance criterion, not a claim that
|
||||
it has passed. Keep the run and its evidence under existing T02.
|
||||
|
|
|
|||
|
|
@ -1,15 +1,15 @@
|
|||
---
|
||||
id: CUST-WP-0073
|
||||
type: workplan
|
||||
title: "Agents cannot read secret values: separate agent and admin credentials"
|
||||
title: "Separate agent credentials and establish supervised privileged execution"
|
||||
domain: infotech
|
||||
repo: the-custodian
|
||||
status: proposed
|
||||
owner: claude-code
|
||||
status: active
|
||||
owner: the-custodian
|
||||
topic_slug: custodian
|
||||
flavor: implementation
|
||||
created: "2026-09-24"
|
||||
updated: "2026-09-24"
|
||||
updated: "2026-09-28"
|
||||
related:
|
||||
- KEY-WP-0033
|
||||
- RPF-WP-0044
|
||||
|
|
@ -18,7 +18,7 @@ origin_ref: key-cape/docs/operations.md#before-any-live-change
|
|||
state_hub_workstream_id: "a98a9f34-83b4-5c8c-8107-e05f7806d50d"
|
||||
---
|
||||
|
||||
# Agents cannot read secret values: separate agent and admin credentials
|
||||
# Separate agent credentials and establish supervised privileged execution
|
||||
|
||||
## Why
|
||||
|
||||
|
|
@ -41,8 +41,12 @@ The root cause is the credentials, not the command:
|
|||
A command denylist chases them one by one, and it only binds the harness that
|
||||
enforces it.
|
||||
|
||||
**Goal:** the identity an agent uses cannot read a secret value by any command.
|
||||
Then a leak needs a human's attended credential, not a mistake.
|
||||
**Goal:** keep privileged credentials outside the agent's direct reach, and
|
||||
mediate privileged actions according to the identified agent's autonomy mode.
|
||||
Agents start supervised; proven agents may later receive scoped autopilot
|
||||
permission with a cost budget and risk limit in EUR. Neither mode implies
|
||||
unrestricted admin credentials. Founder decision, September 28:
|
||||
`docs/agent-autonomy-decision.md`.
|
||||
|
||||
**Scope:** builder mode, founder decision 2026-09-24. Rotating the exposed
|
||||
Secrets is deferred, not dropped (T05). This plan removes the problem class.
|
||||
|
|
@ -51,42 +55,47 @@ Secrets is deferred, not dropped (T05). This plan removes the problem class.
|
|||
|
||||
```task
|
||||
id: CUST-WP-0073-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "4781bb99-020f-59f6-be13-e3b8777d3fd8"
|
||||
```
|
||||
|
||||
Decide, with railiance-platform and ops-warden:
|
||||
**Done 2026-09-28 — policy decision.** The founder chooses autonomy as a
|
||||
characteristic of the agent: supervised-mode initially, promotion only on
|
||||
successful proposals without adaptation/refinement, and autopilot bounded by
|
||||
cost and risk limits in EUR. Decision: `docs/agent-autonomy-decision.md`.
|
||||
|
||||
- **Agent identity:** a dedicated kube identity outside `system:masters`,
|
||||
bound to the built-in `view` role plus the specific write verbs agents need
|
||||
(for example patch and rollout restart on Deployments, ConfigMap updates).
|
||||
No `secrets` verbs at all, since `list` and `watch` return data too. No
|
||||
`pods/exec`, `pods/attach`, `pods/portforward` or `nodes/proxy`. No `helm`,
|
||||
which stores its releases in Secrets.
|
||||
- **Admin identity:** stays `system:admin`, reachable only by an attended step,
|
||||
never readable from the agent's Unix account.
|
||||
- **Where the agent credential lives** on the workstation and on railiance01.
|
||||
ops-warden already distinguishes `adm`/`agt`/`atm` SSH principals. Mapping
|
||||
`agt` to a restricted account on railiance01 is the obvious candidate; how
|
||||
warden provisions those principals is still to be verified.
|
||||
- **Paths that stay attended:** Secret writes, helm releases and break-glass.
|
||||
The supervised starting identity is outside `system:masters` and has a reviewed
|
||||
observation allowlist. Exact privileged actions go through supervisor approval
|
||||
or supervisor execution, with unchanged-acceptance and verified unchanged-success
|
||||
recorded separately. Admin credentials remain in the privileged execution path,
|
||||
outside the agent's direct reach. Later autopilot removes per-action approval
|
||||
only for an explicit grant within scope, cost and risk constraints; it does not
|
||||
hand out unrestricted sudo or an admin kubeconfig.
|
||||
|
||||
Output: a decision record in the-custodian, resolved by the founder
|
||||
(`GOVERN @ estate`).
|
||||
Built-in `view` plus Deployment/ConfigMap writes cannot establish that boundary:
|
||||
workload writes can extract credentials, and ConfigMaps/pod specs/logs can contain
|
||||
values. Agent-visible results must be sanitized. T02 selects and proves the
|
||||
actual account/profile/execution path. This decision resolves the identity and
|
||||
autonomy policy, not the implementation, numeric promotion thresholds, euro
|
||||
limits or authorization of a live cutover. Existing human-only lanes still apply.
|
||||
|
||||
## Build and hand out the agent identity
|
||||
|
||||
```task
|
||||
id: CUST-WP-0073-T02
|
||||
status: todo
|
||||
status: progress
|
||||
priority: high
|
||||
state_hub_task_id: "4b88b0b7-dc7e-5612-aa5d-1a08f0530c35"
|
||||
```
|
||||
|
||||
Owner: railiance-platform (RBAC), railiance-enablement (k3s install),
|
||||
ops-warden (principal mapping).
|
||||
railiance-infra (host principal mapping), ops-warden (certificate issuance).
|
||||
|
||||
- Bind the agent's stable identity and `supervised-mode` to its existing
|
||||
instance/assignment record and a restricted runtime profile. Name its
|
||||
supervisor and privileged execution path. No raw admin credentials in the
|
||||
agent process/account; no unrestricted sudo, socket or GitOps bypass.
|
||||
- Create the ServiceAccount or client certificate, the ClusterRole and the
|
||||
binding in git, applied by the owner's documented path.
|
||||
- Set `write-kubeconfig-mode` to `600` in the k3s install config. Attended admin
|
||||
|
|
@ -96,19 +105,34 @@ ops-warden (principal mapping).
|
|||
issuance through an attended login).
|
||||
- Proof: as the agent identity, `kubectl auth can-i get secrets -A` and
|
||||
`can-i create pods/exec -A` both answer `no`, and `kubectl auth whoami`
|
||||
shows no `system:masters`. Record the output as evidence.
|
||||
shows no `system:masters`. Record the output as evidence. Also verify that
|
||||
an exact supervisor-approved action can execute through the selected privileged
|
||||
path and return sanitized outcome evidence, while an unapproved/revised action
|
||||
cannot use that approval. Test from the actual agent account, including denial
|
||||
of the old admin SSH/sudo/credential paths.
|
||||
- Autopilot remains disabled without a scoped promotion decision, concrete EUR
|
||||
cost/risk limits and verified enforcement. Implementing a general promotion or
|
||||
risk-scoring service is not required for this supervised credential boundary.
|
||||
|
||||
## Reject last-applied annotations on Secrets
|
||||
|
||||
```task
|
||||
id: CUST-WP-0073-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
needs_human: false
|
||||
state_hub_task_id: "5abbfcae-eb65-5b62-a7fa-f4f698f95312"
|
||||
```
|
||||
|
||||
Owner: railiance-platform.
|
||||
|
||||
**Done 2026-09-28.** Explicit metadata fixes landed in all 31 affected ESO
|
||||
declarations. The guard is active and Synced/Healthy; all nine native admission
|
||||
checks and 39/39 fresh ESO refreshes under Deny pass. All 257 active-namespace
|
||||
Secrets were annotation-free. After explicit founder approval, the one orphan
|
||||
drill Secret was deleted with its UID precondition; absence verified and the
|
||||
3 GiB PVC unchanged. Full evidence: `docs/evidence/2026-09-28-secret-annotation-rollout.md`.
|
||||
|
||||
- Add a `ValidatingAdmissionPolicy` (v1.35 is available) with its binding. It
|
||||
rejects any Secret carrying `kubectl.kubernetes.io/last-applied-configuration`.
|
||||
- Strip the annotation from existing Secrets first, cluster-wide, using the
|
||||
|
|
@ -122,7 +146,7 @@ Owner: railiance-platform.
|
|||
|
||||
```task
|
||||
id: CUST-WP-0073-T04
|
||||
status: todo
|
||||
status: progress
|
||||
priority: medium
|
||||
state_hub_task_id: "90725511-4e31-549f-b567-47feff1a9ca4"
|
||||
```
|
||||
|
|
@ -140,8 +164,13 @@ state_hub_task_id: "90725511-4e31-549f-b567-47feff1a9ca4"
|
|||
acceptable for a stopgap.
|
||||
- Offer the same guard to the Codex and Grok harnesses, or record that they have
|
||||
none. Until T02 lands, those agents are protected by instructions only.
|
||||
- Open question for the founder: `Bash(bao read *)`, `vault kv get` and
|
||||
`vault read` are still pre-approved and print secret values the same way.
|
||||
- OpenBao/Vault secret-reading permissions also belong behind the privileged
|
||||
boundary; preapproved command prefixes do not implement supervision.
|
||||
- Document the agent-specific mode and supervisor. Reference exact proposal and
|
||||
execution receipts; track unchanged acceptance separately from verified
|
||||
unchanged success, including refinements, rejections and interventions. Reuse
|
||||
existing records and receipts per `docs/agent-autonomy-decision.md`; no new
|
||||
dashboard, supervisor service or automatic promotion machinery.
|
||||
|
||||
## Rotate what was exposed
|
||||
|
||||
|
|
@ -165,3 +194,86 @@ Reopen on the first of:
|
|||
- a planned key rotation
|
||||
|
||||
The passage of time alone reopens nothing.
|
||||
|
||||
## September 28 implementation review
|
||||
|
||||
The founder asks for minimal additional tasks/workplans/functionality. Keep
|
||||
execution and all unresolved evidence in T01–T05. No new plan or task was opened.
|
||||
|
||||
Reviewable package: `docs/changes/CUST-WP-0073/README.md` and
|
||||
`reject-secret-last-applied.yaml` beside it. Live read-only inspection confirms
|
||||
`tegwick` has unrestricted passwordless sudo, k3s kubeconfig is still 644, and
|
||||
Kubernetes uses `system:admin` / `system:masters`. The public host inventory maps
|
||||
agent and admin principals to this same account. A kubeconfig switch or chmod
|
||||
alone is insufficient; do not claim the agent boundary has landed.
|
||||
|
||||
T01's initial permanent observation-only proposal is superseded by the founder's
|
||||
agent-specific supervised/autopilot decision in `docs/agent-autonomy-decision.md`.
|
||||
T01 is done; T02 must
|
||||
verify the actual agent execution environment has no route back through admin
|
||||
SSH/sudo, tokens, sockets or automated deployment. This adds no new broker.
|
||||
|
||||
T02 in progress: the existing sand-boxer `profile.bwrap-local` passed a
|
||||
synthetic supervised-process proof: admin homes, Kubernetes/container socket
|
||||
paths and privileged environment variables absent; only loopback networking;
|
||||
observation readable; proposal writable; wrong consumer identity rejected;
|
||||
workspace destroyed. Receipt: `docs/evidence/2026-09-28-supervised-sandbox-proof.json`.
|
||||
This was not an interactive agent or a credential migration. Existing GLAS
|
||||
local-profile acceptance and actual admin-path denial remain required in T02.
|
||||
|
||||
T03 in progress: policy source `railiance-platform@800cbfa`, application `54885ac`
|
||||
and nine passing native admission checks were followed by ESO refresh failures.
|
||||
ESO v0.16.1 copies source metadata when an ExternalSecret has no target template;
|
||||
31 declarations need explicit metadata before the strict guard is compatible.
|
||||
The binding was removed and all 39 ExternalSecrets recovered. GitOps now pins
|
||||
policy-only `6016f72` via application commit `c5d65b0`; the application is Synced
|
||||
and Healthy, and enforcement is disabled. Detailed rollout and recovery receipt:
|
||||
`docs/evidence/2026-09-28-secret-annotation-rollout.md`.
|
||||
|
||||
Cleanup removed the duplicate annotation from 49 distinct active-namespace
|
||||
Secrets across the recorded passes, but ESO can regenerate it while enforcement
|
||||
is off. An orphan `platform-pg-drill/drill-minio` Secret cannot be patched because
|
||||
its namespace is absent; a referencing Deployment, PVC and Service remain. No
|
||||
orphan was deleted. Neither stable cluster-wide cleanup nor T03 completion is
|
||||
claimed. Keep remediation and integration proof in this existing task.
|
||||
|
||||
T04 in progress: orientation §6 withdraws the unsafe raw presence template;
|
||||
only the capturing/sanitizing maintenance helper is allowed. A logical
|
||||
per-agent supervised record lives at `.kaizen/agents/custodian-codex/supervision.json`.
|
||||
Its summary separates unchanged acceptance from verified unchanged execution,
|
||||
retains failed outcomes and rescue, and grants no authority. The rollout is an
|
||||
unscored historical approval with a failed outcome and recovery, not promotion
|
||||
evidence. There is no eligible acceptance-rate sample yet, no autopilot grant,
|
||||
and no enforced interactive-runtime migration. Codex/Grok have no established
|
||||
equivalent read-denial hook. Final guidance still needs the actual T02 path.
|
||||
T05 remains the original trigger-based founder deferral, not cancelled or done.
|
||||
|
||||
## Corrected admission rollout — September 28 continuation
|
||||
|
||||
T03: all 31 affected ExternalSecrets now have explicit target metadata in their
|
||||
owner sources (23 files, 12 repositories, committed and published). Server
|
||||
dry-run verified only the target template changes; credential data mappings and
|
||||
policies remain unchanged. All 39 ExternalSecrets refreshed successfully before
|
||||
and after re-enabling Deny enforcement. All nine native admission checks pass.
|
||||
The guard is Synced/Healthy at platform source `7daf7e9`, pinned by `db51ec8`.
|
||||
The earlier rollback is historical, not the current live state. Detailed evidence:
|
||||
`docs/evidence/2026-09-28-secret-annotation-rollout.md`.
|
||||
|
||||
A complete scan of all 257 Secrets in existing namespaces found no forbidden
|
||||
annotation after the writer fixes. T03 now waits only for the orphan
|
||||
`platform-pg-drill/drill-minio`: its namespace is absent, so an annotation patch
|
||||
is refused. UID-bound deletion of that one Secret is prepared and awaits
|
||||
explicit authorization; no PVC deletion or namespace recreation is proposed.
|
||||
All remaining work stays in existing tasks; no new task, workplan, controller
|
||||
or service was introduced. T02 still needs actual supervised-runtime admission;
|
||||
T05 keeps its founder-deferred rotation triggers.
|
||||
|
||||
Post-enforcement scan: all 257 active-namespace Secrets remain annotation-free.
|
||||
The exact orphan deletion also passed server-side dry-run; execution awaits
|
||||
the founder response. Receipt: `docs/evidence/2026-09-28-secret-annotation-scan-enforced.json`.
|
||||
|
||||
Final orphan disposition: the founder explicitly selected “Delete only the
|
||||
orphan Secret.” The UID-bound deletion succeeded and absence was verified;
|
||||
the bound 3 GiB PVC retained the same UID, resourceVersion, volume and status.
|
||||
No other resources were changed. T03 is done and its human-needed flag cleared.
|
||||
Receipt: `docs/evidence/2026-09-28-orphan-secret-deletion.json`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue