railiance-platform/workplans/RPF-WP-0062-custodian-agent-observer-identity.md

id	type	title	domain	repo	status	owner	created	updated	flavor	origin	origin_ref	state_hub_workstream_id
RPF-WP-0062	workplan	Custodian agent observer identity: read-only ServiceAccount and curated ClusterRole on railiance01	financials	railiance-platform	finished	claude	2026-10-20	2026-10-20	implementation	cross-repo	the-custodian CUST-WP-0073-T02 (founder decision 2026-10-20)	2cda239a-c071-5234-a728-7d19c3eff306

Demand: CUST-WP-0073-T02 needs a supervised agent identity on railiance01 that can observe cluster state without reaching credentials or writing anything. This repo owns the RBAC half. The earlier health-only candidate (the-custodian/docs/changes/CUST-WP-0073/credential-boundary-r1/identity-rbac.yaml) is superseded by the founder's 2026-10-20 decision.

Founder decision (Bernd Worsch, 2026-10-20):

• Namespace custodian-agents, ServiceAccount custodian-agent, automountServiceAccountToken: false. No static token Secret.
• Credentials are short-lived TokenRequest tokens issued by the supervisor.
• ClusterRole custodian-supervised-observer: get/list/watch on namespaces, nodes, pods, deployments/statefulsets/daemonsets/replicasets, services, endpoints/endpointslices, ingresses, events (core + events.k8s.io), persistentvolumeclaims, Argo CD applications/appprojects, CNPG clusters/backups/scheduledbackups; get on /version /healthz /livez /readyz.
• Explicitly not: secrets, configmaps, pods/log, pods/exec, pods/attach, pods/portforward, serviceaccounts/token, nodes/proxy, any write verb, any wildcard. Privileged actions go through the separate gated executor.

Layout:

• Manifests: argocd/platform-addons/custodian-agents/
• Child Application (manual sync, pinned, no finalizer): argocd/railiance01/applications/custodian-agents.application.yaml, project railiance-platform-addons (already whitelists Namespace, ClusterRole, ClusterRoleBinding, ServiceAccount).
• Offline guard: tests/test_custodian_agent_rbac.py.

The railiance01 root railiance-apps-root has no automated block, so a push to main changes nothing live. Going live is an attended founder action.

Token issuance (supervisor only)

Recommended TTL 24h (shorter is fine; the API server may cap it via --service-account-max-token-expiration). Audience is the API server's own service-account audience, so the token works against the API and nothing else:

kubectl -n custodian-agents create token custodian-agent \
  --duration=24h --audience=https://kubernetes.default.svc.cluster.local

The token is handed to the agent's restricted kubeconfig by the supervisor; it is never committed, logged, or stored in the State Hub. It cannot be renewed by the agent (no serviceaccounts/token grant).

Place the identity manifests and offline guard

id: RPF-WP-0062-T01
status: done
priority: high
state_hub_task_id: "889272c3-d025-5e1b-b43a-ddd053ee5482"

Namespace, ServiceAccount, ClusterRole and ClusterRoleBinding written; the pytest guard parses the kustomization and asserts read-only verbs, the exact curated resource set, no forbidden resource, no subresource, no wildcard, no Secret, no automount, and a manual pinned Application.

Place the pinned child Application

id: RPF-WP-0062-T02
status: done
priority: high
state_hub_task_id: "e248f206-2b56-5336-a79b-d0a496da6098"

custodian-agents.application.yaml pins targetRevision to the commit that introduced the manifests. No automated block, no finalizer.

Attended sync and identity proof on railiance01

id: RPF-WP-0062-T03
status: done
priority: high
state_hub_task_id: "c1d2bf34-791e-5bfa-beda-94532448c7b3"

2026-10-20 (recorded 2026-10-21): the founder issued the 24h ServiceAccount token for custodian-agents/custodian-agent (CUST-WP-0073 issue-agent-credentials.sh kube). Token-based proof from the agent account: identity custodian-agents:custodian-agent, no system:masters, curated get pods allowed, 597/597 negative probes denied across 48 namespaces (the-custodian docs/evidence/2026-10-20-custodian-agent-kube-denials-receipt.json, passed=true). Done.

2026-10-20: the founder ran the attended custodian-agents child sync, pinned to 567d2c6 with the apply strategy and prune/force off. The operation Succeeded and the app is Synced/Healthy. It created exactly four objects: Namespace custodian-agents, ServiceAccount custodian-agent, ClusterRole custodian-supervised-observer and its ClusterRoleBinding. A pre-check confirmed none of these names existed before the sync.

Impersonated kubectl auth can-i --as=system:serviceaccount:custodian-agents:custodian-agent:

• Allowed (yes): get pods, list deployments, list Argo applications, list CNPG clusters.
• Refused (no): get configmaps, create deployments, delete pods, patch applications.
• Refused (no), checked with --subresource: pods log/exec/attach/portforward, serviceaccounts token, nodes proxy.

Without --subresource, a pods/log argument is parsed as a pod named "log" and reports yes. That result is not a grant.

The secret-read probe is blocked for agents by the guard hook. A binding audit found only the observer binding and the standard system:basic-user, system:discovery, system:public-info-viewer and issuer-discovery bindings applying to this account; none of them grants secret access. Remaining: issuing the token.

2026-10-20: the founder ran the attended railiance-apps-root sync, pinned to 9d5fc42 with the apply strategy and prune/force off. The operation Succeeded and the root is Synced. The custodian-agents Application now exists (OutOfSync/Missing, no automated sync), and its namespace, ServiceAccount and RBAC are not created yet. The same sync also recorded activity-core, informed-decision-requester, issue-core and state-hub-memo-dispatch as Synced; a client-side diff showed no spec change for any of them. Remaining T03 steps: child sync, token, can-i proof.

1. make argocd-pin-drift — stop on any DRIFT line.
2. Sync only this child out of the root at the reviewed revision, then the child (commands in the CUST-WP-0073-T02 handoff / this repo's commit log).
3. Issue a token (above) and, as the agent identity, prove: kubectl auth can-i get secrets -A → no; can-i create pods/exec -A → no; can-i get pods/log -A → no; can-i get configmaps -A → no; can-i list pods -A → yes; kubectl auth whoami shows system:serviceaccount:custodian-agents:custodian-agent, no system:masters.
4. Report the result to CUST-WP-0073-T02.