From 3ed1ac19c59cad10f9ff8a574963318cf3e0c6fb Mon Sep 17 00:00:00 2001 From: codex Date: Tue, 27 Oct 2026 22:31:37 +0100 Subject: [PATCH] RCLUSTER-WP-0021: propose k3s API audit policy and shipping to audit-core (NK-WP-0055-T11) Co-Authored-By: Claude Opus 5.5 Assistant: claude-code Assistant-Model: opus Assistant-Process: 4515@bnt-lap001 Assistant-Session: 0bf96127-6e04-41be-bfda-d11717c3f354 --- WORK-RECORDS.md | 6 + ...TER-WP-0021-k3s-api-audit-to-audit-core.md | 140 ++++++++++++++++++ 2 files changed, 146 insertions(+) create mode 100644 workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 5335e22..395b9d7 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -24,6 +24,7 @@ | workplan | RCLUSTER-WP-0018 | active | — | workplans/RCLUSTER-WP-0018-keycape-original-tls-recovery.md | | workplan | RCLUSTER-WP-0019 | active | — | workplans/RCLUSTER-WP-0019-k3s-admin-kubeconfig-mode.md | | workplan | RCLUSTER-WP-0020 | active | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md | +| workplan | RCLUSTER-WP-0021 | proposed | — | workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md | | task | RCLUSTER-WP-ADHOC-2026-07-27-T01 | done | — | workplans/ADHOC-2026-07-27.md | | task | RAIL-BS-WP-0008-T01 | done | — | workplans/RAIL-BS-WP-0008-activity-core-wp0016-triage-output-deploy.md | | task | RAIL-BS-WP-0008-T02 | done | — | workplans/RAIL-BS-WP-0008-activity-core-wp0016-triage-output-deploy.md | @@ -80,3 +81,8 @@ | task | RCLUSTER-WP-0020-T02 | done | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md | | task | RCLUSTER-WP-0020-T03 | wait | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md | | task | RCLUSTER-WP-0020-T04 | todo | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md | +| task | RCLUSTER-WP-0021-T01 | todo | — | workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md | +| task | RCLUSTER-WP-0021-T02 | todo | — | workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md | +| task | RCLUSTER-WP-0021-T03 | wait | — | workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md | +| task | RCLUSTER-WP-0021-T04 | wait | — | workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md | +| task | RCLUSTER-WP-0021-T05 | wait | — | workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md | diff --git a/workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md b/workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md new file mode 100644 index 0000000..7f2dd29 --- /dev/null +++ b/workplans/RCLUSTER-WP-0021-k3s-api-audit-to-audit-core.md @@ -0,0 +1,140 @@ +--- +id: RCLUSTER-WP-0021 +type: workplan +title: "k3s API-server audit policy, node log backend and durable shipping to audit-core" +domain: financials +repo: railiance-cluster +status: proposed +owner: claude +topic_slug: railiance +created: "2026-10-27" +updated: "2026-10-27" +flavor: implementation +depends_on: [] +related: [NK-WP-0055, AUDIT-WP-0005, AUDIT-WP-0007, AUDIT-WP-0015, RCLUSTER-WP-0019] +--- + +Owner source work for NetKingdom NK-WP-0055-T11 (production-readiness ledger, +area 9 "Audit sink", row "Kubernetes API audit: absent"). The single-node k3s +server on railiance01 runs with no `audit-policy-file` and no audit log, so no +Kubernetes API record exists for secrets access, RBAC changes or exec/attach. + +Boundary: this repo owns the k3s server flags, the audit policy, the node-local +log backend and rotation, and the shipper that runs on the node. audit-core owns +the receiving API, sender registration, schema, hash chain, retention in its +store and its restore drill. Nothing here is auto-applied; every host change runs +through an attended playbook. The k3s restart is a founder-run live step. + +## Audit policy (source) + +```task +id: RCLUSTER-WP-0021-T01 +status: todo +priority: high +assignee: claude +``` + +Add `ansible/files/k3s-audit-policy.yaml` (`audit.k8s.io/v1`, rules first-match): + +1. `omitStages: [RequestReceived]` globally. +2. `secrets`, `serviceaccounts/token`, `tokenreviews`, `subjectaccessreviews`: + **Metadata only, in every namespace**. The policy must contain no + Request/RequestResponse rule that can match `secrets`; a test enforces this. +3. RBAC (`rbac.authorization.k8s.io`: roles, rolebindings, clusterroles, + clusterrolebindings) write verbs: RequestResponse (no secret payloads). +4. `pods/exec`, `pods/attach`, `pods/portforward`, `nodes/proxy`: Metadata + (the request body carries no stream content). +5. cert-manager (`cert-manager.io`, `acme.cert-manager.io`) writes: Request. +6. Namespaces `sso`, `mfa`, `openbao` (exact list confirmed against the cluster + in T05): all write verbs at Request, reads at Metadata; still never above + Metadata for secrets/configmaps (rule 2 precedes; configmaps held at Metadata + here because some carry credentials-adjacent config). +7. Drop noise: `system:kube-proxy` watches, `/healthz*`, `/readyz*`, `/livez*`, + `/metrics`, events, leases and node status heartbeats: None. +8. Catch-all: Metadata for write verbs, None for read verbs elsewhere. + +Done when: policy file, a `make audit-policy-check` target (kubeconform-style +schema check plus a script asserting rule 2 and the absence of any secrets match +above Metadata), and an offline unit test on synthetic request fixtures. + +## Node log backend and rotation (source) + +```task +id: RCLUSTER-WP-0021-T02 +status: todo +priority: high +assignee: claude +``` + +Extend the existing-host pattern of `ansible/k3s-kubeconfig-mode.yml` into +`ansible/k3s-api-audit.yml` (and `bootstrap.yml` for fresh installs), writing +k3s `config.yaml` `kube-apiserver-arg` entries: + +- `audit-policy-file=/etc/rancher/k3s/audit-policy.yaml` (root:root 0600) +- `audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log` (dir 0700) +- `audit-log-maxage=7`, `audit-log-maxbackup=10`, `audit-log-maxsize=100` +- `audit-log-format=json`, `audit-log-mode=batch` (buffer sized so a slow disk + cannot stall the API server) + +Same guards as RCLUSTER-WP-0019: pinned preimage hashes, refuse on drift or +foreign config.yaml keys, root-only backup, `--check`/`--diff` with content +suppressed, a rollback action that removes the args. Validate offline only +(syntax-check, ansible-lint production profile). Disk headroom check: refuse +if `/var/lib/rancher` has less than 5 GiB free. + +## Shipping to audit-core (source; depends on audit-core contract) + +```task +id: RCLUSTER-WP-0021-T03 +status: wait +priority: high +assignee: claude +``` + +A node-local shipper (Fluent Bit DaemonSet with hostPath read-only on the log +dir, or a systemd unit on the host; choose after audit-core states its intake) +tails `audit.log`, keeps a persisted file offset, maps each event to the +audit-core envelope (`auditID` as idempotency key, `stageTimestamp`, user, +verb, objectRef, responseStatus, source `kubernetes-api/railiance01`) and posts +with retry and back-pressure. Sender credential comes from OpenBao via the +`warden route` owner path; never a static token in Git. Fail mode: the node log +is the buffer; shipper lag is exported as a metric and alerted. + +Waits on audit-core: sender registration for `railiance-cluster` / +`kubernetes-api`, the intake endpoint and auth method, envelope/schema version, +expected volume ceiling (estimate in T05), and the correlation-id field from +NK-WP-0055-T11. + +## Retention and restore check (source) + +```task +id: RCLUSTER-WP-0021-T04 +status: wait +priority: medium +assignee: claude +``` + +Node retention is a buffer only (7 days / 10 x 100 MB). Durable retention +(proposed: 400 days for Kubernetes API records) lives in audit-core and its +backups. Add `make audit-k8s-restore-check`: read-only, emits a synthetic +marker request (a labelled `SelfSubjectAccessReview` by a named operator +identity), then confirms its `auditID` is present in audit-core and, during +audit-core's recurring restore drill, in the restored copy with an intact hash +chain. Receipt goes to `docs/evidence/`. Waits on T03 and audit-core's drill. + +## Founder-run live rollout (attended k3s restart) + +```task +id: RCLUSTER-WP-0021-T05 +status: wait +priority: high +assignee: founder +``` + +Founder runs, attended, after T01/T02 are reviewed: +`ansible-playbook ansible/k3s-api-audit.yml --check --diff`, then the apply, +which restarts k3s (single node: brief API outage, workloads keep running). +Verify `/readyz`, node Ready, `audit.log` growing, a test `kubectl get secret` +logged at Metadata with no `requestObject`/`responseObject`, then a 24 h +volume sample to size T03/T04. Rollback is the playbook's rollback action. +Agents prepare and hand over the exact commands; they do not run them.