From ead0f09f38e02c52c63480ff65f46502ffc901c6 Mon Sep 17 00:00:00 2001 From: tegwick Date: Tue, 27 Oct 2026 22:31:48 +0100 Subject: [PATCH] AUDIT-WP-0017: propose production audit sources, correlation id, restore drill (NK-WP-0055-T11) Co-Authored-By: Claude Opus 5.5 Assistant: claude-code Assistant-Model: opus Assistant-Process: 4515@bnt-lap001 Assistant-Session: 0bf96127-6e04-41be-bfda-d11717c3f354 --- ...roduction-audit-sources-and-correlation.md | 177 ++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 workplans/AUDIT-WP-0017-production-audit-sources-and-correlation.md diff --git a/workplans/AUDIT-WP-0017-production-audit-sources-and-correlation.md b/workplans/AUDIT-WP-0017-production-audit-sources-and-correlation.md new file mode 100644 index 0000000..bd3de26 --- /dev/null +++ b/workplans/AUDIT-WP-0017-production-audit-sources-and-correlation.md @@ -0,0 +1,177 @@ +--- +id: AUDIT-WP-0017 +type: workplan +title: "Production audit sources, correlation-id contract, restore drill and offsite chain head" +domain: infotech +repo: audit-core +status: proposed +flavor: implementation +owner: audit-core +topic_slug: audit-evidence-dependencies +created: "2026-10-27" +updated: "2026-10-27" +related: + - NK-WP-0055 + - AUDIT-WP-0013 + - AUDIT-WP-0015 +--- + +# AUDIT-WP-0017 — Production audit sources, correlation-id contract, restore drill and offsite chain head + +Receiving workplan in `audit-core` for NetKingdom NK-WP-0055-T11 (production readiness, area 9 "Audit sink" +in `net-kingdom/docs/evidence/2026-10-27-production-readiness-ledger.md`). The goal is that every production +audit source lands in the durable, hash-chained sink and that records across streams can be joined by one +correlation id. + +Topaz is not a source: NetKingdom ADR-0018 makes the access-engine Go core the production PDP. + +Owner split: + +- audit-core (this plan): sender admission, envelope, scope rows, retention, restore drill, chain-head export. +- railiance-platform: OpenBao audit-device forwarding. +- railiance-cluster: Kubernetes API audit policy and forwarding. +- access-engine (and flex-auth for its pins): decision records, tracked in AUDIT-WP-0013-T02. +- NetKingdom: identity-provider event forwarding and the cross-stream correlation contract. +- resource-control / rapp-postgres: offsite chain-head custody, tracked in AUDIT-WP-0015-T01. + +This file plans work only. It activates no sender, mints no credential and makes no live change. Each sender +task stays `wait` until its owner returns the inputs listed. The general admission rules are in +`docs/cross-repo-evidence-admission.md` and `docs/cadence-registration.md`. + +## Common intake package for every new sender + +Each sender owner must return, before audit-core prepares an overlay: + +1. **Source registration**: stable `source` name, owning repo, namespace/pod selectors, the emitting + identity, and the tenant policy (platform-scoped or per-tenant). +2. **Envelope mapping**: how native records map to the envelope in `docs/event-envelope.md` + (`event_id` derivation for idempotent retry, `observed_at`, `actor`, `action`, `outcome`, `tenant`, + `correlation_id`). The raw native record may travel as payload but must hold no secret value. +3. **Scope rows**: the action vocabulary the source may emit, so refusals are explicit. +4. **Volume and retention**: expected events/day and peak rate, `emission_cadence` with heartbeat and + the allowed silence threshold, and the required retention (default 30-day query window plus archive). +5. **Credential path**: the sender secret is minted through the attended OpenBao path routed by + `warden route`; audit-core never holds or prints it. `may_read` stays false. + +## Admit OpenBao audit forwarding + +```task +id: AUDIT-WP-0017-T01 +status: wait +priority: high +blocking_reason: "railiance-platform must supply a forwarder design for the OpenBao audit device (today audit only lands on a PVC) and the common intake package." +blocked_on: message-from:railiance-platform +``` + +Needs from railiance-platform: the forwarder (socket or file-tail device to the receiver), HMAC'd fields kept +HMAC'd (no plaintext secret or token in the payload), request-id mapping to `event_id`, and the volume estimate. +Fail-closed question for the owner: whether OpenBao should block on audit-sink loss (OpenBao's own semantics) +or buffer locally; audit-core needs the answer recorded, not a particular choice. +**Done when:** a scoped overlay is prepared, acceptance/retry/refusal/heartbeat are verified on synthetic +fixtures, and an owner-run activation receipt lands. + +## Admit Kubernetes API audit + +```task +id: AUDIT-WP-0017-T02 +status: wait +priority: high +blocking_reason: "railiance-cluster must supply the k3s audit policy (levels and excluded noise) and a forwarding path (webhook backend or log shipper) plus the common intake package." +blocked_on: message-from:railiance-cluster +``` + +Needs from railiance-cluster: audit policy that records Secret access at Metadata level only (never +RequestResponse for Secrets), `auditID` as `event_id`, user/impersonation fields as `actor`, and a volume +estimate after noise exclusion. audit-core checks the receiver can sustain that rate and sizes retention. +**Done when:** overlay prepared, fixture tests pass, owner-run activation receipt lands. + +## Admit access-engine decision records + +```task +id: AUDIT-WP-0017-T03 +status: wait +priority: high +depends_on: [AUDIT-WP-0013-T02] +blocking_reason: "Tracked by AUDIT-WP-0013-T02; waits on flex-auth/access-engine registrations and attended custody." +blocked_on: message-from:access-engine +``` + +No separate admission here. This task closes when AUDIT-WP-0013-T02 is done and the decision records carry the +correlation id from T05 (decision id plus the caller's correlation id). + +## Admit identity-provider events + +```task +id: AUDIT-WP-0017-T04 +status: wait +priority: high +blocking_reason: "NetKingdom must supply one intake package per provider (KeyCape, Authelia, LLDAP, privacyIDEA) and say which events are in scope." +blocked_on: message-from:net-kingdom +``` + +Needs from NetKingdom: per provider, the event set (login success/failure, MFA enrolment and challenge, token +issue/revoke, user and group changes, admin actions), the export mechanism (native webhook, log tail or +collector), and how the provider's session or request id maps to `correlation_id`. Four separate `source` +names, not one merged stream, so silence in one provider is detectable. +**Done when:** four overlays prepared, fixture tests pass, owner-run activation receipts land. + +## Define the correlation_id field and propagation contract + +```task +id: AUDIT-WP-0017-T05 +status: todo +priority: high +``` + +The envelope already carries `correlation_id` (free-form string) and the receiver serves +`GET ...?correlation_id=` across tenants (`audit_core/ingestion.py`). Proposal for the envelope side: + +- **Format**: an opaque string of at most 128 chars from `[A-Za-z0-9._:-]`, no PII and no secret. Prefer a + W3C trace-context `trace-id` (32 lowercase hex) when the caller has one, so the same value can sit in logs + and traces. +- **Required** for the new production sources; the receiver refuses an empty value for those sources and + keeps today's behaviour for already-admitted senders until they migrate. +- **Origin rule**: the first component in a chain (normally the IdP login or the onboarding request at + user-engine/tenant-engine) mints it; every later hop copies it unchanged. A component that has none mints one + and records `causation` metadata pointing at the native request id. +- **Optional `parent_event_id`** for the direct cause, so a chain can be ordered without trusting clocks. + +NetKingdom owns the cross-stream contract (which hop mints, the propagation header name, and the onboarding chain +test). audit-core owns the field definition, validation and query. Deliverables: update `docs/event-envelope.md`, +receiver validation behind a per-source flag, tests, and a synthetic onboarding-chain fixture +(IdP login, tenant-engine onboarding, access-engine decision, OpenBao read, Kubernetes API call) that is joined +by one id through the query endpoint. +**Done when:** NetKingdom accepts the field definition, the doc and tests are committed, and the fixture +query returns the whole chain. + +## Replace the single restore walk with a recurring drill + +```task +id: AUDIT-WP-0017-T06 +status: todo +priority: high +``` + +The only restore evidence is the 2026-08-13 walk. Define a recurring drill: quarterly, and after any schema or +backend change. Each drill restores the newest backup into an isolated namespace or database (never over the +live store), runs the integrity verifier over the restored chain, compares head digest, event count and newest +`observed_at` with the live head, and records a dated non-secret receipt under `evidence/`. The drill is +founder- or operator-run; audit-core supplies the script, the runbook section in `docs/operator-runbook.md` +and the receipt schema, tested against a synthetic store. A missed drill is a stale-evidence finding. +**Done when:** the procedure and script are committed, tested on fixtures, and the first owner-run drill +receipt lands. + +## Close the offsite chain head (AUDIT-WP-0015-T01) + +```task +id: AUDIT-WP-0017-T07 +status: wait +priority: high +depends_on: [AUDIT-WP-0015-T01] +blocking_reason: "AUDIT-WP-0015-T01 still waits on resource-control and rapp-postgres to name the chain-head custody owner and return offsite upload/readback receipts." +blocked_on: message-from:resource-control +``` + +Status 2026-10-27: AUDIT-WP-0015-T01 is `wait`. No owner or task for chain-head custody has been named and no +offsite upload/readback receipt exists. This task closes when that one does; it adds only the T06 link: each +drill also checks the restored head against the latest offsite head.