audit-core/workplans/AUDIT-WP-0003-user-engine-event-ingestion-service.md
tegwick 3e9ec78f91
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Rescope WP-0003 and open WP-0004/WP-0005 after pre-deploy review
A pre-deploy review found the receiver is not deployable as built. The
ingestion path never calls AuditBackend.emit(), so a 202 means a SQLite row
exists, not that a backend with a declared retention policy accepted the
event. Storage exceptions escape the handler with start_response never
called. Tenant isolation and source binding are recorded as done but are not
implemented. There is no read, replay, or correlation-lookup surface, so the
failure matrix cannot produce the evidence NK-WP-0024 needs.

WP-0003 closes as finished on its narrowed scope (contract + reference
implementation, T01/T02). T03 and T04 are cancelled with rationale.

WP-0004 covers receiver correctness and hardening, storage-agnostic so it
runs in parallel with the database platform work.

WP-0005 covers the Postgres backend, deployment, SQLite record migration,
and the live failure matrix. Production storage moves from SQLite-on-a-volume
to the Railiance PostgreSQL platform (RAPP-POSTGRES-WP-0002).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 12:44:23 +02:00

4.6 KiB

id type title domain repo status owner topic_slug created updated depends_on state_hub_workstream_id
AUDIT-WP-0003 workplan Provide durable user-engine event ingestion infotech audit-core finished codex netkingdom 2026-08-08 2026-08-10
NK-WP-0024
d2726f51-2beb-4c6b-96c4-98d29d11a93f

AUDIT-WP-0003 - user-engine event ingestion service

Provide the authenticated, durable cluster event receiver for user-engine's transactional outbox. Reuse audit-core's intended POST /v1/events boundary; do not turn user-engine into an audit-retention service.

T01 - Finalize the ingestion contract

id: AUDIT-WP-0003-T01
status: done
priority: high
state_hub_task_id: "620a6910-60d6-4ef0-b82b-37eacb2fcf1d"

Map the user-engine envelope fields (id, type, source, subject, tenant, correlation_id, occurred_at, data) into the normalized audit-core event schema. Define validation, redaction, size limits, tenant isolation, Idempotency-Key behavior, and stable HTTP error semantics.

Done when valid events preserve correlation and duplicate event IDs are accepted idempotently without duplicate custody records.

Done 2026-08-09: the normalized contract preserves event, tenant, correlation, occurrence, source, subject, type, and redacted data; request size, timestamp, source, secret-shaped fields, and Idempotency-Key are validated.

T02 - Implement authenticated durable ingestion

id: AUDIT-WP-0003-T02
status: done
priority: high
state_hub_task_id: "a6fce44e-8977-47f0-8a75-9ce431cc27a6"

Implement POST /v1/events with scoped workload authentication, schema validation, idempotency storage, bounded request handling, and durable backend write acknowledgment. Reject missing/invalid credentials, cross-tenant claims, secret-shaped fields, oversized payloads, and malformed timestamps.

Done when a successful response means the event is durably accepted and retryable failures do not lose or duplicate evidence.

Done 2026-08-09: POST /v1/events requires a constant-time bearer check and stores normalized records plus payload hashes in SQLite. Exact duplicates return 200 without a second record; conflicting IDs and malformed events fail. All 15 repository tests pass.

T03 - Deploy the single-cluster receiver

id: AUDIT-WP-0003-T03
status: cancel
priority: high
state_hub_task_id: "fc0b5850-1954-448c-8729-ffe93d7b530f"

Publish an immutable image and deploy audit-core API plus durable backend on railiance01. Provide Service discovery, health probes, resource/security controls, default-deny NetworkPolicy, backup/restore, retention, and rollback. Deliver the user-engine sender credential through the approved OpenBao lane.

Done when only the user-engine workload can use its sender identity and the receiver survives pod restart without losing idempotency state.

Cancelled 2026-08-10, superseded by AUDIT-WP-0005. A pre-deploy review found that the receiver is not deployable as built: the ingestion path bypasses the AuditBackend contract entirely and writes to SQLite directly, so a 202 does not mean the event reached audit custody. Deployment also presumed SQLite on a volume as the production store; it will instead use the Railiance PostgreSQL platform (RAPP-POSTGRES-WP-0002). The deploy work moves to AUDIT-WP-0005 and the receiver correctness work it depends on moves to AUDIT-WP-0004.

T04 - Prove delivery, retry, and replay

id: AUDIT-WP-0003-T04
status: cancel
priority: high
state_hub_task_id: "8d1624de-3e71-41c9-80ac-ab522313a0da"

Exercise successful delivery, receiver timeout/unavailability, bounded user-engine retry, dead-letter visibility, operator replay, duplicate replay, redaction, and correlation lookup through the deployed path. Hand non-secret evidence back to NK-WP-0024.

Done when one source outbox event produces exactly one durable normalized event across retries and replay.

Cancelled 2026-08-10, superseded by AUDIT-WP-0005-T05. The matrix cannot run against the current build: there is no read surface, no replay tool, and no correlation lookup, so the evidence NK-WP-0024 needs cannot be produced. Those are build work and move to AUDIT-WP-0004-T05; the live run moves to AUDIT-WP-0005-T05.

Scope note (2026-08-10)

This workplan is complete as the contract and reference implementation strand: T01 fixed the ingestion contract and T02 built a working, tested receiver against it. Production hardening and deployment were rescoped out into AUDIT-WP-0004 and AUDIT-WP-0005 after review. WP-0003 closes as finished on that narrowed scope rather than staying open across work that now belongs to two successor plans.