--- id: AUDIT-WP-0004 type: workplan title: "Make the event receiver correct and operable under load" domain: infotech repo: audit-core status: proposed owner: codex topic_slug: netkingdom created: "2026-08-10" updated: "2026-08-10" --- # AUDIT-WP-0004 - receiver correctness and hardening ## Goal Close the gap between what the receiver claims and what it does, and make it survive a real sender. This is the work that must land before audit-core is worth deploying. A pre-deploy review of the WP-0003 implementation found that the ingestion path never calls the `AuditBackend` contract, that several failure modes escape the error handler and drop the connection, and that two isolation properties the workplan claims as done are not implemented. None of these depend on the choice of storage engine, so this workplan runs in parallel with RAPP-POSTGRES-WP-0002 and gates AUDIT-WP-0005. ## Boundaries This workplan owns the receiver's correctness, error semantics, and operable surface. It does not own deployment, the production storage engine, or the live failure matrix — those are AUDIT-WP-0005. Work here stays storage-agnostic: the existing SQLite store remains the development and test backend, and the Postgres implementation lands in WP-0005 against the interface this workplan fixes. ## T01 - Route ingestion through the audit backend contract ```task id: AUDIT-WP-0004-T01 status: todo priority: high ``` `ingestion.py` writes records into SQLite directly and never calls `AuditBackend.emit()`. WP-0002 built a pluggable backend contract with a declared `RetentionPolicy`; WP-0003 built an ingestion path that bypasses it. A 202 currently means "a row exists in a file", against a store that declares no custody class at all. Make the ingestion path write through the backend contract, so that accepting an event means a backend with a declared retention policy acknowledged it. Idempotency state and custody write must not be able to diverge — an event recorded as seen but not durably stored is the failure this whole service exists to prevent. Add a readiness condition that refuses to serve if the configured backend declares `durable=False`, so the development file backend can never silently become the production sink. Done when a successful response is backed by a durable backend acknowledgment, and the mock backend cannot be used to serve production traffic. ## T02 - Fix error semantics and failure handling ```task id: AUDIT-WP-0004-T02 status: todo priority: high ``` The handler catches `ValueError`, `TypeError`, `KeyError`, and `JSONDecodeError`. Storage errors do not inherit from those: `sqlite3.OperationalError` on lock contention and `sqlite3.IntegrityError` on the race between the existence check and the insert both escape with `start_response` never called. The auth check sits outside the try block, so a non-ASCII `Authorization` header raises `TypeError` from `hmac.compare_digest` and crashes the request before any handler runs. Add a catch-all that always produces a response, map storage and unexpected failures to 503 or 500 rather than letting them surface as dropped connections, and move credential comparison inside the guarded path. Correct the status codes: a duplicate event ID carrying a different payload is a conflict and must return 409, not 400. This distinction drives the sender's retry-versus-dead-letter decision, so the full set of responses — accepted, duplicate, conflict, rejected, unavailable — needs to be documented as a contract with the retry semantics each implies. Done when no request path can terminate without a response, and every status code the receiver returns is deliberate and documented. ## T03 - Enforce the isolation properties already claimed ```task id: AUDIT-WP-0004-T03 status: todo priority: high ``` WP-0003 T01 and T02 both record tenant isolation and cross-tenant claim rejection as done. `normalize()` accepts whatever `tenant` string the caller sends and never checks it against the authenticated identity; no test covers it. Separately, `source` is hardcoded to `"user-engine"`, so the allowlist is unbound to the credential — any holder of the token can claim to be user-engine, and a second sender would require a code change. Bind the authenticated identity to the tenants and sources it may write for, and reject claims outside that binding. Drive the binding from configuration rather than literals. Done when a credential scoped to one tenant is refused when it claims another, and that refusal is covered by a test. ## T04 - Settle redaction policy ```task id: AUDIT-WP-0004-T04 status: todo priority: medium ``` `_contains_secret` rejects the entire event when any key name in `data` contains a secret-shaped fragment, and never inspects the envelope. WP-0003 T01 describes "redacted data" and the failure matrix tests redaction, but the implementation only rejects. Decide whether a secret-shaped field is a rejection or a redaction, and note the consequence: rejection means a legitimate outbox event containing a field named `token_count` is dead-lettered at the sender, and the audit trail loses the event entirely. Extend detection to the envelope, and consider value shape rather than key name alone. Whatever is chosen, the contract document and the sender's expectations must agree with the code. Done when the redaction rule is explicit, applied to the whole payload, and consistent between the contract and the implementation. ## T05 - Provide the operator read surface ```task id: AUDIT-WP-0004-T05 status: todo priority: high ``` There is no way to read anything back. The failure matrix requires dead-letter visibility, operator replay, and correlation lookup; none exist, so that evidence cannot be produced regardless of how the deployment goes. Build lookup by event ID and by correlation ID, a dead-letter view of rejected events with their rejection reason, and an operator replay path. Replay must be idempotent against the same durable store, so a replayed event reconciles with the original rather than creating a second custody record. Read access is a separate privilege from write access — a sender credential must not be able to read the audit trail back. Done when an operator can trace one correlation ID through the system and replay a specific event without creating a duplicate. ## T06 - Serving layer and observability ```task id: AUDIT-WP-0004-T06 status: todo priority: high ``` `wsgiref.simple_server` is single-threaded with no request timeout, no graceful shutdown, and no access log. One slow client blocks every sender. It also makes the failure matrix meaningless: "receiver correctly reported unavailable" and "the server stalled" are indistinguishable. Move to a production WSGI server with explicit worker and timeout configuration and graceful shutdown on SIGTERM, so in-flight events are not lost on rollout. Add structured request logging with a request ID, propagate the correlation ID into logs, and expose counters for accepted, duplicate, conflicted, rejected, and failed events plus write latency. Logs and metrics must never carry event payloads. Done when the receiver serves concurrent senders under a bounded timeout, sheds load predictably instead of stalling, and its behaviour is visible from outside. ## T07 - Close the test gaps ```task id: AUDIT-WP-0004-T07 status: todo priority: medium ``` Ingestion has two tests. Uncovered: oversized and zero-length bodies, absent `Content-Length`, malformed JSON, wrong path and method, the health endpoints, event ID conflict, naive and out-of-range timestamps, storage failure, and persistence across restart. `accepted_at` is recorded in local time while every other timestamp is UTC; `datetime.fromisoformat` accepts naive timestamps and arbitrary dates. Fix both and cover them. Done when each rejection reason and each failure mode above has a test that asserts the documented status code.