audit-core/workplans/AUDIT-WP-0004-receiver-correctness-and-hardening.md

---
id: AUDIT-WP-0004
type: workplan
title: "Make the event receiver correct and operable under load"
domain: infotech
repo: audit-core
status: proposed
owner: codex
topic_slug: netkingdom
created: "2026-08-10"
updated: "2026-08-10"
---

# AUDIT-WP-0004 - receiver correctness and hardening

## Goal

Close the gap between what the receiver claims and what it does, and make it
survive a real sender. This is the work that must land before audit-core is
worth deploying.

A pre-deploy review of the WP-0003 implementation found that the ingestion
path never calls the `AuditBackend` contract, that several failure modes
escape the error handler and drop the connection, and that two isolation
properties the workplan claims as done are not implemented. None of these
depend on the choice of storage engine, so this workplan runs in parallel
with RAPP-POSTGRES-WP-0002 and gates AUDIT-WP-0005.

## Boundaries

This workplan owns the receiver's correctness, error semantics, and operable
surface. It does not own deployment, the production storage engine, or the
live failure matrix — those are AUDIT-WP-0005.

Work here stays storage-agnostic: the existing SQLite store remains the
development and test backend, and the Postgres implementation lands in
WP-0005 against the interface this workplan fixes.

## T01 - Route ingestion through the audit backend contract

```task
id: AUDIT-WP-0004-T01
status: todo
priority: high
```

`ingestion.py` writes records into SQLite directly and never calls
`AuditBackend.emit()`. WP-0002 built a pluggable backend contract with a
declared `RetentionPolicy`; WP-0003 built an ingestion path that bypasses it.
A 202 currently means "a row exists in a file", against a store that declares
no custody class at all.

Make the ingestion path write through the backend contract, so that accepting
an event means a backend with a declared retention policy acknowledged it.
Idempotency state and custody write must not be able to diverge — an event
recorded as seen but not durably stored is the failure this whole service
exists to prevent.

Add a readiness condition that refuses to serve if the configured backend
declares `durable=False`, so the development file backend can never silently
become the production sink.

Done when a successful response is backed by a durable backend
acknowledgment, and the mock backend cannot be used to serve production
traffic.

## T02 - Fix error semantics and failure handling

```task
id: AUDIT-WP-0004-T02
status: todo
priority: high
```

The handler catches `ValueError`, `TypeError`, `KeyError`, and
`JSONDecodeError`. Storage errors do not inherit from those:
`sqlite3.OperationalError` on lock contention and `sqlite3.IntegrityError` on
the race between the existence check and the insert both escape with
`start_response` never called. The auth check sits outside the try block, so a
non-ASCII `Authorization` header raises `TypeError` from
`hmac.compare_digest` and crashes the request before any handler runs.

Add a catch-all that always produces a response, map storage and unexpected
failures to 503 or 500 rather than letting them surface as dropped
connections, and move credential comparison inside the guarded path.

Correct the status codes: a duplicate event ID carrying a different payload is
a conflict and must return 409, not 400. This distinction drives the sender's
retry-versus-dead-letter decision, so the full set of responses — accepted,
duplicate, conflict, rejected, unavailable — needs to be documented as a
contract with the retry semantics each implies.

Done when no request path can terminate without a response, and every status
code the receiver returns is deliberate and documented.

## T03 - Enforce the isolation properties already claimed

```task
id: AUDIT-WP-0004-T03
status: todo
priority: high
```

WP-0003 T01 and T02 both record tenant isolation and cross-tenant claim
rejection as done. `normalize()` accepts whatever `tenant` string the caller
sends and never checks it against the authenticated identity; no test covers
it. Separately, `source` is hardcoded to `"user-engine"`, so the allowlist is
unbound to the credential — any holder of the token can claim to be
user-engine, and a second sender would require a code change.

Bind the authenticated identity to the tenants and sources it may write for,
and reject claims outside that binding. Drive the binding from configuration
rather than literals.

Done when a credential scoped to one tenant is refused when it claims
another, and that refusal is covered by a test.

## T04 - Settle redaction policy

```task
id: AUDIT-WP-0004-T04
status: todo
priority: medium
```

`_contains_secret` rejects the entire event when any key name in `data`
contains a secret-shaped fragment, and never inspects the envelope. WP-0003
T01 describes "redacted data" and the failure matrix tests redaction, but the
implementation only rejects.

Decide whether a secret-shaped field is a rejection or a redaction, and note
the consequence: rejection means a legitimate outbox event containing a field
named `token_count` is dead-lettered at the sender, and the audit trail loses
the event entirely. Extend detection to the envelope, and consider value shape
rather than key name alone.

Whatever is chosen, the contract document and the sender's expectations must
agree with the code.

Done when the redaction rule is explicit, applied to the whole payload, and
consistent between the contract and the implementation.

## T05 - Provide the operator read surface

```task
id: AUDIT-WP-0004-T05
status: todo
priority: high
```

There is no way to read anything back. The failure matrix requires dead-letter
visibility, operator replay, and correlation lookup; none exist, so that
evidence cannot be produced regardless of how the deployment goes.

Build lookup by event ID and by correlation ID, a dead-letter view of rejected
events with their rejection reason, and an operator replay path. Replay must
be idempotent against the same durable store, so a replayed event reconciles
with the original rather than creating a second custody record.

Read access is a separate privilege from write access — a sender credential
must not be able to read the audit trail back.

Done when an operator can trace one correlation ID through the system and
replay a specific event without creating a duplicate.

## T06 - Serving layer and observability

```task
id: AUDIT-WP-0004-T06
status: todo
priority: high
```

`wsgiref.simple_server` is single-threaded with no request timeout, no
graceful shutdown, and no access log. One slow client blocks every sender.
It also makes the failure matrix meaningless: "receiver correctly reported
unavailable" and "the server stalled" are indistinguishable.

Move to a production WSGI server with explicit worker and timeout
configuration and graceful shutdown on SIGTERM, so in-flight events are not
lost on rollout.

Add structured request logging with a request ID, propagate the correlation ID
into logs, and expose counters for accepted, duplicate, conflicted, rejected,
and failed events plus write latency. Logs and metrics must never carry event
payloads.

Done when the receiver serves concurrent senders under a bounded timeout,
sheds load predictably instead of stalling, and its behaviour is visible from
outside.

## T07 - Close the test gaps

```task
id: AUDIT-WP-0004-T07
status: todo
priority: medium
```

Ingestion has two tests. Uncovered: oversized and zero-length bodies, absent
`Content-Length`, malformed JSON, wrong path and method, the health
endpoints, event ID conflict, naive and out-of-range timestamps, storage
failure, and persistence across restart.

`accepted_at` is recorded in local time while every other timestamp is UTC;
`datetime.fromisoformat` accepts naive timestamps and arbitrary dates. Fix
both and cover them.

Done when each rejection reason and each failure mode above has a test that
asserts the documented status code.