Route ingestion through the backend contract; fix error semantics
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

AUDIT-WP-0004 T01, T02, T07.

T01 - ingestion wrote to SQLite directly and never called the AuditBackend
contract, so a 202 meant a row existed rather than that a backend with a
declared retention policy had accepted the event. Adds IdempotentAuditBackend
to the contract: duplicate detection lives inside the backend so custody and
idempotency state share a transaction and cannot diverge. SQLiteAuditBackend
implements it with WAL, synchronous=FULL and a busy timeout. Ingestion now
refuses any backend declaring durable=False, so the development file backend
cannot silently become the production sink.

The atomicity claim was tested rather than asserted, and the first attempt
failed: with a single shared connection, 16 racing submissions of one event
told two callers they were first. Storage was correct but the response was
not. Fixed with per-thread connections and BEGIN IMMEDIATE around the
insert/read pair, and locked in by a test.

T02 - storage errors previously escaped the handler with start_response never
called, and the auth check sat outside the try block so a non-ASCII
Authorization header crashed the request. Adds a catch-all, maps conflict to
409, backend unavailability to 503 and unexpected faults to 500, and
documents the full response contract with the retry semantics each status
implies, since senders key their behaviour off it.

T07 - ingestion tests 2 -> 23, suite 15 -> 36. accepted_at is now UTC rather
than local time, and naive timestamps are rejected instead of silently
assumed.

Remaining in WP-0004: T03 tenant/source binding, T04 redaction policy,
T05 operator read surface, T06 production serving layer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-10 14:30:20 +02:00
parent f45c4f2511
commit eb649dd747
8 changed files with 693 additions and 92 deletions

View file

@ -9,6 +9,7 @@ owner: codex
topic_slug: netkingdom
created: "2026-08-10"
updated: "2026-08-10"
state_hub_workstream_id: "f3345e90-f466-4184-b149-9b0be92ffec8"
---
# AUDIT-WP-0004 - receiver correctness and hardening
@ -40,8 +41,9 @@ WP-0005 against the interface this workplan fixes.
```task
id: AUDIT-WP-0004-T01
status: todo
status: done
priority: high
state_hub_task_id: "9b194ebc-f853-48c5-a65b-88b0eae3d7a2"
```
`ingestion.py` writes records into SQLite directly and never calls
@ -64,12 +66,25 @@ Done when a successful response is backed by a durable backend
acknowledgment, and the mock backend cannot be used to serve production
traffic.
Done 2026-08-10: added `IdempotentAuditBackend` to the contract — idempotency
lives inside the backend so custody and duplicate-detection share one
transaction and cannot diverge. `SQLiteAuditBackend` implements it with WAL,
`synchronous=FULL`, and a busy timeout. Ingestion refuses any backend
declaring `durable=False`, so the mock file backend cannot serve.
The atomicity claim was tested rather than asserted, and the first
implementation failed: with one shared connection, 16 racing submissions of
one event told **two** callers they were first. Fixed with per-thread
connections and `BEGIN IMMEDIATE` around the insert/read pair. Now covered by
`test_concurrent_duplicates_produce_exactly_one_record`.
## T02 - Fix error semantics and failure handling
```task
id: AUDIT-WP-0004-T02
status: todo
status: done
priority: high
state_hub_task_id: "b193acaf-9c0e-411a-921b-13282bee8325"
```
The handler catches `ValueError`, `TypeError`, `KeyError`, and
@ -93,12 +108,20 @@ contract with the retry semantics each implies.
Done when no request path can terminate without a response, and every status
code the receiver returns is deliberate and documented.
Done 2026-08-10: catch-all wrapper guarantees a response on every path.
Conflict is now 409, backend unavailability 503, unexpected faults 500.
Credential comparison moved inside the guarded path and a non-ASCII
`Authorization` header is a 401 rather than a crash. The full response
contract with its retry semantics is documented in the `ingestion` module
docstring.
## T03 - Enforce the isolation properties already claimed
```task
id: AUDIT-WP-0004-T03
status: todo
priority: high
state_hub_task_id: "c29be4e7-4c2e-47f0-9d37-7d72061274ee"
```
WP-0003 T01 and T02 both record tenant isolation and cross-tenant claim
@ -121,6 +144,7 @@ another, and that refusal is covered by a test.
id: AUDIT-WP-0004-T04
status: todo
priority: medium
state_hub_task_id: "5cf5c412-965a-4a56-aeef-e965f2861c51"
```
`_contains_secret` rejects the entire event when any key name in `data`
@ -146,6 +170,7 @@ consistent between the contract and the implementation.
id: AUDIT-WP-0004-T05
status: todo
priority: high
state_hub_task_id: "006bc4ca-de36-4152-afae-0eef2a402e73"
```
There is no way to read anything back. The failure matrix requires dead-letter
@ -169,6 +194,7 @@ replay a specific event without creating a duplicate.
id: AUDIT-WP-0004-T06
status: todo
priority: high
state_hub_task_id: "348c2f4c-3ab0-46c7-9ddd-b198122f58ed"
```
`wsgiref.simple_server` is single-threaded with no request timeout, no
@ -193,8 +219,9 @@ outside.
```task
id: AUDIT-WP-0004-T07
status: todo
status: done
priority: medium
state_hub_task_id: "b8141609-858d-41e1-9cc4-eb6f2723d561"
```
Ingestion has two tests. Uncovered: oversized and zero-length bodies, absent
@ -208,3 +235,10 @@ both and cover them.
Done when each rejection reason and each failure mode above has a test that
asserts the documented status code.
Done 2026-08-10: ingestion tests went from 2 to 23 (suite 15 -> 36). Covers
oversized, empty and truncated bodies, absent/invalid `Content-Length`,
malformed JSON, wrong path and method, health endpoints, id conflict,
naive and unparseable timestamps, backend unavailability, unexpected backend
faults, durability across reopen, and the concurrency race. `accepted_at` is
UTC; naive timestamps are rejected rather than silently assumed.

View file

@ -13,6 +13,7 @@ depends_on:
- AUDIT-WP-0004
- RAPP-POSTGRES-WP-0002
- NK-WP-0024
state_hub_workstream_id: "7b24a844-c9f2-4d2d-ac7d-20978bdf6b38"
---
# AUDIT-WP-0005 - Postgres store and production deployment
@ -50,6 +51,7 @@ requirement and consumes it; it does not implement it here.
id: AUDIT-WP-0005-T01
status: todo
priority: high
state_hub_task_id: "b1601d0b-922a-40f7-92c0-ea06af6c4468"
```
Implement `AuditBackend` against PostgreSQL, declaring an honest
@ -80,6 +82,7 @@ restart mid-write does not produce an acknowledged-but-absent event.
id: AUDIT-WP-0005-T02
status: todo
priority: high
state_hub_task_id: "831b2472-0d80-4369-a5e3-eb08ef3526b1"
```
Declare audit-core's database requirement against rapp-postgres and take
@ -101,6 +104,7 @@ events.
id: AUDIT-WP-0005-T03
status: todo
priority: high
state_hub_task_id: "598af2ac-e772-4a4e-9a65-dde9d4ca167f"
```
Publish an immutable image — base pinned by digest, not a mutable tag, and
@ -129,6 +133,7 @@ previous version.
id: AUDIT-WP-0005-T04
status: todo
priority: medium
state_hub_task_id: "9010fb4a-a1b8-4ef7-b143-e33ca7cc0619"
```
Any events accepted by the pre-production SQLite receiver are audit records
@ -148,6 +153,7 @@ and verified, or explicitly and justifiably discarded.
id: AUDIT-WP-0005-T05
status: todo
priority: high
state_hub_task_id: "1da30fec-b9f1-4be0-b42c-15797a8c4392"
```
Exercise the deployed path: successful delivery; receiver timeout and
@ -172,6 +178,7 @@ documented contract.
id: AUDIT-WP-0005-T06
status: todo
priority: medium
state_hub_task_id: "0856c80d-abe1-4bff-ba8d-87295cf76819"
```
Document what an operator needs: how to look up an event by correlation ID,