coulomb/audit-core

Fork 0

History

tegwick 4de672aaf0 Some checks failed CI Smoke / host-smoke (push) Successful in 0s Details CI Smoke / container-smoke (push) Has been cancelled Details Make the failure matrix an executable harness AUDIT-WP-0005-T05 (progress). scripts/failure_matrix.py, make failure-matrix. Two modes: MODE=local stands up PostgreSQL and the receiver in Docker and runs all 15 scenarios including infrastructure disruption; MODE=remote targets a deployed receiver and skips disruption unless DISRUPT=1, since restarting a production database is not this script's call. Rehearsed locally: 15 passed, 0 failed. Delivery and reconciliation, rejection and dead-letter visibility, redaction with per-path counting, correlation lookup, privilege separation both directions, credential rotation mid-ingestion with no delivery gap, operator replay and duplicate replay, receiver unavailability with sender retry, and a database restart mid-ingestion where 5 of 7 attempts were acknowledged and all 5 survived. Two deliberate choices. Stored-record counts are read straight from the database rather than through the API, because the assertion is about what is stored and asking the service to vouch for itself is weaker evidence. The retry policy retries 503/500 and treats 400/401/403/409 as terminal, which is the documented response contract - so what is under test is a sender that follows it. Harness credibility checked rather than assumed: exit 0 on success, exit 2 against an unreachable receiver rather than passing silently, and the evidence JSON carries no tokens, credentials or event payloads so it can go to NK-WP-0024 as-is. The local rehearsal is not a substitute for the live run: it does not exercise CNPG failover, NetworkPolicy enforcement, or OpenBao-leased credentials. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-10 17:49:32 +02:00
..
README.md	Make the failure matrix an executable harness	2026-08-10 17:49:32 +02:00

tegwick 4de672aaf0

CI Smoke / host-smoke (push) Successful in 0s

Details

CI Smoke / container-smoke (push) Has been cancelled

Details

Make the failure matrix an executable harness

AUDIT-WP-0005-T05 (progress). scripts/failure_matrix.py, make failure-matrix.

Two modes: MODE=local stands up PostgreSQL and the receiver in Docker and runs
all 15 scenarios including infrastructure disruption; MODE=remote targets a
deployed receiver and skips disruption unless DISRUPT=1, since restarting a
production database is not this script's call.

Rehearsed locally: 15 passed, 0 failed. Delivery and reconciliation, rejection
and dead-letter visibility, redaction with per-path counting, correlation
lookup, privilege separation both directions, credential rotation mid-ingestion
with no delivery gap, operator replay and duplicate replay, receiver
unavailability with sender retry, and a database restart mid-ingestion where 5
of 7 attempts were acknowledged and all 5 survived.

Two deliberate choices. Stored-record counts are read straight from the
database rather than through the API, because the assertion is about what is
stored and asking the service to vouch for itself is weaker evidence. The
retry policy retries 503/500 and treats 400/401/403/409 as terminal, which is
the documented response contract - so what is under test is a sender that
follows it.

Harness credibility checked rather than assumed: exit 0 on success, exit 2
against an unreachable receiver rather than passing silently, and the evidence
JSON carries no tokens, credentials or event payloads so it can go to
NK-WP-0024 as-is.

The local rehearsal is not a substitute for the live run: it does not exercise
CNPG failover, NetworkPolicy enforcement, or OpenBao-leased credentials.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-10 17:49:32 +02:00

README.md

Make the failure matrix an executable harness

2026-08-10 17:49:32 +02:00

README.md

Evidence

Run artifacts from scripts/failure_matrix.py (AUDIT-WP-0005-T05) land here as failure-matrix-<timestamp>.json. They are gitignored: they accumulate per run and the durable record of an outcome belongs in the workplan.

The JSON is deliberately non-secret — no tokens, credentials, or event payloads — so a run against the deployed receiver can be handed to NK-WP-0024 as-is.

make failure-matrix                 # local: full matrix incl. disruption
MODE=remote BASE_URL=https://audit-core.… make failure-matrix