tegwick
4de672aaf0
AUDIT-WP-0005-T05 (progress). scripts/failure_matrix.py, make failure-matrix. Two modes: MODE=local stands up PostgreSQL and the receiver in Docker and runs all 15 scenarios including infrastructure disruption; MODE=remote targets a deployed receiver and skips disruption unless DISRUPT=1, since restarting a production database is not this script's call. Rehearsed locally: 15 passed, 0 failed. Delivery and reconciliation, rejection and dead-letter visibility, redaction with per-path counting, correlation lookup, privilege separation both directions, credential rotation mid-ingestion with no delivery gap, operator replay and duplicate replay, receiver unavailability with sender retry, and a database restart mid-ingestion where 5 of 7 attempts were acknowledged and all 5 survived. Two deliberate choices. Stored-record counts are read straight from the database rather than through the API, because the assertion is about what is stored and asking the service to vouch for itself is weaker evidence. The retry policy retries 503/500 and treats 400/401/403/409 as terminal, which is the documented response contract - so what is under test is a sender that follows it. Harness credibility checked rather than assumed: exit 0 on success, exit 2 against an unreachable receiver rather than passing silently, and the evidence JSON carries no tokens, credentials or event payloads so it can go to NK-WP-0024 as-is. The local rehearsal is not a substitute for the live run: it does not exercise CNPG failover, NetworkPolicy enforcement, or OpenBao-leased credentials. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
45 lines
1.8 KiB
Makefile
45 lines
1.8 KiB
Makefile
SHELL := /usr/bin/env bash
|
|
.DEFAULT_GOAL := help
|
|
|
|
AUDIT_CORE_MOCK_DIR ?= /tmp/audit-core
|
|
|
|
test: ## Run unit tests
|
|
python3 -m pytest -q
|
|
|
|
mock-audit-smoke: ## Write one non-secret smoke event to the mock audit backend
|
|
AUDIT_CORE_MOCK_DIR="$(AUDIT_CORE_MOCK_DIR)" python3 -m audit_core emit \
|
|
--source audit-core \
|
|
--action audit_core.mock_backend.smoke \
|
|
--resource "$(AUDIT_CORE_MOCK_DIR)" \
|
|
--outcome success \
|
|
--detail backend=mock-file
|
|
|
|
mock-audit-cleanup: ## Remove mock audit files older than the retention window
|
|
AUDIT_CORE_MOCK_DIR="$(AUDIT_CORE_MOCK_DIR)" python3 -m audit_core cleanup
|
|
|
|
help: ## Show this help
|
|
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} \
|
|
/^[a-zA-Z_-]+:.*?##/ { printf " \033[36m%-24s\033[0m %s\n", $$1, $$2 }' $(MAKEFILE_LIST)
|
|
|
|
.PHONY: test test-pg pg-test-up pg-test-down failure-matrix mock-audit-smoke mock-audit-cleanup help
|
|
|
|
PG_TEST_CONTAINER ?= ac-pg-test
|
|
PG_TEST_PORT ?= 55445
|
|
PG_TEST_URL ?= postgresql://postgres:test@127.0.0.1:$(PG_TEST_PORT)/audit_core
|
|
|
|
pg-test-up: ## Start a throwaway PostgreSQL for backend conformance tests
|
|
-docker rm -f $(PG_TEST_CONTAINER) 2>/dev/null
|
|
docker run -d --name $(PG_TEST_CONTAINER) -e POSTGRES_PASSWORD=test \
|
|
-e POSTGRES_DB=audit_core -p 127.0.0.1:$(PG_TEST_PORT):5432 postgres:16-alpine
|
|
@for i in $$(seq 1 40); do \
|
|
docker exec $(PG_TEST_CONTAINER) pg_isready -U postgres >/dev/null 2>&1 && exit 0; \
|
|
sleep 1; done; echo "postgres did not become ready" >&2; exit 1
|
|
|
|
pg-test-down: ## Remove the throwaway PostgreSQL
|
|
-docker rm -f $(PG_TEST_CONTAINER)
|
|
|
|
test-pg: ## Run the suite including PostgreSQL conformance (needs pg-test-up)
|
|
AUDIT_CORE_TEST_DATABASE_URL="$(PG_TEST_URL)" python3 -m pytest -q
|
|
|
|
failure-matrix: ## Run the delivery/retry/replay failure matrix (AUDIT-WP-0005-T05)
|
|
python3 scripts/failure_matrix.py
|