audit-core/docs/evidence/AUDIT-WP-0008-T07-v1-2026-09-24.md
tegwick 8d3328b620
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 4s
Record the availability exercise and finish the tenancy posture plan.
Assistant: grok
Assistant-Session: 01a0d25d-d358-7e13-b84a-d007fbb7e34f
2026-09-24 15:53:09 +02:00

2.8 KiB

AUDIT-WP-0008-T07 availability exercise — 2026-09-24

Operation measured: POST /v1/events accepted into custody. Authority: founder request in the custodian session, recorded as ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED. Not performed: runtime-lease revocation, OpenBao shutdown, node reboot.

The probe ran inside the receiver, used the already-mounted sender registry, and printed only a timestamp and an HTTP class. No credential was printed. Each successful probe is a synthetic audit.recovery.synthetic event from user-engine for tenant:platform.

Receiver

Step Wall clock (UTC) Result
E1 last accept before rollout 13:45:37 202 accepted
E1 rollout restart issued 13:45:35 workstation new pod scheduled
E1 probes 13:45:36, 13:45:39 exec failed, pod not usable
E1 next accept 13:45:46 202 accepted (audit-wp-0008-t07-e1-134541)
E2 pod deleted 13:46:12 audit-core-66767ddd58-mcxtm
E2 probe 13:46:13 exec failed
E2 next accept 13:46:19 202 accepted (audit-wp-0008-t07-e2-134615)

Receiver recreate recovered in about 7 seconds (E2). The rollout (E1) recovered in about 11 seconds. Both new processes accepted an event, so the mounted database credential and the sender registry were readable at process start.

Custody

platform-pg is one primary. tenant-engine and core-hub share it. Both were told immediately before the delete.

Step Wall clock (UTC) Result
Last accept before delete 13:46:56 202 accepted
kubectl delete pod platform-pg-1 issued 13:46:54 workstation command waited for termination
Delete returned 13:49:59 pod object gone
First probe after that 13:50:13 unreachable
Next accept 13:50:21 202 accepted (audit-wp-0008-t07-e3-135015)

The three minutes inside kubectl delete were not sampled, so they are not claimed as an outage. After the primary pod was gone, the next successful accept was 22 seconds later. The recreated platform-pg-1 was Ready, restart count 0. The CloudNativePG cluster reported healthy. tenant-engine and core-hub pods were still Running.

Integrity

python3 -m audit_core verify-chain --schema audit_core after E3: intact: true, events: 125, first_break: null. The count is the pre-exercise chain plus the accepted probes only. No accepted probe is missing.

What this evidences

V1 for the accept path: dependencies were already enumerated in docs/availability-recovery.md, receiver recreate and primary recreate were exercised, and the interruption was measured. The binding recovery is the custody restart, 22 seconds from pod termination to the next accept. V2 remains unreachable while platform-pg has one instance. Review this exercise by 2026-12-31.