coulomb/audit-core
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Record the availability exercise and finish the tenancy posture plan.
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
Details
CI Smoke / container-smoke (push) Successful in 4s
Details

Browse source 
Assistant: grok
Assistant-Session: 01a0d25d-d358-7e13-b84a-d007fbb7e34f
This commit is contained in:
tegwick 2026-09-24 15:53:09 +02:00
parent 6b46d1aaca
commit 8d3328b620
4 changed files with 104 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

42
docs/availability-recovery.md
View file

@ -1,25 +1,15 @@
# Availability and recovery
**Status:** V1 not yet evidenced. Dependency enumeration complete; exercise
pending a live window on railiance01.
**Status:** V1 evidenced 2026-09-24. Review by 2026-12-31.
**Framework:** NetKingdom Tenancy Posture v0.1 (draft-8) §4.6, §13 V-row.
**Declared:** `tenancy.yaml` — `current.V: 0`, `target.V: 1`.
**Declared:** `tenancy.yaml` — `current.V: 1`, `target.V: 1`.
## Why V0 today
## Why this is V1
`V0` is "no availability or recovery position; recovery is untested or depends
on improvisation". That is accurate. Nothing in this repo exercises recovery of
the complete audit path and records a measured recovery time.
What exists is an *observation*, not an exercise: after the railiance01 node
reboot on 2026-08-16, `/readyz` failed for roughly 40 seconds before the pod
went Ready, recorded in `docs/operator-runbook.md`. That is useful and it is not
V1 evidence — it enumerated nothing, measured nothing deliberately, and happened
to us rather than being performed.
Decision 4.6.1 is explicit that a replica count or a status page is not
evidence of a level. audit-core runs one replica; that fact argues for neither
V0 nor V1. Only the exercise settles it.
The 2026-08-16 node reboot was an observation, not an exercise. The 2026-09-24
run was an exercise: the receiver was recreated, the platform-pg primary was
recreated, and the time to the next accepted event was recorded. A replica
count is still not the evidence. The evidence is the exercise.
## Critical dependency enumeration (Decision 4.6.1)
@ -83,10 +73,16 @@ it; E2 and E3 are where it would break.
## Results
*Not yet run.* Needs a live window on railiance01 and coordination with
`user-engine`, since E4 and E5 are visible to the sender.
Run 2026-09-24. Evidence: `docs/evidence/AUDIT-WP-0008-T07-v1-2026-09-24.md`.
On completion, record the measured recovery time per scenario here with dates,
raise `tenancy.yaml` `current.V` to 1, note the exercise as the evidence, and
set a review date — a recovery exercise from a year ago describes a system that
no longer exists.
| Scenario | Measured recovery | Pass |
|---|---|---|
| E1 receiver rollout | about 11s to the next accept | chain intact |
| E2 receiver pod delete | about 7s to the next accept | chain intact |
| E3 `platform-pg` primary recreate | 22s from pod termination to the next accept | chain intact; co-residents still running |
| E4 lease revocation | not run | not claimed |
| E5 node reboot | not run | not claimed |
The binding number is E3. E4 and E5 stay out of this V1 claim: OpenBao being
down does not stop accepts until the current lease expires, and a node reboot
was not part of the approved window. Review by 2026-12-31.

61
docs/evidence/AUDIT-WP-0008-T07-v1-2026-09-24.md Normal file
View file

@ -0,0 +1,61 @@
# AUDIT-WP-0008-T07 availability exercise — 2026-09-24
Operation measured: `POST /v1/events` accepted into custody.
Authority: founder request in the custodian session, recorded as
`ADMINISTER @ realm:kubernetes/railiance01`, `activation=APPROVED`.
Not performed: runtime-lease revocation, OpenBao shutdown, node reboot.
The probe ran inside the receiver, used the already-mounted sender registry,
and printed only a timestamp and an HTTP class. No credential was printed.
Each successful probe is a synthetic `audit.recovery.synthetic` event from
`user-engine` for `tenant:platform`.
## Receiver
| Step | Wall clock (UTC) | Result |
|---|---|---|
| E1 last accept before rollout | 13:45:37 | 202 accepted |
| E1 `rollout restart` issued | 13:45:35 workstation | new pod scheduled |
| E1 probes | 13:45:36, 13:45:39 | exec failed, pod not usable |
| E1 next accept | 13:45:46 | 202 accepted (`audit-wp-0008-t07-e1-134541`) |
| E2 pod deleted | 13:46:12 | `audit-core-66767ddd58-mcxtm` |
| E2 probe | 13:46:13 | exec failed |
| E2 next accept | 13:46:19 | 202 accepted (`audit-wp-0008-t07-e2-134615`) |
Receiver recreate recovered in about 7 seconds (E2). The rollout (E1) recovered
in about 11 seconds. Both new processes accepted an event, so the mounted
database credential and the sender registry were readable at process start.
## Custody
`platform-pg` is one primary. `tenant-engine` and `core-hub` share it.
Both were told immediately before the delete.
| Step | Wall clock (UTC) | Result |
|---|---|---|
| Last accept before delete | 13:46:56 | 202 accepted |
| `kubectl delete pod platform-pg-1` issued | 13:46:54 workstation | command waited for termination |
| Delete returned | 13:49:59 | pod object gone |
| First probe after that | 13:50:13 | unreachable |
| Next accept | 13:50:21 | 202 accepted (`audit-wp-0008-t07-e3-135015`) |
The three minutes inside `kubectl delete` were not sampled, so they are not
claimed as an outage. After the primary pod was gone, the next successful
accept was 22 seconds later. The recreated `platform-pg-1` was Ready, restart
count 0. The CloudNativePG cluster reported healthy. `tenant-engine` and
`core-hub` pods were still Running.
## Integrity
`python3 -m audit_core verify-chain --schema audit_core` after E3:
`intact: true`, `events: 125`, `first_break: null`. The count is the pre-exercise
chain plus the accepted probes only. No accepted probe is missing.
## What this evidences
V1 for the accept path: dependencies were already enumerated in
`docs/availability-recovery.md`, receiver recreate and primary recreate were
exercised, and the interruption was measured. The binding recovery is the
custody restart, 22 seconds from pod termination to the next accept. V2 remains
unreachable while `platform-pg` has one instance. Review this exercise by
2026-12-31.

25
tenancy.yaml
View file

@ -21,7 +21,7 @@ tenancy:
# tenant-engine on platform-pg; the mixture is
# reported by the platform, not hidden.
current: { I: 1, A: 2, E: 2, P: 1, R: 2, V: 0 }
current: { I: 1, A: 2, E: 2, P: 1, R: 2, V: 1 }
target: { I: 1, A: 2, E: 3, P: 1, R: 2, V: 1 }
# §5.2 — declare per path, quote the minimum. The quoted E above is the
@ -104,13 +104,13 @@ tenancy:
under the §4.5 retention trigger, with P2 as the fallback — a worse answer
than archive, named now so it is not improvised later.
V: >-
No exercise establishes restart recovery for the complete audit path.
V1 is the target; replica count or Kubernetes restart policy is not
treated as evidence. Dependencies enumerated and the exercise specified in
docs/availability-recovery.md (AUDIT-WP-0008-T07); the exercise itself
awaits a live window on railiance01. V2 is not reachable at P1 as built:
platform-pg runs instances 1, and Decision 4.6.1 makes V the minimum across
synchronous providers, so V1 is the ceiling here rather than the next step.
V1 evidenced 2026-09-24 (AUDIT-WP-0008-T07). Receiver recreate recovered
in about 7 seconds. Recreating the single platform-pg primary recovered
the accept path 22 seconds after the pod terminated. The chain stayed
intact. Lease revocation and node reboot were not part of the exercise.
V2 is not reachable at P1 as built: platform-pg runs instances 1, and
Decision 4.6.1 makes V the minimum across synchronous providers, so V1
is the ceiling. Review 2026-12-31.
provider:
capability: operations.audit
@ -150,12 +150,17 @@ provider:
- "rapp-postgres/consumers/audit-core.yaml"
- "audit_core/interface.py"
V:
available: 0
available: 1
maximum: 1
conditions:
- "Exercise restart recovery across audit-core, platform-pg and OpenBao."
- "V1 is receiver recreate plus one platform-pg primary recreate, measured 2026-09-24. OpenBao lease revocation was not exercised and is not required for an accept until the current lease expires."
evidence:
- "docs/evidence/AUDIT-WP-0008-T07-v1-2026-09-24.md"
- "docs/availability-recovery.md"
evidence:
V1:
- "docs/evidence/AUDIT-WP-0008-T07-v1-2026-09-24.md"
A2:
- "audit_core/ingestion.py"
- "tests/test_ingestion.py"

12
workplans/AUDIT-WP-0008-tenancy-posture-alignment.md
View file

@ -4,12 +4,12 @@ type: workplan
title: "Tenancy posture declaration and read-path enforcement"
domain: infotech
repo: audit-core
status: active
status: finished
flavor: implementation
owner: claude
topic_slug: railiance
created: "2026-08-17"
updated: "2026-08-22"
updated: "2026-09-24"
depends_on:
- AUDIT-WP-0007
state_hub_workstream_id: "a9a248b2-d26c-503e-a8fc-4f3675e6ed51"
@ -549,7 +549,7 @@ duty, so the correction is ours to send unprompted or it will simply sit stale.
```task
id: AUDIT-WP-0008-T07
status: progress
status: done
priority: high
state_hub_task_id: "4e3ab790-daa6-5fbb-837e-9a9a26848201"
```
@ -640,6 +640,12 @@ trip through audit-core with a dedicated scoped identity: the fixture returned
driver suite is now 12 passing tests; a native full run passed 119 with 24
environment-dependent skips.
Completed 2026-09-24. Receiver recreate recovered in about 7 seconds. The
platform-pg primary recreate recovered the accept path 22 seconds after the
pod terminated. The chain stayed intact. Lease revocation and node reboot were
not run. Evidence: `docs/evidence/AUDIT-WP-0008-T07-v1-2026-09-24.md`.
`current.V` is 1. V2 stays unreachable at one database instance.
```task
id: AUDIT-WP-0008-T08
status: done