Implement AUDIT-WP-0008 T03, T06, T07, T08.
T08 fixes two defects in our own declaration. provider.R.available quoted a 30-day horizon audit-core does not solely control: at P1 the erasure horizon is the instance maximum across co-residents, so a co-resident declaring longer extends what a sender's records remain recoverable for, silently. Decision 4.5.4 names this for tiers; it applies to a provider quoting a number too, and the provider block now says so. And user-engine, the only consumer, was notified under Decision 6.1 -- what we declared, that E4 and R4 are unreachable here, and that the retention number is a floor rather than a ceiling. T03 writes docs/erasure-and-audit.md: the fact/payload split, why shreddability is not retrofittable onto a chain committing to cleartext, and why the retained hash is a confirmation oracle over low-entropy audit records. The framework half was already resolved as Decision 4.5.3, so what remains is our own position. The legal basis for retaining audit facts is routed to risk-nexus, open and visible. T06 closes the review loop with net-kingdom: five findings adopted, declaration validates clean, and the E line will go stale on an upgrade that Decision 6.1 deliberately does not require anyone to announce. T07 enumerates the seven dependencies on the accept path and specifies five recovery scenarios with integrity as a pass condition. It settled one thing: V2 is not reachable from P1 as built, since platform-pg runs instances 1 and Decision 4.6.1 makes V the minimum across synchronous providers. V1 is the ceiling here, not the next step. The exercise needs a live window. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
92c56fe325
commit
58a106aa4d
5 changed files with 242 additions and 9 deletions
|
|
@ -50,6 +50,7 @@
|
|||
| task | AUDIT-WP-0008-T01 | done | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
| task | AUDIT-WP-0008-T02 | done | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
| task | AUDIT-WP-0008-T03 | todo | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
| task | AUDIT-WP-0008-T04 | done | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
| task | AUDIT-WP-0008-T05 | todo | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
| task | AUDIT-WP-0008-T06 | todo | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
| task | AUDIT-WP-0008-T07 | todo | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
| task | AUDIT-WP-0008-T08 | todo | — | workplans/AUDIT-WP-0008-tenancy-posture-alignment.md |
|
||||
|
|
|
|||
92
docs/availability-recovery.md
Normal file
92
docs/availability-recovery.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Availability and recovery
|
||||
|
||||
**Status:** V1 not yet evidenced. Dependency enumeration complete; exercise
|
||||
pending a live window on railiance01.
|
||||
**Framework:** NetKingdom Tenancy Posture v0.1 (draft-8) §4.6, §13 V-row.
|
||||
**Declared:** `tenancy.yaml` — `current.V: 0`, `target.V: 1`.
|
||||
|
||||
## Why V0 today
|
||||
|
||||
`V0` is "no availability or recovery position; recovery is untested or depends
|
||||
on improvisation". That is accurate. Nothing in this repo exercises recovery of
|
||||
the complete audit path and records a measured recovery time.
|
||||
|
||||
What exists is an *observation*, not an exercise: after the railiance01 node
|
||||
reboot on 2026-08-16, `/readyz` failed for roughly 40 seconds before the pod
|
||||
went Ready, recorded in `docs/operator-runbook.md`. That is useful and it is not
|
||||
V1 evidence — it enumerated nothing, measured nothing deliberately, and happened
|
||||
to us rather than being performed.
|
||||
|
||||
Decision 4.6.1 is explicit that a replica count or a status page is not
|
||||
evidence of a level. audit-core runs one replica; that fact argues for neither
|
||||
V0 nor V1. Only the exercise settles it.
|
||||
|
||||
## Critical dependency enumeration (Decision 4.6.1)
|
||||
|
||||
V is **end-to-end**: the minimum across the components and synchronous providers
|
||||
required to serve the operation. An application that restarts in five seconds
|
||||
over a database that takes ninety is not V1 at five seconds.
|
||||
|
||||
The operation being declared is **accept an audit event** (`POST /v1/events`).
|
||||
Read surfaces are operator-facing and are not the availability claim.
|
||||
|
||||
| # | Dependency | Role in the accept path | Failure shows as |
|
||||
|---|---|---|---|
|
||||
| 1 | `audit-core` Deployment (1 replica, namespace `audit-core`) | Serves the receiver | Connection refused; Service has no endpoint |
|
||||
| 2 | `platform-pg` (CNPG, `instances: 1`) | Custody. Synchronous — an accept is not accepted until committed | `/readyz` 503, request 503 `unavailable` |
|
||||
| 3 | Secret `audit-core-database` at `/etc/audit-core/db` | Runtime lease, re-read per connection | 503 `unavailable` on lease expiry or revocation |
|
||||
| 4 | ESO + OpenBao (`database/creds/audit-core-runtime`) | Refreshes 3 every 15 min | Delayed: works until the current lease expires |
|
||||
| 5 | Secret `audit-core-senders` + scope ConfigMap | Sender authentication; read at process start | 401 for every sender after a restart with a bad Secret |
|
||||
| 6 | CoreDNS | Resolves `platform-pg` | 503 `unavailable`, indistinguishable from 2 without logs |
|
||||
| 7 | NetworkPolicy (default-deny + sender ingress) | Admits `user-engine` | Sender timeouts; receiver sees nothing at all |
|
||||
|
||||
**The binding constraint is 2.** `platform-pg` runs `instances: 1` — no HA, single
|
||||
node. §17 of the framework already records that no P1 tenant has HA. audit-core
|
||||
cannot exceed `platform-pg`'s availability level regardless of what it does to
|
||||
its own Deployment, so **V2 is not reachable from P1 as built** and is not a
|
||||
target. V1 is.
|
||||
|
||||
Dependency 4 is worth separating: OpenBao being down does not stop accepts. It
|
||||
stops *renewal*, so the failure is deferred to lease expiry. An exercise that
|
||||
takes OpenBao down and declares success after two minutes has proved nothing —
|
||||
this is the invisible-failure shape §12 warns about.
|
||||
|
||||
## The V1 exercise
|
||||
|
||||
§13's V1 row asks for three things, all mechanical: dependencies enumerated
|
||||
(above), restart/recreate recovery exercised, interruption and measured recovery
|
||||
time recorded.
|
||||
|
||||
**Rules for the run.** Measure from the last successful accept to the next
|
||||
successful accept, not from pod restart to Ready — Ready is our own opinion of
|
||||
ourselves. Drive a real `POST /v1/events` on a fixed interval throughout and
|
||||
count what the sender sees, because the sender's experience is the availability
|
||||
being claimed. Record wall-clock times, not durations someone computed later.
|
||||
|
||||
| # | Scenario | Action | Record |
|
||||
|---|---|---|---|
|
||||
| E1 | Receiver restart | `kubectl -n audit-core rollout restart deploy/audit-core` | Accepts lost; time to first successful accept |
|
||||
| E2 | Receiver recreate | Delete the pod | As E1, plus whether any accepted event is missing after recovery |
|
||||
| E3 | Custody restart | Restart the `platform-pg` primary | Time to first successful accept; 503 count; chain intact after |
|
||||
| E4 | Lease expiry under load | Revoke the runtime lease, wait for ESO refresh | Time to recovery **without** a pod restart — the property `credentials.py` exists to provide |
|
||||
| E5 | Node reboot | Reboot railiance01 | End-to-end recovery; compare against the 2026-08-16 observation of ~40s |
|
||||
|
||||
**Integrity is part of the pass condition, not a separate check.** After every
|
||||
scenario, `python -m audit_core verify-chain` must report intact, and every
|
||||
event the driver believes was accepted must be present. A recovery that loses an
|
||||
accepted event, or forks the chain, is a custody defect and fails the exercise
|
||||
regardless of how fast it was.
|
||||
|
||||
**RPO.** An accept is committed before the sender is told, so the expected RPO
|
||||
for accepted events is zero. The exercise tests that claim rather than assuming
|
||||
it; E2 and E3 are where it would break.
|
||||
|
||||
## Results
|
||||
|
||||
*Not yet run.* Needs a live window on railiance01 and coordination with
|
||||
`user-engine`, since E4 and E5 are visible to the sender.
|
||||
|
||||
On completion, record the measured recovery time per scenario here with dates,
|
||||
raise `tenancy.yaml` `current.V` to 1, note the exercise as the evidence, and
|
||||
set a review date — a recovery exercise from a year ago describes a system that
|
||||
no longer exists.
|
||||
112
docs/erasure-and-audit.md
Normal file
112
docs/erasure-and-audit.md
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
# Erasure versus audit
|
||||
|
||||
**Status:** audit-core's position. Written for AUDIT-WP-0008-T03.
|
||||
**Framework:** NetKingdom Tenancy Posture v0.1 (draft-8), §4.5 and §19.11.
|
||||
**Declared consequence:** `tenancy.yaml` targets `R2`. `R4` is not a target.
|
||||
|
||||
## The question
|
||||
|
||||
Raised by `rapp-postgres` as question 11 of the tenancy posture review. The
|
||||
retention axis tops out at `R4`, verified erasure, reachable either by waiting
|
||||
out the erasure horizon or by encrypting per entity and destroying the key.
|
||||
Crypto-shredding is the established mechanism for erasure under immutable
|
||||
retention.
|
||||
|
||||
Applied to an audit service it destroys the evidence the service exists to hold.
|
||||
|
||||
The question was asked because a naive "R4 everywhere" fleet target would
|
||||
instruct this service to erase its own trail, and nobody would notice until it
|
||||
mattered.
|
||||
|
||||
## The shape of the answer, and why it is not enough
|
||||
|
||||
The literature's usual resolution is to separate the **fact** of an event,
|
||||
retained, from its **personal payload**, encrypted per subject and shreddable.
|
||||
The trail survives erasure with its personal content removed. That shape is
|
||||
right, and we accept it as the shape.
|
||||
|
||||
It does not survive contact with this repo's integrity design, in two ways.
|
||||
|
||||
### 1. It is not retrofittable
|
||||
|
||||
`_record_hash` (`audit_core/postgres_backend.py`) is SHA-256 over the canonical
|
||||
JSON of the **cleartext** record, and the hash chain commits to that value.
|
||||
Encrypting a payload after the fact severs the binding between `payload_hash`
|
||||
and the stored record: the chain still verifies, because `verify_rows` walks
|
||||
stored hashes, but the hash no longer attests to anything present.
|
||||
|
||||
Shreddability has to be built as **encrypt-then-hash at accept time**, with the
|
||||
chain committing to ciphertext. Retrofitting it onto events already accepted
|
||||
means rewriting the chain — which is the single thing AUDIT-WP-0007 was built to
|
||||
make detectable. A design that requires forging the tamper-evidence log in order
|
||||
to comply is not a design.
|
||||
|
||||
### 2. The retained hash is a confirmation oracle
|
||||
|
||||
This is the sharper problem and it is easy to miss.
|
||||
|
||||
Audit records are low-entropy. An event carries a type, an actor, a tenant, a
|
||||
subject and a timestamp, drawn from small enumerable sets. Given the retained
|
||||
SHA-256, an adversary guesses a plausible record, hashes the guess, and compares.
|
||||
A match confirms the content.
|
||||
|
||||
So destroying the key **does not** make the payload unrecoverable while that
|
||||
hash survives. Key destruction removes the ability to read the ciphertext; it
|
||||
does nothing about a separate commitment that anyone can test guesses against.
|
||||
|
||||
The remedies are known — an HMAC under a per-subject key that dies with the key,
|
||||
or a per-record salt destroyed alongside it — and neither is built here.
|
||||
|
||||
**Conclusion: crypto-shredding does not reach `R4` under this design.** Claiming
|
||||
it would be an overclaim of exactly the kind §6 exists to prohibit, and worse
|
||||
than most, because the claim being made would be "this data is gone".
|
||||
|
||||
This finding was general rather than specific to us, and the framework adopted
|
||||
it as **Decision 4.5.3**: the key-destroyed route now requires that no retained
|
||||
commitment reveals the erased content. It applies to anyone hashing, chaining or
|
||||
indexing what they intend to shred.
|
||||
|
||||
## What audit-core therefore declares
|
||||
|
||||
- **Target `R2`**, not `R4`. Retention declared per dataset, erasure horizon
|
||||
published. That is in `tenancy.yaml` and reflected on `/readyz` as
|
||||
`recoverable_days`.
|
||||
- **`R4` is unreachable by design**, and §19.11 records audit-core as explicitly
|
||||
exempt from any fleet `R4` target.
|
||||
- **Our provider block says so to senders.** A consumer with a verified erasure
|
||||
obligation over its audit trail cannot discharge it here, and is told that
|
||||
rather than left to assume.
|
||||
|
||||
## The dependency we are not competent to close
|
||||
|
||||
Declining to build erasure means that if an Article 17 request arrives naming a
|
||||
data subject who appears in the audit trail, audit-core has no mechanism. The
|
||||
answer will rest on audit evidence being exempt — legal obligation, or
|
||||
legitimate interest in fraud and security investigation.
|
||||
|
||||
Those grounds are ordinary and widely relied on. They are also a **legal
|
||||
conclusion**, and this repo is not competent to reach one. Nothing in `INTENT`,
|
||||
`SCOPE`, the capability case, or the framework records anyone in this estate
|
||||
having actually reached it. §19.11 says as much in the framework's own words:
|
||||
"the legal basis for retaining audit facts remains a risk/legal question outside
|
||||
this framework."
|
||||
|
||||
**Routed to `risk-nexus`**, which owns judging what an external rule demands of
|
||||
the estate (moved there from `policy-nexus` on 2026-08-17). Open, unowned inside
|
||||
audit-core, and deliberately visible.
|
||||
|
||||
The distinction being drawn: a *recorded tension* is a position someone has
|
||||
taken and can defend. An *unnoticed liability* is the same technical state with
|
||||
nobody named. An audit service discovering mid-regulator-question that its
|
||||
exemption was assumed rather than established is the specific failure this
|
||||
document exists to prevent.
|
||||
|
||||
## What would change the position
|
||||
|
||||
- A `data.archive` provision with per-subject encryption at accept time, making
|
||||
encrypt-then-hash available for new events. Old events remain unshreddable.
|
||||
- A ruling or advice from `risk-nexus` that the retention basis does not hold,
|
||||
which would make the R4 problem urgent rather than theoretical.
|
||||
- A sender arriving with a genuine per-subject erasure obligation over its audit
|
||||
trail. Today none exists; the provider block is written so one would find out
|
||||
before building on us rather than after.
|
||||
21
tenancy.yaml
21
tenancy.yaml
|
|
@ -106,7 +106,11 @@ tenancy:
|
|||
V: >-
|
||||
No exercise establishes restart recovery for the complete audit path.
|
||||
V1 is the target; replica count or Kubernetes restart policy is not
|
||||
treated as evidence.
|
||||
treated as evidence. Dependencies enumerated and the exercise specified in
|
||||
docs/availability-recovery.md (AUDIT-WP-0008-T07); the exercise itself
|
||||
awaits a live window on railiance01. V2 is not reachable at P1 as built:
|
||||
platform-pg runs instances 1, and Decision 4.6.1 makes V the minimum across
|
||||
synchronous providers, so V1 is the ceiling here rather than the next step.
|
||||
|
||||
provider:
|
||||
capability: operations.audit
|
||||
|
|
@ -126,6 +130,21 @@ provider:
|
|||
maximum: 2
|
||||
conditions:
|
||||
- "R4 is unreachable while the integrity chain commits to cleartext hashes."
|
||||
- >-
|
||||
The 30-day horizon is declared but NOT solely ours to hold. audit-core
|
||||
is at P1, and on shared substrate the erasure horizon is the instance
|
||||
maximum across co-residents (§4.5), not the value this consumer
|
||||
declares. A co-resident on platform-pg declaring a longer window
|
||||
extends what a sender's audit records remain recoverable for, without
|
||||
any change here and without notice to the sender. Decision 4.5.4 names
|
||||
this as why a bare R2 is insufficient for a retention promise; we are
|
||||
not a tier, but we are a provider quoting a number we do not control
|
||||
alone, and a sender relying on 30 days should read it as a floor on
|
||||
our side and a platform-derived maximum in practice.
|
||||
- >-
|
||||
A sender needing a horizon it can actually rely on needs audit-core at
|
||||
P2. That is the §4.5 retention trigger and it is recorded in
|
||||
gap.retention_placement with a 2026-12-31 review.
|
||||
evidence:
|
||||
- "rapp-postgres/consumers/audit-core.yaml"
|
||||
- "audit_core/interface.py"
|
||||
|
|
|
|||
|
|
@ -337,7 +337,7 @@ does not wait on T01. Mark the message read on send.
|
|||
|
||||
```task
|
||||
id: AUDIT-WP-0008-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "0003954e-5033-457f-b9aa-97efdefe9beb"
|
||||
```
|
||||
|
|
@ -373,7 +373,7 @@ and the operator runbook. Raises E to 2 on both paths; update T01's declaration.
|
|||
|
||||
```task
|
||||
id: AUDIT-WP-0008-T05
|
||||
status: todo
|
||||
status: wait
|
||||
priority: medium
|
||||
state_hub_task_id: "30b56b30-e7eb-4873-aa7c-f4a3bf2fcb24"
|
||||
```
|
||||
|
|
@ -385,7 +385,7 @@ per §19.3.
|
|||
|
||||
```task
|
||||
id: AUDIT-WP-0008-T06
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "ed58ca37-755b-4284-b1ac-2a02baaceb21"
|
||||
```
|
||||
|
|
@ -404,8 +404,9 @@ duty, so the correction is ours to send unprompted or it will simply sit stale.
|
|||
|
||||
```task
|
||||
id: AUDIT-WP-0008-T07
|
||||
status: todo
|
||||
status: progress
|
||||
priority: high
|
||||
state_hub_task_id: "e972a8f7-e6ee-4522-bc76-fb69d71e5ed5"
|
||||
```
|
||||
Reach **V1** on the availability axis, new in draft-8.
|
||||
|
||||
|
|
@ -422,13 +423,21 @@ observed ~40s of unreadiness after a node reboot, without enumerating the
|
|||
dependency set or recording a measured recovery time as an exercise. Decision
|
||||
4.6.1 is explicit that a replica count or a status page is not evidence.
|
||||
|
||||
Write the exercise procedure and the dependency enumeration here; the exercise
|
||||
itself runs against railiance01 and needs a live window.
|
||||
Delivered in `docs/availability-recovery.md`: the seven-dependency enumeration,
|
||||
five exercise scenarios with integrity as a pass condition, and the measurement
|
||||
rules. The exercise itself needs a live window on railiance01 and coordination
|
||||
with `user-engine`, since two scenarios are visible to the sender.
|
||||
|
||||
One thing the enumeration settled: **V2 is not reachable from P1 as built.**
|
||||
`platform-pg` runs `instances: 1`, and Decision 4.6.1 makes V the minimum across
|
||||
synchronous providers — so no amount of replication on our Deployment raises it.
|
||||
V1 is the ceiling here, not merely the next step.
|
||||
|
||||
```task
|
||||
id: AUDIT-WP-0008-T08
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "50eed5e2-06d6-4bdc-913d-5a8cd41ce613"
|
||||
```
|
||||
Disclose the shared-substrate retention coupling in our **provider** block, and
|
||||
notify `user-engine` under Decision 6.1.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue