From 5d46723d4eada0f4bf240d74aee85c50c93a118f Mon Sep 17 00:00:00 2001 From: tegwick Date: Sun, 16 Aug 2026 00:57:59 +0200 Subject: [PATCH] Cut over AUDIT-WP-0006 image and add AUDIT-WP-0007. Live receiver now reports custody_class=operational with a cited 30-day recoverable window (sha256:05fe1c06). Next workplan is integrity verification so tamper_evidence can become honest. --- SCOPE.md | 8 +- deploy/README.md | 2 +- deploy/audit-core.yaml | 2 +- deploy/migrate-job.yaml | 2 +- docs/operator-runbook.md | 5 + .../AUDIT-WP-0007-integrity-verification.md | 184 ++++++++++++++++++ 6 files changed, 196 insertions(+), 7 deletions(-) create mode 100644 workplans/AUDIT-WP-0007-integrity-verification.md diff --git a/SCOPE.md b/SCOPE.md index d84371d..60ef68d 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -32,11 +32,11 @@ audit-core exists to provide the capability described in INTENT.md. - Production receiver on railiance01 (`namespace audit-core`), Postgres operational custody on `platform-pg`, sender `user-engine`. - Recovery is the platform `data.backup` window (30 days, RESOURCE-WP-0002 - live). Manifest `/readyz` reports `custody_class=operational` and - `recoverable_days=30` once the AUDIT-WP-0006 image is pinned. + live). Live `/readyz` reports `custody_class=operational` and + `recoverable_days=30` (image `sha256:05fe1c06…`, 2026-08-16 cutover). - ITC-CAP case: `data/capability/audit-core-operational.json`. - `data.archive` is an unmet requirement. -- AUDIT-WP-0001…0006 closed. No open workplan. + `data.archive` is an unmet requirement. `tamper_evidence` is still false. +- Open workplan: `workplans/AUDIT-WP-0007-integrity-verification.md`. ## Getting Oriented diff --git a/deploy/README.md b/deploy/README.md index a23b79d..4922b02 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -8,7 +8,7 @@ the server port or export a copy before applying. Apply order is documented in `docs/operator-runbook.md`. Do not apply the Deployment until: -1. The image digest is pinned (currently `sha256:41493cd5…` from commit `3a7d63e`). +1. The image digest is pinned (currently `sha256:05fe1c06…` from commit `40dcadd`). 2. Secrets `audit-core-database`, `audit-core-database-migrate`, and `audit-core-senders` exist. ConfigMap `audit-core-senders-scope` is applied (`deploy/senders-scope.yaml`) before the Deployment mounts it. diff --git a/deploy/audit-core.yaml b/deploy/audit-core.yaml index 6144c88..d62d65e 100644 --- a/deploy/audit-core.yaml +++ b/deploy/audit-core.yaml @@ -80,7 +80,7 @@ spec: - name: audit-core # REPLACE at release time with the built digest. A mutable tag is not # an immutable image, and `:latest` must never be the only reference. - image: forgejo.coulomb.social/coulomb/audit-core@sha256:aec5575ceadf1dc53d0892c0786f1fcd5cb38077ce36ba271ee7204aacd80df5 + image: forgejo.coulomb.social/coulomb/audit-core@sha256:05fe1c06f809be29309695c397025da755bfe82afed81a7f9b8771e0b9200c17 imagePullPolicy: IfNotPresent ports: - name: http diff --git a/deploy/migrate-job.yaml b/deploy/migrate-job.yaml index 53ed674..dd11e60 100644 --- a/deploy/migrate-job.yaml +++ b/deploy/migrate-job.yaml @@ -33,7 +33,7 @@ spec: type: RuntimeDefault containers: - name: migrate - image: forgejo.coulomb.social/coulomb/audit-core@sha256:aec5575ceadf1dc53d0892c0786f1fcd5cb38077ce36ba271ee7204aacd80df5 + image: forgejo.coulomb.social/coulomb/audit-core@sha256:05fe1c06f809be29309695c397025da755bfe82afed81a7f9b8771e0b9200c17 imagePullPolicy: IfNotPresent command: ["python", "-m", "audit_core", "migrate"] env: diff --git a/docs/operator-runbook.md b/docs/operator-runbook.md index d2c7393..32be270 100644 --- a/docs/operator-runbook.md +++ b/docs/operator-runbook.md @@ -37,6 +37,11 @@ failure (`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=operational`), not a silent downgrade to SQLite. An older manifest that still requires `archive` is accepted as an alias for one mixed rollout. +After a node reboot, `/readyz` failing for tens of seconds is expected: +CoreDNS and `platform-pg` come up after the receiver. Liveness stays on +`/healthz` so that window does not restart the pod. Walked 2026-08-16 +(~40s unreadiness, then Ready). + ## Lookup All read routes require a sender identity with `may_read: true`. A write diff --git a/workplans/AUDIT-WP-0007-integrity-verification.md b/workplans/AUDIT-WP-0007-integrity-verification.md new file mode 100644 index 0000000..5c2e39b --- /dev/null +++ b/workplans/AUDIT-WP-0007-integrity-verification.md @@ -0,0 +1,184 @@ +--- +id: AUDIT-WP-0007 +type: workplan +title: "Integrity verification for operational custody" +domain: infotech +repo: audit-core +status: ready +owner: grok +topic_slug: railiance +created: "2026-08-16" +updated: "2026-08-16" +depends_on: + - AUDIT-WP-0006 +--- + +# AUDIT-WP-0007 — Integrity verification for operational custody + +## Goal + +Make the `operations.audit` integrity claim checkable. + +AUDIT-WP-0006 made custody honest: the live receiver reports +`operational`, not `archive`, and cites the 30-day platform backup. +`tamper_evidence` is still `False`. The append-only trigger rejects +UPDATE/DELETE for the runtime role, and each row stores `payload_hash`, +but a database owner can drop the trigger and rewrite rows. Nothing +would *prove* they had. ITC-CAP hook `integrity_verification` is +therefore `unknown`. + +This workplan adds a hash chain on accept, a verify surface that can +fail, and a chain-head attestation stored outside `platform-pg`. Only +then may `RetentionPolicy.tamper_evidence` become true, and only for +what those artefacts actually detect. + +It does not raise the provision to D5. D5 also needs reliability +measured and controlled (one replica, no drill cadence). Leave maturity +at D4 unless those land elsewhere. + +## Why this is the next improvement + +`data.archive` still needs a founder decision and a resource-control +procurement. A second sender expands the product. Integrity is the +remaining honesty gap on the live provision and is owned here. + +## Boundaries + +**This workplan owns** + +- Per-event hash chain in the Postgres store. +- A verify CLI and a `may_read` HTTP surface. +- A chain-head attestation outside `audit_core` on `platform-pg`. +- Updating `tamper_evidence` and the ITC-CAP `integrity_verification` + hook only when verify can detect a rewritten payload. + +**This workplan does not own** + +- A `data.archive` / WORM object-store sink. +- Booked cost, `rapp.yaml`, public ingest, Barman/S3. +- A public transparency log. +- Raising provision maturity to D5. +- Replacing the append-only trigger (it stays). + +## T01 — Specify the chain and the proof bound + +```task +id: AUDIT-WP-0007-T01 +status: todo +priority: high +``` + +Write a short contract in `docs/audit-backend-contract.md` (or a sibling +`docs/integrity.md`) that states: + +1. Each accepted event carries `payload_hash` (already stored) and a + new `chain_hash` = H(previous `chain_hash` || `payload_hash` || + `event_id`). First event uses a documented genesis value. +2. The chain is per schema / database, not per tenant. Tenant splits + would hide a cross-tenant rewrite. +3. `verify` walks accepted_at/chain order, recomputes, and reports the + first break. A break is a custody defect, not a sender error. +4. Proof bound: a chain inside the same database detects a rewritten + payload *if the attacker does not also recompute the suffix*. A + database owner can. Tamper evidence against that class of attacker + requires a chain-head attestation stored outside `platform-pg`. +5. `tamper_evidence=True` is allowed only when both verify and an + external head exist. It still does not mean WORM or `data.archive`. + +Done when the contract is written and the tests to be added are named. + +## T02 — Persist the chain on accept + +```task +id: AUDIT-WP-0007-T02 +status: todo +priority: high +``` + +Add an additive migration (0006) for `chain_hash` (and `chain_prev` if +the walk needs it). Backfill existing rows in accept-order so a live +database with the AUDIT-WP-0005/0006 events verifies after migrate. + +`accept` must write the next link in the same transaction as the +idempotent insert. A duplicate accept must not mint a second link. +A conflict must not advance the head. + +SQLite (dev) should implement the same chain so conformance stays one +suite. Mock file backend stays `tamper_evidence=False`. + +Done when conformance tests show: first accept sets genesis; second +links; duplicate does not fork; verify is clean on a fresh store. + +## T03 — Verify surface + +```task +id: AUDIT-WP-0007-T03 +status: todo +priority: high +``` + +- CLI: `python -m audit_core verify-chain` (or equivalent) exits 0 + only if the walk is intact. Print first-break event id on failure. +- HTTP: a `may_read` route (suggested `GET /v1/integrity`) returns + `{intact, events, head, first_break}` without event payloads. + +A test must break a payload_hash (or chain_hash) using a superuser +connection and show verify fails. That is the evidence the trigger +alone never gave us. + +Done when CLI and HTTP agree, and the broken-row test is red-then-green +as a detector, not as a repair. + +## T04 — Attest the chain head outside platform-pg + +```task +id: AUDIT-WP-0007-T04 +status: todo +priority: medium +``` + +Publish `{chain_hash, event_id, accepted_at, event_count, observed_at}` +to a location that is not a table in `audit_core` on `platform-pg`. +First implementation: an operator command that writes a dated evidence +file, plus a documented second copy (for example the existing +logical-offsite / Nextcloud path used by RESOURCE-WP-0002-T06 — cite +it, do not invent a new bucket). + +Do not write the head into the Barman prefix. That copy dies with the +same restore as the table. + +Verify must be able to compare the live head to a cited attestation +and report mismatch as a break. + +Done when one production walk produces an attestation file in +`docs/evidence/` and verify-against-attestation is tested. + +## T05 — Declare tamper_evidence only as far as the proof + +```task +id: AUDIT-WP-0007-T05 +status: todo +priority: medium +``` + +After T03 and T04: + +- Postgres `RetentionPolicy.tamper_evidence` becomes `True`. +- `/readyz` may report it. Do not change `custody_class` back to + `archive`. +- Update `data/capability/audit-core-operational.json`: + `integrity_verification` basis `measured`, pointing at the broken-row + test and the attestation walk. Leave maturity at D4. Keep + `data.archive` unmet. +- Refresh the reuse-surface card and operator runbook (how to verify, + what a break means, that reboot unreadiness is unrelated). + +Done when capability-review still `ok` and the hook is no longer +`unknown`. + +## Acceptance + +- A rewritten stored payload makes `verify` fail. +- A chain-head attestation exists outside `platform-pg`. +- `tamper_evidence` is true only after those two exist. +- No `data.archive` sink, no D5 inflation, no new cost stream.