audit-core/workplans/AUDIT-WP-0007-integrity-verification.md
tegwick 61214bc24a
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Close AUDIT-WP-0007 after live chain attestation.
Migration 0006 applied on railiance01. Verify-against the dated
chain-head evidence is intact for 30 events.
2026-08-16 01:23:54 +02:00

209 lines
7.3 KiB
Markdown

---
id: AUDIT-WP-0007
type: workplan
title: "Integrity verification for operational custody"
domain: infotech
repo: audit-core
status: finished
owner: grok
topic_slug: railiance
created: "2026-08-16"
updated: "2026-08-16"
depends_on:
- AUDIT-WP-0006
state_hub_workstream_id: "97946512-eb0a-4282-8c1f-f53d9ee25bfa"
---
# AUDIT-WP-0007 — Integrity verification for operational custody
## Goal
Make the `operations.audit` integrity claim checkable.
AUDIT-WP-0006 made custody honest: the live receiver reports
`operational`, not `archive`, and cites the 30-day platform backup.
`tamper_evidence` is still `False`. The append-only trigger rejects
UPDATE/DELETE for the runtime role, and each row stores `payload_hash`,
but a database owner can drop the trigger and rewrite rows. Nothing
would *prove* they had. ITC-CAP hook `integrity_verification` is
therefore `unknown`.
This workplan adds a hash chain on accept, a verify surface that can
fail, and a chain-head attestation stored outside `platform-pg`. Only
then may `RetentionPolicy.tamper_evidence` become true, and only for
what those artefacts actually detect.
It does not raise the provision to D5. D5 also needs reliability
measured and controlled (one replica, no drill cadence). Leave maturity
at D4 unless those land elsewhere.
## Why this is the next improvement
`data.archive` still needs a founder decision and a resource-control
procurement. A second sender expands the product. Integrity is the
remaining honesty gap on the live provision and is owned here.
## Boundaries
**This workplan owns**
- Per-event hash chain in the Postgres store.
- A verify CLI and a `may_read` HTTP surface.
- A chain-head attestation outside `audit_core` on `platform-pg`.
- Updating `tamper_evidence` and the ITC-CAP `integrity_verification`
hook only when verify can detect a rewritten payload.
**This workplan does not own**
- A `data.archive` / WORM object-store sink.
- Booked cost, `rapp.yaml`, public ingest, Barman/S3.
- A public transparency log.
- Raising provision maturity to D5.
- Replacing the append-only trigger (it stays).
## T01 — Specify the chain and the proof bound
```task
id: AUDIT-WP-0007-T01
status: done
priority: high
state_hub_task_id: "d4423fd5-ad78-47d0-b85e-7ae6c582ec2b"
```
Write a short contract in `docs/audit-backend-contract.md` (or a sibling
`docs/integrity.md`) that states:
1. Each accepted event carries `payload_hash` (already stored) and a
new `chain_hash` = H(previous `chain_hash` || `payload_hash` ||
`event_id`). First event uses a documented genesis value.
2. The chain is per schema / database, not per tenant. Tenant splits
would hide a cross-tenant rewrite.
3. `verify` walks accepted_at/chain order, recomputes, and reports the
first break. A break is a custody defect, not a sender error.
4. Proof bound: a chain inside the same database detects a rewritten
payload *if the attacker does not also recompute the suffix*. A
database owner can. Tamper evidence against that class of attacker
requires a chain-head attestation stored outside `platform-pg`.
5. `tamper_evidence=True` is allowed only when both verify and an
external head exist. It still does not mean WORM or `data.archive`.
Done when the contract is written and the tests to be added are named.
Done 2026-08-16: `docs/integrity.md` names the chain, genesis, proof bound,
and the six tests.
## T02 — Persist the chain on accept
```task
id: AUDIT-WP-0007-T02
status: done
priority: high
state_hub_task_id: "5830bb1a-27b7-4eed-b09f-75ba8cc9f7f1"
```
Add an additive migration (0006) for `chain_hash` (and `chain_prev` if
the walk needs it). Backfill existing rows in accept-order so a live
database with the AUDIT-WP-0005/0006 events verifies after migrate.
`accept` must write the next link in the same transaction as the
idempotent insert. A duplicate accept must not mint a second link.
A conflict must not advance the head.
SQLite (dev) should implement the same chain so conformance stays one
suite. Mock file backend stays `tamper_evidence=False`.
Done when conformance tests show: first accept sets genesis; second
links; duplicate does not fork; verify is clean on a fresh store.
Done 2026-08-16: migration 0006 plus backfill; SQLite and Postgres accept
write the next link under a lock. Conformance + `tests/test_integrity.py`.
## T03 — Verify surface
```task
id: AUDIT-WP-0007-T03
status: done
priority: high
state_hub_task_id: "fee1f93c-dcac-4fc3-9e41-17bb4436e8d6"
```
- CLI: `python -m audit_core verify-chain` (or equivalent) exits 0
only if the walk is intact. Print first-break event id on failure.
- HTTP: a `may_read` route (suggested `GET /v1/integrity`) returns
`{intact, events, head, first_break}` without event payloads.
A test must break a payload_hash (or chain_hash) using a superuser
connection and show verify fails. That is the evidence the trigger
alone never gave us.
Done when CLI and HTTP agree, and the broken-row test is red-then-green
as a detector, not as a repair.
Done 2026-08-16: `verify-chain` CLI, `GET /v1/integrity`, SQLite rewrite
test and Postgres superuser rewrite test both fail verify.
## T04 — Attest the chain head outside platform-pg
```task
id: AUDIT-WP-0007-T04
status: done
priority: medium
state_hub_task_id: "b6d03d6a-0605-4982-a9a3-e18c89319ba9"
```
Publish `{chain_hash, event_id, accepted_at, event_count, observed_at}`
to a location that is not a table in `audit_core` on `platform-pg`.
First implementation: an operator command that writes a dated evidence
file, plus a documented second copy (for example the existing
logical-offsite / Nextcloud path used by RESOURCE-WP-0002-T06 — cite
it, do not invent a new bucket).
Do not write the head into the Barman prefix. That copy dies with the
same restore as the table.
Verify must be able to compare the live head to a cited attestation
and report mismatch as a break.
Done when one production walk produces an attestation file in
`docs/evidence/` and verify-against-attestation is tested.
Done 2026-08-16: live walk 30 events, intact, head
`03fd1773…`. Evidence
`docs/evidence/chain-head-20260816.json`. Verify-against matched.
Do not copy this file into the Barman prefix; a second copy may follow
RESOURCE-WP-0002-T06 logical-offsite.
## T05 — Declare tamper_evidence only as far as the proof
```task
id: AUDIT-WP-0007-T05
status: done
priority: medium
state_hub_task_id: "9fd70d3c-5a64-4b26-967f-4ef45394d81c"
```
After T03 and T04:
- Postgres `RetentionPolicy.tamper_evidence` becomes `True`.
- `/readyz` may report it. Do not change `custody_class` back to
`archive`.
- Update `data/capability/audit-core-operational.json`:
`integrity_verification` basis `measured`, pointing at the broken-row
test and the attestation walk. Leave maturity at D4. Keep
`data.archive` unmet.
- Refresh the reuse-surface card and operator runbook (how to verify,
what a break means, that reboot unreadiness is unrelated).
Done when capability-review still `ok` and the hook is no longer
`unknown`.
Done 2026-08-16: Postgres `tamper_evidence=True`; live `/readyz` reports
it; capability-review `ok`; `integrity_verification` is `measured`.
Maturity stays D4. `data.archive` stays unmet.
## Acceptance
- A rewritten stored payload makes `verify` fail.
- A chain-head attestation exists outside `platform-pg`.
- `tamper_evidence` is true only after those two exist.
- No `data.archive` sink, no D5 inflation, no new cost stream.