From 5433562d7a9e09b565376c0704b28364eeb35366 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 15 Aug 2026 20:45:13 +0200 Subject: [PATCH] Add AUDIT-WP-0006 for honest custody against live canon. Review of info-tech-canon 0.6.0, resource-control, fin-hub, and railiance-master found production still claims archive custody after the platform backup and ITC-CAP catalog made that claim false. --- SCOPE.md | 8 +- WORK-RECORDS.md | 10 +- ...T-WP-0006-honest-custody-and-canon-join.md | 341 ++++++++++++++++++ 3 files changed, 353 insertions(+), 6 deletions(-) create mode 100644 workplans/AUDIT-WP-0006-honest-custody-and-canon-join.md diff --git a/SCOPE.md b/SCOPE.md index 05915e7..7798e1f 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -23,7 +23,13 @@ audit-core exists to provide the capability described in INTENT.md. ## Current State -- Status: active; implementation and stability should be verified by the repo agent. +- Status: active. AUDIT-WP-0001…0005 finished. +- Production receiver on railiance01 (`namespace audit-core`), Postgres + append-only store on `platform-pg`, sender `user-engine`. +- Recovery is the platform `data.backup` window (30 days, RESOURCE-WP-0002 + live), not an audit-core deletion window. Production still reports + `custody_class=archive`; AUDIT-WP-0006 is the honesty/canon-join pass. +- Open workplan: `workplans/AUDIT-WP-0006-honest-custody-and-canon-join.md`. ## Getting Oriented diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 1d2de2d..8bc6bec 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -12,7 +12,7 @@ | workplan | AUDIT-WP-0002 | finished | — | workplans/AUDIT-WP-0002-pluggable-audit-backend.md | | workplan | AUDIT-WP-0003 | finished | — | workplans/AUDIT-WP-0003-user-engine-event-ingestion-service.md | | workplan | AUDIT-WP-0004 | finished | — | workplans/AUDIT-WP-0004-receiver-correctness-and-hardening.md | -| workplan | AUDIT-WP-0005 | active | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | +| workplan | AUDIT-WP-0005 | finished | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | | task | AUDIT-WP-0001-T01 | done | — | workplans/AUDIT-WP-0001-statehub-bootstrap.md | | task | AUDIT-WP-0001-T02 | done | — | workplans/AUDIT-WP-0001-statehub-bootstrap.md | | task | AUDIT-WP-0001-T03 | done | — | workplans/AUDIT-WP-0001-statehub-bootstrap.md | @@ -29,8 +29,8 @@ | task | AUDIT-WP-0004-T06 | done | — | workplans/AUDIT-WP-0004-receiver-correctness-and-hardening.md | | task | AUDIT-WP-0004-T07 | done | — | workplans/AUDIT-WP-0004-receiver-correctness-and-hardening.md | | task | AUDIT-WP-0005-T01 | done | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | -| task | AUDIT-WP-0005-T02 | progress | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | -| task | AUDIT-WP-0005-T03 | progress | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | +| task | AUDIT-WP-0005-T02 | done | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | +| task | AUDIT-WP-0005-T03 | done | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | | task | AUDIT-WP-0005-T04 | done | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | -| task | AUDIT-WP-0005-T05 | progress | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | -| task | AUDIT-WP-0005-T06 | progress | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | +| task | AUDIT-WP-0005-T05 | done | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | +| task | AUDIT-WP-0005-T06 | done | — | workplans/AUDIT-WP-0005-postgres-store-and-production-deployment.md | diff --git a/workplans/AUDIT-WP-0006-honest-custody-and-canon-join.md b/workplans/AUDIT-WP-0006-honest-custody-and-canon-join.md new file mode 100644 index 0000000..f59e24a --- /dev/null +++ b/workplans/AUDIT-WP-0006-honest-custody-and-canon-join.md @@ -0,0 +1,341 @@ +--- +id: AUDIT-WP-0006 +type: workplan +title: "Honest operational custody against ITC-CAP and the live platform backup" +domain: infotech +repo: audit-core +status: ready +owner: grok +topic_slug: railiance +created: "2026-08-15" +updated: "2026-08-15" +depends_on: + - AUDIT-WP-0005 + - RESOURCE-WP-0002 + - ITC-WP-0014 + - ITC-WP-0015 +--- + +# AUDIT-WP-0006 — Honest operational custody against ITC-CAP and the live platform backup + +## Goal + +Stop claiming a capability the live catalog and the live backup provision +contradict, and join the spine neighbours already use. + +After AUDIT-WP-0005 the receiver is in production on railiance01, append-only +in `platform-pg`, fail-closed on `AUDIT_CORE_REQUIRE_CUSTODY_CLASS=archive`. +Since then three things landed that change what that claim means: + +1. **info-tech-canon 0.6.0 / ITC-CAP 0.4.0** names distinct abilities: + `operations.audit` (attributable historical evidence), `data.archive` + (long-term retention under lifecycle/integrity/retrieval), `data.backup` + (recover a store), `governance.evidence`, `governance.lifecycle`. + Consumption must carry an EvidenceBasis. Maturity attaches to a + *provision*, not an abstract capability. +2. **resource-control RESOURCE-WP-0002 is finished.** Off-host Barman to + Scaleway (`resource:platform:audit-storage`) is live. Restore evidence + used `audit_core.events` as the checksum. The provision is scored + `data.backup` D4 against a D5 requirement. The 30-day window is real. +3. **fin-hub / railiance-master** tightened ownership: booked cost stays + in fin-hub; usage and inventory stay in resource-control; `rapp.yaml` + is only valid in a `rapp-*` repo; default exposure is private. + +audit-core still reports `custody_class=archive`, documents archive as +"planned WORM + manifests", leaves `retention_days` unset (read as +indefinite), and keeps a reuse-surface card at A2 / `source module` / +`helix_forge`. The operator runbook still says production Barman is +fail-closed. Those statements are now false against the catalog and +against the platform. + +This workplan makes the **operating claim** match the **live store and +the live backup**, and publishes the join artefacts neighbours can +require the same way they already require `data.backup`. + +## Why this changes how audit-core should work + +Not documentation-only. Three operating surfaces currently encode the +wrong claim: + +| Surface | Today | After this workplan | +| --- | --- | --- | +| `RetentionPolicy.custody_class` + `/readyz` | `archive` | An honest class for durable operational custody. `archive` reserved for a future `data.archive` sink. | +| `AUDIT_CORE_REQUIRE_CUSTODY_CLASS` | `archive` | The honest production class. Still fail-closed: a development backend cannot serve. | +| Recovery promise | "no deletion; Barman fail-closed; do not promise an RPO" | Recoverable history = platform `data.backup` window (30 days, cited, EvidenceBasis `measured` from resource-control T05). Unbounded retention is **unknown**, not a fact. | + +`None` retention is not infinite archive. Canon: unknown carries no +quantity and must name the gap and owner. The gap owner for off-host +recovery is rapp-postgres + resource-control. The gap owner for a +WORM / object-lock archive beyond that window is **this repo**, as an +unmet `data.archive` requirement — not a silent `retention_days=None`. + +## Three names that must stay distinct + +| Name | What it is | Owner | +| --- | --- | --- | +| `capability.audit.event-retain` / ITC-CAP `operations.audit` | Product: collect and retain audit events | **audit-core** | +| `platform:audit-core` | Postgres consumer join key for database `audit_core` | rapp-postgres (`consumers/audit-core.yaml`) | +| `platform:audit-storage` / `resource:platform:audit-storage` | Scaleway bucket for **all of** `platform-pg` Barman | resource-control + reef-storage | + +Do not restatement `data.backup` here. Cite the existing capability +case at `resource-control/data/capability/platform-audit-storage.json`. + +## Boundaries + +**This workplan owns** + +- Honest `RetentionPolicy` and the production fail-closed gate. +- Recovery language in the runbook, backend contract, and `/readyz`. +- An ITC-CAP requirement + provision record for `operations.audit`. +- A Canon Interface Card. +- Refresh of the reuse-surface card so it no longer says A2 / packaging + in progress. +- Persisting the non-secret user-engine sender tenant scope `["*"]`. +- SCOPE.md current state. + +**This workplan does not own** + +- Procuring or operating S3 / Barman / WAL (`resource-control`, + `rapp-postgres`, `reef-storage`). +- Booked Scaleway invoices or a second usage stream for + `platform:audit-storage` (`fin-hub` FIN-WP-0004-T05). +- Emitting `UsageObservation` or `booked_cost`. Keep the existing + `platform:audit-core` key on the postgres consumer. +- A `declarations/rapp.yaml` in this repo. The schema requires + `rapp_id: ^rapp-…`. Extraction to `rapp-audit-core` is a + railiance-master first-wave decision, not this file. +- Public ingest. ClusterIP + default-deny already matches ADR-0008 + private-by-default. Do not add Ingress. +- Hash-chain / tamper evidence, hot search, object-store archive + sink, v1 nested envelope, control-plane APIs, OpenBao/K8s + collectors. Those remain INTENT residuals, not this optimisation. +- OpenBao AppRole-as-ESO-parent for `database/creds`. Residual from + AUDIT-WP-0005; different problem. + +**Founder decision this workplan records, not executes:** if audit +events must survive beyond the 30-day platform backup window, that is +a `data.archive` requirement for resource-control to procure (different +bucket/lifecycle than Barman). T03 writes the requirement as unmet. Do +not build the sink here. + +## T01 — Restate custody vocabulary in the running service + +```task +id: AUDIT-WP-0006-T01 +status: todo +priority: high +``` + +`CustodyClass` is currently `development | archive | hot_search`. The +backend contract defines `archive` as "long-term evidence (planned)… +durable object storage, batch manifests". The Postgres backend reports +`archive` anyway, with `tamper_evidence=False` and `retention_days=None`. +Production will not start unless the class string is `archive`. + +That string is now a catalog collision with ITC-CAP `data.archive`. + +Make `RetentionPolicy` say what the store actually is: + +1. Add an honest production class (suggested `operational`) meaning: + durable, append-only against the runtime role, recoverable through + the platform `data.backup` provision. Keep `archive` in the enum + for a future object-store backend that can satisfy `data.archive` + hooks (`retention_policy`, `integrity_verification`, `retrieval_test`). +2. Postgres reports the honest class. `immutable=True` stays (trigger + 0002). `tamper_evidence=False` stays. Do not inflate. +3. Add explicit recovery fields so `/readyz` can publish them: + recoverable window (days), source of that number, EvidenceBasis. + The 30-day figure is **cited** from the platform provision, basis + `measured` against resource-control T05 / the 2026-08-13 walk — + not re-measured here unless the walk is re-run. +4. Change `AUDIT_CORE_REQUIRE_CUSTODY_CLASS` in + `deploy/audit-core.yaml` to the honest class. Fail-closed remains: + a `development` backend still cannot serve. Accept the old `archive` + value as an alias during one deploy so a mixed rollout cannot + refuse to start. +5. Update `docs/audit-backend-contract.md`. Delete "Production archive + policy (planned)" as if the durable backend did not exist. + +user-engine does not parse `custody_class`. Still treat `/readyz` as a +sender-visible contract: keep `custody_class` in the JSON and add the +new fields rather than removing the old one. + +Done when: unit tests cover the new class and the alias; production +manifest requires the honest class; contract and `/readyz` no longer +call Postgres `data.archive`. + +## T02 — Bind the recovery promise to the live platform backup + +```task +id: AUDIT-WP-0006-T02 +status: todo +priority: high +``` + +`docs/operator-runbook.md` still says production Barman is fail-closed +and "do not promise an RPO until that target exists". RESOURCE-WP-0002 +closed that gap on 2026-08-14. + +Rewrite the Restore section to: + +- Recoverable history is the platform `data.backup` window (30 days, + prefix `platform-pg/`, bucket owned by `resource:platform:audit-storage`). +- Cite, do not copy, `resource-control/data/capability/platform-audit-storage.json` + and `rapp-postgres/docs/restore.md`. +- State the provision honesty: requirement D5, provision D4; RPO/RTO + numbers are theirs (`measured`, single observation). audit-core does + not claim a better grade. +- audit-core still does not expire rows. That is a **lifecycle policy + statement**, not a recovery guarantee. Rows older than the backup + window are not promised after a restore. +- Physical restore remains instance-wide; single-consumer path remains + logical export. Do not invent a second restore procedure. + +Leave the 2026-08-13 walk in place as historical evidence. Add a short +dated note that the fail-closed sentence is superseded by +RESOURCE-WP-0002-T05 (30 events, 65 s full / 65 s PITR). + +Done when the runbook and the T06 evidence note no longer contradict +the live backup provision. + +## T03 — Publish an ITC-CAP case for the live provision + +```task +id: AUDIT-WP-0006-T03 +status: todo +priority: high +``` + +Neighbours already restate real provisions against the live catalog +(`resource-control/data/capability/platform-audit-storage.json`). +audit-core's reuse-surface card uses a private id +(`capability.audit.event-retain`), puts maturity on the capability +(violates CAP-R2), and still says availability A2. + +Add a capability record valid against +`info-tech-canon/infospace/schemas/capability-record.schema.yaml` and +the live `capabilities.yaml`. Suggested path: +`data/capability/audit-core-operational.json` (or equivalent under +`registry/` if a `data/` tree is not wanted — pick one and index it). + +Minimum content: + +- **requires** + - `operations.audit` (profile `administrative` or `security`), + production dependency. + - `data.archive` as an **unmet** requirement if INTENT's unbounded + WORM archive is still wanted. Do not invent a provision for it. + - Do **not** re-require `data.backup`. Record it under + `uses_provisions` pointing at the existing platform provision. +- **provision** + - provider: the live receiver (landscape id, e.g. + `audit-core.railiance01`). + - capability: `operations.audit`. + - maturity: **D4** (approved production dependency). Not D5: + integrity is trigger-enforced, not proven (`tamper_evidence=False`); + reliability is not actively controlled; one replica. + - `uses_provisions`: + - platform-pg / CNPG as the transactional store (`depends_on` + only if the catalog pair exists; otherwise `may_use` and say so). + - `data.backup` provided by `rapp-postgres/platform-pg CNPG + barmanObjectStore` (`may_use` or `depends_on` per catalog). + - OpenBao / ESO as `security.secrets` (`may_use`, never class `P`). + - `consumes` in native units with EvidenceBasis (CAP-R8, CAP-R10): + - `S` stored events — `measured` from `pg_total_relation_size` or + `/v1/stats`, or `unknown` with owner if not yet queried. + - `H` operator hours — `unknown` (owner: audit-core; start a + time record later). + - `I` tokens — `unknown`. + - Do not invent euros. `invoiced` is a fin-hub fact. + - `evidence` hooks: + - `audit_records` → existing failure-matrix / live accept evidence + (`measured`). + - `integrity_verification` → gap, owner audit-core (trigger is + not a proof). Do not borrow the restore drill for this hook. + +Refresh `registry/capabilities/capability.audit.event-retain.md` and +`registry/indexes/capabilities.yaml`: map to `operations.audit`, move +maturity onto the provision, status reflecting production, domain +`infotech`, consumption mode no longer "source module" only. Do not +delete the reuse-surface id; join it. + +Validate with `info-tech-canon capability-review` against the live +catalog. Gaps are allowed when they name owner and disposition. + +Done when the record validates and the reuse-surface card no longer +contradicts the live receiver. + +## T04 — Publish a Canon Interface Card + +```task +id: AUDIT-WP-0006-T04 +status: todo +priority: medium +``` + +ITC-GOV / ITC-SEC / ITC-DATA say subsystems that produce Evidence +SHOULD publish a Canon Interface Card. Schema requires `id`, `title`, +`consumer`, `canon_surfaces`. Use the template at +`info-tech-canon/infospace/agent/templates/canon-interface-card.template.yaml`. + +The card is how senders and reviewers join this repo to the canon +without reading INTENT. It is not a second capability catalog. + +Declare at least: + +- Produced: Evidence / AuditRecord (stored events). This service is + **not** an independent Audit-as-assessment. +- Consumed: ITC-CAP `operations.audit`, ITC-GOV Evidence / EvidenceBasis, + rapp-postgres as store, resource-control `data.backup` as recovery. +- Known deviations: no `data.archive` sink; `tamper_evidence=False`; + no hash-chain; single sender (`user-engine`); no `rapp.yaml` (not a + `rapp-*` repo); `/readyz` custody class historically overclaimed + (closed by T01). +- purpose_fit + gap rule: a gap is acceptable only with owner and + disposition. + +Done when the card exists in-repo and validates against +`interface-card.schema.yaml`. + +## T05 — Persist sender tenant scope and refresh SCOPE + +```task +id: AUDIT-WP-0006-T05 +status: todo +priority: medium +``` + +Inbox 2026-08-13 from net-kingdom (NK-WP-0024): live user-engine sender +was limited to `tenant:friendly:binky`; six legitimate platform events +for other tenants returned 400. The live Secret was corrected to +`tenants: ["*"]` (source `user-engine`, `may_write=true`, +`may_read=false`). That scope is **not a secret**. Persist it in the +authoritative senders document / deployment source so an ExternalSecret +refresh cannot revert it. Do not write token values into Git. + +`docs/senders.example.json` already shows `["*"]`. Confirm the live +Secret and the OpenBao KV / wrap-migrate source match. If the +workstation `~/.audit-core-senders.json` was later tightened for a +matrix run, do not restore that narrow list. + +Refresh `SCOPE.md` Current State: production receiver on railiance01, +Postgres operational custody, recovery bounded by platform 30-day +backup, AUDIT-WP-0001…0005 finished, this workplan active. Out of +scope stays: Barman/S3 procurement, booked cost, rapp extraction. + +Done when the live sender document cannot revert to a single-tenant +list on refresh, SCOPE matches the repo, and the net-kingdom message +is answered. + +## Acceptance + +- Production fail-closed gate no longer keys off the word `archive`. +- `/readyz` and the runbook agree with RESOURCE-WP-0002: 30-day + recoverable window, cited, graded. +- An ITC-CAP case for `operations.audit` validates; `data.archive` is + either unmet-with-owner or explicitly not required. +- Interface card and reuse-surface card no longer describe a draft + module. +- user-engine sender tenant scope `["*"]` survives credential refresh. +- No new cost stream, no `rapp.yaml` in this repo, no public ingest, + no second backup procedure.