audit-core/workplans/AUDIT-WP-0006-honest-custody-and-canon-join.md
tegwick ded432a63f
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 2s
Implement AUDIT-WP-0006 honest operational custody.
Postgres now reports custody_class=operational with a cited 30-day
recoverable window. Join ITC-CAP operations.audit at D4, publish the
interface card, and overlay user-engine tenants [*] from Git so an
ExternalSecret refresh cannot shrink it.
2026-08-16 00:24:33 +02:00

17 KiB

id type title domain repo status owner topic_slug created updated depends_on state_hub_workstream_id
AUDIT-WP-0006 workplan Honest operational custody against ITC-CAP and the live platform backup infotech audit-core finished grok railiance 2026-08-15 2026-08-16
AUDIT-WP-0005
RESOURCE-WP-0002
ITC-WP-0014
ITC-WP-0015
8d775ffb-3c83-4c33-9ffa-05ce52c5ff91

AUDIT-WP-0006 — Honest operational custody against ITC-CAP and the live platform backup

Goal

Stop claiming a capability the live catalog and the live backup provision contradict, and join the spine neighbours already use.

After AUDIT-WP-0005 the receiver is in production on railiance01, append-only in platform-pg, fail-closed on AUDIT_CORE_REQUIRE_CUSTODY_CLASS=archive. Since then three things landed that change what that claim means:

  1. info-tech-canon 0.6.0 / ITC-CAP 0.4.0 names distinct abilities: operations.audit (attributable historical evidence), data.archive (long-term retention under lifecycle/integrity/retrieval), data.backup (recover a store), governance.evidence, governance.lifecycle. Consumption must carry an EvidenceBasis. Maturity attaches to a provision, not an abstract capability.
  2. resource-control RESOURCE-WP-0002 is finished. Off-host Barman to Scaleway (resource:platform:audit-storage) is live. Restore evidence used audit_core.events as the checksum. The provision is scored data.backup D4 against a D5 requirement. The 30-day window is real.
  3. fin-hub / railiance-master tightened ownership: booked cost stays in fin-hub; usage and inventory stay in resource-control; rapp.yaml is only valid in a rapp-* repo; default exposure is private.

audit-core still reports custody_class=archive, documents archive as "planned WORM + manifests", leaves retention_days unset (read as indefinite), and keeps a reuse-surface card at A2 / source module / helix_forge. The operator runbook still says production Barman is fail-closed. Those statements are now false against the catalog and against the platform.

This workplan makes the operating claim match the live store and the live backup, and publishes the join artefacts neighbours can require the same way they already require data.backup.

Why this changes how audit-core should work

Not documentation-only. Three operating surfaces currently encode the wrong claim:

Surface Today After this workplan
RetentionPolicy.custody_class + /readyz archive An honest class for durable operational custody. archive reserved for a future data.archive sink.
AUDIT_CORE_REQUIRE_CUSTODY_CLASS archive The honest production class. Still fail-closed: a development backend cannot serve.
Recovery promise "no deletion; Barman fail-closed; do not promise an RPO" Recoverable history = platform data.backup window (30 days, cited, EvidenceBasis measured from resource-control T05). Unbounded retention is unknown, not a fact.

None retention is not infinite archive. Canon: unknown carries no quantity and must name the gap and owner. The gap owner for off-host recovery is rapp-postgres + resource-control. The gap owner for a WORM / object-lock archive beyond that window is this repo, as an unmet data.archive requirement — not a silent retention_days=None.

Three names that must stay distinct

Name What it is Owner
capability.audit.event-retain / ITC-CAP operations.audit Product: collect and retain audit events audit-core
platform:audit-core Postgres consumer join key for database audit_core rapp-postgres (consumers/audit-core.yaml)
platform:audit-storage / resource:platform:audit-storage Scaleway bucket for all of platform-pg Barman resource-control + reef-storage

Do not restatement data.backup here. Cite the existing capability case at resource-control/data/capability/platform-audit-storage.json.

Boundaries

This workplan owns

  • Honest RetentionPolicy and the production fail-closed gate.
  • Recovery language in the runbook, backend contract, and /readyz.
  • An ITC-CAP requirement + provision record for operations.audit.
  • A Canon Interface Card.
  • Refresh of the reuse-surface card so it no longer says A2 / packaging in progress.
  • Persisting the non-secret user-engine sender tenant scope ["*"].
  • SCOPE.md current state.

This workplan does not own

  • Procuring or operating S3 / Barman / WAL (resource-control, rapp-postgres, reef-storage).
  • Booked Scaleway invoices or a second usage stream for platform:audit-storage (fin-hub FIN-WP-0004-T05).
  • Emitting UsageObservation or booked_cost. Keep the existing platform:audit-core key on the postgres consumer.
  • A declarations/rapp.yaml in this repo. The schema requires rapp_id: ^rapp-…. Extraction to rapp-audit-core is a railiance-master first-wave decision, not this file.
  • Public ingest. ClusterIP + default-deny already matches ADR-0008 private-by-default. Do not add Ingress.
  • Hash-chain / tamper evidence, hot search, object-store archive sink, v1 nested envelope, control-plane APIs, OpenBao/K8s collectors. Those remain INTENT residuals, not this optimisation.
  • OpenBao AppRole-as-ESO-parent for database/creds. Residual from AUDIT-WP-0005; different problem.

Founder decision this workplan records, not executes: if audit events must survive beyond the 30-day platform backup window, that is a data.archive requirement for resource-control to procure (different bucket/lifecycle than Barman). T03 writes the requirement as unmet. Do not build the sink here.

T01 — Restate custody vocabulary in the running service

id: AUDIT-WP-0006-T01
status: done
priority: high
state_hub_task_id: "bbf476eb-7bc3-4cb6-bd1c-9b2c75903906"

CustodyClass is currently development | archive | hot_search. The backend contract defines archive as "long-term evidence (planned)… durable object storage, batch manifests". The Postgres backend reports archive anyway, with tamper_evidence=False and retention_days=None. Production will not start unless the class string is archive.

That string is now a catalog collision with ITC-CAP data.archive.

Make RetentionPolicy say what the store actually is:

  1. Add an honest production class (suggested operational) meaning: durable, append-only against the runtime role, recoverable through the platform data.backup provision. Keep archive in the enum for a future object-store backend that can satisfy data.archive hooks (retention_policy, integrity_verification, retrieval_test).
  2. Postgres reports the honest class. immutable=True stays (trigger 0002). tamper_evidence=False stays. Do not inflate.
  3. Add explicit recovery fields so /readyz can publish them: recoverable window (days), source of that number, EvidenceBasis. The 30-day figure is cited from the platform provision, basis measured against resource-control T05 / the 2026-08-13 walk — not re-measured here unless the walk is re-run.
  4. Change AUDIT_CORE_REQUIRE_CUSTODY_CLASS in deploy/audit-core.yaml to the honest class. Fail-closed remains: a development backend still cannot serve. Accept the old archive value as an alias during one deploy so a mixed rollout cannot refuse to start.
  5. Update docs/audit-backend-contract.md. Delete "Production archive policy (planned)" as if the durable backend did not exist.

user-engine does not parse custody_class. Still treat /readyz as a sender-visible contract: keep custody_class in the JSON and add the new fields rather than removing the old one.

Done when: unit tests cover the new class and the alias; production manifest requires the honest class; contract and /readyz no longer call Postgres data.archive.

Done 2026-08-16: CustodyClass includes operational; Postgres reports it with a 30-day cited recoverable window (measured). AUDIT_CORE_REQUIRE_CUSTODY_CLASS=operational in the manifest; archive is a one-deploy alias. /readyz publishes recovery fields. Contract replaced "Production archive policy (planned)" with the live operational policy. Suite 84 passed. Do not apply the Deployment until a new image is pinned — the live image still reports archive and has no alias.

T02 — Bind the recovery promise to the live platform backup

id: AUDIT-WP-0006-T02
status: done
priority: high
state_hub_task_id: "06907b2c-a822-48f0-8edc-50dd8914342b"

docs/operator-runbook.md still says production Barman is fail-closed and "do not promise an RPO until that target exists". RESOURCE-WP-0002 closed that gap on 2026-08-14.

Rewrite the Restore section to:

  • Recoverable history is the platform data.backup window (30 days, prefix platform-pg/, bucket owned by resource:platform:audit-storage).
  • Cite, do not copy, resource-control/data/capability/platform-audit-storage.json and rapp-postgres/docs/restore.md.
  • State the provision honesty: requirement D5, provision D4; RPO/RTO numbers are theirs (measured, single observation). audit-core does not claim a better grade.
  • audit-core still does not expire rows. That is a lifecycle policy statement, not a recovery guarantee. Rows older than the backup window are not promised after a restore.
  • Physical restore remains instance-wide; single-consumer path remains logical export. Do not invent a second restore procedure.

Leave the 2026-08-13 walk in place as historical evidence. Add a short dated note that the fail-closed sentence is superseded by RESOURCE-WP-0002-T05 (30 events, 65 s full / 65 s PITR).

Done when the runbook and the T06 evidence note no longer contradict the live backup provision.

Done 2026-08-16: Restore section cites resource:platform:audit-storage, D5 requirement / D4 provision, and supersedes the 2026-08-13 fail-closed sentence with RESOURCE-WP-0002-T05. Evidence JSON updated in place.

T03 — Publish an ITC-CAP case for the live provision

id: AUDIT-WP-0006-T03
status: done
priority: high
state_hub_task_id: "eba29df0-36b9-4f43-add6-760285c56bde"

Neighbours already restate real provisions against the live catalog (resource-control/data/capability/platform-audit-storage.json). audit-core's reuse-surface card uses a private id (capability.audit.event-retain), puts maturity on the capability (violates CAP-R2), and still says availability A2.

Add a capability record valid against info-tech-canon/infospace/schemas/capability-record.schema.yaml and the live capabilities.yaml. Suggested path: data/capability/audit-core-operational.json (or equivalent under registry/ if a data/ tree is not wanted — pick one and index it).

Minimum content:

  • requires
    • operations.audit (profile administrative or security), production dependency.
    • data.archive as an unmet requirement if INTENT's unbounded WORM archive is still wanted. Do not invent a provision for it.
    • Do not re-require data.backup. Record it under uses_provisions pointing at the existing platform provision.
  • provision
    • provider: the live receiver (landscape id, e.g. audit-core.railiance01).
    • capability: operations.audit.
    • maturity: D4 (approved production dependency). Not D5: integrity is trigger-enforced, not proven (tamper_evidence=False); reliability is not actively controlled; one replica.
    • uses_provisions:
      • platform-pg / CNPG as the transactional store (depends_on only if the catalog pair exists; otherwise may_use and say so).
      • data.backup provided by rapp-postgres/platform-pg CNPG barmanObjectStore (may_use or depends_on per catalog).
      • OpenBao / ESO as security.secrets (may_use, never class P).
    • consumes in native units with EvidenceBasis (CAP-R8, CAP-R10):
      • S stored events — measured from pg_total_relation_size or /v1/stats, or unknown with owner if not yet queried.
      • H operator hours — unknown (owner: audit-core; start a time record later).
      • I tokens — unknown.
      • Do not invent euros. invoiced is a fin-hub fact.
    • evidence hooks:
      • audit_records → existing failure-matrix / live accept evidence (measured).
      • integrity_verification → gap, owner audit-core (trigger is not a proof). Do not borrow the restore drill for this hook.

Refresh registry/capabilities/capability.audit.event-retain.md and registry/indexes/capabilities.yaml: map to operations.audit, move maturity onto the provision, status reflecting production, domain infotech, consumption mode no longer "source module" only. Do not delete the reuse-surface id; join it.

Validate with info-tech-canon capability-review against the live catalog. Gaps are allowed when they name owner and disposition.

Done when the record validates and the reuse-surface card no longer contradicts the live receiver.

Done 2026-08-16: data/capability/audit-core-operational.json reviews ok against ITC-CAP 0.4.0 (operations.audit met at D4; data.archive unprovided). Reuse-surface card joins that provision, domain infotech, status production.

T04 — Publish a Canon Interface Card

id: AUDIT-WP-0006-T04
status: done
priority: medium
state_hub_task_id: "c5f5a883-834f-42eb-8f05-de2857d3904f"

ITC-GOV / ITC-SEC / ITC-DATA say subsystems that produce Evidence SHOULD publish a Canon Interface Card. Schema requires id, title, consumer, canon_surfaces. Use the template at info-tech-canon/infospace/agent/templates/canon-interface-card.template.yaml.

The card is how senders and reviewers join this repo to the canon without reading INTENT. It is not a second capability catalog.

Declare at least:

  • Produced: Evidence / AuditRecord (stored events). This service is not an independent Audit-as-assessment.
  • Consumed: ITC-CAP operations.audit, ITC-GOV Evidence / EvidenceBasis, rapp-postgres as store, resource-control data.backup as recovery.
  • Known deviations: no data.archive sink; tamper_evidence=False; no hash-chain; single sender (user-engine); no rapp.yaml (not a rapp-* repo); /readyz custody class historically overclaimed (closed by T01).
  • purpose_fit + gap rule: a gap is acceptable only with owner and disposition.

Done when the card exists in-repo and validates against interface-card.schema.yaml.

Done 2026-08-16: docs/interface-card.yaml validates. Schema wants consumer as a string and canon_surfaces as a string array; richer template fields live under consumer_profile / surfaces.

T05 — Persist sender tenant scope and refresh SCOPE

id: AUDIT-WP-0006-T05
status: done
priority: medium
state_hub_task_id: "c629d894-2fe6-4b46-9909-7267d9170c84"

Inbox 2026-08-13 from net-kingdom (NK-WP-0024): live user-engine sender was limited to tenant:friendly:binky; six legitimate platform events for other tenants returned 400. The live Secret was corrected to tenants: ["*"] (source user-engine, may_write=true, may_read=false). That scope is not a secret. Persist it in the authoritative senders document / deployment source so an ExternalSecret refresh cannot revert it. Do not write token values into Git.

docs/senders.example.json already shows ["*"]. Confirm the live Secret and the OpenBao KV / wrap-migrate source match. If the workstation ~/.audit-core-senders.json was later tightened for a matrix run, do not restore that narrow list.

Refresh SCOPE.md Current State: production receiver on railiance01, Postgres operational custody, recovery bounded by platform 30-day backup, AUDIT-WP-0001…0005 finished, this workplan active. Out of scope stays: Barman/S3 procurement, booked cost, rapp extraction.

Done when the live sender document cannot revert to a single-tenant list on refresh, SCOPE matches the repo, and the net-kingdom message is answered.

Done 2026-08-16: Live Secret already has user-engine tenants: ["*"]. deploy/senders-scope.json + ConfigMap audit-core-senders-scope applied on railiance01. The process overlays that file over the Secret so a later KV refresh cannot shrink tenants. Tokens stay out of Git. SCOPE current state updated.

Acceptance

  • Production fail-closed gate no longer keys off the word archive.
  • /readyz and the runbook agree with RESOURCE-WP-0002: 30-day recoverable window, cited, graded.
  • An ITC-CAP case for operations.audit validates; data.archive is either unmet-with-owner or explicitly not required.
  • Interface card and reuse-surface card no longer describe a draft module.
  • user-engine sender tenant scope ["*"] survives credential refresh.
  • No new cost stream, no rapp.yaml in this repo, no public ingest, no second backup procedure.