Implement AUDIT-WP-0006 honest operational custody.
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 2s

Postgres now reports custody_class=operational with a cited 30-day
recoverable window. Join ITC-CAP operations.audit at D4, publish the
interface card, and overlay user-engine tenants [*] from Git so an
ExternalSecret refresh cannot shrink it.
This commit is contained in:
tegwick 2026-08-16 00:24:33 +02:00
parent 0a3d05ff1c
commit ded432a63f
25 changed files with 832 additions and 94 deletions

View file

@ -117,20 +117,26 @@ Compatibility rules (not yet implemented):
| Field | Meaning |
| --- | --- |
| `custody_class` | `development`, `archive`, or `hot_search` |
| `retention_days` | Maximum age before eligible deletion; `None` means indefinite |
| `custody_class` | `development`, `operational`, `archive`, or `hot_search` |
| `retention_days` | Maximum age before eligible deletion; `None` is a lifecycle statement (no expiry), not a recovery guarantee |
| `immutable` | Whether stored records are protected from in-place alteration |
| `tamper_evidence` | Whether manifests, hash chains, or signatures exist |
| `durable` | Whether survival is expected across process restarts and host reboots |
| `recoverable_days` | Cited platform backup window; `None` if not declared |
| `recoverable_source` | Where the recoverable window is cited from |
| `recoverable_basis` | ITC-GOV EvidenceBasis of that citation (`measured`, `quoted`, …) |
### Custody classes
| Class | Purpose | Guarantees |
| --- | --- | --- |
| `development` | Local integration and bootstrap wiring | Ephemeral local files; best-effort cleanup; **not audit custody** |
| `archive` | Long-term evidence (planned) | Durable object storage, batch manifests, explicit retention |
| `operational` | Durable production custody | Append-only Postgres; recoverable through the platform `data.backup` provision; not ITC-CAP `data.archive` |
| `archive` | Long-term evidence (future sink) | Reserved for a backend that can satisfy `data.archive` hooks (retention policy, integrity verification, retrieval test) |
| `hot_search` | Operational investigation (planned) | Shorter retention; searchable; not the evidence record |
`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=operational` is the production fail-closed gate. `archive` is accepted as an alias of `operational` for one mixed rollout so an old manifest cannot refuse a new image. A `development` backend satisfies neither.
### Mock file backend policy
`MockFileAuditBackend.retention_policy`:
@ -152,15 +158,20 @@ Enforcement:
**Not guaranteed:** crash-safe writes, replication, encryption, tenant isolation,
integrity proofs, or survival of `/tmp` across reboots.
### Production archive policy (planned)
### Production operational policy (Postgres, live)
Target guarantees for the first durable backend:
`PostgresAuditBackend.retention_policy`:
- `custody_class`: `archive`
- `retention_days`: scope policy (often years, sometimes indefinite)
- `immutable`: true (WORM / object lock where available)
- `tamper_evidence`: true (batch manifests with content hashes)
- `custody_class`: `operational`
- `retention_days`: unset in production (the service does not expire rows)
- `immutable`: true (trigger `events_append_only`; not a claim against the database owner)
- `tamper_evidence`: false (a superuser can drop the trigger; no hash-chain)
- `durable`: true
- `recoverable_days`: 30, cited from the platform `data.backup` provision
- `recoverable_source`: `resource-control/data/capability/platform-audit-storage.json#provisions[capability=data.backup]`
- `recoverable_basis`: `measured`
`None` retention is a lifecycle statement, not unbounded archive. Rows older than the recoverable window are not promised after a restore. A future `archive` backend that satisfies ITC-CAP `data.archive` is not implemented.
## Migration path: mock file → durable backend
@ -189,15 +200,15 @@ backend.emit(AuditEvent(source="...", action="...", resource="...", outcome="suc
1. Register a durable archive backend implementing `AuditBackend`.
2. Configure routing: development scopes may keep mock; production scopes require
`custody_class=archive`.
`custody_class=operational` (the live Postgres backend).
3. Readiness checks compare `backend.retention_policy` against scope policy and
fail closed when custody is insufficient.
### Phase 2 — archive primary (planned)
### Phase 2 — operational primary (live)
1. Point `emit` calls (or HTTP ingestion) at the archive backend.
1. HTTP ingestion writes through `PostgresAuditBackend` (`custody_class=operational`).
2. Retain mock only for local `make mock-audit-smoke` and unit tests.
3. Export historical mock JSONL into archive batches with manifest generation.
3. A future `data.archive` sink is a separate backend, not a rename of Postgres.
### Phase 3 — hot search adjunct (planned)
@ -220,7 +231,9 @@ Archive remains the evidence record; hot search may use shorter `retention_days`
| Backend | Module | Custody class |
| --- | --- | --- |
| Mock file JSONL | `audit_core.mock_file_backend.MockFileAuditBackend` | `development` |
| Archive (planned) | TBD | `archive` |
| SQLite (local / test) | `audit_core.sqlite_backend.SQLiteAuditBackend` | `development` |
| PostgreSQL (production) | `audit_core.postgres_backend.PostgresAuditBackend` | `operational` |
| Archive (planned `data.archive` sink) | TBD | `archive` |
| Hot search (planned) | TBD | `hot_search` |
## Related documents

View file

@ -27,10 +27,11 @@
"platform_logical_seconds": 3.468,
"retention": {
"audit_core_retention_days": null,
"audit_core_meaning": "does not expire or delete events",
"audit_core_meaning": "does not expire or delete events; lifecycle statement, not a recovery guarantee",
"platform_planned_window_days": 30,
"production_barman": "fail-closed; no off-host copy yet",
"recovery_bound": "platform backup retention, not audit-core deletion"
"production_barman": "superseded 2026-08-14 by RESOURCE-WP-0002-T05: live Scaleway Barman, 30-day window, audit_core.events 30=30, full 65s / PITR 65s",
"recovery_bound": "platform data.backup window (resource:platform:audit-storage), not audit-core deletion",
"provision_honesty": "requirement D5, provision D4; cited, not re-scored here"
},
"verified": true
}

99
docs/interface-card.yaml Normal file
View file

@ -0,0 +1,99 @@
schema: info-tech-canon.interface-card.v1
id: audit-core/interface-card
title: audit-core Canon Interface Card
consumer: audit-core
consumer_profile:
repo: audit-core
domain: infotech
owner: audit-core
intent: >
Provide durable, tenant-aware operational custody for audit events so
senders can treat a 202 as evidence-in-store, not a log-forwarding hint.
scope:
- audit event ingestion
- append-only operational custody
- sender binding
- recovery bound to platform backup
purposes:
- id: audit-core/operational-custody
use_case: Accept normalized events from registered senders and retain them in an append-only store.
consumer_need: A joinable operations.audit provision with an honest recovery claim.
demand_signals:
- user-engine delivers platform and tenant events over POST /v1/events
- neighbours already require data.backup in ITC-CAP terms
canon_surfaces:
- model/capability
- model/governance
- model/data
- model/security
surfaces:
implemented_profiles: []
consumed_artifacts:
- model/capability
- model/governance
- model/data
- model/security
owned_concepts: []
produced_concepts:
- Evidence
- AuditRecord
consumed_concepts:
- Evidence
- EvidenceBasis
- CapabilityProvision
- RetentionRuleReference
mappings:
- from: stored event
to: Evidence / AuditRecord
note: This service stores evidence. It is not an independent Audit-as-assessment.
- from: capability.audit.event-retain
to: operations.audit
note: data/capability/audit-core-operational.json
validation_expectations:
commands:
- PYTHONPATH=src python3 -m info_tech_canon capability-review /home/worsch/audit-core/data/capability/audit-core-operational.json
evidence_required:
- data/capability/audit-core-operational.json
- docs/operator-runbook.md Restore section
- docs/evidence/restore-walk-20260813T121200Z.json
known_gaps:
- id: data.archive-unprovided
owner: audit-core
disposition: unmet requirement recorded on the ITC-CAP case; do not build the sink in AUDIT-WP-0006
- id: tamper-evidence-false
owner: audit-core
disposition: integrity_verification hook is unknown; trigger is not a proof
- id: no-hash-chain
owner: audit-core
disposition: INTENT residual, not this workplan
- id: single-sender
owner: audit-core
disposition: NetworkPolicy admits user-engine only; other sources remain adapters
- id: no-rapp-yaml
owner: railiance-master
disposition: schema requires rapp-*; extraction is a first-wave family decision
- id: historic-archive-overclaim
owner: audit-core
disposition: closed by AUDIT-WP-0006-T01; /readyz reports operational
purpose_fit:
state: partial
matched_capabilities:
- operations.audit
scope_pressure: >
INTENT describes a control plane, object archive, hot search, and export.
The live service is a single-sender operational custody receiver.
recommended_disposition: keep the operational provision honest; do not inflate to data.archive
consumer_needs:
current:
- Honest custody_class and recoverable window on /readyz
- Joinable operations.audit provision
requested_extensions:
- data.archive sink if events must survive past the 30-day backup window
feedback: []
known_deviations:
- no data.archive sink
- tamper_evidence=False
- no hash-chain
- single sender user-engine
- no rapp.yaml (not a rapp-* repo)
- /readyz historically overclaimed archive (closed by T01)

View file

@ -29,12 +29,13 @@ warden route show database-dynamic-credentials --json
| Check | Meaning |
| --- | --- |
| `GET /healthz` | Process is up. Liveness uses this. A database outage must **not** restart the pod. |
| `GET /readyz` | Custody is reachable and `custody_class=archive`. Readiness uses this; the pod leaves the Service rather than accept events it cannot store. |
| `GET /readyz` | Custody is reachable and `custody_class=operational`. Also reports `recoverable_days` (cited platform backup window). Readiness uses this; the pod leaves the Service rather than accept events it cannot store. |
| `GET /v1/stats` | In-process counters since start (`accepted`, `duplicate`, `conflict`, `rejected`, `unauthorized`, `forbidden`, `unavailable`, `error`). Resets on restart. Requires `may_read`. |
A missing `AUDIT_CORE_DATABASE_URL` / credential directory is a startup
failure (`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=archive`), not a silent downgrade
to SQLite.
failure (`AUDIT_CORE_REQUIRE_CUSTODY_CLASS=operational`), not a silent
downgrade to SQLite. An older manifest that still requires `archive` is
accepted as an alias for one mixed rollout.
## Lookup
@ -96,9 +97,14 @@ replace that path with a founder paste. Rotation is overlap-first:
5. Either wait for the 1h refresh or annotate the ExternalSecret to force a
sync, then restart the pod so it re-reads `AUDIT_CORE_SENDERS`.
The write token is bound to `source=user-engine` and the tenants that
identity may claim. The operator token is a separate identity with
`may_read: true`. Do not reuse one token for both.
The write token is bound to `source=user-engine`. Tenant scope for that
identity is **not a secret**: `deploy/senders-scope.json` (ConfigMap
`audit-core-senders-scope`) overlays `tenants: ["*"]` onto the Secret
at start. An ExternalSecret refresh cannot revert it to a single tenant.
Tokens stay in Secret `audit-core-senders` / OpenBao KV.
The operator token is a separate identity with `may_read: true`. Do not
reuse one token for both.
A shape (values are placeholders) is in `docs/senders.example.json`.
@ -142,17 +148,34 @@ behind the read privilege). Watch:
## Restore
audit-core does **not** expire events (`retention_days` unset). Recovery is
bounded by what rapp-postgres can restore, not by an audit-core deletion
window. The platform's planned Barman window is 30 days. Production Barman
is still fail-closed (no governed off-host target). Do not promise an RPO
until that target exists. Local WAL on the node is not an off-host copy.
audit-core does **not** expire events (`retention_days` unset). That is a
lifecycle policy statement, not a recovery guarantee. Recoverable history
is the platform `data.backup` window: **30 days**, prefix `platform-pg/`,
bucket owned by `resource:platform:audit-storage`. Cite, do not copy:
- `resource-control/data/capability/platform-audit-storage.json`
- `rapp-postgres/docs/restore.md`
The platform requirement is `data.backup` profile `database` at D5
(RPO 5 min, RTO 60 min, 30-day retention, not in the railiance01 /
host-europe failure domain). The live provision is **D4**, not D5:
resource-control scored one backup, one full restore, one PITR, and
does not yet emit `wal_archive_gap_minutes`. RPO/RTO numbers are
theirs (`measured`, single observation). audit-core does not claim a
better grade.
Rows older than the 30-day window are not promised after a restore.
Local WAL on the node is not a second copy.
Physical restore is instance-wide. A consumer-only restore is a logical
export of `audit_core` from a scratch physical restore, then a controlled
import. Never recover in place. Procedure: `rapp-postgres/docs/restore.md`.
Walked 2026-08-13:
The 2026-08-13 fail-closed sentence is superseded by RESOURCE-WP-0002-T05
(2026-08-14): production Barman to Scaleway, `audit_core.events` 30=30,
full restore 65 s, PITR 65 s.
Walked 2026-08-13 (pre-commissioning historical evidence):
| Path | Evidence | Elapsed | Result |
| --- | --- | --- | --- |
@ -185,5 +208,6 @@ narrows a column must replace that note before release.
3. Sender registry is Secret `audit-core-senders` (in-cluster mint).
Database leases come from `openbao-audit-core-database`.
4. Job `audit-core-migrate` with `AUDIT_CORE_MIGRATE_ROLE=audit_core_migrate`.
5. Deployment. `/readyz` must report `custody_class=archive`.
5. Deployment. `/readyz` must report `custody_class=operational` and
`recoverable_days=30`.
6. In-pod `MODE=remote DISRUPT=0` failure matrix. Evidence goes to NK-WP-0024.