audit-core/workplans/AUDIT-WP-0009-evidence-role-conformance.md
tegwick cb23dc82fd
All checks were successful
CI Smoke / host-smoke (push) Successful in 1s
CI Smoke / container-smoke (push) Successful in 1s
Prepare compatible audit receiver with verified container lifecycle
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc
2026-09-11 06:46:15 +02:00

26 KiB
Raw Blame History

id type title domain repo status owner topic_slug created updated depends_on state_hub_workstream_id
AUDIT-WP-0009 workplan Evidence-role conformance under Security Layer Model v0.7 infotech audit-core active claude railiance 2026-08-29 2026-09-11
AUDIT-WP-0007
46a96b03-bc08-53b5-9c93-4071adabf734

AUDIT-WP-0009 — Evidence-role conformance under Security Layer Model v0.7

Goal

Make audit-core dischargeable as the estate's Evidence engine under NetKingdom Security Layer Model v0.7 (accepted), and stop it claiming a property its own contract does not permit.

Two things drive this workplan.

First, a live overclaim. audit_core/postgres_backend.py returns tamper_evidence=True as a constant, while docs/integrity.md permits that claim only when a live external chain-head attestation exists. The one attestation on record is 2026-08-16 and nothing renews it. Audit Core is currently telling every sender it has a property whose precondition is unverified — the §9.6 defect it twice corrected in gate-house's doctrine, turned inward.

Second, an obligation audit-core argued for and cannot yet support. v0.7 §9.6 makes a declared emission cadence a MUST for load-bearing sources, with reconciliation or a heartbeat as the required form for low-volume classes. audit-core raised both, moving cadence from SHOULD to MUST. A source cannot declare a cadence to a system with nowhere to put it. Having won the obligation, audit-core owes the surface.

Full assessment: history/2026-08-29-v0.7-alignment-and-scope-assessment.md. Standard: net-kingdom/canon/standards/security-layer-model_v0.7.md, companion net-kingdom/SECURITY-COMPANION.md.

Non-goals

Fixed by the statute; not deferred, not ours:

  • No authorization decision surface, and no approval-validity query (§6, §9.4).
  • No operative approval state (§9.4).
  • No WORM, object lock, or ITC-CAP data.archive — §16 ruled the stronger-custody gap closed, a different store with a different owner.
  • Emission atomicity at the source, which is the emitter's obligation (§9.6).

Tasks

id: AUDIT-WP-0009-T01
status: done
priority: high
state_hub_task_id: "f545b0e4-8c99-5186-affd-ce9a41209ed7"

Make tamper_evidence conditional on live attestation state. Derive the flag rather than hard-coding it: the backend reports True only when a chain-head attestation exists, verifies against the live chain, and is within a declared freshness window. Stale or absent attestation degrades the claim — it does not leave it standing. Assert the degradation with a test, and state the freshness window in docs/integrity.md alongside the two existing preconditions, which today are documented but unenforced.

Done 2026-09-06. evaluate_tamper_evidence in audit_core/integrity.py derives the claim from a ChainReport and the mounted attestation; PostgresAuditBackend reads it per retention_policy call through tamper_evidence_state(), briefly cached because /readyz probes it. Seven degradation reasons are distinguished and asserted — no_attestation, attestation_stale, attestation_mismatch, attestation_undated, chain_break, chain_unreadable, against attested. Unreadable is treated as absent deliberately: a malformed file must not hold up a claim a missing file would drop. The freshness window is 168h against an intended daily cadence, declared in docs/integrity.md with the reasoning that the window is seven cadences so missed runs degrade rather than flap. Twelve tests in tests/test_integrity.py; docs/audit-backend-contract.md now states the field is derived rather than declared.

Consequence to state plainly: production /readyz will report tamper_evidence: false until T02 mounts a renewed attestation. The 2026-08-16 attestation is 21 days old against a 7-day window. That is the overclaim being removed, not a regression — the claim was false before and now says so.

id: AUDIT-WP-0009-T02
status: done
priority: high
state_hub_task_id: "6de9945f-4dd1-57dd-898a-f59c35b1df6c"

Done 2026-09-10. deploy/attest-cronjob.yaml: daily at 03:17 UTC against the 168h window, its own ServiceAccount, and a Role reaching exactly one named ConfigMap — get/update/patch, no create, no list.

Three decisions worth recording, all of them refusals.

The producer is not the receiver. A receiver that could rewrite its own attestation could forge it, so the job is a separate workload and the receiver has no API-server egress. audit-core-egress is now scoped to component: receiver and a new audit-core-attest-egress carries the 6443 rule; tests/test_layer_conformance.py asserts the receiver never gains it.

It refuses to publish over a broken chain. A fresh head written over a break replaces an honest chain_break with a fresh-looking attestation. Stale degrades the claim visibly; false does not.

Mounted as a directory, not subPath. Found while writing the manifest: a subPath ConfigMap mount is resolved once at pod start and never updates, so the daily attestation would land in the ConfigMap and never reach the running receiver — tamper_evidence would age out to false while the job reported success every night, silent in both directions.

audit_core/attest_publish.py does the publish in stdlib rather than shelling out, because the image carries no kubectl and adding one to an audit receiver's image to write a single file is the worse trade.

The offsite copy stays an operator step: audit-core holds no Nextcloud credential and should not acquire one to publish a hash. Stated plainly in docs/integrity.md — until that copy exists, the delivered control is "defends against a database owner, not a cluster owner", and no stronger claim may be made from it. Applying the manifests remains an operator action. Schedule chain-head attestation so the precondition T01 enforces is normally met. attest-chain exists and is operator-run; deploy/ has no job. Add one, write the attestation to the logical-offsite path already used by RESOURCE-WP-0002-T06 (rapp-postgres / Nextcloud + age) and not the Barman prefix, per docs/integrity.md — that copy is restored with the table and proves nothing. Record the cadence in docs/integrity.md.

id: AUDIT-WP-0009-T03
status: done
priority: high
state_hub_task_id: "acae6085-ada7-51b2-8dc0-4abac7f7e7b3"

Represent the §9.6 evidence kind per source. Add evidence_kind (load-bearing | attributive) to SenderIdentity and the sender registration schema, defaulting to attributive — a source that has not declared is not silently treated as load-bearing. Where a source declares attributive and has deliberately traded away atomicity, carry the declared trade with it, because §9.6 requires the trade be declared where the trail is documented. Prerequisite for T04T06.

Done 2026-09-06. evidence_kind and completeness_trade on SenderIdentity, the AUDIT_CORE_SENDERS schema, and the non-secret scope overlay. Two asymmetries are deliberate and asserted:

  • Default is attributive. The other default would have audit-core imply a completeness obligation no source ever accepted.
  • The overlay may raise the kind, never lower it. Same principle that stops an ExternalSecret refresh shrinking user-engine's tenants: a ConfigMap refresh must not be able to drop a source's atomicity and detection obligations without anyone deciding to. Downgrading raises.

A load-bearing source may not carry a completeness_trade — §9.6 requires atomicity of it, so recording a trade would store a contradiction as policy. evidence_declaration() states completeness_claimed: False for both kinds, because neither licenses the claim that the archive proves an event occurred. Twelve tests in tests/test_senders.py.

id: AUDIT-WP-0009-T04
status: done
priority: high
state_hub_task_id: "f36470af-4019-54b2-96d7-e049d867c7db"

Done 2026-09-10. audit_core/stream_findings.py, surfaced at GET /v1/stream-findings, documented in docs/stream-completeness.md.

A heartbeat is an ordinary event — same envelope, same append-only custody, same chain, no special table. A heartbeat stored outside the chain would be the one record in this store that could be back-dated.

Declared per class, not per source, on the sender registration (heartbeat_classes). informed-decision raised the shape first and it generalises: a per-source heartbeat from a mixed-volume emitter is satisfied by its chattiest class and says nothing about the quiet, security-relevant one, which is the only reason heartbeats exist.

no_heartbeat_since_registration is a distinct finding kind rather than a skip — a source that declared a heartbeat and never sent one is the case most likely to be a broken integration, and it is exactly what a "compare against last seen" implementation drops silently. A wildcard source is refused rather than guessed at: there is no determinate set of streams to expect a heartbeat from.

Explicitly not T05's territory. Cadence describes expected rate and belongs to the §17 Taxonomy schema; this says how often a source promises to say nothing to report for a class that may legitimately be silent. Heartbeat ingestion and missing-heartbeat findings. The required form for low-volume load-bearing classes, and the only control covering adversarial omission for exactly the rare negative events — revocation, denial, containment — where suppression is most valuable and least visible. Accept a signed nothing to report assertion per source and class; raise a finding when an expected heartbeat does not arrive. The point is a positive claim that can itself go missing, which rate monitoring can never produce.

id: AUDIT-WP-0009-T05
status: wait
priority: medium
state_hub_task_id: "382575ca-1801-5a32-a93d-90a8b9c8adbf"

Accept and evaluate a declared emission cadence per source, and raise a finding when the stream falls below it. Waiting on the §17 emission-cadence declaration schema, which kings-guard is drafting and Taxonomy will own; do not invent a competing shape. Cadence covers high-volume streams; T04 covers the low-volume load-bearing case, and the two are complementary rather than alternatives.

id: AUDIT-WP-0009-T06
status: done
priority: medium
state_hub_task_id: "0a8d6eed-75dd-5aaa-bf86-60be9dadca03"

Done 2026-09-10. GET /v1/reconciliation?source=&tenant=&since=&until=, per-class counts, event_counts on both the SQLite and Postgres backends. Counts only — never payloads.

The read-model tension, resolved rather than ignored. The task said respect may_read. But every registered sender holds may_read: false, so read literally that makes the §9.6 reconciliation obligation undischargeable by every source actually registered — a strange end for a rule audit-core argued for. The resolution: a source asking how many of its own events audit-core holds is not reading the archive, because it learns nothing it did not itself emit. So the surface is scoped to the caller's own permitted sources and tenants and carries no payloads; another source's counts, or an unscoped aggregate, stay behind may_read and full tenant scope.

Two refusals: another source's counts return 403, not an empty count, since a zero would read as "we hold none of yours" — a false answer to a question about completeness. And there is no default window: a count whose bounds the caller did not choose is not comparable to anything the caller computed, and would be quoted as though it were. Reconciliation surface: per-source, per-class event counts over a bounded window, readable by the source itself, so an emitter can compare audit-core's count against its own state transitions and treat divergence as a finding. GH-WP-0002-T04 is the reference instance on the approval-engine side; this is its counterpart here. Respect the existing read model — may_read, tenant scoping, and no payloads in a counts response.

id: AUDIT-WP-0009-T07
status: done
priority: medium
state_hub_task_id: "f572dfeb-0d1b-58d6-be83-405125189028"

Done 2026-09-10. GET /v1/stream-findings, following the existing findings-surface conventions rather than adding a new shape: may_read plus full tenant scope, because findings span every registered sender and carry no tenant key to filter on — the same rule as dead letters, secret findings and integrity.

INTENT principle 10 had the principle and no mechanism. It has one now.

Every finding and every reconciliation response carries a means field stating the bound — a missing heartbeat is not proof of suppression, and agreement on counts proves neither completeness nor that any event occurred. On the response rather than in a document nobody opens beside it, because these are the numbers most likely to be quoted out of context in someone else's conformance argument. Give stream-completeness findings a home. /v1/dead-letters and /v1/secret-findings exist; a cadence miss (T05) and a missing heartbeat (T04) have nowhere to surface. INTENT principle 10 already says a degraded audit stream is itself an audit and operations event — the principle is in place and the mechanism is not. Follow the existing findings-surface conventions rather than adding a new shape.

id: AUDIT-WP-0009-T08
status: done
priority: medium
state_hub_task_id: "f7b513ee-af48-5c5a-a34e-3e9922d29b76"

Assert the §9.4 approval-validity prohibition with a negative test. It is currently honoured by absence, which is not the estate's idiom: §6.4 obligation 3 requires a PEP's published stance map equal shipped behaviour asserted by test. A prohibition worth stating in INTENT.md, SCOPE.md, and layer.yaml is worth asserting in tests/.

id: AUDIT-WP-0009-T09
status: progress
priority: high
state_hub_task_id: "fd4a4ac3-e525-57c1-9179-e0fcd0226913"

Register approval-engine as a distinct source under §9.4 and AUDIT-IN-0001: sender registration, the four event classes (issuance, use, supersession, revocation), tenancy mapping, retention profile, secret_policy, and evidence_kind: load-bearing from T03. Onboarded under INTENT principle 4 — declared ownership, retention, access, export and evidence policy, not merely events arriving.

Re-priced low → high, 2026-09-06. glas-harness reports (message 85c68e66) that approval-engine production startup now requires audit delivery, putting this on the deployment critical path. The old note — "not blocking: approval-engine is not yet emitting" — read the dependency backwards: audit-core is not waiting for a producer, the producer is waiting for a sender identity. Corrected.

What this does not change: T09 still depends on T03, because §9.4 makes approval evidence load-bearing and there is nowhere to record that until evidence_kind exists. Registering the sender ahead of T03 would admit a load-bearing source into a system that would silently treat it as attributive — the precise defect §9.6 names. T03 is therefore promoted with it. Cadence and reconciliation (T04/T06) remain open gates on the detection half and are not prerequisites for admission; they bound what may be claimed of the stream once it flows, and approval-engine should be told so rather than discovering it later.

Token provisioning goes by the approved custody path (approval-engine/approval-engine-audit, key audit-token) via warden route; no sender token passes through Git, State Hub, or a workplan.

In progress 2026-09-06. T03 landed, so the registration inputs are now expressible. Prepared, non-secret, and recorded in docs/approval-engine-source-registration.md:

  • Non-secret scope in deploy/senders-scope.json and senders-scope.yaml, evidence_kind: load-bearing, may_read: false, exact source match. Landed ahead of the credential deliberately: the overlay only applies to a sender the Secret already carries, so the entry admits nothing until the token exists. A test asserts that inertness rather than trusting the reading.
  • audit-core-approval-engine-ingress in deploy/networkpolicies.yaml: namespace and pod label in one from peer, so both are ANDed. Narrower than user-engine's namespace-only rule on purpose — a new sender should not inherit an older rule's breadth. user-engine's policy is unchanged, and a test asserts it stays that way.
  • The four §9.4 classes, retention (no expiry; 30-day measured recoverable window), custody class, and the claim bound.

Two inputs are owed by approval-engine, not by audit-core. tenants is proposed as ["*"] and needs confirmation with reasoning or a list — senders.py says a missing tenant restriction must be justified per sender, and audit-core cannot justify it on another repo's behalf. secret_policy is recommended redact (losing a revocation record is worse than storing it with one field redacted) but should be chosen explicitly rather than inherited. Applying the manifests is an operator action audit-core does not take unprompted.

T02 is not a gate on this. Asked directly by glas-harness: attestation freshness governs what audit-core may claim, not whether it accepts and durably stores. The append-only trigger and hash chain operate regardless. And it is not even the relevant control — §9.6 is explicit that the acute risk for approvals is omission, which tamper evidence does not address at all. T04 does. A consumer waiting on T02 before trusting approval evidence would be waiting on the wrong thing.

2026-09-08 source return: approval-engine now records the exact existing store scope ["tenant:platform"] and explicitly selects redact in docs/audit-source-registration.md. This supersedes the two input waits above. The JSON and ConfigMap overlays are narrowed together. The overlay preserves secret_policy from the protected registry, so provisioning must set it there. T09 remains progress for the platform custody record, credential, admitted manifest application and positive/negative live ingestion evidence. No secret was created and no production manifest was applied by this source change.

id: AUDIT-WP-0009-T10
status: done
priority: low
state_hub_task_id: "ac3464fd-3df9-51a1-b1f4-3bf3c60e4265"

Done 2026-09-10. layer.yaml now lists four infrastructure contacts — platform-pg, state-hub, kube-apiserver, and the container registry — each with its role and whether another layer reads it. tooling_contacts stays [], which is true under §5 as written; the companion's totality request is met by the uncatalogued list rather than by inventing a Tooling row.

tests/test_layer_conformance.py makes the totality claim mechanical rather than a promise to remember: the egress destinations are derived from deploy/networkpolicies.yaml, the registry from the pinned image digests, and a new contact appearing in the manifests without a row here fails the test. It also pins the declaration's own content — engine/evidence, no decision surface, approval_validity_query: forbidden, and an evidence bound that never claims occurrence. Make the §5 conformance check total. layer.yaml declares tooling_contacts: [], true under §5 as written — audit-core is an Engine and holds no key-cape or OpenBao client. Companion §4 asks that uncatalogued infrastructure be listed anyway, and that carve-out sunsets within two review intervals for a store another layer reads, which platform-pg is. List it and the State Hub now. Add a conformance script and test in the shape ops-warden offered estate-wide.

id: AUDIT-WP-0009-T11
status: progress
priority: high
state_hub_task_id: "96e0cbd5-e3ce-5807-8f64-41244671327b"

Register informed-decision as a distinct load-bearing source under AUDIT-IN-0003, and answer the doctrine question GH-DEC-2026-014 put to audit-core: whether the record shape can carry a source-held-content declaration with a retrieval expectation.

informed-decision is the browser-facing approver surface. GH-DEC-2026-012 limit 3 makes its evidence copy the one that must reach audit-core independently of the emitter, because there the actor being audited and the evidence source are the same component. GH-DEC-2026-014 then granted commitment-only payloads for Stage 1 under a condition Gate House added: the record must assert that committed content exists and where custody sits, so non-production is a finding attributable to the custodian rather than an unremarkable blank.

Done 2026-09-10, prepared and inert: docs/informed-decision-source-registration.md, scope entries in deploy/senders-scope.{json,yaml}, and audit-core-informed-decision-ingress in deploy/networkpolicies.yaml (namespace AND pod label in one peer, following approval-engine rather than user-engine's older breadth). Every proposed field accepted as-is: exact source, ["tenant:platform"], write true, read false, load-bearing, secret_policy: redact.

The answer to Gate House's condition, in two halves that must travel together. The fabric can carry the declaration — data is stored verbatim into details.data and chained, so content_exists and custody need no schema change and become as tamper-evident as the commitment they accompany. It cannot detect non-production: audit-core performs no retrieval and its egress policy permits Postgres and DNS only, so it never learns whether a custodian would produce. Detection happens at retrieval, by the reviewer, and the stored declaration is what turns a blank into an attributable failure. Residual stated rather than left to be found: a custodian that never held the content can emit a false content_exists; audit-core validates the declaration's shape, never its truth. Not a requirement nobody can meet, and not the stronger thing it might be mistaken for.

Cadence answered: reconciliation-per-class plus heartbeat for the low-volume classes is the right shape for a mixed-volume source, with both scoped per class rather than per source — a per-source heartbeat is satisfied by the high-volume presentation stream and says nothing about a quiet month of dispositions. Bound stated: where the emitter itself is compromised, it suppresses the event and its own count together, so neither control covers the residual GH-DEC-2026-012 limit 3 exists for. Depends on T04/T06 for the detection half; declaring the cadence now is correct, describing it as operating is not.

Remaining: token custody, the protected registry entry, operator manifest application, and live ingestion evidence. No secret was created and no production manifest applied.

Factory sender deployment precondition — 2026-09-11

Native metadata still reports 1/1 Ready on image c2fe39a0185b.... The exact image and a synthetic SenderRegistry probe in the native pod confirm that it has no evidence_kind support. Current source supports the contract. The source-complete T03 implementation has not reached this deployment; readiness alone cannot admit a load-bearing sender. T09/T11 retain the compatible image release, current scope/peer manifests and actual ingestion/retrieval acceptance.

Platform RPF-WP-0035-T08 now prepares the separate sender custody records CCR-2026-0021 (approval-engine) and CCR-2026-0022 (informed-decision), with exact-path ESO projections and CAS-preserving first provision/resume. Its receiver check refuses the current image before credential access. Seventeen new tests, including eleven local OpenBao cases, and 53 existing credential-change tests pass. The user approved both named owner review sets on 2026-09-11. Compatible receiver release and target namespace readiness remain prerequisites. See railiance-platform/docs/credential-lane-designs/factory-audit-senders-review.md and its dated preparation evidence. Existing sender/source scopes are preserved.

Acceptance

  • No custody claim is returned unconditionally where docs/integrity.md conditions it, and a stale attestation visibly degrades the claim.
  • A source can declare its evidence kind, and a load-bearing source can discharge the §9.6 cadence MUST against a real audit-core surface.
  • A suppressed low-volume load-bearing class produces a finding rather than silence.
  • The approval-validity prohibition is asserted by test, not by absence.
  • layer.yaml lists every infrastructure contact, catalogued or not, and a test checks it.

Notes

T01 and T02 are separable and T01 is the one that matters: scheduling attestation without making the claim conditional would leave the overclaim in place whenever the job fails, which is precisely the case the claim is supposed to cover.

T05's dependency is real and should not be worked around. audit-core argued the cadence obligation up to a MUST; inventing a local cadence shape ahead of the Taxonomy schema would fragment the very interface it asked for.

Compatible receiver release rehearsal — 2026-09-11

229 receiver/contract tests pass with real disposable PostgreSQL. The full suite exposed five historical SQLite-import failures, now handed off as AUDIT-IN-0004; that importer is not used by the native receiver. No migration SQL changed. The new image uses the factory Alpine baseline and hashed runtime dependencies. Candidate scan went from 81 High/3 Critical to zero reported vulnerabilities, with the Alpine lifecycle-list warning retained. SIGTERM previously killed PID 1 with exit 137; explicit handling now exits 0, preserves one in-flight write and returns duplicate on restart/retry. Both synthetic senders pass exact scope, redaction, independent readback and seven read-route denials.

Rehearsal evidence. Publish and native rollout are the next steps; T09/T11 remain progress until real sender custody, delivery and owner acceptance are evidenced.