informed-decision/docs/durable-review-evidence.md
tegwick 83849b75d4 Connect policy-gated browser review and audit runtime
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc
2026-09-11 00:31:03 +02:00

149 lines
8.6 KiB
Markdown

# Durable review evidence and audit delivery
Implemented under `INFD-WP-0001-T08`. `store.py` replaces the domain tests'
in-memory outbox with a persistent internal store. `review.py` and `web.py` now
connect it to the protected browser flow; the store itself never decides
entitlement. See [protected-browser-review.md](protected-browser-review.md)
for configuration, policy observations, schema v2 migration and readiness.
## Storage and recovery
The store uses a private SQLite file (0600), inside an existing directory owned
by the process user (0700). Symlinks, hard-linked databases, unsafe permissions,
unrecognized databases and unsupported schema versions are refused. WAL,
foreign keys and FULL synchronous commits are enabled. Connections are scoped
to operations; concurrent writers serialize through SQLite transactions.
This is a local Unix filesystem design, not shared network storage.
The existing domain objects are serialized to JSON, without pickle or a second
Decision Memo schema. Memo versions, packet documents, original presentations,
acknowledgments, dispositions and evidence rows are immutable. SQL triggers
reject ordinary updates/deletes; they do not protect against a database
administrator or a compromised process with file access.
Packet bytes must exist in the store before a memo can reference their SHA-256
hashes. Presentation content includes the memo and the exact binding/awareness
documents. A presentation is produced only through `presentation.render`, the
existing writer of `view_hash`. Packet contents are checked against their
hashes on custody retrieval. A retrieval failure names Informed Decision as
the custodian that cannot produce the material; it never returns a blank as
though no evidence had been promised.
Every presentation, explicit acknowledgment, disposition and submission result
commits its evidence and outbox row in the same local transaction. An outbox
write failure rolls the state change back. Acknowledgments append separate
records; they never overwrite the original presentation. A disposition retains
the acknowledgment snapshot that existed at that act, so later acknowledgment
does not retroactively strengthen its evidence.
Actor and presentation recipient must match. Binding also requires verified
human provenance, the current memo version, all required acknowledgments and
the existing step/seal guards. An accept intent requires the native approval id
and carried digest. Return, discussion and decline remain distinct local acts;
only accept creates an external-submission record. A dependency outage is a
stance application, never an invented human decline.
`Store.backup(new_path)` uses SQLite's backup API; it refuses an existing
destination. Restoring the resulting private database preserves evidence,
outbox ids and submission state. Tests cover process death before commit,
reopening after commit and restoring pending/confirmed records. An admitted
production backup location, schedule and restoration drill are still owed.
There is no automatic deletion/retention job in this implementation.
## Approval submission and uncertainty
These are internal delivery records, not cached approval validity:
1. Record the human's accept intent and its evidence before an engine POST.
2. The controller must obtain a fresh applicable access-engine decision, verify
the session and compare the memo's carried digest with the live approval.
`review.py` supplies these checks and persists the exact observation before use.
3. `begin_submission` atomically reserves one attempt. A stale presentation
cannot start; a memo cannot be revised while its entry attempt is in flight
or unresolved. Network I/O happens outside the SQLite transaction.
4. A successful non-duplicate entry response is correlated to the original
disposition using `(approval_id, subject, approved_at)`, atomically with an
evidence record. This confirms delivery, not permission to execute the act.
Operation ids make the same click idempotent; reuse with different content is
refused. A second presentation cannot acquire another intent for the same
approval and approver. A known confirmed correlation retrieves the original
presentation and acknowledgment snapshot.
**An unknown duplicate or lost response is unresolved.** It may refer to an
entry created before this presentation. The engine discards the POST body and
does not store a caller's presentation/operation id, so finding an entry later
does not prove which presentation caused it. The store does not attach it to
the latest view or automatically POST again. A crash after reservation likewise
does not reopen dispatch. The records remain visible for explicit controller/
operator recovery; no recovery path in this slice fabricates that missing link.
The controller and UI now expose this state and remove the submission button;
native operator recovery admission remains in T08.
## Audit Core contract
`audit.py` sends the envelope stored at event creation, with stable event id,
`Idempotency-Key`, source `informed-decision` and tenant `tenant:platform`.
The sender credential is supplied by an owner-provided callback, remains out of
the database and is never sent to a redirected endpoint. No native credential
is provisioned or read by this implementation task.
The receiver must return exactly `202 accepted` or `200 duplicate` with
`reference=audit:<event-id>` before local delivery is marked complete. Lost
responses replay the exact bytes; Audit Core deduplicates them. A 30-second
local lease bounds concurrent claims; stale workers cannot overwrite a newer
delivery result. Retryable failures back off to a five-minute maximum.
Malformed events, conflicts and credential refusals remain visibly blocked;
an explicit requeue after repair preserves the original event id and body.
No failure drops the evidence or stores the upstream response body as an error.
Only commitments and metadata travel: no brief, packet content or discussion
note. Native sender custody, receiver registration/application and independent
operator access still need live proof. Component tests use Audit Core's actual
ingestion/read APIs with a distinct SQLite fixture and synthetic scoped sender
and auditor identities. That is contract proof, not production custody proof.
Per-class heartbeat generation uses the existing 86,400-second declarations.
It does not emit “nothing to report” for a class with undelivered evidence.
The configured `AuditPump` schedules 30-second ticks, bounded draining and
five-minute reconciliation. The deployment owner must admit its private
storage, credentials and monitoring; no running service was installed here.
## Reconciliation has two time bases
Audit Core's current `/v1/reconciliation` counts rows by **accepted_at** and
returns `[{"class": ..., "count": ...}]`. The source counts events by
**occurred_at**. A delayed delivery can fall into different windows even when
nothing is missing. The client verifies source, tenant, explicit time window and
count shape, and reports both time bases with `count_values_match`.
It deliberately sets `automatic_loss_finding=false`, `completeness_proven=false`
and `reconstructability_proven=false`. Compare stable catch-up totals and inspect
pending/blocked events and receiver references before interpreting divergence.
A production reconciliation procedure that accounts for delayed acceptance is
still part of T08 admission. Equal counts cannot establish equal membership,
complete emission, truthful presentation or availability of source-held content.
The GH-DEC-2026-014 commitment-only limitation remains.
## Verification and next integration
```sh
INFD_APPROVAL_ENGINE_SOURCE=/home/worsch/approval-engine \
INFD_AUDIT_CORE_SOURCE=/home/worsch/audit-core \
INFD_FLEX_AUTH_BINARY=/absolute/path/to/flex-auth \
uv run python -m pytest -q
```
The prior 258-test milestone added 52 tests for transactional rollback, process death,
restore, actor/version/ack guards, concurrent clicks, original correlation,
ambiguous submissions, audit retry/refusal and actual receiver contracts.
The opt-in suites use the actual Approval Engine and Audit Core implementations
with synthetic identity/custody. The current 344-test suite also exercises the
actual Flex Auth HTTP server; 12 Chromium checks cover the complete disposable
browser flow. No native credential or paid provider is used.
Remaining under T08: native PDP package/caller/assignment admission against the
[exact consumer contract](flex-auth-review-contract.md), registered human login,
deployed-engine/custody proof and operator recovery/backup admission. No Informed
Decision policy package or registration was found in the checked Flex Auth
examples, registry and docs at `88b3543`; do not substitute a local allow rule.