approval-engine/workplans/APPROVAL-WP-0002-production-readiness-and-consumer-adoption.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	reviewed_at	reviewed_against_commit	reviewed_note	origin	origin_ref	state_hub_workstream_id
APPROVAL-WP-0002	workplan	Production readiness and consumer adoption	infotech	approval-engine	active	codex	netkingdom	2026-09-01	2026-09-06	2026-09-01	ebce5abb27	Reviewed against approval-engine's finished spine, KeyCape's RS256/JWKS and service-token contract, access-engine caller-auth/binding surface, audit-core's deployed authenticated ingestion plus open AUDIT-WP-0009 approval-source/cadence/reconciliation tasks, and secrets-engine's waiting exact-action/decision-consumption tasks. Repo-owned implementation can proceed; live T03-T05 closure remains evidence-gated on those owners.	residual	APPROVAL-WP-0001	4fa25ad5-f5d0-5592-aa59-085f8ee3edaf

APPROVAL-WP-0002 — Production readiness and consumer adoption

Move the completed first-cut engine spine into an authenticated, durable, observable production service and prove one PEP integration end to end. This is the residual production scope deliberately excluded from APPROVAL-WP-0001.

The workplan is proposed pending review against the deployment estate and the current key-cape, access-engine, audit-core, and secrets-engine contracts.

Review completed 2026-09-01. The plan is active. Production mode will verify KeyCape JWT signatures and exact issuer/audience/scopes; local caller-supplied identity never becomes authenticated evidence. SQLite remains the first production store only as a single-replica StatefulSet with explicit migration, backup, integrity, and restore gates. audit-core delivery can be implemented against its existing authenticated idempotent ingest, while heartbeat findings, count reconciliation, and sender registration remain external gates in AUDIT-WP-0009 T04/T06/T09. The first live PEP proof remains jointly gated on secrets-engine T04/T02 and deployment of this service.

Authenticate lifecycle mutations and approver evidence

id: APPROVAL-WP-0002-T01
status: progress
priority: high
state_hub_task_id: "dc4523f5-e0af-5734-a0ef-07aa7f2b2a27"

Bind create, approval-entry, revoke, supersede, and consume callers to authenticated identities. An API-supplied subject_id, actor, or decision_id is provenance only until independently authenticated. Keep authorization decisions in access-engine and approval doctrine in gate-house.

Acceptance: production startup requires a signature-valid KeyCape JWT verifier; every non-health API route requires an explicit scope; approval entry identity and assurance come only from verified claims; create binds binding.actor to the authenticated subject; consume is restricted to service/agent principals; missing, expired, wrong-issuer, wrong-audience, wrong-scope, or unverifiable tokens fail closed without mutation.

Repository implementation complete 2026-09-02: RS256/JWKS verification, issuer/audience/time/profile validation, exact scopes, store-tenant isolation, verified approver evidence, and a deny-all default are covered by tests.

2026-09-02 follow-up: fail-closed coverage now includes wrong signature, HS256, empty/invalid profile claims, deny-all mutation refusal, human-principal consume rejection, and production CLI refusal of static tokens / missing audit delivery. Requested KeyCape registrations are in docs/keycape-service-registrations.md (aud MUST be the resource server approval-engine, not the OAuth client id). Remains progress until KeyCape owns and proves those audience/client/scope registrations.

Harden durable storage and migrations

id: APPROVAL-WP-0002-T02
status: done
priority: high
state_hub_task_id: "2bef94ca-9482-5eff-9bd9-39adef292a78"

Define the production persistence, backup/restore, migration, concurrency, and recovery posture. Prove schema upgrades preserve existing approvals and that crash recovery cannot separate mutations from outbox evidence.

Acceptance: schema version is explicit; production serve refuses an unmigrated or in-memory store; migration is a separate repeatable command; backup uses SQLite's online backup API and integrity verification; restore is documented as a stopped-single-writer operation; tests cover legacy upgrade, backup/restore, and outbox atomicity after restart.

Completed 2026-09-02: schema v2 migration, production no-auto-migrate gate, integrity/status surface, online mode-0600 backup, stopped-writer restore runbook, retry-attempt state, and migration/backup/atomicity tests are in place.

Package and deploy the service

id: APPROVAL-WP-0002-T03
status: wait
priority: high
state_hub_task_id: "f0aa2e6d-19e6-5b43-886c-efa4e3de5f22"

Add the governed image/deployment surface, health and readiness behavior, resource bounds, and fail-closed caller configuration. A local WSGI development server is not production evidence.

Acceptance: a digest-pin-ready image and single-writer StatefulSet manifest exist with non-root/read-only-root controls, PVC, migration init container, resource bounds, probes, and default-deny network policy. Live completion also requires an immutable image digest, KeyCape registrations, audit sender credential, successful rollout, and restart/restore evidence.

Repository implementation complete 2026-09-02: the digest-pin-ready non-root image builds and runs; the single-writer StatefulSet, PVC, migration init, read-only root, resources, probes, and default-deny policies pass client dry-run. Waiting on release digest, KeyCape/audit registrations and credentials, rollout, restart, and restore evidence.

Wire outbox delivery and reconciliation

id: APPROVAL-WP-0002-T04
status: wait
priority: high
state_hub_task_id: "777a5cf5-c1db-5e07-8de6-ab109db5ccdc"

Deliver the local outbox asynchronously to audit-core, preserve event-id deduplication, publish lag/depth signals, emit the declared heartbeat, and prove the Gate House reconciliation contract against accepted event counts.

Acceptance: the sender adapts the local event to audit-core's authenticated HTTP ingest, reuses event id as idempotency key, rereads a mounted token file, marks drained only on accepted/duplicate, retains retryable failures, exposes attempt/lag state, and emits the declared heartbeat. Live reconciliation waits on AUDIT-WP-0009-T04/T06/T09; do not invent that receiver surface here.

Repository implementation complete 2026-09-02: audit-core envelope adaptation, event-id idempotency, mounted-token reread, accepted/duplicate handling, retry-attempt/lag metrics, and periodic heartbeats are tested.

2026-09-02 follow-up: drain tests now cover HTTP 200 duplicate as drained and urllib HTTPError 503 as pending. Waiting on the audit-core sender registration/ingress and receiver-owned reconciliation work (AUDIT-WP-0009-T04/T06/T09).

Prove one live PEP consumption path

id: APPROVAL-WP-0002-T05
status: wait
priority: high
state_hub_task_id: "3196fb77-3df0-5358-ae12-2d07bac12041"

Integrate one protected-system consumer under GH-DEC-2026-003: claim before decision, CAS consume after ALLOW and before side effect, same-digest retry, different-digest conflict, spent-on-failure behavior, and no protected action when approval-engine is unavailable.

Acceptance: a repeatable harness proves the PEP sequence against the real HTTP surface without performing a protected action; live closure requires a secrets-engine-owned handler and evidence that no OpenBao call occurs on every failure case. This repo may supply the protocol client and fixture, but may not claim the consumer's side effect.

Repository implementation complete 2026-09-02: the HTTP PEP client and fail-closed sequencing harness prove claim-before-decision and CAS-consume-before callback, including unavailable, DENY, digest mismatch, and conflict paths.

2026-09-02 follow-up: secrets-engine shipped the PEP consume-before-OpenBao handler (src/secrets_engine/approval_consume.py; inbox 0e04b4e2). This engine's client now maps HTTP 409/401/403/404/503 and unreachability, requires the canonical digest, and refuses a decision-shaped consume payload. The repeatable harness in tests/test_pep.py drives the real HTTP surface: first consume then same-digest retry, different-digest conflict, spent-on-failure claim refusal, and unreachable-engine callback suppression. Waiting on live closure: this service deployed (T03) and a durable consume binding served (SECRETS-WP-0007-T04 / SECRETS-WP-0008-T02). This repo does not claim the OpenBao side effect.

2026-09-06 follow-up: secrets-engine reports its PIP join is implemented and blocked on deployment, not contract (inbox 61ae1174). Review of its validate_action_authorization shows a real envelope divergence: it expects a state-hub-authority ActionAuthorization (id, status, superseded_by, request, approvals.entries, policy pin) while this engine serves the governed approval-claim (approval_id, state/valid_now, binding, freshness, reason_code, issuer: approval-engine). Both declare schema_version 0.1, so the mismatch surfaces as a field/authority error rather than a version error. Recorded in docs/approval-consumption.md; reconciling the envelopes is a GH-DEC-2026-003 cross-repo change, not a unilateral edit here. T05 stays wait: still no deployed base URL (T03).

2026-09-06 ruling: the claim-envelope question is settled. GH-DEC-2026-005 confirms all three requested dispositions — the approval-claim is the step-1 artifact, ActionAuthorization is not required and MUST NOT be served from the claim endpoint, and a PEP validates across the claim and the step-2 DecisionEnvelope. Gate House recorded the split as doctrine (a PIP must not republish the PDP's decision) and struck the provenance.authority == "state-hub" requirement explicitly. flex-auth accepted as FLEX-DEC-2026-006, argued against its own proposal, and traced the bad authority constant to a fixture rather than prose. This engine changes nothing: the claim schema stands as published. APPROVAL-IN-0002 is closed. T05 remains wait on T03 deployment plus the secrets-engine validator split and its secrets-engine-approval KeyCape registration (already requested verbatim in docs/keycape-service-registrations.md).

2026-09-06 follow-on (T01): GH-DEC-2026-005 assigned this engine a compensating obligation. secrets-engine has implemented the split and reports it no longer verifies the distinct-approver threshold independently; Gate House accepted that as correct on layering and as a real reduction in defence in depth, requiring instead that the threshold evaluation be reconstructable from this engine's state transitions and its use outbox row under §9.6. Implemented: approval.issuance and approval.use now carry a threshold object (required_count, distinct_approver_count, threshold_met, and approvers with approved_at plus assurance/evidence refs). Tests prove reconstruction from the use row alone and that the claim still discloses no approver identities. Documented in docs/outbox-contract.md.

2026-09-06 follow-on (T05): flex-auth asked whether this engine should publish an action/target vocabulary mapping between the claim binding and their policy package before SECRETS-WP-0007-T04 makes destroy reachable. Answered no, and recorded why in docs/approval-claim.md: a PIP asserting that one vocabulary's action means another's would author policy semantics it does not own, and a wrong mapping silently accepts a claim approved for a different action. binding.pdp_digest is the correspondence — it compares the PDP's digest to the PDP's digest with no translation. Implemented the part that was ours: pdp_digest is now always present on the claim and null when the approval was not issued against a PDP decision, so absence is a stated fact rather than a missing key, and the schema requires it as nullable. A PEP on a privileged lane must refuse a null. Both published examples were contradicting the schema; fixed, and tests/test_examples.py now validates every example against it.