Implement approval engine production readiness

Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a05e2e-805b-7042-a750-71f473bceea2
This commit is contained in:
tegwick 2026-09-02 00:52:04 +02:00
parent ebce5abb27
commit 2bd2d19a98
30 changed files with 1679 additions and 53 deletions

View file

@ -0,0 +1,34 @@
# Caller authentication
Production accepts only RS256 JWTs verified against KeyCape JWKS with the exact
configured issuer and `approval-engine` audience. `exp`, `iat`, `sub`,
`principal_type`, `tenant`, `roles`, `scope`, and `assurance` are mandatory.
Missing or unverifiable credentials fail closed. The development static-token
mode is explicit, file-backed, and refused with `--production`.
The verified `tenant` must exactly match the service's configured store tenant;
cross-tenant reads and mutations are rejected before object lookup.
| Route | Required scope |
|---|---|
| create approval | `approval:create` |
| get approval or claim | `approval:read` |
| add approval entry | `approval:approve` |
| revoke | `approval:revoke` |
| supersede | `approval:supersede` |
| consume | `approval:consume` and service/agent principal |
| cadence, outbox, storage | `approval:observe` |
| explicit heartbeat | `approval:emit` |
Create additionally requires `binding.actor == sub`. Approval-entry subject,
assurance, and evidence reference are derived from the verified JWT, never the
request body. KeyCape owns client registration and scope grants; approval-engine
only verifies and enforces them. Requested registrations are:
- audience/resource server `approval-engine` with the scopes above;
- the secrets-engine PEP service client with `approval:read` and
`approval:consume`;
- separately governed lifecycle/operator clients with only their needed
mutation or observation scopes.
Client credentials belong in OpenBao/operator custody and must not be placed in
manifests, logs, State Hub, or this repository.

View file

@ -14,9 +14,8 @@ surface; this declaration is what it reads. Machine-readable copy:
### Heartbeat
A signed positive claim: *nothing to report*, together with per-class
transition counts since the previous heartbeat (or since process start on
the first). The claim can itself go missing, which is the point — silence
An authenticated positive claim: *nothing to report*, together with cumulative
per-class committed transition counts. The claim can itself go missing, which is the point — silence
becomes a missing positive rather than a quiet month.
| Field | Value |

View file

@ -41,9 +41,12 @@ emitted class. The validity window is already on the object.
| `event_id` | UUID | Stable. Drain retries reuse it. |
| `class` | enum above | |
| `approval_id` | UUID or null | Null only for heartbeat. |
| `payload` | object | The `audit-core.event.v1alpha1` record, ready to POST. |
| `payload` | object | The source record adapted to audit-core's HTTP envelope at drain time. |
| `created_at` | RFC 3339 UTC | |
| `drained_at` | RFC 3339 UTC or null | Set after a successful audit-core ack. |
| `attempts` | integer | Delivery attempts, including the successful attempt. |
| `last_attempt_at` | RFC 3339 UTC or null | Most recent delivery attempt. |
| `last_error` | class name or null | Bounded failure category; never exception text. |
## Payload (audit-core v1alpha1)
@ -76,9 +79,13 @@ approval-validity query; this payload does not invite one.
## Drain
1. Select undrained rows, oldest first.
2. POST each payload to `audit-core`.
3. On success, set `drained_at`.
4. On failure, leave the row; retry later. **Do not** roll back the object
2. Adapt it to audit-core's `{id,type,source,subject,tenant,correlation_id,
occurred_at,data}` envelope, using `event_id` as both `id` and the
`Idempotency-Key` header.
3. POST with the mounted sender credential, reread on every attempt.
4. On `202 accepted` or `200 duplicate`, set `drained_at`.
5. On failure, leave the row and record only a bounded failure class; retry
later. **Do not** roll back the object
mutation — it already committed with the row.
An `audit-core` outage therefore cannot block a revocation. This engine's

20
docs/pep-integration.md Normal file
View file

@ -0,0 +1,20 @@
# PEP integration sequence
`approval_engine.pep` implements the fail-closed ordering from
`GH-DEC-2026-003` without becoming a PDP:
1. fetch a fresh approval claim;
2. pass that claim to the consumer's authorization decision function;
3. require ALLOW, decision id, and the exact canonical request digest;
4. CAS-consume the approval;
5. only after confirmed consumption invoke the protected callback.
Claim or consume unavailability, invalid/consumed claims, DENY, digest mismatch,
and consume conflicts all prevent the callback. A same-digest retry receives
the engine's idempotent success. If the callback fails after consume, the
approval stays spent; there is no unconsume.
The module rereads the mounted bearer-token file on each HTTP request. Its unit
harness uses a dry-run callback and demonstrates the ordering, but live closure
requires the secrets-engine-owned handler to prove that no OpenBao request is
made in every failure case.

View file

@ -0,0 +1,31 @@
# Storage operations
Production uses one SQLite writer on persistent `ReadWriteOnce` storage.
Production serve disables automatic migration and refuses schema drift or an
in-memory database.
```bash
approval-engine migrate --db /data/approvals.sqlite
approval-engine verify --db /data/approvals.sqlite
approval-engine backup --db /data/approvals.sqlite --output /backup/approval.sqlite
```
Migration is repeatable and sets an explicit `PRAGMA user_version`. Backup uses
SQLite's online backup API, verifies `PRAGMA integrity_check`, writes mode 0600,
and refuses to overwrite a target.
Restore is a stopped-single-writer operation:
1. Stop the StatefulSet and confirm no approval-engine or migration process has
the PVC open.
2. Preserve the failed database and its `-wal`/`-shm` companions for analysis.
3. Copy the verified backup to a new database path with owner 10001 and mode
0600. Do not merge a backup with old WAL files.
4. Run `verify`, then `migrate` if the release schema is newer, then `verify`
again.
5. Start exactly one replica and prove approval/entry/outbox counts, readiness,
claim retrieval, and idempotent audit drain before reopening callers.
Approval mutation and outbox insertion share `BEGIN IMMEDIATE` and one commit;
a failed outbox insert rolls the mutation back. Delivery occurs afterward and
does not roll back a committed mutation.