rein-aharness/docs/spend-admission.md
tegwick c30806b968
Some checks failed
Governed runtime contract / contract (push) Failing after 31s
Reserve provider requests inside the durable worker envelope
Assistant: codex
Assistant-Model: gpt-5.6-luna
Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc
2026-09-09 21:50:12 +02:00

142 lines
8.7 KiB
Markdown

# Durable worker spend admission
The claim worker can reserve a declared maximum liability before invoking Glas.
This is an opt-in control for a single host and one immutable operating envelope.
It adds no service, provider calls or credentials. llm-connect remains the usage
reporting owner; its reporting ledger and bundled FX snapshot are not admission
inputs. Reuse discovery on 2026-09-09 found no reservation capability in the fresh
hosted reuse-surface index (65 capabilities, 61 sources), or in the inspected
llm-connect, Railiance Fabric and agentic-resources implementations. This is a
bounded discovery result, not proof that every fleet capability is registered.
No factory policy is provisioned by this change. HFACT-WP-0001-T01 still owns
provider enforcement/overrun proof, accepted conservative FX, final operating
admission and the deployment return under REINAH-WP-0003-T05/T06.
## Admission and accounting
The worker replays pending terminal closes first, then checks spend capacity
before claiming. A configured worker refuses profile-absent work. Immediately
before invoking Glas it checks the queue owner, ActivityDefinition, resolved
repository, project, actor, repository grant, exact profile and descriptor digests,
and the native USD/turn limits. The same cached catalog supplies the checked
profile to Glas. Policy, ledger and project come from trusted worker configuration;
queue execution references and prompt text cannot supply them.
An SQLite `BEGIN IMMEDIATE` transaction inserts the reservation before dispatch.
The ledger opens in existing-file mode, with synchronous FULL transactions. A
missing/corrupt ledger, changed policy, invalid FX/amount, expired policy, backward
clock, exhausted daily/total capacity or unresolved reservation refuses execution.
Concurrent processes sharing the file cannot admit two active runs. A run ID or
ActivityDefinition/idempotency-key pair cannot be spent again, even on a new
claim attempt, after reconciliation or under a replacement queue row ID.
The reservation is `ceil(max_liability_usd * eur_per_usd * 1,000,000)` integer
micro-euros; envelope ceilings round down. Monetary arithmetic uses decimal
values without reporting-FX defaults. `max_budget_usd` is the native CLI stop
threshold and must fit inside the declared maximum liability. These two values
are separate because admission must account for the provider's demonstrated
maximum exposure, including any bounded in-flight overshoot.
On complete success with finite accounting and confirmed session cleanup and
sandbox destruction, the ledger charges the **full reserved amount**. It never
refunds capacity based on the sandbox's self-reported cost. Missing accounting,
execution failure, cancellation, lost lease or crash leaves the reservation held
and blocks further admission, including after midnight or restart. An observed
cost above declared liability is recorded conservatively and permanently marks
the envelope breached. Reconciliation does not clear that breach.
A completed or reconciled reservation counts in total once and, conservatively,
against every local calendar day from admission through completion/reconciliation.
A run crossing midnight therefore uses its full reservation in both daily totals.
Unresolved work blocks new admission globally; no timer silently releases it.
This intentionally sacrifices some utilization for the first bounded pilot.
More exact billing reconciliation can follow measured use without weakening the
unknown-outcome boundary.
## Operator configuration and recovery
Use a private directory on durable local storage outside every sandbox mount and
target checkout. The directory must be worker-owned mode 0700; policy and ledger
must be worker-owned regular files mode 0600, without hardlinks or symlinks.
All admitted processes must use this one ledger. Separate copies, network-file
locking, multi-host admission, rollback to stale backups and hostile host owners
are outside v1. The protected deployment must keep both files and the launch
configuration inaccessible to the workload and preserve them across restarts.
Configure the following only after the operating envelope is accepted:
```text
AGENT_HARNESS_REQUIRE_SPEND_ADMISSION=1
AGENT_HARNESS_SPEND_POLICY=/absolute/private/spend-policy.json
AGENT_HARNESS_SPEND_LEDGER=/absolute/private/spend.sqlite3
AGENT_HARNESS_EXECUTION_PROJECT=<accepted-project>
```
Any nonempty REQUIRE setting requires admission; missing policy/ledger then
refuses before claim. With all spend settings absent, the existing worker
compatibility behavior remains. The factory deployment must set REQUIRE and
verify missing-configuration denial as part of its owner acceptance.
The strict JSON `SpendPolicy` fields are:
| Fields | Contract |
| --- | --- |
| `version`, `envelope_id`, `authority_ref` | Version `1`, unique envelope identity, reference to the accepted owner record. Local configuration is trusted; this module does not verify signatures or grant authority. |
| `valid_from`, `expires_at`, `timezone` | Explicit timezone-aware validity interval and budget calendar, for example Europe/Berlin. |
| `worker_id`, `activity_definition_id`, `target_repo`, `project` | Exact admitted scope; target is an absolute path. Actor is `agt`. |
| `profile_ref`, `profile_sha256`, `descriptor_sha256`, `repository_grant_id` | Versioned profile plus SHA-256 of canonical `model_dump(mode="json")` for the resolved profile and descriptor; `spend_admission.digest` supplies the canonical serializer. Grant identity is the existing `RepositoryGrant.grant_id`. |
| `max_budget_usd`, `max_liability_usd`, `eur_per_usd`, `per_run_eur`, `daily_eur`, `total_eur` | Positive decimal **strings**. Explicit liability/FX must fit per-run, daily and total ceilings. No implicit exchange rate or provider price. |
| `max_turns` | Positive integer matching the resolved native profile. |
Provision the ledger once using `rein-aharness spend init --policy <path>
--ledger <path>`. Initialization refuses an existing file. Worker execution never
creates an empty replacement. Keep the accepted policy immutable; replacing its
contents under the same ledger refuses, rather than resetting spent capacity.
Inspect with `rein-aharness spend status --policy <path> --ledger <path>`.
If a reservation is held, the operator must first confirm provider execution has
stopped and obtain final accounting. Record the evidence in the owning work record,
then run:
```text
rein-aharness spend reconcile --policy <path> --ledger <path> \
--run-id <id> --cost-usd <final-usd> --receipt <bounded-evidence-reference> \
--provider-stopped
```
The flag is an operator attestation, not an automatic provider query. The local
receipt must refer to reviewed termination and final accounting; it must contain
no secrets. Reconciliation charges at least the full reservation and any higher
observed liability, cannot discard a known higher cost, is idempotent for the same
receipt/cost, and never authorizes rerunning that demand. There is no refund,
reset, policy migration or breach-clear command. A breached envelope requires a
new owner decision that accounts for existing spend before any further admission.
## Proof and remaining boundary
`tests/test_spend_admission.py` covers pre-dispatch denial, concurrent processes,
crash/reopen, identity/profile/grant changes, replay, unknown and overrun outcomes,
midnight, total exhaustion, decimal rounding and operator reconciliation. It is
included in both runtime-contract and recovery gates.
The opt-in `REIN_REAL_BWRAP=1` test also runs actual bwrap, sandbox owner transport,
Glas, repository import and durable-close replay with admission enabled. Queue
responses, accounting and authoring are deterministic fixtures; no model is
called. The fixture's policy and FX have no production authority.
This ledger enforces allocation against **declared** maximum liability. It cannot
make an opaque provider/tool loop honor that maximum, establish a hard EUR limit,
or verify the operator's FX assumption. G0 remains blocked until those semantics
are proven for the pinned provider/CLI, including retries, cache, subagents and
in-flight work. Exact credential/identity/egress/placement and protected-runtime
installation are also still required before natural factory execution.
The [native CLI proof](native-cli-boundary-proof.md) now demonstrates the missing
pre-request bound: one fixture response overshot a USD 0.01 threshold to an
estimated USD 0.18015. The [request extension](request-admission.md) now provides
child holds and the llm-connect transport proves reserve-before-forward with the
installed CLI. HFACT T01 retains owner hosting, actual lease/token integration,
accepted tariffs/FX and direct-provider bypass denial. Parent observation also
revokes request routes and retains unresolved children; operator reconciliation
keeps full liability while closing the route.