rein-aharness/docs/spend-admission.md
tegwick c30806b968
Some checks failed
Governed runtime contract / contract (push) Failing after 31s
Reserve provider requests inside the durable worker envelope
Assistant: codex
Assistant-Model: gpt-5.6-luna
Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc
2026-09-09 21:50:12 +02:00

8.7 KiB

Durable worker spend admission

The claim worker can reserve a declared maximum liability before invoking Glas. This is an opt-in control for a single host and one immutable operating envelope. It adds no service, provider calls or credentials. llm-connect remains the usage reporting owner; its reporting ledger and bundled FX snapshot are not admission inputs. Reuse discovery on 2026-09-09 found no reservation capability in the fresh hosted reuse-surface index (65 capabilities, 61 sources), or in the inspected llm-connect, Railiance Fabric and agentic-resources implementations. This is a bounded discovery result, not proof that every fleet capability is registered.

No factory policy is provisioned by this change. HFACT-WP-0001-T01 still owns provider enforcement/overrun proof, accepted conservative FX, final operating admission and the deployment return under REINAH-WP-0003-T05/T06.

Admission and accounting

The worker replays pending terminal closes first, then checks spend capacity before claiming. A configured worker refuses profile-absent work. Immediately before invoking Glas it checks the queue owner, ActivityDefinition, resolved repository, project, actor, repository grant, exact profile and descriptor digests, and the native USD/turn limits. The same cached catalog supplies the checked profile to Glas. Policy, ledger and project come from trusted worker configuration; queue execution references and prompt text cannot supply them.

An SQLite BEGIN IMMEDIATE transaction inserts the reservation before dispatch. The ledger opens in existing-file mode, with synchronous FULL transactions. A missing/corrupt ledger, changed policy, invalid FX/amount, expired policy, backward clock, exhausted daily/total capacity or unresolved reservation refuses execution. Concurrent processes sharing the file cannot admit two active runs. A run ID or ActivityDefinition/idempotency-key pair cannot be spent again, even on a new claim attempt, after reconciliation or under a replacement queue row ID.

The reservation is ceil(max_liability_usd * eur_per_usd * 1,000,000) integer micro-euros; envelope ceilings round down. Monetary arithmetic uses decimal values without reporting-FX defaults. max_budget_usd is the native CLI stop threshold and must fit inside the declared maximum liability. These two values are separate because admission must account for the provider's demonstrated maximum exposure, including any bounded in-flight overshoot.

On complete success with finite accounting and confirmed session cleanup and sandbox destruction, the ledger charges the full reserved amount. It never refunds capacity based on the sandbox's self-reported cost. Missing accounting, execution failure, cancellation, lost lease or crash leaves the reservation held and blocks further admission, including after midnight or restart. An observed cost above declared liability is recorded conservatively and permanently marks the envelope breached. Reconciliation does not clear that breach.

A completed or reconciled reservation counts in total once and, conservatively, against every local calendar day from admission through completion/reconciliation. A run crossing midnight therefore uses its full reservation in both daily totals. Unresolved work blocks new admission globally; no timer silently releases it. This intentionally sacrifices some utilization for the first bounded pilot. More exact billing reconciliation can follow measured use without weakening the unknown-outcome boundary.

Operator configuration and recovery

Use a private directory on durable local storage outside every sandbox mount and target checkout. The directory must be worker-owned mode 0700; policy and ledger must be worker-owned regular files mode 0600, without hardlinks or symlinks. All admitted processes must use this one ledger. Separate copies, network-file locking, multi-host admission, rollback to stale backups and hostile host owners are outside v1. The protected deployment must keep both files and the launch configuration inaccessible to the workload and preserve them across restarts.

Configure the following only after the operating envelope is accepted:

AGENT_HARNESS_REQUIRE_SPEND_ADMISSION=1
AGENT_HARNESS_SPEND_POLICY=/absolute/private/spend-policy.json
AGENT_HARNESS_SPEND_LEDGER=/absolute/private/spend.sqlite3
AGENT_HARNESS_EXECUTION_PROJECT=<accepted-project>

Any nonempty REQUIRE setting requires admission; missing policy/ledger then refuses before claim. With all spend settings absent, the existing worker compatibility behavior remains. The factory deployment must set REQUIRE and verify missing-configuration denial as part of its owner acceptance.

The strict JSON SpendPolicy fields are:

Fields Contract
version, envelope_id, authority_ref Version 1, unique envelope identity, reference to the accepted owner record. Local configuration is trusted; this module does not verify signatures or grant authority.
valid_from, expires_at, timezone Explicit timezone-aware validity interval and budget calendar, for example Europe/Berlin.
worker_id, activity_definition_id, target_repo, project Exact admitted scope; target is an absolute path. Actor is agt.
profile_ref, profile_sha256, descriptor_sha256, repository_grant_id Versioned profile plus SHA-256 of canonical model_dump(mode="json") for the resolved profile and descriptor; spend_admission.digest supplies the canonical serializer. Grant identity is the existing RepositoryGrant.grant_id.
max_budget_usd, max_liability_usd, eur_per_usd, per_run_eur, daily_eur, total_eur Positive decimal strings. Explicit liability/FX must fit per-run, daily and total ceilings. No implicit exchange rate or provider price.
max_turns Positive integer matching the resolved native profile.

Provision the ledger once using rein-aharness spend init --policy <path> --ledger <path>. Initialization refuses an existing file. Worker execution never creates an empty replacement. Keep the accepted policy immutable; replacing its contents under the same ledger refuses, rather than resetting spent capacity.

Inspect with rein-aharness spend status --policy <path> --ledger <path>. If a reservation is held, the operator must first confirm provider execution has stopped and obtain final accounting. Record the evidence in the owning work record, then run:

rein-aharness spend reconcile --policy <path> --ledger <path> \
  --run-id <id> --cost-usd <final-usd> --receipt <bounded-evidence-reference> \
  --provider-stopped

The flag is an operator attestation, not an automatic provider query. The local receipt must refer to reviewed termination and final accounting; it must contain no secrets. Reconciliation charges at least the full reservation and any higher observed liability, cannot discard a known higher cost, is idempotent for the same receipt/cost, and never authorizes rerunning that demand. There is no refund, reset, policy migration or breach-clear command. A breached envelope requires a new owner decision that accounts for existing spend before any further admission.

Proof and remaining boundary

tests/test_spend_admission.py covers pre-dispatch denial, concurrent processes, crash/reopen, identity/profile/grant changes, replay, unknown and overrun outcomes, midnight, total exhaustion, decimal rounding and operator reconciliation. It is included in both runtime-contract and recovery gates.

The opt-in REIN_REAL_BWRAP=1 test also runs actual bwrap, sandbox owner transport, Glas, repository import and durable-close replay with admission enabled. Queue responses, accounting and authoring are deterministic fixtures; no model is called. The fixture's policy and FX have no production authority.

This ledger enforces allocation against declared maximum liability. It cannot make an opaque provider/tool loop honor that maximum, establish a hard EUR limit, or verify the operator's FX assumption. G0 remains blocked until those semantics are proven for the pinned provider/CLI, including retries, cache, subagents and in-flight work. Exact credential/identity/egress/placement and protected-runtime installation are also still required before natural factory execution.

The native CLI proof now demonstrates the missing pre-request bound: one fixture response overshot a USD 0.01 threshold to an estimated USD 0.18015. The request extension now provides child holds and the llm-connect transport proves reserve-before-forward with the installed CLI. HFACT T01 retains owner hosting, actual lease/token integration, accepted tariffs/FX and direct-provider bypass denial. Parent observation also revokes request routes and retains unresolved children; operator reconciliation keeps full liability while closing the route.