IR-WP-0006-T01/T02: billing reconciliation contract and settlement mechanism
Spec and implement external-evidence settlement for pending trial reservations, so the USD 0.023712 HTTP-401 reservation (and future ones) can be resolved without fabricating a completed trial or treating a failed request as zero-charge proof. intelligence_radar.trials.reconcile (+ CLI reconcile) is write-once under the existing trials lock, rejects unknown reservations, duplicate/conflicting settlement, cross-campaign references, invalid amounts and budget overrun, and records reviewer attribution plus an evidence pointer rather than any secret value. 9 new tests; full suite (48 tests) and repository check pass. T03 (applying this to the real reservation) stays blocked on authoritative provider billing evidence. ADR-0003 closes the IR-WP-0007-T03 uptime question: retain the local systemd timer, no hosted worker, since nothing here needs sub-day freshness and the scheduler already tolerates downtime. Mark IR-WP-0005/0006/0007 workplan status blocked: each still has at least one task genuinely blocked on an external party (secrets-engine human approval, authoritative billing evidence, and paid-execution outcomes respectively) rather than anything actionable in this repo. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: sonnet Assistant-Process: 8472@bnt-lap001 Assistant-Session: 1cac6b6d-c1a5-4a04-a950-c4febb8d3f67
This commit is contained in:
parent
7d03727945
commit
710b2ebbb5
8 changed files with 312 additions and 6 deletions
46
docs/adr/0003-operating-model-disposition.md
Normal file
46
docs/adr/0003-operating-model-disposition.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# ADR 0003 — Operating model disposition (local timer retained)
|
||||
|
||||
Accepted: 2026-09-27.
|
||||
|
||||
## Context
|
||||
|
||||
IR-WP-0007-T03 asks whether continuous uptime is required for Intelligence
|
||||
Radar's scheduled jobs, and to retain the local timer unless a concrete
|
||||
requirement justifies a hosted worker and its own live owner workplan.
|
||||
|
||||
The current operating model is a user-level systemd timer/service pair
|
||||
(`ops/systemd/intelligence-radar.timer`, `ops/systemd/intelligence-radar.service`)
|
||||
invoking `python3 -m intelligence_radar run-due`, which is explicitly
|
||||
idempotent and safe to invoke repeatedly (`ops/RUNBOOK.md`). `run-due` and the
|
||||
scheduler already handle missed/late runs and retries (`intelligence_radar/scheduler.py`,
|
||||
exercised in `tests/test_ingestion_scheduler.py`), so uptime gaps degrade
|
||||
freshness rather than correctness.
|
||||
|
||||
## Decision
|
||||
|
||||
Retain the local systemd user timer as the operating model. No hosted worker
|
||||
is adopted.
|
||||
|
||||
## Rationale
|
||||
|
||||
- Nothing in this repository's current scope requires sub-day freshness: the
|
||||
cadence is daily/weekly digests and periodic source captures, not real-time
|
||||
alerting.
|
||||
- The scheduler design already absorbs downtime (a laptop asleep, a missed
|
||||
boot) by running the latest due job on next wake rather than requiring
|
||||
uninterrupted uptime.
|
||||
- A hosted worker adds a live infrastructure surface (deployment, credentials,
|
||||
monitoring, its own on-call) with no offsetting capability gain today, and
|
||||
would need its own owner workplan to be justified, per this task's own
|
||||
instruction not to start "another infrastructure programme before the
|
||||
research evidence is usable."
|
||||
- The evidence-quality blockers ahead of this project (IR-WP-0005 credential
|
||||
delivery, IR-WP-0006 billing reconciliation) are unrelated to scheduler
|
||||
uptime; hosting the scheduler elsewhere would not unblock them.
|
||||
|
||||
## Revisit trigger
|
||||
|
||||
Reopen this decision only if a concrete requirement appears — e.g. a
|
||||
downstream consumer needs sub-daily freshness guarantees, or the operator
|
||||
machine's uptime becomes materially unreliable for the daily/weekly cadence.
|
||||
Until then this is not tracked as an open task.
|
||||
|
|
@ -0,0 +1,89 @@
|
|||
# IR-WP-0006-T01 — billing evidence and reconciliation contract
|
||||
|
||||
Reviewed 2026-09-27 by codex. Specifies how an unresolved trial reservation is
|
||||
settled from reviewed external billing evidence, without treating absence of
|
||||
a response as proof of zero cost.
|
||||
|
||||
## Inventory: campaign ir-openrouter-pilot-20260913
|
||||
|
||||
One append-only `trial-events` record exists for this campaign:
|
||||
|
||||
```json
|
||||
{"budget_usd":"10.00","campaign":"ir-openrouter-pilot-20260913","case_id":"provenance-1",
|
||||
"created_at":"2026-09-13T21:22:29.301211+00:00","kind":"reserved","model":"anthropic/claude-sonnet-5",
|
||||
"reserved_usd":"0.023712","run_id":"29a46a1d6c2d9ef6cfce8de6efef75cddcfecee346c1b4babf57119f1a241d71"}
|
||||
```
|
||||
|
||||
It is a `reserved` event with no matching `settled` event — the pending
|
||||
USD 0.023712 reservation referenced throughout IR-WP-0004/0005/0006. `budget()`
|
||||
in `intelligence_radar/trials.py` already reports this as `pending_usd`.
|
||||
There is no mechanism today to settle a reservation except `execute()`, which
|
||||
requires a live, successful, billed provider call — the wrong tool for a
|
||||
request that returned HTTP 401 and never authenticated. `HTTP 401 alone is not
|
||||
zero-charge proof`: no evidence in this repository establishes the provider's
|
||||
actual charge for that attempt, and none is invented here.
|
||||
|
||||
## What counts as billing evidence
|
||||
|
||||
Provider-issued evidence of the actual charge (or absence of charge) for a
|
||||
specific reservation's `run_id`: an OpenRouter activity/billing export,
|
||||
dashboard screenshot, or admin API response showing the request's cost. It
|
||||
must be retained as an artifact (content-addressed, like the rest of this
|
||||
repository's evidence) and referenced by a pointer — a SHA-256 or file path —
|
||||
never by embedding a credential or raw authenticated response that could carry
|
||||
a secret. Retrieving that evidence may itself require the admitted read-only
|
||||
path from IR-WP-0005; this contract does not require the credential, only a
|
||||
place to record what it produces.
|
||||
|
||||
## Matching a reservation to evidence
|
||||
|
||||
A settlement targets exactly one `reserved` event, identified by its
|
||||
`run_id` (content-addressed identity of `campaign + case + request payload`,
|
||||
already unique and tamper-evident). The settlement is refused unless:
|
||||
|
||||
- **the reservation exists** — the `run_id` matches a recorded `reserved`
|
||||
event (else: unknown reservation);
|
||||
- **the campaign matches** — the reservation's own `campaign` field equals the
|
||||
campaign the settlement is filed under (else: cross-campaign reference);
|
||||
- **no settlement already exists** for that reservation (else: duplicate or
|
||||
conflicting settlement — settlement is write-once, matching the store's
|
||||
append-only, content-addressed design);
|
||||
- **the amount is a valid, finite, nonnegative decimal** (else: invalid
|
||||
amount);
|
||||
- **recording it would not push the campaign's committed total (other
|
||||
settled + other pending + this settlement) past the reservation's own
|
||||
recorded `budget_usd` ceiling** (else: budget overrun — a discrepancy this
|
||||
large is escalated for review, not silently written).
|
||||
|
||||
## What a settlement records
|
||||
|
||||
A `trial-events` entry, `kind: settled`, exactly like a live execution
|
||||
produces — so `budget()` and `plan()` need no changes to recognize it — but
|
||||
with `settlement_method: external_evidence`, an `evidence_ref` pointer, the
|
||||
`reviewer` who attested the match, and the resulting `discrepancy_usd`
|
||||
(`actual - reserved`, which may be negative, zero, or positive and is always
|
||||
retained, never discarded). **No `trials` record is created.** Fabricating a
|
||||
completed model trial (a response, a verification result, a returned model)
|
||||
to close out a failed request is explicitly excluded — a settlement asserts
|
||||
only a cost, never a synthetic outcome.
|
||||
|
||||
## Reviewer and restart behavior
|
||||
|
||||
The reviewer attribution is a plain identifier (human or agent name), not a
|
||||
credential. Because the settlement check runs under the same file lock as
|
||||
`execute()` and refuses once a `settled` event exists for the reservation,
|
||||
re-running the same reconciliation after a partial failure (e.g. process
|
||||
killed after validation but before write, or after write) is safe: a retry
|
||||
either completes the original write or is refused as already-settled — never
|
||||
double-settled, never partially written (the store's `atomic`/`put` primitives
|
||||
guarantee whole-file, content-addressed writes). Uncertain or partial evidence
|
||||
(no reviewer, no evidence pointer, non-numeric or missing amount) is refused
|
||||
before any write, by the same validation used for a first attempt.
|
||||
|
||||
## Disposition
|
||||
|
||||
This contract and its implementation (`intelligence_radar/trials.reconcile`,
|
||||
CLI `reconcile`) are specified and tested in this change (IR-WP-0006-T02).
|
||||
Applying it to the real USD 0.023712 reservation (IR-WP-0006-T03) still
|
||||
requires authoritative provider billing evidence, which this repository does
|
||||
not have — that step stays blocked until such evidence exists.
|
||||
Loading…
Add table
Add a link
Reference in a new issue