IR-WP-0006-T01/T02: billing reconciliation contract and settlement mechanism
Spec and implement external-evidence settlement for pending trial reservations, so the USD 0.023712 HTTP-401 reservation (and future ones) can be resolved without fabricating a completed trial or treating a failed request as zero-charge proof. intelligence_radar.trials.reconcile (+ CLI reconcile) is write-once under the existing trials lock, rejects unknown reservations, duplicate/conflicting settlement, cross-campaign references, invalid amounts and budget overrun, and records reviewer attribution plus an evidence pointer rather than any secret value. 9 new tests; full suite (48 tests) and repository check pass. T03 (applying this to the real reservation) stays blocked on authoritative provider billing evidence. ADR-0003 closes the IR-WP-0007-T03 uptime question: retain the local systemd timer, no hosted worker, since nothing here needs sub-day freshness and the scheduler already tolerates downtime. Mark IR-WP-0005/0006/0007 workplan status blocked: each still has at least one task genuinely blocked on an external party (secrets-engine human approval, authoritative billing evidence, and paid-execution outcomes respectively) rather than anything actionable in this repo. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: sonnet Assistant-Process: 8472@bnt-lap001 Assistant-Session: 1cac6b6d-c1a5-4a04-a950-c4febb8d3f67
This commit is contained in:
parent
7d03727945
commit
710b2ebbb5
8 changed files with 312 additions and 6 deletions
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Deliver admitted OpenRouter credentials to Radar"
|
||||
domain: infotech
|
||||
repo: intelligence-radar
|
||||
status: ready
|
||||
status: blocked
|
||||
flavor: implementation
|
||||
owner: codex
|
||||
topic_slug: custodian
|
||||
|
|
@ -47,3 +47,10 @@ state_hub_task_id: "c4b2378a-5241-5195-8c4b-d53ec58a7f72"
|
|||
```
|
||||
|
||||
Depends on T01 and native owner execution. Prove value-safe read-only provider key validation, correct recipient success, wrong caller/owner/tenant/path refusal, revocation and unchanged existing llm-connect delivery. Capture source/recipient revisions and sanitized native receipts. Do not widen policies or rotate the provider key merely to unblock Radar. This plan completes on admitted delivery evidence; paid inference and campaign accounting belong elsewhere.
|
||||
|
||||
Blocked, 2026-09-27: secrets-engine confirmed no ops-mason build is needed and withdrew
|
||||
the guest-clock hold (claim/consume uses a Railiance Clock trust file, not the OS clock).
|
||||
The remaining gate is human acceptance of three new apply/verify/exec approvals for the
|
||||
pinned `intelligence-radar-key-check` recipient; Radar requested secrets-engine open them
|
||||
(message `45b52d6e-7a3b-4ebe-ac15-d8137957731f`). Coordination trail:
|
||||
`docs/evidence/2026-09-27-ir-wp-0005-t01-native-delivery.md`.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Reconcile trial billing and establish campaign accounting"
|
||||
domain: infotech
|
||||
repo: intelligence-radar
|
||||
status: ready
|
||||
status: blocked
|
||||
flavor: implementation
|
||||
owner: codex
|
||||
topic_slug: custodian
|
||||
|
|
@ -24,24 +24,35 @@ retain authority over native changes. Review baseline:
|
|||
|
||||
```task
|
||||
id: IR-WP-0006-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "87b26b10-f4d6-5c53-9965-fae61ac0df11"
|
||||
```
|
||||
|
||||
Inventory the append-only events for ir-openrouter-pilot-20260913. Preserve the USD 0.023712 pending reservation and USD 10 total authorization. Define how provider billing evidence matches the failed request, what is unknown, who reviews a settlement, and how discrepancies are retained. HTTP 401 alone is not zero-charge proof. Accounting design can proceed independently of credential delivery; provider evidence retrieval may require an admitted read-only path.
|
||||
|
||||
Done: contract specified in `docs/evidence/2026-09-27-ir-wp-0006-t01-reconciliation-contract.md` —
|
||||
inventories the one pending reservation, defines valid billing evidence, the run_id-based match,
|
||||
the five refusal conditions, and what a settlement record must (and must not) contain.
|
||||
|
||||
## Implement and verify auditable reconciliation
|
||||
|
||||
```task
|
||||
id: IR-WP-0006-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "1637f2ee-2369-5e78-8893-9437d2d6b6fc"
|
||||
```
|
||||
|
||||
Provide a reviewed append-only settlement procedure or command. Preserve original reservations; reject duplicate/conflicting settlement, unknown reservation, invalid amount, cross-campaign references and budget overrun. Retain source billing evidence and reviewer attribution without credentials. Test uncertain/partial evidence and restart behavior. Do not fabricate a completed model trial to settle a failed request.
|
||||
|
||||
Done: `intelligence_radar.trials.reconcile` (+ CLI `reconcile`) implements the T01 contract —
|
||||
write-once settlement under the existing trials file lock, all five refusal conditions covered,
|
||||
no `trials` record fabricated. Tests in `tests/test_trials.py`
|
||||
(`test_reconcile_*`, 9 new cases) cover unknown reservation, duplicate/retry-after-success,
|
||||
cross-campaign, invalid amount, budget overrun, and missing evidence/reviewer attribution.
|
||||
Full suite (`make check`, 48 tests) passes.
|
||||
|
||||
## Reconcile the outstanding reservation and hand off budget readiness
|
||||
|
||||
```task
|
||||
|
|
@ -52,3 +63,8 @@ state_hub_task_id: "441dc053-af7b-5a9c-866f-5b1eb804b94f"
|
|||
```
|
||||
|
||||
Depends on T01/T02 and authoritative provider evidence. Resolve the USD 0.023712 reservation only to the supported actual amount, retaining any unresolved discrepancy explicitly. Rerender pilot/full plans for the same campaign and report spent, pending and remaining authorized budget. No campaign reset, additional authorization, or paid inference is implied. Completion requires reconciled evidence and a usable accounting handoff, not merely implementation tests.
|
||||
|
||||
Blocked: the mechanism is ready (T01/T02 done), but no authoritative provider billing evidence
|
||||
exists for the pending run_id `29a46a1d6c2d9ef6cfce8de6efef75cddcfecee346c1b4babf57119f1a241d71`
|
||||
yet. Applying `reconcile` here without that evidence would be exactly the fabrication this
|
||||
workplan forbids. Waits on external evidence (likely via the IR-WP-0005 admitted read-only path).
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Complete comparable evidence, field outcomes and publication"
|
||||
domain: infotech
|
||||
repo: intelligence-radar
|
||||
status: ready
|
||||
status: blocked
|
||||
flavor: implementation
|
||||
owner: codex
|
||||
topic_slug: custodian
|
||||
|
|
@ -31,6 +31,14 @@ state_hub_task_id: "8d4a62c6-1f0e-5866-8d62-638c300a5b2e"
|
|||
|
||||
Residual IR-WP-0004-T01. Pin a concrete task and five relevant benchmark versions, harnesses/tools, effort, attempts and source captures for at least two complete profiles. Resolve the vendor-instrument versus advisor-selection mismatch. Obtain requester access evidence from IR-WP-0005 before declaring eligibility. Produce the source-backed Frontier/Efficient comparison only when completeness and comparability checks pass. Preserve unknowns and conflicting claims; provider catalog metadata alone is insufficient.
|
||||
|
||||
Blocked, 2026-09-27: requester access evidence still does not exist — IR-WP-0005-T02
|
||||
(scoped delivery verification) is the explicit prerequisite this task names, and it is
|
||||
itself blocked on a pending secrets-engine approval cycle. Pinning concrete benchmark
|
||||
versions/harnesses independent of eligibility was already attempted in the IR-WP-0004
|
||||
review (`docs/evidence/2026-09-14-ir-wp-0004-review.md`) and found to require a
|
||||
dedicated, sourced measurement campaign rather than a redo inside this session —
|
||||
inventing pinned versions without primary-source verification would violate P11.
|
||||
|
||||
## Collect and assess representative field outcomes
|
||||
|
||||
```task
|
||||
|
|
@ -42,6 +50,12 @@ state_hub_task_id: "d78ac4b1-da1b-52a8-8626-cf5c6a499b57"
|
|||
|
||||
Residual IR-WP-0004-T02. Paid execution depends on BOTH IR-WP-0005 credential readiness and IR-WP-0006 budget readiness; sampling-frame/verifier design can proceed earlier. Use the existing USD 10 campaign and reviewed pilot inputs. Define real task origins, inclusion/exclusion, held-out tasks, dependency groups, verifiers and failure taxonomy before claiming representativeness. The 20 close controlled variants are pilot evidence only. Retain successes/failures, actual costs/runtime and verifier artifacts; justify cross-task calibration and intervals before using them predictively.
|
||||
|
||||
Blocked, 2026-09-27: paid execution still needs IR-WP-0005-T02 (wait) and IR-WP-0006-T03
|
||||
(wait) both closed. The sampling-frame/verifier design portion this task allows to proceed
|
||||
earlier is nontrivial methodology work in its own right (real task origins, held-out sets,
|
||||
verifiers, failure taxonomy) and was not attempted in this session to avoid a rushed
|
||||
design under time pressure; it remains open, undecomposed, for a dedicated pass.
|
||||
|
||||
## Complete reviewed publication and operational disposition
|
||||
|
||||
```task
|
||||
|
|
@ -53,6 +67,12 @@ state_hub_task_id: "314f79fd-d95a-5e02-b67d-67e35e4334ed"
|
|||
|
||||
Residual IR-WP-0004-T03. Build on portable bundles and the public catalog adapter. Define reviewed publication artifacts linking claims to captures, revisions and corrections. Keep generated drafts distinct from reviewed publication/notifications. Decide whether continuous uptime is required; retain the local timer unless a concrete requirement justifies a hosted worker and its live owner workplan.
|
||||
|
||||
Partial, 2026-09-27: the uptime question is decided — `docs/adr/0003-operating-model-disposition.md`
|
||||
retains the local systemd timer, no hosted worker. The reviewed-publication-artifact
|
||||
definition (claim-to-capture references, exclusions, revision identity, correction linkage,
|
||||
distinct from a generated draft) is still open; it is a schema/format design decision
|
||||
better made deliberately than folded into this pass. Stays todo.
|
||||
|
||||
## Compare decomposition with measured monolithic execution
|
||||
|
||||
```task
|
||||
|
|
@ -63,3 +83,5 @@ state_hub_task_id: "e36910d6-b7c5-5dad-bc1f-1af9f472ffab"
|
|||
```
|
||||
|
||||
Depends on T02 matched held-out outcome evidence. Count orchestration, retries and verification in total cost/runtime, control task/model selection differences and retain uncertainty. No synthetic superiority or unsupported empirical quantiles. Carry any remaining research or infrastructure scope into live records before closure.
|
||||
|
||||
Blocked, 2026-09-27: no matched held-out outcome evidence exists yet (T02 unresolved).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue