T01: cost-accounting survey — and a 2.7x correction to our own numbers
Tier L (structural L, chaos d10=2). Every candidate exercised locally
against the CB-WP-0001 transcript rather than cited.
Verdict: session transcript JSONL is the only exact instrument; git
commits supply the attribution index it lacks; the State Hub is a
durable sink whose schema cannot express cache tokens at all; the
status bar and billing API are eliminated on availability.
Three errors found in this workplan's own Purpose section, all
overstating cost:
- one API response is written as up to six JSONL lines, each
repeating the same usage object (657 lines, 346 responses)
- a multi-model session was priced entirely at Fable 5 rates,
violating a per-model rule MetricsAndScenarios §1a already had
- the transcript is a live file, so an unpinned total is not
repeatable (346 -> 356 responses between two runs minutes apart)
CB-WP-0001 pinned at fc76445: $92.21, not $248.46. The qualitative
finding sharpens rather than weakens — 88.4% of spend is cache, at
256:1 cache-read to output tokens.
Neither error was of the harness-does-nothing class the positive
control was written for; both sums ran over real data. What caught
them was re-deriving a number instead of quoting it. Raised for T07
and for CB-WP-0003 T01.
Also measured: the one subagent (adversarial review) cost $0.66 and is
invisible to a collector reading only the main transcript.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
fc76445aec
commit
060eb8cf6a
4 changed files with 385 additions and 12 deletions
21
.claude/ralph-loop.local.md
Normal file
21
.claude/ralph-loop.local.md
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
---
|
||||
active: true
|
||||
iteration: 1
|
||||
session_id: 8cbd5701-a096-45a4-a419-9b7b1c9419bc
|
||||
max_iterations: 20
|
||||
completion_promise: "HEUREKA"
|
||||
workplan_id: CB-WP-0002
|
||||
workplan_file: workplans/CB-WP-0002-cost-accounting.md
|
||||
started_at: "2026-07-31T02:22:44Z"
|
||||
---
|
||||
|
||||
Read the workplan at `workplans/CB-WP-0002-cost-accounting.md`.
|
||||
|
||||
If every task has `status: done` AND frontmatter `status: done`:
|
||||
run `rm -f .claude/ralph-loop.local.md` first (deactivates the loop so the stop hook exits cleanly),
|
||||
then output <promise>HEUREKA</promise>.
|
||||
|
||||
Otherwise implement the next `todo` task as described in the workplan.
|
||||
Set task `in_progress` when starting, `done` when complete.
|
||||
When all tasks are done set frontmatter `status: done`.
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue