coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

T01: cost-accounting survey — and a 2.7x correction to our own numbers

Browse source 
Tier L (structural L, chaos d10=2). Every candidate exercised locally
against the CB-WP-0001 transcript rather than cited.

Verdict: session transcript JSONL is the only exact instrument; git
commits supply the attribution index it lacks; the State Hub is a
durable sink whose schema cannot express cache tokens at all; the
status bar and billing API are eliminated on availability.

Three errors found in this workplan's own Purpose section, all
overstating cost:

  - one API response is written as up to six JSONL lines, each
    repeating the same usage object (657 lines, 346 responses)
  - a multi-model session was priced entirely at Fable 5 rates,
    violating a per-model rule MetricsAndScenarios §1a already had
  - the transcript is a live file, so an unpinned total is not
    repeatable (346 -> 356 responses between two runs minutes apart)

CB-WP-0001 pinned at fc76445: $92.21, not $248.46. The qualitative
finding sharpens rather than weakens — 88.4% of spend is cache, at
256:1 cache-read to output tokens.

Neither error was of the harness-does-nothing class the positive
control was written for; both sums ran over real data. What caught
them was re-deriving a number instead of quoting it. Raised for T07
and for CB-WP-0003 T01.

Also measured: the one subagent (adversarial review) cost $0.66 and is
invisible to a collector reading only the main transcript.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 04:27:52 +02:00
parent fc76445aec
commit 060eb8cf6a
4 changed files with 385 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

44
workplans/CB-WP-0002-cost-accounting.md
View file

@ -18,19 +18,39 @@ transcript (`~/.claude/projects/<slug>/<session>.jsonl`) carries exact
per-message `usage`, including the cache breakdown. Reading it for
CB-WP-0001's session gives:
| Component | Tokens | Cost (Fable 5) |
|---|---|---|
| Output | 585,528 | $29.28 |
| Cache read | 131,863,164 | **$131.86** |
| Cache write (1h) | 4,365,668 | $87.31 |
| Input | 1,090 | $0.01 |
| **Total** | | **$248.46** |
> **Corrected 2026-07-31 by T01.** This section originally reported
> **$248.46** from 131,863,164 cache-read tokens priced at Fable 5. That
> figure was wrong by ~2.7×, for two independent reasons found in
> `research/CB-RES-0002-cost-accounting.md` §Correction: one API response
> is written to the transcript as up to six JSONL lines that each repeat
> the *same* `usage` object (657 lines, 346 real responses), and the
> session ran three models, not one. The numbers below are the re-derived
> ones. The workplan's premise is unaffected; its arithmetic was not.
The headline finding from that single reading is the reason this
workplan exists: **53% of the cost is cache reads**, not output. Cost in
an agentic loop is driven by context size × turn count, not by how much
the model writes. No D2 decision made on "tokens per task" would have
surfaced that.
Deduplicated by `requestId` and priced per message at its own model's
rate. **Pinned** to messages at or before `2026-07-31T02:17:59Z` (commit
`fc76445`, the end of CB-WP-0001) — the transcript is a live file that
grows as later sessions append to it, so an unpinned total is not a
repeatable acceptance target:
| Component | Tokens | Cost |
|---|---|---|
| Output | 313,900 | $10.66 |
| Cache read | 80,453,702 | **$59.59** |
| Cache write (1h) | 1,672,854 | $21.95 |
| Input | 676 | $0.00 |
| **Total** (339 responses: 206 opus-5, 118 fable-5, 14 sonnet-5) | | **$92.21** |
The headline finding survives the correction and gets sharper:
**88.4% of the cost is cache, against 314k tokens of output — a 256:1
ratio of context re-read to text written.** Cost in an agentic loop is
driven by context size × turn count, not by how much the model writes. No
D2 decision made on "tokens per task" would have surfaced that.
The correction is itself the lesson: this workplan opened with a
mismeasured cost. Neither error was of the harness-does-nothing class the
positive-control rule was written for — both sums ran over real data.
What caught them was re-deriving the number instead of quoting it.
This workplan makes cost a first-class measured dimension so that
future AM-12 equivalents support conclusions instead of decorating an