clay-borg/research/CB-RES-0002-cost-accounting.md

CB-RES-0002: agentic cost accounting

capability: meta.loop.cost-accounting status: draft tier: L (structural L, chaos d10=2 → no override) runnable-baseline: invoked — every candidate below was exercised against the CB-WP-0001 session on this machine, not cited review-trail: history/260731-cost-accounting-research.md

Survey of instruments that can attribute the USD cost of agentic work to a unit of work, so that M-D2-CST (specs/MetricsAndScenarios.md §1a) becomes computable. CB-WP-0001 specified that metric completely and recorded it as uncomputable; the premise of this workplan is that the data existed the whole time.

That premise survives. The workplan's numbers do not — see §Correction.

---

Correction to this workplan's own Purpose section

CB-WP-0002's Purpose reports the CB-WP-0001 session at $248.46, from 131,863,164 cache-read tokens priced at Fable 5. Both halves are wrong, and in the same direction — too high. The survey found this by re-deriving the number rather than adopting it.

Error 1 — per-line summation double-counts. A single API response is written to the transcript as several JSONL lines, split by content block (thinking, text, tool_use), and every one of those lines repeats the complete usage object. Measured on the CB-WP-0001 transcript: 657 assistant lines carry only 346 distinct requestIds. Group sizes run 1–6:

lines per requestId	1	2	3	4	5	6
groups	140	120	76	6	5	1

Positive control on the dedup: across all 208 multi-line groups the usage object is byte-identical (208/208 identical, 0 differing), and no group mixes models. The duplication is a transcript-format artifact, not repeated billing. Summing per line inflates by ≈1.9×.

Error 2 — single-model pricing on a multi-model session. The session ran three models, not one:

model	API responses (deduped)
claude-opus-5	382 lines → majority
claude-fable-5	250 lines
claude-sonnet-5	24 lines
<synthetic>	1 (no usage; an error placeholder)

Pricing everything at Fable 5's $10/$50 overstates the Opus and Sonnet majority. §1a already required per-model pricing; the Purpose section did not apply its own rule.

Corrected totals for the same transcript, all three methods run over the identical unpinned line set so the methods are comparable:

method	responses	output	cache read	cost
per-line, all-Fable (the Purpose method)	657	700,690	161,408,840	$289.12
per-line, per-model	657	700,690	161,408,840	$210.05
deduped, per-model (correct)	346	318,230	81,100,498	$93.15

Third error, found while verifying the second: the transcript is a live file. Re-running the deduped figure minutes later returned 356 responses and $94.04 — this survey's own session appends to the same JSONL it is measuring. An unpinned total is not a repeatable number. The acceptance target is therefore pinned by timestamp:

CB-WP-0001, pinned ≤ 2026-07-31T02:17:59Z (commit fc76445)	value
responses	339 (206 opus-5, 118 fable-5, 14 sonnet-5, 1 synthetic)
output	313,900 tok → $10.66
cache read	80,453,702 tok → $59.59
cache write 1h	1,672,854 tok → $21.95
input	676 tok → $0.00
total	$92.21 — 88.4% cache, 256:1 cache-read:output

The reported figure was ~2.7× the real cost. This is the fourth instance of the harness-does-nothing error class from history/260731-inner-loop-retrospective.md, wearing a new coat: not a harness that measured nothing, but an arithmetic that measured the same thing twice. Both produce a number that looks fine.

The qualitative headline survives the correction and gets stronger: cache reads are 81.1M tokens against 318k of output, ~255:1. Cost in an agentic loop is context × turns.

---

Candidates

C1 — Session transcript JSONL

~/.claude/projects/<slug>/<session>.jsonl, one JSON object per line. Assistant lines carry message.usage with exact billing counters: input_tokens, output_tokens, cache_read_input_tokens, and cache_creation.{ephemeral_1h,ephemeral_5m}_input_tokens, plus message.model, requestId, and an ISO-8601 timestamp.

• Granularity: per API response, once deduplicated by requestId.
• Accuracy: exact — these are the counters the invoice is computed from. There is no sampling or rounding.
• Verified non-issue: usage.iterations[] is a sub-breakdown, not an additional charge. Checked all 654 usage-bearing lines: the iteration outputs sum exactly to the top-level output_tokens in every case, and no message had more than one iteration. Summing iterations instead of the top-level fields is safe; summing both would double-count.
• Survives compaction: yes. /compact writes a summary message into the same file (isCompactSummary, compactMetadata) and the session continues; no usage is lost. Compaction is visible as an event, so its cost is itself measurable.
• Attribution: none built in — a transcript is a flat message stream with timestamps. It must be joined against an external time index.
• Blind spot found: subagent cost is not in the main transcript. isSidechain is false on all 657 lines; subagent work lives in <session>/subagents/agent-*.jsonl with an agent-*.meta.json naming the agentType and model. CB-WP-0001 spawned one (the adversarial review). A collector reading only the main file silently under-reports.

C2 — Custodian State Hub token API

record_token_event, the update_task_status token tiers, and get_token_summary. Exercised against CB-WP-0001's workplan (a1b434dc-…), which returned:

tokens_in 362,000   tokens_out 39,100   event_count 7   by model: claude-fable-5

• Granularity: per task — the best of any candidate, and the only one that is natively about the unit of work.
• Accuracy: poor, and structurally so. Three independent defects:
  1. The schema has no cache fields. tokens_in/tokens_out cannot represent the finding this workplan exists to report. Cache reads are ~87% of real spend here and the hub cannot express them at any fidelity.
  2. The recorded numbers are estimates. 7 events for 9 tasks, at round figures — the skill's Tier-3 heuristic (1000/500) and Tier-1 eyeball estimates. Against a deduped transcript output of 318,230, the hub's 39,100 is off by ~8×; against total input it is off by ~450×.
  3. Model attribution is wrong. Everything is filed under claude-fable-5 on a session that was majority Opus 5.
• Survives compaction: yes — it is server-side and independent of the client.
• Verdict: durable and task-shaped, but its numbers are unusable as a cost source. Its role is as a sink for numbers computed elsewhere, not a source. Even as a sink it can only carry a lossy projection until the schema grows cache fields.

C3 — Claude Code status bar

• Granularity: whole session, live.
• Accuracy: unknown and unauditable — it is rendered text.
• Machine-readable: no. Not configured here (statusLine is absent from ~/.claude/settings.json), and it is not reachable from inside a tool call regardless.
• Verdict: eliminated. The ralph-workplan skill's "read tokens from the status bar" instruction is the proximate cause of C2's bad numbers — it asks an agent to report a figure it cannot read, and an agent that cannot read it estimates instead. This should be raised against the skill.

C4 — Anthropic usage / billing API

• Granularity: organization and API-key, by day.
• Accuracy: authoritative — it is the invoice.
• Attribution: none to a task, and none to a session. Cannot separate clay-borg from the other twenty-plus projects on this machine.
• Availability: requires an admin key; none is configured here.
• Verdict: not usable for M-D2-CST, but valuable as an external reconciliation check if an admin key is ever provisioned — it is the only candidate that can catch a systematic error in C1's price model. Left as a stated non-dependency.

C5 — Git commit history (attribution index, not a cost source)

Not a cost instrument; the missing half of C1. The loop already commits per task iteration with the task in the subject line, giving durable, timestamped boundaries at exactly the granularity M-D2-CST wants:

a09d76f 2026-07-31T02:14:34+02:00 T08 iter 1: scenario runner executes; …
b58a913 2026-07-31T02:19:34+02:00 T08 iter 2: Reveal, Resolve, End; …

• Boundaries are 3–15 minutes apart across CB-WP-0001 — finer than a task.
• Durable, versioned, and free; requires no change to how work is done.
• Known hazards: commit timestamps are local (+02:00), transcript timestamps are UTC — a naive join is off by the offset. Work before the first commit and after the last has no enclosing interval. Commits made outside a session (or by fix-consistency) create empty intervals.

---

Baselines (benchmark-to-beat)

Dimension	Baseline holder	Metric	Value	Provenance
D1 ease of specification	C2 hub	fields needed to record a task's cost	4 (task_id, tokens_in, tokens_out, model) — but cannot express cache	measured (API schema)
D2 efficiency	C2 hub	cost of producing a number	~0 (one API call) — number is an estimate, off by ~8× on output	measured
D2 efficiency	C1 transcript	cost of producing a number	one file read, 5.1 MB, ~1 s; exact	measured
D3 speed	C1 transcript	parse of a full session	2,040 lines / 5.1 MB in <1 s in CPython	measured
D4 optionality	C4 billing API	authority	invoice-grade, zero attribution, needs admin key	measured (absent)
Accuracy (the deciding axis)	C1 transcript	agreement with billing counters	exact by construction	measured

Benchmark-to-beat for the collector: reproduce $92.21 for session 8cbd5701 pinned at 2026-07-31T02:17:59Z, from the committed price sheet, with an unattributed remainder reported as its own line and reconciliation asserted rather than assumed.

---

Verdict

C1 (transcript) leads on accuracy and is the only exact candidate. C5 (git commits) supplies the attribution index C1 lacks. C2 is the durable sink. C3 is eliminated. C4 is an optional external check.

The expected shape is therefore: parse transcripts → dedup by requestId → price per message at its own model's rate from benchmarks/baselines/model-prices.toml → attribute to a task by joining message timestamps against commit intervals → emit per-task cost and a composition breakdown → push a lossy summary to C2.

What none of them do well — the surpass opportunity. Every candidate reports totals. None reports composition, and composition is where the actionable finding lives: 81.1M cache-read tokens against 318k of output means cost is driven by how much context is re-read per turn, which no total can show. A metric that had reported only dollars would have been correct and useless.

Risks in the baselines themselves.

1. The $248.46 figure was wrong and was nearly adopted as this workplan's acceptance target. T05's reconciliation test must be against a number this survey re-derived, not against the Purpose section. The Purpose section needs correcting.
2. Dedup is load-bearing. If the transcript format ever splits one response across two requestIds, dedup silently under-reports — the opposite error, and the more dangerous one. The collector must assert its dedup assumption (identical usage within a group) at runtime rather than trusting this survey's one-time check.
3. Subagent transcripts are a separate tree. Measured: CB-WP-0001's one subagent (adversarial review, Fable 5, 7 responses, 158,096 cache reads) cost $0.66, invisible to any collector reading only the main file. Small here; not small for a pass that fans out.
4. The price sheet has a 90-day staleness rule (§1a) and no automated check. Every number this capability produces inherits that.
5. The transcript is append-live. It is written by the session that reads it, so any total is a reading at an instant. Every committed number from this capability states its pin (timestamp or commit), and the collector takes a pin argument rather than defaulting to "all".
6. Attribution across /compact and resumed sessions is unproven. The join is by wall-clock; a compaction inside a task boundary is fine, but two sessions interleaved on one repo would mis-attribute. Not exercised here — CB-WP-0001 ran in a single session.