clay-borg/history/260731-cost-accounting-response.md

2026-07-31 — response to the challenge on CB-RES-0002

Answers to 260731-cost-accounting-challenge.md. Every contested number was re-derived here before being conceded — the rule this survey argued for applies to the challenge too, not only to the survey.

Outcome: 16 findings, 15 conceded, 1 partially answered, 0 rejected. Survey moves to status: approved after the folds below.

Blocking — conceded

#1 dedup table fails its own checksum. Confirmed: the group-size distribution is {1:140, 2:120, 3:74, 4:6, 5:5, 6:1} = 346 groups / 657 lines, and multi-line groups are 206. The survey said 76 and 208. The reviewer is right that this is the worst possible place for an arithmetic slip: the table is the positive control for the load-bearing dedup assumption. Corrected in the survey.

Worth naming plainly: a document whose thesis is "re-derive, don't quote" shipped a hand-typed table that did not sum. The finding survives; the discipline did not extend to the evidence for the finding.

#2 multi-model table mislabelled. Confirmed: 382/250/24/1 are per-line counts under a header reading "API responses (deduped)". Deduped over the same prefix: opus 213, fable 118, sonnet 14, synthetic 1 = 346. The reviewer's 206 for opus was the per-line figure quoted from the survey's own text; the correct deduped value is 213. Corrected, and the table now shows both columns so the 1.9× inflation is visible rather than asserted.

#3 the acceptance target excludes the subagent. Confirmed and the most consequential finding. The subagent ran 2026-07-30T23:11:42Z–23:14:12Z, which is inside the pin window. Re-derived:

main transcript   $92.2075  (339 responses)
subagent          $ 0.6580  (7 responses)
TRUE TOTAL        $92.8655

The survey stated the C1 blind-spot correctly and then set a benchmark that only a collector exhibiting that blind-spot could hit. A correct collector would have failed the acceptance test. The target is now $92.87, stated as the sum of both components so a collector reading one tree can be diagnosed rather than merely failed.

Material — conceded

#4 5m vs 1h cache-write pricing. Confirmed in substance. The subagent's cache writes are entirely 5m (37,467 deduped tokens, 1h = 0). The collector's arithmetic was already correct — it prices the ephemeral_5m and ephemeral_1h fields separately, and $0.658 reflects that. What was missing is the rule: the survey named both fields without stating that the top-level cache_creation_input_tokens aggregate must never be priced at a single multiplier. Mispricing that aggregate at 1h yields $0.939, a +43% error. Added as an explicit contract for T03 and a risk-list entry, since fan-out passes are where 5m writes dominate.

#5 concurrent sessions are already in the data. Conceded, with an observation the reviewer could not have had: session f1eb1147 is this review's own session — the background job running the workplan. The survey claimed the interleaving risk was "not exercised here"; it was being exercised as the sentence was written. ~$12 across a 4h13m overlap cannot be separated by a wall-clock join. This kills naive wall-clock attribution outright and is now a stated ADR constraint: attribution must key on sessionId, not only time. The survey's C1 section also now names the second transcript.

#6 a third of the money has no task tag. Re-derived independently with (prev commit, this commit] intervals over the 33 pinned commits: $30.32 of $92.21 = 33% untagged, 1 empty interval, 0 orphans at either end. (The reviewer got $29.58/32%; the small delta is regex scope — theirs matched \bT\d\d\b, mine the same, differing on one boundary commit. Same conclusion, and the survey now carries the re-derived 33%.) C5's claim that the loop "commits per task iteration with the task in the subject" holds for 14 of 33 commits. This is now sized in the survey rather than left as a qualitative hazard, and the unattributed-remainder line in T04 has a known target to beat.

#7 the ~87% figure matches nothing. Conceded — it was a stale hand-carried number from before the pin. The defensible figures are 64.6% (cache read as a share of cost) and 88.4% (all cache as a share of cost). C2's argument is unaffected: the hub schema can express neither. Corrected to 88.4% with the basis named.

Minor — conceded

• #8 <synthetic> has a valid requestId and an all-zero usage, not a missing one. Dollar impact is exactly $0.00, but the two guard styles (if not usage vs if model not in prices) branch differently, so T04's contract now names which it uses.
• #9 the "654 usage-bearing lines" count does not reproduce; all pinned assistant lines carry usage. Conclusion stood, count was wrong. Removed.
• #10 the repo has two commit-offsets (+02:00 ×34, +00:00 ×1). Guidance changed from "off by the offset" to "parse %cI and convert".
• #11 commit spacing is min 0.1 / p50 6.9 / p90 17.7 / max 36.8 minutes, not "3–15". The max is the number that matters.
• #12 interval convention now stated explicitly: ending-at-commit, (prev, this], which is the only one consistent with commit-at-end-of-task.
• #13 the sonnet intro rate is a TOML comment, so a collector silently ignores it ($0.17 at the pin). Raised as a price-sheet schema defect — time-boxed rates need a representation. Same class of defect the survey levels at C2, as the reviewer notes.
• #14 the "Accuracy" row is a fifth non-rubric axis with no beatable value. Folded into D2 with a number attached.
• #16 accepted as the better framing, and promoted out of "minor": fable is 60% of cost on 35% of calls; opus is 39% of cost on 60% of calls. In a document arguing that totals mislead, a count-majority presented where a cost-majority was meant is the same error one level down.

Partially answered

#15 (reviewer found no fault) — noted, with one correction accepted. The reviewer is right that C3/C4 elimination is categorical rather than performance-based and therefore legitimate. Accepted. The one thing folded in anyway: C4's "authoritative — it is the invoice" is an assertion, and is now marked as such rather than reading like a measurement.

What this round cost, and whether it paid

Reviewer: 56,882 tokens / 14 tool uses / 330 s. Priced against the sheet, ~$0.60 — the same order as CB-WP-0001's review at $0.66.

It found three approval-blocking defects, one of which (#3) would have made T05's acceptance test reward a broken collector and fail a correct one. That is the second consecutive pass where a ~$0.60 review caught something a $92 pass had missed. The economics are not close, and this belongs in the CB-WP-0003 T03 argument for pointing review at measurement rather than prose.

The uncomfortable pattern across both rounds: the survey's arithmetic over raw data was flawless — every one of the reviewer's ~20 spot-checks reproduced to the cent — and its hand-written prose tables were not. The errors were all in numbers typed by hand into markdown after the computation, never in the computation. That is a mechanical, fixable class: tables that a tool can emit should be emitted by the tool. Raised for T03 and T06.