clay-borg/history/260731-cost-accounting-challenge.md

# 2026-07-31 — adversarial challenge to CB-RES-0002

Round 1 of 1 (InnerLoop §Step 2). Reviewer: separate agent session, given
only `research/CB-RES-0002-cost-accounting.md` and read-only access to the
raw data. Verbatim below; answers in `260731-cost-accounting-response.md`.

Reviewer cost: 56,882 tokens, 14 tool uses, 330 s.

---

**Reproduced cleanly** (`python3` over the pinned line set, prices from
`benchmarks/baselines/model-prices.toml`): pin `2026-07-31T02:17:59Z` =
commit `fc76445` (`%cI` = `2026-07-31T04:17:59+02:00`) ✓. 339 responses ✓
(206 opus / 118 fable / 14 sonnet / 1 synthetic) ✓. Output 313,900 →
$10.6625 ✓. Cache read 80,453,702 → $59.5880 ✓. Cache write 1h 1,672,854 →
$21.9525 ✓. Input 676 → $0.0045 ✓. **Total $92.2075** ✓. Cache share
88.43% ✓. Ratio 256.30:1 ✓. The three-method table reproduces to the cent
over the first 657 assistant lines: 346 groups, 700,690 out, 161,408,840
cread, **$289.1248 / $210.0536 / $93.1461** ✓. The "356 / $94.04" drift
check reproduces at assistant line 682 ✓. Subagent **$0.6580** ✓. Dedup
positive control holds: 0/200 pinned multi-line groups differ in `usage` or
model; 0 assistant lines anywhere in the file lack a `requestId`.
`iterations` claim holds: 758/760 non-null, 0 sum-mismatches, 0
multi-iteration. `cache_creation_input_tokens` == `ephemeral_1h +
ephemeral_5m` in all 775 usage lines across both transcripts. The survey did
re-derive; the boast is earned.

**1. The dedup positive-control table does not sum to its own totals. —
MATERIAL** Claim: group sizes `140/120/76/6/5/1`, "346 distinct requestIds",
"all 208 multi-line groups". Recomputed over the same 657-line prefix:
`{1:140, 2:120, 3:74, 4:6, 5:5, 6:1}`. The stated table gives 348 groups and
663 lines, contradicting both 346 and 657. Multi-line groups are **206**, not
208. This is the sole evidence for the load-bearing dedup assumption (Risk
2), so a table that fails its own checksum is the wrong place to be sloppy.
The 74 correction makes it sum exactly.

**2. The multi-model table is labelled "API responses (deduped)" but holds
per-line counts. — MATERIAL** `382 + 250 + 24 + 1 = 657` — those are lines,
the exact quantity the preceding paragraph just condemned. Deduped over the
same prefix: opus 213, fable 118, sonnet 14, synthetic 1 = 346. The mislabel
sits three lines below "Summing per line inflates by ≈1.9×."

**3. The $92.21 acceptance target contradicts C1's own blind-spot finding. —
MATERIAL (blocking for T05 as written)** The subagent transcript spans
`23:11:42Z–23:14:12Z` on 2026-07-30 — **inside the pin window**. Its $0.6580
is therefore part of the pinned session's true cost, which is **$92.87**, not
$92.21. The survey never states whether $92.21 is main-file-only. A collector
that correctly implements the C1 blind-spot ("a collector reading only the
main file silently under-reports") produces $92.87 and **fails** the
benchmark-to-beat. Either pin $92.87, or state "$92.21 main transcript +
$0.66 subagent = $92.87" explicitly.

**4. Pricing the top-level `cache_creation_input_tokens` at the 1h rate
overcharges by 60% on 5m blocks — and the only 5m data in the corpus is the
subagent. — MATERIAL** Main session: `w5m = 0`, so the pinned $92.21 is
insensitive to this. The subagent's *entire* cache write is 5m: 37,467
tokens. At `write_5m=1.25` → $0.4683; at `write_1h=2.0` → $0.7493. On a
$0.658 total that is a **+43% error**. C1 lists both field names but never
states that the top-level aggregate must not be priced at a single
multiplier. Given that a fan-out pass is exactly where subagents (and
therefore 5m caching) dominate, this belongs in the risk list.

**5. Concurrent sessions on one repo are not hypothetical — they are already
in the data. — MATERIAL** Risk 6: "two sessions interleaved on one repo would
mis-attribute. Not exercised here — CB-WP-0001 ran in a single session."
Session `f1eb1147` (same project dir) spans `02:18:13Z–06:33:41Z` while
`8cbd5701` runs to `06:30:47Z`. That is a **4 h 13 m overlap** carrying **67
responses / $6.09** from `8cbd5701` and **67 responses / $5.90** from
`f1eb1147` — ~$12 that a wall-clock join cannot separate, starting 14 seconds
after the pin. The survey also never mentions that a second transcript for
this repo exists; C1's blind-spot section names only the subagent tree.

**6. Attribution coverage is never quantified, and a third of the money has
no task. — MATERIAL** Stress-tested with intervals `(prev commit, this
commit]` over the 33 commits at/before pin: 0 responses before the first
commit, 0 after the last, exactly **1** empty interval (`Initial commit`). So
far so good. But **19 of 33 commits carry no `T##` in the subject**
(`chore(consistency)` ×4, `Add CB-WP-000x`, `CI: enforce every gate`, `AM-4:`,
`Adopt TRSL V1C1`, …), and they hold **$29.58 of $92.21 = 32% of cost**.
C5's "the loop already commits per task iteration with the task in the
subject line" is true of 14/33 commits. The benchmark-to-beat correctly
demands an unattributed-remainder line but sizes it nowhere; 32% is the
number that belongs in the survey.

**7. "Cache reads are ~87% of real spend" (C2, defect 1) matches no computed
quantity. — MATERIAL** Cache read alone = $59.588/$92.2075 = **64.6%** of
cost. All cache (read + 1h write) = **88.4%**. Cache-read tokens as a share
of input tokens = **98.0%**. Nothing is 87%. Since C2's elimination rests on
"the hub cannot express the ~87%", fix it to 64.6% (cost) or 88.4% (all
cache) and say which.

**8. The `<synthetic>` message is described wrongly, though its dollar impact
is zero. — MINOR** "1 (no usage; an error placeholder)". It has a valid
`requestId` (`req_011CdZ9m3xZSLgtg2baVuLmG`) and a **complete all-zero
`usage` object**. Zero dollars are dropped — confirmed, the unpriced
remainder is exactly $0.00 — but a collector guarding on `if not usage:` and
one guarding on `if model not in prices:` take different branches, and the
survey's description points at the wrong one.

**9. "Checked all 654 usage-bearing lines" does not reproduce. — MINOR** All
638 pinned assistant lines (and all 760 in the full file) carry `usage`;
637/638 pinned carry non-null `iterations`. There is no state of the file
with 654 usage-bearing lines out of 657. The *conclusion* is verified; the
*count* is not.

**10. The timezone hazard is stated as a single offset; the repo has two. —
MINOR** `git log` offsets: `+02:00` ×34, **`+00:00` ×1** (`8b66604 Initial
commit`). "commit timestamps are local (`+02:00`) … a naive join is off by
the offset" invites a fixed −2 h correction that is wrong for the first
commit. Say "parse `%cI` and convert", not "subtract the offset".

**11. Commit spacing understated. — MINOR** "Boundaries are 3–15 minutes
apart across CB-WP-0001." Measured over the 33 pinned commits: min 0.1, p50
**6.9**, p90 **17.7**, max **36.8** minutes. The max matters — a 37-minute
interval is coarser than a task.

**12. The interval convention is never named. — MINOR** "Join message
timestamps against commit intervals" is ambiguous between *ending-at-commit*
and *starting-at-commit*. The commit-at-end-of-task pattern makes only the
first correct, and under it the boundary conditions are clean (0 orphans
either end, verified). Under the other, every task's cost shifts one interval
and the reviewer's worry is real. One sentence fixes it.

**13. The price sheet's intro rate is a comment, not data. — MINOR**
`model-prices.toml` encodes sonnet at 3/15 with `# intro 2.00 through
2026-08-31`. A collector reading the TOML — which the Verdict prescribes —
silently ignores a live rate. Delta at the pin: **$0.1738** (0.19% of
$92.21). Small now; the failure mode is that a *time-boxed* rate is
unrepresentable in the schema, which is the same class of defect the survey
levels at C2.

**14. Baselines table: D1–D4 are each present, but the deciding axis is a
fifth non-rubric row with no beatable value. — MINOR** D1, D2 (×2), D3, D4
all appear, so the template is satisfied on its face. But the row marked
"**the deciding axis**" is "Accuracy", which is not one of the four
dimensions, and its value — "exact by construction" — is not a number
anything can be measured against. D4's provenance reads "measured (absent)",
which is not a measurement. Only one benchmark-to-beat ($92.21) is given,
globally, not per dimension.

**15. C3/C4 elimination is legitimate — not the sin the survey accuses the
workplan of. — no finding.** Both are eliminated on *categorical* grounds
(C3: not machine-readable, `statusLine` absent; C4: zero session/task
attribution, no admin key), not on unmeasured performance claims. You cannot
measure the cost-accuracy of an instrument that structurally cannot
attribute. C4's "authoritative — it *is* the invoice" is asserted rather than
measured, but it is used only to justify keeping C4 as an optional external
check, which is the conservative direction.

**16. Framing note: Fable is the cost majority, Opus is only the count
majority. — MINOR** Pinned per-model cost: fable $55.50 (60%) on 118
responses, opus $36.19 (39%) on 206, sonnet $0.52 (0.6%) on 14. "Pricing
everything at Fable 5's $10/$50 overstates the Opus and Sonnet majority" is
true by response count but reads as a cost claim, and in a document whose
thesis is that totals mislead, the 35%-of-calls / 60%-of-dollars inversion is
the more interesting fact.

**Bottom line:** the arithmetic is sound — every pinned figure and the entire
three-method table reproduce to the cent, which is a genuinely strong result.
The survey is not approvable as-is because of #1 (the dedup control fails its
own checksum), #2 (mislabelled table), and #3 (the acceptance target is
provably incomplete by $0.66 against the survey's own blind-spot finding).
#4–#7 must be folded in; the rest are corrections.