T02 review round: survey approved after 15 of 16 findings conceded

Adversarial review per InnerLoop Step 2 — separate agent, given only the
survey, one round. Challenge and response in history/.

The arithmetic held: every pinned figure and the whole three-method
correction table reproduced to the cent under ~20 independent spot
checks. The hand-written markdown tables did not. Three blocking
defects:

  - the dedup positive-control table failed its own checksum (group
    sizes summed to 348 groups / 663 lines, not 346 / 657; multi-line
    groups are 206, not 208) — the worst place for a slip, since that
    table is the sole evidence for the load-bearing dedup assumption
  - the multi-model table was labelled "deduped" while holding per-line
    counts, three lines below the paragraph condemning per-line counts
  - the $92.21 acceptance target excluded the subagent tree, which ran
    INSIDE the pin window. A collector correctly implementing the
    survey's own blind-spot finding would have failed the benchmark.
    Target is now $92.87, stated as its two components.

Folded in, each re-derived before conceding:

  - attribution must key on sessionId, not wall-clock: two sessions
    overlap 4h13m on this repo carrying ~$12 (one of them was this
    workplan's own background job)
  - only 14 of 33 commits name a task; the other 19 hold 33% of cost
  - cache writes bill per TTL — pricing the aggregate at 1h inflates
    the subagent by 43%
  - the "~87% cache" figure matched nothing; 64.6% of cost is cache
    read, 88.4% is all cache
  - fable is 35% of calls and 60% of dollars; opus is the inverse

Review cost ~$0.60 against a $92 pass, the second consecutive round
where that trade found something material. Evidence for CB-WP-0003 T03.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

history/260731-cost-accounting-challenge.md

@@ -0,0 +1,163 @@
+# 2026-07-31 — adversarial challenge to CB-RES-0002
+
+Round 1 of 1 (InnerLoop §Step 2). Reviewer: separate agent session, given
+only `research/CB-RES-0002-cost-accounting.md` and read-only access to the
+raw data. Verbatim below; answers in `260731-cost-accounting-response.md`.
+
+Reviewer cost: 56,882 tokens, 14 tool uses, 330 s.
+
+---
+
+**Reproduced cleanly** (`python3` over the pinned line set, prices from
+`benchmarks/baselines/model-prices.toml`): pin `2026-07-31T02:17:59Z` =
+commit `fc76445` (`%cI` = `2026-07-31T04:17:59+02:00`) ✓. 339 responses ✓
+(206 opus / 118 fable / 14 sonnet / 1 synthetic) ✓. Output 313,900 →
+$10.6625 ✓. Cache read 80,453,702 → $59.5880 ✓. Cache write 1h 1,672,854 →
+$21.9525 ✓. Input 676 → $0.0045 ✓. **Total $92.2075** ✓. Cache share
+88.43% ✓. Ratio 256.30:1 ✓. The three-method table reproduces to the cent
+over the first 657 assistant lines: 346 groups, 700,690 out, 161,408,840
+cread, **$289.1248 / $210.0536 / $93.1461** ✓. The "356 / $94.04" drift
+check reproduces at assistant line 682 ✓. Subagent **$0.6580** ✓. Dedup
+positive control holds: 0/200 pinned multi-line groups differ in `usage` or
+model; 0 assistant lines anywhere in the file lack a `requestId`.
+`iterations` claim holds: 758/760 non-null, 0 sum-mismatches, 0
+multi-iteration. `cache_creation_input_tokens` == `ephemeral_1h +
+ephemeral_5m` in all 775 usage lines across both transcripts. The survey did
+re-derive; the boast is earned.
+
+**1. The dedup positive-control table does not sum to its own totals. —
+MATERIAL** Claim: group sizes `140/120/76/6/5/1`, "346 distinct requestIds",
+"all 208 multi-line groups". Recomputed over the same 657-line prefix:
+`{1:140, 2:120, 3:74, 4:6, 5:5, 6:1}`. The stated table gives 348 groups and
+663 lines, contradicting both 346 and 657. Multi-line groups are **206**, not
+208. This is the sole evidence for the load-bearing dedup assumption (Risk
+2), so a table that fails its own checksum is the wrong place to be sloppy.
+The 74 correction makes it sum exactly.
+
+**2. The multi-model table is labelled "API responses (deduped)" but holds
+per-line counts. — MATERIAL** `382 + 250 + 24 + 1 = 657` — those are lines,
+the exact quantity the preceding paragraph just condemned. Deduped over the
+same prefix: opus 213, fable 118, sonnet 14, synthetic 1 = 346. The mislabel
+sits three lines below "Summing per line inflates by ≈1.9×."
+
+**3. The $92.21 acceptance target contradicts C1's own blind-spot finding. —
+MATERIAL (blocking for T05 as written)** The subagent transcript spans
+`23:11:42Z–23:14:12Z` on 2026-07-30 — **inside the pin window**. Its $0.6580
+is therefore part of the pinned session's true cost, which is **$92.87**, not
+$92.21. The survey never states whether $92.21 is main-file-only. A collector
+that correctly implements the C1 blind-spot ("a collector reading only the
+main file silently under-reports") produces $92.87 and **fails** the
+benchmark-to-beat. Either pin $92.87, or state "$92.21 main transcript +
+$0.66 subagent = $92.87" explicitly.
+
+**4. Pricing the top-level `cache_creation_input_tokens` at the 1h rate
+overcharges by 60% on 5m blocks — and the only 5m data in the corpus is the
+subagent. — MATERIAL** Main session: `w5m = 0`, so the pinned $92.21 is
+insensitive to this. The subagent's *entire* cache write is 5m: 37,467
+tokens. At `write_5m=1.25` → $0.4683; at `write_1h=2.0` → $0.7493. On a
+$0.658 total that is a **+43% error**. C1 lists both field names but never
+states that the top-level aggregate must not be priced at a single
+multiplier. Given that a fan-out pass is exactly where subagents (and
+therefore 5m caching) dominate, this belongs in the risk list.
+
+**5. Concurrent sessions on one repo are not hypothetical — they are already
+in the data. — MATERIAL** Risk 6: "two sessions interleaved on one repo would
+mis-attribute. Not exercised here — CB-WP-0001 ran in a single session."
+Session `f1eb1147` (same project dir) spans `02:18:13Z–06:33:41Z` while
+`8cbd5701` runs to `06:30:47Z`. That is a **4 h 13 m overlap** carrying **67
+responses / $6.09** from `8cbd5701` and **67 responses / $5.90** from
+`f1eb1147` — ~$12 that a wall-clock join cannot separate, starting 14 seconds
+after the pin. The survey also never mentions that a second transcript for
+this repo exists; C1's blind-spot section names only the subagent tree.
+
+**6. Attribution coverage is never quantified, and a third of the money has
+no task. — MATERIAL** Stress-tested with intervals `(prev commit, this
+commit]` over the 33 commits at/before pin: 0 responses before the first
+commit, 0 after the last, exactly **1** empty interval (`Initial commit`). So
+far so good. But **19 of 33 commits carry no `T##` in the subject**
+(`chore(consistency)` ×4, `Add CB-WP-000x`, `CI: enforce every gate`, `AM-4:`,
+`Adopt TRSL V1C1`, …), and they hold **$29.58 of $92.21 = 32% of cost**.
+C5's "the loop already commits per task iteration with the task in the
+subject line" is true of 14/33 commits. The benchmark-to-beat correctly
+demands an unattributed-remainder line but sizes it nowhere; 32% is the
+number that belongs in the survey.
+
+**7. "Cache reads are ~87% of real spend" (C2, defect 1) matches no computed
+quantity. — MATERIAL** Cache read alone = $59.588/$92.2075 = **64.6%** of
+cost. All cache (read + 1h write) = **88.4%**. Cache-read tokens as a share
+of input tokens = **98.0%**. Nothing is 87%. Since C2's elimination rests on
+"the hub cannot express the ~87%", fix it to 64.6% (cost) or 88.4% (all
+cache) and say which.
+
+**8. The `<synthetic>` message is described wrongly, though its dollar impact
+is zero. — MINOR** "1 (no usage; an error placeholder)". It has a valid
+`requestId` (`req_011CdZ9m3xZSLgtg2baVuLmG`) and a **complete all-zero
+`usage` object**. Zero dollars are dropped — confirmed, the unpriced
+remainder is exactly $0.00 — but a collector guarding on `if not usage:` and
+one guarding on `if model not in prices:` take different branches, and the
+survey's description points at the wrong one.
+
+**9. "Checked all 654 usage-bearing lines" does not reproduce. — MINOR** All
+638 pinned assistant lines (and all 760 in the full file) carry `usage`;
+637/638 pinned carry non-null `iterations`. There is no state of the file
+with 654 usage-bearing lines out of 657. The *conclusion* is verified; the
+*count* is not.
+
+**10. The timezone hazard is stated as a single offset; the repo has two. —
+MINOR** `git log` offsets: `+02:00` ×34, **`+00:00` ×1** (`8b66604 Initial
+commit`). "commit timestamps are local (`+02:00`) … a naive join is off by
+the offset" invites a fixed −2 h correction that is wrong for the first
+commit. Say "parse `%cI` and convert", not "subtract the offset".
+
+**11. Commit spacing understated. — MINOR** "Boundaries are 3–15 minutes
+apart across CB-WP-0001." Measured over the 33 pinned commits: min 0.1, p50
+**6.9**, p90 **17.7**, max **36.8** minutes. The max matters — a 37-minute
+interval is coarser than a task.
+
+**12. The interval convention is never named. — MINOR** "Join message
+timestamps against commit intervals" is ambiguous between *ending-at-commit*
+and *starting-at-commit*. The commit-at-end-of-task pattern makes only the
+first correct, and under it the boundary conditions are clean (0 orphans
+either end, verified). Under the other, every task's cost shifts one interval
+and the reviewer's worry is real. One sentence fixes it.
+
+**13. The price sheet's intro rate is a comment, not data. — MINOR**
+`model-prices.toml` encodes sonnet at 3/15 with `# intro 2.00 through
+2026-08-31`. A collector reading the TOML — which the Verdict prescribes —
+silently ignores a live rate. Delta at the pin: **$0.1738** (0.19% of
+$92.21). Small now; the failure mode is that a *time-boxed* rate is
+unrepresentable in the schema, which is the same class of defect the survey
+levels at C2.
+
+**14. Baselines table: D1–D4 are each present, but the deciding axis is a
+fifth non-rubric row with no beatable value. — MINOR** D1, D2 (×2), D3, D4
+all appear, so the template is satisfied on its face. But the row marked
+"**the deciding axis**" is "Accuracy", which is not one of the four
+dimensions, and its value — "exact by construction" — is not a number
+anything can be measured against. D4's provenance reads "measured (absent)",
+which is not a measurement. Only one benchmark-to-beat ($92.21) is given,
+globally, not per dimension.
+
+**15. C3/C4 elimination is legitimate — not the sin the survey accuses the
+workplan of. — no finding.** Both are eliminated on *categorical* grounds
+(C3: not machine-readable, `statusLine` absent; C4: zero session/task
+attribution, no admin key), not on unmeasured performance claims. You cannot
+measure the cost-accuracy of an instrument that structurally cannot
+attribute. C4's "authoritative — it *is* the invoice" is asserted rather than
+measured, but it is used only to justify keeping C4 as an optional external
+check, which is the conservative direction.
+
+**16. Framing note: Fable is the cost majority, Opus is only the count
+majority. — MINOR** Pinned per-model cost: fable $55.50 (60%) on 118
+responses, opus $36.19 (39%) on 206, sonnet $0.52 (0.6%) on 14. "Pricing
+everything at Fable 5's $10/$50 overstates the Opus and Sonnet majority" is
+true by response count but reads as a cost claim, and in a document whose
+thesis is that totals mislead, the 35%-of-calls / 60%-of-dollars inversion is
+the more interesting fact.
+
+**Bottom line:** the arithmetic is sound — every pinned figure and the entire
+three-method table reproduce to the cent, which is a genuinely strong result.
+The survey is not approvable as-is because of #1 (the dedup control fails its
+own checksum), #2 (mislabelled table), and #3 (the acceptance target is
+provably incomplete by $0.66 against the survey's own blind-spot finding).
+#4–#7 must be folded in; the rest are corrections.

history/260731-cost-accounting-response.md

@@ -0,0 +1,134 @@
+# 2026-07-31 — response to the challenge on CB-RES-0002
+
+Answers to `260731-cost-accounting-challenge.md`. Every contested number was
+**re-derived here before being conceded** — the rule this survey argued for
+applies to the challenge too, not only to the survey.
+
+Outcome: **16 findings, 15 conceded, 1 partially answered, 0 rejected.**
+Survey moves to `status: approved` after the folds below.
+
+## Blocking — conceded
+
+**#1 dedup table fails its own checksum.** Confirmed: the group-size
+distribution is `{1:140, 2:120, 3:74, 4:6, 5:5, 6:1}` = 346 groups / 657
+lines, and multi-line groups are **206**. The survey said 76 and 208. The
+reviewer is right that this is the worst possible place for an arithmetic
+slip: the table *is* the positive control for the load-bearing dedup
+assumption. Corrected in the survey.
+
+Worth naming plainly: a document whose thesis is "re-derive, don't quote"
+shipped a hand-typed table that did not sum. The finding survives; the
+discipline did not extend to the evidence *for* the finding.
+
+**#2 multi-model table mislabelled.** Confirmed: `382/250/24/1` are per-line
+counts under a header reading "API responses (deduped)". Deduped over the
+same prefix: **opus 213, fable 118, sonnet 14, synthetic 1 = 346**. The
+reviewer's 206 for opus was the per-line figure quoted from the survey's own
+text; the correct deduped value is 213. Corrected, and the table now shows
+both columns so the 1.9× inflation is visible rather than asserted.
+
+**#3 the acceptance target excludes the subagent.** Confirmed and the most
+consequential finding. The subagent ran `2026-07-30T23:11:42Z–23:14:12Z`,
+which is **inside** the pin window. Re-derived:
+
+```text
+main transcript   $92.2075  (339 responses)
+subagent          $ 0.6580  (7 responses)
+TRUE TOTAL        $92.8655
+```
+
+The survey stated the C1 blind-spot correctly and then set a benchmark that
+only a collector *exhibiting* that blind-spot could hit. A correct collector
+would have failed the acceptance test. The target is now **$92.87**, stated
+as the sum of both components so a collector reading one tree can be
+diagnosed rather than merely failed.
+
+## Material — conceded
+
+**#4 5m vs 1h cache-write pricing.** Confirmed in substance. The subagent's
+cache writes are **entirely 5m** (37,467 deduped tokens, 1h = 0). The
+collector's arithmetic was already correct — it prices the `ephemeral_5m` and
+`ephemeral_1h` fields separately, and $0.658 reflects that. What was missing
+is the *rule*: the survey named both fields without stating that the
+top-level `cache_creation_input_tokens` aggregate must never be priced at a
+single multiplier. Mispricing that aggregate at 1h yields $0.939, a +43%
+error. Added as an explicit contract for T03 and a risk-list entry, since
+fan-out passes are where 5m writes dominate.
+
+**#5 concurrent sessions are already in the data.** Conceded, with an
+observation the reviewer could not have had: session `f1eb1147` is *this
+review's own session* — the background job running the workplan. The survey
+claimed the interleaving risk was "not exercised here"; it was being
+exercised as the sentence was written. ~$12 across a 4h13m overlap cannot be
+separated by a wall-clock join. This kills naive wall-clock attribution
+outright and is now a stated ADR constraint: **attribution must key on
+`sessionId`, not only time.** The survey's C1 section also now names the
+second transcript.
+
+**#6 a third of the money has no task tag.** Re-derived independently with
+`(prev commit, this commit]` intervals over the 33 pinned commits:
+**$30.32 of $92.21 = 33% untagged**, 1 empty interval, 0 orphans at either
+end. (The reviewer got $29.58/32%; the small delta is regex scope — theirs
+matched `\bT\d\d\b`, mine the same, differing on one boundary commit. Same
+conclusion, and the survey now carries the re-derived 33%.) C5's claim that
+the loop "commits per task iteration with the task in the subject" holds for
+14 of 33 commits. This is now sized in the survey rather than left as a
+qualitative hazard, and the unattributed-remainder line in T04 has a known
+target to beat.
+
+**#7 the ~87% figure matches nothing.** Conceded — it was a stale
+hand-carried number from before the pin. The defensible figures are **64.6%**
+(cache read as a share of cost) and **88.4%** (all cache as a share of cost).
+C2's argument is unaffected: the hub schema can express neither. Corrected to
+88.4% with the basis named.
+
+## Minor — conceded
+
+- **#8** `<synthetic>` has a valid `requestId` and an all-zero `usage`, not a
+  missing one. Dollar impact is exactly $0.00, but the two guard styles
+  (`if not usage` vs `if model not in prices`) branch differently, so T04's
+  contract now names which it uses.
+- **#9** the "654 usage-bearing lines" count does not reproduce; all pinned
+  assistant lines carry `usage`. Conclusion stood, count was wrong. Removed.
+- **#10** the repo has two commit-offsets (`+02:00` ×34, `+00:00` ×1).
+  Guidance changed from "off by the offset" to "parse `%cI` and convert".
+- **#11** commit spacing is min 0.1 / p50 6.9 / p90 17.7 / **max 36.8**
+  minutes, not "3–15". The max is the number that matters.
+- **#12** interval convention now stated explicitly: *ending-at-commit*,
+  `(prev, this]`, which is the only one consistent with commit-at-end-of-task.
+- **#13** the sonnet intro rate is a TOML comment, so a collector silently
+  ignores it ($0.17 at the pin). Raised as a price-sheet schema defect —
+  time-boxed rates need a representation. Same class of defect the survey
+  levels at C2, as the reviewer notes.
+- **#14** the "Accuracy" row is a fifth non-rubric axis with no beatable
+  value. Folded into D2 with a number attached.
+- **#16** accepted as the better framing, and promoted out of "minor":
+  **fable is 60% of cost on 35% of calls; opus is 39% of cost on 60% of
+  calls.** In a document arguing that totals mislead, a count-majority
+  presented where a cost-majority was meant is the same error one level down.
+
+## Partially answered
+
+**#15 (reviewer found no fault) — noted, with one correction accepted.**
+The reviewer is right that C3/C4 elimination is categorical rather than
+performance-based and therefore legitimate. Accepted. The one thing folded
+in anyway: C4's "authoritative — it *is* the invoice" is an assertion, and
+is now marked as such rather than reading like a measurement.
+
+## What this round cost, and whether it paid
+
+Reviewer: 56,882 tokens / 14 tool uses / 330 s. Priced against the sheet,
+~$0.60 — the same order as CB-WP-0001's review at $0.66.
+
+It found three approval-blocking defects, one of which (#3) would have made
+T05's acceptance test reward a *broken* collector and fail a correct one.
+That is the second consecutive pass where a ~$0.60 review caught something a
+$92 pass had missed. The economics are not close, and this belongs in the
+CB-WP-0003 T03 argument for pointing review at measurement rather than prose.
+
+The uncomfortable pattern across both rounds: **the survey's arithmetic over
+raw data was flawless — every one of the reviewer's ~20 spot-checks
+reproduced to the cent — and its hand-written prose tables were not.** The
+errors were all in numbers typed by hand into markdown after the computation,
+never in the computation. That is a mechanical, fixable class: tables that a
+tool can emit should be emitted by the tool. Raised for T03 and T06.