coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

T05: CB-EV-0002 — all 9 metrics met, and /compact measured at 3.1x

Browse source 
AC-1 $93.32 ($92.21 main + $1.11 subagent), residual $0.000000. No
unmeasured rows. Composition: 88.0% cache, 11.9% output, 249:1 context
re-read to text written.

The question that could not be answered before: what /compact costs.
The transcript records it directly — 542,991 -> 19,974 tokens, a 27x
reduction. Cost per turn fell from $0.457 (136 turns before) to $0.149
(202 turns after). The 202 post-compact turns cost less than half the
136 before them.

This qualifies CB-WP-0003 T04: sessions are bounded-quadratic, not
quadratic. Cost grows with context between compactions and resets at
each one, so the failure mode is a long UNCOMPACTED session. Whether a
fresh session beats a compaction is now measurable, and T04 should
measure it before prescribing one task per session.

Stale $0.66/$92.87 targets updated across survey and workplan.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 08:51:50 +02:00
parent e008b1e787
commit c0eef604e8
4 changed files with 167 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

15
research/CB-RES-0002-cost-accounting.md
View file

@ -90,8 +90,13 @@ acceptance target is therefore pinned by timestamp:
| cache write 1h | 1,672,854 tok → $21.95 |
| input | 676 tok → $0.00 |
| main transcript | **$92.21** — 88.4% cache, 256:1 cache-read:output |
| + subagent tree (7 responses, ran 23:11–23:14Z, inside the pin) | $0.66 |
| **TRUE TOTAL** | **$92.87** |
| + subagent tree (7 responses, ran 23:11–23:14Z, inside the pin) | $1.11 |
| **TRUE TOTAL** | **$93.32** |
*(The subagent figure was $0.66 when this survey was written. T04's
positive control found the cause: `output_tokens` is a running count in
the `subagents/` tree, so first-wins dedup under-counted it. See
`specs/CostAccounting.md` CA-02a.)*
The subagent line is not a footnote. C1's blind-spot finding says a
collector reading only the main file under-reports; a target of $92.21 would
@ -255,9 +260,9 @@ b58a913 2026-07-31T02:19:34+02:00 T08 iter 2: Reveal, Resolve, End; …
| D2 efficiency (**the deciding row**) | C1 transcript | **error against the billing counters** | **$0.00 — the transcript *is* the counter set; C2's error on the same work is $92.21 − $0.03 recorded ≈ 100%** | measured |
| D4 optionality | C4 billing API | authority | invoice-grade, zero attribution, admin key absent (asserted, not measured — used only to keep C4 as an optional check) | availability measured |
**Benchmark-to-beat for the collector:** reproduce **$92.87** for repo
**Benchmark-to-beat for the collector:** reproduce **$93.32** for repo
`clay-borg` pinned at `2026-07-31T02:17:59Z` — as its two components,
**$92.21 main transcript + $0.66 subagent tree** — from the committed price
**$92.21 main transcript + $1.11 subagent tree** — from the committed price
sheet, with the unattributed remainder reported as its own line (expected
**33%**, $30.32) and reconciliation asserted rather than assumed.
@ -297,7 +302,7 @@ correct and useless.
runtime rather than trusting this survey's one-time check.
3. **Subagent transcripts are a separate tree.** Measured: CB-WP-0001's
one subagent (adversarial review, Fable 5, 7 responses, 158,096 cache
reads) cost **$0.66**, invisible to any collector reading only the
reads) cost **$1.11**, invisible to any collector reading only the
main file. Small here; not small for a pass that fans out.
4. **The price sheet has a 90-day staleness rule** (§1a) and no automated
check. Every number this capability produces inherits that.