tegwick
53c460c992
Duplicated-fact drift is the fourth error class and the only one with no executable rule. No positive control catches it (both copies are internally consistent) and re-derivation does not either (the copy reproduces whatever it was copied from). It is caught only by reading a copy against its source, which nothing in the loop required. facts.toml holds 15 facts and is GENERATED by `make facts-gen` from cb-cost, dep-weight and rule-coverage. The trap this task named — a hand-maintained registry that becomes another drifting copy — is closed by facts-check re-running the instruments and failing when the committed registry disagrees with them. A stale registry cannot certify stale artifacts. An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key. 17 occurrences across 5 artifacts are now checked. Falsified before being believed: changing CostAccounting.md line 158 from $93.15 to $92.87 — the exact historical drift — produced exit 1 naming the file, the line and the expected value. Tested against the class it exists to catch, on a real artifact, not only in its self-test. It then caught a live tag inside its own documentation example in InnerLoop.md within the hour. Third time a gate has failed on its own pass's work. What it does not close is stated rather than implied: 22 untagged literal copies remain and are reported, not failed. Tagging is opt-in, a number can legitimately recur, and a gate that fires on coincidence gets routed around. Naming the uncovered surface beats claiming the class is closed. InnerLoop single-source-of-fact moves from prose to executable — v1.3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
7.2 KiB
CB-EV-0002: cost accounting
capability: meta.loop.cost-accounting
spec: CostAccounting.md v1.0
decision: ADR-0003
research: CB-RES-0002
instrument: make cost-pin (tools/cb-cost.py --pin fc76445)
measured: 2026-07-31
Subject of measurement: the CB-WP-0001 session, pinned at commit fc76445
(2026-07-31T02:17:59Z). All numbers below are emitted by the tool, not
transcribed by hand (CA-15).
1. Acceptance table
| ID | Metric | Target | Measured | Verdict |
|---|---|---|---|---|
| AC-1 | pinned total, as two components | $93.15 = $92.03 + $1.11 | $92.03 main + $1.11 subagent = $93.15 | met |
| AC-2 | reconciliation residual | $0.00 | $0.000000 | met |
| AC-3 | unattributed share reported | present, 33% | 32.4%, own line | met |
| AC-4 | composition reported | 5 components | 5 of 5 | met |
| AC-5 | dedup violation aborts | non-zero exit | abort raised | met |
| AC-6 | zero responses refuses to report | non-zero exit | 0 rows, no number emitted | met |
| AC-7 | subagent tree enumerated | >0 found | 2 of 4 transcripts | met |
| AC-8 | 5m cache priced at 1.25× | $1.25/100k @ fable | $1.2500 (1h would be $2.0000) | met |
| AC-9 | streamed partial output → final | 5,5,195 → 195 | 195 | met |
No unmeasured rows. AC-1's target was corrected four times before and after this
run; §4 records why, because the sequence is more useful than the final
number.
2. Composition — the finding the metric exists for
input 690 tok $ 0.00 0.0%
output 323,643 tok $ 11.13 11.9%
cache_read 80,611,798 tok $ 59.67 64.1%
write_5m 37,467 tok $ 0.47 0.5%
write_1h 1,672,854 tok $ 21.87 23.5%
TOTAL $ 93.15
88.0% of spend is cache; 11.9% is output. The ratio of context re-read
to text written is 249:1. A single total would have shown $93.15 and
concealed all of it — which is exactly what CB-WP-0001's M-D2-TOK would
have done, and why that metric is now demoted.
3. Per-task attribution
UNATTRIBUTED $ 30.14 32.4%
T08 $ 21.02 22.6%
T07 $ 12.01 12.9%
T03 $ 9.91 10.6%
T04 $ 8.00 8.6%
T05 $ 7.19 7.7%
T06 $ 3.58 3.8%
T09 $ 1.29 1.4%
Stated limit (CA-10): 32.4% of cost sits in commits whose subject
carries no T## tag, so this table is a view over 67.5% of spend. That is
a property of commit hygiene, not of the collector.
T08 (the GROUND aggregate, six code iterations) at $21.02 is the most
expensive task and was also the one that produced the most rework — the
tuple-map hash panic, the discarded setup.patch, and the 5.6×-wrong
benchmark all originated there. Expensive and error-dense: the correlation
is worth watching, not yet a conclusion from n=1.
4. What it does not support
- The per-task figures are not comparable across passes. They mix models (opus/fable/sonnet) at different price points and different cache states. The dollar figure is comparable; a token count is not.
- AC-3's 32.4% is a fixture pin, not a quality target. Improving commit tagging will move it, and that is the desired direction.
- This is one session. Every ratio here (cache share, $/turn, the compaction effect in §5) is n=1 and should be treated as a hypothesis until a second pass reproduces it.
The price sheet cannot express a time-boxed rate.Fixed 2026-07-31 by CB-WP-0003 T11 (CA-16/CA-17). Applying the sonnet promotional rate moved AC-1 from $93.32 to $93.15 — the $0.17 this section predicted, now collected rather than merely noted.- AC-1 is not invoice-verified. No admin key exists, so the Anthropic billing API could not independently confirm the total. The transcript counters are the same ones billing uses, but that is an argument, not a reconciliation.
5. The question that could not be answered before
"What does /compact actually cost, and is a long session quadratic?"
CB-WP-0003 T04 asserts that cost ≈ turns × mean_context and that mean_context grows with turns, making long sessions quadratic. The data qualifies that claim: it is true only between compactions.
The session compacted twice, and the transcript records both directly:
| compaction | pre-tokens | post-tokens | reduction |
|---|---|---|---|
C1 00:07:14Z (manual) |
542,991 | 19,974 | 27× |
C2 02:22:23Z (manual) |
344,954 | 19,035 | 18× |
Cost per turn across the boundary:
| segment | turns | mean context | total | $/turn |
|---|---|---|---|---|
| start → C1 | 136 | 304,178 | $62.19 | $0.457 |
| C1 → C2 | 202 | 193,493 | $30.01 | $0.149 |
The 202 turns after the first compaction cost less than half of the 136 turns before it — a 3.1× drop in cost per turn. Context growth is the cost driver, and compaction is the control on it. At the pre-compact rate, those 202 turns would have cost ~$92 instead of $30.
Two consequences for CB-WP-0003:
- T04's "one task per session" recommendation is not the only remedy, and may not be the cheapest one. Compaction achieved a 27× context reduction inside a running session at the cost of one summarization call. A fresh session pays a cold-start re-read of the committed artifacts; compaction pays a summary. Which is cheaper is now a measurable question rather than a matter of taste, and T04 should measure it before prescribing.
- The quadratic claim should be restated as bounded-quadratic: cost grows with context between compactions and resets at each one. The failure mode is not "a long session" but "a long uncompacted session".
6. Retrospective note
The positive control paid for itself on its first execution, which is the
strongest evidence this project has produced for the InnerLoop v1.0 rule
that added it. CA-02 asserted that usage is identical across the lines of
one requestId — verified twice on the main transcript, by the survey
(206/206 groups) and independently by the adversarial reviewer. It is false
in the subagents/ tree, where output_tokens is a running count
(5, 5, 195). The assertion fired, the run aborted, and the tool refused to
print a number. Under the prior first-wins rule it would have printed a
plausible one.
The generalization that failed is worth naming: a property verified on the largest sample was assumed to hold on the smallest one. The main transcript is 338 of 346 responses, so 206/206 felt conclusive; the violation lives entirely in the 8 responses nobody checked separately.
Cost of the adversarial review this pass: $1.11, against a $93.15 pass. It found three approval-blocking defects, one of which (the subagent exclusion) would have made this evidence file certify a broken collector. Second consecutive pass where a ~1% spend on review changed the outcome. That is now two data points for CB-WP-0003 T03.