clay-borg/evidence/CB-EV-0011-instrument-corrections.md

CB-EV-0011 — three instruments, and a rule about quoting yourself

CB-WP-0013 T04. Measured 2026-08-02 at d2c3f34. Pass kind meta, tier M (chaos d4=3, no override). Declaration 8 of 12.

The pass was declared to fix two instruments. It found a third defect in one of them, and a fourth in itself.

---

1. What was actually wrong with SH-3

The declaration named one defect. There were three, each deeper.

1. The sample. --shape-budget windows since the last commit and is read while writing the evidence file — right after a commit, when the window holds one or two responses. SH-1/SH-2 are location statistics and survive that; SH-3 is a rate, and at n=2 its only values are 0%, 50%, 100%. Six evidence files quoted the resulting 0.0% as a measurement.

2. The claimed decline — which was this pass's own error. The declaration stated batching "has got worse", citing 7.8–8.6% pinned history against 1.1–6.3% per-pass windows. Those are differently-placed windows, not different behaviour. Withdrawn. The pass declared to correct a trusted-arithmetic error opened by committing one.

3. The one that decided it. Re-deriving batching from the raw transcripts, independently of cb-cost:

window	with tools	batched	rate
CB-WP-0011 pass	54	0	0.0%
gap → next declaration	16	6	37.5%
CB-WP-0012 pass	86	0	0.0%
gap → next declaration	10	1	10.0%
CB-WP-0013 so far	10	0	0.0%

Zero batched turns in 150 in-pass responses. 37.5% in one gap — above the floor the gate had been failing for seven passes.

Batching needs two tool calls whose inputs are known at once. That is orientation: read four files, run three probes. Implementation has almost none, because each step consumes the previous step's result. And SH-3's window is since the last commit, which during a pass is implementation. The metric could not read above ~0% in the window it was gated on.

A gate meetable only by not working is not a target. SH-3 is retired as a gate (ADR-0008 D1) and kept as a diagnostic over the whole session, where it reads 5.8%.

On the appearance of burying it: retiring a metric that shows a regression looks bad, and should. The answer is defect 2 — the regression was an artifact and has been withdrawn, so there is no established decline to bury. The number is still printed; only the verdict is gone. And the floor was not moved to 6%, which is the thing specs/SessionShape.md actually forbids.

2. A near-miss worth recording: I almost reported a defect that was not there

Midway through, a bounded window reported 53 tool calls in 53 responses; 0 in batched turns — exactly one call per response, which looked like the counter had collapsed. I had batched repeatedly in this session, so the reading looked impossible.

It was correct. Re-deriving from the transcripts independently gave the same 0, and the 131 batched responses that do exist all fall outside those windows. The instrument was right and my expectation was wrong.

Recorded because the false-accusation class is already on this project's record, and because the thing that stopped it was cheap: one independent re-derivation before writing anything down. The cost of checking was two minutes; the cost of not checking would have been an evidence file alleging a defect in a tool that did not have one — and this pass's whole subject is instruments reporting things that are not so.

3. AM-4a, corrected, gives back nothing

before   23 crates   246,250 lines   [ok  target 250,000]   headroom 3,750
after    18 crates   157,202 lines   [ok  target 161,000]   headroom 3,798

The instrument dropped 89,048 lines of proc-macro code that never reaches a binary; the target moved down with it (ADR-0008 D3). Leaving the target at 250,000 would have handed this project 89,048 lines of headroom it did not earn, in the same change that revealed the error.

Three controls, all green: the exclusion drops exactly the five expected crates, only ever removes and never adds, and is not a no-op.

The DFD gate then caught the follow-on, which is exactly what it is for: three historical documents carried live fact:am4a_loc tags for a number that had changed. They are history and were not rewritten — the tags came off and both carry a supersession banner. A gate catching the second-order consequence of a correction, unprompted, is the best evidence yet that the registry is doing something.

AM-4b is deliberately not corrected. Its proc-macro share is unmeasured, and correcting a second instrument on the strength of the first one's ratio is the error this pass exists to fix. Owed.

4. The rule this pass should leave behind: stop quoting your own cost

pass	reported mid-pass	final	understated by
CB-WP-0009	$6.73 (CB-EV-0008)	$11.31	68%
CB-WP-0011	$4.23 / 0.094 (CB-EV-0009)	$7.02 / 0.099	66%
CB-WP-0012	$8.82 / 0.123 (CB-EV-0010)	$14.52 / 0.150	65%

Three for three, always low, by a strikingly consistent ~2/3. And this time it changed a conclusion, not just a figure:

CB-EV-0010 §5 reported the first priced tier comparison as "0.123 $/response at L against 0.099 at S — 24% more." The final figures are 0.150 against 0.099 — 51% more. The direction holds; the magnitude was understated by half.

Tier L is twice as expensive as that comparison claimed. It still found errors of 25× and 85× in CB-WP-0012 and deleted a port that would have been rewritten, so it still looks worth it — but the price of the argument doubled and the evidence file that made it could not have known.

Proposed rule, for the next pass to adopt or reject: an evidence file quotes the previous pass's final cost and does not quote its own. Its own is knowable only after the commit that closes it, so any figure it prints is a floor, and this project has now printed three of them and let one carry a conclusion.

5. cb-play's third mode never arrived

CB-EV-0009 §5 said a third mode would be the second use at which cb-play's single-binary shape should be reconsidered. CB-EV-0010 §6 recorded that it had arrived, with --serve, and that the pass had not acted. CB-EV-0010 was wrong.

--serve is registered in parse_args as a play flag: it selects which Policy fills a human seat, terminal or browser. The binary still has exactly two modes — play a game, or read a recording — and they still take disjoint flags. The trigger did not fire.

Carried for two passes, resolved here, and stopped: there is no split to do, and the standing item is closed rather than deferred again.

6. Cost and shape

pass	kind	responses	cost	$/response
CB-WP-0011	product	71	$7.02	0.099
CB-WP-0012	product	97	$14.52	0.150
CB-WP-0013	meta	34+	$5.78+	provisional, see §4

Meta budget: 21%, inside the soft 25%, without needing ADR-0006 D2's repair exemption — though both halves would have qualified for it.

SH-1 290,475 [SOFT], drifting toward the hard line; the remedy is a compaction and it is free. SH-3 now reads insufficient sample (1 response with tool calls, need 14) — the fix working in production on its first real reading, where the old instrument would have printed a confident 0.0%.

gate-review: 0 due, 0 silent, 0 drifted. Every gate in the registry now has a caught entry, for the first time. GATE-REVIEW earned its own by forcing SH-3's re-justification — asking what a gate had ever caught is what exposed that it could not read what it gated on.

7. Open

• AM-4b's proc-macro share is unmeasured. §3.
• The self-quoting rule in §4 is proposed, not adopted.
• AM-4a still cannot survive INTENT stage 2, and this pass made the ratio worse: wgpu + winit is 1,741,979 against a target now 161,000 — 10.8×, up from 7×. Reserved for the maintainer, twice over now.
• ADR-0007 Decision 3's acquisition rule remains an argument awaiting ratification.
• INTENT stage 1 stays open; the emitted JavaScript has still never been executed.
• Tier M's merged survey-and-ADR was the right weight. One document, no separate review, and the pass still found a defect deeper than the one it was declared on — because the finding came from measuring, not from an extra review round. That is one data point for the tier table and against the assumption that depth comes from process weight.