clay-borg/evidence/CB-EV-0011-instrument-corrections.md

# CB-EV-0011 — three instruments, and a rule about quoting yourself

CB-WP-0013 T04. Measured 2026-08-02 at `d2c3f34`. Pass kind `meta`,
tier **M** (chaos d4=3, no override). Declaration 8 of 12.

The pass was declared to fix two instruments. It found a third defect in
one of them, and a fourth in itself.

---

## 1. What was actually wrong with SH-3

The declaration named one defect. There were three, each deeper.

**1. The sample.** `--shape-budget` windows since the last commit and is
read while writing the evidence file — right after a commit, when the
window holds one or two responses. SH-1/SH-2 are location statistics and
survive that; SH-3 is a rate, and at n=2 its only values are 0%, 50%,
100%. Six evidence files quoted the resulting `0.0%` as a measurement.

**2. The claimed decline — which was this pass's own error.** The
declaration stated batching *"has got worse"*, citing 7.8–8.6% pinned
history against 1.1–6.3% per-pass windows. Those are differently-*placed*
windows, not different behaviour. **Withdrawn.** The pass declared to
correct a trusted-arithmetic error opened by committing one.

**3. The one that decided it.** Re-deriving batching from the raw
transcripts, independently of `cb-cost`:

| window | with tools | batched | rate |
|---|---:|---:|---:|
| CB-WP-0011 pass | 54 | 0 | **0.0%** |
| gap → next declaration | 16 | 6 | **37.5%** |
| CB-WP-0012 pass | 86 | 0 | **0.0%** |
| gap → next declaration | 10 | 1 | **10.0%** |
| CB-WP-0013 so far | 10 | 0 | **0.0%** |

**Zero batched turns in 150 in-pass responses. 37.5% in one gap** — above
the floor the gate had been failing for seven passes.

Batching needs two tool calls whose inputs are known at once. That is
orientation: read four files, run three probes. Implementation has almost
none, because each step consumes the previous step's result. And SH-3's
window is *since the last commit*, which during a pass is implementation.
**The metric could not read above ~0% in the window it was gated on.**

A gate meetable only by not working is not a target. SH-3 is retired as a
gate (ADR-0008 D1) and kept as a diagnostic over the whole session, where
it reads 5.8%.

**On the appearance of burying it:** retiring a metric that shows a
regression looks bad, and should. The answer is defect 2 — the regression
was an artifact and has been withdrawn, so there is no established decline
to bury. The number is still printed; only the verdict is gone. And the
floor was *not* moved to 6%, which is the thing `specs/SessionShape.md`
actually forbids.

## 2. A near-miss worth recording: I almost reported a defect that was not there

Midway through, a bounded window reported `53 tool calls in 53 responses;
0 in batched turns` — exactly one call per response, which looked like the
counter had collapsed. I had batched repeatedly in this session, so the
reading looked impossible.

It was correct. Re-deriving from the transcripts independently gave the
same 0, and the 131 batched responses that do exist all fall *outside*
those windows. The instrument was right and my expectation was wrong.

Recorded because the **false-accusation class** is already on this
project's record, and because the thing that stopped it was cheap: one
independent re-derivation before writing anything down. The cost of
checking was two minutes; the cost of not checking would have been an
evidence file alleging a defect in a tool that did not have one — and this
pass's whole subject is instruments reporting things that are not so.

## 3. AM-4a, corrected, gives back nothing

```
before   23 crates   246,250 lines   [ok  target 250,000]   headroom 3,750
after    18 crates   157,202 lines   [ok  target 161,000]   headroom 3,798
```

The instrument dropped 89,048 lines of proc-macro code that never reaches
a binary; **the target moved down with it** (ADR-0008 D3). Leaving the
target at 250,000 would have handed this project 89,048 lines of headroom
it did not earn, in the same change that revealed the error.

Three controls, all green: the exclusion drops **exactly** the five
expected crates, only ever removes and never adds, and is not a no-op.

**The DFD gate then caught the follow-on**, which is exactly what it is
for: three historical documents carried live `fact:am4a_loc` tags for a
number that had changed. They are history and were not rewritten — the
tags came off and both carry a supersession banner. A gate catching the
second-order consequence of a correction, unprompted, is the best evidence
yet that the registry is doing something.

**AM-4b is deliberately not corrected.** Its proc-macro share is
unmeasured, and correcting a second instrument on the strength of the
first one's ratio is the error this pass exists to fix. Owed.

## 4. The rule this pass should leave behind: stop quoting your own cost

| pass | reported mid-pass | final | understated by |
|---|---|---|---|
| CB-WP-0009 | $6.73 (CB-EV-0008) | $11.31 | 68% |
| CB-WP-0011 | $4.23 / 0.094 (CB-EV-0009) | $7.02 / 0.099 | 66% |
| CB-WP-0012 | $8.82 / 0.123 (CB-EV-0010) | **$14.52 / 0.150** | 65% |

Three for three, always low, by a strikingly consistent ~2/3. And this
time it **changed a conclusion**, not just a figure:

> CB-EV-0010 §5 reported the first priced tier comparison as *"0.123
> $/response at L against 0.099 at S — 24% more."* The final figures are
> **0.150 against 0.099 — 51% more.** The direction holds; the magnitude
> was understated by half.

Tier L is twice as expensive as that comparison claimed. It still found
errors of 25× and 85× in CB-WP-0012 and deleted a port that would have
been rewritten, so it still looks worth it — but the price of the argument
doubled and the evidence file that made it could not have known.

**Proposed rule, for the next pass to adopt or reject:** an evidence file
quotes the *previous* pass's final cost and does not quote its own. Its
own is knowable only after the commit that closes it, so any figure it
prints is a floor, and this project has now printed three of them and let
one carry a conclusion.

## 5. `cb-play`'s third mode never arrived

CB-EV-0009 §5 said a third mode would be the second use at which
`cb-play`'s single-binary shape should be reconsidered. CB-EV-0010 §6
recorded that it had arrived, with `--serve`, and that the pass had not
acted. **CB-EV-0010 was wrong.**

`--serve` is registered in `parse_args` as a *play flag*: it selects which
`Policy` fills a human seat, terminal or browser. The binary still has
exactly two modes — play a game, or read a recording — and they still take
disjoint flags. The trigger did not fire.

Carried for two passes, resolved here, and **stopped**: there is no split
to do, and the standing item is closed rather than deferred again.

## 6. Cost and shape

| pass | kind | responses | cost | $/response |
|---|---|---|---|---|
| CB-WP-0011 | product | 71 | $7.02 | 0.099 |
| CB-WP-0012 | product | 97 | $14.52 | 0.150 |
| CB-WP-0013 | meta | 34+ | $5.78+ | provisional, see §4 |

**Meta budget: 21%**, inside the soft 25%, without needing ADR-0006 D2's
repair exemption — though both halves would have qualified for it.

**SH-1 290,475 `[SOFT]`**, drifting toward the hard line; the remedy is a
compaction and it is free. **SH-3 now reads `insufficient sample (1
response with tool calls, need 14)`** — the fix working in production on
its first real reading, where the old instrument would have printed a
confident `0.0%`.

**`gate-review`: 0 due, 0 silent, 0 drifted.** Every gate in the registry
now has a `caught` entry, for the first time. GATE-REVIEW earned its own
by forcing SH-3's re-justification — asking what a gate had ever caught is
what exposed that it could not read what it gated on.

## 7. Open

- **AM-4b's proc-macro share is unmeasured.** §3.
- **The self-quoting rule in §4 is proposed, not adopted.**
- **AM-4a still cannot survive INTENT stage 2**, and this pass made the
  ratio worse: `wgpu` + `winit` is 1,741,979 against a target now 161,000
  — **10.8×**, up from 7×. Reserved for the maintainer, twice over now.
- **ADR-0007 Decision 3's acquisition rule** remains an argument awaiting
  ratification.
- **INTENT stage 1 stays open**; the emitted JavaScript has still never
  been executed.
- **Tier M's merged survey-and-ADR was the right weight.** One document,
  no separate review, and the pass still found a defect deeper than the
  one it was declared on — because the finding came from *measuring*, not
  from an extra review round. That is one data point for the tier table
  and against the assumption that depth comes from process weight.