clay-borg/workplans/CB-WP-0013-instrument-corrections.md

id	kind	title	status	state_hub_workstream_id
CB-WP-0013	meta	Two instruments that were wrong, and the record they wrote	done	b5adc29b-1a83-4958-8ca6-7be072dccacf

Purpose

structural tier  M   (changes how the loop constrains its own operation:
                      a budget instrument and a gate floor — InnerLoop
                      v1.6, ADR-0006 D5)
chaos            d4 = 3  → no override
declared tier    M

Declaration 8 of 12 in the calibration window. Tier M means the survey and the ADR merge into one document and the adversarial review is optional.

Meta budget: 0% of the trailing three, so this pass is in budget without the ADR-0006 D2 repair exemption — though it would qualify for it, since both halves repair an instrument that reports a breach.

Why now

CB-WP-0012 ended owing two corrections, and one of them turns out to be larger than it was filed as.

SH-3 has never read what six evidence files say it read

CB-EV-0010 §5 states "SH-3 has now read 0.0% for six consecutive passes" and calls it the oldest unargued number in the project. The number is an artifact.

--shape-budget measures the window since the last commit, and it is read while writing the evidence file — immediately after a commit, when that window holds one or two responses. SH-1 and SH-2 are location statistics and survive a sample of two. SH-3 is a rate, and at n=2 its only possible values are 0%, 50% and 100%. One window, three metrics, and it is only wrong for one of them.

Measured over real per-pass windows instead:

window since	responses	SH-3
0d2ab22	115	6.2%
6fb0aea	113	6.3%
331e7e9	95	1.1%
c5fa610	74	1.4%
883b608	70	1.4%
84d6886	24	4.3%

So the substantive claim survives — SH-3 is badly unmet — but the figure quoted in six evidence files was never the measurement. Worse, the real numbers say something the artifact hid: the historical rate pinned in specs/SessionShape.md is 7.8–8.6%, and these windows are 1.1–6.3%. Batching has got worse, and a metric stuck at a constant 0.0% could not show a trend at all.

AM-4a overstates every figure it has ever reported by 36%

ADR-0007 Decision 4, filed and unimplemented. AM-4a counts --edges normal, which includes proc-macro crates that run in the compiler and never reach a binary: 89,048 lines, 36.2% of the shipped-runtime figure, syn alone 66,916.

Task: fix SH-3's instrument, and correct the record

id: CB-WP-0013-T01
status: done
priority: high
state_hub_task_id: "0e2b9dc1-d3e7-4833-b03c-24ff987e5b95"

A rate needs a sample. --shape-budget must refuse to report SH-3 below a stated minimum rather than print a number the window cannot support — insufficient sample (n responses), not 0.0%.

The choice to argue, not assume: whether SH-3 should instead be windowed differently from SH-1/SH-2 — over the pass rather than since the last commit — since it is the only one of the three that a two-response window destroys. Pick one and say why the other was rejected.

Controls. A minimum-sample rule is exactly the kind of guard that can be satisfied vacuously: assert that a window above the floor still reports a rate, that one below reports the refusal, and that the refusal is distinguishable from a genuine 0.0%. Mutate each.

Correct the record. Six evidence files quote 0.0%. They are history and are not rewritten; the correction is recorded once, where a reader will find it, and named as what it is — the same class as CB-WP-0002's "0 of 330 tool calls were batched", which specs/SessionShape.md §4 already records as the eighth trusted-arithmetic instance. This is the ninth, and it is in the same metric. That coincidence is worth a sentence: SH-3 is the number this project keeps getting wrong.

Done 2026-08-02. cb-cost.py gains sh3_line(): below a minimum sample it prints insufficient sample and no verdict.

The window was kept, not split. SH-3 could instead have been given a per-pass window while SH-1/SH-2 kept the since-last-commit one. Rejected: the budget's stated purpose is the open remainder since the last commit — the only thing a live budget can fire on — and giving one of three metrics a different window makes "the window" ambiguous in a tool that three specs cite. Refusing to report is honest and keeps one window.

The floor is derived, not round. If the true rate were exactly the 20% target, the chance of observing zero batched turns in n responses is 0.8^n. At n = 14 that is 4.4%, so "0 batched in 14" rules out a target-meeting rate at ~95%. Below 14 the tool has nothing to say.

Four controls, three mutations, each red for its stated reason:

mutation	result
the guard never refuses	SH-3 refuses a window too small red, and a refusal is distinguishable from a genuine 0.0% red
the guard always refuses	reporting at exactly the minimum, and a real 40% rate, both red
the refusal prints as 0.0%	refuses a window too small red — the case the six evidence files actually hit

The correction is recorded in specs/SessionShape.md §4, beside the eighth instance, with the real per-pass figures.

Task: argue SH-3's floor, or move it

id: CB-WP-0013-T02
status: done
priority: high
state_hub_task_id: "c926c370-26fd-4450-aeee-bc2dff69cdbc"

The declaration CB-EV-0010 said was owed. specs/SessionShape.md is explicit that the 20% floor was set above the measured rate deliberately and that "whether 20% is reachable or arbitrary is unknown until a pass tries". Seven passes have now reported it and none has tried.

Decide, with the real numbers rather than the artifact:

• is 20% reachable, and at what cost to how the work is actually done;
• is the measured decline from 7.8–8.6% to 1.1–6.3% a behavioural change worth reversing, or an artifact of tier-L passes doing more sequential reading than tier-S ones;
• and the honest option: retire SH-3. specs/SessionShape.md §4 puts the ceiling at $2–4 on a $93 pass and says "batch because it is free, not because it is the lever." A floor that is 3–20× the measured rate, cannot be moved by the party measured, and is worth single-digit dollars is a candidate for retire_if, not a standing breach.

Do not retarget to match the measurement. specs/SessionShape.md already forbids exactly that, and CB-WP-0003 T07 exists to prevent it. If the floor moves, it moves on an argument about what batching is worth, and the argument is written down before the number changes.

Done 2026-08-02. ADR-0008 D1 — SH-3 is retired as a gate and kept as a diagnostic.

Investigating it turned up a third defect, deeper than the two this workplan was declared on. Measured by re-deriving batching from the raw transcripts, independently of cb-cost:

window	with tools	batched	rate
CB-WP-0011 pass	54	0	0.0%
gap → next declaration	16	6	37.5%
CB-WP-0012 pass	86	0	0.0%
gap → next declaration	10	1	10.0%
CB-WP-0013 so far	10	0	0.0%

Zero batched turns in 150 in-pass responses; 37.5% in one gap. Batching needs two calls whose inputs are known at once — orientation work. Implementation consumes each step's result before the next. And SH-3's window is since the last commit, which during a pass is always implementation. The metric could not read above ~0% in the window it was gated on.

And this workplan's own Purpose was wrong. It stated batching "has got worse" (7.8–8.6% historical against 1.1–6.3% per-pass). Those come from differently-placed windows, not different behaviour. Withdrawn — the same class of error, in the pass written to correct it.

The retirement is not retargeting to match the measurement: the floor was not moved to 6%, the gate was removed on an argument about what the quantity is worth, established before the disposition. The number is still reported; only the verdict is gone.

Task: correct AM-4a to count what ships

id: CB-WP-0013-T03
status: done
priority: high
state_hub_task_id: "51265ba6-2517-49d0-b842-d9a73016124d"

ADR-0007 Decision 4. tools/dep-weight.py counts --edges normal,no-proc-macro.

The targets do not move with the instrument. Correcting a metric that reports 246,250 down to 157,202 while leaving the target at 250,000 hands this project 89,048 lines of headroom it did not earn, in the same commit that reveals the error. Either the target moves down by the same amount the instrument does, or the pass states plainly why the extra headroom is legitimate. It is not obviously legitimate; argue it or give it back.

Positive control. dep-weight.py already has the guard that an unlocatable crate must not silently count as zero. The new flag must not be able to drop crates it should have counted: assert that the proc-macro set is exactly the five expected crates and that removing the flag restores the old figure.

Done 2026-08-02. ADR-0008 D2/D3.

shipped-runtime   18 crates   157,202 lines   [ok  target 161,000]

The target moved down with the instrument, 250,000 → 161,000, so the correction hands back essentially nothing: headroom goes from 3,750 to 3,798. 161,000 keeps ~2.4% of room where 250,000 kept ~1.5%; that small rounding up is the only thing given back, because a target with 1.5% of room fails on a dependency's patch release.

Three controls, all green: the exclusion drops exactly the five expected crates, only ever removes and never adds, and is not a no-op.

The DFD gate then caught the follow-on, which is what it is for: three historical documents carried live fact:am4a_loc tags for a number that had changed. They are history and were not rewritten — the tags were removed and both documents carry a supersession banner instead.

AM-4b is deliberately not corrected. Its proc-macro share is unmeasured, and correcting a second instrument on the strength of the first one's ratio is the error this task exists to fix. Filed as owed.

Task: evidence

id: CB-WP-0013-T04
status: done
priority: medium
state_hub_task_id: "e9b20874-692a-48df-b3ee-c2929d7fe5a8"

evidence/CB-EV-0011-instrument-corrections.md.

Carry: whether tier M's merged survey-and-ADR was the right weight for two instrument corrections; the cost of a meta pass immediately after two product passes; and the standing question of whether three consecutive evidence files reporting their own pass's cost low means the loop should simply stop quoting it.

Also carry the one that is now overdue: cb-play is three modes in one binary. CB-EV-0009 §5 named the third mode as the trigger to reconsider the shape, CB-EV-0010 §6 recorded that it had arrived and that the pass did not act. Either do it or state why not, and stop carrying it.

Done 2026-08-02. CB-EV-0011.

• The self-quoting rule is proposed. Three passes have now reported their own cost mid-flight and all three were low by ~2/3 — and this time it changed a conclusion: CB-EV-0010's tier comparison said L cost 24% more than S; the final figures say 51%.
• cb-play's third mode never arrived. --serve is a play flag selecting which Policy fills a human seat. The binary still has two modes. CB-EV-0010 §6 was wrong; the item is closed, not deferred.
• A near-miss recorded: an instrument reading that looked impossible turned out to be correct, and one independent re-derivation before writing anything stopped a false accusation.
• Tier M was the right weight — one merged document, no separate review, and the pass still found a defect deeper than the one it was declared on, because the finding came from measuring rather than from an extra review round.