CB-WP-0005 T07/T08: control loop, retrospective, InnerLoop v1.4

T07 — CB-EV-0004. 70 responses, $16.03.

Test 1 MET: 1 spec -> 2, 58 rules -> 76, 1 source file -> 10, and the new
denominator came in at 83% naming K10/K14/K18. The stated failure mode — a
widened denominator reporting the same percentage — did not occur.

Test 2 UNMET: M-D1-MUT 4 of 14 against >=10, and wrong about what as well
as how much. The diagnosis was three absent kernel rules; the measurement
found eight rows with no instrument at all.

Test 3: quality held, and widening surfaced far more than the seven known
defects — six further uninstrumented rows, HDN #7 (rule-coverage's
self-test green while the tool was broken), a stale $248.46 invisible to
facts-check because it was untagged, and a fifth error class.

The clean test CB-WP-0004 was owed is now run, on a pass that used the
tools without building them. The two categories whose tools removed the
manual path are at 0 turns two passes on; the two that merely offered a
better option are now the entire mechanical cost of a pass. Absolute
mechanical cost fell $51.76 -> $6.13. The mechanism holds.

Reported because nothing else would: SH-3 batching is 0.0% this pass — 67
tool calls across 67 responses — against a 20% target, and mean context
315,170 against 200,000. SessionShape has stated these since CB-WP-0003
and none has ever been enforced.

T08 — the retrospective answers its question: yes, a weak mutation is the
new grep, and it is worse, because it fails in the opposite direction.
mutation-check's first run produced two SURVIVED verdicts and both were
the author's own no-op mutations.

The fifth error class: false accusation. HDN, TA, SSB and DFD all
under-report — a real problem passes. FA over-reports: it publishes the
claim that working code is broken, sends the next pass to fix something
that is not broken, and is more credible than the truth because it arrives
with a measurement attached. Thirteen instances, five classes, six passes,
and the newest class is one that hardening created.

One correction to CB-WP-0004 T06: "a gate only pays if it removes the
manual path" is a predictor of whether a gate saves money, not a criterion
for whether it is worth having. mutation-check fails that test and
produced the most valuable findings of the pass.

InnerLoop v1.4: where a claim rests on numbers, the adversarial reviewer
must read the assertion behind each quoted number and mutate it.
Re-running the command that prints a number is not verification of that
number. Second verification step to inherit the author's blindness; both
fixes replace re-derivation with adversarial execution.

CB-WP-0005 status -> done, 5 of 8 tasks, 3 cancelled into CB-WP-0006.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

workplans/CB-WP-0005-assertion-coverage.md

@@ -1,7 +1,7 @@
 ---
 id: CB-WP-0005
 title: "Make the instruments count assertions, then fix what they expose"
-status: proposed
+status: done
 state_hub_workstream_id: "0b95a1e3-7780-43d0-81e9-072ef7978734"
 ---
 
@@ -84,23 +84,15 @@ kernel figures (`k_rules`, `k_linked`, `k_unlinked`) are registered facts,
 so they are under `make facts-check` from the day they first exist rather
 than after they drift.
 
-**The self-test passed while the tool was completely broken.** A
-`print(` inside `say()` was rewritten to `say(`, so every real
-`make coverage` died with `RecursionError` — while `--self-test` reported
-all-ok, because it only ever called `kernel_arm(quiet=True)` and never
-executed the reporting path.
+**The self-test passed while the tool was completely broken** — a
+`print(` inside `say()` became `say(`, so every real `make coverage` died
+with `RecursionError` while `--self-test` reported all-ok, because it only
+ever called the quiet path. The control named the behaviour and did not
+assert it: this task's own thesis in miniature, and HDN instance #7. Fixed
+by exercising the loud path, verified by re-breaking `say()`.
 
-That is this task's own thesis in miniature: **the control named the
-behaviour and did not assert it.** Fixed by exercising the loud path under
-`redirect_stdout` and asserting it prints, and verified by re-breaking
-`say()` and confirming the two new checks go red. Seventh instance of the
-harness-does-nothing shape, found in the tool written to find that shape.
-
-**A limit of `facts-check` surfaced here and is recorded, not patched:**
-the check is line-based, so a tagged value that prose-wraps onto the next
-line fails. It cost three edits to place two tags. Reported for T07 —
-either the checker spans a paragraph, or the rule is stated as
-"tagged values must not wrap".
+**A limit of `facts-check`, recorded not patched:** it is line-based, so a
+tagged value that prose-wraps fails. Three edits to place two tags.
 
 ## Task: M-D1-MUT — one mutation per acceptance row
 
@@ -174,13 +166,10 @@ compares any number to 100,000 events/s**, the headline throughput claim.
 The problem is not three unimplemented rules; it is that **more than half
 the acceptance table has nothing behind it.**
 
-Harness controls that earned their place: a stale find-string reports
-`HARNESS-BROKEN` rather than silently scoring the baseline as the mutant;
-a red baseline reports `inconclusive` rather than `red`; the tree is
-restored in a `finally` and the restoration is verified.
-
-`make mutation-check` is deliberately **not** in `make all` — it rebuilds
-the workspace once per mutated row. `--self-test` is in `make self-tests`.
+Harness controls: a stale find-string reports `HARNESS-BROKEN` rather than
+scoring the baseline as the mutant; a red baseline reports `inconclusive`;
+the tree is restored in a `finally` and verified. Not in `make all` — it
+rebuilds the workspace once per row.
 
 **Stop condition: see the note in T07 and the decision recorded there.**
 
@@ -314,6 +303,33 @@ Normalize per unit of work, as CB-WP-0004 T05 established: report share of
 pass alongside absolute figures, and use `cb-cost --since` to window this
 pass against the last.
 
+**Measured — [CB-EV-0004](../evidence/CB-EV-0004-assertion-coverage.md).**
+70 responses, **$16.03**.
+
+1. **Denominators widened, and the number moved.** 1 spec → 2, 58 rules →
+   76, 1 source file → 10, and the new denominator came in at **83%**
+   naming K10/K14/K18. The stated failure mode — a widened denominator
+   reporting the same percentage — did not occur.
+2. **M-D1-MUT 4 of 14 against ≥10. Unmet**, and wrong about *what* as well
+   as how much: the diagnosis was three absent kernel rules, the
+   measurement found eight rows with no instrument at all.
+3. **Quality held**, and widening surfaced far more than the seven known
+   defects: six further uninstrumented rows, HDN #7 (`rule-coverage`'s
+   self-test green while the tool was broken), a stale $248.46 that
+   `facts-check` could not see because it was untagged, and **a fifth
+   error class**.
+
+**The clean test CB-WP-0004 was owed, now run:** the two categories whose
+tools removed the manual path are at **0 turns** two passes on; the two
+that merely offered a better option are now the *entire* mechanical cost
+of a pass. Absolute mechanical cost fell $51.76 → $6.13. The mechanism
+holds.
+
+**A regression, reported because nothing else would:** SH-3 batching is
+**0.0%** (67 tool calls, 67 responses) against a 20% target, and mean
+context 315,170 against 200,000. SessionShape has stated these since
+CB-WP-0003 and none of them has ever been enforced.
+
 ## Task: retrospective and InnerLoop v1.4
 
 ```task
@@ -343,3 +359,40 @@ recovers capacity only where it removes the manual path. Does the same
 test predict which gates work?** `env-test` and `task-done` removed the
 manual path and held. Does a mutation gate remove the manual path — or is
 writing a weak mutation the new `grep`?
+
+**Answered — [260731-assertion-coverage-retrospective.md](../history/260731-assertion-coverage-retrospective.md).
+Yes, it is the new `grep`, and it is worse, because it fails in the
+opposite direction.**
+
+`mutation-check`'s first run produced two `SURVIVED` verdicts and both
+were the author's own no-op mutations — two of fourteen rows, first
+attempt, in a pass whose subject was inert assertions. Writing a weak
+mutation is exactly as easy as writing a strong one and the harness cannot
+tell them apart.
+
+**The fifth error class: false accusation (FA).** HDN, TA, SSB and DFD all
+under-report — a real problem passes. FA over-reports: it publishes the
+claim that working code is broken, sends the next pass to fix something
+that is not broken, and is *more* credible than the truth because it
+arrives with a measurement attached. Thirteen instances, five classes, six
+passes — and the newest class is one that hardening created.
+
+**What would bind:** each mutation records its expected failure message
+and the harness asserts the observed failure matches. A no-op fails that
+immediately, because it produces no failure at all. CB-WP-0006 T08 already
+requires it; T09 evaluates the stronger form — mutations written by
+someone other than the author of the assertion.
+
+**One correction to CB-WP-0004 T06**, stated rather than left implicit:
+"a gate only pays if it removes the manual path" is a **predictor of
+whether a gate saves money, not a criterion for whether it is worth
+having**. `mutation-check` fails that test and still produced the most
+valuable findings of the pass. T06 stated it too strongly; this pass is
+the counter-example.
+
+**InnerLoop v1.4** adopted: where a claim rests on numbers, the
+adversarial reviewer must read the assertion behind each quoted number and
+**mutate it**. Re-running the command that prints a number is not
+verification of that number. Second verification step to inherit the
+author's blindness; both fixes replace re-derivation with adversarial
+execution.