From fd19f4e878b952b5ec97c9a650e8e0ddbb7925e6 Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 31 Jul 2026 18:05:31 +0200 Subject: [PATCH] CB-WP-0005 T07/T08: control loop, retrospective, InnerLoop v1.4 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit T07 — CB-EV-0004. 70 responses, $16.03. Test 1 MET: 1 spec -> 2, 58 rules -> 76, 1 source file -> 10, and the new denominator came in at 83% naming K10/K14/K18. The stated failure mode — a widened denominator reporting the same percentage — did not occur. Test 2 UNMET: M-D1-MUT 4 of 14 against >=10, and wrong about what as well as how much. The diagnosis was three absent kernel rules; the measurement found eight rows with no instrument at all. Test 3: quality held, and widening surfaced far more than the seven known defects — six further uninstrumented rows, HDN #7 (rule-coverage's self-test green while the tool was broken), a stale $248.46 invisible to facts-check because it was untagged, and a fifth error class. The clean test CB-WP-0004 was owed is now run, on a pass that used the tools without building them. The two categories whose tools removed the manual path are at 0 turns two passes on; the two that merely offered a better option are now the entire mechanical cost of a pass. Absolute mechanical cost fell $51.76 -> $6.13. The mechanism holds. Reported because nothing else would: SH-3 batching is 0.0% this pass — 67 tool calls across 67 responses — against a 20% target, and mean context 315,170 against 200,000. SessionShape has stated these since CB-WP-0003 and none has ever been enforced. T08 — the retrospective answers its question: yes, a weak mutation is the new grep, and it is worse, because it fails in the opposite direction. mutation-check's first run produced two SURVIVED verdicts and both were the author's own no-op mutations. The fifth error class: false accusation. HDN, TA, SSB and DFD all under-report — a real problem passes. FA over-reports: it publishes the claim that working code is broken, sends the next pass to fix something that is not broken, and is more credible than the truth because it arrives with a measurement attached. Thirteen instances, five classes, six passes, and the newest class is one that hardening created. One correction to CB-WP-0004 T06: "a gate only pays if it removes the manual path" is a predictor of whether a gate saves money, not a criterion for whether it is worth having. mutation-check fails that test and produced the most valuable findings of the pass. InnerLoop v1.4: where a claim rests on numbers, the adversarial reviewer must read the assertion behind each quoted number and mutate it. Re-running the command that prints a number is not verification of that number. Second verification step to inherit the author's blindness; both fixes replace re-derivation with adversarial execution. CB-WP-0005 status -> done, 5 of 8 tasks, 3 cancelled into CB-WP-0006. Co-Authored-By: Claude Opus 5 --- evidence/CB-EV-0004-assertion-coverage.md | 168 ++++++++++++++++++ ...260731-assertion-coverage-retrospective.md | 145 +++++++++++++++ specs/InnerLoop.md | 23 ++- workplans/CB-WP-0005-assertion-coverage.md | 101 ++++++++--- 4 files changed, 411 insertions(+), 26 deletions(-) create mode 100644 evidence/CB-EV-0004-assertion-coverage.md create mode 100644 history/260731-assertion-coverage-retrospective.md diff --git a/evidence/CB-EV-0004-assertion-coverage.md b/evidence/CB-EV-0004-assertion-coverage.md new file mode 100644 index 0000000..15643a9 --- /dev/null +++ b/evidence/CB-EV-0004-assertion-coverage.md @@ -0,0 +1,168 @@ +# CB-EV-0004: did widening the instruments change what they report? + +research: [CB-RES-0004](../research/CB-RES-0004-replay-and-kernel-coverage.md) +adr: [ADR-0005](../decisions/ADR-0005-assertion-coverage-and-replay.md) +workplan: [CB-WP-0005](../workplans/CB-WP-0005-assertion-coverage.md) +instruments: `make coverage`, `make mutation-check`, `make cost-mix` +window: `cb-cost --since bd4423a` — 70 responses, **$16.03** + +Phase C was deferred unstarted, so this closes T01–T03 only. That is +itself the headline result: **the pass stopped because its own instrument +contradicted the plan it was executing.** + +--- + +## Test 1 — did the denominators widen? + +| | before | after | +|---|---|---| +| specs in scope | 1 (`GroundRules.md`) | **2** (+ `GameKernel.md`) | +| numbered rules in scope | 58 | **76** | +| source files searched for links | 1 | **10** | +| denominators reported | 1 | **3** | + +```text +AM-1 rule coverage: 58/58 (100%) over 21 scenarios +AM-1b spec->code link: 49/58 claimed rules named in the aggregate +AM-1b kernel spec->code link: 15/18 (83%) across 10 source files + unlinked: K10 K14 K18 +``` + +**Met.** The workplan's stated failure mode — *a widened denominator that +reports the same percentage would prove the instruments still count +names* — did not occur. The new denominator came in at **83%**, and named +three rules that four workplans of green gates never mentioned. + +The gate reports without feeding the exit code **until 2026-08-31**, then +binds. The date is in `tools/rule-coverage.py`, the days remaining print +on every run, and the self-test asserts the arm returns 0 before that date +and 2 after — so it cannot quietly become never, which is how AM-4's +targets went unratified for four workplans. + +## Test 2 — M-D1-MUT against the prediction + +**Prediction ≥10 of 14 (the ADR's 9-of-12, 75%). Measured 4. UNMET.** +No target moved in the commit that measured it. + +```text + M-D1-MUT: 4/14 rows enforced + PARTIAL 2 AM-7, AM-8 — some clauses live, some inert + unmutatable 8 no property to invert + SURVIVED 0 +``` + +**Two corrections to our own numbers**, both recorded rather than quietly +absorbed: + +1. **There are 14 acceptance rows, not 12.** ADR-0005 and CB-WP-0005 both + said twelve; AM-4 splits into a/b/c. The prediction is evaluated as + ≥10 of 14 on the same 75% basis. +2. **The first run reported two `SURVIVED` rows and both were my own + no-op mutations.** `pub struct NullRng;` → `pub struct NullRng {}` is + semantically identical; renaming `max_age_days` changes nothing + because CA-17 reads it with `.get(..., 90)`. Replaced with real + inversions, after which both go red. + +The eight `unmutatable` rows, each with the reason the harness records: + +| row | why nothing can be inverted | +|---|---| +| **AM-6** | **nothing compares any number to 100,000 events/s** — the headline throughput claim | +| AM-2 | no instrument divides LOC by rule count or compares to 40 | +| AM-3 | the synthetic workload's LOC is never measured | +| AM-4c | `reported, not targeted` — no threshold, so nothing can fail | +| AM-5 | `recorded not gated`, and not recorded either | +| AM-9 | nothing measures resident memory | +| AM-10 | population empty — no `cb-*-api` crate exists | +| AM-11 | the conformance suite the metric is a bool over does not exist | + +**The prediction was wrong about *what*, not only *how much*.** +CB-RES-0004 diagnosed three absent kernel rules. The measurement found +that **more than half the acceptance table has no instrument at all** — +which Phase C, scoped to five rules, would not have touched. + +## Test 3 — did quality hold, and did widening surface anything new? + +`make all` green throughout, now including two gates that did not exist: +the kernel coverage arm and `mutation-check --self-test`. + +**It surfaced substantially more than the seven defects CB-RES-0004 +named.** Six further acceptance rows with no instrument (above), plus +three new defect instances: + +- **HDN #7** — `rule-coverage.py`'s `--self-test` printed all-ok while + every real `make coverage` died with `RecursionError`. A `print(` inside + `say()` had become `say(`; the control only ever called the quiet path. + *The control named the behaviour and did not assert it* — the pass's own + thesis, inside the tool written to prove it. +- **A new error class (below)** — the two no-op mutations. +- **DFD** — `evidence/CB-EV-0001` still carried **$248.46** for AM-12, the + figure CB-WP-0002 disproved four workplans ago. Untagged, and therefore + invisible to `facts-check`. The gate built to catch duplicated facts + only checks copies that opted in. + +### The fifth error class: false accusation (FA) + +Every class on record under-reports: a harness that does nothing, trusted +arithmetic, a blind sample, a stale copy. All four let a real problem pass. + +The weak mutation is the first that **over-reports**. A no-op mutation +yields `SURVIVED`, which reads as *"this row asserts nothing"* — a +published claim that working code is broken. It sends the next pass to fix +something that is not broken, and it is *more* credible than the truth +because it arrives with a measurement attached. + +| class | direction | caught by | +|---|---|---| +| HDN, TA, SSB, DFD | under-report — a real defect passes | assertions, re-derivation, all-data checks, reading the copy | +| **FA** *(new)* | **over-report — working code is indicted** | checking that the mutation fails *for its stated reason* | + +Ten instances across four classes became **thirteen across five**. + +## Cost, and the clean test CB-WP-0004 was owed + +CB-WP-0004 T05 said its measurement was confounded because the pass built +the tools it measured, and that *"the clean test is the next pass, which +uses the tools without building them."* T01–T03 are largely that pass. + +| category | baseline (662 resp) | CB-WP-0004 build | **this pass (70 resp)** | +|---|---|---|---| +| environment setup | 85 turns, $15.58 | 3 turns, $0.20 | **0 turns** | +| hub task status + workplan edit | 46 turns, $11.52 | 1 turn, $0.13 | **0 turns** | +| orientation / inspect | 49 turns, $6.87 | 41 turns, $4.10 | 7 turns, **$3.85** | +| ad-hoc text patching | 76 turns, $14.11 | 8 turns, $0.75 | 11 turns, **$2.28** | +| **mechanical share of pass** | **38.2%** | 26% | **38%** | + +**The verdict is clean, and it confirms CB-WP-0004 T06's mechanism +exactly.** The two categories whose tools *removed the manual path* went +to **zero and stayed there** with no further work. The two whose tools +merely *offered a better option* — `make status` and `facts-check` — are +now the entire mechanical cost of a pass. + +Mechanical share returned to 38% not because the wins reversed, but +because the denominator shrank while the misses did not. **Absolute +mechanical cost per pass fell from $51.76 to $6.13; mechanical cost per +response rose slightly, $0.078 → $0.088.** Share is noisy at n=70 against +a $16 denominator and should not be read as a trend. + +**A regression to report:** `SH-3` batching is **0.0%** this pass — 67 +tool calls across 67 responses, none batched, against a 20% target. Mean +context 315,170 against a 200,000 target. Both worse than the pass before. +Nothing in this workplan addressed session shape, and nothing gates it. + +## Verdict + +| claim | status | +|---|---| +| denominators widened, and the number moved | **confirmed** | +| kernel rules became visible (K10, K14, K18) | **confirmed** | +| the record was corrected, five rows | **confirmed** | +| M-D1-MUT ≥10 of 14 | **unmet — 4 of 14** | +| the diagnosis in CB-RES-0004 was complete | **refuted** — 6 more rows, 3 new defects, 1 new class | +| CB-WP-0004's "remove the manual path" mechanism | **confirmed on a clean window** | +| session shape | **regressed, ungated** | + +The pass did not finish what it planned. It stopped because the +instrument it built contradicted the plan — which is the outcome the stop +condition existed to produce, and the first time this loop has spent money +to be told it was wrong and then acted on it. diff --git a/history/260731-assertion-coverage-retrospective.md b/history/260731-assertion-coverage-retrospective.md new file mode 100644 index 0000000..73701fc --- /dev/null +++ b/history/260731-assertion-coverage-retrospective.md @@ -0,0 +1,145 @@ +# 2026-07-31 — retrospective: is a weak mutation the new grep? + +CB-WP-0005 T08. The pass produced the widened coverage arm, +`make mutation-check`, five corrections to committed evidence, a deferred +Phase C, and CB-WP-0006. It measured itself in +[CB-EV-0004](../evidence/CB-EV-0004-assertion-coverage.md). + +## The question this task was given + +> **CB-WP-0004 concluded that tooling recovers capacity only where it +> removes the manual path. Does the same test predict which gates work? +> Does a mutation gate remove the manual path — or is writing a weak +> mutation the new `grep`?** + +**Yes, it is the new `grep` — and it is worse than `grep`, because it +fails in the opposite direction.** + +The evidence is not hypothetical. `mutation-check`'s first run produced +two `SURVIVED` verdicts, and **both were my own no-op mutations**: + +- `pub struct NullRng;` → `pub struct NullRng {}` — semantically identical +- renaming `max_age_days` — CA-17 reads it with `.get(..., 90)` + +Two of fourteen rows, on the first attempt, by the author of the tool, +in a pass whose entire subject was inert assertions. + +## Why this is worse than the failure it replaces + +Apply CB-WP-0004 T06's test — *after this fix, is the manual path +impossible, absurd, or merely inferior?* + +| instrument | manual path after the fix | holds? | +|---|---|---| +| `make env-test` | impossible — the build fails if friction returns | **yes** (0 turns, two passes running) | +| `make task-done` | absurd — the old way means knowingly writing fiction | **yes** (0 turns) | +| `make status` | `grep` is one keystroke away | no | +| `facts-check` | patching markdown by hand still works | no | +| **`make mutation-check`** | **writing a weak mutation is exactly as easy as writing a strong one** | **no** | + +The mechanism predicted this correctly, again, for the fifth and sixth +instrument. But mutation-check is a new category of failure. `make status` +failing means someone greps instead — the loop is no worse off. A weak +mutation **publishes a false claim that working code is broken**: + +> **The fifth error class: false accusation (FA).** Every prior class +> under-reports — HDN, TA, SSB, DFD all let a real problem pass. FA +> over-reports. It sends the next pass to fix something that is not +> broken, and it is *more* credible than the truth because it arrives with +> a measurement attached. + +Thirteen instances, five classes. **Six passes, and every one has produced +an instance of a class the previous pass had not seen.** CB-WP-0003's +retrospective concluded that class-by-class hardening does not converge. +Three passes later that conclusion has only strengthened — and the newest +class is one that hardening *created*. + +## What would actually bind + +Stated as a mechanism, per the discipline CB-WP-0004 T06 established. + +A mutation is only evidence if it **fails for its stated reason**. The +harness already asserts the mutation applied, that the baseline was green, +and that the tree was restored. It cannot assert that the mutation +*means* what its author thought. Three candidates, in order of strength: + +1. **Mutations are written by someone other than the author of the + assertion.** The adversarial-review principle applied one level down. + This is the only one that structurally removes the manual path, and + CB-WP-0006 T09 should evaluate it. +2. **Each mutation records its expected failure message, and the harness + asserts the observed failure matches.** A no-op mutation fails this + immediately: it produces no failure at all. Cheap, mechanical, and + catches exactly the two defects this pass produced. +3. **A `SURVIVED` verdict is provisional until a second, differently + shaped mutation of the same property also survives.** Expensive, and it + only reduces the false-accusation rate rather than removing it. + +**(2) is the one to build first** and CB-WP-0006 T08 already requires that +every new mutation be *shown to fail for the stated reason*. + +## Was the prediction discipline worth it? + +CB-WP-0004 T06 introduced quantitative prediction before acting. Second +outing: + +| pass | predicted | measured | error | +|---|---|---|---| +| CB-WP-0004 | 25–30 points recovered | 6 points | 4–5× | +| CB-WP-0005 | ≥10 of 14 rows enforced | 4 of 14 | 2.5× | + +Both badly wrong, both wrong in the same direction — **optimism about how +much of the system was already working**. That consistency is itself +information, and it is the first thing this loop has learned about its own +forecasting rather than about its code. + +But the value was never the accuracy. **CB-WP-0005's prediction was wrong +in a way that changed what the pass did**: it named a stop condition, the +condition fired, Phase C was deferred unstarted, and CB-WP-0006 is scoped +to the eight rows the measurement found rather than the three rules the +survey predicted. Without the number there was nothing to be wrong about +and the pass would have implemented K10/K11 into a table that mostly +measures nothing. + +**That is the first time this project has paid to be told it was wrong and +then acted on it.** Worth more than the $16.03 the pass cost. + +## What the loop should change — InnerLoop v1.4 + +**Adopted (was already earned before this pass began).** §Step 2's +"numbers" row says the reviewer must *reproduce the number +independently* — satisfiable by re-running the command that prints it, +which finds nothing of this class. The reviewer found AM-7 by opening a +test out of curiosity and said so. + +> **v1.4 §Step 2:** where a claim rests on numbers, the reviewer must +> **read the assertion behind each quoted number and mutate it**. Re-running +> the command that prints a number is not verification of that number. + +Second instance of a verification step inheriting the author's blindness — +CB-WP-0002's dedup blind spot was the first. Both fixes replace +re-derivation with **adversarial execution**, and that now looks like the +general shape rather than two coincidences. + +**Not adopted, and stated so it is not quietly forgotten:** a rule that +every gate must remove its manual path. It is true of the four instruments +measured so far, but `mutation-check` shows a gate can be worth building +while failing that test — the eight `unmutatable` rows are real findings +that nothing else would have produced. The rule is a *predictor of +whether a gate saves money*, not a criterion for whether it is worth +having. CB-WP-0004 T06 stated it slightly too strongly and this pass is +the counter-example. + +## Open, not closed + +- **8 of 14 acceptance rows have no instrument.** CB-WP-0006 Phase A. +- **Phase C never started.** K9, K10, K11, K14, K18 unimplemented; + AM-7's hash clause and AM-11 remain withdrawn/unmet. CB-WP-0006 Phase B. +- **`facts-check` only checks copies that opted in** — AM-12's stale + $248.46 survived it for four workplans. Untagged copies are reported, + not failed, and that gap is now measured at 21 copies. +- **Session shape regressed and nothing gates it.** Batching **0.0%** + this pass against a 20% target, mean context 315,170 against 200,000. + `specs/SessionShape.md` has stated SS-01…SS-05 since CB-WP-0003 and + none of them has ever been enforced. +- **The chaos roll is at declaration 2 of 12** (d4=1, no override). diff --git a/specs/InnerLoop.md b/specs/InnerLoop.md index b3e505e..e24a99c 100644 --- a/specs/InnerLoop.md +++ b/specs/InnerLoop.md @@ -1,7 +1,12 @@ # The Inner Loop — Assimilate and Surpass -Status: **v1.3** — corrected from CB-WP-0004 (mechanical work) on -2026-07-31. Change from v1.2: single source of fact is now executable +Status: **v1.4** — corrected from CB-WP-0005 (assertion coverage) on +2026-07-31. Change from v1.3: where a claim rests on numbers, the +adversarial reviewer must read the assertion behind each quoted number and +**mutate it** — re-running the command that prints a number is not +verification of that number (§Step 2). + +v1.3 changed from v1.2: single source of fact is now executable (`make facts-check`, CB-WP-0004 T04), giving the duplicated-fact-drift class its first gate. @@ -165,6 +170,20 @@ sampling. So: > **The reviewer re-derives on a different sample than the author used.** > Where only one sample exists, the review says so rather than reporting a > clean verify. +> +> **And re-derivation is not enough (v1.4).** Where the claim rests on +> numbers, the reviewer must **read the assertion behind each quoted +> number and mutate it**: invert the property and require the suite to go +> red. Re-running the command that prints a number satisfies "reproduce +> independently" and finds nothing of this class. +> +> *(v1.4, from CB-WP-0005: `evidence/CB-EV-0001` reported `AM-7 replay | +> met, 2,290×` for a clause that asserts nothing — the hash reaches only a +> `println!`. The reviewer found it by opening a test out of curiosity and +> said so; no systematic step pointed there. Mutating it settled it in one +> command. This is the second verification step to inherit the author's +> blindness — the first was CB-WP-0002's dedup sample — and both fixes +> replace re-derivation with **adversarial execution**.)* *(v1.1, from CB-WP-0002: the dedup invariant was verified on the main transcript by the survey — 206/206 groups — and independently re-verified diff --git a/workplans/CB-WP-0005-assertion-coverage.md b/workplans/CB-WP-0005-assertion-coverage.md index 0a89d94..ffb6fb3 100644 --- a/workplans/CB-WP-0005-assertion-coverage.md +++ b/workplans/CB-WP-0005-assertion-coverage.md @@ -1,7 +1,7 @@ --- id: CB-WP-0005 title: "Make the instruments count assertions, then fix what they expose" -status: proposed +status: done state_hub_workstream_id: "0b95a1e3-7780-43d0-81e9-072ef7978734" --- @@ -84,23 +84,15 @@ kernel figures (`k_rules`, `k_linked`, `k_unlinked`) are registered facts, so they are under `make facts-check` from the day they first exist rather than after they drift. -**The self-test passed while the tool was completely broken.** A -`print(` inside `say()` was rewritten to `say(`, so every real -`make coverage` died with `RecursionError` — while `--self-test` reported -all-ok, because it only ever called `kernel_arm(quiet=True)` and never -executed the reporting path. +**The self-test passed while the tool was completely broken** — a +`print(` inside `say()` became `say(`, so every real `make coverage` died +with `RecursionError` while `--self-test` reported all-ok, because it only +ever called the quiet path. The control named the behaviour and did not +assert it: this task's own thesis in miniature, and HDN instance #7. Fixed +by exercising the loud path, verified by re-breaking `say()`. -That is this task's own thesis in miniature: **the control named the -behaviour and did not assert it.** Fixed by exercising the loud path under -`redirect_stdout` and asserting it prints, and verified by re-breaking -`say()` and confirming the two new checks go red. Seventh instance of the -harness-does-nothing shape, found in the tool written to find that shape. - -**A limit of `facts-check` surfaced here and is recorded, not patched:** -the check is line-based, so a tagged value that prose-wraps onto the next -line fails. It cost three edits to place two tags. Reported for T07 — -either the checker spans a paragraph, or the rule is stated as -"tagged values must not wrap". +**A limit of `facts-check`, recorded not patched:** it is line-based, so a +tagged value that prose-wraps fails. Three edits to place two tags. ## Task: M-D1-MUT — one mutation per acceptance row @@ -174,13 +166,10 @@ compares any number to 100,000 events/s**, the headline throughput claim. The problem is not three unimplemented rules; it is that **more than half the acceptance table has nothing behind it.** -Harness controls that earned their place: a stale find-string reports -`HARNESS-BROKEN` rather than silently scoring the baseline as the mutant; -a red baseline reports `inconclusive` rather than `red`; the tree is -restored in a `finally` and the restoration is verified. - -`make mutation-check` is deliberately **not** in `make all` — it rebuilds -the workspace once per mutated row. `--self-test` is in `make self-tests`. +Harness controls: a stale find-string reports `HARNESS-BROKEN` rather than +scoring the baseline as the mutant; a red baseline reports `inconclusive`; +the tree is restored in a `finally` and verified. Not in `make all` — it +rebuilds the workspace once per row. **Stop condition: see the note in T07 and the decision recorded there.** @@ -314,6 +303,33 @@ Normalize per unit of work, as CB-WP-0004 T05 established: report share of pass alongside absolute figures, and use `cb-cost --since` to window this pass against the last. +**Measured — [CB-EV-0004](../evidence/CB-EV-0004-assertion-coverage.md).** +70 responses, **$16.03**. + +1. **Denominators widened, and the number moved.** 1 spec → 2, 58 rules → + 76, 1 source file → 10, and the new denominator came in at **83%** + naming K10/K14/K18. The stated failure mode — a widened denominator + reporting the same percentage — did not occur. +2. **M-D1-MUT 4 of 14 against ≥10. Unmet**, and wrong about *what* as well + as how much: the diagnosis was three absent kernel rules, the + measurement found eight rows with no instrument at all. +3. **Quality held**, and widening surfaced far more than the seven known + defects: six further uninstrumented rows, HDN #7 (`rule-coverage`'s + self-test green while the tool was broken), a stale $248.46 that + `facts-check` could not see because it was untagged, and **a fifth + error class**. + +**The clean test CB-WP-0004 was owed, now run:** the two categories whose +tools removed the manual path are at **0 turns** two passes on; the two +that merely offered a better option are now the *entire* mechanical cost +of a pass. Absolute mechanical cost fell $51.76 → $6.13. The mechanism +holds. + +**A regression, reported because nothing else would:** SH-3 batching is +**0.0%** (67 tool calls, 67 responses) against a 20% target, and mean +context 315,170 against 200,000. SessionShape has stated these since +CB-WP-0003 and none of them has ever been enforced. + ## Task: retrospective and InnerLoop v1.4 ```task @@ -343,3 +359,40 @@ recovers capacity only where it removes the manual path. Does the same test predict which gates work?** `env-test` and `task-done` removed the manual path and held. Does a mutation gate remove the manual path — or is writing a weak mutation the new `grep`? + +**Answered — [260731-assertion-coverage-retrospective.md](../history/260731-assertion-coverage-retrospective.md). +Yes, it is the new `grep`, and it is worse, because it fails in the +opposite direction.** + +`mutation-check`'s first run produced two `SURVIVED` verdicts and both +were the author's own no-op mutations — two of fourteen rows, first +attempt, in a pass whose subject was inert assertions. Writing a weak +mutation is exactly as easy as writing a strong one and the harness cannot +tell them apart. + +**The fifth error class: false accusation (FA).** HDN, TA, SSB and DFD all +under-report — a real problem passes. FA over-reports: it publishes the +claim that working code is broken, sends the next pass to fix something +that is not broken, and is *more* credible than the truth because it +arrives with a measurement attached. Thirteen instances, five classes, six +passes — and the newest class is one that hardening created. + +**What would bind:** each mutation records its expected failure message +and the harness asserts the observed failure matches. A no-op fails that +immediately, because it produces no failure at all. CB-WP-0006 T08 already +requires it; T09 evaluates the stronger form — mutations written by +someone other than the author of the assertion. + +**One correction to CB-WP-0004 T06**, stated rather than left implicit: +"a gate only pays if it removes the manual path" is a **predictor of +whether a gate saves money, not a criterion for whether it is worth +having**. `mutation-check` fails that test and still produced the most +valuable findings of the pass. T06 stated it too strongly; this pass is +the counter-example. + +**InnerLoop v1.4** adopted: where a claim rests on numbers, the +adversarial reviewer must read the assertion behind each quoted number and +**mutate it**. Re-running the command that prints a number is not +verification of that number. Second verification step to inherit the +author's blindness; both fixes replace re-derivation with adversarial +execution.