CB-WP-0005 T07/T08: control loop, retrospective, InnerLoop v1.4

T07 — CB-EV-0004. 70 responses, $16.03.

Test 1 MET: 1 spec -> 2, 58 rules -> 76, 1 source file -> 10, and the new
denominator came in at 83% naming K10/K14/K18. The stated failure mode — a
widened denominator reporting the same percentage — did not occur.

Test 2 UNMET: M-D1-MUT 4 of 14 against >=10, and wrong about what as well
as how much. The diagnosis was three absent kernel rules; the measurement
found eight rows with no instrument at all.

Test 3: quality held, and widening surfaced far more than the seven known
defects — six further uninstrumented rows, HDN #7 (rule-coverage's
self-test green while the tool was broken), a stale $248.46 invisible to
facts-check because it was untagged, and a fifth error class.

The clean test CB-WP-0004 was owed is now run, on a pass that used the
tools without building them. The two categories whose tools removed the
manual path are at 0 turns two passes on; the two that merely offered a
better option are now the entire mechanical cost of a pass. Absolute
mechanical cost fell $51.76 -> $6.13. The mechanism holds.

Reported because nothing else would: SH-3 batching is 0.0% this pass — 67
tool calls across 67 responses — against a 20% target, and mean context
315,170 against 200,000. SessionShape has stated these since CB-WP-0003
and none has ever been enforced.

T08 — the retrospective answers its question: yes, a weak mutation is the
new grep, and it is worse, because it fails in the opposite direction.
mutation-check's first run produced two SURVIVED verdicts and both were
the author's own no-op mutations.

The fifth error class: false accusation. HDN, TA, SSB and DFD all
under-report — a real problem passes. FA over-reports: it publishes the
claim that working code is broken, sends the next pass to fix something
that is not broken, and is more credible than the truth because it arrives
with a measurement attached. Thirteen instances, five classes, six passes,
and the newest class is one that hardening created.

One correction to CB-WP-0004 T06: "a gate only pays if it removes the
manual path" is a predictor of whether a gate saves money, not a criterion
for whether it is worth having. mutation-check fails that test and
produced the most valuable findings of the pass.

InnerLoop v1.4: where a claim rests on numbers, the adversarial reviewer
must read the assertion behind each quoted number and mutate it.
Re-running the command that prints a number is not verification of that
number. Second verification step to inherit the author's blindness; both
fixes replace re-derivation with adversarial execution.

CB-WP-0005 status -> done, 5 of 8 tasks, 3 cancelled into CB-WP-0006.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 18:05:31 +02:00
parent e2a2957b3a
commit fd19f4e878
4 changed files with 411 additions and 26 deletions

View file

@ -1,7 +1,12 @@
# The Inner Loop — Assimilate and Surpass
Status: **v1.3** — corrected from CB-WP-0004 (mechanical work) on
2026-07-31. Change from v1.2: single source of fact is now executable
Status: **v1.4** — corrected from CB-WP-0005 (assertion coverage) on
2026-07-31. Change from v1.3: where a claim rests on numbers, the
adversarial reviewer must read the assertion behind each quoted number and
**mutate it** — re-running the command that prints a number is not
verification of that number (§Step 2).
v1.3 changed from v1.2: single source of fact is now executable
(`make facts-check`, CB-WP-0004 T04), giving the duplicated-fact-drift
class its first gate.
@ -165,6 +170,20 @@ sampling. So:
> **The reviewer re-derives on a different sample than the author used.**
> Where only one sample exists, the review says so rather than reporting a
> clean verify.
>
> **And re-derivation is not enough (v1.4).** Where the claim rests on
> numbers, the reviewer must **read the assertion behind each quoted
> number and mutate it**: invert the property and require the suite to go
> red. Re-running the command that prints a number satisfies "reproduce
> independently" and finds nothing of this class.
>
> *(v1.4, from CB-WP-0005: `evidence/CB-EV-0001` reported `AM-7 replay |
> met, 2,290×` for a clause that asserts nothing — the hash reaches only a
> `println!`. The reviewer found it by opening a test out of curiosity and
> said so; no systematic step pointed there. Mutating it settled it in one
> command. This is the second verification step to inherit the author's
> blindness — the first was CB-WP-0002's dedup sample — and both fixes
> replace re-derivation with **adversarial execution**.)*
*(v1.1, from CB-WP-0002: the dedup invariant was verified on the main
transcript by the survey — 206/206 groups — and independently re-verified