CB-WP-0005 T07/T08: control loop, retrospective, InnerLoop v1.4
T07 — CB-EV-0004. 70 responses, $16.03. Test 1 MET: 1 spec -> 2, 58 rules -> 76, 1 source file -> 10, and the new denominator came in at 83% naming K10/K14/K18. The stated failure mode — a widened denominator reporting the same percentage — did not occur. Test 2 UNMET: M-D1-MUT 4 of 14 against >=10, and wrong about what as well as how much. The diagnosis was three absent kernel rules; the measurement found eight rows with no instrument at all. Test 3: quality held, and widening surfaced far more than the seven known defects — six further uninstrumented rows, HDN #7 (rule-coverage's self-test green while the tool was broken), a stale $248.46 invisible to facts-check because it was untagged, and a fifth error class. The clean test CB-WP-0004 was owed is now run, on a pass that used the tools without building them. The two categories whose tools removed the manual path are at 0 turns two passes on; the two that merely offered a better option are now the entire mechanical cost of a pass. Absolute mechanical cost fell $51.76 -> $6.13. The mechanism holds. Reported because nothing else would: SH-3 batching is 0.0% this pass — 67 tool calls across 67 responses — against a 20% target, and mean context 315,170 against 200,000. SessionShape has stated these since CB-WP-0003 and none has ever been enforced. T08 — the retrospective answers its question: yes, a weak mutation is the new grep, and it is worse, because it fails in the opposite direction. mutation-check's first run produced two SURVIVED verdicts and both were the author's own no-op mutations. The fifth error class: false accusation. HDN, TA, SSB and DFD all under-report — a real problem passes. FA over-reports: it publishes the claim that working code is broken, sends the next pass to fix something that is not broken, and is more credible than the truth because it arrives with a measurement attached. Thirteen instances, five classes, six passes, and the newest class is one that hardening created. One correction to CB-WP-0004 T06: "a gate only pays if it removes the manual path" is a predictor of whether a gate saves money, not a criterion for whether it is worth having. mutation-check fails that test and produced the most valuable findings of the pass. InnerLoop v1.4: where a claim rests on numbers, the adversarial reviewer must read the assertion behind each quoted number and mutate it. Re-running the command that prints a number is not verification of that number. Second verification step to inherit the author's blindness; both fixes replace re-derivation with adversarial execution. CB-WP-0005 status -> done, 5 of 8 tasks, 3 cancelled into CB-WP-0006. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
e2a2957b3a
commit
fd19f4e878
4 changed files with 411 additions and 26 deletions
|
|
@ -1,7 +1,12 @@
|
|||
# The Inner Loop — Assimilate and Surpass
|
||||
|
||||
Status: **v1.3** — corrected from CB-WP-0004 (mechanical work) on
|
||||
2026-07-31. Change from v1.2: single source of fact is now executable
|
||||
Status: **v1.4** — corrected from CB-WP-0005 (assertion coverage) on
|
||||
2026-07-31. Change from v1.3: where a claim rests on numbers, the
|
||||
adversarial reviewer must read the assertion behind each quoted number and
|
||||
**mutate it** — re-running the command that prints a number is not
|
||||
verification of that number (§Step 2).
|
||||
|
||||
v1.3 changed from v1.2: single source of fact is now executable
|
||||
(`make facts-check`, CB-WP-0004 T04), giving the duplicated-fact-drift
|
||||
class its first gate.
|
||||
|
||||
|
|
@ -165,6 +170,20 @@ sampling. So:
|
|||
> **The reviewer re-derives on a different sample than the author used.**
|
||||
> Where only one sample exists, the review says so rather than reporting a
|
||||
> clean verify.
|
||||
>
|
||||
> **And re-derivation is not enough (v1.4).** Where the claim rests on
|
||||
> numbers, the reviewer must **read the assertion behind each quoted
|
||||
> number and mutate it**: invert the property and require the suite to go
|
||||
> red. Re-running the command that prints a number satisfies "reproduce
|
||||
> independently" and finds nothing of this class.
|
||||
>
|
||||
> *(v1.4, from CB-WP-0005: `evidence/CB-EV-0001` reported `AM-7 replay |
|
||||
> met, 2,290×` for a clause that asserts nothing — the hash reaches only a
|
||||
> `println!`. The reviewer found it by opening a test out of curiosity and
|
||||
> said so; no systematic step pointed there. Mutating it settled it in one
|
||||
> command. This is the second verification step to inherit the author's
|
||||
> blindness — the first was CB-WP-0002's dedup sample — and both fixes
|
||||
> replace re-derivation with **adversarial execution**.)*
|
||||
|
||||
*(v1.1, from CB-WP-0002: the dedup invariant was verified on the main
|
||||
transcript by the survey — 206/206 groups — and independently re-verified
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue