CB-WP-0022 T02: the separate reviewer found the showcase finding was false
First adversarial review in this repo run by a genuinely separate agent.
CB-RES-0006's reviewer opened by conceding it could not be, and called its
own findings "a lower bound on what a genuinely separate reviewer would
find." This is the measurement: the separate reviewer ran git log against
the survey's central example and found 2da19a4 had falsified it four days
earlier, while the author -- who wrote that commit -- quoted the dead
number twice.
Seven challenges: four conceded, two conceded in part, one answered.
C1 changes the design. "GR-E01 is admissible because 4/6/9 against 5/7/9
is a computation anyone can rerun" was a computation already rerun: the
edition import measured 6/9/12, the conclusion inverted, and the scenario
was renamed -unreachable- to -reachable-. That finding was one of the TWO
that passed the reproduction rule. So three wrong premises have now
reached ground-game and the third satisfied an existence test -- existence
is not the property that was missing. The rule gains shape (ground-game's
row-level deal table, promoted from a T04 addendum) and a clause the
survey never contemplated: a reproduction must be able to fail. Ours went
green and stayed admissible.
C1 also caught a defect in flight. T06's payload, status todo, still named
4/6/9 and was queued to send it to ground-game as "no dataset reconciles
them." Withdrawn before sending -- the fourth wrong premise, and the only
one stopped.
C2 withdraws the baseline's precision: design-baseline.py is a
hand-maintained dict counting itself, has_reproduction never checks the
file exists (its YES-control is green against a deleted path), and
Makefile:127 runs only --self-test so the reporting path has no CI. The
direction survives; 33% is not a measured rate and T05 must not build on
it. C3: "six provisional defaults" is five, GR-E01 double-counted. C4:
GroundRules §Underdetermined was never evaluated as a candidate and
already delivers four of five benchmarks -- T03's burden flips to arguing
extension over replacement.
Survived: the rule's affordability, and reuse of the provisional
machinery.
loop-lint: no findings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
129ed03492
commit
04c3a4977f
3 changed files with 617 additions and 9 deletions
187
history/260805-design-instrument-response.md
Normal file
187
history/260805-design-instrument-response.md
Normal file
|
|
@ -0,0 +1,187 @@
|
|||
# 260805 — response to the challenge on CB-RES-0007
|
||||
|
||||
One round, per InnerLoop §Step 2. **This review was run by a separate
|
||||
agent**, unlike CB-RES-0006's, whose challenge opened with a fidelity note
|
||||
saying it could not be. That note said *"treat the challenges below as a
|
||||
lower bound on what a genuinely separate reviewer would find."* It was
|
||||
right, and this is what the difference looks like: the separate reviewer
|
||||
went to `git log` and found that the survey's showcase example had been
|
||||
falsified four days earlier by our own commit.
|
||||
|
||||
Four challenges conceded, two conceded in part, one answered. **The
|
||||
load-bearing rule survives but does not survive intact** — it is
|
||||
insufficient as stated, and C1 is the reason.
|
||||
|
||||
---
|
||||
|
||||
## C1 — the showcase finding was itself a wrong premise, and it passed the rule — **conceded, and it is larger than the challenge says**
|
||||
|
||||
The challenge is correct and I verified every step of it independently
|
||||
rather than accepting it.
|
||||
|
||||
`CB-WP-0022:61` defines admissibility by example:
|
||||
|
||||
> *"GR-E01 is admissible because 4/6/9 against 5/7/9 is a computation
|
||||
> anyone can rerun."*
|
||||
|
||||
Commit `2da19a4` (2026-08-04, CB-WP-0021 T01/T02/T05) measured the deal
|
||||
after the edition import: **6/9/12 against thresholds 5/7/9 — the game is
|
||||
reachable at every seat count.** The scenario was renamed
|
||||
`gr-e01-threshold-unreachable-2p.yaml` → `gr-e01-threshold-reachable-2p.yaml`.
|
||||
The finding did not merely get a new number. **Its conclusion inverted.**
|
||||
|
||||
So the count is not two wrong premises reaching `ground-game`. It is
|
||||
three, and **the third one was in the 2-of-6 set that passed the
|
||||
reproduction rule.** The rule's own success case is a false finding that
|
||||
shipped a green artifact.
|
||||
|
||||
**And it was still loaded.** `CB-WP-0022:283`, task T06, status `todo`:
|
||||
|
||||
> *"GR-E01 vs GR-S01 — the deal count puts 4/6/9 points in play against
|
||||
> thresholds of 5/7/9, so either the count or the thresholds are wrong and
|
||||
> no dataset reconciles them"*
|
||||
|
||||
That is queued for transmission to `ground-game` as a finding. Had T06 run
|
||||
before this review, it would have sent a **fourth** wrong premise
|
||||
downstream — one asserting no dataset reconciles numbers our own dataset
|
||||
reconciled on 2026-08-04. The review did not catch a stale citation. It
|
||||
caught a defect in flight.
|
||||
|
||||
**What this forces.** The rule at `CB-WP-0022:53` is an *existence* test:
|
||||
a finding is admissible if a reproduction exists. Three of three wrong
|
||||
premises would have passed an existence test, and one did. Existence is
|
||||
therefore not the property that was missing. T03 must adopt the shape
|
||||
requirement — `ground-game`'s row-level deal table, GROUND-WP-0004 T02 —
|
||||
as **part of the rule**, not as the T04 addendum it currently is. And it
|
||||
needs a second clause the survey did not contemplate: **a reproduction
|
||||
must be able to fail.** A finding whose artifact has gone green is not a
|
||||
closed finding until someone says which it is; ours silently became a
|
||||
counterexample to itself.
|
||||
|
||||
`4/6/9` is corrected wherever it appears in this pass's artifacts, and
|
||||
`evidence/CB-EV-0018:153` is left alone — it is a dated record of what was
|
||||
believed then, and rewriting evidence is worse than a stale number in it.
|
||||
|
||||
## C2 — the baseline is a hand-maintained inventory reporting on itself — **conceded**
|
||||
|
||||
`tools/design-baseline.py:16-36` is a literal dict. `n = len(FINDINGS)`
|
||||
(`:89`). **"6 findings" is a count of rows someone typed.** It is not a
|
||||
measurement of anything, and the survey presented it as a measured
|
||||
baseline, in a repo whose §Single source of fact rule exists precisely to
|
||||
stop typed numbers.
|
||||
|
||||
Worse, and the part I would not have found:
|
||||
|
||||
- `has_reproduction` (`:38-43`) is `p.startswith("scenarios/")` and
|
||||
nothing else. **It never checks the file exists.** The self-test's
|
||||
YES-control (`:63`) passes a path — `gr-e01-threshold-unreachable-2p.yaml` —
|
||||
that `2da19a4` deleted. The positive control for "detection can say
|
||||
YES" is green against a file that is not there. That is a positive
|
||||
control that controls nothing, which is the exact class `gates.toml`
|
||||
exists to catch.
|
||||
- The same detector **rejects a named test**, which `CB-WP-0022:55-56`
|
||||
explicitly admits as a reproduction. So `33%` is biased down by the
|
||||
detector's own narrowness.
|
||||
- The 4-day figure is two date literals subtracted (`:96-98`). The
|
||||
reviewer mutated them to 2020-01-01, got **2406 days**, and all five
|
||||
self-test checks stayed green.
|
||||
- `Makefile:127` runs `--self-test` only. **The reporting path is never
|
||||
executed by CI**, so none of the above was ever going to be caught.
|
||||
|
||||
**What survives, and it matters that something does.** The *direction* of
|
||||
every headline number is checkable without the tool: 11 files with no
|
||||
index is verifiable by reading, and U1–U10 being open with 0 of 10 ruled
|
||||
is verifiable against the hub. The baseline's conclusion — *we have
|
||||
findings scattered across the repo with no register and no reproductions* —
|
||||
is not in doubt. **What is withdrawn is its precision**, and any claim
|
||||
that the 33% is a measured rate.
|
||||
|
||||
T05 must not build on this tool. The register itself is the measurement,
|
||||
and `make design` reports over the register's contents, not over a dict.
|
||||
|
||||
## C3 — the arithmetic in the survey's own table — **conceded**
|
||||
|
||||
"Six provisional defaults" is **five**. The tool's own line prints
|
||||
`5 location(s)`; `make coverage` prints 5; the survey table (`:46`) says
|
||||
6. It was 6 at `2da19a4^` and 5 at HEAD — the survey table was written
|
||||
against a repo state that had already moved.
|
||||
|
||||
And **GR-E01 is double-counted**: it is row 4 in its own right and was a
|
||||
member of row 5's six. So the `2/6` denominator contains one finding
|
||||
twice, and both copies are in the numerator. The real figure, on the
|
||||
survey's own definitions, is not 2/6.
|
||||
|
||||
I am not restating a corrected rate here. **The tool that would produce it
|
||||
is the one C2 just withdrew**, and quoting a new number from a discredited
|
||||
instrument is how the first one got in.
|
||||
|
||||
## C4 — §Underdetermined was never evaluated as a candidate — **conceded in part**
|
||||
|
||||
Correct, and it is the omission that embarrasses the survey most, because
|
||||
§Step 1 asks for the best implementation *in existence* and this one is in
|
||||
our own repo. The survey went to Magic to discover the Oracle property —
|
||||
*the fix lands in the source, the annotation is not authoritative* — and
|
||||
`specs/GroundRules.md:231-233` already states it in our own words:
|
||||
*"a ruling flips the scenario, not the kernel."*
|
||||
|
||||
**Conceded in part, not whole,** and the reviewer marked the honest limit
|
||||
themselves: 10 U-items owe provisional scenarios, 5 exist, and
|
||||
`make coverage` never flags the missing five. A section that cannot report
|
||||
its own gaps is not yet the register. But T03's burden has changed
|
||||
direction — it must now argue why §Underdetermined should be **extended**
|
||||
rather than replaced, and the default answer is that it should be.
|
||||
|
||||
## C5 — the third thing exists — **conceded as to the argument, not the conclusion**
|
||||
|
||||
`specs/InnerLoopReference.md` is 383 lines with two sections literally
|
||||
headed *"moved from InnerLoop v1.6"*, and `history/` holds 769 lines of
|
||||
retrospectives. That is content about *how we learn to build this engine*
|
||||
which was placed where it fit rather than where it belonged. My redundancy
|
||||
inventory named `evidence/`, `decisions/`, `gates.toml` and workplans, and
|
||||
**named neither of those** — so the inventory was incomplete and the
|
||||
argument as written does not stand.
|
||||
|
||||
The conclusion probably does, and for a reason the challenge concedes: the
|
||||
homeless content found homes, and both are load-bearing and read. A third
|
||||
register would be a third place to look for material that is currently in
|
||||
two. **T03 records this as argued-and-narrowed rather than settled**, which
|
||||
is the outcome the declaration asked for when it said this was the
|
||||
judgment it wanted reviewed rather than assumed.
|
||||
|
||||
## C6 — "queue that empties" vs keeping resolved findings — **answered**
|
||||
|
||||
Marked weak by the reviewer and it is weak, but it has a real answer.
|
||||
Magic's property is that a finding **closes when the source changes**. It
|
||||
is not that the record is deleted. `design-baseline.py:23-25`'s comment —
|
||||
keep resolved findings or you cannot report a close rate — is compatible:
|
||||
the *queue* of open findings empties, the *log* of closed ones does not.
|
||||
T03's lifecycle should say this in one sentence so the next reader does
|
||||
not re-litigate it.
|
||||
|
||||
Falsifier: if the register's default view shows closed findings mixed with
|
||||
open ones, the queue property is gone regardless of what the lifecycle
|
||||
says.
|
||||
|
||||
## C7 — n=6, all engine-surfaced — **already conceded in §6**
|
||||
|
||||
The survey states this limit itself. Nothing to add.
|
||||
|
||||
---
|
||||
|
||||
## What the review changed
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| the rule | reproduction must **exist** | must exist, **have the ruled shape**, and **be able to fail** |
|
||||
| wrong premises shipped | 2 | **3** — and one passed the rule |
|
||||
| the baseline | a measured 33% | a hand-maintained inventory; direction stands, precision withdrawn |
|
||||
| §Underdetermined | not considered | the incumbent T03 must argue against |
|
||||
| engine-evolution register | redundant | still redundant, on a corrected inventory |
|
||||
| T06's payload | a finding | **a fourth wrong premise, caught before sending** |
|
||||
|
||||
**The strongest thing here is not any single challenge.** It is that a
|
||||
separate reviewer with no stake ran `git log` against the survey's central
|
||||
example and found it had been falsified by our own commit four days
|
||||
earlier — while the author, who wrote that commit, quoted the dead number
|
||||
twice. CB-RES-0006's review said a same-session reviewer would be a lower
|
||||
bound. This is how much lower.
|
||||
Loading…
Add table
Add a link
Reference in a new issue