# CB-EV-0021 — the design instrument CB-WP-0022 T07. Tier L, declaration 5 of chaos window 2 (structural L, chaos d8=6 → no override). Closed 2026-08-05. **Delivered:** INTENT property 4 (*Instrument*), [ADR-0012](../decisions/ADR-0012-the-design-instrument.md) (nine decisions), [specs/GameDesign.md](../specs/GameDesign.md) v1.0, the register in [GroundRules.md](../specs/GroundRules.md) (14 rows), `tools/design.py` + `make design`, and a report file in `ground-game`. --- ## 1. The headline: a separate reviewer found the survey's showcase was false This is the first adversarial review in this repo **run by a genuinely separate agent**. CB-RES-0006's reviewer opened by conceding it could not be, and wrote: > *"Treat the challenges below as a lower bound on what a genuinely > separate reviewer would find."* That was a testable claim. **This is the test.** The separate reviewer ran `git log` against the survey's central example and found our own commit had falsified it four days earlier — while the author, who wrote that commit, quoted the dead number twice in the same workplan. `CB-WP-0022:61` justified the whole proposal with *"GR-E01 is admissible because 4/6/9 against 5/7/9 is a computation anyone can rerun."* `2da19a4` had rerun it: **6/9/12**, scenario renamed `-unreachable-` → `-reachable-`, conclusion inverted. And GR-E01 was **one of the two findings that passed the reproduction rule**. **Verdict on same-session review: it is not a weak form of the control, it is a different control.** Four of the seven challenges rest on facts a same-session reviewer had already seen and not questioned. The cost was ~70k subagent tokens; it caught a defect in flight (§2) and rewrote the pass's central rule. ## 2. What it caught in flight T06's payload, status `todo`, still read *"the deal count puts 4/6/9 points in play … no dataset reconciles them"* and was queued for transmission to `ground-game`. It would have been the **fourth** wrong premise sent downstream, after *"12 in the file"* (a sum with no deal table) and *"SOLVE offered on a face-down Problem"* (the wrong condition named). **It is the only one stopped before sending.** | wrong premise | reached ground-game? | passed an existence test? | |---|---|---| | *"12 in the file"* | yes | — | | *"SOLVE on a face-down Problem"* | yes | — | | *"4/6/9, no dataset reconciles them"* | **stopped 2026-08-05** | **yes** | ## 3. The rule the pass came to build was one clause short The declaration's load-bearing rule was an **existence** test. Three of three wrong premises would have passed it, and one did. What shipped instead (GameDesign §1): a reproduction must **exist**, have the **shape ground-game ruled** (row-level deal table, never a sum — promoted from a T04 addendum to part of the rule), and **be able to fail**. **Clause 3 is the one this pass bought.** GR-E01's scenario went green when the edition landed and the finding stayed admissible and stayed queued, because nothing treated a passing artifact as a signal. *A green reproduction is an alarm, not a reassurance.* ## 4. Building it produced a distinction the ADR does not have `make design`'s first run alarmed on **U2** — and was wrong to. U2's scenario is green **because the provisional default it documents is implemented**, which says nothing about whether ground-game agrees. GR-E01's was a *counterexample* that went green. **Same colour, opposite meaning.** So the register gained a `role` column — `counterexample` | `default` — and only a green counterexample alarms. **A register that cannot tell them apart either alarms constantly or never**, and this was invisible until the metric ran against real rows. Folded back into GameDesign §1.3. ## 5. Did backfilling change the design? **T07 asked to be suspicious if all six findings fit the first taxonomy.** They did — five kinds absorbed everything, no sixth invented. Two reasons that is less reassuring than it looks and one reason it is fine: - the taxonomy was **derived from these six**, so fitting them is nearly circular. Its real test is the seventh finding; - **one of the six was deleted during backfill.** *"Six provisional defaults"* was not entered: C3 showed it double-counted GR-E01, and provisional scenarios are *reproductions for* underdetermined items, not a finding. So the taxonomy absorbed five, not six; - what it did do is **force the note tier to earn itself** — GR-A13 and GR-E03/E04 have no artifact and became notes, which is the first time the distinction cost anything. ## 6. The numbers, and why none of the survey's survive **CB-RES-0007's measurements are withdrawn.** C2 dismantled the instrument: `design-baseline.py` is a hand-maintained dict where `n = len(FINDINGS)`; `has_reproduction` is `startswith("scenarios/")` and never stats the file, so its own YES-control was **green against a path `2da19a4` deleted**; the 4-day figure is two date literals subtracted (mutated to 2020-01-01 → 2406 days, all checks still green); and `Makefile:127` ran only `--self-test`, so the reporting path had no CI. It is **superseded, not deleted** — it is the evidence for how a wrong number got into a survey, and the header now says so. Measured by `make design`, 2026-08-05, over rows that name real files: ``` findings 12 (+2 notes) with a resolving reproduction 3/12 = 25% target 100% open, lacking a reproduction 9 target 0 reproductions green while open 0 target 0 notes past 30 days 0 target 0 closed (log) 2 [F11, F13] ``` **Two survey claims died against measurement:** | survey said | measured | |---|---| | *"six of the ten already have provisional scenarios"* | **one.** `grep -lE "\bU\b" scenarios/ground/*.yaml` → only U2 | | *"2/6 = 33% reproduced"* | withdrawn; GR-E01 was double-counted and the detector was broken | The first is the same defect class as the wrong premises — **an uncheckable link asserted as a fact** — found inside the survey that proposed the fix. ## 7. The register's first run found ten answers nobody had collected The most useful thing `make design` did was not a metric. **GROUND-WP-0002 T05 ruled all ten U-items on 2026-08-03 — every one confirmed.** T03 confirmed five of the six provisional scenarios. The workplan is `finished`. CB-RES-0007 reported *"0 of 10 ruled"* on 2026-08-03, and the register built this session initially recorded them as `reported`. **Both were two days stale on the day they were written.** **This is the unread-inbox failure running in the opposite direction.** The pass was designed around *we send findings and nobody reads them*. The symmetric failure — *they answer and we do not collect it* — was not anticipated by the declaration, the survey, or the review, and the first run of the instrument surfaced it immediately. The items are `ruled`, not `applied`: the rulings confirmed our defaults, so the rules did not move, but the scenarios still carry `provisional: true` for settled choices. **Lifting those flags is owed and is not done** — `make design` shows them open until it is, which is the register working. ## 8. The engine-evolution question, as the review left it Declared judgment: no second register, because engine evolution already lives in `evidence/`, `decisions/`, `gates.toml` and workplans. **The conclusion survives; the argument did not.** C5 found the "third thing" the maintainer meant is visible in `specs/InnerLoopReference.md` (383 lines, two sections literally headed *"moved from InnerLoop v1.6"*) and 769 lines of `history/` retrospectives — **neither of which the inventory named**. So the content exists and found homes somewhat by accident. ADR-0012 D7 records it as **argued and narrowed, not settled**, with the revisit condition: if InnerLoopReference keeps absorbing material that is neither a decision nor a finding, build the register. ## 9. What tier L cost against what it caught Second full-weight L pass in this window, third in the project. | caught | by | |---|---| | the central example was false | **only** the separate reviewer | | a wrong premise in flight to ground-game | the same | | the baseline instrument counts itself | the same | | the incumbent register was never evaluated | the same | | `role`: green default ≠ green counterexample | **building it**, not reviewing it | | ten uncollected rulings | **running it**, not building it | **Two of six were caught by execution, not by process** — the pattern InnerLoop §Design goal predicts (cheap correction beats exhaustive prevention), and an argument against front-loading more review rather than less. **The one number that matters for the loop's own calibration:** the review step is what distinguished this pass from a competent one. Without it the pass ships a rule justified by a false example, sends a fourth bad premise downstream, and builds `make design` on a dict that counts itself. The correct reading of CB-RES-0006's fidelity note is **not** *"our reviews were somewhat weaker"* — it is *"our reviews were not performing this function at all."* ## 10. Cost CB-WP-0021's cost, by re-running the instrument: see `make cost` / `tools/cb-cost.py`. **Not quoted here as a literal** — §Single source of fact, and this pass has just spent itself proving what typed numbers do. ## Open after this pass - **U1–U10 → `applied`**: lift `provisional: true` where a ruling confirmed the default, and record each ruling. Nine items of bookkeeping. - **The U-item ↔ scenario mapping is unwritten** for 9 of 10. Cheap, and it is what makes the register checkable. - **F12 and F14 are notes** and expire 2026-08-31. GROUND-WP-0003's playtest closes F14 and produces its artifact via `cb-play --record`. - **The taxonomy's real test is the seventh finding**, not these six.