CB-WP-0025 T06/T07: a difficulty baseline that reports its own confound
Some checks failed
ci / check (push) Has been cancelled

T06. games/ground/examples/difficulty.rs, make difficulty, wired into make
self-tests, and a report file in ground-game under GROUND-WP-0005 with a
hub message pointing at it.

THE REPORT OPENS WITH THE RETRACTION, because what this task was written
to send was withdrawn by T02 and GROUND-WP-0005 is blocked on exactly that
number. They are told, in the first section, that we nearly sent them "the
game is too easy at 5-6 seats" and why it was wrong.

  seats  winnable  greedy  random  first-legal  spread
   2p      60%     60.0%    5.0%      76.7%      71.7
   3p      93%     88.3%    6.7%      25.0%      81.7
   4p     100%     93.3%    6.7%      30.0%      86.7
   5p     100%    100.0%    3.3%       0.0%     100.0
   6p     100%    100.0%    3.3%       0.0%     100.0

SPREAD justifies the whole redesign: 71.7 to 100.0 points between three
trivial policies. The table now shows why no single rate is a difficulty
rather than asserting it.

And the 5-6 rows point the OPPOSITE way from the withdrawn claim --
first-legal 0% against greedy 100% is the widest spread in the table,
which suggests play matters MORE there, not less. Neither reading is
established and the report says so.

The confound is stated in the tool's own output, not only in prose:
`winnable` is conditioned on greedy's play up to the final round, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would repeat this pass's error in a subtler form -- which is exactly
how a corrected project reintroduces a defect. NO THRESHOLD CHANGES ARE
PROPOSED.

The instrument can fail (spec §5): a witness must replay to a win, an
unwinnable position must report searched-out rather than a budget cut, a
one-node budget must not claim exhaustion, and the policy panel must
actually disagree. difficulty-baseline.rs marked superseded, kept as the
survey's dated snapshot. Registered as F16, inconsistent / withdrawn.

T07. evidence/CB-EV-0024. Five of nine defects came only from the review;
four from execution, and all four of those were in work written after it.
The wrong-denominator family now has five instances and still no control
-- facts-check catches copies that disagree, nothing catches a number
computed correctly against the wrong base. Tier L was an over-declaration
(no port, structurally M) and paid for itself anyway, because the review
is L-only. Chaos window 2 will close with zero overrides, making its
retirement condition untestable.

Named as open rather than implied done: the witness is NOT wired to the
ending page. The search works; the browser cannot ask it yet.

make all: exit 0. loop-lint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-05 19:24:49 +02:00
parent 81e0aba59a
commit 7ed9fc730a
6 changed files with 583 additions and 79 deletions

View file

@ -0,0 +1,161 @@
# CB-EV-0024 — could we have won
CB-WP-0025 T07. Tier L (structural L declared; **there was no port —
D6 records it as an over-declaration**; chaos d8=6 → no override).
Declaration 8 of chaos window 2. Closed 2026-08-05.
**Delivered:** [CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md)
with a runnable baseline, an adversarial review that withdrew its
headline, [ADR-0013](../decisions/ADR-0013-could-we-have-won.md),
[RetrospectiveAnalysis.md](../specs/RetrospectiveAnalysis.md),
`games/ground/src/search.rs`, `benches/search.rs`,
`examples/difficulty.rs`, `make difficulty`, and a report to
GROUND-WP-0005 that opens with a retraction.
---
## 1. Was the maintainer's game winnable?
The question the pass exists for. **Yes, and the tool answers it.**
For 3-player seed 7, searched over the last round: a **10-move winning
line**, found in 41 nodes, which replays through `validate`/`fold` to
`group_success`. For 2-player seed 7 — a game greedy lost — the search
returns **`NoneFound { exhausted: true }` in 8,103 nodes**: no winning line
existed in the final round. That is a real negative, not a budget cut, and
the type keeps the two apart.
**The feature works. The difficulty half is where the trouble was.**
## 2. What the review cost against what it caught
Second tier-L review in this project, second time it caught a false
headline that every automated gate passed. `make all` was green
throughout.
| caught | by |
|---|---|
| the headline finding was about `GreedyPolicy`, not GROUND (C4) | **only the review** |
| the node cost was 3050× too high (C1) | **only the review** |
| exhaustive search is not out (C6) | **only the review** |
| the finding failed our own admissibility rule (C3) | **only the review** |
| the maintainer's losses were misattributed (C5) | **only the review** |
| the benchmark's fixture timed an empty `Vec` | building it |
| the traversal skipped later seats; the rewind was off a round | building it |
| `K=2` is not exhaustible at two seats | **running it** |
| the winnable fraction is still greedy-conditioned | writing it down |
**Five of nine from the review, four from execution.** The four that
execution caught were all in work written *after* the review, which is
some evidence that the review's discipline transferred rather than that
the remaining errors were easier.
**The reviewer under-ranked their own best challenge.** They put C4
fourth. Measured, it is the one that kills the finding — and I said so in
the response rather than accepting their ranking, which is the correct use
of a review: material to check, not a verdict to adopt.
## 3. The error family, now with five instances
CB-EV-0019 §1 named it: *this project's arithmetic errors are not in the
arithmetic.* A number computed correctly against the wrong base, span, or
subject.
| # | pass | the wrong thing |
|---|---|---|
| 1 | CB-WP-0021 | `csv` cost against AM-4a, which never sees the code |
| 2 | ground-game report | *"12 in the file"* — a sum with no deal table |
| 3 | CB-WP-0018 | SOLVE's inertness attributed to the wrong condition |
| 4 | **this pass, C1** | a timer bracketing whole games, divided by decisions |
| 5 | **this pass, C4** | a win rate attributed to the game, not the policy |
**And two projections published in place of measurements**, both in this
pass: the survey's `7.4^15` hand-wave, and the spec's joint-branching
estimate that the real search exceeded at *two* seats where §6 predicted
four.
**`facts-check` catches copies that disagree. Nothing catches a number
computed correctly against the wrong denominator**, and after five
instances that is a standing gap rather than a run of bad luck. Naming it
is what this file can do; closing it is not this pass's.
## 4. What the difficulty measurement actually shows
`make difficulty`, 60 seeds:
| seats | winnable | greedy | random | first-legal | spread |
|---|---:|---:|---:|---:|---:|
| 2 | 60% | 60.0% | 5.0% | 76.7% | 71.7 |
| 3 | 93% | 88.3% | 6.7% | 25.0% | 81.7 |
| 4 | 100% | 93.3% | 6.7% | 30.0% | 86.7 |
| 5 | 100% | 100.0% | 3.3% | 0.0% | 100.0 |
| 6 | 100% | 100.0% | 3.3% | 0.0% | 100.0 |
**The `spread` column is the finding.** 71.7 to 100.0 points between three
trivial policies. It is the evidence for the prohibition in
RetrospectiveAnalysis §4.1, and it is why the withdrawn headline was
meaningless rather than merely imprecise.
**The 56 seat rows point the opposite way from the withdrawn claim.**
`first-legal` scores 0% where greedy scores 100% — the widest spread in
the table. Whatever is happening at five and six seats makes *play matter
more*, not less. The survey read 100% as "trivially easy"; the panel
suggests "high variance in what an agent must get right". **Neither is
established**, and saying so is the honest state.
**The winnable fraction is still not policy-free**, and the tool says so
in its own output: it is *"winnable from where greedy got to"*, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would be this pass's own error in a subtler form — which is exactly
how a corrected project reintroduces a defect, so it is stated in the
code, the spec, the report and here.
## 5. Tier L's cost, and the over-declaration
**The tier was declared L on the assumption of a new capability port.
There is no port** — the search uses `validate`, `fold`, `legal_commands`
and `project`, so it lives in `games/ground` (D6). By the structural
trigger this was an **M**.
**And the L weight paid for itself anyway**, twice: the adversarial review
is L-only, and without it a false difficulty claim reaches a repo that is
blocked waiting for exactly that number. **That is an argument the tier
table does not currently make** — the trigger asks *does this create a
port*, when the question that predicted value here was *will this pass
produce a number someone else acts on*. Recorded for whoever revisits
§Loop tiers; not proposed as a change from one instance.
## 6. Chaos window 2
**Declaration 8 of 12.** Structural L (over-declared), d8 = 6, no
override.
**Eleven declarations, no 8 rolled.** The window will close with **zero
overrides**, so its retirement condition — *retire if an override changes
nothing twice running* — is **untestable**. Window 1 closed with two
overrides at d4 and both changed the outcome.
**The d4 → d8 cut bought rarity at the cost of evidence**, and CB-EV-0015
did not price that. At d8 over twelve declarations the expected count is
1.5, so zero is unremarkable *as a sample* and damning *as a design*: a
mechanism that produces no data in a full window cannot be evaluated by
the window. **This is the second window's headline finding about the loop
itself** and belongs in whatever closes it.
## 7. Cost
CB-WP-0024's cost, by re-running the instrument: `make cost`. Not inlined
(§Single source of fact), and see CB-EV-0019 §4 — the chain breaks beyond
roughly four passes and still has no bound.
## Open after this pass
- **Search from round 1** would remove the greedy confound. Needs
transposition or move-ordering; neither built.
- **Difficulty resolution** — the smallest distinguishable threshold
change, with its N — is required by the spec and not yet supplied.
- **The witness is not wired to the ending page.** T05 built the search;
the browser cannot yet ask it. That is the maintainer-facing half of
remark 2 and it is not done.
- **The wrong-denominator family has no control** (§3), at five instances.
- **The chaos window's retirement condition is untestable** (§6).