--- id: CB-RES-0008 capability: analysis.witness-and-difficulty status: reviewed 2026-08-05 — headline finding WITHDRAWN (C4); numbers corrected (C1) tier: L chaos: d8 = 6 → no override --- # CB-RES-0008 — a path out of a lost game, and how hard the game is CB-WP-0025 T01. Surveyed 2026-08-05. Two maintainer questions that are the same machine asked twice: *"could we have won, and how?"* is a search from a recorded state; *"how hard is this?"* is that search — or a proxy for it — run over many deals and counted. **The runnable baseline is ours and it is the row that matters.** External candidates are algorithms and practices, not software we can run on our workload, so per InnerLoop §Step 1 their rows are **directional and cap at `parity`**. --- ## 1. The baseline, measured `cargo run --release -p games-ground --example difficulty-baseline` (200 seeds per seat count, `GreedyPolicy` and `RandomPolicy`): ``` bot win rate over 200 seeds (GR-E01 group success): 2p greedy 132/200 won = 66.0% mean total 5.3 of 5.0 median margin +1 3p greedy 165/200 won = 82.5% mean total 8.4 of 7.0 median margin +2 4p greedy 190/200 won = 95.0% mean total 8.8 of 7.0 median margin +2 5p greedy 200/200 won = 100.0% mean total 11.8 of 9.0 median margin +3 6p greedy 200/200 won = 100.0% mean total 12.0 of 9.0 median margin +3 2p random 10/200 won = 5.0% mean total 2.4 of 5.0 median margin -3 3p random 19/200 won = 9.5% mean total 3.1 of 7.0 median margin -4 4p random 16/200 won = 8.0% mean total 3.2 of 7.0 median margin -4 search cost — legal_commands at every real decision point: 2p 462 decisions branching mean 4.7 median 5 max 7 3.0 us/call 3p 649 decisions branching mean 7.4 median 8 max 10 3.5 us/call 4p 870 decisions branching mean 9.1 median 10 max 12 4.1 us/call ``` *(Search-cost row re-measured after C1. The first published figures — 161/139/112 "us/node" — timed two `setup`s, a whole greedy game and a validate+fold replay. See §1.2.)* ### 1.1 WITHDRAWN — the finding this claimed, and why it is not one > **Withdrawn 2026-08-05 by the adversarial review (C4), before it left > the repo.** The section is kept, struck through, because a claim > retracted silently is how three earlier wrong premises survived > (ADR-0012 D5). > > A `FirstLegal` policy — take `legal[0]`, no heuristic — scores > **0% at five and six seats** where `GreedyPolicy` scores 100%, and > **77.5% at two seats** where greedy scores 66%. Two unsophisticated > agents span the entire range at the same seat count. **A measurement > that does that is about the policy, not about the game.** > > | seats | greedy | firstlegal | > |---|---:|---:| > | 2 | 66.0% | **77.5%** | > | 5 | **100.0%** | **0.0%** | > | 6 | **100.0%** | **0.0%** | > > The arithmetic below is right; the **interpretation** is not, and the > table also fails GameDesign §1.2 by reporting **sums** where > GROUND-WP-0004 T02 requires per-priority rows. ~~**A greedy bot wins 200 of 200 games at five and six seats.**~~ Not 95%, not 99% — every game, with a median margin of **+3** and mean available points of **11.8–12.0 against a threshold of 9**. The arithmetic behind it, row by row (the shape GROUND-WP-0004 T02 requires): | seats | Surface | hidden dealt | points available | threshold | ratio | |---|---|---|---:|---:|---:| | 2 | priority 1 | 2 | **6** | 5 | 1.20 | | 3–4 | priority 1 | 3 | **9** | 7 | 1.29 | | 5–6 | priority 1 | 4 | **12** | 9 | 1.33 | **The ratio moves the wrong way.** More seats means more points on the table *and* a proportionally lower bar *and* more actions per round to clear it with. Three multipliers all pointing the same direction, which is why the curve is not gentle — it is 66% → 100% across four seat counts. ~~**This is admissible under GameDesign §1.**~~ **It is not (C3), and the rule it fails was one day old.** - **Clause 2, ruled shape.** 6/9/12 are **sums**, and the `hidden dealt` column is a *count*. GROUND-WP-0004 T02 requires Surface and each hidden priority listed separately and explicitly forbids "deal depth N". The table reproduces the prohibited shape while citing the ruling. - **Clause 3, can fail.** The harness has **no assertions, no `--self-test`, and is in no `make` target**. Nothing can turn it red. Under CB-WP-0022 T05's own `role` distinction it is a `default` artifact — it prints what the code does — wearing a `counterexample` label. **A reproduction that cannot fail is a printout.** T05 must fix the harness before any figure from it is quoted again. ~~It also **explains the maintainer's report** — *"I felt it was too easy but then we lost, so who knows"* — without needing a solver. He plays at low seat counts, where 66% is a real game, and had been feeling the 5–6 seat experience from elsewhere in the same session. Both halves of the sentence are true of different seat counts.~~ **Also withdrawn (C5).** `games/ground/src/lib.rs:2487-2493` records what actually happened: *"the maintainer played several 3-player games on 2026-08-03 and could not win any of them, because GR-S01 dealt 2/3/4 Problems worth 3/6/10 against thresholds of 5/7/9."* Three seats, on the pre-ruling deal, **arithmetically unwinnable at 6 against 7** — nothing to do with the curve fitted to it here. ~~**T06 must report this to GROUND-WP-0005.**~~ **It must not.** GROUND-WP-0005 is blocked waiting on a difficulty baseline, and this is not one. Sending it would have invited ground-game to move thresholds on the strength of one bot's behaviour — the **fifth** wrong premise this project has sent them, and the second stopped by a review rather than by a control. ### 1.2 What the search-cost numbers rule out Branching is small — mean 4.7 to 9.1. > **CORRECTED (C1). The published figure of 112–161 µs/node was wrong by > 30–50×**: the timer bracketed two `setup`s, a whole greedy game and a > validate+fold replay, divided by player-decision count. The tell was in > the output — it **fell** as branching **rose**, which no per-enumeration > cost can do. > > Re-measured, clock around `legal_commands` only: **3.0 / 3.5 / 4.1 µs** > at 2/3/4 seats, now rising with branching. **The reviewer measured > 15.6–20.4 µs by a different isolation and we have not settled which is > right** — T04 must benchmark it properly (`criterion` is already a > dev-dependency) rather than adopt either. ~~**Exhaustive search is out at any seat count.**~~ **False (C6).** The reviewer measured ~16 minutes for a full 2p game and **~3 seconds over the last two rounds at 3p**. With C1's correction the affordable budget is **~10⁵–10⁶ nodes**, and bounded exhaustive search over the endgame is inside it. **This changes T03's starting point.** The ADR cannot open with *"exhaustive is impossible, therefore determinized sampling"* — the premise is false, and the alternative carries strategy fusion that exhaustive search does not. Neither the survey nor the review considered transposition or the co-operative single-agent framing, which cut the exponent further. ## 2. Prior art: determinized search, and the failure it is famous for The natural first idea — *deal out the hidden cards, solve the resulting perfect-information game, repeat* — is **Perfect Information Monte Carlo (PIMC)**, and its failure modes were named by Frank, Basin and Matsubara in 1998: - **Strategy fusion** — the search picks *different* actions from two states in the same information set, which no real player could do, because a player cannot tell those states apart. The plan it returns is not executable by someone who does not know which world they are in. - **Non-locality** — subgame values are not well-defined when information is hidden, so recursive search over subgames is unsound. **Strategy fusion is precisely the trap in this pass.** A witness produced by determinized search may be a line that requires knowing which Solution is on top of the deck. Showing the maintainer *"you could have won by playing Repair on turn 3"* — when nothing he could see said a Repair was coming — teaches a false lesson about his own play, which is worse than not shipping the feature. **Long and Sturtevant** later characterized *when* PIMC nonetheless works well, which matters here: its success depends on properties of the game tree (leaf correlation, bias, disambiguation rate). GROUND disambiguates fast — Problems flip face-up, selections reveal every round — which is the regime where PIMC is least bad. **That is an argument the ADR may use, and it is a directional one, not a measurement.** **ISMCTS** (information-set MCTS) searches over information sets directly rather than determinizations, and is the standard answer to strategy fusion. **Benchmark to beat:** a witness that **replays through our existing scenario runner and ends in `group_success`**. That is a stronger and cheaper bar than any of the above, because it is mechanically checkable — and note it does *not* by itself exclude a strategy-fused line. A fused line replays fine. **Checkability and honesty are different properties, and the ADR must not let the first stand in for the second.** *Directional, cited-only.* ## 3. The retrospective question is not the playing question Worth separating, because the prior art is all about *playing*: | question | information | honest? | |---|---|---| | *was this deal winnable at all* | omniscient | **yes** — it is a question about the deal, not about the player | | *was it winnable from what we could see* | the seat's view | yes, and expensive | | *could a reasonable player have found it* | the seat's view, bounded | the only affordable honest one | The first is legitimate and cheap, and answers *"the deal was unwinnable, stop blaming yourself"* — which is a real thing a player wants to hear. It is **not** an answer to *"how could we have won"*, and labelling it as one is the failure mode. **Naming matters more than the algorithm here.** The ADR's first decision is which question is being answered and what it is called on screen. ## 4. Difficulty as a measured quantity Co-operative board games set difficulty with a dial and publish the win rate — Pandemic's number of Epidemic cards is the canonical example. The practice is: **a named dial, a stated player skill, and a target band.** We have the dial candidates already — the threshold (`GR-E01`), and `ground-game`'s proposed Pressure dial (GROUND-WP-0005) — and §1 supplies the first measured band. **The problem the practice does not solve for us:** a published win rate is measured against *humans*. Ours is measured against `GreedyPolicy`. The 5–6 seat 100% is a claim about our bot, and the honest reading is narrower than "the game is too easy at six players" — it is *"a bot that takes the obvious action never fails to clear the threshold at six players."* Whether that is the same statement is **the reviewer's strongest line of attack** and is not settled here. **Benchmark to beat:** a difficulty figure whose resolution is stated — the smallest threshold change it can distinguish, with its N. A rate without that cannot tune anything. *Directional, cited-only.* ## 5. Benchmarks to beat | dimension | today | benchmark | |---|---|---| | witness checkability | no witness exists | **100%** of emitted paths replay to `group_success` through the existing runner | | witness honesty | — | no line that requires unseen information; **the ADR must say how this is enforced, not asserted** | | search cost | 112–161 µs/node measured | a bound in nodes or wall clock, and *"none found within B"* wording that does not claim unwinnability | | difficulty resolution | one band, one policy | the smallest threshold delta distinguishable, with N and policy named | | difficulty honesty | — | the policy and seed range are **in the number's name**, not a footnote | ## 6. What the survey did not settle - **Whether a bot win rate is a difficulty at all.** §4. The strongest counter is that it measures the bot, and improving the bot would "increase the difficulty" without touching the game. - **How witness honesty is enforced rather than asserted.** Running the search on a `GroundView` makes the information boundary structural; running it on `GroundState` makes it a promise. The survey believes the first is right and has **not** measured whether it is affordable — a view-only search cannot fold events, so it needs a state it may not see. **This is the gap most likely to sink the pass, and it is stated here rather than discovered in T05.** - **Whether 100% at 5–6 seats is a rules finding or a bot finding.** §1.1 reports it as measured; which repo owns it is T03's call. - **Whether the cheap honest answer is enough.** *"This deal was unwinnable"* (omniscient, cheap) may satisfy the maintainer's actual need without any information-respecting search at all. Nobody has asked him. That is a one-question experiment this survey did not run.