clay-borg/research/CB-RES-0008-could-we-have-won.md

id	capability	status	tier	chaos
CB-RES-0008	analysis.witness-and-difficulty	reviewed 2026-08-05 — headline finding WITHDRAWN (C4); numbers corrected (C1)	L	d8 = 6 → no override

CB-RES-0008 — a path out of a lost game, and how hard the game is

CB-WP-0025 T01. Surveyed 2026-08-05.

Two maintainer questions that are the same machine asked twice: "could we have won, and how?" is a search from a recorded state; "how hard is this?" is that search — or a proxy for it — run over many deals and counted.

The runnable baseline is ours and it is the row that matters. External candidates are algorithms and practices, not software we can run on our workload, so per InnerLoop §Step 1 their rows are directional and cap at parity.

---

1. The baseline, measured

cargo run --release -p games-ground --example difficulty-baseline (200 seeds per seat count, GreedyPolicy and RandomPolicy):

bot win rate over 200 seeds (GR-E01 group success):
  2p greedy  132/200 won =  66.0%   mean total  5.3 of  5.0   median margin +1
  3p greedy  165/200 won =  82.5%   mean total  8.4 of  7.0   median margin +2
  4p greedy  190/200 won =  95.0%   mean total  8.8 of  7.0   median margin +2
  5p greedy  200/200 won = 100.0%   mean total 11.8 of  9.0   median margin +3
  6p greedy  200/200 won = 100.0%   mean total 12.0 of  9.0   median margin +3
  2p random   10/200 won =   5.0%   mean total  2.4 of  5.0   median margin -3
  3p random   19/200 won =   9.5%   mean total  3.1 of  7.0   median margin -4
  4p random   16/200 won =   8.0%   mean total  3.2 of  7.0   median margin -4

search cost — legal_commands at every real decision point:
  2p  462 decisions   branching mean 4.7  median 5  max 7   3.0 us/call
  3p  649 decisions   branching mean 7.4  median 8  max 10  3.5 us/call
  4p  870 decisions   branching mean 9.1  median 10  max 12  4.1 us/call

(Search-cost row re-measured after C1. The first published figures — 161/139/112 "us/node" — timed two setups, a whole greedy game and a validate+fold replay. See §1.2.)

1.1 WITHDRAWN — the finding this claimed, and why it is not one

Withdrawn 2026-08-05 by the adversarial review (C4), before it left the repo. The section is kept, struck through, because a claim retracted silently is how three earlier wrong premises survived (ADR-0012 D5).

A FirstLegal policy — take legal[0], no heuristic — scores 0% at five and six seats where GreedyPolicy scores 100%, and 77.5% at two seats where greedy scores 66%. Two unsophisticated agents span the entire range at the same seat count. A measurement that does that is about the policy, not about the game.

seats	greedy	firstlegal
2	66.0%	77.5%
5	100.0%	0.0%
6	100.0%	0.0%

The arithmetic below is right; the interpretation is not, and the table also fails GameDesign §1.2 by reporting sums where GROUND-WP-0004 T02 requires per-priority rows.

A greedy bot wins 200 of 200 games at five and six seats. Not 95%, not 99% — every game, with a median margin of +3 and mean available points of 11.8–12.0 against a threshold of 9.

The arithmetic behind it, row by row (the shape GROUND-WP-0004 T02 requires):

seats	Surface	hidden dealt	points available	threshold	ratio
2	priority 1	2	6	5	1.20
3–4	priority 1	3	9	7	1.29
5–6	priority 1	4	12	9	1.33

The ratio moves the wrong way. More seats means more points on the table and a proportionally lower bar and more actions per round to clear it with. Three multipliers all pointing the same direction, which is why the curve is not gentle — it is 66% → 100% across four seat counts.

This is admissible under GameDesign §1. It is not (C3), and the rule it fails was one day old.

• Clause 2, ruled shape. 6/9/12 are sums, and the hidden dealt column is a count. GROUND-WP-0004 T02 requires Surface and each hidden priority listed separately and explicitly forbids "deal depth N". The table reproduces the prohibited shape while citing the ruling.
• Clause 3, can fail. The harness has no assertions, no --self-test, and is in no make target. Nothing can turn it red. Under CB-WP-0022 T05's own role distinction it is a default artifact — it prints what the code does — wearing a counterexample label.

A reproduction that cannot fail is a printout. T05 must fix the harness before any figure from it is quoted again.

It also explains the maintainer's report — "I felt it was too easy but then we lost, so who knows" — without needing a solver. He plays at low seat counts, where 66% is a real game, and had been feeling the 5–6 seat experience from elsewhere in the same session. Both halves of the sentence are true of different seat counts.

Also withdrawn (C5). games/ground/src/lib.rs:2487-2493 records what actually happened: "the maintainer played several 3-player games on 2026-08-03 and could not win any of them, because GR-S01 dealt 2/3/4 Problems worth 3/6/10 against thresholds of 5/7/9." Three seats, on the pre-ruling deal, arithmetically unwinnable at 6 against 7 — nothing to do with the curve fitted to it here.

T06 must report this to GROUND-WP-0005. It must not. GROUND-WP-0005 is blocked waiting on a difficulty baseline, and this is not one. Sending it would have invited ground-game to move thresholds on the strength of one bot's behaviour — the fifth wrong premise this project has sent them, and the second stopped by a review rather than by a control.

1.2 What the search-cost numbers rule out

Branching is small — mean 4.7 to 9.1.

CORRECTED (C1). The published figure of 112–161 µs/node was wrong by 30–50×: the timer bracketed two setups, a whole greedy game and a validate+fold replay, divided by player-decision count. The tell was in the output — it fell as branching rose, which no per-enumeration cost can do.

Re-measured, clock around legal_commands only: 3.0 / 3.5 / 4.1 µs at 2/3/4 seats, now rising with branching. The reviewer measured 15.6–20.4 µs by a different isolation and we have not settled which is right — T04 must benchmark it properly (criterion is already a dev-dependency) rather than adopt either.

Exhaustive search is out at any seat count. False (C6). The reviewer measured ~16 minutes for a full 2p game and ~3 seconds over the last two rounds at 3p. With C1's correction the affordable budget is ~10⁵–10⁶ nodes, and bounded exhaustive search over the endgame is inside it.

This changes T03's starting point. The ADR cannot open with "exhaustive is impossible, therefore determinized sampling" — the premise is false, and the alternative carries strategy fusion that exhaustive search does not. Neither the survey nor the review considered transposition or the co-operative single-agent framing, which cut the exponent further.

2. Prior art: determinized search, and the failure it is famous for

The natural first idea — deal out the hidden cards, solve the resulting perfect-information game, repeat — is Perfect Information Monte Carlo (PIMC), and its failure modes were named by Frank, Basin and Matsubara in 1998:

• Strategy fusion — the search picks different actions from two states in the same information set, which no real player could do, because a player cannot tell those states apart. The plan it returns is not executable by someone who does not know which world they are in.
• Non-locality — subgame values are not well-defined when information is hidden, so recursive search over subgames is unsound.

Strategy fusion is precisely the trap in this pass. A witness produced by determinized search may be a line that requires knowing which Solution is on top of the deck. Showing the maintainer "you could have won by playing Repair on turn 3" — when nothing he could see said a Repair was coming — teaches a false lesson about his own play, which is worse than not shipping the feature.

Long and Sturtevant later characterized when PIMC nonetheless works well, which matters here: its success depends on properties of the game tree (leaf correlation, bias, disambiguation rate). GROUND disambiguates fast — Problems flip face-up, selections reveal every round — which is the regime where PIMC is least bad. That is an argument the ADR may use, and it is a directional one, not a measurement.

ISMCTS (information-set MCTS) searches over information sets directly rather than determinizations, and is the standard answer to strategy fusion.

Benchmark to beat: a witness that replays through our existing scenario runner and ends in group_success. That is a stronger and cheaper bar than any of the above, because it is mechanically checkable — and note it does not by itself exclude a strategy-fused line. A fused line replays fine. Checkability and honesty are different properties, and the ADR must not let the first stand in for the second.

Directional, cited-only.

3. The retrospective question is not the playing question

Worth separating, because the prior art is all about playing:

question	information	honest?
was this deal winnable at all	omniscient	yes — it is a question about the deal, not about the player
was it winnable from what we could see	the seat's view	yes, and expensive
could a reasonable player have found it	the seat's view, bounded	the only affordable honest one

The first is legitimate and cheap, and answers "the deal was unwinnable, stop blaming yourself" — which is a real thing a player wants to hear. It is not an answer to "how could we have won", and labelling it as one is the failure mode.

Naming matters more than the algorithm here. The ADR's first decision is which question is being answered and what it is called on screen.

4. Difficulty as a measured quantity

Co-operative board games set difficulty with a dial and publish the win rate — Pandemic's number of Epidemic cards is the canonical example. The practice is: a named dial, a stated player skill, and a target band.

We have the dial candidates already — the threshold (GR-E01), and ground-game's proposed Pressure dial (GROUND-WP-0005) — and §1 supplies the first measured band.

The problem the practice does not solve for us: a published win rate is measured against humans. Ours is measured against GreedyPolicy. The 5–6 seat 100% is a claim about our bot, and the honest reading is narrower than "the game is too easy at six players" — it is "a bot that takes the obvious action never fails to clear the threshold at six players."

Whether that is the same statement is the reviewer's strongest line of attack and is not settled here.

Benchmark to beat: a difficulty figure whose resolution is stated — the smallest threshold change it can distinguish, with its N. A rate without that cannot tune anything.

Directional, cited-only.

5. Benchmarks to beat

dimension	today	benchmark
witness checkability	no witness exists	100% of emitted paths replay to group_success through the existing runner
witness honesty	—	no line that requires unseen information; the ADR must say how this is enforced, not asserted
search cost	112–161 µs/node measured	a bound in nodes or wall clock, and "none found within B" wording that does not claim unwinnability
difficulty resolution	one band, one policy	the smallest threshold delta distinguishable, with N and policy named
difficulty honesty	—	the policy and seed range are in the number's name, not a footnote

6. What the survey did not settle

• Whether a bot win rate is a difficulty at all. §4. The strongest counter is that it measures the bot, and improving the bot would "increase the difficulty" without touching the game.
• How witness honesty is enforced rather than asserted. Running the search on a GroundView makes the information boundary structural; running it on GroundState makes it a promise. The survey believes the first is right and has not measured whether it is affordable — a view-only search cannot fold events, so it needs a state it may not see. This is the gap most likely to sink the pass, and it is stated here rather than discovered in T05.
• Whether 100% at 5–6 seats is a rules finding or a bot finding. §1.1 reports it as measured; which repo owns it is T03's call.
• Whether the cheap honest answer is enough. "This deal was unwinnable" (omniscient, cheap) may satisfy the maintainer's actual need without any information-respecting search at all. Nobody has asked him. That is a one-question experiment this survey did not run.