affordability projection games/ground/src/search.rs, five tests. It finds real winning lines and replays them through validate/fold to group_success. Two bugs in my own work, found and fixed here. THE TRAVERSAL WAS WRONG. It branched on "the first seat with any legal command" and stopped there, so a later seat never acted if an earlier one was already selected but still had a legal move. Restructured around what the rules oblige: a seat without a selection MUST select (GR-R02) and nothing else can happen first; after Reveal the optional actions branch freely and the aggregate rejects Resolve until the obligatory ones are done -- so the search needs no phase logic of its own. AND MY REWIND WAS OFF BY ONE ROUND, replaying the round it was meant to search. That is why the first run reported 3 nodes and looked like a working search. Measured with the real search, rewinding real games to the start of their last K rounds: 2p K=1 exhausted, 8,103 nodes, ~29 ms 2p K=2 budget cut at 2,000,000 nodes, ~5 s 3p K=2 win found, 41 nodes, ~157 us The spec's own falsifier said "§3 fails if K=2 proves unaffordable at four seats". IT FAILED AT TWO. The projection assumed a joint product per round; the search explores sequential per-seat decisions, so orderings multiply the tree far beyond width^seats. That is the second projection this pass published in place of a measurement -- C1's timer was the first. THE ASYMMETRY IS THE OPERATIVE FINDING. Finding a win is cheap: DFS stumbles onto one in tens of nodes. Proving none exists needs exhaustion. So the witness feature is affordable now at any K a player would ask about, and the winnable fraction (ADR-0013 D4) is NOT, because its negative half must exhaust every deal it counts. K=1 is the honest default for exhaustive answers today; making K=2 exhaustible needs transposition or move-ordering, neither of which this pass built. specs §3 and §3.1 corrected accordingly, and the K=2 default withdrawn. The negative control that makes "winnable" falsifiable: 2p seed 7 over its last round returns NoneFound with exhausted=true in ~8k nodes -- a real negative, not a budget cut wearing a verdict's clothes. And the visible/ hidden marking is tested both ways, since a marking that can only say YES is decoration. make all: exit 0. loop-lint clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
395 lines
17 KiB
Markdown
395 lines
17 KiB
Markdown
---
|
||
id: CB-WP-0025
|
||
kind: product
|
||
title: "Could we have won: a path out of a lost game, and how hard the game actually is"
|
||
status: active
|
||
state_hub_workstream_id: "a866982f-94e7-432b-b0d5-2fefb781a574"
|
||
---
|
||
|
||
# Purpose
|
||
|
||
```
|
||
structural tier L (creates a new capability — a search over game state,
|
||
and a measurement the engine does not currently take;
|
||
both produce numbers ground-game will tune against)
|
||
chaos d8 = 6 → no override
|
||
declared tier L
|
||
```
|
||
|
||
Declaration 8 of chaos window 2. Tier L: separate survey, **adversarial
|
||
review**, ADR, then spec, then code.
|
||
|
||
## Two remarks, and why they are one pass
|
||
|
||
> *"I had a game where we lost and in this case I would have liked to know
|
||
> if and how we could have won… the best path is not computable I guess so
|
||
> a path to win is fine."*
|
||
|
||
> *"Do we have difficulty estimations? If so we should show them. It will
|
||
> help tuning the game. I felt it was too easy but then we lost, so who
|
||
> knows."*
|
||
|
||
They are the same machine asked two questions. *Was this game winnable?*
|
||
is a search from a recorded state. *How hard is this game?* is that search
|
||
run over many deals and counted. Building the second without the first
|
||
gives a win-rate with no witness; building the first without the second
|
||
gives one anecdote per game.
|
||
|
||
**"I felt it was too easy but then we lost, so who knows" is the finding.**
|
||
The maintainer cannot calibrate the game from play, and that is precisely
|
||
the gap `ground-game` is currently blocked in: **GROUND-WP-0005**
|
||
*(Difficulty tiers (threshold ratio) and optional Pressure dial)* is active
|
||
with **both its tasks in `wait`**. Tiers cannot be set without a measured
|
||
baseline, and clay-borg is the thing that can measure. This pass is what
|
||
unblocks it — which is the design-instrument aspect CB-WP-0022 is in the
|
||
middle of stating, arriving with a concrete demand.
|
||
|
||
## What already exists, so the survey does not re-find it
|
||
|
||
The state is replayable (`replay.rs`, `make replay-test`); the move space
|
||
is enumerable (`legal_commands`, narrowed by CB-WP-0023 so branching is
|
||
real rather than inflated by inert moves); bots exist (`bot.rs`); and the
|
||
threshold is public (`OutcomeView`). **A search needs no new persistence
|
||
and no new rules** — which is why D6 put it in `games/ground` with no new
|
||
crate and no port.
|
||
|
||
## What makes this hard, and must not be waved through
|
||
|
||
**Settled by [ADR-0013](../decisions/ADR-0013-could-we-have-won.md).** The
|
||
declaration framed the central risk as *a path computed with the deck
|
||
known is a path the players could never have found*, and asked the ADR to
|
||
choose between an omniscient, an information-respecting, and a bounded
|
||
search.
|
||
|
||
**D1 dissolved the choice**: strategy fusion is a defect of aggregating
|
||
over determinizations to *choose a move*, and after the game there is one
|
||
world — so a line found in it is executable in it. **D2** keeps the
|
||
declaration's real concern by marking each move `visible`/`hidden` rather
|
||
than by refusing to search.
|
||
|
||
The second warning here — *"one win rate over one bot policy is not the
|
||
difficulty"* — was right, and **the survey made exactly that error
|
||
anyway**; see T02.
|
||
|
||
## Task: survey
|
||
|
||
```task
|
||
id: CB-WP-0025-T01
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "556cfd24-5992-4cc0-be90-0b60e989b5bb"
|
||
```
|
||
|
||
`research/CB-RES-0008-*.md`, with `tier: L` and the chaos roll recorded
|
||
(`loop-lint` checks both).
|
||
|
||
Per §Step 1 the survey is done when it can name a **benchmark-to-beat**
|
||
per dimension. Four were asked for: retrospective solvers in
|
||
hidden-information games (and their known failure, strategy fusion); *a
|
||
path to win* as a **witness** rather than a proof; difficulty as a measured
|
||
quantity in co-operative games; and the **per-node cost**, measured on our
|
||
machine — the runnable-baseline option applies, since a search that cannot
|
||
finish while the player is looking at the page is a different feature.
|
||
|
||
**Done 2026-08-05.**
|
||
[CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md), with a
|
||
runnable baseline (`games/ground/examples/difficulty-baseline.rs`).
|
||
|
||
> **Everything this record originally claimed was withdrawn by T02 the
|
||
> same day.** Kept as a pointer rather than rewritten, because a claim
|
||
> retracted silently is how three earlier wrong premises survived
|
||
> (ADR-0012 D5).
|
||
|
||
**Claimed:** a greedy bot wins 200/200 at five and six seats, so the game
|
||
is too easy there; the points-to-threshold ratio rises with seat count and
|
||
explains the curve; `legal_commands` costs 112–161 µs, so exhaustive
|
||
search is out; and all of this explains the maintainer's report.
|
||
|
||
**Withdrawn:** the win rate is `GreedyPolicy`'s, not the game's (C4 —
|
||
`FirstLegal` scores 0% on the same deals); the ratio is identical at 3p
|
||
and 4p, which differ by 12.5 points (C2); the cost figure was wrong by
|
||
30–50× and exhaustive search is affordable (C1, C6); and the maintainer's
|
||
losses were 3-player games on the pre-ruling deal (C5).
|
||
|
||
**What survives:** the harness exists and runs, the 6/9/12 arithmetic is
|
||
right against `Problems.csv`, and the branching widths (4.7–9.1) hold. See
|
||
[CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md) for the
|
||
corrected text and
|
||
[the response](../history/260805-could-we-have-won-response.md) for the
|
||
full accounting.
|
||
|
||
## Task: adversarial review
|
||
|
||
```task
|
||
id: CB-WP-0025-T02
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "bdd9fc97-4706-4d8e-b286-91ff53339680"
|
||
```
|
||
|
||
Tier L requires it. Exactly one round: challenge, then response, trail in
|
||
`history/`, unpolished. Require an attempt at:
|
||
|
||
- **that the honest version is unaffordable** — that a search respecting
|
||
the information rule is too expensive or too weak to find anything, so
|
||
the shipped tool will quietly become the omniscient one with a
|
||
reassuring label;
|
||
- **that a witness misleads more than it helps** — being shown a line that
|
||
needed a card you could not know about teaches a wrong lesson about the
|
||
game, and the tool would be better refusing to answer;
|
||
- **that the difficulty number is a bot benchmark wearing a difficulty
|
||
costume**, and `ground-game` will tune the game against our bot rather
|
||
than against play;
|
||
- **that this is CB-WP-0022's job** — the design instrument is being built
|
||
right now, and a difficulty measurement is a finding-producing tool. The
|
||
strongest counter is that the register records findings and this
|
||
*produces* them, but the reviewer should press whether that is a
|
||
distinction worth a separate capability.
|
||
|
||
**Done 2026-08-05.** Trail:
|
||
[challenge](../history/260805-could-we-have-won-challenge.md),
|
||
[response](../history/260805-could-we-have-won-response.md).
|
||
Separate agent, as in CB-WP-0022.
|
||
|
||
**Six of seven conceded. The survey's headline finding is WITHDRAWN.**
|
||
|
||
**C4 kills it and the reviewer ranked it fourth.** A `FirstLegal` policy —
|
||
`legal[0]`, no heuristic — scores **0% at five and six seats** where greedy
|
||
scores 100%, and **77.5% at two** where greedy scores 66%. Two
|
||
unsophisticated agents span the whole range, so *"too easy at 5–6 seats"*
|
||
was about `GreedyPolicy`, not GROUND.
|
||
|
||
**C1**: the node cost was wrong by 30–50× — the timer bracketed whole
|
||
games — and **the tell was in my own output**, falling as branching rose.
|
||
**C6**: exhaustive search is *not* out, which changes T03's premise.
|
||
**C3**: the finding failed the admissibility rule this project wrote nine
|
||
hours earlier. **C2**: 3p and 4p share deal, threshold and ratio and differ
|
||
by 12.5 points. **C5**: `lib.rs:2487` contradicts the "explains the
|
||
maintainer" claim — his losses were 3-player on the pre-ruling deal,
|
||
unwinnable at 6 against 7.
|
||
|
||
**T06 was pointed at GROUND-WP-0005, which is blocked on this number.**
|
||
Sending it would have been the fifth wrong premise to reach ground-game.
|
||
**Both tier-L reviews here have now caught a false headline that every
|
||
gate passed.**
|
||
|
||
## Task: decide
|
||
|
||
```task
|
||
id: CB-WP-0025-T03
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "055630a1-56e0-4ad4-8796-35f444313371"
|
||
```
|
||
|
||
`decisions/ADR-0013-*.md`. (ADR-0012 is CB-WP-0022 T03's.) At minimum:
|
||
|
||
- **which question the solver answers**, from the three in §What makes
|
||
this hard, and what it is called in the UI — the name must not overclaim;
|
||
- **the information boundary**: whether the search runs on `GroundState`
|
||
or on a `GroundView`, and if on state, what stops it using what the view
|
||
hides. Note that running on the view makes the rule structural rather
|
||
than a promise, and that this is the cheapest guarantee available;
|
||
- **the bound**: depth, node budget, or wall clock, and what *no path
|
||
found* means against it — a bounded search that says "unwinnable" is
|
||
lying, and the wording must say "none found within B";
|
||
- **whether difficulty ships as one number or a small table**, and what it
|
||
is a function of: policy, seat count, threshold, seed range;
|
||
- **where it lives** — a new crate, a mode of `cb-play`, or a tool under
|
||
`tools/`. The tier was declared L on the assumption of a new capability;
|
||
if the ADR concludes it is a mode of an existing one, say so, and the
|
||
over-declaration is a chaos-window data point worth recording.
|
||
|
||
**Done 2026-08-05.**
|
||
[ADR-0013](../decisions/ADR-0013-could-we-have-won.md), seven decisions.
|
||
**Two of them are not what T03 was written expecting**, because the review
|
||
moved the ground under both.
|
||
|
||
- **D1 — strategy fusion does not apply, and that is why this is
|
||
affordable.** Fusion is a defect of *aggregating over determinizations
|
||
to choose a move*. **After the game there is one world**: the deal is
|
||
known, so a search over it produces a line executable in the only world
|
||
there is. The survey treated fusion as this pass's central obstacle; it
|
||
is an obstacle to a *playing* engine, which we are not building.
|
||
- **D4 — difficulty is the WINNABLE FRACTION, not any bot's win rate.**
|
||
C4 killed the bot rate: two trivial policies span 0–100% on the same
|
||
deals, and improving the bot would make the game "easier" without a rule
|
||
changing. The solver supplies a policy-independent measure — *over N
|
||
deals, in what proportion does a winning line exist* — which is a
|
||
property of the deal distribution and the threshold, and is what
|
||
GROUND-WP-0005 actually needs. **The bot rate never was.**
|
||
|
||
**D2** searches `GroundState` — the survey's view-only boundary is not
|
||
implementable, since a view cannot fold events — and moves the guarantee to
|
||
a checkable per-move `visible`/`hidden` marking from `project()`. **D3**
|
||
bounded exhaustive over the last K rounds, with *"no winning line found in
|
||
the last K rounds"* normative, never *"unwinnable"*. **D5** makes the
|
||
harness an instrument before any figure is quoted (C3). **D6** no new
|
||
crate and no port — **the L declaration was an over-declaration, recorded
|
||
as one**. **D7** the node cost is disputed 5×; T04 benchmarks it and
|
||
neither existing figure may be cited.
|
||
|
||
## Task: specify
|
||
|
||
```task
|
||
id: CB-WP-0025-T04
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "7c3ca25d-0001-44ef-a07d-0ab2d51b09a5"
|
||
```
|
||
|
||
`specs/` — extend `MetricsAndScenarios.md` or add a capability spec as the
|
||
ADR directs, with metrics, because a spec without them is prose.
|
||
|
||
Candidates, to be argued not adopted:
|
||
|
||
- **witness checkability** — every path the tool emits replays through the
|
||
existing scenario runner and ends in `group_success`. Target 100%, and it
|
||
is a hard gate, not a metric: a path that does not replay is a bug that
|
||
says the opposite of the truth;
|
||
- **search cost** — nodes and wall clock at the chosen bound, on the
|
||
recorded games we have;
|
||
- **difficulty resolution** — the smallest threshold difference the
|
||
measurement can distinguish, with its N. This is the number
|
||
`ground-game` needs, and stating it as *"we can tell 5 from 7 but not 7
|
||
from 8"* is more useful than a win rate with no error bar.
|
||
|
||
Per `ground-game`'s ruling (GROUND-WP-0004 T02), **any arithmetic finding
|
||
this produces ships a runnable reproduction and a row-level table** — never
|
||
a summed figure. A difficulty number is arithmetic, and it is exactly the
|
||
kind that has already gone wrong twice.
|
||
|
||
**Done 2026-08-05.**
|
||
[specs/RetrospectiveAnalysis.md](../specs/RetrospectiveAnalysis.md) v1.0,
|
||
and `games/ground/benches/search.rs` for D7's disputed number.
|
||
|
||
**The benchmark's own first fixture was defective — the same defect class,
|
||
one layer down.** A fixed stop at step 20 put 2p and 4p where seat 0 had
|
||
**no legal commands**, so it timed an empty `Vec` (~120 ns) and silently
|
||
skipped `validate_fold`. It now advances to a real branch **and asserts
|
||
it**.
|
||
|
||
Measured at real decision points (table in
|
||
[RetrospectiveAnalysis §3.1](../specs/RetrospectiveAnalysis.md)):
|
||
`legal_commands` 4.06–4.76 µs, `clone` 378–639 ns, `validate+fold`
|
||
**0.5–3.8 µs**.
|
||
|
||
**`clone` is in there because a search must copy state per branch**, and
|
||
`iter_batched` excludes setup from timing — leaving the budget on an
|
||
unmeasured span, which is precisely C1's mistake.
|
||
|
||
**The affordability conclusion drawn here was itself falsified by T05.**
|
||
It projected joint branching and concluded `K = 2` holds at two to four
|
||
seats. The real search exceeded 2×10⁶ nodes at **two**. See T05's record
|
||
and [RetrospectiveAnalysis §3.1](../specs/RetrospectiveAnalysis.md) —
|
||
a projection from branch widths is not a timing of a search, and this
|
||
pass has now made that mistake twice.
|
||
|
||
**§4.1 is a normative prohibition**, not a preference: a single-policy win
|
||
rate may not be reported as a difficulty. The spec carries the measured
|
||
reason — greedy 100% vs first-legal 0% on identical deals.
|
||
|
||
## Task: build the witness
|
||
|
||
```task
|
||
id: CB-WP-0025-T05
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "5f1cbda3-7427-4f23-8c67-b2e216b987a8"
|
||
```
|
||
|
||
The search, the bound, and the replayable path. Wire it to the ending page
|
||
so a lost game can be asked the question — the page CB-WP-0024 T01 is
|
||
already reworking, so land that first or expect a conflict.
|
||
|
||
**A game that was won is not asked the question.** The feature exists for
|
||
a loss.
|
||
|
||
**Controls:**
|
||
- every emitted witness replays to `group_success` through the existing
|
||
runner — asserted, not spot-checked;
|
||
- a deal constructed to be unwinnable returns *none found*, and the test
|
||
says which construction makes it so;
|
||
- the information boundary is mutation-provable: relax it, and a test
|
||
naming *that* boundary goes red. If it cannot be mutated, it was a
|
||
comment rather than a rule.
|
||
|
||
**Done 2026-08-05.** `games/ground/src/search.rs`, five tests.
|
||
|
||
**The first traversal was wrong and the diagnostic hid it.** It branched
|
||
on *the first seat with any legal command* and stopped there, so a later
|
||
seat never acted if an earlier one was already selected. Restructured
|
||
around what the rules oblige: a seat without a selection **must** select
|
||
(GR-R02) and nothing else can happen first; after Reveal the optional
|
||
actions branch freely, and the aggregate rejects `Resolve` until the
|
||
obligatory ones are done — **so the search needs no phase logic of its
|
||
own.**
|
||
|
||
**And my rewind was off by one round**, replaying the round it was meant
|
||
to search. That is why the first run reported 3 nodes and looked like a
|
||
working search.
|
||
|
||
**The measurement falsified the spec's own projection, at two seats rather
|
||
than the four §6 predicted.**
|
||
|
||
| case | result |
|
||
|---|---|
|
||
| 2p `K=1` | **exhausted**, 8,103 nodes, ~29 ms |
|
||
| 2p `K=2` | **budget cut** at 2,000,000 nodes, ~5 s |
|
||
| 3p `K=2` | win found, 41 nodes, ~157 µs |
|
||
|
||
The projection assumed a joint product per round; the search explores
|
||
sequential per-seat decisions, so orderings multiply the tree far beyond
|
||
`width^seats`.
|
||
|
||
**The asymmetry is the operative finding.** *Finding* a win is cheap —
|
||
DFS stumbles onto one in tens of nodes. *Proving none exists* needs
|
||
exhaustion. So the **witness feature is affordable now**, and the
|
||
**winnable fraction is not**, because its negative half must exhaust every
|
||
deal it counts. That is T06's problem and the spec now says so.
|
||
|
||
## Task: measure the difficulty, and hand it to ground-game
|
||
|
||
```task
|
||
id: CB-WP-0025-T06
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "723feae8-1ed4-4dd4-9330-8ec84625e5d9"
|
||
```
|
||
|
||
Run the measurement, ship it as a `make` target beside the other
|
||
instruments, and show the result in the game — the maintainer asked for it
|
||
to be visible, and a number in a file will not calibrate anything.
|
||
|
||
Then send it to `ground-game` **against GROUND-WP-0005**, which is active
|
||
with both tasks waiting on exactly this. Per CB-WP-0022 T06, it lands as a
|
||
file in their repo under their workplan, not only an inbox entry — *the
|
||
message that sat unread for four days is the baseline to beat*.
|
||
|
||
**Controls:**
|
||
- the number regenerates from a single command, and `facts.toml` carries
|
||
it if anything else quotes it (§Single source of fact — `make
|
||
facts-check`);
|
||
- the report carries the row-level table the ruling requires;
|
||
- **the seed range and policy are in the number's name**, not in a
|
||
footnote.
|
||
|
||
## Task: evidence
|
||
|
||
```task
|
||
id: CB-WP-0025-T07
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "2616e06c-5d88-4680-be18-4452515dff21"
|
||
```
|
||
|
||
`evidence/CB-EV-0023-*.md`.
|
||
|
||
- **Was the game winnable**, for the maintainer's actual lost game. That is
|
||
the acceptance test with a face on it.
|
||
- **What the honest search cost against the omniscient one**, since the
|
||
review will have pressed hardest there.
|
||
- **Whether the difficulty measurement moved ground-game**, or sat.
|
||
- **What tier L cost against what it caught** — third full-weight L pass in
|
||
the project, and the second in this chaos window.
|
||
- **Quote CB-WP-0024's cost by re-running the instrument.**
|