diff --git a/Makefile b/Makefile index 102f883..7db140f 100644 --- a/Makefile +++ b/Makefile @@ -125,7 +125,6 @@ self-tests: $(PY) $(TOOLS)/runtime-metrics.py --self-test $(PY) $(TOOLS)/replay-test.py --self-test $(PY) $(TOOLS)/design.py --self-test - cargo run --release -q -p games-ground --example difficulty -- --self-test $(PY) $(TOOLS)/edition-check.py --self-test # T01 positive control: prove the environment fix, do not assume it. Runs @@ -161,12 +160,6 @@ facts-check: facts-gen: $(PY) $(TOOLS)/facts.py --gen -# CB-WP-0025 T06: the difficulty table (specs/RetrospectiveAnalysis.md §4). -# Winnable fraction from the solver plus a PLURAL policy panel -- a single -# policy's win rate may not be reported as a difficulty (§4.1). -difficulty: - @cargo run --release -q -p games-ground --example difficulty - # CB-WP-0022 T05: the design-finding register, reported over # specs/GroundRules.md. Shows the QUEUE by default; the log of closed # findings is a line, not a listing, because a default view that mixes diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 72be584..4522972 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -32,7 +32,7 @@ | workplan | CB-WP-0022 | done | — | workplans/CB-WP-0022-the-design-instrument.md | | workplan | CB-WP-0023 | done | — | workplans/CB-WP-0023-solve-legality.md | | workplan | CB-WP-0024 | done | — | workplans/CB-WP-0024-the-table-you-can-watch.md | -| workplan | CB-WP-0025 | active | — | workplans/CB-WP-0025-could-we-have-won.md | +| workplan | CB-WP-0025 | ready | — | workplans/CB-WP-0025-could-we-have-won.md | | workplan | CB-WP-0026 | done | — | workplans/CB-WP-0026-collect-the-rulings.md | | task | CB-WP-0001-T01 | done | — | workplans/CB-WP-0001-inner-loop.md | | task | CB-WP-0001-T02 | done | — | workplans/CB-WP-0001-inner-loop.md | @@ -162,8 +162,8 @@ | task | CB-WP-0024-T03 | done | — | workplans/CB-WP-0024-the-table-you-can-watch.md | | task | CB-WP-0024-T04 | done | — | workplans/CB-WP-0024-the-table-you-can-watch.md | | task | CB-WP-0024-T05 | done | — | workplans/CB-WP-0024-the-table-you-can-watch.md | -| task | CB-WP-0025-T01 | done | — | workplans/CB-WP-0025-could-we-have-won.md | -| task | CB-WP-0025-T02 | done | — | workplans/CB-WP-0025-could-we-have-won.md | +| task | CB-WP-0025-T01 | todo | — | workplans/CB-WP-0025-could-we-have-won.md | +| task | CB-WP-0025-T02 | todo | — | workplans/CB-WP-0025-could-we-have-won.md | | task | CB-WP-0025-T03 | todo | — | workplans/CB-WP-0025-could-we-have-won.md | | task | CB-WP-0025-T04 | todo | — | workplans/CB-WP-0025-could-we-have-won.md | | task | CB-WP-0025-T05 | todo | — | workplans/CB-WP-0025-could-we-have-won.md | diff --git a/decisions/ADR-0013-could-we-have-won.md b/decisions/ADR-0013-could-we-have-won.md deleted file mode 100644 index 730f8ed..0000000 --- a/decisions/ADR-0013-could-we-have-won.md +++ /dev/null @@ -1,230 +0,0 @@ -# ADR-0013: answer the retrospective question, and measure difficulty with the solver rather than with a bot - -status: accepted -date: 2026-08-05 -decided by: agent, under the standing loop authorization -tier: L (structural L — a new analysis capability whose numbers `ground-game` -will tune against; chaos d8=6 → no override). Tier L: separate survey, -adversarial review, then this document. -references: [CB-WP-0025](../workplans/CB-WP-0025-could-we-have-won.md), -[CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md), -[challenge](../history/260805-could-we-have-won-challenge.md) / -[response](../history/260805-could-we-have-won-response.md), -[ADR-0012](ADR-0012-the-design-instrument.md) (admissibility), -[GameDesign.md](../specs/GameDesign.md), -GROUND-WP-0004 T02, GROUND-WP-0005 (blocked on D4) - -## Context - -The maintainer asked two things: *"we lost — could we have won, and how?"* -and *"do we have difficulty estimations?"* - -**The survey answered the second and was wrong.** It measured -`GreedyPolicy` winning 200/200 at five and six seats and called the game -too easy there. A `FirstLegal` policy — `legal[0]`, no heuristic — scores -**0%** on the same deals. Two unsophisticated agents span the whole range, -so the measurement was about the policy. - -That failure is not incidental to this ADR; **it determines D4.** - -## The premise that changed, and it changes the algorithm - -The survey said exhaustive search was impossible and reached for -determinized sampling, which carries strategy fusion. **Both halves were -wrong.** - -- Its per-node cost was **30–50× too high** (a timer bracketing whole - games). Corrected: ~3–4 µs per `legal_commands` call, with the exact - figure still disputed (§D7). -- Bounded exhaustive search is **affordable**: measured ~3 s over the last - two rounds at three seats. - -Joint branching, treating the table as one co-operative agent — the -product over seats of the measured per-seat branching: - -| seats | per-seat mean | joint per round | last 2 rounds | -|---|---:|---:|---:| -| 2 | 4.7 | ~22 | ~5×10² | -| 3 | 7.4 | ~405 | ~1.6×10⁵ | -| 4 | 9.1 | ~754 | ~5.7×10⁵ | - -Against a ~10⁵–10⁶ node budget, **the last two rounds are exhaustively -searchable at two, three and four seats.** Five rounds is not, at any seat -count. - ---- - -## D1 — answer the *retrospective* question, and say so in those words - -Three questions were on the table (CB-RES-0008 §3). The tool answers: - -> **"Given the deal as it actually was, was there a line of play that -> reached the threshold — and here is one."** - -**Strategy fusion does not apply to this question, and that is the whole -reason it is the affordable one.** Fusion is a defect of *aggregating over -determinizations to choose a move*: the search picks different actions in -states the player cannot distinguish. **After the game there is one -world.** The deck is known, the deal is known, and a search over that -single world produces a line that is executable in it — because it is the -only world there is. - -The survey treated fusion as an obstacle to this pass. It is an obstacle -to a *playing* engine. We are not building one. - -**What remains true is that the line may have been unfindable at the -time**, and D2 handles that by annotation rather than by refusing to -answer. - -**On screen it is called** *"was this deal winnable?"* — never *"how you -should have played"*. The distinction is the honest content of the -feature, and a label that overclaims turns a true answer into a false -lesson. - -## D2 — run on `GroundState`, and mark each move's information dependence - -The survey's preferred guarantee was structural: search a `GroundView` so -the boundary cannot be crossed. **It is not implementable** — a view -cannot `fold` events, so a search needs a state it may not see. The survey -said so in §6 and was right to. - -Decision: **search `GroundState`** — legitimate here, because post-game -the deal is public (`solution_discard` already is, and the game is over) — -and move the honesty guarantee to something checkable: - -> **Every move in an emitted witness is marked `visible` or `hidden`.** -> A move is `visible` if, at the point it is played, everything it depends -> on was in the acting seat's projection: the target Problem face-up, the -> Solution in that seat's own hand. Otherwise `hidden`. - -So a witness reads *"you could have won — but two of these six moves -needed a card you had no way to know was coming."* **That is a more useful -answer than either extreme**, and it is computed from `project()`, which -already exists and is already tested. - -**Falsifier:** if a witness is emitted whose moves are all marked -`visible` but which no seat could actually have chosen, the marking is -wrong and D2 has failed. A test constructs exactly that case. - -## D3 — bounded exhaustive over the endgame, `K` rounds, and honest wording - -**Exhaustive search over the last `K` rounds**, with the table treated as -one co-operative agent choosing joint selections. `K = 2` by default, -which the measurements put inside budget at 2–4 seats. - -- The bound is **rounds**, not nodes or seconds, because rounds are what a - player understands: *"winnable from round 4"* means something; *"winnable - within 100,000 nodes"* does not. -- A node budget is a **secondary** cut that aborts with a stated reason, - so a wide table cannot hang the page. -- **Wording is normative.** When no line is found the tool says - **"no winning line found in the last K rounds"** — never *"unwinnable"*. - A bounded search that claims unwinnability is lying, and this is the - sentence the maintainer will read. - -**Not chosen: determinized sampling (PIMC), ISMCTS.** Both are for playing -under uncertainty. Here there is one world (D1), so they would add strategy -fusion to a problem that does not have it. - -## D4 — difficulty is the **winnable fraction**, not any bot's win rate - -**This is the decision the review forced, and it is the useful half of the -pass.** - -A single-policy win rate cannot be a difficulty: two trivial policies span -0–100% on the same deals. Worse, *improving the bot would make the game -"easier"* without a rule changing — a measure that moves when the -measurer improves is not measuring the thing. - -The solver supplies a policy-independent alternative: - -> **Winnable fraction** — over N deals at a seat count, the proportion in -> which the search finds *any* winning line within its bound. - -That is a property of **the deal distribution and the threshold**, which -is exactly what `ground-game` tunes. It is the number GROUND-WP-0005 is -blocked on, and the bot rate never was. - -Difficulty therefore ships as **a small table, never one number**: - -| column | what it is | -|---|---| -| winnable fraction | can the deal be won at all (bounded, K stated) | -| reference-policy win rate | what a stated bot achieves — **named policy** | -| skill gap | the difference: how much play has to supply | - -**Every rate carries its policy, its N, its seed range and its K in the -number's name**, not in a footnote. A figure that loses them is -inadmissible under GameDesign §1.2. - -**Bounded-below caveat, stated because it will be quoted:** the winnable -fraction from a K-round search is a **lower bound** on true winnability — -a deal unwinnable in the last 2 rounds may have been winnable in round 1. -The report says "winnable-from-round-(6−K)", never "winnable". - -## D5 — the harness becomes an instrument before any figure is quoted - -C3 established that `difficulty-baseline.rs` has no assertions, no -`--self-test` and no `make` target — nothing can turn it red. Under -CB-WP-0022 T05's own `role` distinction it is a `default` artifact wearing -a `counterexample` label, and **GameDesign §1.3 makes it inadmissible.** - -Required before T06 reports anything: - -- **positive controls** — a deal constructed to be unwinnable returns - none; a deal constructed to be winnable returns a witness that replays; -- **`--self-test`**, wired into `make self-tests` like every other - reporting tool; -- **`make difficulty`** (or equivalent), so the figure regenerates from - one command; -- the **policy panel is plural**: at least `greedy`, `random` and - `first-legal`, because the spread between them is what C4 exposed and - hiding it would restore the error. - -## D6 — it lives in `games/ground`, not a new crate - -The search needs `validate`, `fold`, `legal_commands` and `project` — -all of `games_ground`. A separate crate would either re-export the -aggregate or take a dependency on it and add nothing. - -**The tier was declared L on the assumption of a new capability port. -There is no port**, and that over-declaration is recorded rather than -hidden — it is a data point for the tier rules, and the L weight paid for -itself twice over regardless (§Consequences). - -`cb-play` gains a mode to ask the question about a finished game; the -difficulty sweep is an example/binary, as the baseline is. - -## D7 — the per-node cost is unsettled and T04 must benchmark it - -The author measured **3.0–4.1 µs**, the reviewer **15.6–20.4 µs**, by -different isolations. Both agree the published 112–161 µs was wrong by -1–2 orders; neither has established which is right. - -**T04 benchmarks it with `criterion`** — already a dev-dependency, already -used by `benches/synthetic.rs` — and the spec quotes that number and no -other. **Neither figure above may be cited**, including by this ADR. - -## Consequences - -- `specs/` gains the witness contract and the difficulty table's shape - (T04), plus the benchmarked node cost. -- T05 builds the K-round search, the `visible`/`hidden` marking, and the - replay check. -- **T06's payload changes completely.** It reports a winnable fraction and - a policy panel to GROUND-WP-0005 — *not* the withdrawn "too easy at 5–6 - seats". The withdrawal itself is reported, per ADR-0012 D5. -- The register gains the withdrawn finding as `inconsistent` / - `withdrawn`, so it is in the log rather than forgotten. - -## What was rejected - -| rejected | why | -|---|---| -| determinized sampling / PIMC / ISMCTS | strategy fusion, added to a question that has one world (D1) | -| a view-only search as a structural boundary | not implementable — a view cannot fold events | -| refusing to answer unless the line was findable at the time | throws away a true and useful answer; annotate instead (D2) | -| a single bot win rate as "difficulty" | two trivial policies span 0–100% on the same deals (C4) | -| "unwinnable" as output wording | a bounded search cannot know it (D3) | -| a new crate | no port exists; it would re-export the aggregate (D6) | -| quoting either measured node cost | they disagree 5× and neither is established (D7) | diff --git a/evidence/CB-EV-0024-could-we-have-won.md b/evidence/CB-EV-0024-could-we-have-won.md deleted file mode 100644 index 94a2b49..0000000 --- a/evidence/CB-EV-0024-could-we-have-won.md +++ /dev/null @@ -1,161 +0,0 @@ -# CB-EV-0024 — could we have won - -CB-WP-0025 T07. Tier L (structural L declared; **there was no port — -D6 records it as an over-declaration**; chaos d8=6 → no override). -Declaration 8 of chaos window 2. Closed 2026-08-05. - -**Delivered:** [CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md) -with a runnable baseline, an adversarial review that withdrew its -headline, [ADR-0013](../decisions/ADR-0013-could-we-have-won.md), -[RetrospectiveAnalysis.md](../specs/RetrospectiveAnalysis.md), -`games/ground/src/search.rs`, `benches/search.rs`, -`examples/difficulty.rs`, `make difficulty`, and a report to -GROUND-WP-0005 that opens with a retraction. - ---- - -## 1. Was the maintainer's game winnable? - -The question the pass exists for. **Yes, and the tool answers it.** - -For 3-player seed 7, searched over the last round: a **10-move winning -line**, found in 41 nodes, which replays through `validate`/`fold` to -`group_success`. For 2-player seed 7 — a game greedy lost — the search -returns **`NoneFound { exhausted: true }` in 8,103 nodes**: no winning line -existed in the final round. That is a real negative, not a budget cut, and -the type keeps the two apart. - -**The feature works. The difficulty half is where the trouble was.** - -## 2. What the review cost against what it caught - -Second tier-L review in this project, second time it caught a false -headline that every automated gate passed. `make all` was green -throughout. - -| caught | by | -|---|---| -| the headline finding was about `GreedyPolicy`, not GROUND (C4) | **only the review** | -| the node cost was 30–50× too high (C1) | **only the review** | -| exhaustive search is not out (C6) | **only the review** | -| the finding failed our own admissibility rule (C3) | **only the review** | -| the maintainer's losses were misattributed (C5) | **only the review** | -| the benchmark's fixture timed an empty `Vec` | building it | -| the traversal skipped later seats; the rewind was off a round | building it | -| `K=2` is not exhaustible at two seats | **running it** | -| the winnable fraction is still greedy-conditioned | writing it down | - -**Five of nine from the review, four from execution.** The four that -execution caught were all in work written *after* the review, which is -some evidence that the review's discipline transferred rather than that -the remaining errors were easier. - -**The reviewer under-ranked their own best challenge.** They put C4 -fourth. Measured, it is the one that kills the finding — and I said so in -the response rather than accepting their ranking, which is the correct use -of a review: material to check, not a verdict to adopt. - -## 3. The error family, now with five instances - -CB-EV-0019 §1 named it: *this project's arithmetic errors are not in the -arithmetic.* A number computed correctly against the wrong base, span, or -subject. - -| # | pass | the wrong thing | -|---|---|---| -| 1 | CB-WP-0021 | `csv` cost against AM-4a, which never sees the code | -| 2 | ground-game report | *"12 in the file"* — a sum with no deal table | -| 3 | CB-WP-0018 | SOLVE's inertness attributed to the wrong condition | -| 4 | **this pass, C1** | a timer bracketing whole games, divided by decisions | -| 5 | **this pass, C4** | a win rate attributed to the game, not the policy | - -**And two projections published in place of measurements**, both in this -pass: the survey's `7.4^15` hand-wave, and the spec's joint-branching -estimate that the real search exceeded at *two* seats where §6 predicted -four. - -**`facts-check` catches copies that disagree. Nothing catches a number -computed correctly against the wrong denominator**, and after five -instances that is a standing gap rather than a run of bad luck. Naming it -is what this file can do; closing it is not this pass's. - -## 4. What the difficulty measurement actually shows - -`make difficulty`, 60 seeds: - -| seats | winnable | greedy | random | first-legal | spread | -|---|---:|---:|---:|---:|---:| -| 2 | 60% | 60.0% | 5.0% | 76.7% | 71.7 | -| 3 | 93% | 88.3% | 6.7% | 25.0% | 81.7 | -| 4 | 100% | 93.3% | 6.7% | 30.0% | 86.7 | -| 5 | 100% | 100.0% | 3.3% | 0.0% | 100.0 | -| 6 | 100% | 100.0% | 3.3% | 0.0% | 100.0 | - -**The `spread` column is the finding.** 71.7 to 100.0 points between three -trivial policies. It is the evidence for the prohibition in -RetrospectiveAnalysis §4.1, and it is why the withdrawn headline was -meaningless rather than merely imprecise. - -**The 5–6 seat rows point the opposite way from the withdrawn claim.** -`first-legal` scores 0% where greedy scores 100% — the widest spread in -the table. Whatever is happening at five and six seats makes *play matter -more*, not less. The survey read 100% as "trivially easy"; the panel -suggests "high variance in what an agent must get right". **Neither is -established**, and saying so is the honest state. - -**The winnable fraction is still not policy-free**, and the tool says so -in its own output: it is *"winnable from where greedy got to"*, because -searching from round 1 is unaffordable. Presenting it as a property of the -deal would be this pass's own error in a subtler form — which is exactly -how a corrected project reintroduces a defect, so it is stated in the -code, the spec, the report and here. - -## 5. Tier L's cost, and the over-declaration - -**The tier was declared L on the assumption of a new capability port. -There is no port** — the search uses `validate`, `fold`, `legal_commands` -and `project`, so it lives in `games/ground` (D6). By the structural -trigger this was an **M**. - -**And the L weight paid for itself anyway**, twice: the adversarial review -is L-only, and without it a false difficulty claim reaches a repo that is -blocked waiting for exactly that number. **That is an argument the tier -table does not currently make** — the trigger asks *does this create a -port*, when the question that predicted value here was *will this pass -produce a number someone else acts on*. Recorded for whoever revisits -§Loop tiers; not proposed as a change from one instance. - -## 6. Chaos window 2 - -**Declaration 8 of 12.** Structural L (over-declared), d8 = 6, no -override. - -**Eleven declarations, no 8 rolled.** The window will close with **zero -overrides**, so its retirement condition — *retire if an override changes -nothing twice running* — is **untestable**. Window 1 closed with two -overrides at d4 and both changed the outcome. - -**The d4 → d8 cut bought rarity at the cost of evidence**, and CB-EV-0015 -did not price that. At d8 over twelve declarations the expected count is -1.5, so zero is unremarkable *as a sample* and damning *as a design*: a -mechanism that produces no data in a full window cannot be evaluated by -the window. **This is the second window's headline finding about the loop -itself** and belongs in whatever closes it. - -## 7. Cost - -CB-WP-0024's cost, by re-running the instrument: `make cost`. Not inlined -(§Single source of fact), and see CB-EV-0019 §4 — the chain breaks beyond -roughly four passes and still has no bound. - -## Open after this pass - -- **Search from round 1** would remove the greedy confound. Needs - transposition or move-ordering; neither built. -- **Difficulty resolution** — the smallest distinguishable threshold - change, with its N — is required by the spec and not yet supplied. -- **The witness is not wired to the ending page.** T05 built the search; - the browser cannot yet ask it. That is the maintainer-facing half of - remark 2 and it is not done. -- **The wrong-denominator family has no control** (§3), at five instances. -- **The chaos window's retirement condition is untestable** (§6). diff --git a/games/ground/Cargo.toml b/games/ground/Cargo.toml index 1f8e627..9d0768b 100644 --- a/games/ground/Cargo.toml +++ b/games/ground/Cargo.toml @@ -27,10 +27,5 @@ serde_json.workspace = true name = "synthetic" harness = false -# CB-WP-0025 T04 / ADR-0013 D7: the node cost the spec quotes. -[[bench]] -name = "search" -harness = false - [lints] workspace = true diff --git a/games/ground/benches/search.rs b/games/ground/benches/search.rs deleted file mode 100644 index 75c7fd5..0000000 --- a/games/ground/benches/search.rs +++ /dev/null @@ -1,128 +0,0 @@ -//! CB-WP-0025 T04 — what one search node actually costs. -//! -//! **ADR-0013 D7 exists because two measurements disagreed by 5×.** The -//! survey published 112–161 µs/node from a timer that bracketed two -//! `setup`s and a whole greedy game (C1). The author's re-measurement said -//! 3.0–4.1 µs with `Instant::now()` around each call; the adversarial -//! reviewer's isolation said 15.6–20.4 µs. Both agreed the published -//! figure was wrong by 1–2 orders and neither established which -//! replacement was right. -//! -//! So the spec quotes **this** and nothing else. `criterion` handles the -//! things hand-rolled timing gets wrong here: per-call clock overhead -//! against a ~microsecond subject, warm-up, and run-to-run variance — -//! which is what let the survey's figure move 161 → 112 between two runs -//! of the same unmodified binary. -//! -//! Two subjects, because a search node is not one call: -//! -//! * `legal_commands` — enumerating a seat's options; -//! * `validate + fold` — taking one branch, which any search does per -//! child and which the survey never separated out. - -use cb_game_runtime::{ScenarioGame, Setup}; -use cb_kernel::{Actor, Aggregate, PlayerId}; -use criterion::{criterion_group, criterion_main, BatchSize, Criterion}; -use games_ground::bot::{legal_commands, play, GreedyPolicy, Policy}; -use games_ground::GroundState; -use std::collections::BTreeMap; - -fn setup(players: u8, seed: u64) -> GroundState { - GroundState::setup( - &Setup { - players, - preset: format!("standard-{players}p"), - patch: BTreeMap::new(), - }, - seed, - ) - .expect("preset") -} - -/// A **mid-game state at a real decision point** for `seat`. -/// -/// Not a fresh deal: at deal time most branches do not exist yet, and a -/// node cost taken there would flatter any search proposal. -/// -/// **And not a fixed step count either.** The first version stopped at -/// step 20 for every seat count, which put 2p and 4p in a state where -/// seat 0 had *no* legal commands at all — so the benchmark reported -/// ~120 ns (the cost of returning an empty `Vec`) and silently skipped -/// `validate_fold` because there was nothing to validate. A fixture that -/// measures the empty case and calls it a node cost is the same defect -/// class this whole pass exists to correct, one layer down. -/// -/// So: advance until the seat genuinely has a choice, and assert it. -fn midgame(players: u8, seed: u64, seat: PlayerId) -> GroundState { - let mut ps: Vec> = (0..players) - .map(|_| Box::new(GreedyPolicy) as Box) - .collect(); - let game = play(setup(players, seed), &mut ps).expect("a complete game"); - let mut state = setup(players, seed); - let mut best: Option = None; - for (i, (actor, cmd)) in game.steps.iter().enumerate() { - // Past the opening, take the first state where the seat has a real - // branch. `> 1` rather than `> 0`: a forced move is not a node. - if i >= 8 && legal_commands(&state, seat).len() > 1 { - best = Some(state.clone()); - break; - } - if let Ok(events) = state.validate(*actor, cmd) { - for e in &events { - state.fold(e); - } - } - } - let state = best.expect("a mid-game state where the seat has a choice"); - assert!( - legal_commands(&state, seat).len() > 1, - "benchmark fixture has no branch to measure — it would time the empty case" - ); - state -} - -fn bench(c: &mut Criterion) { - for players in [2u8, 3, 4] { - let seat = PlayerId(0); - let state = midgame(players, 7, seat); - let width = legal_commands(&state, seat).len(); - println!(" fixture {players}p: {width} legal commands at the measured node"); - - c.bench_function(&format!("legal_commands/{players}p"), |b| { - b.iter(|| std::hint::black_box(legal_commands(&state, seat))) - }); - - // A search must COPY the state per branch (or undo, which we do - // not have). `iter_batched` excludes setup from the timing, so - // without this the budget would rest on an unmeasured span — - // which is the exact mistake C1 caught in the survey. - c.bench_function(&format!("clone/{players}p"), |b| { - b.iter(|| std::hint::black_box(state.clone())) - }); - - // One branch taken: what a search pays per CHILD, on top of - // enumeration. The survey folded this into "us/node" without - // separating it, and a search's real cost is enumeration once plus - // this per child. - let legal = legal_commands(&state, seat); - if let Some(cmd) = legal.first() { - c.bench_function(&format!("validate_fold/{players}p"), |b| { - b.iter_batched( - || state.clone(), - |mut s| { - if let Ok(events) = s.validate(Actor::Player(seat), cmd) { - for e in &events { - s.fold(e); - } - } - std::hint::black_box(s) - }, - BatchSize::SmallInput, - ) - }); - } - } -} - -criterion_group!(benches, bench); -criterion_main!(benches); diff --git a/games/ground/examples/difficulty-baseline.rs b/games/ground/examples/difficulty-baseline.rs index 1737dff..51140ac 100644 --- a/games/ground/examples/difficulty-baseline.rs +++ b/games/ground/examples/difficulty-baseline.rs @@ -1,21 +1,5 @@ //! CB-RES-0008's runnable baseline (CB-WP-0025 T01). //! -//! ── SUPERSEDED 2026-08-05 by `difficulty.rs` (CB-WP-0025 T06) ────────── -//! -//! Kept as the survey's dated snapshot, NOT as a live instrument, for the -//! same reason `tools/design-baseline.py` was kept: it is the evidence for -//! how a wrong number got into a survey. -//! -//! **Its headline was withdrawn.** It reported `GreedyPolicy` winning -//! 200/200 at five and six seats and the survey called the game too easy -//! there. A `FirstLegal` policy scores **0%** on the same deals — the -//! measurement was about the policy. See the challenge (C4) and the -//! response. -//! -//! **And it is inadmissible under GameDesign §1.3**: no assertions, no -//! `--self-test`, no `make` target, so nothing can turn it red. -//! `difficulty.rs` carries all three. -//! //! Two numbers the survey needs and cannot cite from anyone else, because //! they are about **our** game on **our** machine: //! diff --git a/games/ground/examples/difficulty.rs b/games/ground/examples/difficulty.rs deleted file mode 100644 index 4992767..0000000 --- a/games/ground/examples/difficulty.rs +++ /dev/null @@ -1,304 +0,0 @@ -//! CB-WP-0025 T06 — the difficulty table, and the instrument that can fail. -//! -//! Implements `specs/RetrospectiveAnalysis.md` §4. Replaces -//! `difficulty-baseline.rs`, which CB-WP-0022's admissibility rule and -//! this pass's own review both found inadmissible: it had no assertions, -//! no `--self-test` and no `make` target, so nothing could turn it red. -//! -//! ## What it will not print -//! -//! **A single policy's win rate as a difficulty** (§4.1). Measured on -//! identical deals, `GreedyPolicy` wins 100% at five and six seats where -//! `FirstLegal` wins 0%. The panel is plural for that reason, and the -//! spread is reported rather than hidden. -//! -//! ```text -//! cargo run --release -p games-ground --example difficulty [--self-test] -//! ``` - -use cb_game_runtime::{ScenarioGame, Setup}; -use cb_kernel::{Aggregate, PlayerId}; -use games_ground::bot::{play, Choice, GreedyPolicy, Policy, RandomPolicy}; -use games_ground::search::{winnable_within, Verdict}; -use games_ground::{GroundCommand, GroundState}; - -/// Seeds per cell. Small by default: the winnable fraction needs an -/// exhaustive search per deal, which is the expensive half (§3.1). -const SEEDS: u64 = 60; -/// §3's measured limit: `K=1` is exhaustible, `K=2` is not. -const K: u8 = 1; -const BUDGET: usize = 2_000_000; - -/// A policy with no heuristic at all. **It is in the panel because it is -/// what falsified the survey's headline** — it beats greedy at two seats -/// and scores zero at six. -struct FirstLegal; -impl Policy for FirstLegal { - fn name(&self) -> &'static str { - "first-legal" - } - fn choose( - &mut self, - _s: &GroundState, - _seat: PlayerId, - _legal: &[GroundCommand], - _may_pass: bool, - ) -> Choice { - Choice::Command(0) - } -} - -fn setup(players: u8, seed: u64) -> Option { - GroundState::setup( - &Setup { - players, - preset: format!("standard-{players}p"), - patch: Default::default(), - }, - seed, - ) - .ok() -} - -fn policies(kind: &str, players: u8, seed: u64) -> Vec> { - (0..players) - .map(|i| -> Box { - match kind { - "random" => Box::new(RandomPolicy::new(seed ^ u64::from(i))), - "first-legal" => Box::new(FirstLegal), - _ => Box::new(GreedyPolicy), - } - }) - .collect() -} - -/// Games a named policy actually wins. -fn policy_wins(kind: &str, players: u8) -> (u32, u32) { - let (mut wins, mut played) = (0, 0); - for seed in 0..SEEDS { - let Some(state) = setup(players, seed) else { - continue; - }; - let mut ps = policies(kind, players, seed); - let Ok(game) = play(state, &mut ps) else { - continue; - }; - if let Some(o) = &game.state.outcome { - played += 1; - if o.group_success { - wins += 1; - } - } - } - (wins, played) -} - -/// Rewind a played game to the start of its last `k` rounds. -fn last_rounds(players: u8, seed: u64, k: usize) -> Option { - let mut ps = policies("greedy", players, seed); - let game = play(setup(players, seed)?, &mut ps).ok()?; - let total = game - .steps - .iter() - .filter(|(_, c)| matches!(c, GroundCommand::EndRound)) - .count(); - let mut st = setup(players, seed)?; - let mut ends = 0usize; - for (a, c) in &game.steps { - if let Ok(ev) = st.validate(*a, c) { - for e in &ev { - st.fold(e); - } - } - if matches!(c, GroundCommand::EndRound) { - ends += 1; - if ends >= total.saturating_sub(k) { - break; - } - } - } - Some(st) -} - -/// In what fraction of deals does a winning line exist in the last `K` -/// rounds? -/// -/// **Less policy-dependent than a win rate, but NOT policy-free, and -/// saying otherwise would repeat this pass's own error in a subtler -/// form.** The position searched is the one `GreedyPolicy` left at the -/// start of the last round, so the figure is *"winnable from where greedy -/// got to"*. A genuinely policy-free measure would search from round 1, -/// which §3.1 measured as unaffordable. -/// -/// What it does buy: the last round's outcome no longer depends on which -/// agent plays it, so the measure is insensitive to exactly the variation -/// that made the bot rate meaningless (§4.1). -/// -/// Deals where the search was cut by its budget are **not counted either -/// way** — they are reported separately, because folding "we stopped -/// looking" into "not winnable" is exactly the collapse §2.3 forbids. -fn winnable_fraction(players: u8) -> (u32, u32, u32) { - let (mut yes, mut decided, mut undecided) = (0, 0, 0); - for seed in 0..SEEDS { - let Some(state) = last_rounds(players, seed, K as usize) else { - continue; - }; - match winnable_within(&state, K, BUDGET) { - Verdict::Winnable { .. } => { - yes += 1; - decided += 1; - } - Verdict::NoneFound { - exhausted: true, .. - } => decided += 1, - Verdict::NoneFound { - exhausted: false, .. - } => undecided += 1, - } - } - (yes, decided, undecided) -} - -fn report() { - println!("difficulty — specs/RetrospectiveAnalysis.md §4\n"); - println!( - " winnable-from-round-{} over {SEEDS} seeds, budget {BUDGET} nodes", - 6 - K - ); - println!(" policy win rates over the same {SEEDS} seeds\n"); - println!(" seats winnable greedy random first-legal spread undecided"); - - for players in [2u8, 3, 4, 5, 6] { - let (yes, decided, undecided) = winnable_fraction(players); - let pct = |(w, n): (u32, u32)| { - if n == 0 { - -1.0 - } else { - 100.0 * f64::from(w) / f64::from(n) - } - }; - let g = pct(policy_wins("greedy", players)); - let r = pct(policy_wins("random", players)); - let f = pct(policy_wins("first-legal", players)); - let spread = [g, r, f].iter().cloned().fold(f64::MIN, f64::max) - - [g, r, f].iter().cloned().fold(f64::MAX, f64::min); - let wf = if decided == 0 { - "n/a".to_string() - } else { - format!("{:.0}%", 100.0 * f64::from(yes) / f64::from(decided)) - }; - println!( - " {players}p {wf:>6} {g:>5.1}% {r:>5.1}% {f:>5.1}% \ - {spread:>5.1} {undecided:>3}" - ); - } - - println!( - "\n WINNABLE is conditioned on GREEDY's play up to the last round —\n \ - it is 'winnable from where greedy got to', not a property of the\n \ - deal alone. Searching from round 1 is unaffordable (spec §3.1).\n\n \ - It is also a LOWER BOUND: a K={K} search cannot see a line that\n \ - needed an earlier round. `undecided` are deals whose search hit the\n \ - node budget — they are excluded from the fraction, not counted as\n \ - unwinnable.\n\n \ - SPREAD is the range across three policies. Where it is large, no\n \ - single policy's rate says anything about the game (§4.1)." - ); -} - -fn self_test() -> i32 { - let mut ok = true; - let mut check = |name: &str, cond: bool, detail: String| { - ok &= cond; - println!( - " [{}] {name}{}", - if cond { "ok " } else { "FAIL" }, - if detail.is_empty() { - String::new() - } else { - format!(" — {detail}") - } - ); - }; - - // The control the old harness never had: a search that finds a line - // must produce one that REPLAYS. §2.1 is a gate, not a metric. - let state = last_rounds(3, 7, K as usize).expect("a 3p game"); - match winnable_within(&state, K, BUDGET) { - Verdict::Winnable { line, .. } => { - let mut replay = state.clone(); - let mut good = true; - for m in &line { - match replay.validate(m.actor, &m.command) { - Ok(ev) => { - for e in &ev { - replay.fold(e); - } - } - Err(_) => good = false, - } - } - let won = replay.outcome.as_ref().is_some_and(|o| o.group_success); - check( - "a witness replays to a win", - good && won, - format!("{} moves", line.len()), - ); - } - Verdict::NoneFound { .. } => check( - "a witness replays to a win", - false, - "3p seed 7 found no line — the fixture moved".into(), - ), - } - - // The negative: a search must be able to return nothing, exhaustively. - let lost = last_rounds(2, 7, 1).expect("a 2p game"); - match winnable_within(&lost, 1, BUDGET) { - Verdict::NoneFound { exhausted, nodes } => check( - "an unwinnable position is reported as searched-out", - exhausted && nodes > 100, - format!("{nodes} nodes"), - ), - Verdict::Winnable { .. } => check( - "an unwinnable position is reported as searched-out", - false, - "found a win in a game 2p seed 7 lost".into(), - ), - } - - // A budget of one must NOT claim exhaustion — the distinction §2.3 - // rests on. - match winnable_within(&state, K, 1) { - Verdict::NoneFound { exhausted, .. } => check( - "a budget cut is not reported as exhaustion", - !exhausted, - String::new(), - ), - Verdict::Winnable { .. } => check( - "a budget cut is not reported as exhaustion", - false, - String::new(), - ), - } - - // §4.1's reason, asserted rather than asserted-about: the panel must - // actually disagree, or reporting three policies is ceremony. - let g = policy_wins("greedy", 6); - let f = policy_wins("first-legal", 6); - check( - "the policy panel disagrees, so no single rate is a difficulty", - g.0 != f.0, - format!("greedy {}/{}, first-legal {}/{}", g.0, g.1, f.0, f.1), - ); - - println!("difficulty self-test (positive control)"); - i32::from(!ok) -} - -fn main() { - if std::env::args().any(|a| a == "--self-test") { - std::process::exit(self_test()); - } - report(); -} diff --git a/games/ground/src/lib.rs b/games/ground/src/lib.rs index 5f01712..c0e67b5 100644 --- a/games/ground/src/lib.rs +++ b/games/ground/src/lib.rs @@ -20,12 +20,6 @@ pub mod view; #[cfg(feature = "scenarios")] pub mod record; -/// *Was this deal winnable?* — the retrospective search (CB-WP-0025 T05, -/// ADR-0013). Uses only `validate`/`fold`/`legal_commands`, so it lives -/// beside the aggregate rather than in a crate that would re-export it -/// (ADR-0013 D6). -pub mod search; - #[cfg(feature = "scenarios")] use cb_game_runtime::{parse_actor, CommandStep, ScenarioGame, Setup}; use cb_kernel::{Actor, Aggregate, ChaChaRng, KernelRng, PlayerId, Rejection, Seed}; diff --git a/games/ground/src/search.rs b/games/ground/src/search.rs deleted file mode 100644 index aa4a25f..0000000 --- a/games/ground/src/search.rs +++ /dev/null @@ -1,412 +0,0 @@ -//! Was this deal winnable — and here is one line (CB-WP-0025 T05). -//! -//! Implements [`specs/RetrospectiveAnalysis.md`]. The question is -//! **retrospective**: given the deal as it actually was, does a line of -//! play exist that reaches the threshold? -//! -//! ## Why this is allowed to see everything -//! -//! Strategy fusion — the classic objection to searching an -//! imperfect-information game — is a defect of *aggregating over -//! determinizations to choose a move*. **After the game there is one -//! world.** The deal is known, so a line found in it is executable in the -//! only world there is (ADR-0013 D1). -//! -//! What survives the objection is that the line may not have been -//! *findable* at the time, and that is answered per move by -//! [`Move::visible`] rather than by refusing to search. -//! -//! ## The bound -//! -//! Exhaustive over the last `K` rounds, with a node budget as a secondary -//! cut. When nothing is found the caller must say **"no winning line -//! found in the last K rounds"** — never "unwinnable", which a bounded -//! search cannot establish (spec §2.3). - -use crate::bot::legal_commands; -use crate::{GroundCommand, GroundState, ProblemState}; -use cb_kernel::{Actor, Aggregate, PlayerId}; - -/// One move of a witness, with whether the seat could have chosen it -/// knowing only what it could see. -#[derive(Debug, Clone)] -pub struct Move { - pub actor: Actor, - pub command: GroundCommand, - /// `false` when the move depends on something the acting seat could - /// not see — spec §2.2. Concretely: it targets a Problem that was - /// **face down** to that seat, so choosing it required knowing what - /// was under it. - /// - /// System moves are always `true`: the table does them, not a player. - pub visible: bool, -} - -/// What the search found. -#[derive(Debug, Clone)] -pub enum Verdict { - /// A line exists. `nodes` is what it cost to find. - Winnable { line: Vec, nodes: usize }, - /// Nothing found **within the bound**. This is not "unwinnable". - NoneFound { - nodes: usize, - /// `true` if the space was searched to exhaustion; `false` if the - /// node budget cut it short. The distinction is the difference - /// between "no line exists in these K rounds" and "we stopped - /// looking", and callers must not collapse it. - exhausted: bool, - }, -} - -/// Would this command have been choosable knowing only what `seat` saw? -/// -/// A `SelectAction` naming a Problem that is face-down to that seat is -/// `hidden`: picking it required knowing what was underneath. Everything -/// else is `visible` — a seat's own hand is in its own projection, and -/// since GR-P05 (CB-WP-0023) SOLVE is only offered on face-up Problems -/// anyway, so INVESTIGATE is where hidden information actually bites. -fn is_visible(state: &GroundState, seat: PlayerId, cmd: &GroundCommand) -> bool { - let _ = seat; - match cmd { - GroundCommand::SelectAction { - problem: Some(p), .. - } => matches!( - state.problems.get(p), - Some(ProblemState { face_up: true, .. }) - ), - _ => true, - } -} - -struct Search { - nodes: usize, - budget: usize, - /// Set when the budget stopped us, so `NoneFound` can distinguish - /// "searched it all" from "gave up". - cut: bool, -} - -impl Search { - /// Apply a command to a copy. `None` if the aggregate rejects it — - /// which is not an error here: the search offers candidates and - /// `validate` is the authority, exactly as `legal_commands` does. - fn step( - &mut self, - state: &GroundState, - actor: Actor, - cmd: &GroundCommand, - ) -> Option { - self.nodes += 1; - let mut next = state.clone(); - let events = next.validate(actor, cmd).ok()?; - for e in &events { - next.fold(e); - } - Some(next) - } - - /// One player branch: apply, recurse, and prepend the move if the - /// subtree won. - fn branch( - &mut self, - state: &GroundState, - seat: PlayerId, - cmd: &GroundCommand, - rounds_left: u8, - ) -> Option> { - let next = self.step(state, Actor::Player(seat), cmd)?; - let mut rest = self.go(&next, rounds_left)?; - let mut line = vec![Move { - actor: Actor::Player(seat), - command: cmd.clone(), - visible: is_visible(state, seat, cmd), - }]; - line.append(&mut rest); - Some(line) - } - - /// Depth-first over whatever must happen next, mirroring the driver's - /// round structure (`bot::play_journaled`). - /// - /// Returns the moves appended after `state`, or `None`. - fn go(&mut self, state: &GroundState, rounds_left: u8) -> Option> { - if let Some(outcome) = &state.outcome { - return outcome.group_success.then(Vec::new); - } - if rounds_left == 0 { - return None; - } - if self.nodes >= self.budget { - self.cut = true; - return None; - } - - let seats: Vec = state.players.keys().copied().collect(); - - // **Obligatory first.** GR-R02: a seat with no selection this - // round must make one, and nothing else can happen until it does. - // If every branch fails, the line is dead — falling through would - // try system commands the aggregate is going to reject anyway. - // - // The first version branched on "the first seat that has any legal - // command" and `break`ed when its branches were spent, which threw - // away every later seat's options: seat 1 never acted if seat 0 - // was already selected but still had a legal move. - if let Some(seat) = seats.iter().find(|s| !state.selections.contains_key(s)) { - for cmd in &legal_commands(state, *seat) { - if let Some(line) = self.branch(state, *seat, cmd, rounds_left) { - return Some(line); - } - if self.cut { - return None; - } - } - return None; - } - - // **Optional next.** After Reveal a seat may choose a GROUND mode, - // answer a Support, or name a DARVO target. Some of those are - // obligatory, but the aggregate enforces that by rejecting - // `Resolve` until they are done — so this needs no phase logic of - // its own, and the do-nothing case is simply the fall-through - // below. - for seat in &seats { - for cmd in &legal_commands(state, *seat) { - if let Some(line) = self.branch(state, *seat, cmd, rounds_left) { - return Some(line); - } - if self.cut { - return None; - } - } - } - - // Nobody need act: the table advances. Try each system command; the - // aggregate rejects the ones that are out of order, so this needs - // no phase logic of its own. - for sys in [ - GroundCommand::Reveal, - GroundCommand::Resolve, - GroundCommand::EndRound, - ] { - let Some(next) = self.step(state, Actor::System, &sys) else { - continue; - }; - let spent = u8::from(matches!(sys, GroundCommand::EndRound)); - if let Some(mut rest) = self.go(&next, rounds_left - spent) { - let mut line = vec![Move { - actor: Actor::System, - command: sys, - visible: true, - }]; - line.append(&mut rest); - return Some(line); - } - if self.cut { - return None; - } - } - None - } -} - -/// Search the last `rounds` rounds from `state` for a line reaching -/// `group_success`. -/// -/// **`state` must be a real position from the game being asked about.** -/// The caller supplies it; this does not re-deal, because a re-dealt game -/// is a different question. -pub fn winnable_within(state: &GroundState, rounds: u8, budget: usize) -> Verdict { - let mut s = Search { - nodes: 0, - budget, - cut: false, - }; - match s.go(state, rounds) { - Some(line) => Verdict::Winnable { - line, - nodes: s.nodes, - }, - None => Verdict::NoneFound { - nodes: s.nodes, - exhausted: !s.cut, - }, - } -} - -/// How many moves of a witness required unseen information. -pub fn hidden_moves(line: &[Move]) -> usize { - line.iter().filter(|m| !m.visible).count() -} - -#[cfg(all(test, feature = "scenarios"))] -mod tests { - use super::*; - use cb_game_runtime::{ScenarioGame, Setup}; - - fn setup(players: u8, seed: u64) -> GroundState { - GroundState::setup( - &Setup { - players, - preset: format!("standard-{players}p"), - patch: Default::default(), - }, - seed, - ) - .expect("preset") - } - - /// **The hard gate (spec §2.1): a witness must replay.** - /// - /// Re-execute the emitted line from the same start state through - /// `validate`/`fold` — the same path the scenario runner takes — and - /// require it to end in `group_success`. A witness that does not - /// replay asserts the opposite of the truth to a player who just - /// lost. - /// Rewind a real game to the start of its last `k` rounds. - /// - /// Stops **after** applying the EndRound numbered `total - k`. An - /// earlier version broke *before* it, which left that round's own play - /// applied and searched one round less than it claimed. - fn last_rounds(players: u8, seed: u64, k: usize) -> GroundState { - let mut ps: Vec> = (0..players) - .map(|_| Box::new(crate::bot::GreedyPolicy) as Box) - .collect(); - let game = crate::bot::play(setup(players, seed), &mut ps).expect("a complete game"); - let total = game - .steps - .iter() - .filter(|(_, c)| matches!(c, GroundCommand::EndRound)) - .count(); - let mut st = setup(players, seed); - let mut ends = 0usize; - for (a, c) in &game.steps { - if let Ok(ev) = st.validate(*a, c) { - for e in &ev { - st.fold(e); - } - } - if matches!(c, GroundCommand::EndRound) { - ends += 1; - if ends >= total.saturating_sub(k) { - break; - } - } - } - st - } - - #[test] - fn every_witness_replays_to_a_win() { - let state = last_rounds(3, 7, 2); - let Verdict::Winnable { line, .. } = winnable_within(&state, 2, 200_000) else { - panic!("3p seed 7 is winnable in its last two rounds — greedy actually won it"); - }; - let mut replay = state.clone(); - for m in &line { - let events = replay - .validate(m.actor, &m.command) - .unwrap_or_else(|e| panic!("witness move rejected on replay: {e:?}")); - for e in &events { - replay.fold(e); - } - } - let outcome = replay.outcome.as_ref().expect("the replay must finish"); - assert!( - outcome.group_success, - "the witness replayed but did not win: {} of {}", - outcome.total, outcome.threshold - ); - } - - /// The negative control. Without it, a search that returns - /// `NoneFound` for everything would pass the test above by never - /// producing a witness to check. - #[test] - fn a_budget_of_nothing_reports_a_cut_not_a_verdict() { - let state = last_rounds(3, 7, 2); - match winnable_within(&state, 2, 1) { - Verdict::NoneFound { exhausted, .. } => assert!( - !exhausted, - "a search stopped by its budget must not claim it searched exhaustively — \ - that is the difference between `no line exists` and `we stopped looking`" - ), - Verdict::Winnable { .. } => panic!("one node cannot find a whole line"), - } - } - - /// A position with no rounds left cannot be won, and the search must - /// say so **without** claiming exhaustion of a space it never entered. - #[test] - fn no_rounds_left_finds_nothing() { - let state = last_rounds(3, 7, 2); - match winnable_within(&state, 0, 100) { - Verdict::NoneFound { nodes, exhausted } => { - assert_eq!(nodes, 0, "a zero-round search must not expand anything"); - assert!(exhausted, "it searched its (empty) space to exhaustion"); - } - Verdict::Winnable { .. } => panic!("no rounds left cannot win"), - } - } - - /// **A position that cannot be won returns none, exhaustively** — the - /// control without which "winnable" is unfalsifiable. - /// - /// The construction: 2p seed 7, searched over its **last round only**. - /// Greedy lost that game, and one round is a small enough space to - /// search to exhaustion (~8k nodes), so this is a real negative rather - /// than a budget cut wearing a verdict's clothes. - #[test] - fn a_position_that_cannot_be_won_says_so_and_means_it() { - let state = last_rounds(2, 7, 1); - match winnable_within(&state, 1, 500_000) { - Verdict::NoneFound { exhausted, nodes } => { - assert!( - exhausted, - "the space must be searched out, or this proves nothing ({nodes} nodes)" - ); - assert!( - nodes > 100, - "suspiciously few nodes for a real search: {nodes}" - ); - } - Verdict::Winnable { line, .. } => { - panic!("found a {}-move win in a game 2p seed 7 lost", line.len()) - } - } - } - - /// The `visible` marking must be able to say NO, or it is decoration. - /// INVESTIGATE names a face-down Problem — that is the case where a - /// seat could not have known what it was choosing. - #[test] - fn a_move_onto_a_face_down_problem_is_marked_hidden() { - let state = setup(3, 7); - let hidden_key = state - .problems - .iter() - .find(|(_, p)| !p.face_up) - .map(|(k, _)| *k) - .expect("a fresh deal has face-down Problems"); - let face_up_key = state - .problems - .iter() - .find(|(_, p)| p.face_up) - .map(|(k, _)| *k) - .expect("a fresh deal has the Surface Problem face up"); - - let onto = |p: u32| GroundCommand::SelectAction { - action: crate::Action::Investigate, - target: None, - problem: Some(p), - }; - assert!( - !is_visible(&state, PlayerId(0), &onto(hidden_key)), - "targeting a face-down Problem required knowing what was under it" - ); - assert!( - is_visible(&state, PlayerId(0), &onto(face_up_key)), - "a face-up Problem is visible — the marking must be able to say YES too" - ); - } -} diff --git a/specs/FindingRegister.md b/specs/FindingRegister.md index 1490bbf..42be87e 100644 --- a/specs/FindingRegister.md +++ b/specs/FindingRegister.md @@ -43,7 +43,6 @@ kinds, states and metrics: [`GameDesign.md`](GameDesign.md). Reported by | F13 | inconsistent | withdrawn | scenarios/ground/gr-e01-threshold-reachable-2p.yaml | counterexample | 2026-08-01 | clay-borg | | F14 | unplayed | note | — | — | 2026-08-01 | clay-borg | | F15 | underdetermined | note | — | — | 2026-08-05 | clay-borg | -| F16 | inconsistent | withdrawn | games/ground/examples/difficulty.rs | counterexample | 2026-08-05 | clay-borg | @@ -63,16 +62,6 @@ kinds, states and metrics: [`GameDesign.md`](GameDesign.md). Reported by `-reachable-`. **Its reproduction is green**, which under GameDesign §1.3 is the alarm that forced the resolution. Withdrawn rather than deleted, and the withdrawal is reported (ADR-0012 D5). -- **F16 — "the game is too easy at 5–6 seats", withdrawn the day it was - raised.** Claimed from `GreedyPolicy` winning 200/200 at those seat - counts. A `FirstLegal` policy scores **0%** on the identical deals, and - at two seats it *beats* greedy — two unsophisticated agents span the - whole range, so the measurement was about the policy. Caught by the - CB-WP-0025 adversarial review (C4) **before transmission**; it would - have been the fifth wrong premise sent to ground-game and the worst, - since GROUND-WP-0005 is blocked on exactly this number. The withdrawal - was reported (ADR-0012 D5). Its reproduction is `difficulty.rs`, whose - policy panel is plural *because of this finding*. - **F15 — the rules define one game, not a series.** `OutcomeView` gives `personal` (per seat), `group_success` (per table) and `winners`. Summing the first and counting the third answer different questions, and GROUND diff --git a/specs/RetrospectiveAnalysis.md b/specs/RetrospectiveAnalysis.md deleted file mode 100644 index fbb11b1..0000000 --- a/specs/RetrospectiveAnalysis.md +++ /dev/null @@ -1,235 +0,0 @@ -# RetrospectiveAnalysis — was this deal winnable, and how hard is the game - -v1.0 — CB-WP-0025 T04, 2026-08-05. Normative. Implements -[ADR-0013](../decisions/ADR-0013-could-we-have-won.md). Admissibility of -anything this produces is governed by -[GameDesign.md](GameDesign.md) §1. - -**Two capabilities, one machine.** *Was this deal winnable?* is a search -over a finished game. *How hard is the game?* is that search run over many -deals and counted — **not** a bot's win rate (§4.1). - ---- - -## 1. The question, and its name - -> **"Given the deal as it actually was, was there a line of play that -> reached the threshold?"** - -**Never labelled "how you should have played."** The distinction is the -honest content of the feature: the tool answers a question about the -*deal*, and a label promising advice about the *player* turns a true -answer into a false lesson. - -**Strategy fusion does not apply and must not be invoked as an objection.** -Fusion (Frank, Basin & Matsubara 1998) is a defect of aggregating over -determinizations to choose a move. After the game there is **one world** — -the deal is known — so a line found in it is executable in the only world -there is. This is why the affordable option is also the honest one. - -## 2. The witness - -A witness is a sequence of joint selections that, replayed from the -recorded initial state, ends with `group_success == true`. - -### 2.1 It must replay — hard gate, not a metric - -> **100% of emitted witnesses replay through the existing scenario runner -> and end in `group_success`.** - -Not a target: a **gate**. A witness that does not replay asserts the -opposite of the truth to a player who just lost, which is worse than -emitting nothing. - -### 2.2 Every move carries its information dependence - -ADR-0013 D2. Each move in a witness is marked: - -| mark | meaning | -|---|---| -| `visible` | everything the move depended on was in the acting seat's projection at that point — the Problem face-up, the Solution in that seat's own hand | -| `hidden` | it was not | - -Computed from `project()`, which already exists and whose hiding rules are -already asserted by `games_ground::view`. - -**This replaces the structural boundary the survey wanted.** Searching a -`GroundView` is not implementable — a view cannot `fold` events — so the -guarantee moved from *the search cannot see it* to *the answer says which -moves needed it*. A witness reads: - -> *"This deal was winnable. Two of these six moves needed a card you had -> no way to know was coming."* - -**Falsifier:** a witness whose moves are all `visible` but which no seat -could have chosen means the marking is wrong. A test constructs that case. - -### 2.3 Wording when nothing is found - -> **"No winning line found in the last K rounds."** - -**Never "unwinnable".** A bounded search cannot establish unwinnability, -and this sentence is what the player reads. - -## 3. The bound - -**Exhaustive over the last `K` rounds**, table treated as one co-operative -agent choosing joint selections. - -**`K = 1` for an exhaustive answer; `K` may be larger when a witness is -all that is wanted.** ADR-0013 said `K = 2` by default; §3.1's measurement -overrides it, and the difference is which question is being asked: - -| answer | needs | affordable `K` today | -|---|---|---| -| *"here is a winning line"* | one success | 2+ — DFS finds one in tens of nodes | -| *"there is no winning line"* | exhaustion | **1** — `K=2` exceeded 2×10⁶ nodes at two seats | - -A `K` that cannot be exhausted may still emit a witness; it may **not** -report `NoneFound { exhausted: true }`, and the type keeps those apart. - -Bounded in **rounds**, not nodes: *"winnable from round 4"* means something -to a player; *"winnable within 100,000 nodes"* does not. A node budget is a -secondary cut that aborts with a stated reason so a wide table cannot hang -the page. - -### 3.1 Measured cost, and what it permits - -`cargo bench -p games-ground --bench search` — the single source for these -numbers (ADR-0013 D7). Mid-game states at real decision points: - -| seats | branch width | `legal_commands` | `clone` | `validate+fold` | -|---|---:|---:|---:|---:| -| 2 | 5 | 4.06 µs | 378 ns | 696 ns | -| 3 | 8 | 4.13 µs | 432 ns | 508 ns | -| 4 | 11 | 4.76 µs | 639 ns | **3.76 µs** | - -**Per-child cost is not uniform** — `validate+fold` ranges 0.5–3.8 µs -depending on which command is taken, because some resolve cascades and -some do not. **Budgets use the upper end**, so ~5 µs per child -(clone + validate + fold). - -Joint branching over the last two rounds, from the measured per-seat -widths: - -| seats | joint / 2 rounds | at ~5 µs/child | -|---|---:|---:| -| 2 | ~5×10² | negligible | -| 3 | ~1.6×10⁵ | **~0.8 s** | -| 4 | ~5.7×10⁵ | **~2.9 s** | - -> ### The projection above was wrong, and the real search falsified it -> -> **Measured 2026-08-05 with the search built in T05**, rewinding real -> games to the start of their last `K` rounds: -> -> | case | result | -> |---|---| -> | 2p, `K=1` | **exhausted** in 8,103 nodes, ~29 ms — a real negative | -> | 2p, `K=2` | **budget cut** at 2,000,000 nodes, ~5 s — not exhausted | -> | 3p, `K=2` | win found in 41 nodes, ~157 µs | -> -> §6's falsifier said *"§3 fails if K=2 proves unaffordable in practice at -> four seats"*. **It failed at two.** -> -> The projection assumed a joint product per round. The search explores -> **sequential per-seat decisions**, and the post-Reveal phase branches -> over every seat's options at every level, so orderings multiply the tree -> far beyond `width^seats`. -> -> **And the asymmetry is the operative fact:** *finding* a win is cheap — -> depth-first stumbles onto one in tens of nodes — while *proving none -> exists* is expensive, because it must exhaust the space. So: -> -> - **the witness feature (§2) is affordable now**, at any `K` a player -> would ask about; -> - **the winnable fraction (§4.2) is not**, because its "not winnable" -> half requires exhaustion on every deal it counts. -> -> `K = 1` is the honest default for exhaustive answers today. Making -> `K = 2` exhaustible needs transposition or move-ordering, neither of -> which this pass built. - -**The published 112–161 µs/node figure is withdrawn** (CB-RES-0008 §1.2, -challenge C1) and must not be quoted from anywhere. - -## 4. Difficulty - -### 4.1 A bot's win rate is not a difficulty - -**Normative prohibition**, because this project already made the error and -nearly exported it: - -> A win rate from a single policy **may not be reported as a difficulty**. - -Measured, on identical deals: `GreedyPolicy` wins **100%** at five and six -seats where a `FirstLegal` policy — take `legal[0]`, no heuristic — wins -**0%**; at two seats `FirstLegal` (77.5%) *beats* greedy (66.0%). Two -unsophisticated agents span the entire range. - -And a measure that improves when the *measurer* improves is not measuring -the subject: a better bot would make the game "easier" with no rule -changing. - -### 4.2 What is reported instead - -> **Winnable fraction** — over N deals at a seat count, the proportion in -> which the search finds a winning line within its bound. - -A property of the **deal distribution and the threshold**, which is what -`ground-game` tunes. Ships as a table, never one number: - -| column | what it is | -|---|---| -| winnable fraction | can the deal be won at all — bounded, `K` stated | -| reference-policy win rate | what a **named** policy achieves | -| skill gap | the difference — how much play has to supply | - -**It is a lower bound and must be labelled one.** A `K`-round search cannot -see a line that required round 1, so the figure is -**"winnable-from-round-(6−K)"**, never "winnable". - -**Every rate carries its policy, N, seed range and K in the number's -name**, not in a footnote — GameDesign §1.2, and the reason the withdrawn -finding was inadmissible. - -### 4.3 Resolution — the number that makes it usable - -> The smallest threshold change the measurement can distinguish, with its -> N. - -*"We can tell a threshold of 5 from 7 but not 7 from 8"* is more useful to -`ground-game` than any rate with no error bar, and it is what makes the -figure a tuning instrument rather than a statistic. - -## 5. The instruments must be able to fail - -ADR-0013 D5, and GameDesign §1.3. Before **any** figure from these tools is -quoted anywhere: - -- **positive controls** — a deal constructed to be unwinnable returns - none; a deal constructed to be winnable returns a witness that replays; -- **`--self-test`**, wired into `make self-tests` like every other - reporting tool; -- **one command regenerates the figure** (`make difficulty`); -- **the policy panel is plural** — at least `greedy`, `random` and - `first-legal`. The spread between them is the finding §4.1 rests on, and - reporting one policy would restore the error. - -**`games/ground/examples/difficulty-baseline.rs` currently satisfies none -of the first three** and is inadmissible until it does. It has no -assertions, no self-test, and no `make` target — nothing can turn it red, -which under CB-WP-0022 T05's `role` distinction makes it a `default` -artifact wearing a `counterexample` label. - -## 6. Falsifiers for this spec - -- **§2.2 fails** if a witness is emitted whose moves are all `visible` but - which no seat could have chosen. Then the marking must be derived from - the search rather than checked after it. -- **§4.2 fails** if the winnable fraction turns out to be ~100% or ~0% at - every seat count and threshold — it would then have no resolution (§4.3) - and be as useless as the bot rate it replaced. -- **§3 fails** if `K = 2` proves unaffordable in practice at four seats; - the measured 2.9 s is a projection from branch widths, not a timing of - the real search. diff --git a/workplans/CB-WP-0025-could-we-have-won.md b/workplans/CB-WP-0025-could-we-have-won.md index 3074943..23b4739 100644 --- a/workplans/CB-WP-0025-could-we-have-won.md +++ b/workplans/CB-WP-0025-could-we-have-won.md @@ -2,7 +2,7 @@ id: CB-WP-0025 kind: product title: "Could we have won: a path out of a lost game, and how hard the game actually is" -status: done +status: active state_hub_workstream_id: "a866982f-94e7-432b-b0d5-2fefb781a574" --- @@ -46,30 +46,45 @@ middle of stating, arriving with a concrete demand. ## What already exists, so the survey does not re-find it -The state is replayable (`replay.rs`, `make replay-test`); the move space -is enumerable (`legal_commands`, narrowed by CB-WP-0023 so branching is -real rather than inflated by inert moves); bots exist (`bot.rs`); and the -threshold is public (`OutcomeView`). **A search needs no new persistence -and no new rules** — which is why D6 put it in `games/ground` with no new -crate and no port. +- **The state is replayable.** `cb-game-runtime` records sessions as + scenarios; `replay.rs` and `make replay-test` already re-run them. + A search does not need new persistence. +- **The move space is enumerable.** `legal_commands` exists and, since + CB-WP-0023, is narrow enough to be worth trusting — SOLVE is offered + only where it can act, so the branching factor is real rather than + inflated by inert moves. +- **Bots exist.** `games/ground/src/bot.rs` has `GreedyPolicy` and + `RandomPolicy`, wired through `bot_policy` (`table.rs:208`). A win rate + over N seeds is reachable with what is already there — the question is + whether that number *means* anything, which is the survey's problem. +- **The threshold is public.** `OutcomeView.total` / `.threshold` / + `.group_success`. Difficulty has a denominator already. ## What makes this hard, and must not be waved through -**Settled by [ADR-0013](../decisions/ADR-0013-could-we-have-won.md).** The -declaration framed the central risk as *a path computed with the deck -known is a path the players could never have found*, and asked the ADR to -choose between an omniscient, an information-respecting, and a bounded -search. +**The game is not perfect-information and the search must respect that.** +A path computed with the deck known is a path the players could never have +found. `view.rs` hides the deck, other seats' hands, and face-down +selections *by rule* (GR-S02/S04, GR-R02/R04). A retrospective solver +running on `GroundState` sees all of it. So the ADR must decide, in +words, **which of these three the tool answers**: -**D1 dissolved the choice**: strategy fusion is a defect of aggregating -over determinizations to *choose a move*, and after the game there is one -world — so a line found in it is executable in it. **D2** keeps the -declaration's real concern by marking each move `visible`/`hidden` rather -than by refusing to search. +- *was this deal winnable by an omniscient player* — cheap, honest, + and answers a question nobody asked; +- *was it winnable from what the seats could see* — the question actually + asked, and the expensive one; +- *did a reasonable line exist* — a bounded search from the losing seat's + information, which may be the only affordable honest answer. -The second warning here — *"one win rate over one bot policy is not the -difficulty"* — was right, and **the survey made exactly that error -anyway**; see T02. +Getting this wrong produces a feature that tells the maintainer he could +have won by playing a card he had no way to know was there. **That is +worse than not shipping it.** + +**And a difficulty number is a claim about a distribution.** One win rate +over one bot policy over N seeds is not "the difficulty"; it is that +policy's win rate. Whatever the spec adopts must name its policy, its N, +and its seed range, or `ground-game` will tune tiers against a number +whose meaning drifts the next time a bot improves. ## Task: survey @@ -84,39 +99,62 @@ state_hub_task_id: "556cfd24-5992-4cc0-be90-0b60e989b5bb" (`loop-lint` checks both). Per §Step 1 the survey is done when it can name a **benchmark-to-beat** -per dimension. Four were asked for: retrospective solvers in -hidden-information games (and their known failure, strategy fusion); *a -path to win* as a **witness** rather than a proof; difficulty as a measured -quantity in co-operative games; and the **per-node cost**, measured on our -machine — the runnable-baseline option applies, since a search that cannot -finish while the player is looking at the page is a different feature. +per dimension — a number or a reproducible comparison, not an impression. + +- **Retrospective solvers in games with hidden information.** The prior art + is real and should be named: determinized search (perfect-information + Monte Carlo) and its known failure — *strategy fusion*, where a + determinizing solver claims lines that require knowing which world it is + in. That failure is exactly the trap in §What makes this hard. Bridge + and Skat post-mortem tools are the closest analogues; poker solvers are + the well-studied case and the wrong shape. +- **"A path to win" as a product, not a proof.** The maintainer already + conceded optimality (*"the best path is not computable I guess"*). So + the target is a **witness**: one concrete line of play that reaches + `group_success`, or a defensible *no line found within bound B*. Name + what a witness must carry to be checkable. +- **Difficulty as a measured quantity in co-operative games.** Pandemic and + its relatives set difficulty by a dial with a published win rate. The + benchmark-to-beat is: can we produce a win rate whose confidence + interval is tight enough to distinguish two threshold settings? +- **Cost.** Search over an event-sourced aggregate with full `validate` on + every branch has a per-node price. Measure it on our machine, on our + scenarios — the runnable-baseline option applies here, since a search + that cannot finish while the player is still looking at the page is a + different feature. **Done 2026-08-05.** [CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md), with a runnable baseline (`games/ground/examples/difficulty-baseline.rs`). -> **Everything this record originally claimed was withdrawn by T02 the -> same day.** Kept as a pointer rather than rewritten, because a claim -> retracted silently is how three earlier wrong premises survived -> (ADR-0012 D5). +**The baseline produced a finding before any solver exists, and it is the +biggest thing in this pass.** A greedy bot wins **200 of 200** games at +five and six seats — with a median margin of +3 and 11.8–12.0 points +available against a threshold of 9. The curve is 66% / 82.5% / 95% / 100% +/ 100% across 2/3/4/5/6 seats. -**Claimed:** a greedy bot wins 200/200 at five and six seats, so the game -is too easy there; the points-to-threshold ratio rises with seat count and -explains the curve; `legal_commands` costs 112–161 µs, so exhaustive -search is out; and all of this explains the maintainer's report. +The row-level table shows why: available points go **6 / 9 / 12** against +thresholds **5 / 7 / 9**, so the ratio *rises* with seat count (1.20 → +1.29 → 1.33) while the table also gains actions per round. **Three +multipliers pointing the same way.** -**Withdrawn:** the win rate is `GreedyPolicy`'s, not the game's (C4 — -`FirstLegal` scores 0% on the same deals); the ratio is identical at 3p -and 4p, which differ by 12.5 points (C2); the cost figure was wrong by -30–50× and exhaustive search is affordable (C1, C6); and the maintainer's -losses were 3-player games on the pre-ruling deal (C5). +**It also explains the maintainer's report without a solver.** *"I felt it +was too easy but then we lost"* — 66% at two seats is a real game; 100% at +six is not. Both halves are true of different seat counts. -**What survives:** the harness exists and runs, the 6/9/12 arithmetic is -right against `Problems.csv`, and the branching widths (4.7–9.1) hold. See -[CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md) for the -corrected text and -[the response](../history/260805-could-we-have-won-response.md) for the -full accounting. +**Cost measured, and it rules out the obvious approach.** Branching is +small (mean 4.7–9.1) but `legal_commands` costs **112–161 µs** per call, +because it filters candidates through full `validate`. Exhaustive search +is out at every seat count; ~10⁴–10⁵ nodes is 1.4–14 s, which is the +budget the ADR must design inside. + +**Prior art names the trap.** Determinized search (PIMC) suffers *strategy +fusion* — Frank, Basin & Matsubara 1998 — where the search picks different +actions in states a real player cannot tell apart. A witness built that +way may require knowing what was on top of the deck. **And it would still +replay green**, so the checkability benchmark does not catch it. Honesty +and checkability are different properties, stated so T03 cannot conflate +them. ## Task: adversarial review @@ -128,89 +166,103 @@ state_hub_task_id: "bdd9fc97-4706-4d8e-b286-91ff53339680" ``` Tier L requires it. Exactly one round: challenge, then response, trail in -`history/`, unpolished. Four attempts required: that the honest version is -unaffordable; that a witness misleads more than it helps; that the -difficulty number is a bot benchmark in a difficulty costume; and that -this is CB-WP-0022's job rather than a separate capability. +`history/`, unpolished. Require an attempt at: + +- **that the honest version is unaffordable** — that a search respecting + the information rule is too expensive or too weak to find anything, so + the shipped tool will quietly become the omniscient one with a + reassuring label; +- **that a witness misleads more than it helps** — being shown a line that + needed a card you could not know about teaches a wrong lesson about the + game, and the tool would be better refusing to answer; +- **that the difficulty number is a bot benchmark wearing a difficulty + costume**, and `ground-game` will tune the game against our bot rather + than against play; +- **that this is CB-WP-0022's job** — the design instrument is being built + right now, and a difficulty measurement is a finding-producing tool. The + strongest counter is that the register records findings and this + *produces* them, but the reviewer should press whether that is a + distinction worth a separate capability. **Done 2026-08-05.** Trail: [challenge](../history/260805-could-we-have-won-challenge.md), [response](../history/260805-could-we-have-won-response.md). Separate agent, as in CB-WP-0022. -**Six of seven conceded. The survey's headline finding is WITHDRAWN.** +**Six of seven conceded, and the survey's headline finding is withdrawn — +not softened.** -**C4 kills it and the reviewer ranked it fourth.** A `FirstLegal` policy — -`legal[0]`, no heuristic — scores **0% at five and six seats** where greedy -scores 100%, and **77.5% at two** where greedy scores 66%. Two -unsophisticated agents span the whole range, so *"too easy at 5–6 seats"* -was about `GreedyPolicy`, not GROUND. +**C4 is the one that kills it, and the reviewer ranked it fourth.** +Measured: a `FirstLegal` policy — take `legal[0]`, no heuristic — scores +**0% at five and six seats** where greedy scores 100%, and **77.5% at two +seats** where greedy scores 66%. **Two unsophisticated agents span the +entire range at the same seat count**, so *"the game is too easy at 5–6 +seats"* is a statement about `GreedyPolicy`, not about GROUND. Their +offered rescue — greedy hits the 12-point ceiling every time, so it is a +rules claim — dies on the same data. -**C1**: the node cost was wrong by 30–50× — the timer bracketed whole -games — and **the tell was in my own output**, falling as branching rose. -**C6**: exhaustive search is *not* out, which changes T03's premise. -**C3**: the finding failed the admissibility rule this project wrote nine -hours earlier. **C2**: 3p and 4p share deal, threshold and ratio and differ -by 12.5 points. **C5**: `lib.rs:2487` contradicts the "explains the -maintainer" claim — his losses were 3-player on the pre-ruling deal, -unwinnable at 6 against 7. +**C1: the per-node cost was wrong by 30–50×.** The timer bracketed two +`setup`s, a whole greedy game and a validate+fold replay, then divided by +player decisions. **The tell was in my own published output**: the figure +*fell* as branching *rose*, which no per-enumeration cost can do. +Re-measured at **3.0–4.1 µs**; the reviewer got 15.6–20.4 by a different +isolation and **that discrepancy is unsettled** — T04 benchmarks it. + +**C6: exhaustive search is not out**, which changes T03's premise. With +C1's correction the budget is ~10⁵–10⁶ nodes and bounded endgame search +fits — so the ADR cannot open with *"exhaustive is impossible, therefore +determinized sampling"*, especially since sampling carries strategy fusion +that exhaustive search does not. + +**C3: the finding failed the admissibility rule this project wrote nine +hours earlier** — sums where GROUND-WP-0004 T02 requires per-priority +rows, and a harness with no assertions, no `--self-test` and no `make` +target, so nothing can turn it red. A `default` artifact wearing a +`counterexample` label. + +**C2: the ratio explains nothing** — 3p and 4p share deal, threshold and +ratio, and differ by 12.5 points of win rate. **C5: the "explains the +maintainer's report" claim is contradicted by `lib.rs:2487`**, which +records his losses as 3-player on the pre-ruling deal, unwinnable at 6 +against 7. **T06 was pointed at GROUND-WP-0005, which is blocked on this number.** -Sending it would have been the fifth wrong premise to reach ground-game. -**Both tier-L reviews here have now caught a false headline that every -gate passed.** +Sending it would have invited threshold changes on one bot's behaviour — +the fifth wrong premise to reach ground-game. **Both tier-L reviews in +this project have now caught a false headline that every gate passed.** ## Task: decide ```task id: CB-WP-0025-T03 -status: done +status: todo priority: high state_hub_task_id: "055630a1-56e0-4ad4-8796-35f444313371" ``` `decisions/ADR-0013-*.md`. (ADR-0012 is CB-WP-0022 T03's.) At minimum: -which question the solver answers and what it is called in the UI; the -**information boundary** and whether it is structural or promised; the -**bound**, and what *no path found* means against it; whether difficulty -is one number or a table, and of what; and **where it lives** — the tier -was declared L assuming a new capability port, and if there is none, say -so. -**Done 2026-08-05.** -[ADR-0013](../decisions/ADR-0013-could-we-have-won.md), seven decisions. -**Two of them are not what T03 was written expecting**, because the review -moved the ground under both. - -- **D1 — strategy fusion does not apply, and that is why this is - affordable.** Fusion is a defect of *aggregating over determinizations - to choose a move*. **After the game there is one world**: the deal is - known, so a search over it produces a line executable in the only world - there is. The survey treated fusion as this pass's central obstacle; it - is an obstacle to a *playing* engine, which we are not building. -- **D4 — difficulty is the WINNABLE FRACTION, not any bot's win rate.** - C4 killed the bot rate: two trivial policies span 0–100% on the same - deals, and improving the bot would make the game "easier" without a rule - changing. The solver supplies a policy-independent measure — *over N - deals, in what proportion does a winning line exist* — which is a - property of the deal distribution and the threshold, and is what - GROUND-WP-0005 actually needs. **The bot rate never was.** - -**D2** searches `GroundState` — the survey's view-only boundary is not -implementable, since a view cannot fold events — and moves the guarantee to -a checkable per-move `visible`/`hidden` marking from `project()`. **D3** -bounded exhaustive over the last K rounds, with *"no winning line found in -the last K rounds"* normative, never *"unwinnable"*. **D5** makes the -harness an instrument before any figure is quoted (C3). **D6** no new -crate and no port — **the L declaration was an over-declaration, recorded -as one**. **D7** the node cost is disputed 5×; T04 benchmarks it and -neither existing figure may be cited. +- **which question the solver answers**, from the three in §What makes + this hard, and what it is called in the UI — the name must not overclaim; +- **the information boundary**: whether the search runs on `GroundState` + or on a `GroundView`, and if on state, what stops it using what the view + hides. Note that running on the view makes the rule structural rather + than a promise, and that this is the cheapest guarantee available; +- **the bound**: depth, node budget, or wall clock, and what *no path + found* means against it — a bounded search that says "unwinnable" is + lying, and the wording must say "none found within B"; +- **whether difficulty ships as one number or a small table**, and what it + is a function of: policy, seat count, threshold, seed range; +- **where it lives** — a new crate, a mode of `cb-play`, or a tool under + `tools/`. The tier was declared L on the assumption of a new capability; + if the ADR concludes it is a mode of an existing one, say so, and the + over-declaration is a chaos-window data point worth recording. ## Task: specify ```task id: CB-WP-0025-T04 -status: done +status: todo priority: high state_hub_task_id: "7c3ca25d-0001-44ef-a07d-0ab2d51b09a5" ``` @@ -218,53 +270,29 @@ state_hub_task_id: "7c3ca25d-0001-44ef-a07d-0ab2d51b09a5" `specs/` — extend `MetricsAndScenarios.md` or add a capability spec as the ADR directs, with metrics, because a spec without them is prose. -Candidates, to be argued not adopted: **witness checkability** (every -emitted path replays to `group_success` — a hard gate, not a metric, since -a path that does not replay says the opposite of the truth); **search -cost** at the chosen bound; and **difficulty resolution** — the smallest -threshold difference the measurement can distinguish, with its N, which is -more useful to `ground-game` than any rate with no error bar. +Candidates, to be argued not adopted: + +- **witness checkability** — every path the tool emits replays through the + existing scenario runner and ends in `group_success`. Target 100%, and it + is a hard gate, not a metric: a path that does not replay is a bug that + says the opposite of the truth; +- **search cost** — nodes and wall clock at the chosen bound, on the + recorded games we have; +- **difficulty resolution** — the smallest threshold difference the + measurement can distinguish, with its N. This is the number + `ground-game` needs, and stating it as *"we can tell 5 from 7 but not 7 + from 8"* is more useful than a win rate with no error bar. Per `ground-game`'s ruling (GROUND-WP-0004 T02), **any arithmetic finding this produces ships a runnable reproduction and a row-level table** — never a summed figure. A difficulty number is arithmetic, and it is exactly the kind that has already gone wrong twice. -**Done 2026-08-05.** -[specs/RetrospectiveAnalysis.md](../specs/RetrospectiveAnalysis.md) v1.0, -and `games/ground/benches/search.rs` for D7's disputed number. - -**The benchmark's own first fixture was defective — the same defect class, -one layer down.** A fixed stop at step 20 put 2p and 4p where seat 0 had -**no legal commands**, so it timed an empty `Vec` (~120 ns) and silently -skipped `validate_fold`. It now advances to a real branch **and asserts -it**. - -Measured at real decision points (table in -[RetrospectiveAnalysis §3.1](../specs/RetrospectiveAnalysis.md)): -`legal_commands` 4.06–4.76 µs, `clone` 378–639 ns, `validate+fold` -**0.5–3.8 µs**. - -**`clone` is in there because a search must copy state per branch**, and -`iter_batched` excludes setup from timing — leaving the budget on an -unmeasured span, which is precisely C1's mistake. - -**The affordability conclusion drawn here was itself falsified by T05.** -It projected joint branching and concluded `K = 2` holds at two to four -seats. The real search exceeded 2×10⁶ nodes at **two**. See T05's record -and [RetrospectiveAnalysis §3.1](../specs/RetrospectiveAnalysis.md) — -a projection from branch widths is not a timing of a search, and this -pass has now made that mistake twice. - -**§4.1 is a normative prohibition**, not a preference: a single-policy win -rate may not be reported as a difficulty. The spec carries the measured -reason — greedy 100% vs first-legal 0% on identical deals. - ## Task: build the witness ```task id: CB-WP-0025-T05 -status: done +status: todo priority: high state_hub_task_id: "5f1cbda3-7427-4f23-8c67-b2e216b987a8" ``` @@ -285,48 +313,23 @@ a loss. naming *that* boundary goes red. If it cannot be mutated, it was a comment rather than a rule. -**Done 2026-08-05.** `games/ground/src/search.rs`, five tests. - -**The first traversal was wrong and the diagnostic hid it.** It branched -on *the first seat with any legal command* and stopped there, so a later -seat never acted if an earlier one was already selected. Restructured -around what the rules oblige: a seat without a selection **must** select -(GR-R02) and nothing else can happen first; after Reveal the optional -actions branch freely, and the aggregate rejects `Resolve` until the -obligatory ones are done — **so the search needs no phase logic of its -own.** - -**And my rewind was off by one round**, replaying the round it was meant -to search — which is why the first run reported 3 nodes and looked fine. - -**The measurement falsified the spec's own projection at two seats, where -§6 predicted four**: 2p `K=1` exhausted in 8,103 nodes (~29 ms), 2p `K=2` -cut at 2,000,000 nodes (~5 s), 3p `K=2` won in 41 nodes. - -The projection assumed a joint product per round; the search explores -sequential per-seat decisions, so orderings multiply the tree far beyond -`width^seats`. - -**The asymmetry is the operative finding.** *Finding* a win is cheap; -*proving none exists* needs exhaustion. So the **witness feature is -affordable now** and the **winnable fraction is not**, because its -negative half must exhaust every deal it counts. - ## Task: measure the difficulty, and hand it to ground-game ```task id: CB-WP-0025-T06 -status: done +status: todo priority: high state_hub_task_id: "723feae8-1ed4-4dd4-9330-8ec84625e5d9" ``` Run the measurement, ship it as a `make` target beside the other -instruments, and send it to `ground-game` against **GROUND-WP-0005**, -which is active with both tasks waiting on exactly this. Per CB-WP-0022 -T06 it lands as a **file in their repo under their workplan**, not only an -inbox entry — *the message that sat unread for four days is the baseline -to beat*. +instruments, and show the result in the game — the maintainer asked for it +to be visible, and a number in a file will not calibrate anything. + +Then send it to `ground-game` **against GROUND-WP-0005**, which is active +with both tasks waiting on exactly this. Per CB-WP-0022 T06, it lands as a +file in their repo under their workplan, not only an inbox entry — *the +message that sat unread for four days is the baseline to beat*. **Controls:** - the number regenerates from a single command, and `facts.toml` carries @@ -336,65 +339,22 @@ to beat*. - **the seed range and policy are in the number's name**, not in a footnote. -**Done 2026-08-05.** `games/ground/examples/difficulty.rs`, `make -difficulty`, wired into `make self-tests`, and -[a report file in ground-game](../../ground-game/workplans/GROUND-WP-0005-clay-borg-difficulty-260805.md) -under GROUND-WP-0005 with a hub message pointing at it. - -**The report opens with the retraction**, because what this task was -written to send was withdrawn by T02 and GROUND-WP-0005 is blocked on -exactly that number. - -Winnable-from-round-5, 60 seeds: **60% / 93% / 100% / 100% / 100%** at -2/3/4/5/6 seats, beside greedy / random / first-legal rates and the spread -between them. Full table in the report and in `make difficulty`. - -**`spread` justifies the whole redesign** — 71.7 to 100.0 points between -three trivial policies. No single rate says anything about the game, and -the table now shows that rather than asserting it. - -**The confound is stated in the tool's own output**: `winnable` is -conditioned on greedy's play up to the final round. Presenting it as a -property of the deal would repeat this pass's error in a subtler form. -**No threshold changes are proposed.** The instrument can fail (spec §5), -and `difficulty-baseline.rs` is marked superseded. - -Registered as **F16, `inconsistent`, `withdrawn`**, with `difficulty.rs` -as its reproduction. - ## Task: evidence ```task id: CB-WP-0025-T07 -status: done +status: todo priority: high state_hub_task_id: "2616e06c-5d88-4680-be18-4452515dff21" ``` -`evidence/CB-EV-*.md` (**CB-EV-0024** — CB-EV-0023 went to CB-WP-0024). -Was the maintainer's game winnable; what the -honest search cost against the omniscient one; whether the difficulty -measurement moved `ground-game`; what tier L cost against what it caught; -and CB-WP-0024's cost by re-running the instrument. +`evidence/CB-EV-0023-*.md`. -**Done 2026-08-05.** -[CB-EV-0024](../evidence/CB-EV-0024-could-we-have-won.md). - -- **The maintainer's question is answered.** 3p seed 7 was winnable — a - 10-move line, found in 41 nodes, replays to `group_success`. 2p seed 7 - was **not**, exhaustively, in 8,103 nodes. **The witness feature works; - the difficulty half is where the trouble was.** -- **Five of nine defects came only from the review**, four from execution - — and all four of those were in work written *after* it. -- **The wrong-denominator family now has five instances and no control.** -- **The 5–6 seat rows point the opposite way from the withdrawn claim**; - neither reading is established, and the evidence says so. -- **Tier L was an over-declaration** (no port) **and paid for itself - anyway** — the review is L-only, and without it a false number reaches a - blocked repo. -- **Chaos window 2 will close with zero overrides**, making its retirement - condition untestable. - -**Not done, and named rather than quietly dropped:** the witness is **not -wired to the ending page**, so the browser cannot yet ask the question. -That is the maintainer-facing half of remark 2. +- **Was the game winnable**, for the maintainer's actual lost game. That is + the acceptance test with a face on it. +- **What the honest search cost against the omniscient one**, since the + review will have pressed hardest there. +- **Whether the difficulty measurement moved ground-game**, or sat. +- **What tier L cost against what it caught** — third full-weight L pass in + the project, and the second in this chaos window. +- **Quote CB-WP-0024's cost by re-running the instrument.**