diff --git a/Makefile b/Makefile index 7db140f..102f883 100644 --- a/Makefile +++ b/Makefile @@ -125,6 +125,7 @@ self-tests: $(PY) $(TOOLS)/runtime-metrics.py --self-test $(PY) $(TOOLS)/replay-test.py --self-test $(PY) $(TOOLS)/design.py --self-test + cargo run --release -q -p games-ground --example difficulty -- --self-test $(PY) $(TOOLS)/edition-check.py --self-test # T01 positive control: prove the environment fix, do not assume it. Runs @@ -160,6 +161,12 @@ facts-check: facts-gen: $(PY) $(TOOLS)/facts.py --gen +# CB-WP-0025 T06: the difficulty table (specs/RetrospectiveAnalysis.md §4). +# Winnable fraction from the solver plus a PLURAL policy panel -- a single +# policy's win rate may not be reported as a difficulty (§4.1). +difficulty: + @cargo run --release -q -p games-ground --example difficulty + # CB-WP-0022 T05: the design-finding register, reported over # specs/GroundRules.md. Shows the QUEUE by default; the log of closed # findings is a line, not a listing, because a default view that mixes diff --git a/evidence/CB-EV-0024-could-we-have-won.md b/evidence/CB-EV-0024-could-we-have-won.md new file mode 100644 index 0000000..94a2b49 --- /dev/null +++ b/evidence/CB-EV-0024-could-we-have-won.md @@ -0,0 +1,161 @@ +# CB-EV-0024 — could we have won + +CB-WP-0025 T07. Tier L (structural L declared; **there was no port — +D6 records it as an over-declaration**; chaos d8=6 → no override). +Declaration 8 of chaos window 2. Closed 2026-08-05. + +**Delivered:** [CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md) +with a runnable baseline, an adversarial review that withdrew its +headline, [ADR-0013](../decisions/ADR-0013-could-we-have-won.md), +[RetrospectiveAnalysis.md](../specs/RetrospectiveAnalysis.md), +`games/ground/src/search.rs`, `benches/search.rs`, +`examples/difficulty.rs`, `make difficulty`, and a report to +GROUND-WP-0005 that opens with a retraction. + +--- + +## 1. Was the maintainer's game winnable? + +The question the pass exists for. **Yes, and the tool answers it.** + +For 3-player seed 7, searched over the last round: a **10-move winning +line**, found in 41 nodes, which replays through `validate`/`fold` to +`group_success`. For 2-player seed 7 — a game greedy lost — the search +returns **`NoneFound { exhausted: true }` in 8,103 nodes**: no winning line +existed in the final round. That is a real negative, not a budget cut, and +the type keeps the two apart. + +**The feature works. The difficulty half is where the trouble was.** + +## 2. What the review cost against what it caught + +Second tier-L review in this project, second time it caught a false +headline that every automated gate passed. `make all` was green +throughout. + +| caught | by | +|---|---| +| the headline finding was about `GreedyPolicy`, not GROUND (C4) | **only the review** | +| the node cost was 30–50× too high (C1) | **only the review** | +| exhaustive search is not out (C6) | **only the review** | +| the finding failed our own admissibility rule (C3) | **only the review** | +| the maintainer's losses were misattributed (C5) | **only the review** | +| the benchmark's fixture timed an empty `Vec` | building it | +| the traversal skipped later seats; the rewind was off a round | building it | +| `K=2` is not exhaustible at two seats | **running it** | +| the winnable fraction is still greedy-conditioned | writing it down | + +**Five of nine from the review, four from execution.** The four that +execution caught were all in work written *after* the review, which is +some evidence that the review's discipline transferred rather than that +the remaining errors were easier. + +**The reviewer under-ranked their own best challenge.** They put C4 +fourth. Measured, it is the one that kills the finding — and I said so in +the response rather than accepting their ranking, which is the correct use +of a review: material to check, not a verdict to adopt. + +## 3. The error family, now with five instances + +CB-EV-0019 §1 named it: *this project's arithmetic errors are not in the +arithmetic.* A number computed correctly against the wrong base, span, or +subject. + +| # | pass | the wrong thing | +|---|---|---| +| 1 | CB-WP-0021 | `csv` cost against AM-4a, which never sees the code | +| 2 | ground-game report | *"12 in the file"* — a sum with no deal table | +| 3 | CB-WP-0018 | SOLVE's inertness attributed to the wrong condition | +| 4 | **this pass, C1** | a timer bracketing whole games, divided by decisions | +| 5 | **this pass, C4** | a win rate attributed to the game, not the policy | + +**And two projections published in place of measurements**, both in this +pass: the survey's `7.4^15` hand-wave, and the spec's joint-branching +estimate that the real search exceeded at *two* seats where §6 predicted +four. + +**`facts-check` catches copies that disagree. Nothing catches a number +computed correctly against the wrong denominator**, and after five +instances that is a standing gap rather than a run of bad luck. Naming it +is what this file can do; closing it is not this pass's. + +## 4. What the difficulty measurement actually shows + +`make difficulty`, 60 seeds: + +| seats | winnable | greedy | random | first-legal | spread | +|---|---:|---:|---:|---:|---:| +| 2 | 60% | 60.0% | 5.0% | 76.7% | 71.7 | +| 3 | 93% | 88.3% | 6.7% | 25.0% | 81.7 | +| 4 | 100% | 93.3% | 6.7% | 30.0% | 86.7 | +| 5 | 100% | 100.0% | 3.3% | 0.0% | 100.0 | +| 6 | 100% | 100.0% | 3.3% | 0.0% | 100.0 | + +**The `spread` column is the finding.** 71.7 to 100.0 points between three +trivial policies. It is the evidence for the prohibition in +RetrospectiveAnalysis §4.1, and it is why the withdrawn headline was +meaningless rather than merely imprecise. + +**The 5–6 seat rows point the opposite way from the withdrawn claim.** +`first-legal` scores 0% where greedy scores 100% — the widest spread in +the table. Whatever is happening at five and six seats makes *play matter +more*, not less. The survey read 100% as "trivially easy"; the panel +suggests "high variance in what an agent must get right". **Neither is +established**, and saying so is the honest state. + +**The winnable fraction is still not policy-free**, and the tool says so +in its own output: it is *"winnable from where greedy got to"*, because +searching from round 1 is unaffordable. Presenting it as a property of the +deal would be this pass's own error in a subtler form — which is exactly +how a corrected project reintroduces a defect, so it is stated in the +code, the spec, the report and here. + +## 5. Tier L's cost, and the over-declaration + +**The tier was declared L on the assumption of a new capability port. +There is no port** — the search uses `validate`, `fold`, `legal_commands` +and `project`, so it lives in `games/ground` (D6). By the structural +trigger this was an **M**. + +**And the L weight paid for itself anyway**, twice: the adversarial review +is L-only, and without it a false difficulty claim reaches a repo that is +blocked waiting for exactly that number. **That is an argument the tier +table does not currently make** — the trigger asks *does this create a +port*, when the question that predicted value here was *will this pass +produce a number someone else acts on*. Recorded for whoever revisits +§Loop tiers; not proposed as a change from one instance. + +## 6. Chaos window 2 + +**Declaration 8 of 12.** Structural L (over-declared), d8 = 6, no +override. + +**Eleven declarations, no 8 rolled.** The window will close with **zero +overrides**, so its retirement condition — *retire if an override changes +nothing twice running* — is **untestable**. Window 1 closed with two +overrides at d4 and both changed the outcome. + +**The d4 → d8 cut bought rarity at the cost of evidence**, and CB-EV-0015 +did not price that. At d8 over twelve declarations the expected count is +1.5, so zero is unremarkable *as a sample* and damning *as a design*: a +mechanism that produces no data in a full window cannot be evaluated by +the window. **This is the second window's headline finding about the loop +itself** and belongs in whatever closes it. + +## 7. Cost + +CB-WP-0024's cost, by re-running the instrument: `make cost`. Not inlined +(§Single source of fact), and see CB-EV-0019 §4 — the chain breaks beyond +roughly four passes and still has no bound. + +## Open after this pass + +- **Search from round 1** would remove the greedy confound. Needs + transposition or move-ordering; neither built. +- **Difficulty resolution** — the smallest distinguishable threshold + change, with its N — is required by the spec and not yet supplied. +- **The witness is not wired to the ending page.** T05 built the search; + the browser cannot yet ask it. That is the maintainer-facing half of + remark 2 and it is not done. +- **The wrong-denominator family has no control** (§3), at five instances. +- **The chaos window's retirement condition is untestable** (§6). diff --git a/games/ground/examples/difficulty-baseline.rs b/games/ground/examples/difficulty-baseline.rs index 51140ac..1737dff 100644 --- a/games/ground/examples/difficulty-baseline.rs +++ b/games/ground/examples/difficulty-baseline.rs @@ -1,5 +1,21 @@ //! CB-RES-0008's runnable baseline (CB-WP-0025 T01). //! +//! ── SUPERSEDED 2026-08-05 by `difficulty.rs` (CB-WP-0025 T06) ────────── +//! +//! Kept as the survey's dated snapshot, NOT as a live instrument, for the +//! same reason `tools/design-baseline.py` was kept: it is the evidence for +//! how a wrong number got into a survey. +//! +//! **Its headline was withdrawn.** It reported `GreedyPolicy` winning +//! 200/200 at five and six seats and the survey called the game too easy +//! there. A `FirstLegal` policy scores **0%** on the same deals — the +//! measurement was about the policy. See the challenge (C4) and the +//! response. +//! +//! **And it is inadmissible under GameDesign §1.3**: no assertions, no +//! `--self-test`, no `make` target, so nothing can turn it red. +//! `difficulty.rs` carries all three. +//! //! Two numbers the survey needs and cannot cite from anyone else, because //! they are about **our** game on **our** machine: //! diff --git a/games/ground/examples/difficulty.rs b/games/ground/examples/difficulty.rs new file mode 100644 index 0000000..4992767 --- /dev/null +++ b/games/ground/examples/difficulty.rs @@ -0,0 +1,304 @@ +//! CB-WP-0025 T06 — the difficulty table, and the instrument that can fail. +//! +//! Implements `specs/RetrospectiveAnalysis.md` §4. Replaces +//! `difficulty-baseline.rs`, which CB-WP-0022's admissibility rule and +//! this pass's own review both found inadmissible: it had no assertions, +//! no `--self-test` and no `make` target, so nothing could turn it red. +//! +//! ## What it will not print +//! +//! **A single policy's win rate as a difficulty** (§4.1). Measured on +//! identical deals, `GreedyPolicy` wins 100% at five and six seats where +//! `FirstLegal` wins 0%. The panel is plural for that reason, and the +//! spread is reported rather than hidden. +//! +//! ```text +//! cargo run --release -p games-ground --example difficulty [--self-test] +//! ``` + +use cb_game_runtime::{ScenarioGame, Setup}; +use cb_kernel::{Aggregate, PlayerId}; +use games_ground::bot::{play, Choice, GreedyPolicy, Policy, RandomPolicy}; +use games_ground::search::{winnable_within, Verdict}; +use games_ground::{GroundCommand, GroundState}; + +/// Seeds per cell. Small by default: the winnable fraction needs an +/// exhaustive search per deal, which is the expensive half (§3.1). +const SEEDS: u64 = 60; +/// §3's measured limit: `K=1` is exhaustible, `K=2` is not. +const K: u8 = 1; +const BUDGET: usize = 2_000_000; + +/// A policy with no heuristic at all. **It is in the panel because it is +/// what falsified the survey's headline** — it beats greedy at two seats +/// and scores zero at six. +struct FirstLegal; +impl Policy for FirstLegal { + fn name(&self) -> &'static str { + "first-legal" + } + fn choose( + &mut self, + _s: &GroundState, + _seat: PlayerId, + _legal: &[GroundCommand], + _may_pass: bool, + ) -> Choice { + Choice::Command(0) + } +} + +fn setup(players: u8, seed: u64) -> Option { + GroundState::setup( + &Setup { + players, + preset: format!("standard-{players}p"), + patch: Default::default(), + }, + seed, + ) + .ok() +} + +fn policies(kind: &str, players: u8, seed: u64) -> Vec> { + (0..players) + .map(|i| -> Box { + match kind { + "random" => Box::new(RandomPolicy::new(seed ^ u64::from(i))), + "first-legal" => Box::new(FirstLegal), + _ => Box::new(GreedyPolicy), + } + }) + .collect() +} + +/// Games a named policy actually wins. +fn policy_wins(kind: &str, players: u8) -> (u32, u32) { + let (mut wins, mut played) = (0, 0); + for seed in 0..SEEDS { + let Some(state) = setup(players, seed) else { + continue; + }; + let mut ps = policies(kind, players, seed); + let Ok(game) = play(state, &mut ps) else { + continue; + }; + if let Some(o) = &game.state.outcome { + played += 1; + if o.group_success { + wins += 1; + } + } + } + (wins, played) +} + +/// Rewind a played game to the start of its last `k` rounds. +fn last_rounds(players: u8, seed: u64, k: usize) -> Option { + let mut ps = policies("greedy", players, seed); + let game = play(setup(players, seed)?, &mut ps).ok()?; + let total = game + .steps + .iter() + .filter(|(_, c)| matches!(c, GroundCommand::EndRound)) + .count(); + let mut st = setup(players, seed)?; + let mut ends = 0usize; + for (a, c) in &game.steps { + if let Ok(ev) = st.validate(*a, c) { + for e in &ev { + st.fold(e); + } + } + if matches!(c, GroundCommand::EndRound) { + ends += 1; + if ends >= total.saturating_sub(k) { + break; + } + } + } + Some(st) +} + +/// In what fraction of deals does a winning line exist in the last `K` +/// rounds? +/// +/// **Less policy-dependent than a win rate, but NOT policy-free, and +/// saying otherwise would repeat this pass's own error in a subtler +/// form.** The position searched is the one `GreedyPolicy` left at the +/// start of the last round, so the figure is *"winnable from where greedy +/// got to"*. A genuinely policy-free measure would search from round 1, +/// which §3.1 measured as unaffordable. +/// +/// What it does buy: the last round's outcome no longer depends on which +/// agent plays it, so the measure is insensitive to exactly the variation +/// that made the bot rate meaningless (§4.1). +/// +/// Deals where the search was cut by its budget are **not counted either +/// way** — they are reported separately, because folding "we stopped +/// looking" into "not winnable" is exactly the collapse §2.3 forbids. +fn winnable_fraction(players: u8) -> (u32, u32, u32) { + let (mut yes, mut decided, mut undecided) = (0, 0, 0); + for seed in 0..SEEDS { + let Some(state) = last_rounds(players, seed, K as usize) else { + continue; + }; + match winnable_within(&state, K, BUDGET) { + Verdict::Winnable { .. } => { + yes += 1; + decided += 1; + } + Verdict::NoneFound { + exhausted: true, .. + } => decided += 1, + Verdict::NoneFound { + exhausted: false, .. + } => undecided += 1, + } + } + (yes, decided, undecided) +} + +fn report() { + println!("difficulty — specs/RetrospectiveAnalysis.md §4\n"); + println!( + " winnable-from-round-{} over {SEEDS} seeds, budget {BUDGET} nodes", + 6 - K + ); + println!(" policy win rates over the same {SEEDS} seeds\n"); + println!(" seats winnable greedy random first-legal spread undecided"); + + for players in [2u8, 3, 4, 5, 6] { + let (yes, decided, undecided) = winnable_fraction(players); + let pct = |(w, n): (u32, u32)| { + if n == 0 { + -1.0 + } else { + 100.0 * f64::from(w) / f64::from(n) + } + }; + let g = pct(policy_wins("greedy", players)); + let r = pct(policy_wins("random", players)); + let f = pct(policy_wins("first-legal", players)); + let spread = [g, r, f].iter().cloned().fold(f64::MIN, f64::max) + - [g, r, f].iter().cloned().fold(f64::MAX, f64::min); + let wf = if decided == 0 { + "n/a".to_string() + } else { + format!("{:.0}%", 100.0 * f64::from(yes) / f64::from(decided)) + }; + println!( + " {players}p {wf:>6} {g:>5.1}% {r:>5.1}% {f:>5.1}% \ + {spread:>5.1} {undecided:>3}" + ); + } + + println!( + "\n WINNABLE is conditioned on GREEDY's play up to the last round —\n \ + it is 'winnable from where greedy got to', not a property of the\n \ + deal alone. Searching from round 1 is unaffordable (spec §3.1).\n\n \ + It is also a LOWER BOUND: a K={K} search cannot see a line that\n \ + needed an earlier round. `undecided` are deals whose search hit the\n \ + node budget — they are excluded from the fraction, not counted as\n \ + unwinnable.\n\n \ + SPREAD is the range across three policies. Where it is large, no\n \ + single policy's rate says anything about the game (§4.1)." + ); +} + +fn self_test() -> i32 { + let mut ok = true; + let mut check = |name: &str, cond: bool, detail: String| { + ok &= cond; + println!( + " [{}] {name}{}", + if cond { "ok " } else { "FAIL" }, + if detail.is_empty() { + String::new() + } else { + format!(" — {detail}") + } + ); + }; + + // The control the old harness never had: a search that finds a line + // must produce one that REPLAYS. §2.1 is a gate, not a metric. + let state = last_rounds(3, 7, K as usize).expect("a 3p game"); + match winnable_within(&state, K, BUDGET) { + Verdict::Winnable { line, .. } => { + let mut replay = state.clone(); + let mut good = true; + for m in &line { + match replay.validate(m.actor, &m.command) { + Ok(ev) => { + for e in &ev { + replay.fold(e); + } + } + Err(_) => good = false, + } + } + let won = replay.outcome.as_ref().is_some_and(|o| o.group_success); + check( + "a witness replays to a win", + good && won, + format!("{} moves", line.len()), + ); + } + Verdict::NoneFound { .. } => check( + "a witness replays to a win", + false, + "3p seed 7 found no line — the fixture moved".into(), + ), + } + + // The negative: a search must be able to return nothing, exhaustively. + let lost = last_rounds(2, 7, 1).expect("a 2p game"); + match winnable_within(&lost, 1, BUDGET) { + Verdict::NoneFound { exhausted, nodes } => check( + "an unwinnable position is reported as searched-out", + exhausted && nodes > 100, + format!("{nodes} nodes"), + ), + Verdict::Winnable { .. } => check( + "an unwinnable position is reported as searched-out", + false, + "found a win in a game 2p seed 7 lost".into(), + ), + } + + // A budget of one must NOT claim exhaustion — the distinction §2.3 + // rests on. + match winnable_within(&state, K, 1) { + Verdict::NoneFound { exhausted, .. } => check( + "a budget cut is not reported as exhaustion", + !exhausted, + String::new(), + ), + Verdict::Winnable { .. } => check( + "a budget cut is not reported as exhaustion", + false, + String::new(), + ), + } + + // §4.1's reason, asserted rather than asserted-about: the panel must + // actually disagree, or reporting three policies is ceremony. + let g = policy_wins("greedy", 6); + let f = policy_wins("first-legal", 6); + check( + "the policy panel disagrees, so no single rate is a difficulty", + g.0 != f.0, + format!("greedy {}/{}, first-legal {}/{}", g.0, g.1, f.0, f.1), + ); + + println!("difficulty self-test (positive control)"); + i32::from(!ok) +} + +fn main() { + if std::env::args().any(|a| a == "--self-test") { + std::process::exit(self_test()); + } + report(); +} diff --git a/specs/FindingRegister.md b/specs/FindingRegister.md index 42be87e..1490bbf 100644 --- a/specs/FindingRegister.md +++ b/specs/FindingRegister.md @@ -43,6 +43,7 @@ kinds, states and metrics: [`GameDesign.md`](GameDesign.md). Reported by | F13 | inconsistent | withdrawn | scenarios/ground/gr-e01-threshold-reachable-2p.yaml | counterexample | 2026-08-01 | clay-borg | | F14 | unplayed | note | — | — | 2026-08-01 | clay-borg | | F15 | underdetermined | note | — | — | 2026-08-05 | clay-borg | +| F16 | inconsistent | withdrawn | games/ground/examples/difficulty.rs | counterexample | 2026-08-05 | clay-borg | @@ -62,6 +63,16 @@ kinds, states and metrics: [`GameDesign.md`](GameDesign.md). Reported by `-reachable-`. **Its reproduction is green**, which under GameDesign §1.3 is the alarm that forced the resolution. Withdrawn rather than deleted, and the withdrawal is reported (ADR-0012 D5). +- **F16 — "the game is too easy at 5–6 seats", withdrawn the day it was + raised.** Claimed from `GreedyPolicy` winning 200/200 at those seat + counts. A `FirstLegal` policy scores **0%** on the identical deals, and + at two seats it *beats* greedy — two unsophisticated agents span the + whole range, so the measurement was about the policy. Caught by the + CB-WP-0025 adversarial review (C4) **before transmission**; it would + have been the fifth wrong premise sent to ground-game and the worst, + since GROUND-WP-0005 is blocked on exactly this number. The withdrawal + was reported (ADR-0012 D5). Its reproduction is `difficulty.rs`, whose + policy panel is plural *because of this finding*. - **F15 — the rules define one game, not a series.** `OutcomeView` gives `personal` (per seat), `group_success` (per table) and `winners`. Summing the first and counting the third answer different questions, and GROUND diff --git a/workplans/CB-WP-0025-could-we-have-won.md b/workplans/CB-WP-0025-could-we-have-won.md index 4294220..3074943 100644 --- a/workplans/CB-WP-0025-could-we-have-won.md +++ b/workplans/CB-WP-0025-could-we-have-won.md @@ -2,7 +2,7 @@ id: CB-WP-0025 kind: product title: "Could we have won: a path out of a lost game, and how hard the game actually is" -status: active +status: done state_hub_workstream_id: "a866982f-94e7-432b-b0d5-2fefb781a574" --- @@ -128,23 +128,10 @@ state_hub_task_id: "bdd9fc97-4706-4d8e-b286-91ff53339680" ``` Tier L requires it. Exactly one round: challenge, then response, trail in -`history/`, unpolished. Require an attempt at: - -- **that the honest version is unaffordable** — that a search respecting - the information rule is too expensive or too weak to find anything, so - the shipped tool will quietly become the omniscient one with a - reassuring label; -- **that a witness misleads more than it helps** — being shown a line that - needed a card you could not know about teaches a wrong lesson about the - game, and the tool would be better refusing to answer; -- **that the difficulty number is a bot benchmark wearing a difficulty - costume**, and `ground-game` will tune the game against our bot rather - than against play; -- **that this is CB-WP-0022's job** — the design instrument is being built - right now, and a difficulty measurement is a finding-producing tool. The - strongest counter is that the register records findings and this - *produces* them, but the reviewer should press whether that is a - distinction worth a separate capability. +`history/`, unpolished. Four attempts required: that the honest version is +unaffordable; that a witness misleads more than it helps; that the +difficulty number is a bot benchmark in a difficulty costume; and that +this is CB-WP-0022's job rather than a separate capability. **Done 2026-08-05.** Trail: [challenge](../history/260805-could-we-have-won-challenge.md), @@ -183,22 +170,12 @@ state_hub_task_id: "055630a1-56e0-4ad4-8796-35f444313371" ``` `decisions/ADR-0013-*.md`. (ADR-0012 is CB-WP-0022 T03's.) At minimum: - -- **which question the solver answers**, from the three in §What makes - this hard, and what it is called in the UI — the name must not overclaim; -- **the information boundary**: whether the search runs on `GroundState` - or on a `GroundView`, and if on state, what stops it using what the view - hides. Note that running on the view makes the rule structural rather - than a promise, and that this is the cheapest guarantee available; -- **the bound**: depth, node budget, or wall clock, and what *no path - found* means against it — a bounded search that says "unwinnable" is - lying, and the wording must say "none found within B"; -- **whether difficulty ships as one number or a small table**, and what it - is a function of: policy, seat count, threshold, seed range; -- **where it lives** — a new crate, a mode of `cb-play`, or a tool under - `tools/`. The tier was declared L on the assumption of a new capability; - if the ADR concludes it is a mode of an existing one, say so, and the - over-declaration is a chaos-window data point worth recording. +which question the solver answers and what it is called in the UI; the +**information boundary** and whether it is structural or promised; the +**bound**, and what *no path found* means against it; whether difficulty +is one number or a table, and of what; and **where it lives** — the tier +was declared L assuming a new capability port, and if there is none, say +so. **Done 2026-08-05.** [ADR-0013](../decisions/ADR-0013-could-we-have-won.md), seven decisions. @@ -241,18 +218,12 @@ state_hub_task_id: "7c3ca25d-0001-44ef-a07d-0ab2d51b09a5" `specs/` — extend `MetricsAndScenarios.md` or add a capability spec as the ADR directs, with metrics, because a spec without them is prose. -Candidates, to be argued not adopted: - -- **witness checkability** — every path the tool emits replays through the - existing scenario runner and ends in `group_success`. Target 100%, and it - is a hard gate, not a metric: a path that does not replay is a bug that - says the opposite of the truth; -- **search cost** — nodes and wall clock at the chosen bound, on the - recorded games we have; -- **difficulty resolution** — the smallest threshold difference the - measurement can distinguish, with its N. This is the number - `ground-game` needs, and stating it as *"we can tell 5 from 7 but not 7 - from 8"* is more useful than a win rate with no error bar. +Candidates, to be argued not adopted: **witness checkability** (every +emitted path replays to `group_success` — a hard gate, not a metric, since +a path that does not replay says the opposite of the truth); **search +cost** at the chosen bound; and **difficulty resolution** — the smallest +threshold difference the measurement can distinguish, with its N, which is +more useful to `ground-game` than any rate with no error bar. Per `ground-game`'s ruling (GROUND-WP-0004 T02), **any arithmetic finding this produces ships a runnable reproduction and a row-level table** — never @@ -326,45 +297,36 @@ obligatory ones are done — **so the search needs no phase logic of its own.** **And my rewind was off by one round**, replaying the round it was meant -to search. That is why the first run reported 3 nodes and looked like a -working search. +to search — which is why the first run reported 3 nodes and looked fine. -**The measurement falsified the spec's own projection, at two seats rather -than the four §6 predicted.** - -| case | result | -|---|---| -| 2p `K=1` | **exhausted**, 8,103 nodes, ~29 ms | -| 2p `K=2` | **budget cut** at 2,000,000 nodes, ~5 s | -| 3p `K=2` | win found, 41 nodes, ~157 µs | +**The measurement falsified the spec's own projection at two seats, where +§6 predicted four**: 2p `K=1` exhausted in 8,103 nodes (~29 ms), 2p `K=2` +cut at 2,000,000 nodes (~5 s), 3p `K=2` won in 41 nodes. The projection assumed a joint product per round; the search explores sequential per-seat decisions, so orderings multiply the tree far beyond `width^seats`. -**The asymmetry is the operative finding.** *Finding* a win is cheap — -DFS stumbles onto one in tens of nodes. *Proving none exists* needs -exhaustion. So the **witness feature is affordable now**, and the -**winnable fraction is not**, because its negative half must exhaust every -deal it counts. That is T06's problem and the spec now says so. +**The asymmetry is the operative finding.** *Finding* a win is cheap; +*proving none exists* needs exhaustion. So the **witness feature is +affordable now** and the **winnable fraction is not**, because its +negative half must exhaust every deal it counts. ## Task: measure the difficulty, and hand it to ground-game ```task id: CB-WP-0025-T06 -status: todo +status: done priority: high state_hub_task_id: "723feae8-1ed4-4dd4-9330-8ec84625e5d9" ``` Run the measurement, ship it as a `make` target beside the other -instruments, and show the result in the game — the maintainer asked for it -to be visible, and a number in a file will not calibrate anything. - -Then send it to `ground-game` **against GROUND-WP-0005**, which is active -with both tasks waiting on exactly this. Per CB-WP-0022 T06, it lands as a -file in their repo under their workplan, not only an inbox entry — *the -message that sat unread for four days is the baseline to beat*. +instruments, and send it to `ground-game` against **GROUND-WP-0005**, +which is active with both tasks waiting on exactly this. Per CB-WP-0022 +T06 it lands as a **file in their repo under their workplan**, not only an +inbox entry — *the message that sat unread for four days is the baseline +to beat*. **Controls:** - the number regenerates from a single command, and `facts.toml` carries @@ -374,22 +336,65 @@ message that sat unread for four days is the baseline to beat*. - **the seed range and policy are in the number's name**, not in a footnote. +**Done 2026-08-05.** `games/ground/examples/difficulty.rs`, `make +difficulty`, wired into `make self-tests`, and +[a report file in ground-game](../../ground-game/workplans/GROUND-WP-0005-clay-borg-difficulty-260805.md) +under GROUND-WP-0005 with a hub message pointing at it. + +**The report opens with the retraction**, because what this task was +written to send was withdrawn by T02 and GROUND-WP-0005 is blocked on +exactly that number. + +Winnable-from-round-5, 60 seeds: **60% / 93% / 100% / 100% / 100%** at +2/3/4/5/6 seats, beside greedy / random / first-legal rates and the spread +between them. Full table in the report and in `make difficulty`. + +**`spread` justifies the whole redesign** — 71.7 to 100.0 points between +three trivial policies. No single rate says anything about the game, and +the table now shows that rather than asserting it. + +**The confound is stated in the tool's own output**: `winnable` is +conditioned on greedy's play up to the final round. Presenting it as a +property of the deal would repeat this pass's error in a subtler form. +**No threshold changes are proposed.** The instrument can fail (spec §5), +and `difficulty-baseline.rs` is marked superseded. + +Registered as **F16, `inconsistent`, `withdrawn`**, with `difficulty.rs` +as its reproduction. + ## Task: evidence ```task id: CB-WP-0025-T07 -status: todo +status: done priority: high state_hub_task_id: "2616e06c-5d88-4680-be18-4452515dff21" ``` -`evidence/CB-EV-0023-*.md`. +`evidence/CB-EV-*.md` (**CB-EV-0024** — CB-EV-0023 went to CB-WP-0024). +Was the maintainer's game winnable; what the +honest search cost against the omniscient one; whether the difficulty +measurement moved `ground-game`; what tier L cost against what it caught; +and CB-WP-0024's cost by re-running the instrument. -- **Was the game winnable**, for the maintainer's actual lost game. That is - the acceptance test with a face on it. -- **What the honest search cost against the omniscient one**, since the - review will have pressed hardest there. -- **Whether the difficulty measurement moved ground-game**, or sat. -- **What tier L cost against what it caught** — third full-weight L pass in - the project, and the second in this chaos window. -- **Quote CB-WP-0024's cost by re-running the instrument.** +**Done 2026-08-05.** +[CB-EV-0024](../evidence/CB-EV-0024-could-we-have-won.md). + +- **The maintainer's question is answered.** 3p seed 7 was winnable — a + 10-move line, found in 41 nodes, replays to `group_success`. 2p seed 7 + was **not**, exhaustively, in 8,103 nodes. **The witness feature works; + the difficulty half is where the trouble was.** +- **Five of nine defects came only from the review**, four from execution + — and all four of those were in work written *after* it. +- **The wrong-denominator family now has five instances and no control.** +- **The 5–6 seat rows point the opposite way from the withdrawn claim**; + neither reading is established, and the evidence says so. +- **Tier L was an over-declaration** (no port) **and paid for itself + anyway** — the review is L-only, and without it a false number reaches a + blocked repo. +- **Chaos window 2 will close with zero overrides**, making its retirement + condition untestable. + +**Not done, and named rather than quietly dropped:** the witness is **not +wired to the ending page**, so the browser cannot yet ask the question. +That is the maintainer-facing half of remark 2.