coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CB-WP-0025 T06/T07: a difficulty baseline that reports its own confound
Some checks failed
ci / check (push) Has been cancelled
Details

Browse source 
T06. games/ground/examples/difficulty.rs, make difficulty, wired into make
self-tests, and a report file in ground-game under GROUND-WP-0005 with a
hub message pointing at it.

THE REPORT OPENS WITH THE RETRACTION, because what this task was written
to send was withdrawn by T02 and GROUND-WP-0005 is blocked on exactly that
number. They are told, in the first section, that we nearly sent them "the
game is too easy at 5-6 seats" and why it was wrong.

  seats  winnable  greedy  random  first-legal  spread
   2p      60%     60.0%    5.0%      76.7%      71.7
   3p      93%     88.3%    6.7%      25.0%      81.7
   4p     100%     93.3%    6.7%      30.0%      86.7
   5p     100%    100.0%    3.3%       0.0%     100.0
   6p     100%    100.0%    3.3%       0.0%     100.0

SPREAD justifies the whole redesign: 71.7 to 100.0 points between three
trivial policies. The table now shows why no single rate is a difficulty
rather than asserting it.

And the 5-6 rows point the OPPOSITE way from the withdrawn claim --
first-legal 0% against greedy 100% is the widest spread in the table,
which suggests play matters MORE there, not less. Neither reading is
established and the report says so.

The confound is stated in the tool's own output, not only in prose:
`winnable` is conditioned on greedy's play up to the final round, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would repeat this pass's error in a subtler form -- which is exactly
how a corrected project reintroduces a defect. NO THRESHOLD CHANGES ARE
PROPOSED.

The instrument can fail (spec §5): a witness must replay to a win, an
unwinnable position must report searched-out rather than a budget cut, a
one-node budget must not claim exhaustion, and the policy panel must
actually disagree. difficulty-baseline.rs marked superseded, kept as the
survey's dated snapshot. Registered as F16, inconsistent / withdrawn.

T07. evidence/CB-EV-0024. Five of nine defects came only from the review;
four from execution, and all four of those were in work written after it.
The wrong-denominator family now has five instances and still no control
-- facts-check catches copies that disagree, nothing catches a number
computed correctly against the wrong base. Tier L was an over-declaration
(no port, structurally M) and paid for itself anyway, because the review
is L-only. Chaos window 2 will close with zero overrides, making its
retirement condition untestable.

Named as open rather than implied done: the witness is NOT wired to the
ending page. The search works; the browser cannot ask it yet.

make all: exit 0. loop-lint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-05 19:24:49 +02:00
parent 81e0aba59a
commit 7ed9fc730a
6 changed files with 583 additions and 79 deletions
Download patch file Download diff file Expand all files Collapse all files

7
Makefile
View file

@ -125,6 +125,7 @@ self-tests:
$(PY) $(TOOLS)/runtime-metrics.py --self-test
$(PY) $(TOOLS)/replay-test.py --self-test
$(PY) $(TOOLS)/design.py --self-test
cargo run --release -q -p games-ground --example difficulty -- --self-test
$(PY) $(TOOLS)/edition-check.py --self-test
# T01 positive control: prove the environment fix, do not assume it. Runs
@ -160,6 +161,12 @@ facts-check:
facts-gen:
$(PY) $(TOOLS)/facts.py --gen
# CB-WP-0025 T06: the difficulty table (specs/RetrospectiveAnalysis.md §4).
# Winnable fraction from the solver plus a PLURAL policy panel -- a single
# policy's win rate may not be reported as a difficulty (§4.1).
difficulty:
@cargo run --release -q -p games-ground --example difficulty
# CB-WP-0022 T05: the design-finding register, reported over
# specs/GroundRules.md. Shows the QUEUE by default; the log of closed
# findings is a line, not a listing, because a default view that mixes

161
evidence/CB-EV-0024-could-we-have-won.md Normal file
View file

@ -0,0 +1,161 @@
# CB-EV-0024 — could we have won
CB-WP-0025 T07. Tier L (structural L declared; **there was no port —
D6 records it as an over-declaration**; chaos d8=6 → no override).
Declaration 8 of chaos window 2. Closed 2026-08-05.
**Delivered:** [CB-RES-0008](../research/CB-RES-0008-could-we-have-won.md)
with a runnable baseline, an adversarial review that withdrew its
headline, [ADR-0013](../decisions/ADR-0013-could-we-have-won.md),
[RetrospectiveAnalysis.md](../specs/RetrospectiveAnalysis.md),
`games/ground/src/search.rs`, `benches/search.rs`,
`examples/difficulty.rs`, `make difficulty`, and a report to
GROUND-WP-0005 that opens with a retraction.
---
## 1. Was the maintainer's game winnable?
The question the pass exists for. **Yes, and the tool answers it.**
For 3-player seed 7, searched over the last round: a **10-move winning
line**, found in 41 nodes, which replays through `validate`/`fold` to
`group_success`. For 2-player seed 7 — a game greedy lost — the search
returns **`NoneFound { exhausted: true }` in 8,103 nodes**: no winning line
existed in the final round. That is a real negative, not a budget cut, and
the type keeps the two apart.
**The feature works. The difficulty half is where the trouble was.**
## 2. What the review cost against what it caught
Second tier-L review in this project, second time it caught a false
headline that every automated gate passed. `make all` was green
throughout.
| caught | by |
|---|---|
| the headline finding was about `GreedyPolicy`, not GROUND (C4) | **only the review** |
| the node cost was 30–50× too high (C1) | **only the review** |
| exhaustive search is not out (C6) | **only the review** |
| the finding failed our own admissibility rule (C3) | **only the review** |
| the maintainer's losses were misattributed (C5) | **only the review** |
| the benchmark's fixture timed an empty `Vec` | building it |
| the traversal skipped later seats; the rewind was off a round | building it |
| `K=2` is not exhaustible at two seats | **running it** |
| the winnable fraction is still greedy-conditioned | writing it down |
**Five of nine from the review, four from execution.** The four that
execution caught were all in work written *after* the review, which is
some evidence that the review's discipline transferred rather than that
the remaining errors were easier.
**The reviewer under-ranked their own best challenge.** They put C4
fourth. Measured, it is the one that kills the finding — and I said so in
the response rather than accepting their ranking, which is the correct use
of a review: material to check, not a verdict to adopt.
## 3. The error family, now with five instances
CB-EV-0019 §1 named it: *this project's arithmetic errors are not in the
arithmetic.* A number computed correctly against the wrong base, span, or
subject.
| # | pass | the wrong thing |
|---|---|---|
| 1 | CB-WP-0021 | `csv` cost against AM-4a, which never sees the code |
| 2 | ground-game report | *"12 in the file"* — a sum with no deal table |
| 3 | CB-WP-0018 | SOLVE's inertness attributed to the wrong condition |
| 4 | **this pass, C1** | a timer bracketing whole games, divided by decisions |
| 5 | **this pass, C4** | a win rate attributed to the game, not the policy |
**And two projections published in place of measurements**, both in this
pass: the survey's `7.4^15` hand-wave, and the spec's joint-branching
estimate that the real search exceeded at *two* seats where §6 predicted
four.
**`facts-check` catches copies that disagree. Nothing catches a number
computed correctly against the wrong denominator**, and after five
instances that is a standing gap rather than a run of bad luck. Naming it
is what this file can do; closing it is not this pass's.
## 4. What the difficulty measurement actually shows
`make difficulty`, 60 seeds:
| seats | winnable | greedy | random | first-legal | spread |
|---|---:|---:|---:|---:|---:|
| 2 | 60% | 60.0% | 5.0% | 76.7% | 71.7 |
| 3 | 93% | 88.3% | 6.7% | 25.0% | 81.7 |
| 4 | 100% | 93.3% | 6.7% | 30.0% | 86.7 |
| 5 | 100% | 100.0% | 3.3% | 0.0% | 100.0 |
| 6 | 100% | 100.0% | 3.3% | 0.0% | 100.0 |
**The `spread` column is the finding.** 71.7 to 100.0 points between three
trivial policies. It is the evidence for the prohibition in
RetrospectiveAnalysis §4.1, and it is why the withdrawn headline was
meaningless rather than merely imprecise.
**The 5–6 seat rows point the opposite way from the withdrawn claim.**
`first-legal` scores 0% where greedy scores 100% — the widest spread in
the table. Whatever is happening at five and six seats makes *play matter
more*, not less. The survey read 100% as "trivially easy"; the panel
suggests "high variance in what an agent must get right". **Neither is
established**, and saying so is the honest state.
**The winnable fraction is still not policy-free**, and the tool says so
in its own output: it is *"winnable from where greedy got to"*, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would be this pass's own error in a subtler form — which is exactly
how a corrected project reintroduces a defect, so it is stated in the
code, the spec, the report and here.
## 5. Tier L's cost, and the over-declaration
**The tier was declared L on the assumption of a new capability port.
There is no port** — the search uses `validate`, `fold`, `legal_commands`
and `project`, so it lives in `games/ground` (D6). By the structural
trigger this was an **M**.
**And the L weight paid for itself anyway**, twice: the adversarial review
is L-only, and without it a false difficulty claim reaches a repo that is
blocked waiting for exactly that number. **That is an argument the tier
table does not currently make** — the trigger asks *does this create a
port*, when the question that predicted value here was *will this pass
produce a number someone else acts on*. Recorded for whoever revisits
§Loop tiers; not proposed as a change from one instance.
## 6. Chaos window 2
**Declaration 8 of 12.** Structural L (over-declared), d8 = 6, no
override.
**Eleven declarations, no 8 rolled.** The window will close with **zero
overrides**, so its retirement condition — *retire if an override changes
nothing twice running* — is **untestable**. Window 1 closed with two
overrides at d4 and both changed the outcome.
**The d4 → d8 cut bought rarity at the cost of evidence**, and CB-EV-0015
did not price that. At d8 over twelve declarations the expected count is
1.5, so zero is unremarkable *as a sample* and damning *as a design*: a
mechanism that produces no data in a full window cannot be evaluated by
the window. **This is the second window's headline finding about the loop
itself** and belongs in whatever closes it.
## 7. Cost
CB-WP-0024's cost, by re-running the instrument: `make cost`. Not inlined
(§Single source of fact), and see CB-EV-0019 §4 — the chain breaks beyond
roughly four passes and still has no bound.
## Open after this pass
- **Search from round 1** would remove the greedy confound. Needs
transposition or move-ordering; neither built.
- **Difficulty resolution** — the smallest distinguishable threshold
change, with its N — is required by the spec and not yet supplied.
- **The witness is not wired to the ending page.** T05 built the search;
the browser cannot yet ask it. That is the maintainer-facing half of
remark 2 and it is not done.
- **The wrong-denominator family has no control** (§3), at five instances.
- **The chaos window's retirement condition is untestable** (§6).

16
games/ground/examples/difficulty-baseline.rs
View file

@ -1,5 +1,21 @@
//! CB-RES-0008's runnable baseline (CB-WP-0025 T01).
//!
//! ── SUPERSEDED 2026-08-05 by `difficulty.rs` (CB-WP-0025 T06) ──────────
//!
//! Kept as the survey's dated snapshot, NOT as a live instrument, for the
//! same reason `tools/design-baseline.py` was kept: it is the evidence for
//! how a wrong number got into a survey.
//!
//! **Its headline was withdrawn.** It reported `GreedyPolicy` winning
//! 200/200 at five and six seats and the survey called the game too easy
//! there. A `FirstLegal` policy scores **0%** on the same deals — the
//! measurement was about the policy. See the challenge (C4) and the
//! response.
//!
//! **And it is inadmissible under GameDesign §1.3**: no assertions, no
//! `--self-test`, no `make` target, so nothing can turn it red.
//! `difficulty.rs` carries all three.
//!
//! Two numbers the survey needs and cannot cite from anyone else, because
//! they are about **our** game on **our** machine:
//!

304
games/ground/examples/difficulty.rs Normal file
View file

@ -0,0 +1,304 @@
//! CB-WP-0025 T06 — the difficulty table, and the instrument that can fail.
//!
//! Implements `specs/RetrospectiveAnalysis.md` §4. Replaces
//! `difficulty-baseline.rs`, which CB-WP-0022's admissibility rule and
//! this pass's own review both found inadmissible: it had no assertions,
//! no `--self-test` and no `make` target, so nothing could turn it red.
//!
//! ## What it will not print
//!
//! **A single policy's win rate as a difficulty** (§4.1). Measured on
//! identical deals, `GreedyPolicy` wins 100% at five and six seats where
//! `FirstLegal` wins 0%. The panel is plural for that reason, and the
//! spread is reported rather than hidden.
//!
//! ```text
//! cargo run --release -p games-ground --example difficulty [--self-test]
//! ```
use cb_game_runtime::{ScenarioGame, Setup};
use cb_kernel::{Aggregate, PlayerId};
use games_ground::bot::{play, Choice, GreedyPolicy, Policy, RandomPolicy};
use games_ground::search::{winnable_within, Verdict};
use games_ground::{GroundCommand, GroundState};
/// Seeds per cell. Small by default: the winnable fraction needs an
/// exhaustive search per deal, which is the expensive half (§3.1).
const SEEDS: u64 = 60;
/// §3's measured limit: `K=1` is exhaustible, `K=2` is not.
const K: u8 = 1;
const BUDGET: usize = 2_000_000;
/// A policy with no heuristic at all. **It is in the panel because it is
/// what falsified the survey's headline** — it beats greedy at two seats
/// and scores zero at six.
struct FirstLegal;
impl Policy for FirstLegal {
fn name(&self) -> &'static str {
"first-legal"
}
fn choose(
&mut self,
_s: &GroundState,
_seat: PlayerId,
_legal: &[GroundCommand],
_may_pass: bool,
) -> Choice {
Choice::Command(0)
}
}
fn setup(players: u8, seed: u64) -> Option<GroundState> {
GroundState::setup(
&Setup {
players,
preset: format!("standard-{players}p"),
patch: Default::default(),
},
seed,
)
.ok()
}
fn policies(kind: &str, players: u8, seed: u64) -> Vec<Box<dyn Policy>> {
(0..players)
.map(|i| -> Box<dyn Policy> {
match kind {
"random" => Box::new(RandomPolicy::new(seed ^ u64::from(i))),
"first-legal" => Box::new(FirstLegal),
_ => Box::new(GreedyPolicy),
}
})
.collect()
}
/// Games a named policy actually wins.
fn policy_wins(kind: &str, players: u8) -> (u32, u32) {
let (mut wins, mut played) = (0, 0);
for seed in 0..SEEDS {
let Some(state) = setup(players, seed) else {
continue;
};
let mut ps = policies(kind, players, seed);
let Ok(game) = play(state, &mut ps) else {
continue;
};
if let Some(o) = &game.state.outcome {
played += 1;
if o.group_success {
wins += 1;
}
}
}
(wins, played)
}
/// Rewind a played game to the start of its last `k` rounds.
fn last_rounds(players: u8, seed: u64, k: usize) -> Option<GroundState> {
let mut ps = policies("greedy", players, seed);
let game = play(setup(players, seed)?, &mut ps).ok()?;
let total = game
.steps
.iter()
.filter(|(_, c)| matches!(c, GroundCommand::EndRound))
.count();
let mut st = setup(players, seed)?;
let mut ends = 0usize;
for (a, c) in &game.steps {
if let Ok(ev) = st.validate(*a, c) {
for e in &ev {
st.fold(e);
}
}
if matches!(c, GroundCommand::EndRound) {
ends += 1;
if ends >= total.saturating_sub(k) {
break;
}
}
}
Some(st)
}
/// In what fraction of deals does a winning line exist in the last `K`
/// rounds?
///
/// **Less policy-dependent than a win rate, but NOT policy-free, and
/// saying otherwise would repeat this pass's own error in a subtler
/// form.** The position searched is the one `GreedyPolicy` left at the
/// start of the last round, so the figure is *"winnable from where greedy
/// got to"*. A genuinely policy-free measure would search from round 1,
/// which §3.1 measured as unaffordable.
///
/// What it does buy: the last round's outcome no longer depends on which
/// agent plays it, so the measure is insensitive to exactly the variation
/// that made the bot rate meaningless (§4.1).
///
/// Deals where the search was cut by its budget are **not counted either
/// way** — they are reported separately, because folding "we stopped
/// looking" into "not winnable" is exactly the collapse §2.3 forbids.
fn winnable_fraction(players: u8) -> (u32, u32, u32) {
let (mut yes, mut decided, mut undecided) = (0, 0, 0);
for seed in 0..SEEDS {
let Some(state) = last_rounds(players, seed, K as usize) else {
continue;
};
match winnable_within(&state, K, BUDGET) {
Verdict::Winnable { .. } => {
yes += 1;
decided += 1;
}
Verdict::NoneFound {
exhausted: true, ..
} => decided += 1,
Verdict::NoneFound {
exhausted: false, ..
} => undecided += 1,
}
}
(yes, decided, undecided)
}
fn report() {
println!("difficulty — specs/RetrospectiveAnalysis.md §4\n");
println!(
" winnable-from-round-{} over {SEEDS} seeds, budget {BUDGET} nodes",
6 - K
);
println!(" policy win rates over the same {SEEDS} seeds\n");
println!(" seats winnable greedy random first-legal spread undecided");
for players in [2u8, 3, 4, 5, 6] {
let (yes, decided, undecided) = winnable_fraction(players);
let pct = |(w, n): (u32, u32)| {
if n == 0 {
-1.0
} else {
100.0 * f64::from(w) / f64::from(n)
}
};
let g = pct(policy_wins("greedy", players));
let r = pct(policy_wins("random", players));
let f = pct(policy_wins("first-legal", players));
let spread = [g, r, f].iter().cloned().fold(f64::MIN, f64::max)
- [g, r, f].iter().cloned().fold(f64::MAX, f64::min);
let wf = if decided == 0 {
"n/a".to_string()
} else {
format!("{:.0}%", 100.0 * f64::from(yes) / f64::from(decided))
};
println!(
" {players}p {wf:>6} {g:>5.1}% {r:>5.1}% {f:>5.1}% \
{spread:>5.1} {undecided:>3}"
);
}
println!(
"\n WINNABLE is conditioned on GREEDY's play up to the last round —\n \
it is 'winnable from where greedy got to', not a property of the\n \
deal alone. Searching from round 1 is unaffordable (spec §3.1).\n\n \
It is also a LOWER BOUND: a K={K} search cannot see a line that\n \
needed an earlier round. `undecided` are deals whose search hit the\n \
node budget — they are excluded from the fraction, not counted as\n \
unwinnable.\n\n \
SPREAD is the range across three policies. Where it is large, no\n \
single policy's rate says anything about the game (§4.1)."
);
}
fn self_test() -> i32 {
let mut ok = true;
let mut check = |name: &str, cond: bool, detail: String| {
ok &= cond;
println!(
" [{}] {name}{}",
if cond { "ok " } else { "FAIL" },
if detail.is_empty() {
String::new()
} else {
format!(" — {detail}")
}
);
};
// The control the old harness never had: a search that finds a line
// must produce one that REPLAYS. §2.1 is a gate, not a metric.
let state = last_rounds(3, 7, K as usize).expect("a 3p game");
match winnable_within(&state, K, BUDGET) {
Verdict::Winnable { line, .. } => {
let mut replay = state.clone();
let mut good = true;
for m in &line {
match replay.validate(m.actor, &m.command) {
Ok(ev) => {
for e in &ev {
replay.fold(e);
}
}
Err(_) => good = false,
}
}
let won = replay.outcome.as_ref().is_some_and(|o| o.group_success);
check(
"a witness replays to a win",
good && won,
format!("{} moves", line.len()),
);
}
Verdict::NoneFound { .. } => check(
"a witness replays to a win",
false,
"3p seed 7 found no line — the fixture moved".into(),
),
}
// The negative: a search must be able to return nothing, exhaustively.
let lost = last_rounds(2, 7, 1).expect("a 2p game");
match winnable_within(&lost, 1, BUDGET) {
Verdict::NoneFound { exhausted, nodes } => check(
"an unwinnable position is reported as searched-out",
exhausted && nodes > 100,
format!("{nodes} nodes"),
),
Verdict::Winnable { .. } => check(
"an unwinnable position is reported as searched-out",
false,
"found a win in a game 2p seed 7 lost".into(),
),
}
// A budget of one must NOT claim exhaustion — the distinction §2.3
// rests on.
match winnable_within(&state, K, 1) {
Verdict::NoneFound { exhausted, .. } => check(
"a budget cut is not reported as exhaustion",
!exhausted,
String::new(),
),
Verdict::Winnable { .. } => check(
"a budget cut is not reported as exhaustion",
false,
String::new(),
),
}
// §4.1's reason, asserted rather than asserted-about: the panel must
// actually disagree, or reporting three policies is ceremony.
let g = policy_wins("greedy", 6);
let f = policy_wins("first-legal", 6);
check(
"the policy panel disagrees, so no single rate is a difficulty",
g.0 != f.0,
format!("greedy {}/{}, first-legal {}/{}", g.0, g.1, f.0, f.1),
);
println!("difficulty self-test (positive control)");
i32::from(!ok)
}
fn main() {
if std::env::args().any(|a| a == "--self-test") {
std::process::exit(self_test());
}
report();
}

11
specs/FindingRegister.md
View file

@ -43,6 +43,7 @@ kinds, states and metrics: [`GameDesign.md`](GameDesign.md). Reported by
| F13 | inconsistent | withdrawn | scenarios/ground/gr-e01-threshold-reachable-2p.yaml | counterexample | 2026-08-01 | clay-borg |
| F14 | unplayed | note | — | — | 2026-08-01 | clay-borg |
| F15 | underdetermined | note | — | — | 2026-08-05 | clay-borg |
| F16 | inconsistent | withdrawn | games/ground/examples/difficulty.rs | counterexample | 2026-08-05 | clay-borg |
<!-- design-register:end -->
@ -62,6 +63,16 @@ kinds, states and metrics: [`GameDesign.md`](GameDesign.md). Reported by
`-reachable-`. **Its reproduction is green**, which under GameDesign §1.3
is the alarm that forced the resolution. Withdrawn rather than deleted,
and the withdrawal is reported (ADR-0012 D5).
- **F16 — "the game is too easy at 5–6 seats", withdrawn the day it was
raised.** Claimed from `GreedyPolicy` winning 200/200 at those seat
counts. A `FirstLegal` policy scores **0%** on the identical deals, and
at two seats it *beats* greedy — two unsophisticated agents span the
whole range, so the measurement was about the policy. Caught by the
CB-WP-0025 adversarial review (C4) **before transmission**; it would
have been the fifth wrong premise sent to ground-game and the worst,
since GROUND-WP-0005 is blocked on exactly this number. The withdrawal
was reported (ADR-0012 D5). Its reproduction is `difficulty.rs`, whose
policy panel is plural *because of this finding*.
- **F15 — the rules define one game, not a series.** `OutcomeView` gives
`personal` (per seat), `group_success` (per table) and `winners`. Summing
the first and counting the third answer different questions, and GROUND

163
workplans/CB-WP-0025-could-we-have-won.md
View file

@ -2,7 +2,7 @@
id: CB-WP-0025
kind: product
title: "Could we have won: a path out of a lost game, and how hard the game actually is"
status: active
status: done
state_hub_workstream_id: "a866982f-94e7-432b-b0d5-2fefb781a574"
---
@ -128,23 +128,10 @@ state_hub_task_id: "bdd9fc97-4706-4d8e-b286-91ff53339680"
```
Tier L requires it. Exactly one round: challenge, then response, trail in
`history/`, unpolished. Require an attempt at:
- **that the honest version is unaffordable** — that a search respecting
the information rule is too expensive or too weak to find anything, so
the shipped tool will quietly become the omniscient one with a
reassuring label;
- **that a witness misleads more than it helps** — being shown a line that
needed a card you could not know about teaches a wrong lesson about the
game, and the tool would be better refusing to answer;
- **that the difficulty number is a bot benchmark wearing a difficulty
costume**, and `ground-game` will tune the game against our bot rather
than against play;
- **that this is CB-WP-0022's job** — the design instrument is being built
right now, and a difficulty measurement is a finding-producing tool. The
strongest counter is that the register records findings and this
*produces* them, but the reviewer should press whether that is a
distinction worth a separate capability.
`history/`, unpolished. Four attempts required: that the honest version is
unaffordable; that a witness misleads more than it helps; that the
difficulty number is a bot benchmark in a difficulty costume; and that
this is CB-WP-0022's job rather than a separate capability.
**Done 2026-08-05.** Trail:
[challenge](../history/260805-could-we-have-won-challenge.md),
@ -183,22 +170,12 @@ state_hub_task_id: "055630a1-56e0-4ad4-8796-35f444313371"
```
`decisions/ADR-0013-*.md`. (ADR-0012 is CB-WP-0022 T03's.) At minimum:
- **which question the solver answers**, from the three in §What makes
this hard, and what it is called in the UI — the name must not overclaim;
- **the information boundary**: whether the search runs on `GroundState`
or on a `GroundView`, and if on state, what stops it using what the view
hides. Note that running on the view makes the rule structural rather
than a promise, and that this is the cheapest guarantee available;
- **the bound**: depth, node budget, or wall clock, and what *no path
found* means against it — a bounded search that says "unwinnable" is
lying, and the wording must say "none found within B";
- **whether difficulty ships as one number or a small table**, and what it
is a function of: policy, seat count, threshold, seed range;
- **where it lives** — a new crate, a mode of `cb-play`, or a tool under
`tools/`. The tier was declared L on the assumption of a new capability;
if the ADR concludes it is a mode of an existing one, say so, and the
over-declaration is a chaos-window data point worth recording.
which question the solver answers and what it is called in the UI; the
**information boundary** and whether it is structural or promised; the
**bound**, and what *no path found* means against it; whether difficulty
is one number or a table, and of what; and **where it lives** — the tier
was declared L assuming a new capability port, and if there is none, say
so.
**Done 2026-08-05.**
[ADR-0013](../decisions/ADR-0013-could-we-have-won.md), seven decisions.
@ -241,18 +218,12 @@ state_hub_task_id: "7c3ca25d-0001-44ef-a07d-0ab2d51b09a5"
`specs/` — extend `MetricsAndScenarios.md` or add a capability spec as the
ADR directs, with metrics, because a spec without them is prose.
Candidates, to be argued not adopted:
- **witness checkability** — every path the tool emits replays through the
existing scenario runner and ends in `group_success`. Target 100%, and it
is a hard gate, not a metric: a path that does not replay is a bug that
says the opposite of the truth;
- **search cost** — nodes and wall clock at the chosen bound, on the
recorded games we have;
- **difficulty resolution** — the smallest threshold difference the
measurement can distinguish, with its N. This is the number
`ground-game` needs, and stating it as *"we can tell 5 from 7 but not 7
from 8"* is more useful than a win rate with no error bar.
Candidates, to be argued not adopted: **witness checkability** (every
emitted path replays to `group_success` — a hard gate, not a metric, since
a path that does not replay says the opposite of the truth); **search
cost** at the chosen bound; and **difficulty resolution** — the smallest
threshold difference the measurement can distinguish, with its N, which is
more useful to `ground-game` than any rate with no error bar.
Per `ground-game`'s ruling (GROUND-WP-0004 T02), **any arithmetic finding
this produces ships a runnable reproduction and a row-level table** — never
@ -326,45 +297,36 @@ obligatory ones are done — **so the search needs no phase logic of its
own.**
**And my rewind was off by one round**, replaying the round it was meant
to search. That is why the first run reported 3 nodes and looked like a
working search.
to search — which is why the first run reported 3 nodes and looked fine.
**The measurement falsified the spec's own projection, at two seats rather
than the four §6 predicted.**
| case | result |
|---|---|
| 2p `K=1` | **exhausted**, 8,103 nodes, ~29 ms |
| 2p `K=2` | **budget cut** at 2,000,000 nodes, ~5 s |
| 3p `K=2` | win found, 41 nodes, ~157 µs |
**The measurement falsified the spec's own projection at two seats, where
§6 predicted four**: 2p `K=1` exhausted in 8,103 nodes (~29 ms), 2p `K=2`
cut at 2,000,000 nodes (~5 s), 3p `K=2` won in 41 nodes.
The projection assumed a joint product per round; the search explores
sequential per-seat decisions, so orderings multiply the tree far beyond
`width^seats`.
**The asymmetry is the operative finding.** *Finding* a win is cheap —
DFS stumbles onto one in tens of nodes. *Proving none exists* needs
exhaustion. So the **witness feature is affordable now**, and the
**winnable fraction is not**, because its negative half must exhaust every
deal it counts. That is T06's problem and the spec now says so.
**The asymmetry is the operative finding.** *Finding* a win is cheap;
*proving none exists* needs exhaustion. So the **witness feature is
affordable now** and the **winnable fraction is not**, because its
negative half must exhaust every deal it counts.
## Task: measure the difficulty, and hand it to ground-game
```task
id: CB-WP-0025-T06
status: todo
status: done
priority: high
state_hub_task_id: "723feae8-1ed4-4dd4-9330-8ec84625e5d9"
```
Run the measurement, ship it as a `make` target beside the other
instruments, and show the result in the game — the maintainer asked for it
to be visible, and a number in a file will not calibrate anything.
Then send it to `ground-game` **against GROUND-WP-0005**, which is active
with both tasks waiting on exactly this. Per CB-WP-0022 T06, it lands as a
file in their repo under their workplan, not only an inbox entry — *the
message that sat unread for four days is the baseline to beat*.
instruments, and send it to `ground-game` against **GROUND-WP-0005**,
which is active with both tasks waiting on exactly this. Per CB-WP-0022
T06 it lands as a **file in their repo under their workplan**, not only an
inbox entry — *the message that sat unread for four days is the baseline
to beat*.
**Controls:**
- the number regenerates from a single command, and `facts.toml` carries
@ -374,22 +336,65 @@ message that sat unread for four days is the baseline to beat*.
- **the seed range and policy are in the number's name**, not in a
footnote.
**Done 2026-08-05.** `games/ground/examples/difficulty.rs`, `make
difficulty`, wired into `make self-tests`, and
[a report file in ground-game](../../ground-game/workplans/GROUND-WP-0005-clay-borg-difficulty-260805.md)
under GROUND-WP-0005 with a hub message pointing at it.
**The report opens with the retraction**, because what this task was
written to send was withdrawn by T02 and GROUND-WP-0005 is blocked on
exactly that number.
Winnable-from-round-5, 60 seeds: **60% / 93% / 100% / 100% / 100%** at
2/3/4/5/6 seats, beside greedy / random / first-legal rates and the spread
between them. Full table in the report and in `make difficulty`.
**`spread` justifies the whole redesign** — 71.7 to 100.0 points between
three trivial policies. No single rate says anything about the game, and
the table now shows that rather than asserting it.
**The confound is stated in the tool's own output**: `winnable` is
conditioned on greedy's play up to the final round. Presenting it as a
property of the deal would repeat this pass's error in a subtler form.
**No threshold changes are proposed.** The instrument can fail (spec §5),
and `difficulty-baseline.rs` is marked superseded.
Registered as **F16, `inconsistent`, `withdrawn`**, with `difficulty.rs`
as its reproduction.
## Task: evidence
```task
id: CB-WP-0025-T07
status: todo
status: done
priority: high
state_hub_task_id: "2616e06c-5d88-4680-be18-4452515dff21"
```
`evidence/CB-EV-0023-*.md`.
`evidence/CB-EV-*.md` (**CB-EV-0024** — CB-EV-0023 went to CB-WP-0024).
Was the maintainer's game winnable; what the
honest search cost against the omniscient one; whether the difficulty
measurement moved `ground-game`; what tier L cost against what it caught;
and CB-WP-0024's cost by re-running the instrument.
- **Was the game winnable**, for the maintainer's actual lost game. That is
the acceptance test with a face on it.
- **What the honest search cost against the omniscient one**, since the
review will have pressed hardest there.
- **Whether the difficulty measurement moved ground-game**, or sat.
- **What tier L cost against what it caught** — third full-weight L pass in
the project, and the second in this chaos window.
- **Quote CB-WP-0024's cost by re-running the instrument.**
**Done 2026-08-05.**
[CB-EV-0024](../evidence/CB-EV-0024-could-we-have-won.md).
- **The maintainer's question is answered.** 3p seed 7 was winnable — a
10-move line, found in 41 nodes, replays to `group_success`. 2p seed 7
was **not**, exhaustively, in 8,103 nodes. **The witness feature works;
the difficulty half is where the trouble was.**
- **Five of nine defects came only from the review**, four from execution
— and all four of those were in work written *after* it.
- **The wrong-denominator family now has five instances and no control.**
- **The 5–6 seat rows point the opposite way from the withdrawn claim**;
neither reading is established, and the evidence says so.
- **Tier L was an over-declaration** (no port) **and paid for itself
anyway** — the review is L-only, and without it a false number reaches a
blocked repo.
- **Chaos window 2 will close with zero overrides**, making its retirement
condition untestable.
**Not done, and named rather than quietly dropped:** the witness is **not
wired to the ending page**, so the browser cannot yet ask the question.
That is the maintainer-facing half of remark 2.