diff --git a/Makefile b/Makefile index 914d2a2..e97e91a 100644 --- a/Makefile +++ b/Makefile @@ -238,6 +238,7 @@ panels: @cargo run --release -q -p games-ground --example attack-value @cargo run --release -q -p games-ground --example regulation @cargo run --release -q -p games-ground --example perfect-recall + @cargo run --release -q -p games-ground --example h2-panel # CB-WP-0022 T05: the design-finding register, reported over # specs/GroundRules.md. Shows the QUEUE by default; the log of closed diff --git a/evidence/CB-EV-0032-h2-measured.md b/evidence/CB-EV-0032-h2-measured.md new file mode 100644 index 0000000..63ce1af --- /dev/null +++ b/evidence/CB-EV-0032-h2-measured.md @@ -0,0 +1,123 @@ +# CB-EV-0032 — H2 measured + +CB-WP-0042 T05. Tier M. Closed 2026-08-08. **The first pass declared after +the chaos roll was retired** (ADR-0021), so there is no roll to record. + +**Delivered:** H2 implemented from `rules_delta.yaml`, measured against +**ground-game's own §3 criteria** — read from their design note, not +reused from H1 — and a control that isolates the mechanism. + +--- + +## 1. Their criteria, answered + +| # | criterion | verdict | +|---|---|---| +| 1 | greedy SHARED 3–4p **well above H1's 0** | **met, strongly** | +| 2 | Stress variance above baseline *and* H1-greedy | **met at 2p/4p/6p, marginally missed at 3p** | +| 3 | DARVO non-zero for a policy that **still sometimes wins** | **met** — and this was H1's clearest failure | +| 4 | bond cards show elevated SOLVE rate vs personal | **not met, and this panel cannot test it** (§3) | +| 5 | control: force all scopes global → H1-like collapse | **met, decisively** | + +## 2. The numbers + +SHARED GROUND, 200 games per cell. `spread` is the mean max−min final +Stress across seats — their criterion 2. + +**greedy** + +| seats | baseline won / spread | H1 won / spread | H2 won / spread | +|---|---|---|---| +| 2p | 132 / 0.05 | 68 / 0.34 | **87 / 0.47** | +| 3p | 165 / 1.57 | **0** / 0.00 | **120** / 1.50 | +| 4p | 190 / 1.83 | **0** / 0.00 | **175** / 2.40 | +| 6p | 200 / 1.55 | **0** / 0.00 | **199** / 2.02 | + +**reactive** — DARVO arms in brackets + +| seats | H1 won (arms) | H2 won (arms) | +|---|---|---| +| 2p | 66 (363) | 61 (346) | +| 3p | **0** (600) | **13** (499) | +| 4p | **0** (800) | **13** (470) | +| 6p | **0** (1200) | **57** (565) | + +**Criterion 1 is met with room.** H2 restores 73% of baseline wins at 3p +and 92% at 4p, where H1 destroyed them entirely. + +**Criterion 3 is the one that matters most.** Under H1 the unregulated +seat armed DARVO constantly and **never won**; under H2 it arms and wins +13/13/57. *"Non-zero for some policy that still sometimes wins"* is +exactly the shape H1 could not produce. + +**Criterion 2, honestly.** H2's spread beats baseline at 2p, 4p and 6p and +**misses at 3p — 1.50 against 1.57**. It beats H1-greedy everywhere, and +**H1-greedy's spread is 0.00 at 3p+**: a flat tax on every seat creates no +variance at all, which is the clearest statement of why scoping was the +right correction. + +## 3. Criterion 4 — the number is lower, and the number cannot mean what +the criterion asks + +Claim rate by scope, H2, greedy: + +| seats | global | personal | bond | +|---|---|---|---| +| 2p | 70.5% | 54.5% | — (no bond card, as designed) | +| 3p | 85.5% | 80.2% | **60.0%** | +| 4p | 99.0% | 98.5% | **87.5%** | +| 6p | 100% | 95.3% | 99.5% | + +**Bond is lower than personal at 3p and 4p — the opposite direction.** + +**And this panel cannot test the hypothesis.** The claim is that a shared +tick makes a Bond network *jointly motivated* to clear that card. +**No policy here models another seat, or knows what a scope is.** Greedy +picks SOLVE on suit availability. So the bond rate is driven by which +cards happen to be matchable, not by incentive — the mechanism the +criterion is about could not have shown up. + +**This was stated in the workplan before the run**, not discovered in the +numbers. The honest report is: **criterion 4 is untested, and the +incidental figure points the wrong way.** A table can answer it; this +cannot. + +## 4. Criterion 5 — the control isolates the mechanism + +Force every scope to `global` and change nothing else: + +| seats | H2 scoped | H2, all scopes global | +|---|---:|---:| +| 3p | **120** | **0** | +| 4p | **175** | **0** | + +**H1's collapse returns exactly.** That is a strong causal statement and +the best evidence in this file: **the scoping is what saves it**, not any +other difference between the packages. Their optional A/B was worth +running. + +## 5. What this does not say + +- **No felt-play.** H2's central claim is about *motivation* — a bonded + pair caring about each other's card — and §3 says as much. 200-game + aggregates measure dynamics, not motivation + ([`Taxonomy.md`](../specs/Taxonomy.md) §4). +- **The panel is still greedy-family**, as CB-EV-0031 said of itself. + Criterion 3 asks for "some policy that still sometimes wins" and we have + two, both derived from the same ranking. +- **These are single-sample figures.** CB-REV-0003 #10 found the 2p arm + counts varying 354–366 across seed windows; nothing here has been + re-derived on a second sample, and the `won` columns should be read as + approximate. +- **No re-review.** CB-WP-0038's tier-L review found twelve fatal defects + across three rounds, all in the instrument. This pass reuses that + instrument. + +## Open after this pass + +- **Criterion 4 needs a policy that models other seats**, or a table. +- **A second seed window**, to separate the invariant figures from the + sample-specific ones as CB-EV-0031 §1 now does. +- **The 3p variance miss** (1.50 vs 1.57) is within what a second sample + might move; it is reported as a miss because that is what this sample + says. diff --git a/games/ground/examples/h2-panel.rs b/games/ground/examples/h2-panel.rs new file mode 100644 index 0000000..be6680b --- /dev/null +++ b/games/ground/examples/h2-panel.rs @@ -0,0 +1,198 @@ +//! **H2 measured against ground-game's own criteria** (CB-WP-0042 T05). +//! +//! Their §3 targets, not H1's reused from memory: +//! +//! 1. group success (greedy SHARED, 3–4p) well above H1's 0 +//! 2. Stress **variance** — max−min across seats — higher than baseline +//! and H1-greedy +//! 3. DARVO non-zero **for some policy that still sometimes wins** +//! 4. bond cards show an elevated SOLVE rate versus personal +//! 5. control: force every scope global and H1-like collapse returns +//! +//! **Criterion 4 is the one this instrument probably cannot answer**, and +//! that is said up front rather than discovered in the numbers: no policy +//! here models another seat, or knows what a scope is. If bond and +//! personal rates come out equal it is because nothing in the panel could +//! have made them differ — not because the incentive fails at a table. + +use std::collections::BTreeMap; + +use cb_game_runtime::{ScenarioGame, Setup}; +use cb_kernel::PlayerId; +use games_ground::bot::{play, Choice, GreedyPolicy, Policy}; +use games_ground::edition::StressScope; +use games_ground::{Action, GroundCommand, GroundState, ScoringMode, Variant}; + +const GAMES: u32 = 200; + +/// Greedy with GROUND demoted below ATTACK — the unregulated seat from +/// CB-EV-0031, delegating so exactly one preference differs. +struct Reactive; + +impl Policy for Reactive { + fn name(&self) -> &'static str { + "reactive" + } + fn choose( + &mut self, + state: &GroundState, + seat: PlayerId, + legal: &[GroundCommand], + _may_pass: bool, + ) -> Choice { + let gated = state + .players + .get(&seat) + .is_some_and(|p| p.stress >= 4 && !p.freedom_gate_lifted); + let rank = |c: &GroundCommand| -> i32 { + match c { + GroundCommand::SelectAction { + action: Action::Ground, + .. + } if gated => 5, + other => GreedyPolicy::rank(state, seat, other), + } + }; + let mut best = 0; + for (i, c) in legal.iter().enumerate() { + if rank(c) > rank(&legal[best]) { + best = i; + } + } + Choice::Command(best) + } +} + +#[derive(Default)] +struct Cell { + games: u32, + played: u32, + won: u32, + darvo: u32, + /// Summed max−min final Stress, for criterion 2. + spread: u32, + /// Claimed / total, by scope, for criterion 4. + claimed: BTreeMap<&'static str, (u32, u32)>, +} + +fn sweep(variant: Variant, players: u8, reactive: bool, force_global: bool) -> Cell { + let mut c = Cell::default(); + for seed in 0..GAMES as u64 { + let Ok(st) = GroundState::setup( + &Setup { + players, + preset: format!("standard-{players}p"), + patch: Default::default(), + }, + seed, + ) else { + continue; + }; + let mut st = st.with_variant(variant); + st.mode = ScoringMode::SharedGround; + // Criterion 5's control: every scope forced global. + if force_global { + for p in st.problems.values_mut() { + if p.scope.is_some() { + p.scope = Some(StressScope::Global); + p.owner = None; + } + } + } + let mut ps: Vec> = (0..players) + .map(|_| { + if reactive { + Box::new(Reactive) as Box + } else { + Box::new(GreedyPolicy) as Box + } + }) + .collect(); + let Ok(g) = play(st, &mut ps) else { continue }; + c.games += 1; + if g.state.outcome.is_some() && g.rounds == 5 { + c.played += 1; + } + if g.state.outcome.as_ref().is_some_and(|o| o.group_success) { + c.won += 1; + } + for e in &g.events { + if matches!(e, games_ground::GroundEvent::DarvoTriggered { .. }) { + c.darvo += 1; + } + } + let stresses: Vec = g.state.players.values().map(|p| p.stress).collect(); + let (lo, hi) = ( + stresses.iter().copied().min().unwrap_or(0), + stresses.iter().copied().max().unwrap_or(0), + ); + c.spread += u32::from(hi - lo); + for p in g.state.problems.values() { + let name = match p.scope { + Some(StressScope::Global) => "global", + Some(StressScope::Personal) => "personal", + Some(StressScope::Bond) => "bond", + None => continue, + }; + let e = c.claimed.entry(name).or_default(); + e.1 += 1; + if p.claimed_by.is_some() { + e.0 += 1; + } + } + } + assert_eq!(c.games, GAMES, "{players}p {variant:?}: short cell"); + assert_eq!(c.played, GAMES, "{players}p {variant:?}: games did not finish"); + c +} + +fn main() { + println!("H2 — against ground-game's §3 criteria (CB-WP-0042 T05)\n"); + println!("SHARED GROUND, {GAMES} games per cell. `spread` is the mean"); + println!("max−min final Stress across seats — their criterion 2.\n"); + + for (label, reactive) in [("greedy", false), ("reactive", true)] { + println!("{label}"); + println!("seats variant won darvo spread"); + for players in [2u8, 3, 4, 6] { + for (vname, v) in [ + ("baseline", Variant::Baseline), + ("H1 ", Variant::H1ProblemStress), + ("H2 ", Variant::H2ScopedProblemStress), + ] { + let c = sweep(v, players, reactive, false); + println!( + " {players}p {vname} {:>4} {:>5} {:>5.2}", + c.won, + c.darvo, + f64::from(c.spread) / f64::from(GAMES) + ); + } + } + println!(); + } + + println!("criterion 4 — claim rate by scope (H2, greedy)"); + println!("seats global personal bond"); + for players in [2u8, 3, 4, 6] { + let c = sweep(Variant::H2ScopedProblemStress, players, false, false); + let rate = |k: &str| match c.claimed.get(k) { + Some((a, b)) if *b > 0 => format!("{:>6.1}%", 100.0 * f64::from(*a) / f64::from(*b)), + _ => " —".to_string(), + }; + println!( + " {players}p {} {} {}", + rate("global"), + rate("personal"), + rate("bond") + ); + } + + println!("\ncriterion 5 — control: every scope forced global (H2, greedy)"); + println!("seats won (scoped) won (all global)"); + for players in [3u8, 4] { + let scoped = sweep(Variant::H2ScopedProblemStress, players, false, false); + let global = sweep(Variant::H2ScopedProblemStress, players, false, true); + println!(" {players}p {:>4} {:>4}", scoped.won, global.won); + } +} diff --git a/workplans/CB-WP-0042-h2-scoped-problem-stress.md b/workplans/CB-WP-0042-h2-scoped-problem-stress.md index dc0e053..bebebaf 100644 --- a/workplans/CB-WP-0042-h2-scoped-problem-stress.md +++ b/workplans/CB-WP-0042-h2-scoped-problem-stress.md @@ -2,7 +2,7 @@ id: CB-WP-0042 kind: product title: "H2 — scoped problem stress" -status: active +status: done state_hub_workstream_id: "8f11d55b-3452-49f3-9857-d0bf06752f68" --- @@ -176,7 +176,7 @@ The test must fail if it does. ```task id: CB-WP-0042-T05 -status: todo +status: done priority: high state_hub_task_id: "0bcd97da-60b8-46b2-ae86-7a94f6af8dd5" ``` @@ -196,6 +196,34 @@ Same instrument as H1, same panel, both variants in one run. - **`make panels` runs it**, or the figures come from an ungated binary again (CB-REV-0002 #7). +**Done 2026-08-08.** +[CB-EV-0032](../evidence/CB-EV-0032-h2-measured.md). **H2 largely +succeeds where H1 failed.** + +| # | their criterion | verdict | +|---|---|---| +| 1 | greedy SHARED 3–4p above H1's 0 | **met** — 120 and 175 against H1's 0 and 0 | +| 2 | variance above baseline and H1-greedy | **met at 2p/4p/6p, missed at 3p** (1.50 vs 1.57) | +| 3 | DARVO for a policy that still sometimes wins | **met** — H1's clearest failure, now 13/13/57 wins with arms | +| 4 | bond SOLVE rate above personal | **not met, and untestable here** | +| 5 | control: all scopes global → collapse | **met decisively** — 120→0 and 175→0 | + +**Criterion 5 is the best evidence in the pass.** Forcing every scope to +`global` and changing nothing else reproduces H1's collapse exactly, which +says the **scoping** is what saves it rather than any other difference +between the packages. + +**Criterion 2's mechanism is visible in the numbers**: H1-greedy's spread +is **0.00** at 3p+. A flat tax on every seat creates no variance at all, +which is the clearest statement of why scoping was the right correction. + +**Criterion 4 came out backwards, and the prediction held.** The workplan +said our panel might not be able to test it, because no policy here models +another seat or knows what a scope is. Bond claim rates are **lower** than +personal at 3p and 4p — driven by suit availability, not by incentive. +**Reported as untested with an incidental figure pointing the wrong way**, +not as a refutation. + ## Not in this workplan - **No stacking with H1.** The package forbids it explicitly.