From ff892885810ae5684abc4818994b140c969ffca0 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 8 Aug 2026 16:09:16 +0200 Subject: [PATCH] =?UTF-8?q?CB-WP-0042=20T05:=20H2=20measured=20=E2=80=94?= =?UTF-8?q?=20it=20largely=20succeeds=20where=20H1=20failed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured against ground-game's own §3 criteria, read from their design note rather than reused from H1. Criterion 1 met with room: greedy SHARED wins 120 at 3p and 175 at 4p, against H1's 0 and 0, restoring 73% and 92% of baseline. Criterion 3 met, and it was H1's clearest failure. Under H1 the unregulated seat armed DARVO constantly and never won; under H2 it arms and wins 13/13/57. "Non-zero for some policy that still sometimes wins" is exactly the shape H1 could not produce. Criterion 2 met at 2p/4p/6p and missed at 3p — 1.50 against baseline's 1.57 — reported as a miss because that is what this sample says. The mechanism is visible: H1-greedy's spread is 0.00 at 3p+, because a flat tax on every seat creates no variance at all. That is the clearest statement of why scoping was the right correction. Criterion 5 is the best evidence in the pass. Forcing every scope to global and changing nothing else reproduces H1's collapse exactly — 120 to 0 at 3p, 175 to 0 at 4p — so the scoping is what saves it, not any other difference between the packages. Criterion 4 came out backwards and the prediction held. The workplan said this panel might be unable to test it, because no policy here models another seat or knows what a scope is; bond claim rates are LOWER than personal at 3p and 4p, driven by suit availability rather than incentive. Reported as untested with an incidental figure pointing the wrong way, not as a refutation. Wired into make panels. First pass declared after ADR-0021, so no chaos roll is recorded. Co-Authored-By: Claude Opus 5 --- Makefile | 1 + evidence/CB-EV-0032-h2-measured.md | 123 +++++++++++ games/ground/examples/h2-panel.rs | 198 ++++++++++++++++++ .../CB-WP-0042-h2-scoped-problem-stress.md | 32 ++- 4 files changed, 352 insertions(+), 2 deletions(-) create mode 100644 evidence/CB-EV-0032-h2-measured.md create mode 100644 games/ground/examples/h2-panel.rs diff --git a/Makefile b/Makefile index 914d2a2..e97e91a 100644 --- a/Makefile +++ b/Makefile @@ -238,6 +238,7 @@ panels: @cargo run --release -q -p games-ground --example attack-value @cargo run --release -q -p games-ground --example regulation @cargo run --release -q -p games-ground --example perfect-recall + @cargo run --release -q -p games-ground --example h2-panel # CB-WP-0022 T05: the design-finding register, reported over # specs/GroundRules.md. Shows the QUEUE by default; the log of closed diff --git a/evidence/CB-EV-0032-h2-measured.md b/evidence/CB-EV-0032-h2-measured.md new file mode 100644 index 0000000..63ce1af --- /dev/null +++ b/evidence/CB-EV-0032-h2-measured.md @@ -0,0 +1,123 @@ +# CB-EV-0032 — H2 measured + +CB-WP-0042 T05. Tier M. Closed 2026-08-08. **The first pass declared after +the chaos roll was retired** (ADR-0021), so there is no roll to record. + +**Delivered:** H2 implemented from `rules_delta.yaml`, measured against +**ground-game's own §3 criteria** — read from their design note, not +reused from H1 — and a control that isolates the mechanism. + +--- + +## 1. Their criteria, answered + +| # | criterion | verdict | +|---|---|---| +| 1 | greedy SHARED 3–4p **well above H1's 0** | **met, strongly** | +| 2 | Stress variance above baseline *and* H1-greedy | **met at 2p/4p/6p, marginally missed at 3p** | +| 3 | DARVO non-zero for a policy that **still sometimes wins** | **met** — and this was H1's clearest failure | +| 4 | bond cards show elevated SOLVE rate vs personal | **not met, and this panel cannot test it** (§3) | +| 5 | control: force all scopes global → H1-like collapse | **met, decisively** | + +## 2. The numbers + +SHARED GROUND, 200 games per cell. `spread` is the mean max−min final +Stress across seats — their criterion 2. + +**greedy** + +| seats | baseline won / spread | H1 won / spread | H2 won / spread | +|---|---|---|---| +| 2p | 132 / 0.05 | 68 / 0.34 | **87 / 0.47** | +| 3p | 165 / 1.57 | **0** / 0.00 | **120** / 1.50 | +| 4p | 190 / 1.83 | **0** / 0.00 | **175** / 2.40 | +| 6p | 200 / 1.55 | **0** / 0.00 | **199** / 2.02 | + +**reactive** — DARVO arms in brackets + +| seats | H1 won (arms) | H2 won (arms) | +|---|---|---| +| 2p | 66 (363) | 61 (346) | +| 3p | **0** (600) | **13** (499) | +| 4p | **0** (800) | **13** (470) | +| 6p | **0** (1200) | **57** (565) | + +**Criterion 1 is met with room.** H2 restores 73% of baseline wins at 3p +and 92% at 4p, where H1 destroyed them entirely. + +**Criterion 3 is the one that matters most.** Under H1 the unregulated +seat armed DARVO constantly and **never won**; under H2 it arms and wins +13/13/57. *"Non-zero for some policy that still sometimes wins"* is +exactly the shape H1 could not produce. + +**Criterion 2, honestly.** H2's spread beats baseline at 2p, 4p and 6p and +**misses at 3p — 1.50 against 1.57**. It beats H1-greedy everywhere, and +**H1-greedy's spread is 0.00 at 3p+**: a flat tax on every seat creates no +variance at all, which is the clearest statement of why scoping was the +right correction. + +## 3. Criterion 4 — the number is lower, and the number cannot mean what +the criterion asks + +Claim rate by scope, H2, greedy: + +| seats | global | personal | bond | +|---|---|---|---| +| 2p | 70.5% | 54.5% | — (no bond card, as designed) | +| 3p | 85.5% | 80.2% | **60.0%** | +| 4p | 99.0% | 98.5% | **87.5%** | +| 6p | 100% | 95.3% | 99.5% | + +**Bond is lower than personal at 3p and 4p — the opposite direction.** + +**And this panel cannot test the hypothesis.** The claim is that a shared +tick makes a Bond network *jointly motivated* to clear that card. +**No policy here models another seat, or knows what a scope is.** Greedy +picks SOLVE on suit availability. So the bond rate is driven by which +cards happen to be matchable, not by incentive — the mechanism the +criterion is about could not have shown up. + +**This was stated in the workplan before the run**, not discovered in the +numbers. The honest report is: **criterion 4 is untested, and the +incidental figure points the wrong way.** A table can answer it; this +cannot. + +## 4. Criterion 5 — the control isolates the mechanism + +Force every scope to `global` and change nothing else: + +| seats | H2 scoped | H2, all scopes global | +|---|---:|---:| +| 3p | **120** | **0** | +| 4p | **175** | **0** | + +**H1's collapse returns exactly.** That is a strong causal statement and +the best evidence in this file: **the scoping is what saves it**, not any +other difference between the packages. Their optional A/B was worth +running. + +## 5. What this does not say + +- **No felt-play.** H2's central claim is about *motivation* — a bonded + pair caring about each other's card — and §3 says as much. 200-game + aggregates measure dynamics, not motivation + ([`Taxonomy.md`](../specs/Taxonomy.md) §4). +- **The panel is still greedy-family**, as CB-EV-0031 said of itself. + Criterion 3 asks for "some policy that still sometimes wins" and we have + two, both derived from the same ranking. +- **These are single-sample figures.** CB-REV-0003 #10 found the 2p arm + counts varying 354–366 across seed windows; nothing here has been + re-derived on a second sample, and the `won` columns should be read as + approximate. +- **No re-review.** CB-WP-0038's tier-L review found twelve fatal defects + across three rounds, all in the instrument. This pass reuses that + instrument. + +## Open after this pass + +- **Criterion 4 needs a policy that models other seats**, or a table. +- **A second seed window**, to separate the invariant figures from the + sample-specific ones as CB-EV-0031 §1 now does. +- **The 3p variance miss** (1.50 vs 1.57) is within what a second sample + might move; it is reported as a miss because that is what this sample + says. diff --git a/games/ground/examples/h2-panel.rs b/games/ground/examples/h2-panel.rs new file mode 100644 index 0000000..be6680b --- /dev/null +++ b/games/ground/examples/h2-panel.rs @@ -0,0 +1,198 @@ +//! **H2 measured against ground-game's own criteria** (CB-WP-0042 T05). +//! +//! Their §3 targets, not H1's reused from memory: +//! +//! 1. group success (greedy SHARED, 3–4p) well above H1's 0 +//! 2. Stress **variance** — max−min across seats — higher than baseline +//! and H1-greedy +//! 3. DARVO non-zero **for some policy that still sometimes wins** +//! 4. bond cards show an elevated SOLVE rate versus personal +//! 5. control: force every scope global and H1-like collapse returns +//! +//! **Criterion 4 is the one this instrument probably cannot answer**, and +//! that is said up front rather than discovered in the numbers: no policy +//! here models another seat, or knows what a scope is. If bond and +//! personal rates come out equal it is because nothing in the panel could +//! have made them differ — not because the incentive fails at a table. + +use std::collections::BTreeMap; + +use cb_game_runtime::{ScenarioGame, Setup}; +use cb_kernel::PlayerId; +use games_ground::bot::{play, Choice, GreedyPolicy, Policy}; +use games_ground::edition::StressScope; +use games_ground::{Action, GroundCommand, GroundState, ScoringMode, Variant}; + +const GAMES: u32 = 200; + +/// Greedy with GROUND demoted below ATTACK — the unregulated seat from +/// CB-EV-0031, delegating so exactly one preference differs. +struct Reactive; + +impl Policy for Reactive { + fn name(&self) -> &'static str { + "reactive" + } + fn choose( + &mut self, + state: &GroundState, + seat: PlayerId, + legal: &[GroundCommand], + _may_pass: bool, + ) -> Choice { + let gated = state + .players + .get(&seat) + .is_some_and(|p| p.stress >= 4 && !p.freedom_gate_lifted); + let rank = |c: &GroundCommand| -> i32 { + match c { + GroundCommand::SelectAction { + action: Action::Ground, + .. + } if gated => 5, + other => GreedyPolicy::rank(state, seat, other), + } + }; + let mut best = 0; + for (i, c) in legal.iter().enumerate() { + if rank(c) > rank(&legal[best]) { + best = i; + } + } + Choice::Command(best) + } +} + +#[derive(Default)] +struct Cell { + games: u32, + played: u32, + won: u32, + darvo: u32, + /// Summed max−min final Stress, for criterion 2. + spread: u32, + /// Claimed / total, by scope, for criterion 4. + claimed: BTreeMap<&'static str, (u32, u32)>, +} + +fn sweep(variant: Variant, players: u8, reactive: bool, force_global: bool) -> Cell { + let mut c = Cell::default(); + for seed in 0..GAMES as u64 { + let Ok(st) = GroundState::setup( + &Setup { + players, + preset: format!("standard-{players}p"), + patch: Default::default(), + }, + seed, + ) else { + continue; + }; + let mut st = st.with_variant(variant); + st.mode = ScoringMode::SharedGround; + // Criterion 5's control: every scope forced global. + if force_global { + for p in st.problems.values_mut() { + if p.scope.is_some() { + p.scope = Some(StressScope::Global); + p.owner = None; + } + } + } + let mut ps: Vec> = (0..players) + .map(|_| { + if reactive { + Box::new(Reactive) as Box + } else { + Box::new(GreedyPolicy) as Box + } + }) + .collect(); + let Ok(g) = play(st, &mut ps) else { continue }; + c.games += 1; + if g.state.outcome.is_some() && g.rounds == 5 { + c.played += 1; + } + if g.state.outcome.as_ref().is_some_and(|o| o.group_success) { + c.won += 1; + } + for e in &g.events { + if matches!(e, games_ground::GroundEvent::DarvoTriggered { .. }) { + c.darvo += 1; + } + } + let stresses: Vec = g.state.players.values().map(|p| p.stress).collect(); + let (lo, hi) = ( + stresses.iter().copied().min().unwrap_or(0), + stresses.iter().copied().max().unwrap_or(0), + ); + c.spread += u32::from(hi - lo); + for p in g.state.problems.values() { + let name = match p.scope { + Some(StressScope::Global) => "global", + Some(StressScope::Personal) => "personal", + Some(StressScope::Bond) => "bond", + None => continue, + }; + let e = c.claimed.entry(name).or_default(); + e.1 += 1; + if p.claimed_by.is_some() { + e.0 += 1; + } + } + } + assert_eq!(c.games, GAMES, "{players}p {variant:?}: short cell"); + assert_eq!(c.played, GAMES, "{players}p {variant:?}: games did not finish"); + c +} + +fn main() { + println!("H2 — against ground-game's §3 criteria (CB-WP-0042 T05)\n"); + println!("SHARED GROUND, {GAMES} games per cell. `spread` is the mean"); + println!("max−min final Stress across seats — their criterion 2.\n"); + + for (label, reactive) in [("greedy", false), ("reactive", true)] { + println!("{label}"); + println!("seats variant won darvo spread"); + for players in [2u8, 3, 4, 6] { + for (vname, v) in [ + ("baseline", Variant::Baseline), + ("H1 ", Variant::H1ProblemStress), + ("H2 ", Variant::H2ScopedProblemStress), + ] { + let c = sweep(v, players, reactive, false); + println!( + " {players}p {vname} {:>4} {:>5} {:>5.2}", + c.won, + c.darvo, + f64::from(c.spread) / f64::from(GAMES) + ); + } + } + println!(); + } + + println!("criterion 4 — claim rate by scope (H2, greedy)"); + println!("seats global personal bond"); + for players in [2u8, 3, 4, 6] { + let c = sweep(Variant::H2ScopedProblemStress, players, false, false); + let rate = |k: &str| match c.claimed.get(k) { + Some((a, b)) if *b > 0 => format!("{:>6.1}%", 100.0 * f64::from(*a) / f64::from(*b)), + _ => " —".to_string(), + }; + println!( + " {players}p {} {} {}", + rate("global"), + rate("personal"), + rate("bond") + ); + } + + println!("\ncriterion 5 — control: every scope forced global (H2, greedy)"); + println!("seats won (scoped) won (all global)"); + for players in [3u8, 4] { + let scoped = sweep(Variant::H2ScopedProblemStress, players, false, false); + let global = sweep(Variant::H2ScopedProblemStress, players, false, true); + println!(" {players}p {:>4} {:>4}", scoped.won, global.won); + } +} diff --git a/workplans/CB-WP-0042-h2-scoped-problem-stress.md b/workplans/CB-WP-0042-h2-scoped-problem-stress.md index dc0e053..bebebaf 100644 --- a/workplans/CB-WP-0042-h2-scoped-problem-stress.md +++ b/workplans/CB-WP-0042-h2-scoped-problem-stress.md @@ -2,7 +2,7 @@ id: CB-WP-0042 kind: product title: "H2 — scoped problem stress" -status: active +status: done state_hub_workstream_id: "8f11d55b-3452-49f3-9857-d0bf06752f68" --- @@ -176,7 +176,7 @@ The test must fail if it does. ```task id: CB-WP-0042-T05 -status: todo +status: done priority: high state_hub_task_id: "0bcd97da-60b8-46b2-ae86-7a94f6af8dd5" ``` @@ -196,6 +196,34 @@ Same instrument as H1, same panel, both variants in one run. - **`make panels` runs it**, or the figures come from an ungated binary again (CB-REV-0002 #7). +**Done 2026-08-08.** +[CB-EV-0032](../evidence/CB-EV-0032-h2-measured.md). **H2 largely +succeeds where H1 failed.** + +| # | their criterion | verdict | +|---|---|---| +| 1 | greedy SHARED 3–4p above H1's 0 | **met** — 120 and 175 against H1's 0 and 0 | +| 2 | variance above baseline and H1-greedy | **met at 2p/4p/6p, missed at 3p** (1.50 vs 1.57) | +| 3 | DARVO for a policy that still sometimes wins | **met** — H1's clearest failure, now 13/13/57 wins with arms | +| 4 | bond SOLVE rate above personal | **not met, and untestable here** | +| 5 | control: all scopes global → collapse | **met decisively** — 120→0 and 175→0 | + +**Criterion 5 is the best evidence in the pass.** Forcing every scope to +`global` and changing nothing else reproduces H1's collapse exactly, which +says the **scoping** is what saves it rather than any other difference +between the packages. + +**Criterion 2's mechanism is visible in the numbers**: H1-greedy's spread +is **0.00** at 3p+. A flat tax on every seat creates no variance at all, +which is the clearest statement of why scoping was the right correction. + +**Criterion 4 came out backwards, and the prediction held.** The workplan +said our panel might not be able to test it, because no policy here models +another seat or knows what a scope is. Bond claim rates are **lower** than +personal at 3p and 4p — driven by suit availability, not by incentive. +**Reported as untested with an incidental figure pointing the wrong way**, +not as a refutation. + ## Not in this workplan - **No stacking with H1.** The package forbids it explicitly.