CB-WP-0050: every configuration, faceted by aspect — and F31
Some checks failed
ci / check (push) Failing after 4s
Some checks failed
ci / check (push) Failing after 4s
module-panel sweeps points in aspect space the way scenario-panel sweeps boards: one module at a time with every other aspect held, then the profiles where an interaction is the hypothesis. F31: attack_relief.self_soothe_ge4 HAS NEVER FIRED. Zero ATTACKs in every cell, every seat band, both bots. The module acts only on an uncancelled ATTACK by a seat at Stress >= 4, and greedy ranks Ground 100 at the gate against Attack's 10 -- so at exactly the position where self-soothe would pay, GROUND wins. ModuleAwarePolicy adds 55 and that is still not enough. `unplayed`, not `inert`: the kernel implements it correctly and nothing has ever given it an opportunity. This reaches backwards: CB-EV-0030's H1 verdict rests on its flat-pressure half alone, because H1's other half never had an opportunity in those games either. Also: problem_stress.scoped is the only module with a measured effect (3p 85->60 greedy, 86->63 module-aware, peak Stress 2->4); flat_any_open drives group success to 0 at every band, reproducing the H1 rejection from the module side; and scoped_plus_attack_soothe is EXACTLY scoped alone -- necessarily, given F31 -- so the catalog's first intentional multi-aspect combination cannot currently be evaluated as a combination. THE PANEL'S OWN DEFECT, first run: it printed 85/86 for a module that never fired -- real-looking numbers inviting "measured, no effect" when no seat ever created the precondition. Its docstring already said it would not do that; the claim was written before the behaviour was. First fix marked whole rows unmeasured, which threw away h1's real flat-pressure result; the shipped fix names the specific module and keeps the row's numbers, which are real for the modules that did fire. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
3d4818b492
commit
3f2ba2ec5d
4 changed files with 425 additions and 0 deletions
1
Makefile
1
Makefile
|
|
@ -260,6 +260,7 @@ panels:
|
|||
@cargo run --release -q -p games-ground --example perfect-recall
|
||||
@cargo run --release -q -p games-ground --example h2-panel
|
||||
@cargo run --release -q -p games-ground --example scenario-panel
|
||||
@cargo run --release -q -p games-ground --example module-panel
|
||||
|
||||
# CB-WP-0022 T05: the design-finding register, reported over
|
||||
# specs/GroundRules.md. Shows the QUEUE by default; the log of closed
|
||||
|
|
|
|||
297
games/ground/examples/module-panel.rs
Normal file
297
games/ground/examples/module-panel.rs
Normal file
|
|
@ -0,0 +1,297 @@
|
|||
//! Every configuration the catalog offers, faceted by aspect
|
||||
//! (CB-WP-0050, CB-RES-0010 §5.4).
|
||||
//!
|
||||
//! `scenario-panel` sweeps boards. This sweeps **points in aspect
|
||||
//! space**: each aspect held at its default while one module varies, then
|
||||
//! the profiles where the hypothesis is an interaction.
|
||||
//!
|
||||
//! ## What it will not do
|
||||
//!
|
||||
//! **It does not omit what it cannot run.** A module with no kernel path
|
||||
//! is printed as `no kernel path` rather than left out — a panel whose
|
||||
//! absences are invisible is how `end_condition` and `problem_deal` would
|
||||
//! quietly stop being questions. CB-RES-0010 §5.6.
|
||||
//!
|
||||
//! **It does not report a module as failed when nothing could have
|
||||
//! tested it.** Both policies are named per row, and the honest reading
|
||||
//! of a flat row under `greedy` is that greedy cannot see the module —
|
||||
//! F27's shape, and the reason `ModuleAwarePolicy` exists.
|
||||
|
||||
use cb_game_runtime::{ScenarioGame, Setup};
|
||||
use cb_kernel::PlayerId;
|
||||
use games_ground::bot::{play, GreedyPolicy, ModuleAwarePolicy, Policy};
|
||||
use games_ground::config::Configuration;
|
||||
use games_ground::{GroundState, ScoringMode};
|
||||
|
||||
/// Games per cell. Named once so the banner and the assertion cannot
|
||||
/// disagree (CB-REV-0002 #1).
|
||||
const GAMES: u32 = 100;
|
||||
|
||||
#[derive(Default)]
|
||||
struct Cell {
|
||||
games: u32,
|
||||
played: u32,
|
||||
won: u32,
|
||||
/// Peak Stress **held**, not assigned — a table that sat at 2 all
|
||||
/// game reported 1 when this was a max over event payloads
|
||||
/// (CB-REV-0003 #5).
|
||||
peak_stress: u8,
|
||||
darvo: u32,
|
||||
attacks: u32,
|
||||
setup_fails: u32,
|
||||
/// Games in which **every active module actually acted**.
|
||||
///
|
||||
/// A module that never fires has not been measured, and printing a
|
||||
/// group-success number for it is worse than printing nothing: 85 vs
|
||||
/// 86 reads as "measured, no effect" when the truth is that no seat
|
||||
/// ever created the precondition. `attack_relief.self_soothe_ge4`
|
||||
/// showed exactly this on the panel's first run — zero ATTACKs in
|
||||
/// every cell, because GROUND outranks ATTACK at the stress gate, so
|
||||
/// the module could not fire and its row looked like a null result.
|
||||
unfired: std::collections::BTreeMap<String, u32>,
|
||||
}
|
||||
|
||||
/// Did every non-default module in `cfg` actually act in this game?
|
||||
///
|
||||
/// Matched per module rather than inferred, for the same reason `Rules`
|
||||
/// is exhaustive: a module added later must be a compile error here, not
|
||||
/// a silent "fired" that turns an untested row into a measured one.
|
||||
fn unfired_modules(cfg: &Configuration, g: &games_ground::bot::BotGame) -> Vec<String> {
|
||||
cfg.active_modules()
|
||||
.into_iter()
|
||||
.filter(|m| !fired(m, g))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Did this one module act in this game?
|
||||
///
|
||||
/// **Per module, not inferred.** Same reason `Rules` is exhaustive: a
|
||||
/// module added later must be a compile error here rather than a silent
|
||||
/// "fired" that turns an untested row into a measured one.
|
||||
fn fired(m: &str, g: &games_ground::bot::BotGame) -> bool {
|
||||
match m {
|
||||
// Its Stress tick happens at Round End; the ledger moves.
|
||||
"problem_stress.flat_any_open" | "problem_stress.scoped" => g
|
||||
.events
|
||||
.iter()
|
||||
.any(|e| matches!(e, games_ground::GroundEvent::StressSet { .. })),
|
||||
// It only acts on an uncancelled ATTACK by a seat at Stress >= 4.
|
||||
"attack_relief.self_soothe_ge4" => g.steps.iter().any(|(_, c)| {
|
||||
matches!(
|
||||
c,
|
||||
games_ground::GroundCommand::SelectAction {
|
||||
action: games_ground::Action::Attack,
|
||||
..
|
||||
}
|
||||
)
|
||||
}),
|
||||
// Anything else has no kernel path, so the row never gets here.
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, PartialEq)]
|
||||
enum Seat {
|
||||
Greedy,
|
||||
ModuleAware,
|
||||
}
|
||||
|
||||
fn sweep(cfg: &Configuration, players: u8, who: Seat) -> Cell {
|
||||
let mut c = Cell::default();
|
||||
for seed in 0..GAMES as u64 {
|
||||
let Ok(st) = GroundState::setup(
|
||||
&Setup {
|
||||
players,
|
||||
preset: format!("standard-{players}p"),
|
||||
patch: Default::default(),
|
||||
},
|
||||
seed,
|
||||
) else {
|
||||
c.setup_fails += 1;
|
||||
continue;
|
||||
};
|
||||
// `with_config`, never a field write: problem_stress.scoped
|
||||
// assigns owners at setup and a bare write leaves the module
|
||||
// inert (ADR-0022 D3).
|
||||
let mut st = st.with_config(cfg.clone());
|
||||
st.mode = ScoringMode::SharedGround;
|
||||
let start = st.players.values().map(|p| p.stress).max().unwrap_or(0);
|
||||
let mut ps: Vec<Box<dyn Policy>> = (0..players)
|
||||
.map(|_| match who {
|
||||
Seat::Greedy => Box::new(GreedyPolicy) as Box<dyn Policy>,
|
||||
Seat::ModuleAware => Box::new(ModuleAwarePolicy) as Box<dyn Policy>,
|
||||
})
|
||||
.collect();
|
||||
let g = match play(st, &mut ps) {
|
||||
Ok(g) => g,
|
||||
Err(e) => {
|
||||
eprintln!(" !! {players}p seed {seed}: {e:?}");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
c.games += 1;
|
||||
if g.state.outcome.is_some() && g.rounds == 5 {
|
||||
c.played += 1;
|
||||
}
|
||||
if g.state.outcome.as_ref().is_some_and(|o| o.group_success) {
|
||||
c.won += 1;
|
||||
}
|
||||
let mut held: std::collections::BTreeMap<PlayerId, u8> =
|
||||
g.state.players.keys().map(|s| (*s, start)).collect();
|
||||
c.peak_stress = c.peak_stress.max(start);
|
||||
for e in &g.events {
|
||||
match e {
|
||||
games_ground::GroundEvent::StressSet { player, stress } => {
|
||||
held.insert(*player, *stress);
|
||||
c.peak_stress = c.peak_stress.max(held.values().copied().max().unwrap_or(0));
|
||||
}
|
||||
games_ground::GroundEvent::DarvoTriggered { .. } => c.darvo += 1,
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
for m in unfired_modules(cfg, &g) {
|
||||
*c.unfired.entry(m).or_insert(0) += 1;
|
||||
}
|
||||
for (_, cmd) in &g.steps {
|
||||
if let games_ground::GroundCommand::SelectAction {
|
||||
action: games_ground::Action::Attack,
|
||||
..
|
||||
} = cmd
|
||||
{
|
||||
c.attacks += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(
|
||||
c.games, GAMES,
|
||||
"{players}p: only {} of {GAMES} games ran ({} setups refused)",
|
||||
c.games, c.setup_fails
|
||||
);
|
||||
assert_eq!(
|
||||
c.played, GAMES,
|
||||
"{players}p: {} of {} games reached an outcome over five rounds",
|
||||
c.played, c.games
|
||||
);
|
||||
c
|
||||
}
|
||||
|
||||
/// Every point worth measuring, in the order a reader should read them.
|
||||
fn points() -> Vec<(String, String, Result<Configuration, String>)> {
|
||||
let cat = games_ground::catalog::catalog().expect("catalog.yaml");
|
||||
let mut out = vec![(
|
||||
"—".to_string(),
|
||||
"baseline (every aspect at its default)".to_string(),
|
||||
Configuration::select("ground-darvo-r0"),
|
||||
)];
|
||||
|
||||
// ONE MODULE AT A TIME, grouped by aspect: "hold all aspects fixed,
|
||||
// vary one" is what makes a row a measurement of that module rather
|
||||
// than of a bundle (CB-RES-0010 §5.4).
|
||||
for aspect in &cat.aspects {
|
||||
for m in cat.modules.iter().filter(|m| m.aspect == aspect.id) {
|
||||
if m.is_default || m.module_id == aspect.default_module {
|
||||
continue;
|
||||
}
|
||||
out.push((
|
||||
aspect.id.clone(),
|
||||
m.module_id.clone(),
|
||||
Configuration::from_modules(
|
||||
&cat.default_baseline,
|
||||
std::slice::from_ref(&m.module_id),
|
||||
),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
// Then the profiles, which are where an INTERACTION is the
|
||||
// hypothesis. A single-module profile is skipped: it is the row above.
|
||||
for p in &cat.profiles {
|
||||
if p.modules.len() < 2 {
|
||||
continue;
|
||||
}
|
||||
out.push((
|
||||
"(interaction)".to_string(),
|
||||
p.profile_id.clone(),
|
||||
Configuration::from_profile(&p.profile_id),
|
||||
));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn main() {
|
||||
println!("CB-WP-0050 — every configuration, faceted by aspect\n");
|
||||
println!("SHARED GROUND; {GAMES} games per cell; seeds 0..{GAMES}.");
|
||||
println!("One module at a time first, holding every other aspect at its");
|
||||
println!("default; then the profiles, where an interaction is the point.\n");
|
||||
println!("`grdy` is GreedyPolicy, which reads no module at all. `mod` is");
|
||||
println!("ModuleAwarePolicy. A row flat under `grdy` and moving under");
|
||||
println!("`mod` is a module that needs a seat able to see it — not a");
|
||||
println!("module that does nothing (F27's distinction).\n");
|
||||
println!("`never fired: UNMEASURED` means no seat created that module's");
|
||||
println!("precondition in any game, so its contribution to the row is not");
|
||||
println!("a null result — it is an absence of evidence. The other numbers");
|
||||
println!("in such a row are still real for the modules that DID fire.\n");
|
||||
|
||||
for players in [3u8, 4, 6] {
|
||||
println!("{players} players");
|
||||
println!(
|
||||
" {:<18} {:<38} {:>5} {:>5} {:>6} {:>6} {:>6}",
|
||||
"aspect", "module / profile", "grdy", "mod", "peak", "darvo", "atk"
|
||||
);
|
||||
for (aspect, name, cfg) in points() {
|
||||
let cfg = match cfg {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
println!(" {aspect:<18} {name:<38} — — {e}");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
// **Named, never omitted.** A module the kernel cannot run is
|
||||
// a question that stays open, and a blank row is how it would
|
||||
// stop being one.
|
||||
if let Err(e) = cfg.resolve() {
|
||||
let why = e.split(" (catalog status").next().unwrap_or(&e).to_string();
|
||||
let why = why.replace(&format!("{name} is a known module with "), "");
|
||||
println!(" {aspect:<18} {name:<38} {:>5} {:>5} {why}", "—", "—");
|
||||
continue;
|
||||
}
|
||||
let g = sweep(&cfg, players, Seat::Greedy);
|
||||
let m = sweep(&cfg, players, Seat::ModuleAware);
|
||||
// **A module that never acted is UNMEASURED, not null.**
|
||||
// Printing win counts for it invites the reading that it does
|
||||
// nothing, when what happened is that no seat ever created
|
||||
// the precondition. CB-RES-0010 §5.4's last bullet, one level
|
||||
// deeper than F27: not "the bot cannot see the module" but
|
||||
// "the bot never gives the module anything to see".
|
||||
// **Name the module that never acted, not the whole row.**
|
||||
// Reporting a two-module profile as wholly unmeasured throws
|
||||
// away the half that did fire: h1's flat pressure reaches
|
||||
// every game and drives group success to 0, which is a real
|
||||
// result about that module even though its ATTACK half never
|
||||
// gets an opportunity.
|
||||
let never: Vec<&String> = m
|
||||
.unfired
|
||||
.iter()
|
||||
.filter(|(_, n)| **n == GAMES)
|
||||
.map(|(k, _)| k)
|
||||
.collect();
|
||||
let note = if never.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
format!(
|
||||
" \u{2190} {} never fired: UNMEASURED",
|
||||
never
|
||||
.iter()
|
||||
.map(|s| s.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
)
|
||||
};
|
||||
println!(
|
||||
" {aspect:<18} {name:<38} {:>5} {:>5} {:>6} {:>6} {:>6}{note}",
|
||||
g.won, m.won, m.peak_stress, m.darvo, m.attacks
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
}
|
||||
|
|
@ -56,6 +56,7 @@ kinds, states and metrics: [`GameDesign.md`](GameDesign.md). Reported by
|
|||
| F26 | inert | ruled | `crates/cb-render-html/src/lib.rs::a_scoped_table_says_what_a_scope_does` | counterexample | 2026-08-08 | ground-game |
|
||||
| F27 | unplayed | reported | `games/ground/examples/scenario-panel.rs` | counterexample | 2026-08-08 | clay-borg |
|
||||
| F29 | inconsistent | applied | `games_ground::edition::card_text_tests::which_scenarios_are_mechanically_distinct` | counterexample | 2026-08-08 | ground-game |
|
||||
| F31 | unplayed | raised | `games/ground/examples/module-panel.rs` | counterexample | 2026-08-08 | clay-borg |
|
||||
| F30 | degenerate | ruled | `games/ground/examples/scenario-panel.rs` | counterexample | 2026-08-08 | ground-game |
|
||||
| F28 | underdetermined | applied | `games_ground::tests::the_mastery_rating_and_the_shared_score_count_different_things` | counterexample | 2026-08-08 | ground-game |
|
||||
|
||||
|
|
@ -66,6 +67,34 @@ Prose for **closed** findings (withdrawn or applied) lives in
|
|||
are the whole register and `make design` reads them from here.
|
||||
|
||||
|
||||
- **F31 — `attack_relief.self_soothe_ge4` cannot fire, so it has never
|
||||
been measured.** `module-panel` records **zero ATTACKs in every cell**,
|
||||
at every seat band, under both a mode-blind and a module-aware bot. The
|
||||
module only acts on an uncancelled ATTACK by a seat at Stress ≥ 4, so
|
||||
it has had no opportunity in any game this repo has run — including
|
||||
every game behind CB-EV-0030's H1 measurement, where it was half the
|
||||
package.
|
||||
|
||||
The mechanism is legible: `GreedyPolicy` ranks `Ground` at 100 when the
|
||||
stress gate bites and `Attack` at 10, so at exactly the position where
|
||||
self-soothe would pay, GROUND wins the comparison.
|
||||
`ModuleAwarePolicy` raises ATTACK by 55 at the gate and **that is still
|
||||
not enough**, which is a fact about the ranking rather than about the
|
||||
game.
|
||||
|
||||
**`unplayed`, emphatically not `inert`.** The kernel implements the
|
||||
module correctly; nothing has ever given it an opportunity. Reporting
|
||||
it as a null result is the error the panel now refuses to make: its
|
||||
row shows real numbers with `← never fired: UNMEASURED` attached,
|
||||
because the numbers belong to the *other* modules in the row.
|
||||
|
||||
**Sensitivity:** vary only the ATTACK rank. The falsifier is a policy
|
||||
that attacks at the gate — if group success then moves under
|
||||
`attack_relief.self_soothe_ge4` and not under the baseline, the module
|
||||
is real and only our seats were hiding it. Until such a seat exists,
|
||||
**no claim about this module is supported in either direction**, and
|
||||
CB-EV-0030's H1 verdict rests on its flat-pressure half alone.
|
||||
|
||||
- **F30 — SCN_04 is materially harder at 2 players.** 52/100 group
|
||||
success against 67 and 73, with **every other parameter held by the
|
||||
edition itself**: same deal shape, same 6 available points, same
|
||||
|
|
|
|||
|
|
@ -0,0 +1,98 @@
|
|||
---
|
||||
id: CB-WP-0050
|
||||
kind: product
|
||||
title: "Every configuration, faceted by aspect"
|
||||
status: done
|
||||
---
|
||||
|
||||
# Purpose
|
||||
|
||||
```
|
||||
structural tier S (a new panel; no rule, no state, no artifact
|
||||
contract moved)
|
||||
declared tier S
|
||||
```
|
||||
|
||||
CB-RES-0010 §5.4: *"hold all aspects fixed, vary one."* The selector
|
||||
(CB-WP-0048) made configurations selectable and the policies (CB-WP-0049)
|
||||
made them judgeable. This measures them.
|
||||
|
||||
## Task: sweep the points, and say what could not be measured
|
||||
|
||||
```task
|
||||
id: CB-WP-0050-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
**Controls:**
|
||||
- **one module at a time first**, every other aspect held at its default,
|
||||
so a row measures a module rather than a bundle;
|
||||
- **profiles after**, where an interaction is the hypothesis — and a
|
||||
single-module profile is skipped, because it is the row above;
|
||||
- **nothing is omitted**: a module with no kernel path prints as such;
|
||||
- **a module that never acted is reported UNMEASURED**, by name.
|
||||
|
||||
**Done 2026-08-08.** `make panels` gained `module-panel`.
|
||||
|
||||
## What it found
|
||||
|
||||
**1. `attack_relief.self_soothe_ge4` has never fired. Not once.**
|
||||
|
||||
Zero ATTACKs in every cell, at every seat band, under both bots. The
|
||||
module acts only on an uncancelled ATTACK by a seat at Stress ≥ 4, and
|
||||
`GreedyPolicy` ranks `Ground` at 100 when the gate bites against
|
||||
`Attack`'s 10 — so at exactly the position where self-soothe would pay,
|
||||
GROUND wins. `ModuleAwarePolicy` adds 55 and **that is still not
|
||||
enough**.
|
||||
|
||||
Filed as **F31, `unplayed`**. This also reaches backwards:
|
||||
**CB-EV-0030's H1 verdict rests on its flat-pressure half alone**, since
|
||||
H1's other half never had an opportunity in any of those games either.
|
||||
|
||||
**2. `problem_stress.scoped` is the only module with a measured effect.**
|
||||
3p group success 85 → 60 (greedy) and 86 → 63 (module-aware); peak Stress
|
||||
2 → 4. `problem_stress.flat_any_open` drives group success to **0** at
|
||||
every seat band, which is the H1 rejection reproduced from the module
|
||||
side rather than the package side.
|
||||
|
||||
**3. `scoped_plus_attack_soothe` is exactly `problem_stress.scoped`.**
|
||||
Identical in every cell — necessarily, given (1). The first intentional
|
||||
multi-aspect combination in the catalog **cannot currently be evaluated
|
||||
as a combination**, and the panel says so on its own row rather than
|
||||
letting the equality read as "no interaction".
|
||||
|
||||
## The defect the panel had, on its first run
|
||||
|
||||
It printed `85 / 86` for `attack_relief.self_soothe_ge4` — real-looking
|
||||
numbers that invite the reading *"measured, no effect"*, when the truth
|
||||
is that no seat ever created the precondition. **Its own docstring said
|
||||
it would not do this**, which is the same shape as CB-WP-0046's wrong
|
||||
reason for an open item: the claim was written before the behaviour was.
|
||||
|
||||
Two iterations to get the reporting honest:
|
||||
|
||||
- first, whole rows marked `never fired — unmeasured` — which **threw away
|
||||
a real result**, because `h1`'s flat-pressure half fires in every game
|
||||
and drives group success to 0;
|
||||
- then, the **specific module** named, with the row's numbers kept:
|
||||
`0 0 4 0 0 ← attack_relief.self_soothe_ge4 never fired: UNMEASURED`.
|
||||
|
||||
The numbers in such a row are real for the modules that did fire. Saying
|
||||
which one did not is what stops the row being read as a verdict on all of
|
||||
them.
|
||||
|
||||
## Not done here
|
||||
|
||||
- **The panel measures SHARED GROUND only.** Mode × module is a
|
||||
three-dimensional sweep and nothing yet says the modules behave the
|
||||
same under the competitive modes — where, per CB-EV-0033, *who wins*
|
||||
moves even when group success does not.
|
||||
- **F31 has no falsifier running.** A seat that attacks at the gate would
|
||||
settle whether `attack_relief.self_soothe_ge4` is real or merely
|
||||
unreachable; until one exists, no claim about that module is supported
|
||||
in either direction.
|
||||
- **The scope term is inert at round-one positions** (CB-WP-0049 T04), so
|
||||
`problem_stress.scoped`'s measured effect is driven by later rounds.
|
||||
The panel does not report per-round effect, and that is where the
|
||||
module actually lives.
|
||||
Loading…
Add table
Add a link
Reference in a new issue