Some checks failed
ci / check (push) Failing after 3s
The pattern is now measured over three rounds: 5 fatal, then 3 (2 from the previous round's corrections), then 4 (3 from them). The corrections are not getting safer. FATAL 1: round 2's short-cell assertion went into regulation.rs only. attack-value.rs — which produced every number in CB-EV-0030's DARVO table — still just warned, and the gate registered to close the finding claimed the property for both. FATAL 2, the sharpest of the three rounds: counting games proves they STARTED. Stopping the engine after one round gives 200 games, all-zero columns and exit 0 — byte for byte the signature CB-EV-0030 says the instrumentation distinguishes from a real result. Both harnesses now require every counted game to have reached an outcome over five rounds. FATAL 3: round 2's `.csv` filter was applied to all three loops, so catalog.yaml and rules_delta.yaml — whose missing digests were round 1's finding — were recorded and then never compared, and never checked against upstream at all. Only the parser loop filters now. FATAL 4: five of six tiebreak comparators had no coverage. GR-E04's tiebreak never executes in any scenario. All four are now covered and mutation-verified; the Blame key needed compensating claims to be reachable at all, since Blame also lowers the coalition score. SERIOUS: "peak held" computed the same number as "peak assigned" for every possible input — the real gap was that START_STRESS was an unchecked constant, now read off the dealt state; cadence="none" was a pure loophole, removed; sibling discovery swapped a hand-written list for hand-written globs and missed metadata.json and VARIANT.md, both named in the package's own changed_files — now walked, and it found them immediately; and "~72,000 games" was unsourced, make panels runs 17,600. Also separated two kinds of number that were presented alike: seats×games is invariant, 363 and 29 vary 7.1%-11.5% across samples. Round 4 owed. The conclusion is not that the work is nearly right — it is that author-made corrections to measurement work should be assumed defective until a fresh reader has attacked them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
283 lines
11 KiB
Rust
283 lines
11 KiB
Rust
//! Does H1 do nothing, or does it do nothing **to a seat that already
|
||
//! manages its Stress**? (CB-WP-0039)
|
||
//!
|
||
//! [`CB-EV-0030`] measured ground-game's H1 with a panel that was
|
||
//! greedy-family throughout, and found DARVO never arms. But
|
||
//! `GreedyPolicy` ranks `Action::Ground if gated => 100`: the moment the
|
||
//! stress gate bites it grounds, Stress plateaus at 3, and the arm at 5
|
||
//! is unreachable **by construction**.
|
||
//!
|
||
//! H1 was written for the seat that does *not* do that. So this varies
|
||
//! **exactly one preference** — GROUND demoted below ATTACK — and holds
|
||
//! every other ranking identical to greedy's.
|
||
//!
|
||
//! [`CB-EV-0030`]: ../../../evidence/CB-EV-0030-h1-measured.md
|
||
|
||
use cb_game_runtime::{ScenarioGame, Setup};
|
||
use cb_kernel::PlayerId;
|
||
use games_ground::bot::{play, Choice, GreedyPolicy, Policy};
|
||
use games_ground::{Action, GroundCommand, GroundState, ScoringMode, Variant};
|
||
|
||
/// Greedy, with GROUND demoted below ATTACK. Nothing else differs.
|
||
///
|
||
/// **It delegates.** An earlier version re-typed an abridged copy of
|
||
/// greedy's ranking and called changing one line of the copy "one
|
||
/// preference"; it differed in five, and the seat burned its Freedom
|
||
/// token in round one of every game. This comment used to argue for the
|
||
/// copy, which gave a future maintainer written cover to restore it
|
||
/// (CB-REV-0002 #11).
|
||
struct Reactive;
|
||
|
||
impl Policy for Reactive {
|
||
fn name(&self) -> &'static str {
|
||
"reactive"
|
||
}
|
||
fn choose(
|
||
&mut self,
|
||
state: &GroundState,
|
||
seat: PlayerId,
|
||
legal: &[GroundCommand],
|
||
_may_pass: bool,
|
||
) -> Choice {
|
||
let gated = state
|
||
.players
|
||
.get(&seat)
|
||
.is_some_and(|p| p.stress >= 4 && !p.freedom_gate_lifted);
|
||
// ONE ARM OVERRIDDEN, THE REST DELEGATED.
|
||
//
|
||
// The first version re-typed greedy's ranking and changed one
|
||
// line of the copy. It differed in five places, and the review
|
||
// found it: `SpendFreedom` ranked 95 unconditionally instead of
|
||
// greedy's `95 if gated else 0`, so this seat spent its Freedom
|
||
// token in round one of every game — a second change to the very
|
||
// mechanism under study. Delegating makes "exactly one preference
|
||
// differs" structurally true instead of a claim in a comment.
|
||
let rank = |c: &GroundCommand| -> i32 {
|
||
match c {
|
||
GroundCommand::SelectAction {
|
||
action: Action::Ground,
|
||
..
|
||
} if gated => 5,
|
||
other => GreedyPolicy::rank(state, seat, other),
|
||
}
|
||
};
|
||
let mut best = 0;
|
||
for (i, c) in legal.iter().enumerate() {
|
||
if rank(c) > rank(&legal[best]) {
|
||
best = i;
|
||
}
|
||
}
|
||
Choice::Command(best)
|
||
}
|
||
}
|
||
|
||
/// The starting Stress, **read off the dealt state** rather than declared.
|
||
///
|
||
/// It was a `const 2` "named so the peak metric cannot silently disagree
|
||
/// with setup" — and nothing compared it to setup, so raising it to 7
|
||
/// printed `peak 7` in every cell with no warning (CB-REV-0003 #6).
|
||
/// Naming a constant is not checking it.
|
||
fn start_stress(state: &GroundState) -> u8 {
|
||
state.players.values().map(|p| p.stress).max().unwrap_or(0)
|
||
}
|
||
|
||
/// Games per cell. Named once so the assertion and the banner cannot
|
||
/// disagree — the banner said "200 games per cell" over 196-game columns.
|
||
const GAMES: u32 = 200;
|
||
|
||
struct Cell {
|
||
games: u32,
|
||
won: u32,
|
||
atk: u32,
|
||
darvo: u32,
|
||
peak_stress: u8,
|
||
/// Setups the engine refused. **Counted, not skipped** (CB-REV-0001
|
||
/// #11): CB-EV-0030 §3 credited this pass with fixing silent skips
|
||
/// and only the `play` path was instrumented.
|
||
setup_fails: u32,
|
||
/// DARVO arms at the End of Round 5. `GameEnded` is pushed
|
||
/// immediately after, so the sequence never advances a stage — the
|
||
/// arm is real and can do nothing. Reported separately, because
|
||
/// ground-game's criterion 1 is asking about DARVO *mattering*.
|
||
inert_arms: u32,
|
||
/// Games that reached an outcome over the full five rounds.
|
||
///
|
||
/// **`games` counts `play` returning `Ok`, which is not the claim.**
|
||
/// Stopping the engine after one round produced 200 games, all-zero
|
||
/// columns and exit 0 — the exact signature CB-EV-0030 §3 says the
|
||
/// instrumentation distinguishes from a real result.
|
||
played: u32,
|
||
}
|
||
|
||
fn sweep(mode: ScoringMode, variant: Variant, players: u8, reactive: bool) -> Cell {
|
||
let mut c = Cell {
|
||
games: 0,
|
||
won: 0,
|
||
atk: 0,
|
||
darvo: 0,
|
||
peak_stress: 0,
|
||
setup_fails: 0,
|
||
inert_arms: 0,
|
||
played: 0,
|
||
};
|
||
for seed in 0..GAMES as u64 {
|
||
let Ok(mut st) = GroundState::setup(
|
||
&Setup {
|
||
players,
|
||
preset: format!("standard-{players}p"),
|
||
patch: Default::default(),
|
||
},
|
||
seed,
|
||
) else {
|
||
c.setup_fails += 1;
|
||
continue;
|
||
};
|
||
st.mode = mode;
|
||
st.variant = variant;
|
||
// The dealt state, kept so the peak metric can read the real
|
||
// starting Stress rather than trust a constant.
|
||
let started = st.clone();
|
||
let mut ps: Vec<Box<dyn Policy>> = (0..players)
|
||
.map(|_| {
|
||
if reactive {
|
||
Box::new(Reactive) as Box<dyn Policy>
|
||
} else {
|
||
Box::new(GreedyPolicy) as Box<dyn Policy>
|
||
}
|
||
})
|
||
.collect();
|
||
// A refused game is not a lost game (CB-EV-0030 §3).
|
||
let g = match play(st, &mut ps) {
|
||
Ok(g) => g,
|
||
Err(e) => {
|
||
eprintln!(" !! {players}p seed {seed}: {e:?}");
|
||
continue;
|
||
}
|
||
};
|
||
c.games += 1;
|
||
// A game that RAN is not a game that was PLAYED (CB-REV-0003 #2).
|
||
if g.state.outcome.is_some() && g.rounds == 5 {
|
||
c.played += 1;
|
||
}
|
||
if g.state.outcome.as_ref().is_some_and(|o| o.group_success) {
|
||
c.won += 1;
|
||
}
|
||
for (_, cmd) in &g.steps {
|
||
if let GroundCommand::SelectAction {
|
||
action: Action::Attack,
|
||
..
|
||
} = cmd
|
||
{
|
||
c.atk += 1;
|
||
}
|
||
}
|
||
// Peak Stress **held**.
|
||
//
|
||
// The original took a maximum over `StressSet` PAYLOADS, and the
|
||
// starting value is written by `setup` and never by an event — so
|
||
// a table that sat at 2 all game reported 1.
|
||
//
|
||
// **The first correction did not change the number it computes**
|
||
// (CB-REV-0003 #5): `held`'s values are exactly
|
||
// `{start} ∪ {payloads}`, so with the floor applied
|
||
// `peak_held ≡ max(start, peak_payload)` for every possible input,
|
||
// and the comment claiming a change of subject was false of the
|
||
// new code too. The real fix is that `start` is now read off the
|
||
// dealt state instead of asserted by a constant.
|
||
// An arm is inert when NO `RoundEnded` follows it: the game
|
||
// ended in the same End step, so the sequence never advances a
|
||
// stage. `g.rounds >= 5` is a property of the GAME, not of the
|
||
// event, and using it marked every arm in every completed game
|
||
// as inert — which is how a metric ends up equal to the thing it
|
||
// was supposed to be a subset of.
|
||
let last_round_ended = g
|
||
.events
|
||
.iter()
|
||
.rposition(|e| matches!(e, games_ground::GroundEvent::RoundEnded { .. }));
|
||
let start = start_stress(&started);
|
||
let mut held: std::collections::BTreeMap<PlayerId, u8> =
|
||
g.state.players.keys().map(|s| (*s, start)).collect();
|
||
c.peak_stress = c.peak_stress.max(start);
|
||
for (i, e) in g.events.iter().enumerate() {
|
||
if matches!(e, games_ground::GroundEvent::DarvoTriggered { .. }) {
|
||
c.darvo += 1;
|
||
if last_round_ended.is_none_or(|last| i > last) {
|
||
c.inert_arms += 1;
|
||
}
|
||
}
|
||
if let games_ground::GroundEvent::StressSet { player, stress } = e {
|
||
held.insert(*player, *stress);
|
||
}
|
||
// Read from `held`, which is the point. The first correction
|
||
// BUILT `held`, then took the max over event payloads anyway
|
||
// and silenced the unused binding with `let _ = held;` — so
|
||
// the comment described code that did not exist (CB-REV-0002
|
||
// #10). The published figures were right only because the
|
||
// START_STRESS floor made the two agree.
|
||
c.peak_stress = c
|
||
.peak_stress
|
||
.max(held.values().copied().max().unwrap_or(start));
|
||
}
|
||
}
|
||
// **`games`, not `games + setup_fails`** (CB-REV-0002 #1).
|
||
//
|
||
// The first version asserted the SUM — and a refused setup increments
|
||
// `setup_fails` while skipping `games`, so the sum is invariant under
|
||
// exactly the failure it claimed to catch. It could only ever fire on
|
||
// a `play` error, the path that was already instrumented. Verified by
|
||
// injecting setup failures: green, exit 0, and a full table printed
|
||
// over 196-game columns under a banner reading "200 games per cell".
|
||
assert_eq!(
|
||
c.games, GAMES,
|
||
"{players}p {variant:?}: only {} of {GAMES} games ran ({} setups refused) — \
|
||
the cell is short, so every number in it is over a sample nobody chose",
|
||
c.games, c.setup_fails
|
||
);
|
||
assert_eq!(
|
||
c.played, GAMES,
|
||
"{players}p {variant:?}: {} of {} games reached an outcome over five rounds",
|
||
c.played, c.games
|
||
);
|
||
if c.setup_fails > 0 {
|
||
eprintln!(
|
||
" !! {players}p {variant:?}: {} setups refused",
|
||
c.setup_fails
|
||
);
|
||
}
|
||
c
|
||
}
|
||
|
||
fn main() {
|
||
println!("CB-WP-0039 — does H1 reach a seat that does not regulate?\n");
|
||
println!("`reactive` is greedy with ONE preference changed: GROUND is");
|
||
println!("demoted below ATTACK, so the seat never grounds to shed Stress.");
|
||
println!("SHARED GROUND; `won` is group success; {GAMES} games per cell.\n");
|
||
|
||
for (vlabel, variant) in [
|
||
("BASELINE ground-darvo-r0", Variant::Baseline),
|
||
("H1 h1-problem-stress", Variant::H1ProblemStress),
|
||
] {
|
||
println!("{vlabel}");
|
||
println!(" greedy (regulates) reactive (does not)");
|
||
println!("seats games won atk darvo peak games won atk darvo peak inert");
|
||
for players in [2u8, 3, 4, 6] {
|
||
let g = sweep(ScoringMode::SharedGround, variant, players, false);
|
||
let r = sweep(ScoringMode::SharedGround, variant, players, true);
|
||
println!(
|
||
" {players}p {:>4} {:>4} {:>4} {:>5} {:>4} {:>4} {:>4} {:>4} {:>5} {:>4} {:>6}",
|
||
g.games,
|
||
g.won,
|
||
g.atk,
|
||
g.darvo,
|
||
g.peak_stress,
|
||
r.games,
|
||
r.won,
|
||
r.atk,
|
||
r.darvo,
|
||
r.peak_stress,
|
||
r.inert_arms
|
||
);
|
||
}
|
||
println!();
|
||
}
|
||
}
|