clay-borg/games/ground/examples/attack-value.rs
tegwick c4a8a227c0
Some checks failed
ci / check (push) Failing after 3s
CB-REV-0003: round 3, and three of four FATAL came from round 2's fixes
The pattern is now measured over three rounds: 5 fatal, then 3 (2 from the
previous round's corrections), then 4 (3 from them). The corrections are
not getting safer.

FATAL 1: round 2's short-cell assertion went into regulation.rs only.
attack-value.rs — which produced every number in CB-EV-0030's DARVO table
— still just warned, and the gate registered to close the finding claimed
the property for both.

FATAL 2, the sharpest of the three rounds: counting games proves they
STARTED. Stopping the engine after one round gives 200 games, all-zero
columns and exit 0 — byte for byte the signature CB-EV-0030 says the
instrumentation distinguishes from a real result. Both harnesses now
require every counted game to have reached an outcome over five rounds.

FATAL 3: round 2's `.csv` filter was applied to all three loops, so
catalog.yaml and rules_delta.yaml — whose missing digests were round 1's
finding — were recorded and then never compared, and never checked against
upstream at all. Only the parser loop filters now.

FATAL 4: five of six tiebreak comparators had no coverage. GR-E04's
tiebreak never executes in any scenario. All four are now covered and
mutation-verified; the Blame key needed compensating claims to be
reachable at all, since Blame also lowers the coalition score.

SERIOUS: "peak held" computed the same number as "peak assigned" for every
possible input — the real gap was that START_STRESS was an unchecked
constant, now read off the dealt state; cadence="none" was a pure
loophole, removed; sibling discovery swapped a hand-written list for
hand-written globs and missed metadata.json and VARIANT.md, both named in
the package's own changed_files — now walked, and it found them
immediately; and "~72,000 games" was unsourced, make panels runs 17,600.

Also separated two kinds of number that were presented alike: seats×games
is invariant, 363 and 29 vary 7.1%-11.5% across samples.

Round 4 owed. The conclusion is not that the work is nearly right — it is
that author-made corrections to measurement work should be assumed
defective until a fresh reader has attacked them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 10:26:25 +02:00

251 lines
9.6 KiB
Rust

//! F17's reproduction: what is ATTACK worth, in each scoring mode?
//!
//! The maintainer reported *"there is no incentive to play attacks as long
//! as I have positive cards"*. F17 sat as a note because nothing
//! demonstrated it. This is the artifact.
//!
//! **`GreedyPolicy` plays ATTACK zero times, and that measures our bot.**
//! `bot.rs` ranks `Action::Attack => 10`, below everything else. Reporting
//! "the game gives no incentive" from a policy programmed to rank attack
//! last would be CB-WP-0025's C4 error again — one policy's behaviour
//! presented as the game's.
//!
//! So this varies **exactly one number**: ATTACK's rank in an otherwise
//! identical policy. 10 is greedy's. 75 puts it above SUPPORT and below
//! INVESTIGATE — *attack when convenient*. 95 puts it above SOLVE —
//! *attack whenever legal*.
//!
//! And it asks the question in **all three scoring modes**, because Blame
//! costs *personal* score: ATTACK may be worthless in the co-op mode and
//! earn its place in GR-E03 or GR-E04.
use cb_game_runtime::{ScenarioGame, Setup};
use cb_kernel::PlayerId;
use games_ground::bot::{play, Choice, GreedyPolicy, Policy};
use games_ground::{Action, GroundCommand, GroundState, ScoringMode};
/// Games per cell. Named once so the footer and the assertion cannot
/// disagree — the footer said "200 games per cell" over 195-game columns.
const GAMES: u32 = 200;
/// Greedy's ordering with ATTACK's rank as a parameter.
///
/// A copy of the ranking rather than a call into it: `GreedyPolicy::rank`
/// is private, and the point is to vary one number while holding every
/// other preference identical.
struct Attacker(i32);
impl Policy for Attacker {
fn name(&self) -> &'static str {
"attacker"
}
fn choose(
&mut self,
state: &GroundState,
seat: PlayerId,
legal: &[GroundCommand],
_may_pass: bool,
) -> Choice {
let gated = state
.players
.get(&seat)
.is_some_and(|p| p.stress >= 4 && !p.freedom_gate_lifted);
let rank = |c: &GroundCommand| -> i32 {
match c {
GroundCommand::SelectAction {
action, problem, ..
} => match action {
Action::Ground if gated => 100,
Action::Attack => self.0, // the one varying number
Action::Solve
if problem
.and_then(|p| state.problems.get(&p))
.is_some_and(|p| p.claimed_by.is_some()) =>
{
5
}
Action::Solve => 90,
Action::Investigate => 80,
Action::Support => 70,
Action::Ground => 60,
},
GroundCommand::SpendFreedom => {
if gated {
96
} else {
0
}
}
GroundCommand::Reveal | GroundCommand::Resolve | GroundCommand::EndRound => -1,
_ => 50,
}
};
let mut best = 0usize;
for (i, c) in legal.iter().enumerate() {
if rank(c) > rank(&legal[best]) {
best = i;
}
}
Choice::Command(best)
}
}
/// `(games, good outcomes, attacks, DARVO armings)` over 200 seeds.
///
/// **`games` is reported, not assumed to be 200.** A rules variant can
/// make the engine refuse a position, and a harness that skips those
/// silently reports "nobody won" for a run in which nobody played.
fn sweep(
mode: ScoringMode,
variant: games_ground::Variant,
players: u8,
mk: &dyn Fn(u8) -> Vec<Box<dyn Policy>>,
) -> (u32, u32, u32, u32) {
let (mut games, mut won, mut atk, mut darvo) = (0, 0, 0, 0);
let mut played = 0u32;
let mut errs: Vec<String> = Vec::new();
let mut setup_fails = 0u32;
for seed in 0..GAMES as u64 {
let Ok(mut st) = GroundState::setup(
&Setup {
players,
preset: format!("standard-{players}p"),
patch: Default::default(),
},
seed,
) else {
setup_fails += 1;
continue;
};
st.mode = mode;
// CB-WP-0038 T03: the SAME instrument measures both rule sets in
// the same run. Two harnesses would compare harnesses.
st.variant = variant;
let mut ps = mk(players);
// CB-WP-0038 T03: a game the engine REFUSES is not a game the
// table lost, and `else { continue }` made the two identical.
// H1's first run showed won/atk/darvo all exactly 0 at 3+ seats,
// which is the signature of every game being discarded rather
// than played -- so the count is now reported, not swallowed.
let g = match play(st, &mut ps) {
Ok(g) => g,
Err(e) => {
errs.push(format!("{e:?}"));
continue;
}
};
games += 1;
if g.state.outcome.is_some() && g.rounds == 5 {
played += 1;
}
// In the co-op mode the table wins together. In the other two the
// question is whether SEAT 0 is among the winners — because that
// is what an individual player's incentive turns on, and ATTACK is
// an individual's choice.
let good = match mode {
ScoringMode::SharedGround => g.state.outcome.as_ref().is_some_and(|o| o.group_success),
_ => g
.state
.outcome
.as_ref()
.is_some_and(|o| o.winners.contains(&PlayerId(0))),
};
if good {
won += 1;
}
for (_, c) in &g.steps {
if let GroundCommand::SelectAction {
action: Action::Attack,
..
} = c
{
atk += 1;
}
}
for e in &g.events {
if matches!(e, games_ground::GroundEvent::DarvoTriggered { .. }) {
darvo += 1;
}
}
}
// **Asserted, not printed** (CB-REV-0003 #1). This harness produced
// every number in CB-EV-0030's DARVO table, and it still only warned
// on a short cell — the very defect fixed in `regulation.rs` and left
// here, while the gate registered to close it claimed the property
// for both.
assert_eq!(
games,
GAMES,
"{players}p {mode:?} {variant:?}: only {games} of {GAMES} games ran \
({setup_fails} setups refused, {} play errors) — the cell is short, so \
every number in it is over a sample nobody chose",
errs.len()
);
// **And that a game RAN is not that it was PLAYED** (CB-REV-0003 #2).
// Stopping the engine after one round gave 200 games, all-zero
// columns and exit 0 — byte for byte the signature CB-EV-0030 §3
// claims to distinguish from a real result. A counted game must have
// reached an outcome.
assert_eq!(
played, GAMES,
"{players}p {mode:?} {variant:?}: {played} of {games} games reached an \
outcome — the rest stopped early, and all-zero columns would read as \
a result rather than as nothing having happened"
);
(games, won, atk, darvo)
}
fn main() {
println!("F17 — what is ATTACK worth, in each scoring mode?\n");
println!("CB-WP-0038 T03: run for BOTH rule sets. ground-game's H1 asks");
println!("whether problem pressure plus a high-Stress self-soothe make");
println!("ATTACK and DARVO relevant without making always-ATTACK optimal.\n");
println!("`won` is group success in SHARED GROUND, and \"seat 0 is among the");
println!("winners\" in the other two, because ATTACK is an individual's choice");
println!("and that is what an individual's incentive turns on.\n");
for (vlabel, variant) in [
("BASELINE ground-darvo-r0", games_ground::Variant::Baseline),
(
"H1 h1-problem-stress",
games_ground::Variant::H1ProblemStress,
),
] {
println!("================ {vlabel} ================\n");
for (label, mode) in [
("SHARED GROUND (GR-E02, co-op)", ScoringMode::SharedGround),
(
"COMMON PROBLEM (GR-E03, semi-co-op)",
ScoringMode::CommonProblem,
),
("BONDED COALITIONS (GR-E04)", ScoringMode::BondedCoalitions),
] {
println!("{label}");
println!(" rank=10 (greedy) rank=75 (sometimes) rank=95 (always)");
println!("seats won atk darvo won atk darvo won atk darvo");
for players in [2u8, 3, 4, 6] {
let (_, gw, ga, gd) = sweep(mode, variant, players, &|n| {
(0..n)
.map(|_| Box::new(GreedyPolicy) as Box<dyn Policy>)
.collect()
});
let cell = |r: i32| {
sweep(mode, variant, players, &move |n| {
(0..n)
.map(|_| Box::new(Attacker(r)) as Box<dyn Policy>)
.collect()
})
};
let (_, mw, ma, md) = cell(75);
let (_, aw, aa, ad) = cell(95);
println!(
" {players}p {gw:>4} {ga:>4} {gd:>5} {mw:>4} {ma:>4} {md:>5} \
{aw:>4} {aa:>4} {ad:>5}"
);
}
println!();
}
}
println!("(200 games per cell)");
}