Some checks failed
ci / check (push) Failing after 3s
The pattern is now measured over three rounds: 5 fatal, then 3 (2 from the previous round's corrections), then 4 (3 from them). The corrections are not getting safer. FATAL 1: round 2's short-cell assertion went into regulation.rs only. attack-value.rs — which produced every number in CB-EV-0030's DARVO table — still just warned, and the gate registered to close the finding claimed the property for both. FATAL 2, the sharpest of the three rounds: counting games proves they STARTED. Stopping the engine after one round gives 200 games, all-zero columns and exit 0 — byte for byte the signature CB-EV-0030 says the instrumentation distinguishes from a real result. Both harnesses now require every counted game to have reached an outcome over five rounds. FATAL 3: round 2's `.csv` filter was applied to all three loops, so catalog.yaml and rules_delta.yaml — whose missing digests were round 1's finding — were recorded and then never compared, and never checked against upstream at all. Only the parser loop filters now. FATAL 4: five of six tiebreak comparators had no coverage. GR-E04's tiebreak never executes in any scenario. All four are now covered and mutation-verified; the Blame key needed compensating claims to be reachable at all, since Blame also lowers the coalition score. SERIOUS: "peak held" computed the same number as "peak assigned" for every possible input — the real gap was that START_STRESS was an unchecked constant, now read off the dealt state; cadence="none" was a pure loophole, removed; sibling discovery swapped a hand-written list for hand-written globs and missed metadata.json and VARIANT.md, both named in the package's own changed_files — now walked, and it found them immediately; and "~72,000 games" was unsourced, make panels runs 17,600. Also separated two kinds of number that were presented alike: seats×games is invariant, 363 and 29 vary 7.1%-11.5% across samples. Round 4 owed. The conclusion is not that the work is nearly right — it is that author-made corrections to measurement work should be assumed defective until a fresh reader has attacked them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
251 lines
9.6 KiB
Rust
251 lines
9.6 KiB
Rust
//! F17's reproduction: what is ATTACK worth, in each scoring mode?
|
|
//!
|
|
//! The maintainer reported *"there is no incentive to play attacks as long
|
|
//! as I have positive cards"*. F17 sat as a note because nothing
|
|
//! demonstrated it. This is the artifact.
|
|
//!
|
|
//! **`GreedyPolicy` plays ATTACK zero times, and that measures our bot.**
|
|
//! `bot.rs` ranks `Action::Attack => 10`, below everything else. Reporting
|
|
//! "the game gives no incentive" from a policy programmed to rank attack
|
|
//! last would be CB-WP-0025's C4 error again — one policy's behaviour
|
|
//! presented as the game's.
|
|
//!
|
|
//! So this varies **exactly one number**: ATTACK's rank in an otherwise
|
|
//! identical policy. 10 is greedy's. 75 puts it above SUPPORT and below
|
|
//! INVESTIGATE — *attack when convenient*. 95 puts it above SOLVE —
|
|
//! *attack whenever legal*.
|
|
//!
|
|
//! And it asks the question in **all three scoring modes**, because Blame
|
|
//! costs *personal* score: ATTACK may be worthless in the co-op mode and
|
|
//! earn its place in GR-E03 or GR-E04.
|
|
|
|
use cb_game_runtime::{ScenarioGame, Setup};
|
|
use cb_kernel::PlayerId;
|
|
use games_ground::bot::{play, Choice, GreedyPolicy, Policy};
|
|
use games_ground::{Action, GroundCommand, GroundState, ScoringMode};
|
|
|
|
/// Games per cell. Named once so the footer and the assertion cannot
|
|
/// disagree — the footer said "200 games per cell" over 195-game columns.
|
|
const GAMES: u32 = 200;
|
|
|
|
/// Greedy's ordering with ATTACK's rank as a parameter.
|
|
///
|
|
/// A copy of the ranking rather than a call into it: `GreedyPolicy::rank`
|
|
/// is private, and the point is to vary one number while holding every
|
|
/// other preference identical.
|
|
struct Attacker(i32);
|
|
|
|
impl Policy for Attacker {
|
|
fn name(&self) -> &'static str {
|
|
"attacker"
|
|
}
|
|
fn choose(
|
|
&mut self,
|
|
state: &GroundState,
|
|
seat: PlayerId,
|
|
legal: &[GroundCommand],
|
|
_may_pass: bool,
|
|
) -> Choice {
|
|
let gated = state
|
|
.players
|
|
.get(&seat)
|
|
.is_some_and(|p| p.stress >= 4 && !p.freedom_gate_lifted);
|
|
let rank = |c: &GroundCommand| -> i32 {
|
|
match c {
|
|
GroundCommand::SelectAction {
|
|
action, problem, ..
|
|
} => match action {
|
|
Action::Ground if gated => 100,
|
|
Action::Attack => self.0, // the one varying number
|
|
Action::Solve
|
|
if problem
|
|
.and_then(|p| state.problems.get(&p))
|
|
.is_some_and(|p| p.claimed_by.is_some()) =>
|
|
{
|
|
5
|
|
}
|
|
Action::Solve => 90,
|
|
Action::Investigate => 80,
|
|
Action::Support => 70,
|
|
Action::Ground => 60,
|
|
},
|
|
GroundCommand::SpendFreedom => {
|
|
if gated {
|
|
96
|
|
} else {
|
|
0
|
|
}
|
|
}
|
|
GroundCommand::Reveal | GroundCommand::Resolve | GroundCommand::EndRound => -1,
|
|
_ => 50,
|
|
}
|
|
};
|
|
let mut best = 0usize;
|
|
for (i, c) in legal.iter().enumerate() {
|
|
if rank(c) > rank(&legal[best]) {
|
|
best = i;
|
|
}
|
|
}
|
|
Choice::Command(best)
|
|
}
|
|
}
|
|
|
|
/// `(games, good outcomes, attacks, DARVO armings)` over 200 seeds.
|
|
///
|
|
/// **`games` is reported, not assumed to be 200.** A rules variant can
|
|
/// make the engine refuse a position, and a harness that skips those
|
|
/// silently reports "nobody won" for a run in which nobody played.
|
|
fn sweep(
|
|
mode: ScoringMode,
|
|
variant: games_ground::Variant,
|
|
players: u8,
|
|
mk: &dyn Fn(u8) -> Vec<Box<dyn Policy>>,
|
|
) -> (u32, u32, u32, u32) {
|
|
let (mut games, mut won, mut atk, mut darvo) = (0, 0, 0, 0);
|
|
let mut played = 0u32;
|
|
let mut errs: Vec<String> = Vec::new();
|
|
let mut setup_fails = 0u32;
|
|
for seed in 0..GAMES as u64 {
|
|
let Ok(mut st) = GroundState::setup(
|
|
&Setup {
|
|
players,
|
|
preset: format!("standard-{players}p"),
|
|
patch: Default::default(),
|
|
},
|
|
seed,
|
|
) else {
|
|
setup_fails += 1;
|
|
continue;
|
|
};
|
|
st.mode = mode;
|
|
// CB-WP-0038 T03: the SAME instrument measures both rule sets in
|
|
// the same run. Two harnesses would compare harnesses.
|
|
st.variant = variant;
|
|
let mut ps = mk(players);
|
|
// CB-WP-0038 T03: a game the engine REFUSES is not a game the
|
|
// table lost, and `else { continue }` made the two identical.
|
|
// H1's first run showed won/atk/darvo all exactly 0 at 3+ seats,
|
|
// which is the signature of every game being discarded rather
|
|
// than played -- so the count is now reported, not swallowed.
|
|
let g = match play(st, &mut ps) {
|
|
Ok(g) => g,
|
|
Err(e) => {
|
|
errs.push(format!("{e:?}"));
|
|
continue;
|
|
}
|
|
};
|
|
games += 1;
|
|
if g.state.outcome.is_some() && g.rounds == 5 {
|
|
played += 1;
|
|
}
|
|
|
|
// In the co-op mode the table wins together. In the other two the
|
|
// question is whether SEAT 0 is among the winners — because that
|
|
// is what an individual player's incentive turns on, and ATTACK is
|
|
// an individual's choice.
|
|
let good = match mode {
|
|
ScoringMode::SharedGround => g.state.outcome.as_ref().is_some_and(|o| o.group_success),
|
|
_ => g
|
|
.state
|
|
.outcome
|
|
.as_ref()
|
|
.is_some_and(|o| o.winners.contains(&PlayerId(0))),
|
|
};
|
|
if good {
|
|
won += 1;
|
|
}
|
|
for (_, c) in &g.steps {
|
|
if let GroundCommand::SelectAction {
|
|
action: Action::Attack,
|
|
..
|
|
} = c
|
|
{
|
|
atk += 1;
|
|
}
|
|
}
|
|
for e in &g.events {
|
|
if matches!(e, games_ground::GroundEvent::DarvoTriggered { .. }) {
|
|
darvo += 1;
|
|
}
|
|
}
|
|
}
|
|
// **Asserted, not printed** (CB-REV-0003 #1). This harness produced
|
|
// every number in CB-EV-0030's DARVO table, and it still only warned
|
|
// on a short cell — the very defect fixed in `regulation.rs` and left
|
|
// here, while the gate registered to close it claimed the property
|
|
// for both.
|
|
assert_eq!(
|
|
games,
|
|
GAMES,
|
|
"{players}p {mode:?} {variant:?}: only {games} of {GAMES} games ran \
|
|
({setup_fails} setups refused, {} play errors) — the cell is short, so \
|
|
every number in it is over a sample nobody chose",
|
|
errs.len()
|
|
);
|
|
// **And that a game RAN is not that it was PLAYED** (CB-REV-0003 #2).
|
|
// Stopping the engine after one round gave 200 games, all-zero
|
|
// columns and exit 0 — byte for byte the signature CB-EV-0030 §3
|
|
// claims to distinguish from a real result. A counted game must have
|
|
// reached an outcome.
|
|
assert_eq!(
|
|
played, GAMES,
|
|
"{players}p {mode:?} {variant:?}: {played} of {games} games reached an \
|
|
outcome — the rest stopped early, and all-zero columns would read as \
|
|
a result rather than as nothing having happened"
|
|
);
|
|
(games, won, atk, darvo)
|
|
}
|
|
|
|
fn main() {
|
|
println!("F17 — what is ATTACK worth, in each scoring mode?\n");
|
|
println!("CB-WP-0038 T03: run for BOTH rule sets. ground-game's H1 asks");
|
|
println!("whether problem pressure plus a high-Stress self-soothe make");
|
|
println!("ATTACK and DARVO relevant without making always-ATTACK optimal.\n");
|
|
println!("`won` is group success in SHARED GROUND, and \"seat 0 is among the");
|
|
println!("winners\" in the other two, because ATTACK is an individual's choice");
|
|
println!("and that is what an individual's incentive turns on.\n");
|
|
|
|
for (vlabel, variant) in [
|
|
("BASELINE ground-darvo-r0", games_ground::Variant::Baseline),
|
|
(
|
|
"H1 h1-problem-stress",
|
|
games_ground::Variant::H1ProblemStress,
|
|
),
|
|
] {
|
|
println!("================ {vlabel} ================\n");
|
|
for (label, mode) in [
|
|
("SHARED GROUND (GR-E02, co-op)", ScoringMode::SharedGround),
|
|
(
|
|
"COMMON PROBLEM (GR-E03, semi-co-op)",
|
|
ScoringMode::CommonProblem,
|
|
),
|
|
("BONDED COALITIONS (GR-E04)", ScoringMode::BondedCoalitions),
|
|
] {
|
|
println!("{label}");
|
|
println!(" rank=10 (greedy) rank=75 (sometimes) rank=95 (always)");
|
|
println!("seats won atk darvo won atk darvo won atk darvo");
|
|
for players in [2u8, 3, 4, 6] {
|
|
let (_, gw, ga, gd) = sweep(mode, variant, players, &|n| {
|
|
(0..n)
|
|
.map(|_| Box::new(GreedyPolicy) as Box<dyn Policy>)
|
|
.collect()
|
|
});
|
|
let cell = |r: i32| {
|
|
sweep(mode, variant, players, &move |n| {
|
|
(0..n)
|
|
.map(|_| Box::new(Attacker(r)) as Box<dyn Policy>)
|
|
.collect()
|
|
})
|
|
};
|
|
let (_, mw, ma, md) = cell(75);
|
|
let (_, aw, aa, ad) = cell(95);
|
|
println!(
|
|
" {players}p {gw:>4} {ga:>4} {gd:>5} {mw:>4} {ma:>4} {md:>5} \
|
|
{aw:>4} {aa:>4} {ad:>5}"
|
|
);
|
|
}
|
|
println!();
|
|
}
|
|
}
|
|
println!("(200 games per cell)");
|
|
}
|