coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

T08 complete: benchmarks, determinism evidence, and one missed metric

Browse source 
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.

Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.

Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.

AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.

A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.

The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 03:09:14 +02:00
parent 85d93a9e3c
commit eb1378e667
4 changed files with 569 additions and 37 deletions
Download patch file Download diff file Expand all files Collapse all files

161
games/ground/src/lib.rs
View file

@ -2087,3 +2087,164 @@ mod tests {
assert_ne!(state_hash_hex(&a), state_hash_hex(&c));
}
}
#[cfg(test)]
mod bench_shape {
use super::*;
use cb_game_runtime::{ScenarioGame, Setup};
/// AM-6 reports events/second; the evidence file converts that to
/// rounds and commands per second. Both divisors are pinned here so
/// a change to the workload cannot silently rescale the metric.
#[test]
fn synthetic_round_shape_is_pinned() {
let mut state = GroundState::setup(
&Setup {
players: 3,
preset: "standard-3p".into(),
patch: BTreeMap::new(),
},
1,
)
.unwrap();
let mut events = 0;
let mut commands = 0;
let mut run = |state: &mut GroundState, actor: Actor, cmd: &GroundCommand| {
commands += 1;
if let Ok(produced) = state.validate(actor, cmd) {
for e in &produced {
state.fold(e);
}
events += produced.len();
}
};
for (seat, action, target) in [
(0u8, Action::Attack, Some(PlayerId(1))),
(2, Action::Support, Some(PlayerId(1))),
(1, Action::Ground, None),
] {
run(
&mut state,
Actor::Player(PlayerId(seat)),
&GroundCommand::SelectAction {
action,
target,
problem: None,
},
);
}
run(&mut state, Actor::System, &GroundCommand::Reveal);
run(
&mut state,
Actor::Player(PlayerId(1)),
&GroundCommand::ChooseGroundMode {
mode: GroundMode::Gr,
choice: None,
},
);
run(&mut state, Actor::System, &GroundCommand::Resolve);
run(&mut state, Actor::System, &GroundCommand::EndRound);
assert_eq!(commands, 7, "commands per synthetic round");
assert_eq!(events, 13, "events per synthetic round");
}
}
#[cfg(test)]
mod replay_probe {
use super::*;
use cb_events::state_hash_hex;
use cb_game_runtime::{ScenarioGame, Setup};
use std::time::Instant;
fn fresh(seed: u64) -> GroundState {
GroundState::setup(
&Setup {
players: 3,
preset: "standard-3p".into(),
patch: BTreeMap::new(),
},
seed,
)
.unwrap()
}
fn record_round(state: &mut GroundState, log: &mut Vec<GroundEvent>) -> usize {
let mut n = 0;
let mut run = |state: &mut GroundState, actor: Actor, cmd: &GroundCommand| {
if let Ok(produced) = state.validate(actor, cmd) {
for e in &produced {
state.fold(e);
log.push(e.clone());
n += 1;
}
}
};
for (seat, action, target) in [
(0u8, Action::Attack, Some(PlayerId(1))),
(2, Action::Support, Some(PlayerId(1))),
(1, Action::Ground, None),
] {
run(
state,
Actor::Player(PlayerId(seat)),
&GroundCommand::SelectAction {
action,
target,
problem: None,
},
);
}
run(state, Actor::System, &GroundCommand::Reveal);
run(
state,
Actor::Player(PlayerId(1)),
&GroundCommand::ChooseGroundMode {
mode: GroundMode::Gr,
choice: None,
},
);
run(state, Actor::System, &GroundCommand::Resolve);
run(state, Actor::System, &GroundCommand::EndRound);
n
}
/// AM-7: folding a 100k-event log back into state must stay well
/// under the 5s budget, and must be linear in log length.
#[test]
fn replay_100k_events_is_linear_and_fast() {
for target in [10_000usize, 100_000] {
let mut log = Vec::with_capacity(target);
let mut source = fresh(42);
let mut games = 0u64;
let mut stalls = 0;
while log.len() < target {
if source.outcome.is_some() {
games += 1;
source = fresh(42 + games);
}
if record_round(&mut source, &mut log) == 0 {
stalls += 1;
assert!(stalls < 10, "round produced no events; builder stalled");
}
}
let start = Instant::now();
let mut state = fresh(42);
for event in &log {
state.fold(event);
}
let hash = state_hash_hex(&state);
let elapsed = start.elapsed();
println!(
"replay {} events in {:?} ({:.0} events/s), hash {}",
log.len(),
elapsed,
log.len() as f64 / elapsed.as_secs_f64(),
&hash[..8]
);
assert!(elapsed.as_secs_f64() < 5.0, "AM-7: 100k replay under 5s");
}
}
}