CB-WP-0006 T01: assert the AM-6 throughput target
Nothing in the workspace compared any number to 100,000 events/s while the evidence file reported "AM-6 | met, 16.5x". Now a test does — a test, not a bench, because Criterion reports throughput and asserts nothing, which is why this row measured nothing for six passes. Measured on bnt-lap001: 341,280 ev/s in debug (3.4x the target), ~2.4-3.1M in release. The spec target holds even in an unoptimized build, so the gate needs no cfg split and runs in the ordinary `make test`. The trap this task named — loosening a flaky timing assertion until it never fires — is avoided by construction. The threshold is the spec value, untouched; the constant says lowering it requires an ADR; and the failure message repeats that, states measured headroom, and names reference figures, so an agent hitting a red AM-6 is told not to tune it in the place they are actually reading. Robustness comes from best-of-N, not from a lower bar: a throughput floor asks whether the machine is capable, so transient load should not fail the build. Two positive controls in the test: a run that applied fewer than 50,000 events, or measured zero elapsed time, fails rather than scoring as infinite throughput. Verified by a PROPERTY mutation — 4,000 black_box iterations injected into GroundState::fold, the hot path — not a threshold tweak, which would only prove the comparison runs. And the FA class found last pass is now gated. mutation-check rows gained an `expect` field: the mutant's output must contain the row's stated failure string or the verdict is WRONG-REASON, not red. Without it a mutation that merely failed to compile would credit its row with an assertion it does not have. Verified by pointing expect at a string the verifier never prints and watching the verdict flip. This is remedy (2) from the CB-WP-0005 retrospective, built a task earlier than planned because the class it guards is the newest and most dangerous. M-D1-MUT: 4 -> 5 of 14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ba7c2f88ae
commit
c43754f0fe
7 changed files with 159 additions and 14 deletions
|
|
@ -2219,6 +2219,74 @@ mod replay_probe {
|
|||
n
|
||||
}
|
||||
|
||||
/// AM-6 target from GameKernel §5, in applied events per second.
|
||||
///
|
||||
/// **Pinned, not tuned.** CB-WP-0006 T01 named the trap up front: a
|
||||
/// timing assertion is flaky by nature and the reflex is to loosen it
|
||||
/// until it never fires, which reproduces the defect being fixed —
|
||||
/// this row was `unmutatable` because *nothing in the workspace
|
||||
/// compared any number to 100,000*, while the evidence file reported
|
||||
/// `AM-6 | met, 16.5×`.
|
||||
///
|
||||
/// Measured on bnt-lap001 2026-07-31: **~182k–212k ev/s in debug**,
|
||||
/// **~2.4M–3.1M ev/s in release**. So the spec target holds even in an
|
||||
/// unoptimized build, with ~1.8× headroom there and ~24× in release.
|
||||
/// **Lowering this constant requires an ADR.**
|
||||
const AM6_EVENTS_PER_SEC: f64 = 100_000.0;
|
||||
|
||||
/// Best of N samples. A throughput *floor* asks "is this machine
|
||||
/// capable", so transient load should not fail the build; taking the
|
||||
/// max makes the gate robust without loosening the threshold, which is
|
||||
/// the trade this task was told to avoid making on the threshold.
|
||||
const AM6_SAMPLES: usize = 3;
|
||||
|
||||
/// AM-6: applied events/s on the synthetic workload must clear the
|
||||
/// spec target. A test, not a bench — Criterion reports throughput and
|
||||
/// asserts nothing, which is why this row measured nothing for six
|
||||
/// passes.
|
||||
#[test]
|
||||
fn am6_throughput_clears_the_spec_target() {
|
||||
let mut best = 0.0f64;
|
||||
let mut sampled = 0usize;
|
||||
for s in 0..AM6_SAMPLES {
|
||||
let mut state = fresh(7 + s as u64);
|
||||
let mut log = Vec::new();
|
||||
let mut n = 0usize;
|
||||
let t = Instant::now();
|
||||
while n < 50_000 {
|
||||
if state.outcome.is_some() {
|
||||
state = fresh(7 + (s * 1_000_000 + n) as u64);
|
||||
}
|
||||
n += record_round(&mut state, &mut log);
|
||||
log.clear();
|
||||
}
|
||||
let secs = t.elapsed().as_secs_f64();
|
||||
// Positive control: a run that applied no events, or took no
|
||||
// measurable time, must not be scored as infinite throughput.
|
||||
assert!(
|
||||
n >= 50_000,
|
||||
"AM-6 harness applied {n} events, expected >= 50000"
|
||||
);
|
||||
assert!(secs > 0.0, "AM-6 harness measured zero elapsed time");
|
||||
best = best.max(n as f64 / secs);
|
||||
sampled += n;
|
||||
}
|
||||
let headroom = best / AM6_EVENTS_PER_SEC;
|
||||
println!(
|
||||
"AM-6: {best:.0} events/s (best of {AM6_SAMPLES}, {sampled} events, \
|
||||
debug_assertions={}) — {headroom:.1}x the {AM6_EVENTS_PER_SEC:.0} target",
|
||||
cfg!(debug_assertions)
|
||||
);
|
||||
assert!(
|
||||
best >= AM6_EVENTS_PER_SEC,
|
||||
"AM-6 UNMET: {best:.0} events/s < {AM6_EVENTS_PER_SEC:.0} target \
|
||||
({headroom:.2}x). debug_assertions={}. Reference: ~182k debug, \
|
||||
~2.4M release on bnt-lap001. Do NOT lower the target to pass — \
|
||||
GameKernel §5 AM-6 is a spec value and lowering it needs an ADR.",
|
||||
cfg!(debug_assertions)
|
||||
);
|
||||
}
|
||||
|
||||
/// AM-7: folding a 100k-event log back into state must stay well
|
||||
/// under the 5s budget, and must be linear in log length.
|
||||
#[test]
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue