CB-WP-0015: the two inert clauses, AM-7 scaling and AM-8 N=10
Some checks failed
ci / check (push) Failing after 3s
Some checks failed
ci / check (push) Failing after 3s
Provenance (tier S, one paragraph in lieu of survey and ADR): the two clauses mutation-check has reported inert since CB-WP-0005. AM-7's scaling ratio was held up by a test literally named replay_100k_events_is_linear_and_fast that computed both throughputs, printed both, and never divided one by the other. AM-8's N=10 was held up by a runner that does two. Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's >=10-of-14 prediction MET for the first time. Neither was closed by amending the question away, which was the live risk: the denominator is unchanged and the four unenforced rows are the four already unenforceable. AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's, correct for a floor on one number) gave 0.581-1.085 on an unchanged binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU contention while absolute throughput fell 4x. The INDETERMINATE guard demanded unanimity and failed a good measurement over one sample 0.001 under the floor; it now requires a two-thirds majority. The control that matters: AM-6's constant-cost mutation halves throughput and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6. AM-8 kept N=10 because the measurement said so. Perturbing the RNG only from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A late-onset divergence is deterministic, not flaky, so it is a control rather than a coin flip. Ten runs live on one scenario (make am8, ~2s) rather than all 25 (47s a build). GameKernel 5b records it. The full run also found AM-4a's own mutation stale since ADR-0008 D3 moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported HARNESS-BROKEN, no score published. The build-free half of that check is now a --self-test assertion, so make all catches the next one. mutation-check clauses may now carry their own verify and mutation, and then the enforced flag is measured rather than declared; a declaration disagreeing with its measurement is refused. make all exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
7e9ab221a7
commit
ee37b82675
9 changed files with 802 additions and 51 deletions
|
|
@ -17,7 +17,7 @@ const REPLAY_DIR: &str = "replays";
|
|||
fn main() {
|
||||
let argv: Vec<String> = std::env::args().skip(1).collect();
|
||||
if argv.is_empty() {
|
||||
eprintln!("usage: cb-sim <scenario.yaml>...");
|
||||
eprintln!("usage: cb-sim [--runs <n>] <scenario.yaml>...");
|
||||
eprintln!(" cb-sim --replay <bundle.cbreplay>...");
|
||||
std::process::exit(64);
|
||||
}
|
||||
|
|
@ -28,7 +28,28 @@ fn main() {
|
|||
if argv[0] == "--replay" {
|
||||
std::process::exit(replay_bundles(&argv[1..]));
|
||||
}
|
||||
let args = argv;
|
||||
|
||||
// AM-8 (CB-WP-0015 T02): the replay count. Defaults to K8's two, so
|
||||
// `make sim` is unchanged; `make am8` passes 10 on one scenario.
|
||||
let mut runs = scenario::K8_RUNS;
|
||||
let mut args = argv;
|
||||
if args[0] == "--runs" {
|
||||
if args.len() < 3 {
|
||||
eprintln!("usage: cb-sim --runs <n> <scenario.yaml>...");
|
||||
std::process::exit(64);
|
||||
}
|
||||
runs = match args[1].parse::<usize>() {
|
||||
Ok(n) if n >= 2 => n,
|
||||
// A --runs that silently fell back to 2 would report an
|
||||
// N=10 result after doing an N=2 check — the harness-does-
|
||||
// nothing class this binary's header is about.
|
||||
_ => {
|
||||
eprintln!("--runs needs an integer >= 2, got {:?}", args[1]);
|
||||
std::process::exit(64);
|
||||
}
|
||||
};
|
||||
args = args.split_off(2);
|
||||
}
|
||||
|
||||
let mut failed = false;
|
||||
let mut passed = 0usize;
|
||||
|
|
@ -53,7 +74,7 @@ fn main() {
|
|||
};
|
||||
|
||||
let outcome = match sc.scenario.split('/').next() {
|
||||
Some("ground") => scenario::run::<GroundState>(&sc),
|
||||
Some("ground") => scenario::run_n::<GroundState>(&sc, runs),
|
||||
_ => {
|
||||
// A renamed or typo'd prefix would otherwise skip every
|
||||
// scenario while the run still looked clean.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue