CB-WP-0015: the two inert clauses, AM-7 scaling and AM-8 N=10
Some checks failed
ci / check (push) Failing after 3s

Provenance (tier S, one paragraph in lieu of survey and ADR): the two
clauses mutation-check has reported inert since CB-WP-0005. AM-7's
scaling ratio was held up by a test literally named
replay_100k_events_is_linear_and_fast that computed both throughputs,
printed both, and never divided one by the other. AM-8's N=10 was held
up by a runner that does two.

Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's
>=10-of-14 prediction MET for the first time. Neither was closed by
amending the question away, which was the live risk: the denominator is
unchanged and the four unenforced rows are the four already
unenforceable.

AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's,
correct for a floor on one number) gave 0.581-1.085 on an unchanged
binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at
fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU
contention while absolute throughput fell 4x. The INDETERMINATE guard
demanded unanimity and failed a good measurement over one sample
0.001 under the floor; it now requires a two-thirds majority. The
control that matters: AM-6's constant-cost mutation halves throughput
and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6.

AM-8 kept N=10 because the measurement said so. Perturbing the RNG only
from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A
late-onset divergence is deterministic, not flaky, so it is a control
rather than a coin flip. Ten runs live on one scenario (make am8, ~2s)
rather than all 25 (47s a build). GameKernel 5b records it.

The full run also found AM-4a's own mutation stale since ADR-0008 D3
moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported
HARNESS-BROKEN, no score published. The build-free half of that check
is now a --self-test assertion, so make all catches the next one.

mutation-check clauses may now carry their own verify and mutation, and
then the enforced flag is measured rather than declared; a declaration
disagreeing with its measurement is refused.

make all exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-02 14:07:08 +02:00
parent 7e9ab221a7
commit ee37b82675
9 changed files with 802 additions and 51 deletions

View file

@ -2440,4 +2440,216 @@ mod replay_probe {
assert!(elapsed.as_secs_f64() < 5.0, "AM-7: 100k replay under 5s");
}
}
/// AM-7 scaling floor from GameKernel §5: fold throughput at 100k
/// events must be at least this fraction of throughput at 5k.
///
/// **Pinned, not tuned** — same rule as `AM6_EVENTS_PER_SEC`. The
/// baseline it was written against is boardgame.io at 0.450.66×,
/// degrading to DNF at 100k. Lowering it requires an ADR.
const AM7_SCALING_FLOOR: f64 = 0.9;
/// The two sizes the spec names.
const AM7_SMALL: usize = 5_000;
const AM7_LARGE: usize = 100_000;
/// Events applied **per leg** per sample.
///
/// Sized against the machine's drift, not against timer resolution.
/// At 2,000,000 a sample took ~50 ms, and this machine's throughput
/// wanders by 2.5× over a few seconds (CB-EV-0013 §1) — so a 50 ms
/// sample measures whatever the clock happened to be doing. At 10 M
/// each leg runs ~0.35 s and averages over the drift instead of
/// sampling a point on it. Measured: medians 0.987 / 0.991 / 0.989
/// across three runs, and 0.989 under 8-way CPU contention. Doubling
/// this to 20 M cost 9 s more per run and did not tighten them.
const AM7_EVENTS_PER_LEG: usize = 10_000_000;
/// Paired samples per run. Nine rather than AM-6's three because the
/// verdict is a **median**, not a best-of: a median needs enough
/// samples that one excursion cannot move it.
const AM7_SAMPLES: usize = 9;
/// Fraction of samples that must agree with the median's verdict for
/// the run to be a measurement rather than noise.
///
/// **This replaced a unanimity guard that was measurably wrong.** The
/// first version declared INDETERMINATE whenever any sample fell on
/// the other side of the floor. Under deliberate 8-way CPU contention
/// the ratio held at a median of 0.971 — the pairing works, and
/// absolute throughput had dropped 4× — but one sample read 0.899,
/// a thousandth under the floor, and the guard turned a good
/// measurement into a failed build. It also fired intermittently
/// inside `mutation-check`, where this test runs straight after a
/// 50-second rebuild.
///
/// A gate that fails when the machine is busy is a flake, and a flake
/// gets suppressed rather than fixed. Requiring a two-thirds majority
/// keeps the guard's purpose — refusing to read a coin-flip as a
/// verdict — without treating a single outlier as one.
const AM7_AGREEMENT: f64 = 2.0 / 3.0;
/// Fold `log` once from a fresh state, returning only the time inside
/// the fold loop.
///
/// **Setup is outside the clock, and that is the first trap here.**
/// The small leg runs 20× more folds than the large one, so it pays
/// 20× more `fresh()` calls. Timing those would penalise the
/// denominator, inflate the ratio, and make the row pass for a reason
/// that has nothing to do with scaling.
fn fold_once(log: &[GroundEvent]) -> std::time::Duration {
let mut state = fresh(42);
let t = Instant::now();
for event in log {
state.fold(event);
}
let dt = t.elapsed();
// Keep the fold from being optimised out without paying for a hash
// inside the timed region.
std::hint::black_box(&state);
dt
}
/// One paired sample: the two legs **interleaved**, ratio taken inside.
///
/// **Two estimators were wrong before this one, and both looked
/// reasonable.**
///
/// The first took best-of-5 on each leg independently and divided —
/// the estimator AM-6 uses, correct there because AM-6 is a floor on a
/// single number and the question is "is this machine capable". For a
/// *ratio* it is wrong: the legs are measured at different moments and
/// the noise multiplies instead of cancelling. Five runs of an
/// unchanged binary gave **0.581 to 1.085**.
///
/// The second ran the legs back to back inside one sample, expecting
/// the load to be common-mode. It was not enough: this machine's
/// absolute throughput wanders between **22 M and 53 M ev/s within a
/// single run**, and a 50 ms leg samples a point on that wander rather
/// than averaging it. Three runs gave medians 1.004 / 0.931 / 0.956 —
/// clustered near the true value but still straddling the floor.
///
/// So: interleave at *fold* granularity, alternating one large fold
/// against twenty small ones so both legs apply the same number of
/// events, and run long enough that each leg spans the drift instead
/// of sitting inside one excursion of it.
fn paired_ratio(small: &[GroundEvent], large: &[GroundEvent]) -> (f64, f64, f64) {
let per_round = large.len();
let rounds = AM7_EVENTS_PER_LEG.div_ceil(per_round);
let small_folds = per_round.div_ceil(small.len());
let (mut t_small, mut t_large) = (std::time::Duration::ZERO, std::time::Duration::ZERO);
let (mut n_small, mut n_large) = (0usize, 0usize);
for _ in 0..rounds {
for _ in 0..small_folds {
t_small += fold_once(small);
n_small += small.len();
}
t_large += fold_once(large);
n_large += large.len();
}
assert!(
t_small.as_secs_f64() > 0.0 && t_large.as_secs_f64() > 0.0,
"AM-7 measured zero elapsed time"
);
let tp_small = n_small as f64 / t_small.as_secs_f64();
let tp_large = n_large as f64 / t_large.as_secs_f64();
(tp_small, tp_large, tp_large / tp_small)
}
/// Build one growing log of at least `target` events, the same way
/// `replay_100k_events_is_linear_and_fast` does.
fn growing_log(target: usize) -> Vec<GroundEvent> {
let mut log = Vec::with_capacity(target);
let mut seed = 42u64;
let mut source = fresh(seed);
let mut stalls = 0;
while log.len() < target {
if source.outcome.is_some() {
seed += 1;
source = fresh(seed);
}
if record_round(&mut source, &mut log) == 0 {
stalls += 1;
assert!(stalls < 10, "round produced no events; builder stalled");
}
}
log
}
/// AM-7's `scaling >= 0.9x` clause, which was inert for nine passes.
///
/// `mutation-check.py` said it plainly every run: *"no code computes
/// the ratio of throughput @100k to @5k or compares it to 0.9;
/// Criterion reports both and nothing relates them."* The sibling test
/// above is even named `replay_100k_events_is_linear_and_fast` and
/// checks the two sizes **independently** — it computes both numbers,
/// prints both, and never divides one by the other.
///
/// **`#[ignore]` for the same reason AM-6 is** (CB-WP-0006 T04): a
/// throughput assertion inside a parallel `cargo test` harness
/// measures contention. A *ratio* of two such timings is worse, not
/// better — the noise multiplies rather than cancels. Run `make am7`.
#[test]
#[ignore = "throughput ratio — invalid under a parallel harness; run `make am7`"]
fn am7_scaling_holds_from_5k_to_100k_events() {
// Positive control on the shape of the measurement. A harness that
// measured the same size twice would report ~1.0 and look
// excellent; one that swapped the legs would report the reciprocal
// and look excellent for the opposite reason.
let (small, large) = (growing_log(AM7_SMALL), growing_log(AM7_LARGE));
assert!(
large.len() >= 15 * small.len(),
"AM-7 legs are not far enough apart: {} vs {}",
small.len(),
large.len()
);
let mut ratios = Vec::with_capacity(AM7_SAMPLES);
for _ in 0..AM7_SAMPLES {
let (tp_small, tp_large, ratio) = paired_ratio(&small, &large);
println!(
" AM-7 sample: {tp_small:.0} ev/s @{AM7_SMALL} → \
{tp_large:.0} ev/s @{AM7_LARGE} = {ratio:.3}x"
);
ratios.push(ratio);
}
ratios.sort_by(|a, b| a.partial_cmp(b).expect("no NaN ratios"));
let (worst, median, best) = (
ratios[0],
ratios[ratios.len() / 2],
ratios[ratios.len() - 1],
);
println!(
"AM-7 scaling: {worst:.3}x / {median:.3}x / {best:.3}x \
(worst/median/best of {AM7_SAMPLES}, floor {AM7_SCALING_FLOOR})"
);
// The spread is reported, not hidden behind a best-of. The verdict
// is the median, and it counts as a measurement only if a
// two-thirds majority of samples agree with it — see
// AM7_AGREEMENT for the unanimity guard this replaced and why.
let agreeing = ratios
.iter()
.filter(|r| (**r >= AM7_SCALING_FLOOR) == (median >= AM7_SCALING_FLOOR))
.count();
let agreement = agreeing as f64 / ratios.len() as f64;
assert!(
agreement >= AM7_AGREEMENT,
"AM-7 INDETERMINATE: only {agreeing} of {} samples agree with \
the median ({worst:.3}x..{best:.3}x around \
{AM7_SCALING_FLOOR}). The machine is too noisy for this to be \
a measurement; do not read the best sample as a pass.",
ratios.len()
);
assert!(
median >= AM7_SCALING_FLOOR,
"AM-7 UNMET: throughput at {AM7_LARGE} events is {median:.3}x \
throughput at {AM7_SMALL} events (worst {worst:.3}x, best \
{best:.3}x), below the {AM7_SCALING_FLOOR} floor. Baseline: \
boardgame.io 0.45-0.66x, DNF at 100k. Do NOT lower the floor \
to pass GameKernel §5 AM-7 is a spec value and lowering it \
needs an ADR."
);
}
}