Provenance (tier S, one paragraph in lieu of survey and ADR): the two
clauses mutation-check has reported inert since CB-WP-0005. AM-7's
scaling ratio was held up by a test literally named
replay_100k_events_is_linear_and_fast that computed both throughputs,
printed both, and never divided one by the other. AM-8's N=10 was held
up by a runner that does two.
Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's
>=10-of-14 prediction MET for the first time. Neither was closed by
amending the question away, which was the live risk: the denominator is
unchanged and the four unenforced rows are the four already
unenforceable.
AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's,
correct for a floor on one number) gave 0.581-1.085 on an unchanged
binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at
fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU
contention while absolute throughput fell 4x. The INDETERMINATE guard
demanded unanimity and failed a good measurement over one sample
0.001 under the floor; it now requires a two-thirds majority. The
control that matters: AM-6's constant-cost mutation halves throughput
and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6.
AM-8 kept N=10 because the measurement said so. Perturbing the RNG only
from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A
late-onset divergence is deterministic, not flaky, so it is a control
rather than a coin flip. Ten runs live on one scenario (make am8, ~2s)
rather than all 25 (47s a build). GameKernel 5b records it.
The full run also found AM-4a's own mutation stale since ADR-0008 D3
moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported
HARNESS-BROKEN, no score published. The build-free half of that check
is now a --self-test assertion, so make all catches the next one.
mutation-check clauses may now carry their own verify and mutation, and
then the enforced flag is measured rather than declared; a declaration
disagreeing with its measurement is refused.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had
no flag parsing at all, so --replay had nowhere to go.
The bundle is manifest + commands.log + initial.snapshot + expected.yaml,
dev-only behind the scenarios feature and charged to AM-4b. The command
stream goes through the K11 framing built in T05, so a truncated bundle is
detected rather than replayed short — the two tasks compose rather than
duplicating.
The reviewer's D2 correction was real: this was not "a directory of four
files". Pass carried only the end state, RunOutcome::Failed was a
formatted String, and scenario.rs created an EventLog, appended to it and
never read it. All three had to change.
The first round trip failed to reproduce, and the cause is worth keeping:
state_hash_hex over a serde_json::Value is a different canonical form than
over the typed aggregate — Value's map is key-sorted, a struct serializes
in declaration order. The bundle was written with one basis and verified
with the other. A round trip written to recompute its own comparison value
would have PASSED this bug; it failed because the recorded hash came from
the producing process, which is control 2's entire purpose.
make replay-test implements ADR-0005 §6's four controls, 14/14: a
committed deliberately-failing fixture outside the corpus with covers: []
so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash
must fail; a log short by one byte and a corrupted length prefix must be
rejected; and a mutated manifest seed must fail — which bites only because
replay re-derives the initial state from seed+setup and checks it against
the recorded snapshot, since restoring from the snapshot alone would leave
the seed inert. Plus a control on the controls: the bundle must still
replay after every mutation is reverted.
AM-7's hash-identical clause is re-earned. The probe records a hash per
per-game segment and replays each from its own genesis; folding from the
wrong seed now fails. That is the clause ADR-0005 §4 withdrew as
mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so
AM-7 stays PARTIAL — reported, not rounded up.
Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's
copy of that number going stale, on a number that moved the same hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
