CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case. - cb-sim no longer has a "tolerable" non-zero exit. An unregistered game prefix is a failure, and a run in which nothing executed is a failure. Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix would have skipped every scenario while the pipeline stayed green. Verified with a negative control. - CI now runs make coverage (AM-1) and make dep-weight (AM-4), both added after CI was written and neither enforced until now. - dep-weight enforces its targets instead of only reporting them. - CI lints the shipped-runtime configuration separately, so the feature split cannot rot unnoticed. - Dropped the stale `make deps` target, which still measured the retired crate-count metric. The positive-control rule is now executable: CI runs `cargo bench -- --test`, which executes every benchmark once, so a workload that stalls fails the build. That step immediately found a fourth instance of the error class it was written for. The committed replay benchmark was the broken version — an earlier patch never applied, leaving a command sequence that omits Resolve, so every round produced nothing and the log-building loop spun forever. It had never run to completion; the reported AM-7 replay numbers came from a probe test instead. Fixed, given the same positive control as the round loop, and re-measured from the benchmark: 100k events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget. Evidence now reports confidence intervals rather than point estimates, so the 3% regression rule in MetricsAndScenarios is enforceable. The finding worth carrying: writing the positive-control rule into InnerLoop v1.0 did not prevent the next instance. Making it a CI step did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
4be6e020ea
commit
72c594ee49
6 changed files with 169 additions and 70 deletions
14
Makefile
14
Makefile
|
|
@ -3,7 +3,7 @@
|
|||
|
||||
CARGO := cargo
|
||||
|
||||
.PHONY: check test sim bench deps loc all
|
||||
.PHONY: check test sim bench bench-test coverage dep-weight loc all
|
||||
|
||||
## fmt + clippy (deny warnings) + HashMap deny-lint
|
||||
check:
|
||||
|
|
@ -24,13 +24,15 @@ coverage:
|
|||
sim:
|
||||
$(CARGO) run -q -p cb-sim -- scenarios/ground/*.yaml
|
||||
|
||||
## Criterion benches (AM-6/AM-7 wiring)
|
||||
## Criterion benches (AM-6/AM-7)
|
||||
bench:
|
||||
$(CARGO) bench -p games-ground
|
||||
|
||||
## AM-4: transitive crate count (excludes dev/build deps)
|
||||
deps:
|
||||
@$(CARGO) tree --workspace -e normal --prefix none | sed 's/ (\*)//' | sort -u | grep -vE '^(cb-|games-|cb_|$$)' | tee /dev/stderr | wc -l
|
||||
## InnerLoop positive control: run every bench once, no measurement.
|
||||
## Fails if a workload stalls or produces the wrong event count.
|
||||
bench-test:
|
||||
$(CARGO) bench -p games-ground --bench synthetic -- --test
|
||||
|
||||
|
||||
## AM-2/AM-3 input: source LOC per crate (excludes tests would need tokei)
|
||||
loc:
|
||||
|
|
@ -38,4 +40,4 @@ loc:
|
|||
printf '%-28s %s\n' $$d "$$(find $$d/src -name '*.rs' | xargs cat | grep -vcE '^\s*(//|$$)')"; \
|
||||
done
|
||||
|
||||
all: check test sim
|
||||
all: check test sim coverage dep-weight bench-test
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue