coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CI: enforce every gate; close the silent-skip holes

Browse source 
The gates existed; CI ran half of them and tolerated the failure case.

- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
  prefix is a failure, and a run in which nothing executed is a failure.
  Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
  would have skipped every scenario while the pipeline stayed green.
  Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
  added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
  split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
  crate-count metric.

The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.

That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.

Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.

The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 04:02:33 +02:00
parent 4be6e020ea
commit 72c594ee49
6 changed files with 169 additions and 70 deletions
Download patch file Download diff file Expand all files Collapse all files

24
.forgejo/workflows/ci.yml
View file

@ -14,5 +14,27 @@ jobs:
- run: rustup component add rustfmt clippy
- run: cargo fmt --all --check
- run: cargo clippy --workspace --all-targets -- -D warnings
# The shipped runtime is a distinct configuration (AM-4a); it must
# compile and lint on its own, or the feature split rots.
- run: cargo clippy -p games-ground --no-default-features -- -D warnings
- run: cargo test --workspace
- run: cargo run -q -p cb-sim -- scenarios/ground/*.yaml || test $? -eq 2
# No `|| test $? -eq 2`: cb-sim now fails on an unregistered game
# prefix and on a run that executed nothing. A silent skip is the
# failure this step exists to catch.
- run: cargo run -q -p cb-sim -- scenarios/ground/*.yaml
# AM-1: every numbered GR-rule is claimed by a scenario, and no
# scenario claims a rule the spec does not define.
- run: make coverage
# AM-4a/AM-4b: third-party source under audit, per configuration.
- run: make dep-weight
# InnerLoop v1.0 positive control, enforced rather than asserted in
# prose: --test runs every benchmark once, so a workload that
# stalls or produces the wrong event count fails the build instead
# of silently reporting throughput for work that never happened.
- run: cargo bench -p games-ground --bench synthetic -- --test