2026-07-31 01:57:13 +02:00
|
|
|
//! cb-sim — scenario runner binary (GameKernel K17; precursor of `cb sim`).
|
T08 iter 1: scenario runner executes; GROUND setup and Select step
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:14:34 +02:00
|
|
|
//! Parses scenario files, dispatches each to its game by the `<game>/`
|
|
|
|
|
//! prefix of its `scenario` field, and executes it. Exit codes: 0 all
|
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
|
|
|
//! passed, 1 any failure — including a scenario whose game prefix is not
|
|
|
|
|
//! registered, and a run in which nothing executed. There is deliberately
|
|
|
|
|
//! no "tolerable" non-zero exit: a silent skip is the failure mode this
|
|
|
|
|
//! binary exists to catch.
|
2026-07-31 01:57:13 +02:00
|
|
|
|
|
|
|
|
use cb_game_runtime::{scenario, RunOutcome, ScenarioFile};
|
T08 iter 1: scenario runner executes; GROUND setup and Select step
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:14:34 +02:00
|
|
|
use games_ground::GroundState;
|
2026-07-31 01:57:13 +02:00
|
|
|
|
|
|
|
|
fn main() {
|
|
|
|
|
let args: Vec<String> = std::env::args().skip(1).collect();
|
|
|
|
|
if args.is_empty() {
|
|
|
|
|
eprintln!("usage: cb-sim <scenario.yaml>...");
|
|
|
|
|
std::process::exit(64);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let mut failed = false;
|
T08 iter 1: scenario runner executes; GROUND setup and Select step
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:14:34 +02:00
|
|
|
let mut passed = 0usize;
|
|
|
|
|
let mut covered: Vec<String> = Vec::new();
|
2026-07-31 01:57:13 +02:00
|
|
|
|
|
|
|
|
for path in &args {
|
|
|
|
|
let text = match std::fs::read_to_string(path) {
|
|
|
|
|
Ok(t) => t,
|
|
|
|
|
Err(e) => {
|
|
|
|
|
eprintln!("{path}: read error: {e}");
|
|
|
|
|
failed = true;
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
};
|
T08 iter 1: scenario runner executes; GROUND setup and Select step
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:14:34 +02:00
|
|
|
let sc = match ScenarioFile::from_yaml(&text) {
|
2026-07-31 01:57:13 +02:00
|
|
|
Err(e) => {
|
|
|
|
|
eprintln!("{path}: parse error: {e}");
|
|
|
|
|
failed = true;
|
T08 iter 1: scenario runner executes; GROUND setup and Select step
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:14:34 +02:00
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
Ok(sc) => sc,
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
let outcome = match sc.scenario.split('/').next() {
|
|
|
|
|
Some("ground") => scenario::run::<GroundState>(&sc),
|
|
|
|
|
_ => {
|
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
|
|
|
// A renamed or typo'd prefix would otherwise skip every
|
|
|
|
|
// scenario while the run still looked clean.
|
|
|
|
|
eprintln!(
|
|
|
|
|
"FAIL {} — no game registered for prefix {:?}",
|
|
|
|
|
sc.scenario,
|
|
|
|
|
sc.scenario.split('/').next().unwrap_or("")
|
|
|
|
|
);
|
|
|
|
|
failed = true;
|
T08 iter 1: scenario runner executes; GROUND setup and Select step
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:14:34 +02:00
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
match outcome {
|
|
|
|
|
RunOutcome::Passed { covers } => {
|
|
|
|
|
println!(
|
|
|
|
|
"PASS {} covers={}{}",
|
|
|
|
|
sc.scenario,
|
|
|
|
|
covers.join(","),
|
|
|
|
|
if sc.provisional { " provisional" } else { "" }
|
|
|
|
|
);
|
|
|
|
|
covered.extend(covers);
|
|
|
|
|
passed += 1;
|
|
|
|
|
}
|
|
|
|
|
RunOutcome::Failed { reason } => {
|
|
|
|
|
println!("FAIL {} — {reason}", sc.scenario);
|
|
|
|
|
failed = true;
|
2026-07-31 01:57:13 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
T08 iter 1: scenario runner executes; GROUND setup and Select step
Replaces the RunOutcome::Unimplemented stub with a real runner:
- ScenarioGame trait: games own setup presets and the command
vocabulary, the runner owns execution, assertions, and determinism.
- K8 double-run: every scenario runs twice on the same seed and fails
on state-hash divergence.
- K4/K11: applied events go through Envelope into EventLog, so seq
monotonicity is enforced on the real path, not just in unit tests.
- setup.patch was parsed and silently dropped; the runner now applies
it generically and errors on a path that does not exist, so a typo
in a scenario can never pass as a no-op.
- Assertions: dot-path state lookup over objects and arrays, ordered
event subsequence matching by field subset, exact rejects-set match.
GROUND rules realized: GR-S01..S04 setup (seeded shuffle, deal, Lead,
Surface Problem face up), GR-R02 Select commit, GR-R03 stress gate and
Freedom spend, GR-A13 targeting legality.
cb-sim dispatches by the scenario's game prefix and reports rule
coverage. 3 scenarios pass, 7 rules covered; fmt/clippy/tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:14:34 +02:00
|
|
|
covered.sort();
|
|
|
|
|
covered.dedup();
|
|
|
|
|
println!("{passed} passed, {} rules covered", covered.len());
|
|
|
|
|
|
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
|
|
|
// Positive control: a run that executed nothing must not pass. This
|
|
|
|
|
// is the same class of error as a benchmark timing rejected work.
|
|
|
|
|
if passed == 0 {
|
|
|
|
|
eprintln!("FAIL — no scenario executed; refusing to report success");
|
|
|
|
|
failed = true;
|
|
|
|
|
}
|
|
|
|
|
|
2026-07-31 01:57:13 +02:00
|
|
|
if failed {
|
|
|
|
|
std::process::exit(1);
|
|
|
|
|
}
|
|
|
|
|
}
|