CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had no flag parsing at all, so --replay had nowhere to go. The bundle is manifest + commands.log + initial.snapshot + expected.yaml, dev-only behind the scenarios feature and charged to AM-4b. The command stream goes through the K11 framing built in T05, so a truncated bundle is detected rather than replayed short — the two tasks compose rather than duplicating. The reviewer's D2 correction was real: this was not "a directory of four files". Pass carried only the end state, RunOutcome::Failed was a formatted String, and scenario.rs created an EventLog, appended to it and never read it. All three had to change. The first round trip failed to reproduce, and the cause is worth keeping: state_hash_hex over a serde_json::Value is a different canonical form than over the typed aggregate — Value's map is key-sorted, a struct serializes in declaration order. The bundle was written with one basis and verified with the other. A round trip written to recompute its own comparison value would have PASSED this bug; it failed because the recorded hash came from the producing process, which is control 2's entire purpose. make replay-test implements ADR-0005 §6's four controls, 14/14: a committed deliberately-failing fixture outside the corpus with covers: [] so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash must fail; a log short by one byte and a corrupted length prefix must be rejected; and a mutated manifest seed must fail — which bites only because replay re-derives the initial state from seed+setup and checks it against the recorded snapshot, since restoring from the snapshot alone would leave the seed inert. Plus a control on the controls: the bundle must still replay after every mutation is reverted. AM-7's hash-identical clause is re-earned. The probe records a hash per per-game segment and replays each from its own genesis; folding from the wrong seed now fails. That is the clause ADR-0005 §4 withdrew as mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so AM-7 stays PARTIAL — reported, not rounded up. Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's copy of that number going stale, on a number that moved the same hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
6037467478
commit
1edadac9a2
17 changed files with 721 additions and 35 deletions
|
|
@ -6,16 +6,30 @@
|
|||
//! no "tolerable" non-zero exit: a silent skip is the failure mode this
|
||||
//! binary exists to catch.
|
||||
|
||||
use cb_game_runtime::{scenario, RunOutcome, ScenarioFile};
|
||||
use cb_game_runtime::{replay, scenario, RunOutcome, ScenarioFile};
|
||||
use games_ground::GroundState;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// Where a failing run drops its `.cbreplay` bundle
|
||||
/// (MetricsAndScenarios §2).
|
||||
const REPLAY_DIR: &str = "replays";
|
||||
|
||||
fn main() {
|
||||
let args: Vec<String> = std::env::args().skip(1).collect();
|
||||
if args.is_empty() {
|
||||
let argv: Vec<String> = std::env::args().skip(1).collect();
|
||||
if argv.is_empty() {
|
||||
eprintln!("usage: cb-sim <scenario.yaml>...");
|
||||
eprintln!(" cb-sim --replay <bundle.cbreplay>...");
|
||||
std::process::exit(64);
|
||||
}
|
||||
|
||||
// K10: until CB-WP-0006 T06 this binary had no flag parsing at all, so
|
||||
// `--replay` had nowhere to go — every argument was treated as a
|
||||
// scenario path.
|
||||
if argv[0] == "--replay" {
|
||||
std::process::exit(replay_bundles(&argv[1..]));
|
||||
}
|
||||
let args = argv;
|
||||
|
||||
let mut failed = false;
|
||||
let mut passed = 0usize;
|
||||
let mut covered: Vec<String> = Vec::new();
|
||||
|
|
@ -64,8 +78,28 @@ fn main() {
|
|||
covered.extend(covers);
|
||||
passed += 1;
|
||||
}
|
||||
RunOutcome::Failed { reason } => {
|
||||
RunOutcome::Failed { reason, evidence } => {
|
||||
println!("FAIL {} — {reason}", sc.scenario);
|
||||
// K10: a failure becomes work an agent can start, not just
|
||||
// information (MetricsAndScenarios §4).
|
||||
if let Some(ev) = evidence {
|
||||
match std::fs::create_dir_all(REPLAY_DIR)
|
||||
.map_err(|e| e.to_string())
|
||||
.and_then(|()| {
|
||||
replay::write_bundle(
|
||||
Path::new(REPLAY_DIR),
|
||||
&sc,
|
||||
&ev.initial,
|
||||
&ev.initial_hash,
|
||||
&ev.end_state,
|
||||
&ev.end_state_hash,
|
||||
&reason,
|
||||
)
|
||||
}) {
|
||||
Ok(path) => println!(" bundle {}", path.display()),
|
||||
Err(e) => eprintln!(" bundle NOT written: {e}"),
|
||||
}
|
||||
}
|
||||
failed = true;
|
||||
}
|
||||
}
|
||||
|
|
@ -86,3 +120,38 @@ fn main() {
|
|||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
/// `cb-sim --replay <bundle>...` — re-execute recorded bundles.
|
||||
fn replay_bundles(paths: &[String]) -> i32 {
|
||||
if paths.is_empty() {
|
||||
eprintln!("usage: cb-sim --replay <bundle.cbreplay>...");
|
||||
return 64;
|
||||
}
|
||||
let mut ok = 0usize;
|
||||
let mut bad = false;
|
||||
for p in paths {
|
||||
let bundle = PathBuf::from(p);
|
||||
match replay::replay::<GroundState>(&bundle) {
|
||||
Ok(r) => {
|
||||
println!(
|
||||
"REPLAY {} — {} commands, hash {} reproduced",
|
||||
r.scenario,
|
||||
r.commands,
|
||||
&r.hash[..12]
|
||||
);
|
||||
ok += 1;
|
||||
}
|
||||
Err(e) => {
|
||||
println!("REPLAY FAIL {} — {e}", bundle.display());
|
||||
bad = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Same positive control as the scenario path: a run that replayed
|
||||
// nothing must not report success.
|
||||
if ok == 0 {
|
||||
eprintln!("FAIL — no bundle replayed; refusing to report success");
|
||||
bad = true;
|
||||
}
|
||||
i32::from(bad)
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue