AM-4: gate scenario YAML, retarget on audited source, re-measure
Some checks failed
ci / check (push) Failing after 3s
Some checks failed
ci / check (push) Failing after 3s
Adopts both remediations from CB-EV-0001 §4 (maintainer decision). Option A — serde_yaml is now optional behind cb-game-runtime's `scenarios` feature. The scenario module, the ScenarioGame impl and the string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both configurations compile and lint clean under -D warnings. A trap worth recording: `default-features = false` on a *member* dependency is silently ignored when the workspace dependency does not specify it. The first attempt gated nothing while looking correct — the build succeeded and cargo tree still showed all six YAML crates. Fixed by setting it on the workspace dependency. This is the positive-control failure mode in miniature: success was not evidence the change applied. Retarget — AM-4 now measures third-party source under audit, split by build configuration, replacing a crate count that was unreachable without undoing K5/K7 and that does not compare across ecosystems. Re-measured via the new `make dep-weight`, whose own positive control refuses to report when any crate's source cannot be located: shipped runtime 23 crates 246,250 lines target <=250,000 met dev toolchain 29 crates 317,021 lines target <=350,000 met own source 3,408 lines Scenario tooling costs 70,771 lines a shipped game never compiles — the split the single number was hiding. Targets are set at current measurement plus headroom, so they bind on future growth rather than retroactively passing what had failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
8e11fc412e
commit
4be6e020ea
12 changed files with 271 additions and 43 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# CB-EV-0001 — GROUND game kernel: acceptance evidence
|
||||
|
||||
Status: **T08 complete, with one acceptance metric not met (AM-4).**
|
||||
Status: **T08 complete. AM-4 remediated and re-measured 2026-07-31.**
|
||||
Recorded: 2026-07-31. Amended 2026-07-31 — §4 gains measured savings per
|
||||
remediation option, and §5 corrects AM-12 from "uncomputable" to
|
||||
measured-at-session-level; see CB-WP-0002.
|
||||
|
|
@ -17,7 +17,8 @@ Machine: WSL2, Linux 6.18.33.2-microsoft-standard-WSL2, rustc 1.97.1,
|
|||
| Metric | Target | Measured | Verdict |
|
||||
|---|---|---|---|
|
||||
| AM-1 rule coverage | 100% of GR-rules | 58/58 (100%) | **met** |
|
||||
| AM-4 dependency weight | ≤20 crates | 33 | **NOT MET** |
|
||||
| AM-4a dep weight, shipped runtime | ≤250,000 third-party lines | 246,250 (23 crates) | **met** |
|
||||
| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** |
|
||||
| AM-6 throughput | ≥100,000 events/s | 1,651,400 events/s | **met, 16.5×** |
|
||||
| AM-7 scaling | ≥0.9× at 20× workload | 1.08× | **met** |
|
||||
| AM-7 replay | 100k events ≤5s | 4.13 ms | **met, 1,210×** |
|
||||
|
|
@ -111,11 +112,44 @@ panics rather than measuring a stalled loop. The corrected figure is
|
|||
aggregate holds only ordered collections, so iteration order cannot
|
||||
vary between runs.
|
||||
|
||||
## 4. AM-4 — not met, and why it is reported rather than fixed
|
||||
## 4. AM-4 — remediated and re-measured
|
||||
|
||||
**33 transitive crates against a ≤20 target.** The baseline it was set
|
||||
against is boardgame.io's 120 npm packages, so we are 3.6× lighter, but
|
||||
the metric as written is missed and is recorded as missed.
|
||||
**Original result: NOT MET, 33 transitive crates against a ≤20 target.**
|
||||
Resolved by adopting both remediations (maintainer decision,
|
||||
2026-07-31): `serde_yaml` was made optional, and the metric was
|
||||
retargeted onto third-party source under audit.
|
||||
|
||||
### Re-measurement (`make dep-weight`)
|
||||
|
||||
| Configuration | Crates | Third-party LOC | Target | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| Shipped runtime (`--no-default-features`) | 23 | 246,250 | ≤250,000 | **met** |
|
||||
| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** |
|
||||
| Our own source | — | 3,408 | — | — |
|
||||
|
||||
Scenario tooling costs **70,771 lines that a shipped game never
|
||||
compiles**. That split is the substantive result: the single number
|
||||
previously reported conflated a runtime concern with a test concern.
|
||||
|
||||
**What actually changed in the build.** `cb-game-runtime` gained a
|
||||
`scenarios` feature carrying `serde_yaml`; the scenario module, the
|
||||
`ScenarioGame` impl and the string parsers behind it are `#[cfg]`-gated.
|
||||
Both configurations compile and lint clean under `-D warnings`.
|
||||
|
||||
One trap worth recording: setting `default-features = false` on a
|
||||
*member* dependency is silently ignored when the workspace dependency
|
||||
does not specify it, so the first attempt gated nothing while appearing
|
||||
to work — `cargo tree` still showed all six YAML crates. The fix was
|
||||
setting `default-features = false` on the workspace dependency itself,
|
||||
with `cb-sim` opting into `scenarios` explicitly. This is exactly the
|
||||
class of error InnerLoop v1.0's positive-control rule targets: the build
|
||||
succeeded and the feature flag looked applied. It was caught by checking
|
||||
the dependency graph rather than trusting that the edit had worked.
|
||||
|
||||
### Why the target moved, and why that is not moving the goalposts
|
||||
|
||||
The ≤20 crate target was retired for two measured reasons, both
|
||||
recorded before the decision was taken:
|
||||
|
||||
Attribution:
|
||||
|
||||
|
|
@ -128,35 +162,24 @@ Attribution:
|
|||
| `rand_chacha` (K5 seeded RNG) | rand_chacha, rand_core, ppv-lite86, zerocopy | 4 |
|
||||
| Clay-Borg crates | cb-kernel, cb-events, cb-game-runtime, games-ground | 4 |
|
||||
|
||||
The honest options, in order of preference:
|
||||
1. **It was unreachable without undoing the spec's own contracts.**
|
||||
Measured ladder: `serde_yaml` optional −6 (→27), dropping
|
||||
`serde_json` −4 (→23), inlining SHA-256 −8 (→19), inlining ChaCha12
|
||||
−4 (→15). Nothing reaches 20 except reimplementing a primitive that
|
||||
K5 or K7 requires — trading an audited implementation for a
|
||||
scoreboard number.
|
||||
2. **Crate count does not compare across ecosystems.** Rust splits
|
||||
crates far more finely than npm. The same granularity difference made
|
||||
"33 vs 120 npm packages" flatter us *and* made ≤20 punish us.
|
||||
|
||||
1. **Make `serde_yaml` optional** behind a `scenarios` feature. YAML is
|
||||
a test-and-tooling concern; a shipped game runtime does not need it.
|
||||
Removes 6 crates from the default build for no loss of capability
|
||||
(`ryu` belongs to this group, not to serde_json, which uses `zmij`
|
||||
for floats — corrected after measuring the reverse-dependency graph).
|
||||
This is the one to do first, and it improves D4 optionality as well
|
||||
as D2.
|
||||
2. **Revisit the target, and what it measures.** Measured savings per
|
||||
option: serde_yaml optional −6 (→27); replacing serde_json −4 more
|
||||
(→23); inlining SHA-256 −8 (→19); inlining ChaCha12 −4 (→15). **Only
|
||||
reimplementing SHA-256 or ChaCha gets under 20**, so the target is
|
||||
unreachable without undoing K5/K7.
|
||||
Third-party source under audit is what the count was proxying for, is
|
||||
comparable across ecosystems, and cannot be gamed by granularity. The
|
||||
new targets are set at roughly the current measurement plus headroom,
|
||||
so they bind on future growth rather than retroactively passing
|
||||
something that failed: adding another `serde_yaml`-sized dependency to
|
||||
the shipped runtime would breach AM-4a.
|
||||
|
||||
Crate count also compares badly across ecosystems: Rust splits
|
||||
crates far more finely than npm, so "33 vs 120 npm packages" flatters
|
||||
us. The measurable thing crate count proxies for is third-party
|
||||
source under audit: **307,317 lines** across all five groups, against
|
||||
3,398 of our own. Retargeting AM-4 on audited third-party LOC, split
|
||||
into shipped-runtime and dev-toolchain, measures the real concern and
|
||||
cannot be gamed by crate granularity.
|
||||
|
||||
What we are **not** doing: hand-rolling SHA-256 or ChaCha to win a
|
||||
dependency count. That trades an auditable, well-tested primitive for a
|
||||
number on a scoreboard.
|
||||
|
||||
Carried forward as an open decision (see the note at the head of this
|
||||
file): AM-4 is re-measured once the option is chosen.
|
||||
What we did **not** do: hand-roll SHA-256 or ChaCha to win a count.
|
||||
|
||||
## 5. Cost log (AM-12)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue