AM-4: gate scenario YAML, retarget on audited source, re-measure
Some checks failed
ci / check (push) Failing after 3s

Adopts both remediations from CB-EV-0001 §4 (maintainer decision).

Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.

A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.

Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.

Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:

  shipped runtime   23 crates   246,250 lines   target <=250,000  met
  dev toolchain     29 crates   317,021 lines   target <=350,000  met
  own source                      3,408 lines

Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.

Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 03:35:41 +02:00
parent 8e11fc412e
commit 4be6e020ea
12 changed files with 271 additions and 43 deletions

View file

@ -16,8 +16,8 @@ license-file = "LICENSE"
[workspace.dependencies] [workspace.dependencies]
cb-kernel = { path = "crates/cb-kernel" } cb-kernel = { path = "crates/cb-kernel" }
cb-events = { path = "crates/cb-events" } cb-events = { path = "crates/cb-events" }
cb-game-runtime = { path = "crates/cb-game-runtime" } cb-game-runtime = { path = "crates/cb-game-runtime", default-features = false }
games-ground = { path = "games/ground" } games-ground = { path = "games/ground", default-features = false }
serde = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] }
serde_json = "1" serde_json = "1"
serde_yaml = "0.9" serde_yaml = "0.9"

View file

@ -15,6 +15,9 @@ test:
$(CARGO) test --workspace $(CARGO) test --workspace
## run all GROUND scenarios through cb-sim ## run all GROUND scenarios through cb-sim
dep-weight:
python3 tools/dep-weight.py
coverage: coverage:
python3 tools/rule-coverage.py python3 tools/rule-coverage.py

View file

@ -9,6 +9,7 @@
| Kind | ID | Status | Lane | Source | | Kind | ID | Status | Lane | Source |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| workplan | CB-WP-0001 | done | — | workplans/CB-WP-0001-inner-loop.md | | workplan | CB-WP-0001 | done | — | workplans/CB-WP-0001-inner-loop.md |
| workplan | CB-WP-0002 | proposed | — | workplans/CB-WP-0002-cost-accounting.md |
| task | CB-WP-0001-T01 | done | — | workplans/CB-WP-0001-inner-loop.md | | task | CB-WP-0001-T01 | done | — | workplans/CB-WP-0001-inner-loop.md |
| task | CB-WP-0001-T02 | done | — | workplans/CB-WP-0001-inner-loop.md | | task | CB-WP-0001-T02 | done | — | workplans/CB-WP-0001-inner-loop.md |
| task | CB-WP-0001-T03 | done | — | workplans/CB-WP-0001-inner-loop.md | | task | CB-WP-0001-T03 | done | — | workplans/CB-WP-0001-inner-loop.md |
@ -18,3 +19,10 @@
| task | CB-WP-0001-T07 | done | — | workplans/CB-WP-0001-inner-loop.md | | task | CB-WP-0001-T07 | done | — | workplans/CB-WP-0001-inner-loop.md |
| task | CB-WP-0001-T08 | done | — | workplans/CB-WP-0001-inner-loop.md | | task | CB-WP-0001-T08 | done | — | workplans/CB-WP-0001-inner-loop.md |
| task | CB-WP-0001-T09 | done | — | workplans/CB-WP-0001-inner-loop.md | | task | CB-WP-0001-T09 | done | — | workplans/CB-WP-0001-inner-loop.md |
| task | CB-WP-0002-T01 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
| task | CB-WP-0002-T02 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
| task | CB-WP-0002-T03 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
| task | CB-WP-0002-T04 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
| task | CB-WP-0002-T05 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
| task | CB-WP-0002-T06 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
| task | CB-WP-0002-T07 | todo | — | workplans/CB-WP-0002-cost-accounting.md |

View file

@ -9,7 +9,14 @@ cb-kernel.workspace = true
cb-events.workspace = true cb-events.workspace = true
serde.workspace = true serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
serde_yaml.workspace = true serde_yaml = { workspace = true, optional = true }
[features]
# Scenario files are a test-and-tooling concern: a shipped game runtime
# parses no YAML. Default-on so `cargo test`/`cargo run` behave normally;
# measure the shipped runtime with --no-default-features (AM-4).
default = ["scenarios"]
scenarios = ["dep:serde_yaml"]
[lints] [lints]
workspace = true workspace = true

View file

@ -1,8 +1,10 @@
//! cb-game-runtime — round/phase machinery, commit windows, projections, //! cb-game-runtime — round/phase machinery, commit windows, projections,
//! and the scenario runner (GameKernel §2.5, §3). Game-agnostic. //! and the scenario runner (GameKernel §2.5, §3). Game-agnostic.
#[cfg(feature = "scenarios")]
pub mod scenario; pub mod scenario;
#[cfg(feature = "scenarios")]
pub use scenario::{parse_actor, run, CommandStep, RunOutcome, ScenarioFile, ScenarioGame, Setup}; pub use scenario::{parse_actor, run, CommandStep, RunOutcome, ScenarioFile, ScenarioGame, Setup};
use cb_kernel::PlayerId; use cb_kernel::PlayerId;

View file

@ -1,6 +1,6 @@
# CB-EV-0001 — GROUND game kernel: acceptance evidence # CB-EV-0001 — GROUND game kernel: acceptance evidence
Status: **T08 complete, with one acceptance metric not met (AM-4).** Status: **T08 complete. AM-4 remediated and re-measured 2026-07-31.**
Recorded: 2026-07-31. Amended 2026-07-31 — §4 gains measured savings per Recorded: 2026-07-31. Amended 2026-07-31 — §4 gains measured savings per
remediation option, and §5 corrects AM-12 from "uncomputable" to remediation option, and §5 corrects AM-12 from "uncomputable" to
measured-at-session-level; see CB-WP-0002. measured-at-session-level; see CB-WP-0002.
@ -17,7 +17,8 @@ Machine: WSL2, Linux 6.18.33.2-microsoft-standard-WSL2, rustc 1.97.1,
| Metric | Target | Measured | Verdict | | Metric | Target | Measured | Verdict |
|---|---|---|---| |---|---|---|---|
| AM-1 rule coverage | 100% of GR-rules | 58/58 (100%) | **met** | | AM-1 rule coverage | 100% of GR-rules | 58/58 (100%) | **met** |
| AM-4 dependency weight | ≤20 crates | 33 | **NOT MET** | | AM-4a dep weight, shipped runtime | ≤250,000 third-party lines | 246,250 (23 crates) | **met** |
| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** |
| AM-6 throughput | ≥100,000 events/s | 1,651,400 events/s | **met, 16.5×** | | AM-6 throughput | ≥100,000 events/s | 1,651,400 events/s | **met, 16.5×** |
| AM-7 scaling | ≥0.9× at 20× workload | 1.08× | **met** | | AM-7 scaling | ≥0.9× at 20× workload | 1.08× | **met** |
| AM-7 replay | 100k events ≤5s | 4.13 ms | **met, 1,210×** | | AM-7 replay | 100k events ≤5s | 4.13 ms | **met, 1,210×** |
@ -111,11 +112,44 @@ panics rather than measuring a stalled loop. The corrected figure is
aggregate holds only ordered collections, so iteration order cannot aggregate holds only ordered collections, so iteration order cannot
vary between runs. vary between runs.
## 4. AM-4 — not met, and why it is reported rather than fixed ## 4. AM-4 — remediated and re-measured
**33 transitive crates against a ≤20 target.** The baseline it was set **Original result: NOT MET, 33 transitive crates against a ≤20 target.**
against is boardgame.io's 120 npm packages, so we are 3.6× lighter, but Resolved by adopting both remediations (maintainer decision,
the metric as written is missed and is recorded as missed. 2026-07-31): `serde_yaml` was made optional, and the metric was
retargeted onto third-party source under audit.
### Re-measurement (`make dep-weight`)
| Configuration | Crates | Third-party LOC | Target | Verdict |
|---|---|---|---|---|
| Shipped runtime (`--no-default-features`) | 23 | 246,250 | ≤250,000 | **met** |
| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** |
| Our own source | — | 3,408 | — | — |
Scenario tooling costs **70,771 lines that a shipped game never
compiles**. That split is the substantive result: the single number
previously reported conflated a runtime concern with a test concern.
**What actually changed in the build.** `cb-game-runtime` gained a
`scenarios` feature carrying `serde_yaml`; the scenario module, the
`ScenarioGame` impl and the string parsers behind it are `#[cfg]`-gated.
Both configurations compile and lint clean under `-D warnings`.
One trap worth recording: setting `default-features = false` on a
*member* dependency is silently ignored when the workspace dependency
does not specify it, so the first attempt gated nothing while appearing
to work — `cargo tree` still showed all six YAML crates. The fix was
setting `default-features = false` on the workspace dependency itself,
with `cb-sim` opting into `scenarios` explicitly. This is exactly the
class of error InnerLoop v1.0's positive-control rule targets: the build
succeeded and the feature flag looked applied. It was caught by checking
the dependency graph rather than trusting that the edit had worked.
### Why the target moved, and why that is not moving the goalposts
The ≤20 crate target was retired for two measured reasons, both
recorded before the decision was taken:
Attribution: Attribution:
@ -128,35 +162,24 @@ Attribution:
| `rand_chacha` (K5 seeded RNG) | rand_chacha, rand_core, ppv-lite86, zerocopy | 4 | | `rand_chacha` (K5 seeded RNG) | rand_chacha, rand_core, ppv-lite86, zerocopy | 4 |
| Clay-Borg crates | cb-kernel, cb-events, cb-game-runtime, games-ground | 4 | | Clay-Borg crates | cb-kernel, cb-events, cb-game-runtime, games-ground | 4 |
The honest options, in order of preference: 1. **It was unreachable without undoing the spec's own contracts.**
Measured ladder: `serde_yaml` optional 6 (→27), dropping
`serde_json` 4 (→23), inlining SHA-256 8 (→19), inlining ChaCha12
4 (→15). Nothing reaches 20 except reimplementing a primitive that
K5 or K7 requires — trading an audited implementation for a
scoreboard number.
2. **Crate count does not compare across ecosystems.** Rust splits
crates far more finely than npm. The same granularity difference made
"33 vs 120 npm packages" flatter us *and* made ≤20 punish us.
1. **Make `serde_yaml` optional** behind a `scenarios` feature. YAML is Third-party source under audit is what the count was proxying for, is
a test-and-tooling concern; a shipped game runtime does not need it. comparable across ecosystems, and cannot be gamed by granularity. The
Removes 6 crates from the default build for no loss of capability new targets are set at roughly the current measurement plus headroom,
(`ryu` belongs to this group, not to serde_json, which uses `zmij` so they bind on future growth rather than retroactively passing
for floats — corrected after measuring the reverse-dependency graph). something that failed: adding another `serde_yaml`-sized dependency to
This is the one to do first, and it improves D4 optionality as well the shipped runtime would breach AM-4a.
as D2.
2. **Revisit the target, and what it measures.** Measured savings per
option: serde_yaml optional 6 (→27); replacing serde_json 4 more
(→23); inlining SHA-256 8 (→19); inlining ChaCha12 4 (→15). **Only
reimplementing SHA-256 or ChaCha gets under 20**, so the target is
unreachable without undoing K5/K7.
Crate count also compares badly across ecosystems: Rust splits What we did **not** do: hand-roll SHA-256 or ChaCha to win a count.
crates far more finely than npm, so "33 vs 120 npm packages" flatters
us. The measurable thing crate count proxies for is third-party
source under audit: **307,317 lines** across all five groups, against
3,398 of our own. Retargeting AM-4 on audited third-party LOC, split
into shipped-runtime and dev-toolchain, measures the real concern and
cannot be gamed by crate granularity.
What we are **not** doing: hand-rolling SHA-256 or ChaCha to win a
dependency count. That trades an auditable, well-tested primitive for a
number on a scoreboard.
Carried forward as an open decision (see the note at the head of this
file): AM-4 is re-measured once the option is chosen.
## 5. Cost log (AM-12) ## 5. Cost log (AM-12)

View file

@ -7,9 +7,14 @@ license-file.workspace = true
[dependencies] [dependencies]
cb-kernel.workspace = true cb-kernel.workspace = true
cb-events.workspace = true cb-events.workspace = true
cb-game-runtime.workspace = true cb-game-runtime = { workspace = true, default-features = false }
serde.workspace = true serde.workspace = true
[features]
default = ["scenarios"]
# The ScenarioGame impl exists only when scenarios are compiled in.
scenarios = ["cb-game-runtime/scenarios"]
[dev-dependencies] [dev-dependencies]
criterion.workspace = true criterion.workspace = true

View file

@ -2,6 +2,7 @@
//! GameKernel K15K16). Every rule realized here names its GR-id in a doc //! GameKernel K15K16). Every rule realized here names its GR-id in a doc
//! comment, giving a greppable rule→code→scenario chain. //! comment, giving a greppable rule→code→scenario chain.
#[cfg(feature = "scenarios")]
use cb_game_runtime::{parse_actor, CommandStep, ScenarioGame, Setup}; use cb_game_runtime::{parse_actor, CommandStep, ScenarioGame, Setup};
use cb_kernel::{Actor, Aggregate, ChaChaRng, KernelRng, PlayerId, Rejection, Seed}; use cb_kernel::{Actor, Aggregate, ChaChaRng, KernelRng, PlayerId, Rejection, Seed};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@ -235,6 +236,7 @@ impl GroundChoice {
} }
} }
#[cfg(feature = "scenarios")]
fn parse(raw: &str, arg: Option<u64>) -> Result<Self, String> { fn parse(raw: &str, arg: Option<u64>) -> Result<Self, String> {
let need = |what: &str| { let need = |what: &str| {
arg.ok_or_else(|| format!("GROUND choice {raw:?} needs a {what} argument")) arg.ok_or_else(|| format!("GROUND choice {raw:?} needs a {what} argument"))
@ -275,6 +277,7 @@ pub enum SupportResponse {
} }
impl SupportResponse { impl SupportResponse {
#[cfg(feature = "scenarios")]
fn parse(raw: &str) -> Result<Self, String> { fn parse(raw: &str) -> Result<Self, String> {
match raw { match raw {
"accept_bond" => Ok(SupportResponse::AcceptBond), "accept_bond" => Ok(SupportResponse::AcceptBond),
@ -287,6 +290,7 @@ impl SupportResponse {
} }
impl GroundMode { impl GroundMode {
#[cfg(feature = "scenarios")]
fn parse(raw: &str) -> Result<Self, String> { fn parse(raw: &str) -> Result<Self, String> {
match raw { match raw {
"GR" => Ok(GroundMode::Gr), "GR" => Ok(GroundMode::Gr),
@ -329,6 +333,7 @@ pub enum Action {
} }
impl Action { impl Action {
#[cfg(feature = "scenarios")]
fn parse(raw: &str) -> Result<Self, String> { fn parse(raw: &str) -> Result<Self, String> {
match raw { match raw {
"INVESTIGATE" => Ok(Action::Investigate), "INVESTIGATE" => Ok(Action::Investigate),
@ -1741,6 +1746,7 @@ impl GroundState {
} }
/// GR-S01: hidden-Problem priorities admitted per player count. /// GR-S01: hidden-Problem priorities admitted per player count.
#[cfg(feature = "scenarios")]
fn problem_priorities(players: u8) -> Result<u8, String> { fn problem_priorities(players: u8) -> Result<u8, String> {
match players { match players {
2 => Ok(2), 2 => Ok(2),
@ -1752,6 +1758,7 @@ fn problem_priorities(players: u8) -> Result<u8, String> {
/// GR-S04: the 24 core Solution cards, 6 per suit, in canonical order /// GR-S04: the 24 core Solution cards, 6 per suit, in canonical order
/// before the seeded shuffle. /// before the seeded shuffle.
#[cfg(feature = "scenarios")]
fn core_solution_deck() -> Vec<SolutionCard> { fn core_solution_deck() -> Vec<SolutionCard> {
[Suit::Clarify, Suit::Repair, Suit::Boundary, Suit::Change] [Suit::Clarify, Suit::Repair, Suit::Boundary, Suit::Change]
.into_iter() .into_iter()
@ -1759,6 +1766,7 @@ fn core_solution_deck() -> Vec<SolutionCard> {
.collect() .collect()
} }
#[cfg(feature = "scenarios")]
impl ScenarioGame for GroundState { impl ScenarioGame for GroundState {
/// GR-S01..S04. The `standard-Np` presets differ only in seat count; /// GR-S01..S04. The `standard-Np` presets differ only in seat count;
/// Problem content is scenario data, so the preset uses the canonical /// Problem content is scenario data, so the preset uses the canonical

View file

@ -26,8 +26,16 @@ and must never leak types into anything specified here (M-D4-LEAK = 0).
Dependency rule: `games/ground → cb-game-runtime → cb-events → cb-kernel`. Dependency rule: `games/ground → cb-game-runtime → cb-events → cb-kernel`.
No cycle, no skip that bypasses a public API. External crates allowed in No cycle, no skip that bypasses a public API. External crates allowed in
the headless kernel workspace: serde (+format crate), a seedable RNG the headless kernel workspace: serde (+format crate), a seedable RNG
(e.g. chacha), a hash (sha2), thiserror-class error derive — target total (e.g. chacha), a hash (sha2), thiserror-class error derive. Weight is
transitive crates ≤ 20 (AM-4). budgeted as **third-party source under audit**, split by build
configuration — see AM-4.
**Scenario parsing is dev-only.** `cb-game-runtime`'s `scenarios` feature
carries the YAML dependency; a shipped game runtime builds with
`--no-default-features` and parses no YAML. Workspace dependencies on
`cb-game-runtime` and `games-ground` therefore set
`default-features = false`, and consumers that need scenarios (currently
`cb-sim`) opt in explicitly.
## 2. Canonical model ## 2. Canonical model
@ -125,7 +133,23 @@ Command (actor-tagged intent)
--- ---
## 4. Acceptance metrics (the code loop's exit condition) ## 4. Acceptance metrics
**On AM-4's retarget (2026-07-31).** AM-4 originally read "≤20
transitive crates", set against boardgame.io's 120 npm packages. That
target was retired for two measured reasons. First, it was unreachable
without undoing this spec's own contracts: K5 (seeded ChaCha) and K7
(SHA-256) cost 12 crates between them, and the measured ladder showed
nothing reached 20 except reimplementing one of those primitives —
which trades an audited implementation for a scoreboard number. Second,
crate count does not compare across ecosystems: Rust splits crates far
more finely than npm, so the original 33-vs-120 comparison flattered us
while the ≤20 target punished us, both for the same reason.
Third-party source under audit is what the count was a proxy for, it is
comparable across ecosystems, and it cannot be gamed by crate
granularity. Splitting it by build configuration also makes the
dev/shipped distinction visible, which the single number hid. (the code loop's exit condition)
Per InnerLoop step 4/5: T08 iterates until every row meets its target; Per InnerLoop step 4/5: T08 iterates until every row meets its target;
evidence lands in `evidence/CB-EV-0001-game-kernel.md` with no evidence lands in `evidence/CB-EV-0001-game-kernel.md` with no
@ -136,7 +160,9 @@ evidence lands in `evidence/CB-EV-0001-game-kernel.md` with no
| AM-1 | M-D1-COV: GR-rules covered by ≥1 passing scenario | no candidate has any (observation) | **100%** of GR + U rules | measured by runner report | | AM-1 | M-D1-COV: GR-rules covered by ≥1 passing scenario | no candidate has any (observation) | **100%** of GR + U rules | measured by runner report |
| AM-2 | M-D1-SPL: spec lines per rule in `games/ground` rules code (impl LOC ÷ rule count) | boardgame.io ~36 LOC for the 2-move synthetic game | ≤ 40 LOC/rule, paired with AM-1 (anti-gaming pair) | measured (tokei + rule count) | | AM-2 | M-D1-SPL: spec lines per rule in `games/ground` rules code (impl LOC ÷ rule count) | boardgame.io ~36 LOC for the 2-move synthetic game | ≤ 40 LOC/rule, paired with AM-1 (anti-gaming pair) | measured (tokei + rule count) |
| AM-3 | Synthetic-workload definition size: LOC to express the CB-RES-0001 synthetic game on our kernel | ~36 LOC (boardgame.io, measured) | ≤ 50 LOC | measured | | AM-3 | Synthetic-workload definition size: LOC to express the CB-RES-0001 synthetic game on our kernel | ~36 LOC (boardgame.io, measured) | ≤ 50 LOC | measured |
| AM-4 | M-D2-DEP: transitive crates, headless workspace | 120 npm packages | **≤ 20** | measured (cargo tree) | | AM-4a | M-D2-DEP: third-party LOC, **shipped runtime** (`--no-default-features`) | boardgame.io: 120 npm packages / 3.9M LOC | **≤ 250,000 lines** | measured (`make dep-weight`) |
| AM-4b | M-D2-DEP: third-party LOC, **dev toolchain** (default features) | as above | **≤ 350,000 lines** | measured (`make dep-weight`) |
| AM-4c | M-D2-DEP: own source per third-party 100k lines | — | reported, not targeted | measured (`make dep-weight`) |
| AM-5 | M-D2-BLD: clean release build of headless workspace | n/a (npm install ~seconds; not comparable) | ≤ 60 s on bnt-lap001, recorded not gated | measured | | AM-5 | M-D2-BLD: clean release build of headless workspace | n/a (npm install ~seconds; not comparable) | ≤ 60 s on bnt-lap001, recorded not gated | measured |
| AM-6 | M-D3-THR: applied events/s, synthetic workload, same machine | boardgame.io ~1,1001,900 moves/s (best config, degrading) | **≥ 100,000/s** (stipulated target, ADR-0002) | measured | | AM-6 | M-D3-THR: applied events/s, synthetic workload, same machine | boardgame.io ~1,1001,900 moves/s (best config, degrading) | **≥ 100,000/s** (stipulated target, ADR-0002) | measured |
| AM-7 | M-D3 scaling: throughput @100k events vs @5k; and snapshot+replay of 100k events | boardgame.io 0.450.66× @2040k, DNF @100k | **≥ 0.9×** (flat), replay of 100k events ≤ 5 s, hash-identical | measured | | AM-7 | M-D3 scaling: throughput @100k events vs @5k; and snapshot+replay of 100k events | boardgame.io 0.450.66× @2040k, DNF @100k | **≥ 0.9×** (flat), replay of 100k events ≤ 5 s, hash-identical | measured |

View file

@ -5,8 +5,8 @@ version.workspace = true
license-file.workspace = true license-file.workspace = true
[dependencies] [dependencies]
cb-game-runtime.workspace = true cb-game-runtime = { workspace = true, features = ["scenarios"] }
games-ground.workspace = true games-ground = { workspace = true, features = ["scenarios"] }
[lints] [lints]
workspace = true workspace = true

138
tools/dep-weight.py Executable file
View file

@ -0,0 +1,138 @@
#!/usr/bin/env python3
"""AM-4: third-party dependency weight, measured as source under audit.
Crate count is a poor cross-ecosystem proxy Rust splits crates far more
finely than npm, so "33 crates vs 120 npm packages" flatters us in one
direction and a low crate-count target punishes us in the other. What the
count stands in for is how much third-party source a reviewer would have
to audit. This measures that directly, in two configurations:
shipped-runtime cargo build --no-default-features (what a game ships)
dev-toolchain cargo build (adds scenario YAML)
Positive control (InnerLoop v1.0 §Step 5): every crate in the dependency
graph must be located on disk and produce a non-zero line count. A crate
that cannot be found is reported and the run exits non-zero rather than
silently under-reporting the total under-reporting is the exact
direction this metric could be gamed.
Usage: python3 tools/dep-weight.py [--json]
"""
import glob
import json
import os
import shutil
import subprocess
import sys
PACKAGE = "games-ground"
CONFIGS = {
"shipped-runtime": ["--no-default-features"],
"dev-toolchain": [],
}
def crates(extra_args):
"""Third-party crates in the normal (non-dev) dependency graph."""
if not shutil.which("cargo"):
print(
"ERROR: cargo not on PATH. Try: export PATH=\"$HOME/.cargo/bin:$PATH\"",
file=sys.stderr,
)
sys.exit(1)
out = subprocess.run(
["cargo", "tree", "-p", PACKAGE, "--edges", "normal", "--prefix", "none"]
+ extra_args,
capture_output=True,
text=True,
check=True,
).stdout
found = {}
for line in out.splitlines():
parts = line.split()
if len(parts) < 2 or not parts[1].startswith("v"):
continue
name, version = parts[0], parts[1].lstrip("v")
# Path dependencies are our own code, not third-party.
if "(/" in line:
continue
found[name] = version
return found
def source_lines(name, version):
"""Lines of Rust in the vendored source for one crate."""
roots = glob.glob(os.path.expanduser("~/.cargo/registry/src/*/"))
for root in roots:
# Version may carry a build suffix (e.g. 0.9.34+deprecated).
for d in glob.glob(f"{root}{name}-{version}*/") + glob.glob(f"{root}{name}-*/"):
total = 0
for dirpath, _, files in os.walk(d):
for f in files:
if f.endswith(".rs"):
try:
with open(os.path.join(dirpath, f), "rb") as fh:
total += fh.read().count(b"\n")
except OSError:
pass
if total:
return total
return 0
def main():
report = {}
missing = []
for label, args in CONFIGS.items():
found = crates(args)
per_crate = {}
for name, version in sorted(found.items()):
lines = source_lines(name, version)
if lines == 0:
missing.append(f"{name} {version} ({label})")
per_crate[name] = lines
report[label] = {
"crates": len(found),
"third_party_loc": sum(per_crate.values()),
"per_crate": per_crate,
}
own = 0
for base in ("crates", "games", "tools"):
for dirpath, _, files in os.walk(base):
if "target" in dirpath.split(os.sep):
continue
for f in files:
if f.endswith(".rs"):
with open(os.path.join(dirpath, f), "rb") as fh:
own += fh.read().count(b"\n")
report["own_loc"] = own
if "--json" in sys.argv:
print(json.dumps(report, indent=2))
else:
print("AM-4 dependency weight")
print(f" own source {own:>9,} lines")
for label in CONFIGS:
r = report[label]
print(
f" {label:<18}{r['crates']:>3} crates "
f"{r['third_party_loc']:>9,} lines third-party"
)
delta = (
report["dev-toolchain"]["third_party_loc"]
- report["shipped-runtime"]["third_party_loc"]
)
print(f" scenario tooling costs {delta:>9,} lines (dev only)")
if missing:
# Positive control: a crate we could not measure would silently
# shrink the total, so refuse to report rather than under-report.
print("\nERROR — source not found for:", ", ".join(missing), file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())

View file

@ -2,6 +2,7 @@
id: CB-WP-0002 id: CB-WP-0002
title: "Make agentic cost measurable, so D2 claims are falsifiable" title: "Make agentic cost measurable, so D2 claims are falsifiable"
status: proposed status: proposed
state_hub_workstream_id: "b7c22f69-fbe9-48df-9619-007db79ae338"
--- ---
# Purpose # Purpose
@ -45,6 +46,7 @@ positive control.
id: CB-WP-0002-T01 id: CB-WP-0002-T01
status: todo status: todo
priority: high priority: high
state_hub_task_id: "2694c2c1-0070-4d8e-b4fc-196b582b36d5"
``` ```
Produce `research/CB-RES-0002-cost-accounting.md` per the InnerLoop Produce `research/CB-RES-0002-cost-accounting.md` per the InnerLoop
@ -67,6 +69,7 @@ accuracy and the hub to lead on durability.
id: CB-WP-0002-T02 id: CB-WP-0002-T02
status: todo status: todo
priority: high priority: high
state_hub_task_id: "eae248ab-f29f-4f11-9d20-e8145b0d822d"
``` ```
Adversarial review of T01 first (InnerLoop §Step 2), committed as Adversarial review of T01 first (InnerLoop §Step 2), committed as
@ -92,6 +95,7 @@ Gate: no collector code before this ADR is committed.
id: CB-WP-0002-T03 id: CB-WP-0002-T03
status: todo status: todo
priority: high priority: high
state_hub_task_id: "00d42ed2-4391-4580-aae2-06e3e151c69b"
``` ```
Write `specs/CostAccounting.md`: the cost model (input, output, cache Write `specs/CostAccounting.md`: the cost model (input, output, cache
@ -114,6 +118,7 @@ replace AM-12's definition with one that is computable.
id: CB-WP-0002-T04 id: CB-WP-0002-T04
status: todo status: todo
priority: medium priority: medium
state_hub_task_id: "9eb8329b-5f41-477b-8cf3-2cda5ba8dbe8"
``` ```
Implement the tool chosen in T02 (expected: `tools/cb-cost`). It reads Implement the tool chosen in T02 (expected: `tools/cb-cost`). It reads
@ -137,6 +142,7 @@ each message at its own model's rate.
id: CB-WP-0002-T05 id: CB-WP-0002-T05
status: todo status: todo
priority: medium priority: medium
state_hub_task_id: "bea4cc0a-e4d0-4077-9dc7-df7726a48f86"
``` ```
Run the collector over the CB-WP-0001 session and commit Run the collector over the CB-WP-0001 session and commit
@ -160,6 +166,7 @@ cleared the bar that CB-WP-0001's AM-12 failed to clear.
id: CB-WP-0002-T06 id: CB-WP-0002-T06
status: todo status: todo
priority: low priority: low
state_hub_task_id: "1412263b-70c1-43e4-957d-1ad6c3203ca9"
``` ```
Make cost collection automatic rather than remembered: add the Make cost collection automatic rather than remembered: add the
@ -174,6 +181,7 @@ the one command surface.
id: CB-WP-0002-T07 id: CB-WP-0002-T07
status: todo status: todo
priority: low priority: low
state_hub_task_id: "ebe58d91-be5f-4d5b-ba40-b03275b4eefc"
``` ```
Revise `specs/InnerLoop.md` and `specs/MetricsAndScenarios.md` from what Revise `specs/InnerLoop.md` and `specs/MetricsAndScenarios.md` from what