AM-4: gate scenario YAML, retarget on audited source, re-measure
Some checks failed
ci / check (push) Failing after 3s
Some checks failed
ci / check (push) Failing after 3s
Adopts both remediations from CB-EV-0001 §4 (maintainer decision). Option A — serde_yaml is now optional behind cb-game-runtime's `scenarios` feature. The scenario module, the ScenarioGame impl and the string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both configurations compile and lint clean under -D warnings. A trap worth recording: `default-features = false` on a *member* dependency is silently ignored when the workspace dependency does not specify it. The first attempt gated nothing while looking correct — the build succeeded and cargo tree still showed all six YAML crates. Fixed by setting it on the workspace dependency. This is the positive-control failure mode in miniature: success was not evidence the change applied. Retarget — AM-4 now measures third-party source under audit, split by build configuration, replacing a crate count that was unreachable without undoing K5/K7 and that does not compare across ecosystems. Re-measured via the new `make dep-weight`, whose own positive control refuses to report when any crate's source cannot be located: shipped runtime 23 crates 246,250 lines target <=250,000 met dev toolchain 29 crates 317,021 lines target <=350,000 met own source 3,408 lines Scenario tooling costs 70,771 lines a shipped game never compiles — the split the single number was hiding. Targets are set at current measurement plus headroom, so they bind on future growth rather than retroactively passing what had failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
8e11fc412e
commit
4be6e020ea
12 changed files with 271 additions and 43 deletions
|
|
@ -16,8 +16,8 @@ license-file = "LICENSE"
|
|||
[workspace.dependencies]
|
||||
cb-kernel = { path = "crates/cb-kernel" }
|
||||
cb-events = { path = "crates/cb-events" }
|
||||
cb-game-runtime = { path = "crates/cb-game-runtime" }
|
||||
games-ground = { path = "games/ground" }
|
||||
cb-game-runtime = { path = "crates/cb-game-runtime", default-features = false }
|
||||
games-ground = { path = "games/ground", default-features = false }
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
serde_yaml = "0.9"
|
||||
|
|
|
|||
3
Makefile
3
Makefile
|
|
@ -15,6 +15,9 @@ test:
|
|||
$(CARGO) test --workspace
|
||||
|
||||
## run all GROUND scenarios through cb-sim
|
||||
dep-weight:
|
||||
python3 tools/dep-weight.py
|
||||
|
||||
coverage:
|
||||
python3 tools/rule-coverage.py
|
||||
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@
|
|||
| Kind | ID | Status | Lane | Source |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| workplan | CB-WP-0001 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||
| workplan | CB-WP-0002 | proposed | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
| task | CB-WP-0001-T01 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||
| task | CB-WP-0001-T02 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||
| task | CB-WP-0001-T03 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||
|
|
@ -18,3 +19,10 @@
|
|||
| task | CB-WP-0001-T07 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||
| task | CB-WP-0001-T08 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||
| task | CB-WP-0001-T09 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||
| task | CB-WP-0002-T01 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
| task | CB-WP-0002-T02 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
| task | CB-WP-0002-T03 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
| task | CB-WP-0002-T04 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
| task | CB-WP-0002-T05 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
| task | CB-WP-0002-T06 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
| task | CB-WP-0002-T07 | todo | — | workplans/CB-WP-0002-cost-accounting.md |
|
||||
|
|
|
|||
|
|
@ -9,7 +9,14 @@ cb-kernel.workspace = true
|
|||
cb-events.workspace = true
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
serde_yaml.workspace = true
|
||||
serde_yaml = { workspace = true, optional = true }
|
||||
|
||||
[features]
|
||||
# Scenario files are a test-and-tooling concern: a shipped game runtime
|
||||
# parses no YAML. Default-on so `cargo test`/`cargo run` behave normally;
|
||||
# measure the shipped runtime with --no-default-features (AM-4).
|
||||
default = ["scenarios"]
|
||||
scenarios = ["dep:serde_yaml"]
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
|
|
|
|||
|
|
@ -1,8 +1,10 @@
|
|||
//! cb-game-runtime — round/phase machinery, commit windows, projections,
|
||||
//! and the scenario runner (GameKernel §2.5, §3). Game-agnostic.
|
||||
|
||||
#[cfg(feature = "scenarios")]
|
||||
pub mod scenario;
|
||||
|
||||
#[cfg(feature = "scenarios")]
|
||||
pub use scenario::{parse_actor, run, CommandStep, RunOutcome, ScenarioFile, ScenarioGame, Setup};
|
||||
|
||||
use cb_kernel::PlayerId;
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# CB-EV-0001 — GROUND game kernel: acceptance evidence
|
||||
|
||||
Status: **T08 complete, with one acceptance metric not met (AM-4).**
|
||||
Status: **T08 complete. AM-4 remediated and re-measured 2026-07-31.**
|
||||
Recorded: 2026-07-31. Amended 2026-07-31 — §4 gains measured savings per
|
||||
remediation option, and §5 corrects AM-12 from "uncomputable" to
|
||||
measured-at-session-level; see CB-WP-0002.
|
||||
|
|
@ -17,7 +17,8 @@ Machine: WSL2, Linux 6.18.33.2-microsoft-standard-WSL2, rustc 1.97.1,
|
|||
| Metric | Target | Measured | Verdict |
|
||||
|---|---|---|---|
|
||||
| AM-1 rule coverage | 100% of GR-rules | 58/58 (100%) | **met** |
|
||||
| AM-4 dependency weight | ≤20 crates | 33 | **NOT MET** |
|
||||
| AM-4a dep weight, shipped runtime | ≤250,000 third-party lines | 246,250 (23 crates) | **met** |
|
||||
| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** |
|
||||
| AM-6 throughput | ≥100,000 events/s | 1,651,400 events/s | **met, 16.5×** |
|
||||
| AM-7 scaling | ≥0.9× at 20× workload | 1.08× | **met** |
|
||||
| AM-7 replay | 100k events ≤5s | 4.13 ms | **met, 1,210×** |
|
||||
|
|
@ -111,11 +112,44 @@ panics rather than measuring a stalled loop. The corrected figure is
|
|||
aggregate holds only ordered collections, so iteration order cannot
|
||||
vary between runs.
|
||||
|
||||
## 4. AM-4 — not met, and why it is reported rather than fixed
|
||||
## 4. AM-4 — remediated and re-measured
|
||||
|
||||
**33 transitive crates against a ≤20 target.** The baseline it was set
|
||||
against is boardgame.io's 120 npm packages, so we are 3.6× lighter, but
|
||||
the metric as written is missed and is recorded as missed.
|
||||
**Original result: NOT MET, 33 transitive crates against a ≤20 target.**
|
||||
Resolved by adopting both remediations (maintainer decision,
|
||||
2026-07-31): `serde_yaml` was made optional, and the metric was
|
||||
retargeted onto third-party source under audit.
|
||||
|
||||
### Re-measurement (`make dep-weight`)
|
||||
|
||||
| Configuration | Crates | Third-party LOC | Target | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| Shipped runtime (`--no-default-features`) | 23 | 246,250 | ≤250,000 | **met** |
|
||||
| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** |
|
||||
| Our own source | — | 3,408 | — | — |
|
||||
|
||||
Scenario tooling costs **70,771 lines that a shipped game never
|
||||
compiles**. That split is the substantive result: the single number
|
||||
previously reported conflated a runtime concern with a test concern.
|
||||
|
||||
**What actually changed in the build.** `cb-game-runtime` gained a
|
||||
`scenarios` feature carrying `serde_yaml`; the scenario module, the
|
||||
`ScenarioGame` impl and the string parsers behind it are `#[cfg]`-gated.
|
||||
Both configurations compile and lint clean under `-D warnings`.
|
||||
|
||||
One trap worth recording: setting `default-features = false` on a
|
||||
*member* dependency is silently ignored when the workspace dependency
|
||||
does not specify it, so the first attempt gated nothing while appearing
|
||||
to work — `cargo tree` still showed all six YAML crates. The fix was
|
||||
setting `default-features = false` on the workspace dependency itself,
|
||||
with `cb-sim` opting into `scenarios` explicitly. This is exactly the
|
||||
class of error InnerLoop v1.0's positive-control rule targets: the build
|
||||
succeeded and the feature flag looked applied. It was caught by checking
|
||||
the dependency graph rather than trusting that the edit had worked.
|
||||
|
||||
### Why the target moved, and why that is not moving the goalposts
|
||||
|
||||
The ≤20 crate target was retired for two measured reasons, both
|
||||
recorded before the decision was taken:
|
||||
|
||||
Attribution:
|
||||
|
||||
|
|
@ -128,35 +162,24 @@ Attribution:
|
|||
| `rand_chacha` (K5 seeded RNG) | rand_chacha, rand_core, ppv-lite86, zerocopy | 4 |
|
||||
| Clay-Borg crates | cb-kernel, cb-events, cb-game-runtime, games-ground | 4 |
|
||||
|
||||
The honest options, in order of preference:
|
||||
1. **It was unreachable without undoing the spec's own contracts.**
|
||||
Measured ladder: `serde_yaml` optional −6 (→27), dropping
|
||||
`serde_json` −4 (→23), inlining SHA-256 −8 (→19), inlining ChaCha12
|
||||
−4 (→15). Nothing reaches 20 except reimplementing a primitive that
|
||||
K5 or K7 requires — trading an audited implementation for a
|
||||
scoreboard number.
|
||||
2. **Crate count does not compare across ecosystems.** Rust splits
|
||||
crates far more finely than npm. The same granularity difference made
|
||||
"33 vs 120 npm packages" flatter us *and* made ≤20 punish us.
|
||||
|
||||
1. **Make `serde_yaml` optional** behind a `scenarios` feature. YAML is
|
||||
a test-and-tooling concern; a shipped game runtime does not need it.
|
||||
Removes 6 crates from the default build for no loss of capability
|
||||
(`ryu` belongs to this group, not to serde_json, which uses `zmij`
|
||||
for floats — corrected after measuring the reverse-dependency graph).
|
||||
This is the one to do first, and it improves D4 optionality as well
|
||||
as D2.
|
||||
2. **Revisit the target, and what it measures.** Measured savings per
|
||||
option: serde_yaml optional −6 (→27); replacing serde_json −4 more
|
||||
(→23); inlining SHA-256 −8 (→19); inlining ChaCha12 −4 (→15). **Only
|
||||
reimplementing SHA-256 or ChaCha gets under 20**, so the target is
|
||||
unreachable without undoing K5/K7.
|
||||
Third-party source under audit is what the count was proxying for, is
|
||||
comparable across ecosystems, and cannot be gamed by granularity. The
|
||||
new targets are set at roughly the current measurement plus headroom,
|
||||
so they bind on future growth rather than retroactively passing
|
||||
something that failed: adding another `serde_yaml`-sized dependency to
|
||||
the shipped runtime would breach AM-4a.
|
||||
|
||||
Crate count also compares badly across ecosystems: Rust splits
|
||||
crates far more finely than npm, so "33 vs 120 npm packages" flatters
|
||||
us. The measurable thing crate count proxies for is third-party
|
||||
source under audit: **307,317 lines** across all five groups, against
|
||||
3,398 of our own. Retargeting AM-4 on audited third-party LOC, split
|
||||
into shipped-runtime and dev-toolchain, measures the real concern and
|
||||
cannot be gamed by crate granularity.
|
||||
|
||||
What we are **not** doing: hand-rolling SHA-256 or ChaCha to win a
|
||||
dependency count. That trades an auditable, well-tested primitive for a
|
||||
number on a scoreboard.
|
||||
|
||||
Carried forward as an open decision (see the note at the head of this
|
||||
file): AM-4 is re-measured once the option is chosen.
|
||||
What we did **not** do: hand-roll SHA-256 or ChaCha to win a count.
|
||||
|
||||
## 5. Cost log (AM-12)
|
||||
|
||||
|
|
|
|||
|
|
@ -7,9 +7,14 @@ license-file.workspace = true
|
|||
[dependencies]
|
||||
cb-kernel.workspace = true
|
||||
cb-events.workspace = true
|
||||
cb-game-runtime.workspace = true
|
||||
cb-game-runtime = { workspace = true, default-features = false }
|
||||
serde.workspace = true
|
||||
|
||||
[features]
|
||||
default = ["scenarios"]
|
||||
# The ScenarioGame impl exists only when scenarios are compiled in.
|
||||
scenarios = ["cb-game-runtime/scenarios"]
|
||||
|
||||
[dev-dependencies]
|
||||
criterion.workspace = true
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
//! GameKernel K15–K16). Every rule realized here names its GR-id in a doc
|
||||
//! comment, giving a greppable rule→code→scenario chain.
|
||||
|
||||
#[cfg(feature = "scenarios")]
|
||||
use cb_game_runtime::{parse_actor, CommandStep, ScenarioGame, Setup};
|
||||
use cb_kernel::{Actor, Aggregate, ChaChaRng, KernelRng, PlayerId, Rejection, Seed};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
|
@ -235,6 +236,7 @@ impl GroundChoice {
|
|||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "scenarios")]
|
||||
fn parse(raw: &str, arg: Option<u64>) -> Result<Self, String> {
|
||||
let need = |what: &str| {
|
||||
arg.ok_or_else(|| format!("GROUND choice {raw:?} needs a {what} argument"))
|
||||
|
|
@ -275,6 +277,7 @@ pub enum SupportResponse {
|
|||
}
|
||||
|
||||
impl SupportResponse {
|
||||
#[cfg(feature = "scenarios")]
|
||||
fn parse(raw: &str) -> Result<Self, String> {
|
||||
match raw {
|
||||
"accept_bond" => Ok(SupportResponse::AcceptBond),
|
||||
|
|
@ -287,6 +290,7 @@ impl SupportResponse {
|
|||
}
|
||||
|
||||
impl GroundMode {
|
||||
#[cfg(feature = "scenarios")]
|
||||
fn parse(raw: &str) -> Result<Self, String> {
|
||||
match raw {
|
||||
"GR" => Ok(GroundMode::Gr),
|
||||
|
|
@ -329,6 +333,7 @@ pub enum Action {
|
|||
}
|
||||
|
||||
impl Action {
|
||||
#[cfg(feature = "scenarios")]
|
||||
fn parse(raw: &str) -> Result<Self, String> {
|
||||
match raw {
|
||||
"INVESTIGATE" => Ok(Action::Investigate),
|
||||
|
|
@ -1741,6 +1746,7 @@ impl GroundState {
|
|||
}
|
||||
|
||||
/// GR-S01: hidden-Problem priorities admitted per player count.
|
||||
#[cfg(feature = "scenarios")]
|
||||
fn problem_priorities(players: u8) -> Result<u8, String> {
|
||||
match players {
|
||||
2 => Ok(2),
|
||||
|
|
@ -1752,6 +1758,7 @@ fn problem_priorities(players: u8) -> Result<u8, String> {
|
|||
|
||||
/// GR-S04: the 24 core Solution cards, 6 per suit, in canonical order
|
||||
/// before the seeded shuffle.
|
||||
#[cfg(feature = "scenarios")]
|
||||
fn core_solution_deck() -> Vec<SolutionCard> {
|
||||
[Suit::Clarify, Suit::Repair, Suit::Boundary, Suit::Change]
|
||||
.into_iter()
|
||||
|
|
@ -1759,6 +1766,7 @@ fn core_solution_deck() -> Vec<SolutionCard> {
|
|||
.collect()
|
||||
}
|
||||
|
||||
#[cfg(feature = "scenarios")]
|
||||
impl ScenarioGame for GroundState {
|
||||
/// GR-S01..S04. The `standard-Np` presets differ only in seat count;
|
||||
/// Problem content is scenario data, so the preset uses the canonical
|
||||
|
|
|
|||
|
|
@ -26,8 +26,16 @@ and must never leak types into anything specified here (M-D4-LEAK = 0).
|
|||
Dependency rule: `games/ground → cb-game-runtime → cb-events → cb-kernel`.
|
||||
No cycle, no skip that bypasses a public API. External crates allowed in
|
||||
the headless kernel workspace: serde (+format crate), a seedable RNG
|
||||
(e.g. chacha), a hash (sha2), thiserror-class error derive — target total
|
||||
transitive crates ≤ 20 (AM-4).
|
||||
(e.g. chacha), a hash (sha2), thiserror-class error derive. Weight is
|
||||
budgeted as **third-party source under audit**, split by build
|
||||
configuration — see AM-4.
|
||||
|
||||
**Scenario parsing is dev-only.** `cb-game-runtime`'s `scenarios` feature
|
||||
carries the YAML dependency; a shipped game runtime builds with
|
||||
`--no-default-features` and parses no YAML. Workspace dependencies on
|
||||
`cb-game-runtime` and `games-ground` therefore set
|
||||
`default-features = false`, and consumers that need scenarios (currently
|
||||
`cb-sim`) opt in explicitly.
|
||||
|
||||
## 2. Canonical model
|
||||
|
||||
|
|
@ -125,7 +133,23 @@ Command (actor-tagged intent)
|
|||
|
||||
---
|
||||
|
||||
## 4. Acceptance metrics (the code loop's exit condition)
|
||||
## 4. Acceptance metrics
|
||||
|
||||
**On AM-4's retarget (2026-07-31).** AM-4 originally read "≤20
|
||||
transitive crates", set against boardgame.io's 120 npm packages. That
|
||||
target was retired for two measured reasons. First, it was unreachable
|
||||
without undoing this spec's own contracts: K5 (seeded ChaCha) and K7
|
||||
(SHA-256) cost 12 crates between them, and the measured ladder showed
|
||||
nothing reached 20 except reimplementing one of those primitives —
|
||||
which trades an audited implementation for a scoreboard number. Second,
|
||||
crate count does not compare across ecosystems: Rust splits crates far
|
||||
more finely than npm, so the original 33-vs-120 comparison flattered us
|
||||
while the ≤20 target punished us, both for the same reason.
|
||||
|
||||
Third-party source under audit is what the count was a proxy for, it is
|
||||
comparable across ecosystems, and it cannot be gamed by crate
|
||||
granularity. Splitting it by build configuration also makes the
|
||||
dev/shipped distinction visible, which the single number hid. (the code loop's exit condition)
|
||||
|
||||
Per InnerLoop step 4/5: T08 iterates until every row meets its target;
|
||||
evidence lands in `evidence/CB-EV-0001-game-kernel.md` with no
|
||||
|
|
@ -136,7 +160,9 @@ evidence lands in `evidence/CB-EV-0001-game-kernel.md` with no
|
|||
| AM-1 | M-D1-COV: GR-rules covered by ≥1 passing scenario | no candidate has any (observation) | **100%** of GR + U rules | measured by runner report |
|
||||
| AM-2 | M-D1-SPL: spec lines per rule in `games/ground` rules code (impl LOC ÷ rule count) | boardgame.io ~36 LOC for the 2-move synthetic game | ≤ 40 LOC/rule, paired with AM-1 (anti-gaming pair) | measured (tokei + rule count) |
|
||||
| AM-3 | Synthetic-workload definition size: LOC to express the CB-RES-0001 synthetic game on our kernel | ~36 LOC (boardgame.io, measured) | ≤ 50 LOC | measured |
|
||||
| AM-4 | M-D2-DEP: transitive crates, headless workspace | 120 npm packages | **≤ 20** | measured (cargo tree) |
|
||||
| AM-4a | M-D2-DEP: third-party LOC, **shipped runtime** (`--no-default-features`) | boardgame.io: 120 npm packages / 3.9M LOC | **≤ 250,000 lines** | measured (`make dep-weight`) |
|
||||
| AM-4b | M-D2-DEP: third-party LOC, **dev toolchain** (default features) | as above | **≤ 350,000 lines** | measured (`make dep-weight`) |
|
||||
| AM-4c | M-D2-DEP: own source per third-party 100k lines | — | reported, not targeted | measured (`make dep-weight`) |
|
||||
| AM-5 | M-D2-BLD: clean release build of headless workspace | n/a (npm install ~seconds; not comparable) | ≤ 60 s on bnt-lap001, recorded not gated | measured |
|
||||
| AM-6 | M-D3-THR: applied events/s, synthetic workload, same machine | boardgame.io ~1,100–1,900 moves/s (best config, degrading) | **≥ 100,000/s** (stipulated target, ADR-0002) | measured |
|
||||
| AM-7 | M-D3 scaling: throughput @100k events vs @5k; and snapshot+replay of 100k events | boardgame.io 0.45–0.66× @20–40k, DNF @100k | **≥ 0.9×** (flat), replay of 100k events ≤ 5 s, hash-identical | measured |
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ version.workspace = true
|
|||
license-file.workspace = true
|
||||
|
||||
[dependencies]
|
||||
cb-game-runtime.workspace = true
|
||||
games-ground.workspace = true
|
||||
cb-game-runtime = { workspace = true, features = ["scenarios"] }
|
||||
games-ground = { workspace = true, features = ["scenarios"] }
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
|
|
|
|||
138
tools/dep-weight.py
Executable file
138
tools/dep-weight.py
Executable file
|
|
@ -0,0 +1,138 @@
|
|||
#!/usr/bin/env python3
|
||||
"""AM-4: third-party dependency weight, measured as source under audit.
|
||||
|
||||
Crate count is a poor cross-ecosystem proxy — Rust splits crates far more
|
||||
finely than npm, so "33 crates vs 120 npm packages" flatters us in one
|
||||
direction and a low crate-count target punishes us in the other. What the
|
||||
count stands in for is how much third-party source a reviewer would have
|
||||
to audit. This measures that directly, in two configurations:
|
||||
|
||||
shipped-runtime cargo build --no-default-features (what a game ships)
|
||||
dev-toolchain cargo build (adds scenario YAML)
|
||||
|
||||
Positive control (InnerLoop v1.0 §Step 5): every crate in the dependency
|
||||
graph must be located on disk and produce a non-zero line count. A crate
|
||||
that cannot be found is reported and the run exits non-zero rather than
|
||||
silently under-reporting the total — under-reporting is the exact
|
||||
direction this metric could be gamed.
|
||||
|
||||
Usage: python3 tools/dep-weight.py [--json]
|
||||
"""
|
||||
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
PACKAGE = "games-ground"
|
||||
CONFIGS = {
|
||||
"shipped-runtime": ["--no-default-features"],
|
||||
"dev-toolchain": [],
|
||||
}
|
||||
|
||||
|
||||
def crates(extra_args):
|
||||
"""Third-party crates in the normal (non-dev) dependency graph."""
|
||||
if not shutil.which("cargo"):
|
||||
print(
|
||||
"ERROR: cargo not on PATH. Try: export PATH=\"$HOME/.cargo/bin:$PATH\"",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
out = subprocess.run(
|
||||
["cargo", "tree", "-p", PACKAGE, "--edges", "normal", "--prefix", "none"]
|
||||
+ extra_args,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
).stdout
|
||||
found = {}
|
||||
for line in out.splitlines():
|
||||
parts = line.split()
|
||||
if len(parts) < 2 or not parts[1].startswith("v"):
|
||||
continue
|
||||
name, version = parts[0], parts[1].lstrip("v")
|
||||
# Path dependencies are our own code, not third-party.
|
||||
if "(/" in line:
|
||||
continue
|
||||
found[name] = version
|
||||
return found
|
||||
|
||||
|
||||
def source_lines(name, version):
|
||||
"""Lines of Rust in the vendored source for one crate."""
|
||||
roots = glob.glob(os.path.expanduser("~/.cargo/registry/src/*/"))
|
||||
for root in roots:
|
||||
# Version may carry a build suffix (e.g. 0.9.34+deprecated).
|
||||
for d in glob.glob(f"{root}{name}-{version}*/") + glob.glob(f"{root}{name}-*/"):
|
||||
total = 0
|
||||
for dirpath, _, files in os.walk(d):
|
||||
for f in files:
|
||||
if f.endswith(".rs"):
|
||||
try:
|
||||
with open(os.path.join(dirpath, f), "rb") as fh:
|
||||
total += fh.read().count(b"\n")
|
||||
except OSError:
|
||||
pass
|
||||
if total:
|
||||
return total
|
||||
return 0
|
||||
|
||||
|
||||
def main():
|
||||
report = {}
|
||||
missing = []
|
||||
for label, args in CONFIGS.items():
|
||||
found = crates(args)
|
||||
per_crate = {}
|
||||
for name, version in sorted(found.items()):
|
||||
lines = source_lines(name, version)
|
||||
if lines == 0:
|
||||
missing.append(f"{name} {version} ({label})")
|
||||
per_crate[name] = lines
|
||||
report[label] = {
|
||||
"crates": len(found),
|
||||
"third_party_loc": sum(per_crate.values()),
|
||||
"per_crate": per_crate,
|
||||
}
|
||||
|
||||
own = 0
|
||||
for base in ("crates", "games", "tools"):
|
||||
for dirpath, _, files in os.walk(base):
|
||||
if "target" in dirpath.split(os.sep):
|
||||
continue
|
||||
for f in files:
|
||||
if f.endswith(".rs"):
|
||||
with open(os.path.join(dirpath, f), "rb") as fh:
|
||||
own += fh.read().count(b"\n")
|
||||
report["own_loc"] = own
|
||||
|
||||
if "--json" in sys.argv:
|
||||
print(json.dumps(report, indent=2))
|
||||
else:
|
||||
print("AM-4 dependency weight")
|
||||
print(f" own source {own:>9,} lines")
|
||||
for label in CONFIGS:
|
||||
r = report[label]
|
||||
print(
|
||||
f" {label:<18}{r['crates']:>3} crates "
|
||||
f"{r['third_party_loc']:>9,} lines third-party"
|
||||
)
|
||||
delta = (
|
||||
report["dev-toolchain"]["third_party_loc"]
|
||||
- report["shipped-runtime"]["third_party_loc"]
|
||||
)
|
||||
print(f" scenario tooling costs {delta:>9,} lines (dev only)")
|
||||
|
||||
if missing:
|
||||
# Positive control: a crate we could not measure would silently
|
||||
# shrink the total, so refuse to report rather than under-report.
|
||||
print("\nERROR — source not found for:", ", ".join(missing), file=sys.stderr)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -2,6 +2,7 @@
|
|||
id: CB-WP-0002
|
||||
title: "Make agentic cost measurable, so D2 claims are falsifiable"
|
||||
status: proposed
|
||||
state_hub_workstream_id: "b7c22f69-fbe9-48df-9619-007db79ae338"
|
||||
---
|
||||
|
||||
# Purpose
|
||||
|
|
@ -45,6 +46,7 @@ positive control.
|
|||
id: CB-WP-0002-T01
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "2694c2c1-0070-4d8e-b4fc-196b582b36d5"
|
||||
```
|
||||
|
||||
Produce `research/CB-RES-0002-cost-accounting.md` per the InnerLoop
|
||||
|
|
@ -67,6 +69,7 @@ accuracy and the hub to lead on durability.
|
|||
id: CB-WP-0002-T02
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "eae248ab-f29f-4f11-9d20-e8145b0d822d"
|
||||
```
|
||||
|
||||
Adversarial review of T01 first (InnerLoop §Step 2), committed as
|
||||
|
|
@ -92,6 +95,7 @@ Gate: no collector code before this ADR is committed.
|
|||
id: CB-WP-0002-T03
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "00d42ed2-4391-4580-aae2-06e3e151c69b"
|
||||
```
|
||||
|
||||
Write `specs/CostAccounting.md`: the cost model (input, output, cache
|
||||
|
|
@ -114,6 +118,7 @@ replace AM-12's definition with one that is computable.
|
|||
id: CB-WP-0002-T04
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "9eb8329b-5f41-477b-8cf3-2cda5ba8dbe8"
|
||||
```
|
||||
|
||||
Implement the tool chosen in T02 (expected: `tools/cb-cost`). It reads
|
||||
|
|
@ -137,6 +142,7 @@ each message at its own model's rate.
|
|||
id: CB-WP-0002-T05
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "bea4cc0a-e4d0-4077-9dc7-df7726a48f86"
|
||||
```
|
||||
|
||||
Run the collector over the CB-WP-0001 session and commit
|
||||
|
|
@ -160,6 +166,7 @@ cleared the bar that CB-WP-0001's AM-12 failed to clear.
|
|||
id: CB-WP-0002-T06
|
||||
status: todo
|
||||
priority: low
|
||||
state_hub_task_id: "1412263b-70c1-43e4-957d-1ad6c3203ca9"
|
||||
```
|
||||
|
||||
Make cost collection automatic rather than remembered: add the
|
||||
|
|
@ -174,6 +181,7 @@ the one command surface.
|
|||
id: CB-WP-0002-T07
|
||||
status: todo
|
||||
priority: low
|
||||
state_hub_task_id: "ebe58d91-be5f-4d5b-ba40-b03275b4eefc"
|
||||
```
|
||||
|
||||
Revise `specs/InnerLoop.md` and `specs/MetricsAndScenarios.md` from what
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue