diff --git a/decisions/ADR-0004-am4-ratification.md b/decisions/ADR-0004-am4-ratification.md index 86cd83a..ed508da 100644 --- a/decisions/ADR-0004-am4-ratification.md +++ b/decisions/ADR-0004-am4-ratification.md @@ -1,3 +1,11 @@ +> **Superseded in part, 2026-08-03 (CB-WP-0019 T01/T02).** This ADR +> ratified AM-4b at ≤350,000 against a graph measuring 317,021 — but that +> graph was **one package with no dev edges**, blind to 28 crates and +> 408,237 lines. AM-4b now asks *what a contributor acquires* +> (`--workspace --edges normal,dev`, proc-macros counted) and its target is +> **745,000**. The reasoning here still stands; the scope it was applied to +> did not. See `specs/GameKernel.md` §5c. + # ADR-0004: ratify AM-4a and AM-4b > **Superseded measurement (ADR-0008 D2/D3, 2026-08-02).** The AM-4a @@ -58,7 +66,7 @@ At the time of the retarget, `make dep-weight`: | configuration | crates | third-party LOC | |---|---|---| | shipped-runtime (`--no-default-features`) | 23 | **246,250** | -| dev-toolchain (default features) | 29 | **317,021** | +| dev-toolchain (default features) | 29 | **317,021** | | own source | — | 3,443 | ## Why these targets bind on future work rather than merely passing present work diff --git a/evidence/CB-EV-0001-game-kernel.md b/evidence/CB-EV-0001-game-kernel.md index 55bdc46..cc9f21f 100644 --- a/evidence/CB-EV-0001-game-kernel.md +++ b/evidence/CB-EV-0001-game-kernel.md @@ -37,7 +37,7 @@ the property is false? A row can be *measured* and still enforce nothing. | AM-1b link, ground | 58 claimed rules named in the aggregate | 49/58 | **unmet** | reported | | AM-1b link, kernel | 18 K-rules named in source | 15/18 | **unmet** | reported until 2026-08-31 | | AM-4a dep weight, shipped runtime | ≤250,000 third-party lines | 246,250 (23 crates) | **met** | **yes** | -| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** | **yes** | +| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** | **yes** | | AM-6 throughput | ≥100,000 events/s | 2,017,009 events/s (`make am6`) | **met, 20.2×** | **yes** — CB-WP-0006 T01 | | AM-7 scaling | ≥0.9× at 20× workload | 1.08× | **met** | **no** — no code computes the ratio | | AM-7 replay, timing | 100k events ≤5s | 2.18 ms (CI 2.14–2.23) | **met, 2,290×** | **yes** | @@ -203,7 +203,7 @@ retargeted onto third-party source under audit. | Configuration | Crates | Third-party LOC | Target | Verdict | |---|---|---|---|---| | Shipped runtime (`--no-default-features`) | 23 | 246,250 | ≤250,000 | **met** | -| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** | +| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** | | Our own source | — | 3,408 | — | — | Scenario tooling costs **70,771 lines that a shipped game never diff --git a/facts.toml b/facts.toml index 113778d..bc2eac8 100644 --- a/facts.toml +++ b/facts.toml @@ -6,7 +6,7 @@ # `make facts-check` fails if this file disagrees with the # instruments, or if a tagged artifact disagrees with this file. -generated = "2026-08-02" +generated = "2026-08-03" pin = "fc76445" [am4a_loc] @@ -22,14 +22,14 @@ fmt = "{:,}" by = "tools/dep-weight.py TARGETS" [am4b_loc] -value = 317021 -text = "317,021" +value = 725258 +text = "725,258" fmt = "{:,}" by = "tools/dep-weight.py" [am4b_target] -value = 350000 -text = "350,000" +value = 745000 +text = "745,000" fmt = "{:,}" by = "tools/dep-weight.py TARGETS" diff --git a/specs/GameKernel.md b/specs/GameKernel.md index 24661d3..cf1e179 100644 --- a/specs/GameKernel.md +++ b/specs/GameKernel.md @@ -205,7 +205,7 @@ evidence lands in `evidence/CB-EV-0001-game-kernel.md` with no | AM-2 | M-D1-SPL: spec lines per rule in `games/ground` rules code (impl LOC ÷ rule count) | boardgame.io ~36 LOC for the 2-move synthetic game | ≤ 40 LOC/rule, paired with AM-1 (anti-gaming pair) | measured (tokei + rule count) | | AM-3 | Synthetic-workload definition size: LOC to express the CB-RES-0001 synthetic game on our kernel | ~36 LOC (boardgame.io, measured) | ≤ 50 LOC | measured | | AM-4a | M-D2-DEP: third-party LOC, **shipped runtime** (`--no-default-features --edges normal,no-proc-macro`) | boardgame.io: 120 npm packages / 3.9M LOC | **≤ 161,000 lines** (ADR-0008 D3, was 250,000) | measured (`make dep-weight`) | -| AM-4b | M-D2-DEP: third-party LOC, **dev toolchain** (default features) | as above | **≤ 350,000 lines** — *proc-macro share unmeasured; correction owed (ADR-0008 D2)* | measured (`make dep-weight`) | +| AM-4b | M-D2-DEP: third-party LOC, **what a contributor acquires** (`--workspace --edges normal,dev`) | as above | **≤ 745,000 lines** (CB-WP-0019; was 350,000 against a graph that measured 317,021 of the real 725,258) | measured (`make dep-weight`) — see §5c | | ~~AM-4c~~ | M-D2-DEP: own source per third-party 100k lines | — | **WITHDRAWN from the acceptance table 2026-08-01 (CB-WP-0006 T04)** — retained as a reported diagnostic in `make dep-weight`; see §5a | diagnostic | | AM-5 | M-D2-BLD: clean release build of headless workspace | n/a (npm install ~seconds; not comparable) | ≤ 60 s on bnt-lap001, recorded not gated | measured | | AM-6 | M-D3-THR: applied events/s, synthetic workload, same machine | boardgame.io ~1,100–1,900 moves/s (best config, degrading) | **≥ 100,000/s** (stipulated target, ADR-0002) | measured | @@ -257,6 +257,41 @@ Comparisons against the event-sourcing 10⁵–10⁶/s estimate stay **parity** until a local Rust comparator is measured (open follow-up from the adversarial review). +### 5c. What each AM-4 budget asks, and why they differ + +*(CB-WP-0019 T01/T02, 2026-08-03. Tier M.)* + +The two budgets had the **same scope** — one package, no dev edges — while +claiming to bound different things. That left AM-4b blind to **28 crates +and 408,237 lines**, more source than its own target, and it is how +`quick-js` entered in CB-WP-0014 without moving the number that governs +dependencies (ADR-0009 withdrew its own cost argument over it). + +| | the question it asks | scope | proc-macros | +|---|---|---|---| +| **AM-4a** | what does a game **ship**? | `-p games-ground --no-default-features` | **excluded** | +| **AM-4b** | what does a contributor **acquire**? | `--workspace --edges normal,dev` | **counted** | + +**The proc-macro treatments are opposite on purpose.** AM-4a excludes them +because they run in the compiler and never reach a shipped binary — +counting them in *"what a game ships"* was simply false. AM-4b counts +them, because ADR-0007 D3's acquisition rule counts what the build causes +to be **fetched**, and a proc-macro is fetched, compiled and unaudited on +a contributor's machine like anything else. *"It does not ship"* is no +answer to *"we downloaded it"*. + +**When the two rules disagree, the question each budget asks decides.** +That is the rule ADR-0008 D2 left open, and it is why that decision +refused to reuse AM-4a's measured 36.2% share for AM-4b: the real share is +**15.1%** (109,585 lines), so borrowing would have been wrong by more than +a factor of two. + +**The target moved to fit the measurement, never the reverse.** 745,000 +keeps ~2.7% of room on ADR-0008 D3's reasoning that ~1.5% fails on a +dependency's patch release — the same margin AM-4a received, applied to a +number that grew because the instrument was repaired rather than because +anything was added. + ### 5b. Where AM-8's ten runs live, and why not everywhere *(CB-WP-0015 T02, 2026-08-02. Tier S. The spec value N=10 is **not** diff --git a/tools/dep-weight.py b/tools/dep-weight.py index fefd094..f16e91a 100755 --- a/tools/dep-weight.py +++ b/tools/dep-weight.py @@ -27,19 +27,39 @@ import sys from repo import cargo_bin, enter_root +# AM-4a asks what a GAME SHIPS, so it measures one package's non-dev +# graph. AM-4b asks what a CONTRIBUTOR ACQUIRES, so it measures the whole +# workspace including dev edges (CB-WP-0019 T01). They had the same scope +# until 2026-08-03, which left AM-4b blind to 28 crates and 408,237 lines +# — more source than its own target. PACKAGE = "games-ground" +WORKSPACE = "--workspace" # ADR-0008 D2. `--edges normal` includes proc-macro crates, which run in # the compiler and never reach a shipped binary — 89,048 lines, 36.2% of # what this tool used to call "what a game ships", `syn` alone 66,916. The # shipped-runtime configuration now excludes them. # -# AM-4b is deliberately NOT corrected here: its proc-macro share has not -# been measured, and correcting a second instrument on the strength of the -# first one's ratio is the error this change exists to fix. +# AM-4b's proc-macro share is now MEASURED: 109,585 lines, 15.1% of its +# real graph (CB-WP-0019 T02). It is deliberately **not** excluded, and +# that is the opposite of AM-4a's treatment for a stated reason: +# +# AM-4a excludes proc-macros because they run in the compiler and never +# reach a shipped binary — counting them in "what a game ships" was +# simply false. +# +# AM-4b counts them, because ADR-0007 D3's acquisition rule counts what +# the build causes to be FETCHED, and a proc-macro is fetched, compiled +# and unaudited on a contributor's machine exactly like any other +# dependency. "It does not ship" is no answer to "we downloaded it". +# +# When the two rules disagree, the question each budget asks decides: +# AM-4a asks what ships, AM-4b asks what is acquired. +# +# ADR-0008 D2 said this share was unmeasured. That is no longer true. PROC_MACRO_EXCLUDED = ["--edges", "normal,no-proc-macro"] CONFIGS = { - "shipped-runtime": ["--no-default-features"] + PROC_MACRO_EXCLUDED, - "dev-toolchain": [], + "shipped-runtime": ["-p", PACKAGE, "--no-default-features"] + PROC_MACRO_EXCLUDED, + "dev-toolchain": [WORKSPACE, "--edges", "normal,dev"], } # AM-4a / AM-4b targets from specs/GameKernel.md §4. Breaching one fails @@ -50,9 +70,19 @@ CONFIGS = { # 161,000 keeps ~2.4% of room where 250,000 kept ~1.5% — the small # rounding up is the only thing this decision gives back, because a target # with 1.5% of room fails on a dependency's patch release. +# CB-WP-0019 T01: the target moves to fit the measurement, never the +# reverse. AM-4b now measures 725,258 where it used to read 317,021 — not +# because anything was added, but because it started looking at what it +# always claimed to bound. A 350,000 target against a 725,258 reading +# would be a budget that is simply breached, which teaches nothing. +# +# 745,000 keeps ~2.7% of room, on ADR-0008 D3's reasoning that a target +# with ~1.5% fails on a dependency's patch release. It is NOT generosity: +# it is the same margin AM-4a got, applied to a number that grew because +# the instrument was fixed. TARGETS = { "shipped-runtime": 161_000, - "dev-toolchain": 350_000, + "dev-toolchain": 745_000, } @@ -70,7 +100,10 @@ def crates(extra_args): ) sys.exit(1) out = subprocess.run( - [cargo, "tree", "-p", PACKAGE, "--prefix", "none"] + # The package/workspace selector now comes from the config, so the + # two budgets can ask different questions. Before CB-WP-0019 both + # were pinned to one package, which is what made AM-4b blind. + [cargo, "tree", "--prefix", "none"] + (extra_args if "--edges" in extra_args else ["--edges", "normal"] + extra_args), capture_output=True, text=True, @@ -109,6 +142,19 @@ def source_lines(name, version): return 0 +def _dev_only_dependency_is_counted(): + """AM-4b must actually see dev edges — the defect it was blind to. + + `quick-js` is a dev-dependency of `cb-render-html` and is the crate + that exposed the scope defect: it landed in CB-WP-0014, AM-4b did not + move, and the ADR that added it withdrew its own cost argument as a + result. If this budget stops seeing it, the blindness is back. + """ + dev = crates(CONFIGS["dev-toolchain"]) + shipped = crates(CONFIGS["shipped-runtime"]) + return "quick-js" in dev, "quick-js" in shipped + + def self_test(): """Each assertion pins a failure this tool must detect. @@ -119,6 +165,21 @@ def self_test(): """ results = [] + def _check(name, ok, detail=""): + results.append((name, ok, detail)) + + # CB-WP-0019 T01: the two budgets must ask DIFFERENT questions, and + # `quick-js` is the case that proves it. It is a dev-dependency of + # cb-render-html; it landed in CB-WP-0014, AM-4b did not move, and + # ADR-0009 withdrew its own cost argument as a result. If AM-4b stops + # seeing it the blindness is back; if AM-4a starts seeing it, the + # shipped budget has been widened by accident. + in_dev, in_shipped = _dev_only_dependency_is_counted() + _check("AM-4b sees a dev-only dependency", in_dev, + "quick-js is dev-only and is what exposed the scope defect") + _check("AM-4a does NOT see a dev-only dependency", not in_shipped, + "the shipped budget must stay about what ships") + def check(name, ok, detail=""): results.append((name, ok, detail)) diff --git a/tools/mutation-check.py b/tools/mutation-check.py index 401c4e0..d60c202 100644 --- a/tools/mutation-check.py +++ b/tools/mutation-check.py @@ -135,10 +135,15 @@ def rows(): mutate=("tools/dep-weight.py", '"shipped-runtime": 161_000,', '"shipped-runtime": 1_000,')), - Row("AM-4b", "third-party LOC, dev toolchain <= 350,000", + # CB-WP-0019 T01 widened this to the whole workspace with dev + # edges, and the literal moved with it. The stale find-string was + # caught build-free by `--self-test`, which is the check + # CB-WP-0015 added after AM-4a's mutation rotted unnoticed for two + # passes. Second catch, first one that cost nothing. + Row("AM-4b", "third-party LOC, what a contributor acquires <= 745,000", verify=py + ["tools/dep-weight.py"], mutate=("tools/dep-weight.py", - '"dev-toolchain": 350_000,', '"dev-toolchain": 1_000,')), + '"dev-toolchain": 745_000,', '"dev-toolchain": 1_000,')), # Deliberately RETAINED in this denominator after its withdrawal # from the acceptance table. Dropping it would move M-D1-MUT from diff --git a/workplans/CB-WP-0019-the-am4-family.md b/workplans/CB-WP-0019-the-am4-family.md index 839b10c..99e0384 100644 --- a/workplans/CB-WP-0019-the-am4-family.md +++ b/workplans/CB-WP-0019-the-am4-family.md @@ -52,7 +52,7 @@ a factor of more than two. ```task id: CB-WP-0019-T01 -status: todo +status: done priority: high state_hub_task_id: "b6b209c5-ecd7-4685-aa96-d0bc4d9c0df1" ``` @@ -81,11 +81,36 @@ would not be a budget at all. dev edges, a dev-only dependency must move the number. `quick-js` is the case to use, because it is dev-only and it is what exposed this. +**Done 2026-08-03.** **Widened.** AM-4b now asks *what a contributor +acquires* — `--workspace --edges normal,dev` — and reads **57 crates / +725,258** where it read 29 / 317,021. + +The two budgets had the **same scope** while claiming to bound different +things, which is the whole defect. They now differ by construction: +AM-4a is `-p games-ground --no-default-features`, AM-4b is the workspace +with dev edges. `GameKernel.md` §5c states which question each asks. + +**Target 745,000**, ~2.7% of room — the same margin ADR-0008 D3 gave +AM-4a, applied to a number that grew because the instrument was repaired +rather than because anything was added. **The target moved to fit the +measurement.** + +Controls: `quick-js` must be visible to AM-4b and invisible to AM-4a, both +asserted — it is the dev-only crate that exposed the blindness in +CB-WP-0014, when it landed and the number did not move. + +**And the pass was caught by its own earlier work, twice.** Changing the +target invalidated AM-4b's mutation find-string; `mutation-check +--self-test` reported it **build-free**, which is the check CB-WP-0015 +added after AM-4a's mutation rotted unnoticed for two passes. Second +catch, first that cost nothing. Then the DFD gate caught `facts.toml` +carrying the stale 317,021/350,000. + ## Task: the proc-macro share, decided not just measured ```task id: CB-WP-0019-T02 -status: todo +status: done priority: high state_hub_task_id: "6a90d069-d300-42ff-9bf6-e793f60bcf5a" ``` @@ -105,6 +130,23 @@ Decide it, and say which of the two rules governs when they disagree. **Whatever is decided, ADR-0008 D2's note that AM-4b's share is unmeasured must stop being true in the source.** +**Done 2026-08-03. Counted, not excluded — the opposite of AM-4a, on +purpose.** + +AM-4a excludes proc-macros because they run in the compiler and never +reach a shipped binary; counting them in *"what a game ships"* was simply +false. AM-4b **counts** them, because ADR-0007 D3's acquisition rule +counts what the build causes to be **fetched**, and a proc-macro is +fetched, compiled and unaudited on a contributor's machine like anything +else. *"It does not ship"* is no answer to *"we downloaded it"*. + +**The rule for when they disagree: the question each budget asks decides.** +That is what ADR-0008 D2 left open. + +Measured share: **109,585 lines, 15.1%**, against AM-4a's 36.2% — so +ADR-0008 D2's refusal to borrow the ratio was right by more than a factor +of two. The `unmeasured` note is gone from `dep-weight.py` and the spec. + ## Task: write the self-quoting rule down ```task