From ca832329e729dc07b9215d0b0df1ca80bec23553 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 1 Aug 2026 10:07:58 +0200 Subject: [PATCH] Move CB-WP-0006 delivery notes to a log; keep the plan a plan loop-lint failed at 427 lines. Third workplan in this project to cross the ~400-line loadability limit, which makes it structural rather than a formatting nuisance: delivery notes accumulate in a file whose job is to state intent, and every remaining task would push it further over. history/260801-cb-wp-0006-log.md now holds the per-task narrative; the workplan keeps a one-line pointer per task. 427 -> 270 lines, with room for T05-T09. T08 draws on both. Same fix shape as CB-WP-0005, where the limit exposed a circular reference rather than mere length. A gate that keeps finding real structure is worth more than its stated rule. Co-Authored-By: Claude Opus 5 --- history/260801-cb-wp-0006-log.md | 180 +++++++++++++++++++ workplans/CB-WP-0006-instrument-the-table.md | 165 +---------------- 2 files changed, 184 insertions(+), 161 deletions(-) create mode 100644 history/260801-cb-wp-0006-log.md diff --git a/history/260801-cb-wp-0006-log.md b/history/260801-cb-wp-0006-log.md new file mode 100644 index 0000000..c3dd418 --- /dev/null +++ b/history/260801-cb-wp-0006-log.md @@ -0,0 +1,180 @@ +# CB-WP-0006 — delivery log + +Per-task delivery notes, moved out of +[the workplan](../workplans/CB-WP-0006-instrument-the-table.md) when it +crossed the ~400-line loadability limit for the third time in this +project. The workplan holds the plan; this holds what happened. T08 +draws on both. + +## CB-WP-0006-T01 + +**Delivered.** `am6_throughput_clears_the_spec_target` in +`games/ground/src/lib.rs` — a test, not a bench. Best of 3 samples of +50,000 applied events each. + +Measured on bnt-lap001 2026-07-31: **341,280 ev/s in debug (3.4× the +target)**, ~2.4–3.1M in release (~24–30×). **The spec target holds even in +an unoptimized build**, so the gate needed no `cfg` split and runs in the +ordinary `make test`. + +**The trap was avoided by construction, not by intention.** The threshold +is the spec value `100_000`, untouched; the constant carries a comment +saying lowering it requires an ADR; and the failure message repeats that, +states the measured headroom, and names the reference figures — so a +future agent hitting a red AM-6 is told not to tune it, in the place they +will actually be reading. Robustness comes from **best-of-N**, not from a +lower bar: a throughput *floor* asks "is this machine capable", so +transient load should not fail the build. + +Two positive controls in the test itself: a run that applied fewer than +50,000 events, or measured zero elapsed time, fails rather than scoring as +infinite throughput. + +**Verified:** `make mutation-check --row AM-6` → **red**, via a *property* +mutation (4,000 `black_box` iterations injected into `GroundState::fold`, +the hot path) rather than a threshold tweak — raising the target would only +prove the comparison runs. + +**And the FA class is now gated.** `mutation-check` rows gained an +`expect` field: the mutant's output must contain the row's stated failure +string, or the verdict is **`WRONG-REASON`**, not `red`. Without it, a +mutation that merely failed to compile would credit its row with an +assertion it does not have. Verified by pointing `expect` at a string the +verifier never prints and confirming the verdict flips. This is +remedy (2) from the CB-WP-0005 retrospective, built one task earlier than +T08 planned because the class it guards is the newest and the most +dangerous. + +**M-D1-MUT: 4 → 5 of 14.** + +## CB-WP-0006-T02 + +**Delivered — but the two rows resolved differently, and the difference is +the point.** + +**AM-2 is instrumented and enforced.** `tools/size-metrics.py` + +`make size-metrics`, in `make all`: + +```text +AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom) + 1,575 code lines before the first #[cfg(test)] / 58 numbered rules +``` + +Tests are excluded because AM-2 asks what a rule *costs*, not how much it +is exercised — `lib.rs` is ~18% test code and including it would have +flattered the number. Verified red by a **property** mutation: ~800 lines +of filler injected into the impl, pushing the ratio past 40. `expect` is +the precise failure signature `FAIL target <= 40`, not the row name, which +would have matched passing output too. + +**AM-3 is BLOCKED, not uninstrumented — and this is a finding, not a +deferral.** The row measures "LOC to express the CB-RES-0001 synthetic +game on our kernel" against a boardgame.io baseline of ~36 LOC for a +declarative **3p commit/reveal game object**. That artifact has never been +built: `games/` contains only `ground`, and `benches/synthetic.rs` +*drives* GROUND rather than *defining* a synthetic game. + +Measuring GROUND's 1,575 impl lines against a 36-line synthetic game +object would compare **two different games** and call the difference a D1 +result. So the tool ships the measurement mechanism — a marker-delimited +`// AM-3:BEGIN` / `// AM-3:END` region, self-tested — and **reports the row +blocked, naming the missing artifact**. A number here would have been +worse than a blank. + +It therefore stays `unmutatable` and **still counts against M-D1-MUT**, per +ADR-0005 §1: a row that cannot fail asserts nothing, however good the +reason. Resolving it needs an artifact, not a metric tweak — carried +forward, not silently dropped. + +**M-D1-MUT: 5 → 6 of 14.** + +## CB-WP-0006-T03 + +**Delivered — `tools/runtime-metrics.py`. One passes, one breaches.** + +**AM-9: met, and comfortably.** 13.4 MB peak RSS against a ≤64 MB target, +**4.8× headroom**. Gated and in `make all` (`--fast`, ~1 s). The row that +CB-EV-0001 called "very unlikely to bind" was right — but it is now +*measured* rather than assumed, and verified red by a property mutation +(a 300 MB allocation in the workload). + +**AM-5: BREACHED, on both readings, on the machine the spec names.** + +```text +dev toolchain (default features) 87.0 s [FAIL target <= 60 s] +shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s] +``` + +Measured on `bnt-lap001`, 8 cores — the machine `specs/GameKernel.md` §5 +names, so this is a direct comparison, not a directional one. A row +declared `recorded not gated` and never recorded turns out, on first +measurement, to **fail its own target by 45%**. + +The tool **reports and exits 0**, because the spec says the row is not +gated. Gating it is a spec change and needs an ADR; a tool that promotes +itself is how a target starts binding without anyone deciding it should. +So AM-5 stays `unmutatable` — for the accurate reason now — and the breach +is **raised as a maintainer decision**, not resolved here. Three honest +options: speed the build, move the target by ADR (arguing why 60 s was +wrong rather than why 87 s is convenient), or withdraw the row. + +**The measurement had a real bug, found by cross-validation.** +`getrusage(RUSAGE_CHILDREN)` is a high-water mark across *every* reaped +child, so it attributed `cargo`'s memory to the workload and reported +**38.2 MB for a run that used 12.3 MB** — a 3× over-report. Fixed with +`os.wait4`, which returns that specific child's rusage. The self-test now +**cross-checks against `/usr/bin/time -v`** (13.4 vs 12.4 MB), which is +the only reason the bug was visible at all: the wrong number was +plausible, passed its target, and would have been published. + +That is FA in the measurement layer rather than the mutation layer — an +instrument confidently reporting a number it had not earned. + +**M-D1-MUT: 6 → 7 of 14.** + +## CB-WP-0006-T04 + +**Delivered: withdrawn from the acceptance table, retained as a +diagnostic.** `specs/GameKernel.md` §5a carries the argument. + +**The ratio has no monotone better direction.** INTENT's rule is *own the +semantics; assimilate the implementation*. Rising can mean owning +semantics properly, or reimplementing what should have been assimilated. +Falling can mean good leverage, or dependency bloat. A target requires +knowing which way is better, and this metric does not. + +**It is also redundant:** AM-4a/AM-4b bound the denominator and AM-2 bounds +own-source density per rule. AM-4c is a ratio of two already-targeted +quantities. + +Measured at withdrawal: **1,426** own lines per 100k third-party (shipped), +**1,107** (dev). `make dep-weight` now prints both, labelled +`diagnostic, not targeted` — it was never actually reported before. + +**M-D1-MUT keeps AM-4c in its denominator, deliberately**, and says so in +the output. Dropping it would move the score 7/14 → 7/13 without enforcing +anything: a score improved by deleting the question. + +### A T01 correction found here + +**The AM-6 gate failed inside `make all` at 38,753 ev/s against 341,280 +measured in isolation — a 9× drop.** `cargo test` runs test binaries and +threads concurrently, so **a throughput assertion inside a parallel +harness measures contention, not throughput.** T01's measurement was valid +and its gate placement was not. + +Fixed by running the measurement only where it is valid — `#[ignore]`, +plus `make am6` in release with `--test-threads=1`, now **2.0M ev/s, +20.2× headroom** — and *not* by lowering the target, which T01 explicitly +forbade and which would have reproduced the defect being fixed. My first +attempt did drift that way (a debug "sanity floor" of 50,000) and was +backed out: a second threshold is still a second chance to tune. + +**And the mutation went `SURVIVED` on the first run after the move** — +4,000 `black_box` iterations were calibrated against debug's 3.4× +headroom and are invisible against release's 20×. Raised to 100,000; back +to `red`. A weak mutation is not a fixed property of a row: **it can +become weak when the row's measurement conditions change.** + +**M-D1-MUT: 7 of 14** (unchanged — AM-4c was always going to stay +uncounted; what changed is that the reason is now correct and recorded). diff --git a/workplans/CB-WP-0006-instrument-the-table.md b/workplans/CB-WP-0006-instrument-the-table.md index 4ffef57..6486084 100644 --- a/workplans/CB-WP-0006-instrument-the-table.md +++ b/workplans/CB-WP-0006-instrument-the-table.md @@ -63,44 +63,7 @@ names. **Verified by:** `make mutation-check --row AM-6` goes from `unmutatable` to `red`. -**Delivered.** `am6_throughput_clears_the_spec_target` in -`games/ground/src/lib.rs` — a test, not a bench. Best of 3 samples of -50,000 applied events each. - -Measured on bnt-lap001 2026-07-31: **341,280 ev/s in debug (3.4× the -target)**, ~2.4–3.1M in release (~24–30×). **The spec target holds even in -an unoptimized build**, so the gate needed no `cfg` split and runs in the -ordinary `make test`. - -**The trap was avoided by construction, not by intention.** The threshold -is the spec value `100_000`, untouched; the constant carries a comment -saying lowering it requires an ADR; and the failure message repeats that, -states the measured headroom, and names the reference figures — so a -future agent hitting a red AM-6 is told not to tune it, in the place they -will actually be reading. Robustness comes from **best-of-N**, not from a -lower bar: a throughput *floor* asks "is this machine capable", so -transient load should not fail the build. - -Two positive controls in the test itself: a run that applied fewer than -50,000 events, or measured zero elapsed time, fails rather than scoring as -infinite throughput. - -**Verified:** `make mutation-check --row AM-6` → **red**, via a *property* -mutation (4,000 `black_box` iterations injected into `GroundState::fold`, -the hot path) rather than a threshold tweak — raising the target would only -prove the comparison runs. - -**And the FA class is now gated.** `mutation-check` rows gained an -`expect` field: the mutant's output must contain the row's stated failure -string, or the verdict is **`WRONG-REASON`**, not `red`. Without it, a -mutation that merely failed to compile would credit its row with an -assertion it does not have. Verified by pointing `expect` at a string the -verifier never prints and confirming the verdict flips. This is -remedy (2) from the CB-WP-0005 retrospective, built one task earlier than -T08 planned because the class it guards is the newest and the most -dangerous. - -**M-D1-MUT: 4 → 5 of 14.** +**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t01). ## Task: AM-2, AM-3 — instrument the size metrics @@ -124,44 +87,7 @@ AM-3 depends on K18 (benches driven from scenario files); if K18 resolves toward amending the spec rather than implementing it, AM-3 must be restated or withdrawn with an argument rather than left unmeasured. -**Delivered — but the two rows resolved differently, and the difference is -the point.** - -**AM-2 is instrumented and enforced.** `tools/size-metrics.py` + -`make size-metrics`, in `make all`: - -```text -AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom) - 1,575 code lines before the first #[cfg(test)] / 58 numbered rules -``` - -Tests are excluded because AM-2 asks what a rule *costs*, not how much it -is exercised — `lib.rs` is ~18% test code and including it would have -flattered the number. Verified red by a **property** mutation: ~800 lines -of filler injected into the impl, pushing the ratio past 40. `expect` is -the precise failure signature `FAIL target <= 40`, not the row name, which -would have matched passing output too. - -**AM-3 is BLOCKED, not uninstrumented — and this is a finding, not a -deferral.** The row measures "LOC to express the CB-RES-0001 synthetic -game on our kernel" against a boardgame.io baseline of ~36 LOC for a -declarative **3p commit/reveal game object**. That artifact has never been -built: `games/` contains only `ground`, and `benches/synthetic.rs` -*drives* GROUND rather than *defining* a synthetic game. - -Measuring GROUND's 1,575 impl lines against a 36-line synthetic game -object would compare **two different games** and call the difference a D1 -result. So the tool ships the measurement mechanism — a marker-delimited -`// AM-3:BEGIN` / `// AM-3:END` region, self-tested — and **reports the row -blocked, naming the missing artifact**. A number here would have been -worse than a blank. - -It therefore stays `unmutatable` and **still counts against M-D1-MUT**, per -ADR-0005 §1: a row that cannot fail asserts nothing, however good the -reason. Resolving it needs an artifact, not a metric tweak — carried -forward, not silently dropped. - -**M-D1-MUT: 5 → 6 of 14.** +**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t02). ## Task: AM-5, AM-9 — measure or withdraw, but stop leaving them blank @@ -182,47 +108,7 @@ argument**, which is a legitimate outcome and possibly the right one for AM-5 on a machine-dependent number. What is not legitimate is a third pass leaving them blank while the scoreboard reports elsewhere. -**Delivered — `tools/runtime-metrics.py`. One passes, one breaches.** - -**AM-9: met, and comfortably.** 13.4 MB peak RSS against a ≤64 MB target, -**4.8× headroom**. Gated and in `make all` (`--fast`, ~1 s). The row that -CB-EV-0001 called "very unlikely to bind" was right — but it is now -*measured* rather than assumed, and verified red by a property mutation -(a 300 MB allocation in the workload). - -**AM-5: BREACHED, on both readings, on the machine the spec names.** - -```text -dev toolchain (default features) 87.0 s [FAIL target <= 60 s] -shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s] -``` - -Measured on `bnt-lap001`, 8 cores — the machine `specs/GameKernel.md` §5 -names, so this is a direct comparison, not a directional one. A row -declared `recorded not gated` and never recorded turns out, on first -measurement, to **fail its own target by 45%**. - -The tool **reports and exits 0**, because the spec says the row is not -gated. Gating it is a spec change and needs an ADR; a tool that promotes -itself is how a target starts binding without anyone deciding it should. -So AM-5 stays `unmutatable` — for the accurate reason now — and the breach -is **raised as a maintainer decision**, not resolved here. Three honest -options: speed the build, move the target by ADR (arguing why 60 s was -wrong rather than why 87 s is convenient), or withdraw the row. - -**The measurement had a real bug, found by cross-validation.** -`getrusage(RUSAGE_CHILDREN)` is a high-water mark across *every* reaped -child, so it attributed `cargo`'s memory to the workload and reported -**38.2 MB for a run that used 12.3 MB** — a 3× over-report. Fixed with -`os.wait4`, which returns that specific child's rusage. The self-test now -**cross-checks against `/usr/bin/time -v`** (13.4 vs 12.4 MB), which is -the only reason the bug was visible at all: the wrong number was -plausible, passed its target, and would have been published. - -That is FA in the measurement layer rather than the mutation layer — an -instrument confidently reporting a number it had not earned. - -**M-D1-MUT: 6 → 7 of 14.** +**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t03). ## Task: AM-4c — target it or drop it @@ -246,50 +132,7 @@ kernel work in Phase B, so AM-4c will move against us. Setting its threshold *after* seeing that movement would be the retarget InnerLoop §Step 4 forbids — so it is set here, before Phase B, or not at all. -**Delivered: withdrawn from the acceptance table, retained as a -diagnostic.** `specs/GameKernel.md` §5a carries the argument. - -**The ratio has no monotone better direction.** INTENT's rule is *own the -semantics; assimilate the implementation*. Rising can mean owning -semantics properly, or reimplementing what should have been assimilated. -Falling can mean good leverage, or dependency bloat. A target requires -knowing which way is better, and this metric does not. - -**It is also redundant:** AM-4a/AM-4b bound the denominator and AM-2 bounds -own-source density per rule. AM-4c is a ratio of two already-targeted -quantities. - -Measured at withdrawal: **1,426** own lines per 100k third-party (shipped), -**1,107** (dev). `make dep-weight` now prints both, labelled -`diagnostic, not targeted` — it was never actually reported before. - -**M-D1-MUT keeps AM-4c in its denominator, deliberately**, and says so in -the output. Dropping it would move the score 7/14 → 7/13 without enforcing -anything: a score improved by deleting the question. - -### A T01 correction found here - -**The AM-6 gate failed inside `make all` at 38,753 ev/s against 341,280 -measured in isolation — a 9× drop.** `cargo test` runs test binaries and -threads concurrently, so **a throughput assertion inside a parallel -harness measures contention, not throughput.** T01's measurement was valid -and its gate placement was not. - -Fixed by running the measurement only where it is valid — `#[ignore]`, -plus `make am6` in release with `--test-threads=1`, now **2.0M ev/s, -20.2× headroom** — and *not* by lowering the target, which T01 explicitly -forbade and which would have reproduced the defect being fixed. My first -attempt did drift that way (a debug "sanity floor" of 50,000) and was -backed out: a second threshold is still a second chance to tune. - -**And the mutation went `SURVIVED` on the first run after the move** — -4,000 `black_box` iterations were calibrated against debug's 3.4× -headroom and are invisible against release's 20×. Raised to 100,000; back -to `red`. A weak mutation is not a fixed property of a row: **it can -become weak when the row's measurement conditions change.** - -**M-D1-MUT: 7 of 14** (unchanged — AM-4c was always going to stay -uncounted; what changed is that the reason is now correct and recorded). +**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t04). ## Phase B — the kernel work inherited from CB-WP-0005