Move CB-WP-0006 delivery notes to a log; keep the plan a plan
Some checks failed
ci / check (push) Failing after 4s
Some checks failed
ci / check (push) Failing after 4s
loop-lint failed at 427 lines. Third workplan in this project to cross the ~400-line loadability limit, which makes it structural rather than a formatting nuisance: delivery notes accumulate in a file whose job is to state intent, and every remaining task would push it further over. history/260801-cb-wp-0006-log.md now holds the per-task narrative; the workplan keeps a one-line pointer per task. 427 -> 270 lines, with room for T05-T09. T08 draws on both. Same fix shape as CB-WP-0005, where the limit exposed a circular reference rather than mere length. A gate that keeps finding real structure is worth more than its stated rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
959dc39cf0
commit
ca832329e7
2 changed files with 184 additions and 161 deletions
180
history/260801-cb-wp-0006-log.md
Normal file
180
history/260801-cb-wp-0006-log.md
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
# CB-WP-0006 — delivery log
|
||||
|
||||
Per-task delivery notes, moved out of
|
||||
[the workplan](../workplans/CB-WP-0006-instrument-the-table.md) when it
|
||||
crossed the ~400-line loadability limit for the third time in this
|
||||
project. The workplan holds the plan; this holds what happened. T08
|
||||
draws on both.
|
||||
|
||||
## CB-WP-0006-T01
|
||||
|
||||
**Delivered.** `am6_throughput_clears_the_spec_target` in
|
||||
`games/ground/src/lib.rs` — a test, not a bench. Best of 3 samples of
|
||||
50,000 applied events each.
|
||||
|
||||
Measured on bnt-lap001 2026-07-31: **341,280 ev/s in debug (3.4× the
|
||||
target)**, ~2.4–3.1M in release (~24–30×). **The spec target holds even in
|
||||
an unoptimized build**, so the gate needed no `cfg` split and runs in the
|
||||
ordinary `make test`.
|
||||
|
||||
**The trap was avoided by construction, not by intention.** The threshold
|
||||
is the spec value `100_000`, untouched; the constant carries a comment
|
||||
saying lowering it requires an ADR; and the failure message repeats that,
|
||||
states the measured headroom, and names the reference figures — so a
|
||||
future agent hitting a red AM-6 is told not to tune it, in the place they
|
||||
will actually be reading. Robustness comes from **best-of-N**, not from a
|
||||
lower bar: a throughput *floor* asks "is this machine capable", so
|
||||
transient load should not fail the build.
|
||||
|
||||
Two positive controls in the test itself: a run that applied fewer than
|
||||
50,000 events, or measured zero elapsed time, fails rather than scoring as
|
||||
infinite throughput.
|
||||
|
||||
**Verified:** `make mutation-check --row AM-6` → **red**, via a *property*
|
||||
mutation (4,000 `black_box` iterations injected into `GroundState::fold`,
|
||||
the hot path) rather than a threshold tweak — raising the target would only
|
||||
prove the comparison runs.
|
||||
|
||||
**And the FA class is now gated.** `mutation-check` rows gained an
|
||||
`expect` field: the mutant's output must contain the row's stated failure
|
||||
string, or the verdict is **`WRONG-REASON`**, not `red`. Without it, a
|
||||
mutation that merely failed to compile would credit its row with an
|
||||
assertion it does not have. Verified by pointing `expect` at a string the
|
||||
verifier never prints and confirming the verdict flips. This is
|
||||
remedy (2) from the CB-WP-0005 retrospective, built one task earlier than
|
||||
T08 planned because the class it guards is the newest and the most
|
||||
dangerous.
|
||||
|
||||
**M-D1-MUT: 4 → 5 of 14.**
|
||||
|
||||
## CB-WP-0006-T02
|
||||
|
||||
**Delivered — but the two rows resolved differently, and the difference is
|
||||
the point.**
|
||||
|
||||
**AM-2 is instrumented and enforced.** `tools/size-metrics.py` +
|
||||
`make size-metrics`, in `make all`:
|
||||
|
||||
```text
|
||||
AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom)
|
||||
1,575 code lines before the first #[cfg(test)] / 58 numbered rules
|
||||
```
|
||||
|
||||
Tests are excluded because AM-2 asks what a rule *costs*, not how much it
|
||||
is exercised — `lib.rs` is ~18% test code and including it would have
|
||||
flattered the number. Verified red by a **property** mutation: ~800 lines
|
||||
of filler injected into the impl, pushing the ratio past 40. `expect` is
|
||||
the precise failure signature `FAIL target <= 40`, not the row name, which
|
||||
would have matched passing output too.
|
||||
|
||||
**AM-3 is BLOCKED, not uninstrumented — and this is a finding, not a
|
||||
deferral.** The row measures "LOC to express the CB-RES-0001 synthetic
|
||||
game on our kernel" against a boardgame.io baseline of ~36 LOC for a
|
||||
declarative **3p commit/reveal game object**. That artifact has never been
|
||||
built: `games/` contains only `ground`, and `benches/synthetic.rs`
|
||||
*drives* GROUND rather than *defining* a synthetic game.
|
||||
|
||||
Measuring GROUND's 1,575 impl lines against a 36-line synthetic game
|
||||
object would compare **two different games** and call the difference a D1
|
||||
result. So the tool ships the measurement mechanism — a marker-delimited
|
||||
`// AM-3:BEGIN` / `// AM-3:END` region, self-tested — and **reports the row
|
||||
blocked, naming the missing artifact**. A number here would have been
|
||||
worse than a blank.
|
||||
|
||||
It therefore stays `unmutatable` and **still counts against M-D1-MUT**, per
|
||||
ADR-0005 §1: a row that cannot fail asserts nothing, however good the
|
||||
reason. Resolving it needs an artifact, not a metric tweak — carried
|
||||
forward, not silently dropped.
|
||||
|
||||
**M-D1-MUT: 5 → 6 of 14.**
|
||||
|
||||
## CB-WP-0006-T03
|
||||
|
||||
**Delivered — `tools/runtime-metrics.py`. One passes, one breaches.**
|
||||
|
||||
**AM-9: met, and comfortably.** 13.4 MB peak RSS against a ≤64 MB target,
|
||||
**4.8× headroom**. Gated and in `make all` (`--fast`, ~1 s). The row that
|
||||
CB-EV-0001 called "very unlikely to bind" was right — but it is now
|
||||
*measured* rather than assumed, and verified red by a property mutation
|
||||
(a 300 MB allocation in the workload).
|
||||
|
||||
**AM-5: BREACHED, on both readings, on the machine the spec names.**
|
||||
|
||||
```text
|
||||
dev toolchain (default features) 87.0 s [FAIL target <= 60 s]
|
||||
shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s]
|
||||
```
|
||||
|
||||
Measured on `bnt-lap001`, 8 cores — the machine `specs/GameKernel.md` §5
|
||||
names, so this is a direct comparison, not a directional one. A row
|
||||
declared `recorded not gated` and never recorded turns out, on first
|
||||
measurement, to **fail its own target by 45%**.
|
||||
|
||||
The tool **reports and exits 0**, because the spec says the row is not
|
||||
gated. Gating it is a spec change and needs an ADR; a tool that promotes
|
||||
itself is how a target starts binding without anyone deciding it should.
|
||||
So AM-5 stays `unmutatable` — for the accurate reason now — and the breach
|
||||
is **raised as a maintainer decision**, not resolved here. Three honest
|
||||
options: speed the build, move the target by ADR (arguing why 60 s was
|
||||
wrong rather than why 87 s is convenient), or withdraw the row.
|
||||
|
||||
**The measurement had a real bug, found by cross-validation.**
|
||||
`getrusage(RUSAGE_CHILDREN)` is a high-water mark across *every* reaped
|
||||
child, so it attributed `cargo`'s memory to the workload and reported
|
||||
**38.2 MB for a run that used 12.3 MB** — a 3× over-report. Fixed with
|
||||
`os.wait4`, which returns that specific child's rusage. The self-test now
|
||||
**cross-checks against `/usr/bin/time -v`** (13.4 vs 12.4 MB), which is
|
||||
the only reason the bug was visible at all: the wrong number was
|
||||
plausible, passed its target, and would have been published.
|
||||
|
||||
That is FA in the measurement layer rather than the mutation layer — an
|
||||
instrument confidently reporting a number it had not earned.
|
||||
|
||||
**M-D1-MUT: 6 → 7 of 14.**
|
||||
|
||||
## CB-WP-0006-T04
|
||||
|
||||
**Delivered: withdrawn from the acceptance table, retained as a
|
||||
diagnostic.** `specs/GameKernel.md` §5a carries the argument.
|
||||
|
||||
**The ratio has no monotone better direction.** INTENT's rule is *own the
|
||||
semantics; assimilate the implementation*. Rising can mean owning
|
||||
semantics properly, or reimplementing what should have been assimilated.
|
||||
Falling can mean good leverage, or dependency bloat. A target requires
|
||||
knowing which way is better, and this metric does not.
|
||||
|
||||
**It is also redundant:** AM-4a/AM-4b bound the denominator and AM-2 bounds
|
||||
own-source density per rule. AM-4c is a ratio of two already-targeted
|
||||
quantities.
|
||||
|
||||
Measured at withdrawal: **1,426** own lines per 100k third-party (shipped),
|
||||
**1,107** (dev). `make dep-weight` now prints both, labelled
|
||||
`diagnostic, not targeted` — it was never actually reported before.
|
||||
|
||||
**M-D1-MUT keeps AM-4c in its denominator, deliberately**, and says so in
|
||||
the output. Dropping it would move the score 7/14 → 7/13 without enforcing
|
||||
anything: a score improved by deleting the question.
|
||||
|
||||
### A T01 correction found here
|
||||
|
||||
**The AM-6 gate failed inside `make all` at 38,753 ev/s against 341,280
|
||||
measured in isolation — a 9× drop.** `cargo test` runs test binaries and
|
||||
threads concurrently, so **a throughput assertion inside a parallel
|
||||
harness measures contention, not throughput.** T01's measurement was valid
|
||||
and its gate placement was not.
|
||||
|
||||
Fixed by running the measurement only where it is valid — `#[ignore]`,
|
||||
plus `make am6` in release with `--test-threads=1`, now **2.0M ev/s,
|
||||
20.2× headroom** — and *not* by lowering the target, which T01 explicitly
|
||||
forbade and which would have reproduced the defect being fixed. My first
|
||||
attempt did drift that way (a debug "sanity floor" of 50,000) and was
|
||||
backed out: a second threshold is still a second chance to tune.
|
||||
|
||||
**And the mutation went `SURVIVED` on the first run after the move** —
|
||||
4,000 `black_box` iterations were calibrated against debug's 3.4×
|
||||
headroom and are invisible against release's 20×. Raised to 100,000; back
|
||||
to `red`. A weak mutation is not a fixed property of a row: **it can
|
||||
become weak when the row's measurement conditions change.**
|
||||
|
||||
**M-D1-MUT: 7 of 14** (unchanged — AM-4c was always going to stay
|
||||
uncounted; what changed is that the reason is now correct and recorded).
|
||||
|
|
@ -63,44 +63,7 @@ names.
|
|||
**Verified by:** `make mutation-check --row AM-6` goes from `unmutatable`
|
||||
to `red`.
|
||||
|
||||
**Delivered.** `am6_throughput_clears_the_spec_target` in
|
||||
`games/ground/src/lib.rs` — a test, not a bench. Best of 3 samples of
|
||||
50,000 applied events each.
|
||||
|
||||
Measured on bnt-lap001 2026-07-31: **341,280 ev/s in debug (3.4× the
|
||||
target)**, ~2.4–3.1M in release (~24–30×). **The spec target holds even in
|
||||
an unoptimized build**, so the gate needed no `cfg` split and runs in the
|
||||
ordinary `make test`.
|
||||
|
||||
**The trap was avoided by construction, not by intention.** The threshold
|
||||
is the spec value `100_000`, untouched; the constant carries a comment
|
||||
saying lowering it requires an ADR; and the failure message repeats that,
|
||||
states the measured headroom, and names the reference figures — so a
|
||||
future agent hitting a red AM-6 is told not to tune it, in the place they
|
||||
will actually be reading. Robustness comes from **best-of-N**, not from a
|
||||
lower bar: a throughput *floor* asks "is this machine capable", so
|
||||
transient load should not fail the build.
|
||||
|
||||
Two positive controls in the test itself: a run that applied fewer than
|
||||
50,000 events, or measured zero elapsed time, fails rather than scoring as
|
||||
infinite throughput.
|
||||
|
||||
**Verified:** `make mutation-check --row AM-6` → **red**, via a *property*
|
||||
mutation (4,000 `black_box` iterations injected into `GroundState::fold`,
|
||||
the hot path) rather than a threshold tweak — raising the target would only
|
||||
prove the comparison runs.
|
||||
|
||||
**And the FA class is now gated.** `mutation-check` rows gained an
|
||||
`expect` field: the mutant's output must contain the row's stated failure
|
||||
string, or the verdict is **`WRONG-REASON`**, not `red`. Without it, a
|
||||
mutation that merely failed to compile would credit its row with an
|
||||
assertion it does not have. Verified by pointing `expect` at a string the
|
||||
verifier never prints and confirming the verdict flips. This is
|
||||
remedy (2) from the CB-WP-0005 retrospective, built one task earlier than
|
||||
T08 planned because the class it guards is the newest and the most
|
||||
dangerous.
|
||||
|
||||
**M-D1-MUT: 4 → 5 of 14.**
|
||||
**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t01).
|
||||
|
||||
## Task: AM-2, AM-3 — instrument the size metrics
|
||||
|
||||
|
|
@ -124,44 +87,7 @@ AM-3 depends on K18 (benches driven from scenario files); if K18 resolves
|
|||
toward amending the spec rather than implementing it, AM-3 must be
|
||||
restated or withdrawn with an argument rather than left unmeasured.
|
||||
|
||||
**Delivered — but the two rows resolved differently, and the difference is
|
||||
the point.**
|
||||
|
||||
**AM-2 is instrumented and enforced.** `tools/size-metrics.py` +
|
||||
`make size-metrics`, in `make all`:
|
||||
|
||||
```text
|
||||
AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom)
|
||||
1,575 code lines before the first #[cfg(test)] / 58 numbered rules
|
||||
```
|
||||
|
||||
Tests are excluded because AM-2 asks what a rule *costs*, not how much it
|
||||
is exercised — `lib.rs` is ~18% test code and including it would have
|
||||
flattered the number. Verified red by a **property** mutation: ~800 lines
|
||||
of filler injected into the impl, pushing the ratio past 40. `expect` is
|
||||
the precise failure signature `FAIL target <= 40`, not the row name, which
|
||||
would have matched passing output too.
|
||||
|
||||
**AM-3 is BLOCKED, not uninstrumented — and this is a finding, not a
|
||||
deferral.** The row measures "LOC to express the CB-RES-0001 synthetic
|
||||
game on our kernel" against a boardgame.io baseline of ~36 LOC for a
|
||||
declarative **3p commit/reveal game object**. That artifact has never been
|
||||
built: `games/` contains only `ground`, and `benches/synthetic.rs`
|
||||
*drives* GROUND rather than *defining* a synthetic game.
|
||||
|
||||
Measuring GROUND's 1,575 impl lines against a 36-line synthetic game
|
||||
object would compare **two different games** and call the difference a D1
|
||||
result. So the tool ships the measurement mechanism — a marker-delimited
|
||||
`// AM-3:BEGIN` / `// AM-3:END` region, self-tested — and **reports the row
|
||||
blocked, naming the missing artifact**. A number here would have been
|
||||
worse than a blank.
|
||||
|
||||
It therefore stays `unmutatable` and **still counts against M-D1-MUT**, per
|
||||
ADR-0005 §1: a row that cannot fail asserts nothing, however good the
|
||||
reason. Resolving it needs an artifact, not a metric tweak — carried
|
||||
forward, not silently dropped.
|
||||
|
||||
**M-D1-MUT: 5 → 6 of 14.**
|
||||
**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t02).
|
||||
|
||||
## Task: AM-5, AM-9 — measure or withdraw, but stop leaving them blank
|
||||
|
||||
|
|
@ -182,47 +108,7 @@ argument**, which is a legitimate outcome and possibly the right one for
|
|||
AM-5 on a machine-dependent number. What is not legitimate is a third pass
|
||||
leaving them blank while the scoreboard reports elsewhere.
|
||||
|
||||
**Delivered — `tools/runtime-metrics.py`. One passes, one breaches.**
|
||||
|
||||
**AM-9: met, and comfortably.** 13.4 MB peak RSS against a ≤64 MB target,
|
||||
**4.8× headroom**. Gated and in `make all` (`--fast`, ~1 s). The row that
|
||||
CB-EV-0001 called "very unlikely to bind" was right — but it is now
|
||||
*measured* rather than assumed, and verified red by a property mutation
|
||||
(a 300 MB allocation in the workload).
|
||||
|
||||
**AM-5: BREACHED, on both readings, on the machine the spec names.**
|
||||
|
||||
```text
|
||||
dev toolchain (default features) 87.0 s [FAIL target <= 60 s]
|
||||
shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s]
|
||||
```
|
||||
|
||||
Measured on `bnt-lap001`, 8 cores — the machine `specs/GameKernel.md` §5
|
||||
names, so this is a direct comparison, not a directional one. A row
|
||||
declared `recorded not gated` and never recorded turns out, on first
|
||||
measurement, to **fail its own target by 45%**.
|
||||
|
||||
The tool **reports and exits 0**, because the spec says the row is not
|
||||
gated. Gating it is a spec change and needs an ADR; a tool that promotes
|
||||
itself is how a target starts binding without anyone deciding it should.
|
||||
So AM-5 stays `unmutatable` — for the accurate reason now — and the breach
|
||||
is **raised as a maintainer decision**, not resolved here. Three honest
|
||||
options: speed the build, move the target by ADR (arguing why 60 s was
|
||||
wrong rather than why 87 s is convenient), or withdraw the row.
|
||||
|
||||
**The measurement had a real bug, found by cross-validation.**
|
||||
`getrusage(RUSAGE_CHILDREN)` is a high-water mark across *every* reaped
|
||||
child, so it attributed `cargo`'s memory to the workload and reported
|
||||
**38.2 MB for a run that used 12.3 MB** — a 3× over-report. Fixed with
|
||||
`os.wait4`, which returns that specific child's rusage. The self-test now
|
||||
**cross-checks against `/usr/bin/time -v`** (13.4 vs 12.4 MB), which is
|
||||
the only reason the bug was visible at all: the wrong number was
|
||||
plausible, passed its target, and would have been published.
|
||||
|
||||
That is FA in the measurement layer rather than the mutation layer — an
|
||||
instrument confidently reporting a number it had not earned.
|
||||
|
||||
**M-D1-MUT: 6 → 7 of 14.**
|
||||
**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t03).
|
||||
|
||||
## Task: AM-4c — target it or drop it
|
||||
|
||||
|
|
@ -246,50 +132,7 @@ kernel work in Phase B, so AM-4c will move against us. Setting its
|
|||
threshold *after* seeing that movement would be the retarget InnerLoop
|
||||
§Step 4 forbids — so it is set here, before Phase B, or not at all.
|
||||
|
||||
**Delivered: withdrawn from the acceptance table, retained as a
|
||||
diagnostic.** `specs/GameKernel.md` §5a carries the argument.
|
||||
|
||||
**The ratio has no monotone better direction.** INTENT's rule is *own the
|
||||
semantics; assimilate the implementation*. Rising can mean owning
|
||||
semantics properly, or reimplementing what should have been assimilated.
|
||||
Falling can mean good leverage, or dependency bloat. A target requires
|
||||
knowing which way is better, and this metric does not.
|
||||
|
||||
**It is also redundant:** AM-4a/AM-4b bound the denominator and AM-2 bounds
|
||||
own-source density per rule. AM-4c is a ratio of two already-targeted
|
||||
quantities.
|
||||
|
||||
Measured at withdrawal: **1,426** own lines per 100k third-party (shipped),
|
||||
**1,107** (dev). `make dep-weight` now prints both, labelled
|
||||
`diagnostic, not targeted` — it was never actually reported before.
|
||||
|
||||
**M-D1-MUT keeps AM-4c in its denominator, deliberately**, and says so in
|
||||
the output. Dropping it would move the score 7/14 → 7/13 without enforcing
|
||||
anything: a score improved by deleting the question.
|
||||
|
||||
### A T01 correction found here
|
||||
|
||||
**The AM-6 gate failed inside `make all` at 38,753 ev/s against 341,280
|
||||
measured in isolation — a 9× drop.** `cargo test` runs test binaries and
|
||||
threads concurrently, so **a throughput assertion inside a parallel
|
||||
harness measures contention, not throughput.** T01's measurement was valid
|
||||
and its gate placement was not.
|
||||
|
||||
Fixed by running the measurement only where it is valid — `#[ignore]`,
|
||||
plus `make am6` in release with `--test-threads=1`, now **2.0M ev/s,
|
||||
20.2× headroom** — and *not* by lowering the target, which T01 explicitly
|
||||
forbade and which would have reproduced the defect being fixed. My first
|
||||
attempt did drift that way (a debug "sanity floor" of 50,000) and was
|
||||
backed out: a second threshold is still a second chance to tune.
|
||||
|
||||
**And the mutation went `SURVIVED` on the first run after the move** —
|
||||
4,000 `black_box` iterations were calibrated against debug's 3.4×
|
||||
headroom and are invisible against release's 20×. Raised to 100,000; back
|
||||
to `red`. A weak mutation is not a fixed property of a row: **it can
|
||||
become weak when the row's measurement conditions change.**
|
||||
|
||||
**M-D1-MUT: 7 of 14** (unchanged — AM-4c was always going to stay
|
||||
uncounted; what changed is that the reason is now correct and recorded).
|
||||
**Delivered** — see [the delivery log](../history/260801-cb-wp-0006-log.md#cb-wp-0006-t04).
|
||||
|
||||
## Phase B — the kernel work inherited from CB-WP-0005
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue