tegwick
ca832329e7
Some checks failed
ci / check (push) Failing after 4s
loop-lint failed at 427 lines. Third workplan in this project to cross the ~400-line loadability limit, which makes it structural rather than a formatting nuisance: delivery notes accumulate in a file whose job is to state intent, and every remaining task would push it further over. history/260801-cb-wp-0006-log.md now holds the per-task narrative; the workplan keeps a one-line pointer per task. 427 -> 270 lines, with room for T05-T09. T08 draws on both. Same fix shape as CB-WP-0005, where the limit exposed a circular reference rather than mere length. A gate that keeps finding real structure is worth more than its stated rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
180 lines
8.4 KiB
Markdown
180 lines
8.4 KiB
Markdown
# CB-WP-0006 — delivery log
|
||
|
||
Per-task delivery notes, moved out of
|
||
[the workplan](../workplans/CB-WP-0006-instrument-the-table.md) when it
|
||
crossed the ~400-line loadability limit for the third time in this
|
||
project. The workplan holds the plan; this holds what happened. T08
|
||
draws on both.
|
||
|
||
## CB-WP-0006-T01
|
||
|
||
**Delivered.** `am6_throughput_clears_the_spec_target` in
|
||
`games/ground/src/lib.rs` — a test, not a bench. Best of 3 samples of
|
||
50,000 applied events each.
|
||
|
||
Measured on bnt-lap001 2026-07-31: **341,280 ev/s in debug (3.4× the
|
||
target)**, ~2.4–3.1M in release (~24–30×). **The spec target holds even in
|
||
an unoptimized build**, so the gate needed no `cfg` split and runs in the
|
||
ordinary `make test`.
|
||
|
||
**The trap was avoided by construction, not by intention.** The threshold
|
||
is the spec value `100_000`, untouched; the constant carries a comment
|
||
saying lowering it requires an ADR; and the failure message repeats that,
|
||
states the measured headroom, and names the reference figures — so a
|
||
future agent hitting a red AM-6 is told not to tune it, in the place they
|
||
will actually be reading. Robustness comes from **best-of-N**, not from a
|
||
lower bar: a throughput *floor* asks "is this machine capable", so
|
||
transient load should not fail the build.
|
||
|
||
Two positive controls in the test itself: a run that applied fewer than
|
||
50,000 events, or measured zero elapsed time, fails rather than scoring as
|
||
infinite throughput.
|
||
|
||
**Verified:** `make mutation-check --row AM-6` → **red**, via a *property*
|
||
mutation (4,000 `black_box` iterations injected into `GroundState::fold`,
|
||
the hot path) rather than a threshold tweak — raising the target would only
|
||
prove the comparison runs.
|
||
|
||
**And the FA class is now gated.** `mutation-check` rows gained an
|
||
`expect` field: the mutant's output must contain the row's stated failure
|
||
string, or the verdict is **`WRONG-REASON`**, not `red`. Without it, a
|
||
mutation that merely failed to compile would credit its row with an
|
||
assertion it does not have. Verified by pointing `expect` at a string the
|
||
verifier never prints and confirming the verdict flips. This is
|
||
remedy (2) from the CB-WP-0005 retrospective, built one task earlier than
|
||
T08 planned because the class it guards is the newest and the most
|
||
dangerous.
|
||
|
||
**M-D1-MUT: 4 → 5 of 14.**
|
||
|
||
## CB-WP-0006-T02
|
||
|
||
**Delivered — but the two rows resolved differently, and the difference is
|
||
the point.**
|
||
|
||
**AM-2 is instrumented and enforced.** `tools/size-metrics.py` +
|
||
`make size-metrics`, in `make all`:
|
||
|
||
```text
|
||
AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom)
|
||
1,575 code lines before the first #[cfg(test)] / 58 numbered rules
|
||
```
|
||
|
||
Tests are excluded because AM-2 asks what a rule *costs*, not how much it
|
||
is exercised — `lib.rs` is ~18% test code and including it would have
|
||
flattered the number. Verified red by a **property** mutation: ~800 lines
|
||
of filler injected into the impl, pushing the ratio past 40. `expect` is
|
||
the precise failure signature `FAIL target <= 40`, not the row name, which
|
||
would have matched passing output too.
|
||
|
||
**AM-3 is BLOCKED, not uninstrumented — and this is a finding, not a
|
||
deferral.** The row measures "LOC to express the CB-RES-0001 synthetic
|
||
game on our kernel" against a boardgame.io baseline of ~36 LOC for a
|
||
declarative **3p commit/reveal game object**. That artifact has never been
|
||
built: `games/` contains only `ground`, and `benches/synthetic.rs`
|
||
*drives* GROUND rather than *defining* a synthetic game.
|
||
|
||
Measuring GROUND's 1,575 impl lines against a 36-line synthetic game
|
||
object would compare **two different games** and call the difference a D1
|
||
result. So the tool ships the measurement mechanism — a marker-delimited
|
||
`// AM-3:BEGIN` / `// AM-3:END` region, self-tested — and **reports the row
|
||
blocked, naming the missing artifact**. A number here would have been
|
||
worse than a blank.
|
||
|
||
It therefore stays `unmutatable` and **still counts against M-D1-MUT**, per
|
||
ADR-0005 §1: a row that cannot fail asserts nothing, however good the
|
||
reason. Resolving it needs an artifact, not a metric tweak — carried
|
||
forward, not silently dropped.
|
||
|
||
**M-D1-MUT: 5 → 6 of 14.**
|
||
|
||
## CB-WP-0006-T03
|
||
|
||
**Delivered — `tools/runtime-metrics.py`. One passes, one breaches.**
|
||
|
||
**AM-9: met, and comfortably.** 13.4 MB peak RSS against a ≤64 MB target,
|
||
**4.8× headroom**. Gated and in `make all` (`--fast`, ~1 s). The row that
|
||
CB-EV-0001 called "very unlikely to bind" was right — but it is now
|
||
*measured* rather than assumed, and verified red by a property mutation
|
||
(a 300 MB allocation in the workload).
|
||
|
||
**AM-5: BREACHED, on both readings, on the machine the spec names.**
|
||
|
||
```text
|
||
dev toolchain (default features) 87.0 s [FAIL target <= 60 s]
|
||
shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s]
|
||
```
|
||
|
||
Measured on `bnt-lap001`, 8 cores — the machine `specs/GameKernel.md` §5
|
||
names, so this is a direct comparison, not a directional one. A row
|
||
declared `recorded not gated` and never recorded turns out, on first
|
||
measurement, to **fail its own target by 45%**.
|
||
|
||
The tool **reports and exits 0**, because the spec says the row is not
|
||
gated. Gating it is a spec change and needs an ADR; a tool that promotes
|
||
itself is how a target starts binding without anyone deciding it should.
|
||
So AM-5 stays `unmutatable` — for the accurate reason now — and the breach
|
||
is **raised as a maintainer decision**, not resolved here. Three honest
|
||
options: speed the build, move the target by ADR (arguing why 60 s was
|
||
wrong rather than why 87 s is convenient), or withdraw the row.
|
||
|
||
**The measurement had a real bug, found by cross-validation.**
|
||
`getrusage(RUSAGE_CHILDREN)` is a high-water mark across *every* reaped
|
||
child, so it attributed `cargo`'s memory to the workload and reported
|
||
**38.2 MB for a run that used 12.3 MB** — a 3× over-report. Fixed with
|
||
`os.wait4`, which returns that specific child's rusage. The self-test now
|
||
**cross-checks against `/usr/bin/time -v`** (13.4 vs 12.4 MB), which is
|
||
the only reason the bug was visible at all: the wrong number was
|
||
plausible, passed its target, and would have been published.
|
||
|
||
That is FA in the measurement layer rather than the mutation layer — an
|
||
instrument confidently reporting a number it had not earned.
|
||
|
||
**M-D1-MUT: 6 → 7 of 14.**
|
||
|
||
## CB-WP-0006-T04
|
||
|
||
**Delivered: withdrawn from the acceptance table, retained as a
|
||
diagnostic.** `specs/GameKernel.md` §5a carries the argument.
|
||
|
||
**The ratio has no monotone better direction.** INTENT's rule is *own the
|
||
semantics; assimilate the implementation*. Rising can mean owning
|
||
semantics properly, or reimplementing what should have been assimilated.
|
||
Falling can mean good leverage, or dependency bloat. A target requires
|
||
knowing which way is better, and this metric does not.
|
||
|
||
**It is also redundant:** AM-4a/AM-4b bound the denominator and AM-2 bounds
|
||
own-source density per rule. AM-4c is a ratio of two already-targeted
|
||
quantities.
|
||
|
||
Measured at withdrawal: **1,426** own lines per 100k third-party (shipped),
|
||
**1,107** (dev). `make dep-weight` now prints both, labelled
|
||
`diagnostic, not targeted` — it was never actually reported before.
|
||
|
||
**M-D1-MUT keeps AM-4c in its denominator, deliberately**, and says so in
|
||
the output. Dropping it would move the score 7/14 → 7/13 without enforcing
|
||
anything: a score improved by deleting the question.
|
||
|
||
### A T01 correction found here
|
||
|
||
**The AM-6 gate failed inside `make all` at 38,753 ev/s against 341,280
|
||
measured in isolation — a 9× drop.** `cargo test` runs test binaries and
|
||
threads concurrently, so **a throughput assertion inside a parallel
|
||
harness measures contention, not throughput.** T01's measurement was valid
|
||
and its gate placement was not.
|
||
|
||
Fixed by running the measurement only where it is valid — `#[ignore]`,
|
||
plus `make am6` in release with `--test-threads=1`, now **2.0M ev/s,
|
||
20.2× headroom** — and *not* by lowering the target, which T01 explicitly
|
||
forbade and which would have reproduced the defect being fixed. My first
|
||
attempt did drift that way (a debug "sanity floor" of 50,000) and was
|
||
backed out: a second threshold is still a second chance to tune.
|
||
|
||
**And the mutation went `SURVIVED` on the first run after the move** —
|
||
4,000 `black_box` iterations were calibrated against debug's 3.4×
|
||
headroom and are invisible against release's 20×. Raised to 100,000; back
|
||
to `red`. A weak mutation is not a fixed property of a row: **it can
|
||
become weak when the row's measurement conditions change.**
|
||
|
||
**M-D1-MUT: 7 of 14** (unchanged — AM-4c was always going to stay
|
||
uncounted; what changed is that the reason is now correct and recorded).
|