CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches

AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x
headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind"
was right, but it is now measured rather than assumed, and verified red by
a property mutation (a 300 MB allocation in the workload).

AM-5 is BREACHED on both readings, on the machine the spec names:

  dev toolchain (default features)        87.0 s  [FAIL target <= 60 s]
  shipped runtime (--no-default-features) 61.3 s  [FAIL target <= 60 s]

bnt-lap001, 8 cores — a direct comparison, not a directional one. A row
declared "recorded not gated" and never recorded fails its own target by
45% on first measurement.

The tool reports and exits 0 because the spec says the row is ungated.
Gating it is a spec change needing an ADR; a tool that promotes itself is
how a target starts binding without anyone deciding it should. So AM-5
stays unmutatable — for the accurate reason now — and the breach is raised
as a maintainer decision: speed the build, move the target by ADR (arguing
why 60 s was wrong rather than why 87 s is convenient), or withdraw the
row.

The measurement itself had a real bug, found only by cross-validation.
getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child,
so it attributed cargo's memory to the workload and reported 38.2 MB for a
run that used 12.3 MB — a 3x over-report that was plausible, passed its
target, and would have been published. Fixed with os.wait4, which returns
that specific child's rusage, and the self-test now cross-checks against
/usr/bin/time -v.

That is the false-accusation shape in the measurement layer rather than
the mutation layer: an instrument confidently reporting a number it had
not earned.

Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather
than running `cargo clean`, so measuring the metric does not cost several
minutes of rebuild afterwards. A metric that punishes its own measurement
gets measured once and never again.

M-D1-MUT: 6 -> 7 of 14.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 18:59:32 +02:00
parent fd67813876
commit e7312b3b8a
8 changed files with 301 additions and 11 deletions

View file

@ -182,6 +182,48 @@ argument**, which is a legitimate outcome and possibly the right one for
AM-5 on a machine-dependent number. What is not legitimate is a third pass
leaving them blank while the scoreboard reports elsewhere.
**Delivered — `tools/runtime-metrics.py`. One passes, one breaches.**
**AM-9: met, and comfortably.** 13.4 MB peak RSS against a ≤64 MB target,
**4.8× headroom**. Gated and in `make all` (`--fast`, ~1 s). The row that
CB-EV-0001 called "very unlikely to bind" was right — but it is now
*measured* rather than assumed, and verified red by a property mutation
(a 300 MB allocation in the workload).
**AM-5: BREACHED, on both readings, on the machine the spec names.**
```text
dev toolchain (default features) 87.0 s [FAIL target <= 60 s]
shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s]
```
Measured on `bnt-lap001`, 8 cores — the machine `specs/GameKernel.md` §5
names, so this is a direct comparison, not a directional one. A row
declared `recorded not gated` and never recorded turns out, on first
measurement, to **fail its own target by 45%**.
The tool **reports and exits 0**, because the spec says the row is not
gated. Gating it is a spec change and needs an ADR; a tool that promotes
itself is how a target starts binding without anyone deciding it should.
So AM-5 stays `unmutatable` — for the accurate reason now — and the breach
is **raised as a maintainer decision**, not resolved here. Three honest
options: speed the build, move the target by ADR (arguing why 60 s was
wrong rather than why 87 s is convenient), or withdraw the row.
**The measurement had a real bug, found by cross-validation.**
`getrusage(RUSAGE_CHILDREN)` is a high-water mark across *every* reaped
child, so it attributed `cargo`'s memory to the workload and reported
**38.2 MB for a run that used 12.3 MB** — a 3× over-report. Fixed with
`os.wait4`, which returns that specific child's rusage. The self-test now
**cross-checks against `/usr/bin/time -v`** (13.4 vs 12.4 MB), which is
the only reason the bug was visible at all: the wrong number was
plausible, passed its target, and would have been published.
That is FA in the measurement layer rather than the mutation layer — an
instrument confidently reporting a number it had not earned.
**M-D1-MUT: 6 → 7 of 14.**
## Task: AM-4c — target it or drop it
```task