CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches

AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x
headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind"
was right, but it is now measured rather than assumed, and verified red by
a property mutation (a 300 MB allocation in the workload).

AM-5 is BREACHED on both readings, on the machine the spec names:

  dev toolchain (default features)        87.0 s  [FAIL target <= 60 s]
  shipped runtime (--no-default-features) 61.3 s  [FAIL target <= 60 s]

bnt-lap001, 8 cores — a direct comparison, not a directional one. A row
declared "recorded not gated" and never recorded fails its own target by
45% on first measurement.

The tool reports and exits 0 because the spec says the row is ungated.
Gating it is a spec change needing an ADR; a tool that promotes itself is
how a target starts binding without anyone deciding it should. So AM-5
stays unmutatable — for the accurate reason now — and the breach is raised
as a maintainer decision: speed the build, move the target by ADR (arguing
why 60 s was wrong rather than why 87 s is convenient), or withdraw the
row.

The measurement itself had a real bug, found only by cross-validation.
getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child,
so it attributed cargo's memory to the workload and reported 38.2 MB for a
run that used 12.3 MB — a 3x over-report that was plausible, passed its
target, and would have been published. Fixed with os.wait4, which returns
that specific child's rusage, and the self-test now cross-checks against
/usr/bin/time -v.

That is the false-accusation shape in the measurement layer rather than
the mutation layer: an instrument confidently reporting a number it had
not earned.

Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather
than running `cargo clean`, so measuring the metric does not cost several
minutes of rebuild afterwards. A metric that punishes its own measurement
gets measured once and never again.

M-D1-MUT: 6 -> 7 of 14.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 18:59:32 +02:00
parent fd67813876
commit e7312b3b8a
8 changed files with 301 additions and 11 deletions

View file

@ -24,7 +24,7 @@ TOOLS := $(REPO)/tools
# Every cargo recipe runs at the repo root; the shell does not persist cd.
IN_REPO := cd $(REPO) &&
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics loc all
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time loc all
## fmt + clippy (deny warnings) + HashMap deny-lint
check:
@ -42,6 +42,15 @@ dep-weight:
coverage:
$(PY) $(TOOLS)/rule-coverage.py
# AM-9 peak RSS (fast, gated). AM-5 needs a clean build — see build-time.
runtime-metrics:
$(PY) $(TOOLS)/runtime-metrics.py --fast
# AM-5: clean release build, ~90s, into a throwaway CARGO_TARGET_DIR so the
# working cache survives. Reported not gated per GameKernel §5.
build-time:
$(PY) $(TOOLS)/runtime-metrics.py
# AM-2 (M-D1-SPL, AM-1's anti-gaming pair) and AM-3.
size-metrics:
$(PY) $(TOOLS)/size-metrics.py
@ -70,6 +79,7 @@ self-tests:
$(PY) $(TOOLS)/facts.py --self-test
$(PY) $(TOOLS)/mutation-check.py --self-test
$(PY) $(TOOLS)/size-metrics.py --self-test
$(PY) $(TOOLS)/runtime-metrics.py --self-test
# T01 positive control: prove the environment fix, do not assume it. Runs
# every tool from a foreign working directory with a PATH that has no
@ -147,4 +157,4 @@ loc:
printf '%-28s %s\n' $$d "$$(find $$d/src -name '*.rs' | xargs cat | grep -vcE '^\s*(//|$$)')"; \
done
all: check test sim coverage size-metrics dep-weight self-tests env-test facts-check loop-lint bench-test
all: check test sim coverage size-metrics runtime-metrics dep-weight self-tests env-test facts-check loop-lint bench-test