CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches
AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind" was right, but it is now measured rather than assumed, and verified red by a property mutation (a 300 MB allocation in the workload). AM-5 is BREACHED on both readings, on the machine the spec names: dev toolchain (default features) 87.0 s [FAIL target <= 60 s] shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s] bnt-lap001, 8 cores — a direct comparison, not a directional one. A row declared "recorded not gated" and never recorded fails its own target by 45% on first measurement. The tool reports and exits 0 because the spec says the row is ungated. Gating it is a spec change needing an ADR; a tool that promotes itself is how a target starts binding without anyone deciding it should. So AM-5 stays unmutatable — for the accurate reason now — and the breach is raised as a maintainer decision: speed the build, move the target by ADR (arguing why 60 s was wrong rather than why 87 s is convenient), or withdraw the row. The measurement itself had a real bug, found only by cross-validation. getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child, so it attributed cargo's memory to the workload and reported 38.2 MB for a run that used 12.3 MB — a 3x over-report that was plausible, passed its target, and would have been published. Fixed with os.wait4, which returns that specific child's rusage, and the self-test now cross-checks against /usr/bin/time -v. That is the false-accusation shape in the measurement layer rather than the mutation layer: an instrument confidently reporting a number it had not earned. Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather than running `cargo clean`, so measuring the metric does not cost several minutes of rebuild afterwards. A metric that punishes its own measurement gets measured once and never again. M-D1-MUT: 6 -> 7 of 14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
fd67813876
commit
e7312b3b8a
8 changed files with 301 additions and 11 deletions
14
Makefile
14
Makefile
|
|
@ -24,7 +24,7 @@ TOOLS := $(REPO)/tools
|
|||
# Every cargo recipe runs at the repo root; the shell does not persist cd.
|
||||
IN_REPO := cd $(REPO) &&
|
||||
|
||||
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics loc all
|
||||
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time loc all
|
||||
|
||||
## fmt + clippy (deny warnings) + HashMap deny-lint
|
||||
check:
|
||||
|
|
@ -42,6 +42,15 @@ dep-weight:
|
|||
coverage:
|
||||
$(PY) $(TOOLS)/rule-coverage.py
|
||||
|
||||
# AM-9 peak RSS (fast, gated). AM-5 needs a clean build — see build-time.
|
||||
runtime-metrics:
|
||||
$(PY) $(TOOLS)/runtime-metrics.py --fast
|
||||
|
||||
# AM-5: clean release build, ~90s, into a throwaway CARGO_TARGET_DIR so the
|
||||
# working cache survives. Reported not gated per GameKernel §5.
|
||||
build-time:
|
||||
$(PY) $(TOOLS)/runtime-metrics.py
|
||||
|
||||
# AM-2 (M-D1-SPL, AM-1's anti-gaming pair) and AM-3.
|
||||
size-metrics:
|
||||
$(PY) $(TOOLS)/size-metrics.py
|
||||
|
|
@ -70,6 +79,7 @@ self-tests:
|
|||
$(PY) $(TOOLS)/facts.py --self-test
|
||||
$(PY) $(TOOLS)/mutation-check.py --self-test
|
||||
$(PY) $(TOOLS)/size-metrics.py --self-test
|
||||
$(PY) $(TOOLS)/runtime-metrics.py --self-test
|
||||
|
||||
# T01 positive control: prove the environment fix, do not assume it. Runs
|
||||
# every tool from a foreign working directory with a PATH that has no
|
||||
|
|
@ -147,4 +157,4 @@ loc:
|
|||
printf '%-28s %s\n' $$d "$$(find $$d/src -name '*.rs' | xargs cat | grep -vcE '^\s*(//|$$)')"; \
|
||||
done
|
||||
|
||||
all: check test sim coverage size-metrics dep-weight self-tests env-test facts-check loop-lint bench-test
|
||||
all: check test sim coverage size-metrics runtime-metrics dep-weight self-tests env-test facts-check loop-lint bench-test
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue