T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ee1ee485b2
commit
fed422a3a3
10 changed files with 827 additions and 242 deletions
|
|
@ -16,7 +16,7 @@ that cannot be found is reported and the run exits non-zero rather than
|
|||
silently under-reporting the total — under-reporting is the exact
|
||||
direction this metric could be gamed.
|
||||
|
||||
Usage: python3 tools/dep-weight.py [--json]
|
||||
Usage: python3 tools/dep-weight.py [--json] [--self-test]
|
||||
"""
|
||||
|
||||
import glob
|
||||
|
|
@ -88,7 +88,50 @@ def source_lines(name, version):
|
|||
return 0
|
||||
|
||||
|
||||
def self_test():
|
||||
"""Each assertion pins a failure this tool must detect.
|
||||
|
||||
The controls that matter here are: an unlocatable crate must not be
|
||||
silently counted as zero lines (that under-reports, the direction this
|
||||
metric could be gamed), and a target breach must fail rather than
|
||||
merely print.
|
||||
"""
|
||||
results = []
|
||||
|
||||
def check(name, ok, detail=""):
|
||||
results.append((name, ok, detail))
|
||||
|
||||
# A crate that does not exist must measure zero, so the caller's
|
||||
# `lines == 0` guard fires rather than silently shrinking the total.
|
||||
check("unlocatable crate measures zero (so the guard fires)",
|
||||
source_lines("definitely-not-a-real-crate-xyz", "9.9.9") == 0)
|
||||
|
||||
# A crate we do depend on must measure non-zero, or the guard above
|
||||
# would fire on everything and the tool would never report at all.
|
||||
real = source_lines("serde", "1")
|
||||
check("a real vendored crate measures non-zero", real > 0,
|
||||
f"{real:,} lines")
|
||||
|
||||
# Targets must be present and numeric — a missing target would make
|
||||
# the breach check vacuous.
|
||||
check("targets defined for every configuration",
|
||||
set(TARGETS) == set(CONFIGS) and all(
|
||||
isinstance(v, int) and v > 0 for v in TARGETS.values()),
|
||||
f"{TARGETS}")
|
||||
|
||||
print("dep-weight self-test (positive control)")
|
||||
ok = True
|
||||
for name, passed, detail in results:
|
||||
print(f" [{'ok ' if passed else 'FAIL'}] {name}"
|
||||
+ (f" — {detail}" if detail else ""))
|
||||
ok &= passed
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
def main():
|
||||
if "--self-test" in sys.argv:
|
||||
return self_test()
|
||||
|
||||
report = {}
|
||||
missing = []
|
||||
for label, args in CONFIGS.items():
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue