T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
#!/usr/bin/env python3
|
|
|
|
|
"""Executable checks for specs/InnerLoop.md rules (CB-WP-0003 T01).
|
|
|
|
|
|
|
|
|
|
The loop's own rules were prose. A rule nobody can run is a suggestion,
|
|
|
|
|
and the audit in history/260731-inner-loop-rule-audit.md found several
|
|
|
|
|
that were already being violated with no signal. This makes the
|
|
|
|
|
mechanically-checkable ones fail a command.
|
|
|
|
|
|
|
|
|
|
Each check names the InnerLoop rule it enforces. Checks that cannot be
|
|
|
|
|
made mechanical are recorded in the audit as `checkable` or `decorative`
|
|
|
|
|
and are deliberately absent here — see the audit for why.
|
|
|
|
|
|
|
|
|
|
Positive control (InnerLoop v1.1 §Step 5): --self-test asserts each check
|
|
|
|
|
actually detects its failure, using fixtures with known answers. A linter
|
|
|
|
|
that passes everything because its matcher is broken is the same defect
|
|
|
|
|
class as a benchmark timing rejected work.
|
|
|
|
|
|
|
|
|
|
Usage:
|
|
|
|
|
python3 tools/loop-lint.py # lint the repo
|
|
|
|
|
python3 tools/loop-lint.py --self-test # positive control
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
import os
|
|
|
|
|
import re
|
|
|
|
|
import sys
|
|
|
|
|
|
2026-07-31 10:13:52 +02:00
|
|
|
from repo import ROOT as REPO # noqa: E402 (single source of fact, T01)
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
LOADABILITY_LIMIT = 400
|
|
|
|
|
|
|
|
|
|
# Artifact classes the loop produces. history/ is an append-only trail
|
|
|
|
|
# (verbatim challenge text is not something to split), so it is exempt.
|
|
|
|
|
LOOP_DIRS = ("specs", "research", "decisions", "evidence", "workplans")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class Finding:
|
|
|
|
|
def __init__(self, rule, path, detail):
|
|
|
|
|
self.rule, self.path, self.detail = rule, path, detail
|
|
|
|
|
|
|
|
|
|
def __str__(self):
|
|
|
|
|
return f" [{self.rule}] {self.path}\n {self.detail}"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _md_files(root=REPO):
|
|
|
|
|
"""Loop artifacts only.
|
|
|
|
|
|
|
|
|
|
Vendored third-party trees (a baseline harness ships its own
|
|
|
|
|
node_modules) are not artifacts this loop produces, and linting them
|
|
|
|
|
buries the two real findings under eighteen irrelevant ones.
|
|
|
|
|
"""
|
|
|
|
|
for d in LOOP_DIRS:
|
|
|
|
|
base = os.path.join(root, d)
|
|
|
|
|
for dirpath, dirnames, files in os.walk(base):
|
|
|
|
|
dirnames[:] = [x for x in dirnames if x != "node_modules"]
|
|
|
|
|
for f in sorted(files):
|
|
|
|
|
if f.endswith(".md"):
|
|
|
|
|
yield os.path.relpath(os.path.join(dirpath, f), root)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def check_loadability(root=REPO):
|
|
|
|
|
"""§Agentic-efficiency 1 — every loop artifact stays under ~400 lines."""
|
|
|
|
|
out = []
|
|
|
|
|
for rel in _md_files(root):
|
|
|
|
|
with open(os.path.join(root, rel)) as fh:
|
|
|
|
|
n = sum(1 for _ in fh)
|
|
|
|
|
if n > LOADABILITY_LIMIT:
|
|
|
|
|
out.append(
|
|
|
|
|
Finding(
|
|
|
|
|
"loadability",
|
|
|
|
|
rel,
|
|
|
|
|
f"{n} lines exceeds the ~{LOADABILITY_LIMIT}-line limit; "
|
|
|
|
|
f"split and link with relative paths",
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def check_evidence_no_unmeasured(root=REPO):
|
|
|
|
|
"""§Rubric — `unmeasured` is legal in a survey, illegal in an evidence file."""
|
|
|
|
|
out = []
|
|
|
|
|
base = os.path.join(root, "evidence")
|
|
|
|
|
if not os.path.isdir(base):
|
|
|
|
|
return out
|
|
|
|
|
for f in sorted(os.listdir(base)):
|
|
|
|
|
if not f.endswith(".md"):
|
|
|
|
|
continue
|
|
|
|
|
rel = os.path.join("evidence", f)
|
|
|
|
|
for i, line in enumerate(open(os.path.join(root, rel)), 1):
|
|
|
|
|
# A row asserting the verdict, not prose discussing the word.
|
|
|
|
|
if re.search(r"\|\s*unmeasured\s*\|", line):
|
|
|
|
|
out.append(
|
|
|
|
|
Finding("evidence-unmeasured", f"{rel}:{i}",
|
|
|
|
|
"verdict `unmeasured` in an evidence table")
|
|
|
|
|
)
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
|
|
|
def check_own_cost_not_quoted(root=REPO):
|
|
|
|
|
"""§Quoting a cost — an evidence file may not quote its own pass's
|
|
|
|
|
cost as final.
|
|
|
|
|
|
|
|
|
|
Six passes under-reported themselves by 30-45%, never once high, so a
|
|
|
|
|
self-quoted figure is not a rounding error but a known bias. The check
|
|
|
|
|
is deliberately narrow: it fires only when a line names the file's OWN
|
|
|
|
|
workplan beside a dollar amount and does not mark it provisional.
|
|
|
|
|
Quoting an earlier pass is exactly what the rule asks for.
|
|
|
|
|
|
|
|
|
|
**Binds forward, from the pass that wrote it down.** The rule was
|
|
|
|
|
written in CB-WP-0019 after six passes had each under-reported
|
|
|
|
|
themselves, and those six evidence files are the *evidence for the
|
|
|
|
|
rule*. Firing on them would demand the record be edited to remove the
|
|
|
|
|
thing it proves — the same category error as putting a live `fact:`
|
|
|
|
|
tag on a dated measurement. So the check applies from CB-WP-0019 on,
|
|
|
|
|
and the older figures stay as they were reported.
|
|
|
|
|
"""
|
|
|
|
|
BINDS_FROM = 19
|
|
|
|
|
out = []
|
|
|
|
|
base = os.path.join(root, "evidence")
|
|
|
|
|
if not os.path.isdir(base):
|
|
|
|
|
return out
|
|
|
|
|
for f in sorted(os.listdir(base)):
|
|
|
|
|
if not f.endswith(".md"):
|
|
|
|
|
continue
|
|
|
|
|
rel = os.path.join("evidence", f)
|
|
|
|
|
text = open(os.path.join(root, rel)).read()
|
|
|
|
|
# The pass an evidence file belongs to is named in its header.
|
|
|
|
|
m = re.search(r"\b(CB-WP-\d{4})\b", text)
|
|
|
|
|
if not m:
|
|
|
|
|
continue
|
|
|
|
|
own = m.group(1)
|
|
|
|
|
if int(own.rsplit("-", 1)[1]) < BINDS_FROM:
|
|
|
|
|
continue
|
|
|
|
|
for i, line in enumerate(text.splitlines(), 1):
|
|
|
|
|
if own not in line or "$" not in line:
|
|
|
|
|
continue
|
|
|
|
|
if "provisional" in line.lower() or "not quoted" in line.lower():
|
|
|
|
|
continue
|
|
|
|
|
out.append(
|
|
|
|
|
Finding("own-cost", f"{rel}:{i}",
|
|
|
|
|
f"quotes {own}'s own cost as final; re-run the "
|
|
|
|
|
f"instrument and quote a settled pass, or mark it "
|
|
|
|
|
f"provisional")
|
|
|
|
|
)
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def check_workplan_lifecycle(root=REPO):
|
|
|
|
|
"""§Workplan lifecycle — `ready` means declared and NOT started.
|
|
|
|
|
|
|
|
|
|
A workplan with work done in it that still says `ready` answers the
|
|
|
|
|
wrong question: the status should say whether anyone is on it, not
|
|
|
|
|
only whether it is finished.
|
|
|
|
|
"""
|
|
|
|
|
out = []
|
|
|
|
|
base = os.path.join(root, "workplans")
|
|
|
|
|
if not os.path.isdir(base):
|
|
|
|
|
return out
|
|
|
|
|
for f in sorted(os.listdir(base)):
|
|
|
|
|
if not f.endswith(".md"):
|
|
|
|
|
continue
|
|
|
|
|
rel = os.path.join("workplans", f)
|
|
|
|
|
text = open(os.path.join(root, rel)).read()
|
|
|
|
|
m = re.search(r"^status:\s*(\S+)", text, re.M)
|
|
|
|
|
if not m:
|
|
|
|
|
continue
|
|
|
|
|
status = m.group(1)
|
|
|
|
|
# Parse the ```task blocks, not every `status:` in the file. The
|
|
|
|
|
# first version stripped the leading match assuming it was the
|
|
|
|
|
# frontmatter — which silently dropped a real task the moment the
|
|
|
|
|
# frontmatter said `ready` or `active`, because those do not match
|
|
|
|
|
# the task vocabulary. Caught by this check's own self-test.
|
|
|
|
|
tasks = [
|
|
|
|
|
t.group(1)
|
|
|
|
|
for block in re.findall(r"```task\n(.*?)```", text, re.S)
|
|
|
|
|
for t in [re.search(r"^status:\s*(\S+)", block, re.M)]
|
|
|
|
|
if t
|
|
|
|
|
]
|
|
|
|
|
if not tasks:
|
|
|
|
|
continue
|
|
|
|
|
closed = sum(1 for t in tasks if t in ("done", "cancel"))
|
|
|
|
|
started = closed > 0
|
|
|
|
|
if status == "ready" and started:
|
|
|
|
|
out.append(Finding("lifecycle", rel,
|
|
|
|
|
"still `ready` but work has started — use `active`"))
|
|
|
|
|
elif status == "active" and closed == len(tasks):
|
|
|
|
|
out.append(Finding("lifecycle", rel,
|
|
|
|
|
"every task is closed but status is `active` — use `done`"))
|
|
|
|
|
elif status == "done" and closed != len(tasks):
|
|
|
|
|
out.append(Finding("lifecycle", rel,
|
|
|
|
|
f"`done` with {len(tasks) - closed} task(s) still open"))
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
def check_survey_tier_and_chaos(root=REPO):
|
|
|
|
|
"""§Loop tiers — tier declared, and the chaos roll recorded every time."""
|
|
|
|
|
out = []
|
|
|
|
|
base = os.path.join(root, "research")
|
|
|
|
|
if not os.path.isdir(base):
|
|
|
|
|
return out
|
|
|
|
|
for f in sorted(os.listdir(base)):
|
|
|
|
|
if not f.endswith(".md"):
|
|
|
|
|
continue
|
|
|
|
|
rel = os.path.join("research", f)
|
|
|
|
|
text = open(os.path.join(root, rel)).read()
|
|
|
|
|
if not re.search(r"^tier:\s*[SML]\b", text, re.M):
|
|
|
|
|
out.append(Finding("tier-declared", rel, "no `tier:` declaration"))
|
|
|
|
|
elif "chaos" not in text.lower():
|
|
|
|
|
out.append(
|
|
|
|
|
Finding("chaos-recorded", rel,
|
|
|
|
|
"tier declared without the chaos roll; the rule requires "
|
|
|
|
|
"recording it even when it changes nothing")
|
|
|
|
|
)
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def check_review_trail(root=REPO):
|
|
|
|
|
"""§Step 2 — a tier-L survey carries research/challenge/response history."""
|
|
|
|
|
out = []
|
|
|
|
|
base = os.path.join(root, "research")
|
|
|
|
|
hist = os.path.join(root, "history")
|
|
|
|
|
if not (os.path.isdir(base) and os.path.isdir(hist)):
|
|
|
|
|
return out
|
|
|
|
|
files = os.listdir(hist)
|
|
|
|
|
for f in sorted(os.listdir(base)):
|
|
|
|
|
if not f.endswith(".md"):
|
|
|
|
|
continue
|
|
|
|
|
rel = os.path.join("research", f)
|
|
|
|
|
text = open(os.path.join(root, rel)).read()
|
|
|
|
|
if not re.search(r"^tier:\s*L\b", text, re.M):
|
|
|
|
|
continue
|
|
|
|
|
if not re.search(r"^status:\s*approved", text, re.M):
|
|
|
|
|
continue
|
|
|
|
|
for kind in ("challenge", "response"):
|
|
|
|
|
if not any(x.endswith(f"-{kind}.md") and kind in x for x in files):
|
|
|
|
|
out.append(
|
|
|
|
|
Finding("review-trail", rel,
|
|
|
|
|
f"tier-L approved survey with no history/*-{kind}.md")
|
|
|
|
|
)
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def check_reporting_tools_self_test(root=REPO):
|
|
|
|
|
"""§Step 5 v1.1 — every tool that reports a number exposes --self-test."""
|
|
|
|
|
out = []
|
|
|
|
|
base = os.path.join(root, "tools")
|
|
|
|
|
if not os.path.isdir(base):
|
|
|
|
|
return out
|
|
|
|
|
for f in sorted(os.listdir(base)):
|
|
|
|
|
if not f.endswith(".py"):
|
|
|
|
|
continue
|
|
|
|
|
rel = os.path.join("tools", f)
|
|
|
|
|
text = open(os.path.join(root, rel)).read()
|
|
|
|
|
if "--self-test" not in text:
|
|
|
|
|
out.append(
|
|
|
|
|
Finding("self-test", rel,
|
|
|
|
|
"reporting tool with no --self-test entry point; "
|
|
|
|
|
"nothing verifies its positive control still works")
|
|
|
|
|
)
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
def check_gate_registry(root=REPO):
|
|
|
|
|
"""ADR-0006 D3 — every control gate is in `gates.toml`, and every
|
|
|
|
|
entry names a real target.
|
|
|
|
|
|
|
|
|
|
The failure this prevents is drift in the direction nobody notices: a
|
|
|
|
|
gate added to `make all` with no registry entry never acquires a
|
|
|
|
|
review date, which is how five mechanisms accumulated with no way to
|
|
|
|
|
retire any of them.
|
|
|
|
|
"""
|
|
|
|
|
out = []
|
|
|
|
|
registry = os.path.join(root, "gates.toml")
|
|
|
|
|
makefile = os.path.join(root, "Makefile")
|
|
|
|
|
if not (os.path.exists(registry) and os.path.exists(makefile)):
|
|
|
|
|
return out
|
|
|
|
|
try:
|
|
|
|
|
import tomllib
|
|
|
|
|
except ModuleNotFoundError: # pragma: no cover
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
with open(registry, "rb") as fh:
|
|
|
|
|
data = tomllib.load(fh)
|
|
|
|
|
gates = data.get("gate") or []
|
|
|
|
|
if not gates:
|
|
|
|
|
return [Finding("gates", "gates.toml", "registry contains no gates")]
|
|
|
|
|
registered = {g.get("target") for g in gates if g.get("target")}
|
|
|
|
|
exempt = set(data.get("not_control_gates") or [])
|
|
|
|
|
|
|
|
|
|
text = open(makefile).read()
|
|
|
|
|
m = re.search(r"^all:(.*)$", text, re.M)
|
|
|
|
|
deps = m.group(1).split() if m else []
|
|
|
|
|
targets = {ln.split(":", 1)[0].strip() for ln in text.splitlines()
|
|
|
|
|
if ln and not ln[0].isspace() and ":" in ln and not ln.startswith(".")}
|
|
|
|
|
|
|
|
|
|
for dep in deps:
|
|
|
|
|
if dep not in registered and dep not in exempt:
|
|
|
|
|
out.append(Finding(
|
|
|
|
|
"gates", "gates.toml",
|
|
|
|
|
f"`make all` runs {dep!r}, which is neither a registered "
|
|
|
|
|
f"control gate nor listed in not_control_gates — classify it, "
|
|
|
|
|
f"so it cannot acquire permanence without a review date"))
|
|
|
|
|
for target in sorted(registered):
|
|
|
|
|
if target not in targets:
|
|
|
|
|
out.append(Finding(
|
|
|
|
|
"gates", "gates.toml",
|
|
|
|
|
f"entry names target {target!r}, which the Makefile lacks"))
|
2026-08-08 09:55:41 +02:00
|
|
|
|
|
|
|
|
# CB-REV-0002 #7, generalised. The registry checked that a gate's
|
|
|
|
|
# target EXISTS; it never checked that the gate RUNS. `panels` was
|
|
|
|
|
# added to close "the harness is run by no gate" and passed lint while
|
|
|
|
|
# running nowhere -- the same defect, one level up.
|
|
|
|
|
#
|
|
|
|
|
# A gate may legitimately be manual. It must SAY so, and one that says
|
|
|
|
|
# it runs in `make all` must be there.
|
|
|
|
|
for g in gates:
|
|
|
|
|
target = g.get("target")
|
|
|
|
|
if not target:
|
|
|
|
|
continue
|
|
|
|
|
cadence = g.get("cadence")
|
CB-REV-0003: round 3, and three of four FATAL came from round 2's fixes
The pattern is now measured over three rounds: 5 fatal, then 3 (2 from the
previous round's corrections), then 4 (3 from them). The corrections are
not getting safer.
FATAL 1: round 2's short-cell assertion went into regulation.rs only.
attack-value.rs — which produced every number in CB-EV-0030's DARVO table
— still just warned, and the gate registered to close the finding claimed
the property for both.
FATAL 2, the sharpest of the three rounds: counting games proves they
STARTED. Stopping the engine after one round gives 200 games, all-zero
columns and exit 0 — byte for byte the signature CB-EV-0030 says the
instrumentation distinguishes from a real result. Both harnesses now
require every counted game to have reached an outcome over five rounds.
FATAL 3: round 2's `.csv` filter was applied to all three loops, so
catalog.yaml and rules_delta.yaml — whose missing digests were round 1's
finding — were recorded and then never compared, and never checked against
upstream at all. Only the parser loop filters now.
FATAL 4: five of six tiebreak comparators had no coverage. GR-E04's
tiebreak never executes in any scenario. All four are now covered and
mutation-verified; the Blame key needed compensating claims to be
reachable at all, since Blame also lowers the coalition score.
SERIOUS: "peak held" computed the same number as "peak assigned" for every
possible input — the real gap was that START_STRESS was an unchecked
constant, now read off the dealt state; cadence="none" was a pure
loophole, removed; sibling discovery swapped a hand-written list for
hand-written globs and missed metadata.json and VARIANT.md, both named in
the package's own changed_files — now walked, and it found them
immediately; and "~72,000 games" was unsourced, make panels runs 17,600.
Also separated two kinds of number that were presented alike: seats×games
is invariant, 363 and 29 vary 7.1%-11.5% across samples.
Round 4 owed. The conclusion is not that the work is nearly right — it is
that author-made corrections to measurement work should be assumed
defective until a fresh reader has attacked them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 10:26:25 +02:00
|
|
|
if cadence not in ("all", "manual"):
|
2026-08-08 09:55:41 +02:00
|
|
|
out.append(Finding(
|
|
|
|
|
"gates", "gates.toml",
|
CB-REV-0003: round 3, and three of four FATAL came from round 2's fixes
The pattern is now measured over three rounds: 5 fatal, then 3 (2 from the
previous round's corrections), then 4 (3 from them). The corrections are
not getting safer.
FATAL 1: round 2's short-cell assertion went into regulation.rs only.
attack-value.rs — which produced every number in CB-EV-0030's DARVO table
— still just warned, and the gate registered to close the finding claimed
the property for both.
FATAL 2, the sharpest of the three rounds: counting games proves they
STARTED. Stopping the engine after one round gives 200 games, all-zero
columns and exit 0 — byte for byte the signature CB-EV-0030 says the
instrumentation distinguishes from a real result. Both harnesses now
require every counted game to have reached an outcome over five rounds.
FATAL 3: round 2's `.csv` filter was applied to all three loops, so
catalog.yaml and rules_delta.yaml — whose missing digests were round 1's
finding — were recorded and then never compared, and never checked against
upstream at all. Only the parser loop filters now.
FATAL 4: five of six tiebreak comparators had no coverage. GR-E04's
tiebreak never executes in any scenario. All four are now covered and
mutation-verified; the Blame key needed compensating claims to be
reachable at all, since Blame also lowers the coalition score.
SERIOUS: "peak held" computed the same number as "peak assigned" for every
possible input — the real gap was that START_STRESS was an unchecked
constant, now read off the dealt state; cadence="none" was a pure
loophole, removed; sibling discovery swapped a hand-written list for
hand-written globs and missed metadata.json and VARIANT.md, both named in
the package's own changed_files — now walked, and it found them
immediately; and "~72,000 games" was unsourced, make panels runs 17,600.
Also separated two kinds of number that were presented alike: seats×games
is invariant, 363 and 29 vary 7.1%-11.5% across samples.
Round 4 owed. The conclusion is not that the work is nearly right — it is
that author-made corrections to measurement work should be assumed
defective until a fresh reader has attacked them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 10:26:25 +02:00
|
|
|
f"{target!r} declares no cadence — say `all` or `manual`. "
|
|
|
|
|
f"`none` was a pure loophole: a real target that runs "
|
|
|
|
|
f"nowhere and passes, which is the condition this rule "
|
|
|
|
|
f"exists to prevent (CB-REV-0003 #7)"))
|
2026-08-08 09:55:41 +02:00
|
|
|
elif cadence == "all" and target not in deps:
|
|
|
|
|
out.append(Finding(
|
|
|
|
|
"gates", "gates.toml",
|
|
|
|
|
f"{target!r} declares cadence=\"all\" and `make all` does not "
|
|
|
|
|
f"run it — a gate that does not run is not a gate"))
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
CB-WP-0036 done: the pace flag and the first ornament declarations
--pace speed|interactive, defaulting to Speed. Nothing reads it yet, and
that is the point: it is the seam clay-animate attaches to, and a seam is
cheap now where a retrofit would not be. A misspelt pace is refused rather
than defaulting, because quietly falling back to Speed would look exactly
like the renderer being broken.
I3 is asserted rather than intended: the same scripted game at both paces
must produce a byte-identical serialised recording and the same end state
hash. Mutation-proven — leak the pace into the seed and it fails with "the
recording differs by pace, so a renderer has become mechanism".
specs/OrnamentRegister.md carries four declarations. This reverses the
reasoning written in T03 earlier, which said the first declarations would
come from F18's unvendored files: instances already existed. Hand order is
what prompted the category, and "who deals" was the maintainer's own
example. O3 is the interesting one — seat ORDER is mechanism because
GR-R08 rotates Lead, while where a seat is drawn is not.
I5 is executable: check_ornament_falsifier fails any row still declared
that names no falsifier, mutation-proven red on O1. Presence, never
adequacy, and the finding text says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 22:15:38 +02:00
|
|
|
def check_ornament_falsifier(root=REPO):
|
|
|
|
|
"""Ornamentation I5 — an open declaration names what would refute it.
|
|
|
|
|
|
|
|
|
|
**A declaration is a claim that something does not matter**, and this
|
|
|
|
|
project's finding register is largely a list of times that claim was
|
|
|
|
|
wrong. One that cannot be wrong is not a claim, it is a preference
|
|
|
|
|
with a table row.
|
|
|
|
|
|
|
|
|
|
Presence, never adequacy -- the same split as ADR-0018 D5.
|
|
|
|
|
"""
|
|
|
|
|
out = []
|
|
|
|
|
reg = os.path.join(root, "specs", "OrnamentRegister.md")
|
|
|
|
|
if not os.path.exists(reg):
|
|
|
|
|
return out
|
|
|
|
|
with open(reg) as fh:
|
|
|
|
|
text = fh.read()
|
|
|
|
|
m = re.search(r"<!-- ornament-register:begin -->(.*?)"
|
|
|
|
|
r"<!-- ornament-register:end -->", text, re.S)
|
|
|
|
|
if not m:
|
|
|
|
|
return out
|
|
|
|
|
for line in m.group(1).splitlines():
|
|
|
|
|
line = line.strip()
|
|
|
|
|
if not line.startswith("|") or line.startswith("|---"):
|
|
|
|
|
continue
|
|
|
|
|
cells = [c.strip() for c in line.strip("|").split("|")]
|
|
|
|
|
if len(cells) != 5 or cells[0] == "id":
|
|
|
|
|
continue
|
|
|
|
|
oid, state = cells[0], cells[3]
|
|
|
|
|
# A refuted or withdrawn row is history; the rule binds a claim
|
|
|
|
|
# that is still being made.
|
|
|
|
|
if state != "declared":
|
|
|
|
|
continue
|
|
|
|
|
body = re.search(rf"^- \*\*{re.escape(oid)} [^\n]*(?:\n(?!- \*\*O\d).*)*",
|
|
|
|
|
text, re.M)
|
|
|
|
|
if not body or "Falsifier:" not in body.group(0):
|
|
|
|
|
out.append(Finding(
|
|
|
|
|
"ornament",
|
|
|
|
|
"specs/OrnamentRegister.md",
|
|
|
|
|
f"{oid} is declared ornamentation and names no falsifier "
|
|
|
|
|
f"(Ornamentation.md I5). Say what would make it mechanism. "
|
|
|
|
|
f"NOTE: this checks presence, not adequacy.",
|
|
|
|
|
))
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
2026-08-07 11:35:41 +02:00
|
|
|
def check_sensitivity_stated(root=REPO):
|
|
|
|
|
"""GameDesign §1.4 / ADR-0018 — a finding whose claim is arithmetic
|
|
|
|
|
must name the variable it depends on.
|
|
|
|
|
|
|
|
|
|
**This checks PRESENCE, NEVER ADEQUACY.** It cannot tell whether the
|
|
|
|
|
variable named is the right one; that is the judgement the rule exists
|
|
|
|
|
to force, and it stays with the author and the reviewer. A green run
|
|
|
|
|
here means "somebody wrote a sensitivity line", not "the claim was
|
|
|
|
|
verified" -- and if it is ever read as the second, the control has
|
|
|
|
|
become a way of not looking.
|
|
|
|
|
|
|
|
|
|
Seven claims in this project were arithmetically correct about the
|
|
|
|
|
wrong subject. Three of them would have been caught by varying
|
|
|
|
|
something; this is the half of that a machine can see.
|
|
|
|
|
"""
|
|
|
|
|
out = []
|
|
|
|
|
reg = os.path.join(root, "specs", "FindingRegister.md")
|
|
|
|
|
if not os.path.exists(reg):
|
|
|
|
|
return out
|
|
|
|
|
text = open(reg).read()
|
|
|
|
|
try:
|
|
|
|
|
block = text.split("<!-- design-register:begin -->")[1] \
|
|
|
|
|
.split("<!-- design-register:end -->")[0]
|
|
|
|
|
except IndexError:
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
# Kinds whose claim is a quantity. `inert` and `unplayed` are about
|
|
|
|
|
# whether a thing happens at all, which has no denominator to get
|
|
|
|
|
# wrong.
|
|
|
|
|
ARITHMETIC = {"inconsistent", "degenerate", "underdetermined"}
|
|
|
|
|
exempt = set(re.findall(r"^<!-- sensitivity-exempt:\s*(\S+)\s+(.+?)\s*-->$",
|
|
|
|
|
text, re.M))
|
|
|
|
|
exempt_ids = {e[0] for e in exempt}
|
|
|
|
|
|
|
|
|
|
for line in block.splitlines():
|
|
|
|
|
line = line.strip()
|
|
|
|
|
if not line.startswith("|") or line.startswith("|---"):
|
|
|
|
|
continue
|
|
|
|
|
cells = [c.strip() for c in line.strip("|").split("|")]
|
|
|
|
|
if len(cells) != 7 or cells[0] == "id":
|
|
|
|
|
continue
|
|
|
|
|
fid, kind, state = cells[0], cells[1], cells[2]
|
|
|
|
|
# Closed rows are history; the rule binds what is still claimed.
|
|
|
|
|
# And a `note` is by definition a finding WITHOUT a reproduction
|
|
|
|
|
# (GameDesign §3.1) -- there is no measurement to be sensitive
|
|
|
|
|
# about, so requiring one would be asking for a sensitivity
|
|
|
|
|
# statement about nothing.
|
|
|
|
|
if state in ("withdrawn", "applied", "note") or kind not in ARITHMETIC:
|
|
|
|
|
continue
|
|
|
|
|
if fid in exempt_ids:
|
|
|
|
|
continue
|
|
|
|
|
# The prose block for this finding must say what moves it.
|
|
|
|
|
body = ""
|
|
|
|
|
m = re.search(rf"^- \*\*{re.escape(fid)} [^\n]*(?:\n(?!- \*\*F?U?\d).*)*",
|
|
|
|
|
text, re.M)
|
|
|
|
|
if m:
|
|
|
|
|
body = m.group(0)
|
|
|
|
|
if not re.search(r"varie[sd]|varying|sensitivit|moves with|held fixed|"
|
|
|
|
|
r"one number|second policy", body, re.I):
|
|
|
|
|
out.append(Finding(
|
|
|
|
|
"sensitivity", "specs/FindingRegister.md",
|
|
|
|
|
f"{fid} ({kind}) states a quantity and names no variable it "
|
|
|
|
|
f"depends on (GameDesign §1.4). Say what would move it, or "
|
|
|
|
|
f"add `<!-- sensitivity-exempt: {fid} <reason> -->`. "
|
|
|
|
|
f"NOTE: this checks presence, not adequacy."))
|
|
|
|
|
return out
|
|
|
|
|
|
|
|
|
|
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
CHECKS = (
|
|
|
|
|
check_loadability,
|
|
|
|
|
check_evidence_no_unmeasured,
|
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
|
|
|
check_own_cost_not_quoted,
|
|
|
|
|
check_workplan_lifecycle,
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
check_survey_tier_and_chaos,
|
|
|
|
|
check_review_trail,
|
|
|
|
|
check_reporting_tools_self_test,
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
check_gate_registry,
|
2026-08-07 11:35:41 +02:00
|
|
|
check_sensitivity_stated,
|
CB-WP-0036 done: the pace flag and the first ornament declarations
--pace speed|interactive, defaulting to Speed. Nothing reads it yet, and
that is the point: it is the seam clay-animate attaches to, and a seam is
cheap now where a retrofit would not be. A misspelt pace is refused rather
than defaulting, because quietly falling back to Speed would look exactly
like the renderer being broken.
I3 is asserted rather than intended: the same scripted game at both paces
must produce a byte-identical serialised recording and the same end state
hash. Mutation-proven — leak the pace into the seed and it fails with "the
recording differs by pace, so a renderer has become mechanism".
specs/OrnamentRegister.md carries four declarations. This reverses the
reasoning written in T03 earlier, which said the first declarations would
come from F18's unvendored files: instances already existed. Hand order is
what prompted the category, and "who deals" was the maintainer's own
example. O3 is the interesting one — seat ORDER is mechanism because
GR-R08 rotates Lead, while where a seat is drawn is not.
I5 is executable: check_ornament_falsifier fails any row still declared
that names no falsifier, mutation-proven red on O1. Presence, never
adequacy, and the finding text says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 22:15:38 +02:00
|
|
|
check_ornament_falsifier,
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def self_test():
|
|
|
|
|
"""Each check must DETECT its failure, not merely run."""
|
|
|
|
|
import shutil
|
|
|
|
|
import tempfile
|
|
|
|
|
|
|
|
|
|
results = []
|
|
|
|
|
|
|
|
|
|
def check(name, ok, detail=""):
|
|
|
|
|
results.append((name, ok, detail))
|
|
|
|
|
|
|
|
|
|
tmp = tempfile.mkdtemp()
|
|
|
|
|
try:
|
|
|
|
|
for d in LOOP_DIRS + ("tools", "history"):
|
|
|
|
|
os.makedirs(os.path.join(tmp, d), exist_ok=True)
|
|
|
|
|
|
|
|
|
|
# loadability: 401 lines must trip, 400 must not.
|
|
|
|
|
with open(os.path.join(tmp, "specs", "Big.md"), "w") as fh:
|
|
|
|
|
fh.write("x\n" * (LOADABILITY_LIMIT + 1))
|
|
|
|
|
with open(os.path.join(tmp, "specs", "Ok.md"), "w") as fh:
|
|
|
|
|
fh.write("x\n" * LOADABILITY_LIMIT)
|
|
|
|
|
f = check_loadability(tmp)
|
|
|
|
|
check("loadability detects overlong artifact",
|
|
|
|
|
len(f) == 1 and "Big.md" in f[0].path,
|
|
|
|
|
f"{len(f)} finding(s)")
|
|
|
|
|
|
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
|
|
|
# own-cost: a file quoting its OWN pass beside a dollar amount
|
|
|
|
|
# trips; the same line marked provisional does not; and a file
|
|
|
|
|
# quoting an EARLIER pass does not, because that is the rule.
|
|
|
|
|
def ev(name, body):
|
|
|
|
|
with open(os.path.join(tmp, "evidence", name), "w") as fh:
|
|
|
|
|
fh.write(body)
|
|
|
|
|
ev("CB-EV-0100-self.md", "CB-WP-0019 T04.\n| CB-WP-0019 | $9.99 |\n")
|
|
|
|
|
f = check_own_cost_not_quoted(tmp)
|
|
|
|
|
check("own-cost detects a pass quoting itself", len(f) == 1,
|
|
|
|
|
f"{len(f)} finding(s)")
|
|
|
|
|
ev("CB-EV-0100-self.md",
|
|
|
|
|
"CB-WP-0019 T04.\n| CB-WP-0019 | $9.99 provisional |\n")
|
|
|
|
|
check("own-cost accepts a figure marked provisional",
|
|
|
|
|
not check_own_cost_not_quoted(tmp))
|
|
|
|
|
ev("CB-EV-0100-self.md", "CB-WP-0019 T04.\n| CB-WP-0018 | $28.08 |\n")
|
|
|
|
|
check("own-cost accepts quoting an EARLIER pass",
|
|
|
|
|
not check_own_cost_not_quoted(tmp))
|
|
|
|
|
# and it binds forward: the six passes that PROVE the rule are the
|
|
|
|
|
# evidence for it, and must not be edited to satisfy it.
|
|
|
|
|
ev("CB-EV-0100-self.md", "CB-WP-0009 T04.\n| CB-WP-0009 | $6.73 |\n")
|
|
|
|
|
check("own-cost binds forward, not over the record it rests on",
|
|
|
|
|
not check_own_cost_not_quoted(tmp))
|
|
|
|
|
os.remove(os.path.join(tmp, "evidence", "CB-EV-0100-self.md"))
|
|
|
|
|
|
|
|
|
|
# lifecycle: `ready` with work started trips; `active` does not.
|
|
|
|
|
def wp(status, tasks):
|
|
|
|
|
body = f"---\nid: CB-WP-0100\nstatus: {status}\n---\n"
|
|
|
|
|
for t in tasks:
|
|
|
|
|
body += f"\n```task\nid: CB-WP-0100-T\nstatus: {t}\npriority: high\n```\n"
|
|
|
|
|
with open(os.path.join(tmp, "workplans", "CB-WP-0100-x.md"), "w") as fh:
|
|
|
|
|
fh.write(body)
|
2026-08-07 11:35:41 +02:00
|
|
|
# GameDesign §1.4 / ADR-0018. Four controls, because a check that
|
|
|
|
|
# cannot say NO is decoration and one that cannot say YES fires on
|
|
|
|
|
# everything.
|
|
|
|
|
os.makedirs(os.path.join(tmp, "specs"), exist_ok=True)
|
|
|
|
|
|
|
|
|
|
def reg(kind, state, prose):
|
|
|
|
|
body = ("<!-- design-register:begin -->\n\n"
|
|
|
|
|
"| id | kind | state | reproduction | role | raised | owner |\n"
|
|
|
|
|
"|---|---|---|---|---|---|---|\n"
|
|
|
|
|
f"| F99 | {kind} | {state} | x.rs | counterexample | 2026-01-01 | us |\n"
|
|
|
|
|
"\n<!-- design-register:end -->\n\n"
|
|
|
|
|
f"- **F99 — a claim.** {prose}\n")
|
|
|
|
|
with open(os.path.join(tmp, "specs", "FindingRegister.md"), "w") as fh:
|
|
|
|
|
fh.write(body)
|
|
|
|
|
|
|
|
|
|
reg("degenerate", "raised", "It is 42.")
|
|
|
|
|
check("sensitivity: an arithmetic claim with no variable is caught",
|
|
|
|
|
len(check_sensitivity_stated(tmp)) == 1)
|
|
|
|
|
reg("degenerate", "raised", "It is 42, and it varies with seat count.")
|
|
|
|
|
check("sensitivity: naming the variable clears it",
|
|
|
|
|
not check_sensitivity_stated(tmp),
|
|
|
|
|
"without this it would fire on everything")
|
|
|
|
|
reg("inert", "raised", "It is 42.")
|
|
|
|
|
check("sensitivity: a non-arithmetic kind is not asked",
|
|
|
|
|
not check_sensitivity_stated(tmp),
|
|
|
|
|
"`inert` is about whether a thing happens, not how much")
|
|
|
|
|
reg("degenerate", "note", "It is 42.")
|
|
|
|
|
check("sensitivity: a note has no measurement to be sensitive about",
|
|
|
|
|
not check_sensitivity_stated(tmp), "GameDesign §3.1")
|
|
|
|
|
|
CB-WP-0036 done: the pace flag and the first ornament declarations
--pace speed|interactive, defaulting to Speed. Nothing reads it yet, and
that is the point: it is the seam clay-animate attaches to, and a seam is
cheap now where a retrofit would not be. A misspelt pace is refused rather
than defaulting, because quietly falling back to Speed would look exactly
like the renderer being broken.
I3 is asserted rather than intended: the same scripted game at both paces
must produce a byte-identical serialised recording and the same end state
hash. Mutation-proven — leak the pace into the seed and it fails with "the
recording differs by pace, so a renderer has become mechanism".
specs/OrnamentRegister.md carries four declarations. This reverses the
reasoning written in T03 earlier, which said the first declarations would
come from F18's unvendored files: instances already existed. Hand order is
what prompted the category, and "who deals" was the maintainer's own
example. O3 is the interesting one — seat ORDER is mechanism because
GR-R08 rotates Lead, while where a seat is drawn is not.
I5 is executable: check_ornament_falsifier fails any row still declared
that names no falsifier, mutation-proven red on O1. Presence, never
adequacy, and the finding text says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 22:15:38 +02:00
|
|
|
# Ornamentation I5. Same shape: it must say NO and it must say YES.
|
|
|
|
|
def orn(state, prose):
|
|
|
|
|
body = ("<!-- ornament-register:begin -->\n\n"
|
|
|
|
|
"| id | ornaments | grounded | state | raised |\n"
|
|
|
|
|
"|---|---|---|---|---|\n"
|
|
|
|
|
f"| O9 | a thing | provisional | {state} | 2026-01-01 |\n"
|
|
|
|
|
"\n<!-- ornament-register:end -->\n\n"
|
|
|
|
|
f"- **O9 — a thing.** {prose}\n")
|
|
|
|
|
with open(os.path.join(tmp, "specs", "OrnamentRegister.md"), "w") as fh:
|
|
|
|
|
fh.write(body)
|
|
|
|
|
|
|
|
|
|
orn("declared", "It does not matter.")
|
|
|
|
|
check("ornament: a declaration with no falsifier is caught",
|
|
|
|
|
len(check_ornament_falsifier(tmp)) == 1,
|
|
|
|
|
"a claim that cannot be wrong is not a claim")
|
|
|
|
|
orn("declared", "It does not matter. **Falsifier:** a rule naming it.")
|
|
|
|
|
check("ornament: naming a falsifier clears it",
|
|
|
|
|
not check_ornament_falsifier(tmp),
|
|
|
|
|
"without this it would fire on everything")
|
|
|
|
|
orn("refuted", "It does not matter.")
|
|
|
|
|
check("ornament: a refuted row is history, not a live claim",
|
|
|
|
|
not check_ornament_falsifier(tmp))
|
|
|
|
|
|
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
|
|
|
wp("ready", ["done", "todo"])
|
|
|
|
|
f = check_workplan_lifecycle(tmp)
|
|
|
|
|
check("lifecycle detects `ready` after work has started",
|
|
|
|
|
len(f) == 1 and "active" in f[0].detail, f"{len(f)} finding(s)")
|
|
|
|
|
wp("active", ["done", "todo"])
|
|
|
|
|
check("lifecycle accepts `active` mid-flight",
|
|
|
|
|
not check_workplan_lifecycle(tmp))
|
|
|
|
|
wp("active", ["done", "cancel"])
|
|
|
|
|
f = check_workplan_lifecycle(tmp)
|
|
|
|
|
check("lifecycle detects `active` when everything is closed",
|
|
|
|
|
len(f) == 1 and "done" in f[0].detail, f"{len(f)} finding(s)")
|
|
|
|
|
wp("done", ["done", "todo"])
|
|
|
|
|
check("lifecycle detects `done` with an open task",
|
|
|
|
|
len(check_workplan_lifecycle(tmp)) == 1)
|
|
|
|
|
os.remove(os.path.join(tmp, "workplans", "CB-WP-0100-x.md"))
|
|
|
|
|
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
# gates: an unclassified `all:` dependency trips, and so does an
|
|
|
|
|
# entry naming a target the Makefile lacks.
|
|
|
|
|
with open(os.path.join(tmp, "Makefile"), "w") as fh:
|
|
|
|
|
fh.write("all: coverage newthing\ncoverage:\n\techo\n")
|
|
|
|
|
with open(os.path.join(tmp, "gates.toml"), "w") as fh:
|
|
|
|
|
fh.write('not_control_gates = []\n\n[[gate]]\nid = "G"\n'
|
2026-08-08 09:57:39 +02:00
|
|
|
'name = "n"\ntarget = "coverage"\ncadence = "all"\n'
|
|
|
|
|
'checks = "c"\n'
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
'added = "2026-01-01"\nreview_by = "2026-02-01"\n'
|
|
|
|
|
'retire_if = "r"\n')
|
|
|
|
|
f = check_gate_registry(tmp)
|
|
|
|
|
check("gate registry detects an unclassified all: dependency",
|
|
|
|
|
len(f) == 1 and "newthing" in f[0].detail, f"{len(f)} finding(s)")
|
|
|
|
|
with open(os.path.join(tmp, "gates.toml"), "w") as fh:
|
|
|
|
|
fh.write('not_control_gates = ["newthing", "coverage"]\n\n[[gate]]\nid = "G"\n'
|
2026-08-08 09:57:39 +02:00
|
|
|
'name = "n"\ntarget = "ghost"\ncadence = "manual"\n'
|
|
|
|
|
'checks = "c"\n'
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
'added = "2026-01-01"\nreview_by = "2026-02-01"\n'
|
|
|
|
|
'retire_if = "r"\n')
|
|
|
|
|
f = check_gate_registry(tmp)
|
|
|
|
|
check("gate registry detects an entry naming a missing target",
|
|
|
|
|
len(f) == 1 and "ghost" in f[0].detail, f"{len(f)} finding(s)")
|
2026-08-08 09:57:39 +02:00
|
|
|
|
|
|
|
|
# CB-REV-0002 #7 generalised: a gate that does not run is not a
|
|
|
|
|
# gate. Both directions, because a rule that cannot say NO is
|
|
|
|
|
# decoration and one that cannot say YES fires on everything.
|
|
|
|
|
with open(os.path.join(tmp, "Makefile"), "w") as fh:
|
|
|
|
|
fh.write("all: coverage\ncoverage:\n\techo\npanels:\n\techo\n")
|
|
|
|
|
with open(os.path.join(tmp, "gates.toml"), "w") as fh:
|
|
|
|
|
fh.write('not_control_gates = []\n\n[[gate]]\nid = "G"\n'
|
|
|
|
|
'name = "n"\ntarget = "coverage"\ncadence = "all"\n'
|
|
|
|
|
'checks = "c"\n\n[[gate]]\nid = "P"\nname = "p"\n'
|
|
|
|
|
'target = "panels"\ncadence = "all"\nchecks = "c"\n')
|
|
|
|
|
f = check_gate_registry(tmp)
|
|
|
|
|
check("gates: a cadence=all gate absent from `make all` is caught",
|
|
|
|
|
len(f) == 1 and "does not run it" in f[0].detail,
|
|
|
|
|
"the panels gate passed lint while running nowhere")
|
|
|
|
|
with open(os.path.join(tmp, "gates.toml"), "w") as fh:
|
|
|
|
|
fh.write('not_control_gates = []\n\n[[gate]]\nid = "G"\n'
|
|
|
|
|
'name = "n"\ntarget = "coverage"\nchecks = "c"\n')
|
|
|
|
|
check("gates: a gate with no declared cadence is caught",
|
|
|
|
|
any("declares no cadence" in x.detail for x in check_gate_registry(tmp)))
|
|
|
|
|
with open(os.path.join(tmp, "gates.toml"), "w") as fh:
|
|
|
|
|
fh.write('not_control_gates = []\n\n[[gate]]\nid = "G"\n'
|
|
|
|
|
'name = "n"\ntarget = "panels"\ncadence = "manual"\n'
|
|
|
|
|
'checks = "c"\n\n[[gate]]\nid = "C"\nname = "c"\n'
|
|
|
|
|
'target = "coverage"\ncadence = "all"\nchecks = "c"\n')
|
|
|
|
|
check("gates: a gate may be manual, and saying so clears it",
|
|
|
|
|
not check_gate_registry(tmp),
|
|
|
|
|
"without this every manual gate would fire")
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
os.unlink(os.path.join(tmp, "gates.toml"))
|
|
|
|
|
os.unlink(os.path.join(tmp, "Makefile"))
|
|
|
|
|
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
# evidence: a table verdict trips; the word in prose does not.
|
|
|
|
|
with open(os.path.join(tmp, "evidence", "E.md"), "w") as fh:
|
|
|
|
|
fh.write("| AC-1 | x | unmeasured |\n"
|
|
|
|
|
"the word unmeasured appearing in prose is fine\n")
|
|
|
|
|
f = check_evidence_no_unmeasured(tmp)
|
|
|
|
|
check("evidence-unmeasured detects a table verdict, not prose",
|
|
|
|
|
len(f) == 1, f"{len(f)} finding(s), expected exactly 1")
|
|
|
|
|
|
|
|
|
|
# tier/chaos: missing tier trips; tier without chaos trips.
|
|
|
|
|
with open(os.path.join(tmp, "research", "A.md"), "w") as fh:
|
|
|
|
|
fh.write("# survey\nno tier here\n")
|
|
|
|
|
with open(os.path.join(tmp, "research", "B.md"), "w") as fh:
|
|
|
|
|
fh.write("tier: L (structural L)\n")
|
|
|
|
|
f = check_survey_tier_and_chaos(tmp)
|
|
|
|
|
rules = sorted(x.rule for x in f)
|
|
|
|
|
check("tier/chaos detects both omissions",
|
|
|
|
|
rules == ["chaos-recorded", "tier-declared"], f"{rules}")
|
|
|
|
|
|
|
|
|
|
# self-test: a tool without the flag trips.
|
|
|
|
|
with open(os.path.join(tmp, "tools", "silent.py"), "w") as fh:
|
|
|
|
|
fh.write("print(42)\n")
|
|
|
|
|
f = check_reporting_tools_self_test(tmp)
|
|
|
|
|
check("self-test detects a tool lacking --self-test",
|
|
|
|
|
len(f) == 1 and "silent.py" in f[0].path, f"{len(f)} finding(s)")
|
|
|
|
|
|
|
|
|
|
# review trail: approved tier-L survey with no history trips.
|
|
|
|
|
with open(os.path.join(tmp, "research", "C.md"), "w") as fh:
|
|
|
|
|
fh.write("tier: L (structural L, chaos 3)\nstatus: approved\n")
|
|
|
|
|
f = check_review_trail(tmp)
|
|
|
|
|
check("review-trail detects a missing challenge/response",
|
|
|
|
|
len(f) == 2, f"{len(f)} finding(s), expected 2")
|
|
|
|
|
finally:
|
|
|
|
|
shutil.rmtree(tmp, ignore_errors=True)
|
|
|
|
|
|
|
|
|
|
print("loop-lint self-test (positive control)")
|
|
|
|
|
ok = True
|
|
|
|
|
for name, passed, detail in results:
|
|
|
|
|
print(f" [{'ok ' if passed else 'FAIL'}] {name}"
|
|
|
|
|
+ (f" — {detail}" if detail else ""))
|
|
|
|
|
ok &= passed
|
|
|
|
|
return 0 if ok else 1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def main():
|
|
|
|
|
if "--self-test" in sys.argv:
|
|
|
|
|
return self_test()
|
|
|
|
|
|
|
|
|
|
findings = []
|
|
|
|
|
for c in CHECKS:
|
|
|
|
|
findings.extend(c())
|
|
|
|
|
|
|
|
|
|
print("loop-lint — executable InnerLoop rules")
|
|
|
|
|
if not findings:
|
|
|
|
|
print(" no findings")
|
|
|
|
|
return 0
|
|
|
|
|
by_rule = {}
|
|
|
|
|
for f in findings:
|
|
|
|
|
by_rule.setdefault(f.rule, []).append(f)
|
|
|
|
|
for rule, fs in sorted(by_rule.items()):
|
|
|
|
|
print(f"\n{rule} ({len(fs)}):")
|
|
|
|
|
for f in fs:
|
|
|
|
|
print(str(f))
|
|
|
|
|
print(f"\n{len(findings)} finding(s)")
|
|
|
|
|
return 1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
if __name__ == "__main__":
|
|
|
|
|
sys.exit(main())
|