coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CB-WP-0009-T02: gates.toml and make gate-review

Browse source 
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.

Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.

First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-01 15:42:13 +02:00
parent 38f237fc5c
commit cd2dc5380a
5 changed files with 436 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

6
Makefile
View file

@ -24,7 +24,7 @@ TOOLS := $(REPO)/tools
# Every cargo recipe runs at the repo root; the shell does not persist cd.
IN_REPO := cd $(REPO) &&
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget shape-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time am6 replay-test loc play all
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget shape-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time am6 replay-test loc play gate-review all
## fmt + clippy (deny warnings) + HashMap deny-lint
check:
@ -36,6 +36,10 @@ check:
play:
$(IN_REPO) $(CARGO) run -q -p cb-play -- $(ARGS)
## which control gates are due for a keep-or-kill argument (ADR-0006 D3)
gate-review:
$(PY) $(TOOLS)/gate-review.py
## unit + scenario-format tests
test:
$(IN_REPO) $(CARGO) test --workspace

134
gates.toml Normal file
View file

@ -0,0 +1,134 @@
# The gate registry (ADR-0006 D3, CB-WP-0009 T02).
#
# A gate without an expiry is a permanent tax justified once. Every
# standing control mechanism gets an entry here saying what it checks,
# what it has actually **caught**, when its keep-or-kill argument is due,
# and what would retire it.
#
# `make gate-review` reports what is overdue and what has caught nothing.
# It reports; it does not fail the build — CB-RES-0005 §4: a gate that
# blocks the remedy when the metric breaches is a trap, not a gate.
#
# `caught` is the load-bearing field. An empty `caught` is not proof a
# gate is useless — it may be preventing rather than missing — but it
# means the argument has to be made out loud on `review_by`.
# Targets in `make all` that are build or acceptance steps rather than
# *control* gates — they measure the product, not how we work. Listed
# explicitly so a new target has to be classified rather than ignored;
# `loop-lint` fails when a target is in neither list.
not_control_gates = [
"check", "test", "sim", "bench-test", "size-metrics", "runtime-metrics",
"am6", "replay-test", "dep-weight", "self-tests", "env-test",
]
[[gate]]
id = "CB-01/CB-02"
name = "cost budget"
target = "cost-budget"
checks = "spend since the last commit; soft $10, hard $22"
added = "2026-07-30"
review_by = "2026-11-30"
caught = [
"CB-WP-0005: hard breach forced the Phase C re-plan",
"CB-WP-0006 T07: $12.06 in one task, the pass's most expensive",
]
retire_if = "two consecutive passes never approach the soft line, or commits get small enough that the window is always trivial"
[[gate]]
id = "SH-1/SH-2/SH-3"
name = "session-shape budget"
target = "shape-budget"
checks = "mean and p90 context, and batching rate, since the last commit"
added = "2026-08-01"
review_by = "2026-11-30"
caught = [
"first run fired HARD at 656,574 against a 300,000 ceiling, which is what prompted the compaction before CB-WP-0008",
]
retire_if = "context stops correlating with cost, or the model's context handling makes the number unactionable"
[[gate]]
id = "M-D1-MUT"
name = "mutation coverage of acceptance rows"
target = "mutation-check"
checks = "each acceptance row's assertion must go red for a stated reason when mutated"
added = "2026-07-31"
review_by = "2026-12-31"
caught = [
"AM-6 measuring contention, not throughput",
"AM-5's 61% measurement error under load",
"peak RSS over-reported 3x",
"K10's first round trip not reproducing",
"the bench workload existing twice",
"AM-2's expect matching its own passing output (EXPECT-VACUOUS)",
]
retire_if = "a full pass adds rows without finding anything, twice running — the harness costs real money per run"
[[gate]]
id = "DFD"
name = "single source of fact"
target = "facts-check"
checks = "every tagged number in the docs matches the tool that measures it"
added = "2026-07-31"
review_by = "2026-12-31"
caught = [
"gr_scenarios stale at 21 after CB-WP-0008 T03 added three scenarios",
]
retire_if = "the untagged-literal count reaches zero and stays there, meaning the docs stopped restating measured numbers"
[[gate]]
id = "AM-1b"
name = "kernel spec->code link"
target = "coverage"
checks = "every numbered K-rule is named in the source; binds 2026-08-31"
added = "2026-07-31"
review_by = "2026-08-31"
caught = [
"3 unlinked K-rules at introduction (15/18); 18/18 today",
]
retire_if = "it stays at 100% through two passes that add kernel rules — at that point it is measuring a habit, not enforcing one"
[[gate]]
id = "META-25"
name = "meta budget"
target = "status"
checks = "share of the trailing 3 passes spent on the loop itself; soft 25%"
added = "2026-08-01"
review_by = "2026-11-30"
caught = [
"its own cumulative-window defect, reported in CB-EV-0007 §3 and fixed by CB-WP-0009 T01",
]
retire_if = "product and meta stop being separable, or the share sits under the line for four passes without anyone consulting it"
[[gate]]
id = "LOOP-LINT"
name = "executable InnerLoop rules"
target = "loop-lint"
checks = "loadability, unmeasured verdicts, tier and chaos declarations, review trails, self-test entry points, and this registry"
added = "2026-07-30"
review_by = "2026-12-31"
caught = [
"four loadability breaches (401, 427, 406, 409 lines), each fixed structurally rather than by raising the limit",
"a reporting tool with no --self-test entry point (tools/repo.py)",
]
retire_if = "two passes run with no finding while artifacts keep growing — that would mean it is measuring the wrong properties"
[[gate]]
id = "CHAOS"
name = "the chaos roll"
target = ""
checks = "d4 on each tier declaration, 12-declaration calibration window"
added = "2026-07-30"
review_by = "2026-09-30"
caught = []
retire_if = "the window closes with no overridden tier producing a different outcome than the argued one — the evaluation this window exists to make possible"
[[gate]]
id = "GATE-REVIEW"
name = "this registry"
target = "gate-review"
checks = "gates past their review date, and gates that have caught nothing"
added = "2026-08-01"
review_by = "2026-12-31"
caught = []
retire_if = "it has retired, tightened, or forced the re-justification of nothing by its review date — then it is a ritual, and ADR-0006 D4 says rituals cash out or go"

218
tools/gate-review.py Normal file
View file

@ -0,0 +1,218 @@
#!/usr/bin/env python3
"""Which control gates are due for a keep-or-kill argument? (ADR-0006 D3)
CB-WP-0009 T02. Six passes produced five standing control mechanisms and
no way to retire any of them. Gates accumulate monotonically because each
one was justified once, at the moment it was cheapest to justify.
This reads `gates.toml` and reports two things:
* gates past `review_by` — the date someone said they would argue for
keeping it;
* gates whose `caught` list is **empty** — which is not proof a gate is
useless (it may be preventing rather than missing), but is the
argument that has to be made out loud rather than never.
It **reports**. It does not fail the build, for CB-RES-0005 §4's reason:
a gate that blocks the remedy when the metric breaches is a trap. The
only failure exit here is a broken registry — a file that cannot be read
would otherwise report "0 gates, all healthy".
Usage:
python3 tools/gate-review.py
python3 tools/gate-review.py --self-test
"""
import datetime
import os
import sys
from repo import ROOT, enter_root
REGISTRY = os.path.join(ROOT, "gates.toml")
try:
import tomllib
except ModuleNotFoundError: # pragma: no cover - Python < 3.11
import tomli as tomllib
class Fail(Exception):
pass
REQUIRED = ("id", "name", "checks", "added", "review_by", "retire_if")
def load(path=REGISTRY):
"""Every gate, validated. A registry that parses but says nothing is
the harness-does-nothing failure this project keeps finding."""
if not os.path.exists(path):
raise Fail(f"{os.path.relpath(path, ROOT)} is missing")
with open(path, "rb") as fh:
data = tomllib.load(fh)
gates = data.get("gate") or []
if not gates:
raise Fail("registry parsed but contains no gates")
for g in gates:
missing = [k for k in REQUIRED if not g.get(k)]
if missing:
raise Fail(f"gate {g.get('id', '?')!r} is missing {missing}")
for field in ("added", "review_by"):
try:
datetime.date.fromisoformat(g[field])
except ValueError as e:
raise Fail(f"gate {g['id']!r} has a bad {field}: {e}") from e
return gates
def make_targets(path=None):
"""Targets declared in the Makefile, so an entry cannot name a
command that does not exist."""
path = path or os.path.join(ROOT, "Makefile")
targets = set()
for line in open(path):
if line and not line[0].isspace() and ":" in line and not line.startswith("."):
name = line.split(":", 1)[0].strip()
if name and " " not in name:
targets.add(name)
return targets
def report(today=None):
today = today or datetime.date.today()
gates = load()
targets = make_targets()
overdue, quiet, broken = [], [], []
for g in gates:
if datetime.date.fromisoformat(g["review_by"]) <= today:
overdue.append(g)
if not g.get("caught"):
quiet.append(g)
target = g.get("target") or ""
if target and target not in targets:
broken.append((g, target))
print("gate review — every gate is an experiment (ADR-0006 D3)")
print(f" registry {len(gates)} gate(s), {REGISTRY.split('/')[-1]}")
print(f" today {today.isoformat()}")
print("\n gates")
for g in gates:
due = datetime.date.fromisoformat(g["review_by"])
days = (due - today).days
mark = "DUE " if days <= 0 else ("soon" if days <= 30 else "ok ")
print(f" [{mark}] {g['id']:<14} {g['name']:<28} "
f"review {g['review_by']} ({days:+}d) caught {len(g.get('caught', []))}")
if quiet:
print("\n caught nothing yet — not a verdict, an argument that is owed")
for g in quiet:
print(f" {g['id']:<14} retire if: {g['retire_if']}")
if overdue:
print("\n DUE for a keep-or-kill argument")
for g in overdue:
print(f" {g['id']:<14} {g['retire_if']}")
if broken:
print("\n registry drift — entry names a target the Makefile lacks")
for g, target in broken:
print(f" {g['id']:<14} target {target!r}")
print(f"\n {len(overdue)} due, {len(quiet)} silent, {len(broken)} drifted")
print(" reporting only — never fails the build (CB-RES-0005 §4)")
return 0
def self_test():
"""Each check pins a way this tool could report a comfortable lie."""
results = []
def check(name, ok, detail=""):
results.append((name, ok, detail))
gates = load()
check("the real registry loads", len(gates) >= 5, f"{len(gates)} gate(s)")
check("every gate names what would retire it",
all(len(g["retire_if"]) > 20 for g in gates))
# Registry drift is the failure this exists to prevent: a gate added
# to the Makefile with no entry, or an entry for a deleted target.
targets = make_targets()
named = [(g["id"], g["target"]) for g in gates if g.get("target")]
check("every named target exists in the Makefile",
all(t in targets for _i, t in named),
", ".join(f"{i}:{t}" for i, t in named if t not in targets) or "all present")
check("Makefile targets were actually parsed", len(targets) >= 15,
f"{len(targets)} target(s)")
import tempfile
def registry(text):
fh = tempfile.NamedTemporaryFile("w", suffix=".toml", delete=False)
fh.write(text)
fh.close()
return fh.name
# A registry that parses to nothing must abort, not report "0 due".
empty = registry("# no gates here\n")
try:
load(empty)
check("an empty registry aborts", False, "reported a clean bill")
except Fail:
check("an empty registry aborts", True)
finally:
os.unlink(empty)
# A missing required field must abort rather than be treated as absent
# evidence — `caught` is optional, the rest are not.
partial = registry('[[gate]]\nid = "X"\nname = "n"\nchecks = "c"\n'
'added = "2026-01-01"\nreview_by = "2026-02-01"\n')
try:
load(partial)
check("a gate with no retire_if aborts", False, "accepted")
except Fail:
check("a gate with no retire_if aborts", True)
finally:
os.unlink(partial)
# The overdue arithmetic must actually fire.
import io
from contextlib import redirect_stdout
buf = io.StringIO()
with redirect_stdout(buf):
report(today=datetime.date(2099, 1, 1))
late = buf.getvalue()
check("a far-future date marks every gate due",
late.count("[DUE ]") == len(gates),
f"{late.count('[DUE ]')} of {len(gates)}")
buf = io.StringIO()
with redirect_stdout(buf):
report(today=datetime.date(2020, 1, 1))
early = buf.getvalue()
check("a far-past date marks none due", "[DUE ]" not in early)
check("silent gates are named either way", "caught nothing yet" in early)
print("gate-review self-test (positive control)")
ok = True
for name, passed, det in results:
print(f" [{'ok ' if passed else 'FAIL'}] {name}" + (f" — {det}" if det else ""))
ok &= passed
return 0 if ok else 1
def main():
enter_root()
try:
if "--self-test" in sys.argv:
return self_test()
return report()
except Fail as e:
print(f"gate-review: {e}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())

72
tools/loop-lint.py
View file

@ -162,12 +162,61 @@ def check_reporting_tools_self_test(root=REPO):
return out
def check_gate_registry(root=REPO):
"""ADR-0006 D3 — every control gate is in `gates.toml`, and every
entry names a real target.
The failure this prevents is drift in the direction nobody notices: a
gate added to `make all` with no registry entry never acquires a
review date, which is how five mechanisms accumulated with no way to
retire any of them.
"""
out = []
registry = os.path.join(root, "gates.toml")
makefile = os.path.join(root, "Makefile")
if not (os.path.exists(registry) and os.path.exists(makefile)):
return out
try:
import tomllib
except ModuleNotFoundError: # pragma: no cover
return out
with open(registry, "rb") as fh:
data = tomllib.load(fh)
gates = data.get("gate") or []
if not gates:
return [Finding("gates", "gates.toml", "registry contains no gates")]
registered = {g.get("target") for g in gates if g.get("target")}
exempt = set(data.get("not_control_gates") or [])
text = open(makefile).read()
m = re.search(r"^all:(.*)$", text, re.M)
deps = m.group(1).split() if m else []
targets = {ln.split(":", 1)[0].strip() for ln in text.splitlines()
if ln and not ln[0].isspace() and ":" in ln and not ln.startswith(".")}
for dep in deps:
if dep not in registered and dep not in exempt:
out.append(Finding(
"gates", "gates.toml",
f"`make all` runs {dep!r}, which is neither a registered "
f"control gate nor listed in not_control_gates — classify it, "
f"so it cannot acquire permanence without a review date"))
for target in sorted(registered):
if target not in targets:
out.append(Finding(
"gates", "gates.toml",
f"entry names target {target!r}, which the Makefile lacks"))
return out
CHECKS = (
check_loadability,
check_evidence_no_unmeasured,
check_survey_tier_and_chaos,
check_review_trail,
check_reporting_tools_self_test,
check_gate_registry,
)
@ -196,6 +245,29 @@ def self_test():
len(f) == 1 and "Big.md" in f[0].path,
f"{len(f)} finding(s)")
# gates: an unclassified `all:` dependency trips, and so does an
# entry naming a target the Makefile lacks.
with open(os.path.join(tmp, "Makefile"), "w") as fh:
fh.write("all: coverage newthing\ncoverage:\n\techo\n")
with open(os.path.join(tmp, "gates.toml"), "w") as fh:
fh.write('not_control_gates = []\n\n[[gate]]\nid = "G"\n'
'name = "n"\ntarget = "coverage"\nchecks = "c"\n'
'added = "2026-01-01"\nreview_by = "2026-02-01"\n'
'retire_if = "r"\n')
f = check_gate_registry(tmp)
check("gate registry detects an unclassified all: dependency",
len(f) == 1 and "newthing" in f[0].detail, f"{len(f)} finding(s)")
with open(os.path.join(tmp, "gates.toml"), "w") as fh:
fh.write('not_control_gates = ["newthing", "coverage"]\n\n[[gate]]\nid = "G"\n'
'name = "n"\ntarget = "ghost"\nchecks = "c"\n'
'added = "2026-01-01"\nreview_by = "2026-02-01"\n'
'retire_if = "r"\n')
f = check_gate_registry(tmp)
check("gate registry detects an entry naming a missing target",
len(f) == 1 and "ghost" in f[0].detail, f"{len(f)} finding(s)")
os.unlink(os.path.join(tmp, "gates.toml"))
os.unlink(os.path.join(tmp, "Makefile"))
# evidence: a table verdict trips; the word in prose does not.
with open(os.path.join(tmp, "evidence", "E.md"), "w") as fh:
fh.write("| AC-1 | x | unmeasured |\n"

8
workplans/CB-WP-0009-adaptive-gates.md
View file

@ -62,7 +62,7 @@ the last 3 passes** against **49% lifetime** — CB-WP-0008 alone reads 0%.
```task
id: CB-WP-0009-T02
status: todo
status: done
priority: high
state_hub_task_id: "0fb6cfaf-9304-481a-a96f-1f0150f4d81d"
```
@ -82,6 +82,12 @@ behind the Makefile is worse than none, so `loop-lint` gains a check that
every gate target has an entry and every entry names a real target. Its
own self-test must fail when an entry is removed.
**Done 2026-08-01.** `gates.toml` (9 gates), `make gate-review`, and a
`loop-lint` check in both directions — an unclassified `make all`
dependency and an entry naming a target the Makefile lacks each produce
a finding, both pinned by self-tests. First run: **0 due, 2 silent**
(the chaos roll and `gate-review` itself).
## Task: InnerLoop v1.6
```task