From cd2dc5380abd8ebee714b5b1309f125f3b675dd3 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 1 Aug 2026 15:42:13 +0200 Subject: [PATCH] CB-WP-0009-T02: gates.toml and make gate-review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ADR-0006 D3. Nine standing control gates now say what they check, what they have caught with pointers, when their keep-or-kill argument is due, and what would retire them. make gate-review reports what is overdue and what has caught nothing; it never fails the build, for CB-RES-0005 §4's reason. Drift is checked in both directions and both are pinned by self-tests: a dependency of `make all` that is neither a registered control gate nor listed in not_control_gates is a loop-lint finding, so a new gate cannot acquire permanence without a review date, and an entry naming a target the Makefile lacks is a finding too. First run: 0 due, 2 silent. The silent two are the chaos roll, whose 12-declaration window exists precisely to find out, and gate-review itself, which is not exempt from its own rule — if it has retired, tightened or forced the re-justification of nothing by 2026-12-31 it is a ritual and goes. Co-Authored-By: Claude Opus 5 --- Makefile | 6 +- gates.toml | 134 +++++++++++++++ tools/gate-review.py | 218 +++++++++++++++++++++++++ tools/loop-lint.py | 72 ++++++++ workplans/CB-WP-0009-adaptive-gates.md | 8 +- 5 files changed, 436 insertions(+), 2 deletions(-) create mode 100644 gates.toml create mode 100644 tools/gate-review.py diff --git a/Makefile b/Makefile index 1af25c3..221d8a2 100644 --- a/Makefile +++ b/Makefile @@ -24,7 +24,7 @@ TOOLS := $(REPO)/tools # Every cargo recipe runs at the repo root; the shell does not persist cd. IN_REPO := cd $(REPO) && -.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget shape-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time am6 replay-test loc play all +.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget shape-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time am6 replay-test loc play gate-review all ## fmt + clippy (deny warnings) + HashMap deny-lint check: @@ -36,6 +36,10 @@ check: play: $(IN_REPO) $(CARGO) run -q -p cb-play -- $(ARGS) +## which control gates are due for a keep-or-kill argument (ADR-0006 D3) +gate-review: + $(PY) $(TOOLS)/gate-review.py + ## unit + scenario-format tests test: $(IN_REPO) $(CARGO) test --workspace diff --git a/gates.toml b/gates.toml new file mode 100644 index 0000000..111de34 --- /dev/null +++ b/gates.toml @@ -0,0 +1,134 @@ +# The gate registry (ADR-0006 D3, CB-WP-0009 T02). +# +# A gate without an expiry is a permanent tax justified once. Every +# standing control mechanism gets an entry here saying what it checks, +# what it has actually **caught**, when its keep-or-kill argument is due, +# and what would retire it. +# +# `make gate-review` reports what is overdue and what has caught nothing. +# It reports; it does not fail the build — CB-RES-0005 §4: a gate that +# blocks the remedy when the metric breaches is a trap, not a gate. +# +# `caught` is the load-bearing field. An empty `caught` is not proof a +# gate is useless — it may be preventing rather than missing — but it +# means the argument has to be made out loud on `review_by`. + +# Targets in `make all` that are build or acceptance steps rather than +# *control* gates — they measure the product, not how we work. Listed +# explicitly so a new target has to be classified rather than ignored; +# `loop-lint` fails when a target is in neither list. +not_control_gates = [ + "check", "test", "sim", "bench-test", "size-metrics", "runtime-metrics", + "am6", "replay-test", "dep-weight", "self-tests", "env-test", +] + +[[gate]] +id = "CB-01/CB-02" +name = "cost budget" +target = "cost-budget" +checks = "spend since the last commit; soft $10, hard $22" +added = "2026-07-30" +review_by = "2026-11-30" +caught = [ + "CB-WP-0005: hard breach forced the Phase C re-plan", + "CB-WP-0006 T07: $12.06 in one task, the pass's most expensive", +] +retire_if = "two consecutive passes never approach the soft line, or commits get small enough that the window is always trivial" + +[[gate]] +id = "SH-1/SH-2/SH-3" +name = "session-shape budget" +target = "shape-budget" +checks = "mean and p90 context, and batching rate, since the last commit" +added = "2026-08-01" +review_by = "2026-11-30" +caught = [ + "first run fired HARD at 656,574 against a 300,000 ceiling, which is what prompted the compaction before CB-WP-0008", +] +retire_if = "context stops correlating with cost, or the model's context handling makes the number unactionable" + +[[gate]] +id = "M-D1-MUT" +name = "mutation coverage of acceptance rows" +target = "mutation-check" +checks = "each acceptance row's assertion must go red for a stated reason when mutated" +added = "2026-07-31" +review_by = "2026-12-31" +caught = [ + "AM-6 measuring contention, not throughput", + "AM-5's 61% measurement error under load", + "peak RSS over-reported 3x", + "K10's first round trip not reproducing", + "the bench workload existing twice", + "AM-2's expect matching its own passing output (EXPECT-VACUOUS)", +] +retire_if = "a full pass adds rows without finding anything, twice running — the harness costs real money per run" + +[[gate]] +id = "DFD" +name = "single source of fact" +target = "facts-check" +checks = "every tagged number in the docs matches the tool that measures it" +added = "2026-07-31" +review_by = "2026-12-31" +caught = [ + "gr_scenarios stale at 21 after CB-WP-0008 T03 added three scenarios", +] +retire_if = "the untagged-literal count reaches zero and stays there, meaning the docs stopped restating measured numbers" + +[[gate]] +id = "AM-1b" +name = "kernel spec->code link" +target = "coverage" +checks = "every numbered K-rule is named in the source; binds 2026-08-31" +added = "2026-07-31" +review_by = "2026-08-31" +caught = [ + "3 unlinked K-rules at introduction (15/18); 18/18 today", +] +retire_if = "it stays at 100% through two passes that add kernel rules — at that point it is measuring a habit, not enforcing one" + +[[gate]] +id = "META-25" +name = "meta budget" +target = "status" +checks = "share of the trailing 3 passes spent on the loop itself; soft 25%" +added = "2026-08-01" +review_by = "2026-11-30" +caught = [ + "its own cumulative-window defect, reported in CB-EV-0007 §3 and fixed by CB-WP-0009 T01", +] +retire_if = "product and meta stop being separable, or the share sits under the line for four passes without anyone consulting it" + +[[gate]] +id = "LOOP-LINT" +name = "executable InnerLoop rules" +target = "loop-lint" +checks = "loadability, unmeasured verdicts, tier and chaos declarations, review trails, self-test entry points, and this registry" +added = "2026-07-30" +review_by = "2026-12-31" +caught = [ + "four loadability breaches (401, 427, 406, 409 lines), each fixed structurally rather than by raising the limit", + "a reporting tool with no --self-test entry point (tools/repo.py)", +] +retire_if = "two passes run with no finding while artifacts keep growing — that would mean it is measuring the wrong properties" + +[[gate]] +id = "CHAOS" +name = "the chaos roll" +target = "" +checks = "d4 on each tier declaration, 12-declaration calibration window" +added = "2026-07-30" +review_by = "2026-09-30" +caught = [] +retire_if = "the window closes with no overridden tier producing a different outcome than the argued one — the evaluation this window exists to make possible" + +[[gate]] +id = "GATE-REVIEW" +name = "this registry" +target = "gate-review" +checks = "gates past their review date, and gates that have caught nothing" +added = "2026-08-01" +review_by = "2026-12-31" +caught = [] +retire_if = "it has retired, tightened, or forced the re-justification of nothing by its review date — then it is a ritual, and ADR-0006 D4 says rituals cash out or go" diff --git a/tools/gate-review.py b/tools/gate-review.py new file mode 100644 index 0000000..0ee6890 --- /dev/null +++ b/tools/gate-review.py @@ -0,0 +1,218 @@ +#!/usr/bin/env python3 +"""Which control gates are due for a keep-or-kill argument? (ADR-0006 D3) + +CB-WP-0009 T02. Six passes produced five standing control mechanisms and +no way to retire any of them. Gates accumulate monotonically because each +one was justified once, at the moment it was cheapest to justify. + +This reads `gates.toml` and reports two things: + + * gates past `review_by` — the date someone said they would argue for + keeping it; + * gates whose `caught` list is **empty** — which is not proof a gate is + useless (it may be preventing rather than missing), but is the + argument that has to be made out loud rather than never. + +It **reports**. It does not fail the build, for CB-RES-0005 §4's reason: +a gate that blocks the remedy when the metric breaches is a trap. The +only failure exit here is a broken registry — a file that cannot be read +would otherwise report "0 gates, all healthy". + +Usage: + python3 tools/gate-review.py + python3 tools/gate-review.py --self-test +""" +import datetime +import os +import sys + +from repo import ROOT, enter_root + +REGISTRY = os.path.join(ROOT, "gates.toml") + +try: + import tomllib +except ModuleNotFoundError: # pragma: no cover - Python < 3.11 + import tomli as tomllib + + +class Fail(Exception): + pass + + +REQUIRED = ("id", "name", "checks", "added", "review_by", "retire_if") + + +def load(path=REGISTRY): + """Every gate, validated. A registry that parses but says nothing is + the harness-does-nothing failure this project keeps finding.""" + if not os.path.exists(path): + raise Fail(f"{os.path.relpath(path, ROOT)} is missing") + with open(path, "rb") as fh: + data = tomllib.load(fh) + gates = data.get("gate") or [] + if not gates: + raise Fail("registry parsed but contains no gates") + for g in gates: + missing = [k for k in REQUIRED if not g.get(k)] + if missing: + raise Fail(f"gate {g.get('id', '?')!r} is missing {missing}") + for field in ("added", "review_by"): + try: + datetime.date.fromisoformat(g[field]) + except ValueError as e: + raise Fail(f"gate {g['id']!r} has a bad {field}: {e}") from e + return gates + + +def make_targets(path=None): + """Targets declared in the Makefile, so an entry cannot name a + command that does not exist.""" + path = path or os.path.join(ROOT, "Makefile") + targets = set() + for line in open(path): + if line and not line[0].isspace() and ":" in line and not line.startswith("."): + name = line.split(":", 1)[0].strip() + if name and " " not in name: + targets.add(name) + return targets + + +def report(today=None): + today = today or datetime.date.today() + gates = load() + targets = make_targets() + + overdue, quiet, broken = [], [], [] + for g in gates: + if datetime.date.fromisoformat(g["review_by"]) <= today: + overdue.append(g) + if not g.get("caught"): + quiet.append(g) + target = g.get("target") or "" + if target and target not in targets: + broken.append((g, target)) + + print("gate review — every gate is an experiment (ADR-0006 D3)") + print(f" registry {len(gates)} gate(s), {REGISTRY.split('/')[-1]}") + print(f" today {today.isoformat()}") + + print("\n gates") + for g in gates: + due = datetime.date.fromisoformat(g["review_by"]) + days = (due - today).days + mark = "DUE " if days <= 0 else ("soon" if days <= 30 else "ok ") + print(f" [{mark}] {g['id']:<14} {g['name']:<28} " + f"review {g['review_by']} ({days:+}d) caught {len(g.get('caught', []))}") + + if quiet: + print("\n caught nothing yet — not a verdict, an argument that is owed") + for g in quiet: + print(f" {g['id']:<14} retire if: {g['retire_if']}") + + if overdue: + print("\n DUE for a keep-or-kill argument") + for g in overdue: + print(f" {g['id']:<14} {g['retire_if']}") + + if broken: + print("\n registry drift — entry names a target the Makefile lacks") + for g, target in broken: + print(f" {g['id']:<14} target {target!r}") + + print(f"\n {len(overdue)} due, {len(quiet)} silent, {len(broken)} drifted") + print(" reporting only — never fails the build (CB-RES-0005 §4)") + return 0 + + +def self_test(): + """Each check pins a way this tool could report a comfortable lie.""" + results = [] + + def check(name, ok, detail=""): + results.append((name, ok, detail)) + + gates = load() + check("the real registry loads", len(gates) >= 5, f"{len(gates)} gate(s)") + check("every gate names what would retire it", + all(len(g["retire_if"]) > 20 for g in gates)) + # Registry drift is the failure this exists to prevent: a gate added + # to the Makefile with no entry, or an entry for a deleted target. + targets = make_targets() + named = [(g["id"], g["target"]) for g in gates if g.get("target")] + check("every named target exists in the Makefile", + all(t in targets for _i, t in named), + ", ".join(f"{i}:{t}" for i, t in named if t not in targets) or "all present") + check("Makefile targets were actually parsed", len(targets) >= 15, + f"{len(targets)} target(s)") + + import tempfile + + def registry(text): + fh = tempfile.NamedTemporaryFile("w", suffix=".toml", delete=False) + fh.write(text) + fh.close() + return fh.name + + # A registry that parses to nothing must abort, not report "0 due". + empty = registry("# no gates here\n") + try: + load(empty) + check("an empty registry aborts", False, "reported a clean bill") + except Fail: + check("an empty registry aborts", True) + finally: + os.unlink(empty) + + # A missing required field must abort rather than be treated as absent + # evidence — `caught` is optional, the rest are not. + partial = registry('[[gate]]\nid = "X"\nname = "n"\nchecks = "c"\n' + 'added = "2026-01-01"\nreview_by = "2026-02-01"\n') + try: + load(partial) + check("a gate with no retire_if aborts", False, "accepted") + except Fail: + check("a gate with no retire_if aborts", True) + finally: + os.unlink(partial) + + # The overdue arithmetic must actually fire. + import io + from contextlib import redirect_stdout + + buf = io.StringIO() + with redirect_stdout(buf): + report(today=datetime.date(2099, 1, 1)) + late = buf.getvalue() + check("a far-future date marks every gate due", + late.count("[DUE ]") == len(gates), + f"{late.count('[DUE ]')} of {len(gates)}") + + buf = io.StringIO() + with redirect_stdout(buf): + report(today=datetime.date(2020, 1, 1)) + early = buf.getvalue() + check("a far-past date marks none due", "[DUE ]" not in early) + check("silent gates are named either way", "caught nothing yet" in early) + + print("gate-review self-test (positive control)") + ok = True + for name, passed, det in results: + print(f" [{'ok ' if passed else 'FAIL'}] {name}" + (f" — {det}" if det else "")) + ok &= passed + return 0 if ok else 1 + + +def main(): + enter_root() + try: + if "--self-test" in sys.argv: + return self_test() + return report() + except Fail as e: + print(f"gate-review: {e}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/loop-lint.py b/tools/loop-lint.py index 82e4163..be436b7 100644 --- a/tools/loop-lint.py +++ b/tools/loop-lint.py @@ -162,12 +162,61 @@ def check_reporting_tools_self_test(root=REPO): return out +def check_gate_registry(root=REPO): + """ADR-0006 D3 — every control gate is in `gates.toml`, and every + entry names a real target. + + The failure this prevents is drift in the direction nobody notices: a + gate added to `make all` with no registry entry never acquires a + review date, which is how five mechanisms accumulated with no way to + retire any of them. + """ + out = [] + registry = os.path.join(root, "gates.toml") + makefile = os.path.join(root, "Makefile") + if not (os.path.exists(registry) and os.path.exists(makefile)): + return out + try: + import tomllib + except ModuleNotFoundError: # pragma: no cover + return out + + with open(registry, "rb") as fh: + data = tomllib.load(fh) + gates = data.get("gate") or [] + if not gates: + return [Finding("gates", "gates.toml", "registry contains no gates")] + registered = {g.get("target") for g in gates if g.get("target")} + exempt = set(data.get("not_control_gates") or []) + + text = open(makefile).read() + m = re.search(r"^all:(.*)$", text, re.M) + deps = m.group(1).split() if m else [] + targets = {ln.split(":", 1)[0].strip() for ln in text.splitlines() + if ln and not ln[0].isspace() and ":" in ln and not ln.startswith(".")} + + for dep in deps: + if dep not in registered and dep not in exempt: + out.append(Finding( + "gates", "gates.toml", + f"`make all` runs {dep!r}, which is neither a registered " + f"control gate nor listed in not_control_gates — classify it, " + f"so it cannot acquire permanence without a review date")) + for target in sorted(registered): + if target not in targets: + out.append(Finding( + "gates", "gates.toml", + f"entry names target {target!r}, which the Makefile lacks")) + return out + + CHECKS = ( check_loadability, check_evidence_no_unmeasured, check_survey_tier_and_chaos, check_review_trail, check_reporting_tools_self_test, + check_gate_registry, ) @@ -196,6 +245,29 @@ def self_test(): len(f) == 1 and "Big.md" in f[0].path, f"{len(f)} finding(s)") + # gates: an unclassified `all:` dependency trips, and so does an + # entry naming a target the Makefile lacks. + with open(os.path.join(tmp, "Makefile"), "w") as fh: + fh.write("all: coverage newthing\ncoverage:\n\techo\n") + with open(os.path.join(tmp, "gates.toml"), "w") as fh: + fh.write('not_control_gates = []\n\n[[gate]]\nid = "G"\n' + 'name = "n"\ntarget = "coverage"\nchecks = "c"\n' + 'added = "2026-01-01"\nreview_by = "2026-02-01"\n' + 'retire_if = "r"\n') + f = check_gate_registry(tmp) + check("gate registry detects an unclassified all: dependency", + len(f) == 1 and "newthing" in f[0].detail, f"{len(f)} finding(s)") + with open(os.path.join(tmp, "gates.toml"), "w") as fh: + fh.write('not_control_gates = ["newthing", "coverage"]\n\n[[gate]]\nid = "G"\n' + 'name = "n"\ntarget = "ghost"\nchecks = "c"\n' + 'added = "2026-01-01"\nreview_by = "2026-02-01"\n' + 'retire_if = "r"\n') + f = check_gate_registry(tmp) + check("gate registry detects an entry naming a missing target", + len(f) == 1 and "ghost" in f[0].detail, f"{len(f)} finding(s)") + os.unlink(os.path.join(tmp, "gates.toml")) + os.unlink(os.path.join(tmp, "Makefile")) + # evidence: a table verdict trips; the word in prose does not. with open(os.path.join(tmp, "evidence", "E.md"), "w") as fh: fh.write("| AC-1 | x | unmeasured |\n" diff --git a/workplans/CB-WP-0009-adaptive-gates.md b/workplans/CB-WP-0009-adaptive-gates.md index 041a2b2..6db3480 100644 --- a/workplans/CB-WP-0009-adaptive-gates.md +++ b/workplans/CB-WP-0009-adaptive-gates.md @@ -62,7 +62,7 @@ the last 3 passes** against **49% lifetime** — CB-WP-0008 alone reads 0%. ```task id: CB-WP-0009-T02 -status: todo +status: done priority: high state_hub_task_id: "0fb6cfaf-9304-481a-a96f-1f0150f4d81d" ``` @@ -82,6 +82,12 @@ behind the Makefile is worse than none, so `loop-lint` gains a check that every gate target has an entry and every entry names a real target. Its own self-test must fail when an entry is removed. +**Done 2026-08-01.** `gates.toml` (9 gates), `make gate-review`, and a +`loop-lint` check in both directions — an unclassified `make all` +dependency and an entry naming a target the Makefile lacks each produce +a finding, both pinned by self-tests. First run: **0 due, 2 silent** +(the chaos roll and `gate-review` itself). + ## Task: InnerLoop v1.6 ```task