T01: audit every InnerLoop rule, and make the checkable ones executable

41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md

New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:

  - specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
    the loop has stated since v0.2 and never measured. Split at its own
    section boundaries into Blueprint (1-8) + Runtime (9-15).
  - tools/dep-weight.py and tools/rule-coverage.py had positive-control
    logic and no --self-test, so nothing verified the control worked.

Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.

Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.

The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.

Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.

make loop-lint and make self-tests wired into `make all` and CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 09:16:00 +02:00
parent ee1ee485b2
commit fed422a3a3
10 changed files with 827 additions and 242 deletions

View file

@ -151,11 +151,12 @@ as a target, or it is cited as unverified.** Quoting is not measuring.
from a prior pass. Re-derivation put it at $92.21 — the quoted figure
double-counted transcript lines and priced a three-model session at one
model's rate. Neither error was of the harness-does-nothing class; both
sums ran over real data, and a positive control would have passed them.)* A metric must
also be checked against the contracts in the *same spec*: if a contract
makes a target unreachable, one of the two is wrong and the conflict is
resolved when it is noticed, not at the acceptance run. Re-check the
table whenever a contract is added.
sums ran over real data, and a positive control would have passed them.)*
**A metric is checked against the contracts in its own spec.** If a
contract makes a target unreachable, one of the two is wrong and the
conflict is resolved when it is noticed, not at the acceptance run.
Re-check the table whenever a contract is added.
*(v1.0, from CB-WP-0001: AM-4's ≤20-crate target was made unreachable by
the K5 and K7 contracts written after it, and AM-12's cost metric was
@ -297,8 +298,9 @@ The loop exists to be driven by agents. Therefore:
1. **Whole-file loadability** — every loop artifact stays under ~400 lines;
split before exceeding, link with relative paths.
2. **Structured over prose** — tables and fenced blocks for anything a
later step must parse (baselines, acceptance metrics, evidence rows).
2. **Structured over prose** *(guidance, not a requirement — nothing can
check it)* — tables and fenced blocks for anything a later step must
parse (baselines, acceptance metrics, evidence rows).
3. **One command surface** — all checks runnable through repo-root
commands (eventually `cb *`; until then, `make`/`cargo` aliases declared
in one place), each supporting deterministic, greppable output.
@ -307,9 +309,20 @@ The loop exists to be driven by agents. Therefore:
task text plus linked files alone.
5. **Evidence or it didn't happen** — claims of "better" live in committed
evidence files with numbers, never only in commit messages or chat.
6. **Token discipline** — per the global budget policy, a loop iteration
6. ~~**Token discipline** — per the global budget policy, a loop iteration
that exceeds its budget without measurable progress is stopped and
decomposed, not pushed through.
decomposed, not pushed through.~~ **DEAD POLICY.** The 8k/10k per-task
token budget was never referenced or enforced, and CB-WP-0001 T08
exceeded it by orders of magnitude with no signal. It implies a control
that does not exist. Replacement in USD is CB-WP-0003 T05; until then
this is documentation of a gap, not a rule.
**Enforcement status.** Rules above that a command can check are enforced
by `make loop-lint`; the full classification of every InnerLoop rule as
executable / checkable / decorative, with the failure class each catches,
is in `history/260731-inner-loop-rule-audit.md`. Rules marked *guidance*
or *dead* say so where they appear, so a reader can tell a requirement
from a preference without consulting the audit.
---
@ -335,4 +348,3 @@ A capability has completed the loop when all of the following are committed:
([CostAccounting.md](CostAccounting.md))
- [ ] retrospective note (may be one paragraph appended to the evidence
file): what the loop itself should change
```