2026-07-31 00:31:14 +02:00
# The Inner Loop — Assimilate and Surpass
2026-08-01 15:43:34 +02:00
Status: **v1.6** — corrected from CB-WP-0008 (stage 0) and ADR-0006 on
2026-08-01. Changes from v1.5: budgets measure the window they govern and
a breach never blocks its own repair (D1/D2); loop work cashes out into
something runnable in the pass that invents it (D4); the tier table
covers changes to the loop's own operation (D5).
CB-WP-0019-T05: what the meta budget is for, at 80/20
InnerLoop v1.7. The purpose is written first and the number follows from
it: most spend on the task at hand, some on control, review and
improving the process. make status prints it above the figure, because a
threshold with no stated purpose is what let this number be argued three
times.
Soft 20% over a trailing 5, and the self-test enforces that the ratio and
the window are a PAIR: META_SOFT_PCT == 100 / TRAILING_PASSES. One meta
pass among n at parity cost reads 1/n, so 80/20 is one pass in five at
normal cost -- a five-pass window. The same 20% over three would have
silently also demanded the meta pass be half-price, which makes meta work
rushed rather than rare. Moving the ratio without the window goes red.
The phase setting is declared, argued and expiring in gates.toml, and
reverts on review_by unless re-argued. Verified live at 35%. One with no
reason or no expiry is refused rather than honoured, because a threshold
anyone may move is not a threshold.
Measured: the last five passes read 7% against the new line.
InnerLoop.md crossed the 400-line limit three times while this was
written and was fixed structurally each time -- the arithmetic, the
cost-per-response basis and the two review case studies moved to
InnerLoopReference.md. The limit was not raised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 18:35:05 +02:00
> **What the budget is for (v1.7, CB-WP-0019 T05).** Most of what this
> project spends should go on **the task at hand**. Some of it must go on
> **control, review, and improving the process that carries the work
> forward** — a loop that never inspects itself accumulates exactly the
> defects this one keeps finding, and a loop that only inspects itself
> ships nothing. The budget names the split so that neither happens by
> drift.
>
> **Meta budget — soft, 20% of the trailing five passes.** Measured by
> `make status` from each workplan's `kind:` frontmatter (`product` |
> `meta` | `mixed`), over a **trailing 5-pass window**; the lifetime
> figure is reported as history and is not the metric (ADR-0006 D1).
>
> **The ratio and the window are a pair** — 20% over five is *one pass in
> five at normal cost*; the same number over three would mean *one in five
> and at half price*. `InnerLoopReference.md` §Meta budget has the
> arithmetic.
CB-WP-0007 T01+T03: window the metric, budget it, cap meta at 25%
Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:07 +02:00
>
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
> **Soft on purpose.** A task already started may be finished; what the
> budget constrains is *opening* new meta work above the line, with two
2026-08-01 15:43:34 +02:00
> exemptions:
>
> 1. **Repairing the instrument that reports a breach is always in
> budget.** A gate must never block its own remedy (ADR-0006 D2,
> CB-RES-0005 §4). v1.5 had no such exemption, and the first thing it
> did was forbid fixing the budget's own defect.
> 2. **The maintainer may authorize above-line work**, recorded as
> `authorized_above_budget:` in the workplan frontmatter. Undocumented
> above-line meta work is a violation the evidence file must report.
CB-WP-0007 T01+T03: window the metric, budget it, cap meta at 25%
Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:07 +02:00
>
CB-WP-0019-T05: what the meta budget is for, at 80/20
InnerLoop v1.7. The purpose is written first and the number follows from
it: most spend on the task at hand, some on control, review and
improving the process. make status prints it above the figure, because a
threshold with no stated purpose is what let this number be argued three
times.
Soft 20% over a trailing 5, and the self-test enforces that the ratio and
the window are a PAIR: META_SOFT_PCT == 100 / TRAILING_PASSES. One meta
pass among n at parity cost reads 1/n, so 80/20 is one pass in five at
normal cost -- a five-pass window. The same 20% over three would have
silently also demanded the meta pass be half-price, which makes meta work
rushed rather than rare. Moving the ratio without the window goes red.
The phase setting is declared, argued and expiring in gates.toml, and
reverts on review_by unless re-argued. Verified live at 35%. One with no
reason or no expiry is refused rather than honoured, because a threshold
anyone may move is not a threshold.
Measured: the last five passes read 7% against the new line.
InnerLoop.md crossed the 400-line limit three times while this was
written and was fixed structurally each time -- the arithmetic, the
cost-per-response basis and the two review case studies moved to
InnerLoopReference.md. The limit was not raised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 18:35:05 +02:00
> **The phase setting is not a dial.** The split may move for a phase, but
> only **declared, argued, and expiring** — `meta_phase = { pct, reason,
> review_by }` in `gates.toml`, reverting to 20 on `review_by` unless
> re-argued. A setting with no reason or no expiry fails
> `status.py --self-test`, so the line cannot be raised quietly to pass a
> breach.
>
CB-WP-0007 T01+T03: window the metric, budget it, cap meta at 25%
Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:07 +02:00
> A pass that exceeds it **says so in its evidence file and names the
2026-08-01 15:43:34 +02:00
> product work displaced**. That is the whole enforcement: a reporting
> budget, not a gate, for the same reason the session-shape budget is —
> it constrains judgment, not artifacts.
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
> **Workplan lifecycle (v1.7).** `ready` → `active` → `done`. **`ready`
> means declared and not yet started**; a workplan being worked is
> `active`, so the status answers *"is anyone on this?"* rather than only
> *"is it finished?"*. `loop-lint` fails when a workplan has started work
> and still says `ready`, or has finished it and does not say `done`.
> Tasks keep `todo` | `done` | `cancel`; `cancel` is a **closed** outcome
> and is counted as such.
> **Quoting a cost (v1.7).** An evidence file **may not quote its own
> pass's cost as final** — six passes under-reported themselves by 30–
> never once high. **Re-run the instrument at the moment of quoting**;
> quoting from memory defeats the rule even when the boundary is right.
> `loop-lint` fails an evidence file naming its own workplan beside a
> dollar figure without marking it provisional.
> `InnerLoopReference.md` §Quoting a cost.
2026-08-01 15:43:34 +02:00
> **Loop work cashes out (v1.6, ADR-0006 D4).** A `meta` pass must ship at
> least one executable check, command, or gate. Findings that stay prose
> are recorded as findings and do **not** count as delivery.
CB-WP-0007 T01+T03: window the metric, budget it, cap meta at 25%
Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:07 +02:00
>
CB-WP-0019-T05: what the meta budget is for, at 80/20
InnerLoop v1.7. The purpose is written first and the number follows from
it: most spend on the task at hand, some on control, review and
improving the process. make status prints it above the figure, because a
threshold with no stated purpose is what let this number be argued three
times.
Soft 20% over a trailing 5, and the self-test enforces that the ratio and
the window are a PAIR: META_SOFT_PCT == 100 / TRAILING_PASSES. One meta
pass among n at parity cost reads 1/n, so 80/20 is one pass in five at
normal cost -- a five-pass window. The same 20% over three would have
silently also demanded the meta pass be half-price, which makes meta work
rushed rather than rare. Moving the ratio without the window goes red.
The phase setting is declared, argued and expiring in gates.toml, and
reverts on review_by unless re-argued. Verified live at 35%. One with no
reason or no expiry is refused rather than honoured, because a threshold
anyone may move is not a threshold.
Measured: the last five passes read 7% against the new line.
InnerLoop.md crossed the 400-line limit three times while this was
written and was fixed structurally each time -- the arithmetic, the
cost-per-response basis and the two review case studies moved to
InnerLoopReference.md. The limit was not raised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 18:35:05 +02:00
> Measured basis and its falsifier: `InnerLoopReference.md` §Loop work
> cashes out — **cost tracks distance from a runnable check**.
2026-08-01 15:43:34 +02:00
> **Every gate is an experiment (v1.6, ADR-0006 D3).** Each standing
> control gate has an entry in `gates.toml` naming what it has caught, a
> `review_by` date, and what would retire it. `make gate-review` reports
> what is overdue and what has caught nothing; `loop-lint` fails when the
> registry drifts from the Makefile in either direction.
CB-WP-0007 T01+T03: window the metric, budget it, cap meta at 25%
Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:07 +02:00
v1.4 — corrected from CB-WP-0005 (assertion coverage) on
CB-WP-0005 T07/T08: control loop, retrospective, InnerLoop v1.4
T07 — CB-EV-0004. 70 responses, $16.03.
Test 1 MET: 1 spec -> 2, 58 rules -> 76, 1 source file -> 10, and the new
denominator came in at 83% naming K10/K14/K18. The stated failure mode — a
widened denominator reporting the same percentage — did not occur.
Test 2 UNMET: M-D1-MUT 4 of 14 against >=10, and wrong about what as well
as how much. The diagnosis was three absent kernel rules; the measurement
found eight rows with no instrument at all.
Test 3: quality held, and widening surfaced far more than the seven known
defects — six further uninstrumented rows, HDN #7 (rule-coverage's
self-test green while the tool was broken), a stale $248.46 invisible to
facts-check because it was untagged, and a fifth error class.
The clean test CB-WP-0004 was owed is now run, on a pass that used the
tools without building them. The two categories whose tools removed the
manual path are at 0 turns two passes on; the two that merely offered a
better option are now the entire mechanical cost of a pass. Absolute
mechanical cost fell $51.76 -> $6.13. The mechanism holds.
Reported because nothing else would: SH-3 batching is 0.0% this pass — 67
tool calls across 67 responses — against a 20% target, and mean context
315,170 against 200,000. SessionShape has stated these since CB-WP-0003
and none has ever been enforced.
T08 — the retrospective answers its question: yes, a weak mutation is the
new grep, and it is worse, because it fails in the opposite direction.
mutation-check's first run produced two SURVIVED verdicts and both were
the author's own no-op mutations.
The fifth error class: false accusation. HDN, TA, SSB and DFD all
under-report — a real problem passes. FA over-reports: it publishes the
claim that working code is broken, sends the next pass to fix something
that is not broken, and is more credible than the truth because it arrives
with a measurement attached. Thirteen instances, five classes, six passes,
and the newest class is one that hardening created.
One correction to CB-WP-0004 T06: "a gate only pays if it removes the
manual path" is a predictor of whether a gate saves money, not a criterion
for whether it is worth having. mutation-check fails that test and
produced the most valuable findings of the pass.
InnerLoop v1.4: where a claim rests on numbers, the adversarial reviewer
must read the assertion behind each quoted number and mutate it.
Re-running the command that prints a number is not verification of that
number. Second verification step to inherit the author's blindness; both
fixes replace re-derivation with adversarial execution.
CB-WP-0005 status -> done, 5 of 8 tasks, 3 cancelled into CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:05:31 +02:00
2026-07-31. Change from v1.3: where a claim rests on numbers, the
adversarial reviewer must read the assertion behind each quoted number and
**mutate it** — re-running the command that prints a number is not
verification of that number (§Step 2).
2026-08-01 15:43:34 +02:00
v1.5 (meta budget), v1.3 (executable single source of fact), v1.2
(design goal: cheap correction over exhaustive prevention), v1.1
(instruments emit their own targets) and v1.0: see
`specs/InnerLoopReference.md` §Change log.
2026-07-31 00:31:14 +02:00
Normative process for building every Clay-Borg capability. Referenced by
T10: InnerLoop v1.2 — hardening does not converge, so optimize correction
The retrospective question was whether the mechanism set is complete or
each pass still finds a new class. This pass produced both an eighth
instance AND a fourth class, so the answer is the uncomfortable one.
Ledger: 10 instances, 4 classes, across 3 workplans. Every pass has
produced at least one class the previous pass had not seen.
HDN harness-does-nothing 5 executable assertions
TA trusted arithmetic 3 re-derivation
SSB same-sample blind spot 1 assertions over ALL the data
DFD duplicated-fact drift 2 NEW -- reading a copy against source
DFD is genuinely distinct: no positive control catches it, because both
copies are internally consistent, and re-derivation does not either,
because the copy faithfully reproduces what it was copied from. Found
when an inlined price sheet went stale within an hour of T11 changing
the real one.
So v1.2 stops trying to enumerate classes in advance. Every error in
three passes was corrected in-session for under ~1% of the pass, so the
stated design goal is now cheap CORRECTION: keep raw data so numbers are
re-derivable, keep artifacts small and committed so a wrong number is
one grep from everywhere quoting it, give every number a command.
Plus the one rule the new class earns: single source of fact.
The original hypothesis is revised rather than confirmed. "A rule that
cannot be executed is not a rule" is wrong -- the two most valuable
corrections in the project came from a decorative rule that cannot be
automated (re-derive inherited numbers). An executable rule fires
reliably and catches one class; a decorative one fires unreliably and
can catch any class, including unnamed ones. Keep both.
Gates this pass: loop-lint caught 3 real violations on first run, then
failed on its own author within the hour when a T07 edit pushed
InnerLoop.md to 407 lines against its own 400 limit.
CB-WP-0003 complete: 11 of 11 tasks done.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:31:19 +02:00
all workplans.
**Design goal (v1.2).** Three passes produced ten error instances across
four classes, and every pass produced a class the previous one had not
seen — prevention is not converging. Every one of those errors was
corrected inside the same session for under ~1% of the pass. The loop
therefore optimizes for **cheap correction** : keep the raw data so numbers
can be re-derived, keep artifacts small and committed so a wrong number is
one grep from everywhere that quotes it, and give every reported number a
command so re-running is free.
> **Single source of fact.** A number, rate, or target lives in exactly
> one place; everywhere else links to it. Where a copy is unavoidable it
> is generated by a command, not typed. *(v1.2: a price sheet inlined into
> a spec went stale within the hour of the real sheet changing, and one
> acceptance figure had to be chased across three artifacts each time it
> moved. No positive control catches this — both copies are internally
> consistent — and re-derivation does not either, because the copy
2026-07-31 10:24:39 +02:00
> reproduces whatever it was copied from.)*
>
> **Now executable (v1.3, CB-WP-0004 T04):** `make facts-check`. `facts.toml`
> is generated from the instruments, never edited; an artifact quoting a
> registry value tags it `<!-- fact:<key> -->` and the gate fails when the
> two disagree. Untagged literal copies are reported, not failed — that is
> the drift surface still uncovered, and naming it is more useful than
> pretending it is closed.
The loop's own optimization target is **agentic efficiency** :
2026-07-31 00:31:14 +02:00
every artifact it produces must be small enough to load whole, structured
enough to act on without interpretation, and falsifiable enough that an
agent can judge its own work without a human in the iteration.
---
## The five steps
```text
1 RESEARCH → research/CB-RES-NNNN-< slug > .md (survey, baselines)
2 APPROVE → decision recorded in the ADR (gate: survey complete?)
3 DECIDE → decisions/ADR-NNNN-< slug > .md (assimilate/reimplement/hybrid)
4 SPECIFY → specs/< Capability > .md (contracts + acceptance metrics)
5 CODE LOOP → code + scenarios + benchmarks (iterate until metrics beat baseline)
```
**Hard gate: no implementation code for a capability exists before its ADR
2026-07-31 00:48:04 +02:00
(step 3) is committed.**
## Loop tiers and the chaos roll
Every work packet declares a tier before work starts. The tier sets how
heavy steps 1– –
are never skipped for code-producing work.
| Tier | Weight of steps 1–
|---|---|---|
| **L** | Full: separate survey, adversarial review, ADR | Creates a new capability port, or is named a high-leverage pass by the maintainer |
2026-08-01 15:43:34 +02:00
| **M** | Survey and ADR merged into one document; review optional | Touches a canonical interface, adds/updates an external dependency, **or changes whether or how the loop constrains its own operation** — budgets, gates, review requirements, or these tier rules (v1.6, ADR-0006 D5) |
2026-07-31 00:48:04 +02:00
| **S** | One provenance paragraph in the commit message | Everything else (utilities, fixes, refactors inside a boundary) |
CB-WP-0018 T03/T04: explanations, and window 1's verdict
T03: input::describe writes a sentence per legal command; data-descs
carries them in step with data-targets; the ghost already following the
pointer shows the one for whatever legal target is under it, so the
explanation lands beside the target with no overlay layer to keep
aligned. ADR-0010 D1 binds -- the page renders it, never composes it.
Both mutations INITIALLY SURVIVED because the fixture's Attack card had
exactly one target, where an off-by-one shift and a truncation are both
no-ops. CB-EV-0014's lesson one level in: a fixture too thin to express
a failure is how the failure survives. Two attack targets now, both red.
T04: chaos rate d4 -> d8, window 2 open at 12 declarations, retiring if
an override changes nothing twice running. Window 1's condition was NOT
met -- both overrides changed the outcome -- so the mechanism is kept.
The weakest part of the decision is that it is a rate change argued from
n=2, so window 2 carries a falsifier: no override at all is evidence the
rate went too far, not that the mechanism is healthy.
InnerLoop.md hit 401 lines and the loadability gate fired; the rationale
moved to InnerLoopReference.md, structurally, per the standing precedent
that limits are not raised.
CB-WP-0017 settled at $9.48/40 against $5.19/23 reported mid-flight,
83% higher. Six for six, always low -- read by re-running the instrument
at the moment of quoting, which is CB-EV-0015's correction applied for
the first time.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 02:24:13 +02:00
**The chaos roll.** After deriving the structural tier, roll **d8**
(`shuf -i 1-8 -n 1` ). On an **8** , the tier is instead picked uniformly at
2026-07-31 00:48:04 +02:00
random (`shuf -e S M L -n 1` ), overriding the structural derivation — up or
2026-07-31 09:24:12 +02:00
down.
ADR-0017: window 2's verdict — the mechanism worked, my account of it did
not
Tier M (changes how the loop constrains its own operation), declared at
d8 because the rate for window 3 is what this document decides and
declaring at a rate it invents would be circular. chaos d8 = 7, no
override.
I CLAIMED WINDOW 2 PRODUCED ZERO OVERRIDES, FIVE TIMES, AND IT IS FALSE.
Declaration 3 (CB-WP-0020) rolled d8 = 8, overrode, drew S against a
structural S, and changed nothing -- and CB-WP-0020 recorded it correctly
at the time, in those words: "the first override at d8... It changed
nothing... One."
Counting the workplans takes one command and I never ran it. CB-EV-0024
asserted "zero" without checking; CB-EV-0025, 0026, 0027 and CB-WP-0029
each cited the one before. A claim propagated five times by citation
rather than by measurement, in files whose subject was that exact failure.
facts-check catches a copied number that disagrees with its source;
nothing catches a number with NO source, asserted once and repeated.
Registered F23, and all four evidence files carry an in-place correction
rather than a silent edit (ADR-0012 D5).
THE ACTUAL VERDICT: THE RATE IS WORKING. Eleven rolls at d8 -- declaration
1 opened the window at the old d4 -- against 1.375 eights expected, 1
observed. Not a shortage of evidence; the design.
BUT THE RETIREMENT CONDITION GENUINELY CANNOT FIRE, and that took
computing to see. "An override changes nothing twice running" needs a
consecutive pair at P=1/3 each, so ~12 overrides expected, at ~1.4 per
window: ~9 windows, roughly 100 declarations. A gate that cannot cash out
on any realistic horizon is decoration, which ADR-0006 D3 forbids.
Restated to be evaluated PER WINDOW: retire if a full window's overrides
all change nothing, met in two consecutive windows. A window with no
overrides is inconclusive and advances nothing. ~24 declarations rather
than ~100. Window 2 counts as the first; window 3 opens at d8 and decides.
Recorded and deliberately not acted on: declarations 5-9 rolled six five
times running, ~1 in 370 for some run of five in eleven rolls. shuf tested
over 200 rapid successive calls looks uniform, longest run three. Found
post hoc, which is how coincidences become findings, so it is logged for a
future window to check rather than treated as evidence.
InnerLoop.md then crossed the loadability limit, and so did
InnerLoopReference.md. The window log moved to specs/ChaosRollHistory.md:
it grows by one entry per window, and a log inside a reference eventually
crowds out the reference.
make all: exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:55:54 +02:00
> **Window 3, opened 2026-08-07 at d8**, running to 12 declarations.
CB-WP-0018 T03/T04: explanations, and window 1's verdict
T03: input::describe writes a sentence per legal command; data-descs
carries them in step with data-targets; the ghost already following the
pointer shows the one for whatever legal target is under it, so the
explanation lands beside the target with no overlay layer to keep
aligned. ADR-0010 D1 binds -- the page renders it, never composes it.
Both mutations INITIALLY SURVIVED because the fixture's Attack card had
exactly one target, where an off-by-one shift and a truncation are both
no-ops. CB-EV-0014's lesson one level in: a fixture too thin to express
a failure is how the failure survives. Two attack targets now, both red.
T04: chaos rate d4 -> d8, window 2 open at 12 declarations, retiring if
an override changes nothing twice running. Window 1's condition was NOT
met -- both overrides changed the outcome -- so the mechanism is kept.
The weakest part of the decision is that it is a rate change argued from
n=2, so window 2 carries a falsifier: no override at all is evidence the
rate went too far, not that the mechanism is healthy.
InnerLoop.md hit 401 lines and the loadability gate fired; the rationale
moved to InnerLoopReference.md, structurally, per the standing precedent
that limits are not raised.
CB-WP-0017 settled at $9.48/40 against $5.19/23 reported mid-flight,
83% higher. Six for six, always low -- read by re-running the instrument
at the moment of quoting, which is CB-EV-0015's correction applied for
the first time.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 02:24:13 +02:00
>
ADR-0017: window 2's verdict — the mechanism worked, my account of it did
not
Tier M (changes how the loop constrains its own operation), declared at
d8 because the rate for window 3 is what this document decides and
declaring at a rate it invents would be circular. chaos d8 = 7, no
override.
I CLAIMED WINDOW 2 PRODUCED ZERO OVERRIDES, FIVE TIMES, AND IT IS FALSE.
Declaration 3 (CB-WP-0020) rolled d8 = 8, overrode, drew S against a
structural S, and changed nothing -- and CB-WP-0020 recorded it correctly
at the time, in those words: "the first override at d8... It changed
nothing... One."
Counting the workplans takes one command and I never ran it. CB-EV-0024
asserted "zero" without checking; CB-EV-0025, 0026, 0027 and CB-WP-0029
each cited the one before. A claim propagated five times by citation
rather than by measurement, in files whose subject was that exact failure.
facts-check catches a copied number that disagrees with its source;
nothing catches a number with NO source, asserted once and repeated.
Registered F23, and all four evidence files carry an in-place correction
rather than a silent edit (ADR-0012 D5).
THE ACTUAL VERDICT: THE RATE IS WORKING. Eleven rolls at d8 -- declaration
1 opened the window at the old d4 -- against 1.375 eights expected, 1
observed. Not a shortage of evidence; the design.
BUT THE RETIREMENT CONDITION GENUINELY CANNOT FIRE, and that took
computing to see. "An override changes nothing twice running" needs a
consecutive pair at P=1/3 each, so ~12 overrides expected, at ~1.4 per
window: ~9 windows, roughly 100 declarations. A gate that cannot cash out
on any realistic horizon is decoration, which ADR-0006 D3 forbids.
Restated to be evaluated PER WINDOW: retire if a full window's overrides
all change nothing, met in two consecutive windows. A window with no
overrides is inconclusive and advances nothing. ~24 declarations rather
than ~100. Window 2 counts as the first; window 3 opens at d8 and decides.
Recorded and deliberately not acted on: declarations 5-9 rolled six five
times running, ~1 in 370 for some run of five in eleven rolls. shuf tested
over 200 rapid successive calls looks uniform, longest run three. Found
post hoc, which is how coincidences become findings, so it is logged for a
future window to check rather than treated as evidence.
InnerLoop.md then crossed the loadability limit, and so did
InnerLoopReference.md. The window log moved to specs/ChaosRollHistory.md:
it grows by one entry per window, and a log inside a reference eventually
crowds out the reference.
make all: exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:55:54 +02:00
> **Retirement condition (ADR-0017 D2):** retire the roll if **a full
> window's overrides all change nothing**, met in **two consecutive
> windows**. A window with **no** overrides is *inconclusive* and advances
> nothing. **Window 2 counts as the first.**
>
> Both windows' full records, the rate history, and why the old
> "twice running" condition was replaced:
> [`specs/ChaosRollHistory.md`](ChaosRollHistory.md), and
> `specs/InnerLoopReference.md` §Chaos roll — calibration.
CB-WP-0018 T03/T04: explanations, and window 1's verdict
T03: input::describe writes a sentence per legal command; data-descs
carries them in step with data-targets; the ghost already following the
pointer shows the one for whatever legal target is under it, so the
explanation lands beside the target with no overlay layer to keep
aligned. ADR-0010 D1 binds -- the page renders it, never composes it.
Both mutations INITIALLY SURVIVED because the fixture's Attack card had
exactly one target, where an off-by-one shift and a truncation are both
no-ops. CB-EV-0014's lesson one level in: a fixture too thin to express
a failure is how the failure survives. Two attack targets now, both red.
T04: chaos rate d4 -> d8, window 2 open at 12 declarations, retiring if
an override changes nothing twice running. Window 1's condition was NOT
met -- both overrides changed the outcome -- so the mechanism is kept.
The weakest part of the decision is that it is a rate change argued from
n=2, so window 2 carries a falsifier: no override at all is evidence the
rate went too far, not that the mechanism is healthy.
InnerLoop.md hit 401 lines and the loadability gate fired; the rationale
moved to InnerLoopReference.md, structurally, per the standing precedent
that limits are not raised.
CB-WP-0017 settled at $9.48/40 against $5.19/23 reported mid-flight,
83% higher. Six for six, always low -- read by re-running the instrument
at the moment of quoting, which is CB-EV-0015's correction applied for
the first time.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 02:24:13 +02:00
>
> Record the roll every time, including when it changes nothing
> (`tier: L (structural L, chaos 8)`). Why the rate fell, why n=2 makes
> that the weakest part of the decision, and the dead rolls:
ADR-0017: window 2's verdict — the mechanism worked, my account of it did
not
Tier M (changes how the loop constrains its own operation), declared at
d8 because the rate for window 3 is what this document decides and
declaring at a rate it invents would be circular. chaos d8 = 7, no
override.
I CLAIMED WINDOW 2 PRODUCED ZERO OVERRIDES, FIVE TIMES, AND IT IS FALSE.
Declaration 3 (CB-WP-0020) rolled d8 = 8, overrode, drew S against a
structural S, and changed nothing -- and CB-WP-0020 recorded it correctly
at the time, in those words: "the first override at d8... It changed
nothing... One."
Counting the workplans takes one command and I never ran it. CB-EV-0024
asserted "zero" without checking; CB-EV-0025, 0026, 0027 and CB-WP-0029
each cited the one before. A claim propagated five times by citation
rather than by measurement, in files whose subject was that exact failure.
facts-check catches a copied number that disagrees with its source;
nothing catches a number with NO source, asserted once and repeated.
Registered F23, and all four evidence files carry an in-place correction
rather than a silent edit (ADR-0012 D5).
THE ACTUAL VERDICT: THE RATE IS WORKING. Eleven rolls at d8 -- declaration
1 opened the window at the old d4 -- against 1.375 eights expected, 1
observed. Not a shortage of evidence; the design.
BUT THE RETIREMENT CONDITION GENUINELY CANNOT FIRE, and that took
computing to see. "An override changes nothing twice running" needs a
consecutive pair at P=1/3 each, so ~12 overrides expected, at ~1.4 per
window: ~9 windows, roughly 100 declarations. A gate that cannot cash out
on any realistic horizon is decoration, which ADR-0006 D3 forbids.
Restated to be evaluated PER WINDOW: retire if a full window's overrides
all change nothing, met in two consecutive windows. A window with no
overrides is inconclusive and advances nothing. ~24 declarations rather
than ~100. Window 2 counts as the first; window 3 opens at d8 and decides.
Recorded and deliberately not acted on: declarations 5-9 rolled six five
times running, ~1 in 370 for some run of five in eleven rolls. shuf tested
over 200 rapid successive calls looks uniform, longest run three. Found
post hoc, which is how coincidences become findings, so it is logged for a
future window to check rather than treated as evidence.
InnerLoop.md then crossed the loadability limit, and so did
InnerLoopReference.md. The window log moved to specs/ChaosRollHistory.md:
it grows by one entry per window, and a log inside a reference eventually
crowds out the reference.
make all: exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:55:54 +02:00
> [`specs/ChaosRollHistory.md`](ChaosRollHistory.md), and
2026-08-01 15:43:34 +02:00
> `specs/InnerLoopReference.md` §Chaos roll — calibration.
2026-07-31 00:48:04 +02:00
Chaos limits: a rolled-down tier relaxes *process* weight only. Invariants
(zero foreign types in canonical interfaces, determinism, passing
conformance suites) bind at every tier, and a rolled-down pass touching a
canonical interface still requires the interface change to be flagged in
the commit for retrospective review.
2026-07-31 00:31:14 +02:00
### Step 1 — Research
Identify the best implementation in existence for this capability. Produce
`research/CB-RES-NNNN-<slug>.md` following the survey template (below).
The survey is done when it can name, per dimension, a concrete
**benchmark-to-beat**: a number, a property, or a reproducible comparison —
not an impression.
2026-07-31 00:48:04 +02:00
**Runnable-baseline option.** For passes judged high-leverage (declared by
the maintainer or proposed in the survey and confirmed in the ADR), cited
numbers are not enough: the survey must ship a reproducible **baseline
harness** that runs the leading candidate on our machine against our
workload — the same scenario files where feasible. The harness ships with a
*fidelity note* stating what was and wasn't faithfully reproduced, so a
hastily wired competitor setup cannot silently inflate our advantage.
Where the option is not invoked (or the candidate isn't practically
runnable), comparisons against cited-only numbers are **directional** : the
evidence verdict for those rows caps at `parity` , never `better` .
### Step 2 — Approve (adversarial review)
For tier-L passes, approval is earned through an **adversarial review** : a
T03: point review at the harness, and state what review cannot catch
The review step targeted the survey. Every serious error in this project
has been in measurement or build configuration, so the loop was
adversarially reviewing the artifact cheapest to fix and leaving
unreviewed the one where errors occur.
Step 2 now routes by risk: when the claim rests on numbers, the reviewer
gets the harness and the evidence file too, and must reproduce the
number independently rather than read about it.
The addition that matters more, because it was learned the hard way: a
reviewer re-derives the author's claims and therefore inherits the
author's SAMPLING. CB-WP-0002's dedup invariant was checked twice --
survey 206/206 groups, then the reviewer independently -- and both used
the main transcript. It is false in the 8-response subagent tree neither
looked at. Two independent verifications, one shared blind spot.
Rule: the reviewer re-derives on a different sample than the author
used, and where only one sample exists, says so rather than reporting
a clean verify.
Also recorded: what review demonstrably DOES do. $0.66 and $1.11 across
two passes, ~1% of each, both finding approval-blocking defects. Cost is
not a reason to skip it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:26:01 +02:00
separate session (or agent) attempts to break the work. Exactly **one
round**: challenge, then response. The work is approvable only when every
challenge is either answered with evidence or conceded and folded in.
**The review target follows the risk.** Reviewing the survey was the
original rule, and it is the wrong target for a capability whose claim
rests on numbers — every serious error in this project has been in
measurement or build configuration, not in prose.
| the claim rests on | the reviewer is given | and must attempt |
|---|---|---|
| a survey of candidates | the survey | an omitted candidate; a stale or unverifiable benchmark; an unmeasured claim presented as measured |
| **numbers** | the survey **and the harness and the evidence file** | **reproduce the number independently** ; state what the harness would report if the work silently stopped |
**What review cannot do — state this, do not discover it.** A reviewer
re-derives the author's claims and therefore inherits the author's
sampling. So:
> **The reviewer re-derives on a different sample than the author used.**
> Where only one sample exists, the review says so rather than reporting a
> clean verify.
CB-WP-0005 T07/T08: control loop, retrospective, InnerLoop v1.4
T07 — CB-EV-0004. 70 responses, $16.03.
Test 1 MET: 1 spec -> 2, 58 rules -> 76, 1 source file -> 10, and the new
denominator came in at 83% naming K10/K14/K18. The stated failure mode — a
widened denominator reporting the same percentage — did not occur.
Test 2 UNMET: M-D1-MUT 4 of 14 against >=10, and wrong about what as well
as how much. The diagnosis was three absent kernel rules; the measurement
found eight rows with no instrument at all.
Test 3: quality held, and widening surfaced far more than the seven known
defects — six further uninstrumented rows, HDN #7 (rule-coverage's
self-test green while the tool was broken), a stale $248.46 invisible to
facts-check because it was untagged, and a fifth error class.
The clean test CB-WP-0004 was owed is now run, on a pass that used the
tools without building them. The two categories whose tools removed the
manual path are at 0 turns two passes on; the two that merely offered a
better option are now the entire mechanical cost of a pass. Absolute
mechanical cost fell $51.76 -> $6.13. The mechanism holds.
Reported because nothing else would: SH-3 batching is 0.0% this pass — 67
tool calls across 67 responses — against a 20% target, and mean context
315,170 against 200,000. SessionShape has stated these since CB-WP-0003
and none has ever been enforced.
T08 — the retrospective answers its question: yes, a weak mutation is the
new grep, and it is worse, because it fails in the opposite direction.
mutation-check's first run produced two SURVIVED verdicts and both were
the author's own no-op mutations.
The fifth error class: false accusation. HDN, TA, SSB and DFD all
under-report — a real problem passes. FA over-reports: it publishes the
claim that working code is broken, sends the next pass to fix something
that is not broken, and is more credible than the truth because it arrives
with a measurement attached. Thirteen instances, five classes, six passes,
and the newest class is one that hardening created.
One correction to CB-WP-0004 T06: "a gate only pays if it removes the
manual path" is a predictor of whether a gate saves money, not a criterion
for whether it is worth having. mutation-check fails that test and
produced the most valuable findings of the pass.
InnerLoop v1.4: where a claim rests on numbers, the adversarial reviewer
must read the assertion behind each quoted number and mutate it.
Re-running the command that prints a number is not verification of that
number. Second verification step to inherit the author's blindness; both
fixes replace re-derivation with adversarial execution.
CB-WP-0005 status -> done, 5 of 8 tasks, 3 cancelled into CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:05:31 +02:00
>
> **And re-derivation is not enough (v1.4).** Where the claim rests on
> numbers, the reviewer must **read the assertion behind each quoted
> number and mutate it**: invert the property and require the suite to go
> red. Re-running the command that prints a number satisfies "reproduce
> independently" and finds nothing of this class.
>
CB-WP-0019-T05: what the meta budget is for, at 80/20
InnerLoop v1.7. The purpose is written first and the number follows from
it: most spend on the task at hand, some on control, review and
improving the process. make status prints it above the figure, because a
threshold with no stated purpose is what let this number be argued three
times.
Soft 20% over a trailing 5, and the self-test enforces that the ratio and
the window are a PAIR: META_SOFT_PCT == 100 / TRAILING_PASSES. One meta
pass among n at parity cost reads 1/n, so 80/20 is one pass in five at
normal cost -- a five-pass window. The same 20% over three would have
silently also demanded the meta pass be half-price, which makes meta work
rushed rather than rare. Moving the ratio without the window goes red.
The phase setting is declared, argued and expiring in gates.toml, and
reverts on review_by unless re-argued. Verified live at 35%. One with no
reason or no expiry is refused rather than honoured, because a threshold
anyone may move is not a threshold.
Measured: the last five passes read 7% against the new line.
InnerLoop.md crossed the 400-line limit three times while this was
written and was fixed structurally each time -- the arithmetic, the
cost-per-response basis and the two review case studies moved to
InnerLoopReference.md. The limit was not raised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 18:35:05 +02:00
*Both cases that produced these rules — CB-WP-0005's inert `AM-7 replay`
clause and CB-WP-0002's same-sample dedup blind spot — are in
`InnerLoopReference.md` §Why review is adversarial execution.*
T03: point review at the harness, and state what review cannot catch
The review step targeted the survey. Every serious error in this project
has been in measurement or build configuration, so the loop was
adversarially reviewing the artifact cheapest to fix and leaving
unreviewed the one where errors occur.
Step 2 now routes by risk: when the claim rests on numbers, the reviewer
gets the harness and the evidence file too, and must reproduce the
number independently rather than read about it.
The addition that matters more, because it was learned the hard way: a
reviewer re-derives the author's claims and therefore inherits the
author's SAMPLING. CB-WP-0002's dedup invariant was checked twice --
survey 206/206 groups, then the reviewer independently -- and both used
the main transcript. It is false in the 8-response subagent tree neither
looked at. Two independent verifications, one shared blind spot.
Rule: the reviewer re-derives on a different sample than the author
used, and where only one sample exists, says so rather than reporting
a clean verify.
Also recorded: what review demonstrably DOES do. $0.66 and $1.11 across
two passes, ~1% of each, both finding approval-blocking defects. Cost is
not a reason to skip it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:26:01 +02:00
**What review demonstrably does do.** Measured across two passes:
**$0.66** and ** $1.11**, roughly 1% of each pass, each finding
approval-blocking defects — in CB-WP-0002 a target that would have made
the evidence file certify a broken collector. The step pays for itself by
a wide margin and the cost is not a reason to skip it.
2026-07-31 00:48:04 +02:00
**Documentation requirement:** the research process, the challenge, and the
resulting improvements to the research are each documented in timestamped
markdown files under `history/` :
```text
history/YYMMDD-< slug > -research.md # how the survey was conducted: sources,
# queries, what was measured vs cited, dead ends
history/YYMMDD-< slug > -challenge.md # the adversarial attack, verbatim
history/YYMMDD-< slug > -response.md # answers/concessions and what changed in the survey
```
2026-07-31 00:31:14 +02:00
2026-07-31 00:48:04 +02:00
The polished survey artifact remains `research/CB-RES-NNNN-<slug>.md` ; the
history files preserve the unpolished trail so a later reader can judge how
hard the survey was actually tested. For tier-M passes the review is
optional but, when performed, follows the same format. If not approvable
after the round, the loop returns to step 1 with the named gaps.
2026-07-31 00:31:14 +02:00
### Step 3 — Decide
`decisions/ADR-NNNN-<slug>.md` : assimilate behind a port, reimplement, or
hybrid — with the **expected advantage stated per dimension** (see rubric).
An honest "worse here, better there, and why that trade is right" beats a
claimed sweep of all four dimensions.
### Step 4 — Specify
`specs/<Capability>.md` : the contracts, invariants, and — mandatory — the
**acceptance metrics table**, each row tied to a baseline from step 1.
A spec without measurable acceptance criteria is not done. Metrics follow
the conventions in [MetricsAndScenarios.md ](MetricsAndScenarios.md ),
including the rule that metric selection itself passes through a mini
research step (metric provenance).
2026-07-31 03:11:19 +02:00
**Every metric names its instrument, and is checked reachable.** A row
in the acceptance table carries the command that produces its number.
2026-07-31 08:53:52 +02:00
A metric with no named instrument is a wish, not a metric.
**The instrument must exist, and the target must come out of it.**
Naming a command is not the same as running one. A target computed by
hand and merely *labelled* with a command is the same defect the rule
was written to stop, one level down. Where the instrument is built later
in the pass, the target is marked `provisional:` until the instrument
emits it, and the spec is amended to whatever the instrument returns.
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
*Case: `InnerLoopReference.md` §Numbers you inherit.*
2026-07-31 08:53:52 +02:00
**A number inherited from earlier work is re-derived before it is used
as a target, or it is cited as unverified.** Quoting is not measuring.
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
*Case: `InnerLoopReference.md` §Numbers you inherit.*
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
2026-07-31 09:25:22 +02:00
**Retargeting: the instrument may move a target, the implementation may
not.** A metric's target changes in only two ways, and they are not
treated alike:
| | trigger | requirement |
|---|---|---|
| **corrected** | the instrument disproved the target — the target was computed by hand, or by an earlier tool with a defect | legitimate in the same commit, **provided the instrument's output is in that commit** |
| **retargeted** | the implementation missed the target and the target moves to accommodate it | requires an **ADR** : old target, the measurement, and why the new target binds on *future* work rather than merely passing present work |
The distinction is not the implementer's self-report of intent. It is
mechanical: **a correction is one where the target moves and the
implementation does not.** If the same commit changes both the target and
the code the target measures, it is a retarget and needs the ADR.
CB-WP-0019 T03/T04: the cost rule written down, and the lifecycle
T03: InnerLoop v1.7 plus loop-lint's own-cost check. Six passes
under-reported themselves by 30-45%, never once high, and the rule lived
only in evidence files having been re-derived three times. The READING
is load bearing, not the boundary: CB-WP-0018 T04 applied 're-run the
instrument at the moment of quoting' alone and its figure was correct.
So the operative instruction is re-run when you quote, and loop-lint
fails an evidence file naming its own workplan beside a dollar amount
without marking it provisional.
It binds forward from this pass. The check fires on seven historical
files which ARE the evidence for the rule; making them comply would edit
the record to remove the thing it proves -- the same category error as a
live fact: tag on a dated measurement, which this pass also hit.
Lifecycle, at the maintainer's instruction: ready -> active -> done,
where ready means declared and not started. loop-lint fails a workplan
that has started and still says ready, one that is active with
everything closed, and one that is done with an open task. The first
version of that check was WRONG and its own self-test caught it: it
stripped the leading status: assuming frontmatter, which silently
dropped a real task once the frontmatter said ready or active.
Both new checks then fired on this pass's own artifacts and both were
right to.
T04: CB-EV-0017. The new meta budget's first reading is a breach it
caused -- 27% against the 20% line, because this pass cost $31.18
against product passes averaging ~$21. Reported rather than exempted:
ADR-0006 D2 covers the instrument repairs but not the rule-writing, and
the honest reading is that this should have been two passes.
CB-WP-0018 settled at $36.53/95 against $28.08/82 last reported, 30%
higher. Seven for seven.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:25:18 +02:00
*The case that produced this rule, and the four legitimate corrections a
blanket prohibition would have forbidden: `InnerLoopReference.md`
§Retargeting.*
2026-07-31 09:25:22 +02:00
**Applied retroactively:** AM-4a and AM-4b are **unratified** until an ADR
is written or they are changed. They were set by the implementer after
seeing the measurement, in the commit that produced it, and the
implementation changed in that same commit — a retarget by the test above.
Tracked as an open item in `history/260731-inner-loop-rule-audit.md` .
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
**A metric is checked against the contracts in its own spec.** If a
contract makes a target unreachable, one of the two is wrong and the
conflict is resolved when it is noticed, not at the acceptance run.
Re-check the table whenever a contract is added.
2026-07-31 03:11:19 +02:00
*(v1.0, from CB-WP-0001: AM-4's ≤20-crate target was made unreachable by
the K5 and K7 contracts written after it, and AM-12's cost metric was
fully specified and never instrumented, so it could not be computed.)*
2026-07-31 00:31:14 +02:00
### Step 5 — Code loop
Implement iteratively. Each iteration:
```text
change → cb-check (fmt, clippy, tests) → scenarios → benchmarks
→ compare against acceptance table → evidence row appended
```
Done when every acceptance metric meets or beats its baseline and the
comparison numbers are committed as an evidence file
(`evidence/CB-EV-NNNN-<slug>.md` ). A failed scenario must yield a replay
artifact an agent can re-execute locally.
2026-07-31 03:11:19 +02:00
#### Measurement validity — the positive control
**Every benchmark and harness must assert that it performed the work it
reports.** Completing without error is not evidence of having done
anything: a loop whose commands are all rejected runs fast and reports a
throughput for work that never happened.
Concretely, a measurement harness must, on every run:
- assert the unit of work produced its expected effect (events applied,
rows written, moves accepted) — not merely that the call returned;
- fail loudly rather than report a number when that assertion fails;
- state the divisor used to convert raw timings into the metric's unit,
pinned by a test so a workload change cannot silently rescale it.
**A number from a run that cannot prove it did the work is void** and
must not reach an evidence file.
2026-07-31 08:52:33 +02:00
**Every tool that reports a number exposes `--self-test` **, and that
self-test runs before the number is produced (`make cost` depends on
`make cost-test` ). The assertion must name a failure it detects, not
merely exercise the happy path.
*(v1.0+, from CB-WP-0002: `cb-cost` 's dedup assertion fired on its first
run against real data and aborted, catching a rule that was verified on
206/206 groups of the main transcript and false in the 8-response
subagent tree. The generalization that failed — a property confirmed on
the largest sample assumed to hold on the smallest — is not one review
catches, because both the survey and the adversarial reviewer checked
the same large sample.)*
2026-07-31 03:11:19 +02:00
*(v1.0, from CB-WP-0001: both serious errors in the first pass were of
exactly this shape. A JS harness reported 8.4s for 100k moves while
every move was being rejected, and a Rust benchmark reported 9.3M
events/s — a 93×
stress gate rejected one player's action. The corrected figure was 5.6×
lower. Adversarial review caught neither; both were claims about
numbers, and review reads prose.)*
#### Evidence states what it does not support
An evidence file that compares across runtimes, languages, or feature
sets **names the disanalogies explicitly** , in the same section as the
number. The reader must not have to infer that a ratio is not
like-for-like. This is the parity-cap rule applied to the write-up:
state the claim you will defend, and the claim you are not making.
2026-07-31 00:31:14 +02:00
---
---
2026-07-31 09:25:22 +02:00
## Reference material
2026-07-31 03:11:19 +02:00
2026-07-31 09:25:22 +02:00
The four-dimension rubric, the survey template, the implementation rules
each pass earned, the agentic-efficiency requirements, and the
definition of done live in
**[InnerLoopReference.md ](InnerLoopReference.md )**. They are normative;
they are separated only so both files load whole.
2026-07-31 00:31:14 +02:00
2026-07-31 09:25:22 +02:00
*(Split 2026-07-31: this file reached 407 lines against its own ~400-line
loadability rule, and `make loop-lint` — added the same day to make that
rule executable — failed on the commit that pushed it over. The rule
caught its own author within an hour of being written.)*
