tegwick
05d1a9aada
Some checks failed
ci / check (push) Failing after 4s
The condition is met and the tally was verified rather than recalled. ADR-0017 D2 named window 3 as the decider; windows 2 and 3 each produced exactly one override and each changed nothing. Every roll in window 3 was cross-checked against the workplan that made it, because the last time this project tallied chaos rolls from memory it was wrong and asserted the wrong figure four times (F23). All twelve agree. But meeting the condition is not evidence. P(an override changes nothing) is 1/3, and each window had exactly one override, so the condition fires on a 1/9 coincidence. ADR-0017 restated it to be REACHABLE and made it weak in the process; reachability was checked and discriminating power was not. Worse, it measures the wrong subject. It asks whether an override changed the tier; the question is whether changing the tier helped. Under it, a die that always changed the tier could never be retired however useless its changes were. The real ground is stronger. Four overrides across roughly forty declarations, and the mechanism's value has never once been demonstrated. The one substantive intervention dropped CB-WP-0011 from a structural L to S, and that work then needed CB-WP-0016 and CB-WP-0017 to fix defects a human found by playing. Not offered as causation — a tier is process weight, not a guarantee — but it is the only evidence we have about an override's consequences and it points the wrong way. And the purpose has no live evidence of need: 17 M, 11 S, 4 L across every workplan, with the only two structural/declared mismatches being the window-1 overrides themselves. Tier declaration has not ossified. So: retired, with nothing replacing it. Adding a successor to guard against ossification that has not occurred would invent a gate for an instance we do not have. The revival trigger is stated: tiers collapsing toward one value, or a pass declaring below its structural tier to dodge a review. InnerLoop loses the chaos paragraph, loop-lint loses the chaos-recorded check — a check that outlives its rule becomes an obstruction — and its self-test now asserts the opposite: a note with no roll must pass. ChaosRollHistory is closed. Window 4 ends incomplete at four declarations, the last of which is this one, rolled because the rule was still in force. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
387 lines
19 KiB
Markdown
387 lines
19 KiB
Markdown
# The Inner Loop — Assimilate and Surpass
|
||
|
||
Status: **v1.6** — corrected from CB-WP-0008 (stage 0) and ADR-0006 on
|
||
2026-08-01. Changes from v1.5: budgets measure the window they govern and
|
||
a breach never blocks its own repair (D1/D2); loop work cashes out into
|
||
something runnable in the pass that invents it (D4); the tier table
|
||
covers changes to the loop's own operation (D5).
|
||
|
||
> **What the budget is for (v1.7, CB-WP-0019 T05).** Most of what this
|
||
> project spends should go on **the task at hand**. Some of it must go on
|
||
> **control, review, and improving the process that carries the work
|
||
> forward** — a loop that never inspects itself accumulates exactly the
|
||
> defects this one keeps finding, and a loop that only inspects itself
|
||
> ships nothing. The budget names the split so that neither happens by
|
||
> drift.
|
||
>
|
||
> **Meta budget — soft, 20% of the trailing five passes.** Measured by
|
||
> `make status` from each workplan's `kind:` frontmatter (`product` |
|
||
> `meta` | `mixed`), over a **trailing 5-pass window**; the lifetime
|
||
> figure is reported as history and is not the metric (ADR-0006 D1).
|
||
>
|
||
> **The ratio and the window are a pair** — 20% over five is *one pass in
|
||
> five at normal cost*; the same number over three would mean *one in five
|
||
> and at half price*. `InnerLoopReference.md` §Meta budget has the
|
||
> arithmetic.
|
||
>
|
||
> **Soft on purpose.** A task already started may be finished; what the
|
||
> budget constrains is *opening* new meta work above the line, with two
|
||
> exemptions:
|
||
>
|
||
> 1. **Repairing the instrument that reports a breach is always in
|
||
> budget.** A gate must never block its own remedy (ADR-0006 D2,
|
||
> CB-RES-0005 §4). v1.5 had no such exemption, and the first thing it
|
||
> did was forbid fixing the budget's own defect.
|
||
> 2. **The maintainer may authorize above-line work**, recorded as
|
||
> `authorized_above_budget:` in the workplan frontmatter. Undocumented
|
||
> above-line meta work is a violation the evidence file must report.
|
||
>
|
||
> **The phase setting is not a dial.** The split may move for a phase, but
|
||
> only **declared, argued, and expiring** — `meta_phase = { pct, reason,
|
||
> review_by }` in `gates.toml`, reverting to 20 on `review_by` unless
|
||
> re-argued. A setting with no reason or no expiry fails
|
||
> `status.py --self-test`, so the line cannot be raised quietly to pass a
|
||
> breach.
|
||
>
|
||
> A pass that exceeds it **says so in its evidence file and names the
|
||
> product work displaced**. That is the whole enforcement: a reporting
|
||
> budget, not a gate, for the same reason the session-shape budget is —
|
||
> it constrains judgment, not artifacts.
|
||
|
||
> **Workplan lifecycle (v1.7).** `ready` → `active` → `done`. **`ready`
|
||
> means declared and not yet started**; a workplan being worked is
|
||
> `active`, so the status answers *"is anyone on this?"* rather than only
|
||
> *"is it finished?"*. `loop-lint` fails when a workplan has started work
|
||
> and still says `ready`, or has finished it and does not say `done`.
|
||
> Tasks keep `todo` | `done` | `cancel`; `cancel` is a **closed** outcome
|
||
> and is counted as such.
|
||
|
||
> **Quoting a cost (v1.7).** An evidence file **may not quote its own
|
||
> pass's cost as final** — six passes under-reported themselves by 30–45%,
|
||
> never once high. **Re-run the instrument at the moment of quoting**;
|
||
> quoting from memory defeats the rule even when the boundary is right.
|
||
> `loop-lint` fails an evidence file naming its own workplan beside a
|
||
> dollar figure without marking it provisional.
|
||
> `InnerLoopReference.md` §Quoting a cost.
|
||
|
||
> **Loop work cashes out (v1.6, ADR-0006 D4).** A `meta` pass must ship at
|
||
> least one executable check, command, or gate. Findings that stay prose
|
||
> are recorded as findings and do **not** count as delivery.
|
||
>
|
||
> Measured basis and its falsifier: `InnerLoopReference.md` §Loop work
|
||
> cashes out — **cost tracks distance from a runnable check**.
|
||
|
||
> **Every gate is an experiment (v1.6, ADR-0006 D3).** Each standing
|
||
> control gate has an entry in `gates.toml` naming what it has caught, a
|
||
> `review_by` date, and what would retire it. `make gate-review` reports
|
||
> what is overdue and what has caught nothing; `loop-lint` fails when the
|
||
> registry drifts from the Makefile in either direction.
|
||
|
||
v1.4 — corrected from CB-WP-0005 (assertion coverage) on
|
||
2026-07-31. Change from v1.3: where a claim rests on numbers, the
|
||
adversarial reviewer must read the assertion behind each quoted number and
|
||
**mutate it** — re-running the command that prints a number is not
|
||
verification of that number (§Step 2).
|
||
|
||
v1.5 (meta budget), v1.3 (executable single source of fact), v1.2
|
||
(design goal: cheap correction over exhaustive prevention), v1.1
|
||
(instruments emit their own targets) and v1.0: see
|
||
`specs/InnerLoopReference.md` §Change log.
|
||
|
||
Normative process for building every Clay-Borg capability. Referenced by
|
||
all workplans.
|
||
|
||
**Design goal (v1.2).** Three passes produced ten error instances across
|
||
four classes, and every pass produced a class the previous one had not
|
||
seen — prevention is not converging. Every one of those errors was
|
||
corrected inside the same session for under ~1% of the pass. The loop
|
||
therefore optimizes for **cheap correction**: keep the raw data so numbers
|
||
can be re-derived, keep artifacts small and committed so a wrong number is
|
||
one grep from everywhere that quotes it, and give every reported number a
|
||
command so re-running is free.
|
||
|
||
> **Single source of fact.** A number, rate, or target lives in exactly
|
||
> one place; everywhere else links to it. Where a copy is unavoidable it
|
||
> is generated by a command, not typed. *(v1.2: a price sheet inlined into
|
||
> a spec went stale within the hour of the real sheet changing, and one
|
||
> acceptance figure had to be chased across three artifacts each time it
|
||
> moved. No positive control catches this — both copies are internally
|
||
> consistent — and re-derivation does not either, because the copy
|
||
> reproduces whatever it was copied from.)*
|
||
>
|
||
> **Now executable (v1.3, CB-WP-0004 T04):** `make facts-check`. `facts.toml`
|
||
> is generated from the instruments, never edited; an artifact quoting a
|
||
> registry value tags it `<!-- fact:<key> -->` and the gate fails when the
|
||
> two disagree. Untagged literal copies are reported, not failed — that is
|
||
> the drift surface still uncovered, and naming it is more useful than
|
||
> pretending it is closed.
|
||
|
||
The loop's own optimization target is **agentic efficiency**:
|
||
every artifact it produces must be small enough to load whole, structured
|
||
enough to act on without interpretation, and falsifiable enough that an
|
||
agent can judge its own work without a human in the iteration.
|
||
|
||
---
|
||
|
||
## The five steps
|
||
|
||
```text
|
||
1 RESEARCH → research/CB-RES-NNNN-<slug>.md (survey, baselines)
|
||
2 APPROVE → decision recorded in the ADR (gate: survey complete?)
|
||
3 DECIDE → decisions/ADR-NNNN-<slug>.md (assimilate/reimplement/hybrid)
|
||
4 SPECIFY → specs/<Capability>.md (contracts + acceptance metrics)
|
||
5 CODE LOOP → code + scenarios + benchmarks (iterate until metrics beat baseline)
|
||
```
|
||
|
||
**Hard gate: no implementation code for a capability exists before its ADR
|
||
(step 3) is committed.**
|
||
|
||
## Loop tiers and the chaos roll
|
||
|
||
Every work packet declares a tier before work starts. The tier sets how
|
||
heavy steps 1–3 are; steps 4–5 (spec with metrics, code loop with evidence)
|
||
are never skipped for code-producing work.
|
||
|
||
| Tier | Weight of steps 1–3 | Structural trigger (forces at least this tier) |
|
||
|---|---|---|
|
||
| **L** | Full: separate survey, adversarial review, ADR | Creates a new capability port, or is named a high-leverage pass by the maintainer |
|
||
| **M** | Survey and ADR merged into one document; review optional | Touches a canonical interface, adds/updates an external dependency, **or changes whether or how the loop constrains its own operation** — budgets, gates, review requirements, or these tier rules (v1.6, ADR-0006 D5) |
|
||
| **S** | One provenance paragraph in the commit message | Everything else (utilities, fixes, refactors inside a boundary) |
|
||
|
||
**The chaos roll is retired** ([ADR-0021](../decisions/ADR-0021-the-chaos-roll-is-retired.md),
|
||
2026-08-08). The tier is the structural derivation above, and nothing
|
||
overrides it.
|
||
|
||
> It ran for four windows and acted four times. Its retirement condition
|
||
> was met — and **the condition was wrong**: it fires on a 1/9 coincidence
|
||
> and it measured whether an override *changed the tier*, not whether the
|
||
> change *helped*. The real ground was that **no override ever
|
||
> demonstrated a benefit**, and the one substantive intervention dropped a
|
||
> structural L to S on a pass that then needed two follow-ups.
|
||
>
|
||
> The log is closed at [`ChaosRollHistory.md`](ChaosRollHistory.md).
|
||
> **The trigger for reviving it** is ossification: declared tiers
|
||
> collapsing toward one value, or a pass declaring below its structural
|
||
> tier to dodge a review.
|
||
|
||
### Step 1 — Research
|
||
|
||
Identify the best implementation in existence for this capability. Produce
|
||
`research/CB-RES-NNNN-<slug>.md` following the survey template (below).
|
||
The survey is done when it can name, per dimension, a concrete
|
||
**benchmark-to-beat**: a number, a property, or a reproducible comparison —
|
||
not an impression.
|
||
|
||
**Runnable-baseline option.** For passes judged high-leverage (declared by
|
||
the maintainer or proposed in the survey and confirmed in the ADR), cited
|
||
numbers are not enough: the survey must ship a reproducible **baseline
|
||
harness** that runs the leading candidate on our machine against our
|
||
workload — the same scenario files where feasible. The harness ships with a
|
||
*fidelity note* stating what was and wasn't faithfully reproduced, so a
|
||
hastily wired competitor setup cannot silently inflate our advantage.
|
||
Where the option is not invoked (or the candidate isn't practically
|
||
runnable), comparisons against cited-only numbers are **directional**: the
|
||
evidence verdict for those rows caps at `parity`, never `better`.
|
||
|
||
### Step 2 — Approve (adversarial review)
|
||
|
||
For tier-L passes, approval is earned through an **adversarial review**: a
|
||
separate session (or agent) attempts to break the work. Exactly **one
|
||
round**: challenge, then response. The work is approvable only when every
|
||
challenge is either answered with evidence or conceded and folded in.
|
||
|
||
**The review target follows the risk.** Reviewing the survey was the
|
||
original rule, and it is the wrong target for a capability whose claim
|
||
rests on numbers — every serious error in this project has been in
|
||
measurement or build configuration, not in prose.
|
||
|
||
| the claim rests on | the reviewer is given | and must attempt |
|
||
|---|---|---|
|
||
| a survey of candidates | the survey | an omitted candidate; a stale or unverifiable benchmark; an unmeasured claim presented as measured |
|
||
| **numbers** | the survey **and the harness and the evidence file** | **reproduce the number independently**; state what the harness would report if the work silently stopped |
|
||
|
||
**What review cannot do — state this, do not discover it.** A reviewer
|
||
re-derives the author's claims and therefore inherits the author's
|
||
sampling. So:
|
||
|
||
> **The reviewer re-derives on a different sample than the author used.**
|
||
> Where only one sample exists, the review says so rather than reporting a
|
||
> clean verify.
|
||
>
|
||
> **And re-derivation is not enough (v1.4).** Where the claim rests on
|
||
> numbers, the reviewer must **read the assertion behind each quoted
|
||
> number and mutate it**: invert the property and require the suite to go
|
||
> red. Re-running the command that prints a number satisfies "reproduce
|
||
> independently" and finds nothing of this class.
|
||
>
|
||
*Both cases that produced these rules — CB-WP-0005's inert `AM-7 replay`
|
||
clause and CB-WP-0002's same-sample dedup blind spot — are in
|
||
`InnerLoopReference.md` §Why review is adversarial execution.*
|
||
|
||
**What review demonstrably does do.** Measured across two passes:
|
||
**$0.66** and **$1.11**, roughly 1% of each pass, each finding
|
||
approval-blocking defects — in CB-WP-0002 a target that would have made
|
||
the evidence file certify a broken collector. The step pays for itself by
|
||
a wide margin and the cost is not a reason to skip it.
|
||
|
||
**Documentation requirement:** the research process, the challenge, and the
|
||
resulting improvements to the research are each documented in timestamped
|
||
markdown files under `history/`:
|
||
|
||
```text
|
||
history/YYMMDD-<slug>-research.md # how the survey was conducted: sources,
|
||
# queries, what was measured vs cited, dead ends
|
||
history/YYMMDD-<slug>-challenge.md # the adversarial attack, verbatim
|
||
history/YYMMDD-<slug>-response.md # answers/concessions and what changed in the survey
|
||
```
|
||
|
||
The polished survey artifact remains `research/CB-RES-NNNN-<slug>.md`; the
|
||
history files preserve the unpolished trail so a later reader can judge how
|
||
hard the survey was actually tested. For tier-M passes the review is
|
||
optional but, when performed, follows the same format. If not approvable
|
||
after the round, the loop returns to step 1 with the named gaps.
|
||
|
||
### Step 3 — Decide
|
||
|
||
`decisions/ADR-NNNN-<slug>.md`: assimilate behind a port, reimplement, or
|
||
hybrid — with the **expected advantage stated per dimension** (see rubric).
|
||
An honest "worse here, better there, and why that trade is right" beats a
|
||
claimed sweep of all four dimensions.
|
||
|
||
### Step 4 — Specify
|
||
|
||
`specs/<Capability>.md`: the contracts, invariants, and — mandatory — the
|
||
**acceptance metrics table**, each row tied to a baseline from step 1.
|
||
A spec without measurable acceptance criteria is not done. Metrics follow
|
||
the conventions in [MetricsAndScenarios.md](MetricsAndScenarios.md),
|
||
including the rule that metric selection itself passes through a mini
|
||
research step (metric provenance).
|
||
|
||
**Every metric names its instrument, and is checked reachable.** A row
|
||
in the acceptance table carries the command that produces its number.
|
||
A metric with no named instrument is a wish, not a metric.
|
||
|
||
**The instrument must exist, and the target must come out of it.**
|
||
Naming a command is not the same as running one. A target computed by
|
||
hand and merely *labelled* with a command is the same defect the rule
|
||
was written to stop, one level down. Where the instrument is built later
|
||
in the pass, the target is marked `provisional:` until the instrument
|
||
emits it, and the spec is amended to whatever the instrument returns.
|
||
|
||
*Case: `InnerLoopReference.md` §Numbers you inherit.*
|
||
|
||
**A number inherited from earlier work is re-derived before it is used
|
||
as a target, or it is cited as unverified.** Quoting is not measuring.
|
||
|
||
*Case: `InnerLoopReference.md` §Numbers you inherit.*
|
||
|
||
**Retargeting: the instrument may move a target, the implementation may
|
||
not.** A metric's target changes in only two ways, and they are not
|
||
treated alike:
|
||
|
||
| | trigger | requirement |
|
||
|---|---|---|
|
||
| **corrected** | the instrument disproved the target — the target was computed by hand, or by an earlier tool with a defect | legitimate in the same commit, **provided the instrument's output is in that commit** |
|
||
| **retargeted** | the implementation missed the target and the target moves to accommodate it | requires an **ADR**: old target, the measurement, and why the new target binds on *future* work rather than merely passing present work |
|
||
|
||
The distinction is not the implementer's self-report of intent. It is
|
||
mechanical: **a correction is one where the target moves and the
|
||
implementation does not.** If the same commit changes both the target and
|
||
the code the target measures, it is a retarget and needs the ADR.
|
||
|
||
*The case that produced this rule, and the four legitimate corrections a
|
||
blanket prohibition would have forbidden: `InnerLoopReference.md`
|
||
§Retargeting.*
|
||
|
||
**Applied retroactively:** AM-4a and AM-4b are **unratified** until an ADR
|
||
is written or they are changed. They were set by the implementer after
|
||
seeing the measurement, in the commit that produced it, and the
|
||
implementation changed in that same commit — a retarget by the test above.
|
||
Tracked as an open item in `history/260731-inner-loop-rule-audit.md`.
|
||
|
||
**A metric is checked against the contracts in its own spec.** If a
|
||
contract makes a target unreachable, one of the two is wrong and the
|
||
conflict is resolved when it is noticed, not at the acceptance run.
|
||
Re-check the table whenever a contract is added.
|
||
|
||
*(v1.0, from CB-WP-0001: AM-4's ≤20-crate target was made unreachable by
|
||
the K5 and K7 contracts written after it, and AM-12's cost metric was
|
||
fully specified and never instrumented, so it could not be computed.)*
|
||
|
||
### Step 5 — Code loop
|
||
|
||
Implement iteratively. Each iteration:
|
||
|
||
```text
|
||
change → cb-check (fmt, clippy, tests) → scenarios → benchmarks
|
||
→ compare against acceptance table → evidence row appended
|
||
```
|
||
|
||
Done when every acceptance metric meets or beats its baseline and the
|
||
comparison numbers are committed as an evidence file
|
||
(`evidence/CB-EV-NNNN-<slug>.md`). A failed scenario must yield a replay
|
||
artifact an agent can re-execute locally.
|
||
|
||
#### Measurement validity — the positive control
|
||
|
||
**Every benchmark and harness must assert that it performed the work it
|
||
reports.** Completing without error is not evidence of having done
|
||
anything: a loop whose commands are all rejected runs fast and reports a
|
||
throughput for work that never happened.
|
||
|
||
Concretely, a measurement harness must, on every run:
|
||
|
||
- assert the unit of work produced its expected effect (events applied,
|
||
rows written, moves accepted) — not merely that the call returned;
|
||
- fail loudly rather than report a number when that assertion fails;
|
||
- state the divisor used to convert raw timings into the metric's unit,
|
||
pinned by a test so a workload change cannot silently rescale it.
|
||
|
||
**A number from a run that cannot prove it did the work is void** and
|
||
must not reach an evidence file.
|
||
|
||
**Every tool that reports a number exposes `--self-test`**, and that
|
||
self-test runs before the number is produced (`make cost` depends on
|
||
`make cost-test`). The assertion must name a failure it detects, not
|
||
merely exercise the happy path.
|
||
|
||
*(v1.0+, from CB-WP-0002: `cb-cost`'s dedup assertion fired on its first
|
||
run against real data and aborted, catching a rule that was verified on
|
||
206/206 groups of the main transcript and false in the 8-response
|
||
subagent tree. The generalization that failed — a property confirmed on
|
||
the largest sample assumed to hold on the smallest — is not one review
|
||
catches, because both the survey and the adversarial reviewer checked
|
||
the same large sample.)*
|
||
|
||
*(v1.0, from CB-WP-0001: both serious errors in the first pass were of
|
||
exactly this shape. A JS harness reported 8.4s for 100k moves while
|
||
every move was being rejected, and a Rust benchmark reported 9.3M
|
||
events/s — a 93× beat — while most rounds never completed because a
|
||
stress gate rejected one player's action. The corrected figure was 5.6×
|
||
lower. Adversarial review caught neither; both were claims about
|
||
numbers, and review reads prose.)*
|
||
|
||
#### Evidence states what it does not support
|
||
|
||
An evidence file that compares across runtimes, languages, or feature
|
||
sets **names the disanalogies explicitly**, in the same section as the
|
||
number. The reader must not have to infer that a ratio is not
|
||
like-for-like. This is the parity-cap rule applied to the write-up:
|
||
state the claim you will defend, and the claim you are not making.
|
||
|
||
---
|
||
|
||
---
|
||
|
||
## Reference material
|
||
|
||
The four-dimension rubric, the survey template, the implementation rules
|
||
each pass earned, the agentic-efficiency requirements, and the
|
||
definition of done live in
|
||
**[InnerLoopReference.md](InnerLoopReference.md)**. They are normative;
|
||
they are separated only so both files load whole.
|
||
|
||
*(Split 2026-07-31: this file reached 407 lines against its own ~400-line
|
||
loadability rule, and `make loop-lint` — added the same day to make that
|
||
rule executable — failed on the commit that pushed it over. The rule
|
||
caught its own author within an hour of being written.)*
|