D2 the two exemptions — repairing the instrument that reports a breach is always in budget, and the maintainer may authorize above-line work in the workplan frontmatter. v1.5 had neither, and the first thing it did was forbid fixing the budget's own defect. D4 loop work cashes out into something runnable in the pass that invents it, with the measured basis and the refutation condition stated. D5 the tier table now triggers at M for changes to the loop's own operation, closing the gap CB-RES-0005 §6 opened and this pass's own ADR had to declare around. The file went from 399 lines to 386: the chaos-roll calibration rationale and the v1.0-v1.3 change log moved to InnerLoopReference.md. Loadability is a gate too, and growing the file to fit a rule about gates would have been the joke writing itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
197 lines
10 KiB
Markdown
197 lines
10 KiB
Markdown
# The Inner Loop — Reference
|
||
|
||
Normative companion to **[InnerLoop.md](InnerLoop.md)**, which holds the
|
||
five steps, the tiers, and the chaos roll. Split from it on 2026-07-31 so
|
||
both files stay under the loop's own whole-file loadability limit.
|
||
|
||
Everything here binds exactly as the process document does.
|
||
|
||
## The four-dimension rubric
|
||
|
||
Every survey, ADR, and acceptance table is organized by these dimensions:
|
||
|
||
| Dimension | Question | Example measurable proxies |
|
||
|---|---|---|
|
||
| **D1 Ease of specification** | How simply can behavior be stated, tested, understood? | rules-to-scenario coverage %, spec lines per rule, time-to-first-correct-scenario for a fresh agent session |
|
||
| **D2 Efficiency of implementation** | How cheap to build and keep building? | source LOC, dependency count/weight, clean-build and incremental-build time, tokens-per-completed-task |
|
||
| **D3 Speed of execution** | How fast does it run? | benchmark wall-time vs baseline, events/sec, memory footprint, determinism overhead |
|
||
| **D4 Optionality** | How cleanly does it integrate, extend, get replaced? | public API surface size, count of leaked foreign types (must be 0), effort-to-swap measured by null/reference impl existence, WIT-expressibility |
|
||
|
||
Scoring is always **relative to the step-1 baseline**, never absolute:
|
||
`better / parity / worse / unmeasured` per proxy, with the number attached.
|
||
`unmeasured` is legal in a survey, illegal in an evidence file.
|
||
|
||
---
|
||
|
||
## Survey template (research/CB-RES-NNNN-<slug>.md)
|
||
|
||
```markdown
|
||
# CB-RES-NNNN: <capability>
|
||
capability: <canonical.capability.id>
|
||
status: draft | approved
|
||
|
||
## Candidates
|
||
Per candidate: origin, license, maturity, adoption; data model; mutation
|
||
mechanism; determinism/replay story; relevant performance (measured if
|
||
runnable locally, cited with source otherwise).
|
||
|
||
## Baselines (benchmark-to-beat)
|
||
| Dimension | Baseline holder | Metric | Value | Provenance |
|
||
(one row minimum per dimension; provenance = measured / cited / estimated)
|
||
|
||
## Verdict
|
||
Which candidate leads per dimension; what none of them do well
|
||
(the surpass opportunity); risks in the baselines themselves.
|
||
```
|
||
|
||
---
|
||
|
||
## Implementation rules the first pass earned
|
||
|
||
These are cheap, and each exists because its absence cost something in
|
||
CB-WP-0001. See `history/260731-inner-loop-retrospective.md`.
|
||
|
||
1. **No silently-ignored input.** A field that is parsed and then unused
|
||
is a defect, not a stub. Inputs are honoured or rejected with an
|
||
error — never dropped. *(A scenario `setup.patch` was parsed and
|
||
discarded; every scenario using it would have tested the wrong
|
||
initial state while passing.)*
|
||
2. **Decisions get commands, not defaults.** A rule that requires a
|
||
participant's choice is implemented as a command carrying that
|
||
choice. Until it is, **nothing claims coverage of it** — no tag, no
|
||
scenario, no acceptance row. Inventing a default to make a rule
|
||
"done" is the failure this prevents.
|
||
3. **Scaffolds are exercised or marked.** A scaffold's green gates are
|
||
not evidence. Any scaffold path no test reaches is marked as
|
||
unexercised. *(A compiling, fully-green scaffold shipped a state-hash
|
||
that would panic on any state holding a relation.)*
|
||
4. **Coverage gates that count tags say so.** A gate comparing rule IDs
|
||
against `covers:` lists proves no rule is unclaimed and no claimed
|
||
rule is invented. It does **not** prove a scenario exercises what it
|
||
names. Wherever such a number is reported, that limit is reported
|
||
with it.
|
||
|
||
---
|
||
|
||
## Agentic-efficiency requirements
|
||
|
||
The loop exists to be driven by agents. Therefore:
|
||
|
||
1. **Whole-file loadability** — every loop artifact stays under ~400 lines;
|
||
split before exceeding, link with relative paths.
|
||
2. **Structured over prose** *(guidance, not a requirement — nothing can
|
||
check it)* — tables and fenced blocks for anything a later step must
|
||
parse (baselines, acceptance metrics, evidence rows).
|
||
3. **One command surface** — all checks runnable through repo-root
|
||
commands (eventually `cb *`; until then, `make`/`cargo` aliases declared
|
||
in one place), each supporting deterministic, greppable output.
|
||
4. **Self-contained tasks** — a workplan task names its input artifacts and
|
||
output artifacts; a fresh session must be able to execute it from the
|
||
task text plus linked files alone.
|
||
5. **Evidence or it didn't happen** — claims of "better" live in committed
|
||
evidence files with numbers, never only in commit messages or chat.
|
||
6. ~~**Token discipline** — per the global budget policy, a loop iteration
|
||
that exceeds its budget without measurable progress is stopped and
|
||
decomposed, not pushed through.~~ **DEAD POLICY.** The 8k/10k per-task
|
||
token budget was never referenced or enforced, and CB-WP-0001 T08
|
||
exceeded it by orders of magnitude with no signal. It implies a control
|
||
that does not exist. Replacement in USD is CB-WP-0003 T05; until then
|
||
this is documentation of a gap, not a rule.
|
||
|
||
6a. **Live cost budget** *(replaces the above)* — spend since the last
|
||
commit, soft **$10.00**, hard **$22.00**, checked by `make cost-budget`.
|
||
Calibrated on the 32 commit intervals of CB-WP-0001 (p50 $1.40, p90
|
||
$9.36, max $10.80), so both thresholds bind on future work rather than
|
||
ratifying past work. Contract: [CostAccounting.md](CostAccounting.md)
|
||
§7 (CB-01, CB-02). It fires on the open remainder rather than per task
|
||
because per-task cost needs the commit that closes the task, and a
|
||
budget that can only report after the money is spent is the dead policy
|
||
this replaces.
|
||
|
||
**Enforcement status.** Rules above that a command can check are enforced
|
||
by `make loop-lint`; the full classification of every InnerLoop rule as
|
||
executable / checkable / decorative, with the failure class each catches,
|
||
is in `history/260731-inner-loop-rule-audit.md`. Rules marked *guidance*
|
||
or *dead* say so where they appear, so a reader can tell a requirement
|
||
from a preference without consulting the audit.
|
||
|
||
---
|
||
|
||
## Definition of done — one loop pass
|
||
|
||
A capability has completed the loop when all of the following are committed:
|
||
|
||
- [ ] research/CB-RES-NNNN with approved status and full baseline table
|
||
- [ ] decisions/ADR-NNNN with per-dimension expected advantage
|
||
- [ ] specs/<Capability>.md with acceptance-metrics table
|
||
- [ ] passing scenarios covering every numbered spec rule
|
||
- [ ] evidence/CB-EV-NNNN with final comparison vs baseline, no `unmeasured`
|
||
- [ ] every reported number produced by a harness with a positive
|
||
control; any metric that could not be instrumented is recorded as
|
||
uncomputable rather than estimated
|
||
- [ ] every unmet metric reported as unmet, with attribution and the
|
||
options for resolving it — a missed target is an output of the
|
||
loop, not a reason to move the target quietly
|
||
- [ ] **cost recorded**: `make cost` run for the pass, its composition
|
||
(not only its total) in the evidence file, and the per-task figures
|
||
pushed to the hub. M-D2-CST is no longer allowed to be
|
||
`uncomputable` — the instrument exists
|
||
([CostAccounting.md](CostAccounting.md))
|
||
- [ ] retrospective note (may be one paragraph appended to the evidence
|
||
file): what the loop itself should change
|
||
|
||
## Chaos roll — calibration (moved from InnerLoop v1.6)
|
||
|
||
> **Calibration window, opened 2026-07-31 (CB-WP-0003 T06).** The rate was
|
||
> d10 and the mechanism **never fired**: two rolls across two workplans
|
||
> (CB-WP-0001: 9, CB-WP-0002: 2), against ~0.2 expected firings. At d10 and
|
||
> ~2 tier decisions per workplan it would take roughly twenty workplans to
|
||
> observe four overrides, so the mechanism was set at a rate that prevented
|
||
> its own evaluation — the one option T06 ruled out.
|
||
>
|
||
> Raised to **d4 (25%) for the next 12 tier declarations**, then evaluated
|
||
> and either kept, returned to d10, or deleted. Expected ~3 firings in the
|
||
> window, which is enough to see whether an overridden tier produces a
|
||
> different outcome than the argued one.
|
||
>
|
||
> **Stated cost:** a chaos-L override on work that would have been S buys a
|
||
> full survey, adversarial review, and ADR. Measured comparable: CB-WP-0001
|
||
> T03 (a tier-L survey) cost **$9.91**. At 25% over 12 declarations the
|
||
> window is expected to cost **$20–30**. That is the price of finding out
|
||
> whether the mechanism is worth keeping, and it is cheaper than carrying an
|
||
> unevaluated ritual indefinitely. Both rolls are recorded in the tier declaration
|
||
(`tier: M (structural L, chaos 10→M)`). **Record the roll every time,
|
||
including when it changes nothing** (`tier: L (structural L, chaos 4)`),
|
||
so a mechanism that never fires is visible rather than assumed. Purpose:
|
||
an occasional random
|
||
reweighting keeps the classification honest — arguing everything into S
|
||
stops paying off when audits can compare argued tiers against the random
|
||
sample — and occasionally forces a deep look at something "obviously
|
||
trivial", which is where local optima hide.
|
||
|
||
|
||
## Change log — v1.0 to v1.3 (moved from InnerLoop v1.6)
|
||
|
||
v1.3 changed from v1.2: single source of fact is now executable
|
||
(`make facts-check`, CB-WP-0004 T04), giving the duplicated-fact-drift
|
||
class its first gate.
|
||
|
||
v1.2 changes from v1.1: single source of fact; review targets the
|
||
harness and states its sampling limit; correction vs retarget; the chaos
|
||
roll's calibration window; the live cost budget. The design goal is now
|
||
stated: **optimize for cheap correction, not for exhaustive prevention.**
|
||
Rationale: `history/260731-loop-hardening-retrospective.md`.
|
||
|
||
v1.1 — corrected from CB-WP-0002 (cost accounting) on
|
||
2026-07-31. Changes from v1.0: the instrument must exist and emit its own
|
||
target; inherited numbers are re-derived before use; every reporting tool
|
||
exposes `--self-test`; cost is in the definition of done. Rationale:
|
||
`history/260731-cost-accounting-retrospective.md`.
|
||
|
||
v1.0 — survived its first full pass (CB-WP-0001, the GROUND game kernel)
|
||
and was corrected from it on 2026-07-31. Changes from v0.2: measurement
|
||
validity (the positive control), metric feasibility and instrument naming,
|
||
four implementation rules the pass earned, and the requirement that
|
||
evidence state what it does not support. Rationale and the failures behind
|
||
each: `history/260731-inner-loop-retrospective.md`.
|
||
|