clay-borg/specs/InnerLoop.md

The Inner Loop — Assimilate and Surpass

Status: v1.1 — corrected from CB-WP-0002 (cost accounting) on 2026-07-31. Changes from v1.0: the instrument must exist and emit its own target; inherited numbers are re-derived before use; every reporting tool exposes --self-test; cost is in the definition of done. Rationale: history/260731-cost-accounting-retrospective.md.

v1.0 — survived its first full pass (CB-WP-0001, the GROUND game kernel) and was corrected from it on 2026-07-31. Changes from v0.2: measurement validity (the positive control), metric feasibility and instrument naming, four implementation rules the pass earned, and the requirement that evidence state what it does not support. Rationale and the failures behind each: history/260731-inner-loop-retrospective.md.

Normative process for building every Clay-Borg capability. Referenced by all workplans. The loop's own optimization target is agentic efficiency: every artifact it produces must be small enough to load whole, structured enough to act on without interpretation, and falsifiable enough that an agent can judge its own work without a human in the iteration.

---

The five steps

1 RESEARCH  → research/CB-RES-NNNN-<slug>.md     (survey, baselines)
2 APPROVE   → decision recorded in the ADR        (gate: survey complete?)
3 DECIDE    → decisions/ADR-NNNN-<slug>.md        (assimilate/reimplement/hybrid)
4 SPECIFY   → specs/<Capability>.md               (contracts + acceptance metrics)
5 CODE LOOP → code + scenarios + benchmarks       (iterate until metrics beat baseline)

Hard gate: no implementation code for a capability exists before its ADR (step 3) is committed.

Loop tiers and the chaos roll

Every work packet declares a tier before work starts. The tier sets how heavy steps 1–3 are; steps 4–5 (spec with metrics, code loop with evidence) are never skipped for code-producing work.

Tier	Weight of steps 1–3	Structural trigger (forces at least this tier)
L	Full: separate survey, adversarial review, ADR	Creates a new capability port, or is named a high-leverage pass by the maintainer
M	Survey and ADR merged into one document; review optional	Touches a canonical interface, or adds/updates an external dependency
S	One provenance paragraph in the commit message	Everything else (utilities, fixes, refactors inside a boundary)

The chaos roll. After deriving the structural tier, roll d10 (shuf -i 1-10 -n 1). On a 10, the tier is instead picked uniformly at random (shuf -e S M L -n 1), overriding the structural derivation — up or down. Both rolls are recorded in the tier declaration (tier: M (structural L, chaos 10→M)). Record the roll every time, including when it changes nothing (tier: L (structural L, chaos 4)), so a mechanism that never fires is visible rather than assumed. Purpose: an occasional random reweighting keeps the classification honest — arguing everything into S stops paying off when audits can compare argued tiers against the random sample — and occasionally forces a deep look at something "obviously trivial", which is where local optima hide.

Chaos limits: a rolled-down tier relaxes process weight only. Invariants (zero foreign types in canonical interfaces, determinism, passing conformance suites) bind at every tier, and a rolled-down pass touching a canonical interface still requires the interface change to be flagged in the commit for retrospective review.

Step 1 — Research

Identify the best implementation in existence for this capability. Produce research/CB-RES-NNNN-<slug>.md following the survey template (below). The survey is done when it can name, per dimension, a concrete benchmark-to-beat: a number, a property, or a reproducible comparison — not an impression.

Runnable-baseline option. For passes judged high-leverage (declared by the maintainer or proposed in the survey and confirmed in the ADR), cited numbers are not enough: the survey must ship a reproducible baseline harness that runs the leading candidate on our machine against our workload — the same scenario files where feasible. The harness ships with a fidelity note stating what was and wasn't faithfully reproduced, so a hastily wired competitor setup cannot silently inflate our advantage. Where the option is not invoked (or the candidate isn't practically runnable), comparisons against cited-only numbers are directional: the evidence verdict for those rows caps at parity, never better.

Step 2 — Approve (adversarial review)

For tier-L passes, approval is earned through an adversarial review: a separate session (or agent), given only the survey document, attempts to break it — an omitted candidate, a stale or unverifiable benchmark, an unmeasured claim presented as measured. Exactly one round: challenge, then response. The survey is approvable only when every challenge is either answered with evidence or conceded and folded into the survey.

Documentation requirement: the research process, the challenge, and the resulting improvements to the research are each documented in timestamped markdown files under history/:

history/YYMMDD-<slug>-research.md    # how the survey was conducted: sources,
                                     # queries, what was measured vs cited, dead ends
history/YYMMDD-<slug>-challenge.md   # the adversarial attack, verbatim
history/YYMMDD-<slug>-response.md    # answers/concessions and what changed in the survey

The polished survey artifact remains research/CB-RES-NNNN-<slug>.md; the history files preserve the unpolished trail so a later reader can judge how hard the survey was actually tested. For tier-M passes the review is optional but, when performed, follows the same format. If not approvable after the round, the loop returns to step 1 with the named gaps.

Step 3 — Decide

decisions/ADR-NNNN-<slug>.md: assimilate behind a port, reimplement, or hybrid — with the expected advantage stated per dimension (see rubric). An honest "worse here, better there, and why that trade is right" beats a claimed sweep of all four dimensions.

Step 4 — Specify

specs/<Capability>.md: the contracts, invariants, and — mandatory — the acceptance metrics table, each row tied to a baseline from step 1. A spec without measurable acceptance criteria is not done. Metrics follow the conventions in MetricsAndScenarios.md, including the rule that metric selection itself passes through a mini research step (metric provenance).

Every metric names its instrument, and is checked reachable. A row in the acceptance table carries the command that produces its number. A metric with no named instrument is a wish, not a metric.

The instrument must exist, and the target must come out of it. Naming a command is not the same as running one. A target computed by hand and merely labelled with a command is the same defect the rule was written to stop, one level down. Where the instrument is built later in the pass, the target is marked provisional: until the instrument emits it, and the spec is amended to whatever the instrument returns.

(v1.1, from CB-WP-0002: specs/CostAccounting.md AC-1 named cb-cost --pin fc76445 before that tool existed, and set the target to a hand-computed $92.87. When the tool was built it returned $93.32 — the hand computation carried a dedup bug the tool's own positive control caught. The metric satisfied v1.0's rule completely and was still wrong.)

A number inherited from earlier work is re-derived before it is used as a target, or it is cited as unverified. Quoting is not measuring.

(v1.1, from CB-WP-0002: the workplan opened with $248.46, inherited from a prior pass. Re-derivation put it at $92.21 — the quoted figure double-counted transcript lines and priced a three-model session at one model's rate. Neither error was of the harness-does-nothing class; both sums ran over real data, and a positive control would have passed them.) A metric must also be checked against the contracts in the same spec: if a contract makes a target unreachable, one of the two is wrong and the conflict is resolved when it is noticed, not at the acceptance run. Re-check the table whenever a contract is added.

(v1.0, from CB-WP-0001: AM-4's ≤20-crate target was made unreachable by the K5 and K7 contracts written after it, and AM-12's cost metric was fully specified and never instrumented, so it could not be computed.)

Step 5 — Code loop

Implement iteratively. Each iteration:

change → cb-check (fmt, clippy, tests) → scenarios → benchmarks
       → compare against acceptance table → evidence row appended

Done when every acceptance metric meets or beats its baseline and the comparison numbers are committed as an evidence file (evidence/CB-EV-NNNN-<slug>.md). A failed scenario must yield a replay artifact an agent can re-execute locally.

Measurement validity — the positive control

Every benchmark and harness must assert that it performed the work it reports. Completing without error is not evidence of having done anything: a loop whose commands are all rejected runs fast and reports a throughput for work that never happened.

Concretely, a measurement harness must, on every run:

• assert the unit of work produced its expected effect (events applied, rows written, moves accepted) — not merely that the call returned;
• fail loudly rather than report a number when that assertion fails;
• state the divisor used to convert raw timings into the metric's unit, pinned by a test so a workload change cannot silently rescale it.

A number from a run that cannot prove it did the work is void and must not reach an evidence file.

Every tool that reports a number exposes --self-test, and that self-test runs before the number is produced (make cost depends on make cost-test). The assertion must name a failure it detects, not merely exercise the happy path.

(v1.0+, from CB-WP-0002: cb-cost's dedup assertion fired on its first run against real data and aborted, catching a rule that was verified on 206/206 groups of the main transcript and false in the 8-response subagent tree. The generalization that failed — a property confirmed on the largest sample assumed to hold on the smallest — is not one review catches, because both the survey and the adversarial reviewer checked the same large sample.)

(v1.0, from CB-WP-0001: both serious errors in the first pass were of exactly this shape. A JS harness reported 8.4s for 100k moves while every move was being rejected, and a Rust benchmark reported 9.3M events/s — a 93× beat — while most rounds never completed because a stress gate rejected one player's action. The corrected figure was 5.6× lower. Adversarial review caught neither; both were claims about numbers, and review reads prose.)

Evidence states what it does not support

An evidence file that compares across runtimes, languages, or feature sets names the disanalogies explicitly, in the same section as the number. The reader must not have to infer that a ratio is not like-for-like. This is the parity-cap rule applied to the write-up: state the claim you will defend, and the claim you are not making.

---

The four-dimension rubric

Every survey, ADR, and acceptance table is organized by these dimensions:

Dimension	Question	Example measurable proxies
D1 Ease of specification	How simply can behavior be stated, tested, understood?	rules-to-scenario coverage %, spec lines per rule, time-to-first-correct-scenario for a fresh agent session
D2 Efficiency of implementation	How cheap to build and keep building?	source LOC, dependency count/weight, clean-build and incremental-build time, tokens-per-completed-task
D3 Speed of execution	How fast does it run?	benchmark wall-time vs baseline, events/sec, memory footprint, determinism overhead
D4 Optionality	How cleanly does it integrate, extend, get replaced?	public API surface size, count of leaked foreign types (must be 0), effort-to-swap measured by null/reference impl existence, WIT-expressibility

Scoring is always relative to the step-1 baseline, never absolute: better / parity / worse / unmeasured per proxy, with the number attached. unmeasured is legal in a survey, illegal in an evidence file.

---

Survey template (research/CB-RES-NNNN-.md)

# CB-RES-NNNN: <capability>
capability: <canonical.capability.id>
status: draft | approved

## Candidates
Per candidate: origin, license, maturity, adoption; data model; mutation
mechanism; determinism/replay story; relevant performance (measured if
runnable locally, cited with source otherwise).

## Baselines (benchmark-to-beat)
| Dimension | Baseline holder | Metric | Value | Provenance |
(one row minimum per dimension; provenance = measured / cited / estimated)

## Verdict
Which candidate leads per dimension; what none of them do well
(the surpass opportunity); risks in the baselines themselves.

---

Implementation rules the first pass earned

These are cheap, and each exists because its absence cost something in CB-WP-0001. See history/260731-inner-loop-retrospective.md.

1. No silently-ignored input. A field that is parsed and then unused is a defect, not a stub. Inputs are honoured or rejected with an error — never dropped. (A scenario setup.patch was parsed and discarded; every scenario using it would have tested the wrong initial state while passing.)
2. Decisions get commands, not defaults. A rule that requires a participant's choice is implemented as a command carrying that choice. Until it is, nothing claims coverage of it — no tag, no scenario, no acceptance row. Inventing a default to make a rule "done" is the failure this prevents.
3. Scaffolds are exercised or marked. A scaffold's green gates are not evidence. Any scaffold path no test reaches is marked as unexercised. (A compiling, fully-green scaffold shipped a state-hash that would panic on any state holding a relation.)
4. Coverage gates that count tags say so. A gate comparing rule IDs against covers: lists proves no rule is unclaimed and no claimed rule is invented. It does not prove a scenario exercises what it names. Wherever such a number is reported, that limit is reported with it.

---

Agentic-efficiency requirements

The loop exists to be driven by agents. Therefore:

1. Whole-file loadability — every loop artifact stays under ~400 lines; split before exceeding, link with relative paths.
2. Structured over prose — tables and fenced blocks for anything a later step must parse (baselines, acceptance metrics, evidence rows).
3. One command surface — all checks runnable through repo-root commands (eventually cb *; until then, make/cargo aliases declared in one place), each supporting deterministic, greppable output.
4. Self-contained tasks — a workplan task names its input artifacts and output artifacts; a fresh session must be able to execute it from the task text plus linked files alone.
5. Evidence or it didn't happen — claims of "better" live in committed evidence files with numbers, never only in commit messages or chat.
6. Token discipline — per the global budget policy, a loop iteration that exceeds its budget without measurable progress is stopped and decomposed, not pushed through.

---

Definition of done — one loop pass

A capability has completed the loop when all of the following are committed:

• research/CB-RES-NNNN with approved status and full baseline table
• decisions/ADR-NNNN with per-dimension expected advantage
• specs/.md with acceptance-metrics table
• passing scenarios covering every numbered spec rule
• evidence/CB-EV-NNNN with final comparison vs baseline, no unmeasured
• every reported number produced by a harness with a positive control; any metric that could not be instrumented is recorded as uncomputable rather than estimated
• every unmet metric reported as unmet, with attribution and the options for resolving it — a missed target is an output of the loop, not a reason to move the target quietly
• cost recorded: make cost run for the pass, its composition (not only its total) in the evidence file, and the per-task figures pushed to the hub. M-D2-CST is no longer allowed to be uncomputable — the instrument exists (CostAccounting.md)
• retrospective note (may be one paragraph appended to the evidence file): what the loop itself should change