clay-borg/history/260731-inner-loop-retrospective.md

# Inner-loop retrospective — after CB-WP-0001 (first full pass)

Date: 2026-07-31
Subject: `specs/InnerLoop.md` v0.2 → v1.0
Evidence: `evidence/CB-EV-0001-game-kernel.md`

The loop has now survived one full pass, from SOTA survey through an
adversarial review, an ADR, two specs, and six code iterations that
ended in a committed acceptance run. This is what it got right, what it
missed, and what changes in v1.0.

## What earned its cost

**The adversarial review (Step 2).** On its single use it produced one
blocking finding and three significant ones. The blocking one mattered:
the survey claimed boardgame.io's degradation was "architectural, not a
tuning artifact", and the challenge pointed at the `disableUndo` flag
that the claim had not accounted for. The resolution was a rerun with
undo disabled, not a rhetorical defence — degradation persisted (0.66×
at 40k, still DNF at 100k), so the overclaim was retracted and two
mechanisms were attributed separately. A survey that had gone straight
to an ADR would have carried a false claim into the decision.

**The parity-cap rule.** It stopped the boardgame.io comparison from
being written up as a verdict. The final evidence file says the ~450×
command-rate ratio is a direction, not a verdict, and lists the four
reasons it is not like-for-like. That paragraph exists because the rule
required it.

**Deferring rules that need a decision.** Four rules (GR-L02, GR-A05,
GR-A11, GR-A12) were left unimplemented across two iterations rather
than given invented defaults, then implemented as explicit commands.
No scenario claimed coverage of them in the interim. This is the single
most useful habit the pass produced and v1.0 promotes it to a rule.

**The provisional mechanism.** Ten U-items with `provisional: true`
scenarios meant an underdetermined rule cost a scenario tag, not a
kernel decision. One further ambiguity (GR-E02's "successes") was found
during implementation and captured the same way.

## What the loop missed

**Both serious errors in this pass were measurement errors, and the
loop caught neither.** Review caught claims; nothing caught numbers.

1. The boardgame.io harness v1 reported "8.4s for 100k moves". It was
   measuring rejected no-ops — every move was refused. Caught by
   noticing the error text in output, not by any gate.
2. The Rust benchmark reported **9.3M events/s on a flat curve** and
   would have been published as a 93× beat of AM-6. The workload had a
   player selecting SUPPORT while parked at Stress 4, so GR-R03 rejected
   it, rounds never completed, and throughput was computed for rounds
   that never happened. Caught only because a probe test asserted a
   round produces events at all. The corrected number is 5.6× lower.

Both share a shape: **the harness ran successfully while doing no
work**, and success was inferred from "it completed" rather than from
"it did the thing". This is the dominant failure mode of an agent-driven
loop, because an agent will happily report a fast number.

**Scaffold code with unexercised paths.** T07 produced a compiling,
"green" workspace that contained two latent defects: relations keyed by
a tuple, which JSON cannot serialize, so `state_hash` would have
panicked on any state holding a relation; and `setup.patch`, which was
parsed and silently discarded, so every scenario using it would have
tested the wrong initial state. Both were invisible until real scenarios
exercised them. Green gates on a scaffold mean very little.

**Metrics were specified without checking they were reachable or
instrumented.** AM-4 (≤20 crates) is unreachable given the spec's own
K5 and K7 contracts, which require ChaCha and SHA-256 and cost 12 crates
between them — the metric was written before those contracts existed and
never revisited. AM-12 (cost in USD) was specified in detail, with a
price sheet and a formula, and then never instrumented, so it could not
be computed at all. A metric with no named instrument is a wish.

**The chaos roll never fired.** Across the pass the d10 never selected
a tier different from the structural one, so the mechanism is untested
in practice. It is retained, but v1.0 requires the roll be recorded even
when it changes nothing, so its absence is visible rather than assumed.

**The coverage gate counts tags, not behaviour.** `make coverage`
reports 58/58 by comparing rule IDs in the spec against `covers:` lists.
It catches invented IDs and outright gaps, which is real, but a scenario
can name a rule it does not exercise. 100% on that gate is not proof
AM-1 is met, and v1.0 says so where the number is reported.

## Changes in v1.0

1. **Measurement validity (new, Step 5).** Every benchmark and harness
   must assert it performed the work it reports — a positive control.
   A number from a run that cannot prove it did the work is void.
2. **Metric feasibility (Step 4).** Each acceptance metric names its
   instrument and is checked reachable against the contracts in the same
   spec. Re-checked whenever a contract is added.
3. **No silently-ignored input (new).** A parsed-but-unused field is a
   defect. Inputs are honoured or rejected, never dropped.
4. **Decisions get commands, not defaults (new).** A rule requiring a
   participant's choice is implemented as a command; if it is not
   implemented yet, nothing claims coverage of it.
5. **Scaffolds are exercised or marked (new).** Scaffold code paths not
   reached by a test are marked as unexercised; a scaffold's green gates
   are not evidence.
6. **Evidence states what it does not support (Step 5).** Every
   cross-runtime comparison names its disanalogies explicitly.
7. **The chaos roll is always recorded**, including when it does not
   change the tier.

## What did not need changing

The five steps, the four-dimension rubric, the tier system's structural
triggers, the ADR gate ("no implementation code before the ADR is
committed"), the survey template, and the agentic-efficiency
requirements all held up. The gate in particular was never
uncomfortable — by the time the ADR was written the decision was easy,
which is what a good gate feels like.