T09: InnerLoop v1.0 — corrected from the first full pass
Some checks failed
ci / check (push) Has been cancelled

The loop has now survived CB-WP-0001 end to end and is revised from what
that pass actually taught, recorded in
history/260731-inner-loop-retrospective.md.

Held up: the adversarial review (one blocking and three significant
findings on its single use, resolved with new measurements rather than
argument), the parity-cap rule, the provisional U-item mechanism, and
the ADR gate.

The gap: both serious errors in the pass were measurement errors, and
the loop caught neither. Review reads prose; these were claims about
numbers. In both cases a harness ran successfully while doing no work —
a JS harness timing rejected no-ops, and a Rust benchmark computing
throughput for rounds that never completed.

v1.0 adds:
- Measurement validity: a harness must assert it performed the work it
  reports. A number from a run that cannot prove it did the work is void.
- Metric feasibility: every metric names its instrument and is checked
  reachable against the contracts in its own spec.
- No silently-ignored input; decisions get commands, not defaults;
  scaffolds are exercised or marked; coverage gates that count tags say
  so.
- Evidence must state what a comparison does not support.
- The chaos roll is recorded even when it changes nothing, so a
  mechanism that never fires is visible rather than assumed.

CB-WP-0001 is complete: 9/9 tasks done.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 03:11:19 +02:00
parent eb1378e667
commit 63dcffd9f1
3 changed files with 215 additions and 7 deletions

View file

@ -1,9 +1,11 @@
# The Inner Loop — Assimilate and Surpass
Status: **v0.2 draft** — becomes v1.0 only after surviving its first full
pass (CB-WP-0001-T09 retrospective). v0.2 adds loop tiers with the chaos
roll, adversarial survey review, and the runnable-baseline option
(maintainer decision, 2026-07-31).
Status: **v1.0** — survived its first full pass (CB-WP-0001, the GROUND
game kernel) and was corrected from it on 2026-07-31. Changes from v0.2:
measurement validity (the positive control), metric feasibility and
instrument naming, four implementation rules the pass earned, and the
requirement that evidence state what it does not support. Rationale and
the failures behind each: `history/260731-inner-loop-retrospective.md`.
Normative process for building every Clay-Borg capability. Referenced by
all workplans. The loop's own optimization target is **agentic efficiency**:
@ -42,7 +44,10 @@ are never skipped for code-producing work.
(`shuf -i 1-10 -n 1`). On a **10**, the tier is instead picked uniformly at
random (`shuf -e S M L -n 1`), overriding the structural derivation — up or
down. Both rolls are recorded in the tier declaration
(`tier: M (structural L, chaos 10→M)`). Purpose: an occasional random
(`tier: M (structural L, chaos 10→M)`). **Record the roll every time,
including when it changes nothing** (`tier: L (structural L, chaos 4)`),
so a mechanism that never fires is visible rather than assumed. Purpose:
an occasional random
reweighting keeps the classification honest — arguing everything into S
stops paying off when audits can compare argued tiers against the random
sample — and occasionally forces a deep look at something "obviously
@ -115,6 +120,18 @@ the conventions in [MetricsAndScenarios.md](MetricsAndScenarios.md),
including the rule that metric selection itself passes through a mini
research step (metric provenance).
**Every metric names its instrument, and is checked reachable.** A row
in the acceptance table carries the command that produces its number.
A metric with no named instrument is a wish, not a metric. A metric must
also be checked against the contracts in the *same spec*: if a contract
makes a target unreachable, one of the two is wrong and the conflict is
resolved when it is noticed, not at the acceptance run. Re-check the
table whenever a contract is added.
*(v1.0, from CB-WP-0001: AM-4's ≤20-crate target was made unreachable by
the K5 and K7 contracts written after it, and AM-12's cost metric was
fully specified and never instrumented, so it could not be computed.)*
### Step 5 — Code loop
Implement iteratively. Each iteration:
@ -129,6 +146,40 @@ comparison numbers are committed as an evidence file
(`evidence/CB-EV-NNNN-<slug>.md`). A failed scenario must yield a replay
artifact an agent can re-execute locally.
#### Measurement validity — the positive control
**Every benchmark and harness must assert that it performed the work it
reports.** Completing without error is not evidence of having done
anything: a loop whose commands are all rejected runs fast and reports a
throughput for work that never happened.
Concretely, a measurement harness must, on every run:
- assert the unit of work produced its expected effect (events applied,
rows written, moves accepted) — not merely that the call returned;
- fail loudly rather than report a number when that assertion fails;
- state the divisor used to convert raw timings into the metric's unit,
pinned by a test so a workload change cannot silently rescale it.
**A number from a run that cannot prove it did the work is void** and
must not reach an evidence file.
*(v1.0, from CB-WP-0001: both serious errors in the first pass were of
exactly this shape. A JS harness reported 8.4s for 100k moves while
every move was being rejected, and a Rust benchmark reported 9.3M
events/s — a 93× beat — while most rounds never completed because a
stress gate rejected one player's action. The corrected figure was 5.6×
lower. Adversarial review caught neither; both were claims about
numbers, and review reads prose.)*
#### Evidence states what it does not support
An evidence file that compares across runtimes, languages, or feature
sets **names the disanalogies explicitly**, in the same section as the
number. The reader must not have to infer that a ratio is not
like-for-like. This is the parity-cap rule applied to the write-up:
state the claim you will defend, and the claim you are not making.
---
## The four-dimension rubric
@ -171,6 +222,33 @@ Which candidate leads per dimension; what none of them do well
---
## Implementation rules the first pass earned
These are cheap, and each exists because its absence cost something in
CB-WP-0001. See `history/260731-inner-loop-retrospective.md`.
1. **No silently-ignored input.** A field that is parsed and then unused
is a defect, not a stub. Inputs are honoured or rejected with an
error — never dropped. *(A scenario `setup.patch` was parsed and
discarded; every scenario using it would have tested the wrong
initial state while passing.)*
2. **Decisions get commands, not defaults.** A rule that requires a
participant's choice is implemented as a command carrying that
choice. Until it is, **nothing claims coverage of it** — no tag, no
scenario, no acceptance row. Inventing a default to make a rule
"done" is the failure this prevents.
3. **Scaffolds are exercised or marked.** A scaffold's green gates are
not evidence. Any scaffold path no test reaches is marked as
unexercised. *(A compiling, fully-green scaffold shipped a state-hash
that would panic on any state holding a relation.)*
4. **Coverage gates that count tags say so.** A gate comparing rule IDs
against `covers:` lists proves no rule is unclaimed and no claimed
rule is invented. It does **not** prove a scenario exercises what it
names. Wherever such a number is reported, that limit is reported
with it.
---
## Agentic-efficiency requirements
The loop exists to be driven by agents. Therefore:
@ -202,6 +280,12 @@ A capability has completed the loop when all of the following are committed:
- [ ] specs/<Capability>.md with acceptance-metrics table
- [ ] passing scenarios covering every numbered spec rule
- [ ] evidence/CB-EV-NNNN with final comparison vs baseline, no `unmeasured`
- [ ] every reported number produced by a harness with a positive
control; any metric that could not be instrumented is recorded as
uncomputable rather than estimated
- [ ] every unmet metric reported as unmet, with attribution and the
options for resolving it — a missed target is an output of the
loop, not a reason to move the target quietly
- [ ] retrospective note (may be one paragraph appended to the evidence
file): what the loop itself should change
```