T07: separate a correction from a retarget with a mechanical test
A blanket "no retargeting in the measuring commit" rule would have been
wrong. CB-WP-0002 moved AC-1 three times in exactly that shape and every
move was correct -- each time a new instrument disproved the old figure.
Four legitimate corrections would have been forbidden to catch one bad
retarget.
The test is mechanical rather than a statement of intent:
correction the target moves and the implementation does not; legal in
the same commit provided the instrument's output is there
retarget the same commit changes both the target and the code the
target measures; requires an ADR stating why the new target
binds on future work
Applied retroactively: AM-4a/AM-4b are UNRATIFIED. They were set after
seeing the measurement, in the commit that produced it, with the
implementation changing too -- a retarget by this test. make dep-weight
is currently enforcing a target no reviewed decision stands behind.
Recorded as an open item; ratifying or changing them is a maintainer
decision, not an implementer's.
Also: specs/InnerLoop.md split into InnerLoop.md (process) and
InnerLoopReference.md (rubric, template, rules, definition of done).
Not a stylistic choice -- `make loop-lint` failed on the commit that
pushed the file to 407 lines against its own ~400 limit. The gate added
this morning to make that rule executable caught its own author within
the hour, which is the cheapest possible demonstration that it works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ced573ba75
commit
e21b9f4250
4 changed files with 189 additions and 130 deletions
141
specs/InnerLoopReference.md
Normal file
141
specs/InnerLoopReference.md
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
# The Inner Loop — Reference
|
||||
|
||||
Normative companion to **[InnerLoop.md](InnerLoop.md)**, which holds the
|
||||
five steps, the tiers, and the chaos roll. Split from it on 2026-07-31 so
|
||||
both files stay under the loop's own whole-file loadability limit.
|
||||
|
||||
Everything here binds exactly as the process document does.
|
||||
|
||||
## The four-dimension rubric
|
||||
|
||||
Every survey, ADR, and acceptance table is organized by these dimensions:
|
||||
|
||||
| Dimension | Question | Example measurable proxies |
|
||||
|---|---|---|
|
||||
| **D1 Ease of specification** | How simply can behavior be stated, tested, understood? | rules-to-scenario coverage %, spec lines per rule, time-to-first-correct-scenario for a fresh agent session |
|
||||
| **D2 Efficiency of implementation** | How cheap to build and keep building? | source LOC, dependency count/weight, clean-build and incremental-build time, tokens-per-completed-task |
|
||||
| **D3 Speed of execution** | How fast does it run? | benchmark wall-time vs baseline, events/sec, memory footprint, determinism overhead |
|
||||
| **D4 Optionality** | How cleanly does it integrate, extend, get replaced? | public API surface size, count of leaked foreign types (must be 0), effort-to-swap measured by null/reference impl existence, WIT-expressibility |
|
||||
|
||||
Scoring is always **relative to the step-1 baseline**, never absolute:
|
||||
`better / parity / worse / unmeasured` per proxy, with the number attached.
|
||||
`unmeasured` is legal in a survey, illegal in an evidence file.
|
||||
|
||||
---
|
||||
|
||||
## Survey template (research/CB-RES-NNNN-<slug>.md)
|
||||
|
||||
```markdown
|
||||
# CB-RES-NNNN: <capability>
|
||||
capability: <canonical.capability.id>
|
||||
status: draft | approved
|
||||
|
||||
## Candidates
|
||||
Per candidate: origin, license, maturity, adoption; data model; mutation
|
||||
mechanism; determinism/replay story; relevant performance (measured if
|
||||
runnable locally, cited with source otherwise).
|
||||
|
||||
## Baselines (benchmark-to-beat)
|
||||
| Dimension | Baseline holder | Metric | Value | Provenance |
|
||||
(one row minimum per dimension; provenance = measured / cited / estimated)
|
||||
|
||||
## Verdict
|
||||
Which candidate leads per dimension; what none of them do well
|
||||
(the surpass opportunity); risks in the baselines themselves.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation rules the first pass earned
|
||||
|
||||
These are cheap, and each exists because its absence cost something in
|
||||
CB-WP-0001. See `history/260731-inner-loop-retrospective.md`.
|
||||
|
||||
1. **No silently-ignored input.** A field that is parsed and then unused
|
||||
is a defect, not a stub. Inputs are honoured or rejected with an
|
||||
error — never dropped. *(A scenario `setup.patch` was parsed and
|
||||
discarded; every scenario using it would have tested the wrong
|
||||
initial state while passing.)*
|
||||
2. **Decisions get commands, not defaults.** A rule that requires a
|
||||
participant's choice is implemented as a command carrying that
|
||||
choice. Until it is, **nothing claims coverage of it** — no tag, no
|
||||
scenario, no acceptance row. Inventing a default to make a rule
|
||||
"done" is the failure this prevents.
|
||||
3. **Scaffolds are exercised or marked.** A scaffold's green gates are
|
||||
not evidence. Any scaffold path no test reaches is marked as
|
||||
unexercised. *(A compiling, fully-green scaffold shipped a state-hash
|
||||
that would panic on any state holding a relation.)*
|
||||
4. **Coverage gates that count tags say so.** A gate comparing rule IDs
|
||||
against `covers:` lists proves no rule is unclaimed and no claimed
|
||||
rule is invented. It does **not** prove a scenario exercises what it
|
||||
names. Wherever such a number is reported, that limit is reported
|
||||
with it.
|
||||
|
||||
---
|
||||
|
||||
## Agentic-efficiency requirements
|
||||
|
||||
The loop exists to be driven by agents. Therefore:
|
||||
|
||||
1. **Whole-file loadability** — every loop artifact stays under ~400 lines;
|
||||
split before exceeding, link with relative paths.
|
||||
2. **Structured over prose** *(guidance, not a requirement — nothing can
|
||||
check it)* — tables and fenced blocks for anything a later step must
|
||||
parse (baselines, acceptance metrics, evidence rows).
|
||||
3. **One command surface** — all checks runnable through repo-root
|
||||
commands (eventually `cb *`; until then, `make`/`cargo` aliases declared
|
||||
in one place), each supporting deterministic, greppable output.
|
||||
4. **Self-contained tasks** — a workplan task names its input artifacts and
|
||||
output artifacts; a fresh session must be able to execute it from the
|
||||
task text plus linked files alone.
|
||||
5. **Evidence or it didn't happen** — claims of "better" live in committed
|
||||
evidence files with numbers, never only in commit messages or chat.
|
||||
6. ~~**Token discipline** — per the global budget policy, a loop iteration
|
||||
that exceeds its budget without measurable progress is stopped and
|
||||
decomposed, not pushed through.~~ **DEAD POLICY.** The 8k/10k per-task
|
||||
token budget was never referenced or enforced, and CB-WP-0001 T08
|
||||
exceeded it by orders of magnitude with no signal. It implies a control
|
||||
that does not exist. Replacement in USD is CB-WP-0003 T05; until then
|
||||
this is documentation of a gap, not a rule.
|
||||
|
||||
6a. **Live cost budget** *(replaces the above)* — spend since the last
|
||||
commit, soft **$10.00**, hard **$22.00**, checked by `make cost-budget`.
|
||||
Calibrated on the 32 commit intervals of CB-WP-0001 (p50 $1.40, p90
|
||||
$9.36, max $10.80), so both thresholds bind on future work rather than
|
||||
ratifying past work. Contract: [CostAccounting.md](CostAccounting.md)
|
||||
§7 (CB-01, CB-02). It fires on the open remainder rather than per task
|
||||
because per-task cost needs the commit that closes the task, and a
|
||||
budget that can only report after the money is spent is the dead policy
|
||||
this replaces.
|
||||
|
||||
**Enforcement status.** Rules above that a command can check are enforced
|
||||
by `make loop-lint`; the full classification of every InnerLoop rule as
|
||||
executable / checkable / decorative, with the failure class each catches,
|
||||
is in `history/260731-inner-loop-rule-audit.md`. Rules marked *guidance*
|
||||
or *dead* say so where they appear, so a reader can tell a requirement
|
||||
from a preference without consulting the audit.
|
||||
|
||||
---
|
||||
|
||||
## Definition of done — one loop pass
|
||||
|
||||
A capability has completed the loop when all of the following are committed:
|
||||
|
||||
- [ ] research/CB-RES-NNNN with approved status and full baseline table
|
||||
- [ ] decisions/ADR-NNNN with per-dimension expected advantage
|
||||
- [ ] specs/<Capability>.md with acceptance-metrics table
|
||||
- [ ] passing scenarios covering every numbered spec rule
|
||||
- [ ] evidence/CB-EV-NNNN with final comparison vs baseline, no `unmeasured`
|
||||
- [ ] every reported number produced by a harness with a positive
|
||||
control; any metric that could not be instrumented is recorded as
|
||||
uncomputable rather than estimated
|
||||
- [ ] every unmet metric reported as unmet, with attribution and the
|
||||
options for resolving it — a missed target is an output of the
|
||||
loop, not a reason to move the target quietly
|
||||
- [ ] **cost recorded**: `make cost` run for the pass, its composition
|
||||
(not only its total) in the evidence file, and the per-task figures
|
||||
pushed to the hub. M-D2-CST is no longer allowed to be
|
||||
`uncomputable` — the instrument exists
|
||||
([CostAccounting.md](CostAccounting.md))
|
||||
- [ ] retrospective note (may be one paragraph appended to the evidence
|
||||
file): what the loop itself should change
|
||||
Loading…
Add table
Add a link
Reference in a new issue