T07: InnerLoop v1.1 — the instrument must emit its own target
Some checks failed
ci / check (push) Failing after 3s
Some checks failed
ci / check (push) Failing after 3s
Answers the question the workplan posed: did v1.0's "every metric names
its instrument" rule stop a metric being written without a working
instrument? No. AC-1 named `cb-cost --pin` before that tool existed and
set a hand-computed target of $92.87; the tool returned $93.32. The rule
was satisfied completely and the metric was still wrong.
v1.1 adds:
- the instrument must exist and the target must come out of it;
targets are provisional until the tool emits them
- a number inherited from earlier work is re-derived before use as a
target, or cited as unverified
- every reporting tool exposes --self-test, run before the number
- cost is in the definition of done; M-D2-CST may not be uncomputable
The cost of CB-WP-0001 was stated four times before it was right --
$248.46, $92.21, $92.87, $93.32 -- and each correction came from a
different mechanism: re-derivation, adversarial review, and the positive
control. None found more than one. That is the case for keeping all
three.
CB-WP-0003 T10 predicted the next error would be harness-does-nothing.
It was not, twice. Trusted arithmetic over real data would pass a
positive control; and a property verified on 206/206 groups of the main
transcript is false in the 8-response subagent tree that neither the
survey nor the reviewer examined separately. Review structurally cannot
catch the second -- re-deriving on the same sample reproduces the same
blind spot.
Workplan status: done.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
2f086d26b6
commit
42180fbc57
3 changed files with 150 additions and 9 deletions
|
|
@ -1,11 +1,17 @@
|
|||
# The Inner Loop — Assimilate and Surpass
|
||||
|
||||
Status: **v1.0** — survived its first full pass (CB-WP-0001, the GROUND
|
||||
game kernel) and was corrected from it on 2026-07-31. Changes from v0.2:
|
||||
measurement validity (the positive control), metric feasibility and
|
||||
instrument naming, four implementation rules the pass earned, and the
|
||||
requirement that evidence state what it does not support. Rationale and
|
||||
the failures behind each: `history/260731-inner-loop-retrospective.md`.
|
||||
Status: **v1.1** — corrected from CB-WP-0002 (cost accounting) on
|
||||
2026-07-31. Changes from v1.0: the instrument must exist and emit its own
|
||||
target; inherited numbers are re-derived before use; every reporting tool
|
||||
exposes `--self-test`; cost is in the definition of done. Rationale:
|
||||
`history/260731-cost-accounting-retrospective.md`.
|
||||
|
||||
v1.0 — survived its first full pass (CB-WP-0001, the GROUND game kernel)
|
||||
and was corrected from it on 2026-07-31. Changes from v0.2: measurement
|
||||
validity (the positive control), metric feasibility and instrument naming,
|
||||
four implementation rules the pass earned, and the requirement that
|
||||
evidence state what it does not support. Rationale and the failures behind
|
||||
each: `history/260731-inner-loop-retrospective.md`.
|
||||
|
||||
Normative process for building every Clay-Borg capability. Referenced by
|
||||
all workplans. The loop's own optimization target is **agentic efficiency**:
|
||||
|
|
@ -122,7 +128,30 @@ research step (metric provenance).
|
|||
|
||||
**Every metric names its instrument, and is checked reachable.** A row
|
||||
in the acceptance table carries the command that produces its number.
|
||||
A metric with no named instrument is a wish, not a metric. A metric must
|
||||
A metric with no named instrument is a wish, not a metric.
|
||||
|
||||
**The instrument must exist, and the target must come out of it.**
|
||||
Naming a command is not the same as running one. A target computed by
|
||||
hand and merely *labelled* with a command is the same defect the rule
|
||||
was written to stop, one level down. Where the instrument is built later
|
||||
in the pass, the target is marked `provisional:` until the instrument
|
||||
emits it, and the spec is amended to whatever the instrument returns.
|
||||
|
||||
*(v1.1, from CB-WP-0002: `specs/CostAccounting.md` AC-1 named
|
||||
`cb-cost --pin fc76445` before that tool existed, and set the target to
|
||||
a hand-computed $92.87. When the tool was built it returned $93.32 —
|
||||
the hand computation carried a dedup bug the tool's own positive control
|
||||
caught. The metric satisfied v1.0's rule completely and was still
|
||||
wrong.)*
|
||||
|
||||
**A number inherited from earlier work is re-derived before it is used
|
||||
as a target, or it is cited as unverified.** Quoting is not measuring.
|
||||
|
||||
*(v1.1, from CB-WP-0002: the workplan opened with $248.46, inherited
|
||||
from a prior pass. Re-derivation put it at $92.21 — the quoted figure
|
||||
double-counted transcript lines and priced a three-model session at one
|
||||
model's rate. Neither error was of the harness-does-nothing class; both
|
||||
sums ran over real data, and a positive control would have passed them.)* A metric must
|
||||
also be checked against the contracts in the *same spec*: if a contract
|
||||
makes a target unreachable, one of the two is wrong and the conflict is
|
||||
resolved when it is noticed, not at the acceptance run. Re-check the
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue