- InnerLoop definition-of-done gains a cost row: M-D2-CST may no longer
be recorded uncomputable, and composition must be reported, not only a
total.
- InnerLoop Step 5 gains the --self-test contract: every tool that
reports a number exposes one, and it runs before the number does.
Rationale attached, because the case that motivated it is the one
review cannot catch — survey and reviewer both verified the same large
sample and both missed the small one.
- make cost / cost-test / cost-pin on the one command surface; cost-test
in `make all` and in CI.
- Hub now holds the measured figure for CB-WP-0001: 80.6M in / 323.6k
out against the 401,100 it previously estimated, low by ~200x. The
event states plainly that the hub schema cannot represent the 88% of
cost that is cache, and names `make cost-pin` as the authority.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Normative spec for the cost model (dedup by requestId, per-model and
per-TTL pricing), the attribution contract (ending-at-commit intervals
scoped by sessionId), and the reported shape (composition, not a total).
Every acceptance row names the command that produces its number, per
InnerLoop Step 4. AC-5..AC-8 are the positive control: cb-cost
--self-test must fail on a dedup violation, on zero responses, on a
missing subagent tree, and on 5m cache priced at the 1h rate.
The feasibility check earns its place here: AC-1's $92.87 is reachable
only if CA-06 holds. The earlier $92.21 target was reachable only by a
collector with the exact blind spot the survey documented.
MetricsAndScenarios 1a superseded. Two of its rules struck through
rather than deleted, because both were wrong in instructive ways: "if
the cache split is unknown, count all input at full price" would have
priced 80.5M cache reads at 10x, and "the hub already records tokens"
named as a source what is only ever a lossy sink. M-D2-TOK demoted --
tokens are not comparable across models or cache states.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The loop has now survived CB-WP-0001 end to end and is revised from what
that pass actually taught, recorded in
history/260731-inner-loop-retrospective.md.
Held up: the adversarial review (one blocking and three significant
findings on its single use, resolved with new measurements rather than
argument), the parity-cap rule, the provisional U-item mechanism, and
the ADR gate.
The gap: both serious errors in the pass were measurement errors, and
the loop caught neither. Review reads prose; these were claims about
numbers. In both cases a harness ran successfully while doing no work —
a JS harness timing rejected no-ops, and a Rust benchmark computing
throughput for rounds that never completed.
v1.0 adds:
- Measurement validity: a harness must assert it performed the work it
reports. A number from a run that cannot prove it did the work is void.
- Metric feasibility: every metric names its instrument and is checked
reachable against the contracts in its own spec.
- No silently-ignored input; decisions get commands, not defaults;
scaffolds are exercised or marked; coverage gates that count tags say
so.
- Evidence must state what a comparison does not support.
- The chaos roll is recorded even when it changes nothing, so a
mechanism that never fires is visible rather than assumed.
CB-WP-0001 is complete: 9/9 tasks done.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
