T07 — CB-EV-0004. 70 responses, $16.03.
Test 1 MET: 1 spec -> 2, 58 rules -> 76, 1 source file -> 10, and the new
denominator came in at 83% naming K10/K14/K18. The stated failure mode — a
widened denominator reporting the same percentage — did not occur.
Test 2 UNMET: M-D1-MUT 4 of 14 against >=10, and wrong about what as well
as how much. The diagnosis was three absent kernel rules; the measurement
found eight rows with no instrument at all.
Test 3: quality held, and widening surfaced far more than the seven known
defects — six further uninstrumented rows, HDN #7 (rule-coverage's
self-test green while the tool was broken), a stale $248.46 invisible to
facts-check because it was untagged, and a fifth error class.
The clean test CB-WP-0004 was owed is now run, on a pass that used the
tools without building them. The two categories whose tools removed the
manual path are at 0 turns two passes on; the two that merely offered a
better option are now the entire mechanical cost of a pass. Absolute
mechanical cost fell $51.76 -> $6.13. The mechanism holds.
Reported because nothing else would: SH-3 batching is 0.0% this pass — 67
tool calls across 67 responses — against a 20% target, and mean context
315,170 against 200,000. SessionShape has stated these since CB-WP-0003
and none has ever been enforced.
T08 — the retrospective answers its question: yes, a weak mutation is the
new grep, and it is worse, because it fails in the opposite direction.
mutation-check's first run produced two SURVIVED verdicts and both were
the author's own no-op mutations.
The fifth error class: false accusation. HDN, TA, SSB and DFD all
under-report — a real problem passes. FA over-reports: it publishes the
claim that working code is broken, sends the next pass to fix something
that is not broken, and is more credible than the truth because it arrives
with a measurement attached. Thirteen instances, five classes, six passes,
and the newest class is one that hardening created.
One correction to CB-WP-0004 T06: "a gate only pays if it removes the
manual path" is a predictor of whether a gate saves money, not a criterion
for whether it is worth having. mutation-check fails that test and
produced the most valuable findings of the pass.
InnerLoop v1.4: where a claim rests on numbers, the adversarial reviewer
must read the assertion behind each quoted number and mutate it.
Re-running the command that prints a number is not verification of that
number. Second verification step to inherit the author's blindness; both
fixes replace re-derivation with adversarial execution.
CB-WP-0005 status -> done, 5 of 8 tasks, 3 cancelled into CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AGGREGATE becomes a list of source roots and rule patterns become
per-spec, so the link runs over every numbered spec x every crate rather
than GroundRules.md x games/ground/src/lib.rs.
The prediction held on the first run:
AM-1b kernel spec->code link: 15/18 (83%) across 10 source files
unlinked: K10 K14 K18
Kernel rules are link-only by design, and the output says so: they are
kernel invariants with no aggregate, setup preset or command vocabulary,
so scenarios/kernel/*.yaml with covers: [K11] would be a tag in a
directory the runner cannot dispatch. Claiming scenario coverage for them
is the inflation this gate exists to prevent.
Per ADR-0005 §5 the kernel arm reports without feeding the exit code
until 2026-08-31, then binds — the date in the tool, not in prose, with
days remaining printed every run, because open-ended "gate it later" is
how AM-4's targets went unratified for four workplans. The self-test
asserts the gate returns 0 before that date and 2 after.
The zero-rules positive control is replicated on the new denominator: a
kernel regex that stops matching aborts rather than printing 0/0 as
though it were 100%.
The self-test passed while the tool was completely broken. A print(
inside say() became say(), so every real `make coverage` died with
RecursionError while --self-test reported all-ok — it only ever called
kernel_arm(quiet=True) and never executed the reporting path. The control
named the behaviour and did not assert it, which is precisely what this
workplan is about. Fixed by exercising the loud path and asserting it
prints, then verified by re-breaking say() and confirming both new checks
go red. Seventh instance of the harness-does-nothing shape, in the tool
written to find that shape.
Also caught by its own gate: a self-test label that printed "0 K-ids"
beside a passing ">5" assertion, because the detail string rebuilt the
pattern with different escaping. A label that contradicts its own check
is worse than no label.
k_rules, k_linked and k_unlinked are registered facts under facts-check.
A limit of that checker is recorded rather than patched: it is
line-based, so a tagged value that prose-wraps fails.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Duplicated-fact drift is the fourth error class and the only one with no
executable rule. No positive control catches it (both copies are
internally consistent) and re-derivation does not either (the copy
reproduces whatever it was copied from). It is caught only by reading a
copy against its source, which nothing in the loop required.
facts.toml holds 15 facts and is GENERATED by `make facts-gen` from
cb-cost, dep-weight and rule-coverage. The trap this task named — a
hand-maintained registry that becomes another drifting copy — is closed
by facts-check re-running the instruments and failing when the committed
registry disagrees with them. A stale registry cannot certify stale
artifacts.
An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key.
17 occurrences across 5 artifacts are now checked.
Falsified before being believed: changing CostAccounting.md line 158 from
$93.15 to $92.87 — the exact historical drift — produced exit 1 naming
the file, the line and the expected value. Tested against the class it
exists to catch, on a real artifact, not only in its self-test.
It then caught a live tag inside its own documentation example in
InnerLoop.md within the hour. Third time a gate has failed on its own
pass's work.
What it does not close is stated rather than implied: 22 untagged literal
copies remain and are reported, not failed. Tagging is opt-in, a number
can legitimately recur, and a gate that fires on coincidence gets routed
around. Naming the uncovered surface beats claiming the class is closed.
InnerLoop single-source-of-fact moves from prose to executable — v1.3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Discharges the open item T07 raised. Values unchanged at 250,000 and
350,000; what was missing was a reviewed decision behind them, since
they had been set by the implementer in the commit that measured them
and that also changed the feature gating being measured.
The ADR supplies the argument T07's test requires -- why the targets
bind on FUTURE work rather than merely passing present work:
AM-4a leaves 3,750 lines of headroom (1.5%), about one small crate,
so any new shipped-runtime dependency breaches it almost immediately.
That is intended: the shipped runtime should be effectively frozen.
AM-4b leaves 32,979 lines (10.4%), deliberately looser -- dev tooling
should absorb one moderate dependency without a spec change, not two.
Both are ceilings on a quantity that only grows by choice. Nothing
drifts across them; only adding a dependency does.
Falsification condition stated: if a later pass raises AM-4a to
accommodate a dependency it wants, that is the failure the ceiling
exists to catch, and the answer is an ADR arguing for the dependency.
First ADR written under the correction/retarget test; sets the shape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The retrospective question was whether the mechanism set is complete or
each pass still finds a new class. This pass produced both an eighth
instance AND a fourth class, so the answer is the uncomfortable one.
Ledger: 10 instances, 4 classes, across 3 workplans. Every pass has
produced at least one class the previous pass had not seen.
HDN harness-does-nothing 5 executable assertions
TA trusted arithmetic 3 re-derivation
SSB same-sample blind spot 1 assertions over ALL the data
DFD duplicated-fact drift 2 NEW -- reading a copy against source
DFD is genuinely distinct: no positive control catches it, because both
copies are internally consistent, and re-derivation does not either,
because the copy faithfully reproduces what it was copied from. Found
when an inlined price sheet went stale within an hour of T11 changing
the real one.
So v1.2 stops trying to enumerate classes in advance. Every error in
three passes was corrected in-session for under ~1% of the pass, so the
stated design goal is now cheap CORRECTION: keep raw data so numbers are
re-derivable, keep artifacts small and committed so a wrong number is
one grep from everywhere quoting it, give every number a command.
Plus the one rule the new class earns: single source of fact.
The original hypothesis is revised rather than confirmed. "A rule that
cannot be executed is not a rule" is wrong -- the two most valuable
corrections in the project came from a decorative rule that cannot be
automated (re-derive inherited numbers). An executable rule fires
reliably and catches one class; a decorative one fires unreliably and
can catch any class, including unnamed ones. Keep both.
Gates this pass: loop-lint caught 3 real violations on first run, then
failed on its own author within the hour when a T07 edit pushed
InnerLoop.md to 407 lines against its own 400 limit.
CB-WP-0003 complete: 11 of 11 tasks done.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five scenarios encoded U-item defaults with no owner and no review date,
so they could shape the kernel indefinitely while looking handled.
Each now carries provisional_owner and provisional_raised, and the
runtime's ScenarioFile learned both fields (deny_unknown_fields meant
adding them to YAML alone would have failed every scenario -- the parser
had to agree).
make coverage reports every provisional item with its owner and age in
days, warns on any with no owner, and warns past 30 days. It WARNS
rather than breaking the build, on purpose: the ruling is ground-game's
to make and the kernel cannot make it for them. What the loop can
enforce is that evidence files list them, which is now stated in
GroundRules.
rule-coverage --self-test gained an assertion that every provisional
item has both fields, so the next one added without them fails a
command rather than passing quietly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The copy in MetricsAndScenarios 1a went stale within hours of T11: it
still showed the sonnet intro rate as a comment after the real sheet
moved dated rates into data. A spec that duplicates a data file will
always drift from it; the copy is now a pointer that says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
make coverage now reports a second number: how many claimed rules are
also NAMED in the aggregate source. 58/58 tag coverage was weaker
evidence than it read as, and this says how much weaker.
Measured: 49 of 58. Nine rules are claimed by a scenario and appear
nowhere in games/ground/src/lib.rs --
GR-D07 GR-F02 GR-L03 GR-O03 GR-P01 GR-P02 GR-P03 GR-P04 GR-T01.
The gate REPORTS rather than fails, on purpose. Closing the gap by
adding those IDs to comments would satisfy the check without
establishing that any of the nine is implemented -- the overclaim
InnerLoop implementation rule 2 exists to prevent, and one CB-WP-0001
already committed once. Each needs its implementation confirmed before
it is tagged; promoting M-D1-LNK to a failing gate is correct after
that, not before.
Also added: a phantom check that fails when a rule id appears in code
that the spec does not define (currently zero).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The review step targeted the survey. Every serious error in this project
has been in measurement or build configuration, so the loop was
adversarially reviewing the artifact cheapest to fix and leaving
unreviewed the one where errors occur.
Step 2 now routes by risk: when the claim rests on numbers, the reviewer
gets the harness and the evidence file too, and must reproduce the
number independently rather than read about it.
The addition that matters more, because it was learned the hard way: a
reviewer re-derives the author's claims and therefore inherits the
author's SAMPLING. CB-WP-0002's dedup invariant was checked twice --
survey 206/206 groups, then the reviewer independently -- and both used
the main transcript. It is false in the 8-response subagent tree neither
looked at. Two independent verifications, one shared blind spot.
Rule: the reviewer re-derives on a different sample than the author
used, and where only one sample exists, says so rather than reporting
a clean verify.
Also recorded: what review demonstrably DOES do. $0.66 and $1.11 across
two passes, ~1% of each, both finding approval-blocking defects. Cost is
not a reason to skip it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A blanket "no retargeting in the measuring commit" rule would have been
wrong. CB-WP-0002 moved AC-1 three times in exactly that shape and every
move was correct -- each time a new instrument disproved the old figure.
Four legitimate corrections would have been forbidden to catch one bad
retarget.
The test is mechanical rather than a statement of intent:
correction the target moves and the implementation does not; legal in
the same commit provided the instrument's output is there
retarget the same commit changes both the target and the code the
target measures; requires an ADR stating why the new target
binds on future work
Applied retroactively: AM-4a/AM-4b are UNRATIFIED. They were set after
seeing the measurement, in the commit that produced it, with the
implementation changing too -- a retarget by this test. make dep-weight
is currently enforcing a target no reviewed decision stands behind.
Recorded as an open item; ratifying or changing them is a maintainer
decision, not an implementer's.
Also: specs/InnerLoop.md split into InnerLoop.md (process) and
InnerLoopReference.md (rubric, template, rules, definition of done).
Not a stylistic choice -- `make loop-lint` failed on the commit that
pushed the file to 407 lines against its own ~400 limit. The gate added
this morning to make that rule executable caught its own author within
the hour, which is the cheapest possible demonstration that it works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mechanism never fired. Two rolls across two workplans (9, then 2)
against ~0.2 expected firings at d10. At ~2 tier decisions per workplan
it would take roughly twenty workplans to see four overrides -- a rate
that prevents its own evaluation, which T06 named as the one option to
reject.
Raised to d4 for the next 12 tier declarations, then kept, reverted to
d10, or deleted on the evidence. Expected ~3 firings.
Cost stated up front rather than discovered: a chaos-L override on
S-shaped work buys a full survey + review + ADR, and the comparable
measured figure is CB-WP-0001 T03 at $9.91. The window should cost
$20-30. Cheaper than carrying an unevaluated ritual indefinitely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The price sheet had two defects of one shape -- a schema that could not
hold the fact it needed, the same criticism the cost survey levelled at
the State Hub.
CA-16 time-boxed rates are DATA. Sonnet's intro price lived in a
`# intro ...` comment and was invisible to the collector that
reads the file. Now promo_input/promo_output/promo_until,
applied per response at its own timestamp.
CA-17 the 90-day staleness rule was prose in MetricsAndScenarios 1a
that every M-D2-CST verdict silently inherited. Now `recorded`
+ `max_age_days` in the sheet, and a stale sheet ABORTS.
Both are exercised by make cost-test: the promo rate must apply before
2026-08-31 and lapse after, and a 102-day-old sheet must trip.
Applying CA-16 moved AC-1 from $93.32 to $93.15 -- the $0.17 CB-EV-0002
predicted, now collected rather than noted. That is a legitimate
retarget under T07's distinction: the instrument disproved the target,
and its output is in this commit. The number has now been stated five
times ($248.46, $92.21, $92.87, $93.32, $93.15), each correction from a
different mechanism.
Evidence tables regenerated from the tool rather than hand-patched,
per CA-15 -- which is the rule that exists because hand-typed tables
were the only thing the adversarial review found wrong.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 8k/10k per-task token budget was never referenced or enforced and
T08 blew past it silently. Replaced with a budget that can actually
fire.
The design constraint is the interesting part: per-task cost needs the
commit that CLOSES the task, so a per-task budget is unavoidably
retrospective -- it can only report a breach after the money is spent,
which is the dead-policy failure again. What IS observable mid-task is
spend since the last commit, because the transcript is append-live. So
the budget binds on the open remainder.
CB-01 budget = USD since the last commit, via `make cost-budget`
CB-02 soft $10.00 (state progress, decide), hard $22.00 (stop)
Calibrated on the 32 non-empty commit intervals of CB-WP-0001: p50
$1.40, p90 $9.36, max $10.80. Soft sits just below the observed maximum
-- it would have fired exactly once on the calibration pass. Hard is ~2x
the observed max, a value never reached in 32 intervals, so reaching it
means the session is doing something the data has no example of.
Both thresholds are set ABOVE every observed value, so they bind on
future work rather than ratifying present work -- the distinction T07
is about.
Stated limit: it is a command, not a daemon. An agent that never runs it
gets no signal, which is the dead-policy failure one level up. Mitigated
only by being free to run and on the one command surface.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The task's original premise was wrong and is recorded rather than
deleted. It was written to prescribe one task per session; measurement
says the variable is context, not turn count.
Measured:
- compaction cut context 27x (542,991 -> 19,974) and cost/turn 3.1x;
the 202 turns after C1 cost less than half the 136 before it
- a turn costs $0.010 at 20k context and $0.270 at 540k
- break-even for a compaction is 2-11 turns, so: compact whenever
context exceeds ~300k and work remains
- a fresh session is NOT free -- cold start floors at ~51k and must
then re-read the artifacts a compact summary already holds (~66k).
Prefer compaction to continue work; prefer a fresh session when the
task changes, because then prior context is pure overhead.
cb-cost now emits SH-1..SH-3 so the targets come from the instrument
rather than from analysis, per InnerLoop v1.1. All three are UNMET
(mean context 232,982 vs 200,000; p90 492,042 vs 300,000; batching 7.8%
vs 20%) and are reported unmet rather than retargeted -- retargeting in
the commit that first measures is precisely what T07 exists to prevent.
Eighth error instance found while writing this: CB-WP-0001's claim that
"0 of 330 tool calls were batched" is wrong. 330 was the count of
single-call responses, not the total; 31 responses batched, covering 76
calls. It was carried into this workplan unverified. Trusted-arithmetic
class -- the one the T01 audit flagged as having no executable defence,
confirming that finding within hours of making it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Answers the question the workplan posed: did v1.0's "every metric names
its instrument" rule stop a metric being written without a working
instrument? No. AC-1 named `cb-cost --pin` before that tool existed and
set a hand-computed target of $92.87; the tool returned $93.32. The rule
was satisfied completely and the metric was still wrong.
v1.1 adds:
- the instrument must exist and the target must come out of it;
targets are provisional until the tool emits them
- a number inherited from earlier work is re-derived before use as a
target, or cited as unverified
- every reporting tool exposes --self-test, run before the number
- cost is in the definition of done; M-D2-CST may not be uncomputable
The cost of CB-WP-0001 was stated four times before it was right --
$248.46, $92.21, $92.87, $93.32 -- and each correction came from a
different mechanism: re-derivation, adversarial review, and the positive
control. None found more than one. That is the case for keeping all
three.
CB-WP-0003 T10 predicted the next error would be harness-does-nothing.
It was not, twice. Trusted arithmetic over real data would pass a
positive control; and a property verified on 206/206 groups of the main
transcript is false in the 8-response subagent tree that neither the
survey nor the reviewer examined separately. Review structurally cannot
catch the second -- re-deriving on the same sample reproduces the same
blind spot.
Workplan status: done.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- InnerLoop definition-of-done gains a cost row: M-D2-CST may no longer
be recorded uncomputable, and composition must be reported, not only a
total.
- InnerLoop Step 5 gains the --self-test contract: every tool that
reports a number exposes one, and it runs before the number does.
Rationale attached, because the case that motivated it is the one
review cannot catch — survey and reviewer both verified the same large
sample and both missed the small one.
- make cost / cost-test / cost-pin on the one command surface; cost-test
in `make all` and in CI.
- Hub now holds the measured figure for CB-WP-0001: 80.6M in / 323.6k
out against the 401,100 it previously estimated, low by ~200x. The
event states plainly that the hub schema cannot represent the 88% of
cost that is cache, and names `make cost-pin` as the authority.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Normative spec for the cost model (dedup by requestId, per-model and
per-TTL pricing), the attribution contract (ending-at-commit intervals
scoped by sessionId), and the reported shape (composition, not a total).
Every acceptance row names the command that produces its number, per
InnerLoop Step 4. AC-5..AC-8 are the positive control: cb-cost
--self-test must fail on a dedup violation, on zero responses, on a
missing subagent tree, and on 5m cache priced at the 1h rate.
The feasibility check earns its place here: AC-1's $92.87 is reachable
only if CA-06 holds. The earlier $92.21 target was reachable only by a
collector with the exact blind spot the survey documented.
MetricsAndScenarios 1a superseded. Two of its rules struck through
rather than deleted, because both were wrong in instructive ways: "if
the cache split is unknown, count all input at full price" would have
priced 80.5M cache reads at 10x, and "the hub already records tokens"
named as a source what is only ever a lossy sink. M-D2-TOK demoted --
tokens are not comparable across models or cache states.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The loop has now survived CB-WP-0001 end to end and is revised from what
that pass actually taught, recorded in
history/260731-inner-loop-retrospective.md.
Held up: the adversarial review (one blocking and three significant
findings on its single use, resolved with new measurements rather than
argument), the parity-cap rule, the provisional U-item mechanism, and
the ADR gate.
The gap: both serious errors in the pass were measurement errors, and
the loop caught neither. Review reads prose; these were claims about
numbers. In both cases a harness ran successfully while doing no work —
a JS harness timing rejected no-ops, and a Rust benchmark computing
throughput for rounds that never completed.
v1.0 adds:
- Measurement validity: a harness must assert it performed the work it
reports. A number from a run that cannot prove it did the work is void.
- Metric feasibility: every metric names its instrument and is checked
reachable against the contracts in its own spec.
- No silently-ignored input; decisions get commands, not defaults;
scaffolds are exercised or marked; coverage gates that count tags say
so.
- Evidence must state what a comparison does not support.
- The chaos roll is recorded even when it changes nothing, so a
mechanism that never fires is visible rather than assumed.
CB-WP-0001 is complete: 9/9 tasks done.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
