T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
# CB-EV-0001 — GROUND game kernel: acceptance evidence
CB-WP-0013-T02/T03: retire SH-3 as a gate; correct AM-4a and its target
ADR-0008, tier M (survey and ADR merged).
D1 — SH-3 retired as a gate, kept as a diagnostic. Investigating it
found a third defect, deeper than the two this pass was declared on.
Re-deriving batching from the raw transcripts, independently of cb-cost:
CB-WP-0011 pass 54 with tools 0 batched 0.0%
gap -> next decl 16 with tools 6 batched 37.5%
CB-WP-0012 pass 86 with tools 0 batched 0.0%
gap -> next decl 10 with tools 1 batched 10.0%
CB-WP-0013 so far 10 with tools 0 batched 0.0%
Zero batched turns in 150 in-pass responses; 37.5% in one gap, above the
20% floor. Batching needs two calls whose inputs are known at once —
orientation work. Implementation consumes each step's result before the
next. SH-3's window is since the last commit, which during a pass is
always implementation. The metric could not read above ~0% in the window
it was gated on. A floor the window structurally excludes is not a
target.
This pass's own declaration was also wrong: it claimed batching "has got
worse" (7.8-8.6% vs 1.1-6.3%). Differently-placed windows, not different
behaviour. Withdrawn — the same class of error, in the pass written to
correct it.
Not retargeting to match the measurement: the floor was not moved to 6%,
the gate was removed on an argument about what the quantity is worth.
The number is still reported; only the verdict is gone.
D2/D3 — AM-4a counts --edges normal,no-proc-macro: 157,202, not 246,250.
The target moves down with it, 250,000 -> 161,000, so the correction
hands back essentially nothing (headroom 3,750 -> 3,798). Three controls:
the exclusion drops exactly the five expected crates, only removes and
never adds, and is not a no-op.
The DFD gate then caught the follow-on it exists for — three historical
documents carrying live fact tags for a number that had changed. Not
rewritten; untagged, with a supersession banner.
AM-4b is deliberately not corrected: its proc-macro share is unmeasured.
gate-review now reads 0 due, 0 silent, 0 drifted — GATE-REVIEW earns its
first caught entry by forcing SH-3's re-justification, and the registry
has no silent gates left.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 07:30:14 +02:00
> **Superseded measurement (ADR-0008 D2/D3, 2026-08-02).** The AM-4a
> figures below were taken with an instrument that counted proc-macro
> crates — 89,048 lines, 36.2% — which run in the compiler and never
> reach a binary. Corrected, the same tree measures **157,202** against
> a target moved to **161,000**. The numbers below are left as the
> record of what was measured then, and are no longer live facts.
AM-4: gate scenario YAML, retarget on audited source, re-measure
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
Status: **T08 complete. AM-4 remediated and re-measured 2026-07-31.**
Amend CB-EV-0001; add CB-WP-0002 for cost accounting
AM-4: measured what each remediation option actually buys, rather than
leaving one recommendation unquantified. serde_yaml optional removes 6
crates, not 5 — ryu belongs to that group, since serde_json now uses
zmij for floats. Full ladder: -6 to 27, serde_json -4 more to 23,
inlining SHA-256 -8 to 19, inlining ChaCha12 -4 to 15. Only
reimplementing a primitive gets under 20, so the target is unreachable
without undoing K5/K7.
Also records that crate count compares badly across ecosystems, and
offers the alternative the count is a proxy for: 307,317 lines of
third-party source under audit against 3,398 of our own.
AM-12: corrected from "uncomputable" to measured. The refusal to
estimate was right; the claim that no instrument existed was wrong.
Session transcripts carry exact per-message usage including the cache
breakdown. This session cost $248.46 at Fable 5 rates, of which 53% is
cache reads — cost is driven by context size times turn count, not by
output volume. What is still missing is per-task attribution, since
nothing marks task boundaries in a transcript.
CB-WP-0002 makes cost measurable and attributable: survey the
instruments, decide the attribution model by ADR, spec metrics that
include cost composition rather than a bare total, build a collector
whose positive control refuses to emit unreconciled numbers, and prove
it by answering a question that could not be answered before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:26:03 +02:00
Recorded: 2026-07-31. Amended 2026-07-31 — §4 gains measured savings per
remediation option, and §5 corrects AM-12 from "uncomputable" to
measured-at-session-level; see CB-WP-0002.
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
Workplan: CB-WP-0001, task T08
Spec: `specs/GameKernel.md` §4 (AM-1..AM-12)
Baseline: `research/CB-RES-0001-game-kernel.md` , measurements in
`research/CB-RES-0001-harness/boardgame-io/results-260731.json`
Machine: WSL2, Linux 6.18.33.2-microsoft-standard-WSL2, rustc 1.97.1,
`--release` , Criterion 1s warm-up / 3s measurement.
## 1. Scoreboard
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
> **Corrected 2026-07-31 (CB-WP-0005 T03, per ADR-0005 §4).** Four rows
> below were wrong or misleading as originally committed. They are
> corrected **in place with this note**, not silently edited — the
> correction trail is the artifact. What changed, and why, is in §1a.
The **Enforced** column is new and is the point of the correction. It
carries M-D1-MUT (`make mutation-check` ): does anything actually fail when
the property is false? A row can be *measured* and still enforce nothing.
| Metric | Target | Measured | Verdict | Enforced |
|---|---|---|---|---|
| AM-1 rule coverage (GR only) | 100% of GR-rules | 58/58 (100%) | **met** | **yes** |
| AM-1b link, ground | 58 claimed rules named in the aggregate | 49/58 | **unmet** | reported |
| AM-1b link, kernel | 18 K-rules named in source | 15/18 | **unmet** | reported until 2026-08-31 |
CB-WP-0013-T02/T03: retire SH-3 as a gate; correct AM-4a and its target
ADR-0008, tier M (survey and ADR merged).
D1 — SH-3 retired as a gate, kept as a diagnostic. Investigating it
found a third defect, deeper than the two this pass was declared on.
Re-deriving batching from the raw transcripts, independently of cb-cost:
CB-WP-0011 pass 54 with tools 0 batched 0.0%
gap -> next decl 16 with tools 6 batched 37.5%
CB-WP-0012 pass 86 with tools 0 batched 0.0%
gap -> next decl 10 with tools 1 batched 10.0%
CB-WP-0013 so far 10 with tools 0 batched 0.0%
Zero batched turns in 150 in-pass responses; 37.5% in one gap, above the
20% floor. Batching needs two calls whose inputs are known at once —
orientation work. Implementation consumes each step's result before the
next. SH-3's window is since the last commit, which during a pass is
always implementation. The metric could not read above ~0% in the window
it was gated on. A floor the window structurally excludes is not a
target.
This pass's own declaration was also wrong: it claimed batching "has got
worse" (7.8-8.6% vs 1.1-6.3%). Differently-placed windows, not different
behaviour. Withdrawn — the same class of error, in the pass written to
correct it.
Not retargeting to match the measurement: the floor was not moved to 6%,
the gate was removed on an argument about what the quantity is worth.
The number is still reported; only the verdict is gone.
D2/D3 — AM-4a counts --edges normal,no-proc-macro: 157,202, not 246,250.
The target moves down with it, 250,000 -> 161,000, so the correction
hands back essentially nothing (headroom 3,750 -> 3,798). Three controls:
the exclusion drops exactly the five expected crates, only removes and
never adds, and is not a no-op.
The DFD gate then caught the follow-on it exists for — three historical
documents carrying live fact tags for a number that had changed. Not
rewritten; untagged, with a supersession banner.
AM-4b is deliberately not corrected: its proc-macro share is unmeasured.
gate-review now reads 0 due, 0 silent, 0 drifted — GATE-REVIEW earns its
first caught entry by forcing SH-3's re-justification, and the registry
has no silent gates left.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 07:30:14 +02:00
| AM-4a dep weight, shipped runtime | ≤250,000 third-party lines | 246,250 (23 crates) | **met** | **yes** | <!-- historical: measured under the pre-ADR-0008 instrument; not a live fact -->
CB-WP-0019 T01/T02: AM-4b asks what a contributor acquires
The two AM-4 budgets had the SAME scope -- one package, no dev edges --
while claiming to bound different things. AM-4b now measures the
workspace with dev edges: 57 crates / 725,258 lines where it read 29 /
317,021, having been blind to 28 crates and 408,237 lines, more source
than its own target.
Target 745,000, ~2.7% of room -- the same margin ADR-0008 D3 gave AM-4a,
applied to a number that grew because the instrument was repaired, not
because anything was added. The target moved to fit the measurement.
T02: proc-macros are COUNTED here and excluded from AM-4a, on purpose.
AM-4a asks what ships and a proc-macro never ships. AM-4b asks what is
acquired, and ADR-0007 D3's acquisition rule counts what the build
fetches -- 'it does not ship' is no answer to 'we downloaded it'. When
the rules disagree, the question each budget asks decides. Measured
share 109,585 lines / 15.1% against AM-4a's 36.2%, so ADR-0008 D2's
refusal to borrow the ratio was right by more than a factor of two.
Caught by this project's own earlier work twice: the mutation
find-string went stale and --self-test reported it BUILD-FREE (the check
CB-WP-0015 added after AM-4a's rotted for two passes), then the DFD gate
caught facts.toml carrying the old numbers.
CB-EV-0001 and ADR-0004 carried live fact: tags on historical readings.
A dated record asserting a CURRENT value is a category error, so those
occurrences are marked as-measured instead of retro-edited, and ADR-0004
gains a supersession note.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:04:54 +02:00
| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** | **yes** | <!-- as-measured 2026-07-31; AM-4b was rescoped by CB-WP-0019, see GameKernel 5c -->
CB-WP-0006 T05: K9's assertion, K11's format, and the AM-11 suites
K11 is implemented: crates/cb-events/src/store.rs, magic + version header,
4-byte little-endian length prefix, append-only. Reimplemented not
assimilated per ADR-0005 §2 — no new dependency, and AM-4a/AM-4b are
unchanged at 246,250 / 317,021 because nothing entered the graph.
The operative clause is "detected", so corruption is tested rather than
assumed: a tail short by one byte, a half-written length prefix, a length
prefix corrupted to claim more than the file holds, foreign magic, and a
future format version are each rejected with a distinct error. A reader
that accepts a truncated tail is worse than no format, because it silently
returns a short history that looks complete.
AM-11 is earned. LogStore has two impls — MemLogStore and FileLogStore —
driven through ONE conformance(). The trait carries raw/set_raw precisely
so the corruption controls live in the shared suite: a format contract
that only one impl enforces is not a contract. The same shape is
retro-fitted to KernelRng, which is what AM-11 actually names: ChaChaRng
and NullRng now pass one suite asserting bounds, draw(1) == 0, determinism
across fresh instances, and shuffle preserving the multiset. They were
previously exercised by two separate tests, which is why "met, narrow" was
never earned and ADR-0005 §4 downgraded it.
K9 gets the assertion it did not have: snapshot at seq N + events N+1..M
must equal the from-genesis fold, hash-compared, on GroundState,
single-seed on purpose — AM-7's probe folds a multi-seed log, which is not
a replay of anything, and that defect is not repeated. Two positive
controls: the log must exceed 50 events, and the mid-log snapshot must
differ from the end state or "apply the remainder" is vacuous.
Proof it works: the exact mutation that SURVIVED in CB-WP-0005 — making
Snapshot::take discard its EventSeq — now fails on the K9 assertion.
AM-11's mutation breaks NullRng::draw to return its bound and the shared
suite fails. That is what M-D4-SWAP claims — either impl substitutable —
and exactly what two separate per-impl tests could never demonstrate.
M-D1-MUT: 7 -> 8 of 14. CB-EV-0001's scoreboard is refreshed: AM-2, AM-5
and AM-9 added, AM-6 moved to enforced, and the headline total corrected
from 4 to 8 — it had gone stale inside the same workplan that produced it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:50:52 +02:00
| AM-6 throughput | ≥100,000 events/s | 2,017,009 events/s (`make am6` ) | **met, 20.2× | **yes** — CB-WP-0006 T01 |
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
| AM-7 scaling | ≥0.9× × × **met** | **no** — no code computes the ratio |
| AM-7 replay, timing | 100k events ≤5s | 2.18 ms (CI 2.14– **met, 2,290× | **yes** |
CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had
no flag parsing at all, so --replay had nowhere to go.
The bundle is manifest + commands.log + initial.snapshot + expected.yaml,
dev-only behind the scenarios feature and charged to AM-4b. The command
stream goes through the K11 framing built in T05, so a truncated bundle is
detected rather than replayed short — the two tasks compose rather than
duplicating.
The reviewer's D2 correction was real: this was not "a directory of four
files". Pass carried only the end state, RunOutcome::Failed was a
formatted String, and scenario.rs created an EventLog, appended to it and
never read it. All three had to change.
The first round trip failed to reproduce, and the cause is worth keeping:
state_hash_hex over a serde_json::Value is a different canonical form than
over the typed aggregate — Value's map is key-sorted, a struct serializes
in declaration order. The bundle was written with one basis and verified
with the other. A round trip written to recompute its own comparison value
would have PASSED this bug; it failed because the recorded hash came from
the producing process, which is control 2's entire purpose.
make replay-test implements ADR-0005 §6's four controls, 14/14: a
committed deliberately-failing fixture outside the corpus with covers: []
so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash
must fail; a log short by one byte and a corrupted length prefix must be
rejected; and a mutated manifest seed must fail — which bites only because
replay re-derives the initial state from seed+setup and checks it against
the recorded snapshot, since restoring from the snapshot alone would leave
the seed inert. Plus a control on the controls: the bundle must still
replay after every mutation is reverted.
AM-7's hash-identical clause is re-earned. The probe records a hash per
per-game segment and replays each from its own genesis; folding from the
wrong seed now fails. That is the clause ADR-0005 §4 withdrew as
mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so
AM-7 stays PARTIAL — reported, not rounded up.
Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's
copy of that number going stale, on a number that moved the same hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 11:05:37 +02:00
| AM-7 replay, hash-identical | bit-identical fold | per-segment replay reproduces each recorded hash | **re-earned 2026-08-01** | **yes** — CB-WP-0006 T06 |
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
| AM-8 determinism | zero divergence, 10 replays | 1 distinct hash / 10 runs (one-off probe) | **met, narrow** | **partial** — the runner double-runs; nothing enforces N=10 |
| AM-8 lint | fmt + clippy clean | clean, `-D warnings` | **met** | **yes** |
| AM-10 foreign types | 0 in `cb-*-api` signatures | — | **WITHDRAWN** | **no** — no such crate; population empty |
| AM-10′ `HashMap` /`HashSet` in game state | 0 | **met** | **yes** |
CB-WP-0006 T05: K9's assertion, K11's format, and the AM-11 suites
K11 is implemented: crates/cb-events/src/store.rs, magic + version header,
4-byte little-endian length prefix, append-only. Reimplemented not
assimilated per ADR-0005 §2 — no new dependency, and AM-4a/AM-4b are
unchanged at 246,250 / 317,021 because nothing entered the graph.
The operative clause is "detected", so corruption is tested rather than
assumed: a tail short by one byte, a half-written length prefix, a length
prefix corrupted to claim more than the file holds, foreign magic, and a
future format version are each rejected with a distinct error. A reader
that accepts a truncated tail is worse than no format, because it silently
returns a short history that looks complete.
AM-11 is earned. LogStore has two impls — MemLogStore and FileLogStore —
driven through ONE conformance(). The trait carries raw/set_raw precisely
so the corruption controls live in the shared suite: a format contract
that only one impl enforces is not a contract. The same shape is
retro-fitted to KernelRng, which is what AM-11 actually names: ChaChaRng
and NullRng now pass one suite asserting bounds, draw(1) == 0, determinism
across fresh instances, and shuffle preserving the multiset. They were
previously exercised by two separate tests, which is why "met, narrow" was
never earned and ADR-0005 §4 downgraded it.
K9 gets the assertion it did not have: snapshot at seq N + events N+1..M
must equal the from-genesis fold, hash-compared, on GroundState,
single-seed on purpose — AM-7's probe folds a multi-seed log, which is not
a replay of anything, and that defect is not repeated. Two positive
controls: the log must exceed 50 events, and the mid-log snapshot must
differ from the end state or "apply the remainder" is vacuous.
Proof it works: the exact mutation that SURVIVED in CB-WP-0005 — making
Snapshot::take discard its EventSeq — now fails on the K9 assertion.
AM-11's mutation breaks NullRng::draw to return its bound and the shared
suite fails. That is what M-D4-SWAP claims — either impl substitutable —
and exactly what two separate per-impl tests could never demonstrate.
M-D1-MUT: 7 -> 8 of 14. CB-EV-0001's scoreboard is refreshed: AM-2, AM-5
and AM-9 added, AM-6 moved to enforced, and the headline total corrected
from 4 to 8 — it had gone stale inside the same workplan that produced it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:50:52 +02:00
| AM-11 impl pairs | ≥2 impls under **one conformance suite** | 2 ports, 2 impls each, one shared suite per port | **met 2026-08-01** | **yes** — break one impl and the shared suite fails |
| AM-2 LOC per rule | ≤40 | 27.2 (1,575 impl lines / 58 rules) | **met** | **yes** — CB-WP-0006 T02 |
| AM-3 synthetic workload LOC | ≤50 | — | **blocked** — the artifact has never been built | **no** |
| AM-5 clean release build | ≤60 s on bnt-lap001 | 37.3 s dev / 41.2 s shipped, best of 3, quiet | **met, 1.6× | **no** — spec declares it ungated |
| AM-9 peak RSS | ≤64 MB | 13.4 MB | **met, 4.8× | **yes** — CB-WP-0006 T03 |
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
| AM-12 cost | per-task USD | ** $93.15** pinned, per task via `make cost` | **met** | **yes** | <!-- fact:pinned_total -->
CB-WP-0006 T05: K9's assertion, K11's format, and the AM-11 suites
K11 is implemented: crates/cb-events/src/store.rs, magic + version header,
4-byte little-endian length prefix, append-only. Reimplemented not
assimilated per ADR-0005 §2 — no new dependency, and AM-4a/AM-4b are
unchanged at 246,250 / 317,021 because nothing entered the graph.
The operative clause is "detected", so corruption is tested rather than
assumed: a tail short by one byte, a half-written length prefix, a length
prefix corrupted to claim more than the file holds, foreign magic, and a
future format version are each rejected with a distinct error. A reader
that accepts a truncated tail is worse than no format, because it silently
returns a short history that looks complete.
AM-11 is earned. LogStore has two impls — MemLogStore and FileLogStore —
driven through ONE conformance(). The trait carries raw/set_raw precisely
so the corruption controls live in the shared suite: a format contract
that only one impl enforces is not a contract. The same shape is
retro-fitted to KernelRng, which is what AM-11 actually names: ChaChaRng
and NullRng now pass one suite asserting bounds, draw(1) == 0, determinism
across fresh instances, and shuffle preserving the multiset. They were
previously exercised by two separate tests, which is why "met, narrow" was
never earned and ADR-0005 §4 downgraded it.
K9 gets the assertion it did not have: snapshot at seq N + events N+1..M
must equal the from-genesis fold, hash-compared, on GroundState,
single-seed on purpose — AM-7's probe folds a multi-seed log, which is not
a replay of anything, and that defect is not repeated. Two positive
controls: the log must exceed 50 events, and the mid-log snapshot must
differ from the end state or "apply the remainder" is vacuous.
Proof it works: the exact mutation that SURVIVED in CB-WP-0005 — making
Snapshot::take discard its EventSeq — now fails on the K9 assertion.
AM-11's mutation breaks NullRng::draw to return its bound and the shared
suite fails. That is what M-D4-SWAP claims — either impl substitutable —
and exactly what two separate per-impl tests could never demonstrate.
M-D1-MUT: 7 -> 8 of 14. CB-EV-0001's scoreboard is refreshed: AM-2, AM-5
and AM-9 added, AM-6 moved to enforced, and the headline total corrected
from 4 to 8 — it had gone stale inside the same workplan that produced it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:50:52 +02:00
**M-D1-MUT over the whole acceptance table: 8 of 14 rows enforced**
(`make mutation-check` ), up from 4 when the instrument was first run.
AM-4c is retained in that denominator after its withdrawal, deliberately:
a score improved by deleting the question is not an improvement.
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
CB-WP-0006 T05: K9's assertion, K11's format, and the AM-11 suites
K11 is implemented: crates/cb-events/src/store.rs, magic + version header,
4-byte little-endian length prefix, append-only. Reimplemented not
assimilated per ADR-0005 §2 — no new dependency, and AM-4a/AM-4b are
unchanged at 246,250 / 317,021 because nothing entered the graph.
The operative clause is "detected", so corruption is tested rather than
assumed: a tail short by one byte, a half-written length prefix, a length
prefix corrupted to claim more than the file holds, foreign magic, and a
future format version are each rejected with a distinct error. A reader
that accepts a truncated tail is worse than no format, because it silently
returns a short history that looks complete.
AM-11 is earned. LogStore has two impls — MemLogStore and FileLogStore —
driven through ONE conformance(). The trait carries raw/set_raw precisely
so the corruption controls live in the shared suite: a format contract
that only one impl enforces is not a contract. The same shape is
retro-fitted to KernelRng, which is what AM-11 actually names: ChaChaRng
and NullRng now pass one suite asserting bounds, draw(1) == 0, determinism
across fresh instances, and shuffle preserving the multiset. They were
previously exercised by two separate tests, which is why "met, narrow" was
never earned and ADR-0005 §4 downgraded it.
K9 gets the assertion it did not have: snapshot at seq N + events N+1..M
must equal the from-genesis fold, hash-compared, on GroundState,
single-seed on purpose — AM-7's probe folds a multi-seed log, which is not
a replay of anything, and that defect is not repeated. Two positive
controls: the log must exceed 50 events, and the mid-log snapshot must
differ from the end state or "apply the remainder" is vacuous.
Proof it works: the exact mutation that SURVIVED in CB-WP-0005 — making
Snapshot::take discard its EventSeq — now fails on the K9 assertion.
AM-11's mutation breaks NullRng::draw to return its bound and the shared
suite fails. That is what M-D4-SWAP claims — either impl substitutable —
and exactly what two separate per-impl tests could never demonstrate.
M-D1-MUT: 7 -> 8 of 14. CB-EV-0001's scoreboard is refreshed: AM-2, AM-5
and AM-9 added, AM-6 moved to enforced, and the headline total corrected
from 4 to 8 — it had gone stale inside the same workplan that produced it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:50:52 +02:00
Of the six not enforced: **AM-3** is blocked on an artifact that was never
built; **AM-4c** and **AM-5** cannot fail because the spec declares them
untargeted/ungated; **AM-10** was withdrawn; **AM-7** and **AM-8** are
partial — some clauses live, some inert. §6's note that AM-2/AM-5/AM-9 are
"not reported" was true until CB-WP-0006 and is superseded by the rows
above.
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
### 1a. What was corrected, and why
| row | as committed | corrected to | found by |
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
|---|---|---|---|
CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had
no flag parsing at all, so --replay had nowhere to go.
The bundle is manifest + commands.log + initial.snapshot + expected.yaml,
dev-only behind the scenarios feature and charged to AM-4b. The command
stream goes through the K11 framing built in T05, so a truncated bundle is
detected rather than replayed short — the two tasks compose rather than
duplicating.
The reviewer's D2 correction was real: this was not "a directory of four
files". Pass carried only the end state, RunOutcome::Failed was a
formatted String, and scenario.rs created an EventLog, appended to it and
never read it. All three had to change.
The first round trip failed to reproduce, and the cause is worth keeping:
state_hash_hex over a serde_json::Value is a different canonical form than
over the typed aggregate — Value's map is key-sorted, a struct serializes
in declaration order. The bundle was written with one basis and verified
with the other. A round trip written to recompute its own comparison value
would have PASSED this bug; it failed because the recorded hash came from
the producing process, which is control 2's entire purpose.
make replay-test implements ADR-0005 §6's four controls, 14/14: a
committed deliberately-failing fixture outside the corpus with covers: []
so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash
must fail; a log short by one byte and a corrupted length prefix must be
rejected; and a mutated manifest seed must fail — which bites only because
replay re-derives the initial state from seed+setup and checks it against
the recorded snapshot, since restoring from the snapshot alone would leave
the seed inert. Plus a control on the controls: the bundle must still
replay after every mutation is reverted.
AM-7's hash-identical clause is re-earned. The probe records a hash per
per-game segment and replays each from its own genesis; folding from the
wrong seed now fails. That is the clause ADR-0005 §4 withdrew as
mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so
AM-7 stays PARTIAL — reported, not rounded up.
Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's
copy of that number going stale, on a number that moved the same hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 11:05:37 +02:00
| **AM-7 replay** | `met, 2,290× | split: timing **met** , `hash-identical` **withdrawn** — *re-earned 2026-08-01, CB-WP-0006 T06* | mutation — folding the log from `fresh(999)` instead of `fresh(42)` , an unrelated genesis state, leaves the test green. The hash reaches only a `println!` . It could not be asserted as written anyway: the log spans games seeded 42, 43, 44… so it is not a replay of anything |
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
| **AM-10** | `met` | **withdrawn as written** ; restated as AM-10′ `cb-*-api` crate, so `0 foreign types` was true over an empty set. What was actually measured is a `clippy.toml` deny of `HashMap` /`HashSet` whose stated reason cites **K6 (determinism)** , reported under a **D4 leak** row |
| **AM-11** | `met, narrow` | **unmet** | adversarial review — M-D4-SWAP is a bool over "the same conformance suite"; `grep -rn conformance` over every `.rs` returns one doc comment describing future work, and the RNG pair is exercised by two separate, non-shared tests |
| **AM-12** | `$248.46 session` | ** $93.15** | CB-WP-0002 re-derivation — the original figure double-counted per-line and priced a three-model session at one model's rate. It had been stale in this file since, untagged and therefore invisible to `facts-check` |
| **AM-1b** | *absent from the scoreboard* | **added, both denominators** | adversarial review — `make coverage` printed `49/58` two lines below the `100%` this table carried, and the table kept the flattering half |
Nothing here was found by re-running the command that printed the number.
Three of the five were found by an adversarial reviewer reading the
assertion behind the number, and two of those required a mutation to
settle. That is the loop change recorded in CB-WP-0005 T08.
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
## 2. Throughput and scaling (AM-6, AM-7)
Workload: 3-player GROUND rounds, 7 commands and 13 events per round
(12 on a game's fifth round, where GR-R09 ends the game). Pinned by
`bench_shape` in `games/ground/src/lib.rs` , so a change to the workload
breaks the test rather than silently rescaling the metric.
| Rounds | Throughput (events/s) | vs 5k |
|---|---|---|
| 5,000 | 1,524,200 | 1.00×
| 10,000 | 1,638,200 | 1.07×
| 20,000 | 1,577,000 | 1.03×
| 40,000 | 1,626,000 | 1.07×
| 100,000 | 1,651,400 | 1.08×
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
Replay — folding one growing event log back into state (Criterion
95% CI, low/median/high):
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
| Events | Time (median) | 95% CI | Rate |
|---|---|---|---|
| 10,010 | 184.8 µs | 180.9 –
| 100,072 | 2.183 ms | 2.142 –
**Correction (2026-07-31).** These originally read 465 µs / 4.13 ms and
were taken from the `replay_probe` **test** , not from the benchmark —
because the benchmark did not work. See §2a.
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
### The comparison against boardgame.io, stated carefully
boardgame.io measured **1,930 moves/s at 5,000 moves** , falling to
**870 moves/s at 20,000**, and **did not finish 100,000 moves in 300 s** .
Our figure in the same unit is ~129,000 rounds/s × ** ~903,000
commands/s**, and 100,000 rounds complete in 776 ms.
That is roughly a 400– ×
measurement**. Four differences matter, all favouring us:
1. **Different language and process model.** Rust in-process against
Node.js with immutable state, patch generation and undo history.
2. **Different feature set.** boardgame.io's per-move cost includes
producing client patches and maintaining undo state; the run with
`--disable-undo` still degraded (0.66×
3. **Different workload shape.** Our 5-round games (GR-R09) bound state
size by construction. The boardgame.io harness ran one match with
unbounded history, which is exactly the axis it degraded on.
4. **No network or storage layer** on our side.
Point 3 is the important one and it is why the flat curve in the table
above is *weak evidence on its own* — a game that resets every five
rounds cannot exhibit history-growth degradation. The replay benchmark
is the honest test of that axis, because there the log grows without
bound, and it stays linear (21.5M → 24.2M events/s from 10k to 100k).
**Claim we are willing to defend:** the kernel meets AM-6 and AM-7 with
large margin, and does not degrade as event-log length grows.
**Claim we are not making:** that Clay-Borg is ~450×
boardgame.io" as a like-for-like engine comparison. Per the
InnerLoop parity-cap rule, a cross-runtime ratio this coarse is not a
verdict, it is a direction.
### A measurement error found and corrected
The first run of this benchmark reported **9.3M events/s with a
perfectly flat curve** — a number that would have been reported as a
93×
every round while being attacked; after three rounds P2 sat at Stress 4,
GR-R03 rejected the SUPPORT, the round never completed, and the loop
spun on rejected commands. Throughput was computed as
`rounds × while most rounds produced 2.
Found by a probe test asserting that a round produces events at all.
The benchmark now asserts the per-round event count on every round and
panics rather than measuring a stalled loop. The corrected figure is
**5.6×
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
### 2a. A fourth measurement error, found by enforcing the rule
The replay benchmark committed alongside this evidence was **the broken
version**. A `python3` patch that was supposed to replace its
log-building loop never applied, leaving a sequence that omits `Resolve`
— so `EndRound` was rejected, every round produced no events, and the
`while log.len() < target` loop spun forever. It was never run to
completion; the AM-7 replay numbers were taken from a separate probe
test instead, and the dead benchmark was committed and left hanging.
Found by adding `cargo bench -- --test` to CI, which runs every
benchmark once. That is the fourth instance of one error class in this
project — a harness that appears to work while doing no work — and the
**first one caught by a gate rather than by noticing**.
The replay loop now carries the positive control the round loop already
had: it asserts each round appended events and fails rather than
spinning. Corrected figures are in the table above; both configurations
still clear the AM-7 budget by three orders of magnitude.
The lesson recorded for the loop: writing the positive-control rule into
`specs/InnerLoop.md` did **not** prevent the next instance. Making it a
CI step did. Prose rules do not enforce themselves.
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
## 3. Determinism (AM-8)
- Every scenario runs twice per invocation with the same seed and fails
on state-hash divergence (K8). 21/21 pass.
- Ten consecutive full runs of all 21 scenarios produced **one distinct
output hash**, i.e. zero divergence.
- `cargo fmt --check` and `cargo clippy --workspace --all-targets
-D warnings` are clean.
- `clippy.toml` denies `HashMap` /`HashSet` workspace-wide (K6); the
aggregate holds only ordered collections, so iteration order cannot
vary between runs.
AM-4: gate scenario YAML, retarget on audited source, re-measure
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
## 4. AM-4 — remediated and re-measured
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
AM-4: gate scenario YAML, retarget on audited source, re-measure
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
**Original result: NOT MET, 33 transitive crates against a ≤20 target.**
Resolved by adopting both remediations (maintainer decision,
2026-07-31): `serde_yaml` was made optional, and the metric was
retargeted onto third-party source under audit.
### Re-measurement (`make dep-weight`)
| Configuration | Crates | Third-party LOC | Target | Verdict |
|---|---|---|---|---|
CB-WP-0013-T02/T03: retire SH-3 as a gate; correct AM-4a and its target
ADR-0008, tier M (survey and ADR merged).
D1 — SH-3 retired as a gate, kept as a diagnostic. Investigating it
found a third defect, deeper than the two this pass was declared on.
Re-deriving batching from the raw transcripts, independently of cb-cost:
CB-WP-0011 pass 54 with tools 0 batched 0.0%
gap -> next decl 16 with tools 6 batched 37.5%
CB-WP-0012 pass 86 with tools 0 batched 0.0%
gap -> next decl 10 with tools 1 batched 10.0%
CB-WP-0013 so far 10 with tools 0 batched 0.0%
Zero batched turns in 150 in-pass responses; 37.5% in one gap, above the
20% floor. Batching needs two calls whose inputs are known at once —
orientation work. Implementation consumes each step's result before the
next. SH-3's window is since the last commit, which during a pass is
always implementation. The metric could not read above ~0% in the window
it was gated on. A floor the window structurally excludes is not a
target.
This pass's own declaration was also wrong: it claimed batching "has got
worse" (7.8-8.6% vs 1.1-6.3%). Differently-placed windows, not different
behaviour. Withdrawn — the same class of error, in the pass written to
correct it.
Not retargeting to match the measurement: the floor was not moved to 6%,
the gate was removed on an argument about what the quantity is worth.
The number is still reported; only the verdict is gone.
D2/D3 — AM-4a counts --edges normal,no-proc-macro: 157,202, not 246,250.
The target moves down with it, 250,000 -> 161,000, so the correction
hands back essentially nothing (headroom 3,750 -> 3,798). Three controls:
the exclusion drops exactly the five expected crates, only removes and
never adds, and is not a no-op.
The DFD gate then caught the follow-on it exists for — three historical
documents carrying live fact tags for a number that had changed. Not
rewritten; untagged, with a supersession banner.
AM-4b is deliberately not corrected: its proc-macro share is unmeasured.
gate-review now reads 0 due, 0 silent, 0 drifted — GATE-REVIEW earns its
first caught entry by forcing SH-3's re-justification, and the registry
has no silent gates left.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 07:30:14 +02:00
| Shipped runtime (`--no-default-features` ) | 23 | 246,250 | ≤250,000 | **met** | <!-- historical: measured under the pre-ADR-0008 instrument; not a live fact -->
CB-WP-0019 T01/T02: AM-4b asks what a contributor acquires
The two AM-4 budgets had the SAME scope -- one package, no dev edges --
while claiming to bound different things. AM-4b now measures the
workspace with dev edges: 57 crates / 725,258 lines where it read 29 /
317,021, having been blind to 28 crates and 408,237 lines, more source
than its own target.
Target 745,000, ~2.7% of room -- the same margin ADR-0008 D3 gave AM-4a,
applied to a number that grew because the instrument was repaired, not
because anything was added. The target moved to fit the measurement.
T02: proc-macros are COUNTED here and excluded from AM-4a, on purpose.
AM-4a asks what ships and a proc-macro never ships. AM-4b asks what is
acquired, and ADR-0007 D3's acquisition rule counts what the build
fetches -- 'it does not ship' is no answer to 'we downloaded it'. When
the rules disagree, the question each budget asks decides. Measured
share 109,585 lines / 15.1% against AM-4a's 36.2%, so ADR-0008 D2's
refusal to borrow the ratio was right by more than a factor of two.
Caught by this project's own earlier work twice: the mutation
find-string went stale and --self-test reported it BUILD-FREE (the check
CB-WP-0015 added after AM-4a's rotted for two passes), then the DFD gate
caught facts.toml carrying the old numbers.
CB-EV-0001 and ADR-0004 carried live fact: tags on historical readings.
A dated record asserting a CURRENT value is a category error, so those
occurrences are marked as-measured instead of retro-edited, and ADR-0004
gains a supersession note.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:04:54 +02:00
| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** | <!-- as-measured 2026-07-31; AM-4b was rescoped by CB-WP-0019, see GameKernel 5c -->
AM-4: gate scenario YAML, retarget on audited source, re-measure
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
| Our own source | — | 3,408 | — | — |
Scenario tooling costs **70,771 lines that a shipped game never
compiles**. That split is the substantive result: the single number
previously reported conflated a runtime concern with a test concern.
**What actually changed in the build.** `cb-game-runtime` gained a
`scenarios` feature carrying `serde_yaml` ; the scenario module, the
`ScenarioGame` impl and the string parsers behind it are `#[cfg]` -gated.
Both configurations compile and lint clean under `-D warnings` .
One trap worth recording: setting `default-features = false` on a
*member* dependency is silently ignored when the workspace dependency
does not specify it, so the first attempt gated nothing while appearing
to work — `cargo tree` still showed all six YAML crates. The fix was
setting `default-features = false` on the workspace dependency itself,
with `cb-sim` opting into `scenarios` explicitly. This is exactly the
class of error InnerLoop v1.0's positive-control rule targets: the build
succeeded and the feature flag looked applied. It was caught by checking
the dependency graph rather than trusting that the edit had worked.
### Why the target moved, and why that is not moving the goalposts
The ≤20 crate target was retired for two measured reasons, both
recorded before the decision was taken:
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
Attribution:
| Group | Crates | Count |
|---|---|---|
| `sha2` (K7 state hashing) | sha2, digest, block-buffer, crypto-common, generic-array, typenum, cpufeatures, cfg-if | 8 |
| serde derive chain | serde_derive, proc-macro2, quote, syn, unicode-ident | 5 |
Amend CB-EV-0001; add CB-WP-0002 for cost accounting
AM-4: measured what each remediation option actually buys, rather than
leaving one recommendation unquantified. serde_yaml optional removes 6
crates, not 5 — ryu belongs to that group, since serde_json now uses
zmij for floats. Full ladder: -6 to 27, serde_json -4 more to 23,
inlining SHA-256 -8 to 19, inlining ChaCha12 -4 to 15. Only
reimplementing a primitive gets under 20, so the target is unreachable
without undoing K5/K7.
Also records that crate count compares badly across ecosystems, and
offers the alternative the count is a proxy for: 307,317 lines of
third-party source under audit against 3,398 of our own.
AM-12: corrected from "uncomputable" to measured. The refusal to
estimate was right; the claim that no instrument existed was wrong.
Session transcripts carry exact per-message usage including the cache
breakdown. This session cost $248.46 at Fable 5 rates, of which 53% is
cache reads — cost is driven by context size times turn count, not by
output volume. What is still missing is per-task attribution, since
nothing marks task boundaries in a transcript.
CB-WP-0002 makes cost measurable and attributable: survey the
instruments, decide the attribution model by ADR, spec metrics that
include cost composition rather than a bare total, build a collector
whose positive control refuses to emit unreconciled numbers, and prove
it by answering a question that could not be answered before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:26:03 +02:00
| serde runtime + json | serde, serde_core, serde_json, itoa, memchr, zmij | 6 |
| `serde_yaml` (scenario files only) | serde_yaml, unsafe-libyaml, indexmap, hashbrown, equivalent, ryu | 6 |
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
| `rand_chacha` (K5 seeded RNG) | rand_chacha, rand_core, ppv-lite86, zerocopy | 4 |
| Clay-Borg crates | cb-kernel, cb-events, cb-game-runtime, games-ground | 4 |
AM-4: gate scenario YAML, retarget on audited source, re-measure
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
1. **It was unreachable without undoing the spec's own contracts.**
Measured ladder: `serde_yaml` optional −
`serde_json` − −
−
K5 or K7 requires — trading an audited implementation for a
scoreboard number.
2. **Crate count does not compare across ecosystems.** Rust splits
crates far more finely than npm. The same granularity difference made
"33 vs 120 npm packages" flatter us *and* made ≤20 punish us.
Third-party source under audit is what the count was proxying for, is
comparable across ecosystems, and cannot be gamed by granularity. The
new targets are set at roughly the current measurement plus headroom,
so they bind on future growth rather than retroactively passing
something that failed: adding another `serde_yaml` -sized dependency to
the shipped runtime would breach AM-4a.
What we did **not** do: hand-roll SHA-256 or ChaCha to win a count.
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
## 5. Cost log (AM-12)
Per `specs/MetricsAndScenarios.md` §1a. Model: Claude Fable 5, at
`benchmarks/baselines/model-prices.toml` rates ($10/$50 per MTok).
| Task | Model | Iterations | Notes |
|---|---|---|---|
Amend CB-EV-0001; add CB-WP-0002 for cost accounting
AM-4: measured what each remediation option actually buys, rather than
leaving one recommendation unquantified. serde_yaml optional removes 6
crates, not 5 — ryu belongs to that group, since serde_json now uses
zmij for floats. Full ladder: -6 to 27, serde_json -4 more to 23,
inlining SHA-256 -8 to 19, inlining ChaCha12 -4 to 15. Only
reimplementing a primitive gets under 20, so the target is unreachable
without undoing K5/K7.
Also records that crate count compares badly across ecosystems, and
offers the alternative the count is a proxy for: 307,317 lines of
third-party source under audit against 3,398 of our own.
AM-12: corrected from "uncomputable" to measured. The refusal to
estimate was right; the claim that no instrument existed was wrong.
Session transcripts carry exact per-message usage including the cache
breakdown. This session cost $248.46 at Fable 5 rates, of which 53% is
cache reads — cost is driven by context size times turn count, not by
output volume. What is still missing is per-task attribution, since
nothing marks task boundaries in a transcript.
CB-WP-0002 makes cost measurable and attributable: survey the
instruments, decide the attribution model by ADR, spec metrics that
include cost composition rather than a bare total, build a collector
whose positive control refuses to emit unreconciled numbers, and prove
it by answering a question that could not be answered before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:26:03 +02:00
| CB-WP-0001 (whole session, T01–
**Correction (2026-07-31).** This section originally recorded AM-12 as
*uncomputable*. That was wrong. The declining to estimate was right; the
conclusion that no instrument existed was not. Every session transcript
(`~/.claude/projects/<slug>/<session>.jsonl` ) carries exact per-message
`usage` including the cache breakdown. Read for this session:
| Component | Tokens | Cost (Fable 5) |
|---|---|---|
| Output | 585,528 | $29.28 |
| Cache read | 131,863,164 | $131.86 |
| Cache write (1h) | 4,365,668 | $87.31 |
| Input | 1,090 | $0.01 |
| **Session total** | | ** $248.46** (~$124 on Opus 5) |
**53% of the cost is cache reads**, not output. Cost in an agentic loop
is driven by context size ×
metric would have surfaced.
Still missing is *attribution* : this is a whole-session figure, not a
per-task one, because nothing marks task boundaries in the transcript.
That is what CB-WP-0002 is for. The AM-12 row above should be read as
"session-level cost measured; per-task attribution pending CB-WP-0002".
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
## 6. Metrics not reported
- **AM-2, AM-3, AM-5** — specification-quality metrics that need a
second capability to compare against; a single data point is not a
measurement.
- **AM-9 (≤64MB)** — not instrumented. The aggregate is a few KB and
the largest log measured here is 100k events, so the budget is very
unlikely to bind, but "unlikely" is not "measured" and it is left
unclaimed.
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
- **AM-11** — ~~the `KernelRng` null/reference pair exists and is
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
exercised. It is the only port with a pair so far, so the metric is
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
met narrowly and will mean more once storage has one.~~
**Corrected 2026-07-31:** the pair exists; the **conformance suite does
CB-WP-0006 T05: K9's assertion, K11's format, and the AM-11 suites
K11 is implemented: crates/cb-events/src/store.rs, magic + version header,
4-byte little-endian length prefix, append-only. Reimplemented not
assimilated per ADR-0005 §2 — no new dependency, and AM-4a/AM-4b are
unchanged at 246,250 / 317,021 because nothing entered the graph.
The operative clause is "detected", so corruption is tested rather than
assumed: a tail short by one byte, a half-written length prefix, a length
prefix corrupted to claim more than the file holds, foreign magic, and a
future format version are each rejected with a distinct error. A reader
that accepts a truncated tail is worse than no format, because it silently
returns a short history that looks complete.
AM-11 is earned. LogStore has two impls — MemLogStore and FileLogStore —
driven through ONE conformance(). The trait carries raw/set_raw precisely
so the corruption controls live in the shared suite: a format contract
that only one impl enforces is not a contract. The same shape is
retro-fitted to KernelRng, which is what AM-11 actually names: ChaChaRng
and NullRng now pass one suite asserting bounds, draw(1) == 0, determinism
across fresh instances, and shuffle preserving the multiset. They were
previously exercised by two separate tests, which is why "met, narrow" was
never earned and ADR-0005 §4 downgraded it.
K9 gets the assertion it did not have: snapshot at seq N + events N+1..M
must equal the from-genesis fold, hash-compared, on GroundState,
single-seed on purpose — AM-7's probe folds a multi-seed log, which is not
a replay of anything, and that defect is not repeated. Two positive
controls: the log must exceed 50 events, and the mid-log snapshot must
differ from the end state or "apply the remainder" is vacuous.
Proof it works: the exact mutation that SURVIVED in CB-WP-0005 — making
Snapshot::take discard its EventSeq — now fails on the K9 assertion.
AM-11's mutation breaks NullRng::draw to return its bound and the shared
suite fails. That is what M-D4-SWAP claims — either impl substitutable —
and exactly what two separate per-impl tests could never demonstrate.
M-D1-MUT: 7 -> 8 of 14. CB-EV-0001's scoreboard is refreshed: AM-2, AM-5
and AM-9 added, AM-6 moved to enforced, and the headline total corrected
from 4 to 8 — it had gone stale inside the same workplan that produced it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:50:52 +02:00
not**. **Resolved 2026-08-01 (CB-WP-0006 T05):** `cb_kernel::rng::conformance`
drives `ChaChaRng` and `NullRng` ; `cb_events::store::conformance` drives
`MemLogStore` and `FileLogStore` . One suite per port, both impls through
it. AM-11 is **met** and mutation-verified — breaking `NullRng::draw` to
return its bound fails the shared suite. The original text follows.
<!-- historical --> M-D4-SWAP is a bool over impls "passing the same conformance
CB-WP-0005 T03: correct the record, and defer Phase C
Four verdicts in evidence/CB-EV-0001 corrected in place with a dated
note, per ADR-0005 §4: AM-7 replay split (timing met, hash-identical
withdrawn), AM-10 withdrawn as written and restated as AM-10' (the K6
determinism lint it actually measured), AM-11 downgraded to unmet, and
AM-1b added to the scoreboard it was missing from.
The scoreboard gains an Enforced column carrying M-D1-MUT, because a row
can be measured and still enforce nothing and the table had no way to say
so. AM-6 now reads "met, 16.5x" alongside "not enforced — nothing
compares any number to 100,000".
A fifth correction surfaced that ADR-0005 did not list: AM-12 still read
$248.46, the figure CB-WP-0002 disproved and corrected to $93.15 four
workplans ago. It was stale in the evidence file ever since — untagged,
and therefore invisible to facts-check. Now tagged. A duplicated-fact
instance that survived the gate built to catch duplicated facts, because
that gate only checks copies that opted in. Recorded for T07.
GameKernel §5 carries the AM-10 withdrawal and AM-11 downgrade inline so
a reader of the spec cannot reach the old claim.
Phase C is deferred before starting, per the stop condition T02 wrote and
the maintainer's decision. It is scoped to five rules; the measurement
says eight acceptance rows have no instrument at all. Building it as
written would proceed on a diagnosis the instrument had just
contradicted. T04-T06 stay in the file with their analysis intact and
move to CB-WP-0006.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:54:29 +02:00
suite", and the two impls are exercised by two separate, non-shared
tests — there is no `fn conformance<R: KernelRng>(…)` that both are
driven through. `grep -rn conformance` over every `.rs` returns a single
doc comment describing future work. **AM-11 is unmet** , and was never
earned. Building the suite is deferred to the re-planned pass, not to
this one.
T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.
Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.
Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.
AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.
A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.
The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00
## 7. Rules implemented under a provisional default
Ten U-items in `specs/GroundRules.md` carry PROVISIONAL defaults. Those
realized here are U2 (clamp on every application), U3 (DENY with no
legal target is a no-op that still advances), U4 (deck reshuffle), U5
(REVERSE owner relief applies whether or not the Reverse was rejected)
and U8 (GROUND—OU cancellation precedes Protection).
One further ambiguity was found during T08 and is **not** in the U-list:
**GR-E02's "successes"** is undefined in dataset 0.1. It is implemented
as the count of claimed Problems. Both scoring scenarios are marked
`provisional: true` .
All provisional behaviour lives behind named functions and is covered by
scenarios tagged `provisional: true` , so a ground-game ruling flips a
scenario rather than the kernel (K16). **Action for ground-game:** rule
on the ten U-items and on GR-E02's "successes".
