Commit graph

3 commits

Author SHA1 Message Date
4be6e020ea AM-4: gate scenario YAML, retarget on audited source, re-measure
Some checks failed
ci / check (push) Failing after 3s
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).

Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.

A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.

Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.

Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:

  shipped runtime   23 crates   246,250 lines   target <=250,000  met
  dev toolchain     29 crates   317,021 lines   target <=350,000  met
  own source                      3,408 lines

Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.

Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
8e11fc412e Amend CB-EV-0001; add CB-WP-0002 for cost accounting
Some checks failed
ci / check (push) Failing after 4s
AM-4: measured what each remediation option actually buys, rather than
leaving one recommendation unquantified. serde_yaml optional removes 6
crates, not 5 — ryu belongs to that group, since serde_json now uses
zmij for floats. Full ladder: -6 to 27, serde_json -4 more to 23,
inlining SHA-256 -8 to 19, inlining ChaCha12 -4 to 15. Only
reimplementing a primitive gets under 20, so the target is unreachable
without undoing K5/K7.

Also records that crate count compares badly across ecosystems, and
offers the alternative the count is a proxy for: 307,317 lines of
third-party source under audit against 3,398 of our own.

AM-12: corrected from "uncomputable" to measured. The refusal to
estimate was right; the claim that no instrument existed was wrong.
Session transcripts carry exact per-message usage including the cache
breakdown. This session cost $248.46 at Fable 5 rates, of which 53% is
cache reads — cost is driven by context size times turn count, not by
output volume. What is still missing is per-task attribution, since
nothing marks task boundaries in a transcript.

CB-WP-0002 makes cost measurable and attributable: survey the
instruments, decide the attribution model by ADR, spec metrics that
include cost composition rather than a bare total, build a collector
whose positive control refuses to emit unreconciled numbers, and prove
it by answering a question that could not be answered before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:26:03 +02:00
eb1378e667 T08 complete: benchmarks, determinism evidence, and one missed metric
evidence/CB-EV-0001-game-kernel.md records the acceptance run against
the CB-RES-0001 baseline.

Met: AM-1 rule coverage 58/58; AM-6 throughput 1.65M events/s against a
100k target; AM-7 scaling 1.08x at 20x workload and a 100k-event replay
in 4.13ms against a 5s budget; AM-8 zero divergence over 10 full runs
with fmt and clippy clean; AM-10 zero foreign collection types.

Not met and reported as such: AM-4 at 33 transitive crates against a
<=20 target. Attribution is in the evidence file. The recommended fix
is making serde_yaml optional (-5, a test-only concern), after which
the remainder is sha2 and rand_chacha, which K5 and K7 require. We are
not hand-rolling crypto primitives to win a dependency count.

AM-12 is recorded as uncomputable: per-task token counts were never
instrumented, and inventing a USD figure would defeat the metric.

A measurement error was found and corrected before publication. The
first benchmark reported 9.3M events/s on a flat curve. The workload
had a player selecting SUPPORT while parked at Stress 4, so GR-R03
rejected it, rounds never completed, and throughput was computed for
rounds that never happened. The bench now asserts the per-round event
count and panics rather than measuring a stalled loop. The corrected
figure is 5.6x lower.

The evidence file states plainly what the boardgame.io comparison does
and does not support: the ~450x command-rate ratio is cross-runtime and
cross-feature-set, so it is a direction, not a verdict, per the
InnerLoop parity-cap rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:09:14 +02:00