clay-borg/history/260731-replay-kernel-coverage-research.md

# 2026-07-31 — how CB-RES-0004 was conducted

The unpolished trail, so a later reader can judge how hard the survey was
actually tested.

## How the finding was reached

Not by looking for it. CB-WP-0004 closed, `make status` reported every
workplan done and every gate green, and the question was what the next
pass should be. `INTENT.md` implementation order stage 0 lists "replay
and scenario tests" among the headless-GROUND deliverables, so the first
check was whether that was done.

Sequence of commands, in order:

```sh
find crates games tools/cb-sim -name '*.rs' | xargs wc -l   # 3,443 lines total
grep -n "replay\|cbreplay\|--replay" specs/GameKernel.md    # K10 found
ls replays/                                                  # no such directory
grep -rn "cbreplay\|--replay" crates tools --include=*.rs    # zero matches
grep -n "pub fn\|pub struct" crates/cb-events/src/lib.rs     # K9 present, K10/K11 absent
grep -rn "File::\|fs::\|write_all\|BufWriter" crates games tools/cb-sim --include=*.rs
```

The last one is the load-bearing check and it is worth stating what it
found: **the only `fs::` call in the entire workspace is
`read_to_string` for scenario YAML.** There is no write path anywhere. A
"durable log format" rule (K11) is being satisfied by a `Vec`.

## What was measured vs cited

**Measured on the tree at `59b73c3`:**

- absence of `.cbreplay` reader/writer — grep, zero matches
- absence of a `--replay` flag — read `tools/cb-sim/src/main.rs` in full;
  it collects `args` and treats every one as a scenario path. There is no
  flag parsing at all, so the flag is not merely missing, there is no
  place to put it
- absence of any file-write path — grep for five different write idioms
- 18 K-rules in `specs/GameKernel.md` (`grep -o "K[0-9]\+" | sort -u`)
- `rule-coverage.py` `RULE_RE = \*\*(GR-[A-Z]+\d+)` against
  `specs/GroundRules.md` only — read the source, not inferred from output

**Cited, not measured:** every row of the state-of-the-art table in §2.
boardgame.io, Rune, rr/Pernosco, Playwright and Kafka/EventStore are
described from prior knowledge. None was run. Per MetricsAndScenarios §3
this caps any comparison verdict at `parity`, and the survey says so.

This is a real weakness of the survey and is not hidden: the *gap*
finding is measured and the *design* recommendation rests on cited
precedent.

## Dead ends

- **Looking for the gap in the scenario corpus first.** 21 scenarios, 58
  GR-rules, 100% covered — the corpus looks complete and says nothing
  about the kernel. Roughly ten minutes were spent reading scenario names
  before it became clear the coverage instrument's *denominator* was the
  question, not its numerator.
- **Assuming AM-7 covered replay.** `evidence/CB-EV-0001` reports
  `AM-7 replay | 100k events ≤5s | 2.18 ms | met, 2,290×`, which reads
  like replay is done and fast. It measures an in-memory fold — the K9
  path. The evidence is honest (it even documents that this benchmark was
  the fourth harness-does-nothing instance) but the row's wording carried
  me past the gap on first reading. Worth recording because a future
  reader will hit the same row.
- **Considering an embedded store (sled/redb) for K11.** Dropped once
  ADR-0004's 1.5% AM-4a headroom was recalled: a new shipped-runtime
  dependency now requires an argued decision, and the format is a
  paragraph.

## The chaos roll

`shuf -i 1-4 -n 1` → **1**. Structural tier L (new capability port;
changes the meaning of a headline metric). No override. Recorded per the
rule that the roll is written down even when it changes nothing. This is
declaration **1** of the 12-declaration d4 calibration window opened by
CB-WP-0003 T06.

## Known weaknesses of this survey, stated before review

1. **All baseline rows are cited.** No competitor was run.
2. **The author found the gap and also proposes the fix**, so the
   framing ("coverage scope is the root cause") is the author's and has
   not been independently attacked.
3. **The predicted outcome — kernel coverage below 100% on first run —
   is unfalsified.** The instrument that would test it has not been
   written yet, which is exactly the shape of claim this project has
   been wrong about before.
4. **The single-sample problem applies.** One repo, one spec pair. The
   claim "an instrument reports a percentage over the sample it was
   pointed at" is general, but the evidence for it here is n=1.