2026-07-31 — adversarial challenge: CB-RES-0001 (T04)

Verdict: approvable with corrections — the measured numbers are real and the determinism/dependency claims verify against the harness, but the survey's central architectural claim about boardgame.io's superlinear degradation is contradicted by a documented tuning option the harness never tested, the memory methodology is invalid, and at least two squarely in-scope candidates (OpenSpiel, Ludii) are omitted.

Reviewed cold: only research/CB-RES-0001-game-kernel.md plus the harness under research/CB-RES-0001-harness/boardgame-io/ (bench.js, results-260731.json, package.json, and the installed boardgame.io@0.50.2 source in node_modules).

Blocking

B1. "Architectural, not a tuning artifact" is asserted, untested, and likely false as stated

Survey text: "Per-move cost grows with history length (log accumulation + state pipeline): throughput halves as move count doubles — superlinear total cost. This is architectural (unbounded redux log per client), not a tuning artifact." This attribution underwrites the D3-scaling baseline row (0.45×), the "boardgame.io is slow and degrades" verdict, and plausibly the "did not finish in 300s" @100k row.

I inspected the installed library. boardgame.io v0.50.2 has a documented game option disableUndo (node_modules/boardgame.io/dist/cjs/initialize-648ccd94.js:49, server.js:1912). Without it, every move executes _undo: [...state._undo, undoEntry] (reducer-6f7cf6b0.js:853) — an O(n) array spread per move, i.e. O(n²) total, retaining a G/ctx/plugins snapshot per move. That alone is sufficient to produce exactly the observed halving-per-doubling and the RSS growth, and it is switched off by a one-line, documented config flag. The harness (bench.js) never sets disableUndo, never tests the bare Master, and the survey nowhere acknowledges the flag. The client-side log accumulation the survey blames may or may not remain superlinear with undo disabled — nobody measured it.

The survey's Risks section hedges the measurement surface ("headless client pipeline... a bare server-side master could differ") but not the attribution: "architectural, not a tuning artifact" is precisely a claim that no tuning knob exists, and one demonstrably does. Required correction: rerun the harness with disableUndo: true (and ideally the server Master path). If scaling flattens, the D3-scaling row, the @100k DNF row, and the "degrades" verdict language must be rewritten; if it stays superlinear, the claim becomes defensible and should cite the flag explicitly.

Significant

S1. Memory baseline (RSS 224 MB @5k) is methodologically invalid

bench.js measures process.memoryUsage() (line 101) after the warmup run (5,000 moves, line 92), the main run, and three additional determinism runs of 4,000 moves each (lines 96–98) — five client instances, each with its own unbounded log and undo stack, with no GC or isolation between them. The "D3 memory | RSS @5k moves | 224 MB | measured" baseline row attributes to a 5k-move workload an RSS that actually reflects ~22k moves across five clients plus Node baseline. The per-run rss_mb column in results-260731.json inherits the same flaw. The row should either be re-measured with an isolated single run (report heap delta, not raw RSS) or downgraded from "measured" to "indicative".

S2. Omitted candidates: OpenSpiel and Ludii are squarely in scope; BGA Studio and Colyseus deserve at least a line

The survey's stated scope is "deterministic authoritative state, command → validation → events, simultaneous commit/reveal, hidden information, replay". Two mature, well-known systems sit dead-center in that scope and are absent:

OpenSpiel (DeepMind, C++/Python, active): a games kernel explicitly built for simultaneous-move and imperfect-information games, deterministic, with serializable state and replay via action histories. It is arguably the best existing reference for the commit/reveal + hidden-information combination the verdict claims "none of them do", and as an in-process C++ kernel it is a real D3 comparator that could replace the unsourced event-sourcing row (S3).
Ludii (and the GGP/GDL lineage): general game systems whose entire point is ease of rule specification — the D1 dimension. A survey whose D1 baseline is "~45 LOC of JS" while ignoring the ludeme/GDL literature has a hole exactly where it sets a benchmark-to-beat.

Secondary: Board Game Arena Studio (the dominant commercial turn-based rules framework: server-authoritative state machine, hidden info, full replay — a stronger commercial reference than TTS, which the survey itself calls the anti-model) and Colyseus (active JS authoritative-state server — the natural "active JS alternative" counterweight to inactive boardgame.io). The verdict's surpass claim ("what none of them do") cannot stand on a candidate set that excludes the systems most likely to falsify it, OpenSpiel in particular.

S3. The D3 ceiling row — and the headline ADR benchmark derived from it — rests on an unsourced number

"~10⁵–10⁶ events/s per core is the commonly reported range" carries no citation: no benchmark link, no repo, no paper. The survey is honest that it is "cited/estimated — directional" and caps verdicts at parity — good — but then the headline benchmark-to-beat (≥100,000 events/s, flat scaling) is set from exactly this unsourced range. A benchmark-to-beat whose floor comes from a number nobody can check is soft. Correction: either cite a concrete reproducible source (e.g. a named cqrs-es or fold-benchmark), or measure a minimal Rust comparator, or explicitly label the 100k floor as a stipulated engineering target rather than a competitor-derived baseline.

S4. Timed-loop contamination: `updatePlayerID` inside the measured region

bench.js calls client.updatePlayerID(...) once per move inside the hrtime window (lines 73, 76). That is client-object bookkeeping, not move application, and its cost is unquantified. At 1 call per move it plausibly inflates per-move cost by a constant factor; it does not explain the superlinear shape, but the absolute moves/s baseline (1,930 @5k) is not purely "applied moves". Either hoist per-player clients out of the loop or note the contamination in the provenance column.

Minor

M1 — vacuous baseline row: D1 "rule-to-scenario traceability: 0% — none of the candidates link rules to tests" is trivially beatable by any nonzero effort; as a "benchmark-to-beat" it is gameable to the point of being a free pass. Keep it as an observation, not a baseline row.
M2 — D1 LOC number is generous and gameable: the game definition in bench.js (lines 15–50) is ~36 lines, not ~45; more importantly LOC-of-a-synthetic-game is an easily gamed metric (formatting, helper extraction). Acceptable as a rough bar; should not be the only quantitative D1 row (see S2 — Ludii/GDL is the real D1 literature).
M3 — hidden information claimed as a feature, never exercised: the survey credits boardgame.io's playerView/STRIP_SECRET, but the harness never invokes a playerView, and the synthetic "hands" never deplete (commitAction reads hands[playerID][0] and never removes cards — line 30), so the hidden-info and hand mechanics are dead weight in the workload. The commit/reveal shape is real; the hidden-info cost is unmeasured. Provenance should say so.
M4 — unverifiable maturity citations: "12.4k stars", "Snyk: Inactive" are plausible but not independently checkable from the artifact; low stakes since the ~4-years-since-release fact is checkable from npm.

What was attacked and survived

Determinism claim (measured ✅): verified in bench.js lines 96–98 — same-seed double run compared by SHA-256 of G, different-seed run diverges; results-260731.json records true/true. The seeded RNG is exercised (resolveAll calls random.D6()). Sound.
Dependency/weight numbers: independently re-verified — node_modules contains exactly 120 packages, 37 MB (du -sh), matching the D2 row precisely. Trustworthy.
Warmup: present (line 92, 5,000-move JIT warmup before the timed run). Sound.
Throughput arithmetic: applied-moves accounting (rounds × 4), elapsed_ms, and moves/s are internally consistent between bench.js and results-260731.json; the 2.6 s / 1,930 moves/s @5k row reproduces from the JSON.
Measured-vs-cited labeling: with the exceptions in B1/S1/S4, the survey is unusually disciplined about tagging rows measured/cited/observed and its Risks section preemptively flags the client-pipeline and cited-not-measured caveats. No claim tagged "measured" is fabricated; the failures are attribution (B1) and methodology (S1), not honesty.
Verdict internal consistency: the "×50 headline" arithmetic (100k vs 1,930) checks out, and the verdict correctly refuses to claim a D3 win over the event-sourcing class it hasn't measured. The one overclaim is the B1 "architectural" sentence.

Disposition

Approvable after: (1) rerunning the harness with disableUndo: true and revising or vindicating the superlinear-is-architectural claim; (2) fixing or relabeling the memory row; (3) adding OpenSpiel and Ludii (at minimum as cited candidates, with the verdict's "what none of them do" paragraph re-checked against OpenSpiel specifically); (4) sourcing or relabeling the event-sourcing throughput range. None of these require new candidate-grade engineering; all are within a day's work.

9.4 KiB Raw Blame History Unescape Escape