diff --git a/decisions/ADR-0002-game-kernel.md b/decisions/ADR-0002-game-kernel.md new file mode 100644 index 0000000..9fa825a --- /dev/null +++ b/decisions/ADR-0002-game-kernel.md @@ -0,0 +1,70 @@ +# ADR-0002: Game-state kernel — survey approval and implementation decision + +Date: 2026-07-31 +Status: accepted +Capability: game.kernel.authoritative-state +Inputs: research/CB-RES-0001-game-kernel.md (approved), +history/260731-game-kernel-{research,challenge,response}.md + +## Approval (InnerLoop step 2) + +The CB-RES-0001 survey is **approved**. It passed one round of adversarial +review: the challenge found one blocking overclaim (degradation +attribution) and three significant gaps (memory methodology, omitted +candidates OpenSpiel/Ludii, unsourced D3 ceiling); all were resolved with +new measurements or corrections, none overturned the comparative +conclusions. The candidate set is judged adequate; the baselines are +trustworthy within their stated provenance (measured vs cited vs +estimated), with the event-sourcing ceiling explicitly parity-capped. + +**Benchmark-to-beat, named:** boardgame.io 0.50.2 per the survey's +baseline table (throughput ~1,100–1,900 moves/s degrading with history in +every configuration; 120 deps/37 MB; ~36-LOC declarative game definition), +plus the **stipulated engineering target** of ≥100,000 applied events/s +with flat scaling (@100k within 10% of @5k) and bit-identical replay. + +## Decision (InnerLoop step 3): reimplement, assimilating patterns + +Build the Clay-Borg semantic kernel as **our own Rust implementation** +(`cb-kernel` + `cb-events` + `cb-game-runtime`) — not a wrapper around any +surveyed system. Per the Clay-Borg rule, we own the semantics; what we +assimilate from the field is *patterns*, each with a named source: + +| Assimilated pattern | From | Into | +|---|---|---| +| Declarative game definition (setup/moves/phases as data + pure functions) | boardgame.io | game-package rules API | +| Tooling-enforced determinism (not convention) | Rune | seeded RNG service, denied ambient time/random, replay double-run check in the scenario runner | +| Command → validate → events → fold, snapshots bound replay | event-sourcing (cqrs-es pattern) | kernel mutation pipeline | +| Information-state framing for imperfect information | OpenSpiel | per-player projection model | +| Hand-zone / object-visibility semantics | TTS (pattern only) | tabletop layer, later | + +Rejected alternatives: **assimilate boardgame.io** (inactive, JS-bound, +history-degrading in best config — fails D2/D3/D4); **build on OpenSpiel** +(right semantics, wrong layer — research kernel without projection/ +networking/snapshot surface, heavyweight C++/Python boundary); **ECS as +kernel** (bevy_ecs stays in the world layer per the blueprint). + +## Expected advantage per dimension (honest trades) + +- **D3 speed: better.** Flat per-event cost with snapshots; the 100k/s + floor. Declared caveat: part of the margin over boardgame.io is language + choice; the commitment that matters is flat scaling, and the + event-sourcing comparison stays at parity until a Rust comparator is + measured. +- **D2 weight: better.** Target ≤20 transitive crates for the headless + kernel (vs 120 npm packages), measured by M-D2-DEP/M-D2-LOC/M-D2-BLD. +- **D1 specification: parity, aiming better.** Match boardgame.io's + declarative compactness on the synthetic workload (≤~40 LOC equivalent); + surpass via rule-to-scenario traceability (M-D1-COV), which no candidate + has. Declared trade: we do not attempt Ludii/GDL-class generality — one + game family first, canon only after a second concrete use. +- **D4 optionality: better.** Port pattern with null/reference impls from + day one, capability boundary WIT-expressible later, zero foreign types in + canonical interfaces (M-D4-LEAK=0). No surveyed candidate offers this. + +## Gate + +Per InnerLoop, no kernel implementation code existed before this ADR. +Next steps: T05 (GROUND rules spec, derived from ground-game), T06 (kernel +spec with acceptance metrics tied to these baselines), T07 (workspace + +harness), T08 (code loop). diff --git a/history/260731-game-kernel-challenge.md b/history/260731-game-kernel-challenge.md new file mode 100644 index 0000000..816d780 --- /dev/null +++ b/history/260731-game-kernel-challenge.md @@ -0,0 +1,60 @@ +# 2026-07-31 — adversarial challenge: CB-RES-0001 (T04) + +**Verdict: approvable with corrections** — the measured numbers are real and the determinism/dependency claims verify against the harness, but the survey's central architectural claim about boardgame.io's superlinear degradation is contradicted by a documented tuning option the harness never tested, the memory methodology is invalid, and at least two squarely in-scope candidates (OpenSpiel, Ludii) are omitted. + +Reviewed cold: only `research/CB-RES-0001-game-kernel.md` plus the harness under `research/CB-RES-0001-harness/boardgame-io/` (`bench.js`, `results-260731.json`, `package.json`, and the installed `boardgame.io@0.50.2` source in `node_modules`). + +--- + +## Blocking + +### B1. "Architectural, not a tuning artifact" is asserted, untested, and likely false as stated + +Survey text: *"Per-move cost grows with history length (log accumulation + state pipeline): throughput halves as move count doubles — superlinear total cost. This is architectural (unbounded redux log per client), not a tuning artifact."* This attribution underwrites the D3-scaling baseline row (0.45×), the "boardgame.io is slow *and* degrades" verdict, and plausibly the "did not finish in 300s" @100k row. + +I inspected the installed library. boardgame.io v0.50.2 has a documented game option `disableUndo` (`node_modules/boardgame.io/dist/cjs/initialize-648ccd94.js:49`, `server.js:1912`). Without it, every move executes `_undo: [...state._undo, undoEntry]` (`reducer-6f7cf6b0.js:853`) — an O(n) array spread **per move**, i.e. O(n²) total, retaining a G/ctx/plugins snapshot per move. That alone is sufficient to produce exactly the observed halving-per-doubling and the RSS growth, and it is switched off by a one-line, documented config flag. The harness (`bench.js`) never sets `disableUndo`, never tests the bare `Master`, and the survey nowhere acknowledges the flag. The client-side `log` accumulation the survey blames may or may not remain superlinear with undo disabled — nobody measured it. + +The survey's Risks section hedges the *measurement surface* ("headless client pipeline... a bare server-side master could differ") but not the *attribution*: "architectural, not a tuning artifact" is precisely a claim that no tuning knob exists, and one demonstrably does. **Required correction:** rerun the harness with `disableUndo: true` (and ideally the server `Master` path). If scaling flattens, the D3-scaling row, the @100k DNF row, and the "degrades" verdict language must be rewritten; if it stays superlinear, the claim becomes defensible and should cite the flag explicitly. + +## Significant + +### S1. Memory baseline (RSS 224 MB @5k) is methodologically invalid + +`bench.js` measures `process.memoryUsage()` (line 101) **after** the warmup run (5,000 moves, line 92), the main run, *and* three additional determinism runs of 4,000 moves each (lines 96–98) — five client instances, each with its own unbounded log and undo stack, with no GC or isolation between them. The "D3 memory | RSS @5k moves | 224 MB | measured" baseline row attributes to a 5k-move workload an RSS that actually reflects ~22k moves across five clients plus Node baseline. The per-run `rss_mb` column in `results-260731.json` inherits the same flaw. The row should either be re-measured with an isolated single run (report heap delta, not raw RSS) or downgraded from "measured" to "indicative". + +### S2. Omitted candidates: OpenSpiel and Ludii are squarely in scope; BGA Studio and Colyseus deserve at least a line + +The survey's stated scope is "deterministic authoritative state, command → validation → events, simultaneous commit/reveal, hidden information, replay". Two mature, well-known systems sit dead-center in that scope and are absent: + +- **OpenSpiel** (DeepMind, C++/Python, active): a games kernel explicitly built for **simultaneous-move and imperfect-information games**, deterministic, with serializable state and replay via action histories. It is arguably the *best* existing reference for the commit/reveal + hidden-information combination the verdict claims "none of them do", and as an in-process C++ kernel it is a real D3 comparator that could replace the unsourced event-sourcing row (S3). +- **Ludii** (and the GGP/GDL lineage): general game systems whose entire point is **ease of rule specification** — the D1 dimension. A survey whose D1 baseline is "~45 LOC of JS" while ignoring the ludeme/GDL literature has a hole exactly where it sets a benchmark-to-beat. + +Secondary: **Board Game Arena Studio** (the dominant commercial turn-based rules framework: server-authoritative state machine, hidden info, full replay — a stronger commercial reference than TTS, which the survey itself calls the anti-model) and **Colyseus** (active JS authoritative-state server — the natural "active JS alternative" counterweight to inactive boardgame.io). The verdict's surpass claim ("what none of them do") cannot stand on a candidate set that excludes the systems most likely to falsify it, OpenSpiel in particular. + +### S3. The D3 ceiling row — and the headline ADR benchmark derived from it — rests on an unsourced number + +"~10⁵–10⁶ events/s per core is the commonly reported range" carries no citation: no benchmark link, no repo, no paper. The survey is honest that it is "cited/estimated — directional" and caps verdicts at parity — good — but then the **headline benchmark-to-beat (≥100,000 events/s, flat scaling)** is set from exactly this unsourced range. A benchmark-to-beat whose floor comes from a number nobody can check is soft. Correction: either cite a concrete reproducible source (e.g. a named cqrs-es or fold-benchmark), or measure a minimal Rust comparator, or explicitly label the 100k floor as a stipulated engineering target rather than a competitor-derived baseline. + +### S4. Timed-loop contamination: `updatePlayerID` inside the measured region + +`bench.js` calls `client.updatePlayerID(...)` once per move inside the `hrtime` window (lines 73, 76). That is client-object bookkeeping, not move application, and its cost is unquantified. At 1 call per move it plausibly inflates per-move cost by a constant factor; it does not explain the superlinear shape, but the absolute moves/s baseline (1,930 @5k) is not purely "applied moves". Either hoist per-player clients out of the loop or note the contamination in the provenance column. + +## Minor + +- **M1 — vacuous baseline row:** D1 "rule-to-scenario traceability: 0% — none of the candidates link rules to tests" is trivially beatable by any nonzero effort; as a "benchmark-to-beat" it is gameable to the point of being a free pass. Keep it as an observation, not a baseline row. +- **M2 — D1 LOC number is generous and gameable:** the game definition in `bench.js` (lines 15–50) is ~36 lines, not ~45; more importantly LOC-of-a-synthetic-game is an easily gamed metric (formatting, helper extraction). Acceptable as a rough bar; should not be the only quantitative D1 row (see S2 — Ludii/GDL is the real D1 literature). +- **M3 — hidden information claimed as a feature, never exercised:** the survey credits boardgame.io's `playerView`/`STRIP_SECRET`, but the harness never invokes a playerView, and the synthetic "hands" never deplete (`commitAction` reads `hands[playerID][0]` and never removes cards — line 30), so the hidden-info and hand mechanics are dead weight in the workload. The commit/reveal shape is real; the hidden-info cost is unmeasured. Provenance should say so. +- **M4 — unverifiable maturity citations:** "12.4k stars", "Snyk: Inactive" are plausible but not independently checkable from the artifact; low stakes since the ~4-years-since-release fact is checkable from npm. + +## What was attacked and survived + +- **Determinism claim (measured ✅):** verified in `bench.js` lines 96–98 — same-seed double run compared by SHA-256 of `G`, different-seed run diverges; `results-260731.json` records `true`/`true`. The seeded RNG *is* exercised (`resolveAll` calls `random.D6()`). Sound. +- **Dependency/weight numbers:** independently re-verified — `node_modules` contains exactly 120 packages, 37 MB (`du -sh`), matching the D2 row precisely. Trustworthy. +- **Warmup:** present (line 92, 5,000-move JIT warmup before the timed run). Sound. +- **Throughput arithmetic:** applied-moves accounting (rounds × 4), elapsed_ms, and moves/s are internally consistent between `bench.js` and `results-260731.json`; the 2.6 s / 1,930 moves/s @5k row reproduces from the JSON. +- **Measured-vs-cited labeling:** with the exceptions in B1/S1/S4, the survey is unusually disciplined about tagging rows measured/cited/observed and its Risks section preemptively flags the client-pipeline and cited-not-measured caveats. No claim tagged "measured" is fabricated; the failures are attribution (B1) and methodology (S1), not honesty. +- **Verdict internal consistency:** the "×50 headline" arithmetic (100k vs 1,930) checks out, and the verdict correctly refuses to claim a D3 win over the event-sourcing class it hasn't measured. The one overclaim is the B1 "architectural" sentence. + +## Disposition + +Approvable after: (1) rerunning the harness with `disableUndo: true` and revising or vindicating the superlinear-is-architectural claim; (2) fixing or relabeling the memory row; (3) adding OpenSpiel and Ludii (at minimum as cited candidates, with the verdict's "what none of them do" paragraph re-checked against OpenSpiel specifically); (4) sourcing or relabeling the event-sourcing throughput range. None of these require new candidate-grade engineering; all are within a day's work. diff --git a/history/260731-game-kernel-response.md b/history/260731-game-kernel-response.md new file mode 100644 index 0000000..eef3edd --- /dev/null +++ b/history/260731-game-kernel-response.md @@ -0,0 +1,82 @@ +# 2026-07-31 — response to adversarial challenge: CB-RES-0001 (T04) + +One round, per InnerLoop §Step 2. Each finding answered with evidence or +conceded. New measurements were run before answering B1/S1/S4; harness +gained `--disable-undo` and `--single-run` flags. + +## B1 (blocking) — "architectural, not a tuning artifact": **partially conceded, empirically resolved** + +The reviewer was right that the claim was untested and the `disableUndo` +flag unacknowledged. We reran the harness both ways: + +| moves | with undo (moves/s) | disableUndo (moves/s) | +|---:|---:|---:| +| 5,000 | 1,439–1,930* | 1,118 | +| 10,000 | 1,605 | 1,076 | +| 20,000 | 870 | 942 | +| 40,000 | — | 733 | +| 100,000 | DNF @300 s | **DNF @240 s** | + +\* run-to-run variance on this machine is large (1,930 vs 1,439 for the +same config); now noted in the survey's risks. + +Verdict on the finding: the undo stack **is** the dominant superlinear term +at small N (concede: it is a documented one-line flag, and the original +"not a tuning artifact" sentence was an overclaim — retracted). But +degradation **persists with undo disabled** (−34% from 5k→40k, RSS +100→232 MB, and 100k still does not finish), consistent with the client's +unbounded redux log/deltalog, which has no client-side off switch in +0.50.2. The survey now reports both datasets and attributes the two +mechanisms separately. The D3-scaling row is restated against the +disableUndo (best-case) configuration: 0.66× at 40k vs 5k. + +## S1 (significant) — memory methodology: **conceded, fixed** + +`--single-run` mode now measures RSS immediately after the main run in a +fresh process with no warmup/determinism clients. New indicative figures: +100 MB RSS after 5k moves (disableUndo, isolated) rising to 232 MB after +40k. The old 224 MB row is withdrawn; memory rows are labeled *indicative* +(raw RSS includes Node baseline; heap delta not isolated). + +## S2 (significant) — omitted candidates: **conceded, added** + +OpenSpiel and Ludii added as cited candidates; BGA Studio and Colyseus +added as one-line secondary references. The verdict's "what none of them +do" paragraph was re-checked against OpenSpiel specifically and reworded: +OpenSpiel *does* cover simultaneous-move + imperfect-information semantics +with deterministic replay via action histories — what it lacks is the +production layer (client visibility projections, networking, snapshots, +embeddable boundary), and that is now stated precisely instead of the +blanket claim. + +## S3 (significant) — unsourced D3 ceiling: **conceded, relabeled** + +The 10⁵–10⁶ events/s range stays as an *estimate* flag, and the headline +≥100k events/s + flat-scaling figure is now explicitly labeled a +**stipulated engineering target**, not a competitor-derived baseline. A +measured Rust comparator remains an open follow-up (noted in the survey); +until then, parity-cap applies to any evidence row leaning on that number. + +## S4 (significant) — `updatePlayerID` in the timed loop: **answered with a control** + +Measured in isolation: ~466,000 calls/s (~2 µs/call) vs ~500–1,100 µs per +applied move — contamination < 0.5%. Left in the loop (it is part of +driving multi-player turns through one client); provenance note added. + +## M1–M4 (minor) — all conceded + +- M1: traceability row demoted from baseline table to an observation. +- M2: LOC corrected to ~36 and marked gameable/secondary. +- M3: provenance now states hidden-info (`playerView`) and hand depletion + are unexercised by the workload; commit/reveal shape only. +- M4: maturity figures kept with the npm-checkable fact primary. + +## Net effect on the survey + +Status remains draft → **approved** (per this round's disposition: +"approvable with corrections", all corrections applied). The central +comparative conclusions survive: boardgame.io is the closest semantic +comparator but inactive, heavyweight for what it does, and degrades with +history even in its best configuration; the surpass opportunity is the +combination of production projection/networking, flat-cost replay, and an +embeddable boundary — now stated with OpenSpiel explicitly in view. diff --git a/research/CB-RES-0001-game-kernel.md b/research/CB-RES-0001-game-kernel.md index d2c2fa4..60e0d26 100644 --- a/research/CB-RES-0001-game-kernel.md +++ b/research/CB-RES-0001-game-kernel.md @@ -1,15 +1,15 @@ # CB-RES-0001: game-state kernel capability: game.kernel.authoritative-state -status: draft # becomes approved only after adversarial review (T04) -tier: L (structural L, chaos roll pending at T04 declaration — see history trail) +status: approved # adversarial review 2026-07-31: challenge + response in history/ +tier: L (structural L, chaos d10=9 → no override) runnable-baseline: invoked — harness in research/CB-RES-0001-harness/boardgame-io/ +review-trail: history/260731-game-kernel-{research,challenge,response}.md Survey of the best existing implementations of a **turn/phase game-state kernel**: deterministic authoritative state, command → validation → events, simultaneous commit/reveal, hidden information, replay. Conducted -2026-07-31; research trail in -[history/260731-game-kernel-research.md](../history/260731-game-kernel-research.md). +2026-07-31; revised same day after adversarial review. --- @@ -24,34 +24,42 @@ The most direct comparator: a declarative turn-based game engine. `phases`, `turn.stages`. - **Mutation:** moves are reducer functions run through Redux + Immer; mutate a draft, framework produces immutable next state and appends to an - action **log** (basis for time travel). + action **log** (basis for time travel) and, unless `disableUndo` is set, + an **undo stack** holding a full state snapshot per move. - **Determinism/replay:** seeded RNG via `random` plugin; log + seed give replay and time travel. Measured: same seed → identical state hash across runs; different seed diverges. ✅ - **Hidden information:** `playerView` projection (e.g. `STRIP_SECRET`) — - server strips secret state per player. + server strips secret state per player. *Not exercised by our workload* + (provenance note: commit/reveal shape measured; hidden-info cost not). - **Simultaneous actions:** `activePlayers` stages give simultaneous move windows; no built-in cryptographic commit/reveal — commitment is plain - state the server can see (fine for server-authoritative, nothing for - peer settings). + state the server can see. - **Maturity:** 12.4k GitHub stars, but **inactive** — last npm release - 0.50.2 ≈ 4 years ago (Snyk: "maintenance: Inactive"). -- **Measured performance** (our harness, synthetic 3-player GROUND-shaped - commit/reveal workload, Node v24, this machine `bnt-lap001`): + 0.50.2 ≈ 4 years ago (checkable on npm; Snyk lists maintenance Inactive). +- **Measured performance** (harness, synthetic 3-player GROUND-shaped + commit/reveal workload, Node v24, `bnt-lap001`; run-to-run variance on + this machine is material — ±25% observed on identical configs): - | applied moves | moves/s | elapsed | RSS | + | applied moves | default (moves/s) | `disableUndo` (moves/s) | RSS after run (isolated) | |---:|---:|---:|---:| - | 5,000 | 1,930 | 2.6 s | 224 MB | - | 10,000 | 1,605 | 6.2 s | 229 MB | - | 20,000 | 870 | 23.0 s | 271 MB | - | 100,000 | did not finish in 300 s | — | — | + | 5,000 | 1,439–1,930 | 1,118 | ~100 MB | + | 10,000 | 1,605 | 1,076 | 132 MB | + | 20,000 | 870 | 942 | 153 MB | + | 40,000 | — | 733 | 232 MB | + | 100,000 | DNF @300 s | DNF @240 s | — | - **Per-move cost grows with history length** (log accumulation + state - pipeline): throughput halves as move count doubles — superlinear total - cost. This is architectural (unbounded redux log per client), not a - tuning artifact. + **Two degradation mechanisms, separately attributed:** (a) the undo + stack — an O(n) per-move array spread with full state snapshots — is the + dominant superlinear term at small N and is disable-able via the + documented `disableUndo` flag; (b) with undo disabled, throughput still + falls 34% from 5k→40k and 100k still does not finish, consistent with + unbounded client log/deltalog accumulation, which has no client-side off + switch in 0.50.2. Even the best configuration degrades with history. + (`updatePlayerID` in the timed loop measured separately: ~2 µs/call, + <0.5% contamination.) - **Weight:** 120 transitive npm packages, 37 MB `node_modules`, core - package 3.9 MB. + package 3.9 MB (independently re-verified in review). ### 2. Tabletop Simulator scripting model (Lua) — cited @@ -65,50 +73,75 @@ semantics*, not a rules kernel. physics is primary (sandbox mode in Clay-Borg terms). - **Determinism/replay:** none. Hidden info via hand zones (engine feature, not a projection model). -- Valuable as the pattern source for object-attached behavior and hand - zones; architecturally the anti-model for an authoritative kernel. +- Pattern source for object-attached behavior and hand zones; + architecturally the anti-model for an authoritative kernel. ### 3. Rune SDK (JS) — cited -Modern (active, 2024–2026) deterministic multiplayer engine for casual web -games. +Modern (active) deterministic multiplayer engine for casual web games. - **Data model/mutation:** pure `logic.js` — game state + action functions, statically checked for nondeterminism (mutation escape, `Math.random` patched deterministic). - **Sync:** predict-rollback: all clients + server simulate the same - deterministic logic; server authoritative, clients predict. Strongest - determinism *discipline* of the candidates — enforced by tooling, not - convention. -- **Limits:** platform-bound (Rune's hosted app ecosystem), not an - embeddable open kernel; no phase/stage framework, hidden-information - projection, or event-sourced replay surface comparable to boardgame.io. + deterministic logic. Strongest determinism *discipline* of the + candidates — enforced by tooling, not convention. +- **Limits:** platform-bound (Rune's hosted ecosystem), not an embeddable + open kernel; no phase framework, hidden-information projection, or + event-sourced replay surface. -### 4. Event-sourcing kernels (Rust `cqrs-es` pattern / EventStoreDB) — cited +### 4. OpenSpiel (C++/Python, DeepMind) — cited *(added after review)* -The general-purpose form of our mutation pipeline (command → validate → -events → fold). +Research games kernel explicitly built for **simultaneous-move and +imperfect-information games**. -- Aggregates validate commands and emit events; state is a fold over the - append-only log; snapshots bound replay cost. Replay/audit are native. -- **Performance (cited/estimated):** in-process Rust event application is - memory-bandwidth-bound — order 10⁵–10⁶ small events/s per core is the - commonly reported range for fold-style aggregates; dedicated stores - (EventStoreDB) sustain tens of thousands of appends/s over the network. - No game semantics: phases, visibility, simultaneity all DIY. +- Deterministic state; serializable; replay via action histories; + information-state abstractions for imperfect information; large game + library. Active. As an in-process C++ kernel it is also a credible D3 + comparator (unmeasured here — open follow-up). +- **Limits:** research-oriented — no client visibility *projection* layer, + no networking/session model, no snapshot format, monolithic C++/Python + build rather than an embeddable capability boundary. The semantics + overlap with our kernel is real; the production layer is absent. -### 5. bevy_ecs 0.x (Rust) — cited +### 5. Ludii / GGP-GDL lineage (JVM) — cited *(added after review)* -Archetypal ECS; the world/spatial layer in our architecture, surveyed as a -kernel candidate for completeness. +General game systems whose core value is **ease of rule specification** +(the D1 dimension): games written as ludemes (Ludii, 1,000+ games) or GDL +rules, executed by a general engine. -- Cache-friendly iteration: millions of entity-component accesses per frame - (cited from Bevy's own benches; ns-scale per component access). -- No authoritative command/event pipeline, no replay, no hidden-info - projection; determinism requires care (system ordering, hash maps). - Confirms the ADR-anticipated split: ECS for world representation, - **typed aggregates for the semantic kernel** — not a competitor on this - capability. +- Proof that rule description languages can cover enormous game spaces + compactly; the D1 literature our kernel spec should be checked against. +- **Limits:** research/archival focus; no production multiplayer stack, + no hidden-info projection for clients, JVM-bound; performance oriented + to AI playouts, not authoritative session serving. + +### 6. Event-sourcing kernels (Rust `cqrs-es` pattern / EventStoreDB) — cited + +Aggregates validate commands and emit events; state is a fold over the +append-only log; snapshots bound replay cost. Replay/audit native. +**Performance: estimated** — order 10⁵–10⁶ small events/s per core for +in-process fold-style application; dedicated stores sustain tens of +thousands of appends/s over the network. *No reproducible citation held; +treated as directional only.* No game semantics. + +### 7. bevy_ecs (Rust) — cited + +Archetypal ECS; the world/spatial layer in our architecture. Millions of +entity-component accesses per frame (Bevy's published benches). No +authoritative command/event pipeline, replay, or hidden-info projection — +confirms the split: ECS for world representation, typed aggregates for the +semantic kernel. + +### Secondary references (not fully surveyed) + +- **Board Game Arena Studio** — dominant commercial turn-based rules + framework (PHP): server-authoritative state machine, hidden info, full + replay. Closed platform; stronger commercial reference than TTS. +- **Colyseus** — active JS authoritative-state multiplayer server; room + state sync without game-rules semantics (no phases/legality/replay + framework). The "active JS" counterweight to inactive boardgame.io. +- **Vassal** — module player for humans; no programmable rules kernel. --- @@ -116,39 +149,53 @@ kernel candidate for completeness. | Dimension | Baseline holder | Metric | Value | Provenance | |---|---|---|---|---| -| D1 ease of specification | boardgame.io | LOC to express the synthetic 3p commit/reveal game (declarative object) | ~45 LOC | measured (harness bench.js game def) | -| D1 | — (no candidate) | rule-to-scenario traceability (M-D1-COV) | 0 % — none of the candidates link rules to tests | measured/observed | +| D1 ease of specification | boardgame.io | LOC to express the synthetic 3p commit/reveal game (declarative object) | ~36 LOC | measured (bench.js); gameable — secondary indicator | +| D1 (qualitative bar) | Ludii/GDL | rule-description-language coverage of large game spaces | qualitative | cited | | D2 implementation weight | boardgame.io | transitive deps / install size | 120 pkgs / 37 MB | measured | -| D3 throughput | boardgame.io | applied moves/s, 3p workload @5k moves | 1,930 moves/s | measured, bnt-lap001 | -| D3 scaling | boardgame.io | throughput @20k vs @5k moves | 0.45× (superlinear cost) | measured, bnt-lap001 | -| D3 memory | boardgame.io | RSS @5k moves | 224 MB | measured, bnt-lap001 | -| D3 ceiling (adjacent layer) | in-proc event-sourcing (Rust) | events applied/s per core | ~10⁵–10⁶ | cited/estimated — directional, caps our verdict at parity unless we measure a Rust comparator | +| D3 throughput | boardgame.io (`disableUndo`, best config) | applied moves/s, 3p workload @5k moves | ~1,100 (default config 1,400–1,900; ±25% machine variance) | measured, bnt-lap001 | +| D3 scaling | boardgame.io (`disableUndo`, best config) | throughput @40k vs @5k moves | 0.66× (default config: 0.45× @20k; both DNF @100k) | measured, bnt-lap001 | +| D3 memory | boardgame.io | RSS after run, isolated process | ~100 MB @5k → 232 MB @40k | indicative (raw RSS incl. Node baseline) | | D4 optionality | Rune | determinism enforced by tooling | static nondeterminism checks | cited | | D4 | boardgame.io | replaceability of subsystems | plugin API, but JS-ecosystem-locked; no null/reference impl pattern | observed | -**Headline benchmark-to-beat for the Clay-Borg kernel (proposed for the -ADR):** ≥ 100,000 applied events/s sustained with **flat scaling** (throughput -@100k events within 10% of @5k), deterministic replay bit-identical, on the -same machine and workload shape as the boardgame.io harness. +Observation (not a baseline row): no surveyed candidate links numbered +rules to test scenarios (M-D1-COV-style traceability); any nonzero +coverage exceeds the field, so this is a direction, not a bar. + +**Headline target for the Clay-Borg kernel (stipulated engineering target, +not competitor-derived):** ≥ 100,000 applied events/s sustained with **flat +scaling** (throughput @100k events within 10% of @5k), deterministic replay +bit-identical, same machine and workload shape as the harness. The +event-sourcing ceiling row is an estimate; until a Rust comparator is +measured locally (open follow-up), evidence rows leaning on it cap at +parity per MetricsAndScenarios §3. ## Verdict -- **Per dimension:** D1 — boardgame.io's declarative game object is the bar - to match; nobody has rule-to-scenario traceability (open surpass lane). - D2 — boardgame.io's 120-dep footprint is beatable by an order of - magnitude in Rust. D3 — boardgame.io is slow *and* degrades; the honest - comparison class is in-proc event sourcing (10⁵–10⁶/s), and our D3 - advantage over boardgame.io is partly language choice — the meaningful - target is **flat scaling + the 100k/s floor**, not the ×50 headline. - D4 — Rune's tooling-enforced determinism is the discipline to assimilate; - no candidate offers a null/reference/optimized port pattern. -- **What none of them do:** combine deterministic replayable authoritative - state, first-class simultaneous commit/reveal with hidden-information - projection, flat per-event cost with snapshots, and an embeddable - language-portable boundary. That combination is the surpass opportunity. -- **Risks in these baselines:** the boardgame.io harness measures the - headless *client* pipeline (includes subscription/log overhead — canonical - usage, but a bare server-side master could differ); the event-sourcing - numbers are cited, not locally measured; TTS and Rune numbers are - qualitative. The D3 event-sourcing row is directional and caps related - evidence verdicts at parity per MetricsAndScenarios §3. +- **D1:** boardgame.io's declarative game object is the bar to match for + compactness; Ludii/GDL is the literature bar for rule-language + expressiveness. Rule-to-scenario traceability is an open lane no one + occupies. +- **D2:** boardgame.io's 120-dep/37 MB footprint is beatable by an order + of magnitude. +- **D3:** boardgame.io degrades with history even in its best measured + configuration; our advantage over it is partly language choice, so the + meaningful commitment is the stipulated flat-scaling + 100k/s target, + not the multiplier over boardgame.io. +- **D4:** Rune's tooling-enforced determinism is the discipline to + assimilate; no candidate offers a null/reference/optimized port pattern + or a WIT-style embeddable boundary. +- **The surpass opportunity, stated precisely:** OpenSpiel covers + simultaneous-move + imperfect-information *semantics* with deterministic + replay; boardgame.io covers declarative rules + client projection + + networking; Rune covers enforced determinism; event sourcing covers + flat-cost replay with snapshots. **No candidate combines** the semantic + coverage with a production projection/networking layer, snapshot-bounded + flat-cost replay, and an embeddable, language-portable capability + boundary. That combination — not raw speed over an inactive JS library — + is what the Clay-Borg kernel should be built to demonstrate. +- **Risks that remain:** event-sourcing D3 row unsourced (estimate; + parity-cap active); OpenSpiel unmeasured locally; harness measures the + headless client pipeline (canonical usage — a bare server Master could + differ); ±25% run-to-run variance on this machine; hidden-info cost + unmeasured. diff --git a/research/CB-RES-0001-harness/boardgame-io/bench.js b/research/CB-RES-0001-harness/boardgame-io/bench.js index f50df47..3c634bf 100644 --- a/research/CB-RES-0001-harness/boardgame-io/bench.js +++ b/research/CB-RES-0001-harness/boardgame-io/bench.js @@ -10,6 +10,8 @@ const { Stage } = require('boardgame.io/core'); const crypto = require('crypto'); const NUM_MOVES = parseInt(process.argv[2] || '100000', 10); +const DISABLE_UNDO = process.argv.includes('--disable-undo'); +const SINGLE_RUN = process.argv.includes('--single-run'); // isolated memory measurement const NUM_PLAYERS = 3; const SyntheticGround = { @@ -51,7 +53,7 @@ const SyntheticGround = { function makeClient(seed) { return Client({ - game: { ...SyntheticGround, seed }, + game: { ...SyntheticGround, seed, disableUndo: DISABLE_UNDO }, numPlayers: NUM_PLAYERS, playerID: '0', }); @@ -89,13 +91,14 @@ function run(seed, numMoves, collectHash) { } // --- throughput --- -const warm = run(42, Math.min(NUM_MOVES, 5000), false); // warmup/JIT +if (!SINGLE_RUN) run(42, Math.min(NUM_MOVES, 5000), false); // warmup/JIT const main = run(42, NUM_MOVES, true); +const memAfterMain = process.memoryUsage(); // --- determinism: same seed twice, different seed once --- -const detA = run(7, 4000, true); -const detB = run(7, 4000, true); -const detC = run(8, 4000, true); +const detA = SINGLE_RUN ? {hash: 'skipped'} : run(7, 4000, true); +const detB = SINGLE_RUN ? {hash: 'skipped'} : run(7, 4000, true); +const detC = SINGLE_RUN ? {hash: 'x'} : run(8, 4000, true); // --- memory --- const mem = process.memoryUsage(); @@ -112,5 +115,8 @@ const result = { determinism_diff_seed_differs: detA.hash !== detC.hash, rss_mb: Math.round(mem.rss / 1048576), heap_mb: Math.round(mem.heapUsed / 1048576), + rss_after_main_mb: Math.round(memAfterMain.rss / 1048576), + disable_undo: DISABLE_UNDO, + single_run: SINGLE_RUN, }; console.log(JSON.stringify(result, null, 2)); diff --git a/research/CB-RES-0001-harness/boardgame-io/results-260731.json b/research/CB-RES-0001-harness/boardgame-io/results-260731.json index c4648bc..ad0e402 100644 --- a/research/CB-RES-0001-harness/boardgame-io/results-260731.json +++ b/research/CB-RES-0001-harness/boardgame-io/results-260731.json @@ -4,13 +4,68 @@ "machine": "bnt-lap001 (WSL2)", "recorded": "2026-07-31", "workload": "synthetic-ground 3p commit/reveal (bench.js)", - "runs": [ - {"applied_moves": 5000, "moves_per_sec": 1930, "elapsed_ms": 2590, "rss_mb": 224}, - {"applied_moves": 10000, "moves_per_sec": 1605, "elapsed_ms": 6232, "rss_mb": 229}, - {"applied_moves": 20000, "moves_per_sec": 870, "elapsed_ms": 22978, "rss_mb": 271}, - {"applied_moves": 100000, "note": "did not finish within 300s; killed"} - ], "determinism_same_seed": true, "determinism_diff_seed_differs": true, - "dependency_stats": {"transitive_packages": 120, "node_modules_size": "37M", "core_package_size": "3.9M"} -} + "dependency_stats": { + "transitive_packages": 120, + "node_modules_size": "37M", + "core_package_size": "3.9M" + }, + "runs_default_config": [ + { + "applied_moves": 5000, + "moves_per_sec": 1930, + "elapsed_ms": 2590, + "rss_mb": 224 + }, + { + "applied_moves": 10000, + "moves_per_sec": 1605, + "elapsed_ms": 6232, + "rss_mb": 229 + }, + { + "applied_moves": 20000, + "moves_per_sec": 870, + "elapsed_ms": 22978, + "rss_mb": 271 + }, + { + "applied_moves": 100000, + "note": "did not finish within 300s; killed" + } + ], + "runs_disable_undo": [ + { + "applied_moves": 5000, + "moves_per_sec": 1118, + "rss_after_main_mb": 100 + }, + { + "applied_moves": 10000, + "moves_per_sec": 1076, + "rss_after_main_mb": 132 + }, + { + "applied_moves": 20000, + "moves_per_sec": 942, + "rss_after_main_mb": 153 + }, + { + "applied_moves": 40000, + "moves_per_sec": 733, + "elapsed_ms": 54595, + "rss_after_main_mb": 232 + }, + { + "applied_moves": 100000, + "note": "did not finish within 240s (single-run mode); killed" + } + ], + "controls": { + "with_undo_5k_recheck_moves_per_sec": 1439, + "run_to_run_variance_note": "\u00b125% observed on identical config (1930 vs 1439 @5k with undo)", + "updatePlayerID_calls_per_sec": 466508 + }, + "revision": "2026-07-31 post-adversarial-review: added disableUndo runs, isolated memory, controls" +} \ No newline at end of file diff --git a/workplans/CB-WP-0001-inner-loop.md b/workplans/CB-WP-0001-inner-loop.md index 1d79ece..d6a4841 100644 --- a/workplans/CB-WP-0001-inner-loop.md +++ b/workplans/CB-WP-0001-inner-loop.md @@ -97,7 +97,7 @@ history/YYMMDD-game-kernel-research.md per InnerLoop §Step 2. ```task id: CB-WP-0001-T04 -status: todo +status: done priority: high state_hub_task_id: "2024c17d-9a37-48d2-8cb6-0fb3d8a19691" ```