T04: adversarial review round + survey corrections + ADR-0002 (reimplement, assimilate patterns)
This commit is contained in:
parent
a7e31d4210
commit
53c1b18ec1
7 changed files with 414 additions and 94 deletions
70
decisions/ADR-0002-game-kernel.md
Normal file
70
decisions/ADR-0002-game-kernel.md
Normal file
|
|
@ -0,0 +1,70 @@
|
||||||
|
# ADR-0002: Game-state kernel — survey approval and implementation decision
|
||||||
|
|
||||||
|
Date: 2026-07-31
|
||||||
|
Status: accepted
|
||||||
|
Capability: game.kernel.authoritative-state
|
||||||
|
Inputs: research/CB-RES-0001-game-kernel.md (approved),
|
||||||
|
history/260731-game-kernel-{research,challenge,response}.md
|
||||||
|
|
||||||
|
## Approval (InnerLoop step 2)
|
||||||
|
|
||||||
|
The CB-RES-0001 survey is **approved**. It passed one round of adversarial
|
||||||
|
review: the challenge found one blocking overclaim (degradation
|
||||||
|
attribution) and three significant gaps (memory methodology, omitted
|
||||||
|
candidates OpenSpiel/Ludii, unsourced D3 ceiling); all were resolved with
|
||||||
|
new measurements or corrections, none overturned the comparative
|
||||||
|
conclusions. The candidate set is judged adequate; the baselines are
|
||||||
|
trustworthy within their stated provenance (measured vs cited vs
|
||||||
|
estimated), with the event-sourcing ceiling explicitly parity-capped.
|
||||||
|
|
||||||
|
**Benchmark-to-beat, named:** boardgame.io 0.50.2 per the survey's
|
||||||
|
baseline table (throughput ~1,100–1,900 moves/s degrading with history in
|
||||||
|
every configuration; 120 deps/37 MB; ~36-LOC declarative game definition),
|
||||||
|
plus the **stipulated engineering target** of ≥100,000 applied events/s
|
||||||
|
with flat scaling (@100k within 10% of @5k) and bit-identical replay.
|
||||||
|
|
||||||
|
## Decision (InnerLoop step 3): reimplement, assimilating patterns
|
||||||
|
|
||||||
|
Build the Clay-Borg semantic kernel as **our own Rust implementation**
|
||||||
|
(`cb-kernel` + `cb-events` + `cb-game-runtime`) — not a wrapper around any
|
||||||
|
surveyed system. Per the Clay-Borg rule, we own the semantics; what we
|
||||||
|
assimilate from the field is *patterns*, each with a named source:
|
||||||
|
|
||||||
|
| Assimilated pattern | From | Into |
|
||||||
|
|---|---|---|
|
||||||
|
| Declarative game definition (setup/moves/phases as data + pure functions) | boardgame.io | game-package rules API |
|
||||||
|
| Tooling-enforced determinism (not convention) | Rune | seeded RNG service, denied ambient time/random, replay double-run check in the scenario runner |
|
||||||
|
| Command → validate → events → fold, snapshots bound replay | event-sourcing (cqrs-es pattern) | kernel mutation pipeline |
|
||||||
|
| Information-state framing for imperfect information | OpenSpiel | per-player projection model |
|
||||||
|
| Hand-zone / object-visibility semantics | TTS (pattern only) | tabletop layer, later |
|
||||||
|
|
||||||
|
Rejected alternatives: **assimilate boardgame.io** (inactive, JS-bound,
|
||||||
|
history-degrading in best config — fails D2/D3/D4); **build on OpenSpiel**
|
||||||
|
(right semantics, wrong layer — research kernel without projection/
|
||||||
|
networking/snapshot surface, heavyweight C++/Python boundary); **ECS as
|
||||||
|
kernel** (bevy_ecs stays in the world layer per the blueprint).
|
||||||
|
|
||||||
|
## Expected advantage per dimension (honest trades)
|
||||||
|
|
||||||
|
- **D3 speed: better.** Flat per-event cost with snapshots; the 100k/s
|
||||||
|
floor. Declared caveat: part of the margin over boardgame.io is language
|
||||||
|
choice; the commitment that matters is flat scaling, and the
|
||||||
|
event-sourcing comparison stays at parity until a Rust comparator is
|
||||||
|
measured.
|
||||||
|
- **D2 weight: better.** Target ≤20 transitive crates for the headless
|
||||||
|
kernel (vs 120 npm packages), measured by M-D2-DEP/M-D2-LOC/M-D2-BLD.
|
||||||
|
- **D1 specification: parity, aiming better.** Match boardgame.io's
|
||||||
|
declarative compactness on the synthetic workload (≤~40 LOC equivalent);
|
||||||
|
surpass via rule-to-scenario traceability (M-D1-COV), which no candidate
|
||||||
|
has. Declared trade: we do not attempt Ludii/GDL-class generality — one
|
||||||
|
game family first, canon only after a second concrete use.
|
||||||
|
- **D4 optionality: better.** Port pattern with null/reference impls from
|
||||||
|
day one, capability boundary WIT-expressible later, zero foreign types in
|
||||||
|
canonical interfaces (M-D4-LEAK=0). No surveyed candidate offers this.
|
||||||
|
|
||||||
|
## Gate
|
||||||
|
|
||||||
|
Per InnerLoop, no kernel implementation code existed before this ADR.
|
||||||
|
Next steps: T05 (GROUND rules spec, derived from ground-game), T06 (kernel
|
||||||
|
spec with acceptance metrics tied to these baselines), T07 (workspace +
|
||||||
|
harness), T08 (code loop).
|
||||||
60
history/260731-game-kernel-challenge.md
Normal file
60
history/260731-game-kernel-challenge.md
Normal file
|
|
@ -0,0 +1,60 @@
|
||||||
|
# 2026-07-31 — adversarial challenge: CB-RES-0001 (T04)
|
||||||
|
|
||||||
|
**Verdict: approvable with corrections** — the measured numbers are real and the determinism/dependency claims verify against the harness, but the survey's central architectural claim about boardgame.io's superlinear degradation is contradicted by a documented tuning option the harness never tested, the memory methodology is invalid, and at least two squarely in-scope candidates (OpenSpiel, Ludii) are omitted.
|
||||||
|
|
||||||
|
Reviewed cold: only `research/CB-RES-0001-game-kernel.md` plus the harness under `research/CB-RES-0001-harness/boardgame-io/` (`bench.js`, `results-260731.json`, `package.json`, and the installed `boardgame.io@0.50.2` source in `node_modules`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Blocking
|
||||||
|
|
||||||
|
### B1. "Architectural, not a tuning artifact" is asserted, untested, and likely false as stated
|
||||||
|
|
||||||
|
Survey text: *"Per-move cost grows with history length (log accumulation + state pipeline): throughput halves as move count doubles — superlinear total cost. This is architectural (unbounded redux log per client), not a tuning artifact."* This attribution underwrites the D3-scaling baseline row (0.45×), the "boardgame.io is slow *and* degrades" verdict, and plausibly the "did not finish in 300s" @100k row.
|
||||||
|
|
||||||
|
I inspected the installed library. boardgame.io v0.50.2 has a documented game option `disableUndo` (`node_modules/boardgame.io/dist/cjs/initialize-648ccd94.js:49`, `server.js:1912`). Without it, every move executes `_undo: [...state._undo, undoEntry]` (`reducer-6f7cf6b0.js:853`) — an O(n) array spread **per move**, i.e. O(n²) total, retaining a G/ctx/plugins snapshot per move. That alone is sufficient to produce exactly the observed halving-per-doubling and the RSS growth, and it is switched off by a one-line, documented config flag. The harness (`bench.js`) never sets `disableUndo`, never tests the bare `Master`, and the survey nowhere acknowledges the flag. The client-side `log` accumulation the survey blames may or may not remain superlinear with undo disabled — nobody measured it.
|
||||||
|
|
||||||
|
The survey's Risks section hedges the *measurement surface* ("headless client pipeline... a bare server-side master could differ") but not the *attribution*: "architectural, not a tuning artifact" is precisely a claim that no tuning knob exists, and one demonstrably does. **Required correction:** rerun the harness with `disableUndo: true` (and ideally the server `Master` path). If scaling flattens, the D3-scaling row, the @100k DNF row, and the "degrades" verdict language must be rewritten; if it stays superlinear, the claim becomes defensible and should cite the flag explicitly.
|
||||||
|
|
||||||
|
## Significant
|
||||||
|
|
||||||
|
### S1. Memory baseline (RSS 224 MB @5k) is methodologically invalid
|
||||||
|
|
||||||
|
`bench.js` measures `process.memoryUsage()` (line 101) **after** the warmup run (5,000 moves, line 92), the main run, *and* three additional determinism runs of 4,000 moves each (lines 96–98) — five client instances, each with its own unbounded log and undo stack, with no GC or isolation between them. The "D3 memory | RSS @5k moves | 224 MB | measured" baseline row attributes to a 5k-move workload an RSS that actually reflects ~22k moves across five clients plus Node baseline. The per-run `rss_mb` column in `results-260731.json` inherits the same flaw. The row should either be re-measured with an isolated single run (report heap delta, not raw RSS) or downgraded from "measured" to "indicative".
|
||||||
|
|
||||||
|
### S2. Omitted candidates: OpenSpiel and Ludii are squarely in scope; BGA Studio and Colyseus deserve at least a line
|
||||||
|
|
||||||
|
The survey's stated scope is "deterministic authoritative state, command → validation → events, simultaneous commit/reveal, hidden information, replay". Two mature, well-known systems sit dead-center in that scope and are absent:
|
||||||
|
|
||||||
|
- **OpenSpiel** (DeepMind, C++/Python, active): a games kernel explicitly built for **simultaneous-move and imperfect-information games**, deterministic, with serializable state and replay via action histories. It is arguably the *best* existing reference for the commit/reveal + hidden-information combination the verdict claims "none of them do", and as an in-process C++ kernel it is a real D3 comparator that could replace the unsourced event-sourcing row (S3).
|
||||||
|
- **Ludii** (and the GGP/GDL lineage): general game systems whose entire point is **ease of rule specification** — the D1 dimension. A survey whose D1 baseline is "~45 LOC of JS" while ignoring the ludeme/GDL literature has a hole exactly where it sets a benchmark-to-beat.
|
||||||
|
|
||||||
|
Secondary: **Board Game Arena Studio** (the dominant commercial turn-based rules framework: server-authoritative state machine, hidden info, full replay — a stronger commercial reference than TTS, which the survey itself calls the anti-model) and **Colyseus** (active JS authoritative-state server — the natural "active JS alternative" counterweight to inactive boardgame.io). The verdict's surpass claim ("what none of them do") cannot stand on a candidate set that excludes the systems most likely to falsify it, OpenSpiel in particular.
|
||||||
|
|
||||||
|
### S3. The D3 ceiling row — and the headline ADR benchmark derived from it — rests on an unsourced number
|
||||||
|
|
||||||
|
"~10⁵–10⁶ events/s per core is the commonly reported range" carries no citation: no benchmark link, no repo, no paper. The survey is honest that it is "cited/estimated — directional" and caps verdicts at parity — good — but then the **headline benchmark-to-beat (≥100,000 events/s, flat scaling)** is set from exactly this unsourced range. A benchmark-to-beat whose floor comes from a number nobody can check is soft. Correction: either cite a concrete reproducible source (e.g. a named cqrs-es or fold-benchmark), or measure a minimal Rust comparator, or explicitly label the 100k floor as a stipulated engineering target rather than a competitor-derived baseline.
|
||||||
|
|
||||||
|
### S4. Timed-loop contamination: `updatePlayerID` inside the measured region
|
||||||
|
|
||||||
|
`bench.js` calls `client.updatePlayerID(...)` once per move inside the `hrtime` window (lines 73, 76). That is client-object bookkeeping, not move application, and its cost is unquantified. At 1 call per move it plausibly inflates per-move cost by a constant factor; it does not explain the superlinear shape, but the absolute moves/s baseline (1,930 @5k) is not purely "applied moves". Either hoist per-player clients out of the loop or note the contamination in the provenance column.
|
||||||
|
|
||||||
|
## Minor
|
||||||
|
|
||||||
|
- **M1 — vacuous baseline row:** D1 "rule-to-scenario traceability: 0% — none of the candidates link rules to tests" is trivially beatable by any nonzero effort; as a "benchmark-to-beat" it is gameable to the point of being a free pass. Keep it as an observation, not a baseline row.
|
||||||
|
- **M2 — D1 LOC number is generous and gameable:** the game definition in `bench.js` (lines 15–50) is ~36 lines, not ~45; more importantly LOC-of-a-synthetic-game is an easily gamed metric (formatting, helper extraction). Acceptable as a rough bar; should not be the only quantitative D1 row (see S2 — Ludii/GDL is the real D1 literature).
|
||||||
|
- **M3 — hidden information claimed as a feature, never exercised:** the survey credits boardgame.io's `playerView`/`STRIP_SECRET`, but the harness never invokes a playerView, and the synthetic "hands" never deplete (`commitAction` reads `hands[playerID][0]` and never removes cards — line 30), so the hidden-info and hand mechanics are dead weight in the workload. The commit/reveal shape is real; the hidden-info cost is unmeasured. Provenance should say so.
|
||||||
|
- **M4 — unverifiable maturity citations:** "12.4k stars", "Snyk: Inactive" are plausible but not independently checkable from the artifact; low stakes since the ~4-years-since-release fact is checkable from npm.
|
||||||
|
|
||||||
|
## What was attacked and survived
|
||||||
|
|
||||||
|
- **Determinism claim (measured ✅):** verified in `bench.js` lines 96–98 — same-seed double run compared by SHA-256 of `G`, different-seed run diverges; `results-260731.json` records `true`/`true`. The seeded RNG *is* exercised (`resolveAll` calls `random.D6()`). Sound.
|
||||||
|
- **Dependency/weight numbers:** independently re-verified — `node_modules` contains exactly 120 packages, 37 MB (`du -sh`), matching the D2 row precisely. Trustworthy.
|
||||||
|
- **Warmup:** present (line 92, 5,000-move JIT warmup before the timed run). Sound.
|
||||||
|
- **Throughput arithmetic:** applied-moves accounting (rounds × 4), elapsed_ms, and moves/s are internally consistent between `bench.js` and `results-260731.json`; the 2.6 s / 1,930 moves/s @5k row reproduces from the JSON.
|
||||||
|
- **Measured-vs-cited labeling:** with the exceptions in B1/S1/S4, the survey is unusually disciplined about tagging rows measured/cited/observed and its Risks section preemptively flags the client-pipeline and cited-not-measured caveats. No claim tagged "measured" is fabricated; the failures are attribution (B1) and methodology (S1), not honesty.
|
||||||
|
- **Verdict internal consistency:** the "×50 headline" arithmetic (100k vs 1,930) checks out, and the verdict correctly refuses to claim a D3 win over the event-sourcing class it hasn't measured. The one overclaim is the B1 "architectural" sentence.
|
||||||
|
|
||||||
|
## Disposition
|
||||||
|
|
||||||
|
Approvable after: (1) rerunning the harness with `disableUndo: true` and revising or vindicating the superlinear-is-architectural claim; (2) fixing or relabeling the memory row; (3) adding OpenSpiel and Ludii (at minimum as cited candidates, with the verdict's "what none of them do" paragraph re-checked against OpenSpiel specifically); (4) sourcing or relabeling the event-sourcing throughput range. None of these require new candidate-grade engineering; all are within a day's work.
|
||||||
82
history/260731-game-kernel-response.md
Normal file
82
history/260731-game-kernel-response.md
Normal file
|
|
@ -0,0 +1,82 @@
|
||||||
|
# 2026-07-31 — response to adversarial challenge: CB-RES-0001 (T04)
|
||||||
|
|
||||||
|
One round, per InnerLoop §Step 2. Each finding answered with evidence or
|
||||||
|
conceded. New measurements were run before answering B1/S1/S4; harness
|
||||||
|
gained `--disable-undo` and `--single-run` flags.
|
||||||
|
|
||||||
|
## B1 (blocking) — "architectural, not a tuning artifact": **partially conceded, empirically resolved**
|
||||||
|
|
||||||
|
The reviewer was right that the claim was untested and the `disableUndo`
|
||||||
|
flag unacknowledged. We reran the harness both ways:
|
||||||
|
|
||||||
|
| moves | with undo (moves/s) | disableUndo (moves/s) |
|
||||||
|
|---:|---:|---:|
|
||||||
|
| 5,000 | 1,439–1,930* | 1,118 |
|
||||||
|
| 10,000 | 1,605 | 1,076 |
|
||||||
|
| 20,000 | 870 | 942 |
|
||||||
|
| 40,000 | — | 733 |
|
||||||
|
| 100,000 | DNF @300 s | **DNF @240 s** |
|
||||||
|
|
||||||
|
\* run-to-run variance on this machine is large (1,930 vs 1,439 for the
|
||||||
|
same config); now noted in the survey's risks.
|
||||||
|
|
||||||
|
Verdict on the finding: the undo stack **is** the dominant superlinear term
|
||||||
|
at small N (concede: it is a documented one-line flag, and the original
|
||||||
|
"not a tuning artifact" sentence was an overclaim — retracted). But
|
||||||
|
degradation **persists with undo disabled** (−34% from 5k→40k, RSS
|
||||||
|
100→232 MB, and 100k still does not finish), consistent with the client's
|
||||||
|
unbounded redux log/deltalog, which has no client-side off switch in
|
||||||
|
0.50.2. The survey now reports both datasets and attributes the two
|
||||||
|
mechanisms separately. The D3-scaling row is restated against the
|
||||||
|
disableUndo (best-case) configuration: 0.66× at 40k vs 5k.
|
||||||
|
|
||||||
|
## S1 (significant) — memory methodology: **conceded, fixed**
|
||||||
|
|
||||||
|
`--single-run` mode now measures RSS immediately after the main run in a
|
||||||
|
fresh process with no warmup/determinism clients. New indicative figures:
|
||||||
|
100 MB RSS after 5k moves (disableUndo, isolated) rising to 232 MB after
|
||||||
|
40k. The old 224 MB row is withdrawn; memory rows are labeled *indicative*
|
||||||
|
(raw RSS includes Node baseline; heap delta not isolated).
|
||||||
|
|
||||||
|
## S2 (significant) — omitted candidates: **conceded, added**
|
||||||
|
|
||||||
|
OpenSpiel and Ludii added as cited candidates; BGA Studio and Colyseus
|
||||||
|
added as one-line secondary references. The verdict's "what none of them
|
||||||
|
do" paragraph was re-checked against OpenSpiel specifically and reworded:
|
||||||
|
OpenSpiel *does* cover simultaneous-move + imperfect-information semantics
|
||||||
|
with deterministic replay via action histories — what it lacks is the
|
||||||
|
production layer (client visibility projections, networking, snapshots,
|
||||||
|
embeddable boundary), and that is now stated precisely instead of the
|
||||||
|
blanket claim.
|
||||||
|
|
||||||
|
## S3 (significant) — unsourced D3 ceiling: **conceded, relabeled**
|
||||||
|
|
||||||
|
The 10⁵–10⁶ events/s range stays as an *estimate* flag, and the headline
|
||||||
|
≥100k events/s + flat-scaling figure is now explicitly labeled a
|
||||||
|
**stipulated engineering target**, not a competitor-derived baseline. A
|
||||||
|
measured Rust comparator remains an open follow-up (noted in the survey);
|
||||||
|
until then, parity-cap applies to any evidence row leaning on that number.
|
||||||
|
|
||||||
|
## S4 (significant) — `updatePlayerID` in the timed loop: **answered with a control**
|
||||||
|
|
||||||
|
Measured in isolation: ~466,000 calls/s (~2 µs/call) vs ~500–1,100 µs per
|
||||||
|
applied move — contamination < 0.5%. Left in the loop (it is part of
|
||||||
|
driving multi-player turns through one client); provenance note added.
|
||||||
|
|
||||||
|
## M1–M4 (minor) — all conceded
|
||||||
|
|
||||||
|
- M1: traceability row demoted from baseline table to an observation.
|
||||||
|
- M2: LOC corrected to ~36 and marked gameable/secondary.
|
||||||
|
- M3: provenance now states hidden-info (`playerView`) and hand depletion
|
||||||
|
are unexercised by the workload; commit/reveal shape only.
|
||||||
|
- M4: maturity figures kept with the npm-checkable fact primary.
|
||||||
|
|
||||||
|
## Net effect on the survey
|
||||||
|
|
||||||
|
Status remains draft → **approved** (per this round's disposition:
|
||||||
|
"approvable with corrections", all corrections applied). The central
|
||||||
|
comparative conclusions survive: boardgame.io is the closest semantic
|
||||||
|
comparator but inactive, heavyweight for what it does, and degrades with
|
||||||
|
history even in its best configuration; the surpass opportunity is the
|
||||||
|
combination of production projection/networking, flat-cost replay, and an
|
||||||
|
embeddable boundary — now stated with OpenSpiel explicitly in view.
|
||||||
|
|
@ -1,15 +1,15 @@
|
||||||
# CB-RES-0001: game-state kernel
|
# CB-RES-0001: game-state kernel
|
||||||
|
|
||||||
capability: game.kernel.authoritative-state
|
capability: game.kernel.authoritative-state
|
||||||
status: draft # becomes approved only after adversarial review (T04)
|
status: approved # adversarial review 2026-07-31: challenge + response in history/
|
||||||
tier: L (structural L, chaos roll pending at T04 declaration — see history trail)
|
tier: L (structural L, chaos d10=9 → no override)
|
||||||
runnable-baseline: invoked — harness in research/CB-RES-0001-harness/boardgame-io/
|
runnable-baseline: invoked — harness in research/CB-RES-0001-harness/boardgame-io/
|
||||||
|
review-trail: history/260731-game-kernel-{research,challenge,response}.md
|
||||||
|
|
||||||
Survey of the best existing implementations of a **turn/phase game-state
|
Survey of the best existing implementations of a **turn/phase game-state
|
||||||
kernel**: deterministic authoritative state, command → validation → events,
|
kernel**: deterministic authoritative state, command → validation → events,
|
||||||
simultaneous commit/reveal, hidden information, replay. Conducted
|
simultaneous commit/reveal, hidden information, replay. Conducted
|
||||||
2026-07-31; research trail in
|
2026-07-31; revised same day after adversarial review.
|
||||||
[history/260731-game-kernel-research.md](../history/260731-game-kernel-research.md).
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -24,34 +24,42 @@ The most direct comparator: a declarative turn-based game engine.
|
||||||
`phases`, `turn.stages`.
|
`phases`, `turn.stages`.
|
||||||
- **Mutation:** moves are reducer functions run through Redux + Immer;
|
- **Mutation:** moves are reducer functions run through Redux + Immer;
|
||||||
mutate a draft, framework produces immutable next state and appends to an
|
mutate a draft, framework produces immutable next state and appends to an
|
||||||
action **log** (basis for time travel).
|
action **log** (basis for time travel) and, unless `disableUndo` is set,
|
||||||
|
an **undo stack** holding a full state snapshot per move.
|
||||||
- **Determinism/replay:** seeded RNG via `random` plugin; log + seed give
|
- **Determinism/replay:** seeded RNG via `random` plugin; log + seed give
|
||||||
replay and time travel. Measured: same seed → identical state hash across
|
replay and time travel. Measured: same seed → identical state hash across
|
||||||
runs; different seed diverges. ✅
|
runs; different seed diverges. ✅
|
||||||
- **Hidden information:** `playerView` projection (e.g. `STRIP_SECRET`) —
|
- **Hidden information:** `playerView` projection (e.g. `STRIP_SECRET`) —
|
||||||
server strips secret state per player.
|
server strips secret state per player. *Not exercised by our workload*
|
||||||
|
(provenance note: commit/reveal shape measured; hidden-info cost not).
|
||||||
- **Simultaneous actions:** `activePlayers` stages give simultaneous move
|
- **Simultaneous actions:** `activePlayers` stages give simultaneous move
|
||||||
windows; no built-in cryptographic commit/reveal — commitment is plain
|
windows; no built-in cryptographic commit/reveal — commitment is plain
|
||||||
state the server can see (fine for server-authoritative, nothing for
|
state the server can see.
|
||||||
peer settings).
|
|
||||||
- **Maturity:** 12.4k GitHub stars, but **inactive** — last npm release
|
- **Maturity:** 12.4k GitHub stars, but **inactive** — last npm release
|
||||||
0.50.2 ≈ 4 years ago (Snyk: "maintenance: Inactive").
|
0.50.2 ≈ 4 years ago (checkable on npm; Snyk lists maintenance Inactive).
|
||||||
- **Measured performance** (our harness, synthetic 3-player GROUND-shaped
|
- **Measured performance** (harness, synthetic 3-player GROUND-shaped
|
||||||
commit/reveal workload, Node v24, this machine `bnt-lap001`):
|
commit/reveal workload, Node v24, `bnt-lap001`; run-to-run variance on
|
||||||
|
this machine is material — ±25% observed on identical configs):
|
||||||
|
|
||||||
| applied moves | moves/s | elapsed | RSS |
|
| applied moves | default (moves/s) | `disableUndo` (moves/s) | RSS after run (isolated) |
|
||||||
|---:|---:|---:|---:|
|
|---:|---:|---:|---:|
|
||||||
| 5,000 | 1,930 | 2.6 s | 224 MB |
|
| 5,000 | 1,439–1,930 | 1,118 | ~100 MB |
|
||||||
| 10,000 | 1,605 | 6.2 s | 229 MB |
|
| 10,000 | 1,605 | 1,076 | 132 MB |
|
||||||
| 20,000 | 870 | 23.0 s | 271 MB |
|
| 20,000 | 870 | 942 | 153 MB |
|
||||||
| 100,000 | did not finish in 300 s | — | — |
|
| 40,000 | — | 733 | 232 MB |
|
||||||
|
| 100,000 | DNF @300 s | DNF @240 s | — |
|
||||||
|
|
||||||
**Per-move cost grows with history length** (log accumulation + state
|
**Two degradation mechanisms, separately attributed:** (a) the undo
|
||||||
pipeline): throughput halves as move count doubles — superlinear total
|
stack — an O(n) per-move array spread with full state snapshots — is the
|
||||||
cost. This is architectural (unbounded redux log per client), not a
|
dominant superlinear term at small N and is disable-able via the
|
||||||
tuning artifact.
|
documented `disableUndo` flag; (b) with undo disabled, throughput still
|
||||||
|
falls 34% from 5k→40k and 100k still does not finish, consistent with
|
||||||
|
unbounded client log/deltalog accumulation, which has no client-side off
|
||||||
|
switch in 0.50.2. Even the best configuration degrades with history.
|
||||||
|
(`updatePlayerID` in the timed loop measured separately: ~2 µs/call,
|
||||||
|
<0.5% contamination.)
|
||||||
- **Weight:** 120 transitive npm packages, 37 MB `node_modules`, core
|
- **Weight:** 120 transitive npm packages, 37 MB `node_modules`, core
|
||||||
package 3.9 MB.
|
package 3.9 MB (independently re-verified in review).
|
||||||
|
|
||||||
### 2. Tabletop Simulator scripting model (Lua) — cited
|
### 2. Tabletop Simulator scripting model (Lua) — cited
|
||||||
|
|
||||||
|
|
@ -65,50 +73,75 @@ semantics*, not a rules kernel.
|
||||||
physics is primary (sandbox mode in Clay-Borg terms).
|
physics is primary (sandbox mode in Clay-Borg terms).
|
||||||
- **Determinism/replay:** none. Hidden info via hand zones (engine feature,
|
- **Determinism/replay:** none. Hidden info via hand zones (engine feature,
|
||||||
not a projection model).
|
not a projection model).
|
||||||
- Valuable as the pattern source for object-attached behavior and hand
|
- Pattern source for object-attached behavior and hand zones;
|
||||||
zones; architecturally the anti-model for an authoritative kernel.
|
architecturally the anti-model for an authoritative kernel.
|
||||||
|
|
||||||
### 3. Rune SDK (JS) — cited
|
### 3. Rune SDK (JS) — cited
|
||||||
|
|
||||||
Modern (active, 2024–2026) deterministic multiplayer engine for casual web
|
Modern (active) deterministic multiplayer engine for casual web games.
|
||||||
games.
|
|
||||||
|
|
||||||
- **Data model/mutation:** pure `logic.js` — game state + action functions,
|
- **Data model/mutation:** pure `logic.js` — game state + action functions,
|
||||||
statically checked for nondeterminism (mutation escape, `Math.random`
|
statically checked for nondeterminism (mutation escape, `Math.random`
|
||||||
patched deterministic).
|
patched deterministic).
|
||||||
- **Sync:** predict-rollback: all clients + server simulate the same
|
- **Sync:** predict-rollback: all clients + server simulate the same
|
||||||
deterministic logic; server authoritative, clients predict. Strongest
|
deterministic logic. Strongest determinism *discipline* of the
|
||||||
determinism *discipline* of the candidates — enforced by tooling, not
|
candidates — enforced by tooling, not convention.
|
||||||
convention.
|
- **Limits:** platform-bound (Rune's hosted ecosystem), not an embeddable
|
||||||
- **Limits:** platform-bound (Rune's hosted app ecosystem), not an
|
open kernel; no phase framework, hidden-information projection, or
|
||||||
embeddable open kernel; no phase/stage framework, hidden-information
|
event-sourced replay surface.
|
||||||
projection, or event-sourced replay surface comparable to boardgame.io.
|
|
||||||
|
|
||||||
### 4. Event-sourcing kernels (Rust `cqrs-es` pattern / EventStoreDB) — cited
|
### 4. OpenSpiel (C++/Python, DeepMind) — cited *(added after review)*
|
||||||
|
|
||||||
The general-purpose form of our mutation pipeline (command → validate →
|
Research games kernel explicitly built for **simultaneous-move and
|
||||||
events → fold).
|
imperfect-information games**.
|
||||||
|
|
||||||
- Aggregates validate commands and emit events; state is a fold over the
|
- Deterministic state; serializable; replay via action histories;
|
||||||
append-only log; snapshots bound replay cost. Replay/audit are native.
|
information-state abstractions for imperfect information; large game
|
||||||
- **Performance (cited/estimated):** in-process Rust event application is
|
library. Active. As an in-process C++ kernel it is also a credible D3
|
||||||
memory-bandwidth-bound — order 10⁵–10⁶ small events/s per core is the
|
comparator (unmeasured here — open follow-up).
|
||||||
commonly reported range for fold-style aggregates; dedicated stores
|
- **Limits:** research-oriented — no client visibility *projection* layer,
|
||||||
(EventStoreDB) sustain tens of thousands of appends/s over the network.
|
no networking/session model, no snapshot format, monolithic C++/Python
|
||||||
No game semantics: phases, visibility, simultaneity all DIY.
|
build rather than an embeddable capability boundary. The semantics
|
||||||
|
overlap with our kernel is real; the production layer is absent.
|
||||||
|
|
||||||
### 5. bevy_ecs 0.x (Rust) — cited
|
### 5. Ludii / GGP-GDL lineage (JVM) — cited *(added after review)*
|
||||||
|
|
||||||
Archetypal ECS; the world/spatial layer in our architecture, surveyed as a
|
General game systems whose core value is **ease of rule specification**
|
||||||
kernel candidate for completeness.
|
(the D1 dimension): games written as ludemes (Ludii, 1,000+ games) or GDL
|
||||||
|
rules, executed by a general engine.
|
||||||
|
|
||||||
- Cache-friendly iteration: millions of entity-component accesses per frame
|
- Proof that rule description languages can cover enormous game spaces
|
||||||
(cited from Bevy's own benches; ns-scale per component access).
|
compactly; the D1 literature our kernel spec should be checked against.
|
||||||
- No authoritative command/event pipeline, no replay, no hidden-info
|
- **Limits:** research/archival focus; no production multiplayer stack,
|
||||||
projection; determinism requires care (system ordering, hash maps).
|
no hidden-info projection for clients, JVM-bound; performance oriented
|
||||||
Confirms the ADR-anticipated split: ECS for world representation,
|
to AI playouts, not authoritative session serving.
|
||||||
**typed aggregates for the semantic kernel** — not a competitor on this
|
|
||||||
capability.
|
### 6. Event-sourcing kernels (Rust `cqrs-es` pattern / EventStoreDB) — cited
|
||||||
|
|
||||||
|
Aggregates validate commands and emit events; state is a fold over the
|
||||||
|
append-only log; snapshots bound replay cost. Replay/audit native.
|
||||||
|
**Performance: estimated** — order 10⁵–10⁶ small events/s per core for
|
||||||
|
in-process fold-style application; dedicated stores sustain tens of
|
||||||
|
thousands of appends/s over the network. *No reproducible citation held;
|
||||||
|
treated as directional only.* No game semantics.
|
||||||
|
|
||||||
|
### 7. bevy_ecs (Rust) — cited
|
||||||
|
|
||||||
|
Archetypal ECS; the world/spatial layer in our architecture. Millions of
|
||||||
|
entity-component accesses per frame (Bevy's published benches). No
|
||||||
|
authoritative command/event pipeline, replay, or hidden-info projection —
|
||||||
|
confirms the split: ECS for world representation, typed aggregates for the
|
||||||
|
semantic kernel.
|
||||||
|
|
||||||
|
### Secondary references (not fully surveyed)
|
||||||
|
|
||||||
|
- **Board Game Arena Studio** — dominant commercial turn-based rules
|
||||||
|
framework (PHP): server-authoritative state machine, hidden info, full
|
||||||
|
replay. Closed platform; stronger commercial reference than TTS.
|
||||||
|
- **Colyseus** — active JS authoritative-state multiplayer server; room
|
||||||
|
state sync without game-rules semantics (no phases/legality/replay
|
||||||
|
framework). The "active JS" counterweight to inactive boardgame.io.
|
||||||
|
- **Vassal** — module player for humans; no programmable rules kernel.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -116,39 +149,53 @@ kernel candidate for completeness.
|
||||||
|
|
||||||
| Dimension | Baseline holder | Metric | Value | Provenance |
|
| Dimension | Baseline holder | Metric | Value | Provenance |
|
||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| D1 ease of specification | boardgame.io | LOC to express the synthetic 3p commit/reveal game (declarative object) | ~45 LOC | measured (harness bench.js game def) |
|
| D1 ease of specification | boardgame.io | LOC to express the synthetic 3p commit/reveal game (declarative object) | ~36 LOC | measured (bench.js); gameable — secondary indicator |
|
||||||
| D1 | — (no candidate) | rule-to-scenario traceability (M-D1-COV) | 0 % — none of the candidates link rules to tests | measured/observed |
|
| D1 (qualitative bar) | Ludii/GDL | rule-description-language coverage of large game spaces | qualitative | cited |
|
||||||
| D2 implementation weight | boardgame.io | transitive deps / install size | 120 pkgs / 37 MB | measured |
|
| D2 implementation weight | boardgame.io | transitive deps / install size | 120 pkgs / 37 MB | measured |
|
||||||
| D3 throughput | boardgame.io | applied moves/s, 3p workload @5k moves | 1,930 moves/s | measured, bnt-lap001 |
|
| D3 throughput | boardgame.io (`disableUndo`, best config) | applied moves/s, 3p workload @5k moves | ~1,100 (default config 1,400–1,900; ±25% machine variance) | measured, bnt-lap001 |
|
||||||
| D3 scaling | boardgame.io | throughput @20k vs @5k moves | 0.45× (superlinear cost) | measured, bnt-lap001 |
|
| D3 scaling | boardgame.io (`disableUndo`, best config) | throughput @40k vs @5k moves | 0.66× (default config: 0.45× @20k; both DNF @100k) | measured, bnt-lap001 |
|
||||||
| D3 memory | boardgame.io | RSS @5k moves | 224 MB | measured, bnt-lap001 |
|
| D3 memory | boardgame.io | RSS after run, isolated process | ~100 MB @5k → 232 MB @40k | indicative (raw RSS incl. Node baseline) |
|
||||||
| D3 ceiling (adjacent layer) | in-proc event-sourcing (Rust) | events applied/s per core | ~10⁵–10⁶ | cited/estimated — directional, caps our verdict at parity unless we measure a Rust comparator |
|
|
||||||
| D4 optionality | Rune | determinism enforced by tooling | static nondeterminism checks | cited |
|
| D4 optionality | Rune | determinism enforced by tooling | static nondeterminism checks | cited |
|
||||||
| D4 | boardgame.io | replaceability of subsystems | plugin API, but JS-ecosystem-locked; no null/reference impl pattern | observed |
|
| D4 | boardgame.io | replaceability of subsystems | plugin API, but JS-ecosystem-locked; no null/reference impl pattern | observed |
|
||||||
|
|
||||||
**Headline benchmark-to-beat for the Clay-Borg kernel (proposed for the
|
Observation (not a baseline row): no surveyed candidate links numbered
|
||||||
ADR):** ≥ 100,000 applied events/s sustained with **flat scaling** (throughput
|
rules to test scenarios (M-D1-COV-style traceability); any nonzero
|
||||||
@100k events within 10% of @5k), deterministic replay bit-identical, on the
|
coverage exceeds the field, so this is a direction, not a bar.
|
||||||
same machine and workload shape as the boardgame.io harness.
|
|
||||||
|
**Headline target for the Clay-Borg kernel (stipulated engineering target,
|
||||||
|
not competitor-derived):** ≥ 100,000 applied events/s sustained with **flat
|
||||||
|
scaling** (throughput @100k events within 10% of @5k), deterministic replay
|
||||||
|
bit-identical, same machine and workload shape as the harness. The
|
||||||
|
event-sourcing ceiling row is an estimate; until a Rust comparator is
|
||||||
|
measured locally (open follow-up), evidence rows leaning on it cap at
|
||||||
|
parity per MetricsAndScenarios §3.
|
||||||
|
|
||||||
## Verdict
|
## Verdict
|
||||||
|
|
||||||
- **Per dimension:** D1 — boardgame.io's declarative game object is the bar
|
- **D1:** boardgame.io's declarative game object is the bar to match for
|
||||||
to match; nobody has rule-to-scenario traceability (open surpass lane).
|
compactness; Ludii/GDL is the literature bar for rule-language
|
||||||
D2 — boardgame.io's 120-dep footprint is beatable by an order of
|
expressiveness. Rule-to-scenario traceability is an open lane no one
|
||||||
magnitude in Rust. D3 — boardgame.io is slow *and* degrades; the honest
|
occupies.
|
||||||
comparison class is in-proc event sourcing (10⁵–10⁶/s), and our D3
|
- **D2:** boardgame.io's 120-dep/37 MB footprint is beatable by an order
|
||||||
advantage over boardgame.io is partly language choice — the meaningful
|
of magnitude.
|
||||||
target is **flat scaling + the 100k/s floor**, not the ×50 headline.
|
- **D3:** boardgame.io degrades with history even in its best measured
|
||||||
D4 — Rune's tooling-enforced determinism is the discipline to assimilate;
|
configuration; our advantage over it is partly language choice, so the
|
||||||
no candidate offers a null/reference/optimized port pattern.
|
meaningful commitment is the stipulated flat-scaling + 100k/s target,
|
||||||
- **What none of them do:** combine deterministic replayable authoritative
|
not the multiplier over boardgame.io.
|
||||||
state, first-class simultaneous commit/reveal with hidden-information
|
- **D4:** Rune's tooling-enforced determinism is the discipline to
|
||||||
projection, flat per-event cost with snapshots, and an embeddable
|
assimilate; no candidate offers a null/reference/optimized port pattern
|
||||||
language-portable boundary. That combination is the surpass opportunity.
|
or a WIT-style embeddable boundary.
|
||||||
- **Risks in these baselines:** the boardgame.io harness measures the
|
- **The surpass opportunity, stated precisely:** OpenSpiel covers
|
||||||
headless *client* pipeline (includes subscription/log overhead — canonical
|
simultaneous-move + imperfect-information *semantics* with deterministic
|
||||||
usage, but a bare server-side master could differ); the event-sourcing
|
replay; boardgame.io covers declarative rules + client projection +
|
||||||
numbers are cited, not locally measured; TTS and Rune numbers are
|
networking; Rune covers enforced determinism; event sourcing covers
|
||||||
qualitative. The D3 event-sourcing row is directional and caps related
|
flat-cost replay with snapshots. **No candidate combines** the semantic
|
||||||
evidence verdicts at parity per MetricsAndScenarios §3.
|
coverage with a production projection/networking layer, snapshot-bounded
|
||||||
|
flat-cost replay, and an embeddable, language-portable capability
|
||||||
|
boundary. That combination — not raw speed over an inactive JS library —
|
||||||
|
is what the Clay-Borg kernel should be built to demonstrate.
|
||||||
|
- **Risks that remain:** event-sourcing D3 row unsourced (estimate;
|
||||||
|
parity-cap active); OpenSpiel unmeasured locally; harness measures the
|
||||||
|
headless client pipeline (canonical usage — a bare server Master could
|
||||||
|
differ); ±25% run-to-run variance on this machine; hidden-info cost
|
||||||
|
unmeasured.
|
||||||
|
|
|
||||||
|
|
@ -10,6 +10,8 @@ const { Stage } = require('boardgame.io/core');
|
||||||
const crypto = require('crypto');
|
const crypto = require('crypto');
|
||||||
|
|
||||||
const NUM_MOVES = parseInt(process.argv[2] || '100000', 10);
|
const NUM_MOVES = parseInt(process.argv[2] || '100000', 10);
|
||||||
|
const DISABLE_UNDO = process.argv.includes('--disable-undo');
|
||||||
|
const SINGLE_RUN = process.argv.includes('--single-run'); // isolated memory measurement
|
||||||
const NUM_PLAYERS = 3;
|
const NUM_PLAYERS = 3;
|
||||||
|
|
||||||
const SyntheticGround = {
|
const SyntheticGround = {
|
||||||
|
|
@ -51,7 +53,7 @@ const SyntheticGround = {
|
||||||
|
|
||||||
function makeClient(seed) {
|
function makeClient(seed) {
|
||||||
return Client({
|
return Client({
|
||||||
game: { ...SyntheticGround, seed },
|
game: { ...SyntheticGround, seed, disableUndo: DISABLE_UNDO },
|
||||||
numPlayers: NUM_PLAYERS,
|
numPlayers: NUM_PLAYERS,
|
||||||
playerID: '0',
|
playerID: '0',
|
||||||
});
|
});
|
||||||
|
|
@ -89,13 +91,14 @@ function run(seed, numMoves, collectHash) {
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- throughput ---
|
// --- throughput ---
|
||||||
const warm = run(42, Math.min(NUM_MOVES, 5000), false); // warmup/JIT
|
if (!SINGLE_RUN) run(42, Math.min(NUM_MOVES, 5000), false); // warmup/JIT
|
||||||
const main = run(42, NUM_MOVES, true);
|
const main = run(42, NUM_MOVES, true);
|
||||||
|
const memAfterMain = process.memoryUsage();
|
||||||
|
|
||||||
// --- determinism: same seed twice, different seed once ---
|
// --- determinism: same seed twice, different seed once ---
|
||||||
const detA = run(7, 4000, true);
|
const detA = SINGLE_RUN ? {hash: 'skipped'} : run(7, 4000, true);
|
||||||
const detB = run(7, 4000, true);
|
const detB = SINGLE_RUN ? {hash: 'skipped'} : run(7, 4000, true);
|
||||||
const detC = run(8, 4000, true);
|
const detC = SINGLE_RUN ? {hash: 'x'} : run(8, 4000, true);
|
||||||
|
|
||||||
// --- memory ---
|
// --- memory ---
|
||||||
const mem = process.memoryUsage();
|
const mem = process.memoryUsage();
|
||||||
|
|
@ -112,5 +115,8 @@ const result = {
|
||||||
determinism_diff_seed_differs: detA.hash !== detC.hash,
|
determinism_diff_seed_differs: detA.hash !== detC.hash,
|
||||||
rss_mb: Math.round(mem.rss / 1048576),
|
rss_mb: Math.round(mem.rss / 1048576),
|
||||||
heap_mb: Math.round(mem.heapUsed / 1048576),
|
heap_mb: Math.round(mem.heapUsed / 1048576),
|
||||||
|
rss_after_main_mb: Math.round(memAfterMain.rss / 1048576),
|
||||||
|
disable_undo: DISABLE_UNDO,
|
||||||
|
single_run: SINGLE_RUN,
|
||||||
};
|
};
|
||||||
console.log(JSON.stringify(result, null, 2));
|
console.log(JSON.stringify(result, null, 2));
|
||||||
|
|
|
||||||
|
|
@ -4,13 +4,68 @@
|
||||||
"machine": "bnt-lap001 (WSL2)",
|
"machine": "bnt-lap001 (WSL2)",
|
||||||
"recorded": "2026-07-31",
|
"recorded": "2026-07-31",
|
||||||
"workload": "synthetic-ground 3p commit/reveal (bench.js)",
|
"workload": "synthetic-ground 3p commit/reveal (bench.js)",
|
||||||
"runs": [
|
|
||||||
{"applied_moves": 5000, "moves_per_sec": 1930, "elapsed_ms": 2590, "rss_mb": 224},
|
|
||||||
{"applied_moves": 10000, "moves_per_sec": 1605, "elapsed_ms": 6232, "rss_mb": 229},
|
|
||||||
{"applied_moves": 20000, "moves_per_sec": 870, "elapsed_ms": 22978, "rss_mb": 271},
|
|
||||||
{"applied_moves": 100000, "note": "did not finish within 300s; killed"}
|
|
||||||
],
|
|
||||||
"determinism_same_seed": true,
|
"determinism_same_seed": true,
|
||||||
"determinism_diff_seed_differs": true,
|
"determinism_diff_seed_differs": true,
|
||||||
"dependency_stats": {"transitive_packages": 120, "node_modules_size": "37M", "core_package_size": "3.9M"}
|
"dependency_stats": {
|
||||||
}
|
"transitive_packages": 120,
|
||||||
|
"node_modules_size": "37M",
|
||||||
|
"core_package_size": "3.9M"
|
||||||
|
},
|
||||||
|
"runs_default_config": [
|
||||||
|
{
|
||||||
|
"applied_moves": 5000,
|
||||||
|
"moves_per_sec": 1930,
|
||||||
|
"elapsed_ms": 2590,
|
||||||
|
"rss_mb": 224
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"applied_moves": 10000,
|
||||||
|
"moves_per_sec": 1605,
|
||||||
|
"elapsed_ms": 6232,
|
||||||
|
"rss_mb": 229
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"applied_moves": 20000,
|
||||||
|
"moves_per_sec": 870,
|
||||||
|
"elapsed_ms": 22978,
|
||||||
|
"rss_mb": 271
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"applied_moves": 100000,
|
||||||
|
"note": "did not finish within 300s; killed"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"runs_disable_undo": [
|
||||||
|
{
|
||||||
|
"applied_moves": 5000,
|
||||||
|
"moves_per_sec": 1118,
|
||||||
|
"rss_after_main_mb": 100
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"applied_moves": 10000,
|
||||||
|
"moves_per_sec": 1076,
|
||||||
|
"rss_after_main_mb": 132
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"applied_moves": 20000,
|
||||||
|
"moves_per_sec": 942,
|
||||||
|
"rss_after_main_mb": 153
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"applied_moves": 40000,
|
||||||
|
"moves_per_sec": 733,
|
||||||
|
"elapsed_ms": 54595,
|
||||||
|
"rss_after_main_mb": 232
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"applied_moves": 100000,
|
||||||
|
"note": "did not finish within 240s (single-run mode); killed"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"controls": {
|
||||||
|
"with_undo_5k_recheck_moves_per_sec": 1439,
|
||||||
|
"run_to_run_variance_note": "\u00b125% observed on identical config (1930 vs 1439 @5k with undo)",
|
||||||
|
"updatePlayerID_calls_per_sec": 466508
|
||||||
|
},
|
||||||
|
"revision": "2026-07-31 post-adversarial-review: added disableUndo runs, isolated memory, controls"
|
||||||
|
}
|
||||||
|
|
@ -97,7 +97,7 @@ history/YYMMDD-game-kernel-research.md per InnerLoop §Step 2.
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: CB-WP-0001-T04
|
id: CB-WP-0001-T04
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "2024c17d-9a37-48d2-8cb6-0fb3d8a19691"
|
state_hub_task_id: "2024c17d-9a37-48d2-8cb6-0fb3d8a19691"
|
||||||
```
|
```
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue