---
id: CB-WP-0022
kind: product
title: "The design instrument: findings about the game, with their reproductions"
status: active
state_hub_workstream_id: "fda16340-0049-4acf-884b-a5cfdbde47c0"
---

# Purpose

```
structural tier  L   (named a high-leverage pass by the maintainer, and it
                      amends INTENT — clay-borg gains a stated aspect)
chaos            d8 = 6  → no override
declared tier    L
```

Declaration 5 of chaos window 2. Tier L: separate survey, **adversarial
review**, ADR, then spec, then code.

## The insight, in the maintainer's words

> *"We should consider ourselves testing the game and document
> inconsistencies to report them back to the ground-game repo, so that the
> game designer can improve the rules accordingly… We should have the
> rigorous game simulation engine and a meta scope to capture notes about
> game design flaws, questions, results and protocols about trial games…
> This will provide clay-borg an additional aspect as a valuable game
> design tool."*

**This is already happening and has no home.** In five passes the engine
has produced, as a by-product of being rigorous:

| finding | how it surfaced |
|---|---|
| ten underdetermined rules points (U1–U10) | formalizing the dataset into testable rules |
| SOLVE offered on a face-down Problem, always inert | a human dragging it three rounds running |
| GR-A13 "wasted SOLVE" on a claimed Problem | a scenario that had to pick a default |
| GR-E01 unreachable below 5 seats | arithmetic over the deal count |
| six provisional scenario defaults | scenarios that could not be written without deciding something |
| GR-E03 / GR-E04 never played to the end | nobody noticed for nineteen passes |

Every one was found by *building the simulator*, not by playing. That is
the thing worth naming: **a simulator rigorous enough to refuse ambiguity
is a design instrument, because it cannot proceed past a rule that does
not decide.**

And every one of them has been carried in prose, in six different places,
and one sat unread in an inbox for four days.

## The load-bearing rule this must have

The project's standing failure is *unexecuted verification*. A design
register that collects opinions would reproduce it in a new medium.

> **A design finding is not admissible without its reproduction.**

Concretely: a scenario that fails, an arithmetic check that prints the
contradiction, a recorded game the reader can replay, or a named test.
*"This feels unbalanced"* is a note, not a finding. **The SOLVE inertness
is admissible because a recorded session shows three no-ops.**

> **The example that stood here was GR-E01, and the adversarial review
> killed it (C1, 2026-08-05).** *"4/6/9 against 5/7/9 is a computation
> anyone can rerun"* was a computation that had already been rerun:
> `2da19a4` measured **6/9/12 against 5/7/9** and renamed the scenario
> `-unreachable-` → `-reachable-`. The finding's conclusion inverted, and
> it was one of the two findings that **passed** this rule.
>
> So existence is not the property that was missing — three wrong premises
> have now reached `ground-game`, and the third satisfied an existence
> test. T03 must adopt the shape requirement as part of the rule, plus a
> clause the survey never contemplated: **a reproduction must be able to
> fail.** Ours went green and stayed admissible.

This is what would make clay-borg a design tool rather than a suggestion
box, and it is the one part of this proposal that must not be traded away
for convenience.

## The judgment I want reviewed, not assumed

The maintainer asked whether this should extend to *"a meta about the
clay-borg engine evolution itself."*

**My answer is no, and it should be argued rather than accepted.** That
register already exists and is load-bearing: `evidence/CB-EV-*` records
what each pass found, `decisions/ADR-*` records what was decided and what
was rejected, `gates.toml` records what every control has caught and what
would retire it, and workplans record what was attempted. Nineteen passes
of engine evolution are already captured, with dates, costs and
falsifiers.

**Building a second register for the same subject would be ceremony**, and
this project has a standing rule that a gate must cash out. The asymmetry
is the point: engine evolution has a home and game design does not.

If the adversarial review disagrees, that is exactly the kind of thing
tier L exists to surface.

## Task: survey how this is done elsewhere, and what we already have

```task
id: CB-WP-0022-T01
status: done
priority: high
state_hub_task_id: "6b8663b8-4143-4361-ad9e-7df0c4f4a19d"
```

`research/CB-RES-0007-*.md`.

**Do not survey issue trackers.** The question is narrower and more
interesting: how do rigorous rule systems record *the ambiguity they
found*? Candidates worth a benchmark-to-beat:

- **Errata and rulings practice** in published games (Magic's
  comprehensive-rules + rulings split, Netrunner's NAPD card rulings) —
  what makes a ruling *findable* years later.
- **Formal-methods counterexample traces** — a model checker's output is
  precisely a reproduction attached to a claim, which is the shape wanted
  here.
- **Conformance-suite provisional behaviour** — how W3C/WHATWG mark
  "implementation-defined" and how a spec later absorbs it.
- **What this repo already has**: `provisional: true` scenarios,
  `§Underdetermined`, `gates.toml`'s `caught`/`retire_if` shape, and the
  hub message that went unread. **The register must reuse the provisional
  machinery rather than compete with it.**

Name, per dimension, the property to beat — findability, reproducibility,
and whether a ruling can *close* a finding mechanically.

**Done 2026-08-03.**
[CB-RES-0007](../research/CB-RES-0007-design-instrument.md), with a
runnable baseline (`tools/design-baseline.py`).

**The baseline is us, and it is measured**: 6 findings across **11 files**
with no index, **2 of 6 (33%) with a runnable reproduction**, and U1–U10
raised 2026-07-30, first *read* 2026-08-03 — **4 days**, 0 of 10 ruled.

**The uncomfortable number is stated up front rather than left for the
review to find: the reproduction rule would reject four of our six
existing findings.** The survey answers it — none of the four is
expensive to reproduce, so the 33% is evidence that nobody was ever asked
for one, not that the rule is unaffordable.

**Magic corrected an assumption this pass was about to build on.** I
expected a ruling to be the authoritative resolution. It is not: rulings
are *"reminder information with no actual weight or rules meaning"*, and
the authoritative fix folds into the **Oracle** card text. **A finding
closes when the source changes, not when an annotation is added** — so
the register must be a queue that empties, not an archive that grows.
That is now a constraint on T03's lifecycle.

Model checkers supplied the reproduction rule independently (a
counterexample trace *is* the finding), and W3C's *implementation-defined*
mark is the one piece of machinery we already have and must reuse rather
than duplicate.

## Task: adversarial review

```task
id: CB-WP-0022-T02
status: done
priority: high
state_hub_task_id: "d2597895-fe2e-4f1e-a3db-a5fb8833434c"
```

Tier L requires it. Give the reviewer the survey **and** the §judgment
above, and require an attempt at:

- **that the engine-evolution register is redundant** — the strongest
  counter is that ADRs record *decisions* and evidence records *findings*,
  but nothing records *what we learned about building engines*, which is a
  third thing;
- **that "carries its reproduction" is affordable** — if half the real
  findings cannot be reproduced cheaply, the rule will be quietly dropped
  and the register becomes a suggestion box anyway. *(Since this task was
  written the answer has hardened: two findings reached ground-game on
  wrong premises, and ground-game has itself ruled that reproductions are
  required. The reviewer should now press the opposite way — whether the
  rule is **sufficient**, given both bad premises would have shipped a
  passing test alongside a wrong description.)*
- **that a register is needed at all**, rather than one more section in
  `GroundRules.md §Underdetermined`, which already exists and already
  works.

Record the trail in `history/`, unpolished.

**Done 2026-08-05.** Trail:
[challenge](../history/260805-design-instrument-challenge.md),
[response](../history/260805-design-instrument-response.md).

**Run by a separate agent** — the first in this repo that was. CB-RES-0006's
review opened by conceding it could not be, and called its own findings
*"a lower bound on what a genuinely separate reviewer would find."* That
was measurable, and this is the measurement: the separate reviewer ran
`git log` against the survey's central example and found our own commit
had falsified it four days earlier, while the author — who wrote that
commit — quoted the dead number twice.

**Seven challenges: four conceded, two conceded in part, one answered.**

- **C1 lands hardest and changed the design.** The rule's showcase finding
  was false and had *passed* the rule. Existence is not the missing
  property; **shape** and **falsifiability** are. Folded into §The
  load-bearing rule above, and it is T03's to settle.
- **C1 also caught a defect in flight** — T06's payload, `todo`, still
  named the dead number. Withdrawn above before sending.
- **C2 withdrew the baseline's precision.** `tools/design-baseline.py` is a
  hand-maintained dict counting itself (`:16-36`, `:89`); `has_reproduction`
  (`:38-43`) never checks the file exists, so the self-test's YES-control
  (`:63`) is green against a path `2da19a4` deleted. `Makefile:127` runs
  only `--self-test`, so the reporting path has no CI. The direction
  stands — 11 files, no index, 0 of 10 ruled are all checkable without the
  tool — but **33% is not a measured rate and T05 must not build on it.**
- **C3**: "six provisional defaults" is five, and GR-E01 is double-counted
  in the `2/6`. No corrected rate is quoted here; the instrument that would
  produce it is the one C2 withdrew.
- **C4**: `§Underdetermined` was never evaluated as a candidate, and it
  already delivers four of five benchmarks including the Oracle property
  the survey went to Magic to find. **T03's burden flips: argue why it is
  extended, not replaced.**
- **C5**: the engine-evolution "third thing" is visible in
  `specs/InnerLoopReference.md` and `history/`'s retrospectives, neither of
  which my redundancy inventory named. Conclusion narrowed, not settled.
- **Survived**: the reproduction rule's *affordability*, and §4's reuse of
  the provisional machinery. Both with stated falsifiers.

## Task: decide

```task
id: CB-WP-0022-T03
status: done
priority: high
state_hub_task_id: "a2e85810-e949-4ea0-81fc-36b912af326c"
```

`decisions/ADR-0012-*.md`. At minimum:

- whether clay-borg's **INTENT gains a stated aspect** as a design
  instrument, and in what words — this is the change with the longest
  half-life in the pass;
- the finding **taxonomy**, and it should be grounded in the six findings
  above rather than invented: *underdetermined* (rules do not say),
  *inconsistent* (rules disagree with each other or the data), *inert* (a
  rule that cannot fire), *degenerate* (fires, but collapses play),
  *unplayed* (implemented, never played);
- the **lifecycle** and who owns each state: raised → reported → ruled →
  applied, or withdrawn;
- whether a finding without a reproduction is **rejected** or **admitted
  as a note** — and if admitted, how it is prevented from aging into an
  apparent finding.

**Done 2026-08-05.**
[ADR-0012](../decisions/ADR-0012-the-design-instrument.md), nine
decisions. The two that were not on this list are the two the review
forced:

- **D2 — `§Underdetermined` *is* the register; nothing parallel is built.**
  C4's point, checked and conceded: against the survey's own five
  benchmarks the incumbent already delivers four, including the Oracle
  property the survey went to Magic to find and we had written down
  ourselves eight days earlier (`GroundRules.md:231-233`). What it lacks
  is reproductions. So the pass **extends** a section instead of building a
  register — no new file, no new schema.
- **D3 — admissibility is three clauses, not one.** Exists, has the ruled
  shape (row-level table, never a sum), **and can fail.** The third is
  C1's: GR-E01's artifact went green when the edition landed and the
  finding stayed admissible and stayed queued, because nothing said a
  passing artifact was a signal. **A green reproduction is an alarm.**

The rest: **D1** INTENT gains property 4, *Instrument* — applied, with the
falsifier in the text. **D4** five kinds, each forced by an existing
finding, and a sixth appearing during backfill means the taxonomy was
invented. **D5** lifecycle with `applied` meaning *the source changed*, the
queue emptying while the log accumulates, and **withdrawals reported, not
deleted** — GR-E01 is why. **D6** notes admitted but never reportable, with
a 30-day expiry on the existing age machinery; refusing them would have
thrown away the only class of finding the engine cannot produce itself,
which is CB-WP-0025's entire input. **D7** no engine-evolution register,
on an inventory C5 corrected — argued and narrowed, not settled. **D8**
`design-baseline.py` retired, not extended; kept as a dated snapshot
because deleting it erases the evidence for how 33% got in. **D9** the
finding and its artifact stay here; ground-game gets a generated file
under its own workplan, citing the reproduction by path.

## Task: specify

```task
id: CB-WP-0022-T04
status: todo
priority: high
state_hub_task_id: "60ddfeab-81b1-45e4-9ec9-91cc8ea7fb72"
```

`specs/GameDesign.md`, with metrics, because a spec without them is prose.

Candidate measures, to be argued not adopted:

- **findings with a runnable reproduction** — target 100%, and the
  denominator includes withdrawn ones;
- **time from raised to reported** — the U-items took four days to be
  *read*; that is the number this exists to fix;
- **findings closed by a ruling** vs **findings still open**, with age.

**ground-game has ruled on what a finding must carry** (GROUND-WP-0004
T02, 2026-08-04), and it is stricter than this pass proposed. Adopt it:

> 1. Arithmetic findings must ship a **runnable reproduction** *and* a
>    **row-level deal table** listing Surface and each hidden priority
>    separately — **never only "sum of file" or "deal depth N"**.
> 2. ground-game rulings that depend on arithmetic **should cite** that
>    reproduction id or path, so a later reader can re-run it.

The second half is theirs to keep. The first is ours, and it is a direct
consequence of both premises we got wrong: *"12 in the file"* was a sum
without a deal table, and *"offered on a face-down Problem"* was a
condition named without checking which one actually fired.

**So the reproduction rule gains a shape requirement, not just an
existence one.** A finding that ships a passing test but describes the
wrong quantity is still a bad finding — which is exactly what happened
twice. The spec must say what a reproduction has to *show*, not only that
one exists.

Also specify the **trial protocol**, which is the part with no precedent
here: a trial game is a `--record`ed session plus an observation log, so
*"we played it and X happened"* is replayable rather than remembered. The
engine already records sessions as scenarios; a trial is that plus notes,
and it must cost almost nothing or it will not be done.

## Task: build it, and backfill what is already known

```task
id: CB-WP-0022-T05
status: todo
priority: high
state_hub_task_id: "469413ba-ed47-4840-a60e-ee7f95dc06ff"
```

The register, the tool, and then **the six findings above entered into
it** — backfilling is the test. A register that cannot express findings
the project already has is the wrong register, and discovering that after
designing it is the point of doing it in this order.

`make design` (or equivalent) must report: open findings by kind, those
without a reproduction, and those never reported to their owner.

## Task: report to ground-game, mechanically

```task
id: CB-WP-0022-T06
status: todo
priority: high
state_hub_task_id: "c6a659bd-c495-4754-9332-baadad51012a"
```

Generate the report and send it. **The message that sat unread for four
days is the baseline to beat**, and the failure was not the message — it
was that nothing pointed at it and nothing tracked whether it was
answered.

So the report must land somewhere that persists: a file in `ground-game`
under its own workplan, not only an inbox entry. GROUND-WP-0002 already
holds the ten U-items; this should extend it rather than duplicate it.

Include the findings this pass has sharpened:

- **SOLVE's legality** against a face-down Problem or an unmatchable suit —
  and note that the case we *reported* was not the case that fired
  (CB-WP-0023 T01).
- ~~**GR-E01 vs GR-S01** — 4/6/9 against 5/7/9, no dataset reconciles
  them~~ — **withdrawn 2026-08-05, before sending.** The adversarial
  review (C1) found `2da19a4` had already measured **6/9/12 against
  5/7/9**: the dataset reconciles them and the scenario is now
  `-reachable-`. Sending this would have been the **fourth** wrong premise
  to reach `ground-game`, and the only one caught before transmission.
  **Report the withdrawal, not the finding** — GROUND-WP-0002 holds the
  original, and a claim retracted silently is how the first three
  survived.

## Task: evidence

```task
id: CB-WP-0022-T07
status: todo
priority: high
state_hub_task_id: "4fb77911-486d-4d9e-a53d-c4e486c74ce2"
```

`evidence/CB-EV-0021-*.md`. *(Was CB-EV-0020 when written; CB-WP-0023
shipped that number first — `evidence/CB-EV-0020-solve-legality.md` — so
this one moves rather than collides.)*

- **Whether backfilling changed the design** — if all six findings fit the
  first taxonomy, say so and be suspicious of it.
- **What tier L cost against what it caught**, since this is the second
  full-weight L pass and CB-WP-0012's deleted its own structural trigger.
- **The engine-evolution question**, as the review left it.
- **Quote CB-WP-0021's cost by re-running the instrument.**