diff --git a/specs/GameDesign.md b/specs/GameDesign.md new file mode 100644 index 0000000..9dd605f --- /dev/null +++ b/specs/GameDesign.md @@ -0,0 +1,169 @@ +# GameDesign — findings about the game, and the trials that produce them + +v1.0 — CB-WP-0022 T04, 2026-08-05. Normative. Implements +[ADR-0012](../decisions/ADR-0012-the-design-instrument.md); INTENT +property 4 (*Instrument*) is what this spec discharges. + +**This is not a register.** The register is +[`GroundRules.md §Underdetermined`](GroundRules.md) — ADR-0012 D2. This +document says what may go in it, what a reproduction must show, how a +finding moves and dies, and how a trial game is run so *"we played it and +X happened"* is replayable rather than remembered. + +--- + +## 1. Admissibility + +> A finding is admissible when its reproduction **exists**, **has the +> ruled shape**, and **can fail**. All three. (ADR-0012 D3.) + +### 1.1 Exists + +One of: a scenario under `scenarios/ground/`, an arithmetic check that +prints the contradiction, a `.cbreplay` bundle or recorded scenario the +reader can replay, or a **named test** — `crate::module::test_name`, so it +can be run. + +The register entry names the artifact **by path**. A finding whose +artifact path does not resolve is not admissible, and §4's check stats the +file rather than pattern-matching the string — `design-baseline.py`'s +`has_reproduction` matched `startswith("scenarios/")` and its own positive +control was green against a file that had been deleted (ADR-0012 D8). + +### 1.2 Has the ruled shape + +Per `ground-game`, GROUND-WP-0004 T02: + +> An arithmetic finding ships a **row-level table** — Surface and each +> hidden priority listed **separately** — never *"sum of file"*, never +> *"deal depth N"*. + +**A reproduction must show the quantity the prose claims.** This is the +clause that exists because passing artifacts accompanied wrong +descriptions: + +| wrong premise | what shipped | what a row-level table would have shown | +|---|---|---| +| *"12 in the file"* | a sum | which rows, at which priorities | +| *"4/6/9 against 5/7/9"* | a scenario, green | 6/9/12 — the deal, per row | +| *"SOLVE offered on a face-down Problem"* | a condition named | which filter actually fired | + +**Requirement.** Where a finding asserts a number, the register entry +quotes the number **and** the artifact prints the rows it came from. A +finding stating a total without its rows is inadmissible even if the total +is right — *"12"* was arithmetically defensible and still wrong about the +game. + +### 1.3 Can fail + +The artifact must be capable of going red, and the register records its +**current colour**. A reproduction that has gone **green is not a closed +finding** — it is a finding whose claim is now in question, and §4 reports +it as needing resolution to `applied` or `withdrawn`. + +**Rationale, and it is the expensive one.** `gr-e01-threshold-…-2p.yaml` +went green when the edition landed on 2026-08-04. The finding remained +admissible, remained in the register, and remained queued for transmission +to `ground-game` for four days, because nothing treated a passing artifact +as a signal. It was stopped by an adversarial review, not by a control. + +## 2. Kinds + +Five (ADR-0012 D4). **No kind exists without a finding that instantiates +it**; a sixth appearing during backfill means the taxonomy was invented +rather than observed, and T07 must say so. + +`underdetermined` · `inconsistent` · `inert` · `degenerate` · `unplayed` + +## 3. States, owners, and death + +``` +raised ──► reported ──► ruled ──► applied + │ + └──────────► withdrawn ◄──────────┘ +``` + +- `applied` means **the source changed** — dataset or rules text — and our + provisional default was deleted. Not *"a ruling was recorded."* +- **The queue is `raised` + `reported`. The log is `applied` + + `withdrawn`.** `make design` reports the queue by default; the log only + on request. If the default view mixes them, the queue property is gone. +- **`withdrawn` is reported downstream, not deleted.** A claim retracted + silently is how three wrong premises survived. + +### 3.1 Notes + +A finding without a reproduction is a **note** (ADR-0012 D6). A note: + +1. **has no `reported` state** — it may not cross the repo boundary; +2. **expires at 30 days**, flagged by §4 on the same mechanism that ages + provisional items (`rule-coverage.py:54`, `PROVISIONAL_WARN_DAYS`); +3. is promoted only by acquiring a reproduction meeting §1 — there is no + path from `note` to `reported` that skips it. + +## 4. Metrics + +`make design` reports these. Every number is computed **over the +register**, never from a maintained list (ADR-0012 D8). + +| metric | target | notes | +|---|---|---| +| findings with a resolving reproduction | **100%** | denominator includes `withdrawn`; notes counted separately, never folded in | +| **reproductions that have gone green while open** | **0** | the §1.3 alarm. Non-zero is a queue that needs reading, not a passing suite | +| arithmetic findings whose artifact prints rows | **100%** | §1.2 | +| median days `raised` → `reported` | the number this exists to fix | the baseline it must beat is **4 days** to first *read* | +| open findings by kind, with age | reported, not targeted | | +| notes past 30 days | **0** | | +| findings closed by a ruling vs still open | reported, with age | | + +**Falsifier for this spec.** If a finding reaches `ground-game` whose +prose describes a quantity its own artifact does not compute, §1.2 has +failed and the next revision must **generate the description from the +artifact** rather than let it be written alongside. + +**No baseline rate is quoted here.** CB-RES-0007's 33% was withdrawn as a +measured figure (ADR-0012 D8); the first honest denominator is the one +T05's backfill produces. + +## 5. The trial protocol + +The part with no precedent in this repo. A **trial game** is a recorded +session plus an observation log — so a claim from play is replayable. + +**It must cost almost nothing or it will not be done.** The engine already +records: `cb-play --record FILE` writes the finished game as a scenario +YAML (`main.rs:30`), and `games/ground/src/record.rs` is trustworthy +because its round trip is the test. + +A trial is therefore: + +``` +cb-play --players N --serve PORT --record trials/-.yaml +``` + +plus a sibling `trials/-.md` carrying, and nothing more: + +- **who played which seats**, and which were bots (with policy); +- **what was observed** — in the player's own words, not cleaned up; +- **the seed**, which the scenario already carries; +- **which observations became findings or notes**, by register id, or + explicitly none. + +**Rules.** + +1. An observation is a **note** until it has a reproduction (§3.1). A + trial log is not a finding source by itself — it is where findings are + *noticed*. +2. **The recording is the artifact for `unplayed` findings.** GR-E03/E04 + were "never played to the end"; a trial that plays them to the end both + closes that finding and produces the reproduction for anything found on + the way. +3. A trial whose recording does not replay is a **bug in the engine**, + reported as one, not a design finding. + +**Why the log is prose and stays prose.** The maintainer's *"I felt it was +too easy but then we lost, so who knows"* is exactly the observation this +protocol exists to catch: real, unreproducible as stated, and the seed of +[CB-WP-0025](../workplans/CB-WP-0025-could-we-have-won.md). Forcing it +into a schema at the moment of observation would lose it. §3.1 is what +stops it aging into an apparent finding. diff --git a/workplans/CB-WP-0022-the-design-instrument.md b/workplans/CB-WP-0022-the-design-instrument.md index 3e38523..90325b6 100644 --- a/workplans/CB-WP-0022-the-design-instrument.md +++ b/workplans/CB-WP-0022-the-design-instrument.md @@ -287,7 +287,7 @@ under its own workplan, citing the reproduction by path. ```task id: CB-WP-0022-T04 -status: todo +status: done priority: high state_hub_task_id: "60ddfeab-81b1-45e4-9ec9-91cc8ea7fb72" ``` @@ -328,6 +328,36 @@ here: a trial game is a `--record`ed session plus an observation log, so engine already records sessions as scenarios; a trial is that plus notes, and it must cost almost nothing or it will not be done. +**Done 2026-08-05.** [specs/GameDesign.md](../specs/GameDesign.md) v1.0. + +**It is not a register, and says so in its second line** — ADR-0012 D2 put +the register in `§Underdetermined`, so this spec says what may go in it, +what a reproduction must *show*, and how a finding dies. + +**§1.2 had to be written against evidence rather than principle.** A +finding must now print the rows behind any number it claims, and the spec +carries the table of what shipped instead: a sum, a green scenario, and a +condition named without checking which one fired. *"12" was arithmetically +defensible and still wrong about the game* — that sentence is the whole +requirement. + +**§1.3's target is `0` reproductions that have gone green while open.** +That is the metric GR-E01 would have tripped four days before a human +caught it. + +**No baseline rate is quoted.** C2 withdrew the 33%, and the first honest +denominator is T05's backfill — quoting a new number from a discredited +instrument is how the first one got in. + +**The trial protocol costs one flag.** `cb-play --record` already writes a +finished game as a scenario and `record.rs`'s round trip is its own test, +so a trial is that plus a sibling `.md` in the player's own words. The +rule that makes it safe: an observation is a **note** until it has a +reproduction, and a trial log is where findings are *noticed*, not a +finding source. The maintainer's *"I felt it was too easy but then we +lost"* is the case it is shaped around — forcing that into a schema at the +moment of observation would lose it. + ## Task: build it, and backfill what is already known ```task