tegwick
561facce43
Some checks failed
ci / check (push) Has been cancelled
T05. tools/design.py, make design, and the register in GroundRules.md --
14 rows, no new file, because ADR-0012 D2 made §Underdetermined the
register rather than building one beside it.
Backfill was the test and it caught two things the ADR did not have.
First, a `role` column. The first report alarmed on U2 and was wrong to:
U2's scenario is green BECAUSE the provisional default it documents is
implemented, which says nothing about whether ground-game agrees. GR-E01's
was a counterexample that went green. Same colour, opposite meaning -- a
register that cannot tell them apart either alarms constantly or never.
Only a green counterexample alarms. Folded back into GameDesign §1.3.
Second, it contradicted the survey. CB-RES-0007 said "six of the ten
already have provisional scenarios." Measured -- grep -lE "\bU<n>\b" over
scenarios/ground -- exactly ONE U-item names itself. Five provisional
scenarios exist and four probably encode U-item defaults, but the mapping
is not written down, so it is not checkable. Same defect class as the
wrong premises, found inside the survey that proposed the fix. Now a
reported debt: open, lacking a reproduction: 9, target 0.
design.py carries the control design-baseline.py never had, asserted
directly: a row citing a nonexistent file must not count as reproduced,
using the exact path 2da19a4 deleted -- which the old tool called green.
design-baseline.py is marked superseded rather than deleted; it is the
evidence for how a wrong number got into a survey.
T06. The report is a FILE in ground-game under GROUND-WP-0002, committed
there, with a hub message that only points at it. It asks for no ruling:
it carries GR-E01's withdrawal, our reproduction debt, and two notes that
are explicitly not findings.
And it had to acknowledge something nobody anticipated. GROUND-WP-0002 is
finished -- all ten U-items were RULED 2026-08-03, every one confirmed as
the default we simulate, plus five of six provisional scenarios. The
survey said "0 of 10 ruled" two days later and this register was built
saying `reported`. That is the unread-inbox failure running in the
opposite direction: they answered and we did not collect it. The
instrument's first run surfaced it. They are `ruled`, not `applied` --
lifting the now-settled provisional flags is owed and is not done, and
make design shows them open until it is.
T07. evidence/CB-EV-0021. Two of six catches in this pass came from
execution rather than process (the role distinction from building it, the
ten uncollected rulings from running it), which is InnerLoop §Design
goal's prediction holding.
make self-tests, facts-check, loop-lint: clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
183 lines
8.1 KiB
Markdown
183 lines
8.1 KiB
Markdown
# GameDesign — findings about the game, and the trials that produce them
|
|
|
|
v1.0 — CB-WP-0022 T04, 2026-08-05. Normative. Implements
|
|
[ADR-0012](../decisions/ADR-0012-the-design-instrument.md); INTENT
|
|
property 4 (*Instrument*) is what this spec discharges.
|
|
|
|
**This is not a register.** The register is
|
|
[`GroundRules.md §Underdetermined`](GroundRules.md) — ADR-0012 D2. This
|
|
document says what may go in it, what a reproduction must show, how a
|
|
finding moves and dies, and how a trial game is run so *"we played it and
|
|
X happened"* is replayable rather than remembered.
|
|
|
|
---
|
|
|
|
## 1. Admissibility
|
|
|
|
> A finding is admissible when its reproduction **exists**, **has the
|
|
> ruled shape**, and **can fail**. All three. (ADR-0012 D3.)
|
|
|
|
### 1.1 Exists
|
|
|
|
One of: a scenario under `scenarios/ground/`, an arithmetic check that
|
|
prints the contradiction, a `.cbreplay` bundle or recorded scenario the
|
|
reader can replay, or a **named test** — `crate::module::test_name`, so it
|
|
can be run.
|
|
|
|
The register entry names the artifact **by path**. A finding whose
|
|
artifact path does not resolve is not admissible, and §4's check stats the
|
|
file rather than pattern-matching the string — `design-baseline.py`'s
|
|
`has_reproduction` matched `startswith("scenarios/")` and its own positive
|
|
control was green against a file that had been deleted (ADR-0012 D8).
|
|
|
|
### 1.2 Has the ruled shape
|
|
|
|
Per `ground-game`, GROUND-WP-0004 T02:
|
|
|
|
> An arithmetic finding ships a **row-level table** — Surface and each
|
|
> hidden priority listed **separately** — never *"sum of file"*, never
|
|
> *"deal depth N"*.
|
|
|
|
**A reproduction must show the quantity the prose claims.** This is the
|
|
clause that exists because passing artifacts accompanied wrong
|
|
descriptions:
|
|
|
|
| wrong premise | what shipped | what a row-level table would have shown |
|
|
|---|---|---|
|
|
| *"12 in the file"* | a sum | which rows, at which priorities |
|
|
| *"4/6/9 against 5/7/9"* | a scenario, green | 6/9/12 — the deal, per row |
|
|
| *"SOLVE offered on a face-down Problem"* | a condition named | which filter actually fired |
|
|
|
|
**Requirement.** Where a finding asserts a number, the register entry
|
|
quotes the number **and** the artifact prints the rows it came from. A
|
|
finding stating a total without its rows is inadmissible even if the total
|
|
is right — *"12"* was arithmetically defensible and still wrong about the
|
|
game.
|
|
|
|
### 1.3 Can fail
|
|
|
|
The artifact must be capable of going red, and the register records its
|
|
**current colour**. A reproduction that has gone **green is not a closed
|
|
finding** — it is a finding whose claim is now in question, and §4 reports
|
|
it as needing resolution to `applied` or `withdrawn`.
|
|
|
|
**A reproduction has a `role`, and only one of them alarms.** This
|
|
distinction was not in ADR-0012; **T05's backfill produced it**, when the
|
|
first report cried wolf over U2.
|
|
|
|
| role | what it is | green means |
|
|
|---|---|---|
|
|
| `counterexample` | demonstrates the defect — the model-checker shape | **alarm.** It should be red while the finding is open |
|
|
| `default` | encodes a provisional choice so it is testable and visible | **expected.** It passes because our default is implemented, which says nothing about whether the owner agrees |
|
|
|
|
U2's scenario is green because the default it documents is implemented.
|
|
GR-E01's was a counterexample that went green when the edition landed, and
|
|
sat that way for four days. **Same colour, opposite meaning** — a register
|
|
that cannot tell them apart either alarms constantly or never.
|
|
|
|
**Rationale, and it is the expensive one.** `gr-e01-threshold-…-2p.yaml`
|
|
went green when the edition landed on 2026-08-04. The finding remained
|
|
admissible, remained in the register, and remained queued for transmission
|
|
to `ground-game` for four days, because nothing treated a passing artifact
|
|
as a signal. It was stopped by an adversarial review, not by a control.
|
|
|
|
## 2. Kinds
|
|
|
|
Five (ADR-0012 D4). **No kind exists without a finding that instantiates
|
|
it**; a sixth appearing during backfill means the taxonomy was invented
|
|
rather than observed, and T07 must say so.
|
|
|
|
`underdetermined` · `inconsistent` · `inert` · `degenerate` · `unplayed`
|
|
|
|
## 3. States, owners, and death
|
|
|
|
```
|
|
raised ──► reported ──► ruled ──► applied
|
|
│
|
|
└──────────► withdrawn ◄──────────┘
|
|
```
|
|
|
|
- `applied` means **the source changed** — dataset or rules text — and our
|
|
provisional default was deleted. Not *"a ruling was recorded."*
|
|
- **The queue is `raised` + `reported`. The log is `applied` +
|
|
`withdrawn`.** `make design` reports the queue by default; the log only
|
|
on request. If the default view mixes them, the queue property is gone.
|
|
- **`withdrawn` is reported downstream, not deleted.** A claim retracted
|
|
silently is how three wrong premises survived.
|
|
|
|
### 3.1 Notes
|
|
|
|
A finding without a reproduction is a **note** (ADR-0012 D6). A note:
|
|
|
|
1. **has no `reported` state** — it may not cross the repo boundary;
|
|
2. **expires at 30 days**, flagged by §4 on the same mechanism that ages
|
|
provisional items (`rule-coverage.py:54`, `PROVISIONAL_WARN_DAYS`);
|
|
3. is promoted only by acquiring a reproduction meeting §1 — there is no
|
|
path from `note` to `reported` that skips it.
|
|
|
|
## 4. Metrics
|
|
|
|
`make design` reports these. Every number is computed **over the
|
|
register**, never from a maintained list (ADR-0012 D8).
|
|
|
|
| metric | target | notes |
|
|
|---|---|---|
|
|
| findings with a resolving reproduction | **100%** | denominator includes `withdrawn`; notes counted separately, never folded in |
|
|
| **reproductions that have gone green while open** | **0** | the §1.3 alarm. Non-zero is a queue that needs reading, not a passing suite |
|
|
| arithmetic findings whose artifact prints rows | **100%** | §1.2 |
|
|
| median days `raised` → `reported` | the number this exists to fix | the baseline it must beat is **4 days** to first *read* |
|
|
| open findings by kind, with age | reported, not targeted | |
|
|
| notes past 30 days | **0** | |
|
|
| findings closed by a ruling vs still open | reported, with age | |
|
|
|
|
**Falsifier for this spec.** If a finding reaches `ground-game` whose
|
|
prose describes a quantity its own artifact does not compute, §1.2 has
|
|
failed and the next revision must **generate the description from the
|
|
artifact** rather than let it be written alongside.
|
|
|
|
**No baseline rate is quoted here.** CB-RES-0007's 33% was withdrawn as a
|
|
measured figure (ADR-0012 D8); the first honest denominator is the one
|
|
T05's backfill produces.
|
|
|
|
## 5. The trial protocol
|
|
|
|
The part with no precedent in this repo. A **trial game** is a recorded
|
|
session plus an observation log — so a claim from play is replayable.
|
|
|
|
**It must cost almost nothing or it will not be done.** The engine already
|
|
records: `cb-play --record FILE` writes the finished game as a scenario
|
|
YAML (`main.rs:30`), and `games/ground/src/record.rs` is trustworthy
|
|
because its round trip is the test.
|
|
|
|
A trial is therefore:
|
|
|
|
```
|
|
cb-play --players N --serve PORT --record trials/<date>-<slug>.yaml
|
|
```
|
|
|
|
plus a sibling `trials/<date>-<slug>.md` carrying, and nothing more:
|
|
|
|
- **who played which seats**, and which were bots (with policy);
|
|
- **what was observed** — in the player's own words, not cleaned up;
|
|
- **the seed**, which the scenario already carries;
|
|
- **which observations became findings or notes**, by register id, or
|
|
explicitly none.
|
|
|
|
**Rules.**
|
|
|
|
1. An observation is a **note** until it has a reproduction (§3.1). A
|
|
trial log is not a finding source by itself — it is where findings are
|
|
*noticed*.
|
|
2. **The recording is the artifact for `unplayed` findings.** GR-E03/E04
|
|
were "never played to the end"; a trial that plays them to the end both
|
|
closes that finding and produces the reproduction for anything found on
|
|
the way.
|
|
3. A trial whose recording does not replay is a **bug in the engine**,
|
|
reported as one, not a design finding.
|
|
|
|
**Why the log is prose and stays prose.** The maintainer's *"I felt it was
|
|
too easy but then we lost, so who knows"* is exactly the observation this
|
|
protocol exists to catch: real, unreproducible as stated, and the seed of
|
|
[CB-WP-0025](../workplans/CB-WP-0025-could-we-have-won.md). Forcing it
|
|
into a schema at the moment of observation would lose it. §3.1 is what
|
|
stops it aging into an apparent finding.
|