tegwick
bff64053ca
The maintainer's observation decided the design: aspects partition the GAME, strata partition our apparatus, and they are orthogonal. A module is one coordinate change in aspect space with an obligation in every stratum. So aspect identity must NOT be Rust types -- an aspect ground-game adds would make clay-borg fail to parse a configuration rather than fail to run it, welding the two coordinate systems at the one place they must stay independent. Chosen: identity as data (Configuration round-trips anything the catalog names), behaviour exhaustive (Rules, no catch-all), resolve() between. Decisive argument: the catalog ALREADY ships modules with a rules_delta and status: proposed, so a per-aspect enum would report them as "unknown module" -- indistinguishable from a typo, a false statement about the edition, and this project's signature failure shape. Two facts need two errors. Federating design authority is permanent, so the representation must outlive the implementation. Legacy ids alias forever through the catalog's own legacy_experiment_id, on the standard-Np precedent: 26 recordings name them and the expansion is exact, so there is nothing to deprecate. T00 done: the schema-2 mirror had arrived with no digests (19 files) and edition-check was red. Digests are now generated by WALKING editions/, not typed -- two reviews already found hand-written lists that made their own controls vacuous, and a mirror that grows a directory is what breaks a maintained list. PROVENANCE-catalog.md was a file inside the mirrored tree that upstream does not have; folded into our own PROVENANCE.md, since provenance about the mirror does not belong inside the thing it describes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
227 lines
10 KiB
Markdown
227 lines
10 KiB
Markdown
# GameDesign — findings about the game, and the trials that produce them
|
|
|
|
v1.0 — CB-WP-0022 T04, 2026-08-05. Normative. Implements
|
|
[ADR-0012](../decisions/ADR-0012-the-design-instrument.md); INTENT
|
|
property 4 (*Instrument*) is what this spec discharges.
|
|
|
|
**This is not a register.** The register is
|
|
[`GroundRules.md §Underdetermined`](GroundRules.md) — ADR-0012 D2. This
|
|
document says what may go in it, what a reproduction must show, how a
|
|
finding moves and dies, and how a trial game is run so *"we played it and
|
|
X happened"* is replayable rather than remembered.
|
|
|
|
---
|
|
|
|
## 1. Admissibility
|
|
|
|
> A finding is admissible when its reproduction **exists**, **has the
|
|
> ruled shape**, and **can fail**. All three. (ADR-0012 D3.)
|
|
|
|
### 1.1 Exists
|
|
|
|
One of: a scenario under `scenarios/ground/`, an arithmetic check that
|
|
prints the contradiction, a `.cbreplay` bundle or recorded scenario the
|
|
reader can replay, or a **named test** — `crate::module::test_name`, so it
|
|
can be run.
|
|
|
|
The register entry names the artifact **by path**. A finding whose
|
|
artifact path does not resolve is not admissible, and §4's check stats the
|
|
file rather than pattern-matching the string — `design-baseline.py`'s
|
|
`has_reproduction` matched `startswith("scenarios/")` and its own positive
|
|
control was green against a file that had been deleted (ADR-0012 D8).
|
|
|
|
### 1.2 Has the ruled shape
|
|
|
|
Per `ground-game`, GROUND-WP-0004 T02:
|
|
|
|
> An arithmetic finding ships a **row-level table** — Surface and each
|
|
> hidden priority listed **separately** — never *"sum of file"*, never
|
|
> *"deal depth N"*.
|
|
|
|
**A reproduction must show the quantity the prose claims.** This is the
|
|
clause that exists because passing artifacts accompanied wrong
|
|
descriptions:
|
|
|
|
| wrong premise | what shipped | what a row-level table would have shown |
|
|
|---|---|---|
|
|
| *"12 in the file"* | a sum | which rows, at which priorities |
|
|
| *"4/6/9 against 5/7/9"* | a scenario, green | 6/9/12 — the deal, per row |
|
|
| *"SOLVE offered on a face-down Problem"* | a condition named | which filter actually fired |
|
|
|
|
**Requirement.** Where a finding asserts a number, the register entry
|
|
quotes the number **and** the artifact prints the rows it came from. A
|
|
finding stating a total without its rows is inadmissible even if the total
|
|
is right — *"12"* was arithmetically defensible and still wrong about the
|
|
game.
|
|
|
|
### 1.3 Can fail
|
|
|
|
The artifact must be capable of going red, and the register records its
|
|
**current colour**. A reproduction that has gone **green is not a closed
|
|
finding** — it is a finding whose claim is now in question, and §4 reports
|
|
it as needing resolution to `applied` or `withdrawn`.
|
|
|
|
**A reproduction has a `role`, and only one of them alarms.** This
|
|
distinction was not in ADR-0012; **T05's backfill produced it**, when the
|
|
first report cried wolf over U2.
|
|
|
|
| role | what it is | green means |
|
|
|---|---|---|
|
|
| `counterexample` | demonstrates the defect — the model-checker shape | **alarm.** It should be red while the finding is open |
|
|
| `default` | encodes a provisional choice so it is testable and visible | **expected.** It passes because our default is implemented, which says nothing about whether the owner agrees |
|
|
|
|
U2's scenario is green because the default it documents is implemented.
|
|
GR-E01's was a counterexample that went green when the edition landed, and
|
|
sat that way for four days. **Same colour, opposite meaning** — a register
|
|
that cannot tell them apart either alarms constantly or never.
|
|
|
|
**Rationale, and it is the expensive one.** `gr-e01-threshold-…-2p.yaml`
|
|
went green when the edition landed on 2026-08-04. The finding remained
|
|
admissible, remained in the register, and remained queued for transmission
|
|
to `ground-game` for four days, because nothing treated a passing artifact
|
|
as a signal. It was stopped by an adversarial review, not by a control.
|
|
|
|
### 1.4 It must move when its subject moves
|
|
|
|
> **A measured claim names the variable it depends on, and shows the
|
|
> measurement moving with it.** A number that does not move when its
|
|
> subject moves is measuring something else.
|
|
|
|
ADR-0018. This is the fourth clause and the first three are unchanged —
|
|
each was bought by an earlier mistake and a rewrite would drop what they
|
|
paid for.
|
|
|
|
**Why it exists.** Seven claims in this project have been wrong the same
|
|
way: the arithmetic right, the subject wrong. Two of them were caught only
|
|
because someone varied something —
|
|
|
|
- a per-node cost that **fell** as branching **rose** (no per-node cost
|
|
can), and
|
|
- a win rate attributed to *the game* that collapsed to 0% under a second
|
|
policy on the same deals.
|
|
|
|
**Both were already printed when they shipped.** Nobody looked at whether
|
|
the number moved the way its subject demanded.
|
|
|
|
**Where it binds:** register findings whose claim is arithmetic, and
|
|
anything leaving the repo. **Not every number everywhere** — a rule that
|
|
binds everywhere is obeyed nowhere.
|
|
|
|
**What it does not catch, stated so nobody stops looking:** a count that
|
|
was never made (there is no sensitivity to state), and a claim that is not
|
|
a number. **No syntactic gate detects a wrong subject in general.** It is
|
|
a semantic error; `loop-lint` checks that a sensitivity line is *present*,
|
|
never that it is *adequate*.
|
|
|
|
## 2. Kinds
|
|
|
|
Five (ADR-0012 D4). **No kind exists without a finding that instantiates
|
|
it**; a sixth appearing during backfill means the taxonomy was invented
|
|
rather than observed, and T07 must say so.
|
|
|
|
`underdetermined` · `inconsistent` · `inert` · `degenerate` · `unplayed`
|
|
|
|
## 3. States, owners, and death
|
|
|
|
```
|
|
raised ──► reported ──► ruled ──► applied
|
|
│
|
|
└──────────► withdrawn ◄──────────┘
|
|
```
|
|
|
|
- `applied` means **the source changed** — dataset or rules text — and our
|
|
provisional default was deleted. Not *"a ruling was recorded."*
|
|
- **The queue is `raised` + `reported`. The log is `applied` +
|
|
`withdrawn`.** `make design` reports the queue by default; the log only
|
|
on request. If the default view mixes them, the queue property is gone.
|
|
- **`withdrawn` is reported downstream, not deleted.** A claim retracted
|
|
silently is how three wrong premises survived.
|
|
|
|
### 3.1 Notes
|
|
|
|
A finding without a reproduction is a **note** (ADR-0012 D6). A note:
|
|
|
|
1. **has no `reported` state** — it may not cross the repo boundary;
|
|
2. **expires at 30 days**, flagged by §4 on the same mechanism that ages
|
|
provisional items (`rule-coverage.py:54`, `PROVISIONAL_WARN_DAYS`);
|
|
3. is promoted only by acquiring a reproduction meeting §1 — there is no
|
|
path from `note` to `reported` that skips it.
|
|
|
|
## 4. Metrics
|
|
|
|
`make design` reports these. Every number is computed **over the
|
|
register**, never from a maintained list (ADR-0012 D8).
|
|
|
|
| metric | target | notes |
|
|
|---|---|---|
|
|
| findings with a resolving reproduction | **100%** | denominator includes `withdrawn`; notes counted separately, never folded in |
|
|
| **reproductions that have gone green while open** | **0** | the §1.3 alarm. Non-zero is a queue that needs reading, not a passing suite |
|
|
| arithmetic findings whose artifact prints rows | **100%** | §1.2 |
|
|
| median days `raised` → `reported` | the number this exists to fix | the baseline it must beat is **4 days** to first *read* |
|
|
| open findings by kind, with age | reported, not targeted | |
|
|
| notes past 30 days | **0** | |
|
|
| findings closed by a ruling vs still open | reported, with age | |
|
|
|
|
**Falsifier for this spec.** If a finding reaches `ground-game` whose
|
|
prose describes a quantity its own artifact does not compute, §1.2 has
|
|
failed and the next revision must **generate the description from the
|
|
artifact** rather than let it be written alongside.
|
|
|
|
**No baseline rate is quoted here.** CB-RES-0007's 33% was withdrawn as a
|
|
measured figure (ADR-0012 D8); the first honest denominator is the one
|
|
T05's backfill produces.
|
|
|
|
## 5. The trial protocol
|
|
|
|
The part with no precedent in this repo. A **trial game** is a recorded
|
|
session plus an observation log — so a claim from play is replayable.
|
|
|
|
**It must cost almost nothing or it will not be done.** The engine already
|
|
records: `cb-play --record FILE` writes the finished game as a scenario
|
|
YAML (`main.rs:30`), and `games/ground/src/record.rs` is trustworthy
|
|
because its round trip is the test.
|
|
|
|
A trial is therefore:
|
|
|
|
```
|
|
cb-play --players N --serve PORT --record trials/<date>-<slug>.yaml
|
|
```
|
|
|
|
plus a sibling `trials/<date>-<slug>.md` carrying, and nothing more:
|
|
|
|
- **who played which seats**, and which were bots (with policy);
|
|
- **what was observed** — in the player's own words, not cleaned up;
|
|
- **the seed**, which the scenario already carries;
|
|
- **which observations became findings or notes**, by register id, or
|
|
explicitly none.
|
|
|
|
**Rules.**
|
|
|
|
1. An observation is a **note** until it has a reproduction (§3.1). A
|
|
trial log is not a finding source by itself — it is where findings are
|
|
*noticed*.
|
|
2. **The recording is the artifact for `unplayed` findings.** GR-E03/E04
|
|
were "never played to the end"; a trial that plays them to the end both
|
|
closes that finding and produces the reproduction for anything found on
|
|
the way.
|
|
3. A trial whose recording does not replay is a **bug in the engine**,
|
|
reported as one, not a design finding.
|
|
|
|
**Why the log is prose and stays prose.** The maintainer's *"I felt it was
|
|
too easy but then we lost, so who knows"* is exactly the observation this
|
|
protocol exists to catch: real, unreproducible as stated, and the seed of
|
|
[CB-WP-0025](../workplans/CB-WP-0025-could-we-have-won.md). Forcing it
|
|
into a schema at the moment of observation would lose it. §3.1 is what
|
|
stops it aging into an apparent finding.
|
|
|
|
---
|
|
|
|
## Appendix — Aspects and design space (2026-08-08)
|
|
|
|
Rules configurations are points in a **multi-aspect design space**. See:
|
|
|
|
- [`research/CB-RES-0010-game-aspects-and-design-space.md`](../research/CB-RES-0010-game-aspects-and-design-space.md) — concepts, sources, clay-borg duties
|
|
- [`editions/ASPECTS.md`](../editions/ASPECTS.md) — GROUND aspect inventory
|
|
- [`editions/CATALOG.md`](../editions/CATALOG.md) — module/profile selection (schema 2)
|
|
|
|
Findings and trials should name the **resolved module list** (configuration), not only a legacy experiment id.
|