tegwick
713a9df7fd
Some checks failed
ci / check (push) Failing after 3s
The maintainer could not tell whether "error", "failure", "finding" or "correction" referred to the game's design, our formalisation of it, the code, the measuring apparatus, or the sentences we wrote. Three review rounds produced twenty-odd defect statements spanning five systems, all called errors. The confusion was ours. specs/Taxonomy.md, grounded in named canon rather than invented here: six strata from Sargent's problem entity / conceptual model / computerized model, extended where a simulation-V&V frame stops — we also own an instrument and an account. The two relations are what was missing: GAME<->MODEL is validation, MODEL<->ENGINE is verification, and nearly every argument about "our bug or their gap" was that distinction going unnamed. Fault/error/failure from Avizienis et al., applied within a stratum, plus the rule that explains the review history: a failure in one stratum is a fault in the next. And it finally defines the family ADR-0018 could only point at — a wrong-subject error is an ACCOUNT failure with no INSTRUMENT fault, which is why tests never catch them. MDA supplies the game-facing layers and one hard limit: our panels measure dynamics, our trial logs sample aesthetics, and a win rate does not answer "is it fun". specs/Positioning.md names the field fairly — Ludii is the closest relative and the right benchmark — and the four differentiators, each already built rather than aspired to. Clay-borg is a design-evidence instrument; anyone can produce the number. Three tracks named and none started: a second game, game theory as the lens on dynamics, and assimilated knowledge about why games work. Track A is the falsifier for the whole positioning: every abstraction here has exactly one instance, which by our own rule may mean invented rather than observed. Chaos window 3 closes at 12 declarations with one override that changed nothing. Its verdict is due and is deliberately not written here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
185 lines
9 KiB
Markdown
185 lines
9 KiB
Markdown
# Taxonomy — which system is a statement about?
|
|
|
|
Status: **draft**, normative for new work. Changes go through a decision
|
|
record in `decisions/`.
|
|
|
|
This document exists because the words were doing too much work. *"Error"*,
|
|
*"failure"*, *"finding"* and *"correction"* were each being used for the
|
|
game's design, our formalisation of it, the code, the measuring apparatus,
|
|
and the sentences we wrote — five different things, one vocabulary.
|
|
|
|
> **Every defect statement names a stratum before it names anything else.**
|
|
> "A bug in the simulator" and "a gap in the game" are not the same kind of
|
|
> claim, do not have the same owner, and are not fixed by the same person.
|
|
|
|
Grounded in existing practice rather than invented here: the
|
|
fault/error/failure chain from **Avizienis, Laprie, Randell & Landwehr**
|
|
(*Basic Concepts and Taxonomy of Dependable and Secure Computing*, IEEE
|
|
TDSC 1:1, 2004); the model strata and the verification/validation split
|
|
from **Sargent** (*Verification and Validation of Simulation Models*); and
|
|
**MDA** (Hunicke, LeBlanc & Zubek, 2004) for the game-facing layers.
|
|
|
|
---
|
|
|
|
## 1. The strata
|
|
|
|
| stratum | canon | what it is | who owns it |
|
|
|---|---|---|---|
|
|
| **GAME** | Sargent's *problem entity*; MDA *mechanics as designed* | the rules as the game's owner defines them — the edition, the printed text | **the game's owner**, not us |
|
|
| **MODEL** | Sargent's *conceptual model* | our formalisation: `GroundRules.md`, the rule ids, the state we chose to represent | us, answerable to the owner |
|
|
| **ENGINE** | Sargent's *computerized model* | the kernel that implements the model | us |
|
|
| **INSTRUMENT** | measurement apparatus | harnesses, policies, metrics, gates, the trial protocol | us |
|
|
| **ACCOUNT** | — | evidence files, register entries, messages, this document | us |
|
|
| **PRESENTATION** | MDA *aesthetics*-facing | the page, and everything [`Ornamentation.md`](Ornamentation.md) covers | us, later `clay-animate` |
|
|
|
|
### 1.1 The two relations that matter
|
|
|
|
> **GAME ↔ MODEL is validation.** *Did we formalise the right game?*
|
|
>
|
|
> **MODEL ↔ ENGINE is verification.** *Did we implement the formalisation
|
|
> correctly?*
|
|
|
|
This is Sargent's split and it is the one we kept losing. A statement that
|
|
the engine mis-scores a Bond is **verification**. A statement that the
|
|
rules do not say what happens when two Bonds break simultaneously is
|
|
**validation** — and it is not ours to answer.
|
|
|
|
**Sargent's third leg is `data validity`**, and it is where
|
|
`editions/` lives: the vendored CSVs, their digests, and the tripwires in
|
|
`edition.rs` are data validation, not verification.
|
|
|
|
### 1.2 What each stratum's defects look like
|
|
|
|
| stratum | a defect here looks like |
|
|
|---|---|
|
|
| GAME | a rule that does not decide; a dominant strategy; an action that earns its place nowhere |
|
|
| MODEL | we represented a rule the edition does not have, or omitted one it does |
|
|
| ENGINE | the code does not do what `GroundRules.md` says |
|
|
| INSTRUMENT | the measurement answers a different question than the one asked |
|
|
| ACCOUNT | the sentence claims more than the artifact shows |
|
|
| PRESENTATION | the player cannot see, or misreads, what the state says |
|
|
|
|
---
|
|
|
|
## 2. Fault, error, failure
|
|
|
|
Avizienis' chain, unchanged, applied **within** a stratum.
|
|
|
|
| term | definition | example, INSTRUMENT stratum |
|
|
|---|---|---|
|
|
| **fault** | the defect in the artifact — the wrong line, the wrong column, the wrong constant | `peak` maximised over `StressSet` payloads |
|
|
| **error** | the wrong internal state it produces | the metric holds 1 where the game held 2 |
|
|
| **failure** | the deviation visible at the interface | the evidence file says *"peak Stress was 1"* |
|
|
|
|
**A fault is dormant until activated.** `attack-value.rs`'s missing
|
|
assertion was a fault for weeks and produced no error, because no cell was
|
|
ever short.
|
|
|
|
### 2.1 The rule that explains our history
|
|
|
|
> **A failure in one stratum is a fault in the next.**
|
|
|
|
An INSTRUMENT failure (a metric reporting the wrong number) is an ACCOUNT
|
|
fault (a sentence built on it). An ACCOUNT failure that reaches the owner
|
|
is a fault in *their* decision-making. That is why
|
|
[`GameDesign.md`](GameDesign.md) §1 gates what may leave the repo: it is a
|
|
firebreak in this chain, not politeness.
|
|
|
|
### 2.2 The wrong-subject error, defined at last
|
|
|
|
[ADR-0018](../decisions/ADR-0018-a-number-that-does-not-move.md) named a
|
|
family and could not say precisely what it was. In this vocabulary:
|
|
|
|
> **A wrong-subject error is an ACCOUNT failure with no INSTRUMENT fault.**
|
|
> The measurement is correct about *its* subject; the sentence is about a
|
|
> different one.
|
|
|
|
`peak Stress was 1` — the metric correctly reported the maximum *assigned*
|
|
Stress. `Reactive is greedy with one preference changed` — the policy
|
|
correctly implemented what was written. **Nothing was miscomputed in
|
|
either case**, which is exactly why tests did not catch them and why the
|
|
fix is never "add a test to the engine".
|
|
|
|
---
|
|
|
|
## 3. The vocabulary, fixed
|
|
|
|
| word | means | does not mean |
|
|
|---|---|---|
|
|
| **finding** | a statement about the **GAME** stratum, admissible under GameDesign §1, addressed to the owner | any defect we happen to notice |
|
|
| **defect** | a fault in any stratum we own — MODEL, ENGINE, INSTRUMENT, ACCOUNT, PRESENTATION | a design question |
|
|
| **fix** | a change to the **artifact** that removes a fault | a change to what we said |
|
|
| **correction** | a change to the **ACCOUNT** — a claim withdrawn, restated or re-derived | a code change |
|
|
| **retraction** | a correction of something that has already **left the repo** | an internal edit |
|
|
| **regression** | a fault introduced by a change, in any stratum | any failure |
|
|
| **inert control** | a control that cannot fail, so proves nothing (ADR-0006 D3) | a control that happens to be green |
|
|
|
|
**The register carries findings.** Where an entry is really a defect in a
|
|
stratum we own — F18, F24, F25 are ENGINE/data — the row says so, because
|
|
"the engine has not read this file" and "the game does not decide this"
|
|
are different claims to a reader deciding what to act on.
|
|
|
|
---
|
|
|
|
## 4. What the layers measure, and what they cannot
|
|
|
|
MDA gives the game-facing split, and it maps onto instruments we already
|
|
have:
|
|
|
|
| MDA layer | where it lives here | what measures it |
|
|
|---|---|---|
|
|
| **Mechanics** | GAME → MODEL → ENGINE | the rule-coverage gate, the edition tripwires |
|
|
| **Dynamics** — the behaviour that emerges in play | what the ENGINE produces over many games | the policy panels: win rates, DARVO arm rates, action selection |
|
|
| **Aesthetics** — the player's response | not representable in state | **trial notes, and nothing else** |
|
|
|
|
> **Our panels measure dynamics. Our trial logs sample aesthetics. Neither
|
|
> substitutes for the other**, and the recurring temptation is to report a
|
|
> dynamics number as though it settled an aesthetics question — *"is the
|
|
> game fun at 5 seats"* is not answered by a win rate.
|
|
|
|
MDA's own directional insight is the reason: **the designer builds
|
|
mechanics upward hoping for a feeling; the player meets the feeling first
|
|
and infers the rules.** A simulator sits at the bottom of that arrow and
|
|
can only ever report upward.
|
|
|
|
---
|
|
|
|
## 5. Worked examples from this repo
|
|
|
|
Because a taxonomy with no instances was invented rather than observed
|
|
(GameDesign §2).
|
|
|
|
| what happened | stratum | fault / error / failure |
|
|
|---|---|---|
|
|
| `solution_deck()` is a Rust literal that never reads `Solutions.csv` | ENGINE (data validity) | dormant **fault**; no error, because the literal agrees |
|
|
| `score()` read `self` while H1's pressure went into `work` | ENGINE | **fault**; error only at round 5; no measured failure |
|
|
| GR-P05 — SOLVE offered where it could not act | MODEL, then GAME | our **failure**, escalated to a GAME finding, ruled by the owner |
|
|
| ATTACK earns its place in no mode (F17) | **GAME** | a finding — not a defect at all |
|
|
| `peak Stress was 1` | ACCOUNT | **failure** with no instrument fault (§2.2) |
|
|
| the panels ran under no gate | INSTRUMENT | **fault** — every published figure came from an ungated binary |
|
|
| the note channel closed when the game ended | PRESENTATION | **failure**; the state was right, the player could not reach it |
|
|
|
|
**Read the stratum column.** Six rows, six different owners and six
|
|
different fixes, and every one of them was called "an error" at the time.
|
|
|
|
---
|
|
|
|
## 6. Falsifier
|
|
|
|
If a defect arises that cannot be placed in exactly one stratum, the
|
|
strata are wrong — either too few, or cut in the wrong place. **Record the
|
|
instance rather than widening a definition to swallow it**, which is how a
|
|
taxonomy stops distinguishing anything.
|
|
|
|
A second falsifier, from §2.2: if a wrong-subject error is ever found that
|
|
*did* have an instrument fault, then the family is not what ADR-0018 and
|
|
this document say it is.
|
|
|
|
## Sources
|
|
|
|
- Avizienis, Laprie, Randell & Landwehr, *Basic Concepts and Taxonomy of
|
|
Dependable and Secure Computing*, IEEE TDSC 1(1), 2004.
|
|
- Sargent, *Verification and Validation of Simulation Models*, Winter
|
|
Simulation Conference.
|
|
- Hunicke, LeBlanc & Zubek, *MDA: A Formal Approach to Game Design and
|
|
Game Research*, AAAI Workshop, 2004.
|