clay-borg/specs/Taxonomy.md

# Taxonomy — which system is a statement about?

Status: **draft**, normative for new work. Changes go through a decision
record in `decisions/`.

This document exists because the words were doing too much work. *"Error"*,
*"failure"*, *"finding"* and *"correction"* were each being used for the
game's design, our formalisation of it, the code, the measuring apparatus,
and the sentences we wrote — five different things, one vocabulary.

> **Every defect statement names a stratum before it names anything else.**
> "A bug in the simulator" and "a gap in the game" are not the same kind of
> claim, do not have the same owner, and are not fixed by the same person.

Grounded in existing practice rather than invented here: the
fault/error/failure chain from **Avizienis, Laprie, Randell & Landwehr**
(*Basic Concepts and Taxonomy of Dependable and Secure Computing*, IEEE
TDSC 1:1, 2004); the model strata and the verification/validation split
from **Sargent** (*Verification and Validation of Simulation Models*); and
**MDA** (Hunicke, LeBlanc & Zubek, 2004) for the game-facing layers.

---

## 1. The strata

| stratum | canon | what it is | who owns it |
|---|---|---|---|
| **GAME** | Sargent's *problem entity*; MDA *mechanics as designed* | the rules as the game's owner defines them — the edition, the printed text | **the game's owner**, not us |
| **MODEL** | Sargent's *conceptual model* | our formalisation: `GroundRules.md`, the rule ids, the state we chose to represent | us, answerable to the owner |
| **ENGINE** | Sargent's *computerized model* | the kernel that implements the model | us |
| **INSTRUMENT** | measurement apparatus | harnesses, policies, metrics, gates, the trial protocol | us |
| **ACCOUNT** | — | evidence files, register entries, messages, this document | us |
| **PRESENTATION** | MDA *aesthetics*-facing | the page, and everything [`Ornamentation.md`](Ornamentation.md) covers | us, later `clay-animate` |

### 1.1 The two relations that matter

> **GAME ↔ MODEL is validation.** *Did we formalise the right game?*
>
> **MODEL ↔ ENGINE is verification.** *Did we implement the formalisation
> correctly?*

This is Sargent's split and it is the one we kept losing. A statement that
the engine mis-scores a Bond is **verification**. A statement that the
rules do not say what happens when two Bonds break simultaneously is
**validation** — and it is not ours to answer.

**Sargent's third leg is `data validity`**, and it is where
`editions/` lives: the vendored CSVs, their digests, and the tripwires in
`edition.rs` are data validation, not verification.

### 1.2 What each stratum's defects look like

| stratum | a defect here looks like |
|---|---|
| GAME | a rule that does not decide; a dominant strategy; an action that earns its place nowhere |
| MODEL | we represented a rule the edition does not have, or omitted one it does |
| ENGINE | the code does not do what `GroundRules.md` says |
| INSTRUMENT | the measurement answers a different question than the one asked |
| ACCOUNT | the sentence claims more than the artifact shows |
| PRESENTATION | the player cannot see, or misreads, what the state says |

---

## 2. Fault, error, failure

Avizienis' chain, unchanged, applied **within** a stratum.

| term | definition | example, INSTRUMENT stratum |
|---|---|---|
| **fault** | the defect in the artifact — the wrong line, the wrong column, the wrong constant | `peak` maximised over `StressSet` payloads |
| **error** | the wrong internal state it produces | the metric holds 1 where the game held 2 |
| **failure** | the deviation visible at the interface | the evidence file says *"peak Stress was 1"* |

**A fault is dormant until activated.** `attack-value.rs`'s missing
assertion was a fault for weeks and produced no error, because no cell was
ever short.

### 2.1 The rule that explains our history

> **A failure in one stratum is a fault in the next.**

An INSTRUMENT failure (a metric reporting the wrong number) is an ACCOUNT
fault (a sentence built on it). An ACCOUNT failure that reaches the owner
is a fault in *their* decision-making. That is why
[`GameDesign.md`](GameDesign.md) §1 gates what may leave the repo: it is a
firebreak in this chain, not politeness.

### 2.2 The wrong-subject error, defined at last

[ADR-0018](../decisions/ADR-0018-a-number-that-does-not-move.md) named a
family and could not say precisely what it was. In this vocabulary:

> **A wrong-subject error is an ACCOUNT failure with no INSTRUMENT fault.**
> The measurement is correct about *its* subject; the sentence is about a
> different one.

`peak Stress was 1` — the metric correctly reported the maximum *assigned*
Stress. `Reactive is greedy with one preference changed` — the policy
correctly implemented what was written. **Nothing was miscomputed in
either case**, which is exactly why tests did not catch them and why the
fix is never "add a test to the engine".

---

## 3. The vocabulary, fixed

| word | means | does not mean |
|---|---|---|
| **finding** | a statement about the **GAME** stratum, admissible under GameDesign §1, addressed to the owner | any defect we happen to notice |
| **defect** | a fault in any stratum we own — MODEL, ENGINE, INSTRUMENT, ACCOUNT, PRESENTATION | a design question |
| **fix** | a change to the **artifact** that removes a fault | a change to what we said |
| **correction** | a change to the **ACCOUNT** — a claim withdrawn, restated or re-derived | a code change |
| **retraction** | a correction of something that has already **left the repo** | an internal edit |
| **regression** | a fault introduced by a change, in any stratum | any failure |
| **inert control** | a control that cannot fail, so proves nothing (ADR-0006 D3) | a control that happens to be green |

**The register carries findings.** Where an entry is really a defect in a
stratum we own — F18, F24, F25 are ENGINE/data — the row says so, because
"the engine has not read this file" and "the game does not decide this"
are different claims to a reader deciding what to act on.

---

## 4. What the layers measure, and what they cannot

MDA gives the game-facing split, and it maps onto instruments we already
have:

| MDA layer | where it lives here | what measures it |
|---|---|---|
| **Mechanics** | GAME → MODEL → ENGINE | the rule-coverage gate, the edition tripwires |
| **Dynamics** — the behaviour that emerges in play | what the ENGINE produces over many games | the policy panels: win rates, DARVO arm rates, action selection |
| **Aesthetics** — the player's response | not representable in state | **trial notes, and nothing else** |

> **Our panels measure dynamics. Our trial logs sample aesthetics. Neither
> substitutes for the other**, and the recurring temptation is to report a
> dynamics number as though it settled an aesthetics question — *"is the
> game fun at 5 seats"* is not answered by a win rate.

MDA's own directional insight is the reason: **the designer builds
mechanics upward hoping for a feeling; the player meets the feeling first
and infers the rules.** A simulator sits at the bottom of that arrow and
can only ever report upward.

---

## 5. Worked examples from this repo

Because a taxonomy with no instances was invented rather than observed
(GameDesign §2).

| what happened | stratum | fault / error / failure |
|---|---|---|
| `solution_deck()` is a Rust literal that never reads `Solutions.csv` | ENGINE (data validity) | dormant **fault**; no error, because the literal agrees |
| `score()` read `self` while H1's pressure went into `work` | ENGINE | **fault**; error only at round 5; no measured failure |
| GR-P05 — SOLVE offered where it could not act | MODEL, then GAME | our **failure**, escalated to a GAME finding, ruled by the owner |
| ATTACK earns its place in no mode (F17) | **GAME** | a finding — not a defect at all |
| `peak Stress was 1` | ACCOUNT | **failure** with no instrument fault (§2.2) |
| the panels ran under no gate | INSTRUMENT | **fault** — every published figure came from an ungated binary |
| the note channel closed when the game ended | PRESENTATION | **failure**; the state was right, the player could not reach it |

**Read the stratum column.** Six rows, six different owners and six
different fixes, and every one of them was called "an error" at the time.

---

## 6. Falsifier

If a defect arises that cannot be placed in exactly one stratum, the
strata are wrong — either too few, or cut in the wrong place. **Record the
instance rather than widening a definition to swallow it**, which is how a
taxonomy stops distinguishing anything.

A second falsifier, from §2.2: if a wrong-subject error is ever found that
*did* have an instrument fault, then the family is not what ADR-0018 and
this document say it is.

## Sources

- Avizienis, Laprie, Randell & Landwehr, *Basic Concepts and Taxonomy of
  Dependable and Secure Computing*, IEEE TDSC 1(1), 2004.
- Sargent, *Verification and Validation of Simulation Models*, Winter
  Simulation Conference.
- Hunicke, LeBlanc & Zubek, *MDA: A Formal Approach to Game Design and
  Game Research*, AAAI Workshop, 2004.