clay-borg/specs/RetrospectiveAnalysis.md

# RetrospectiveAnalysis — was this deal winnable, and how hard is the game

v1.0 — CB-WP-0025 T04, 2026-08-05. Normative. Implements
[ADR-0013](../decisions/ADR-0013-could-we-have-won.md). Admissibility of
anything this produces is governed by
[GameDesign.md](GameDesign.md) §1.

**Two capabilities, one machine.** *Was this deal winnable?* is a search
over a finished game. *How hard is the game?* is that search run over many
deals and counted — **not** a bot's win rate (§4.1).

---

## 1. The question, and its name

> **"Given the deal as it actually was, was there a line of play that
> reached the threshold?"**

**Never labelled "how you should have played."** The distinction is the
honest content of the feature: the tool answers a question about the
*deal*, and a label promising advice about the *player* turns a true
answer into a false lesson.

**Strategy fusion does not apply and must not be invoked as an objection.**
Fusion (Frank, Basin & Matsubara 1998) is a defect of aggregating over
determinizations to choose a move. After the game there is **one world** —
the deal is known — so a line found in it is executable in the only world
there is. This is why the affordable option is also the honest one.

## 2. The witness

A witness is a sequence of joint selections that, replayed from the
recorded initial state, ends with `group_success == true`.

### 2.1 It must replay — hard gate, not a metric

> **100% of emitted witnesses replay through the existing scenario runner
> and end in `group_success`.**

Not a target: a **gate**. A witness that does not replay asserts the
opposite of the truth to a player who just lost, which is worse than
emitting nothing.

### 2.2 Every move carries its information dependence

ADR-0013 D2. Each move in a witness is marked:

| mark | meaning |
|---|---|
| `visible` | everything the move depended on was in the acting seat's projection at that point — the Problem face-up, the Solution in that seat's own hand |
| `hidden` | it was not |

Computed from `project()`, which already exists and whose hiding rules are
already asserted by `games_ground::view`.

**This replaces the structural boundary the survey wanted.** Searching a
`GroundView` is not implementable — a view cannot `fold` events — so the
guarantee moved from *the search cannot see it* to *the answer says which
moves needed it*. A witness reads:

> *"This deal was winnable. Two of these six moves needed a card you had
> no way to know was coming."*

**Falsifier:** a witness whose moves are all `visible` but which no seat
could have chosen means the marking is wrong. A test constructs that case.

### 2.3 Wording when nothing is found

> **"No winning line found in the last K rounds."**

**Never "unwinnable".** A bounded search cannot establish unwinnability,
and this sentence is what the player reads.

## 3. The bound

**Exhaustive over the last `K` rounds**, table treated as one co-operative
agent choosing joint selections. `K = 2` by default.

Bounded in **rounds**, not nodes: *"winnable from round 4"* means something
to a player; *"winnable within 100,000 nodes"* does not. A node budget is a
secondary cut that aborts with a stated reason so a wide table cannot hang
the page.

### 3.1 Measured cost, and what it permits

`cargo bench -p games-ground --bench search` — the single source for these
numbers (ADR-0013 D7). Mid-game states at real decision points:

| seats | branch width | `legal_commands` | `clone` | `validate+fold` |
|---|---:|---:|---:|---:|
| 2 | 5 | 4.06 µs | 378 ns | 696 ns |
| 3 | 8 | 4.13 µs | 432 ns | 508 ns |
| 4 | 11 | 4.76 µs | 639 ns | **3.76 µs** |

**Per-child cost is not uniform** — `validate+fold` ranges 0.5–3.8 µs
depending on which command is taken, because some resolve cascades and
some do not. **Budgets use the upper end**, so ~5 µs per child
(clone + validate + fold).

Joint branching over the last two rounds, from the measured per-seat
widths:

| seats | joint / 2 rounds | at ~5 µs/child |
|---|---:|---:|
| 2 | ~5×10² | negligible |
| 3 | ~1.6×10⁵ | **~0.8 s** |
| 4 | ~5.7×10⁵ | **~2.9 s** |

**So `K = 2` is affordable at two, three and four seats, and is not at
five or six** — joint branching there exceeds 10⁶ per round. Five and six
seats require a smaller `K`, and the tool must reduce it and **say that it
did** rather than silently searching less.

**The published 112–161 µs/node figure is withdrawn** (CB-RES-0008 §1.2,
challenge C1) and must not be quoted from anywhere.

## 4. Difficulty

### 4.1 A bot's win rate is not a difficulty

**Normative prohibition**, because this project already made the error and
nearly exported it:

> A win rate from a single policy **may not be reported as a difficulty**.

Measured, on identical deals: `GreedyPolicy` wins **100%** at five and six
seats where a `FirstLegal` policy — take `legal[0]`, no heuristic — wins
**0%**; at two seats `FirstLegal` (77.5%) *beats* greedy (66.0%). Two
unsophisticated agents span the entire range.

And a measure that improves when the *measurer* improves is not measuring
the subject: a better bot would make the game "easier" with no rule
changing.

### 4.2 What is reported instead

> **Winnable fraction** — over N deals at a seat count, the proportion in
> which the search finds a winning line within its bound.

A property of the **deal distribution and the threshold**, which is what
`ground-game` tunes. Ships as a table, never one number:

| column | what it is |
|---|---|
| winnable fraction | can the deal be won at all — bounded, `K` stated |
| reference-policy win rate | what a **named** policy achieves |
| skill gap | the difference — how much play has to supply |

**It is a lower bound and must be labelled one.** A `K`-round search cannot
see a line that required round 1, so the figure is
**"winnable-from-round-(6−K)"**, never "winnable".

**Every rate carries its policy, N, seed range and K in the number's
name**, not in a footnote — GameDesign §1.2, and the reason the withdrawn
finding was inadmissible.

### 4.3 Resolution — the number that makes it usable

> The smallest threshold change the measurement can distinguish, with its
> N.

*"We can tell a threshold of 5 from 7 but not 7 from 8"* is more useful to
`ground-game` than any rate with no error bar, and it is what makes the
figure a tuning instrument rather than a statistic.

## 5. The instruments must be able to fail

ADR-0013 D5, and GameDesign §1.3. Before **any** figure from these tools is
quoted anywhere:

- **positive controls** — a deal constructed to be unwinnable returns
  none; a deal constructed to be winnable returns a witness that replays;
- **`--self-test`**, wired into `make self-tests` like every other
  reporting tool;
- **one command regenerates the figure** (`make difficulty`);
- **the policy panel is plural** — at least `greedy`, `random` and
  `first-legal`. The spread between them is the finding §4.1 rests on, and
  reporting one policy would restore the error.

**`games/ground/examples/difficulty-baseline.rs` currently satisfies none
of the first three** and is inadmissible until it does. It has no
assertions, no self-test, and no `make` target — nothing can turn it red,
which under CB-WP-0022 T05's `role` distinction makes it a `default`
artifact wearing a `counterexample` label.

## 6. Falsifiers for this spec

- **§2.2 fails** if a witness is emitted whose moves are all `visible` but
  which no seat could have chosen. Then the marking must be derived from
  the search rather than checked after it.
- **§4.2 fails** if the winnable fraction turns out to be ~100% or ~0% at
  every seat count and threshold — it would then have no resolution (§4.3)
  and be as useless as the bot rate it replaced.
- **§3 fails** if `K = 2` proves unaffordable in practice at four seats;
  the measured 2.9 s is a projection from branch widths, not a timing of
  the real search.