clay-borg difficulty report 2026-08-05: a baseline, and a retraction

GROUND-WP-0005 has both tasks blocked on a measured difficulty baseline.
This is that baseline, with its confound stated rather than hidden.

It opens with a retraction. Earlier today clay-borg was going to report
that a greedy bot wins 200 of 200 games at five and six seats and the game
is too easy there -- which would have invited threshold changes. A
FirstLegal policy scores 0% on the identical deals where greedy scores
100%, and at two seats it beats greedy. Two unsophisticated agents span
the entire range, so a single policy's win rate is a statement about the
policy. clay-borg's adversarial review caught it before it left the repo.
Fifth wrong premise avoided, second stopped before sending.

What is reported instead is the WINNABLE FRACTION -- the proportion of
deals in which an exhaustive search finds a winning line in the final
round -- alongside a plural policy panel and the SPREAD between policies.
The spread is 71.7 to 100.0 percentage points, which is the direct
evidence for why the single-policy figure was meaningless.

Stated limits: the winnable figure is conditioned on greedy's play up to
the final round (searching from round 1 exceeded 2x10^6 nodes at two
seats), it is a lower bound, and budget-cut deals are excluded rather than
counted as losses.

No threshold changes are proposed. The confound is large enough that
clay-borg would rather this repo saw it than acted on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

workplans/GROUND-WP-0005-clay-borg-difficulty-260805.md

@@ -0,0 +1,131 @@
+---
+id: GROUND-WP-0005-REPORT-260805
+type: report
+title: "clay-borg difficulty measurement, 2026-08-05 — and a retraction of what we nearly sent instead"
+domain: consumer
+repo: ground-game
+status: informational
+owner: bernd
+topic_slug: whynot
+created: "2026-08-05"
+extends: GROUND-WP-0005
+---
+
+# clay-borg difficulty report — 2026-08-05
+
+GROUND-WP-0005 (*difficulty tiers and the Pressure dial*) has both tasks
+in `wait`, blocked on a measured baseline. This is that baseline, with its
+limits stated.
+
+Generated by `make difficulty`
+(`games/ground/examples/difficulty.rs`), governed by
+`clay-borg/specs/RetrospectiveAnalysis.md` §4, under
+[CB-WP-0025](../../clay-borg/workplans/CB-WP-0025-could-we-have-won.md) T06.
+
+---
+
+## 0. First, what we nearly sent you and did not
+
+**Earlier today this report was going to say: "a greedy bot wins 200 of
+200 games at five and six seats — the game is too easy there."**
+
+It would have invited you to move thresholds. **It was wrong**, and an
+adversarial review caught it before it left our repo.
+
+A `FirstLegal` policy — take the first legal command, no heuristic
+whatsoever — scores **0%** at five and six seats on the identical deals
+where `GreedyPolicy` scores **100%**. At two seats it *beats* greedy
+(76.7% vs 60.0%).
+
+**Two unsophisticated agents span the entire range.** A single policy's
+win rate is therefore a statement about the policy, not about GROUND, and
+we have written that into our spec as a prohibition rather than a
+preference.
+
+This is the fifth claim we would have sent you on a wrong premise. It is
+the second stopped before sending.
+
+## 1. The measurement
+
+`make difficulty`, 60 seeds per cell:
+
+| seats | winnable | greedy | random | first-legal | spread | undecided |
+|---|---:|---:|---:|---:|---:|---:|
+| 2 | **60%** | 60.0% | 5.0% | 76.7% | 71.7 | 0 |
+| 3 | **93%** | 88.3% | 6.7% | 25.0% | 81.7 | 2 |
+| 4 | **100%** | 93.3% | 6.7% | 30.0% | 86.7 | 4 |
+| 5 | **100%** | 100.0% | 3.3% | 0.0% | 100.0 | 0 |
+| 6 | **100%** | 100.0% | 3.3% | 0.0% | 100.0 | 0 |
+
+**`winnable`** is the column that is not about a bot. It is the fraction
+of deals in which an **exhaustive search finds a winning line in the final
+round**. The other three columns are what named policies actually achieve,
+and **`spread`** is the range between them.
+
+## 2. How to read it, including what it is not
+
+**Read `spread` first.** It ranges from 71.7 to 100.0 percentage points.
+Wherever it is large — everywhere — **no single policy's rate tells you
+anything about the game.** That is the finding behind §0.
+
+**`winnable` is conditioned on greedy's play up to the final round.** It
+is *"winnable from where greedy got to"*, not a property of the deal
+alone. A genuinely policy-free figure would search from round 1, which we
+measured as unaffordable: our search exceeded 2×10⁶ nodes at **two** seats
+over two rounds. We are telling you this rather than presenting the number
+as cleaner than it is.
+
+**It is a lower bound.** A one-round search cannot see a line that needed
+an earlier round, so the true winnable fraction is **at least** these
+figures.
+
+**`undecided` deals are excluded, not counted as losses.** Those are deals
+whose search hit the node budget. Folding *"we stopped looking"* into
+*"not winnable"* is the collapse our spec forbids.
+
+## 3. What we think this supports, offered as reading not as ruling
+
+**The rules are yours. This is what the numbers appear to say.**
+
+- **Two seats is the tight configuration.** 60% winnable in the final
+  round, and every policy struggles. If any seat count is under-tuned for
+  *difficulty*, the evidence does not point here.
+- **Four seats and up: every deal we sampled was still winnable in the
+  final round.** 100% at 4, 5 and 6 seats. That is compatible with "too
+  generous at high seat counts" — but it is **not** the claim we withdrew
+  in §0, because it is a statement about deal winnability rather than
+  about a bot's success.
+- **The 5–6 seat rows deserve the most suspicion.** `first-legal` scores
+  0% there while `greedy` scores 100%, which is the widest spread in the
+  table. Something about those configurations makes play matter *more*,
+  not less — the opposite of "too easy" — and we do not yet understand it.
+
+**We are not proposing threshold changes.** The measurement's confound
+(§2) is large enough that we would rather you saw it than acted on it.
+
+## 4. What would make this a better instrument
+
+- **Search from round 1** rather than the last round, which removes the
+  greedy confound entirely. Needs transposition or move-ordering; we have
+  neither.
+- **A resolution figure** — the smallest threshold change the measurement
+  can distinguish, with its N. Our spec requires it and we cannot yet
+  supply it.
+- **Your Pressure dial**, if it lands, gives a second axis to measure
+  against and would make the table two-dimensional.
+
+## 5. Reproduction
+
+Per GROUND-WP-0004 T02, arithmetic findings cite their reproduction by
+path:
+
+```
+clay-borg$ make difficulty              # the table above
+clay-borg$ make self-tests              # includes the difficulty controls
+```
+
+Controls, so the figure can fail rather than merely print: a witness must
+replay to a win; an unwinnable position must be reported as **searched
+out** rather than as a budget cut; a budget of one node must not claim
+exhaustion; and the policy panel must actually disagree, or reporting
+three policies would be ceremony.