tegwick
92f52e777d
The workplan list read as though GROUND-WP-0002, 0003 and 0005 each existed twice. They did not: three inbound clay-borg finding reports were filed in workplans/ borrowing the number of the workplan they bear on, and each was registered in State Hub as a workplan. A report is correspondence, not a unit of work. The three move to reports/YYMMDD-<slug>.md with their own id space (GROUND-RPT-0001..0003), keeping type: report, status: informational and the extends: link to the workplan they relate to. Their state_hub_workstream_id bindings are dropped and the hub registrations archived, so the hub's workplan list matches the files. Five workplans, one file per number. AGENTS.md records the convention, since the reports arrive regularly and nothing in the repo said where they go. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
131 lines
5.3 KiB
Markdown
131 lines
5.3 KiB
Markdown
---
|
||
id: GROUND-RPT-0002
|
||
type: report
|
||
title: "clay-borg difficulty measurement, 2026-08-05 — and a retraction of what we nearly sent instead"
|
||
domain: consumer
|
||
repo: ground-game
|
||
status: informational
|
||
owner: bernd
|
||
topic_slug: whynot
|
||
created: "2026-08-05"
|
||
extends: GROUND-WP-0005
|
||
---
|
||
|
||
# clay-borg difficulty report — 2026-08-05
|
||
|
||
GROUND-WP-0005 (*difficulty tiers and the Pressure dial*) has both tasks
|
||
in `wait`, blocked on a measured baseline. This is that baseline, with its
|
||
limits stated.
|
||
|
||
Generated by `make difficulty`
|
||
(`games/ground/examples/difficulty.rs`), governed by
|
||
`clay-borg/specs/RetrospectiveAnalysis.md` §4, under
|
||
[CB-WP-0025](../../clay-borg/workplans/CB-WP-0025-could-we-have-won.md) T06.
|
||
|
||
---
|
||
|
||
## 0. First, what we nearly sent you and did not
|
||
|
||
**Earlier today this report was going to say: "a greedy bot wins 200 of
|
||
200 games at five and six seats — the game is too easy there."**
|
||
|
||
It would have invited you to move thresholds. **It was wrong**, and an
|
||
adversarial review caught it before it left our repo.
|
||
|
||
A `FirstLegal` policy — take the first legal command, no heuristic
|
||
whatsoever — scores **0%** at five and six seats on the identical deals
|
||
where `GreedyPolicy` scores **100%**. At two seats it *beats* greedy
|
||
(76.7% vs 60.0%).
|
||
|
||
**Two unsophisticated agents span the entire range.** A single policy's
|
||
win rate is therefore a statement about the policy, not about GROUND, and
|
||
we have written that into our spec as a prohibition rather than a
|
||
preference.
|
||
|
||
This is the fifth claim we would have sent you on a wrong premise. It is
|
||
the second stopped before sending.
|
||
|
||
## 1. The measurement
|
||
|
||
`make difficulty`, 60 seeds per cell:
|
||
|
||
| seats | winnable | greedy | random | first-legal | spread | undecided |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| 2 | **60%** | 60.0% | 5.0% | 76.7% | 71.7 | 0 |
|
||
| 3 | **93%** | 88.3% | 6.7% | 25.0% | 81.7 | 2 |
|
||
| 4 | **100%** | 93.3% | 6.7% | 30.0% | 86.7 | 4 |
|
||
| 5 | **100%** | 100.0% | 3.3% | 0.0% | 100.0 | 0 |
|
||
| 6 | **100%** | 100.0% | 3.3% | 0.0% | 100.0 | 0 |
|
||
|
||
**`winnable`** is the column that is not about a bot. It is the fraction
|
||
of deals in which an **exhaustive search finds a winning line in the final
|
||
round**. The other three columns are what named policies actually achieve,
|
||
and **`spread`** is the range between them.
|
||
|
||
## 2. How to read it, including what it is not
|
||
|
||
**Read `spread` first.** It ranges from 71.7 to 100.0 percentage points.
|
||
Wherever it is large — everywhere — **no single policy's rate tells you
|
||
anything about the game.** That is the finding behind §0.
|
||
|
||
**`winnable` is conditioned on greedy's play up to the final round.** It
|
||
is *"winnable from where greedy got to"*, not a property of the deal
|
||
alone. A genuinely policy-free figure would search from round 1, which we
|
||
measured as unaffordable: our search exceeded 2×10⁶ nodes at **two** seats
|
||
over two rounds. We are telling you this rather than presenting the number
|
||
as cleaner than it is.
|
||
|
||
**It is a lower bound.** A one-round search cannot see a line that needed
|
||
an earlier round, so the true winnable fraction is **at least** these
|
||
figures.
|
||
|
||
**`undecided` deals are excluded, not counted as losses.** Those are deals
|
||
whose search hit the node budget. Folding *"we stopped looking"* into
|
||
*"not winnable"* is the collapse our spec forbids.
|
||
|
||
## 3. What we think this supports, offered as reading not as ruling
|
||
|
||
**The rules are yours. This is what the numbers appear to say.**
|
||
|
||
- **Two seats is the tight configuration.** 60% winnable in the final
|
||
round, and every policy struggles. If any seat count is under-tuned for
|
||
*difficulty*, the evidence does not point here.
|
||
- **Four seats and up: every deal we sampled was still winnable in the
|
||
final round.** 100% at 4, 5 and 6 seats. That is compatible with "too
|
||
generous at high seat counts" — but it is **not** the claim we withdrew
|
||
in §0, because it is a statement about deal winnability rather than
|
||
about a bot's success.
|
||
- **The 5–6 seat rows deserve the most suspicion.** `first-legal` scores
|
||
0% there while `greedy` scores 100%, which is the widest spread in the
|
||
table. Something about those configurations makes play matter *more*,
|
||
not less — the opposite of "too easy" — and we do not yet understand it.
|
||
|
||
**We are not proposing threshold changes.** The measurement's confound
|
||
(§2) is large enough that we would rather you saw it than acted on it.
|
||
|
||
## 4. What would make this a better instrument
|
||
|
||
- **Search from round 1** rather than the last round, which removes the
|
||
greedy confound entirely. Needs transposition or move-ordering; we have
|
||
neither.
|
||
- **A resolution figure** — the smallest threshold change the measurement
|
||
can distinguish, with its N. Our spec requires it and we cannot yet
|
||
supply it.
|
||
- **Your Pressure dial**, if it lands, gives a second axis to measure
|
||
against and would make the table two-dimensional.
|
||
|
||
## 5. Reproduction
|
||
|
||
Per GROUND-WP-0004 T02, arithmetic findings cite their reproduction by
|
||
path:
|
||
|
||
```
|
||
clay-borg$ make difficulty # the table above
|
||
clay-borg$ make self-tests # includes the difficulty controls
|
||
```
|
||
|
||
Controls, so the figure can fail rather than merely print: a witness must
|
||
replay to a win; an unwinnable position must be reported as **searched
|
||
out** rather than as a budget cut; a budget of one node must not claim
|
||
exhaustion; and the policy panel must actually disagree, or reporting
|
||
three policies would be ceremony.
|