CB-WP-0025 T02: the review withdrew the finding, and the fifth wrong
Some checks are pending
ci / check (push) Waiting to run
Some checks are pending
ci / check (push) Waiting to run
premise never left the repo Separate agent, second tier-L review in this project. Six of seven challenges conceded. The survey's headline finding is WITHDRAWN, not softened. C4 kills it, and the reviewer ranked it fourth. A FirstLegal policy -- take legal[0], no heuristic at all -- scores 0% at five and six seats where GreedyPolicy scores 100%, and 77.5% at two seats where greedy scores 66%. Two unsophisticated agents span the entire range at the same seat count. "The game is too easy at 5-6 seats" is therefore a statement about GreedyPolicy, not about GROUND. The rescue the reviewer offered -- greedy hits the 12-point ceiling in 200/200 deals, so the 6p row is a rules claim -- dies on the same data: FirstLegal reaches that ceiling never. C1: the per-node cost was wrong by 30-50x. The timer started before the seed loop, so "us/node" included two setups, an entire greedy game and a full validate+fold replay, divided by player-decision count. The tell was in my own published output and I did not look at it: the figure FELL (161/139/112) as branching ROSE (4.7/7.4/9.1), which no per-enumeration cost can do. Re-measured with the clock around legal_commands alone: 3.0/3.5/4.1 us, now rising with branching. The reviewer measured 15.6-20.4 by a different isolation; we disagree by ~5x and neither has established which is right, so T04 must benchmark it with criterion rather than adopt either number. C6: "exhaustive search is out at any seat count" is false -- ~3 seconds over the last two rounds at 3p. With C1's correction the budget is ~10^5-10^6 nodes and bounded endgame search fits, so ADR-0013 cannot open with "exhaustive is impossible, therefore determinized sampling" -- especially as sampling carries strategy fusion that exhaustive search does not. C3: the finding failed the admissibility rule this project wrote nine hours earlier. 6/9/12 are sums where GROUND-WP-0004 T02 requires per-priority rows, and the harness has no assertions, no --self-test and no make target, so nothing can turn it red -- a `default` artifact wearing a `counterexample` label, by CB-WP-0022 T05's own distinction. C2: the ratio story explains nothing; 3p and 4p share deal, threshold and ratio and differ by 12.5 points of win rate. C5: "explains the maintainer's report" is contradicted by lib.rs:2487, which records his losses as 3-player games on the pre-ruling deal, arithmetically unwinnable at 6 against 7. T06 exists to report to GROUND-WP-0005, which is BLOCKED waiting on a difficulty baseline. Had this proceeded they would have been invited to move thresholds on the strength of one bot's behaviour. That is the fifth wrong premise this project would have sent them, and the second stopped by an adversarial review rather than by a control. Both tier-L reviews here have now caught a false headline that every gate passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
469d00d679
commit
1f0f652920
5 changed files with 804 additions and 31 deletions
|
|
@ -160,7 +160,7 @@ them.
|
|||
|
||||
```task
|
||||
id: CB-WP-0025-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "bdd9fc97-4706-4d8e-b286-91ff53339680"
|
||||
```
|
||||
|
|
@ -184,6 +184,53 @@ Tier L requires it. Exactly one round: challenge, then response, trail in
|
|||
*produces* them, but the reviewer should press whether that is a
|
||||
distinction worth a separate capability.
|
||||
|
||||
**Done 2026-08-05.** Trail:
|
||||
[challenge](../history/260805-could-we-have-won-challenge.md),
|
||||
[response](../history/260805-could-we-have-won-response.md).
|
||||
Separate agent, as in CB-WP-0022.
|
||||
|
||||
**Six of seven conceded, and the survey's headline finding is withdrawn —
|
||||
not softened.**
|
||||
|
||||
**C4 is the one that kills it, and the reviewer ranked it fourth.**
|
||||
Measured: a `FirstLegal` policy — take `legal[0]`, no heuristic — scores
|
||||
**0% at five and six seats** where greedy scores 100%, and **77.5% at two
|
||||
seats** where greedy scores 66%. **Two unsophisticated agents span the
|
||||
entire range at the same seat count**, so *"the game is too easy at 5–6
|
||||
seats"* is a statement about `GreedyPolicy`, not about GROUND. Their
|
||||
offered rescue — greedy hits the 12-point ceiling every time, so it is a
|
||||
rules claim — dies on the same data.
|
||||
|
||||
**C1: the per-node cost was wrong by 30–50×.** The timer bracketed two
|
||||
`setup`s, a whole greedy game and a validate+fold replay, then divided by
|
||||
player decisions. **The tell was in my own published output**: the figure
|
||||
*fell* as branching *rose*, which no per-enumeration cost can do.
|
||||
Re-measured at **3.0–4.1 µs**; the reviewer got 15.6–20.4 by a different
|
||||
isolation and **that discrepancy is unsettled** — T04 benchmarks it.
|
||||
|
||||
**C6: exhaustive search is not out**, which changes T03's premise. With
|
||||
C1's correction the budget is ~10⁵–10⁶ nodes and bounded endgame search
|
||||
fits — so the ADR cannot open with *"exhaustive is impossible, therefore
|
||||
determinized sampling"*, especially since sampling carries strategy fusion
|
||||
that exhaustive search does not.
|
||||
|
||||
**C3: the finding failed the admissibility rule this project wrote nine
|
||||
hours earlier** — sums where GROUND-WP-0004 T02 requires per-priority
|
||||
rows, and a harness with no assertions, no `--self-test` and no `make`
|
||||
target, so nothing can turn it red. A `default` artifact wearing a
|
||||
`counterexample` label.
|
||||
|
||||
**C2: the ratio explains nothing** — 3p and 4p share deal, threshold and
|
||||
ratio, and differ by 12.5 points of win rate. **C5: the "explains the
|
||||
maintainer's report" claim is contradicted by `lib.rs:2487`**, which
|
||||
records his losses as 3-player on the pre-ruling deal, unwinnable at 6
|
||||
against 7.
|
||||
|
||||
**T06 was pointed at GROUND-WP-0005, which is blocked on this number.**
|
||||
Sending it would have invited threshold changes on one bot's behaviour —
|
||||
the fifth wrong premise to reach ground-game. **Both tier-L reviews in
|
||||
this project have now caught a false headline that every gate passed.**
|
||||
|
||||
## Task: decide
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue