3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 7ed9fc730a |
CB-WP-0025 T06/T07: a difficulty baseline that reports its own confound
Some checks failed
ci / check (push) Has been cancelled
T06. games/ground/examples/difficulty.rs, make difficulty, wired into make self-tests, and a report file in ground-game under GROUND-WP-0005 with a hub message pointing at it. THE REPORT OPENS WITH THE RETRACTION, because what this task was written to send was withdrawn by T02 and GROUND-WP-0005 is blocked on exactly that number. They are told, in the first section, that we nearly sent them "the game is too easy at 5-6 seats" and why it was wrong. seats winnable greedy random first-legal spread 2p 60% 60.0% 5.0% 76.7% 71.7 3p 93% 88.3% 6.7% 25.0% 81.7 4p 100% 93.3% 6.7% 30.0% 86.7 5p 100% 100.0% 3.3% 0.0% 100.0 6p 100% 100.0% 3.3% 0.0% 100.0 SPREAD justifies the whole redesign: 71.7 to 100.0 points between three trivial policies. The table now shows why no single rate is a difficulty rather than asserting it. And the 5-6 rows point the OPPOSITE way from the withdrawn claim -- first-legal 0% against greedy 100% is the widest spread in the table, which suggests play matters MORE there, not less. Neither reading is established and the report says so. The confound is stated in the tool's own output, not only in prose: `winnable` is conditioned on greedy's play up to the final round, because searching from round 1 is unaffordable. Presenting it as a property of the deal would repeat this pass's error in a subtler form -- which is exactly how a corrected project reintroduces a defect. NO THRESHOLD CHANGES ARE PROPOSED. The instrument can fail (spec §5): a witness must replay to a win, an unwinnable position must report searched-out rather than a budget cut, a one-node budget must not claim exhaustion, and the policy panel must actually disagree. difficulty-baseline.rs marked superseded, kept as the survey's dated snapshot. Registered as F16, inconsistent / withdrawn. T07. evidence/CB-EV-0024. Five of nine defects came only from the review; four from execution, and all four of those were in work written after it. The wrong-denominator family now has five instances and still no control -- facts-check catches copies that disagree, nothing catches a number computed correctly against the wrong base. Tier L was an over-declaration (no port, structurally M) and paid for itself anyway, because the review is L-only. Chaos window 2 will close with zero overrides, making its retirement condition untestable. Named as open rather than implied done: the witness is NOT wired to the ending page. The search works; the browser cannot ask it yet. make all: exit 0. loop-lint clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
|||
| 1f0f652920 |
CB-WP-0025 T02: the review withdrew the finding, and the fifth wrong
Some checks are pending
ci / check (push) Waiting to run
premise never left the repo Separate agent, second tier-L review in this project. Six of seven challenges conceded. The survey's headline finding is WITHDRAWN, not softened. C4 kills it, and the reviewer ranked it fourth. A FirstLegal policy -- take legal[0], no heuristic at all -- scores 0% at five and six seats where GreedyPolicy scores 100%, and 77.5% at two seats where greedy scores 66%. Two unsophisticated agents span the entire range at the same seat count. "The game is too easy at 5-6 seats" is therefore a statement about GreedyPolicy, not about GROUND. The rescue the reviewer offered -- greedy hits the 12-point ceiling in 200/200 deals, so the 6p row is a rules claim -- dies on the same data: FirstLegal reaches that ceiling never. C1: the per-node cost was wrong by 30-50x. The timer started before the seed loop, so "us/node" included two setups, an entire greedy game and a full validate+fold replay, divided by player-decision count. The tell was in my own published output and I did not look at it: the figure FELL (161/139/112) as branching ROSE (4.7/7.4/9.1), which no per-enumeration cost can do. Re-measured with the clock around legal_commands alone: 3.0/3.5/4.1 us, now rising with branching. The reviewer measured 15.6-20.4 by a different isolation; we disagree by ~5x and neither has established which is right, so T04 must benchmark it with criterion rather than adopt either number. C6: "exhaustive search is out at any seat count" is false -- ~3 seconds over the last two rounds at 3p. With C1's correction the budget is ~10^5-10^6 nodes and bounded endgame search fits, so ADR-0013 cannot open with "exhaustive is impossible, therefore determinized sampling" -- especially as sampling carries strategy fusion that exhaustive search does not. C3: the finding failed the admissibility rule this project wrote nine hours earlier. 6/9/12 are sums where GROUND-WP-0004 T02 requires per-priority rows, and the harness has no assertions, no --self-test and no make target, so nothing can turn it red -- a `default` artifact wearing a `counterexample` label, by CB-WP-0022 T05's own distinction. C2: the ratio story explains nothing; 3p and 4p share deal, threshold and ratio and differ by 12.5 points of win rate. C5: "explains the maintainer's report" is contradicted by lib.rs:2487, which records his losses as 3-player games on the pre-ruling deal, arithmetically unwinnable at 6 against 7. T06 exists to report to GROUND-WP-0005, which is BLOCKED waiting on a difficulty baseline. Had this proceeded they would have been invited to move thresholds on the strength of one bot's behaviour. That is the fifth wrong premise this project would have sent them, and the second stopped by an adversarial review rather than by a control. Both tier-L reviews here have now caught a false headline that every gate passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
|||
| 469d00d679 |
CB-WP-0025 T01: survey -- the baseline found the game before the solver did
CB-RES-0008 with a runnable baseline (games/ground/examples/difficulty-baseline.rs), and the measurement produced a finding before any solver exists. A GREEDY BOT WINS 200 OF 200 GAMES AT FIVE AND SIX SEATS. Median margin +3, 11.8-12.0 points available against a threshold of 9. The curve is 66% / 82.5% / 95% / 100% / 100% across 2/3/4/5/6 seats. Row-level, as GROUND-WP-0004 T02 requires: available points 6 / 9 / 12 against thresholds 5 / 7 / 9, so the ratio RISES with seat count (1.20, 1.29, 1.33) while the table also gains actions per round to clear it with. Three multipliers pointing the same direction. It also explains the maintainer's report without needing a solver at all: "I felt it was too easy but then we lost" is two true statements about different seat counts. Cost measured and it rules out the obvious approach. Branching is small (mean 4.7-9.1) but legal_commands costs 112-161 us per call because it filters candidates through full validate. Exhaustive search is out at every seat count; 10^4-10^5 nodes is 1.4-14 seconds, which is the budget the ADR must design inside. Prior art names the trap: determinized search (PIMC) suffers strategy fusion (Frank, Basin & Matsubara 1998) -- the search picks different actions in states a real player cannot distinguish, so the witness may require knowing what was on top of the deck. Such a line still replays green, so the checkability benchmark does not catch it. Honesty and checkability are different properties; stated explicitly so T03 cannot conflate them. The survey states its own most likely killer up front (§6): a view-only search cannot fold events, so making the information boundary structural rather than a promise may not be affordable. Better found here than in T05. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
