premise never left the repo
Separate agent, second tier-L review in this project. Six of seven
challenges conceded. The survey's headline finding is WITHDRAWN, not
softened.
C4 kills it, and the reviewer ranked it fourth. A FirstLegal policy --
take legal[0], no heuristic at all -- scores 0% at five and six seats
where GreedyPolicy scores 100%, and 77.5% at two seats where greedy scores
66%. Two unsophisticated agents span the entire range at the same seat
count. "The game is too easy at 5-6 seats" is therefore a statement about
GreedyPolicy, not about GROUND. The rescue the reviewer offered -- greedy
hits the 12-point ceiling in 200/200 deals, so the 6p row is a rules claim
-- dies on the same data: FirstLegal reaches that ceiling never.
C1: the per-node cost was wrong by 30-50x. The timer started before the
seed loop, so "us/node" included two setups, an entire greedy game and a
full validate+fold replay, divided by player-decision count. The tell was
in my own published output and I did not look at it: the figure FELL
(161/139/112) as branching ROSE (4.7/7.4/9.1), which no per-enumeration
cost can do. Re-measured with the clock around legal_commands alone:
3.0/3.5/4.1 us, now rising with branching. The reviewer measured 15.6-20.4
by a different isolation; we disagree by ~5x and neither has established
which is right, so T04 must benchmark it with criterion rather than adopt
either number.
C6: "exhaustive search is out at any seat count" is false -- ~3 seconds
over the last two rounds at 3p. With C1's correction the budget is
~10^5-10^6 nodes and bounded endgame search fits, so ADR-0013 cannot open
with "exhaustive is impossible, therefore determinized sampling" --
especially as sampling carries strategy fusion that exhaustive search does
not.
C3: the finding failed the admissibility rule this project wrote nine
hours earlier. 6/9/12 are sums where GROUND-WP-0004 T02 requires
per-priority rows, and the harness has no assertions, no --self-test and
no make target, so nothing can turn it red -- a `default` artifact wearing
a `counterexample` label, by CB-WP-0022 T05's own distinction.
C2: the ratio story explains nothing; 3p and 4p share deal, threshold and
ratio and differ by 12.5 points of win rate. C5: "explains the
maintainer's report" is contradicted by lib.rs:2487, which records his
losses as 3-player games on the pre-ruling deal, arithmetically unwinnable
at 6 against 7.
T06 exists to report to GROUND-WP-0005, which is BLOCKED waiting on a
difficulty baseline. Had this proceeded they would have been invited to
move thresholds on the strength of one bot's behaviour. That is the fifth
wrong premise this project would have sent them, and the second stopped by
an adversarial review rather than by a control. Both tier-L reviews here
have now caught a false headline that every gate passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CB-RES-0008 with a runnable baseline
(games/ground/examples/difficulty-baseline.rs), and the measurement
produced a finding before any solver exists.
A GREEDY BOT WINS 200 OF 200 GAMES AT FIVE AND SIX SEATS. Median margin
+3, 11.8-12.0 points available against a threshold of 9. The curve is
66% / 82.5% / 95% / 100% / 100% across 2/3/4/5/6 seats.
Row-level, as GROUND-WP-0004 T02 requires: available points 6 / 9 / 12
against thresholds 5 / 7 / 9, so the ratio RISES with seat count (1.20,
1.29, 1.33) while the table also gains actions per round to clear it with.
Three multipliers pointing the same direction.
It also explains the maintainer's report without needing a solver at all:
"I felt it was too easy but then we lost" is two true statements about
different seat counts.
Cost measured and it rules out the obvious approach. Branching is small
(mean 4.7-9.1) but legal_commands costs 112-161 us per call because it
filters candidates through full validate. Exhaustive search is out at
every seat count; 10^4-10^5 nodes is 1.4-14 seconds, which is the budget
the ADR must design inside.
Prior art names the trap: determinized search (PIMC) suffers strategy
fusion (Frank, Basin & Matsubara 1998) -- the search picks different
actions in states a real player cannot distinguish, so the witness may
require knowing what was on top of the deck. Such a line still replays
green, so the checkability benchmark does not catch it. Honesty and
checkability are different properties; stated explicitly so T03 cannot
conflate them.
The survey states its own most likely killer up front (§6): a view-only
search cannot fold events, so making the information boundary structural
rather than a promise may not be affordable. Better found here than in
T05.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CB-RES-0007 plus a runnable baseline harness. Tier L invokes the
runnable-baseline option; the external candidates are practices rather
than software, so their rows are directional and cap at parity, and the
row that CAN be run is our own.
Measured: 6 findings across 11 files with no index, 2 of 6 (33%) with a
runnable reproduction, U1-U10 raised 2026-07-30 and first READ
2026-08-03 -- 4 days, 0 of 10 ruled.
The uncomfortable number is stated before the review can find it: the
proposed 'no finding without its reproduction' rule would reject four of
our six existing findings. The survey answers rather than routes around
it -- none of the four is expensive to reproduce, so 33% is evidence
nobody was ever asked for one.
Magic corrected an assumption this pass was about to build on. Rulings
are NOT authoritative -- they are 'reminder information with no actual
weight or rules meaning' -- and the authoritative fix folds into the
Oracle card text. So a finding closes when the SOURCE changes, not when
an annotation is added, and the register must be a queue that empties
rather than an archive that grows. That is now a constraint on the ADR's
lifecycle.
Model checkers supply the reproduction rule independently: a
counterexample trace IS the finding. W3C's implementation-defined mark is
the machinery we already have in provisional: scenarios and must reuse.
The loop-lint gate caught the new tool with no --self-test; it has one,
pinning the 2-of-6 baseline so a later edit cannot move it silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Delivers ADR-0007 Decision 1: visualization, drag-to-propose and hot-seat
play, at a measured marginal AM-4a cost of zero.
games-ground shipped: 23 third-party crates
cb-render-html: 23 third-party crates
new crates introduced: 0
Measured, not asserted — the survey's own lesson. AM-4a is unmoved at
246,250; own source is 7,636 -> 9,652.
What shipped:
crates/cb-render-html doc.rs (HTML/SVG emission, incl. the relationship
graph), input.rs (pointer facts -> commands),
serve.rs (Guard, Request, loopback bind)
tools/cb-play hotseat.rs + `--serve PORT`
Per ADR-0007 Decision 2 there is NO cb-render-api and NO cb-render-null.
The renderer targets the existing Project trait; the port waits for
stage 2's wgpu implementation to be its second use.
The six controls, all live, all mutation-checked (8 mutations, each red
for its stated reason):
1-3 token / Origin+Sec-Fetch-Site / explicit 127.0.0.1 bind
4 a token-less request is refused, in the unit AND over a real socket
5 JS may not construct commands — the page reports pointer facts, Rust
resolves them against the legal list the aggregate already offered,
and a test asserts the emitted script contains no game vocabulary
6 the coverage gate crosses the language boundary: it walks the
serialized view for leaf paths and requires each token to appear in
the PARSED emitted document, with a test that the parse really is a
parse (script/style contents must not count as rendered)
The gate fired on its author again, on its first run: ground_choices.*.
choice, ground_choices.*.problem and players.*.blame_from were in neither
list. The last is the one worth keeping — an EMPTY vector is a leaf path
of its own, and it now renders as an explicit absence.
Also, a mutation that did not go red: removing the Sec-Fetch-Site arm
alone left the cross-site test green, because the Origin check caught it
independently. Both had to be removed before the control bit. Recorded
because a control that passes for a reason you did not intend has not
been demonstrated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four of six challenges conceded; the survey's two main arguments are
replaced rather than clarified.
C1 The survey concluded the sub-100k region was empty without ever
measuring it. It is not empty: tiny-skia fits at 83,956, and the
real windowed floor is fltk at 140,079 — 1.5x corrected headroom,
not 128x. "Two orders of magnitude" is withdrawn.
C1b Following that concession: wgpu+winit is 1,741,979 marginal lines
against a 250,000 target. AM-4a is incompatible with INTENT stage 2
by 7x, and has been since both were written. Raised for the
maintainer; deliberately not decided by this pass.
C2 "Marginal cost zero" was scored on an axis chosen to produce zero,
and the same relocation already flattered sdl2 and fltk in the
survey's own table. One acquisition rule now covers all three, and
it raises two candidates' cost rather than lowering the winner's.
C3 The survey contradicted itself on the second-use rule. cb-render-api
and cb-render-null are withdrawn; cb-render-html ships against the
existing Project trait, and the port waits for stage 2's second use.
C6 The candidate measurements did carry a positive control. The C1
batch did not — it copied dep-weight.py's measurement function
without its guards — and was re-measured under them before being
cited. Figures reproduced unchanged. The HTML row is relabelled
0 by construction, not by measurement.
C4/C5 become six controls binding on ADR-0007.
The recommendation stands. The reasoning that reached it does not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured with dep-weight.py's own method so the numbers subtract.
Marginal cost against the 23-crate base graph, not headline size:
egui+eframe 2,782,849 742x headroom
wgpu+winit 1,741,979 465x
softbuffer+tiny-skia 1,147,081 306x
ratatui+crossterm 1,067,013 285x
macroquad 480,501 128x
HTML/SVG emission 0 0x
ratatui costing more than macroquad is the surprise: rustix pulls
linux-raw-sys at 479,901 lines.
Separately, AM-4a counts proc-macro crates that never reach a binary:
89,048 lines, 36.2% of the shipped-runtime figure, syn alone 66,916.
Real headroom is 92,798, not the 3,750 this repo has been citing.
The recommendation is unchanged by that correction — macroquad is 5.2x
over even at 92,798 — which is the only condition under which a survey
may propose correcting a number in its own favour. The metric fix is
filed separately, not bundled with the decision it unblocks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tier M — structural S by the letter of InnerLoop's table (no capability
port, no canonical interface, no new dependency), declared M because the
table's triggers only cover code capabilities and this decides whether the
loop constrains its own operation. Chaos d4=2, no override. Survey and ADR
merged per tier M.
The gap is not the one CB-WP-0006 assumed. SH-1/2/3 ARE measured —
cb-cost emits all three every run. The defect is that SH-1 and SH-2 are
cumulative means over every response ever recorded, so they cannot detect
a worsening trend: the history outvotes the present.
cumulative -> dfd0d6d 923 resp mean 206,952 p90 389,733 9.6%
CB-WP-0006 alone 156 resp mean 503,464 p90 607,105 0.0%
Read cumulatively, session shape looks stable and improving. Read by
window, the last pass ran at 2.4x the mean of all preceding history and
batched nothing at all — 146 tool calls in 146 responses.
This is the third shape of one disease. CB-WP-0005 found numbers that were
unmeasured; CB-WP-0006 found numbers measured but unasserted; these are
measured AND asserted and still cannot fire, because they are aggregated
over a window in which nothing can move them.
Decisions: window SH-1/SH-2 and keep cumulative as labelled history;
enforce with a live budget modelled on CB-01/CB-02, NOT `make all` —
failing the build on context would block committing, and committing is
what closes the attribution window and is the natural point to compact, so
a gate that blocks the remedy is a trap; targets do not move, since all
three are unmet by wide margins and adjusting them in the pass that first
enforces them is the retarget §Step 4 forbids; and SH-3 is tested by
batching deliberately rather than by argument.
Recorded and not fixed: InnerLoop's tier table triggers on code
properties, so this pass would be tier S despite deciding how the loop
governs itself. Changing the taxonomy on one instance is the
invention-in-isolation INTENT warns against — the same argument used to
amend K14 and to leave v1.4 alone. A second instance should trigger it.
CB-WP-0007 executes in 6 tasks. T04's third outcome is the interesting
one: if batching stays below 5% despite trying, 20% is the number to
challenge with evidence, in an ADR rather than a quiet edit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tier-L survey plus the full review trail. v1 was REJECTED by adversarial
review with four blocking findings; this is v2, and the finding below is
the reviewer's as much as mine.
The root cause is neither v1's story ("coverage scope is too narrow" —
true but shallow) nor the reviewer's cheaper one ("the link check points
at one file" — true but catches 3 of 7). It is:
Every coverage instrument in this project counts NAMES. None counts
ASSERTIONS. M-D1-COV counts covers: tags; M-D1-LNK counts rule-ID
strings. Both answer "is this rule mentioned?" and neither answers
"does anything fail if this rule is violated?"
Four of seven defects found this pass are named in the source and inert,
so no name-based check finds them. This subsumes the oldest error class:
all five prior harness-does-nothing instances have the same shape.
Two are mutation-proven, because the reviewer set that bar for findings
they had only read:
AM-7 — folding a 100k-event log from fresh(999) instead of fresh(42),
an unrelated genesis state, leaves the test green. The hash reaches
only a println!; the sole assertion is on elapsed time. And the log is
built across games seeded 42,43,44... so it could not be asserted as
written. evidence/CB-EV-0001 reports "AM-7 replay met, 2,290x" —
unearned on the hash-identical clause. HDN instance #6.
K9 — making Snapshot::take discard its EventSeq and store 0, which
destroys the half of K9 that says "+ the EventId it includes", leaves
the test green. On the acceptance criterion GameKernel §2.4 is zero of
three, not two of three.
Also found: K10, K14, K18 named nowhere in source; AM-10 vacuous (a K6
determinism lint reported under a D4 leak row, over a cb-*-api crate that
does not exist); AM-11's conformance suite does not exist; AM-1b measured
and omitted from the evidence scoreboard.
All twelve challenge findings conceded, none defended on its facts. v1's
prediction was rejected as unfalsifiable AND already determined; the
replacement names its refuting outcome and is beatable in both
directions.
Chaos d4=1, no override. Declaration 1 of the 12-declaration window.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review of where token-priced turns did work a deterministic tool could
do. Method: classify every turn in both transcripts by the tool calls it
made. The classifier is committed in tools/cb-cost.py and emitted by
`make cost-mix`, so the baseline is reproducible and the same command
can later falsify the predictions.
mech environment setup 84 turns $15.33
mech ad-hoc text patching 75 turns $13.86
git 37 turns $13.85
mech hub task status 25 turns $ 7.46
mech orientation / inspect 49 turns $ 6.87
hub other 32 turns $ 6.22
mech ad-hoc transcript 39 turns $ 4.56
mech workplan status edit 21 turns $ 4.06
MECHANICAL (dedup) 290 turns $51.26 = 38% of pass
Largest category is `cd` and `export PATH` -- pure friction, and
dep-weight.py already patched it at the leaf, which is evidence it was
noticed and fixed in the wrong place. Second is heredocs string-patching
markdown, which is also the mechanism behind duplicated-fact drift, the
error class InnerLoop v1.2 names and cannot gate.
Explicitly NOT automated: git (37 turns, $13.85) is mostly commit
message authorship -- the highest-output turns in the corpus and the
project's reasoning record. Automating it would save money and destroy
what makes corrections cheap.
CB-WP-0004 implements five candidates and predicts $33-41 recovery
(25-30%), below the 38% measured share on purpose: some inspection and
patching is genuinely exploratory.
The control loop is the deliverable, not a formality. T05 tests three
things and must report all: did mechanical turns disappear, did they
RELOCATE into prose, and did quality hold. If mechanical turns fall and
prose rises by as much, the saving is zero and that is the result to
publish.
Also a self-indictment worth recording: every hub update_task_status in
this project carried hand-typed token estimates, in a repo whose central
finding is that estimated token counts are worthless. T02 fixes it by
reading measured values from cb-cost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- InnerLoop definition-of-done gains a cost row: M-D2-CST may no longer
be recorded uncomputable, and composition must be reported, not only a
total.
- InnerLoop Step 5 gains the --self-test contract: every tool that
reports a number exposes one, and it runs before the number does.
Rationale attached, because the case that motivated it is the one
review cannot catch — survey and reviewer both verified the same large
sample and both missed the small one.
- make cost / cost-test / cost-pin on the one command surface; cost-test
in `make all` and in CI.
- Hub now holds the measured figure for CB-WP-0001: 80.6M in / 323.6k
out against the 401,100 it previously estimated, low by ~200x. The
event states plainly that the hub schema cannot represent the 88% of
cost that is cache, and names `make cost-pin` as the authority.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AC-1 $93.32 ($92.21 main + $1.11 subagent), residual $0.000000. No
unmeasured rows. Composition: 88.0% cache, 11.9% output, 249:1 context
re-read to text written.
The question that could not be answered before: what /compact costs.
The transcript records it directly — 542,991 -> 19,974 tokens, a 27x
reduction. Cost per turn fell from $0.457 (136 turns before) to $0.149
(202 turns after). The 202 post-compact turns cost less than half the
136 before them.
This qualifies CB-WP-0003 T04: sessions are bounded-quadratic, not
quadratic. Cost grows with context between compactions and resets at
each one, so the failure mode is a long UNCOMPACTED session. Whether a
fresh session beats a compaction is now measurable, and T04 should
measure it before prescribing one task per session.
Stale $0.66/$92.87 targets updated across survey and workplan.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adversarial review per InnerLoop Step 2 — separate agent, given only the
survey, one round. Challenge and response in history/.
The arithmetic held: every pinned figure and the whole three-method
correction table reproduced to the cent under ~20 independent spot
checks. The hand-written markdown tables did not. Three blocking
defects:
- the dedup positive-control table failed its own checksum (group
sizes summed to 348 groups / 663 lines, not 346 / 657; multi-line
groups are 206, not 208) — the worst place for a slip, since that
table is the sole evidence for the load-bearing dedup assumption
- the multi-model table was labelled "deduped" while holding per-line
counts, three lines below the paragraph condemning per-line counts
- the $92.21 acceptance target excluded the subagent tree, which ran
INSIDE the pin window. A collector correctly implementing the
survey's own blind-spot finding would have failed the benchmark.
Target is now $92.87, stated as its two components.
Folded in, each re-derived before conceding:
- attribution must key on sessionId, not wall-clock: two sessions
overlap 4h13m on this repo carrying ~$12 (one of them was this
workplan's own background job)
- only 14 of 33 commits name a task; the other 19 hold 33% of cost
- cache writes bill per TTL — pricing the aggregate at 1h inflates
the subagent by 43%
- the "~87% cache" figure matched nothing; 64.6% of cost is cache
read, 88.4% is all cache
- fable is 35% of calls and 60% of dollars; opus is the inverse
Review cost ~$0.60 against a $92 pass, the second consecutive round
where that trade found something material. Evidence for CB-WP-0003 T03.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tier L (structural L, chaos d10=2). Every candidate exercised locally
against the CB-WP-0001 transcript rather than cited.
Verdict: session transcript JSONL is the only exact instrument; git
commits supply the attribution index it lacks; the State Hub is a
durable sink whose schema cannot express cache tokens at all; the
status bar and billing API are eliminated on availability.
Three errors found in this workplan's own Purpose section, all
overstating cost:
- one API response is written as up to six JSONL lines, each
repeating the same usage object (657 lines, 346 responses)
- a multi-model session was priced entirely at Fable 5 rates,
violating a per-model rule MetricsAndScenarios §1a already had
- the transcript is a live file, so an unpinned total is not
repeatable (346 -> 356 responses between two runs minutes apart)
CB-WP-0001 pinned at fc76445: $92.21, not $248.46. The qualitative
finding sharpens rather than weakens — 88.4% of spend is cache, at
256:1 cache-read to output tokens.
Neither error was of the harness-does-nothing class the positive
control was written for; both sums ran over real data. What caught
them was re-deriving a number instead of quoting it. Raised for T07
and for CB-WP-0003 T01.
Also measured: the one subagent (adversarial review) cost $0.66 and is
invisible to a collector reading only the main transcript.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
