T06. games/ground/examples/difficulty.rs, make difficulty, wired into make
self-tests, and a report file in ground-game under GROUND-WP-0005 with a
hub message pointing at it.
THE REPORT OPENS WITH THE RETRACTION, because what this task was written
to send was withdrawn by T02 and GROUND-WP-0005 is blocked on exactly that
number. They are told, in the first section, that we nearly sent them "the
game is too easy at 5-6 seats" and why it was wrong.
seats winnable greedy random first-legal spread
2p 60% 60.0% 5.0% 76.7% 71.7
3p 93% 88.3% 6.7% 25.0% 81.7
4p 100% 93.3% 6.7% 30.0% 86.7
5p 100% 100.0% 3.3% 0.0% 100.0
6p 100% 100.0% 3.3% 0.0% 100.0
SPREAD justifies the whole redesign: 71.7 to 100.0 points between three
trivial policies. The table now shows why no single rate is a difficulty
rather than asserting it.
And the 5-6 rows point the OPPOSITE way from the withdrawn claim --
first-legal 0% against greedy 100% is the widest spread in the table,
which suggests play matters MORE there, not less. Neither reading is
established and the report says so.
The confound is stated in the tool's own output, not only in prose:
`winnable` is conditioned on greedy's play up to the final round, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would repeat this pass's error in a subtler form -- which is exactly
how a corrected project reintroduces a defect. NO THRESHOLD CHANGES ARE
PROPOSED.
The instrument can fail (spec §5): a witness must replay to a win, an
unwinnable position must report searched-out rather than a budget cut, a
one-node budget must not claim exhaustion, and the policy panel must
actually disagree. difficulty-baseline.rs marked superseded, kept as the
survey's dated snapshot. Registered as F16, inconsistent / withdrawn.
T07. evidence/CB-EV-0024. Five of nine defects came only from the review;
four from execution, and all four of those were in work written after it.
The wrong-denominator family now has five instances and still no control
-- facts-check catches copies that disagree, nothing catches a number
computed correctly against the wrong base. Tier L was an over-declaration
(no port, structurally M) and paid for itself anyway, because the review
is L-only. Chaos window 2 will close with zero overrides, making its
retirement condition untestable.
Named as open rather than implied done: the witness is NOT wired to the
ending page. The search works; the browser cannot ask it yet.
make all: exit 0. loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ground-game ruled all ten U-items on 2026-08-03, every one CONFIRMED as
the default clay-borg simulates, and confirmed five of six provisional
scenarios. clay-borg never collected the answers: CB-RES-0007 reported "0
of 10 ruled" the same day, and CB-WP-0022 built the finding register two
days later still recording them as `reported`. make design's first run is
what noticed -- not a human, not the adversarial review that found four
other things.
That is the unread-inbox failure running in the opposite direction, and it
appears nowhere in the declaration, survey, ADR or spec of the pass that
was built entirely around the forward version. It is arguably worse: an
unread message is visible as silence, while a collected-but-unapplied
ruling looks exactly like work in progress.
Ten rulings quoted into §Underdetermined (the three conditional ones
verbatim -- U1's designer note, U2's End-only trigger, U8's
consume-only-if-it-cancels). Five provisional flags lifted, replaced by
ruled/ruled_by/ruled_note so the flag went and the provenance stayed.
Register queue 9 -> 0.
T02's control came back clean: make sim is 26 passed, 59 rules covered,
nothing red. Had a scenario gone red it would have meant we described our
own behaviour incorrectly to ground-game.
I wrote two U-item mappings and both were wrong. gr-a04 -> U1 (it asserts
consent is REQUIRED; U1 asks WHEN the target accepts) and gr-d05 -> U5 (it
exercises the UNREJECTED Reverse; U5 is the rejected one). Both plausible
from covers:, neither survived reading the description. Third and fourth
instance of this defect; the first two reached ground-game. So encodes_u_item
is now a declaration and design.py asserts the file names what it claims --
and that check's own first version grepped for mentions and went red when
two files recorded why they do NOT encode U1 and U5. A mention is not a
claim, which is exactly the looseness that let "six of the ten have
provisional scenarios" stand.
Two positive controls went red for the best possible reason, both broken
the same way -- asserting against live repo data instead of constructing
their condition. rule-coverage.py required at least one provisional item
to EXIST; it now builds a fixture and reports the live count as a
diagnostic, because there is no number of provisional items this project
should have. design-baseline.py pinned "2 of 6" while recomputing one row
from a live glob, so the dated snapshot was never a snapshot; frozen to
its 2026-08-03 list and unwired from self-tests, since per ADR-0012 D8 it
is no longer a reporting tool.
ScenarioFile is deny_unknown_fields and refused the four new fields until
declared -- correct: a corpus accepting unknown metadata would let a typo'd
encodes_u_iem sit there claiming nothing.
DEVIATION: ADR-0012 D2 said "no new file". GroundRules.md crossed the
loadability limit, so the register moved to specs/FindingRegister.md. D2's
substance holds -- one register, same machinery, nothing competing -- but
the literal instruction did not, and it resolves an awkwardness D2 named
itself.
make all: exit 0. loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T05. tools/design.py, make design, and the register in GroundRules.md --
14 rows, no new file, because ADR-0012 D2 made §Underdetermined the
register rather than building one beside it.
Backfill was the test and it caught two things the ADR did not have.
First, a `role` column. The first report alarmed on U2 and was wrong to:
U2's scenario is green BECAUSE the provisional default it documents is
implemented, which says nothing about whether ground-game agrees. GR-E01's
was a counterexample that went green. Same colour, opposite meaning -- a
register that cannot tell them apart either alarms constantly or never.
Only a green counterexample alarms. Folded back into GameDesign §1.3.
Second, it contradicted the survey. CB-RES-0007 said "six of the ten
already have provisional scenarios." Measured -- grep -lE "\bU<n>\b" over
scenarios/ground -- exactly ONE U-item names itself. Five provisional
scenarios exist and four probably encode U-item defaults, but the mapping
is not written down, so it is not checkable. Same defect class as the
wrong premises, found inside the survey that proposed the fix. Now a
reported debt: open, lacking a reproduction: 9, target 0.
design.py carries the control design-baseline.py never had, asserted
directly: a row citing a nonexistent file must not count as reproduced,
using the exact path 2da19a4 deleted -- which the old tool called green.
design-baseline.py is marked superseded rather than deleted; it is the
evidence for how a wrong number got into a survey.
T06. The report is a FILE in ground-game under GROUND-WP-0002, committed
there, with a hub message that only points at it. It asks for no ruling:
it carries GR-E01's withdrawal, our reproduction debt, and two notes that
are explicitly not findings.
And it had to acknowledge something nobody anticipated. GROUND-WP-0002 is
finished -- all ten U-items were RULED 2026-08-03, every one confirmed as
the default we simulate, plus five of six provisional scenarios. The
survey said "0 of 10 ruled" two days later and this register was built
saying `reported`. That is the unread-inbox failure running in the
opposite direction: they answered and we did not collect it. The
instrument's first run surfaced it. They are `ruled`, not `applied` --
lifting the now-settled provisional flags is owed and is not done, and
make design shows them open until it is.
T07. evidence/CB-EV-0021. Two of six catches in this pass came from
execution rather than process (the role distinction from building it, the
ten uncollected rulings from running it), which is InnerLoop §Design
goal's prediction holding.
make self-tests, facts-check, loop-lint: clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The row folded a 5,000-event log against a 100,000-event log and compared
throughputs, which confounds 'does cost per event grow with history'
(the property it claims) with 'does streaming a 20x longer Vec cost more
per element' (a memory-hierarchy fact true of any program). It measured
the second and reported it as the first: importing the edition enlarged
the aggregate and the ratio fell to 0.845 with the state bounded.
Corrected to time the SAME 5,000 events on a state at depth 0 and on a
state at depth 100,000. Equal windows, equal event mix, so the only
difference left is history depth.
corrected: clean 1.004, mutated 0.589 (red)
old: clean 0.845 (red on healthy code), mutated 0.751
It also runs in 8.5s instead of timing out: the first version re-walked
the 100k prefix every repetition, 200M untimed folds per sample, which
under the mutation never finished. A control that cannot be run is not a
control. It now advances to depth once per sample and clones.
Two of my own measurements here were wrong and both were caught by
measuring again. A 2-minute timeout killed the shell line before its
restoring cp ran, so three readings were taken on MUTATED code -- I
diagnosed an event-mix confound that did not exist and 'fixed' it. The
fix is kept on its merits; the justification was fiction. And the probe
that proved state was bounded had checked four of eleven collections.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ADR-0011 decided it: vendor the CSV with a checked digest, read it with
a ~50-line reader, and let the hashes move.
The declaration's constraint was measured against the WRONG BUDGET. It
said a CSV crate costs 21,613 against AM-4a's 3,798 of headroom, '5.7x
over, settled by measurement'. But setup and problem_priorities are
cfg(scenarios) and are not in the shipped runtime at all, so AM-4a never
sees them. Against AM-4b, csv costs 17,651 against 19,742 -- it FITS,
with 2,091 to spare. It is refused anyway, on proportion: 89% of the
budget's remaining capacity to read 20 rows. The revisit condition is
stated (nested quoting, embedded newlines, multiple dialects).
GR-S01 now deals Surface + hidden 1..=k as ruled, with edition values and
suits. Measured: 6/9/12 available against thresholds 5/7/9 -- the game is
winnable at every seat count, which is what the maintainer could not do.
gd0001 is INVERTED, not deleted, and now also asserts the 6/9/12 so a
deal that is reachable for the wrong reason still fails.
Blast radius was scenario expectations, exactly as the ADR predicted: no
scenario pinned a hash and no bundle is committed. Six scenarios and two
unit tests updated, each with a note. gr-e01-threshold-unreachable-2p is
RENAMED to -reachable- and rewritten as the non-provisional import check
ground-game asked for by name. gr-e03's setup was restructured, not just
renumbered: with values 2,2,2 its personal-edge test would have tied
three ways and asserted nothing.
BLOCKING: AM-7 fails at median 0.845 against its 0.9 floor. Isolated
across three runs -- 3 problems + stand-in 0.97, 3 problems + edition
0.909, 4 problems + edition 0.845. State is BOUNDED (proven: identical
after 5k and 100k events), so this is not the unbounded-growth defect
AM-7 exists to catch; it is a bigger working set streaming a long log.
Whether AM-7's floor is still right for a larger aggregate is a spec
question and lowering it requires an ADR, so it is not being tuned here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CB-RES-0007 plus a runnable baseline harness. Tier L invokes the
runnable-baseline option; the external candidates are practices rather
than software, so their rows are directional and cap at parity, and the
row that CAN be run is our own.
Measured: 6 findings across 11 files with no index, 2 of 6 (33%) with a
runnable reproduction, U1-U10 raised 2026-07-30 and first READ
2026-08-03 -- 4 days, 0 of 10 ruled.
The uncomfortable number is stated before the review can find it: the
proposed 'no finding without its reproduction' rule would reject four of
our six existing findings. The survey answers rather than routes around
it -- none of the four is expensive to reproduce, so 33% is evidence
nobody was ever asked for one.
Magic corrected an assumption this pass was about to build on. Rulings
are NOT authoritative -- they are 'reminder information with no actual
weight or rules meaning' -- and the authoritative fix folds into the
Oracle card text. So a finding closes when the SOURCE changes, not when
an annotation is added, and the register must be a queue that empties
rather than an archive that grows. That is now a constraint on the ADR's
lifecycle.
Model checkers supply the reproduction rule independently: a
counterexample trace IS the finding. W3C's implementation-defined mark is
the machinery we already have in provisional: scenarios and must reuse.
The loop-lint gate caught the new tool with no --self-test; it has one,
pinning the 2-of-6 baseline so a later edit cannot move it silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Provenance (tier S, one paragraph in lieu of survey and ADR): the two
clauses mutation-check has reported inert since CB-WP-0005. AM-7's
scaling ratio was held up by a test literally named
replay_100k_events_is_linear_and_fast that computed both throughputs,
printed both, and never divided one by the other. AM-8's N=10 was held
up by a runner that does two.
Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's
>=10-of-14 prediction MET for the first time. Neither was closed by
amending the question away, which was the live risk: the denominator is
unchanged and the four unenforced rows are the four already
unenforceable.
AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's,
correct for a floor on one number) gave 0.581-1.085 on an unchanged
binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at
fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU
contention while absolute throughput fell 4x. The INDETERMINATE guard
demanded unanimity and failed a good measurement over one sample
0.001 under the floor; it now requires a two-thirds majority. The
control that matters: AM-6's constant-cost mutation halves throughput
and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6.
AM-8 kept N=10 because the measurement said so. Perturbing the RNG only
from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A
late-onset divergence is deterministic, not flaky, so it is a control
rather than a coin flip. Ten runs live on one scenario (make am8, ~2s)
rather than all 25 (47s a build). GameKernel 5b records it.
The full run also found AM-4a's own mutation stale since ADR-0008 D3
moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported
HARNESS-BROKEN, no score published. The build-free half of that check
is now a --self-test assertion, so make all catches the next one.
mutation-check clauses may now carry their own verify and mutation, and
then the enforced flag is measured rather than declared; a declaration
disagreeing with its measurement is refused.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A human seat is a Policy like any bot, so the CLI adds no second driver:
HumanPolicy renders the projection, lists the legal commands and reads an
index or `pass`. `make play` runs it; `--all-bots` watches one.
K13's Project trait gains its first implementor after six passes with
none. Hidden: other seats' face-down selections until Reveal, hands and
deck (counts only), a face-down Problem's suit and value, and the seed —
not secret content, but a seat holding it can compute the deck.
A played session becomes an artifact: --record writes it as a scenario
the runner executes, --replay writes a .cbreplay bundle. record.rs is the
inverse of parse_command and its warrant is a round-trip test over every
command shape.
The acceptance test for the projection passed vacuously twice. First it
asserted the text contained "face-down", which every render does because
of Problems. Counted, it then reported zero inspected entries: seats are
asked in order, so a human at P1 is prompted before anyone has selected.
Seated at P3 it inspects ten entries and dies when the projection is
mutated to reveal everything. Counting what the harness examined caught
both, which is the second time that remedy has worked where a stronger
predicate would not have.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had
no flag parsing at all, so --replay had nowhere to go.
The bundle is manifest + commands.log + initial.snapshot + expected.yaml,
dev-only behind the scenarios feature and charged to AM-4b. The command
stream goes through the K11 framing built in T05, so a truncated bundle is
detected rather than replayed short — the two tasks compose rather than
duplicating.
The reviewer's D2 correction was real: this was not "a directory of four
files". Pass carried only the end state, RunOutcome::Failed was a
formatted String, and scenario.rs created an EventLog, appended to it and
never read it. All three had to change.
The first round trip failed to reproduce, and the cause is worth keeping:
state_hash_hex over a serde_json::Value is a different canonical form than
over the typed aggregate — Value's map is key-sorted, a struct serializes
in declaration order. The bundle was written with one basis and verified
with the other. A round trip written to recompute its own comparison value
would have PASSED this bug; it failed because the recorded hash came from
the producing process, which is control 2's entire purpose.
make replay-test implements ADR-0005 §6's four controls, 14/14: a
committed deliberately-failing fixture outside the corpus with covers: []
so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash
must fail; a log short by one byte and a corrupted length prefix must be
rejected; and a mutated manifest seed must fail — which bites only because
replay re-derives the initial state from seed+setup and checks it against
the recorded snapshot, since restoring from the snapshot alone would leave
the seed inert. Plus a control on the controls: the bundle must still
replay after every mutation is reverted.
AM-7's hash-identical clause is re-earned. The probe records a hash per
per-game segment and replays each from its own genesis; folding from the
wrong seed now fails. That is the clause ADR-0005 §4 withdrew as
mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so
AM-7 stays PARTIAL — reported, not rounded up.
Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's
copy of that number going stale, on a number that moved the same hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AM-4c is withdrawn from the acceptance table and retained as a reported
diagnostic. GameKernel §5a carries the argument.
The ratio has no monotone better direction. INTENT's rule is "own the
semantics, assimilate the implementation": rising can mean owning
semantics properly or reimplementing what should have been assimilated;
falling can mean leverage or dependency bloat. A target requires knowing
which way is better. It is also redundant — AM-4a/AM-4b bound the
denominator and AM-2 bounds own-source density, so AM-4c is a ratio of two
already-targeted quantities.
Measured at withdrawal: 1,426 own lines per 100k third-party (shipped),
1,107 (dev). make dep-weight now prints both, labelled diagnostic — the
row was never actually reported before.
M-D1-MUT keeps AM-4c in its denominator on purpose and says so in the
output. Dropping it would move the score 7/14 -> 7/13 without enforcing
anything: a score improved by deleting the question.
Decided before Phase B deliberately, since ADR-0005 predicts own-source
growth that will move this ratio; deciding after would be the retarget
§Step 4 forbids.
A T01 correction found here. The AM-6 gate failed inside `make all` at
38,753 ev/s against 341,280 in isolation — a 9x drop, because cargo test
runs binaries and threads concurrently. A throughput assertion inside a
parallel harness measures contention, not throughput. T01's measurement
was valid; its gate placement was not.
Fixed by running it only where valid — #[ignore] plus `make am6` in
release with --test-threads=1, now 2.0M ev/s at 20.2x headroom — and not
by lowering the target, which T01 forbade. My first attempt did drift that
way, adding a debug "sanity floor" of 50,000, and was backed out: a second
threshold is still a second chance to tune.
The mutation then went SURVIVED on the first run after the move. 4,000
black_box iterations were calibrated against debug's 3.4x headroom and are
invisible against release's 20x. Raised to 100,000; back to red. A weak
mutation is not a fixed property of a row — it can become weak when the
row's measurement conditions change.
Tier S (amends one row, creates no capability), chaos d4=2, no override.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x
headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind"
was right, but it is now measured rather than assumed, and verified red by
a property mutation (a 300 MB allocation in the workload).
AM-5 is BREACHED on both readings, on the machine the spec names:
dev toolchain (default features) 87.0 s [FAIL target <= 60 s]
shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s]
bnt-lap001, 8 cores — a direct comparison, not a directional one. A row
declared "recorded not gated" and never recorded fails its own target by
45% on first measurement.
The tool reports and exits 0 because the spec says the row is ungated.
Gating it is a spec change needing an ADR; a tool that promotes itself is
how a target starts binding without anyone deciding it should. So AM-5
stays unmutatable — for the accurate reason now — and the breach is raised
as a maintainer decision: speed the build, move the target by ADR (arguing
why 60 s was wrong rather than why 87 s is convenient), or withdraw the
row.
The measurement itself had a real bug, found only by cross-validation.
getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child,
so it attributed cargo's memory to the workload and reported 38.2 MB for a
run that used 12.3 MB — a 3x over-report that was plausible, passed its
target, and would have been published. Fixed with os.wait4, which returns
that specific child's rusage, and the self-test now cross-checks against
/usr/bin/time -v.
That is the false-accusation shape in the measurement layer rather than
the mutation layer: an instrument confidently reporting a number it had
not earned.
Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather
than running `cargo clean`, so measuring the metric does not cost several
minutes of rebuild afterwards. A metric that punishes its own measurement
gets measured once and never again.
M-D1-MUT: 6 -> 7 of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both rows were unmutatable for the same stated reason. They resolved
differently, and the difference is the point.
AM-2 is instrumented and enforced — tools/size-metrics.py, in `make all`:
AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom)
1,575 impl lines / 58 rules
Tests are excluded because AM-2 asks what a rule costs, not how much it is
exercised; lib.rs is ~18% test code and including it would have flattered
the number. This matters because AM-2 is AM-1's anti-gaming pair: 100%
rule coverage means nothing if the rules are trivially small, and AM-1 has
been reported met since CB-WP-0001 with its pair uninstrumented.
Verified red by a property mutation — ~800 lines of filler injected into
the impl, pushing the ratio past 40 — not a threshold tweak. The expect
string is the precise failure signature "FAIL target <= 40"; my first
attempt used "AM-2", which also matches passing output and would have
made the FA guard vacuous.
AM-3 is BLOCKED, not uninstrumented, and that is a finding rather than a
deferral. It measures LOC to express the CB-RES-0001 synthetic game on our
kernel, against a boardgame.io baseline of ~36 LOC for a declarative 3p
commit/reveal game object. That artifact has never been built: games/
contains only ground, and benches/synthetic.rs drives GROUND rather than
defining a synthetic game. Measuring GROUND's 1,575 impl lines against a
36-line synthetic game object would compare two different games and call
the difference a D1 result.
So the tool ships the measurement — a marker-delimited region, self-tested
— and reports the row blocked, naming the missing artifact. A number would
have been worse than a blank. It stays unmutatable and still counts
against M-D1-MUT per ADR-0005 §1: a row that cannot fail asserts nothing,
however good the reason.
M-D1-MUT: 5 -> 6 of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
For each acceptance row in GameKernel §5, invert the property and require
the verifying command to go red. adapted:mutation-testing, with the
denominator changed from source lines to acceptance rows.
M-D1-MUT: 4/14 rows enforced
PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert)
unmutatable 8 (no property to invert, reason stated per row)
SURVIVED 0
Two corrections to our own numbers. There are 14 rows, not the twelve
ADR-0005 and CB-WP-0005 both asserted — AM-4 splits into a/b/c. And the
prediction of 9-of-12 (75%) becomes >=10 of 14; measured 4 (29%), badly
unmet. No target moved in this commit.
The second correction matters more. My first run reported two SURVIVED
rows and both were my own no-op mutations: `pub struct NullRng;` ->
`pub struct NullRng {}` is semantically identical, and renaming
max_age_days does nothing because CA-17 reads it with a default of 90.
Both would have been published as "this row asserts nothing" — a false
accusation against code that is fine. Replaced with real inversions (a
per-construction counter in the ChaCha seed; reverting AC-9's output
resolution to the first-wins bug it was fixed for), after which both go
red. T08 asks whether writing a weak mutation is the new grep. It is,
demonstrably, on the first attempt.
The finding is larger than the workplan assumed. 8 of 14 rows are
unmutatable — AM-2, AM-3, AM-4c, AM-5, AM-6, AM-9, AM-10, AM-11 have no
instrument at all. AM-6 is the sharpest: nothing in the workspace
compares any number to 100,000 events/s, the headline throughput claim.
The problem is not three unimplemented rules, it is that more than half
the acceptance table has nothing behind it.
Harness controls: a stale find-string reports HARNESS-BROKEN rather than
scoring the baseline as the mutant; a red baseline reports inconclusive
rather than red; the tree is restored in a finally and the restoration is
verified. Not in `make all` — it rebuilds the workspace once per row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Duplicated-fact drift is the fourth error class and the only one with no
executable rule. No positive control catches it (both copies are
internally consistent) and re-derivation does not either (the copy
reproduces whatever it was copied from). It is caught only by reading a
copy against its source, which nothing in the loop required.
facts.toml holds 15 facts and is GENERATED by `make facts-gen` from
cb-cost, dep-weight and rule-coverage. The trap this task named — a
hand-maintained registry that becomes another drifting copy — is closed
by facts-check re-running the instruments and failing when the committed
registry disagrees with them. A stale registry cannot certify stale
artifacts.
An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key.
17 occurrences across 5 artifacts are now checked.
Falsified before being believed: changing CostAccounting.md line 158 from
$93.15 to $92.87 — the exact historical drift — produced exit 1 naming
the file, the line and the expected value. Tested against the class it
exists to catch, on a real artifact, not only in its self-test.
It then caught a live tag inside its own documentation example in
InnerLoop.md within the hour. Third time a gate has failed on its own
pass's work.
What it does not close is stated rather than implied: 22 untagged literal
copies remain and are reported, not failed. Tagging is opt-in, a number
can legitimately recur, and a gate that fires on coincidence gets routed
around. Naming the uncovered surface beats claiming the class is closed.
InnerLoop single-source-of-fact moves from prose to executable — v1.3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
49 turns and $6.87 per CB-RES-0003 went to grep/ls/wc answering "what is
the state of this repo". tools/status.py answers it in 22 lines: commit
and working-tree state, every workplan with task counts, the next todo
task, spend since the last commit against CB-01/CB-02, and the three fast
gates.
Two constraints are deliberate. It does not build — a status command that
takes two minutes gets replaced by `ls` within a day. And it states its
own limit in the output ("slow gates not run here"), so a green status
cannot be misread as a green `make all`; the self-test asserts that line
is present, and that the whole report stays under 40 lines.
The positive control is the refusal to be confidently empty: a parser
that found zero workplans and zero tasks would print a clean, wrong
picture, which is worse than the greps it replaces. It also caught a
dot-all regex that returned twenty paragraphs of the preceding task's
prose as the "task heading".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the three hand-done steps of a task close (46 turns, $11.52 per
CB-RES-0003): the heredoc flipping status in the workplan file, the
hand-written hub call, and the hand-typed token counts.
The third is the reason this task exists. Every update_task_status this
repo produced carried estimated tokens_in/tokens_out — in a project whose
central finding is that estimated token counts are worthless. task-done
reads the measured figure from the transcripts, or refuses; there is no
path through it that emits an estimate.
cb-cost gains by_task_detail: cost, response count, model histogram and
token components per task. task-done imports cb-cost rather than parsing
its printed table, so the hub figure is not a copy that can drift from
its source.
The positive control found a real defect before the tool ran once.
Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002
T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test
reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x
overstatement would have been pushed to the hub as a *measured* number —
the same fiction in a new form. task_label() now keys qualified subjects
on the full id and leaves unqualified ones bare rather than
retro-assigning them to a workplan. The pinned $93.15 benchmark is
unchanged, so historical attribution was not disturbed.
Fourth instance of trusted arithmetic: a number believed because a
program produced it rather than a hand.
Refusals, all exercised by --self-test: unknown id, typo'd id,
already-done task, missing state_hub_task_id, no measured spend, and a
status flip that produced no change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical
category — spent prefixing commands with `cd` and
`export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once
instead of per-leaf.
tools/repo.py resolves the repo root from __file__ and cargo from PATH
then the standard rustup locations. Every tool imports ROOT from it, so
the repo path is stated once rather than redefined in four files —
single source of fact, the rule DFD earned in InnerLoop v1.2.
rule-coverage and dep-weight now call enter_root(), which is why their
relative paths did not need rewriting one by one.
The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same
way, so `make -C <repo> <target>` works from any directory with no
prefix.
make env-test is the positive control, and is in `make all`: every tool
runs from / with PATH=/usr/bin:/bin. Without it this fix could regress
silently and invalidate T05's measurement — the whole point of the
control loop.
dep-weight's "cargo not on PATH" error is kept rather than deleted. It
should now be unreachable, and --self-test asserts cargo_bin() resolves
unaided; a control that never fires is cheaper than a regression.
loop-lint failed on repo.py on its first run — a reporting tool with a
positive control but no --self-test entry point. Second time the gate
has caught work from its own pass within the hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review of where token-priced turns did work a deterministic tool could
do. Method: classify every turn in both transcripts by the tool calls it
made. The classifier is committed in tools/cb-cost.py and emitted by
`make cost-mix`, so the baseline is reproducible and the same command
can later falsify the predictions.
mech environment setup 84 turns $15.33
mech ad-hoc text patching 75 turns $13.86
git 37 turns $13.85
mech hub task status 25 turns $ 7.46
mech orientation / inspect 49 turns $ 6.87
hub other 32 turns $ 6.22
mech ad-hoc transcript 39 turns $ 4.56
mech workplan status edit 21 turns $ 4.06
MECHANICAL (dedup) 290 turns $51.26 = 38% of pass
Largest category is `cd` and `export PATH` -- pure friction, and
dep-weight.py already patched it at the leaf, which is evidence it was
noticed and fixed in the wrong place. Second is heredocs string-patching
markdown, which is also the mechanism behind duplicated-fact drift, the
error class InnerLoop v1.2 names and cannot gate.
Explicitly NOT automated: git (37 turns, $13.85) is mostly commit
message authorship -- the highest-output turns in the corpus and the
project's reasoning record. Automating it would save money and destroy
what makes corrections cheap.
CB-WP-0004 implements five candidates and predicts $33-41 recovery
(25-30%), below the 38% measured share on purpose: some inspection and
patching is genuinely exploratory.
The control loop is the deliverable, not a formality. T05 tests three
things and must report all: did mechanical turns disappear, did they
RELOCATE into prose, and did quality hold. If mechanical turns fall and
prose rises by as much, the saving is zero and that is the result to
publish.
Also a self-indictment worth recording: every hub update_task_status in
this project carried hand-typed token estimates, in a repo whose central
finding is that estimated token counts are worthless. T02 fixes it by
reading measured values from cb-cost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 8k/10k per-task token budget was never referenced or enforced and
T08 blew past it silently. Replaced with a budget that can actually
fire.
The design constraint is the interesting part: per-task cost needs the
commit that CLOSES the task, so a per-task budget is unavoidably
retrospective -- it can only report a breach after the money is spent,
which is the dead-policy failure again. What IS observable mid-task is
spend since the last commit, because the transcript is append-live. So
the budget binds on the open remainder.
CB-01 budget = USD since the last commit, via `make cost-budget`
CB-02 soft $10.00 (state progress, decide), hard $22.00 (stop)
Calibrated on the 32 non-empty commit intervals of CB-WP-0001: p50
$1.40, p90 $9.36, max $10.80. Soft sits just below the observed maximum
-- it would have fired exactly once on the calibration pass. Hard is ~2x
the observed max, a value never reached in 32 intervals, so reaching it
means the session is doing something the data has no example of.
Both thresholds are set ABOVE every observed value, so they bind on
future work rather than ratifying present work -- the distinction T07
is about.
Stated limit: it is a command, not a daemon. An agent that never runs it
gets no signal, which is the dead-policy failure one level up. Mitigated
only by being free to run and on the one command surface.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
make coverage compares the GR-rule IDs defined in specs/GroundRules.md
against the covers: lists in scenarios/ground/*.yaml. It exits 1 if a
scenario claims a rule the spec does not define, so coverage cannot be
inflated by an invented ID, and exits 2 while rules remain uncovered.
Current reading: 34/58 (58%). AM-1 requires 100%.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
