2026-07-31 01:57:13 +02:00
|
|
|
# One command surface (InnerLoop §agentic-efficiency #3). Deterministic,
|
|
|
|
|
# greppable output; precursor of the `cb` CLI.
|
2026-07-31 10:13:52 +02:00
|
|
|
#
|
|
|
|
|
# CB-WP-0004 T01: every target here runs from a clean shell, from any
|
|
|
|
|
# directory, with no prefix. Invoke as `make -C <repo> <target>` from
|
|
|
|
|
# elsewhere. No target requires `cd` or `export PATH` — CB-RES-0003
|
|
|
|
|
# measured 84 turns and $15.33 spent on exactly those two prefixes.
|
2026-07-31 01:57:13 +02:00
|
|
|
|
2026-07-31 10:13:52 +02:00
|
|
|
# Absolute path to this Makefile's directory, so recipes never depend on
|
|
|
|
|
# the caller's working directory.
|
|
|
|
|
REPO := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
|
2026-07-31 01:57:13 +02:00
|
|
|
|
2026-07-31 10:13:52 +02:00
|
|
|
# Locate cargo instead of requiring it on the inherited PATH. Mirrors
|
|
|
|
|
# tools/repo.py:cargo_bin() — kept in sync by `make env-test`.
|
|
|
|
|
CARGO := $(firstword $(shell command -v cargo 2>/dev/null) \
|
|
|
|
|
$(wildcard $(HOME)/.cargo/bin/cargo) \
|
|
|
|
|
$(wildcard /usr/local/cargo/bin/cargo) \
|
|
|
|
|
cargo)
|
|
|
|
|
export PATH := $(dir $(CARGO)):$(PATH)
|
|
|
|
|
|
|
|
|
|
PY := python3
|
|
|
|
|
TOOLS := $(REPO)/tools
|
|
|
|
|
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
# `make` with no target lists what there is, rather than running the
|
|
|
|
|
# heaviest thing in the file. The Makefile's own header calls itself "one
|
|
|
|
|
# command surface" -- a surface you have to read the source of is not one.
|
|
|
|
|
.DEFAULT_GOAL := help
|
|
|
|
|
|
|
|
|
|
# A trial's name. Timestamped so two sessions on one day cannot overwrite
|
|
|
|
|
# each other's notes, and prefixed with the date because `tools/trials.py`
|
|
|
|
|
# reads the age from there (falling back to mtime).
|
|
|
|
|
TRIAL_NAME := $(shell date +%Y-%m-%d-%H%M)$(if $(SLUG),-$(SLUG),)
|
|
|
|
|
PLAYERS ?= 3
|
|
|
|
|
PORT ?= 0
|
|
|
|
|
|
2026-07-31 10:13:52 +02:00
|
|
|
# Every cargo recipe runs at the repo root; the shell does not persist cd.
|
|
|
|
|
IN_REPO := cd $(REPO) &&
|
|
|
|
|
|
CB-REV-0002: round 2, and the corrections were not approvable either
Three FATAL, five SERIOUS. The substance of round 1's corrections held —
Reactive is genuinely one arm different, the five replacement controls are
non-inert, the inert metric is right, the numbers reproduce. What failed
were the CLAIMS about them, and two defects the corrections introduced.
FATAL 1: the fix for round 1's #11 did not fix it. The assertion was
`games + setup_fails == 200`, and a refused setup increments setup_fails
while skipping games — so the sum is invariant under exactly the failure
it claimed to catch. Injecting setup failures gave exit 0 over 196-game
columns. Now asserts games == GAMES, verified to exit 101.
FATAL 2: the correction to the selective-column FATAL was itself
selective. "81-1000 per cell, baseline AND H1" and "31-1000" twelve lines
apart, both taken from the baseline row; under H1 rank-75 arms are
59/0/0/0. Every cell is now printed rather than summarised, and the
corrected verdict is the opposite of the one it replaced: under rank-75,
H1 REDUCES DARVO arms to zero at 3p and above.
FATAL 3: "DARVO arms 2 per seat per game" is 1 per seat per game, exactly,
at every band.
SERIOUS: the tiebreak oracle asserted only that the winner set CHANGED, so
reversing the tiebreak left it green; the #13 defect's impact was claimed
and never measured (72,000 games: zero divergences — real in principle,
witnessed only by a constructed board); a 29-of-363 citation pointed at a
file that did not contain it (round 1's reviewer did report it, and it was
never transcribed — the record was wrong, not the number); the harnesses
were run by NO GATE, so every published figure came from a manual run of
an ungated binary, including the assertion added for #1; and edition-check's
sibling handling — added by the last correction — was self-certifying,
crashed instead of failing, and counted Markdown lines as coverage. Now
discovered on disk, and it found a real gap on its first run: Rules_Text.csv
vendored with no digest.
Also: "peak Stress held" was dead code kept quiet by `let _ = held;` — the
numbers were right by coincidence.
make panels is now a registered gate. Round 3 is owed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 09:44:00 +02:00
|
|
|
.PHONY: help ground check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget shape-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time am6 am7 am8 edition-check replay-test loc play gate-review panels all \
|
CB-WP-0027 T01-T04: the commentary track
The meta view beside the table, and a note channel that provably cannot
carry a move.
T01 (ADR-0014). ADR-0007 D5 is SCOPED, NOT AMENDED, and the reason it was
easy is that PointerFact::parse already refuses any unrecognised field --
a comment could not reach the command path even by accident. So /command
carries pointer facts, /note carries text, and Note has no code path to
GroundCommand. Comments live in trials/<date>-<slug>.md, not in
ScenarioFile: a scenario is executed, replayed and hashed, and prose in it
is data the runner must ignore, which is how a format rots. The state hash
binds; round and step are for reading. And the retention question, decided
before any comment was written: RAW NOTES NEVER LEAVE clay-borg. A note
reaches ground-game only by being promoted to a register finding, by a
human, with the wording chosen then -- "the DARVO sequence is infuriating"
is useful signal and a bad way to open a message to the game's designer.
T02. CSS grid, minmax(0,1fr) on both tracks -- load-bearing, because a
grid child defaults to min-content width and without it the SVG table
refuses to shrink and pushes the meta column off-screen, looking correct
on the developer's monitor and broken everywhere else. Single-column
fallback under 64rem. The running tally moved into the panel so it is
visible WHILE PLAYING; it only appeared on the ending page before, and a
score you see once the game is over informs nothing.
T03. A plain <form method="post">, so the box works with the script
disabled; the command channel needs JavaScript because a drag is not a
form submission, a comment is one. 303 See Other so a reload does not
re-post. esc()'s first hostile input: <script>alert(1)</script> renders
escaped AND STILL READABLE -- escaping that eats the player's words is its
own defect. Verified over real HTTP: note posted 303, hostile note stored
as text, empty note refused 400, game did not advance.
T04. tools/trials.py and make trials. THE REPORT'S DESIGN CHANGED BECAUSE
I RAN IT: the first version called any note without a recording an orphan,
so a live session reported every note as broken -- the recording is only
written at game end. A metric that cries wolf is one nobody reads, which
is the exact failure this pass exists to prevent. Now ok / pending /
orphan, and only orphan is a target-0 number. The self-test exercises the
REPORTING path, not just the parser, because design-baseline.py had a
green self-test and an unexercised reporting path and that is where it
rotted.
And a latent Makefile defect surfaced: make trials did nothing, because
trials is also a directory and Make saw an up-to-date file. design,
difficulty and trials -- added by CB-WP-0022, CB-WP-0025 and this pass --
were ALL missing from .PHONY; only the one that collided revealed it.
make all: exit 0. 49 render tests, 26 cb-play, loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 10:37:46 +02:00
|
|
|
design difficulty trials
|
|
|
|
|
# `design`, `difficulty` and `trials` were added by CB-WP-0022, CB-WP-0025
|
|
|
|
|
# and CB-WP-0027 and none was declared here. Only `trials` revealed it, by
|
|
|
|
|
# colliding with the trials/ DIRECTORY -- Make saw an up-to-date file and
|
|
|
|
|
# ran nothing. The other two work by luck: no file happens to share their
|
|
|
|
|
# name. A target that is a command, not a file, belongs on this line.
|
2026-07-31 01:57:13 +02:00
|
|
|
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## list every target, with what it does
|
|
|
|
|
## `make help` is also what bare `make` runs.
|
|
|
|
|
help:
|
|
|
|
|
@echo "clay-borg — make <target> (bare \`make\` shows this)"
|
|
|
|
|
@echo
|
|
|
|
|
@awk '/^## /{ sub(/^## /,""); d[++n]=$$0; next } \
|
|
|
|
|
/^[a-zA-Z][a-zA-Z0-9_-]*:/{ \
|
|
|
|
|
if (n) { split($$1,a,":"); printf " %-16s %s\n", a[1], d[1]; \
|
|
|
|
|
for (i=2;i<=n;i++) printf " %-16s %s\n", "", d[i]; print "" } \
|
|
|
|
|
n=0; next } \
|
|
|
|
|
{ n=0 }' $(MAKEFILE_LIST)
|
|
|
|
|
@echo " Variables: PLAYERS=3 PORT=0 SLUG=<name> ARGS=\"...\""
|
|
|
|
|
@echo
|
|
|
|
|
@echo " Undocumented (instruments and gate internals):"
|
|
|
|
|
@awk '/^## /{ d=1; next } \
|
|
|
|
|
/^[a-zA-Z][a-zA-Z0-9_-]*:/{ if (!d) { split($$1,a,":"); print a[1] } d=0; next } \
|
|
|
|
|
{ d=0 }' $(MAKEFILE_LIST) | sort -u | tr "\n" " " | fold -s -w 66 | sed "s/^/ /"
|
|
|
|
|
@echo
|
|
|
|
|
|
|
|
|
|
## play GROUND in a browser, recording a trial (the usual way in)
|
|
|
|
|
## Opens a URL; game on the left, notes on the right. Everything you
|
|
|
|
|
## type in the notes panel is bound to the position you typed it at.
|
|
|
|
|
## `make ground PLAYERS=2 SLUG=darvo-confusion`
|
|
|
|
|
## Read the notes back afterwards with `make trials`.
|
|
|
|
|
ground:
|
|
|
|
|
@mkdir -p $(REPO)/trials
|
|
|
|
|
@echo " trial: trials/$(TRIAL_NAME).md (notes) + .yaml (the game)"
|
|
|
|
|
$(IN_REPO) $(CARGO) run -q -p cb-play -- \
|
|
|
|
|
--players $(PLAYERS) --serve $(PORT) \
|
|
|
|
|
--record trials/$(TRIAL_NAME).yaml \
|
|
|
|
|
--trial trials/$(TRIAL_NAME).md $(ARGS)
|
|
|
|
|
|
2026-07-31 01:57:13 +02:00
|
|
|
## fmt + clippy (deny warnings) + HashMap deny-lint
|
|
|
|
|
check:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(IN_REPO) $(CARGO) fmt --all --check
|
|
|
|
|
$(IN_REPO) $(CARGO) clippy --workspace --all-targets -- -D warnings
|
2026-07-31 01:57:13 +02:00
|
|
|
|
CB-WP-0008-T02: cb-play — INTENT stage 0's CLI player
A human seat is a Policy like any bot, so the CLI adds no second driver:
HumanPolicy renders the projection, lists the legal commands and reads an
index or `pass`. `make play` runs it; `--all-bots` watches one.
K13's Project trait gains its first implementor after six passes with
none. Hidden: other seats' face-down selections until Reveal, hands and
deck (counts only), a face-down Problem's suit and value, and the seed —
not secret content, but a seat holding it can compute the deck.
A played session becomes an artifact: --record writes it as a scenario
the runner executes, --replay writes a .cbreplay bundle. record.rs is the
inverse of parse_command and its warrant is a round-trip test over every
command shape.
The acceptance test for the projection passed vacuously twice. First it
asserted the text contained "face-down", which every render does because
of Problems. Counted, it then reported zero inspected entries: seats are
asked in order, so a human at P1 is prompted before anyone has selected.
Seated at P3 it inspects ten entries and dies when the projection is
mutated to reveal everything. Counting what the harness examined caught
both, which is the second time that remedy has worked where a stronger
predicate would not have.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:43:53 +02:00
|
|
|
## play GROUND from the terminal (INTENT stage 0's CLI player).
|
|
|
|
|
## `make play ARGS="--all-bots --bot random"` to watch one instead.
|
|
|
|
|
play:
|
|
|
|
|
$(IN_REPO) $(CARGO) run -q -p cb-play -- $(ARGS)
|
|
|
|
|
|
CB-WP-0009-T02: gates.toml and make gate-review
ADR-0006 D3. Nine standing control gates now say what they check, what
they have caught with pointers, when their keep-or-kill argument is due,
and what would retire them. make gate-review reports what is overdue and
what has caught nothing; it never fails the build, for CB-RES-0005 §4's
reason.
Drift is checked in both directions and both are pinned by self-tests: a
dependency of `make all` that is neither a registered control gate nor
listed in not_control_gates is a loop-lint finding, so a new gate cannot
acquire permanence without a review date, and an entry naming a target
the Makefile lacks is a finding too.
First run: 0 due, 2 silent. The silent two are the chaos roll, whose
12-declaration window exists precisely to find out, and gate-review
itself, which is not exempt from its own rule — if it has retired,
tightened or forced the re-justification of nothing by 2026-12-31 it is
a ritual and goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
|
|
|
## which control gates are due for a keep-or-kill argument (ADR-0006 D3)
|
|
|
|
|
gate-review:
|
|
|
|
|
$(PY) $(TOOLS)/gate-review.py
|
|
|
|
|
|
2026-07-31 01:57:13 +02:00
|
|
|
## unit + scenario-format tests
|
|
|
|
|
test:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(IN_REPO) $(CARGO) test --workspace
|
2026-07-31 01:57:13 +02:00
|
|
|
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## AM-4 dependency weight: third-party lines behind each budget
|
AM-4: gate scenario YAML, retarget on audited source, re-measure
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
|
|
|
dep-weight:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/dep-weight.py
|
AM-4: gate scenario YAML, retarget on audited source, re-measure
Adopts both remediations from CB-EV-0001 §4 (maintainer decision).
Option A — serde_yaml is now optional behind cb-game-runtime's
`scenarios` feature. The scenario module, the ScenarioGame impl and the
string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both
configurations compile and lint clean under -D warnings.
A trap worth recording: `default-features = false` on a *member*
dependency is silently ignored when the workspace dependency does not
specify it. The first attempt gated nothing while looking correct — the
build succeeded and cargo tree still showed all six YAML crates. Fixed
by setting it on the workspace dependency. This is the positive-control
failure mode in miniature: success was not evidence the change applied.
Retarget — AM-4 now measures third-party source under audit, split by
build configuration, replacing a crate count that was unreachable
without undoing K5/K7 and that does not compare across ecosystems.
Re-measured via the new `make dep-weight`, whose own positive control
refuses to report when any crate's source cannot be located:
shipped runtime 23 crates 246,250 lines target <=250,000 met
dev toolchain 29 crates 317,021 lines target <=350,000 met
own source 3,408 lines
Scenario tooling costs 70,771 lines a shipped game never compiles —
the split the single number was hiding.
Targets are set at current measurement plus headroom, so they bind on
future growth rather than retroactively passing what had failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
|
|
|
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## AM-1 rule coverage: which GROUND rules a scenario exercises
|
2026-07-31 02:23:38 +02:00
|
|
|
coverage:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/rule-coverage.py
|
2026-07-31 02:23:38 +02:00
|
|
|
|
CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had
no flag parsing at all, so --replay had nowhere to go.
The bundle is manifest + commands.log + initial.snapshot + expected.yaml,
dev-only behind the scenarios feature and charged to AM-4b. The command
stream goes through the K11 framing built in T05, so a truncated bundle is
detected rather than replayed short — the two tasks compose rather than
duplicating.
The reviewer's D2 correction was real: this was not "a directory of four
files". Pass carried only the end state, RunOutcome::Failed was a
formatted String, and scenario.rs created an EventLog, appended to it and
never read it. All three had to change.
The first round trip failed to reproduce, and the cause is worth keeping:
state_hash_hex over a serde_json::Value is a different canonical form than
over the typed aggregate — Value's map is key-sorted, a struct serializes
in declaration order. The bundle was written with one basis and verified
with the other. A round trip written to recompute its own comparison value
would have PASSED this bug; it failed because the recorded hash came from
the producing process, which is control 2's entire purpose.
make replay-test implements ADR-0005 §6's four controls, 14/14: a
committed deliberately-failing fixture outside the corpus with covers: []
so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash
must fail; a log short by one byte and a corrupted length prefix must be
rejected; and a mutated manifest seed must fail — which bites only because
replay re-derives the initial state from seed+setup and checks it against
the recorded snapshot, since restoring from the snapshot alone would leave
the seed inert. Plus a control on the controls: the bundle must still
replay after every mutation is reverted.
AM-7's hash-identical clause is re-earned. The probe records a hash per
per-game segment and replays each from its own genesis; folding from the
wrong seed now fails. That is the clause ADR-0005 §4 withdrew as
mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so
AM-7 stays PARTIAL — reported, not rounded up.
Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's
copy of that number going stale, on a number that moved the same hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 11:05:37 +02:00
|
|
|
# K10: a bundle must re-execute, and must be able to fail. Four controls
|
|
|
|
|
# from ADR-0005 §6, including truncate-by-one-byte and mutated-seed.
|
|
|
|
|
replay-test:
|
|
|
|
|
$(PY) $(TOOLS)/replay-test.py
|
|
|
|
|
|
CB-WP-0006 T04: withdraw AM-4c; and fix where AM-6 is measured
AM-4c is withdrawn from the acceptance table and retained as a reported
diagnostic. GameKernel §5a carries the argument.
The ratio has no monotone better direction. INTENT's rule is "own the
semantics, assimilate the implementation": rising can mean owning
semantics properly or reimplementing what should have been assimilated;
falling can mean leverage or dependency bloat. A target requires knowing
which way is better. It is also redundant — AM-4a/AM-4b bound the
denominator and AM-2 bounds own-source density, so AM-4c is a ratio of two
already-targeted quantities.
Measured at withdrawal: 1,426 own lines per 100k third-party (shipped),
1,107 (dev). make dep-weight now prints both, labelled diagnostic — the
row was never actually reported before.
M-D1-MUT keeps AM-4c in its denominator on purpose and says so in the
output. Dropping it would move the score 7/14 -> 7/13 without enforcing
anything: a score improved by deleting the question.
Decided before Phase B deliberately, since ADR-0005 predicts own-source
growth that will move this ratio; deciding after would be the retarget
§Step 4 forbids.
A T01 correction found here. The AM-6 gate failed inside `make all` at
38,753 ev/s against 341,280 in isolation — a 9x drop, because cargo test
runs binaries and threads concurrently. A throughput assertion inside a
parallel harness measures contention, not throughput. T01's measurement
was valid; its gate placement was not.
Fixed by running it only where valid — #[ignore] plus `make am6` in
release with --test-threads=1, now 2.0M ev/s at 20.2x headroom — and not
by lowering the target, which T01 forbade. My first attempt did drift that
way, adding a debug "sanity floor" of 50,000, and was backed out: a second
threshold is still a second chance to tune.
The mutation then went SURVIVED on the first run after the move. 4,000
black_box iterations were calibrated against debug's 3.4x headroom and are
invisible against release's 20x. Raised to 100,000; back to red. A weak
mutation is not a fixed property of a row — it can become weak when the
row's measurement conditions change.
Tier S (amends one row, creates no capability), chaos d4=2, no override.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:06:00 +02:00
|
|
|
# AM-6 gate. Runs in RELEASE, where headroom is ~24x; the same assertion
|
|
|
|
|
# in debug has ~3.4x and flaked under load. The target is unchanged — only
|
|
|
|
|
# where it is measured.
|
|
|
|
|
am6:
|
|
|
|
|
$(IN_REPO) $(CARGO) test --release -p games-ground --all-features \
|
|
|
|
|
am6_throughput -- --ignored --nocapture --test-threads=1
|
|
|
|
|
|
CB-WP-0021 T01/T02/T05: the engine plays its own data — AM-7 blocks
ADR-0011 decided it: vendor the CSV with a checked digest, read it with
a ~50-line reader, and let the hashes move.
The declaration's constraint was measured against the WRONG BUDGET. It
said a CSV crate costs 21,613 against AM-4a's 3,798 of headroom, '5.7x
over, settled by measurement'. But setup and problem_priorities are
cfg(scenarios) and are not in the shipped runtime at all, so AM-4a never
sees them. Against AM-4b, csv costs 17,651 against 19,742 -- it FITS,
with 2,091 to spare. It is refused anyway, on proportion: 89% of the
budget's remaining capacity to read 20 rows. The revisit condition is
stated (nested quoting, embedded newlines, multiple dialects).
GR-S01 now deals Surface + hidden 1..=k as ruled, with edition values and
suits. Measured: 6/9/12 available against thresholds 5/7/9 -- the game is
winnable at every seat count, which is what the maintainer could not do.
gd0001 is INVERTED, not deleted, and now also asserts the 6/9/12 so a
deal that is reachable for the wrong reason still fails.
Blast radius was scenario expectations, exactly as the ADR predicted: no
scenario pinned a hash and no bundle is committed. Six scenarios and two
unit tests updated, each with a note. gr-e01-threshold-unreachable-2p is
RENAMED to -reachable- and rewritten as the non-provisional import check
ground-game asked for by name. gr-e03's setup was restructured, not just
renumbered: with values 2,2,2 its personal-edge test would have tied
three ways and asserted nothing.
BLOCKING: AM-7 fails at median 0.845 against its 0.9 floor. Isolated
across three runs -- 3 problems + stand-in 0.97, 3 problems + edition
0.909, 4 problems + edition 0.845. State is BOUNDED (proven: identical
after 5k and 100k events), so this is not the unbounded-growth defect
AM-7 exists to catch; it is a bigger working set streaming a long log.
Whether AM-7's floor is still right for a larger aggregate is a spec
question and lowering it requires an ADR, so it is not being tuned here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:47:56 +02:00
|
|
|
# ADR-0011 D2: is the vendored edition still what ground-game published?
|
|
|
|
|
# Reports "upstream not checked out" as its own outcome, never a pass.
|
|
|
|
|
edition-check:
|
|
|
|
|
$(PY) $(TOOLS)/edition-check.py
|
|
|
|
|
|
CB-WP-0015: the two inert clauses, AM-7 scaling and AM-8 N=10
Provenance (tier S, one paragraph in lieu of survey and ADR): the two
clauses mutation-check has reported inert since CB-WP-0005. AM-7's
scaling ratio was held up by a test literally named
replay_100k_events_is_linear_and_fast that computed both throughputs,
printed both, and never divided one by the other. AM-8's N=10 was held
up by a runner that does two.
Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's
>=10-of-14 prediction MET for the first time. Neither was closed by
amending the question away, which was the live risk: the denominator is
unchanged and the four unenforced rows are the four already
unenforceable.
AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's,
correct for a floor on one number) gave 0.581-1.085 on an unchanged
binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at
fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU
contention while absolute throughput fell 4x. The INDETERMINATE guard
demanded unanimity and failed a good measurement over one sample
0.001 under the floor; it now requires a two-thirds majority. The
control that matters: AM-6's constant-cost mutation halves throughput
and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6.
AM-8 kept N=10 because the measurement said so. Perturbing the RNG only
from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A
late-onset divergence is deterministic, not flaky, so it is a control
rather than a coin flip. Ten runs live on one scenario (make am8, ~2s)
rather than all 25 (47s a build). GameKernel 5b records it.
The full run also found AM-4a's own mutation stale since ADR-0008 D3
moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported
HARNESS-BROKEN, no score published. The build-free half of that check
is now a --self-test assertion, so make all catches the next one.
mutation-check clauses may now carry their own verify and mutation, and
then the enforced flag is measured rather than declared; a declaration
disagreeing with its measurement is refused.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 14:07:08 +02:00
|
|
|
# AM-8 N=10 determinism gate. One scenario, ten same-seed replays, all
|
|
|
|
|
# compared to the first. Not all 25: `make sim` already runs K8's double-
|
|
|
|
|
# run over every scenario, and repeating that eight more times costs 47 s
|
|
|
|
|
# per build to re-answer a question the second run already answered. The
|
|
|
|
|
# extra runs exist for the probabilistic class, and one workload gives
|
|
|
|
|
# that class its ten samples — see scenario::run_n.
|
|
|
|
|
am8:
|
|
|
|
|
$(IN_REPO) $(CARGO) run -q -p cb-sim -- --runs 10 \
|
|
|
|
|
$(REPO)/scenarios/ground/gr-r06-round-resolve.yaml
|
|
|
|
|
|
|
|
|
|
# AM-7 scaling gate. Same release / single-thread reasoning as am6 and more
|
|
|
|
|
# so: a *ratio* of two timings taken under varying contention is worse than
|
|
|
|
|
# one reading, because the noise multiplies rather than cancels.
|
|
|
|
|
am7:
|
|
|
|
|
$(IN_REPO) $(CARGO) test --release -p games-ground --all-features \
|
CB-WP-0021 T06: fix AM-7's measurement, not its floor
The row folded a 5,000-event log against a 100,000-event log and compared
throughputs, which confounds 'does cost per event grow with history'
(the property it claims) with 'does streaming a 20x longer Vec cost more
per element' (a memory-hierarchy fact true of any program). It measured
the second and reported it as the first: importing the edition enlarged
the aggregate and the ratio fell to 0.845 with the state bounded.
Corrected to time the SAME 5,000 events on a state at depth 0 and on a
state at depth 100,000. Equal windows, equal event mix, so the only
difference left is history depth.
corrected: clean 1.004, mutated 0.589 (red)
old: clean 0.845 (red on healthy code), mutated 0.751
It also runs in 8.5s instead of timing out: the first version re-walked
the 100k prefix every repetition, 200M untimed folds per sample, which
under the mutation never finished. A control that cannot be run is not a
control. It now advances to depth once per sample and clones.
Two of my own measurements here were wrong and both were caught by
measuring again. A 2-minute timeout killed the shell line before its
restoring cp ran, so three readings were taken on MUTATED code -- I
diagnosed an event-mix confound that did not exist and 'fixed' it. The
fix is kept on its merits; the justification was fiction. And the probe
that proved state was bounded had checked four of eleven collections.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 01:10:38 +02:00
|
|
|
am7_cost_per_event -- --ignored --nocapture --test-threads=1
|
CB-WP-0015: the two inert clauses, AM-7 scaling and AM-8 N=10
Provenance (tier S, one paragraph in lieu of survey and ADR): the two
clauses mutation-check has reported inert since CB-WP-0005. AM-7's
scaling ratio was held up by a test literally named
replay_100k_events_is_linear_and_fast that computed both throughputs,
printed both, and never divided one by the other. AM-8's N=10 was held
up by a runner that does two.
Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's
>=10-of-14 prediction MET for the first time. Neither was closed by
amending the question away, which was the live risk: the denominator is
unchanged and the four unenforced rows are the four already
unenforceable.
AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's,
correct for a floor on one number) gave 0.581-1.085 on an unchanged
binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at
fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU
contention while absolute throughput fell 4x. The INDETERMINATE guard
demanded unanimity and failed a good measurement over one sample
0.001 under the floor; it now requires a two-thirds majority. The
control that matters: AM-6's constant-cost mutation halves throughput
and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6.
AM-8 kept N=10 because the measurement said so. Perturbing the RNG only
from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A
late-onset divergence is deterministic, not flaky, so it is a control
rather than a coin flip. Ten runs live on one scenario (make am8, ~2s)
rather than all 25 (47s a build). GameKernel 5b records it.
The full run also found AM-4a's own mutation stale since ADR-0008 D3
moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported
HARNESS-BROKEN, no score published. The build-free half of that check
is now a --self-test assertion, so make all catches the next one.
mutation-check clauses may now carry their own verify and mutation, and
then the enforced flag is measured rather than declared; a declaration
disagreeing with its measurement is refused.
make all exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 14:07:08 +02:00
|
|
|
|
CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches
AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x
headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind"
was right, but it is now measured rather than assumed, and verified red by
a property mutation (a 300 MB allocation in the workload).
AM-5 is BREACHED on both readings, on the machine the spec names:
dev toolchain (default features) 87.0 s [FAIL target <= 60 s]
shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s]
bnt-lap001, 8 cores — a direct comparison, not a directional one. A row
declared "recorded not gated" and never recorded fails its own target by
45% on first measurement.
The tool reports and exits 0 because the spec says the row is ungated.
Gating it is a spec change needing an ADR; a tool that promotes itself is
how a target starts binding without anyone deciding it should. So AM-5
stays unmutatable — for the accurate reason now — and the breach is raised
as a maintainer decision: speed the build, move the target by ADR (arguing
why 60 s was wrong rather than why 87 s is convenient), or withdraw the
row.
The measurement itself had a real bug, found only by cross-validation.
getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child,
so it attributed cargo's memory to the workload and reported 38.2 MB for a
run that used 12.3 MB — a 3x over-report that was plausible, passed its
target, and would have been published. Fixed with os.wait4, which returns
that specific child's rusage, and the self-test now cross-checks against
/usr/bin/time -v.
That is the false-accusation shape in the measurement layer rather than
the mutation layer: an instrument confidently reporting a number it had
not earned.
Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather
than running `cargo clean`, so measuring the metric does not cost several
minutes of rebuild afterwards. A metric that punishes its own measurement
gets measured once and never again.
M-D1-MUT: 6 -> 7 of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:59:32 +02:00
|
|
|
# AM-9 peak RSS (fast, gated). AM-5 needs a clean build — see build-time.
|
|
|
|
|
runtime-metrics:
|
|
|
|
|
$(PY) $(TOOLS)/runtime-metrics.py --fast
|
|
|
|
|
|
|
|
|
|
# AM-5: clean release build, ~90s, into a throwaway CARGO_TARGET_DIR so the
|
|
|
|
|
# working cache survives. Reported not gated per GameKernel §5.
|
|
|
|
|
build-time:
|
|
|
|
|
$(PY) $(TOOLS)/runtime-metrics.py
|
|
|
|
|
|
CB-WP-0006 T02: instrument AM-2; report AM-3 blocked, with the argument
Both rows were unmutatable for the same stated reason. They resolved
differently, and the difference is the point.
AM-2 is instrumented and enforced — tools/size-metrics.py, in `make all`:
AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom)
1,575 impl lines / 58 rules
Tests are excluded because AM-2 asks what a rule costs, not how much it is
exercised; lib.rs is ~18% test code and including it would have flattered
the number. This matters because AM-2 is AM-1's anti-gaming pair: 100%
rule coverage means nothing if the rules are trivially small, and AM-1 has
been reported met since CB-WP-0001 with its pair uninstrumented.
Verified red by a property mutation — ~800 lines of filler injected into
the impl, pushing the ratio past 40 — not a threshold tweak. The expect
string is the precise failure signature "FAIL target <= 40"; my first
attempt used "AM-2", which also matches passing output and would have
made the FA guard vacuous.
AM-3 is BLOCKED, not uninstrumented, and that is a finding rather than a
deferral. It measures LOC to express the CB-RES-0001 synthetic game on our
kernel, against a boardgame.io baseline of ~36 LOC for a declarative 3p
commit/reveal game object. That artifact has never been built: games/
contains only ground, and benches/synthetic.rs drives GROUND rather than
defining a synthetic game. Measuring GROUND's 1,575 impl lines against a
36-line synthetic game object would compare two different games and call
the difference a D1 result.
So the tool ships the measurement — a marker-delimited region, self-tested
— and reports the row blocked, naming the missing artifact. A number would
have been worse than a blank. It stays unmutatable and still counts
against M-D1-MUT per ADR-0005 §1: a row that cannot fail asserts nothing,
however good the reason.
M-D1-MUT: 5 -> 6 of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:38:15 +02:00
|
|
|
# AM-2 (M-D1-SPL, AM-1's anti-gaming pair) and AM-3.
|
|
|
|
|
size-metrics:
|
|
|
|
|
$(PY) $(TOOLS)/size-metrics.py
|
|
|
|
|
|
T04: tools/cb-cost.py — and its positive control fires on first contact
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
|
|
|
# M-D2-CST (specs/CostAccounting.md). cost-test is the positive control and
|
|
|
|
|
# runs first: a cost number from an unverified collector is void.
|
|
|
|
|
cost: cost-test
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/cb-cost.py --composition --by-task
|
T04: tools/cb-cost.py — and its positive control fires on first contact
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
|
|
|
|
|
|
|
|
cost-test:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/cb-cost.py --self-test
|
T04: tools/cb-cost.py — and its positive control fires on first contact
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
|
|
|
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
# InnerLoop rules that are mechanically checkable (CB-WP-0003 T01).
|
|
|
|
|
loop-lint:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/loop-lint.py
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
|
|
|
|
|
# Positive control for every reporting tool, per InnerLoop v1.1 Step 5.
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## positive control for every reporting tool
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
self-tests:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/cb-cost.py --self-test
|
|
|
|
|
$(PY) $(TOOLS)/loop-lint.py --self-test
|
|
|
|
|
$(PY) $(TOOLS)/rule-coverage.py --self-test
|
|
|
|
|
$(PY) $(TOOLS)/dep-weight.py --self-test
|
|
|
|
|
$(PY) $(TOOLS)/repo.py --self-test
|
CB-WP-0004 T02: make task-done — close a task on measured numbers
Replaces the three hand-done steps of a task close (46 turns, $11.52 per
CB-RES-0003): the heredoc flipping status in the workplan file, the
hand-written hub call, and the hand-typed token counts.
The third is the reason this task exists. Every update_task_status this
repo produced carried estimated tokens_in/tokens_out — in a project whose
central finding is that estimated token counts are worthless. task-done
reads the measured figure from the transcripts, or refuses; there is no
path through it that emits an estimate.
cb-cost gains by_task_detail: cost, response count, model histogram and
token components per task. task-done imports cb-cost rather than parsing
its printed table, so the hub figure is not a copy that can drift from
its source.
The positive control found a real defect before the tool ran once.
Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002
T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test
reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x
overstatement would have been pushed to the hub as a *measured* number —
the same fiction in a new form. task_label() now keys qualified subjects
on the full id and leaves unqualified ones bare rather than
retro-assigning them to a workplan. The pinned $93.15 benchmark is
unchanged, so historical attribution was not disturbed.
Fourth instance of trusted arithmetic: a number believed because a
program produced it rather than a hand.
Refusals, all exercised by --self-test: unknown id, typo'd id,
already-done task, missing state_hub_task_id, no measured spend, and a
status flip that produced no change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:17:51 +02:00
|
|
|
$(PY) $(TOOLS)/task-done.py --self-test
|
2026-07-31 10:20:27 +02:00
|
|
|
$(PY) $(TOOLS)/status.py --self-test
|
2026-07-31 10:24:39 +02:00
|
|
|
$(PY) $(TOOLS)/facts.py --self-test
|
CB-WP-0005 T02: M-D1-MUT — 4 of 14 acceptance rows are enforced
For each acceptance row in GameKernel §5, invert the property and require
the verifying command to go red. adapted:mutation-testing, with the
denominator changed from source lines to acceptance rows.
M-D1-MUT: 4/14 rows enforced
PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert)
unmutatable 8 (no property to invert, reason stated per row)
SURVIVED 0
Two corrections to our own numbers. There are 14 rows, not the twelve
ADR-0005 and CB-WP-0005 both asserted — AM-4 splits into a/b/c. And the
prediction of 9-of-12 (75%) becomes >=10 of 14; measured 4 (29%), badly
unmet. No target moved in this commit.
The second correction matters more. My first run reported two SURVIVED
rows and both were my own no-op mutations: `pub struct NullRng;` ->
`pub struct NullRng {}` is semantically identical, and renaming
max_age_days does nothing because CA-17 reads it with a default of 90.
Both would have been published as "this row asserts nothing" — a false
accusation against code that is fine. Replaced with real inversions (a
per-construction counter in the ChaCha seed; reverting AC-9's output
resolution to the first-wins bug it was fixed for), after which both go
red. T08 asks whether writing a weak mutation is the new grep. It is,
demonstrably, on the first attempt.
The finding is larger than the workplan assumed. 8 of 14 rows are
unmutatable — AM-2, AM-3, AM-4c, AM-5, AM-6, AM-9, AM-10, AM-11 have no
instrument at all. AM-6 is the sharpest: nothing in the workspace
compares any number to 100,000 events/s, the headline throughput claim.
The problem is not three unimplemented rules, it is that more than half
the acceptance table has nothing behind it.
Harness controls: a stale find-string reports HARNESS-BROKEN rather than
scoring the baseline as the mutant; a red baseline reports inconclusive
rather than red; the tree is restored in a finally and the restoration is
verified. Not in `make all` — it rebuilds the workspace once per row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:27:23 +02:00
|
|
|
$(PY) $(TOOLS)/mutation-check.py --self-test
|
CB-WP-0006 T02: instrument AM-2; report AM-3 blocked, with the argument
Both rows were unmutatable for the same stated reason. They resolved
differently, and the difference is the point.
AM-2 is instrumented and enforced — tools/size-metrics.py, in `make all`:
AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom)
1,575 impl lines / 58 rules
Tests are excluded because AM-2 asks what a rule costs, not how much it is
exercised; lib.rs is ~18% test code and including it would have flattered
the number. This matters because AM-2 is AM-1's anti-gaming pair: 100%
rule coverage means nothing if the rules are trivially small, and AM-1 has
been reported met since CB-WP-0001 with its pair uninstrumented.
Verified red by a property mutation — ~800 lines of filler injected into
the impl, pushing the ratio past 40 — not a threshold tweak. The expect
string is the precise failure signature "FAIL target <= 40"; my first
attempt used "AM-2", which also matches passing output and would have
made the FA guard vacuous.
AM-3 is BLOCKED, not uninstrumented, and that is a finding rather than a
deferral. It measures LOC to express the CB-RES-0001 synthetic game on our
kernel, against a boardgame.io baseline of ~36 LOC for a declarative 3p
commit/reveal game object. That artifact has never been built: games/
contains only ground, and benches/synthetic.rs drives GROUND rather than
defining a synthetic game. Measuring GROUND's 1,575 impl lines against a
36-line synthetic game object would compare two different games and call
the difference a D1 result.
So the tool ships the measurement — a marker-delimited region, self-tested
— and reports the row blocked, naming the missing artifact. A number would
have been worse than a blank. It stays unmutatable and still counts
against M-D1-MUT per ADR-0005 §1: a row that cannot fail asserts nothing,
however good the reason.
M-D1-MUT: 5 -> 6 of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:38:15 +02:00
|
|
|
$(PY) $(TOOLS)/size-metrics.py --self-test
|
CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches
AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x
headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind"
was right, but it is now measured rather than assumed, and verified red by
a property mutation (a 300 MB allocation in the workload).
AM-5 is BREACHED on both readings, on the machine the spec names:
dev toolchain (default features) 87.0 s [FAIL target <= 60 s]
shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s]
bnt-lap001, 8 cores — a direct comparison, not a directional one. A row
declared "recorded not gated" and never recorded fails its own target by
45% on first measurement.
The tool reports and exits 0 because the spec says the row is ungated.
Gating it is a spec change needing an ADR; a tool that promotes itself is
how a target starts binding without anyone deciding it should. So AM-5
stays unmutatable — for the accurate reason now — and the breach is raised
as a maintainer decision: speed the build, move the target by ADR (arguing
why 60 s was wrong rather than why 87 s is convenient), or withdraw the
row.
The measurement itself had a real bug, found only by cross-validation.
getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child,
so it attributed cargo's memory to the workload and reported 38.2 MB for a
run that used 12.3 MB — a 3x over-report that was plausible, passed its
target, and would have been published. Fixed with os.wait4, which returns
that specific child's rusage, and the self-test now cross-checks against
/usr/bin/time -v.
That is the false-accusation shape in the measurement layer rather than
the mutation layer: an instrument confidently reporting a number it had
not earned.
Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather
than running `cargo clean`, so measuring the metric does not cost several
minutes of rebuild afterwards. A metric that punishes its own measurement
gets measured once and never again.
M-D1-MUT: 6 -> 7 of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:59:32 +02:00
|
|
|
$(PY) $(TOOLS)/runtime-metrics.py --self-test
|
CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had
no flag parsing at all, so --replay had nowhere to go.
The bundle is manifest + commands.log + initial.snapshot + expected.yaml,
dev-only behind the scenarios feature and charged to AM-4b. The command
stream goes through the K11 framing built in T05, so a truncated bundle is
detected rather than replayed short — the two tasks compose rather than
duplicating.
The reviewer's D2 correction was real: this was not "a directory of four
files". Pass carried only the end state, RunOutcome::Failed was a
formatted String, and scenario.rs created an EventLog, appended to it and
never read it. All three had to change.
The first round trip failed to reproduce, and the cause is worth keeping:
state_hash_hex over a serde_json::Value is a different canonical form than
over the typed aggregate — Value's map is key-sorted, a struct serializes
in declaration order. The bundle was written with one basis and verified
with the other. A round trip written to recompute its own comparison value
would have PASSED this bug; it failed because the recorded hash came from
the producing process, which is control 2's entire purpose.
make replay-test implements ADR-0005 §6's four controls, 14/14: a
committed deliberately-failing fixture outside the corpus with covers: []
so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash
must fail; a log short by one byte and a corrupted length prefix must be
rejected; and a mutated manifest seed must fail — which bites only because
replay re-derives the initial state from seed+setup and checks it against
the recorded snapshot, since restoring from the snapshot alone would leave
the seed inert. Plus a control on the controls: the bundle must still
replay after every mutation is reverted.
AM-7's hash-identical clause is re-earned. The probe records a hash per
per-game segment and replays each from its own genesis; folding from the
wrong seed now fails. That is the clause ADR-0005 §4 withdrew as
mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so
AM-7 stays PARTIAL — reported, not rounded up.
Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's
copy of that number going stale, on a number that moved the same hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 11:05:37 +02:00
|
|
|
$(PY) $(TOOLS)/replay-test.py --self-test
|
CB-WP-0022 T05/T06/T07: the register, and what its first run found
T05. tools/design.py, make design, and the register in GroundRules.md --
14 rows, no new file, because ADR-0012 D2 made §Underdetermined the
register rather than building one beside it.
Backfill was the test and it caught two things the ADR did not have.
First, a `role` column. The first report alarmed on U2 and was wrong to:
U2's scenario is green BECAUSE the provisional default it documents is
implemented, which says nothing about whether ground-game agrees. GR-E01's
was a counterexample that went green. Same colour, opposite meaning -- a
register that cannot tell them apart either alarms constantly or never.
Only a green counterexample alarms. Folded back into GameDesign §1.3.
Second, it contradicted the survey. CB-RES-0007 said "six of the ten
already have provisional scenarios." Measured -- grep -lE "\bU<n>\b" over
scenarios/ground -- exactly ONE U-item names itself. Five provisional
scenarios exist and four probably encode U-item defaults, but the mapping
is not written down, so it is not checkable. Same defect class as the
wrong premises, found inside the survey that proposed the fix. Now a
reported debt: open, lacking a reproduction: 9, target 0.
design.py carries the control design-baseline.py never had, asserted
directly: a row citing a nonexistent file must not count as reproduced,
using the exact path 2da19a4 deleted -- which the old tool called green.
design-baseline.py is marked superseded rather than deleted; it is the
evidence for how a wrong number got into a survey.
T06. The report is a FILE in ground-game under GROUND-WP-0002, committed
there, with a hub message that only points at it. It asks for no ruling:
it carries GR-E01's withdrawal, our reproduction debt, and two notes that
are explicitly not findings.
And it had to acknowledge something nobody anticipated. GROUND-WP-0002 is
finished -- all ten U-items were RULED 2026-08-03, every one confirmed as
the default we simulate, plus five of six provisional scenarios. The
survey said "0 of 10 ruled" two days later and this register was built
saying `reported`. That is the unread-inbox failure running in the
opposite direction: they answered and we did not collect it. The
instrument's first run surfaced it. They are `ruled`, not `applied` --
lifting the now-settled provisional flags is owed and is not done, and
make design shows them open until it is.
T07. evidence/CB-EV-0021. Two of six catches in this pass came from
execution rather than process (the role distinction from building it, the
ten uncollected rulings from running it), which is InnerLoop §Design
goal's prediction holding.
make self-tests, facts-check, loop-lint: clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 15:22:33 +02:00
|
|
|
$(PY) $(TOOLS)/design.py --self-test
|
CB-WP-0027 T01-T04: the commentary track
The meta view beside the table, and a note channel that provably cannot
carry a move.
T01 (ADR-0014). ADR-0007 D5 is SCOPED, NOT AMENDED, and the reason it was
easy is that PointerFact::parse already refuses any unrecognised field --
a comment could not reach the command path even by accident. So /command
carries pointer facts, /note carries text, and Note has no code path to
GroundCommand. Comments live in trials/<date>-<slug>.md, not in
ScenarioFile: a scenario is executed, replayed and hashed, and prose in it
is data the runner must ignore, which is how a format rots. The state hash
binds; round and step are for reading. And the retention question, decided
before any comment was written: RAW NOTES NEVER LEAVE clay-borg. A note
reaches ground-game only by being promoted to a register finding, by a
human, with the wording chosen then -- "the DARVO sequence is infuriating"
is useful signal and a bad way to open a message to the game's designer.
T02. CSS grid, minmax(0,1fr) on both tracks -- load-bearing, because a
grid child defaults to min-content width and without it the SVG table
refuses to shrink and pushes the meta column off-screen, looking correct
on the developer's monitor and broken everywhere else. Single-column
fallback under 64rem. The running tally moved into the panel so it is
visible WHILE PLAYING; it only appeared on the ending page before, and a
score you see once the game is over informs nothing.
T03. A plain <form method="post">, so the box works with the script
disabled; the command channel needs JavaScript because a drag is not a
form submission, a comment is one. 303 See Other so a reload does not
re-post. esc()'s first hostile input: <script>alert(1)</script> renders
escaped AND STILL READABLE -- escaping that eats the player's words is its
own defect. Verified over real HTTP: note posted 303, hostile note stored
as text, empty note refused 400, game did not advance.
T04. tools/trials.py and make trials. THE REPORT'S DESIGN CHANGED BECAUSE
I RAN IT: the first version called any note without a recording an orphan,
so a live session reported every note as broken -- the recording is only
written at game end. A metric that cries wolf is one nobody reads, which
is the exact failure this pass exists to prevent. Now ok / pending /
orphan, and only orphan is a target-0 number. The self-test exercises the
REPORTING path, not just the parser, because design-baseline.py had a
green self-test and an unexercised reporting path and that is where it
rotted.
And a latent Makefile defect surfaced: make trials did nothing, because
trials is also a directory and Make saw an up-to-date file. design,
difficulty and trials -- added by CB-WP-0022, CB-WP-0025 and this pass --
were ALL missing from .PHONY; only the one that collided revealed it.
make all: exit 0. 49 render tests, 26 cb-play, loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 10:37:46 +02:00
|
|
|
$(PY) $(TOOLS)/trials.py --self-test
|
CB-WP-0025 T06/T07: a difficulty baseline that reports its own confound
T06. games/ground/examples/difficulty.rs, make difficulty, wired into make
self-tests, and a report file in ground-game under GROUND-WP-0005 with a
hub message pointing at it.
THE REPORT OPENS WITH THE RETRACTION, because what this task was written
to send was withdrawn by T02 and GROUND-WP-0005 is blocked on exactly that
number. They are told, in the first section, that we nearly sent them "the
game is too easy at 5-6 seats" and why it was wrong.
seats winnable greedy random first-legal spread
2p 60% 60.0% 5.0% 76.7% 71.7
3p 93% 88.3% 6.7% 25.0% 81.7
4p 100% 93.3% 6.7% 30.0% 86.7
5p 100% 100.0% 3.3% 0.0% 100.0
6p 100% 100.0% 3.3% 0.0% 100.0
SPREAD justifies the whole redesign: 71.7 to 100.0 points between three
trivial policies. The table now shows why no single rate is a difficulty
rather than asserting it.
And the 5-6 rows point the OPPOSITE way from the withdrawn claim --
first-legal 0% against greedy 100% is the widest spread in the table,
which suggests play matters MORE there, not less. Neither reading is
established and the report says so.
The confound is stated in the tool's own output, not only in prose:
`winnable` is conditioned on greedy's play up to the final round, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would repeat this pass's error in a subtler form -- which is exactly
how a corrected project reintroduces a defect. NO THRESHOLD CHANGES ARE
PROPOSED.
The instrument can fail (spec §5): a witness must replay to a win, an
unwinnable position must report searched-out rather than a budget cut, a
one-node budget must not claim exhaustion, and the policy panel must
actually disagree. difficulty-baseline.rs marked superseded, kept as the
survey's dated snapshot. Registered as F16, inconsistent / withdrawn.
T07. evidence/CB-EV-0024. Five of nine defects came only from the review;
four from execution, and all four of those were in work written after it.
The wrong-denominator family now has five instances and still no control
-- facts-check catches copies that disagree, nothing catches a number
computed correctly against the wrong base. Tier L was an over-declaration
(no port, structurally M) and paid for itself anyway, because the review
is L-only. Chaos window 2 will close with zero overrides, making its
retirement condition untestable.
Named as open rather than implied done: the witness is NOT wired to the
ending page. The search works; the browser cannot ask it yet.
make all: exit 0. loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 19:24:49 +02:00
|
|
|
cargo run --release -q -p games-ground --example difficulty -- --self-test
|
CB-WP-0021 T01/T02/T05: the engine plays its own data — AM-7 blocks
ADR-0011 decided it: vendor the CSV with a checked digest, read it with
a ~50-line reader, and let the hashes move.
The declaration's constraint was measured against the WRONG BUDGET. It
said a CSV crate costs 21,613 against AM-4a's 3,798 of headroom, '5.7x
over, settled by measurement'. But setup and problem_priorities are
cfg(scenarios) and are not in the shipped runtime at all, so AM-4a never
sees them. Against AM-4b, csv costs 17,651 against 19,742 -- it FITS,
with 2,091 to spare. It is refused anyway, on proportion: 89% of the
budget's remaining capacity to read 20 rows. The revisit condition is
stated (nested quoting, embedded newlines, multiple dialects).
GR-S01 now deals Surface + hidden 1..=k as ruled, with edition values and
suits. Measured: 6/9/12 available against thresholds 5/7/9 -- the game is
winnable at every seat count, which is what the maintainer could not do.
gd0001 is INVERTED, not deleted, and now also asserts the 6/9/12 so a
deal that is reachable for the wrong reason still fails.
Blast radius was scenario expectations, exactly as the ADR predicted: no
scenario pinned a hash and no bundle is committed. Six scenarios and two
unit tests updated, each with a note. gr-e01-threshold-unreachable-2p is
RENAMED to -reachable- and rewritten as the non-provisional import check
ground-game asked for by name. gr-e03's setup was restructured, not just
renumbered: with values 2,2,2 its personal-edge test would have tied
three ways and asserted nothing.
BLOCKING: AM-7 fails at median 0.845 against its 0.9 floor. Isolated
across three runs -- 3 problems + stand-in 0.97, 3 problems + edition
0.909, 4 problems + edition 0.845. State is BOUNDED (proven: identical
after 5k and 100k events), so this is not the unbounded-growth defect
AM-7 exists to catch; it is a bigger working set streaming a long log.
Whether AM-7's floor is still right for a larger aggregate is a spec
question and lowering it requires an ADR, so it is not being tuned here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:47:56 +02:00
|
|
|
$(PY) $(TOOLS)/edition-check.py --self-test
|
2026-07-31 10:13:52 +02:00
|
|
|
|
|
|
|
|
# T01 positive control: prove the environment fix, do not assume it. Runs
|
|
|
|
|
# every tool from a foreign working directory with a PATH that has no
|
|
|
|
|
# cargo on it. Before T01 this failed; if it fails again, the friction is
|
|
|
|
|
# back and CB-EV-0003's measurement is invalid.
|
|
|
|
|
env-test:
|
|
|
|
|
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/repo.py --self-test
|
|
|
|
|
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/rule-coverage.py --self-test >/dev/null \
|
|
|
|
|
&& echo " [ok ] rule-coverage runs from / with no cargo on PATH"
|
|
|
|
|
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/dep-weight.py --self-test >/dev/null \
|
|
|
|
|
&& echo " [ok ] dep-weight runs from / with no cargo on PATH"
|
|
|
|
|
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/cb-cost.py --self-test >/dev/null \
|
|
|
|
|
&& echo " [ok ] cb-cost runs from / with no cargo on PATH"
|
|
|
|
|
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/loop-lint.py --self-test >/dev/null \
|
|
|
|
|
&& echo " [ok ] loop-lint runs from / with no cargo on PATH"
|
|
|
|
|
@$(MAKE) -C $(REPO) coverage >/dev/null \
|
|
|
|
|
&& echo " [ok ] make -C <repo> works from any directory"
|
T01: audit every InnerLoop rule, and make the checkable ones executable
41 rules classified executable / checkable / decorative, each tagged with
the failure class it catches. Counts: 11 executable, 22 checkable, 4
decorative (one of them dead policy).
Audit: history/260731-inner-loop-rule-audit.md
New tools/loop-lint.py makes 7 rules executable (tier declared, chaos
roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file
loadability, reporting tools expose --self-test). It found three real
violations on its first run, none previously visible:
- specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit
the loop has stated since v0.2 and never measured. Split at its own
section boundaries into Blueprint (1-8) + Runtime (9-15).
- tools/dep-weight.py and tools/rule-coverage.py had positive-control
logic and no --self-test, so nothing verified the control worked.
Adding rule-coverage's self-test exposed a latent instance of the exact
class this workplan is about: if the spec regex stopped matching, rules
was empty, missing was empty, and the tool exited 0 reporting "0/0" --
a silent pass, in the tool that reports our headline AM-1 number. Both
tools now assert they found something before reporting.
Two demotions applied in the spec rather than left implicit: "structured
over prose" is marked guidance (nothing can check it), and the 8k/10k
token budget is struck through and marked DEAD POLICY pointing at T05.
The audit's uncomfortable finding: rule 13 (re-derive inherited numbers)
has no mechanical form, is deliberately left decorative, and caught the
LARGEST error in CB-WP-0002. That is a counter-example to this
workplan's own hypothesis. "A rule that cannot be executed is not a
rule" is wrong as stated; the defensible version is that such a rule
cannot be relied on to fire, so it must not be the only defence for a
class that matters.
Class coverage: harness-does-nothing has five executable rules;
trusted-arithmetic has ZERO and produced the largest single error.
make loop-lint and make self-tests wired into `make all` and CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
|
|
|
|
CB-WP-0005 T02: M-D1-MUT — 4 of 14 acceptance rows are enforced
For each acceptance row in GameKernel §5, invert the property and require
the verifying command to go red. adapted:mutation-testing, with the
denominator changed from source lines to acceptance rows.
M-D1-MUT: 4/14 rows enforced
PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert)
unmutatable 8 (no property to invert, reason stated per row)
SURVIVED 0
Two corrections to our own numbers. There are 14 rows, not the twelve
ADR-0005 and CB-WP-0005 both asserted — AM-4 splits into a/b/c. And the
prediction of 9-of-12 (75%) becomes >=10 of 14; measured 4 (29%), badly
unmet. No target moved in this commit.
The second correction matters more. My first run reported two SURVIVED
rows and both were my own no-op mutations: `pub struct NullRng;` ->
`pub struct NullRng {}` is semantically identical, and renaming
max_age_days does nothing because CA-17 reads it with a default of 90.
Both would have been published as "this row asserts nothing" — a false
accusation against code that is fine. Replaced with real inversions (a
per-construction counter in the ChaCha seed; reverting AC-9's output
resolution to the first-wins bug it was fixed for), after which both go
red. T08 asks whether writing a weak mutation is the new grep. It is,
demonstrably, on the first attempt.
The finding is larger than the workplan assumed. 8 of 14 rows are
unmutatable — AM-2, AM-3, AM-4c, AM-5, AM-6, AM-9, AM-10, AM-11 have no
instrument at all. AM-6 is the sharpest: nothing in the workspace
compares any number to 100,000 events/s, the headline throughput claim.
The problem is not three unimplemented rules, it is that more than half
the acceptance table has nothing behind it.
Harness controls: a stale find-string reports HARNESS-BROKEN rather than
scoring the baseline as the mutant; a red baseline reports inconclusive
rather than red; the tree is restored in a finally and the restoration is
verified. Not in `make all` — it rebuilds the workspace once per row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:27:23 +02:00
|
|
|
# M-D1-MUT (CB-WP-0005 T02): invert each acceptance row's property and
|
|
|
|
|
# require the verifying command to go red. Deliberately NOT in `make all`:
|
|
|
|
|
# it rebuilds the workspace once per mutated row. Run it on demand and in
|
|
|
|
|
# CI, not in the inner loop.
|
|
|
|
|
mutation-check:
|
|
|
|
|
$(PY) $(TOOLS)/mutation-check.py $(ARGS)
|
|
|
|
|
|
2026-07-31 10:24:39 +02:00
|
|
|
# T04: single source of fact (InnerLoop v1.2) — the DFD gate.
|
|
|
|
|
# facts.toml is GENERATED; facts-check fails if it disagrees with the
|
|
|
|
|
# instruments, or if a tagged artifact disagrees with it.
|
|
|
|
|
facts-check:
|
|
|
|
|
$(PY) $(TOOLS)/facts.py --check
|
|
|
|
|
|
|
|
|
|
facts-gen:
|
|
|
|
|
$(PY) $(TOOLS)/facts.py --gen
|
|
|
|
|
|
CB-WP-0027 T01-T04: the commentary track
The meta view beside the table, and a note channel that provably cannot
carry a move.
T01 (ADR-0014). ADR-0007 D5 is SCOPED, NOT AMENDED, and the reason it was
easy is that PointerFact::parse already refuses any unrecognised field --
a comment could not reach the command path even by accident. So /command
carries pointer facts, /note carries text, and Note has no code path to
GroundCommand. Comments live in trials/<date>-<slug>.md, not in
ScenarioFile: a scenario is executed, replayed and hashed, and prose in it
is data the runner must ignore, which is how a format rots. The state hash
binds; round and step are for reading. And the retention question, decided
before any comment was written: RAW NOTES NEVER LEAVE clay-borg. A note
reaches ground-game only by being promoted to a register finding, by a
human, with the wording chosen then -- "the DARVO sequence is infuriating"
is useful signal and a bad way to open a message to the game's designer.
T02. CSS grid, minmax(0,1fr) on both tracks -- load-bearing, because a
grid child defaults to min-content width and without it the SVG table
refuses to shrink and pushes the meta column off-screen, looking correct
on the developer's monitor and broken everywhere else. Single-column
fallback under 64rem. The running tally moved into the panel so it is
visible WHILE PLAYING; it only appeared on the ending page before, and a
score you see once the game is over informs nothing.
T03. A plain <form method="post">, so the box works with the script
disabled; the command channel needs JavaScript because a drag is not a
form submission, a comment is one. 303 See Other so a reload does not
re-post. esc()'s first hostile input: <script>alert(1)</script> renders
escaped AND STILL READABLE -- escaping that eats the player's words is its
own defect. Verified over real HTTP: note posted 303, hostile note stored
as text, empty note refused 400, game did not advance.
T04. tools/trials.py and make trials. THE REPORT'S DESIGN CHANGED BECAUSE
I RAN IT: the first version called any note without a recording an orphan,
so a live session reported every note as broken -- the recording is only
written at game end. A metric that cries wolf is one nobody reads, which
is the exact failure this pass exists to prevent. Now ok / pending /
orphan, and only orphan is a target-0 number. The self-test exercises the
REPORTING path, not just the parser, because design-baseline.py had a
green self-test and an unexercised reporting path and that is where it
rotted.
And a latent Makefile defect surfaced: make trials did nothing, because
trials is also a directory and Make saw an up-to-date file. design,
difficulty and trials -- added by CB-WP-0022, CB-WP-0025 and this pass --
were ALL missing from .PHONY; only the one that collided revealed it.
make all: exit 0. 49 render tests, 26 cb-play, loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 10:37:46 +02:00
|
|
|
# CB-WP-0027 T04: what the players said, and where. Surfacing is the
|
|
|
|
|
# deliverable -- a commentary feature nobody can read is this project's
|
|
|
|
|
# signature failure in a new medium (ADR-0014 D6).
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## what the players said while playing, and where
|
CB-WP-0027 T01-T04: the commentary track
The meta view beside the table, and a note channel that provably cannot
carry a move.
T01 (ADR-0014). ADR-0007 D5 is SCOPED, NOT AMENDED, and the reason it was
easy is that PointerFact::parse already refuses any unrecognised field --
a comment could not reach the command path even by accident. So /command
carries pointer facts, /note carries text, and Note has no code path to
GroundCommand. Comments live in trials/<date>-<slug>.md, not in
ScenarioFile: a scenario is executed, replayed and hashed, and prose in it
is data the runner must ignore, which is how a format rots. The state hash
binds; round and step are for reading. And the retention question, decided
before any comment was written: RAW NOTES NEVER LEAVE clay-borg. A note
reaches ground-game only by being promoted to a register finding, by a
human, with the wording chosen then -- "the DARVO sequence is infuriating"
is useful signal and a bad way to open a message to the game's designer.
T02. CSS grid, minmax(0,1fr) on both tracks -- load-bearing, because a
grid child defaults to min-content width and without it the SVG table
refuses to shrink and pushes the meta column off-screen, looking correct
on the developer's monitor and broken everywhere else. Single-column
fallback under 64rem. The running tally moved into the panel so it is
visible WHILE PLAYING; it only appeared on the ending page before, and a
score you see once the game is over informs nothing.
T03. A plain <form method="post">, so the box works with the script
disabled; the command channel needs JavaScript because a drag is not a
form submission, a comment is one. 303 See Other so a reload does not
re-post. esc()'s first hostile input: <script>alert(1)</script> renders
escaped AND STILL READABLE -- escaping that eats the player's words is its
own defect. Verified over real HTTP: note posted 303, hostile note stored
as text, empty note refused 400, game did not advance.
T04. tools/trials.py and make trials. THE REPORT'S DESIGN CHANGED BECAUSE
I RAN IT: the first version called any note without a recording an orphan,
so a live session reported every note as broken -- the recording is only
written at game end. A metric that cries wolf is one nobody reads, which
is the exact failure this pass exists to prevent. Now ok / pending /
orphan, and only orphan is a target-0 number. The self-test exercises the
REPORTING path, not just the parser, because design-baseline.py had a
green self-test and an unexercised reporting path and that is where it
rotted.
And a latent Makefile defect surfaced: make trials did nothing, because
trials is also a directory and Make saw an up-to-date file. design,
difficulty and trials -- added by CB-WP-0022, CB-WP-0025 and this pass --
were ALL missing from .PHONY; only the one that collided revealed it.
make all: exit 0. 49 render tests, 26 cb-play, loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 10:37:46 +02:00
|
|
|
trials:
|
|
|
|
|
@$(PY) $(TOOLS)/trials.py
|
|
|
|
|
|
CB-WP-0025 T06/T07: a difficulty baseline that reports its own confound
T06. games/ground/examples/difficulty.rs, make difficulty, wired into make
self-tests, and a report file in ground-game under GROUND-WP-0005 with a
hub message pointing at it.
THE REPORT OPENS WITH THE RETRACTION, because what this task was written
to send was withdrawn by T02 and GROUND-WP-0005 is blocked on exactly that
number. They are told, in the first section, that we nearly sent them "the
game is too easy at 5-6 seats" and why it was wrong.
seats winnable greedy random first-legal spread
2p 60% 60.0% 5.0% 76.7% 71.7
3p 93% 88.3% 6.7% 25.0% 81.7
4p 100% 93.3% 6.7% 30.0% 86.7
5p 100% 100.0% 3.3% 0.0% 100.0
6p 100% 100.0% 3.3% 0.0% 100.0
SPREAD justifies the whole redesign: 71.7 to 100.0 points between three
trivial policies. The table now shows why no single rate is a difficulty
rather than asserting it.
And the 5-6 rows point the OPPOSITE way from the withdrawn claim --
first-legal 0% against greedy 100% is the widest spread in the table,
which suggests play matters MORE there, not less. Neither reading is
established and the report says so.
The confound is stated in the tool's own output, not only in prose:
`winnable` is conditioned on greedy's play up to the final round, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would repeat this pass's error in a subtler form -- which is exactly
how a corrected project reintroduces a defect. NO THRESHOLD CHANGES ARE
PROPOSED.
The instrument can fail (spec §5): a witness must replay to a win, an
unwinnable position must report searched-out rather than a budget cut, a
one-node budget must not claim exhaustion, and the policy panel must
actually disagree. difficulty-baseline.rs marked superseded, kept as the
survey's dated snapshot. Registered as F16, inconsistent / withdrawn.
T07. evidence/CB-EV-0024. Five of nine defects came only from the review;
four from execution, and all four of those were in work written after it.
The wrong-denominator family now has five instances and still no control
-- facts-check catches copies that disagree, nothing catches a number
computed correctly against the wrong base. Tier L was an over-declaration
(no port, structurally M) and paid for itself anyway, because the review
is L-only. Chaos window 2 will close with zero overrides, making its
retirement condition untestable.
Named as open rather than implied done: the witness is NOT wired to the
ending page. The search works; the browser cannot ask it yet.
make all: exit 0. loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 19:24:49 +02:00
|
|
|
# CB-WP-0025 T06: the difficulty table (specs/RetrospectiveAnalysis.md §4).
|
|
|
|
|
# Winnable fraction from the solver plus a PLURAL policy panel -- a single
|
|
|
|
|
# policy's win rate may not be reported as a difficulty (§4.1).
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## winnable fraction + a plural policy panel (never one bot's win rate)
|
CB-WP-0025 T06/T07: a difficulty baseline that reports its own confound
T06. games/ground/examples/difficulty.rs, make difficulty, wired into make
self-tests, and a report file in ground-game under GROUND-WP-0005 with a
hub message pointing at it.
THE REPORT OPENS WITH THE RETRACTION, because what this task was written
to send was withdrawn by T02 and GROUND-WP-0005 is blocked on exactly that
number. They are told, in the first section, that we nearly sent them "the
game is too easy at 5-6 seats" and why it was wrong.
seats winnable greedy random first-legal spread
2p 60% 60.0% 5.0% 76.7% 71.7
3p 93% 88.3% 6.7% 25.0% 81.7
4p 100% 93.3% 6.7% 30.0% 86.7
5p 100% 100.0% 3.3% 0.0% 100.0
6p 100% 100.0% 3.3% 0.0% 100.0
SPREAD justifies the whole redesign: 71.7 to 100.0 points between three
trivial policies. The table now shows why no single rate is a difficulty
rather than asserting it.
And the 5-6 rows point the OPPOSITE way from the withdrawn claim --
first-legal 0% against greedy 100% is the widest spread in the table,
which suggests play matters MORE there, not less. Neither reading is
established and the report says so.
The confound is stated in the tool's own output, not only in prose:
`winnable` is conditioned on greedy's play up to the final round, because
searching from round 1 is unaffordable. Presenting it as a property of the
deal would repeat this pass's error in a subtler form -- which is exactly
how a corrected project reintroduces a defect. NO THRESHOLD CHANGES ARE
PROPOSED.
The instrument can fail (spec §5): a witness must replay to a win, an
unwinnable position must report searched-out rather than a budget cut, a
one-node budget must not claim exhaustion, and the policy panel must
actually disagree. difficulty-baseline.rs marked superseded, kept as the
survey's dated snapshot. Registered as F16, inconsistent / withdrawn.
T07. evidence/CB-EV-0024. Five of nine defects came only from the review;
four from execution, and all four of those were in work written after it.
The wrong-denominator family now has five instances and still no control
-- facts-check catches copies that disagree, nothing catches a number
computed correctly against the wrong base. Tier L was an over-declaration
(no port, structurally M) and paid for itself anyway, because the review
is L-only. Chaos window 2 will close with zero overrides, making its
retirement condition untestable.
Named as open rather than implied done: the witness is NOT wired to the
ending page. The search works; the browser cannot ask it yet.
make all: exit 0. loop-lint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 19:24:49 +02:00
|
|
|
difficulty:
|
|
|
|
|
@cargo run --release -q -p games-ground --example difficulty
|
|
|
|
|
|
CB-REV-0002: round 2, and the corrections were not approvable either
Three FATAL, five SERIOUS. The substance of round 1's corrections held —
Reactive is genuinely one arm different, the five replacement controls are
non-inert, the inert metric is right, the numbers reproduce. What failed
were the CLAIMS about them, and two defects the corrections introduced.
FATAL 1: the fix for round 1's #11 did not fix it. The assertion was
`games + setup_fails == 200`, and a refused setup increments setup_fails
while skipping games — so the sum is invariant under exactly the failure
it claimed to catch. Injecting setup failures gave exit 0 over 196-game
columns. Now asserts games == GAMES, verified to exit 101.
FATAL 2: the correction to the selective-column FATAL was itself
selective. "81-1000 per cell, baseline AND H1" and "31-1000" twelve lines
apart, both taken from the baseline row; under H1 rank-75 arms are
59/0/0/0. Every cell is now printed rather than summarised, and the
corrected verdict is the opposite of the one it replaced: under rank-75,
H1 REDUCES DARVO arms to zero at 3p and above.
FATAL 3: "DARVO arms 2 per seat per game" is 1 per seat per game, exactly,
at every band.
SERIOUS: the tiebreak oracle asserted only that the winner set CHANGED, so
reversing the tiebreak left it green; the #13 defect's impact was claimed
and never measured (72,000 games: zero divergences — real in principle,
witnessed only by a constructed board); a 29-of-363 citation pointed at a
file that did not contain it (round 1's reviewer did report it, and it was
never transcribed — the record was wrong, not the number); the harnesses
were run by NO GATE, so every published figure came from a manual run of
an ungated binary, including the assertion added for #1; and edition-check's
sibling handling — added by the last correction — was self-certifying,
crashed instead of failing, and counted Markdown lines as coverage. Now
discovered on disk, and it found a real gap on its first run: Rules_Text.csv
vendored with no digest.
Also: "peak Stress held" was dead code kept quiet by `let _ = held;` — the
numbers were right by coincidence.
make panels is now a registered gate. Round 3 is owed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 09:44:00 +02:00
|
|
|
# CB-REV-0002 #7: the H1 measurement harnesses were run by NO gate. Every
|
|
|
|
|
# number in CB-EV-0030 and CB-EV-0031 came from a manual invocation of an
|
|
|
|
|
# ungated binary -- so the assertion added to catch short cells was
|
|
|
|
|
# unreachable from `make`, and "what would the harness report if the work
|
|
|
|
|
# silently stopped" answered: green, and nothing else.
|
|
|
|
|
## the variant panels: ATTACK's value, and regulation under H1
|
|
|
|
|
panels:
|
|
|
|
|
@cargo run --release -q -p games-ground --example attack-value
|
|
|
|
|
@cargo run --release -q -p games-ground --example regulation
|
CB-WP-0041 T01: perfect recall fails on the projection, holds on the history
The answer is "it depends what you call an information set", and the
distinction is the result. 44,938 decision points, random play, 2/3/4/6
seats.
Reading A — information set = the seat's current projection, which is what
project(Viewer::Player(seat)) returns and what the page renders: 22
violations. Reading B — information set = the seat's observation history,
every view seen and action taken in order: 0.
The Reading A witness is concrete. Two histories reach a byte-identical
view — round 3, Select step, same hand, same claimed Problem — where the
seat had played SOLVE then GROUND-OU(protect) in one and SUPPORT then
SOLVE in the other. The view does not tell the seat what it did, because
our state is a snapshot rather than a history: selections clear each round
and effects coincide, so a player cannot reconstruct their own past from
the present. In a real game the player's memory supplies it; in the state,
nothing does.
That is precisely OpenSpiel's ObservationString vs InformationStateString
split, arrived at here by measurement rather than read off. project() is
an observation, not an information state.
So Track B is not closed, it is constrained, and usefully: an
extensive-form game built from this engine must key information sets on
observation histories, never on project().
Both directions are asserted — Reading B empty AND Reading A non-empty —
because if the sample stops finding Reading A violations the conclusion is
unsupported and must be re-derived rather than quietly kept. And the check
samples, so it can falsify perfect recall and cannot establish it: Reading
B's zero means no counterexample was drawn, which is printed as such.
Wired into make panels, so it is re-derived by the gate rather than by
hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 15:18:36 +02:00
|
|
|
@cargo run --release -q -p games-ground --example perfect-recall
|
CB-REV-0002: round 2, and the corrections were not approvable either
Three FATAL, five SERIOUS. The substance of round 1's corrections held —
Reactive is genuinely one arm different, the five replacement controls are
non-inert, the inert metric is right, the numbers reproduce. What failed
were the CLAIMS about them, and two defects the corrections introduced.
FATAL 1: the fix for round 1's #11 did not fix it. The assertion was
`games + setup_fails == 200`, and a refused setup increments setup_fails
while skipping games — so the sum is invariant under exactly the failure
it claimed to catch. Injecting setup failures gave exit 0 over 196-game
columns. Now asserts games == GAMES, verified to exit 101.
FATAL 2: the correction to the selective-column FATAL was itself
selective. "81-1000 per cell, baseline AND H1" and "31-1000" twelve lines
apart, both taken from the baseline row; under H1 rank-75 arms are
59/0/0/0. Every cell is now printed rather than summarised, and the
corrected verdict is the opposite of the one it replaced: under rank-75,
H1 REDUCES DARVO arms to zero at 3p and above.
FATAL 3: "DARVO arms 2 per seat per game" is 1 per seat per game, exactly,
at every band.
SERIOUS: the tiebreak oracle asserted only that the winner set CHANGED, so
reversing the tiebreak left it green; the #13 defect's impact was claimed
and never measured (72,000 games: zero divergences — real in principle,
witnessed only by a constructed board); a 29-of-363 citation pointed at a
file that did not contain it (round 1's reviewer did report it, and it was
never transcribed — the record was wrong, not the number); the harnesses
were run by NO GATE, so every published figure came from a manual run of
an ungated binary, including the assertion added for #1; and edition-check's
sibling handling — added by the last correction — was self-certifying,
crashed instead of failing, and counted Markdown lines as coverage. Now
discovered on disk, and it found a real gap on its first run: Rules_Text.csv
vendored with no digest.
Also: "peak Stress held" was dead code kept quiet by `let _ = held;` — the
numbers were right by coincidence.
make panels is now a registered gate. Round 3 is owed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 09:44:00 +02:00
|
|
|
|
CB-WP-0022 T05/T06/T07: the register, and what its first run found
T05. tools/design.py, make design, and the register in GroundRules.md --
14 rows, no new file, because ADR-0012 D2 made §Underdetermined the
register rather than building one beside it.
Backfill was the test and it caught two things the ADR did not have.
First, a `role` column. The first report alarmed on U2 and was wrong to:
U2's scenario is green BECAUSE the provisional default it documents is
implemented, which says nothing about whether ground-game agrees. GR-E01's
was a counterexample that went green. Same colour, opposite meaning -- a
register that cannot tell them apart either alarms constantly or never.
Only a green counterexample alarms. Folded back into GameDesign §1.3.
Second, it contradicted the survey. CB-RES-0007 said "six of the ten
already have provisional scenarios." Measured -- grep -lE "\bU<n>\b" over
scenarios/ground -- exactly ONE U-item names itself. Five provisional
scenarios exist and four probably encode U-item defaults, but the mapping
is not written down, so it is not checkable. Same defect class as the
wrong premises, found inside the survey that proposed the fix. Now a
reported debt: open, lacking a reproduction: 9, target 0.
design.py carries the control design-baseline.py never had, asserted
directly: a row citing a nonexistent file must not count as reproduced,
using the exact path 2da19a4 deleted -- which the old tool called green.
design-baseline.py is marked superseded rather than deleted; it is the
evidence for how a wrong number got into a survey.
T06. The report is a FILE in ground-game under GROUND-WP-0002, committed
there, with a hub message that only points at it. It asks for no ruling:
it carries GR-E01's withdrawal, our reproduction debt, and two notes that
are explicitly not findings.
And it had to acknowledge something nobody anticipated. GROUND-WP-0002 is
finished -- all ten U-items were RULED 2026-08-03, every one confirmed as
the default we simulate, plus five of six provisional scenarios. The
survey said "0 of 10 ruled" two days later and this register was built
saying `reported`. That is the unread-inbox failure running in the
opposite direction: they answered and we did not collect it. The
instrument's first run surfaced it. They are `ruled`, not `applied` --
lifting the now-settled provisional flags is owed and is not done, and
make design shows them open until it is.
T07. evidence/CB-EV-0021. Two of six catches in this pass came from
execution rather than process (the role distinction from building it, the
ten uncollected rulings from running it), which is InnerLoop §Design
goal's prediction holding.
make self-tests, facts-check, loop-lint: clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 15:22:33 +02:00
|
|
|
# CB-WP-0022 T05: the design-finding register, reported over
|
|
|
|
|
# specs/GroundRules.md. Shows the QUEUE by default; the log of closed
|
|
|
|
|
# findings is a line, not a listing, because a default view that mixes
|
|
|
|
|
# them loses the queue property (ADR-0012 D5).
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## the design-finding register: what is open, and what lacks a reproduction
|
CB-WP-0022 T05/T06/T07: the register, and what its first run found
T05. tools/design.py, make design, and the register in GroundRules.md --
14 rows, no new file, because ADR-0012 D2 made §Underdetermined the
register rather than building one beside it.
Backfill was the test and it caught two things the ADR did not have.
First, a `role` column. The first report alarmed on U2 and was wrong to:
U2's scenario is green BECAUSE the provisional default it documents is
implemented, which says nothing about whether ground-game agrees. GR-E01's
was a counterexample that went green. Same colour, opposite meaning -- a
register that cannot tell them apart either alarms constantly or never.
Only a green counterexample alarms. Folded back into GameDesign §1.3.
Second, it contradicted the survey. CB-RES-0007 said "six of the ten
already have provisional scenarios." Measured -- grep -lE "\bU<n>\b" over
scenarios/ground -- exactly ONE U-item names itself. Five provisional
scenarios exist and four probably encode U-item defaults, but the mapping
is not written down, so it is not checkable. Same defect class as the
wrong premises, found inside the survey that proposed the fix. Now a
reported debt: open, lacking a reproduction: 9, target 0.
design.py carries the control design-baseline.py never had, asserted
directly: a row citing a nonexistent file must not count as reproduced,
using the exact path 2da19a4 deleted -- which the old tool called green.
design-baseline.py is marked superseded rather than deleted; it is the
evidence for how a wrong number got into a survey.
T06. The report is a FILE in ground-game under GROUND-WP-0002, committed
there, with a hub message that only points at it. It asks for no ruling:
it carries GR-E01's withdrawal, our reproduction debt, and two notes that
are explicitly not findings.
And it had to acknowledge something nobody anticipated. GROUND-WP-0002 is
finished -- all ten U-items were RULED 2026-08-03, every one confirmed as
the default we simulate, plus five of six provisional scenarios. The
survey said "0 of 10 ruled" two days later and this register was built
saying `reported`. That is the unread-inbox failure running in the
opposite direction: they answered and we did not collect it. The
instrument's first run surfaced it. They are `ruled`, not `applied` --
lifting the now-settled provisional flags is owed and is not done, and
make design shows them open until it is.
T07. evidence/CB-EV-0021. Two of six catches in this pass came from
execution rather than process (the role distinction from building it, the
ten uncollected rulings from running it), which is InnerLoop §Design
goal's prediction holding.
make self-tests, facts-check, loop-lint: clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 15:22:33 +02:00
|
|
|
design:
|
|
|
|
|
@$(PY) $(TOOLS)/design.py
|
|
|
|
|
|
2026-07-31 10:20:27 +02:00
|
|
|
# T03: one-shot orientation — workplans, next task, spend, fast gates.
|
|
|
|
|
# Cheap by design: no build. Start a session with this instead of grepping.
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## one-shot orientation: workplans, next task, spend, fast gates
|
2026-07-31 10:20:27 +02:00
|
|
|
status:
|
|
|
|
|
@$(PY) $(TOOLS)/status.py
|
|
|
|
|
|
CB-WP-0004 T02: make task-done — close a task on measured numbers
Replaces the three hand-done steps of a task close (46 turns, $11.52 per
CB-RES-0003): the heredoc flipping status in the workplan file, the
hand-written hub call, and the hand-typed token counts.
The third is the reason this task exists. Every update_task_status this
repo produced carried estimated tokens_in/tokens_out — in a project whose
central finding is that estimated token counts are worthless. task-done
reads the measured figure from the transcripts, or refuses; there is no
path through it that emits an estimate.
cb-cost gains by_task_detail: cost, response count, model histogram and
token components per task. task-done imports cb-cost rather than parsing
its printed table, so the hub figure is not a copy that can drift from
its source.
The positive control found a real defect before the tool ran once.
Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002
T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test
reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x
overstatement would have been pushed to the hub as a *measured* number —
the same fiction in a new form. task_label() now keys qualified subjects
on the full id and leaves unqualified ones bare rather than
retro-assigning them to a workplan. The pinned $93.15 benchmark is
unchanged, so historical attribution was not disturbed.
Fourth instance of trusted arithmetic: a number believed because a
program produced it rather than a hand.
Refusals, all exercised by --self-test: unknown id, typo'd id,
already-done task, missing state_hub_task_id, no measured spend, and a
status flip that produced no change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:17:51 +02:00
|
|
|
# T02: close a task — flip the workplan file, read the *measured* cost
|
|
|
|
|
# from the transcripts, push the hub event with real numbers. Refuses on an
|
|
|
|
|
# unknown or already-done task, and refuses to report an estimate.
|
|
|
|
|
# make task-done T=CB-WP-0004-T02
|
|
|
|
|
task-done:
|
|
|
|
|
@test -n "$(T)" || { echo "usage: make task-done T=CB-WP-0004-T02" >&2; exit 2; }
|
|
|
|
|
$(PY) $(TOOLS)/task-done.py $(T) $(ARGS)
|
|
|
|
|
|
CB-WP-0007 T01+T03: window the metric, budget it, cap meta at 25%
Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:07 +02:00
|
|
|
# CB-WP-0007 T03 / InnerLoop v1.5: session shape for the window since the
|
|
|
|
|
# last commit. Deliberately NOT in `make all` — failing the build on
|
|
|
|
|
# context would block committing, and committing is the natural point to
|
|
|
|
|
# compact. A gate that blocks the remedy is a trap.
|
|
|
|
|
shape-budget: cost-test
|
|
|
|
|
$(PY) $(TOOLS)/cb-cost.py --shape-budget
|
|
|
|
|
|
T05: live cost budget replaces the dead token budget
The 8k/10k per-task token budget was never referenced or enforced and
T08 blew past it silently. Replaced with a budget that can actually
fire.
The design constraint is the interesting part: per-task cost needs the
commit that CLOSES the task, so a per-task budget is unavoidably
retrospective -- it can only report a breach after the money is spent,
which is the dead-policy failure again. What IS observable mid-task is
spend since the last commit, because the transcript is append-live. So
the budget binds on the open remainder.
CB-01 budget = USD since the last commit, via `make cost-budget`
CB-02 soft $10.00 (state progress, decide), hard $22.00 (stop)
Calibrated on the 32 non-empty commit intervals of CB-WP-0001: p50
$1.40, p90 $9.36, max $10.80. Soft sits just below the observed maximum
-- it would have fired exactly once on the calibration pass. Hard is ~2x
the observed max, a value never reached in 32 intervals, so reaching it
means the session is doing something the data has no example of.
Both thresholds are set ABOVE every observed value, so they bind on
future work rather than ratifying present work -- the distinction T07
is about.
Stated limit: it is a command, not a daemon. An agent that never runs it
gets no signal, which is the dead-policy failure one level up. Mitigated
only by being free to run and on the one command surface.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:21:30 +02:00
|
|
|
# CB-01/CB-02: live spend since the last commit.
|
|
|
|
|
cost-budget: cost-test
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/cb-cost.py --budget
|
T05: live cost budget replaces the dead token budget
The 8k/10k per-task token budget was never referenced or enforced and
T08 blew past it silently. Replaced with a budget that can actually
fire.
The design constraint is the interesting part: per-task cost needs the
commit that CLOSES the task, so a per-task budget is unavoidably
retrospective -- it can only report a breach after the money is spent,
which is the dead-policy failure again. What IS observable mid-task is
spend since the last commit, because the transcript is append-live. So
the budget binds on the open remainder.
CB-01 budget = USD since the last commit, via `make cost-budget`
CB-02 soft $10.00 (state progress, decide), hard $22.00 (stop)
Calibrated on the 32 non-empty commit intervals of CB-WP-0001: p50
$1.40, p90 $9.36, max $10.80. Soft sits just below the observed maximum
-- it would have fired exactly once on the calibration pass. Hard is ~2x
the observed max, a value never reached in 32 intervals, so reaching it
means the session is doing something the data has no example of.
Both thresholds are set ABOVE every observed value, so they bind on
future work rather than ratifying present work -- the distinction T07
is about.
Stated limit: it is a command, not a daemon. An agent that never runs it
gets no signal, which is the dead-policy failure one level up. Mitigated
only by being free to run and on the one command surface.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:21:30 +02:00
|
|
|
|
CB-RES-0003 + CB-WP-0004: 38% of pass cost is mechanical turns
Review of where token-priced turns did work a deterministic tool could
do. Method: classify every turn in both transcripts by the tool calls it
made. The classifier is committed in tools/cb-cost.py and emitted by
`make cost-mix`, so the baseline is reproducible and the same command
can later falsify the predictions.
mech environment setup 84 turns $15.33
mech ad-hoc text patching 75 turns $13.86
git 37 turns $13.85
mech hub task status 25 turns $ 7.46
mech orientation / inspect 49 turns $ 6.87
hub other 32 turns $ 6.22
mech ad-hoc transcript 39 turns $ 4.56
mech workplan status edit 21 turns $ 4.06
MECHANICAL (dedup) 290 turns $51.26 = 38% of pass
Largest category is `cd` and `export PATH` -- pure friction, and
dep-weight.py already patched it at the leaf, which is evidence it was
noticed and fixed in the wrong place. Second is heredocs string-patching
markdown, which is also the mechanism behind duplicated-fact drift, the
error class InnerLoop v1.2 names and cannot gate.
Explicitly NOT automated: git (37 turns, $13.85) is mostly commit
message authorship -- the highest-output turns in the corpus and the
project's reasoning record. Automating it would save money and destroy
what makes corrections cheap.
CB-WP-0004 implements five candidates and predicts $33-41 recovery
(25-30%), below the 38% measured share on purpose: some inspection and
patching is genuinely exploratory.
The control loop is the deliverable, not a formality. T05 tests three
things and must report all: did mechanical turns disappear, did they
RELOCATE into prose, and did quality hold. If mechanical turns fall and
prose rises by as much, the saving is zero and that is the result to
publish.
Also a self-indictment worth recording: every hub update_task_status in
this project carried hand-typed token estimates, in a repo whose central
finding is that estimated token counts are worthless. T02 fixes it by
reading measured values from cb-cost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:49:57 +02:00
|
|
|
# CB-RES-0003 baseline: mechanical vs judgment turns.
|
|
|
|
|
cost-mix: cost-test
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/cb-cost.py --composition
|
CB-RES-0003 + CB-WP-0004: 38% of pass cost is mechanical turns
Review of where token-priced turns did work a deterministic tool could
do. Method: classify every turn in both transcripts by the tool calls it
made. The classifier is committed in tools/cb-cost.py and emitted by
`make cost-mix`, so the baseline is reproducible and the same command
can later falsify the predictions.
mech environment setup 84 turns $15.33
mech ad-hoc text patching 75 turns $13.86
git 37 turns $13.85
mech hub task status 25 turns $ 7.46
mech orientation / inspect 49 turns $ 6.87
hub other 32 turns $ 6.22
mech ad-hoc transcript 39 turns $ 4.56
mech workplan status edit 21 turns $ 4.06
MECHANICAL (dedup) 290 turns $51.26 = 38% of pass
Largest category is `cd` and `export PATH` -- pure friction, and
dep-weight.py already patched it at the leaf, which is evidence it was
noticed and fixed in the wrong place. Second is heredocs string-patching
markdown, which is also the mechanism behind duplicated-fact drift, the
error class InnerLoop v1.2 names and cannot gate.
Explicitly NOT automated: git (37 turns, $13.85) is mostly commit
message authorship -- the highest-output turns in the corpus and the
project's reasoning record. Automating it would save money and destroy
what makes corrections cheap.
CB-WP-0004 implements five candidates and predicts $33-41 recovery
(25-30%), below the 38% measured share on purpose: some inspection and
patching is genuinely exploratory.
The control loop is the deliverable, not a formality. T05 tests three
things and must report all: did mechanical turns disappear, did they
RELOCATE into prose, and did quality hold. If mechanical turns fall and
prose rises by as much, the saving is zero and that is the result to
publish.
Also a self-indictment worth recording: every hub update_task_status in
this project carried hand-typed token estimates, in a repo whose central
finding is that estimated token counts are worthless. T02 fixes it by
reading measured values from cb-cost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:49:57 +02:00
|
|
|
|
T04: tools/cb-cost.py — and its positive control fires on first contact
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
|
|
|
cost-pin: cost-test
|
2026-07-31 10:13:52 +02:00
|
|
|
$(PY) $(TOOLS)/cb-cost.py --pin fc76445 --composition --by-task
|
T04: tools/cb-cost.py — and its positive control fires on first contact
Collector per ADR-0003: enumerates every transcript including the
subagents/ tree, dedups by requestId, prices per model and per cache
TTL, attributes on (prev_commit, this_commit] intervals, and reconciles
to the cent or aborts.
The positive control caught a real defect on its very first run against
real data, which is the entire argument for writing it:
CA-02 assumed usage is identical across the lines of one requestId.
True in the main transcript (206/206 groups, verified twice — by the
survey and by the adversarial reviewer). FALSE in the subagents/
tree, where output_tokens is a running count: one response reads
5, 5, 195 across its three lines. First-wins scored it at 5.
So CA-02 now splits: input-side counters are charged once and must be
identical (assertion retained); output_tokens resolves to the max
(CA-02a). AC-9 pins the exact 5,5,195 case as a regression test.
The acceptance target moved again as a result, for the third time:
$248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same
first-wins method the tool just disproved, so the tool failing its
target was the tool being correct. Target updated, not the tool.
Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual
$0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10.
make cost / cost-test / cost-pin wired; cost-test added to `make all`
and to CI, where it gates the collector's assertions without needing
transcripts present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
|
|
|
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## run all GROUND scenarios through cb-sim
|
2026-07-31 01:57:13 +02:00
|
|
|
sim:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(IN_REPO) $(CARGO) run -q -p cb-sim -- $(REPO)/scenarios/ground/*.yaml
|
2026-07-31 01:57:13 +02:00
|
|
|
|
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
|
|
|
## Criterion benches (AM-6/AM-7)
|
2026-07-31 01:57:13 +02:00
|
|
|
bench:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(IN_REPO) $(CARGO) bench -p games-ground
|
2026-07-31 01:57:13 +02:00
|
|
|
|
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
|
|
|
## InnerLoop positive control: run every bench once, no measurement.
|
|
|
|
|
## Fails if a workload stalls or produces the wrong event count.
|
|
|
|
|
bench-test:
|
2026-07-31 10:13:52 +02:00
|
|
|
$(IN_REPO) $(CARGO) bench -p games-ground --bench synthetic -- --test
|
CI: enforce every gate; close the silent-skip holes
The gates existed; CI ran half of them and tolerated the failure case.
- cb-sim no longer has a "tolerable" non-zero exit. An unregistered game
prefix is a failure, and a run in which nothing executed is a failure.
Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix
would have skipped every scenario while the pipeline stayed green.
Verified with a negative control.
- CI now runs make coverage (AM-1) and make dep-weight (AM-4), both
added after CI was written and neither enforced until now.
- dep-weight enforces its targets instead of only reporting them.
- CI lints the shipped-runtime configuration separately, so the feature
split cannot rot unnoticed.
- Dropped the stale `make deps` target, which still measured the retired
crate-count metric.
The positive-control rule is now executable: CI runs
`cargo bench -- --test`, which executes every benchmark once, so a
workload that stalls fails the build.
That step immediately found a fourth instance of the error class it was
written for. The committed replay benchmark was the broken version — an
earlier patch never applied, leaving a command sequence that omits
Resolve, so every round produced nothing and the log-building loop spun
forever. It had never run to completion; the reported AM-7 replay
numbers came from a probe test instead. Fixed, given the same positive
control as the round loop, and re-measured from the benchmark: 100k
events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget.
Evidence now reports confidence intervals rather than point estimates,
so the 3% regression rule in MetricsAndScenarios is enforceable.
The finding worth carrying: writing the positive-control rule into
InnerLoop v1.0 did not prevent the next instance. Making it a CI step
did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
|
|
|
|
2026-07-31 01:57:13 +02:00
|
|
|
|
|
|
|
|
## AM-2/AM-3 input: source LOC per crate (excludes tests would need tokei)
|
|
|
|
|
loc:
|
2026-07-31 10:13:52 +02:00
|
|
|
@$(IN_REPO) for d in crates/cb-kernel crates/cb-events crates/cb-game-runtime games/ground tools/cb-sim; do \
|
2026-07-31 01:57:13 +02:00
|
|
|
printf '%-28s %s\n' $$d "$$(find $$d/src -name '*.rs' | xargs cat | grep -vcE '^\s*(//|$$)')"; \
|
|
|
|
|
done
|
|
|
|
|
|
make help, and make ground — one way in, and a list of the rest
Tier S (utility inside a boundary; chaos d8=5, no override). Provenance in
this message per InnerLoop §Loop tiers.
The maintainer asked how to start the UI. The answer was a five-flag
cb-play invocation nobody could be expected to remember, in a Makefile
whose own header calls itself "one command surface" -- a surface you have
to read the source of is not one.
make ground # play, recording a trial
make ground PLAYERS=2 SLUG=darvo-confusion
make trials # read the notes back
`ground` timestamps the trial name so two sessions in one day cannot
overwrite each other's notes, and keeps the date prefix because
tools/trials.py reads the age from there.
`make` with no target now lists targets instead of running the heaviest
thing in the file. It extracts the `## ` comments that were already the
convention -- ten of them existed and NOTHING HAD EVER RENDERED THEM.
Which is how the first run found a real defect: `## run all GROUND
scenarios through cb-sim` sat above `dep-weight:`, 167 lines from the
`sim:` it describes. A doc comment nothing reads rots, and this one had --
same family as the message that sat unread for four days and the ten
rulings nobody collected, in a third medium. Moved to `sim`, and
`dep-weight` given its own.
Documented the targets a person reaches for (coverage, status, all,
self-tests, sim, design, difficulty, trials) and listed the undocumented
instruments compactly, so nothing in the file is invisible.
Verified end to end rather than by inspection: `make ground` served a
page (200), a note posted through it (303), and `make trials` reported it
bound to its position.
loop-lint clean, check clean, self-tests 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:53:07 +02:00
|
|
|
## every gate, in order. The one CI would run.
|
2026-08-08 09:55:41 +02:00
|
|
|
all: check test sim coverage size-metrics runtime-metrics am6 am7 am8 edition-check replay-test dep-weight self-tests env-test facts-check loop-lint bench-test panels
|