coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
clay-borg/Makefile

221 lines
8.9 KiB
Makefile
Raw Normal View History

T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
# One command surface (InnerLoop §agentic-efficiency #3). Deterministic,
# greppable output; precursor of the `cb` CLI.
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
#
# CB-WP-0004 T01: every target here runs from a clean shell, from any
# directory, with no prefix. Invoke as `make -C <repo> <target>` from
# elsewhere. No target requires `cd` or `export PATH` — CB-RES-0003
# measured 84 turns and $15.33 spent on exactly those two prefixes.
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
# Absolute path to this Makefile's directory, so recipes never depend on
# the caller's working directory.
REPO := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
# Locate cargo instead of requiring it on the inherited PATH. Mirrors
# tools/repo.py:cargo_bin() — kept in sync by `make env-test`.
CARGO := $(firstword $(shell command -v cargo 2>/dev/null) \
$(wildcard $(HOME)/.cargo/bin/cargo) \
$(wildcard /usr/local/cargo/bin/cargo) \
cargo)
export PATH := $(dir $(CARGO)):$(PATH)
PY := python3
TOOLS := $(REPO)/tools
# Every cargo recipe runs at the repo root; the shell does not persist cd.
IN_REPO := cd $(REPO) &&
CB-WP-0021 T01/T02/T05: the engine plays its own data — AM-7 blocks ADR-0011 decided it: vendor the CSV with a checked digest, read it with a ~50-line reader, and let the hashes move. The declaration's constraint was measured against the WRONG BUDGET. It said a CSV crate costs 21,613 against AM-4a's 3,798 of headroom, '5.7x over, settled by measurement'. But setup and problem_priorities are cfg(scenarios) and are not in the shipped runtime at all, so AM-4a never sees them. Against AM-4b, csv costs 17,651 against 19,742 -- it FITS, with 2,091 to spare. It is refused anyway, on proportion: 89% of the budget's remaining capacity to read 20 rows. The revisit condition is stated (nested quoting, embedded newlines, multiple dialects). GR-S01 now deals Surface + hidden 1..=k as ruled, with edition values and suits. Measured: 6/9/12 available against thresholds 5/7/9 -- the game is winnable at every seat count, which is what the maintainer could not do. gd0001 is INVERTED, not deleted, and now also asserts the 6/9/12 so a deal that is reachable for the wrong reason still fails. Blast radius was scenario expectations, exactly as the ADR predicted: no scenario pinned a hash and no bundle is committed. Six scenarios and two unit tests updated, each with a note. gr-e01-threshold-unreachable-2p is RENAMED to -reachable- and rewritten as the non-provisional import check ground-game asked for by name. gr-e03's setup was restructured, not just renumbered: with values 2,2,2 its personal-edge test would have tied three ways and asserted nothing. BLOCKING: AM-7 fails at median 0.845 against its 0.9 floor. Isolated across three runs -- 3 problems + stand-in 0.97, 3 problems + edition 0.909, 4 problems + edition 0.845. State is BOUNDED (proven: identical after 5k and 100k events), so this is not the unbounded-growth defect AM-7 exists to catch; it is a bigger working set streaming a long log. Whether AM-7's floor is still right for a larger aggregate is a spec question and lowering it requires an ADR, so it is not being tuned here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:47:56 +02:00
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget shape-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen mutation-check size-metrics runtime-metrics build-time am6 am7 am8 edition-check replay-test loc play gate-review all
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
## fmt + clippy (deny warnings) + HashMap deny-lint
check:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(IN_REPO) $(CARGO) fmt --all --check
$(IN_REPO) $(CARGO) clippy --workspace --all-targets -- -D warnings
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
CB-WP-0008-T02: cb-play — INTENT stage 0's CLI player A human seat is a Policy like any bot, so the CLI adds no second driver: HumanPolicy renders the projection, lists the legal commands and reads an index or `pass`. `make play` runs it; `--all-bots` watches one. K13's Project trait gains its first implementor after six passes with none. Hidden: other seats' face-down selections until Reveal, hands and deck (counts only), a face-down Problem's suit and value, and the seed — not secret content, but a seat holding it can compute the deck. A played session becomes an artifact: --record writes it as a scenario the runner executes, --replay writes a .cbreplay bundle. record.rs is the inverse of parse_command and its warrant is a round-trip test over every command shape. The acceptance test for the projection passed vacuously twice. First it asserted the text contained "face-down", which every render does because of Problems. Counted, it then reported zero inspected entries: seats are asked in order, so a human at P1 is prompted before anyone has selected. Seated at P3 it inspects ten entries and dies when the projection is mutated to reveal everything. Counting what the harness examined caught both, which is the second time that remedy has worked where a stronger predicate would not have. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:43:53 +02:00
## play GROUND from the terminal (INTENT stage 0's CLI player).
## `make play ARGS="--all-bots --bot random"` to watch one instead.
play:
$(IN_REPO) $(CARGO) run -q -p cb-play -- $(ARGS)
CB-WP-0009-T02: gates.toml and make gate-review ADR-0006 D3. Nine standing control gates now say what they check, what they have caught with pointers, when their keep-or-kill argument is due, and what would retire them. make gate-review reports what is overdue and what has caught nothing; it never fails the build, for CB-RES-0005 §4's reason. Drift is checked in both directions and both are pinned by self-tests: a dependency of `make all` that is neither a registered control gate nor listed in not_control_gates is a loop-lint finding, so a new gate cannot acquire permanence without a review date, and an entry naming a target the Makefile lacks is a finding too. First run: 0 due, 2 silent. The silent two are the chaos roll, whose 12-declaration window exists precisely to find out, and gate-review itself, which is not exempt from its own rule — if it has retired, tightened or forced the re-justification of nothing by 2026-12-31 it is a ritual and goes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:42:13 +02:00
## which control gates are due for a keep-or-kill argument (ADR-0006 D3)
gate-review:
$(PY) $(TOOLS)/gate-review.py
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
## unit + scenario-format tests
test:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(IN_REPO) $(CARGO) test --workspace
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
## run all GROUND scenarios through cb-sim
AM-4: gate scenario YAML, retarget on audited source, re-measure Adopts both remediations from CB-EV-0001 §4 (maintainer decision). Option A — serde_yaml is now optional behind cb-game-runtime's `scenarios` feature. The scenario module, the ScenarioGame impl and the string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both configurations compile and lint clean under -D warnings. A trap worth recording: `default-features = false` on a *member* dependency is silently ignored when the workspace dependency does not specify it. The first attempt gated nothing while looking correct — the build succeeded and cargo tree still showed all six YAML crates. Fixed by setting it on the workspace dependency. This is the positive-control failure mode in miniature: success was not evidence the change applied. Retarget — AM-4 now measures third-party source under audit, split by build configuration, replacing a crate count that was unreachable without undoing K5/K7 and that does not compare across ecosystems. Re-measured via the new `make dep-weight`, whose own positive control refuses to report when any crate's source cannot be located: shipped runtime 23 crates 246,250 lines target <=250,000 met dev toolchain 29 crates 317,021 lines target <=350,000 met own source 3,408 lines Scenario tooling costs 70,771 lines a shipped game never compiles — the split the single number was hiding. Targets are set at current measurement plus headroom, so they bind on future growth rather than retroactively passing what had failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
dep-weight:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/dep-weight.py
AM-4: gate scenario YAML, retarget on audited source, re-measure Adopts both remediations from CB-EV-0001 §4 (maintainer decision). Option A — serde_yaml is now optional behind cb-game-runtime's `scenarios` feature. The scenario module, the ScenarioGame impl and the string parsers behind it are cfg-gated; cb-sim opts in explicitly. Both configurations compile and lint clean under -D warnings. A trap worth recording: `default-features = false` on a *member* dependency is silently ignored when the workspace dependency does not specify it. The first attempt gated nothing while looking correct — the build succeeded and cargo tree still showed all six YAML crates. Fixed by setting it on the workspace dependency. This is the positive-control failure mode in miniature: success was not evidence the change applied. Retarget — AM-4 now measures third-party source under audit, split by build configuration, replacing a crate count that was unreachable without undoing K5/K7 and that does not compare across ecosystems. Re-measured via the new `make dep-weight`, whose own positive control refuses to report when any crate's source cannot be located: shipped runtime 23 crates 246,250 lines target <=250,000 met dev toolchain 29 crates 317,021 lines target <=350,000 met own source 3,408 lines Scenario tooling costs 70,771 lines a shipped game never compiles — the split the single number was hiding. Targets are set at current measurement plus headroom, so they bind on future growth rather than retroactively passing what had failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 03:35:41 +02:00
Add the AM-1 rule-coverage gate make coverage compares the GR-rule IDs defined in specs/GroundRules.md against the covers: lists in scenarios/ground/*.yaml. It exits 1 if a scenario claims a rule the spec does not define, so coverage cannot be inflated by an invented ID, and exits 2 while rules remain uncovered. Current reading: 34/58 (58%). AM-1 requires 100%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:23:38 +02:00
coverage:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/rule-coverage.py
Add the AM-1 rule-coverage gate make coverage compares the GR-rule IDs defined in specs/GroundRules.md against the covers: lists in scenarios/ground/*.yaml. It exits 1 if a scenario claims a rule the spec does not define, so coverage cannot be inflated by an invented ID, and exits 2 while rules remain uncovered. Current reading: 34/58 (58%). AM-1 requires 100%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 02:23:38 +02:00
CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had no flag parsing at all, so --replay had nowhere to go. The bundle is manifest + commands.log + initial.snapshot + expected.yaml, dev-only behind the scenarios feature and charged to AM-4b. The command stream goes through the K11 framing built in T05, so a truncated bundle is detected rather than replayed short — the two tasks compose rather than duplicating. The reviewer's D2 correction was real: this was not "a directory of four files". Pass carried only the end state, RunOutcome::Failed was a formatted String, and scenario.rs created an EventLog, appended to it and never read it. All three had to change. The first round trip failed to reproduce, and the cause is worth keeping: state_hash_hex over a serde_json::Value is a different canonical form than over the typed aggregate — Value's map is key-sorted, a struct serializes in declaration order. The bundle was written with one basis and verified with the other. A round trip written to recompute its own comparison value would have PASSED this bug; it failed because the recorded hash came from the producing process, which is control 2's entire purpose. make replay-test implements ADR-0005 §6's four controls, 14/14: a committed deliberately-failing fixture outside the corpus with covers: [] so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash must fail; a log short by one byte and a corrupted length prefix must be rejected; and a mutated manifest seed must fail — which bites only because replay re-derives the initial state from seed+setup and checks it against the recorded snapshot, since restoring from the snapshot alone would leave the seed inert. Plus a control on the controls: the bundle must still replay after every mutation is reverted. AM-7's hash-identical clause is re-earned. The probe records a hash per per-game segment and replays each from its own genesis; folding from the wrong seed now fails. That is the clause ADR-0005 §4 withdrew as mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so AM-7 stays PARTIAL — reported, not rounded up. Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's copy of that number going stale, on a number that moved the same hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 11:05:37 +02:00
# K10: a bundle must re-execute, and must be able to fail. Four controls
# from ADR-0005 §6, including truncate-by-one-byte and mutated-seed.
replay-test:
$(PY) $(TOOLS)/replay-test.py
CB-WP-0006 T04: withdraw AM-4c; and fix where AM-6 is measured AM-4c is withdrawn from the acceptance table and retained as a reported diagnostic. GameKernel §5a carries the argument. The ratio has no monotone better direction. INTENT's rule is "own the semantics, assimilate the implementation": rising can mean owning semantics properly or reimplementing what should have been assimilated; falling can mean leverage or dependency bloat. A target requires knowing which way is better. It is also redundant — AM-4a/AM-4b bound the denominator and AM-2 bounds own-source density, so AM-4c is a ratio of two already-targeted quantities. Measured at withdrawal: 1,426 own lines per 100k third-party (shipped), 1,107 (dev). make dep-weight now prints both, labelled diagnostic — the row was never actually reported before. M-D1-MUT keeps AM-4c in its denominator on purpose and says so in the output. Dropping it would move the score 7/14 -> 7/13 without enforcing anything: a score improved by deleting the question. Decided before Phase B deliberately, since ADR-0005 predicts own-source growth that will move this ratio; deciding after would be the retarget §Step 4 forbids. A T01 correction found here. The AM-6 gate failed inside `make all` at 38,753 ev/s against 341,280 in isolation — a 9x drop, because cargo test runs binaries and threads concurrently. A throughput assertion inside a parallel harness measures contention, not throughput. T01's measurement was valid; its gate placement was not. Fixed by running it only where valid — #[ignore] plus `make am6` in release with --test-threads=1, now 2.0M ev/s at 20.2x headroom — and not by lowering the target, which T01 forbade. My first attempt did drift that way, adding a debug "sanity floor" of 50,000, and was backed out: a second threshold is still a second chance to tune. The mutation then went SURVIVED on the first run after the move. 4,000 black_box iterations were calibrated against debug's 3.4x headroom and are invisible against release's 20x. Raised to 100,000; back to red. A weak mutation is not a fixed property of a row — it can become weak when the row's measurement conditions change. Tier S (amends one row, creates no capability), chaos d4=2, no override. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:06:00 +02:00
# AM-6 gate. Runs in RELEASE, where headroom is ~24x; the same assertion
# in debug has ~3.4x and flaked under load. The target is unchanged — only
# where it is measured.
am6:
$(IN_REPO) $(CARGO) test --release -p games-ground --all-features \
am6_throughput -- --ignored --nocapture --test-threads=1
CB-WP-0021 T01/T02/T05: the engine plays its own data — AM-7 blocks ADR-0011 decided it: vendor the CSV with a checked digest, read it with a ~50-line reader, and let the hashes move. The declaration's constraint was measured against the WRONG BUDGET. It said a CSV crate costs 21,613 against AM-4a's 3,798 of headroom, '5.7x over, settled by measurement'. But setup and problem_priorities are cfg(scenarios) and are not in the shipped runtime at all, so AM-4a never sees them. Against AM-4b, csv costs 17,651 against 19,742 -- it FITS, with 2,091 to spare. It is refused anyway, on proportion: 89% of the budget's remaining capacity to read 20 rows. The revisit condition is stated (nested quoting, embedded newlines, multiple dialects). GR-S01 now deals Surface + hidden 1..=k as ruled, with edition values and suits. Measured: 6/9/12 available against thresholds 5/7/9 -- the game is winnable at every seat count, which is what the maintainer could not do. gd0001 is INVERTED, not deleted, and now also asserts the 6/9/12 so a deal that is reachable for the wrong reason still fails. Blast radius was scenario expectations, exactly as the ADR predicted: no scenario pinned a hash and no bundle is committed. Six scenarios and two unit tests updated, each with a note. gr-e01-threshold-unreachable-2p is RENAMED to -reachable- and rewritten as the non-provisional import check ground-game asked for by name. gr-e03's setup was restructured, not just renumbered: with values 2,2,2 its personal-edge test would have tied three ways and asserted nothing. BLOCKING: AM-7 fails at median 0.845 against its 0.9 floor. Isolated across three runs -- 3 problems + stand-in 0.97, 3 problems + edition 0.909, 4 problems + edition 0.845. State is BOUNDED (proven: identical after 5k and 100k events), so this is not the unbounded-growth defect AM-7 exists to catch; it is a bigger working set streaming a long log. Whether AM-7's floor is still right for a larger aggregate is a spec question and lowering it requires an ADR, so it is not being tuned here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:47:56 +02:00
# ADR-0011 D2: is the vendored edition still what ground-game published?
# Reports "upstream not checked out" as its own outcome, never a pass.
edition-check:
$(PY) $(TOOLS)/edition-check.py
CB-WP-0015: the two inert clauses, AM-7 scaling and AM-8 N=10 Provenance (tier S, one paragraph in lieu of survey and ADR): the two clauses mutation-check has reported inert since CB-WP-0005. AM-7's scaling ratio was held up by a test literally named replay_100k_events_is_linear_and_fast that computed both throughputs, printed both, and never divided one by the other. AM-8's N=10 was held up by a runner that does two. Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's >=10-of-14 prediction MET for the first time. Neither was closed by amending the question away, which was the live risk: the denominator is unchanged and the four unenforced rows are the four already unenforceable. AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's, correct for a floor on one number) gave 0.581-1.085 on an unchanged binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU contention while absolute throughput fell 4x. The INDETERMINATE guard demanded unanimity and failed a good measurement over one sample 0.001 under the floor; it now requires a two-thirds majority. The control that matters: AM-6's constant-cost mutation halves throughput and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6. AM-8 kept N=10 because the measurement said so. Perturbing the RNG only from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A late-onset divergence is deterministic, not flaky, so it is a control rather than a coin flip. Ten runs live on one scenario (make am8, ~2s) rather than all 25 (47s a build). GameKernel 5b records it. The full run also found AM-4a's own mutation stale since ADR-0008 D3 moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported HARNESS-BROKEN, no score published. The build-free half of that check is now a --self-test assertion, so make all catches the next one. mutation-check clauses may now carry their own verify and mutation, and then the enforced flag is measured rather than declared; a declaration disagreeing with its measurement is refused. make all exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 14:07:08 +02:00
# AM-8 N=10 determinism gate. One scenario, ten same-seed replays, all
# compared to the first. Not all 25: `make sim` already runs K8's double-
# run over every scenario, and repeating that eight more times costs 47 s
# per build to re-answer a question the second run already answered. The
# extra runs exist for the probabilistic class, and one workload gives
# that class its ten samples — see scenario::run_n.
am8:
$(IN_REPO) $(CARGO) run -q -p cb-sim -- --runs 10 \
$(REPO)/scenarios/ground/gr-r06-round-resolve.yaml
# AM-7 scaling gate. Same release / single-thread reasoning as am6 and more
# so: a *ratio* of two timings taken under varying contention is worse than
# one reading, because the noise multiplies rather than cancels.
am7:
$(IN_REPO) $(CARGO) test --release -p games-ground --all-features \
CB-WP-0021 T06: fix AM-7's measurement, not its floor The row folded a 5,000-event log against a 100,000-event log and compared throughputs, which confounds 'does cost per event grow with history' (the property it claims) with 'does streaming a 20x longer Vec cost more per element' (a memory-hierarchy fact true of any program). It measured the second and reported it as the first: importing the edition enlarged the aggregate and the ratio fell to 0.845 with the state bounded. Corrected to time the SAME 5,000 events on a state at depth 0 and on a state at depth 100,000. Equal windows, equal event mix, so the only difference left is history depth. corrected: clean 1.004, mutated 0.589 (red) old: clean 0.845 (red on healthy code), mutated 0.751 It also runs in 8.5s instead of timing out: the first version re-walked the 100k prefix every repetition, 200M untimed folds per sample, which under the mutation never finished. A control that cannot be run is not a control. It now advances to depth once per sample and clones. Two of my own measurements here were wrong and both were caught by measuring again. A 2-minute timeout killed the shell line before its restoring cp ran, so three readings were taken on MUTATED code -- I diagnosed an event-mix confound that did not exist and 'fixed' it. The fix is kept on its merits; the justification was fiction. And the probe that proved state was bounded had checked four of eleven collections. make all exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 01:10:38 +02:00
am7_cost_per_event -- --ignored --nocapture --test-threads=1
CB-WP-0015: the two inert clauses, AM-7 scaling and AM-8 N=10 Provenance (tier S, one paragraph in lieu of survey and ADR): the two clauses mutation-check has reported inert since CB-WP-0005. AM-7's scaling ratio was held up by a test literally named replay_100k_events_is_linear_and_fast that computed both throughputs, printed both, and never divided one by the other. AM-8's N=10 was held up by a runner that does two. Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's >=10-of-14 prediction MET for the first time. Neither was closed by amending the question away, which was the live risk: the denominator is unchanged and the four unenforced rows are the four already unenforceable. AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's, correct for a floor on one number) gave 0.581-1.085 on an unchanged binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU contention while absolute throughput fell 4x. The INDETERMINATE guard demanded unanimity and failed a good measurement over one sample 0.001 under the floor; it now requires a two-thirds majority. The control that matters: AM-6's constant-cost mutation halves throughput and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6. AM-8 kept N=10 because the measurement said so. Perturbing the RNG only from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A late-onset divergence is deterministic, not flaky, so it is a control rather than a coin flip. Ten runs live on one scenario (make am8, ~2s) rather than all 25 (47s a build). GameKernel 5b records it. The full run also found AM-4a's own mutation stale since ADR-0008 D3 moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported HARNESS-BROKEN, no score published. The build-free half of that check is now a --self-test assertion, so make all catches the next one. mutation-check clauses may now carry their own verify and mutation, and then the enforced flag is measured rather than declared; a declaration disagreeing with its measurement is refused. make all exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 14:07:08 +02:00
CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind" was right, but it is now measured rather than assumed, and verified red by a property mutation (a 300 MB allocation in the workload). AM-5 is BREACHED on both readings, on the machine the spec names: dev toolchain (default features) 87.0 s [FAIL target <= 60 s] shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s] bnt-lap001, 8 cores — a direct comparison, not a directional one. A row declared "recorded not gated" and never recorded fails its own target by 45% on first measurement. The tool reports and exits 0 because the spec says the row is ungated. Gating it is a spec change needing an ADR; a tool that promotes itself is how a target starts binding without anyone deciding it should. So AM-5 stays unmutatable — for the accurate reason now — and the breach is raised as a maintainer decision: speed the build, move the target by ADR (arguing why 60 s was wrong rather than why 87 s is convenient), or withdraw the row. The measurement itself had a real bug, found only by cross-validation. getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child, so it attributed cargo's memory to the workload and reported 38.2 MB for a run that used 12.3 MB — a 3x over-report that was plausible, passed its target, and would have been published. Fixed with os.wait4, which returns that specific child's rusage, and the self-test now cross-checks against /usr/bin/time -v. That is the false-accusation shape in the measurement layer rather than the mutation layer: an instrument confidently reporting a number it had not earned. Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather than running `cargo clean`, so measuring the metric does not cost several minutes of rebuild afterwards. A metric that punishes its own measurement gets measured once and never again. M-D1-MUT: 6 -> 7 of 14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:59:32 +02:00
# AM-9 peak RSS (fast, gated). AM-5 needs a clean build — see build-time.
runtime-metrics:
$(PY) $(TOOLS)/runtime-metrics.py --fast
# AM-5: clean release build, ~90s, into a throwaway CARGO_TARGET_DIR so the
# working cache survives. Reported not gated per GameKernel §5.
build-time:
$(PY) $(TOOLS)/runtime-metrics.py
CB-WP-0006 T02: instrument AM-2; report AM-3 blocked, with the argument Both rows were unmutatable for the same stated reason. They resolved differently, and the difference is the point. AM-2 is instrumented and enforced — tools/size-metrics.py, in `make all`: AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom) 1,575 impl lines / 58 rules Tests are excluded because AM-2 asks what a rule costs, not how much it is exercised; lib.rs is ~18% test code and including it would have flattered the number. This matters because AM-2 is AM-1's anti-gaming pair: 100% rule coverage means nothing if the rules are trivially small, and AM-1 has been reported met since CB-WP-0001 with its pair uninstrumented. Verified red by a property mutation — ~800 lines of filler injected into the impl, pushing the ratio past 40 — not a threshold tweak. The expect string is the precise failure signature "FAIL target <= 40"; my first attempt used "AM-2", which also matches passing output and would have made the FA guard vacuous. AM-3 is BLOCKED, not uninstrumented, and that is a finding rather than a deferral. It measures LOC to express the CB-RES-0001 synthetic game on our kernel, against a boardgame.io baseline of ~36 LOC for a declarative 3p commit/reveal game object. That artifact has never been built: games/ contains only ground, and benches/synthetic.rs drives GROUND rather than defining a synthetic game. Measuring GROUND's 1,575 impl lines against a 36-line synthetic game object would compare two different games and call the difference a D1 result. So the tool ships the measurement — a marker-delimited region, self-tested — and reports the row blocked, naming the missing artifact. A number would have been worse than a blank. It stays unmutatable and still counts against M-D1-MUT per ADR-0005 §1: a row that cannot fail asserts nothing, however good the reason. M-D1-MUT: 5 -> 6 of 14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:38:15 +02:00
# AM-2 (M-D1-SPL, AM-1's anti-gaming pair) and AM-3.
size-metrics:
$(PY) $(TOOLS)/size-metrics.py
T04: tools/cb-cost.py — and its positive control fires on first contact Collector per ADR-0003: enumerates every transcript including the subagents/ tree, dedups by requestId, prices per model and per cache TTL, attributes on (prev_commit, this_commit] intervals, and reconciles to the cent or aborts. The positive control caught a real defect on its very first run against real data, which is the entire argument for writing it: CA-02 assumed usage is identical across the lines of one requestId. True in the main transcript (206/206 groups, verified twice — by the survey and by the adversarial reviewer). FALSE in the subagents/ tree, where output_tokens is a running count: one response reads 5, 5, 195 across its three lines. First-wins scored it at 5. So CA-02 now splits: input-side counters are charged once and must be identical (assertion retained); output_tokens resolves to the max (CA-02a). AC-9 pins the exact 5,5,195 case as a regression test. The acceptance target moved again as a result, for the third time: $248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same first-wins method the tool just disproved, so the tool failing its target was the tool being correct. Target updated, not the tool. Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual $0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10. make cost / cost-test / cost-pin wired; cost-test added to `make all` and to CI, where it gates the collector's assertions without needing transcripts present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
# M-D2-CST (specs/CostAccounting.md). cost-test is the positive control and
# runs first: a cost number from an unverified collector is void.
cost: cost-test
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/cb-cost.py --composition --by-task
T04: tools/cb-cost.py — and its positive control fires on first contact Collector per ADR-0003: enumerates every transcript including the subagents/ tree, dedups by requestId, prices per model and per cache TTL, attributes on (prev_commit, this_commit] intervals, and reconciles to the cent or aborts. The positive control caught a real defect on its very first run against real data, which is the entire argument for writing it: CA-02 assumed usage is identical across the lines of one requestId. True in the main transcript (206/206 groups, verified twice — by the survey and by the adversarial reviewer). FALSE in the subagents/ tree, where output_tokens is a running count: one response reads 5, 5, 195 across its three lines. First-wins scored it at 5. So CA-02 now splits: input-side counters are charged once and must be identical (assertion retained); output_tokens resolves to the max (CA-02a). AC-9 pins the exact 5,5,195 case as a regression test. The acceptance target moved again as a result, for the third time: $248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same first-wins method the tool just disproved, so the tool failing its target was the tool being correct. Target updated, not the tool. Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual $0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10. make cost / cost-test / cost-pin wired; cost-test added to `make all` and to CI, where it gates the collector's assertions without needing transcripts present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
cost-test:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/cb-cost.py --self-test
T04: tools/cb-cost.py — and its positive control fires on first contact Collector per ADR-0003: enumerates every transcript including the subagents/ tree, dedups by requestId, prices per model and per cache TTL, attributes on (prev_commit, this_commit] intervals, and reconciles to the cent or aborts. The positive control caught a real defect on its very first run against real data, which is the entire argument for writing it: CA-02 assumed usage is identical across the lines of one requestId. True in the main transcript (206/206 groups, verified twice — by the survey and by the adversarial reviewer). FALSE in the subagents/ tree, where output_tokens is a running count: one response reads 5, 5, 195 across its three lines. First-wins scored it at 5. So CA-02 now splits: input-side counters are charged once and must be identical (assertion retained); output_tokens resolves to the max (CA-02a). AC-9 pins the exact 5,5,195 case as a regression test. The acceptance target moved again as a result, for the third time: $248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same first-wins method the tool just disproved, so the tool failing its target was the tool being correct. Target updated, not the tool. Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual $0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10. make cost / cost-test / cost-pin wired; cost-test added to `make all` and to CI, where it gates the collector's assertions without needing transcripts present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
T01: audit every InnerLoop rule, and make the checkable ones executable 41 rules classified executable / checkable / decorative, each tagged with the failure class it catches. Counts: 11 executable, 22 checkable, 4 decorative (one of them dead policy). Audit: history/260731-inner-loop-rule-audit.md New tools/loop-lint.py makes 7 rules executable (tier declared, chaos roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file loadability, reporting tools expose --self-test). It found three real violations on its first run, none previously visible: - specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit the loop has stated since v0.2 and never measured. Split at its own section boundaries into Blueprint (1-8) + Runtime (9-15). - tools/dep-weight.py and tools/rule-coverage.py had positive-control logic and no --self-test, so nothing verified the control worked. Adding rule-coverage's self-test exposed a latent instance of the exact class this workplan is about: if the spec regex stopped matching, rules was empty, missing was empty, and the tool exited 0 reporting "0/0" -- a silent pass, in the tool that reports our headline AM-1 number. Both tools now assert they found something before reporting. Two demotions applied in the spec rather than left implicit: "structured over prose" is marked guidance (nothing can check it), and the 8k/10k token budget is struck through and marked DEAD POLICY pointing at T05. The audit's uncomfortable finding: rule 13 (re-derive inherited numbers) has no mechanical form, is deliberately left decorative, and caught the LARGEST error in CB-WP-0002. That is a counter-example to this workplan's own hypothesis. "A rule that cannot be executed is not a rule" is wrong as stated; the defensible version is that such a rule cannot be relied on to fire, so it must not be the only defence for a class that matters. Class coverage: harness-does-nothing has five executable rules; trusted-arithmetic has ZERO and produced the largest single error. make loop-lint and make self-tests wired into `make all` and CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
# InnerLoop rules that are mechanically checkable (CB-WP-0003 T01).
loop-lint:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/loop-lint.py
T01: audit every InnerLoop rule, and make the checkable ones executable 41 rules classified executable / checkable / decorative, each tagged with the failure class it catches. Counts: 11 executable, 22 checkable, 4 decorative (one of them dead policy). Audit: history/260731-inner-loop-rule-audit.md New tools/loop-lint.py makes 7 rules executable (tier declared, chaos roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file loadability, reporting tools expose --self-test). It found three real violations on its first run, none previously visible: - specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit the loop has stated since v0.2 and never measured. Split at its own section boundaries into Blueprint (1-8) + Runtime (9-15). - tools/dep-weight.py and tools/rule-coverage.py had positive-control logic and no --self-test, so nothing verified the control worked. Adding rule-coverage's self-test exposed a latent instance of the exact class this workplan is about: if the spec regex stopped matching, rules was empty, missing was empty, and the tool exited 0 reporting "0/0" -- a silent pass, in the tool that reports our headline AM-1 number. Both tools now assert they found something before reporting. Two demotions applied in the spec rather than left implicit: "structured over prose" is marked guidance (nothing can check it), and the 8k/10k token budget is struck through and marked DEAD POLICY pointing at T05. The audit's uncomfortable finding: rule 13 (re-derive inherited numbers) has no mechanical form, is deliberately left decorative, and caught the LARGEST error in CB-WP-0002. That is a counter-example to this workplan's own hypothesis. "A rule that cannot be executed is not a rule" is wrong as stated; the defensible version is that such a rule cannot be relied on to fire, so it must not be the only defence for a class that matters. Class coverage: harness-does-nothing has five executable rules; trusted-arithmetic has ZERO and produced the largest single error. make loop-lint and make self-tests wired into `make all` and CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
# Positive control for every reporting tool, per InnerLoop v1.1 Step 5.
self-tests:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/cb-cost.py --self-test
$(PY) $(TOOLS)/loop-lint.py --self-test
$(PY) $(TOOLS)/rule-coverage.py --self-test
$(PY) $(TOOLS)/dep-weight.py --self-test
$(PY) $(TOOLS)/repo.py --self-test
CB-WP-0004 T02: make task-done — close a task on measured numbers Replaces the three hand-done steps of a task close (46 turns, $11.52 per CB-RES-0003): the heredoc flipping status in the workplan file, the hand-written hub call, and the hand-typed token counts. The third is the reason this task exists. Every update_task_status this repo produced carried estimated tokens_in/tokens_out — in a project whose central finding is that estimated token counts are worthless. task-done reads the measured figure from the transcripts, or refuses; there is no path through it that emits an estimate. cb-cost gains by_task_detail: cost, response count, model histogram and token components per task. task-done imports cb-cost rather than parsing its printed table, so the hub figure is not a copy that can drift from its source. The positive control found a real defect before the tool ran once. Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002 T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x overstatement would have been pushed to the hub as a *measured* number — the same fiction in a new form. task_label() now keys qualified subjects on the full id and leaves unqualified ones bare rather than retro-assigning them to a workplan. The pinned $93.15 benchmark is unchanged, so historical attribution was not disturbed. Fourth instance of trusted arithmetic: a number believed because a program produced it rather than a hand. Refusals, all exercised by --self-test: unknown id, typo'd id, already-done task, missing state_hub_task_id, no measured spend, and a status flip that produced no change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:17:51 +02:00
$(PY) $(TOOLS)/task-done.py --self-test
CB-WP-0004 T03: make status — one-shot orientation 49 turns and $6.87 per CB-RES-0003 went to grep/ls/wc answering "what is the state of this repo". tools/status.py answers it in 22 lines: commit and working-tree state, every workplan with task counts, the next todo task, spend since the last commit against CB-01/CB-02, and the three fast gates. Two constraints are deliberate. It does not build — a status command that takes two minutes gets replaced by `ls` within a day. And it states its own limit in the output ("slow gates not run here"), so a green status cannot be misread as a green `make all`; the self-test asserts that line is present, and that the whole report stays under 40 lines. The positive control is the refusal to be confidently empty: a parser that found zero workplans and zero tasks would print a clean, wrong picture, which is worse than the greps it replaces. It also caught a dot-all regex that returned twenty paragraphs of the preceding task's prose as the "task heading". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:20:27 +02:00
$(PY) $(TOOLS)/status.py --self-test
CB-WP-0004 T04: fact registry and make facts-check — DFD gets a gate Duplicated-fact drift is the fourth error class and the only one with no executable rule. No positive control catches it (both copies are internally consistent) and re-derivation does not either (the copy reproduces whatever it was copied from). It is caught only by reading a copy against its source, which nothing in the loop required. facts.toml holds 15 facts and is GENERATED by `make facts-gen` from cb-cost, dep-weight and rule-coverage. The trap this task named — a hand-maintained registry that becomes another drifting copy — is closed by facts-check re-running the instruments and failing when the committed registry disagrees with them. A stale registry cannot certify stale artifacts. An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key. 17 occurrences across 5 artifacts are now checked. Falsified before being believed: changing CostAccounting.md line 158 from $93.15 to $92.87 — the exact historical drift — produced exit 1 naming the file, the line and the expected value. Tested against the class it exists to catch, on a real artifact, not only in its self-test. It then caught a live tag inside its own documentation example in InnerLoop.md within the hour. Third time a gate has failed on its own pass's work. What it does not close is stated rather than implied: 22 untagged literal copies remain and are reported, not failed. Tagging is opt-in, a number can legitimately recur, and a gate that fires on coincidence gets routed around. Naming the uncovered surface beats claiming the class is closed. InnerLoop single-source-of-fact moves from prose to executable — v1.3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:24:39 +02:00
$(PY) $(TOOLS)/facts.py --self-test
CB-WP-0005 T02: M-D1-MUT — 4 of 14 acceptance rows are enforced For each acceptance row in GameKernel §5, invert the property and require the verifying command to go red. adapted:mutation-testing, with the denominator changed from source lines to acceptance rows. M-D1-MUT: 4/14 rows enforced PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert) unmutatable 8 (no property to invert, reason stated per row) SURVIVED 0 Two corrections to our own numbers. There are 14 rows, not the twelve ADR-0005 and CB-WP-0005 both asserted — AM-4 splits into a/b/c. And the prediction of 9-of-12 (75%) becomes >=10 of 14; measured 4 (29%), badly unmet. No target moved in this commit. The second correction matters more. My first run reported two SURVIVED rows and both were my own no-op mutations: `pub struct NullRng;` -> `pub struct NullRng {}` is semantically identical, and renaming max_age_days does nothing because CA-17 reads it with a default of 90. Both would have been published as "this row asserts nothing" — a false accusation against code that is fine. Replaced with real inversions (a per-construction counter in the ChaCha seed; reverting AC-9's output resolution to the first-wins bug it was fixed for), after which both go red. T08 asks whether writing a weak mutation is the new grep. It is, demonstrably, on the first attempt. The finding is larger than the workplan assumed. 8 of 14 rows are unmutatable — AM-2, AM-3, AM-4c, AM-5, AM-6, AM-9, AM-10, AM-11 have no instrument at all. AM-6 is the sharpest: nothing in the workspace compares any number to 100,000 events/s, the headline throughput claim. The problem is not three unimplemented rules, it is that more than half the acceptance table has nothing behind it. Harness controls: a stale find-string reports HARNESS-BROKEN rather than scoring the baseline as the mutant; a red baseline reports inconclusive rather than red; the tree is restored in a finally and the restoration is verified. Not in `make all` — it rebuilds the workspace once per row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:27:23 +02:00
$(PY) $(TOOLS)/mutation-check.py --self-test
CB-WP-0006 T02: instrument AM-2; report AM-3 blocked, with the argument Both rows were unmutatable for the same stated reason. They resolved differently, and the difference is the point. AM-2 is instrumented and enforced — tools/size-metrics.py, in `make all`: AM-2: 27.2 LOC/rule [ok target <= 40] (1.47x headroom) 1,575 impl lines / 58 rules Tests are excluded because AM-2 asks what a rule costs, not how much it is exercised; lib.rs is ~18% test code and including it would have flattered the number. This matters because AM-2 is AM-1's anti-gaming pair: 100% rule coverage means nothing if the rules are trivially small, and AM-1 has been reported met since CB-WP-0001 with its pair uninstrumented. Verified red by a property mutation — ~800 lines of filler injected into the impl, pushing the ratio past 40 — not a threshold tweak. The expect string is the precise failure signature "FAIL target <= 40"; my first attempt used "AM-2", which also matches passing output and would have made the FA guard vacuous. AM-3 is BLOCKED, not uninstrumented, and that is a finding rather than a deferral. It measures LOC to express the CB-RES-0001 synthetic game on our kernel, against a boardgame.io baseline of ~36 LOC for a declarative 3p commit/reveal game object. That artifact has never been built: games/ contains only ground, and benches/synthetic.rs drives GROUND rather than defining a synthetic game. Measuring GROUND's 1,575 impl lines against a 36-line synthetic game object would compare two different games and call the difference a D1 result. So the tool ships the measurement — a marker-delimited region, self-tested — and reports the row blocked, naming the missing artifact. A number would have been worse than a blank. It stays unmutatable and still counts against M-D1-MUT per ADR-0005 §1: a row that cannot fail asserts nothing, however good the reason. M-D1-MUT: 5 -> 6 of 14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:38:15 +02:00
$(PY) $(TOOLS)/size-metrics.py --self-test
CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind" was right, but it is now measured rather than assumed, and verified red by a property mutation (a 300 MB allocation in the workload). AM-5 is BREACHED on both readings, on the machine the spec names: dev toolchain (default features) 87.0 s [FAIL target <= 60 s] shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s] bnt-lap001, 8 cores — a direct comparison, not a directional one. A row declared "recorded not gated" and never recorded fails its own target by 45% on first measurement. The tool reports and exits 0 because the spec says the row is ungated. Gating it is a spec change needing an ADR; a tool that promotes itself is how a target starts binding without anyone deciding it should. So AM-5 stays unmutatable — for the accurate reason now — and the breach is raised as a maintainer decision: speed the build, move the target by ADR (arguing why 60 s was wrong rather than why 87 s is convenient), or withdraw the row. The measurement itself had a real bug, found only by cross-validation. getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child, so it attributed cargo's memory to the workload and reported 38.2 MB for a run that used 12.3 MB — a 3x over-report that was plausible, passed its target, and would have been published. Fixed with os.wait4, which returns that specific child's rusage, and the self-test now cross-checks against /usr/bin/time -v. That is the false-accusation shape in the measurement layer rather than the mutation layer: an instrument confidently reporting a number it had not earned. Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather than running `cargo clean`, so measuring the metric does not cost several minutes of rebuild afterwards. A metric that punishes its own measurement gets measured once and never again. M-D1-MUT: 6 -> 7 of 14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:59:32 +02:00
$(PY) $(TOOLS)/runtime-metrics.py --self-test
CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had no flag parsing at all, so --replay had nowhere to go. The bundle is manifest + commands.log + initial.snapshot + expected.yaml, dev-only behind the scenarios feature and charged to AM-4b. The command stream goes through the K11 framing built in T05, so a truncated bundle is detected rather than replayed short — the two tasks compose rather than duplicating. The reviewer's D2 correction was real: this was not "a directory of four files". Pass carried only the end state, RunOutcome::Failed was a formatted String, and scenario.rs created an EventLog, appended to it and never read it. All three had to change. The first round trip failed to reproduce, and the cause is worth keeping: state_hash_hex over a serde_json::Value is a different canonical form than over the typed aggregate — Value's map is key-sorted, a struct serializes in declaration order. The bundle was written with one basis and verified with the other. A round trip written to recompute its own comparison value would have PASSED this bug; it failed because the recorded hash came from the producing process, which is control 2's entire purpose. make replay-test implements ADR-0005 §6's four controls, 14/14: a committed deliberately-failing fixture outside the corpus with covers: [] so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash must fail; a log short by one byte and a corrupted length prefix must be rejected; and a mutated manifest seed must fail — which bites only because replay re-derives the initial state from seed+setup and checks it against the recorded snapshot, since restoring from the snapshot alone would leave the seed inert. Plus a control on the controls: the bundle must still replay after every mutation is reverted. AM-7's hash-identical clause is re-earned. The probe records a hash per per-game segment and replays each from its own genesis; folding from the wrong seed now fails. That is the clause ADR-0005 §4 withdrew as mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so AM-7 stays PARTIAL — reported, not rounded up. Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's copy of that number going stale, on a number that moved the same hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 11:05:37 +02:00
$(PY) $(TOOLS)/replay-test.py --self-test
CB-WP-0022 T05/T06/T07: the register, and what its first run found T05. tools/design.py, make design, and the register in GroundRules.md -- 14 rows, no new file, because ADR-0012 D2 made §Underdetermined the register rather than building one beside it. Backfill was the test and it caught two things the ADR did not have. First, a `role` column. The first report alarmed on U2 and was wrong to: U2's scenario is green BECAUSE the provisional default it documents is implemented, which says nothing about whether ground-game agrees. GR-E01's was a counterexample that went green. Same colour, opposite meaning -- a register that cannot tell them apart either alarms constantly or never. Only a green counterexample alarms. Folded back into GameDesign §1.3. Second, it contradicted the survey. CB-RES-0007 said "six of the ten already have provisional scenarios." Measured -- grep -lE "\bU<n>\b" over scenarios/ground -- exactly ONE U-item names itself. Five provisional scenarios exist and four probably encode U-item defaults, but the mapping is not written down, so it is not checkable. Same defect class as the wrong premises, found inside the survey that proposed the fix. Now a reported debt: open, lacking a reproduction: 9, target 0. design.py carries the control design-baseline.py never had, asserted directly: a row citing a nonexistent file must not count as reproduced, using the exact path 2da19a4 deleted -- which the old tool called green. design-baseline.py is marked superseded rather than deleted; it is the evidence for how a wrong number got into a survey. T06. The report is a FILE in ground-game under GROUND-WP-0002, committed there, with a hub message that only points at it. It asks for no ruling: it carries GR-E01's withdrawal, our reproduction debt, and two notes that are explicitly not findings. And it had to acknowledge something nobody anticipated. GROUND-WP-0002 is finished -- all ten U-items were RULED 2026-08-03, every one confirmed as the default we simulate, plus five of six provisional scenarios. The survey said "0 of 10 ruled" two days later and this register was built saying `reported`. That is the unread-inbox failure running in the opposite direction: they answered and we did not collect it. The instrument's first run surfaced it. They are `ruled`, not `applied` -- lifting the now-settled provisional flags is owed and is not done, and make design shows them open until it is. T07. evidence/CB-EV-0021. Two of six catches in this pass came from execution rather than process (the role distinction from building it, the ten uncollected rulings from running it), which is InnerLoop §Design goal's prediction holding. make self-tests, facts-check, loop-lint: clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 15:22:33 +02:00
$(PY) $(TOOLS)/design.py --self-test
CB-WP-0021 T01/T02/T05: the engine plays its own data — AM-7 blocks ADR-0011 decided it: vendor the CSV with a checked digest, read it with a ~50-line reader, and let the hashes move. The declaration's constraint was measured against the WRONG BUDGET. It said a CSV crate costs 21,613 against AM-4a's 3,798 of headroom, '5.7x over, settled by measurement'. But setup and problem_priorities are cfg(scenarios) and are not in the shipped runtime at all, so AM-4a never sees them. Against AM-4b, csv costs 17,651 against 19,742 -- it FITS, with 2,091 to spare. It is refused anyway, on proportion: 89% of the budget's remaining capacity to read 20 rows. The revisit condition is stated (nested quoting, embedded newlines, multiple dialects). GR-S01 now deals Surface + hidden 1..=k as ruled, with edition values and suits. Measured: 6/9/12 available against thresholds 5/7/9 -- the game is winnable at every seat count, which is what the maintainer could not do. gd0001 is INVERTED, not deleted, and now also asserts the 6/9/12 so a deal that is reachable for the wrong reason still fails. Blast radius was scenario expectations, exactly as the ADR predicted: no scenario pinned a hash and no bundle is committed. Six scenarios and two unit tests updated, each with a note. gr-e01-threshold-unreachable-2p is RENAMED to -reachable- and rewritten as the non-provisional import check ground-game asked for by name. gr-e03's setup was restructured, not just renumbered: with values 2,2,2 its personal-edge test would have tied three ways and asserted nothing. BLOCKING: AM-7 fails at median 0.845 against its 0.9 floor. Isolated across three runs -- 3 problems + stand-in 0.97, 3 problems + edition 0.909, 4 problems + edition 0.845. State is BOUNDED (proven: identical after 5k and 100k events), so this is not the unbounded-growth defect AM-7 exists to catch; it is a bigger working set streaming a long log. Whether AM-7's floor is still right for a larger aggregate is a spec question and lowering it requires an ADR, so it is not being tuned here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:47:56 +02:00
$(PY) $(TOOLS)/edition-check.py --self-test
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
# T01 positive control: prove the environment fix, do not assume it. Runs
# every tool from a foreign working directory with a PATH that has no
# cargo on it. Before T01 this failed; if it fails again, the friction is
# back and CB-EV-0003's measurement is invalid.
env-test:
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/repo.py --self-test
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/rule-coverage.py --self-test >/dev/null \
&& echo " [ok ] rule-coverage runs from / with no cargo on PATH"
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/dep-weight.py --self-test >/dev/null \
&& echo " [ok ] dep-weight runs from / with no cargo on PATH"
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/cb-cost.py --self-test >/dev/null \
&& echo " [ok ] cb-cost runs from / with no cargo on PATH"
@cd / && env PATH=/usr/bin:/bin $(PY) $(TOOLS)/loop-lint.py --self-test >/dev/null \
&& echo " [ok ] loop-lint runs from / with no cargo on PATH"
@$(MAKE) -C $(REPO) coverage >/dev/null \
&& echo " [ok ] make -C <repo> works from any directory"
T01: audit every InnerLoop rule, and make the checkable ones executable 41 rules classified executable / checkable / decorative, each tagged with the failure class it catches. Counts: 11 executable, 22 checkable, 4 decorative (one of them dead policy). Audit: history/260731-inner-loop-rule-audit.md New tools/loop-lint.py makes 7 rules executable (tier declared, chaos roll recorded, tier-L review trail, unmeasured-in-evidence, whole-file loadability, reporting tools expose --self-test). It found three real violations on its first run, none previously visible: - specs/ArchitectureBlueprint.md was 543 lines against a ~400 limit the loop has stated since v0.2 and never measured. Split at its own section boundaries into Blueprint (1-8) + Runtime (9-15). - tools/dep-weight.py and tools/rule-coverage.py had positive-control logic and no --self-test, so nothing verified the control worked. Adding rule-coverage's self-test exposed a latent instance of the exact class this workplan is about: if the spec regex stopped matching, rules was empty, missing was empty, and the tool exited 0 reporting "0/0" -- a silent pass, in the tool that reports our headline AM-1 number. Both tools now assert they found something before reporting. Two demotions applied in the spec rather than left implicit: "structured over prose" is marked guidance (nothing can check it), and the 8k/10k token budget is struck through and marked DEAD POLICY pointing at T05. The audit's uncomfortable finding: rule 13 (re-derive inherited numbers) has no mechanical form, is deliberately left decorative, and caught the LARGEST error in CB-WP-0002. That is a counter-example to this workplan's own hypothesis. "A rule that cannot be executed is not a rule" is wrong as stated; the defensible version is that such a rule cannot be relied on to fire, so it must not be the only defence for a class that matters. Class coverage: harness-does-nothing has five executable rules; trusted-arithmetic has ZERO and produced the largest single error. make loop-lint and make self-tests wired into `make all` and CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:16:00 +02:00
CB-WP-0005 T02: M-D1-MUT — 4 of 14 acceptance rows are enforced For each acceptance row in GameKernel §5, invert the property and require the verifying command to go red. adapted:mutation-testing, with the denominator changed from source lines to acceptance rows. M-D1-MUT: 4/14 rows enforced PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert) unmutatable 8 (no property to invert, reason stated per row) SURVIVED 0 Two corrections to our own numbers. There are 14 rows, not the twelve ADR-0005 and CB-WP-0005 both asserted — AM-4 splits into a/b/c. And the prediction of 9-of-12 (75%) becomes >=10 of 14; measured 4 (29%), badly unmet. No target moved in this commit. The second correction matters more. My first run reported two SURVIVED rows and both were my own no-op mutations: `pub struct NullRng;` -> `pub struct NullRng {}` is semantically identical, and renaming max_age_days does nothing because CA-17 reads it with a default of 90. Both would have been published as "this row asserts nothing" — a false accusation against code that is fine. Replaced with real inversions (a per-construction counter in the ChaCha seed; reverting AC-9's output resolution to the first-wins bug it was fixed for), after which both go red. T08 asks whether writing a weak mutation is the new grep. It is, demonstrably, on the first attempt. The finding is larger than the workplan assumed. 8 of 14 rows are unmutatable — AM-2, AM-3, AM-4c, AM-5, AM-6, AM-9, AM-10, AM-11 have no instrument at all. AM-6 is the sharpest: nothing in the workspace compares any number to 100,000 events/s, the headline throughput claim. The problem is not three unimplemented rules, it is that more than half the acceptance table has nothing behind it. Harness controls: a stale find-string reports HARNESS-BROKEN rather than scoring the baseline as the mutant; a red baseline reports inconclusive rather than red; the tree is restored in a finally and the restoration is verified. Not in `make all` — it rebuilds the workspace once per row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:27:23 +02:00
# M-D1-MUT (CB-WP-0005 T02): invert each acceptance row's property and
# require the verifying command to go red. Deliberately NOT in `make all`:
# it rebuilds the workspace once per mutated row. Run it on demand and in
# CI, not in the inner loop.
mutation-check:
$(PY) $(TOOLS)/mutation-check.py $(ARGS)
CB-WP-0004 T04: fact registry and make facts-check — DFD gets a gate Duplicated-fact drift is the fourth error class and the only one with no executable rule. No positive control catches it (both copies are internally consistent) and re-derivation does not either (the copy reproduces whatever it was copied from). It is caught only by reading a copy against its source, which nothing in the loop required. facts.toml holds 15 facts and is GENERATED by `make facts-gen` from cb-cost, dep-weight and rule-coverage. The trap this task named — a hand-maintained registry that becomes another drifting copy — is closed by facts-check re-running the instruments and failing when the committed registry disagrees with them. A stale registry cannot certify stale artifacts. An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key. 17 occurrences across 5 artifacts are now checked. Falsified before being believed: changing CostAccounting.md line 158 from $93.15 to $92.87 — the exact historical drift — produced exit 1 naming the file, the line and the expected value. Tested against the class it exists to catch, on a real artifact, not only in its self-test. It then caught a live tag inside its own documentation example in InnerLoop.md within the hour. Third time a gate has failed on its own pass's work. What it does not close is stated rather than implied: 22 untagged literal copies remain and are reported, not failed. Tagging is opt-in, a number can legitimately recur, and a gate that fires on coincidence gets routed around. Naming the uncovered surface beats claiming the class is closed. InnerLoop single-source-of-fact moves from prose to executable — v1.3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:24:39 +02:00
# T04: single source of fact (InnerLoop v1.2) — the DFD gate.
# facts.toml is GENERATED; facts-check fails if it disagrees with the
# instruments, or if a tagged artifact disagrees with it.
facts-check:
$(PY) $(TOOLS)/facts.py --check
facts-gen:
$(PY) $(TOOLS)/facts.py --gen
CB-WP-0022 T05/T06/T07: the register, and what its first run found T05. tools/design.py, make design, and the register in GroundRules.md -- 14 rows, no new file, because ADR-0012 D2 made §Underdetermined the register rather than building one beside it. Backfill was the test and it caught two things the ADR did not have. First, a `role` column. The first report alarmed on U2 and was wrong to: U2's scenario is green BECAUSE the provisional default it documents is implemented, which says nothing about whether ground-game agrees. GR-E01's was a counterexample that went green. Same colour, opposite meaning -- a register that cannot tell them apart either alarms constantly or never. Only a green counterexample alarms. Folded back into GameDesign §1.3. Second, it contradicted the survey. CB-RES-0007 said "six of the ten already have provisional scenarios." Measured -- grep -lE "\bU<n>\b" over scenarios/ground -- exactly ONE U-item names itself. Five provisional scenarios exist and four probably encode U-item defaults, but the mapping is not written down, so it is not checkable. Same defect class as the wrong premises, found inside the survey that proposed the fix. Now a reported debt: open, lacking a reproduction: 9, target 0. design.py carries the control design-baseline.py never had, asserted directly: a row citing a nonexistent file must not count as reproduced, using the exact path 2da19a4 deleted -- which the old tool called green. design-baseline.py is marked superseded rather than deleted; it is the evidence for how a wrong number got into a survey. T06. The report is a FILE in ground-game under GROUND-WP-0002, committed there, with a hub message that only points at it. It asks for no ruling: it carries GR-E01's withdrawal, our reproduction debt, and two notes that are explicitly not findings. And it had to acknowledge something nobody anticipated. GROUND-WP-0002 is finished -- all ten U-items were RULED 2026-08-03, every one confirmed as the default we simulate, plus five of six provisional scenarios. The survey said "0 of 10 ruled" two days later and this register was built saying `reported`. That is the unread-inbox failure running in the opposite direction: they answered and we did not collect it. The instrument's first run surfaced it. They are `ruled`, not `applied` -- lifting the now-settled provisional flags is owed and is not done, and make design shows them open until it is. T07. evidence/CB-EV-0021. Two of six catches in this pass came from execution rather than process (the role distinction from building it, the ten uncollected rulings from running it), which is InnerLoop §Design goal's prediction holding. make self-tests, facts-check, loop-lint: clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 15:22:33 +02:00
# CB-WP-0022 T05: the design-finding register, reported over
# specs/GroundRules.md. Shows the QUEUE by default; the log of closed
# findings is a line, not a listing, because a default view that mixes
# them loses the queue property (ADR-0012 D5).
design:
@$(PY) $(TOOLS)/design.py
CB-WP-0004 T03: make status — one-shot orientation 49 turns and $6.87 per CB-RES-0003 went to grep/ls/wc answering "what is the state of this repo". tools/status.py answers it in 22 lines: commit and working-tree state, every workplan with task counts, the next todo task, spend since the last commit against CB-01/CB-02, and the three fast gates. Two constraints are deliberate. It does not build — a status command that takes two minutes gets replaced by `ls` within a day. And it states its own limit in the output ("slow gates not run here"), so a green status cannot be misread as a green `make all`; the self-test asserts that line is present, and that the whole report stays under 40 lines. The positive control is the refusal to be confidently empty: a parser that found zero workplans and zero tasks would print a clean, wrong picture, which is worse than the greps it replaces. It also caught a dot-all regex that returned twenty paragraphs of the preceding task's prose as the "task heading". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:20:27 +02:00
# T03: one-shot orientation — workplans, next task, spend, fast gates.
# Cheap by design: no build. Start a session with this instead of grepping.
status:
@$(PY) $(TOOLS)/status.py
CB-WP-0004 T02: make task-done — close a task on measured numbers Replaces the three hand-done steps of a task close (46 turns, $11.52 per CB-RES-0003): the heredoc flipping status in the workplan file, the hand-written hub call, and the hand-typed token counts. The third is the reason this task exists. Every update_task_status this repo produced carried estimated tokens_in/tokens_out — in a project whose central finding is that estimated token counts are worthless. task-done reads the measured figure from the transcripts, or refuses; there is no path through it that emits an estimate. cb-cost gains by_task_detail: cost, response count, model histogram and token components per task. task-done imports cb-cost rather than parsing its printed table, so the hub figure is not a copy that can drift from its source. The positive control found a real defect before the tool ran once. Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002 T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x overstatement would have been pushed to the hub as a *measured* number — the same fiction in a new form. task_label() now keys qualified subjects on the full id and leaves unqualified ones bare rather than retro-assigning them to a workplan. The pinned $93.15 benchmark is unchanged, so historical attribution was not disturbed. Fourth instance of trusted arithmetic: a number believed because a program produced it rather than a hand. Refusals, all exercised by --self-test: unknown id, typo'd id, already-done task, missing state_hub_task_id, no measured spend, and a status flip that produced no change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:17:51 +02:00
# T02: close a task — flip the workplan file, read the *measured* cost
# from the transcripts, push the hub event with real numbers. Refuses on an
# unknown or already-done task, and refuses to report an estimate.
# make task-done T=CB-WP-0004-T02
task-done:
@test -n "$(T)" || { echo "usage: make task-done T=CB-WP-0004-T02" >&2; exit 2; }
$(PY) $(TOOLS)/task-done.py $(T) $(ARGS)
CB-WP-0007 T01+T03: window the metric, budget it, cap meta at 25% Scope cut first, on the maintainer's decision after a spend review: the project is 38% product / 62% loop-meta, cost per response is 2.9x worse than its best window, and INTENT stage 0 still lacks a CLI player and bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero measured efficiency gain. T02 and T04 are cancelled unstarted. T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and the cumulative figure is retained but labelled "history, NOT the metric". The prediction held decisively — window 655,744 mean context against cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A cumulative mean over 1,094 responses cannot detect a worsening trend because the history outvotes the present. T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the existing SessionShape targets; hard is 1.5x, set before the next measurement per §Step 4. Deliberately not in `make all` — failing the build on context would block committing, and committing is what closes the attribution window and is the natural point to compact, so a gate that blocks the remedy is a trap. It fires HARD on its first run: 656,574 against a 300,000 ceiling. InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare kind: product|meta|mixed and `make status` reports the share; mixed splits 50/50 and says so. Soft on purpose — a task already started may be finished, because stopping mid-task to satisfy a ratio wastes the work. What it forbids is opening new meta work above the line. A pass that exceeds it must say so in its evidence and name the product work displaced. First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because the only product workplan, CB-WP-0001, predates qualified task ids and its bare T## labels collide across passes — stated in the output rather than papered over. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:07 +02:00
# CB-WP-0007 T03 / InnerLoop v1.5: session shape for the window since the
# last commit. Deliberately NOT in `make all` — failing the build on
# context would block committing, and committing is the natural point to
# compact. A gate that blocks the remedy is a trap.
shape-budget: cost-test
$(PY) $(TOOLS)/cb-cost.py --shape-budget
T05: live cost budget replaces the dead token budget The 8k/10k per-task token budget was never referenced or enforced and T08 blew past it silently. Replaced with a budget that can actually fire. The design constraint is the interesting part: per-task cost needs the commit that CLOSES the task, so a per-task budget is unavoidably retrospective -- it can only report a breach after the money is spent, which is the dead-policy failure again. What IS observable mid-task is spend since the last commit, because the transcript is append-live. So the budget binds on the open remainder. CB-01 budget = USD since the last commit, via `make cost-budget` CB-02 soft $10.00 (state progress, decide), hard $22.00 (stop) Calibrated on the 32 non-empty commit intervals of CB-WP-0001: p50 $1.40, p90 $9.36, max $10.80. Soft sits just below the observed maximum -- it would have fired exactly once on the calibration pass. Hard is ~2x the observed max, a value never reached in 32 intervals, so reaching it means the session is doing something the data has no example of. Both thresholds are set ABOVE every observed value, so they bind on future work rather than ratifying present work -- the distinction T07 is about. Stated limit: it is a command, not a daemon. An agent that never runs it gets no signal, which is the dead-policy failure one level up. Mitigated only by being free to run and on the one command surface. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:21:30 +02:00
# CB-01/CB-02: live spend since the last commit.
cost-budget: cost-test
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/cb-cost.py --budget
T05: live cost budget replaces the dead token budget The 8k/10k per-task token budget was never referenced or enforced and T08 blew past it silently. Replaced with a budget that can actually fire. The design constraint is the interesting part: per-task cost needs the commit that CLOSES the task, so a per-task budget is unavoidably retrospective -- it can only report a breach after the money is spent, which is the dead-policy failure again. What IS observable mid-task is spend since the last commit, because the transcript is append-live. So the budget binds on the open remainder. CB-01 budget = USD since the last commit, via `make cost-budget` CB-02 soft $10.00 (state progress, decide), hard $22.00 (stop) Calibrated on the 32 non-empty commit intervals of CB-WP-0001: p50 $1.40, p90 $9.36, max $10.80. Soft sits just below the observed maximum -- it would have fired exactly once on the calibration pass. Hard is ~2x the observed max, a value never reached in 32 intervals, so reaching it means the session is doing something the data has no example of. Both thresholds are set ABOVE every observed value, so they bind on future work rather than ratifying present work -- the distinction T07 is about. Stated limit: it is a command, not a daemon. An agent that never runs it gets no signal, which is the dead-policy failure one level up. Mitigated only by being free to run and on the one command surface. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:21:30 +02:00
CB-RES-0003 + CB-WP-0004: 38% of pass cost is mechanical turns Review of where token-priced turns did work a deterministic tool could do. Method: classify every turn in both transcripts by the tool calls it made. The classifier is committed in tools/cb-cost.py and emitted by `make cost-mix`, so the baseline is reproducible and the same command can later falsify the predictions. mech environment setup 84 turns $15.33 mech ad-hoc text patching 75 turns $13.86 git 37 turns $13.85 mech hub task status 25 turns $ 7.46 mech orientation / inspect 49 turns $ 6.87 hub other 32 turns $ 6.22 mech ad-hoc transcript 39 turns $ 4.56 mech workplan status edit 21 turns $ 4.06 MECHANICAL (dedup) 290 turns $51.26 = 38% of pass Largest category is `cd` and `export PATH` -- pure friction, and dep-weight.py already patched it at the leaf, which is evidence it was noticed and fixed in the wrong place. Second is heredocs string-patching markdown, which is also the mechanism behind duplicated-fact drift, the error class InnerLoop v1.2 names and cannot gate. Explicitly NOT automated: git (37 turns, $13.85) is mostly commit message authorship -- the highest-output turns in the corpus and the project's reasoning record. Automating it would save money and destroy what makes corrections cheap. CB-WP-0004 implements five candidates and predicts $33-41 recovery (25-30%), below the 38% measured share on purpose: some inspection and patching is genuinely exploratory. The control loop is the deliverable, not a formality. T05 tests three things and must report all: did mechanical turns disappear, did they RELOCATE into prose, and did quality hold. If mechanical turns fall and prose rises by as much, the saving is zero and that is the result to publish. Also a self-indictment worth recording: every hub update_task_status in this project carried hand-typed token estimates, in a repo whose central finding is that estimated token counts are worthless. T02 fixes it by reading measured values from cb-cost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:49:57 +02:00
# CB-RES-0003 baseline: mechanical vs judgment turns.
cost-mix: cost-test
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/cb-cost.py --composition
CB-RES-0003 + CB-WP-0004: 38% of pass cost is mechanical turns Review of where token-priced turns did work a deterministic tool could do. Method: classify every turn in both transcripts by the tool calls it made. The classifier is committed in tools/cb-cost.py and emitted by `make cost-mix`, so the baseline is reproducible and the same command can later falsify the predictions. mech environment setup 84 turns $15.33 mech ad-hoc text patching 75 turns $13.86 git 37 turns $13.85 mech hub task status 25 turns $ 7.46 mech orientation / inspect 49 turns $ 6.87 hub other 32 turns $ 6.22 mech ad-hoc transcript 39 turns $ 4.56 mech workplan status edit 21 turns $ 4.06 MECHANICAL (dedup) 290 turns $51.26 = 38% of pass Largest category is `cd` and `export PATH` -- pure friction, and dep-weight.py already patched it at the leaf, which is evidence it was noticed and fixed in the wrong place. Second is heredocs string-patching markdown, which is also the mechanism behind duplicated-fact drift, the error class InnerLoop v1.2 names and cannot gate. Explicitly NOT automated: git (37 turns, $13.85) is mostly commit message authorship -- the highest-output turns in the corpus and the project's reasoning record. Automating it would save money and destroy what makes corrections cheap. CB-WP-0004 implements five candidates and predicts $33-41 recovery (25-30%), below the 38% measured share on purpose: some inspection and patching is genuinely exploratory. The control loop is the deliverable, not a formality. T05 tests three things and must report all: did mechanical turns disappear, did they RELOCATE into prose, and did quality hold. If mechanical turns fall and prose rises by as much, the saving is zero and that is the result to publish. Also a self-indictment worth recording: every hub update_task_status in this project carried hand-typed token estimates, in a repo whose central finding is that estimated token counts are worthless. T02 fixes it by reading measured values from cb-cost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:49:57 +02:00
T04: tools/cb-cost.py — and its positive control fires on first contact Collector per ADR-0003: enumerates every transcript including the subagents/ tree, dedups by requestId, prices per model and per cache TTL, attributes on (prev_commit, this_commit] intervals, and reconciles to the cent or aborts. The positive control caught a real defect on its very first run against real data, which is the entire argument for writing it: CA-02 assumed usage is identical across the lines of one requestId. True in the main transcript (206/206 groups, verified twice — by the survey and by the adversarial reviewer). FALSE in the subagents/ tree, where output_tokens is a running count: one response reads 5, 5, 195 across its three lines. First-wins scored it at 5. So CA-02 now splits: input-side counters are charged once and must be identical (assertion retained); output_tokens resolves to the max (CA-02a). AC-9 pins the exact 5,5,195 case as a regression test. The acceptance target moved again as a result, for the third time: $248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same first-wins method the tool just disproved, so the tool failing its target was the tool being correct. Target updated, not the tool. Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual $0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10. make cost / cost-test / cost-pin wired; cost-test added to `make all` and to CI, where it gates the collector's assertions without needing transcripts present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
cost-pin: cost-test
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(PY) $(TOOLS)/cb-cost.py --pin fc76445 --composition --by-task
T04: tools/cb-cost.py — and its positive control fires on first contact Collector per ADR-0003: enumerates every transcript including the subagents/ tree, dedups by requestId, prices per model and per cache TTL, attributes on (prev_commit, this_commit] intervals, and reconciles to the cent or aborts. The positive control caught a real defect on its very first run against real data, which is the entire argument for writing it: CA-02 assumed usage is identical across the lines of one requestId. True in the main transcript (206/206 groups, verified twice — by the survey and by the adversarial reviewer). FALSE in the subagents/ tree, where output_tokens is a running count: one response reads 5, 5, 195 across its three lines. First-wins scored it at 5. So CA-02 now splits: input-side counters are charged once and must be identical (assertion retained); output_tokens resolves to the max (CA-02a). AC-9 pins the exact 5,5,195 case as a regression test. The acceptance target moved again as a result, for the third time: $248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same first-wins method the tool just disproved, so the tool failing its target was the tool being correct. Target updated, not the tool. Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual $0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10. make cost / cost-test / cost-pin wired; cost-test added to `make all` and to CI, where it gates the collector's assertions without needing transcripts present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:50:04 +02:00
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
sim:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(IN_REPO) $(CARGO) run -q -p cb-sim -- $(REPO)/scenarios/ground/*.yaml
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
CI: enforce every gate; close the silent-skip holes The gates existed; CI ran half of them and tolerated the failure case. - cb-sim no longer has a "tolerable" non-zero exit. An unregistered game prefix is a failure, and a run in which nothing executed is a failure. Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix would have skipped every scenario while the pipeline stayed green. Verified with a negative control. - CI now runs make coverage (AM-1) and make dep-weight (AM-4), both added after CI was written and neither enforced until now. - dep-weight enforces its targets instead of only reporting them. - CI lints the shipped-runtime configuration separately, so the feature split cannot rot unnoticed. - Dropped the stale `make deps` target, which still measured the retired crate-count metric. The positive-control rule is now executable: CI runs `cargo bench -- --test`, which executes every benchmark once, so a workload that stalls fails the build. That step immediately found a fourth instance of the error class it was written for. The committed replay benchmark was the broken version — an earlier patch never applied, leaving a command sequence that omits Resolve, so every round produced nothing and the log-building loop spun forever. It had never run to completion; the reported AM-7 replay numbers came from a probe test instead. Fixed, given the same positive control as the round loop, and re-measured from the benchmark: 100k events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget. Evidence now reports confidence intervals rather than point estimates, so the 3% regression rule in MetricsAndScenarios is enforceable. The finding worth carrying: writing the positive-control rule into InnerLoop v1.0 did not prevent the next instance. Making it a CI step did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
## Criterion benches (AM-6/AM-7)
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
bench:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(IN_REPO) $(CARGO) bench -p games-ground
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
CI: enforce every gate; close the silent-skip holes The gates existed; CI ran half of them and tolerated the failure case. - cb-sim no longer has a "tolerable" non-zero exit. An unregistered game prefix is a failure, and a run in which nothing executed is a failure. Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix would have skipped every scenario while the pipeline stayed green. Verified with a negative control. - CI now runs make coverage (AM-1) and make dep-weight (AM-4), both added after CI was written and neither enforced until now. - dep-weight enforces its targets instead of only reporting them. - CI lints the shipped-runtime configuration separately, so the feature split cannot rot unnoticed. - Dropped the stale `make deps` target, which still measured the retired crate-count metric. The positive-control rule is now executable: CI runs `cargo bench -- --test`, which executes every benchmark once, so a workload that stalls fails the build. That step immediately found a fourth instance of the error class it was written for. The committed replay benchmark was the broken version — an earlier patch never applied, leaving a command sequence that omits Resolve, so every round produced nothing and the log-building loop spun forever. It had never run to completion; the reported AM-7 replay numbers came from a probe test instead. Fixed, given the same positive control as the round loop, and re-measured from the benchmark: 100k events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget. Evidence now reports confidence intervals rather than point estimates, so the 3% regression rule in MetricsAndScenarios is enforceable. The finding worth carrying: writing the positive-control rule into InnerLoop v1.0 did not prevent the next instance. Making it a CI step did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
## InnerLoop positive control: run every bench once, no measurement.
## Fails if a workload stalls or produces the wrong event count.
bench-test:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
$(IN_REPO) $(CARGO) bench -p games-ground --bench synthetic -- --test
CI: enforce every gate; close the silent-skip holes The gates existed; CI ran half of them and tolerated the failure case. - cb-sim no longer has a "tolerable" non-zero exit. An unregistered game prefix is a failure, and a run in which nothing executed is a failure. Previously CI carried `|| test $? -eq 2`, so renaming a scenario prefix would have skipped every scenario while the pipeline stayed green. Verified with a negative control. - CI now runs make coverage (AM-1) and make dep-weight (AM-4), both added after CI was written and neither enforced until now. - dep-weight enforces its targets instead of only reporting them. - CI lints the shipped-runtime configuration separately, so the feature split cannot rot unnoticed. - Dropped the stale `make deps` target, which still measured the retired crate-count metric. The positive-control rule is now executable: CI runs `cargo bench -- --test`, which executes every benchmark once, so a workload that stalls fails the build. That step immediately found a fourth instance of the error class it was written for. The committed replay benchmark was the broken version — an earlier patch never applied, leaving a command sequence that omits Resolve, so every round produced nothing and the log-building loop spun forever. It had never run to completion; the reported AM-7 replay numbers came from a probe test instead. Fixed, given the same positive control as the round loop, and re-measured from the benchmark: 100k events fold in 2.18ms (95% CI 2.14-2.23), against a 5s budget. Evidence now reports confidence intervals rather than point estimates, so the 3% regression rule in MetricsAndScenarios is enforceable. The finding worth carrying: writing the positive-control rule into InnerLoop v1.0 did not prevent the next instance. Making it a CI step did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 04:02:33 +02:00
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
## AM-2/AM-3 input: source LOC per crate (excludes tests would need tokei)
loc:
CB-WP-0004 T01: fix environment friction at the root CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical category — spent prefixing commands with `cd` and `export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once instead of per-leaf. tools/repo.py resolves the repo root from __file__ and cargo from PATH then the standard rustup locations. Every tool imports ROOT from it, so the repo path is stated once rather than redefined in four files — single source of fact, the rule DFD earned in InnerLoop v1.2. rule-coverage and dep-weight now call enter_root(), which is why their relative paths did not need rewriting one by one. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way, so `make -C <repo> <target>` works from any directory with no prefix. make env-test is the positive control, and is in `make all`: every tool runs from / with PATH=/usr/bin:/bin. Without it this fix could regress silently and invalidate T05's measurement — the whole point of the control loop. dep-weight's "cargo not on PATH" error is kept rather than deleted. It should now be unreachable, and --self-test asserts cargo_bin() resolves unaided; a control that never fires is cheaper than a regression. loop-lint failed on repo.py on its first run — a reporting tool with a positive control but no --self-test entry point. Second time the gate has caught work from its own pass within the hour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:13:52 +02:00
@$(IN_REPO) for d in crates/cb-kernel crates/cb-events crates/cb-game-runtime games/ground tools/cb-sim; do \
T07: Cargo workspace scaffold — cb-kernel/cb-events/cb-game-runtime/games-ground/cb-sim, HashMap deny-lint, scenario format + runner stub, Criterion skeleton, Makefile, CI
2026-07-31 01:57:13 +02:00
printf '%-28s %s\n' $$d "$$(find $$d/src -name '*.rs' | xargs cat | grep -vcE '^\s*(//|$$)')"; \
done
CB-WP-0021 T01/T02/T05: the engine plays its own data — AM-7 blocks ADR-0011 decided it: vendor the CSV with a checked digest, read it with a ~50-line reader, and let the hashes move. The declaration's constraint was measured against the WRONG BUDGET. It said a CSV crate costs 21,613 against AM-4a's 3,798 of headroom, '5.7x over, settled by measurement'. But setup and problem_priorities are cfg(scenarios) and are not in the shipped runtime at all, so AM-4a never sees them. Against AM-4b, csv costs 17,651 against 19,742 -- it FITS, with 2,091 to spare. It is refused anyway, on proportion: 89% of the budget's remaining capacity to read 20 rows. The revisit condition is stated (nested quoting, embedded newlines, multiple dialects). GR-S01 now deals Surface + hidden 1..=k as ruled, with edition values and suits. Measured: 6/9/12 available against thresholds 5/7/9 -- the game is winnable at every seat count, which is what the maintainer could not do. gd0001 is INVERTED, not deleted, and now also asserts the 6/9/12 so a deal that is reachable for the wrong reason still fails. Blast radius was scenario expectations, exactly as the ADR predicted: no scenario pinned a hash and no bundle is committed. Six scenarios and two unit tests updated, each with a note. gr-e01-threshold-unreachable-2p is RENAMED to -reachable- and rewritten as the non-provisional import check ground-game asked for by name. gr-e03's setup was restructured, not just renumbered: with values 2,2,2 its personal-edge test would have tied three ways and asserted nothing. BLOCKING: AM-7 fails at median 0.845 against its 0.9 floor. Isolated across three runs -- 3 problems + stand-in 0.97, 3 problems + edition 0.909, 4 problems + edition 0.845. State is BOUNDED (proven: identical after 5k and 100k events), so this is not the unbounded-growth defect AM-7 exists to catch; it is a bigger working set streaming a long log. Whether AM-7's floor is still right for a larger aggregate is a spec question and lowering it requires an ADR, so it is not being tuned here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:47:56 +02:00
all: check test sim coverage size-metrics runtime-metrics am6 am7 am8 edition-check replay-test dep-weight self-tests env-test facts-check loop-lint bench-test
Reference in a new issue Copy permalink