Scope cut first, on the maintainer's decision after a spend review: the
project is 38% product / 62% loop-meta, cost per response is 2.9x worse
than its best window, and INTENT stage 0 still lacks a CLI player and
bots. CB-WP-0005 and CB-WP-0006 cost ~$74 — 31% of all spend — for zero
measured efficiency gain. T02 and T04 are cancelled unstarted.
T01: SH-1/SH-2/SH-3 now report over the window since the last commit, and
the cumulative figure is retained but labelled "history, NOT the metric".
The prediction held decisively — window 655,744 mean context against
cumulative 255,307, a 2.6x gap against a 20% refutation threshold. A
cumulative mean over 1,094 responses cannot detect a worsening trend
because the history outvotes the present.
T03: `make shape-budget`, modelled on CB-01/CB-02. Soft thresholds are the
existing SessionShape targets; hard is 1.5x, set before the next
measurement per §Step 4. Deliberately not in `make all` — failing the
build on context would block committing, and committing is what closes the
attribution window and is the natural point to compact, so a gate that
blocks the remedy is a trap. It fires HARD on its first run: 656,574
against a 300,000 ceiling.
InnerLoop v1.5 establishes the soft 25% meta budget. Workplans declare
kind: product|meta|mixed and `make status` reports the share; mixed splits
50/50 and says so. Soft on purpose — a task already started may be
finished, because stopping mid-task to satisfy a ratio wastes the work.
What it forbids is opening new meta work above the line. A pass that
exceeds it must say so in its evidence and name the product work
displaced.
First reading: 68% OVER, of $74.22 attributed. Product reads $0.00 because
the only product workplan, CB-WP-0001, predates qualified task ids and its
bare T## labels collide across passes — stated in the output rather than
papered over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The question was whether converting agent work to deterministic tooling
recovers capacity or the work reappears elsewhere. Neither, cleanly:
tooling recovers capacity where it removes the possibility of doing the
work by hand, and recovers nothing where it merely offers a better
option.
Sorting the five candidates by that test reproduces the results exactly.
env-setup and task-done left no manual path — env-test fails the build if
the friction returns, and task-done refuses to emit an estimate, so the
old way means knowingly writing fiction. Both met. facts-check and
`make status` left grep and heredocs one keystroke away. Both missed.
This is the principle ADR-0003 already used to reject explicit session
markers — nothing that depends on discipline gets to be the primary
index. It governs tooling economics too, and CB-RES-0003 did not apply
it.
Stated confidence sorted the extremes and said nothing in the middle,
where every judgement call lives: both high met, both medium missed, and
the single low also met, so it was miscalibrated at that end too. Future
reviews state the mechanism instead — after this fix, is the manual path
impossible, absurd, or merely inferior? — and only the first two may
carry a cost prediction.
The confound is symmetric and does not explain the split: building
repo.py and task-done.py also required inspection and patching, and those
categories still went to zero. It inflates the size of the miss, not its
location.
The most valuable output is not the 6 points of 38 recovered. It is that
the loop made a falsifiable quantitative forecast, was wrong by 4-5x, and
could say exactly which parts were wrong and why.
CB-WP-0004 status -> done.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cb-cost gains --since, so the baseline (--pin 578dcbe, 662 responses,
$135.60) and this pass (--since 578dcbe, 83 responses, $7.82) are two
disjoint windows over the same transcripts. Every verdict is on share of
pass, since the windows differ 17x in size.
Test 1 — did mechanical turns disappear? Partly. Mechanical share fell
38.2% -> 32.4%. Six points against a predicted 25-30; reported unmet, and
no target was moved in this commit.
environment setup 11.5% -> 0.6% (85 turns -> 1) met, decisively
hub + workplan 8.5% -> 0.0% (46 turns -> 0) met
text patching 10.4% -> 9.6% not met
orientation 5.1% -> 21.2% not met, worse
The confound is stated before any defence of the numbers: this is the
pass that built the tools, and the two categories that missed are exactly
the two whose tools were under construction. The clean test is the next
pass, and it is carried forward rather than waived.
Test 2 — did the work relocate? Not into prose. Output tokens per
response fell 896 -> 681. Output's rising share of cost is a shrinking
denominator, not more writing. Cost per response halved and the evidence
refuses to claim it: that is compaction (mean context 232,982 ->
117,822), and attributing it to tooling would repeat CB-WP-0002's
original error in a new direction.
Test 3 — did quality hold? Yes, recorded as explicit judgment. make all
green with two gates that did not exist before, and four findings
surfaced this pass, three caught by controls written this pass — one of
them a 5.2x attribution error that would have reached the hub as a
measured number.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Duplicated-fact drift is the fourth error class and the only one with no
executable rule. No positive control catches it (both copies are
internally consistent) and re-derivation does not either (the copy
reproduces whatever it was copied from). It is caught only by reading a
copy against its source, which nothing in the loop required.
facts.toml holds 15 facts and is GENERATED by `make facts-gen` from
cb-cost, dep-weight and rule-coverage. The trap this task named — a
hand-maintained registry that becomes another drifting copy — is closed
by facts-check re-running the instruments and failing when the committed
registry disagrees with them. A stale registry cannot certify stale
artifacts.
An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key.
17 occurrences across 5 artifacts are now checked.
Falsified before being believed: changing CostAccounting.md line 158 from
$93.15 to $92.87 — the exact historical drift — produced exit 1 naming
the file, the line and the expected value. Tested against the class it
exists to catch, on a real artifact, not only in its self-test.
It then caught a live tag inside its own documentation example in
InnerLoop.md within the hour. Third time a gate has failed on its own
pass's work.
What it does not close is stated rather than implied: 22 untagged literal
copies remain and are reported, not failed. Tagging is opt-in, a number
can legitimately recur, and a gate that fires on coincidence gets routed
around. Naming the uncovered surface beats claiming the class is closed.
InnerLoop single-source-of-fact moves from prose to executable — v1.3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
49 turns and $6.87 per CB-RES-0003 went to grep/ls/wc answering "what is
the state of this repo". tools/status.py answers it in 22 lines: commit
and working-tree state, every workplan with task counts, the next todo
task, spend since the last commit against CB-01/CB-02, and the three fast
gates.
Two constraints are deliberate. It does not build — a status command that
takes two minutes gets replaced by `ls` within a day. And it states its
own limit in the output ("slow gates not run here"), so a green status
cannot be misread as a green `make all`; the self-test asserts that line
is present, and that the whole report stays under 40 lines.
The positive control is the refusal to be confidently empty: a parser
that found zero workplans and zero tasks would print a clean, wrong
picture, which is worse than the greps it replaces. It also caught a
dot-all regex that returned twenty paragraphs of the preceding task's
prose as the "task heading".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both hub events carry measured tokens for the first time in this repo;
--hub-only backfills T01, which was closed in the file before the tool
that closes it existed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the three hand-done steps of a task close (46 turns, $11.52 per
CB-RES-0003): the heredoc flipping status in the workplan file, the
hand-written hub call, and the hand-typed token counts.
The third is the reason this task exists. Every update_task_status this
repo produced carried estimated tokens_in/tokens_out — in a project whose
central finding is that estimated token counts are worthless. task-done
reads the measured figure from the transcripts, or refuses; there is no
path through it that emits an estimate.
cb-cost gains by_task_detail: cost, response count, model histogram and
token components per task. task-done imports cb-cost rather than parsing
its printed table, so the hub figure is not a copy that can drift from
its source.
The positive control found a real defect before the tool ran once.
Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002
T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test
reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x
overstatement would have been pushed to the hub as a *measured* number —
the same fiction in a new form. task_label() now keys qualified subjects
on the full id and leaves unqualified ones bare rather than
retro-assigning them to a workplan. The pinned $93.15 benchmark is
unchanged, so historical attribution was not disturbed.
Fourth instance of trusted arithmetic: a number believed because a
program produced it rather than a hand.
Refusals, all exercised by --self-test: unknown id, typo'd id,
already-done task, missing state_hub_task_id, no measured spend, and a
status flip that produced no change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CB-RES-0003 measured 84 turns and $15.33 — the largest mechanical
category — spent prefixing commands with `cd` and
`export PATH="$HOME/.cargo/bin:$PATH"`. Both causes are now fixed once
instead of per-leaf.
tools/repo.py resolves the repo root from __file__ and cargo from PATH
then the standard rustup locations. Every tool imports ROOT from it, so
the repo path is stated once rather than redefined in four files —
single source of fact, the rule DFD earned in InnerLoop v1.2.
rule-coverage and dep-weight now call enter_root(), which is why their
relative paths did not need rewriting one by one.
The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same
way, so `make -C <repo> <target>` works from any directory with no
prefix.
make env-test is the positive control, and is in `make all`: every tool
runs from / with PATH=/usr/bin:/bin. Without it this fix could regress
silently and invalidate T05's measurement — the whole point of the
control loop.
dep-weight's "cargo not on PATH" error is kept rather than deleted. It
should now be unreachable, and --self-test asserts cargo_bin() resolves
unaided; a control that never fires is cheaper than a regression.
loop-lint failed on repo.py on its first run — a reporting tool with a
positive control but no --self-test entry point. Second time the gate
has caught work from its own pass within the hour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fix-consistency appended state_hub_workstream_id rather than replacing
the empty one, leaving a duplicate YAML key.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review of where token-priced turns did work a deterministic tool could
do. Method: classify every turn in both transcripts by the tool calls it
made. The classifier is committed in tools/cb-cost.py and emitted by
`make cost-mix`, so the baseline is reproducible and the same command
can later falsify the predictions.
mech environment setup 84 turns $15.33
mech ad-hoc text patching 75 turns $13.86
git 37 turns $13.85
mech hub task status 25 turns $ 7.46
mech orientation / inspect 49 turns $ 6.87
hub other 32 turns $ 6.22
mech ad-hoc transcript 39 turns $ 4.56
mech workplan status edit 21 turns $ 4.06
MECHANICAL (dedup) 290 turns $51.26 = 38% of pass
Largest category is `cd` and `export PATH` -- pure friction, and
dep-weight.py already patched it at the leaf, which is evidence it was
noticed and fixed in the wrong place. Second is heredocs string-patching
markdown, which is also the mechanism behind duplicated-fact drift, the
error class InnerLoop v1.2 names and cannot gate.
Explicitly NOT automated: git (37 turns, $13.85) is mostly commit
message authorship -- the highest-output turns in the corpus and the
project's reasoning record. Automating it would save money and destroy
what makes corrections cheap.
CB-WP-0004 implements five candidates and predicts $33-41 recovery
(25-30%), below the 38% measured share on purpose: some inspection and
patching is genuinely exploratory.
The control loop is the deliverable, not a formality. T05 tests three
things and must report all: did mechanical turns disappear, did they
RELOCATE into prose, and did quality hold. If mechanical turns fall and
prose rises by as much, the saving is zero and that is the result to
publish.
Also a self-indictment worth recording: every hub update_task_status in
this project carried hand-typed token estimates, in a repo whose central
finding is that estimated token counts are worthless. T02 fixes it by
reading measured values from cb-cost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
