CB-WP-0004 T06: retrospective — what actually predicted the outcome

The question was whether converting agent work to deterministic tooling
recovers capacity or the work reappears elsewhere. Neither, cleanly:
tooling recovers capacity where it removes the possibility of doing the
work by hand, and recovers nothing where it merely offers a better
option.

Sorting the five candidates by that test reproduces the results exactly.
env-setup and task-done left no manual path — env-test fails the build if
the friction returns, and task-done refuses to emit an estimate, so the
old way means knowingly writing fiction. Both met. facts-check and
`make status` left grep and heredocs one keystroke away. Both missed.

This is the principle ADR-0003 already used to reject explicit session
markers — nothing that depends on discipline gets to be the primary
index. It governs tooling economics too, and CB-RES-0003 did not apply
it.

Stated confidence sorted the extremes and said nothing in the middle,
where every judgement call lives: both high met, both medium missed, and
the single low also met, so it was miscalibrated at that end too. Future
reviews state the mechanism instead — after this fix, is the manual path
impossible, absurd, or merely inferior? — and only the first two may
carry a cost prediction.

The confound is symmetric and does not explain the split: building
repo.py and task-done.py also required inspection and patching, and those
categories still went to zero. It inflates the size of the miss, not its
location.

The most valuable output is not the 6 points of 38 recovered. It is that
the loop made a falsifiable quantitative forecast, was wrong by 4-5x, and
could say exactly which parts were wrong and why.

CB-WP-0004 status -> done.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 10:28:35 +02:00
parent 7a374d37d2
commit 7b4d9e359f
2 changed files with 148 additions and 2 deletions

View file

@ -1,7 +1,7 @@
---
id: CB-WP-0004
title: "Move mechanical turns off the token budget, and prove it worked"
status: in_progress
status: done
state_hub_workstream_id: "6880ac78-d817-41b9-b267-f12ff9deea28"
---
@ -241,7 +241,7 @@ InnerLoop's single-source-of-fact rule moves from prose to executable —
```task
id: CB-WP-0004-T05
status: todo
status: done
priority: high
state_hub_task_id: "922346ac-1e99-4e94-9dc6-35c8fa0eddc5"
```
@ -330,3 +330,28 @@ that cannot is guessing with numbers attached.
Record the prediction error per candidate, and whether the review's stated
confidence levels (high/medium/low) tracked reality. If they did not, the
next review should stop stating confidence, or state it differently.
**Recorded — [260731-mechanical-work-retrospective.md](../history/260731-mechanical-work-retrospective.md).**
The answer to the question is neither of the two offered:
**tooling recovers capacity where it removes the possibility of doing the
work by hand, and recovers nothing where it merely offers a better
option.** Sorting the five candidates by that test reproduces the results
exactly — the two that removed the manual path met their predictions, the
two that left `grep` and heredocs one keystroke away did not.
Confidence sorted the extremes and said nothing in the middle, which is
where every judgement call lives: both `high` met, both `medium` missed,
and the single `low` also met — miscalibrated at that end too. So the
next review **states the mechanism, not a confidence level**: after this
fix, is the manual path impossible, absurd, or merely inferior? Only the
first two may carry a cost prediction.
This is the same principle ADR-0003 used to reject explicit session
markers — *nothing that depends on discipline gets to be the primary
index*. It governs tooling economics too, and this review did not apply
it.
The most valuable output is not the $6 of 38 points recovered. It is that
the loop made a falsifiable forecast, was wrong by 45×, and could say
exactly which parts were wrong and why.