CB-WP-0004 T06: retrospective — what actually predicted the outcome
The question was whether converting agent work to deterministic tooling recovers capacity or the work reappears elsewhere. Neither, cleanly: tooling recovers capacity where it removes the possibility of doing the work by hand, and recovers nothing where it merely offers a better option. Sorting the five candidates by that test reproduces the results exactly. env-setup and task-done left no manual path — env-test fails the build if the friction returns, and task-done refuses to emit an estimate, so the old way means knowingly writing fiction. Both met. facts-check and `make status` left grep and heredocs one keystroke away. Both missed. This is the principle ADR-0003 already used to reject explicit session markers — nothing that depends on discipline gets to be the primary index. It governs tooling economics too, and CB-RES-0003 did not apply it. Stated confidence sorted the extremes and said nothing in the middle, where every judgement call lives: both high met, both medium missed, and the single low also met, so it was miscalibrated at that end too. Future reviews state the mechanism instead — after this fix, is the manual path impossible, absurd, or merely inferior? — and only the first two may carry a cost prediction. The confound is symmetric and does not explain the split: building repo.py and task-done.py also required inspection and patching, and those categories still went to zero. It inflates the size of the miss, not its location. The most valuable output is not the 6 points of 38 recovered. It is that the loop made a falsifiable quantitative forecast, was wrong by 4-5x, and could say exactly which parts were wrong and why. CB-WP-0004 status -> done. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
7a374d37d2
commit
7b4d9e359f
2 changed files with 148 additions and 2 deletions
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
id: CB-WP-0004
|
||||
title: "Move mechanical turns off the token budget, and prove it worked"
|
||||
status: in_progress
|
||||
status: done
|
||||
state_hub_workstream_id: "6880ac78-d817-41b9-b267-f12ff9deea28"
|
||||
---
|
||||
|
||||
|
|
@ -241,7 +241,7 @@ InnerLoop's single-source-of-fact rule moves from prose to executable —
|
|||
|
||||
```task
|
||||
id: CB-WP-0004-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "922346ac-1e99-4e94-9dc6-35c8fa0eddc5"
|
||||
```
|
||||
|
|
@ -330,3 +330,28 @@ that cannot is guessing with numbers attached.
|
|||
Record the prediction error per candidate, and whether the review's stated
|
||||
confidence levels (high/medium/low) tracked reality. If they did not, the
|
||||
next review should stop stating confidence, or state it differently.
|
||||
|
||||
**Recorded — [260731-mechanical-work-retrospective.md](../history/260731-mechanical-work-retrospective.md).**
|
||||
|
||||
The answer to the question is neither of the two offered:
|
||||
**tooling recovers capacity where it removes the possibility of doing the
|
||||
work by hand, and recovers nothing where it merely offers a better
|
||||
option.** Sorting the five candidates by that test reproduces the results
|
||||
exactly — the two that removed the manual path met their predictions, the
|
||||
two that left `grep` and heredocs one keystroke away did not.
|
||||
|
||||
Confidence sorted the extremes and said nothing in the middle, which is
|
||||
where every judgement call lives: both `high` met, both `medium` missed,
|
||||
and the single `low` also met — miscalibrated at that end too. So the
|
||||
next review **states the mechanism, not a confidence level**: after this
|
||||
fix, is the manual path impossible, absurd, or merely inferior? Only the
|
||||
first two may carry a cost prediction.
|
||||
|
||||
This is the same principle ADR-0003 used to reject explicit session
|
||||
markers — *nothing that depends on discipline gets to be the primary
|
||||
index*. It governs tooling economics too, and this review did not apply
|
||||
it.
|
||||
|
||||
The most valuable output is not the $6 of 38 points recovered. It is that
|
||||
the loop made a falsifiable forecast, was wrong by 4–5×, and could say
|
||||
exactly which parts were wrong and why.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue