clay-borg/workplans/CB-WP-0004-mechanical-work.md

id	title	status	state_hub_workstream_id
CB-WP-0004	Move mechanical turns off the token budget, and prove it worked	in_progress	6880ac78-d817-41b9-b267-f12ff9deea28

Purpose

research/CB-RES-0003-agent-vs-deterministic.md measured every turn in both clay-borg sessions by the tool calls it made:

    MECHANICAL (deduplicated)          290 turns  $   51.26  = 38% of pass

38% of spend went through turns doing work a deterministic tool could do. The largest single category is cd and export PATH — 84 turns and $15.33 of pure environment friction. The second is inline heredocs string-patching markdown, which is also the mechanism behind the duplicated-fact-drift error class InnerLoop v1.2 named and could not gate.

This workplan converts the five worthwhile categories to classic compute and measures whether that actually recovered anything. The predicted recovery is $33–41 per pass, 25–30%.

The control loop is the point, not a formality. The named failure mode is relocation: an agent that can no longer write a heredoc may simply write more prose, and the pass costs the same. make cost-mix emits the same categories that produced the baseline, so the claim is falsifiable by the same instrument that made it. A saving that cannot be demonstrated in that table did not happen.

Per InnerLoop v1.2, targets here are provisional until the instrument emits them, and no target may be moved in the commit that measures it unless the instrument disproved it (§Step 4, correction vs retarget).

Phase A — The certain wins

Task: Remove environment friction

id: CB-WP-0004-T01
status: done
priority: high
state_hub_task_id: "3ddfd3e2-8596-4969-a069-09577933fcbc"

84 turns / $15.33, the largest category and the least interesting work in the corpus. cargo is not on the default path, so every Rust-touching command carried export PATH="$HOME/.cargo/bin:$PATH", and the shell does not persist cd.

Fix at the root, not the leaf: tools/dep-weight.py already special-cases the missing cargo with a helpful error, which is evidence the friction was noticed and patched in the wrong place.

Deliver: every make target runs from a clean shell with no prefix, from any directory. Document the one-line environment requirement in README.md if one remains. Remove the leaf workaround in dep-weight.py only if it becomes unreachable — a positive control that never fires is still cheaper than a regression.

Predicted: environment-setup turns → < 10 (from 84), $12–15 recovered. Highest confidence in the review.

Delivered. tools/repo.py resolves the repo root from __file__ and cargo from PATH-then-rustup-locations; every tool imports it, so REPO is now stated once rather than four times. The Makefile derives REPO from MAKEFILE_LIST and resolves CARGO the same way. make env-test is the positive control — it runs every tool from / with PATH=/usr/bin:/bin, and is wired into make all, so this cannot silently regress and invalidate T05's measurement.

The leaf workaround in dep-weight.py was kept, not deleted: it is now unreachable on a rustup machine, and --self-test asserts cargo_bin() resolves unaided. Per the task text, a control that never fires is cheaper than a regression.

loop-lint failed on repo.py immediately — a new tool with a positive control but no --self-test entry point. Second time the gate has caught its own pass's work within the hour.

Task: make task-done — one command for a task close

id: CB-WP-0004-T02
status: done
priority: high
state_hub_task_id: "8a1b59c7-6318-47ba-8cbc-83bb44dd221f"

Merges two categories: workplan status edits (21 turns, $4.06) and hub task-status calls (25 turns, $7.46).

make task-done T=CB-WP-0004-T02 must:

1. flip status: todo → done in the workplan file, failing loudly on an unknown or already-done task — the heredocs it replaces silently no-op'd on a typo;
2. read that task's measured cost and tokens from cb-cost --by-task;
3. push the hub event with the real numbers.

The third point is the one that matters beyond cost. Every update_task_status in this project so far carried hand-typed token estimates, in a repo whose central finding is that estimated token counts are worthless. The hub currently holds fiction produced by the exact habit CB-WP-0002 disproved.

Predicted: those 46 turns → ~6, $9–11 recovered, and the hub stops holding estimates. Add --self-test per InnerLoop v1.1.

Delivered. tools/task-done.py + make task-done T=<id>. It refuses on an unknown id, a typo'd id, an already-done task, a task with no state_hub_task_id, and — the one that matters — a task with no measured spend, rather than reporting an estimate. cb-cost gained by_task_detail (cost, response count, model histogram, and token components per task), and task-done imports cb-cost rather than parsing its printed table, so the hub number is not a copy that can drift.

The positive control found a real defect before the tool was used once. Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002 T01, CB-WP-0003 T01 and CB-WP-0004 T01 all landed in one bucket. The self-test reported $12.10 for "T01"; the qualified figure is $2.33 — a 5.2× overstatement that would have been pushed to the hub as a measured number, reproducing the fiction this task exists to end, in a new form. Fixed by task_label(): qualified subjects (CB-WP-0004 T01) key on the full id, unqualified ones stay bare and are never retro-assigned to a workplan. The pinned $93.15 benchmark is unchanged, confirming historical attribution was not disturbed.

That is the fourth instance of trusted arithmetic (TA) — a number believed because it was produced by a program rather than by hand.

Task: make status — one-shot orientation

id: CB-WP-0004-T03
status: todo
priority: medium
state_hub_task_id: "5466a510-37a5-4491-b93e-509cd400cc23"

49 turns / $6.87 of grep/ls/wc answering "what is the state of this repo". Replace with one command printing: active workplan and task counts, gate results, open spend since the last commit (CB-01), provisional item ages, and any loop-lint findings.

Also shrinks cold-start context, which specs/SessionShape.md §3 measures at ~51k for a fresh session — the artifacts a new session reads to orient are exactly what this prints.

Predicted: ~10 orientation turns → 1 per session, $4 recovered. Confidence medium: some inspection is genuinely exploratory and will not disappear, and the review says so.

Delivered. tools/status.py + make status, 22 lines of output: commit and working-tree state, every workplan with its task counts, the next todo task with its heading, spend since the last commit against CB-01/CB-02, and the three fast gates.

Two deliberate constraints. It does not build — a status command that takes two minutes gets replaced by ls within a day — and it states its own limit in the output (slow gates not run here), so a green status cannot be misread as a green make all. The self-test asserts the output stays under 40 lines, because the failure mode for an orientation tool is becoming another thing to skim.

Its positive control refuses the confidently-empty report: a parser that found zero workplans and zero tasks would print a clean, wrong picture, which is worse than the greps it replaces. That control also caught a dot-all regex that returned twenty paragraphs of a previous task's prose as the "task heading".

Phase B — The one that also closes an error class

Task: Fact registry and make facts-check

id: CB-WP-0004-T04
status: todo
priority: high
state_hub_task_id: "b5dfbf1e-619b-4e64-8463-49ff2ce38b48"

75 turns / $13.86 of heredocs opening a markdown file, string-replacing a number, and writing it back — the mechanism behind duplicated-fact drift, the fourth error class, which InnerLoop v1.2 states as prose and cannot currently gate.

Two instances on record: a price sheet inlined into a spec went stale within an hour of the real sheet changing, and the acceptance figure $92.21 → $92.87 → $93.32 → $93.15 had to be chased across a survey, a workplan, and an evidence file on every move.

Deliver a registry where a number appearing in more than one artifact is declared once — generated by the instrument that measures it wherever possible, not hand-maintained — plus make facts-check failing when a committed artifact disagrees with it.

The trap to avoid, stated up front: a hand-maintained registry moves the problem rather than solving it, and would itself become a copy that drifts. If generation from instruments proves impractical, deliver only the check (detect the same number stated differently in two artifacts) and say so — a gate with no generator still closes the class.

Predicted: $6–9 recovered, plus DFD's first executable gate. Confidence medium; this is the hardest task here and the most valuable.

Phase C — Prove it, or withdraw the claim

Task: Control loop — measure recovery and test for relocation

id: CB-WP-0004-T05
status: todo
priority: high
state_hub_task_id: "922346ac-1e99-4e94-9dc6-35c8fa0eddc5"

The task this workplan exists for. Run make cost-mix over the sessions that executed T01–T04 and commit evidence/CB-EV-0003-mechanical-work.md comparing against the committed baseline:

category	baseline turns	baseline $	predicted	measured	verdict
environment setup	84	$15.33	<10 turns
ad-hoc text patching	75	$13.86	$6–9 saved
hub task status + workplan edit	46	$11.52	~6 turns
orientation / inspect	49	$6.87	$4 saved
mechanical total	290	$51.26	$33–41 saved

Three tests, all of which must be reported:

1. Did the mechanical turns disappear? Per-category, against prediction. An unmet prediction is reported unmet, not retargeted.
2. Did they relocate? Total pass cost and non-mechanical turn counts must be compared too. If mechanical turns fell and prose turns rose by as much, the saving is zero and this workplan failed — that is the result to publish.
3. Did quality hold? make all green, and the same class of findings still surfacing. A cheaper pass that catches fewer errors is worse, and the loop has no metric for this yet — record the judgment explicitly rather than implying the cost number settles it.

Normalize per unit of work, not per session: passes differ in size, so report mechanical share of pass cost (baseline: 38%) alongside absolute dollars.

Task: Retrospective

id: CB-WP-0004-T06
status: todo
priority: low
state_hub_task_id: "47892c23-ef09-4910-a9af-327494289e05"

The question to answer honestly: does converting agent work to deterministic tooling actually recover capacity, or does the work reappear elsewhere?

This is the first pass in the project to make a quantitative prediction before acting. Whether the prediction held is more informative than the saving itself — a loop that can forecast its own economics can plan; one that cannot is guessing with numbers attached.

Record the prediction error per candidate, and whether the review's stated confidence levels (high/medium/low) tracked reality. If they did not, the next review should stop stating confidence, or state it differently.