CB-WP-0004 T02: make task-done — close a task on measured numbers

Replaces the three hand-done steps of a task close (46 turns, $11.52 per
CB-RES-0003): the heredoc flipping status in the workplan file, the
hand-written hub call, and the hand-typed token counts.

The third is the reason this task exists. Every update_task_status this
repo produced carried estimated tokens_in/tokens_out — in a project whose
central finding is that estimated token counts are worthless. task-done
reads the measured figure from the transcripts, or refuses; there is no
path through it that emits an estimate.

cb-cost gains by_task_detail: cost, response count, model histogram and
token components per task. task-done imports cb-cost rather than parsing
its printed table, so the hub figure is not a copy that can drift from
its source.

The positive control found a real defect before the tool ran once.
Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002
T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test
reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x
overstatement would have been pushed to the hub as a *measured* number —
the same fiction in a new form. task_label() now keys qualified subjects
on the full id and leaves unqualified ones bare rather than
retro-assigning them to a workplan. The pinned $93.15 benchmark is
unchanged, so historical attribution was not disturbed.

Fourth instance of trusted arithmetic: a number believed because a
program produced it rather than a hand.

Refusals, all exercised by --self-test: unknown id, typo'd id,
already-done task, missing state_hub_task_id, no measured spend, and a
status flip that produced no change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Makefile

@@ -24,7 +24,7 @@ TOOLS := $(REPO)/tools
 # Every cargo recipe runs at the repo root; the shell does not persist cd.
 IN_REPO := cd $(REPO) &&
 
-.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test loc all
+.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done loc all
 
 ## fmt + clippy (deny warnings) + HashMap deny-lint
 check:
@@ -61,6 +61,7 @@ self-tests:
 	$(PY) $(TOOLS)/rule-coverage.py --self-test
 	$(PY) $(TOOLS)/dep-weight.py --self-test
 	$(PY) $(TOOLS)/repo.py --self-test
+	$(PY) $(TOOLS)/task-done.py --self-test
 
 # T01 positive control: prove the environment fix, do not assume it. Runs
 # every tool from a foreign working directory with a PATH that has no
@@ -79,6 +80,14 @@ env-test:
 	@$(MAKE) -C $(REPO) coverage >/dev/null \
 	  && echo "  [ok ] make -C <repo> works from any directory"
 
+# T02: close a task — flip the workplan file, read the *measured* cost
+# from the transcripts, push the hub event with real numbers. Refuses on an
+# unknown or already-done task, and refuses to report an estimate.
+#   make task-done T=CB-WP-0004-T02
+task-done:
+	@test -n "$(T)" || { echo "usage: make task-done T=CB-WP-0004-T02" >&2; exit 2; }
+	$(PY) $(TOOLS)/task-done.py $(T) $(ARGS)
+
 # CB-01/CB-02: live spend since the last commit.
 cost-budget: cost-test
 	$(PY) $(TOOLS)/cb-cost.py --budget

tools/cb-cost.py

@@ -44,7 +44,26 @@ from repo import ROOT as REPO  # noqa: E402  (single source of fact, T01)
 
 PRICES = os.path.join(REPO, "benchmarks", "baselines", "model-prices.toml")
 COMPONENTS = ("input", "output", "cache_read", "write_5m", "write_1h")
+# CA-08 attribution key. Qualified first: a bare `T\d\d` collides across
+# workplans — CB-WP-0002 T01, CB-WP-0003 T01 and CB-WP-0004 T01 all matched
+# the same bucket, so a per-task figure summed three unrelated tasks. Found
+# by T02's positive control, which reported $12.10 for "T01".
+TASK_QUALIFIED_RE = re.compile(r"\b(CB-WP-\d+)[ -](T\d\d)\b")
 TASK_RE = re.compile(r"\bT\d\d\b")
+
+
+def task_label(subject):
+    """Attribution label for a commit subject, or None.
+
+    Qualified ids (`CB-WP-0004 T01`) become `CB-WP-0004-T01`. Bare ids
+    (`T01: ...`) stay bare — commits predating this convention did not name
+    their workplan and must not be retroactively assigned to one.
+    """
+    m = TASK_QUALIFIED_RE.search(subject)
+    if m:
+        return f"{m.group(1)}-{m.group(2)}"
+    m = TASK_RE.search(subject)
+    return m.group(0) if m else None
 UNATTRIBUTED = "UNATTRIBUTED"
 OPEN_REMAINDER = "OPEN (uncommitted)"
 
@@ -323,8 +342,7 @@ def attribute(responses, commits):
     label_for = []
     prev = ""
     for ts, subject in commits:
-        m = TASK_RE.search(subject)
+        label_for.append((prev, ts, task_label(subject) or UNATTRIBUTED))
-        label_for.append((prev, ts, m.group(0) if m else UNATTRIBUTED))
         prev = ts
     last = commits[-1][0] if commits else ""
 
@@ -369,6 +387,13 @@ def collect(slug, pin_ref=None):
     attribute(responses, commit_index(pin))
 
     by_task = collections.defaultdict(float)
+    # T02: per-task token detail, so a task close can report *measured*
+    # tokens. Every update_task_status in this project before T02 carried
+    # hand-typed estimates — in a repo whose finding is that estimated
+    # token counts are worthless.
+    task_detail = collections.defaultdict(
+        lambda: {"cost": 0.0, "responses": 0, "models": collections.Counter(),
+                 "toks": collections.Counter()})
     by_component = collections.Counter()
     by_component_cost = collections.defaultdict(float)
     by_model = collections.defaultdict(float)
@@ -379,6 +404,11 @@ def collect(slug, pin_ref=None):
             unpriced.append(r)
             continue
         by_task[r["task"]] += r["cost"]
+        d = task_detail[r["task"]]
+        d["cost"] += r["cost"]
+        d["responses"] += 1
+        d["models"][r["model"]] += 1
+        d["toks"].update(r["toks"])
         by_model[r["model"]] += r["cost"]
         rin, rout = rates_at(prices, r["model"], r["timestamp"])
         unit = rin / 1e6
@@ -429,6 +459,27 @@ def collect(slug, pin_ref=None):
         "subagent_total": sub,
         "main_total": total - sub,
         "by_task": dict(by_task),
+        "by_task_detail": {
+            k: {
+                "cost": v["cost"],
+                "responses": v["responses"],
+                # The model that ran the most responses for this task. The
+                # hub schema has one model field per event; a task split
+                # across models cannot be represented faithfully, so the
+                # full count travels in the note.
+                "model": v["models"].most_common(1)[0][0],
+                "models": dict(v["models"]),
+                "toks": dict(v["toks"]),
+                # CA-03/CA-04: the hub has no cache fields. tokens_in is
+                # the honest sum of everything billed on the input side.
+                "tokens_in": (v["toks"].get("input", 0)
+                              + v["toks"].get("cache_read", 0)
+                              + v["toks"].get("write_5m", 0)
+                              + v["toks"].get("write_1h", 0)),
+                "tokens_out": v["toks"].get("output", 0),
+            }
+            for k, v in task_detail.items()
+        },
         "by_model": dict(by_model),
         "tokens": dict(by_component),
         "cost_by_component": dict(by_component_cost),
@@ -566,6 +617,18 @@ def self_test():
     finally:
         os.unlink(partial)
 
+    # CA-08 label collision (found by T02's positive control): three
+    # workplans each had a T01, and a bare-label bucket summed all three
+    # into one figure — $12.10 for a task that cost $2.33. Qualified
+    # subjects must produce distinct buckets.
+    check("CA-08 qualified task ids do not collide across workplans",
+          task_label("CB-WP-0004 T01: fix env") == "CB-WP-0004-T01"
+          and task_label("CB-WP-0002 T01: survey") == "CB-WP-0002-T01"
+          and task_label("CB-WP-0004-T01: hyphenated") == "CB-WP-0004-T01")
+    check("CA-08 unqualified subjects stay bare, not retro-assigned",
+          task_label("T01: cost-accounting survey") == "T01"
+          and task_label("chore: no task here") is None)
+
     # CA-16: a dated promo rate must apply before its expiry and lapse after.
     pr = prices
     before = rates_at(pr, "claude-sonnet-5", "2026-07-31T00:00:00Z")

tools/task-done.py

@@ -0,0 +1,322 @@
+#!/usr/bin/env python3
+"""Close a workplan task: flip the file, read the measured cost, tell the hub.
+
+CB-WP-0004 T02. Replaces three things that were done by hand on every task
+close, at a measured 46 turns and $11.52 per pass (CB-RES-0003 candidates
+3 and 5):
+
+  1. a heredoc doing `s.replace("status: todo", "status: done")` on the
+     workplan file — which silently did nothing on a typo'd task id or an
+     already-closed task;
+  2. a hand-written `update_task_status` hub call;
+  3. **hand-typed `tokens_in` / `tokens_out`**, in a project whose central
+     finding is that estimated token counts are worthless. Every hub token
+     event this repo produced before T02 was an estimate. This tool reads
+     the measured figure from the transcripts via cb-cost, or refuses.
+
+Ordering matters and is enforced, not assumed. Attribution is by commit
+subject (CA-08), so the work commit naming the task must exist *before*
+the close: otherwise the spend still sits in OPEN (uncommitted) and there
+is nothing measured to report. The usual sequence is
+
+    <do the work>
+    git commit -m "CB-WP-0004 T02: ..."     # subject names the task
+    make task-done T=CB-WP-0004-T02          # measures, flips, pushes
+    git commit -m "chore: mark T02 done (measured: ...)"
+
+Usage:
+  python3 tools/task-done.py CB-WP-0004-T02
+  python3 tools/task-done.py CB-WP-0004-T02 --dry-run    # measure, change nothing
+  python3 tools/task-done.py CB-WP-0004-T02 --no-hub     # file only
+  python3 tools/task-done.py --self-test
+"""
+import glob
+import json
+import os
+import re
+import subprocess
+import sys
+import urllib.error
+import urllib.request
+
+from repo import ROOT, enter_root
+
+HUB = os.environ.get("CUSTODIAN_HUB", "http://127.0.0.1:8000")
+AGENT = "custodian"
+TASK_ID_RE = re.compile(r"^(?P<wp>CB-WP-\d+)-(?P<label>T\d\d)$")
+
+
+class Fail(Exception):
+    """A loud refusal. The heredocs this replaces failed silently."""
+
+
+# ------------------------------------------------------------ workplan file
+
+
+def find_task_block(text, task_id):
+    """(start, end, body) of the ```task block declaring task_id.
+
+    Returns None when the id is absent — the caller turns that into a
+    non-zero exit. A string replace over the whole file would instead have
+    flipped whichever `status: todo` came first, which is worse than
+    doing nothing.
+    """
+    for m in re.finditer(r"```task\n(.*?)```", text, re.S):
+        body = m.group(1)
+        if re.search(rf"^id:\s*{re.escape(task_id)}\s*$", body, re.M):
+            return m.start(1), m.end(1), body
+    return None
+
+
+def block_status(body):
+    m = re.search(r"^status:\s*(\S+)\s*$", body, re.M)
+    return m.group(1) if m else None
+
+
+def set_status(body, new):
+    return re.sub(r"^status:\s*\S+\s*$", f"status: {new}", body, count=1, flags=re.M)
+
+
+def hub_id(body):
+    m = re.search(r'^state_hub_task_id:\s*"?([0-9a-f-]{36})"?\s*$', body, re.M)
+    return m.group(1) if m else None
+
+
+def locate(task_id):
+    """(path, text, span, body) for the one workplan declaring this task."""
+    hits = []
+    for path in sorted(glob.glob(os.path.join(ROOT, "workplans", "*.md"))):
+        text = open(path).read()
+        found = find_task_block(text, task_id)
+        if found:
+            hits.append((path, text, (found[0], found[1]), found[2]))
+    if not hits:
+        raise Fail(f"no workplan declares task {task_id}")
+    if len(hits) > 1:
+        raise Fail(f"{task_id} declared in {len(hits)} workplans: "
+                   + ", ".join(os.path.basename(h[0]) for h in hits))
+    return hits[0]
+
+
+# ---------------------------------------------------------------- measuring
+
+
+def measured(task_id):
+    """Measured cost and tokens for this task, from the transcripts.
+
+    Looks up the **qualified** label (`CB-WP-0004-T01`) only. A bare `T01`
+    bucket collides across workplans — three of them existed when this was
+    written, and summing them reported $12.10 for a task that cost a
+    fraction of it. Falling back to the bare label would reintroduce
+    exactly the inflated number this tool exists to prevent, so the commit
+    subject must name the workplan.
+
+    Imports cb-cost rather than shelling out and parsing its printed
+    table: a number re-parsed out of formatted text is a copy that can
+    drift from its source (the DFD class).
+    """
+    import importlib.util
+
+    spec = importlib.util.spec_from_file_location(
+        "cb_cost", os.path.join(ROOT, "tools", "cb-cost.py"))
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    rep = mod.collect("-home-worsch-clay-borg", None)
+    detail = rep["by_task_detail"].get(task_id)
+    if not detail:
+        bare = task_id.rsplit("-", 1)[-1]
+        hint = ""
+        if bare in rep["by_task_detail"]:
+            hint = (f" A bare '{bare}' bucket exists, but it is shared with "
+                    f"other workplans and must not be used for this task.")
+        raise Fail(
+            f"no spend attributed to {task_id}. Attribution is by commit "
+            f"subject (CA-08) — commit the work with '{task_id}' (or "
+            f"'{task_id.rsplit('-', 1)[0]} {bare}') in the subject line "
+            f"first, then close the task.{hint} Refusing to report an "
+            f"estimate: that is the habit this tool exists to end.")
+    return detail
+
+
+# --------------------------------------------------------------------- hub
+
+
+def push(task_uuid, detail, label):
+    body = {
+        "status": "done",
+        "tokens_in": detail["tokens_in"],
+        "tokens_out": detail["tokens_out"],
+        "model": detail["model"],
+        "agent": AGENT,
+        "token_note": (
+            f"measured by tools/cb-cost.py (CA-08 attribution); "
+            f"${detail['cost']:.2f} over {detail['responses']} responses; "
+            f"models {detail['models']}; tokens_in includes cache reads and "
+            f"writes, which the hub schema cannot separate"
+        ),
+    }
+    req = urllib.request.Request(
+        f"{HUB}/tasks/{task_uuid}",
+        data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"},
+        method="PATCH",
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=10) as r:
+            return r.status
+    except urllib.error.URLError as e:
+        raise Fail(f"hub PATCH failed for {label} ({task_uuid}): {e}")
+
+
+# --------------------------------------------------------------- self-test
+
+
+def self_test():
+    """Each assertion pins a failure this tool must detect.
+
+    The controls that matter: the silent no-op that motivated the tool
+    (unknown id, already-done task, wrong block flipped), and the refusal
+    to invent token counts.
+    """
+    results = []
+
+    def check(name, ok, detail=""):
+        results.append((name, ok, detail))
+
+    sample = (
+        "## Task: one\n\n```task\nid: CB-WP-0009-T01\nstatus: done\n"
+        'state_hub_task_id: "11111111-2222-3333-4444-555555555555"\n```\n\n'
+        "## Task: two\n\n```task\nid: CB-WP-0009-T02\nstatus: todo\n"
+        'state_hub_task_id: "66666666-7777-8888-9999-aaaaaaaaaaaa"\n```\n'
+    )
+
+    # The defect: a whole-file replace flips the first `status: todo` it
+    # sees, not the one asked for.
+    found = find_task_block(sample, "CB-WP-0009-T02")
+    check("finds the block for the id asked for, not the first block",
+          found is not None and "T02" in found[2])
+    check("already-done task is visible as such",
+          block_status(find_task_block(sample, "CB-WP-0009-T01")[2]) == "done")
+    check("unknown id yields nothing (caller must abort)",
+          find_task_block(sample, "CB-WP-0009-T99") is None)
+    check("typo'd id yields nothing rather than the nearest match",
+          find_task_block(sample, "CB-WP-0009-TO2") is None)
+    check("status flip touches only the target block",
+          set_status(found[2], "done").count("done") == 1)
+    check("hub uuid is extracted from the right block",
+          hub_id(found[2]) == "66666666-7777-8888-9999-aaaaaaaaaaaa")
+    check("missing hub uuid is None, not a guess",
+          hub_id("id: X\nstatus: todo\n") is None)
+    check("task id format is validated",
+          bool(TASK_ID_RE.match("CB-WP-0004-T02"))
+          and not TASK_ID_RE.match("T02")
+          and not TASK_ID_RE.match("CB-WP-0004"))
+
+    # The estimate refusal, exercised against a label that cannot have spend.
+    try:
+        measured("CB-WP-9999-T99")
+        check("refuses to report a task with no attributed spend", False,
+              "returned a figure for a task with no commit")
+    except Fail as e:
+        check("refuses to report a task with no attributed spend",
+              "Refusing to report an estimate" in str(e))
+    except Exception as e:  # cb-cost aborted for an unrelated reason
+        check("refuses to report a task with no attributed spend", False, str(e))
+
+    # And that it does return real numbers for a task that has them.
+    try:
+        d = measured("CB-WP-0004-T01")
+        check("returns measured tokens for a committed task",
+              d["tokens_in"] > 0 and d["cost"] > 0,
+              f"${d['cost']:.2f}, {d['tokens_in']:,} in / {d['tokens_out']:,} out")
+    except Fail as e:
+        check("returns measured tokens for a committed task", False, str(e))
+
+    print("task-done self-test (positive control)")
+    ok = True
+    for name, passed, det in results:
+        print(f"  [{'ok ' if passed else 'FAIL'}] {name}"
+              + (f"  — {det}" if det else ""))
+        ok &= passed
+    return 0 if ok else 1
+
+
+# -------------------------------------------------------------------- main
+
+
+def main():
+    enter_root()
+    argv = [a for a in sys.argv[1:] if not a.startswith("--")]
+    flags = {a for a in sys.argv[1:] if a.startswith("--")}
+    if "--self-test" in flags:
+        return self_test()
+    if len(argv) != 1:
+        print(__doc__.strip().split("Usage:")[-1], file=sys.stderr)
+        return 2
+
+    task_id = argv[0]
+    m = TASK_ID_RE.match(task_id)
+    if not m:
+        print(f"ERROR — expected a task id like CB-WP-0004-T02, got {task_id!r}",
+              file=sys.stderr)
+        return 1
+
+    try:
+        path, text, (lo, hi), body = locate(task_id)
+        status = block_status(body)
+        if status == "done":
+            raise Fail(f"{task_id} is already done in "
+                       f"{os.path.relpath(path, ROOT)} — refusing to re-close")
+        uuid = hub_id(body)
+        if not uuid and "--no-hub" not in flags:
+            raise Fail(f"{task_id} has no state_hub_task_id; run fix-consistency "
+                       f"first, or pass --no-hub")
+        detail = measured(task_id)
+    except Fail as e:
+        print(f"ERROR — {e}", file=sys.stderr)
+        return 1
+
+    rel = os.path.relpath(path, ROOT)
+    print(f"closing {task_id}  ({rel}, status {status} -> done)")
+    print(f"  measured   ${detail['cost']:,.2f} over {detail['responses']} responses")
+    print(f"  tokens     {detail['tokens_in']:,} in / {detail['tokens_out']:,} out")
+    print(f"  models     {detail['models']}")
+
+    if "--dry-run" in flags:
+        print("  dry-run    file unchanged, hub not called")
+        return 0
+
+    new = text[:lo] + set_status(body, "done") + text[hi:]
+    if new == text:
+        # Positive control for the exact defect being replaced.
+        print("ERROR — status flip produced no change; refusing to claim a close",
+              file=sys.stderr)
+        return 1
+    with open(path, "w") as fh:
+        fh.write(new)
+    print(f"  file       {rel} updated")
+
+    if "--no-hub" in flags:
+        print("  hub        skipped (--no-hub)")
+    else:
+        try:
+            code = push(uuid, detail, task_id)
+            print(f"  hub        {uuid} -> done (HTTP {code}), measured tokens")
+        except Fail as e:
+            # The file is already flipped and that is recoverable by git;
+            # a wrong exit code is not. Report and fail.
+            print(f"ERROR — {e}", file=sys.stderr)
+            return 1
+
+    subject = (f"chore: mark {m.group('label')} done (measured: {detail['responses']} responses, "
+               f"${detail['cost']:,.2f}, {detail['model']})")
+    print(f"\nnext: git commit -m {subject!r}")
+    return 0
+
+
+if __name__ == "__main__":
+    try:
+        sys.exit(main())
+    except subprocess.CalledProcessError as e:
+        print(f"ERROR — {e}", file=sys.stderr)
+        sys.exit(1)

workplans/CB-WP-0004-mechanical-work.md

@@ -110,6 +110,28 @@ CB-WP-0002 disproved.
 **Predicted:** those 46 turns → **~6**, **$9–11** recovered, and the hub
 stops holding estimates. Add `--self-test` per InnerLoop v1.1.
 
+**Delivered.** `tools/task-done.py` + `make task-done T=<id>`. It refuses
+on an unknown id, a typo'd id, an already-done task, a task with no
+`state_hub_task_id`, and — the one that matters — **a task with no
+measured spend**, rather than reporting an estimate. `cb-cost` gained
+`by_task_detail` (cost, response count, model histogram, and token
+components per task), and `task-done` imports cb-cost rather than parsing
+its printed table, so the hub number is not a copy that can drift.
+
+**The positive control found a real defect before the tool was used
+once.** Attribution keyed on a bare `T\d\d` from the commit subject, so
+`CB-WP-0002 T01`, `CB-WP-0003 T01` and `CB-WP-0004 T01` all landed in one
+bucket. The self-test reported **$12.10** for "T01"; the qualified figure
+is **$2.33** — a 5.2× overstatement that would have been pushed to the
+hub as a measured number, reproducing the fiction this task exists to
+end, in a new form. Fixed by `task_label()`: qualified subjects
+(`CB-WP-0004 T01`) key on the full id, unqualified ones stay bare and are
+never retro-assigned to a workplan. The pinned $93.15 benchmark is
+unchanged, confirming historical attribution was not disturbed.
+
+That is the **fourth** instance of trusted arithmetic (TA) — a number
+believed because it was produced by a program rather than by hand.
+
 ## Task: `make status` — one-shot orientation
 
 ```task