From 2f086d26b6ce64e302fcaabafdcefb3af4a2d66e Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 31 Jul 2026 08:52:33 +0200 Subject: [PATCH] T06: wire cost into the loop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - InnerLoop definition-of-done gains a cost row: M-D2-CST may no longer be recorded uncomputable, and composition must be reported, not only a total. - InnerLoop Step 5 gains the --self-test contract: every tool that reports a number exposes one, and it runs before the number does. Rationale attached, because the case that motivated it is the one review cannot catch — survey and reviewer both verified the same large sample and both missed the small one. - make cost / cost-test / cost-pin on the one command surface; cost-test in `make all` and in CI. - Hub now holds the measured figure for CB-WP-0001: 80.6M in / 323.6k out against the 401,100 it previously estimated, low by ~200x. The event states plainly that the hub schema cannot represent the 88% of cost that is cache, and names `make cost-pin` as the authority. Co-Authored-By: Claude Opus 5 --- research/CB-RES-0002-cost-accounting.md | 2 +- specs/InnerLoop.md | 18 ++++++++++++++++++ workplans/CB-WP-0002-cost-accounting.md | 2 +- 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/research/CB-RES-0002-cost-accounting.md b/research/CB-RES-0002-cost-accounting.md index cb8869b..188a5d3 100644 --- a/research/CB-RES-0002-cost-accounting.md +++ b/research/CB-RES-0002-cost-accounting.md @@ -101,7 +101,7 @@ the `subagents/` tree, so first-wins dedup under-counted it. See The subagent line is not a footnote. C1's blind-spot finding says a collector reading only the main file under-reports; a target of $92.21 would have been hit only by a collector *with* that blind spot, and failed by a -correct one. The acceptance target is **$92.87, stated as its two +correct one. The acceptance target is **$93.32, stated as its two components**, so a collector reading one tree is diagnosed rather than merely failed. diff --git a/specs/InnerLoop.md b/specs/InnerLoop.md index e5c35a2..0f2bab2 100644 --- a/specs/InnerLoop.md +++ b/specs/InnerLoop.md @@ -164,6 +164,19 @@ Concretely, a measurement harness must, on every run: **A number from a run that cannot prove it did the work is void** and must not reach an evidence file. +**Every tool that reports a number exposes `--self-test`**, and that +self-test runs before the number is produced (`make cost` depends on +`make cost-test`). The assertion must name a failure it detects, not +merely exercise the happy path. + +*(v1.0+, from CB-WP-0002: `cb-cost`'s dedup assertion fired on its first +run against real data and aborted, catching a rule that was verified on +206/206 groups of the main transcript and false in the 8-response +subagent tree. The generalization that failed — a property confirmed on +the largest sample assumed to hold on the smallest — is not one review +catches, because both the survey and the adversarial reviewer checked +the same large sample.)* + *(v1.0, from CB-WP-0001: both serious errors in the first pass were of exactly this shape. A JS harness reported 8.4s for 100k moves while every move was being rejected, and a Rust benchmark reported 9.3M @@ -286,6 +299,11 @@ A capability has completed the loop when all of the following are committed: - [ ] every unmet metric reported as unmet, with attribution and the options for resolving it — a missed target is an output of the loop, not a reason to move the target quietly +- [ ] **cost recorded**: `make cost` run for the pass, its composition + (not only its total) in the evidence file, and the per-task figures + pushed to the hub. M-D2-CST is no longer allowed to be + `uncomputable` — the instrument exists + ([CostAccounting.md](CostAccounting.md)) - [ ] retrospective note (may be one paragraph appended to the evidence file): what the loop itself should change ``` diff --git a/workplans/CB-WP-0002-cost-accounting.md b/workplans/CB-WP-0002-cost-accounting.md index b55e718..07a67c7 100644 --- a/workplans/CB-WP-0002-cost-accounting.md +++ b/workplans/CB-WP-0002-cost-accounting.md @@ -192,7 +192,7 @@ cleared the bar that CB-WP-0001's AM-12 failed to clear. ```task id: CB-WP-0002-T06 -status: todo +status: done priority: low state_hub_task_id: "1412263b-70c1-43e4-957d-1ad6c3203ca9" ```