CB-WP-0013-T01: SH-3 refuses a window that cannot carry a rate
The metric six evidence files reported as 0.0% was never measured. --shape-budget windows since the last commit, and it is read while writing the evidence file — right after a commit, when the window holds one or two responses. SH-1/SH-2 are location statistics and survive n=2. SH-3 is a rate: at n=2 its only possible values are 0%, 50%, 100%. cb-cost.py gains sh3_line(), which below a minimum sample prints "insufficient sample" and no verdict. The floor is derived: if the true rate were exactly the 20% target, P(zero batched in n) = 0.8^n, and at n=14 that is 4.4% — so "0 batched in 14" rules out a target-meeting rate at ~95%. Below that the tool has nothing to say and now says so. The window was kept rather than split. SH-3 could have been given a per-pass window, but the budget's purpose is the open remainder since the last commit, and giving one of three metrics a different window makes "the window" ambiguous in a tool three specs cite. Four controls, three mutations, each red for its stated reason — including the one the evidence files actually hit, where a refusal is printed as a measured zero. SessionShape.md §4 carries the correction with the real per-pass figures (1.1%-6.3%), beside the eighth trusted-arithmetic instance. This is the ninth, and the second in this same metric. It also shows what the frozen 0.0% hid: against the pinned 7.8%-8.6%, batching has got worse, and six passes reported a breach that was moving underneath them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
7105027173
commit
6080c4cf60
3 changed files with 129 additions and 6 deletions
|
|
@ -136,6 +136,47 @@ workplan — the failure mode InnerLoop v1.1 rule 13 exists for, and the
|
|||
`history/260731-inner-loop-rule-audit.md` identified as having no
|
||||
executable defence.
|
||||
|
||||
### Correction (CB-WP-0013 T01): SH-3 was never read as measured
|
||||
|
||||
**Six evidence files quote `SH-3 0.0%`, and none of them measured it.**
|
||||
|
||||
`cb-cost --shape-budget` reports the window **since the last commit**, and
|
||||
it is read while writing the evidence file — immediately *after* a commit,
|
||||
when that window holds one or two responses. SH-1 and SH-2 are location
|
||||
statistics and survive a sample of two. SH-3 is a **rate**, and at n=2 its
|
||||
only possible values are 0%, 50% and 100%. One window, three metrics, and
|
||||
it is only wrong for one of them.
|
||||
|
||||
Measured over real per-pass windows instead:
|
||||
|
||||
| window since | responses | SH-3 |
|
||||
|---|---:|---:|
|
||||
| `0d2ab22` | 115 | 6.2% |
|
||||
| `6fb0aea` | 113 | 6.3% |
|
||||
| `331e7e9` | 95 | 1.1% |
|
||||
| `c5fa610` | 74 | 1.4% |
|
||||
| `883b608` | 70 | 1.4% |
|
||||
| `84d6886` | 24 | 4.3% |
|
||||
|
||||
The substantive claim survives — SH-3 is badly unmet — but the constant
|
||||
0.0% hid something the real numbers show: against the **7.8–8.6%** pinned
|
||||
above, **batching has got worse**, and a metric frozen at a constant could
|
||||
not have shown a trend at all. Six passes reported a breach and the breach
|
||||
was moving underneath them.
|
||||
|
||||
**This is the ninth recorded instance of the trusted-arithmetic class, and
|
||||
the second in this same metric** — the eighth is the `0 of 330 tool calls`
|
||||
figure corrected immediately above. SH-3 is the number this project keeps
|
||||
getting wrong, which is itself a reason to doubt the metric rather than the
|
||||
behaviour (§SH-3 disposition).
|
||||
|
||||
**The fix**, `cb-cost.py` `sh3_line()`: below a minimum sample the tool
|
||||
prints `insufficient sample` and no verdict. The floor is derived, not
|
||||
round — if the true rate were exactly the 20% target, the chance of seeing
|
||||
zero batched turns in `n` responses is `0.8^n`; at **n = 14** that is 4.4%,
|
||||
so "0 batched in 14" rules out a target-meeting rate at ~95%. Below that
|
||||
the tool has nothing to say and now says so.
|
||||
|
||||
At an 8.6% batching rate there is real headroom, but the saving is bounded:
|
||||
eliminating every avoidable single-call turn would remove at most a few
|
||||
percent of turns, worth roughly $2–4 on a $93 pass. **Batch because it is
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue