Tier M — structural S by the letter of InnerLoop's table (no capability
port, no canonical interface, no new dependency), declared M because the
table's triggers only cover code capabilities and this decides whether the
loop constrains its own operation. Chaos d4=2, no override. Survey and ADR
merged per tier M.
The gap is not the one CB-WP-0006 assumed. SH-1/2/3 ARE measured —
cb-cost emits all three every run. The defect is that SH-1 and SH-2 are
cumulative means over every response ever recorded, so they cannot detect
a worsening trend: the history outvotes the present.
cumulative -> dfd0d6d 923 resp mean 206,952 p90 389,733 9.6%
CB-WP-0006 alone 156 resp mean 503,464 p90 607,105 0.0%
Read cumulatively, session shape looks stable and improving. Read by
window, the last pass ran at 2.4x the mean of all preceding history and
batched nothing at all — 146 tool calls in 146 responses.
This is the third shape of one disease. CB-WP-0005 found numbers that were
unmeasured; CB-WP-0006 found numbers measured but unasserted; these are
measured AND asserted and still cannot fire, because they are aggregated
over a window in which nothing can move them.
Decisions: window SH-1/SH-2 and keep cumulative as labelled history;
enforce with a live budget modelled on CB-01/CB-02, NOT `make all` —
failing the build on context would block committing, and committing is
what closes the attribution window and is the natural point to compact, so
a gate that blocks the remedy is a trap; targets do not move, since all
three are unmet by wide margins and adjusting them in the pass that first
enforces them is the retarget §Step 4 forbids; and SH-3 is tested by
batching deliberately rather than by argument.
Recorded and not fixed: InnerLoop's tier table triggers on code
properties, so this pass would be tier S despite deciding how the loop
governs itself. Changing the taxonomy on one instance is the
invention-in-isolation INTENT warns against — the same argument used to
amend K14 and to leave v1.4 alone. A second instance should trigger it.
CB-WP-0007 executes in 6 tasks. T04's third outcome is the interesting
one: if batching stays below 5% despite trying, 20% is the number to
challenge with evidence, in an ADR rather than a quiet edit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
6.9 KiB
| id | title | status |
|---|---|---|
| CB-WP-0007 | Make session shape measurable in the window that matters, then enforce it | proposed |
Purpose
CB-RES-0005 (tier M, survey and ADR merged; chaos d4=2, no override):
SH-1 and SH-2 are cumulative means over every response ever recorded. They cannot detect a worsening trend, because the history outvotes the present.
| window | responses | SH-1 mean | SH-2 p90 | SH-3 batching |
|---|---|---|---|---|
cumulative → dfd0d6d |
923 | 206,952 | 389,733 | 9.6% |
| CB-WP-0006 alone | 156 | 503,464 | 607,105 | 0.0% |
Read cumulatively, session shape looks stable and improving. Read by window, the last pass ran at 2.4× the mean of all preceding history and batched nothing at all — 146 tool calls in 146 responses.
This is the third shape of one disease. CB-WP-0005 found numbers that were unmeasured; CB-WP-0006 found numbers that were measured but unasserted; this pass has numbers that are measured and asserted and still cannot fire, because they are aggregated over a window in which nothing can move them.
Per InnerLoop §Step 4, no target moves in the commit that measures it — and CB-RES-0005 D3 states that up front, because all three targets are currently unmet by wide margins and the temptation is obvious.
Phase A — measure the right window
Task: window the session-shape metrics
id: CB-WP-0007-T01
status: todo
priority: high
cb-cost's session-shape block reports over every response in scope. With
--since it already windows correctly; without it, it reports a
cumulative mean that a bad pass cannot move.
Deliver: the session-shape block states its window explicitly and, when
run without --since, reports both — the window since the last commit
and the cumulative history, labelled so they cannot be confused. The
cumulative figure stays useful for the long view and stops being what a
target compares against.
Predicted: the windowed figure differs sharply from cumulative. Refuted if it lands within 20% of the cumulative number, in which case the aggregation was not the problem and this pass should re-plan.
Task: SH-4 — a windowed trend the instrument can see
id: CB-WP-0007-T02
status: todo
priority: medium
A ceiling catches a bad pass. It does not catch a slow slide, which is how SH-1 got from 232k to 503k without any single pass looking anomalous.
Deliver a reported (not gated) SH-4: mean context of the current window
against the previous window, so a rising trend is visible before it
breaches. Provenance adapted:criterion — the same shape as the >3%
merge-blocking regression rule against our own last evidence file, but
reported rather than blocking, because CB-RES-0005 §4 argues a build gate
is the wrong instrument for behaviour.
State the limit in the output, as every other gate here does: two windows is not a trend, and the first run has nothing to compare against.
Phase B — enforce it where enforcement belongs
Task: make shape-budget — a live budget, not a build gate
id: CB-WP-0007-T03
status: todo
priority: high
Per CB-RES-0005 D2, modelled on CB-01/CB-02, which measures spend since
the last commit, warns at $10, stops at $22, and is surfaced by
make status — never by make all.
Why not a build gate, restated so it is not quietly reversed later:
failing make all on context size would block committing, and
committing is what closes the attribution window and is the natural point
to compact. A gate that blocks the remedy when the metric breaches is a
trap, not a gate. It also punishes the wrong artifact — a long session
does not break the tree.
Deliver make shape-budget and a line in make status:
| threshold | value | action |
|---|---|---|
| SH-1 soft | 200,000 mean context in the window | warn |
| SH-1 hard | 300,000 | report loudly: compact before continuing |
| SH-2 soft | 300,000 p90 | warn |
| SH-3 floor | 20% batching | warn |
The hard threshold is set here, before the next measurement, per §Step 4. It is 1.5× the soft target, matching CB-02's 2.2× ratio on spend scaled down because context is bounded by the model and spend is not.
Carries --self-test. Its positive control is the one this project keeps
needing: a budget that reports ok because it measured nothing must
abort instead.
Task: batch deliberately, and report what the rate reaches
id: CB-WP-0007-T04
status: todo
priority: medium
SH-3 is 0.0% across two consecutive passes. The target is 20%. Nobody has ever tried to hit it, so its reachability is unknown and the number is currently unfalsified rather than wrong.
This task is the trial: batch independent tool calls deliberately through the pass and report the rate the instrument measures.
Three outcomes, all publishable, and the third is the interesting one:
- ≥20% — the target is reachable; it stops being aspirational.
- 5–20% — partially reachable; report the ceiling and what blocked it.
- <5% despite trying — then 20% is the number to challenge, with evidence, and the challenge belongs in an ADR rather than a quiet edit. Per §Step 4 the target does not move in this pass regardless.
Phase C — measure, and say what it cost
Task: control loop — did windowing change what fires?
id: CB-WP-0007-T05
status: todo
priority: high
Commit evidence/CB-EV-0006-session-shape.md. Four tests, all reported:
- Does the windowed metric differ from cumulative? Against T01's prediction; refuted if within 20%.
- Did the budget fire on this pass? A budget that never reports a breach across a full pass is either miscalibrated or measuring nothing — both are findings.
- What did SH-3 reach? T04's three outcomes.
- Did quality hold?
make allgreen, and the honest question this pass cannot dodge: did compacting to satisfy a context budget cause work to be repeated or detail to be lost?SessionShape.md§5 says the loop cannot detect this. Record the judgment explicitly rather than letting the cost number imply it.
Task: retrospective
id: CB-WP-0007-T06
status: todo
priority: medium
The question: three passes have now found three different ways a number can exist and not bind — unmeasured, unasserted, and aggregated so it cannot fire. Is there a fourth, and is the loop discovering a pattern or enumerating an infinite set?
CB-WP-0003 concluded that class-by-class hardening does not converge. Three passes later, the instrument failures have their own taxonomy and it is also not converging. Say plainly whether that changes the design goal, or confirms it.
Also due: the tier-taxonomy gap CB-RES-0005 §6 recorded but did not fix. This pass is the first instance. Per the standing argument used for K14 and InnerLoop v1.4, a second instance triggers the change — say whether this counts as one or two.