coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CB-WP-0013-T02/T03: retire SH-3 as a gate; correct AM-4a and its target

Browse source 
ADR-0008, tier M (survey and ADR merged).

D1 — SH-3 retired as a gate, kept as a diagnostic. Investigating it
found a third defect, deeper than the two this pass was declared on.
Re-deriving batching from the raw transcripts, independently of cb-cost:

  CB-WP-0011 pass      54 with tools    0 batched    0.0%
    gap -> next decl   16 with tools    6 batched   37.5%
  CB-WP-0012 pass      86 with tools    0 batched    0.0%
    gap -> next decl   10 with tools    1 batched   10.0%
  CB-WP-0013 so far    10 with tools    0 batched    0.0%

Zero batched turns in 150 in-pass responses; 37.5% in one gap, above the
20% floor. Batching needs two calls whose inputs are known at once —
orientation work. Implementation consumes each step's result before the
next. SH-3's window is since the last commit, which during a pass is
always implementation. The metric could not read above ~0% in the window
it was gated on. A floor the window structurally excludes is not a
target.

This pass's own declaration was also wrong: it claimed batching "has got
worse" (7.8-8.6% vs 1.1-6.3%). Differently-placed windows, not different
behaviour. Withdrawn — the same class of error, in the pass written to
correct it.

Not retargeting to match the measurement: the floor was not moved to 6%,
the gate was removed on an argument about what the quantity is worth.
The number is still reported; only the verdict is gone.

D2/D3 — AM-4a counts --edges normal,no-proc-macro: 157,202, not 246,250.
The target moves down with it, 250,000 -> 161,000, so the correction
hands back essentially nothing (headroom 3,750 -> 3,798). Three controls:
the exclusion drops exactly the five expected crates, only removes and
never adds, and is not a no-op.

The DFD gate then caught the follow-on it exists for — three historical
documents carrying live fact tags for a number that had changed. Not
rewritten; untagged, with a supersession banner.

AM-4b is deliberately not corrected: its proc-macro share is unmeasured.

gate-review now reads 0 due, 0 silent, 0 drifted — GATE-REVIEW earns its
first caught entry by forcing SH-3's re-justification, and the registry
has no silent gates left.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-02 07:30:14 +02:00
parent 6080c4cf60
commit d2c3f340f9
9 changed files with 306 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

29
tools/cb-cost.py
View file

@ -632,7 +632,7 @@ def render(rep, by_task=False, composition=False):
f"need {SH3_MIN_SAMPLE}]")
else:
print(f"{indent}SH-3 batching rate {100*sh['SH-3_batching_rate']:>11.1f}% "
f"[{'ok ' if sh['SH-3_batching_rate']>=0.20 else 'FAIL'} target 20.0%]")
f"[diagnostic, no floor — ADR-0008 D1]")
print(f"{indent} {sh['tool_calls']} tool calls in "
f"{sh['responses_with_tools']} responses; "
f"{sh['calls_in_batched_turns']} in batched turns")
@ -808,8 +808,14 @@ def self_test():
below.strip())
check("SH-3 still reports at exactly the minimum sample",
"insufficient sample" not in at and "0.0%" in at, at.strip())
check("SH-3 reports a real rate above the floor",
"40.0%" in good and "ok" in good, good.strip())
check("SH-3 reports a real rate, with no verdict attached",
"40.0%" in good and "no floor" in good and "FAIL" not in good,
good.strip())
# ADR-0008 D1: the gate is gone. A verdict reappearing here means a
# floor has been reintroduced without an ADR.
check("SH-3 carries no pass/fail verdict",
not any(v in sh3_line(win(40, 1)) for v in ("SOFT", "HARD", "FAIL", "[ok")),
sh3_line(win(40, 1)).strip())
# The one that matters: a refusal must not read as a measured zero.
check("a refusal is distinguishable from a genuine 0.0%",
below != sh3_line(win(SH3_MIN_SAMPLE, 0)))
@ -853,9 +859,20 @@ SH3_MIN_SAMPLE = 14
def sh3_line(win, min_sample=SH3_MIN_SAMPLE):
"""SH-3, or an explicit refusal when the window cannot support a rate.
"""SH-3 as a DIAGNOSTIC — no floor, no verdict (ADR-0008 D1).
Returns the line rather than printing it so the refusal is testable
Retired as a gate because the window and the metric are systematically
anti-correlated: batching needs two tool calls whose inputs are known
at once, which is orientation work, and this window is *since the last
commit*, which during a pass is implementation work. Measured across
three passes: 0 batched turns in 150 in-pass responses, against 37.5%
in the gap between two of them. A floor the window structurally
excludes is not a target.
The minimum-sample refusal is kept. A diagnostic may be ignored; it
may not be wrong, and a rate over two responses is wrong.
Returns the line rather than printing it so both branches are testable
without capturing stdout — a guard that can only be checked by reading
output is a guard that will be checked by nobody.
"""
@ -865,7 +882,7 @@ def sh3_line(win, min_sample=SH3_MIN_SAMPLE):
f"[----] {n} response(s) with tool calls, need {min_sample}")
rate = win["SH-3_batching_rate"]
return (f" SH-3 batching {100*rate:>9.1f}% "
f"[{'ok ' if rate >= 0.20 else 'SOFT'}] floor 20.0%")
f"[diag] no floor — retired as a gate, ADR-0008 D1")
def shape_budget(slug):