coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
clay-borg/decisions/ADR-0008-instrument-corrections.md
tegwick d2c3f340f9 CB-WP-0013-T02/T03: retire SH-3 as a gate; correct AM-4a and its target 
ADR-0008, tier M (survey and ADR merged).

D1 — SH-3 retired as a gate, kept as a diagnostic. Investigating it
found a third defect, deeper than the two this pass was declared on.
Re-deriving batching from the raw transcripts, independently of cb-cost:

  CB-WP-0011 pass      54 with tools    0 batched    0.0%
    gap -> next decl   16 with tools    6 batched   37.5%
  CB-WP-0012 pass      86 with tools    0 batched    0.0%
    gap -> next decl   10 with tools    1 batched   10.0%
  CB-WP-0013 so far    10 with tools    0 batched    0.0%

Zero batched turns in 150 in-pass responses; 37.5% in one gap, above the
20% floor. Batching needs two calls whose inputs are known at once —
orientation work. Implementation consumes each step's result before the
next. SH-3's window is since the last commit, which during a pass is
always implementation. The metric could not read above ~0% in the window
it was gated on. A floor the window structurally excludes is not a
target.

This pass's own declaration was also wrong: it claimed batching "has got
worse" (7.8-8.6% vs 1.1-6.3%). Differently-placed windows, not different
behaviour. Withdrawn — the same class of error, in the pass written to
correct it.

Not retargeting to match the measurement: the floor was not moved to 6%,
the gate was removed on an argument about what the quantity is worth.
The number is still reported; only the verdict is gone.

D2/D3 — AM-4a counts --edges normal,no-proc-macro: 157,202, not 246,250.
The target moves down with it, 250,000 -> 161,000, so the correction
hands back essentially nothing (headroom 3,750 -> 3,798). Three controls:
the exclusion drops exactly the five expected crates, only removes and
never adds, and is not a no-op.

The DFD gate then caught the follow-on it exists for — three historical
documents carrying live fact tags for a number that had changed. Not
rewritten; untagged, with a supersession banner.

AM-4b is deliberately not corrected: its proc-macro share is unmeasured.

gate-review now reads 0 due, 0 silent, 0 drifted — GATE-REVIEW earns its
first caught entry by forcing SH-3's re-justification, and the registry
has no silent gates left.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 07:30:14 +02:00

7.4 KiB
Raw Blame History

ADR-0008: retire SH-3 as a gate; correct AM-4a to count what ships

status: accepted date: 2026-08-02 decided by: agent, under the standing loop authorization. One item is reserved for the maintainer — see §Reserved. tier: M (structural M — changes how the loop constrains its own operation: a gate floor and a budget instrument, InnerLoop v1.6 / ADR-0006 D5; chaos d4=3 → no override). Tier M merges survey and ADR into one document, which this is; the adversarial review is optional and was not run. references: CB-WP-0013, SessionShape.md §4, ADR-0006 D1/D2/D4, ADR-0007 D4, CB-EV-0010 §5

Context

Two instruments were found wrong by the passes that used them. Neither error was in the direction that flattered us, and one of them has been producing a standing breach report for seven passes.

Decision 1 — SH-3 is retired as a gate and kept as a diagnostic

The survey: three defects, each deeper than the last

First (CB-WP-0013 T01): the sample. --shape-budget windows since the last commit and is read while writing the evidence file — right after a commit, when the window holds one or two responses. SH-1/SH-2 are location statistics and survive that. SH-3 is a rate; at n=2 its only values are 0%, 50%, 100%. Six evidence files quoted the resulting 0.0% as a measurement. Fixed: below a derived minimum of 14 the tool now prints insufficient sample and no verdict.

Second: the claimed decline was also an artifact. This workplan's own Purpose stated that batching "has got worse" — pinned history 7.8–8.6% against per-pass windows of 1.1–6.3%. That was wrong, and by the same class of error it was written to correct. The two figures come from differently-placed windows, not from different behaviour. Withdrawn.

Third, and the one that decides this: the window and the metric are systematically anti-correlated. Measured, by re-deriving batching independently of cb-cost from the raw transcripts:

window responses with tools batched rate
CB-WP-0011 pass 54 0 0.0%
gap → next declaration 16 6 37.5%
CB-WP-0012 pass 86 0 0.0%
gap → next declaration 10 1 10.0%
CB-WP-0013 so far 10 0 0.0%

Zero batched turns in 150 in-pass responses across three passes, and 37.5% in the gap between two of them — comfortably above the 20% floor.

The cause is not discipline. Batching requires two tool calls whose inputs are both known at once. Orientation and survey work is full of those: read four files, run three probes. Implementation has almost none, because each step consumes the previous step's result — read, edit, test, read the failure, fix. So SH-3 measures what kind of work is happening.

And SH-3's window is since the last commit, which during a pass is always in-pass work. The metric is structurally incapable of reading above ~0% in the window it is gated on. It is not a strict gate — it can only be met by not working.

The decision

SH-3 stops being a gate. The 20% floor and the [SOFT] verdict are removed. cb-cost continues to report the batching rate as a diagnostic, over the whole session, where it reads 5.8% (57 batched of 984 responses with tool calls).

Four reasons, strongest first:

  1. It cannot report what it gates on. §above. A floor that the measurement window structurally excludes is not a target.
  2. It measures composition, not waste. Penalising an implementation pass for not batching penalises it for being an implementation pass.
  3. Its own spec bounds its value. specs/SessionShape.md §4: "eliminating every avoidable single-call turn would remove at most a few percent of turns, worth roughly $2–4 on a $93 pass", and "batch because it is free, not because it is the lever." It also calls SH-3 "the least valuable" of the three.
  4. Seven passes, seven breach reports, zero actions. ADR-0006 D4: rituals cash out or go.

This is not retargeting to match the measurement

specs/SessionShape.md forbids that explicitly, and CB-WP-0003 T07 exists to prevent it, so the distinction has to be stated rather than assumed:

  • Retargeting would be moving the floor from 20% to 6% because 6% is what we measure. That is not done and is still forbidden.
  • This removes the gate on the argument that the quantity does not deserve one — established before the disposition, from the window structure and from the spec's own valuation.

And the obvious objection — retiring a metric right when it shows a regression looks like burying it — is answered by the second defect: the regression was an artifact of window placement and has been withdrawn. There is no established decline to bury. The number is still reported; only the verdict is gone.

Decision 2 — AM-4a counts --edges normal,no-proc-macro

ADR-0007 D4, owed. AM-4a's figure includes proc-macro crates that run in the compiler and never reach a binary: 89,048 lines, 36.2% of the shipped-runtime figure, syn alone 66,916.

tools/dep-weight.py measures --edges normal,no-proc-macro for the shipped-runtime configuration.

Decision 3 — the target moves down with the instrument

Correcting the metric from 246,250 to 157,202 while leaving the target at 250,000 would hand this project 89,048 lines of headroom it did not earn, in the same commit that reveals the error. That is the shape of change this repo has twice refused, and refusing it here costs nothing:

AM-4a's shipped-runtime target moves from 250,000 to 161,000.

161,000 preserves the ratio the ratified target expressed — 250,000 was set against a 246,250 measurement, i.e. ~1.5% of headroom; 161,000 against 157,202 is ~2.4%. Rounding to a round number slightly above strict proportionality is deliberate and is the only place this decision gives anything back, because a target that leaves 1.5% of room is a target that fails on a dependency's patch release.

AM-4b (dev-toolchain) is left alone, at 350,000 against a 317,021 measurement. It is not corrected in this pass because its proc-macro share has not been measured, and correcting a second instrument on the strength of the first one's ratio is exactly the error this ADR exists to fix. Filed as owed.

Reserved for the maintainer

AM-4a still cannot survive INTENT stage 2, and this decision makes that worse rather than better. wgpu + winit is 1,741,979 marginal lines against a target now proposed at 161,000 — 10.8×, up from 7×. ADR-0007 raised it; nothing here settles it. The correction below is honest book-keeping, not headroom for stage 2, and it should not be read as having made the stage-2 problem smaller.

Consequences

  • One fewer gate, and the gate registry gets its first retirement. That is also GATE-REVIEW's first caught entry — the registry's retire_if says it must have "retired, tightened, or forced the re-justification of" something by its review date, and it now has.
  • Every AM-4a figure in this repo's history overstates the load by 36%. Those documents are history and are not rewritten; the correction lives in specs/GameKernel.md §4 and here.
  • The batching diagnostic stays visible, so if in-pass batching ever becomes possible the number will show it without a gate having to be re-litigated first.