clay-borg/reviews/CB-REV-0001-h1.md
tegwick 041c0e7c3e
Some checks failed
ci / check (push) Failing after 4s
CB-REV-0001: the adversarial review, and it was not approvable
Thirteen challenges, five FATAL, all five conceded. Nothing had reached
ground-game, which is the only reason this is a correction and not a
retraction.

The worst: `Reactive` was not "greedy with one preference changed". It
differed in five, including SpendFreedom — ranked 95 unconditionally
against greedy's `95 if gated else 0` — so the seat burned its Freedom
token in round one of every game. A second change to the exact mechanism
under study, and every number in CB-EV-0031 was measuring it. The pass
claimed ADR-0018's one-varying-parameter discipline in its own workplan
while violating it. GreedyPolicy::rank is now public and the policy
delegates, overriding one match arm, so the control is structurally true.

Withdrawn entirely: "H1-B suppresses DARVO in the attacker". Disabling
H1-B under the corrected policy changes the arm count by exactly zero.
The pass hedged the wrong variable — it disclaimed "the number 2" and
defended "the direction", and the direction is what failed. The
supporting inference was invalid anyway: final Stress cannot show who
armed, because DarvoEnded resets the stage and REVERSE gives its owner -2.

Corrected: criterion 1 was failed on the greedy column while the pass's
own printed table showed 31-1000 arms in the other columns — the
selective-column move, in the file that names it. "Peak Stress was 1" was
a maximum over StressSet payloads, not held state (true: 2); the baseline
game count was 1,600 not 3,200; and "a reckless policy plays identically
to a careful one" is refuted by this repo's own rank-95 policy.

Inert controls replaced, each verified red against the reviewer's own
mutation: the baseline hash test compared two identically-constructed
states (serde(skip) on variant left 57/57 green); the `unchanged:` test
checked 3 of 7 entries and passed with SOLVE made illegal; H1-A's ordering
and H1-B's OU-cancel path had no test at all.

edition-check now covers catalog.yaml and rules_delta.yaml, whose digests
CB-WP-0038 claimed and never recorded — the review found it and reported
it unverified rather than absent, which was the right call.

Still open: H1-B on the DARVO extra-Attack path is untested, regulation.rs
still skips setup failures silently, and round-5 arms are counted though
they can never act.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 02:02:35 +02:00

6.9 KiB
Raw Blame History

CB-REV-0001 — adversarial review of the H1 measurement

The tier-L review CB-WP-0038 owed (InnerLoop Step 2). One round: challenge, then response. Run 2026-08-08 by a separate agent against CB-WP-0038/CB-EV-0030 and CB-WP-0039/CB-EV-0031.

Verdict: not approvable as submitted. Thirteen challenges, five rated FATAL. Every FATAL is conceded. Nothing from either evidence file had reached ground-game, which is the only reason this is a correction rather than a retraction.

The reviewer reproduced every number, re-derived on three samples (seeds 0..200, 1000..1200, 5000..5500) and ran 14 kernel mutations.


The five that were fatal

1. Reactive was not "greedy with one preference changed" — it differed in five

Conceded, and it is the worst thing in the pass. The policy re-typed an abridged copy of greedy's ranking and changed one line of the copy, under a comment reading // THE ONE LINE. It also differed in SpendFreedom (95 unconditional against greedy's 95 if gated else 0), Solve on a claimed Problem, ChooseGroundMode and RespondToSupport.

The SpendFreedom difference is a second change to the exact mechanism under study: the seat burned its Freedom token in round one of every game, ungated. Measured by the reviewer: 1200 spends against greedy's 0.

This pass claimed ADR-0018's one-varying-parameter discipline in its own workplan while violating it. That is worse than not claiming it.

Fixed structurally rather than by testing: GreedyPolicy::rank is now pub, and Reactive delegates to it and overrides a single match arm. A caller that delegates cannot drift. The reviewer's StrictReactive numbers reproduce exactly.

2. The H1-B suppression mechanism does not survive the corrected policy

Conceded and withdrawn. The reviewer disabled H1-B and re-ran: darvo is identical in every cell. The effect CB-EV-0031 §3 attributed to "H1-B's arithmetic" was an interaction with challenge 1's bugs.

The hedge was on the wrong variable. §3 disclaimed "the number 2" and defended "the direction". The direction is what failed.

3. [5,5,4,4,4,4] does not show that two seats armed

Conceded. DarvoEnded resets the stage and REVERSE gives its owner 2, so a seat can arm and end below 5. The reviewer exhibited six seats arming behind the same signature. A terminal snapshot was used to prove a path property.

4. Criterion 1 was failed on the greedy column alone

Conceded. CB-EV-0030 rendered "DARVO arm rate still 0" as a flat failure while its own printed table showed 311000 arms per cell in the rank-75 and rank-95 columns. The selective-column move, in the file that names selective reporting as the thing to avoid. Restated per column.

5. "Peak Stress was 1" was a wrong-subject error, with two more defects

Conceded, all three. peak maximised over StressSet event payloads; starting Stress is written by setup and never by an event, so a table sitting at 2 reported 1. True peak held: 2. The baseline game count was 1,600, not 3,200. And the generalisation — "a reckless policy plays identically to a careful one" — is refuted by this repo's own rank-95 policy, which under baseline drives Stress to 5.

What survives is narrower and is now stated that way: under the two policies measured, neither of which selects ATTACK under baseline, no Stress is ever added. ATTACK is the sole inbound pressure.


The serious ones

# challenge outcome
6 baseline_is_bit_for_bit_what_it_was compared two identically-constructed states — inert against its own threat; #[serde(skip)] on variant left 57/57 green conceded. Replaced by the_variant_reaches_the_hash_and_baseline_is_not_a_change, which asserts baseline and H1 hash differently. The reviewer's M6 now goes red
7 the unchanged: test checked 3 of 7 entries; making SOLVE illegal under H1 left it green conceded. Now compares legal_commands per seat (catches M9), checks relation slots at the boundary rather than on an empty seat, and asserts the DARVO arm is at 5 (catches M7). Both verified red
8 "criteria met" rests on a forced move — reactive ranks ATTACK at 10 and picks it only when the gate leaves nothing else conceded as a limitation, recorded, not fixed. It is true that the instrument cannot show a null result once H1-A reaches Stress 4. That is a real weakness of the measurement and is now stated in CB-EV-0031 §4
9 three claimed properties had no failing test: H1-A's ordering, H1-B on the DARVO extra Attack, H1-B on an OU-cancelled Attack two fixedh1a_pressure_arms_darvo_in_the_same_round_end and h1b_does_not_soothe_an_ou_cancelled_attack, both mutation-verified. The DARVO extra-Attack path remains untested and is carried as open

The rest

  • 10 — H1-B's after_target_and_relation_effects ordering is vacuous here. Conceded: nothing between the two positions touches the attacker's Stress, so the clause cannot be checked in this kernel. The honest statement replaces the claim that it was got right.
  • 11 — regulation.rs still skips setup failures silently. Conceded; the credit CB-EV-0030 §3 took was half-earned. The play path was instrumented, the setup path was not.
  • 13 — round-5 DARVO arms are counted but can never act, and end_round_events scores from self rather than work, so round-5 H1-A Stress is invisible to the CommonProblem tiebreak. Accepted as a real defect in the other two modes, which CB-EV-0030 reports on. Carried open.
  • 12 — reproduction and sample robustness: no problem found. Every number reproduced; no conclusion was seed-specific. The failures were of construction and interpretation, not sampling.

What the reviewer could not check, and it is recorded rather than glossed

The design note itself (criteria quoted from CB-EV-0030, not read from source); the catalog digest, which the reviewer could not find and reported as unverified rather than absent; --variant on the CLI; make cost; and the two non-SHARED modes under reactive.

The catalog digest is a real gap. CB-WP-0038 T01's control said "the catalog is vendored with a digest, like every other borrowed file", and PROVENANCE.md records digests for the ten CSVs and not for catalog.yaml or the H1 package. The control was claimed and not met.


What this cost, and what it bought

The review found five fatal defects in numbers that were one step from another repository's design decision, and four of the five were errors of the exact class this project has been cataloguing since ADR-0018 — correct computation, wrong subject.

The instrument caught the instrument's author. CB-WP-0039 was itself written to correct CB-EV-0030, and it introduced worse errors than the ones it fixed. The lesson is not "review works" — it is that a pass written to correct a previous pass inherits none of its caution.