diff --git a/specs/InnerLoop.md b/specs/InnerLoop.md index de5d34f..e0df11d 100644 --- a/specs/InnerLoop.md +++ b/specs/InnerLoop.md @@ -106,11 +106,40 @@ evidence verdict for those rows caps at `parity`, never `better`. ### Step 2 — Approve (adversarial review) For tier-L passes, approval is earned through an **adversarial review**: a -separate session (or agent), given only the survey document, attempts to -break it — an omitted candidate, a stale or unverifiable benchmark, an -unmeasured claim presented as measured. Exactly **one round**: challenge, -then response. The survey is approvable only when every challenge is either -answered with evidence or conceded and folded into the survey. +separate session (or agent) attempts to break the work. Exactly **one +round**: challenge, then response. The work is approvable only when every +challenge is either answered with evidence or conceded and folded in. + +**The review target follows the risk.** Reviewing the survey was the +original rule, and it is the wrong target for a capability whose claim +rests on numbers — every serious error in this project has been in +measurement or build configuration, not in prose. + +| the claim rests on | the reviewer is given | and must attempt | +|---|---|---| +| a survey of candidates | the survey | an omitted candidate; a stale or unverifiable benchmark; an unmeasured claim presented as measured | +| **numbers** | the survey **and the harness and the evidence file** | **reproduce the number independently**; state what the harness would report if the work silently stopped | + +**What review cannot do — state this, do not discover it.** A reviewer +re-derives the author's claims and therefore inherits the author's +sampling. So: + +> **The reviewer re-derives on a different sample than the author used.** +> Where only one sample exists, the review says so rather than reporting a +> clean verify. + +*(v1.1, from CB-WP-0002: the dedup invariant was verified on the main +transcript by the survey — 206/206 groups — and independently re-verified +by the reviewer, who used the same transcript. It is false in the +8-response `subagents/` tree neither examined. Two independent checks, +one blind spot, because both sampled the same way. Only an assertion +running over all the data at execution time caught it.)* + +**What review demonstrably does do.** Measured across two passes: +**$0.66** and **$1.11**, roughly 1% of each pass, each finding +approval-blocking defects — in CB-WP-0002 a target that would have made +the evidence file certify a broken collector. The step pays for itself by +a wide margin and the cost is not a reason to skip it. **Documentation requirement:** the research process, the challenge, and the resulting improvements to the research are each documented in timestamped diff --git a/workplans/CB-WP-0003-loop-hardening.md b/workplans/CB-WP-0003-loop-hardening.md index a4ac8c8..bdb52f1 100644 --- a/workplans/CB-WP-0003-loop-hardening.md +++ b/workplans/CB-WP-0003-loop-hardening.md @@ -124,7 +124,7 @@ dedup. ```task id: CB-WP-0003-T03 -status: todo +status: done priority: medium state_hub_task_id: "1365b35d-4539-489a-beff-c072221e4702" ```