frontend-patterns/research/evolution/cycles/2026-10-04-presentation-replication.md
tegwick 1e2741a959 Preregister fresh atomic-criteria replication with 12-call budget
Assistant: codex
Assistant-Model: gpt-6.1-sol
Assistant-Session: 01a1045b-933d-7aa1-b2db-104557487a79
2026-10-04 11:08:31 +02:00

2.6 KiB

LEARN-0019 — Fresh atomic-criteria presentation replication

Status: preregistered before inference. Owner/reviewer: Codex, author-model. Date: 2026-10-04. Work owner: FEP-WP-0009 T02. Stream: cross-stream. New comparison group: presentation-atomic-replication-v2. This protocol differs from the consumed retrospective pilot and from static/runtime mechanism studies; it cannot be pooled into their comparison windows.

Question: does named organization change constraint/recovery recommendations versus the same guidance under numbered headings on fresh authored variants? Both conditions share eight exact bodies and caveats. Previous pilot-01 model-review adjudication was 18/24 in each condition; that supplies no naming benefit or equivalence claim. Material disconfirmation of an assumed benefit includes tied/mixed criterion outcomes, additional unsupported guarantees or common failures despite shared guidance.

Execute the frozen replication packet with 12 isolated llm-connect calls, requested 1,400-output-token caps and no retries, as explicitly authorized by the user. Six pairs have six atomic criteria each, not the old pilot's four. Commit inputs/runner/prompts before execution. Retain failed generations and exclude their pairs from quality comparison. Score masked outputs against the exact rubric and commit initial author-model judgments before unmasking. Exposure and uncertainty stay explicit; human validation remains separate.

Acceptance: a retained bounded replication, descriptive per-criterion/scenario comparison and supported editorial conclusion. No M1 pass, independent human evidence, production applicability or general efficacy estimate follows. A consistent descriptive advantage warrants independent follow-up; tied/mixed outcomes retain hypotheses and are inconclusive. Shared errors may motivate a separate correction, not post-hoc rubric changes or selected reruns. One question counts one outcome, not 72 criterion ratings.

Active research time is unknown: no timer was started before preparation. Do not infer it from the preregistration commit, chat duration, tokens or check subprocesses. Sum measured request durations as execution minutes separately; use token/cost values only where actual receipts establish them. No extra model review is within this grant.

Outcome is recorded after the authorized run in the replication review and LEARN-0019 ledger entry. WP-0009 T01 retains independent human pilot-01 ratings; T03 retains M1 re-assessment. WP-0008 retains actual consumer decision/outcome evidence. A new comparison group does not close WP-0006's comparable-window requirement.