coulomb/frontend-patterns
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Preregister fresh atomic-criteria replication with 12-call budget

Browse source 
Assistant: codex
Assistant-Model: gpt-6.1-sol
Assistant-Session: 01a1045b-933d-7aa1-b2db-104557487a79
This commit is contained in:
tegwick 2026-10-04 11:08:31 +02:00
parent bd85fcd179
commit 1e2741a959
30 changed files with 2229 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

39
research/evolution/cycles/2026-10-04-presentation-replication.md Normal file
View file

@ -0,0 +1,39 @@
# LEARN-0019 — Fresh atomic-criteria presentation replication
Status: preregistered before inference. Owner/reviewer: Codex, author-model.
Date: 2026-10-04. Work owner: FEP-WP-0009 T02.
Stream: cross-stream. New comparison group: presentation-atomic-replication-v2.
This protocol differs from the consumed retrospective pilot and from static/runtime
mechanism studies; it cannot be pooled into their comparison windows.
Question: does named organization change constraint/recovery recommendations versus
the same guidance under numbered headings on fresh authored variants? Both conditions
share eight exact bodies and caveats. Previous pilot-01 model-review adjudication was
18/24 in each condition; that supplies no naming benefit or equivalence claim.
Material disconfirmation of an assumed benefit includes tied/mixed criterion outcomes,
additional unsupported guarantees or common failures despite shared guidance.
Execute the [frozen replication packet](../../evaluations/language-replication-02/protocol.md)
with 12 isolated llm-connect calls, requested 1,400-output-token caps and no retries,
as explicitly authorized by the user. Six pairs have six atomic criteria each,
not the old pilot's four. Commit inputs/runner/prompts before execution. Retain
failed generations and exclude their pairs from quality comparison. Score masked
outputs against the exact rubric and commit initial author-model judgments before
unmasking. Exposure and uncertainty stay explicit; human validation remains separate.
Acceptance: a retained bounded replication, descriptive per-criterion/scenario
comparison and supported editorial conclusion. No M1 pass, independent human evidence,
production applicability or general efficacy estimate follows. A consistent descriptive
advantage warrants independent follow-up; tied/mixed outcomes retain hypotheses and
are inconclusive. Shared errors may motivate a separate correction, not post-hoc rubric
changes or selected reruns. One question counts one outcome, not 72 criterion ratings.
Active research time is unknown: no timer was started before preparation. Do not infer
it from the preregistration commit, chat duration, tokens or check subprocesses. Sum
measured request durations as execution minutes separately; use token/cost values
only where actual receipts establish them. No extra model review is within this grant.
Outcome is recorded after the authorized run in the replication review and LEARN-0019
ledger entry. WP-0009 T01 retains independent human pilot-01 ratings; T03 retains
M1 re-assessment. WP-0008 retains actual consumer decision/outcome evidence. A new
comparison group does not close WP-0006's comparable-window requirement.