tegwick
1e2741a959
Assistant: codex Assistant-Model: gpt-6.1-sol Assistant-Session: 01a1045b-933d-7aa1-b2db-104557487a79
39 lines
2.6 KiB
Markdown
39 lines
2.6 KiB
Markdown
# LEARN-0019 — Fresh atomic-criteria presentation replication
|
|
|
|
Status: preregistered before inference. Owner/reviewer: Codex, author-model.
|
|
Date: 2026-10-04. Work owner: FEP-WP-0009 T02.
|
|
Stream: cross-stream. New comparison group: presentation-atomic-replication-v2.
|
|
This protocol differs from the consumed retrospective pilot and from static/runtime
|
|
mechanism studies; it cannot be pooled into their comparison windows.
|
|
|
|
Question: does named organization change constraint/recovery recommendations versus
|
|
the same guidance under numbered headings on fresh authored variants? Both conditions
|
|
share eight exact bodies and caveats. Previous pilot-01 model-review adjudication was
|
|
18/24 in each condition; that supplies no naming benefit or equivalence claim.
|
|
Material disconfirmation of an assumed benefit includes tied/mixed criterion outcomes,
|
|
additional unsupported guarantees or common failures despite shared guidance.
|
|
|
|
Execute the [frozen replication packet](../../evaluations/language-replication-02/protocol.md)
|
|
with 12 isolated llm-connect calls, requested 1,400-output-token caps and no retries,
|
|
as explicitly authorized by the user. Six pairs have six atomic criteria each,
|
|
not the old pilot's four. Commit inputs/runner/prompts before execution. Retain
|
|
failed generations and exclude their pairs from quality comparison. Score masked
|
|
outputs against the exact rubric and commit initial author-model judgments before
|
|
unmasking. Exposure and uncertainty stay explicit; human validation remains separate.
|
|
|
|
Acceptance: a retained bounded replication, descriptive per-criterion/scenario
|
|
comparison and supported editorial conclusion. No M1 pass, independent human evidence,
|
|
production applicability or general efficacy estimate follows. A consistent descriptive
|
|
advantage warrants independent follow-up; tied/mixed outcomes retain hypotheses and
|
|
are inconclusive. Shared errors may motivate a separate correction, not post-hoc rubric
|
|
changes or selected reruns. One question counts one outcome, not 72 criterion ratings.
|
|
|
|
Active research time is unknown: no timer was started before preparation. Do not infer
|
|
it from the preregistration commit, chat duration, tokens or check subprocesses. Sum
|
|
measured request durations as execution minutes separately; use token/cost values
|
|
only where actual receipts establish them. No extra model review is within this grant.
|
|
|
|
Outcome is recorded after the authorized run in the replication review and LEARN-0019
|
|
ledger entry. WP-0009 T01 retains independent human pilot-01 ratings; T03 retains
|
|
M1 re-assessment. WP-0008 retains actual consumer decision/outcome evidence. A new
|
|
comparison group does not close WP-0006's comparable-window requirement.
|