# Replication 03 — draft criteria revision

Status: **draft, not frozen, not authorized for execution.** Owner: FEP-WP-0009 T02.
Prepared 2026-10-20 by Claude Code (author-model) at the owner's request, using no
model calls. It applies the results of the pilot-01 human review and adjudication
([review](../language-pilot-01/human-review/review-bernd-worsch-2026-10-19.json),
[adjudication](../language-pilot-01/human-review/disagreements-2026-10-19.json)).
Replication-02 and its spent run identities stay unchanged.

## Why a revision

Replication-02 already uses six single-obligation criteria per scenario. The pilot-01
adjudication found 11 of 16 disagreements were caused by rubric wording, not by the
reviewers. Four kinds of ambiguity recur, and several replication-02 criteria still
admit them:

| Ambiguity found in pilot-01 | Example | Replication-02 criteria affected |
| --- | --- | --- |
| Generic vs specific contract naming | S01/S02/S04 Q | S01-A3, S02-A6, S03-A5, S05-A6, S06-A6 |
| Condition stated elsewhere vs in the recommendation | S03 R | S01-A4, S01-A5 |
| End state vs stated atomic step | S05 C1 | S03-A1 |
| Flat assertion of an unknown fact | S01-V "stale"; S03-V "must be retained" | S01-A6, S06-A5 |

The reviewer's recurring guidance was to pair abstract wording with a concrete
example. Each criterion in [scenarios.draft.json](scenarios.draft.json) therefore
carries `qualifies` and `does_not_qualify` examples, a `placement` and a
`change_from_r02` note. Briefs and prohibited lists are copied unchanged.

## Draft scoring rules

- **G1 Placement.** A `recommendation` criterion is met only inside the recommended
  state or action. A matching statement only in a later questions section does not count.
  An `anywhere` criterion may be met in any section.
- **G2 Named subject.** A contract-identification criterion requires the guarantee's
  subject (for example lookup, replay window, read-after-write). A generic "define the
  contract" does not qualify. Listed alternatives joined by "or" are each sufficient.
- **G3 Examples are illustrative.** Equivalent wording qualifies; the examples show
  where the line is, not required phrases.
- **G4 Unknown facts.** Asserting an unspecified fact as known (stale, permanent,
  succeeded) is recorded as an unsupported claim.
- **G5 Overstated caution** (for example "cannot recover" where it is only untested)
  is recorded as a separate observation, not an unsupported claim, following the
  owner's S06-V adjudication.
- **G6 Ambiguity is a valid result.** A reviewer may mark a criterion ambiguous (X)
  instead of forcing a score; ambiguity counts are reported per criterion.

## Reviewer support

The pilot-01 human review worked only in interview form, with plain-language context
per scenario. A replication-03 worksheet should include that context, written before
any answers exist, without hints about any answer.

## Decisions needed before freezing

- **D1 Reuse briefs.** The six replication-02 briefs were never shown to a solver, so
  reuse keeps them unseen. The author has seen them, as before. Recommended: reuse.
- **D2 "Impossible" claims.** Replication-02 S04-A4 scored "recovery is impossible"
  as a miss. Under G5 it becomes an observation. Recommended: apply G5.
- **D3 Reviewer.** Owner review again, an external reviewer, or both. An external
  reviewer strengthens M1 independence.
- **D4 Execution.** A new run identity, restored llm-connect reachability from this
  workstation and a fresh call-budget authorization (12 capped calls, no retries,
  as before).

## Not yet done

No freeze, prompts, assignment, runner wiring or tests. These follow D1–D4 and reuse
the replication-02 tooling with a new packet identity. Packs stay byte-identical to
replication-02 unless a decision changes them.