coulomb/frontend-patterns
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Freeze replication-03 with clarified criteria (FEP-WP-0009-T02)

Browse source 
Owner decisions D1-D3 applied; execution not authorized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 182330@bnt-lap001
Assistant-Session: 242866cf-789d-4e24-ac5e-0a1d7b47a701
This commit is contained in:
tegwick 2026-10-20 00:19:00 +02:00
parent 559d9a4f99
commit f8c3ce3938
28 changed files with 2096 additions and 540 deletions
Download patch file Download diff file Expand all files Collapse all files

41
research/evaluations/language-replication-03/README.md
View file

@ -1,6 +1,6 @@
# Replication 03 — draft criteria revision
Status: **draft, not frozen, not authorized for execution.** Owner: FEP-WP-0009 T02.
Status: **frozen 2026-10-20, not authorized for execution.** Owner: FEP-WP-0009 T02.
Prepared 2026-10-20 by Claude Code (author-model) at the owner's request, using no
model calls. It applies the results of the pilot-01 human review and adjudication
([review](../language-pilot-01/human-review/review-bernd-worsch-2026-10-19.json),
@ -50,20 +50,31 @@ The pilot-01 human review worked only in interview form, with plain-language con
per scenario. A replication-03 worksheet should include that context, written before
any answers exist, without hints about any answer.
## Decisions needed before freezing
## Owner decisions (2026-10-20)
- **D1 Reuse briefs.** The six replication-02 briefs were never shown to a solver, so
reuse keeps them unseen. The author has seen them, as before. Recommended: reuse.
- **D2 "Impossible" claims.** Replication-02 S04-A4 scored "recovery is impossible"
as a miss. Under G5 it becomes an observation. Recommended: apply G5.
- **D3 Reviewer.** Owner review again, an external reviewer, or both. An external
reviewer strengthens M1 independence.
- **D4 Execution.** A new run identity, restored llm-connect reachability from this
workstation and a fresh call-budget authorization (12 capped calls, no retries,
as before).
D1 reuse briefs: yes. D2 overstated-caution rule G5: yes. D3 reviewers: owner and an
external human reviewer, independently. D4 execution: not yet authorized. Details and
the binding rules are in the [protocol](protocol.md); G6 there supersedes the draft
wording above (a binary value is still required alongside an ambiguity entry).
## Not yet done
## Commands
No freeze, prompts, assignment, runner wiring or tests. These follow D1–D4 and reuse
the replication-02 tooling with a new packet identity. Packs stay byte-identical to
replication-02 unless a decision changes them.
From the repository root, without inference:
```bash
make replication-03-check replication-03-test
```
Execution needs, in order: `make inference-preflight` passing against a reachable
llm-connect, fresh explicit authorization of up to 12 capped calls with no retries,
and a committed freeze. Then:
```bash
PYTHONDONTWRITEBYTECODE=1 ../infospace-bench/.venv/bin/python scripts/language_replication_03.py run --max-calls 12
```
After answers exist, build the reviewer worksheet from `masked/` plus each scenario's
`reviewer_context` and criteria examples; each reviewer copies `scores-template.json`.
`scripts/language_replication_03.py scores-check|summarize --scores <file>` validate
and summarise a sheet. Replication-02 stays frozen and verifiable with
`make replication-check`.