frontend-patterns/workplans/FEP-WP-0009-m1-independent-validation.md

id	type	title	domain	repo	status	owner	topic_slug	origin	origin_ref	created	updated	state_hub_workstream_id
FEP-WP-0009	workplan	Obtain independent validation for the M1 seed gate	infotech	frontend-patterns	blocked	codex	canon	residual	FEP-WP-0002	2026-10-03	2026-10-20	78374f2e-006f-5faf-9053-786d54a4488a

Obtain independent validation for the M1 seed gate

Residual of FEP-WP-0002, handed off at its closeout on 2026-10-03 at the user's direction. WP-0002 delivered its phase acceptance (eight connected hypothesis drafts, VC-01/VC-02, M1-SeedReview), but the M1 gate remains revise/defer. This workplan owns the evidence that gate still lacks. Transfer does not satisfy any obligation; downstream tasks that waited on FEP-WP-0002-T03 now wait on T03 here.

Independent human review of language pilot 01

id: FEP-WP-0009-T01
status: done
needs_human: true
priority: high
blocking_reason: "Needs an independent human reviewer to rate the masked pilot-01 answers; no model may supply or infer these scores."
state_hub_task_id: "c7e0d957-d7ee-591f-a086-f7c262a2e27a"
decision_id: "6f698650-de45-4402-a84f-ea19ae5f96af"

Use research/evaluations/language-pilot-01/human-review/README.md and its empty score sheet. Retain disagreements and adjudicate against the frozen rubric. The submitted-review-01 worksheet was a model review and does not count.

Fresh-scenario replication with atomic criteria

id: FEP-WP-0009-T02
status: wait
blocking_reason: "Replication-03 packet frozen (2026-10-20). Needs llm-connect reachable from the workstation and fresh explicit authorization of up to 12 capped calls; no retries."
priority: medium
state_hub_task_id: "8e0ac763-eba4-5625-ad63-af53322bcac4"

Preregister unseen scenarios with atomic, no-partial-credit criteria targeting policy-dependent recovery, cancellation/reversal contracts, atomic supersession and backup/restore. The pilot-01 briefs are consumed. Preparation needs no inference; any paid llm-connect execution needs explicit user authorization of its call budget.

Re-assess the M1 gate

id: FEP-WP-0009-T03
status: wait
depends_on: [FEP-WP-0009-T01, FEP-WP-0009-T02]
priority: high
blocking_reason: "Human ratings obtained (T01, owner reviewer). M1 re-review still needs the fresh replication (T02)."
state_hub_task_id: "121de012-b99a-5615-8d97-cd2f66e50cbd"

Update M1-SeedReview with pass/revise/defer per criterion from the new evidence. Hypotheses may be rejected or deferred with reasons rather than confirmed to close. Storage/receiver validation stays with FEP-WP-0003-T04.

Parked in backlog — 2026-10-03

User set this workplan to backlog for later pickup. Loose-end audit the same day found no open task in WP-0003/0004/0006/0008 finishable in this repo: each chain ends at T01's human gate here, or at external owners (GLAS-WP-0012, ACTIVITY-WP-0041) and an actual consumer for WP-0008. Those plans stay blocked; no task or workplan opened.

2026-10-04 CUST-WP-0075 owner routing: FEP-WP-0009-T01 remains a human wait under open decision 6f698650-de45-4402-a84f-ea19ae5f96af. Exact scoped human act and native receipts remain required; this metadata records no approval or execution.

Gap implementation resumed — 2026-10-04

The user asked to work the assessed gaps, reopening this parked plan. Coverage audit: WP-0008 owns real consumers/outcomes and admitted improvement; WP-0009 owns independent validation/replication; WP-0003/0004 owns broader/gated applicability; WP-0006 owns comparable learning. No uncovered ownership requires a duplicate workplan.

T02 starts with a fresh replication packet: authored variants separate from pilot-01, atomic binary criteria, equivalent named/prose guidance, frozen release and input pins, condition-isolated prompts and evidence-bearing scoring validation. Preparation makes no inference call and cannot satisfy the independent human gate. Explicit paid-call budget is requested separately; absent authorization, execution remains pending.

The user then explicitly authorized up to 12 calls, 1,400 output tokens per call, no retries. This is the fresh replication budget only, not a grant for further model reviews, autonomous editing, policy changes or human-score substitution. Freeze and commit the packet/runner before execution; reserve every attempt and retain failures.

Retained failed replication — 2026-10-04

The preregistered packet/runner was committed and synced at 1e2741a before execution. All twelve reserved attempts failed with no retained answer or provider usage receipt. Both sandbox and host health probes found the configured llm-connect endpoint unreachable. No call was retried. The budget is conservatively exhausted as twelve reserved attempts, without claiming twelve provider executions or zero cost.

Execution record retains all failures, null ratings and six excluded pairs. Packet construction, validation and bounded execution are implemented; T02 remains wait because no empirical comparison exists. Future execution needs a new registered packet/run identity and fresh explicit authorization after service restoration. T01 and T03 retain their existing gates. LEARN-0019 records one inconclusive attempted cycle, not useful validation or independent evidence.

Blocked-task review — 2026-10-04

Dependency requirements establish exact owner returns and local evidence gates for every remaining task. All unfinished tasks stay wait; this plan is blocked because no currently available implementation can meet its remaining acceptance. Completed deliverables remain done with their recorded limits. Cross-repo source/synthetic proofs and finished editorial reviews are not the missing runtime admission, actual consumer or human evidence.

Implemented scripts/inference_preflight.py under T02: one bounded sanitized health-only GET, no redirects, provider execution, reservations or retries. Frozen replication-02 inputs/runner are unchanged. A reachable service, owner-confirmed profile/cap behavior and fresh explicitly authorized packet remain required.

Dependency recheck — 2026-10-19

make inference-preflight still reports health_unreachable (zero attempts reserved); no llm-connect tunnel is configured, although the in-cluster service activity-core/llm-connect is running — the gap is workstation reachability. No independent human ratings have been supplied. T01–T03 remain wait; a fresh replication still needs restored service and new explicit execution authorization. Evidence.

Human pilot-01 review and adjudication — 2026-10-19

T01 done. Bernd Worsch (founder, repository owner) rated all 12 masked answers by interview: Claude Code presented each brief, rubric criterion and answer and recorded his judgements verbatim, supplying no scores. He declared no prior exposure to scores or the key. Review. U and V scored identically in every scenario (36 of 48 criteria met per condition); misses are concentrated in Q. Human vs editorial agreement 36/48 criteria; 4 claim disagreements. Adjudication of all 16: agent upheld 3, reviewer 2, rubric ambiguous 11. Disagreements.

Limits for T03: the reviewer is the repository owner, not an external reviewer; the M1 re-assessment must weigh that. The dominant result is rubric ambiguity in compound criteria, which supports T02's atomic, no-partial-credit design. Reviewer design notes (three-way diff, criticality-based duplicate policy, revocation quarantine, multi-step cancellation, git-style resolution menu, synchronous vs asynchronous systems, abstract wording plus concrete example) are recorded per answer as candidate input, not validated patterns. T03 still waits on T02.

Decision 6f698650-de45-4402-a84f-ea19ae5f96af resolved by Bernd Worsch: "The review is fine, the concept is sound and can be refined based on the results provided by the review and its discussion." This closes the T01 human gate; it does not pass M1.

Replication-03 criteria draft — 2026-10-20

At the owner's request, a draft packet applies the pilot-01 adjudication to replication-02's unseen briefs: per-criterion qualifies/does-not-qualify examples, placement rules and an equivalence rule for contract naming. Draft only: no freeze, prompts, run identity or execution authority. Open decisions D1–D4 (brief reuse, overstated-caution rule, reviewer, execution) precede freezing. T02 remains wait.

Replication-03 frozen — 2026-10-20

Owner decisions: reuse the unseen replication-02 briefs (D1), record overstated caution as an observation (D2), and use both the owner and an external human reviewer (D3). Replication-03 is frozen with its own identity via a wrapper that leaves replication-02's frozen bytes intact; packs are byte-identical to replication-02 and both packet checks plus tests pass. No inference was made. T02 remains wait on llm-connect reachability from the workstation and fresh explicit authorization of up to 12 capped calls (D4).