---
id: FEP-WP-0004
type: workplan
title: "Evaluate retrieval and real repository applicability"
domain: infotech
repo: frontend-patterns
status: proposed
owner: codex
topic_slug: canon
created: "2026-09-24"
updated: "2026-09-27"
state_hub_workstream_id: "831bbc30-30c0-5df3-a315-5355cd3b8ee5"
---

# Evaluate retrieval and real repository applicability

Part of `research/plan/ResearchProgram.md`. Phase prerequisite: FEP-WP-0003's evidence
gate. This is planned follow-up, not an active background execution request.
Individual task descriptions identify any work that can start earlier.

## Validate metadata and export the graph

```task
id: FEP-WP-0004-T01
status: todo
priority: high
state_hub_task_id: "4fec98f6-fee2-52fc-a01c-f490c79ee0f8"
```

After the schema and seed examples pass M1, implement the smallest validator/exporter needed to check unique IDs, allowed values, relation targets and evidence references. Add meaningful invalid and valid fixtures; preserve prose and source provenance.

## Evaluate held-out retrieval cases

```task
id: FEP-WP-0004-T02
status: todo
priority: medium
state_hub_task_id: "4bdd4c53-fcdb-5dfd-9a37-ebc3fee4d445"
```

After M2, freeze a corpus revision and evaluate at least six unseen prompts following ValidationCases.md. Record expected alternatives, abstentions, unsupported claims and missed constraints; compare with manual navigation. Address Q07.

## Test against real repository revisions

```task
id: FEP-WP-0004-T03
status: todo
priority: medium
state_hub_task_id: "ab66848b-1045-54dd-9711-52c5560faafa"
```

Study at least two contrasting accessible repositories at recorded revisions. Separate observed implementation from inferred intent. Write research/reviews/M3-EmpiricalReview.md, publish limitations, and hand remaining findings into live work records.

## User-directed early reuse loop — 2026-09-27

WP-0008 owns an early qualified-reference pilot and consumer outcome instrumentation,
authorized by the user's request for autonomous ecosystem improvement. It may proceed
before M1/M2. It is distinct from this workplan's canonical export and held-out M3
claims; it does not rename consumed feedback into unseen evaluation cases. This plan
remains proposed with its original task gates. Routine reuse must not await those
claims or require a human survey; preserve evidence provenance and honest unknowns.