intelligence-radar/workplans/IR-WP-0004-evidence-coverage-and-field-calibration.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
IR-WP-0004	workplan	Expand verified evidence and calibrate against field outcomes	infotech	intelligence-radar	active	codex	custodian	2026-09-13	2026-09-13	9df17f9e-40b5-5df2-b3d2-a236d3915d71

Expand verified evidence and calibrate against field outcomes

Residual research and product scope from IR-WP-0003. v0.2 implements the mechanics; this work grows the evidence and operational reach without inventing calibration.

Build complete comparable profile evidence

id: IR-WP-0004-T01
status: progress
priority: high
state_hub_task_id: "4f9538f6-3bf0-513f-baa0-5540430a2254"

Review exact versions, harnesses, effort and scores for five task-relevant instruments. Expand beyond the two-profile dated price comparison. Preserve conflicting results and independently verify requester access. Acceptance: at least two eligible profiles with complete comparable evidence and a source-backed Frontier/Efficient comparison. All other imported model/benchmark claims remain research leads until reviewed.

2026-09-13: Retained two additional source captures and a vendor-reported five-instrument transcription. Exact harness/effort differences remain explicit; these results do not establish comparable OpenRouter execution profiles. T01 acceptance remains open.

Collect representative real task outcomes

id: IR-WP-0004-T02
status: wait
priority: high
state_hub_task_id: "50522d33-d3e4-5a5e-ab8c-397a38c1ab17"

Run an explicitly budgeted sample of representative tasks with retained verifier artifacts. Record successes and failures, actual cost and runtime. Evaluate selection bias and dependence before treating Wilson bounds as predictive. Develop versioned cross-task calibration and measured cost/runtime intervals only when justified.

2026-09-13: User authorized OpenRouter within USD 10. Implemented a serialized, append-only budgeted runner and 20 controlled cases across five families. The first request failed with HTTP 401; a read-only credential check also returns 401 after resumption. No completed trial or verified spend is recorded. Keep the USD 0.023712 reservation pending until billing is reconciled. Requires a valid OpenRouter credential delivered to the local environment. The discovered openrouter-llm-connect route is scoped to a separate service and is not a generic client credential. See docs/evidence/2026-09-13-resume.md.

Extend publication and hosted operation

id: IR-WP-0004-T03
status: progress
priority: medium
state_hub_task_id: "e865449d-d53d-50bc-8983-2bd3605dfaee"

Add curated source adapters and editorial synthesis, artifact retention/export, reviewed source coverage, and a deployment-independent worker if continuous uptime is required. Add decomposition comparisons against measured monolithic outcomes. Keep draft creation separate from reviewed publication and any notifications.

2026-09-13: Portable recommendation export/import implemented with checksum, path, dependency and replay checks before publication. Default make lists targets. All 32 tests and repository/archive integrity checks pass. Curated publication, hosted workers and decomposition comparisons remain open.