intelligence-radar/docs/evidence/v0.2-validation.md
tegwick 29ceb77938 Build reproducible capability advisor and scheduled source capture
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97
2026-09-13 23:07:41 +02:00

2 KiB

v0.2 implementation evidence — 2026-09-13

make check: 21 behavior tests cover rank/eligibility, missing or stale evidence, price conflicts, numeric validation, aggregate exclusion, outcome deduplication, empirical bounds, historical replay including JSON ordering, source integrity, retry/size limits, scheduler idempotence and ISO-week handling. Repository checks validate archive hashes, observation envelopes, retained captures and report replay.

make demo-advisor: full synthetic ranking returns the fixture Frontier and Efficient, withholds Budget without outcomes, and reproduces the saved report. The fixture remains isolated from the operational ledger.

Four primary pages were captured from SWE-bench and Anthropic. Original bytes, retrieval receipts and ingestion results are retained under data/. A source-backed comparison is in source-comparison-2026-09-13.md. It explicitly withholds winners because matching benchmark measurements and requester access are not evidenced.

The initial price search contained stale promotion information. Inspection of Anthropic's launch-page changelog resolved it: the August update superseded the previously scheduled price increase. The reviewed price observation reflects the current captured pricing page, with a one-day validity policy.

run-due completed both 2026-09-13 daily and 2026-W37 weekly jobs successfully. Their receipts say existing_preserved, accurately distinguishing the imported historical draft from a newly generated publication. A repeat invocation performed no completed jobs. No external app automation or notifications were used.

The user timer was installed on bnt-lap001. systemctl --user reports timer ActiveState=active, SubState=waiting; an explicit service invocation returned Result=success and ExecMainStatus=0. The timer checks every 15 minutes while the workstation user manager runs; it is not a continuously available hosted worker.