Build reproducible capability advisor and scheduled source capture

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97

docs/adr/0002-reproducible-advisor-and-source-capture.md

@@ -0,0 +1,47 @@
+# ADR 0002 — Reproducible advisor and source capture
+
+Accepted: 2026-09-13.
+
+The v0.2 CLI uses standard-library Python and content-addressed JSON. Source bytes
+are retained by hash. Capture receipts record requested/final URLs, publisher,
+retrieval time, media type and change detection; capture does not certify a claim.
+Changed sources create review-needed observations rather than invented extracted facts.
+
+`advisor-v0.2.json` is the executable policy. It carries forward the baseline
+selection weights, inclusive .90 Topnotch ratio and risk thresholds. It adds a
+90-day evidence freshness limit, a .15 construct-overlap redundancy penalty,
+and a minimum of 20 exact-task/profile outcomes. Earlier YAML remains an immutable
+design baseline. v0.2 is explicitly provisional and curator-driven.
+
+Five distinct nonaggregate instruments are selected greedily with construct-based
+task relevance, curator-assigned health and redundancy penalties. Raw scores use
+an explicit increasing linear scale per benchmark version. Missing scores are not
+imputed. Nonlinear metrics, Elo, lower-is-better metrics and composite indices need
+separate adapters before use. Pending benchmark versions cannot support ranking.
+
+Eligible means all declared task constraints match evidenced deployment attributes.
+Minima use `min_` keys; other constraints require exact equality. Production catalog
+attributes require curator review; available does not imply requester access.
+Fitness is comparative, not calibrated task success. Rankings cover only evidenced,
+eligible candidates, not the entire model market.
+
+Costs are explicit workload scenarios: disjoint input/cache/output buckets, tools,
+declared attempts and verification. Output includes billable reasoning. Unknown or
+conflicting prices prevent claims of cheapest. These are not estimated p50/p90s.
+
+Outcomes bind a recommendation, exact task and exact profile snapshot. Unique run
+IDs prevent duplicate evidence. Wilson bounds are reported over matching observed
+trials; Sufficient requires the lower bound above the policy threshold, with at
+least 20 trials. High-assurance also requires independent-verification assertions.
+This assumes representative independent trials; it is not a guarantee or a
+learned cross-task calibration model. Real operational outcome data is still absent.
+
+Recommendations freeze task, catalog, policy, outcomes and report. Canonical key
+ordering makes replay deterministic. Source captures and bytes remain referenced
+by immutable IDs. Replay checks algorithm version and rejects divergence.
+
+The scheduler checks the latest due daily/weekly slot in Europe/Berlin, catches up
+one slot after downtime, serializes runs, persists successes, and retries failures
+on later invocations. Each source fetch allows three bounded attempts and 5 MiB.
+Existing digest files are preserved. Receipt history distinguishes a new draft
+from an existing preserved artifact. Editorial synthesis remains a human/agent step.

docs/data-contracts.md

@@ -0,0 +1,94 @@
+# Data contracts and commands
+
+Python 3.11+; standard library only. Run commands from the checkout root.
+
+## Source collection
+
+```bash
+python3 -m intelligence_radar ingest catalog/sources.json
+python3 -m intelligence_radar validate
+make check
+```
+
+Source registry: `sources` array with unique `id`, `publisher`, and HTTPS `url`.
+The registry is trusted operator configuration; review URLs before adding sources.
+Captures live in `data/captures/<id>.json`; original bytes are in
+`data/artifacts/<content_sha256>`. Preserve both for replay. `ingestion-runs` records
+success/failure per source. HTML changes may be incidental, not newsworthy.
+
+Review actual source content before marking structured facts `reviewed`. Each
+result, price and availability item requires `capture_id`, timezone-aware
+`observed_at`, `review_status: reviewed`, and a named `reviewer`. Timestamps cannot
+precede the supporting capture date. An evidence ID proves linkage, not correct
+extraction. The curated initial catalog documents its missing measurement data.
+
+## Capability Advisor
+
+```bash
+python3 -m intelligence_radar advise examples/source-backed-task.json \
+  --catalog catalog/reviewed-2026-09-13.json --as-of 2026-09-13
+python3 -m intelligence_radar report <recommendation-id>
+python3 -m intelligence_radar replay <recommendation-id>
+```
+
+The source-backed example deliberately withholds winners. Its candidate benchmarks
+lack verified versions/results and its deployment access has not been established.
+This is distinct from the synthetic demonstration:
+
+```bash
+make demo-advisor
+```
+
+Task JSON: objective, acceptance, environment, risk, capability weights (0–1), hard
+constraints and workload. See examples/task.json. Input and cache token buckets are
+disjoint; output includes reasoning. Attempts are a declared scenario, never `1/fitness`.
+
+Catalog JSON: benchmark definitions (id/version/constructs/health/limitations/scale),
+exact profiles (id/model/harness/effort/provider/deployment/attributes/availability),
+results (profile_id/benchmark_id/benchmark_version/raw_score/evidence), and prices
+(profile_id/currency/valid_from/valid_until/usd_per_million/evidence). See
+examples/advisor-catalog.json for a complete synthetic shape. A synthetic catalog
+requires explicit `--synthetic` and never mixes with production evidence.
+
+For benchmark health, provide all non-task-relevance weights named in the active
+policy. Values are curator judgments, not automated validation. Duplicate IDs are
+rejected. Missing, stale, future or conflicting results remain unknown. Price
+validity is date-inclusive and was deliberately limited to the review day in the
+initial live catalog. New days need new observations.
+
+## Actual outcomes
+
+```bash
+python3 -m intelligence_radar outcome /tmp/outcome.json
+```
+
+An outcome contains recommendation_id, profile_id, unique run_id, accepted (boolean),
+independent_verification (boolean), verifier_evidence (artifact reference or notes),
+actual_usd, runtime_seconds, and completed_at (timezone-aware timestamp).
+Outcomes are immutable and run IDs are idempotent; changed duplicate runs are rejected.
+Keep the verifier artifact where its reference can be resolved. These are operator
+assertions. No actual provider execution or paid inference happens in this command.
+
+A subsequent recommendation uses matching outcomes available at its cutoff.
+A historical replay uses its original frozen set. The current baseline reports
+95% Wilson intervals, not a learned cross-task success predictor.
+
+## Scheduled operation
+
+```bash
+python3 -m intelligence_radar run-due
+python3 -m intelligence_radar health
+systemctl --user status intelligence-radar.timer
+```
+
+`ops/schedule.json` configures daily 21:30 and Sunday 20:00 Europe/Berlin.
+The shipped user timer checks every 15 minutes while this workstation's user service
+manager is running. It does not promise execution while the workstation is off.
+Each job is at most once per successful slot; failures remain retryable. Only the
+latest missed slot is caught up. Runtime state lives in ignored `.runtime/`;
+`data/scheduler-runs/` retains auditable receipts. Existing digests are preserved,
+so review the receipt's digest_status before assuming a new draft was produced.
+
+Install the reviewed units with `make install-timer`; uninstall/disable with
+`systemctl --user disable --now intelligence-radar.timer`. That command leaves
+source evidence and publications intact. No external app notifications are sent.

docs/evidence/source-comparison-2026-09-13.md

@@ -0,0 +1,28 @@
+# Capability comparison — 2026-09-13
+
+Source-backed comparison; evidence gaps remain explicit.
+
+Recommendation: `024a468396eb7d73840df04271cff4c42cd14ced1b249810c399b5a565dea5f1`
+Policy: `advisor-v0.2`
+
+| Profile | Eligible | Fitness proxy | Scenario USD | Outcome trials |
+| --- | --- | --- | --- | --- |
+| claude-sonnet-5-api-default | False | unknown | 0.0700 | 0 |
+| claude-haiku-4-5-api-default | False | unknown | 0.0500 | 0 |
+
+Frontier: undetermined
+Efficient: undetermined
+Budget: undetermined
+
+## Evidence gaps
+
+- Fewer than five relevant distinct measurements; ranking withheld.
+- No eligible profile has complete benchmark evidence.
+- no_evidenced_sufficient_model
+- claude-sonnet-5-api-default: constraint min_context not satisfied; constraint requester_access not satisfied
+- claude-sonnet-5-api-default: missing scicode, terminal-bench, browsecomp, bfcl, swe-bench-verified
+- claude-haiku-4-5-api-default: constraint min_context not satisfied; constraint requester_access not satisfied
+- claude-haiku-4-5-api-default: missing scicode, terminal-bench, browsecomp, bfcl, swe-bench-verified
+
+Cost is a declared token/tool/retry/verification scenario, not a measured success-cost distribution.
+Source review does not prove requester access or task success. See saved inputs for capture references.

docs/evidence/v0.2-validation.md

@@ -0,0 +1,32 @@
+# v0.2 implementation evidence — 2026-09-13
+
+`make check`: 21 behavior tests cover rank/eligibility, missing or stale evidence,
+price conflicts, numeric validation, aggregate exclusion, outcome deduplication,
+empirical bounds, historical replay including JSON ordering, source integrity,
+retry/size limits, scheduler idempotence and ISO-week handling. Repository checks
+validate archive hashes, observation envelopes, retained captures and report replay.
+
+`make demo-advisor`: full synthetic ranking returns the fixture Frontier and
+Efficient, withholds Budget without outcomes, and reproduces the saved report.
+The fixture remains isolated from the operational ledger.
+
+Four primary pages were captured from SWE-bench and Anthropic. Original bytes,
+retrieval receipts and ingestion results are retained under data/. A source-backed
+comparison is in source-comparison-2026-09-13.md. It explicitly withholds winners
+because matching benchmark measurements and requester access are not evidenced.
+
+The initial price search contained stale promotion information. Inspection of
+[Anthropic's launch-page changelog](https://www.anthropic.com/news/claude-sonnet-5)
+resolved it: the August update superseded the previously scheduled price increase.
+The reviewed price observation reflects the current captured
+[pricing page](https://claude.com/pricing), with a one-day validity policy.
+
+`run-due` completed both 2026-09-13 daily and 2026-W37 weekly jobs successfully.
+Their receipts say `existing_preserved`, accurately distinguishing the imported
+historical draft from a newly generated publication. A repeat invocation performed
+no completed jobs. No external app automation or notifications were used.
+
+The user timer was installed on bnt-lap001. `systemctl --user` reports timer
+ActiveState=active, SubState=waiting; an explicit service invocation returned
+Result=success and ExecMainStatus=0. The timer checks every 15 minutes while the
+workstation user manager runs; it is not a continuously available hosted worker.