intelligence-radar/docs/data-contracts.md

Data contracts and commands

Python 3.11+; standard library only. Run commands from the checkout root.

Source collection

python3 -m intelligence_radar ingest catalog/sources.json
python3 -m intelligence_radar validate
make check

Source registry: sources array with unique id, publisher, and HTTPS url. The registry is trusted operator configuration; review URLs before adding sources. Captures live in data/captures/<id>.json; original bytes are in data/artifacts/<content_sha256>. Preserve both for replay. ingestion-runs records success/failure per source. HTML changes may be incidental, not newsworthy.

Review actual source content before marking structured facts reviewed. Each result, price and availability item requires capture_id, timezone-aware observed_at, review_status: reviewed, and a named reviewer. Timestamps cannot precede the supporting capture date. An evidence ID proves linkage, not correct extraction. The curated initial catalog documents its missing measurement data.

Capability Advisor

python3 -m intelligence_radar advise examples/source-backed-task.json \
  --catalog catalog/reviewed-2026-09-13.json --as-of 2026-09-13
python3 -m intelligence_radar report <recommendation-id>
python3 -m intelligence_radar replay <recommendation-id>

The source-backed example deliberately withholds winners. Its candidate benchmarks lack verified versions/results and its deployment access has not been established. This is distinct from the synthetic demonstration:

make demo-advisor

Task JSON: objective, acceptance, environment, risk, capability weights (0–1), hard constraints and workload. See examples/task.json. Input and cache token buckets are disjoint; output includes reasoning. Attempts are a declared scenario, never 1/fitness.

Catalog JSON: benchmark definitions (id/version/constructs/health/limitations/scale), exact profiles (id/model/harness/effort/provider/deployment/attributes/availability), results (profile_id/benchmark_id/benchmark_version/raw_score/evidence), and prices (profile_id/currency/valid_from/valid_until/usd_per_million/evidence). See examples/advisor-catalog.json for a complete synthetic shape. A synthetic catalog requires explicit --synthetic and never mixes with production evidence.

For benchmark health, provide all non-task-relevance weights named in the active policy. Values are curator judgments, not automated validation. Duplicate IDs are rejected. Missing, stale, future or conflicting results remain unknown. Price validity is date-inclusive and was deliberately limited to the review day in the initial live catalog. New days need new observations.

Actual outcomes

python3 -m intelligence_radar outcome /tmp/outcome.json

An outcome contains recommendation_id, profile_id, unique run_id, accepted (boolean), independent_verification (boolean), verifier_evidence (artifact reference or notes), actual_usd, runtime_seconds, and completed_at (timezone-aware timestamp). Outcomes are immutable and run IDs are idempotent; changed duplicate runs are rejected. Keep the verifier artifact where its reference can be resolved. These are operator assertions. No actual provider execution or paid inference happens in this command.

A subsequent recommendation uses matching outcomes available at its cutoff. A historical replay uses its original frozen set. The current baseline reports 95% Wilson intervals, not a learned cross-task success predictor.

Scheduled operation

python3 -m intelligence_radar run-due
python3 -m intelligence_radar health
systemctl --user status intelligence-radar.timer

ops/schedule.json configures daily 21:30 and Sunday 20:00 Europe/Berlin. The shipped user timer checks every 15 minutes while this workstation's user service manager is running. It does not promise execution while the workstation is off. Each job is at most once per successful slot; failures remain retryable. Only the latest missed slot is caught up. Runtime state lives in ignored .runtime/; data/scheduler-runs/ retains auditable receipts. Existing digests are preserved, so review the receipt's digest_status before assuming a new draft was produced.

Install the reviewed units with make install-timer; uninstall/disable with systemctl --user disable --now intelligence-radar.timer. That command leaves source evidence and publications intact. No external app notifications are sent.

Portable evidence bundles

python3 -m intelligence_radar export <recommendation-id> /tmp/report.tar
python3 -m intelligence_radar --root /tmp/radar-copy import /tmp/report.tar

Bundles retain the frozen recommendation, parent recommendations, referenced capture receipts and original source artifacts. Import validates paths, checksums, content IDs, dependency presence and replay in staging before writing to the ledger. Existing conflicting evidence is refused. Import is bounded to 128 MiB and 10,000 members; export is deterministic. External verifier references in outcome notes are not automatically packaged.

Source configuration optionally accepts max_bytes from 1 through 20 MiB; the default remains 5 MiB.

Budgeted provider work samples

python3 -m intelligence_radar trials --config ops/openrouter-pilot.json \
  --suite examples/work-sample-suite.json --pilot

This command makes paid OpenRouter calls using OPENROUTER_API_KEY from the process environment. The recorded authorization is USD 10 total for this campaign. --pilot selects the first two cases for each configured model; omit it for all 20 cases. Do not change the campaign identifier to bypass existing accounting.

Reservations precede requests, completed trials retain response/usage/verifier results, and successful identical runs return their retained result. There are no automatic inference retries. Unsettled reservations stop new work; an actual cost exceeding its reservation halts the campaign across restarts. The prompt reservation uses UTF-8 bytes plus framing allowance and configured provider price caps; it is an estimate, not a provider-enforced account spending limit.

The interrupted first request returned HTTP 401. Before resuming, supply a valid credential and reconcile the retained reservation against provider billing. Reconciliation is currently manual and must preserve old events. Never delete reservations or assume a timeout was free. Controlled work samples and close variants are not representative production outcomes; they are not automatically added to advisor calibration.