tegwick
29ceb77938
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97
94 lines
4.4 KiB
Markdown
94 lines
4.4 KiB
Markdown
# Data contracts and commands
|
||
|
||
Python 3.11+; standard library only. Run commands from the checkout root.
|
||
|
||
## Source collection
|
||
|
||
```bash
|
||
python3 -m intelligence_radar ingest catalog/sources.json
|
||
python3 -m intelligence_radar validate
|
||
make check
|
||
```
|
||
|
||
Source registry: `sources` array with unique `id`, `publisher`, and HTTPS `url`.
|
||
The registry is trusted operator configuration; review URLs before adding sources.
|
||
Captures live in `data/captures/<id>.json`; original bytes are in
|
||
`data/artifacts/<content_sha256>`. Preserve both for replay. `ingestion-runs` records
|
||
success/failure per source. HTML changes may be incidental, not newsworthy.
|
||
|
||
Review actual source content before marking structured facts `reviewed`. Each
|
||
result, price and availability item requires `capture_id`, timezone-aware
|
||
`observed_at`, `review_status: reviewed`, and a named `reviewer`. Timestamps cannot
|
||
precede the supporting capture date. An evidence ID proves linkage, not correct
|
||
extraction. The curated initial catalog documents its missing measurement data.
|
||
|
||
## Capability Advisor
|
||
|
||
```bash
|
||
python3 -m intelligence_radar advise examples/source-backed-task.json \
|
||
--catalog catalog/reviewed-2026-09-13.json --as-of 2026-09-13
|
||
python3 -m intelligence_radar report <recommendation-id>
|
||
python3 -m intelligence_radar replay <recommendation-id>
|
||
```
|
||
|
||
The source-backed example deliberately withholds winners. Its candidate benchmarks
|
||
lack verified versions/results and its deployment access has not been established.
|
||
This is distinct from the synthetic demonstration:
|
||
|
||
```bash
|
||
make demo-advisor
|
||
```
|
||
|
||
Task JSON: objective, acceptance, environment, risk, capability weights (0–1), hard
|
||
constraints and workload. See examples/task.json. Input and cache token buckets are
|
||
disjoint; output includes reasoning. Attempts are a declared scenario, never `1/fitness`.
|
||
|
||
Catalog JSON: benchmark definitions (id/version/constructs/health/limitations/scale),
|
||
exact profiles (id/model/harness/effort/provider/deployment/attributes/availability),
|
||
results (profile_id/benchmark_id/benchmark_version/raw_score/evidence), and prices
|
||
(profile_id/currency/valid_from/valid_until/usd_per_million/evidence). See
|
||
examples/advisor-catalog.json for a complete synthetic shape. A synthetic catalog
|
||
requires explicit `--synthetic` and never mixes with production evidence.
|
||
|
||
For benchmark health, provide all non-task-relevance weights named in the active
|
||
policy. Values are curator judgments, not automated validation. Duplicate IDs are
|
||
rejected. Missing, stale, future or conflicting results remain unknown. Price
|
||
validity is date-inclusive and was deliberately limited to the review day in the
|
||
initial live catalog. New days need new observations.
|
||
|
||
## Actual outcomes
|
||
|
||
```bash
|
||
python3 -m intelligence_radar outcome /tmp/outcome.json
|
||
```
|
||
|
||
An outcome contains recommendation_id, profile_id, unique run_id, accepted (boolean),
|
||
independent_verification (boolean), verifier_evidence (artifact reference or notes),
|
||
actual_usd, runtime_seconds, and completed_at (timezone-aware timestamp).
|
||
Outcomes are immutable and run IDs are idempotent; changed duplicate runs are rejected.
|
||
Keep the verifier artifact where its reference can be resolved. These are operator
|
||
assertions. No actual provider execution or paid inference happens in this command.
|
||
|
||
A subsequent recommendation uses matching outcomes available at its cutoff.
|
||
A historical replay uses its original frozen set. The current baseline reports
|
||
95% Wilson intervals, not a learned cross-task success predictor.
|
||
|
||
## Scheduled operation
|
||
|
||
```bash
|
||
python3 -m intelligence_radar run-due
|
||
python3 -m intelligence_radar health
|
||
systemctl --user status intelligence-radar.timer
|
||
```
|
||
|
||
`ops/schedule.json` configures daily 21:30 and Sunday 20:00 Europe/Berlin.
|
||
The shipped user timer checks every 15 minutes while this workstation's user service
|
||
manager is running. It does not promise execution while the workstation is off.
|
||
Each job is at most once per successful slot; failures remain retryable. Only the
|
||
latest missed slot is caught up. Runtime state lives in ignored `.runtime/`;
|
||
`data/scheduler-runs/` retains auditable receipts. Existing digests are preserved,
|
||
so review the receipt's digest_status before assuming a new draft was produced.
|
||
|
||
Install the reviewed units with `make install-timer`; uninstall/disable with
|
||
`systemctl --user disable --now intelligence-radar.timer`. That command leaves
|
||
source evidence and publications intact. No external app notifications are sent.
|