tegwick
29ceb77938
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97
47 lines
2.9 KiB
Markdown
47 lines
2.9 KiB
Markdown
# ADR 0002 — Reproducible advisor and source capture
|
|
|
|
Accepted: 2026-09-13.
|
|
|
|
The v0.2 CLI uses standard-library Python and content-addressed JSON. Source bytes
|
|
are retained by hash. Capture receipts record requested/final URLs, publisher,
|
|
retrieval time, media type and change detection; capture does not certify a claim.
|
|
Changed sources create review-needed observations rather than invented extracted facts.
|
|
|
|
`advisor-v0.2.json` is the executable policy. It carries forward the baseline
|
|
selection weights, inclusive .90 Topnotch ratio and risk thresholds. It adds a
|
|
90-day evidence freshness limit, a .15 construct-overlap redundancy penalty,
|
|
and a minimum of 20 exact-task/profile outcomes. Earlier YAML remains an immutable
|
|
design baseline. v0.2 is explicitly provisional and curator-driven.
|
|
|
|
Five distinct nonaggregate instruments are selected greedily with construct-based
|
|
task relevance, curator-assigned health and redundancy penalties. Raw scores use
|
|
an explicit increasing linear scale per benchmark version. Missing scores are not
|
|
imputed. Nonlinear metrics, Elo, lower-is-better metrics and composite indices need
|
|
separate adapters before use. Pending benchmark versions cannot support ranking.
|
|
|
|
Eligible means all declared task constraints match evidenced deployment attributes.
|
|
Minima use `min_` keys; other constraints require exact equality. Production catalog
|
|
attributes require curator review; available does not imply requester access.
|
|
Fitness is comparative, not calibrated task success. Rankings cover only evidenced,
|
|
eligible candidates, not the entire model market.
|
|
|
|
Costs are explicit workload scenarios: disjoint input/cache/output buckets, tools,
|
|
declared attempts and verification. Output includes billable reasoning. Unknown or
|
|
conflicting prices prevent claims of cheapest. These are not estimated p50/p90s.
|
|
|
|
Outcomes bind a recommendation, exact task and exact profile snapshot. Unique run
|
|
IDs prevent duplicate evidence. Wilson bounds are reported over matching observed
|
|
trials; Sufficient requires the lower bound above the policy threshold, with at
|
|
least 20 trials. High-assurance also requires independent-verification assertions.
|
|
This assumes representative independent trials; it is not a guarantee or a
|
|
learned cross-task calibration model. Real operational outcome data is still absent.
|
|
|
|
Recommendations freeze task, catalog, policy, outcomes and report. Canonical key
|
|
ordering makes replay deterministic. Source captures and bytes remain referenced
|
|
by immutable IDs. Replay checks algorithm version and rejects divergence.
|
|
|
|
The scheduler checks the latest due daily/weekly slot in Europe/Berlin, catches up
|
|
one slot after downtime, serializes runs, persists successes, and retries failures
|
|
on later invocations. Each source fetch allows three bounded attempts and 5 MiB.
|
|
Existing digest files are preserved. Receipt history distinguishes a new draft
|
|
from an existing preserved artifact. Editorial synthesis remains a human/agent step.
|