41 lines
1.4 KiB
Markdown
41 lines
1.4 KiB
Markdown
|
# Capability Advisor ticket
|
|||
|
|
|
|||
|
|
Status: draft / reviewed
|
|||
|
|
Date and evidence cutoff:
|
|||
|
|
Policy IDs: benchmark-selection-v0.1; model-ranking-v0.1
|
|||
|
|
Task / deliverable / acceptance criteria:
|
|||
|
|
Risk class and explicit sufficiency threshold:
|
|||
|
|
Hard gates / tools / privacy / region / budget:
|
|||
|
|
Clarity components (0–1): objective, context, constraints, acceptance, output, environment
|
|||
|
|
Capability profile:
|
|||
|
|
|
|||
|
|
## Five measurements
|
|||
|
|
|
|||
|
|
| Benchmark + version | Observation ID / source / date | Weight | Relevance and validity | Gaps / redundancy |
|
|||
|
|
| --- | --- | --- | --- | --- |
|
|||
|
|
| 1 | | | | |
|
|||
|
|
| 2 | | | | |
|
|||
|
|
| 3 | | | | |
|
|||
|
|
| 4 | | | | |
|
|||
|
|
| 5 | | | | |
|
|||
|
|
|
|||
|
|
Weights sum to one. No composite index substituted for a measurement.
|
|||
|
|
|
|||
|
|
## Candidates and decision
|
|||
|
|
|
|||
|
|
| Profile + deployment + harness + effort | Eligibility | Fitness proxy | Calibrated success / evidence | Expected cost |
|
|||
|
|
| --- | --- | --- | --- | --- |
|
|||
|
|
|
|||
|
|
Frontier:
|
|||
|
|
Topnotch:
|
|||
|
|
Efficient:
|
|||
|
|
Sufficient: no_evidenced_sufficient_model until supported
|
|||
|
|
Budget: undetermined until supported
|
|||
|
|
|
|||
|
|
Price observation IDs and dates; input/cache/output/reasoning/tool/retry/verifier assumptions:
|
|||
|
|
Cost and runtime p50/p90 (or explicitly labeled scenario ranges):
|
|||
|
|
Confidence by component and what would change the decision:
|
|||
|
|
Evidence gaps:
|
|||
|
|
Optional decomposition vs monolithic baseline:
|
|||
|
|
Outcome: actual configuration, acceptance result, cost, tokens, runtime, retries, failure mode:
|