2026-09-13 22:42:09 +02:00
# Capability Advisor — operational method
The normative method is [TaskModelEvaluationMethod ](../TaskModelEvaluationMethod.md ).
Use policy IDs `benchmark-selection-v0.1` and `model-ranking-v0.1` .
1. Record task, deliverable, acceptance criteria, risk class, constraints, tools,
budget and requester-accessible deployments. Score the six clarity components 0–
2. Map required capabilities and hard gates. Select five distinct benchmark versions
with task-specific weights summing to one; document relevance, redundancy,
weaknesses, provenance and dates. The atlas is only a discovery aid.
3. Preserve model, effort, harness, provider and deployment identity for results.
Treat missing results as missing. Label normalized benchmark scores as proxies.
4. Rank eligible profiles by comparative task fitness. Frontier maximizes fitness;
Topnotch includes fitness >= .90 of Frontier. Efficient minimizes expected total
task cost within Topnotch, using dated prices and explicit workload assumptions.
5. Evaluate Sufficient against the risk threshold using calibrated success evidence.
If missing, report `no_evidenced_sufficient_model` ; Budget is then undetermined.
Do not derive retries by dividing by an uncalibrated benchmark score.
6. Explain confidence dimensions and evidence gaps. Estimate cost/runtime p50/p90
only where evidence supports quantiles; otherwise label a scenario range.
7. Compare optional decomposition with a monolithic baseline including coordination,
verification and failure costs. Record actual outcomes for later calibration.
2026-09-13 23:07:41 +02:00
Use [TEMPLATE ](TEMPLATE.md ). The v0.2 CLI implements selection, eligibility, ranking, scenario costs, outcomes and replay under advisor-v0.2.json. See ../docs/data-contracts.md; decomposition remains manual.
