Integrate founding explorations and bootstrap operational radar
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97
This commit is contained in:
parent
775f93b920
commit
d0e6438609
57 changed files with 4723 additions and 2 deletions
24
task-router/METHOD.md
Normal file
24
task-router/METHOD.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# Capability Advisor — operational method
|
||||
|
||||
The normative method is [TaskModelEvaluationMethod](../TaskModelEvaluationMethod.md).
|
||||
Use policy IDs `benchmark-selection-v0.1` and `model-ranking-v0.1`.
|
||||
|
||||
1. Record task, deliverable, acceptance criteria, risk class, constraints, tools,
|
||||
budget and requester-accessible deployments. Score the six clarity components 0–1.
|
||||
2. Map required capabilities and hard gates. Select five distinct benchmark versions
|
||||
with task-specific weights summing to one; document relevance, redundancy,
|
||||
weaknesses, provenance and dates. The atlas is only a discovery aid.
|
||||
3. Preserve model, effort, harness, provider and deployment identity for results.
|
||||
Treat missing results as missing. Label normalized benchmark scores as proxies.
|
||||
4. Rank eligible profiles by comparative task fitness. Frontier maximizes fitness;
|
||||
Topnotch includes fitness >= .90 of Frontier. Efficient minimizes expected total
|
||||
task cost within Topnotch, using dated prices and explicit workload assumptions.
|
||||
5. Evaluate Sufficient against the risk threshold using calibrated success evidence.
|
||||
If missing, report `no_evidenced_sufficient_model`; Budget is then undetermined.
|
||||
Do not derive retries by dividing by an uncalibrated benchmark score.
|
||||
6. Explain confidence dimensions and evidence gaps. Estimate cost/runtime p50/p90
|
||||
only where evidence supports quantiles; otherwise label a scenario range.
|
||||
7. Compare optional decomposition with a monolithic baseline including coordination,
|
||||
verification and failure costs. Record actual outcomes for later calibration.
|
||||
|
||||
Use [TEMPLATE](TEMPLATE.md). This is a manual workflow in v0.1, not an API promise.
|
||||
Loading…
Add table
Add a link
Reference in a new issue