Integrate founding explorations and bootstrap operational radar

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97

task-router/BENCHMARK-ATLAS.md

@@ -0,0 +1,58 @@
+> Imported discovery draft; recipes are non-normative. Verify versions and use METHOD.md for selection.
+
+# Benchmark atlas — Task Router
+
+Version 0.1 · 13 September 2026  
+Pick five. Recipes at the bottom.
+
+## Living instruments (preferred)
+
+| ID | Benchmark | Best-used-for layers | Still discriminating? |
+| --- | --- | --- | --- |
+| HLE | Humanity’s Last Exam | know + reason (expert) | yes |
+| FMath | FrontierMath / Erdős | reason (research math) | yes, tiny N |
+| GPQA | GPQA Diamond | know + reason (grad science) | fading at the very top |
+| SciCode | SciCode | reason + tool (scientific code) | yes |
+| SWE-V | SWE-bench Verified | tool + plan (repo repair) | yes, some saturation at top |
+| SWE-P | SWE-bench Pro | tool + plan (harder engineering) | yes |
+| DeepSWE | DeepSWE | tool + plan (agent coding) | yes |
+| FCode | Frontier Code | tool + plan (front of coding agents) | yes |
+| Term4 | Terminal-Bench 4.0 | tool + plan (CLI / research ops) | yes (v4 reset) |
+| OSW2 | OSWorld 2.0 | perceive + tool + plan (desktop) | yes, lots of headroom |
+| ARC3 | ARC-AGI-3 | reason (interactive abstraction) | yes; **harness-sensitive** |
+| GDPv | GDPval-AA v2 | know + reason (occupation work) | yes |
+| GDPpdf | GDP.pdf | know + reason (long professional docs) | yes |
+| Brief | AA-Briefcase | know + reason (knowledge work pack) | yes |
+| AutoB | AutomationBench-AA | tool + plan (office agents) | yes |
+| Omni | AA-Omniscience / SimpleQA | know + reliable (facts) | yes |
+| Crit | CritPt | reliable (critique / catch errors) | yes |
+| LCR | AA-LCR | reason (long-context) | yes |
+| AAIX | AA Intelligence Index (current ver.) | composite | yes, version-locked |
+| ECI | Epoch Capabilities Index | long-run general capability | yes, for history |
+| LiveC | LiveCodeBench | tool (contest coding, contamination-aware) | yes |
+| Browse | BrowseComp / web-research evals | tool + know (search) | use when task is open-web |
+
+## Saturated / secondary (at most one of five)
+
+MMLU-Pro, MATH-500, AIME (year-locked), HumanEval, IFEval, Arena Elo.
+
+## Default five-benchmark recipes
+
+| Task family | Five |
+| --- | --- |
+| Hard general question / research memo | HLE, GDPpdf, Brief, Omni, Crit |
+| Graduate science explanation | GPQA, HLE, SciCode, Crit, Omni |
+| Research mathematics | FMath, HLE, SciCode, LCR, Crit |
+| Production software change | SWE-P, DeepSWE, Term4, LiveC, Crit |
+| Greenfield coding / scripts | LiveC, SWE-V, Term4, Crit, Omni |
+| Computer-use / “do this in the GUI” | OSW2, Term4, AutoB, ARC3, Crit |
+| Office automation / agent workflow | AutoB, GDPv, Term4, OSW2, Brief |
+| Long-document diligence | GDPpdf, LCR, Brief, Crit, Omni |
+| Open-web investigation | Browse, Omni, Crit, Brief, HLE |
+| Multimodal perception + act | OSW2, ARC3, AutoB, GDPv, Crit |
+| Cheap classification / extraction | Omni, Crit, Brief, GDPv, AAIX |
+| “What model should we buy?” meta | AAIX, GDPv, SWE-P, Term4, OSW2 |
+
+## Changelog
+
+- 2026-09-13: v0.1 atlas. Rebuild whenever a major eval saturates or an index rebases.

task-router/METHOD.md

@@ -0,0 +1,24 @@
+# Capability Advisor — operational method
+
+The normative method is [TaskModelEvaluationMethod](../TaskModelEvaluationMethod.md).
+Use policy IDs `benchmark-selection-v0.1` and `model-ranking-v0.1`.
+
+1. Record task, deliverable, acceptance criteria, risk class, constraints, tools,
+   budget and requester-accessible deployments. Score the six clarity components 0–1.
+2. Map required capabilities and hard gates. Select five distinct benchmark versions
+   with task-specific weights summing to one; document relevance, redundancy,
+   weaknesses, provenance and dates. The atlas is only a discovery aid.
+3. Preserve model, effort, harness, provider and deployment identity for results.
+   Treat missing results as missing. Label normalized benchmark scores as proxies.
+4. Rank eligible profiles by comparative task fitness. Frontier maximizes fitness;
+   Topnotch includes fitness >= .90 of Frontier. Efficient minimizes expected total
+   task cost within Topnotch, using dated prices and explicit workload assumptions.
+5. Evaluate Sufficient against the risk threshold using calibrated success evidence.
+   If missing, report `no_evidenced_sufficient_model`; Budget is then undetermined.
+   Do not derive retries by dividing by an uncalibrated benchmark score.
+6. Explain confidence dimensions and evidence gaps. Estimate cost/runtime p50/p90
+   only where evidence supports quantiles; otherwise label a scenario range.
+7. Compare optional decomposition with a monolithic baseline including coordination,
+   verification and failure costs. Record actual outcomes for later calibration.
+
+Use [TEMPLATE](TEMPLATE.md). This is a manual workflow in v0.1, not an API promise.

task-router/TEMPLATE.md

@@ -0,0 +1,40 @@
+# Capability Advisor ticket
+
+Status: draft / reviewed
+Date and evidence cutoff:
+Policy IDs: benchmark-selection-v0.1; model-ranking-v0.1
+Task / deliverable / acceptance criteria:
+Risk class and explicit sufficiency threshold:
+Hard gates / tools / privacy / region / budget:
+Clarity components (0–1): objective, context, constraints, acceptance, output, environment
+Capability profile:
+
+## Five measurements
+
+| Benchmark + version | Observation ID / source / date | Weight | Relevance and validity | Gaps / redundancy |
+| --- | --- | --- | --- | --- |
+| 1 | | | | |
+| 2 | | | | |
+| 3 | | | | |
+| 4 | | | | |
+| 5 | | | | |
+
+Weights sum to one. No composite index substituted for a measurement.
+
+## Candidates and decision
+
+| Profile + deployment + harness + effort | Eligibility | Fitness proxy | Calibrated success / evidence | Expected cost |
+| --- | --- | --- | --- | --- |
+
+Frontier:
+Topnotch:
+Efficient:
+Sufficient: no_evidenced_sufficient_model until supported
+Budget: undetermined until supported
+
+Price observation IDs and dates; input/cache/output/reasoning/tool/retry/verifier assumptions:
+Cost and runtime p50/p90 (or explicitly labeled scenario ranges):
+Confidence by component and what would change the decision:
+Evidence gaps:
+Optional decomposition vs monolithic baseline:
+Outcome: actual configuration, acceptance result, cost, tokens, runtime, retries, failure mode:

task-router/examples/2026-09-13-worked-example.md

@@ -0,0 +1,24 @@
+# Worked example — evidence-limited routing
+
+Task: produce a sourced technical diligence memo and migration plan for a coding-agent stack.
+This is a manual workflow illustration, not a live model recommendation.
+
+Missing context: current stack, task distribution, acceptable failure rate, region,
+and access to candidate deployments. Acceptance: reviewed memo plus reproducible cost model.
+Risk policy: important, provisional threshold .90.
+
+Candidate instruments: SWE-bench Multilingual (repository work), Terminal-Bench
+(shell work), BrowseComp (source discovery), GDPval (deliverables), BFCL (tool use).
+Their exact versions, result dates, independence and requester harness match need
+verification. No weights or model results are invented to fill the ticket.
+
+Frontier / Topnotch / Efficient: undetermined pending eligible profile evidence.
+Sufficient: `no_evidenced_sufficient_model`. Budget: undetermined.
+Cost/runtime: unknown until prices and representative traces are observed.
+
+Next: collect versioned results and deployment prices, run representative work samples,
+then compare retrieval → analysis → independent verification against monolithic execution.
+Record measured outcomes; do not assert an unsupported percentage saving.
+
+The original Grok example, with named winners and assumed savings, is preserved in
+the source ZIP and superseded by this evidence-limited example.