141 lines
15 KiB
Markdown
141 lines
15 KiB
Markdown
# Intelligence Radar Daily 2026 09 18
|
||
|
||
**Intelligence Radar — DAILY BRIEF**
|
||
**18 September 2026 (Europe/Berlin)**
|
||
|
||
Frontier weather: dense, post-burst consolidation. Early-September dual launches of Claude Fable 5.1 and GPT-6 Astra continue to define the available ceiling on agentic coding and knowledge work; open-weight efficiency pressure from DeepSeek V4.1 Flash is immediate and measurable.
|
||
|
||
### 1. Frontier & markets
|
||
Confirmed releases cluster tightly. Anthropic shipped Claude Fable 5.1 (and invitation-only Mythos 5.1) on 1 September: same $10 / $50 per MTok headline pricing as Fable 5, but cache reads cut to $0.25 / MTok (2.5 % of input). Context remains 1 M tokens, adaptive thinking always on, knowledge cutoff June 2026. System card and model card confirm stronger long-horizon agentic coding and research capabilities; Mythos variant relaxes dual-use classifiers for verified cybersecurity and life-sciences users.
|
||
|
||
OpenAI followed on 3–4 September with GPT-6 Astra (API id `gpt-6-astra`). Standard pricing matches Fable at $10 input / $50 output (cache $1 / $12.50); long-context (>272 k input) doubles most rates. Context 1.05 M tokens. Roll-out reached Plus/Pro/Business/Enterprise and Azure/Bedrock within days; Fast mode at 2× price for ~2× speed. OpenAI rates it Critical under its Preparedness Framework for cybersecurity.
|
||
|
||
DeepSeek released V4.1 Flash (MIT open weights) on 10 September: 552 B MoE backbone, Causal Encoder–Decoder architecture activating ~8 B parameters on prefill / 16 B on decode, native vision, 1 M context, KV cache compressed to ~890 bytes/token. Peak pricing ~$0.30 / $1.20 (off-peak half); API id `deepseek-flash`. DeepSeek reports it ahead of its own prior V4-Pro on multiple agentic metrics and is phasing the older Pro routing.
|
||
|
||
Meta Muse Spark 1.3 (2 September) and Google Gemini 3.8 Flash / Live variants filled the same window. Artificial Analysis Intelligence Index v4.3 (updated early September, incorporating Terminal-Bench 4.0 and AutomationBench-AA) places both Fable 5.1 (max + fallback) and GPT-6 Astra (max) at 53, with Astra delivering the same score at materially lower cost-per-task. Open-weight leaders remain GLM-5.x and Qwen3.8 variants in the low-40s. Never mix Index versions; v4.3 is the current public board.
|
||
|
||
Availability remains capacity-constrained at the absolute frontier; region blocks and export-control residual effects from earlier 2026 still shape certain Mythos-class access. Cache-price cuts (Anthropic) and extreme compression (DeepSeek) are the clearest near-term economic events.
|
||
|
||
### 2. Science of intelligence
|
||
Measurement continues to migrate away from saturated static exams toward interactive and agentic harnesses. ARC-AGI-3 (interactive reasoning) saw GPT-6 Astra report 99.9 % under the Provider Adapter at high effort — a large jump from prior GPT-5.6-class scores — while the ARC Prize 2026 Milestone #2 window remains open until 30 September. Chollet’s framing (skill-acquisition efficiency under novelty) remains the explicit reference for these boards; saturated ARC-AGI-1 numbers are no longer treated as rankers.
|
||
|
||
Primary theoretical work remains sparse this week. Tenenbaum’s Bayesian cognitive-modeling program (Daedalus 2026) continues to explore integration of LLMs with probabilistic world models. Ganguli’s call for a unified physics–neuroscience–AI science of intelligence (same volume) emphasizes emergent collective dynamics and the still-large efficiency gap between biological and artificial systems. New axiomatic proposals (SANC(E₃), quantitative intelligence-density measures) appear on arXiv but have not yet shifted laboratory evaluation practice. No major primary publications from Amodei, Hassabis, Clark, or Epoch authors landed in the last 48 h that alter measurement doctrine.
|
||
|
||
### 3. Engineering
|
||
Agent harnesses dominate. Terminal-Bench 2.1 / 4.0 and OSWorld-Verified remain the practical signal for long-horizon tool use; Cognition’s SWE-2 leads certain Terminal-Bench snapshots, while Qwen3.8 Max and Claude Fable variants trade the top of OSWorld-Verified. DeepSeek’s KV-cache compression and Anthropic’s cache-price cut are the clearest inference-cost events. Nvidia continues power-efficiency messaging around Vera Rubin / DSX MaxLPS (higher tokens per megawatt, flexible load programs). Custom silicon roadmaps (OpenAI–Broadcom Jalapeño, Meta MTIA generations) remain multi-quarter away from broad availability; current economics are still dominated by software-level cache and sparsity wins.
|
||
|
||
Energy and capacity remain first-class constraints: token throughput per megawatt is the metric hyperscalers now track publicly.
|
||
|
||
### 4. Political / legal / social
|
||
EU AI Act GPAI obligations and AI Office enforcement powers are live; transparency rules (labelling, chatbot disclosure) apply. The Digital Omnibus deferred certain high-risk system deadlines. US federal posture remains light-touch / voluntary (EO framework); California frontier-transparency rules continue in force. G20 discussions this month reflected US preference for non-technology-specific principles versus EU product-safety framing. No new binding global licensing regime emerged. Cybersecurity capability thresholds (OpenAI Critical rating for Astra; Anthropic dual-use classifiers) are now operational access controls rather than purely internal policy.
|
||
|
||
### 5. Watchlist hits (primary)
|
||
- Anthropic Claude Fable / Mythos 5.1 announcement & system card: https://www.anthropic.com/claude-fable-and-mythos-5-1
|
||
- OpenAI GPT-6 Astra: https://openai.com/index/gpt-6-astra/
|
||
- DeepSeek V4.1 Flash: https://www.deepseek.com/en/news/deepseek-v4-1-flash/ and HF model card
|
||
- Artificial Analysis Intelligence Index v4.3: https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v4-3
|
||
- ARC Prize 2026 Milestone updates: https://arcprize.org
|
||
- Epoch AI Capabilities / ECI data (updated continuously): https://epoch.ai/eci
|
||
|
||
### 6. Ledger actions
|
||
- Frontier label: retain Claude Fable 5.1 and GPT-6 Astra (AA Intelligence Index v4.3 = 53; task-dependent on Terminal-Bench / OSWorld / ARC-AGI-3). No single best model without a specified board and harness.
|
||
- Efficient label: elevate DeepSeek V4.1 Flash on cost-per-task and open-weight availability; monitor GLM / Qwen3.8 Max for agentic boards.
|
||
- Budget label: unchanged — Flash-tier and open-weight mid-range remain the practical floor.
|
||
- Five-benchmark atlas (AA Index v4.3, Terminal-Bench 4.0, OSWorld-Verified, ARC-AGI-3, SWE-bench / coding-agent suite) requires no rebase this cycle; cache economics and KV compression are the variables that would move Efficient/Budget thresholds first.
|
||
|
||
Ask Intelligence Radar a task for a Task Router ticket (5 benchmarks → Frontier / Efficient / Budget + optional decomposition).
|
||
|
||
## Intelligence Radar — Daily Delta, 18 Sep 2026
|
||
|
||
**Net assessment:** there is **no verified new general-purpose flagship model or broad API-price change** since the previous scan. The material change is more structural: Anthropic has published the clearest quantitative picture yet of **AI helping to build its own successors**, OpenAI has shown how much a **domain-configured system** can outperform its underlying frontier model, and Anthropic has produced a concrete example of AI materially lowering the cost of scientific-model engineering.
|
||
|
||
### 1. Anthropic publishes a metrology for “AI building AI”
|
||
|
||
On **September 17**, Anthropic published three internal measures intended to make the pace of frontier development externally legible: the share of AI R&D performed by AI, the quality of oversight over internal agents, and the allocation of compute between capability and safety work. ([Anthropic][1])
|
||
|
||
Its most important result is the new **Anthropic R&D Automation Index**. As of August 2026, Claude did not operate fully autonomously on any measured category, but it **“led” 26% of Anthropic’s AI R&D work**—meaning it could perform most of a task end-to-end from a high-level prompt with human supervision—and more than **90%** of R&D work was at least at the “AI collaborates” level. The underlying chart says the “leads” share had risen from under 1% in February. ([Anthropic][1])
|
||
|
||
Anthropic also reports roughly **30,000 concurrent research/engineering agents** on its most-used internal platform. All their actions pass through online monitoring; in August, the company analyzed more than a billion agent decisions and says **0.002%—about 1 in 47,000—were blocked**. Offline monitoring escalates roughly 50 high-priority cases per week to humans. ([Anthropic][1])
|
||
|
||
Its compute snapshot is also novel: during one measured week, Anthropic classified about **6% of AI-R&D compute as safety work**, rising to about **12% within AI-driven AI R&D**. Anthropic explicitly warns that these shares are imperfect and methodology-dependent. ([Anthropic][1])
|
||
|
||
**Radar impact:** yesterday’s `recursive_self_improvement` watch item can now be sharpened considerably. The evidence supports:
|
||
|
||
> **AI-assisted recursive development is already substantial and rapidly increasing, but fully autonomous successor development has not been observed in Anthropic’s own measurement.**
|
||
|
||
So I would classify:
|
||
|
||
* `AI_accelerates_AI_R&D` → **observed / strong**
|
||
* `AI_leads_substantial_fraction_of_frontier_R&D` → **observed / provider-measured**
|
||
* `fully_autonomous_recursive_self_improvement` → **not observed**
|
||
* `recursive_self_improvement_is_imminent` → **contested/watch**
|
||
|
||
More importantly, Intelligence Radar should add a new **Development Process Metrics** family alongside capability benchmarks:
|
||
|
||
`R&D automation level · agent count · oversight coverage · review latency · escalation/block rate · safety-compute share`
|
||
|
||
These are not conventional model benchmarks; they measure the **production system that creates the next generation of models**.
|
||
|
||
[Anthropic — Measurements for understanding the pace of AI development inside frontier labs](https://www.anthropic.com/institute/measuring-pace-of-ai-development?utm_source=chatgpt.com)
|
||
|
||
### 2. Astra for Law demonstrates that the execution system can matter almost as much as the base model
|
||
|
||
OpenAI introduced **Astra for Law on September 17**. Crucially, this is **not a new foundation model**. It combines GPT‑6 Astra with legal-specific instructions and a dedicated search index covering U.S. case law, statutes, regulations, court rules and administrative decisions across more than 230 million URLs. ([OpenAI][2])
|
||
|
||
On 200 private validation questions from Vals AI’s Legal Research Bench, OpenAI reports that the complete Astra-for-Law configuration passed the benchmark’s overall correctness check on **54.0%** of questions, versus **38.7%** for ordinary GPT‑6 Astra with web search at the same highest reasoning setting—a **40% relative improvement**. It also found 24% more reference cases on case-law questions. ([OpenAI][2])
|
||
|
||
Availability is initially restricted to selected U.S. law firms through Trusted Access; API access is planned but not yet generally available. ([OpenAI][2])
|
||
|
||
**Radar impact:** this strongly validates our decision not to rank only “models.”
|
||
|
||
For legal research, the measured entity is:
|
||
|
||
`GPT‑6 Astra + legal instructions + specialized corpus/index + retrieval + access policy`
|
||
|
||
not simply `GPT‑6 Astra`.
|
||
|
||
I would make one taxonomy refinement:
|
||
|
||
> **Execution Profile → Domain Profile**
|
||
|
||
A Domain Profile is an execution profile whose capability comes materially from specialist retrieval, tools, instructions, or workflow constraints.
|
||
|
||
The result also creates a benchmark warning: **generic frontier-model scores can substantially underpredict task performance once a high-quality domain environment is added.** The five-benchmark selector should therefore prefer evaluations whose environment resembles the proposed deployment.
|
||
|
||
Astra for Law does **not** change the global Frontier ranking, but it likely changes the **Available Frontier for U.S. legal research** for users with access.
|
||
|
||
[OpenAI — Introducing Astra for Law](https://openai.com/index/astra-for-law/?utm_source=chatgpt.com)
|
||
|
||
### 3. Claude materially reduces the computational cost of biomolecular AI
|
||
|
||
Anthropic also published a substantial science/engineering result on **September 17**. Working with its Claude Science setup, Claude optimized more than **30 open-source biomolecular deep-learning models in under four weeks**, achieving roughly **4× average speed-up with minimal precision loss** and nearly **2× with identical outputs**. ([Anthropic][3])
|
||
|
||
For protein-structure systems, Anthropic reports custom kernels outperforming the existing field standard by roughly **2.7–2.9× for triangle attention** and **1.7–3.2× for triangle multiplication**. Claude also created a low-memory mode enabling accurate prediction of biomolecular systems above **10,000 tokens on a single GPU node**. ([Anthropic][3])
|
||
|
||
The economics are particularly relevant. Anthropic’s earlier agentic protein-design setup allowed up to roughly 2,500 H100 GPU-hours per target. With the new optimized tooling, a single Claude model using one H200 for 24 hours reportedly achieved comparable **in-silico** design scores using roughly **100× fewer GPU-hours**, with about **$150 total GPU-plus-model spend**. This is computational evidence only; wet-lab validation is still the important downstream test. ([Anthropic][3])
|
||
|
||
**Radar impact:** this exposes another benchmark gap. Scientific capability should not be represented only by answering scientific questions or writing research code. Add constructs such as:
|
||
|
||
`scientific_toolchain_engineering`
|
||
`domain_model_optimization`
|
||
`compute_efficiency_improvement`
|
||
`AI_improving_other_AI_systems`
|
||
|
||
This also reinforces the previous finding about AI-assisted AI R&D: a general-purpose frontier model can improve **specialized learned systems**, not merely write glue code around them.
|
||
|
||
[Anthropic — How Claude is uplifting biomolecular modeling](https://www.anthropic.com/research/claude-uplifts-biomolecular-modeling?utm_source=chatgpt.com)
|
||
|
||
### Recommended Intelligence Radar state changes
|
||
|
||
I would make three concrete changes today: add **Development Process Metrics** as a new measurement family; introduce **Domain Profile** as a named subtype of Execution Profile; and expand scientific capability measurement from reasoning/programming into **optimization of scientific AI systems and their resource economics**.
|
||
|
||
There is **no reason from today’s evidence to alter the general Frontier / Topnotch / Efficient / Budget baseline**. The more consequential delta is that the unit of analysis keeps widening:
|
||
|
||
**model → execution profile → domain system → development organization.**
|
||
|
||
That increasingly suggests Intelligence Radar should measure not only *what an AI can do*, but also **how much of the machinery that produces the next AI is itself being operated by AI**.
|
||
|
||
[1]: https://www.anthropic.com/institute/measuring-pace-of-ai-development "Measurements for understanding the pace of AI development inside frontier labs \ Anthropic"
|
||
[2]: https://openai.com/index/astra-for-law/ "Introducing Astra for Law | OpenAI"
|
||
[3]: https://www.anthropic.com/research/claude-uplifts-biomolecular-modeling "How Claude is uplifting biomolecular modeling \ Anthropic"
|
||
|