Research deliverbles standard, concept on earned equity, data collection and digests added

This commit is contained in:
tegwick 2026-09-14 22:04:44 +02:00
parent c70b671a38
commit 83c8a763a1
16 changed files with 4465 additions and 0 deletions

120
digests/daily/2026-09-14.md Executable file
View file

@ -0,0 +1,120 @@
# Intelligence Radar — Daily Digest
**Date:** 2026-09-14
**Scope:** Daily delta since the previous Intelligence Radar scan
## Net assessment
The capability/economics frontier itself appears **stable since yesterday’s baseline**. No verified new flagship-model launch, major API-price change, or benchmark result was identified that would justify changing the current Frontier / Topnotch / Efficient picture.
Two developments are worth recording: one as a **new policy/safety signal** and one as a **baseline correction to the benchmark catalog**.
## 1. Frontier-lab safety rhetoric shifts toward capability pacing
Dario Amodei is explicitly arguing for pacing capability progress. In a new essay, Anthropic’s CEO says he has become convinced that risk prevention alone is insufficient and that frontier labs should pace capability advancement so safety work has time to catch up.
He gives two principal reasons:
- accelerating AI-assisted AI development, which he characterizes as the beginning of recursive self-improvement;
- recent agentic cybersecurity incidents.
### Why this matters
This is important less as empirical evidence than as a change in the governance position of a major frontier-lab leader. The argument moves from:
> advance quickly while investing heavily in safety
toward:
> the rate of capability improvement itself may need active control.
### Intelligence Radar impact
Add a monitored claim along the lines of:
> **Frontier capability pacing is becoming an explicit policy proposal from frontier-lab leadership.**
Keep the stronger claim that recursive self-improvement is already beginning as `contested/watch`, not `observed`.
Amodei’s essay is evidence that the claim is being made by a highly informed actor; it is not independent proof that an RSI regime has begun.
**Primary source:**
https://darioamodei.com/post/we-must-pace-the-frontier
## 2. Benchmark-catalog correction: AWS Deception Benchmark exposes a security-evaluation gap
AWS published its Deception Benchmark on 2026-09-09. It is therefore not a new release since yesterday, but it was absent from the initial Intelligence Radar benchmark seed and is important enough to correct immediately.
AWS created **14,822 code-security examples across 16 languages and more than 70 CWE categories** to test whether models can distinguish genuine vulnerabilities from code that merely looks dangerous.
According to AWS, none of the 12 tested general-purpose model/configuration combinations achieved its proposed production bar of both:
- false-positive rate below 10%; and
- false-negative rate below 10%.
Direct prompting often caught vulnerabilities but produced very high false-positive rates. Proof-of-exploit prompting reduced false alarms while missing more genuine vulnerabilities.
### Why this matters
This is a useful counterweight to offensive-cyber benchmarks.
Intelligence Radar currently risks treating:
> good at finding or exploiting vulnerabilities
as evidence for:
> good defensive security reviewer.
The AWS result suggests these are materially different constructs.
### Intelligence Radar impact
Add a benchmark/construct class such as:
`security_vulnerability_discrimination`
and retain false-positive and false-negative rates independently rather than collapsing them into a single accuracy score.
The result also strengthens an existing Intelligence Radar principle:
> **Scaffolding and verification can change the operating point without changing the underlying measured competence.**
AWS deliberately uses single-pass classification because it wants to expose whether the model itself understands why a mitigation closes an exploit path. Multi-step agent systems should therefore be evaluated separately instead of attributing all resulting performance to the base model.
**Primary source:**
https://aws.amazon.com/blogs/security/the-state-of-ai-for-security-measuring-what-matters-most-for-building-trust/
## Recommended state changes
### Benchmark policy
Split cybersecurity capability into at least:
- offensive exploitation;
- vulnerability discovery;
- vulnerability discrimination;
- defensive remediation.
Do not infer one capability directly from another.
### Governance watch
Add:
`capability_pacing / recursive_self_improvement_governance`
as a dedicated Radar topic.
Explicitly distinguish:
- evidence of AI-assisted AI R&D;
- acceleration of AI research workflows;
- claims of recursive self-improvement;
- evidence of an actual self-accelerating capability regime.
## Frontier assessment
For 2026-09-14, **no change is recommended to the current model Frontier / Efficient / Budget baseline** based on the material identified in this scan.
The more meaningful movement today is in how frontier AI is being **measured and governed**, rather than where the capability frontier itself sits.