Integrate founding explorations and bootstrap operational radar
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97
This commit is contained in:
parent
775f93b920
commit
d0e6438609
57 changed files with 4723 additions and 2 deletions
38
compendium/00-what-this-is.md
Normal file
38
compendium/00-what-this-is.md
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
> Imported exploration, 2026-09-13. Source claims have not been independently verified by this repository. See [integration review](../docs/integration-review.md).
|
||||
|
||||
# Compendium — what this is
|
||||
|
||||
Version 0.1 · 13 September 2026
|
||||
|
||||
This folder is the **introductory map** Intelligence Radar keeps of how humans currently *model* intelligence. It is meant to improve over time: weekly digests propose edits; accepted edits bump the version and leave a short changelog.
|
||||
|
||||
It is not a complete literature review. It is a working ontology plus the disputes that matter for monitoring the frontier.
|
||||
|
||||
## Scope of “intelligence” here
|
||||
|
||||
Intelligence Radar tracks four overlapping objects:
|
||||
|
||||
1. **Biological intelligence** — nervous systems and the cognitive capacities they support in humans and other animals.
|
||||
2. **Artificial intelligence** — engineered systems that perform cognitive work, currently dominated by large trained models and agent scaffolds.
|
||||
3. **Collective / institutional intelligence** — markets, science, firms, states, open-source communities: systems that know and decide without a single mind.
|
||||
4. **Otherwise generated** — hybrid human–machine systems, in-silico cognitive science, digital twins of brains, proposed non-neural substrates (quantum, analog, neuromorphic, latent-space recurrent architectures).
|
||||
|
||||
The facility’s special focus is the **capability frontier of artificial systems** and the **price and availability** of those capabilities. The other three objects exist so the frontier is not mistaken for the whole phenomenon.
|
||||
|
||||
## What “humanity knows”
|
||||
|
||||
A useful humility: humanity does not have one settled theory of intelligence. It has:
|
||||
|
||||
- **Measurement traditions** (psychometrics, animal cognition batteries, AI benchmarks, economic productivity studies).
|
||||
- **Mechanistic sketches** (predictive processing, global workspace, network neuroscience, scaling laws, Bayesian program learning).
|
||||
- **Engineering recipes** that work without a complete theory (pretrain–reason–act loops, RL from verifiable rewards, tool use, long-horizon agents).
|
||||
- **Open disputes** that are load-bearing for policy and investment (is general intelligence a real factor? is consciousness computational? do current agents generalize or memorize? is the bottleneck data, compute, architecture, or evaluation?).
|
||||
|
||||
The compendium tracks those four layers separately so a new model release does not get written up as a new theory of mind.
|
||||
|
||||
## Reading order
|
||||
|
||||
1. `01-maps-of-intelligence.md` — the main human models
|
||||
2. `02-measurement.md` — how intelligence is scored
|
||||
3. `03-frontier-as-of-2026-09.md` — snapshot of the artificial frontier
|
||||
4. Changelog at the bottom of each file
|
||||
85
compendium/01-maps-of-intelligence.md
Normal file
85
compendium/01-maps-of-intelligence.md
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
> Imported exploration, 2026-09-13. Source claims have not been independently verified by this repository. See [integration review](../docs/integration-review.md).
|
||||
|
||||
# Maps of intelligence
|
||||
|
||||
Version 0.1 · 13 September 2026
|
||||
|
||||
Humans have invented several mutually incompatible but productive ways to talk about intelligence. Intelligence Radar treats them as **maps**, not as a single territory.
|
||||
|
||||
## 1. Psychometric / difference map
|
||||
|
||||
**Claim:** Cognitive performances covary. A general factor *g* plus group factors (verbal, spatial, processing speed) describe individual differences.
|
||||
|
||||
**What it is good for:** Predicting educational and occupational outcomes; grounding “general” as a statistical regularity rather than a slogan.
|
||||
|
||||
**Limits:** *g* is a covariance structure, not a mechanism. It does not say how a brain or a model produces competence. Transfer to machines is metaphorical unless the same battery is used.
|
||||
|
||||
**2026 note:** Network Neuroscience Theory treats *g* as a property of global connectome topology (distributed processing, weak long-range links, modal control hubs, small-world communication) rather than a single region. That is a live empirical program, not settled doctrine.
|
||||
|
||||
## 2. Cognitive-architecture map
|
||||
|
||||
**Claim:** Intelligence is a set of processes — perception, working memory, attention, planning, learning, language, motor control — arranged in an architecture (ACT-R, SOAR, dual-process, global workspace, predictive processing).
|
||||
|
||||
**What it is good for:** Task analysis and decomposition. This is the map the Task Router inherits when it splits a job into subtasks.
|
||||
|
||||
**Limits:** Architectures multiply. “System 1 / System 2” is a useful cartoon that current reasoning models have made commercially real without proving the cartoon.
|
||||
|
||||
**2026 note:** Complementary-intelligence programs distinguish *cognitive AI* (built to model human processes) from *machine AI* (optimized for scale). Integration routes being discussed: shared embeddings, instruction encoding, training agents on cognitive-AI data, and coevolving agents.
|
||||
|
||||
## 3. Computational / Bayesian map
|
||||
|
||||
**Claim:** Intelligence is approximate rational inference under resource limits: priors, likelihoods, programs, search.
|
||||
|
||||
**Champions:** Tenenbaum and the probabilistic-programs tradition; the “language is not all you need” line that wants LLMs as *language modules* attached to a world-model / program-induction core.
|
||||
|
||||
**What it is good for:** Explaining few-shot concept learning, causal reasoning, and why pure next-token prediction still fails some structured tasks.
|
||||
|
||||
**Limits:** Full Bayesian program learning does not yet scale to frontier product systems. The engineering winning streak is still large transformers plus RL and tools.
|
||||
|
||||
## 4. Connectionist / scaling map
|
||||
|
||||
**Claim:** Competence emerges from gradient-trained networks at sufficient scale of data, parameters, and compute, plus the right post-training (instruction, preference, RL, tools).
|
||||
|
||||
**What it is good for:** Predicting that many new skills appear as a function of scale and that evaluation sets saturate. This is the map the AI industry actually ships against.
|
||||
|
||||
**Limits:** Poor sample and energy efficiency versus brains; weak accounts of agency, grounded world models, and long-horizon reliability; benchmarks leak.
|
||||
|
||||
**2026 note:** Alternative architectures (latent-space recurrent “brain-inspired” models such as Pathway BDH) are trying to move reasoning *off* the token tape. Watch these as potential cost-curve breakers, not as proven replacements.
|
||||
|
||||
## 5. Neuroscientific / mechanistic map
|
||||
|
||||
**Claim:** Intelligence is what certain recurrent, sparse, multi-scale biological circuits do — especially prefrontal control interacting with specialized systems — under severe energy and developmental constraints.
|
||||
|
||||
**2026 note:** Proposals for a “foundational brain model of intelligence” want connectome + cell types + recurrent dynamics + multi-task training, with PFC as the control core. Separate work (Ganguli and others) calls for a science of intelligence that unifies physics of complex systems, neuroscience, and AI.
|
||||
|
||||
**Limits:** We still lack a causal, multi-scale model that predicts both a mouse navigating and GPT-class language.
|
||||
|
||||
## 6. Consciousness / sentience map (disputed)
|
||||
|
||||
**Claim variants:** Integrated information; global workspace broadcast; higher-order thought; life-first / enactive views that deny substrate-neutral computation.
|
||||
|
||||
**Radar rule:** Track the *dispute* and any policy consequence (welfare of models, “critical” capability designations). Do not treat a lab’s anthropomorphic demo as evidence of sentience.
|
||||
|
||||
## 7. Collective / economic map
|
||||
|
||||
**Claim:** A lot of “what humanity knows” lives in institutions, prices, papers, and code, not in individual skulls. AI changes the cost of contributing to that collective.
|
||||
|
||||
**What it is good for:** The weekly digest’s economic, legal, and political sections. Capability without deployment, capital, chips, or permission is not a social fact.
|
||||
|
||||
## Working synthesis used by this facility
|
||||
|
||||
For routing work to models, Intelligence Radar uses a **capability stack**, not a theory of mind:
|
||||
|
||||
1. Perception and representation
|
||||
2. Knowledge and retrieval
|
||||
3. Reasoning and search
|
||||
4. Tool and environment action
|
||||
5. Planning over long horizons
|
||||
6. Coordination (multi-agent / human-in-the-loop)
|
||||
7. Reliability under distribution shift and adversarial pressure
|
||||
|
||||
A task is scored by which layers it actually needs. Benchmarks are chosen to hit those layers. Theories from sections 1–7 explain *why* a layer is hard; they do not replace the score.
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2026-09-13: v0.1 initial map.
|
||||
58
compendium/02-measurement.md
Normal file
58
compendium/02-measurement.md
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
> Imported exploration, 2026-09-13. Source claims have not been independently verified by this repository. See [integration review](../docs/integration-review.md).
|
||||
|
||||
# Measurement of intelligence
|
||||
|
||||
Version 0.1 · 13 September 2026
|
||||
|
||||
If the maps in `01` are theories, this file is the **instrument panel**. Intelligence Radar refuses to rank models from a single number, but it also refuses to treat every eval as equally informative.
|
||||
|
||||
## Human measurement (background)
|
||||
|
||||
- **Psychometric batteries** — WAIS-class tests, Raven matrices, specialized factor batteries. Human *g* research lives here.
|
||||
- **Expert work samples** — the closest human analogue to GDPval / OSWorld: can a person produce the artifact a job requires.
|
||||
- **Animal batteries** — delayed match, tool use, social inference. Relevant when someone claims “animal-level AGI.”
|
||||
|
||||
These are monitored for conceptual transfer, not for daily model ranking.
|
||||
|
||||
## Artificial measurement — what still discriminates in September 2026
|
||||
|
||||
Classic exams (MMLU, GSM8K, HumanEval, HellaSwag, ARC-Challenge) are largely **saturated** at the frontier. They remain useful as regression tests and as historical series, not as rankers.
|
||||
|
||||
Instruments that still move, with caveats:
|
||||
|
||||
| Instrument | Layer it hits | Caveat |
|
||||
| --- | --- | --- |
|
||||
| Humanity’s Last Exam (HLE) | expert knowledge + reasoning | tool-use variants change the number |
|
||||
| FrontierMath / FrontierMath Erdős | deep mathematical research | tiny N; one problem moves the headline |
|
||||
| GPQA Diamond | graduate science QA | approaching saturation at the very top |
|
||||
| SciCode | scientific coding | scaffold-sensitive |
|
||||
| SWE-bench Verified / Pro, DeepSWE, Frontier Code | software engineering agents | harness and test-contamination risk |
|
||||
| Terminal-Bench 4.0 | CLI / sysadmin / research agents | version jumps rewrite history |
|
||||
| OSWorld 2.0 | long-horizon computer use | live desktop; expensive to run |
|
||||
| ARC-AGI-3 | abstract interactive reasoning | wrapper/harness can swing scores by tens of points |
|
||||
| GDPval-AA v2, GDP.pdf, AA-Briefcase | professional knowledge work | closer to economic value |
|
||||
| AutomationBench-AA | agentic office workflows | private set; good contamination resistance |
|
||||
| AA-Omniscience, SimpleQA, CritPt | factuality / calibration | “smart but wrong” detector |
|
||||
| Artificial Analysis Intelligence Index v4.3 | composite of the above class | re-bases; never compare raw scores across index versions |
|
||||
| Epoch Capabilities Index (ECI) | long-run capability scale | best for *history*, not shopping |
|
||||
| LMSYS / Arena-style preference | human taste | not a substitute for task success |
|
||||
|
||||
## Rules for using scores
|
||||
|
||||
1. **Same harness, same date, same effort setting.** “Max” vs “medium” reasoning is a different product.
|
||||
2. **Price per completed task beats price per million tokens** when the model burns hidden reasoning tokens.
|
||||
3. **Availability is part of the score.** A preview-only or capacity-gated model is tagged, not treated as generally usable.
|
||||
4. **Open weights vs API** are different goods (latency, privacy, sovereignty, fine-tune rights).
|
||||
5. **A 10% degradation** in Task Router language means: on the *task-weighted* blend of the five mapped benchmarks, comparative task-fit proxy is within 10% relative of the Frontier model (see `task-router/METHOD.md`). It is not “10 Elo” and it is not “10 index points.”
|
||||
|
||||
## What we do not pretend to measure well
|
||||
|
||||
- Scientific novelty (a model can ace HLE and still not find a new theorem).
|
||||
- Taste, aesthetics, and institutional judgment.
|
||||
- Long-run reliability in production (weeks of agent autonomy).
|
||||
- Alignment, refusal quality, and dual-use except via lab preparedness designations and independent red-team reports.
|
||||
- Consciousness.
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2026-09-13: v0.1 instrument panel.
|
||||
50
compendium/03-frontier-as-of-2026-09.md
Normal file
50
compendium/03-frontier-as-of-2026-09.md
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
> Imported exploration, 2026-09-13. Source claims have not been independently verified by this repository. See [integration review](../docs/integration-review.md).
|
||||
|
||||
# Artificial capability frontier — snapshot 13 September 2026
|
||||
|
||||
This is a **dated reading**, assembled from public leaderboards and lab announcements. Independent aggregators disagree. Treat ranks as a cloud, not a podium.
|
||||
|
||||
## What “frontier” means here
|
||||
|
||||
A model is **frontier-class** if it is inside the current top cluster on at least one of: Artificial Analysis Intelligence Index (current version), Epoch ECI, a hard agentic coding/computer-use board, or a hard science/math board — *and* it is a product someone can actually call or download.
|
||||
|
||||
Peak-score models that are invitation-only or unreleased (e.g. some Mythos-class preview lanes) are **frontier-adjacent**, not generally usable Frontier.
|
||||
|
||||
## Cluster as of this week
|
||||
|
||||
Rough usable cluster (not a single official ranking):
|
||||
|
||||
| Role this week | Model | Lab | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| Joint intelligence-index lead (AA v4.3, max effort) | Claude Fable 5.1 and GPT-6 Astra | Anthropic / OpenAI | Tied at 53 on AA Intelligence Index v4.3 after the 4–7 Sep rebase |
|
||||
| Strong general / coding workhorse at mid price | Claude Opus 5 | Anthropic | Same $5/$25 list as older Opus; SWE-bench Verified ~96, Pro ~79.2 (lab-reported) |
|
||||
| Previous OpenAI flagship still widely deployed | GPT-5.6 Sol / Terra / Luna | OpenAI | Sol remains a reference; Terra often the efficiency pick inside the family |
|
||||
| Cheap multimodal workhorse | Gemini 3.8 Flash | Google | GA 2 Sep; Gemini 3.5 Pro still not generally shipped |
|
||||
| Real-time / long-horizon value hosted model | Grok 4.6 | xAI | Current xAI flagship; Grok 4.7 rumored, not shipped as of 13 Sep |
|
||||
| Closed Meta line | Muse Spark 1.3 | Meta Superintelligence Labs | Low list price vs US lab peers on some boards |
|
||||
| Leading open / exportable cluster | Kimi K3, Qwen3.8-Max, DeepSeek V4.x (V4-Pro and V4.1 Flash), GLM-5.3 | Moonshot, Alibaba, DeepSeek, Z.ai | Open or semi-open paths; distillation and export-control fights are political facts |
|
||||
|
||||
**Headline capability claims to verify, not swallow:** GPT-6 Astra’s ARC-AGI-3 number is wrapper-sensitive (public reports of ~63% on ARC Prize harness vs 99.9% on a provider adapter). That gap is itself an Intelligence Radar finding: **the harness is part of the model.**
|
||||
|
||||
## Economics and availability (the other half of the frontier)
|
||||
|
||||
- **Price span** among near-frontier models is two orders of magnitude on blended $/M tokens and still large on *cost per completed hard task*.
|
||||
- **Token cost ≠ task cost.** Reasoning models that emit fewer hidden tokens can beat a cheaper list price.
|
||||
- **Capacity is a constraint.** Mid-September reporting: OpenAI paused some $200 ChatGPT Pro signups/upgrades amid Astra demand. A sold-out Frontier is not an available Frontier.
|
||||
- **Access tiers matter:** Fable 5.1 general vs Mythos 5.1 restricted; Astra first to cybersecurity “Daybreak” customers then broader; some Chinese weights are legally or practically unavailable in parts of the West.
|
||||
- **Hardware sovereignty** is now a first-class story (DeepSeek training paths on non-NVIDIA stacks; EU/US compute buildout; export controls).
|
||||
|
||||
Detailed prices live in `../frontier/ledger-2026-09.md` and will drift. Re-check before any buying decision.
|
||||
|
||||
## What moved this month (for the weekly series)
|
||||
|
||||
- 1 Sep — Claude Fable 5.1 / Mythos 5.1
|
||||
- 2 Sep — Gemini 3.8 Flash; Muse Spark 1.3
|
||||
- 3 Sep — GPT-6 Astra
|
||||
- 4–7 Sep — Artificial Analysis Index v4.2 then v4.3 rebase (harder terminal bench + AutomationBench-AA)
|
||||
- 10 Sep — DeepSeek V4.1 Flash; Cognition SWE-2 (near-frontier coding at lower cost)
|
||||
- Ongoing — Grok 4.7 slip; Gemini 3.5 Pro still unshipped; policy noise in Washington and Beijing
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2026-09-13: v0.1 snapshot.
|
||||
Loading…
Add table
Add a link
Reference in a new issue