intelligence-radar/ResearchProgram.md

# Intelligence Radar Research Program

**Status:** Draft 0.1  
**Date:** 2026-09-13

## 1. Research objective

Maintain a continuously improving map of:

1. what intelligence is proposed to mean;
2. how intelligence is generated and constrained across substrates;
3. how it can be measured;
4. what artificial systems can currently do;
5. what those capabilities cost and where they are available;
6. what the consequences are for engineering, science, economics, society, safety, law, and governance.

The program intentionally joins **slow epistemic research** with **fast operational monitoring**.

## 2. Research streams

### R1 — Models of intelligence

Questions:

- What definitions and operationalizations exist?
- Which phenomena do they explain well?
- Which assumptions do they make about goals, representation, embodiment, learning, or agency?
- Which theories are complementary versus genuinely contradictory?
- What evidence could discriminate among them?

### R2 — Biological intelligence

- neural computation;
- memory and learning;
- animal cognition;
- development;
- evolution;
- perception/action loops;
- energetic efficiency;
- social cognition.

### R3 — Systems, cybernetics, and control

- requisite variety;
- regulation;
- viability;
- feedback;
- adaptation;
- recursive organization;
- observer dependence;
- self-reference;
- autopoiesis/enaction.

### R4 — Artificial architectures

- deep learning;
- transformers and alternatives;
- recurrent/state-space systems;
- world models;
- reinforcement learning;
- memory;
- retrieval;
- planning;
- tool use;
- program synthesis;
- neuro-symbolic systems;
- robotics;
- multi-agent systems.

### R5 — Capability measurement

- construct validity;
- benchmark design;
- contamination;
- saturation;
- evaluator/judge reliability;
- reproducibility;
- ecological validity;
- long-horizon evaluation;
- hidden/private tests;
- human baselines;
- task distributions.

### R6 — Capability frontier

Track:

- model releases;
- execution profiles;
- agent scaffolds;
- context and memory;
- multimodality;
- tool/computer use;
- reliability;
- autonomy horizon;
- coding;
- research;
- scientific reasoning;
- professional knowledge work;
- security-relevant capability.

### R7 — Economics of intelligence

Track:

- input/output/cache/reasoning token prices;
- tool charges;
- cost per solved task;
- runtime;
- throughput;
- latency;
- energy;
- accelerator economics;
- self-hosting economics;
- open-weight availability;
- batching/flex/priority tiers;
- geography and access restrictions.

Goal:

> Build a quality-adjusted price surface for intelligence rather than a token-price table.

### R8 — Intelligence engineering

- inference infrastructure;
- accelerators;
- memory bandwidth;
- networking;
- compilers;
- quantization;
- serving;
- speculative decoding;
- distributed inference;
- edge intelligence;
- agent runtimes;
- observability;
- evaluation harnesses.

### R9 — Safety, security, and reliability

- dangerous capabilities;
- cybersecurity;
- biosecurity;
- alignment;
- deception/manipulation;
- robustness;
- privacy;
- misuse;
- incident evidence;
- control and monitoring methods.

### R10 — Economic and social effects

- productivity;
- task substitution/complementarity;
- labor;
- organizational design;
- education;
- science;
- inequality;
- access;
- market concentration;
- new business models.

### R11 — Law, policy, standards, and institutions

Track original texts and official implementation material for:

- AI regulation;
- safety obligations;
- privacy/data law interactions;
- copyright;
- liability;
- competition;
- standards;
- procurement;
- sector regulation;
- export controls and compute policy where relevant.

## 3. Monitoring cadence

### Daily

Detect high-value deltas.

Output:

- event;
- primary source;
- what changed;
- why it matters;
- affected catalog entities;
- affected claims/policies;
- confidence;
- follow-up required.

### Weekly

Synthesize rather than concatenate.

Recommended sections:

1. **Five most consequential changes**
2. **Capability frontier**
3. **Cost and availability frontier**
4. **Benchmark/evaluation health**
5. **Science of intelligence**
6. **Agents/software/hardware**
7. **Safety and security**
8. **Economics and society**
9. **Politics, law, regulation, standards**
10. **Compendium changes**
11. **What became less certain**
12. **Watchlist for next week**

### Monthly

Perform maintenance that daily/weekly news cannot:

- benchmark health review;
- source quality review;
- thinker/source additions/removals;
- stale claims;
- pricing/catalog reconciliation;
- calibration analysis from real tasks;
- research-gap prioritization.

### Quarterly

Produce a deeper **State of Intelligence** revision:

- major model/theory changes;
- frontier movement;
- economics;
- evaluation validity;
- capability-to-cost curves;
- autonomy horizon;
- open vs closed availability;
- important regulatory changes;
- newly emerging research questions.

## 4. Evidence discipline

### Tier A — Primary evidence

Preferred for durable facts:

- peer-reviewed papers / primary preprints;
- official datasets/evaluation repos;
- benchmark maintainers;
- model/system cards;
- provider documentation;
- official pricing;
- source code/releases;
- regulator/legislator texts;
- standards bodies;
- original experiment data.

### Tier B — Independent measurement / synthesis

Examples:

- independent evaluation organizations;
- systematic benchmark comparisons;
- high-quality academic syntheses;
- major index reports.

Useful for triangulation and cross-provider comparison.

### Tier C — Expert interpretation

Researchers, engineers, economists, legal experts, and other thinkers.

Useful for hypotheses, interpretation, and discovery.

Not equivalent to primary evidence.

### Tier D — General reporting/social signal

Useful as an early warning or discovery stream.

A durable Radar claim should normally resolve to stronger evidence.

## 5. Research item template

```yaml
id: research-item-...
title: ...
research_streams: [R5, R6]
question: ...
status: open
priority: high
claims:
  - ...
evidence_needed:
  - ...
sources:
  - ...
next_actions:
  - ...
last_reviewed: ...
```

## 6. Claim lifecycle

```text
candidate
  → sourced
  → structured
  → challenged
  → triangulated
  → compendium claim
  → periodically re-evaluated
```

No claim becomes durable merely because many secondary articles repeat it.

## 7. Benchmark research lifecycle

Every benchmark gets a health record.

Review:

- intended construct;
- real task distribution;
- scoring correctness;
- ceiling/saturation;
- public-test contamination;
- benchmark gaming incentives;
- scaffold sensitivity;
- judge sensitivity;
- version comparability;
- human baseline quality;
- cost to run;
- licensing/access;
- predictive correlation with Intelligence Radar task outcomes.

A benchmark can be:

- `candidate`
- `active`
- `watch`
- `degraded`
- `saturated`
- `retired`

Retirement does not delete historical results.

## 8. Frontier research question

Do not ask only:

> What is the best model?

Track multiple frontiers:

- maximal quality;
- quality per dollar;
- quality per joule;
- quality per second;
- quality per GB of memory;
- quality under local/self-hosted constraints;
- quality under privacy/ZDR constraints;
- autonomy horizon;
- open-weight frontier;
- deployable frontier by region/provider;
- multimodal frontier;
- specialized domain frontiers.

## 9. Calibration research

The most strategically important proprietary dataset will be the Task → Model Performance Ledger.

Research questions:

- Which public benchmarks best predict which task clusters?
- When do model-family/vendor effects remain after benchmark scores?
- How stable are rankings across scaffolds?
- Which task features predict failures?
- How much does decomposition help?
- When is a verifier more valuable than a larger generator?
- How do costs scale with ambiguity?
- How well does model self-estimated confidence predict actual success?
- Which benchmarks lose predictive value first as the frontier moves?

## 10. Anti-goals

Avoid:

- benchmark tourism;
- leaderboard worship;
- treating press-release evaluation tables as neutral evidence;
- silently mixing model-only and agent-scaffold results;
- using stale prices;
- measuring task success only with model judges;
- collapsing heterogeneous scientific concepts into a single AGI narrative;
- overreacting to every daily release.

## 11. First twelve research tasks

1. Build a genealogy of definitions of intelligence across psychometrics, cybernetics, cognitive science, neuroscience, AI, and collective intelligence.
2. Establish a benchmark construct taxonomy.
3. Create a benchmark health scorecard.
4. Model `execution_profile` explicitly and normalize existing leaderboard evidence into it.
5. Build price-observation ingestion for major providers.
6. Build a first capability-vector ontology from 50 representative real tasks.
7. Test the five-benchmark selector manually on those 50 tasks.
8. Establish initial success-probability calibration using repeated model trials.
9. Compare monolithic vs decomposed execution on at least 20 complex tasks.
10. Measure benchmark correlation and redundancy.
11. Define availability/privacy constraints as first-class ranking filters.
12. Publish the first quarterly State of Intelligence report from accumulated Radar evidence.