coulomb/intelligence-radar
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Integrate founding explorations and bootstrap operational radar

Browse source 
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a09c77-b1a9-7eb3-b501-6cb24b458c97
This commit is contained in:
tegwick 2026-09-13 22:42:09 +02:00
parent 775f93b920
commit d0e6438609
57 changed files with 4723 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

296
INTENT.md Normal file
View file

@ -0,0 +1,296 @@
# INTENT — Intelligence Radar
**Status:** Adopted baseline 0.1 (2026-09-13)
**Date:** 2026-09-13
## 1. Intent
Intelligence Radar exists to improve our ability to reason about **intelligence itself** and to make practical decisions about **where, when, and at what price machine intelligence can be applied**.
The facility shall continuously answer two different classes of questions:
### Epistemic questions
- What do humans mean when they use the word *intelligence*?
- Which models of intelligence exist across biology, cognitive science, cybernetics, neuroscience, psychology, computer science, AI, economics, organizational theory, collective intelligence, and related disciplines?
- Which claims are supported, contested, speculative, superseded, or refuted?
- Which observations or experiments would distinguish competing models?
- Where are we confusing measurement convenience with the underlying construct?
### Operational questions
For a concrete task:
- What capabilities does the task require?
- Which five current benchmarks provide the most useful evidence about those capabilities, and with what relevance?
- Which available execution profile is the **Frontier** choice?
- Which profiles are **Topnotch** — at most 10% predicted degradation from Frontier?
- Which Topnotch profile is the cheapest expected solution — the **Efficient** choice?
- Which profiles are credibly **Sufficient** to satisfy the task?
- Which Sufficient profile is the cheapest — the **Budget** choice?
- How clear is the task?
- How confident are we in the benchmark mapping, capability estimate, availability, cost estimate, and final recommendation?
- What is the expected cost and runtime?
- Would decomposition into subtasks and collaboration among specialized models improve quality, confidence, cost, or runtime?
## 2. Scope
Intelligence Radar spans:
- biological intelligence;
- cognition and neuroscience;
- psychometrics and measurement;
- cybernetics, control, adaptation, and viability;
- embodied and situated intelligence;
- collective, organizational, swarm, and social intelligence;
- artificial intelligence and machine learning;
- foundation models and generative systems;
- agents and multi-agent systems;
- software and systems engineering for intelligent systems;
- AI accelerators, inference systems, networks, storage, and energy;
- capability evaluation and benchmark science;
- economics, price, availability, and accessibility of intelligence;
- safety, security, alignment, reliability, and misuse;
- economic and social effects;
- law, regulation, standards, and public policy.
The facility must not equate *AI progress* with *progress in understanding intelligence*.
## 3. Three planes
### 3.1 Intelligence Compendium
A durable, versioned epistemic graph describing concepts, theories, claims, evidence, objections, thinkers, experiments, systems, and historical development.
It answers:
> What does humanity currently have reason to believe about intelligence?
### 3.2 Intelligence Radar
A high-frequency event and delta layer.
It answers:
> What changed, and which previous beliefs, measurements, policies, capability estimates, or economic assumptions should change because of it?
### 3.3 Capability Advisor
A task-specific decision system.
It answers:
> What intelligence should be used for this task, under these conditions, today?
## 4. Normative vocabulary
### Model
A base trained model, e.g. a named model family/version.
### Execution Profile
A model plus the conditions that materially affect measured capability:
- reasoning/effort level;
- prompting/scaffold;
- system instructions;
- agent harness;
- available tools;
- context strategy;
- retrieval;
- memory;
- sampling parameters;
- number of attempts or ensemble strategy.
### Deployment
An execution profile bound to practical availability:
- provider;
- API/product;
- region;
- service tier;
- rate limits;
- context/output limits;
- quantization or serving stack;
- price;
- latency;
- data-handling constraints;
- availability date/status.
**Benchmarks primarily measure execution profiles, not Platonic model capability.**
### Frontier
The eligible execution profile with the highest predicted task performance, regardless of cost.
### Topnotch
Eligible profiles whose predicted task performance is at most 10% below Frontier under the active ranking policy.
Default policy:
`task_score(profile) >= 0.90 × task_score(frontier)`
### Efficient
The Topnotch profile with the lowest expected total task cost.
### Sufficient
A profile whose predicted probability of satisfying the task's acceptance criteria exceeds the task's declared or inferred sufficiency threshold.
### Budget
The Sufficient profile with the lowest expected total task cost.
### Available Frontier
The highest-performing profile the requester can actually use under jurisdiction, access, privacy, product, and deployment constraints.
## 5. Foundational principles
### P1 — Intelligence is contextual
A capability exists relative to an environment, objective, resource budget, interface, and observation method.
### P2 — Do not collapse intelligence into one scalar
Aggregate indices are useful summaries, not ontologies.
### P3 — Benchmarks are instruments
Every benchmark has a construct, validity envelope, error model, saturation state, contamination risk, ecological validity, and expiration risk.
### P4 — Five benchmarks are selected per task
There is no permanent canonical five. The selector chooses five task-relevant measurements while penalizing redundancy.
If evidence coverage is poor, the system still returns its best five candidates but explicitly marks the mapping as weak.
### P5 — Prefer independent evidence
Five correlated benchmarks do not constitute five independent observations.
### P6 — Distinguish observed fact from policy
Raw benchmark results, prices, availability observations, and task outcomes are facts with provenance.
`topnotch degradation <= 10%`, benchmark weighting, sufficiency thresholds, and confidence aggregation are policies and must be versioned.
### P7 — Price is time indexed
Promotional rates, negotiated rates, region, provider, caching, batch/flex tiers, token lengths, and tool charges can change the economics drastically.
### P8 — Cost per token is not cost per solved task
Expected cost includes retries, reasoning tokens, tool calls, cache behavior, failures, and verification.
### P9 — Outcome evidence outranks leaderboard intuition
Public benchmarks seed predictions. Verified real-world outcomes on structurally similar tasks should progressively calibrate or override generic benchmark priors.
### P10 — Preserve disagreement
The Compendium records competing hypotheses and unresolved contradictions rather than prematurely synthesizing them away.
### P11 — Primary sources first
Secondary reporting discovers events. Durable claims should resolve to primary research, official documentation, benchmark maintainers, regulator texts, standards, or first-party measurements whenever possible.
### P12 — Decomposition is an optimization problem
A task may be better solved by a workplan of heterogeneous models than by one global Frontier model.
## 6. Core service contract
Input:
- task/question;
- supplied context;
- constraints;
- required output;
- acceptance criteria, if known;
- risk/reliability requirements;
- environment/tool constraints;
- optional budget/runtime limits.
Output:
1. **Task interpretation**
2. **Task clarity score** and missing/ambiguous information
3. **Capability profile**
4. **Five selected benchmarks**
- relevance weight;
- rationale;
- measurement quality;
- data freshness;
- known weaknesses
5. **Candidate execution profiles**
6. **Frontier**
7. **Topnotch set**
8. **Efficient**
9. **Sufficient set**
10. **Budget**
11. **Expected cost** with p50/p90 or equivalent interval
12. **Expected runtime** with uncertainty
13. **Recommendation confidence**
14. **Evidence gaps**
15. **Optional optimized workplan**
16. **Comparison against monolithic execution**
17. **Post-execution outcome record** when results become available
## 7. Feedback loop
Every completed task should be eligible to create a calibration observation:
- canonical task profile;
- selected benchmarks and policy versions;
- predicted performance;
- chosen execution profile;
- actual model/provider configuration;
- input/output/cached/reasoning token use;
- tool calls;
- retries;
- elapsed and active runtime;
- acceptance-test results;
- verifier/judge outcomes;
- human assessment;
- failure mode;
- environmental context.
These observations form the **Task → Model Performance Ledger**.
The long-term goal is:
> Replace generic benchmark inference with empirically calibrated predictions for classes of real work while retaining benchmarks as transferable prior evidence.
## 8. Success criteria
Intelligence Radar succeeds when it can:
- explain why a recommendation was made;
- reproduce a historical recommendation from its dated evidence and policy versions;
- detect when a benchmark ceases to predict real outcomes;
- distinguish model capability from scaffolding and deployment effects;
- identify cost/performance Pareto frontiers rather than merely rank models;
- provide actionable model choices for tasks with explicit uncertainty;
- reduce cost without materially reducing task success;
- improve its own predictions from observed outcomes;
- maintain a useful, non-dogmatic map of scientific and philosophical models of intelligence.
## 9. Non-goals
Intelligence Radar is not:
- a universal AGI score;
- a model fan leaderboard;
- a news-content farm;
- an automatic claim that benchmark performance equals real-world competence;
- an authority deciding what intelligence “really is”;
- a system that hides uncertainty behind precise-looking numbers.
## 10. One-line description
> **Intelligence Radar continuously maps what intelligence appears to be, what intelligent systems can demonstrably do, and the cheapest reliable way to apply those capabilities to a concrete task.**