intelligence-radar/ArchitectureBlueprint.md

> Target architecture. The implemented local v0.1 boundary is in [SCOPE.md](SCOPE.md) and [ADR 0001](docs/adr/0001-initial-operational-baseline.md).

# Intelligence Radar — Architecture Blueprint

**Status:** Draft 0.1  
**Date:** 2026-09-13

## 1. Architectural objective

Build a system that can continuously ingest evidence about intelligence and AI capability, preserve provenance, update a structured knowledge model, and answer task-specific model-selection questions reproducibly.

The design separates:

1. **observations** — immutable source-derived facts;
2. **knowledge** — claims and relationships built from observations;
3. **policy** — versioned interpretation and ranking rules;
4. **recommendations** — time-indexed outputs produced from observations + policy;
5. **outcomes** — real task results used for calibration.

## 2. High-level architecture

```text
                         ┌──────────────────────┐
                         │  Primary Sources     │
                         │ papers/docs/repos    │
                         │ labs/regulators/etc. │
                         └──────────┬───────────┘
                                    │
                        discovery / fetch / poll
                                    │
                  ┌─────────────────▼─────────────────┐
                  │          Ingestion Plane          │
                  │ source registry · fetchers        │
                  │ dedupe · provenance · extraction │
                  └─────────────────┬─────────────────┘
                                    │
                           immutable observations
                                    │
        ┌───────────────────────────▼───────────────────────────┐
        │                  Knowledge Plane                     │
        │ events · claims · evidence · models · deployments   │
        │ benchmarks · prices · hardware · regulations        │
        │ task outcomes · historical versions                 │
        └───────────────┬───────────────────────┬──────────────┘
                        │                       │
             compendium synthesis        capability data
                        │                       │
          ┌─────────────▼─────────┐   ┌────────▼────────────────┐
          │  Intelligence        │   │ Capability Advisor       │
          │  Compendium          │   │ task profiler            │
          │  claim graph         │   │ benchmark selector       │
          │  theory maps         │   │ model scorer             │
          └─────────────┬────────┘   │ economics estimator      │
                        │            │ workplan optimizer       │
                        │            └──────────┬───────────────┘
                        │                       │
              ┌─────────▼───────────────────────▼────────┐
              │             Publication Plane            │
              │ daily delta · weekly digest · query API  │
              │ recommendation report · dashboards      │
              └───────────────────┬──────────────────────┘
                                  │
                                  ▼
                         execution / feedback
                                  │
                      ┌───────────▼───────────┐
                      │ Outcome & Calibration│
                      │ task-performance     │
                      │ ledger               │
                      └───────────┬───────────┘
                                  └──────► model calibration
```

## 3. Data layers

### 3.1 Source registry

A versioned registry describing what should be monitored.

Fields:

- source id;
- publisher/person;
- source type;
- URL/API/feed;
- topic tags;
- trust tier;
- primary/secondary;
- expected cadence;
- retrieval method;
- license/copyright constraints;
- last successful observation;
- importance;
- notes.

### 3.2 Raw observation store

Append-only.

Stores:

- retrieval timestamp;
- publication/event timestamp;
- source URI;
- content hash;
- document metadata;
- extracted structured facts;
- retrieval agent/version;
- link to raw artifact where legally appropriate;
- provenance.

Never overwrite a historical observation because a source page later changes.

### 3.3 Event store

Normalized events such as:

- model released;
- model deprecated;
- benchmark version released;
- benchmark score published;
- price changed;
- provider added/removed;
- paper published;
- regulation enacted/applies;
- hardware announced/shipped;
- safety incident;
- capability claim;
- benchmark invalidation/saturation evidence.

### 3.4 Claim graph

Compendium entities and evidence relationships described in `IntelligenceConceptModel.md`.

### 3.5 Capability catalog

Contains separate entities for:

- `model`;
- `execution_profile`;
- `deployment`;
- `benchmark`;
- `benchmark_version`;
- `benchmark_result`;
- `price_observation`;
- `availability_observation`;
- `hardware/system_result`.

### 3.6 Task-performance ledger

Stores real task predictions and outcomes.

This becomes the calibration dataset for task-specific performance prediction.

## 4. Suggested initial physical implementation

Start boring and auditable:

- **Git** for normative Markdown, policies, schemas, hand-curated catalog seeds, and digest artifacts.
- **PostgreSQL** for indexed structured observations, claim relationships, benchmark results, model/deployment catalog, recommendation history, and task-outcome ledger.
- **Object storage** for larger source artifacts and generated reports.
- **Parquet/Iceberg later** when observation volume, historical analytics, or multi-engine processing warrants it.
- **Embeddings/vector search** as an index over evidence, never as the canonical source of truth.

The architecture should not require a graph database initially. PostgreSQL can represent graph edges cleanly; a specialized graph engine can be added when query patterns justify it.

## 5. Ingestion pipeline

```text
discover
  → fetch
  → fingerprint
  → deduplicate
  → classify
  → extract candidate facts/events
  → resolve entities
  → attach provenance
  → confidence checks
  → append observation
  → emit event
  → candidate claim update
```

### Agentic extraction rule

Agents may propose structured facts and claims.

They must not silently turn their inference into source fact.

Every extracted field should carry one of:

- `explicit_source_statement`
- `derived_from_source`
- `cross_source_inference`
- `system_estimate`

## 6. Daily Radar pipeline

Daily runs should answer:

1. What new source artifacts appeared?
2. What facts/events are genuinely new?
3. What existing observations were corrected or contradicted?
4. Did any benchmark materially change?
5. Did price, availability, context limit, or model status change?
6. Did the capability frontier move?
7. Did the cost/performance Pareto frontier move?
8. Did any event affect a Compendium claim?
9. Did any current recommendation policy become questionable?

Output should be a **delta**, not a generic summary of the news.

## 7. Weekly synthesis pipeline

The weekly digest ranks events by estimated significance.

Recommended significance dimensions:

- epistemic impact;
- capability impact;
- economic impact;
- breadth of affected tasks;
- confidence;
- novelty;
- persistence;
- governance/social impact.

The weekly synthesis should explicitly state:

> What do we have more reason to believe than seven days ago?

and:

> What do we have less reason to believe?

## 8. Capability Advisor pipeline

```text
task
 ↓
Task Interpreter
 ↓
Clarity + constraints + risk class
 ↓
Capability Profile
 ↓
Benchmark Candidate Retrieval
 ↓
Five-Benchmark Selector
 ↓
Result Freshness / Validity Filter
 ↓
Execution Profile Candidate Builder
 ↓
Task-Fit Estimator
 ↓
Availability Filter
 ↓
Cost + Runtime Estimator
 ↓
Frontier / Topnotch / Efficient / Sufficient / Budget
 ↓
Confidence decomposition
 ↓
optional Workplan Optimizer
```

## 9. Workplan optimizer

The optimizer converts a task into a DAG.

Each node contains:

- subtask;
- inputs/outputs;
- capability profile;
- dependency edges;
- acceptance/verifier;
- parallelism constraints;
- privacy/tool constraints;
- candidate execution profiles;
- predicted success;
- cost distribution;
- runtime distribution.

It should generate several plans, at minimum:

1. **Monolithic Frontier**
2. **Monolithic Efficient**
3. **Decomposed Quality**
4. **Decomposed Efficient**
5. **Decomposed Budget**

Optimization objectives can include:

- maximize predicted success subject to budget;
- minimize expected cost subject to success threshold;
- minimize p90 runtime subject to quality threshold;
- maximize quality-adjusted value.

### Verification as a first-class subtask

Do not allocate all budget to generation.

A cheaper producer plus an independent verifier can outperform an expensive monolithic attempt when errors are detectable.

## 10. Confidence model

Store components independently:

- `task_clarity`
- `benchmark_coverage`
- `benchmark_validity`
- `measurement_reliability`
- `result_freshness`
- `deployment_match`
- `price_confidence`
- `task_calibration_evidence`
- `overall_recommendation_confidence`

The overall confidence number is a convenience. The components are the actual explanation.

## 11. Temporal semantics

Every mutable external fact must support:

- `observed_at`
- `valid_from` if known
- `valid_to` if known
- `source_published_at`
- `supersedes_observation_id` where applicable.

A model price without a timestamp is not valid capability-economics data.

## 12. Policy versioning

Examples:

- benchmark selection policy;
- task capability ontology;
- benchmark-score normalization;
- Topnotch threshold;
- sufficiency threshold inference;
- cost model;
- confidence aggregation;
- source trust;
- model eligibility.

Every recommendation stores policy hashes/versions.

Historical recommendations therefore remain reproducible.

## 13. API surface — first draft

### `POST /evaluate-task`

Returns task profile and model recommendation.

### `POST /optimize-workplan`

Returns model-allocated DAG alternatives.

### `POST /record-outcome`

Adds verified outcome evidence.

### `GET /models`

Current model catalog.

### `GET /deployments`

Provider-specific availability and economics.

### `GET /benchmarks`

Benchmark catalog and health.

### `GET /radar/events`

Normalized recent events.

### `GET /digests/{period}`

Daily/weekly Radar products.

### `GET /compendium/claims/{id}`

Claim + evidence graph.

## 14. Minimal recommendation record

```yaml
task_id: task-...
evaluated_at: 2026-09-13T...
policy:
  task_profile: v0.1
  benchmark_selection: v0.1
  model_ranking: v0.1
clarity:
  score: 0.78
  ambiguities: []
benchmarks:
  - id: ...
    weight: 0.27
frontier:
  execution_profile: ...
topnotch: [...]
efficient: ...
sufficient: [...]
budget: ...
economics:
  currency: USD
  expected: ...
  p90: ...
runtime:
  expected_seconds: ...
confidence:
  overall: ...
  components: ...
evidence_gaps: [...]
```

## 15. Initial milestones

### M0 — Repository kernel

Documents, schemas, policies, source catalog.

### M1 — Dated model/economics catalog

Automated collection of model releases, availability, context limits, and API prices from major providers.

### M2 — Benchmark health catalog

Benchmark metadata, versions, results, saturation and contamination notes.

### M3 — Task profiler

Produces stable capability profiles and clarity assessment.

### M4 — Five-benchmark selector

Produces explainable benchmark mapping with independence penalties.

### M5 — Model ranking

Frontier / Topnotch / Efficient / Sufficient / Budget with cost intervals.

### M6 — Workplan optimizer

DAG decomposition, heterogeneous model allocation, verification, parallel runtime estimate.

### M7 — Outcome ledger

Capture actual task success, cost, runtime, and failure modes.

### M8 — Calibration engine

Learn task-cluster performance models from the outcome ledger.

### M9 — Self-improving Radar

Detect benchmark drift/saturation and automatically propose policy/catalog changes for review.