intelligence-radar/ArchitectureBlueprint.md

Target architecture. The implemented local v0.1 boundary is in SCOPE.md and ADR 0001.

Intelligence Radar — Architecture Blueprint

Status: Draft 0.1
Date: 2026-09-13

1. Architectural objective

Build a system that can continuously ingest evidence about intelligence and AI capability, preserve provenance, update a structured knowledge model, and answer task-specific model-selection questions reproducibly.

The design separates:

1. observations — immutable source-derived facts;
2. knowledge — claims and relationships built from observations;
3. policy — versioned interpretation and ranking rules;
4. recommendations — time-indexed outputs produced from observations + policy;
5. outcomes — real task results used for calibration.

2. High-level architecture

                         ┌──────────────────────┐
                         │  Primary Sources     │
                         │ papers/docs/repos    │
                         │ labs/regulators/etc. │
                         └──────────┬───────────┘
                                    │
                        discovery / fetch / poll
                                    │
                  ┌─────────────────▼─────────────────┐
                  │          Ingestion Plane          │
                  │ source registry · fetchers        │
                  │ dedupe · provenance · extraction │
                  └─────────────────┬─────────────────┘
                                    │
                           immutable observations
                                    │
        ┌───────────────────────────▼───────────────────────────┐
        │                  Knowledge Plane                     │
        │ events · claims · evidence · models · deployments   │
        │ benchmarks · prices · hardware · regulations        │
        │ task outcomes · historical versions                 │
        └───────────────┬───────────────────────┬──────────────┘
                        │                       │
             compendium synthesis        capability data
                        │                       │
          ┌─────────────▼─────────┐   ┌────────▼────────────────┐
          │  Intelligence        │   │ Capability Advisor       │
          │  Compendium          │   │ task profiler            │
          │  claim graph         │   │ benchmark selector       │
          │  theory maps         │   │ model scorer             │
          └─────────────┬────────┘   │ economics estimator      │
                        │            │ workplan optimizer       │
                        │            └──────────┬───────────────┘
                        │                       │
              ┌─────────▼───────────────────────▼────────┐
              │             Publication Plane            │
              │ daily delta · weekly digest · query API  │
              │ recommendation report · dashboards      │
              └───────────────────┬──────────────────────┘
                                  │
                                  ▼
                         execution / feedback
                                  │
                      ┌───────────▼───────────┐
                      │ Outcome & Calibration│
                      │ task-performance     │
                      │ ledger               │
                      └───────────┬───────────┘
                                  └──────► model calibration

3. Data layers

3.1 Source registry

A versioned registry describing what should be monitored.

Fields:

• source id;
• publisher/person;
• source type;
• URL/API/feed;
• topic tags;
• trust tier;
• primary/secondary;
• expected cadence;
• retrieval method;
• license/copyright constraints;
• last successful observation;
• importance;
• notes.

3.2 Raw observation store

Append-only.

Stores:

• retrieval timestamp;
• publication/event timestamp;
• source URI;
• content hash;
• document metadata;
• extracted structured facts;
• retrieval agent/version;
• link to raw artifact where legally appropriate;
• provenance.

Never overwrite a historical observation because a source page later changes.

3.3 Event store

Normalized events such as:

• model released;
• model deprecated;
• benchmark version released;
• benchmark score published;
• price changed;
• provider added/removed;
• paper published;
• regulation enacted/applies;
• hardware announced/shipped;
• safety incident;
• capability claim;
• benchmark invalidation/saturation evidence.

3.4 Claim graph

Compendium entities and evidence relationships described in IntelligenceConceptModel.md.

3.5 Capability catalog

Contains separate entities for:

• model;
• execution_profile;
• deployment;
• benchmark;
• benchmark_version;
• benchmark_result;
• price_observation;
• availability_observation;
• hardware/system_result.

3.6 Task-performance ledger

Stores real task predictions and outcomes.

This becomes the calibration dataset for task-specific performance prediction.

4. Suggested initial physical implementation

Start boring and auditable:

• Git for normative Markdown, policies, schemas, hand-curated catalog seeds, and digest artifacts.
• PostgreSQL for indexed structured observations, claim relationships, benchmark results, model/deployment catalog, recommendation history, and task-outcome ledger.
• Object storage for larger source artifacts and generated reports.
• Parquet/Iceberg later when observation volume, historical analytics, or multi-engine processing warrants it.
• Embeddings/vector search as an index over evidence, never as the canonical source of truth.

The architecture should not require a graph database initially. PostgreSQL can represent graph edges cleanly; a specialized graph engine can be added when query patterns justify it.

5. Ingestion pipeline

discover
  → fetch
  → fingerprint
  → deduplicate
  → classify
  → extract candidate facts/events
  → resolve entities
  → attach provenance
  → confidence checks
  → append observation
  → emit event
  → candidate claim update

Agentic extraction rule

Agents may propose structured facts and claims.

They must not silently turn their inference into source fact.

Every extracted field should carry one of:

• explicit_source_statement
• derived_from_source
• cross_source_inference
• system_estimate

6. Daily Radar pipeline

Daily runs should answer:

1. What new source artifacts appeared?
2. What facts/events are genuinely new?
3. What existing observations were corrected or contradicted?
4. Did any benchmark materially change?
5. Did price, availability, context limit, or model status change?
6. Did the capability frontier move?
7. Did the cost/performance Pareto frontier move?
8. Did any event affect a Compendium claim?
9. Did any current recommendation policy become questionable?

Output should be a delta, not a generic summary of the news.

7. Weekly synthesis pipeline

The weekly digest ranks events by estimated significance.

Recommended significance dimensions:

• epistemic impact;
• capability impact;
• economic impact;
• breadth of affected tasks;
• confidence;
• novelty;
• persistence;
• governance/social impact.

The weekly synthesis should explicitly state:

What do we have more reason to believe than seven days ago?

and:

What do we have less reason to believe?

8. Capability Advisor pipeline

task
 ↓
Task Interpreter
 ↓
Clarity + constraints + risk class
 ↓
Capability Profile
 ↓
Benchmark Candidate Retrieval
 ↓
Five-Benchmark Selector
 ↓
Result Freshness / Validity Filter
 ↓
Execution Profile Candidate Builder
 ↓
Task-Fit Estimator
 ↓
Availability Filter
 ↓
Cost + Runtime Estimator
 ↓
Frontier / Topnotch / Efficient / Sufficient / Budget
 ↓
Confidence decomposition
 ↓
optional Workplan Optimizer

9. Workplan optimizer

The optimizer converts a task into a DAG.

Each node contains:

• subtask;
• inputs/outputs;
• capability profile;
• dependency edges;
• acceptance/verifier;
• parallelism constraints;
• privacy/tool constraints;
• candidate execution profiles;
• predicted success;
• cost distribution;
• runtime distribution.

It should generate several plans, at minimum:

1. Monolithic Frontier
2. Monolithic Efficient
3. Decomposed Quality
4. Decomposed Efficient
5. Decomposed Budget

Optimization objectives can include:

• maximize predicted success subject to budget;
• minimize expected cost subject to success threshold;
• minimize p90 runtime subject to quality threshold;
• maximize quality-adjusted value.

Verification as a first-class subtask

Do not allocate all budget to generation.

A cheaper producer plus an independent verifier can outperform an expensive monolithic attempt when errors are detectable.

10. Confidence model

Store components independently:

• task_clarity
• benchmark_coverage
• benchmark_validity
• measurement_reliability
• result_freshness
• deployment_match
• price_confidence
• task_calibration_evidence
• overall_recommendation_confidence

The overall confidence number is a convenience. The components are the actual explanation.

11. Temporal semantics

Every mutable external fact must support:

• observed_at
• valid_from if known
• valid_to if known
• source_published_at
• supersedes_observation_id where applicable.

A model price without a timestamp is not valid capability-economics data.

12. Policy versioning

Examples:

• benchmark selection policy;
• task capability ontology;
• benchmark-score normalization;
• Topnotch threshold;
• sufficiency threshold inference;
• cost model;
• confidence aggregation;
• source trust;
• model eligibility.

Every recommendation stores policy hashes/versions.

Historical recommendations therefore remain reproducible.

13. API surface — first draft

POST /evaluate-task

Returns task profile and model recommendation.

POST /optimize-workplan

Returns model-allocated DAG alternatives.

POST /record-outcome

Adds verified outcome evidence.

GET /models

Current model catalog.

GET /deployments

Provider-specific availability and economics.

GET /benchmarks

Benchmark catalog and health.

GET /radar/events

Normalized recent events.

GET /digests/{period}

Daily/weekly Radar products.

GET /compendium/claims/{id}

Claim + evidence graph.

14. Minimal recommendation record

task_id: task-...
evaluated_at: 2026-09-13T...
policy:
  task_profile: v0.1
  benchmark_selection: v0.1
  model_ranking: v0.1
clarity:
  score: 0.78
  ambiguities: []
benchmarks:
  - id: ...
    weight: 0.27
frontier:
  execution_profile: ...
topnotch: [...]
efficient: ...
sufficient: [...]
budget: ...
economics:
  currency: USD
  expected: ...
  p90: ...
runtime:
  expected_seconds: ...
confidence:
  overall: ...
  components: ...
evidence_gaps: [...]

15. Initial milestones

M0 — Repository kernel

Documents, schemas, policies, source catalog.

M1 — Dated model/economics catalog

Automated collection of model releases, availability, context limits, and API prices from major providers.

M2 — Benchmark health catalog

Benchmark metadata, versions, results, saturation and contamination notes.

M3 — Task profiler

Produces stable capability profiles and clarity assessment.

M4 — Five-benchmark selector

Produces explainable benchmark mapping with independence penalties.

M5 — Model ranking

Frontier / Topnotch / Efficient / Sufficient / Budget with cost intervals.

M6 — Workplan optimizer

DAG decomposition, heterogeneous model allocation, verification, parallel runtime estimate.

M7 — Outcome ledger

Capture actual task success, cost, runtime, and failure modes.

M8 — Calibration engine

Learn task-cluster performance models from the outcome ledger.

M9 — Self-improving Radar

Detect benchmark drift/saturation and automatically propose policy/catalog changes for review.