freedom-intelligence/research/2026-07-24-baseline-field-survey.md

Baseline field survey — Freedom Intelligence starting point

Date: 2026-07-24
Type: deep baseline (not a daily delta brief)
Purpose: Establish longitudinal memory so the first daily research brief can be a true delta, not a full survey. Also recommend the initial open-weight collection for the lab reserve.
Axes: A frontier · B edge/local · C training · D harness/fleet
Sources: public web surveys, HF/model docs, pricing roundups, harness project sites (mid–late 2026 public reporting). Prices and leaderboard claims move weekly — treat numbers as order-of-magnitude and re-check on first daily.

---

1. Executive snapshot

Theme	State of the field (mid-2026)	Lab implication
Frontier	Multi-vendor race: OpenAI GPT-5.x, Anthropic Claude 4.x/Opus, Google Gemini 3.x, xAI Grok 4.x; coding agents dominate product narrative	Keep API access via llm-connect; do not try to host frontier closed weights
Price collapse	DeepSeek-class APIs often 10–100× cheaper than flagship closed models for many tasks	Route cheap/easy work to low-cost APIs; reserve local for privacy, offline, specialization
Open weights	Strong mid-size dense + MoE story (Qwen3, DeepSeek distill/V3-class, Gemma/Llama small, Mistral small)	Homelab can be useful without 8×H100
Local stack	llama.cpp / vLLM / MLX / Ollama mature; Q4–Q5 quants standard	Axis B is operationally ready
Training	QLoRA/Unsloth/TRL still the homelab path; full pretrain not lab-realistic	Axis C = adapters + domain SFT, not from-scratch
Harness	OpenHands, Aider, SWE-agent, OpenCode, Cline; closed CLIs (Claude Code, Codex) lead quality	Axis D: watch + integrate with sand-boxer / agent-harness, don’t fork everything

Central proposition for the reserve: collect a compact spine of (1) small general instruct, (2) mid-size code/reason distill, (3) multilingual embed, (4) optional larger “lab ceiling” base — not a mirror of Hugging Face.

---

2. Axis A — Frontier capabilities, benchmarks, price

2.1 Capability landscape

Public 2026 reporting groups the closed frontier into:

• Flagship general / agentic — OpenAI GPT-5.x family, Anthropic Claude Opus/Sonnet 4.x+, Google Gemini 3.x Pro
• Fast / cheap tiers — GPT mini/nano, Claude Haiku, Gemini Flash
• Reasoning-heavy — “thinking” / extended variants; cost dominated by output tokens
• Coding productization — agent harnesses matter as much as base model (SWE-bench Verified numbers often quoted 70%+ with strong closed models + good harness; novel-issue sets much lower ~18–20%)

Benchmark caveats for the lab:

• Arena and public leaderboards are gamed and lag product quality
• SWE-bench Verified ≠ “our monorepos + NetKingdom tools”
• Prefer internal harness scores once we have them; use public numbers only as directional

2.2 Price-point trends (API, ~per 1M tokens)

Synthesized from 2026 pricing roundups (verify on provider pages before budget decisions):

Band	Examples (reported ranges)	Role
Ultra-cheap	DeepSeek V3.x / V4 Flash-class (~$0.14–0.28 in / ~$0.28–0.42 out; cache hits lower)	Bulk classify, draft, simple tools
Budget closed	Gemini Flash, GPT mini/nano, Haiku-class	Default automation when quality bar is modest
Mid flagship	Claude Sonnet-class, GPT mid-tier (~$2–5 in / ~$10–15 out)	Daily agent coding, serious writing
Top flagship	Opus / GPT pro tiers (~$5+ in / $25–30+ out; pro tiers much higher)	Hard reasoning, high-stakes review

Trend: continuous compression of mid-tier quality cost; flagship output tokens remain expensive. Caching and batch APIs swing real cost 50%+.

Lab stance:

• Frontier APIs stay first-class for hard work (via llm-connect / OpenRouter-class)
• Daily research about price moves is high value for routing policy
• Local models win on privacy, offline, predictable unit cost, specialization — not always raw SOTA

2.3 Open near-frontier (API or weights)

DeepSeek open-weight + cheap API continues to be the structural disruptor: MIT-leaning releases for R1/V3-class artifacts, distill series for single-GPU reasoning. Track V3/V4 Flash vs Pro naming carefully — product lines rename often.

---

3. Axis B — Edge, local, open compute

3.1 What “homelab useful” means now

Class	Params (ballpark)	Typical local use
Edge / CPU-OK	≤3B Q4	Chat toys, classification, always-on small agents
Consumer GPU	7–14B Q4/Q5	Daily coding assist, RAG chat, tool loops
Single enthusiast GPU	32B Q4	Stronger reason/code; slower
Lab multi-GPU	70B Q4 or sparse MoE	Optional ceiling; not required for v1 reserve

Runtimes to track: llama.cpp, vLLM, MLX (Apple), Ollama (UX), TensorRT-LLM (NVIDIA optimize). Speculative decoding + good quant matter more than chasing every new 70B.

3.2 Open model families to watch (weights)

Family	Why it matters	License caution
Qwen3 (Alibaba)	Dense + MoE line; strong multilingual & tool use; embeddings/VL variants	Apache-class for many; check model card
DeepSeek (V3/R1 + distills)	Reasoning + cost; MIT for key R1/distill artifacts	Confirm per-revision card
Llama 3.x / 4 text	Ecosystem, tooling, small 1B–3B instruct	Llama community license (not pure MIT)
Gemma 3	Google small/efficient; EmbeddingGemma	Gemma terms
Mistral Small / Ministral	EU-friendly commercial open options	Apache / research licenses vary
Phi / Smol class	Edge CPU experiments	Check commercial terms

3.3 Embeddings & retrieval (local RAG spine)

For NetKingdom docs, code, and ops notes:

Model	Role	Notes
BGE-M3 (BAAI)	Default multilingual dense (+ multi-granularity)	MIT; ~0.5B; self-host staple
Nomic Embed text v1/v2	Strong English/multilingual text	Apache
Nomic Embed Code / code-specialized	Code retrieval	Larger; optional P1
EmbeddingGemma-300M	Tiny multilingual	Edge RAG
Qwen3 embedding / VL-small	Multimodal retrieval experiments	Watch cards

Rerankers (BGE reranker, mini cross-encoders) are high leverage and small — good companions once embed base is reserved.

---

4. Axis C — Homelab training & domain specialization

4.1 Realistic lab methods

Method	Homelab fit	Use
QLoRA / LoRA (Unsloth, PEFT, TRL)	Excellent	Domain assistants (security, ops, railiance)
Preference (DPO/ORPO/KTO)	Good on 7–14B	Style and safety for agents
Continued pretrain	Possible on small bases	Domain language (logs, policies)
Full SFT of 70B+	Poor without multi-GPU budget	Prefer distill/adapter on smaller
From-scratch pretrain	Out of scope	—

4.2 Bases worth specializing

1. 7–8B instruct (Qwen3-8B or Llama-3.1-8B-Instruct class) — default FT target
2. 14B — quality step if VRAM allows
3. DeepSeek-R1-Distill 14B/32B — reason-heavy agents after FT discipline
4. Code-specialized mid — if code assistants dominate workload

Data: prefer synthetic + curated NetKingdom docs over scraping; keep PII out of training sets (separate policy later).

4.3 Tooling to track (not collect as weights)

Unsloth, Axolotl, TRL, llama-factory, mergekit, preference datasets (UltraFeedback-class), eval harnesses (lm-eval, custom domain suites).

---

5. Axis D — Agent harnesses, orchestration, sandboxing, fleet

5.1 Open harness landscape (2026)

Project	Shape	Why watch
OpenHands	Full agent platform + Canvas; MIT; multi-backend	Leading open coding-agent platform; multi-agent
Aider	Git-native CLI pair programmer	Atomic commits; simple; great for ops scripts
SWE-agent / mini-SWE-agent	Research harness, clean ACI	Benchmark + security CTF modes
OpenCode	Open Claude Code alternative	High interest community
Cline	IDE agent	Editor-centric workflows
Closed CLIs	Claude Code, Codex, Gemini CLI	Quality bar for comparison

SWE-bench: harness + model co-determine scores; Mini-SWE-Agent claims strong results in very little code — architecture lessons for Coulomb harnesses.

5.2 NetKingdom adjacency

Coulomb piece	Relation to D
sand-boxer	Where agents run safely
activity-core	When work is due (this lab’s daily brief)
agent-harness / kaizen	Who executes with profiles
state-hub	What changed; task spawn completion
llm-connect	Which model backend

Freedom Intelligence reports on harness tech; it does not replace sand-boxer.

5.3 Sandboxing & fleet themes to track

• Default-deny network + TTL sandboxes
• Credential routing (OpenBao / ops-warden) separate from model weights
• Multi-agent kanban / workplan-tied loops (Ralph-workplan class)
• Cost and rate-limit fleet management for API models

---

6. Collection recommendations (open-weight reserve)

6.1 Decision framework (recap)

• Prefer B/C leverage + clear license + stable org provenance
• Size gates per inventory/collection-policy.md
• Backup path must be pinned (FI-WP-0001-T04) before large pulls
• Quants: prefer official or well-known GGUF/AWQ only after base id is cataloged; v1 recommends safetensors official first, quant as companion later

6.2 P0 — seed now (approve after storage pin)

#	Model (representative HF id)	Est. size class	Axes	Why
1	Qwen/Qwen3-8B (or current 8B-Instruct sibling on card)	~16 GB fp16 / ~5 GB Q4	B, C	Best default general + FT base; multilingual; tool-friendly
2	meta-llama/Llama-3.2-3B-Instruct	~6 GB fp16 / ~2 GB Q4	B	Tiny always-on / edge; huge ecosystem
3	BAAI/bge-m3	~2 GB	B	Multilingual RAG staple; MIT
4	deepseek-ai/DeepSeek-R1-Distill-Qwen-14B (or 8B if VRAM tight)	~28 GB fp16 / ~9 GB Q4	B, C	Local reasoning without full R1; MIT distill story
5	nomic-ai/nomic-embed-text-v1.5 (or current v2 text)	<1 GB	B	Lightweight embed alternative / A/B with BGE

Rough P0 cold storage (fp16 worst case): ~50–60 GB.
With Q4 preference for LLMs + fp for embeds: often ~20–30 GB. Prefer quants only when catalog notes the exact quant provenance.

6.3 P1 — next wave (quota permitting)

#	Model	Why
6	Qwen/Qwen3-14B (instruct)	Quality step for single-GPU chat/code
7	google/gemma-3-4b-it or EmbeddingGemma-300M	Efficient Google line; embed edge
8	mistralai/Mistral-Small-3.x instruct (current card)	EU-adjacent open option
9	deepseek-ai/DeepSeek-R1-Distill-Qwen-32B	Stronger local reason if ≥24 GB VRAM class
10	Code embed: nomic-ai/nomic-embed-code or Jina code embed	Code RAG for repos
11	Small reranker: BAAI/bge-reranker-v2-m3	Cheap quality win for RAG

6.4 Strategic large (updated 2026-07-24)

Superseded by operator decision: 1 TB local NAS + capability-first reserve. Full DeepSeek-V3/R1-class and 70B dense opens are in scope even if unrunnable on current GPUs. See:

• docs/decisions/2026-07-24-nas-strategic-reserve.md
• research/2026-07-24-nas-strategic-collection-plan.md

Item	Guidance
Full DeepSeek-V3 / R1 class	Strategic (S) — collect under 850 GiB soft quota; prefer official compressed if needed
Llama 70B / Qwen 72B class	Strategic (S) dense — good second fill after one primary MoE
Every new quant repack on HF	Skip; track base official only

6.5 Explicit non-recommendations (for now)

• Closed API-only weights
• Anonymous “uncensored” reuploads without provenance
• Duplicate quants of the same revision
• Giant pretraining corpora as “models”

6.6 Catalog seed

YAML candidates for P0–P1 are under inventory/catalog/ with status: candidate and brief_refs pointing at this survey. Approval and download are FI-WP-0003.

---

7. Implications for the first daily brief

The first briefs/YYYY/MM/YYYY-MM-DD.md after this survey should:

1. Not restate this whole document
2. Cover only deltas since 2026-07-24 (new releases, price changes, harness versions)
3. Revisit P0/P1 if a new open model clearly displaces a candidate
4. Note any license or access shocks (takedowns, geo blocks)
5. Optionally mark FI-WP-0001-T04 / FI-WP-0003 progress under Lab implications

Automation: activity-core def fi-daily-research-brief (disabled until FI-WP-0002 resolver + dry-run). Manual playbook: docs/daily-brief-playbook.md.

---

8. Open questions for later briefs

1. Exact hardware envelope measured VRAM on lab hosts?
2. Prefer GGUF Q5_K_M vs official safetensors for P0?
3. Is German-first multilingual quality a hard requirement (affects embed + instruct choice)?
4. Will local models serve air-gapped NetKingdom scenarios or only cost/privacy convenience?
5. Single code model vs general instruct for domain FT?

---

9. Source anchors (non-exhaustive)

• DeepSeek R1 release / MIT notes — DeepSeek API docs & HF model cards
• Pricing roundups 2026 — IntuitionLabs, MorphLLM, provider pricing pages
• Embeddings 2026 comparisons — BentoML, Milvus/Zilliz, BGE-M3 cards
• Harness landscape — OpenHands, Aider, SWE-agent comparisons (LocalAI Master, vendor blogs)
• Open-weight commercial lists — community curations (verify each card)

Re-verify any number before spend or multi-hundred-GB download.

---

Baseline established 2026-07-24 for Freedom Intelligence. Daily briefs begin as deltas from here.