Establish Freedom Intelligence lab foundation and baseline research.
Add INTENT/SCOPE, daily-brief playbook, activity-core definition (disabled), workplans FI-WP-0001..0003, baseline field survey with open-weight collection recommendations, and inventory catalog candidates for the model reserve.
This commit is contained in:
parent
1dc1e09517
commit
b5f911140b
28 changed files with 2651 additions and 1 deletions
119
INTENT.md
Normal file
119
INTENT.md
Normal file
|
|
@ -0,0 +1,119 @@
|
|||
# INTENT
|
||||
|
||||
> This file captures **why this repository exists**,
|
||||
> the **direction it is moving toward**, and
|
||||
> the **kind of system it is meant to become**.
|
||||
> It is intentionally **aspirational and stable**, not a description of current implementation.
|
||||
>
|
||||
> Founding record: `history/InitialExploration.md`
|
||||
|
||||
---
|
||||
|
||||
## One-liner
|
||||
|
||||
**The just-in-case open-weights model lab** — Coulomb/NetKingdom sensing and reserve for AI capability, cost, and operable local intelligence.
|
||||
|
||||
---
|
||||
|
||||
## Why This Exists
|
||||
|
||||
AI capability is moving faster than any single product roadmap. Frontier models, open weights, training recipes, and agent harnesses shift monthly. Most organizations either:
|
||||
|
||||
* chase every vendor announcement without durable memory, or
|
||||
* lock into a single cloud API and lose optionality when prices, licenses, or access change.
|
||||
|
||||
Coulomb NetKingdom needs a third posture:
|
||||
|
||||
1. **Sense** what actually moves capability, cost, accessibility, and operability.
|
||||
2. **Reserve** open-weight artifacts that may matter later for local training or operations.
|
||||
3. **Bias** toward intelligence that can be inspected, specialized, and run under our control.
|
||||
|
||||
Freedom Intelligence is that lab — not a chat product, not a public model CDN, and not a commitment to host every frontier model.
|
||||
|
||||
---
|
||||
|
||||
## Mission
|
||||
|
||||
> *Where we are going.*
|
||||
|
||||
Become the Coulomb NetKingdom **intelligence sensing loop and open-weight strategic reserve**:
|
||||
|
||||
* a **daily research brief** that filters noise into lab-relevant signal across four standing axes;
|
||||
* a **deliberate collection** of open-weight models (and justified companions) on backup-class storage;
|
||||
* durable in-repo memory (briefs, inventory metadata, research notes) that outlives any single session or vendor.
|
||||
|
||||
---
|
||||
|
||||
## Standing axes (the permanent lens)
|
||||
|
||||
| Axis | Focus |
|
||||
| ---- | ----- |
|
||||
| **A — Frontier** | Best-of-class capabilities, benchmarks, and price-point trends |
|
||||
| **B — Edge / local / open** | Low-cost open and local compute intelligence and specializations |
|
||||
| **C — Homelab training** | Pre-, mid-, post-, and domain specialization recipes that fit lab hardware |
|
||||
| **D — Harness & fleet** | Agent harnesses, orchestration, sandboxing, and fleet management |
|
||||
|
||||
These axes define *what we watch*, not a promise to implement every technology we report on.
|
||||
|
||||
---
|
||||
|
||||
## Dual loops
|
||||
|
||||
```text
|
||||
Daily research brief ──► Sensing loop (awareness, prioritization)
|
||||
Open-weight reserve ──► Strategic reserve (optionality for train / run)
|
||||
```
|
||||
|
||||
The brief feeds collection candidates. Collection never mirrors the entire Hub — only high-leverage, clearly licensed, hard-to-reobtain artifacts.
|
||||
|
||||
---
|
||||
|
||||
## Design principles
|
||||
|
||||
* **Signal over noise** — report what changes capability, cost, access, or operability.
|
||||
* **Just in case** — hold optionality without obligation to serve every model.
|
||||
* **Catalog in git, blobs on backup storage** — metadata is versioned; weights are bulk media.
|
||||
* **License and integrity first** — no collection that terms forbid; checksums and provenance required.
|
||||
* **Homelab honesty** — prefer methods and models that fit our real hardware envelope.
|
||||
* **Operable intelligence** — models without harness, sandbox, and fleet thinking are incomplete for NetKingdom.
|
||||
* **Inspectable and open-leaning** — bias toward stacks we can audit and specialize.
|
||||
|
||||
---
|
||||
|
||||
## Strategic role in Coulomb / NetKingdom
|
||||
|
||||
| Freedom Intelligence supplies | Adjacent systems own |
|
||||
| ----------------------------- | -------------------- |
|
||||
| Field sensing and briefs | State Hub distribution / inbox (optional consumers) |
|
||||
| Open-weight reserve + inventory | Backup facilities, storage ops, disaster-control policy |
|
||||
| Guidance on what can run locally | sand-boxer, agent harnesses, local runtimes, llm-connect |
|
||||
| Domain-specialization *candidates* | Domain repos that consume specialized models later |
|
||||
|
||||
It does **not** replace NetKingdom security, identity, ops, or application products.
|
||||
|
||||
---
|
||||
|
||||
## Intended users
|
||||
|
||||
* Operators and agents maintaining Coulomb/NetKingdom infrastructure who need a trusted daily AI field picture
|
||||
* People deciding when to use frontier APIs vs local open weights
|
||||
* Future local training / inference facilities that need a curated reserve and inventory
|
||||
|
||||
---
|
||||
|
||||
## Maturity target
|
||||
|
||||
A mature Freedom Intelligence lab should:
|
||||
|
||||
* produce a reliable **daily brief** with stable structure and cited sources;
|
||||
* maintain a **living inventory** of reserved model revisions with integrity metadata;
|
||||
* enforce a **collection policy** (approval thresholds, license gate, capacity budget);
|
||||
* know its **hardware envelope** for “homelab accessible”;
|
||||
* optionally distribute briefs into operator workflows (inbox / State Hub) without becoming a general news product;
|
||||
* feed domain specialization work without owning every consumer pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Stability note
|
||||
|
||||
Changes to this file represent a **deliberate shift** in why the lab exists or what dual loops it owns. Such changes should be rare. Near-term deliverables and boundaries live in `SCOPE.md`.
|
||||
81
README.md
81
README.md
|
|
@ -1,3 +1,82 @@
|
|||
# freedom-intelligence
|
||||
|
||||
The just in case open weights model lab
|
||||
**The just in case open weights model lab**
|
||||
|
||||
Coulomb / NetKingdom lab for:
|
||||
|
||||
1. **Daily research briefs** on AI models, methods, harness, and tooling
|
||||
2. An **open-weight model reserve** on backup storage for possible local training or ops
|
||||
|
||||
Standing lenses: **A** frontier & price · **B** edge/local open · **C** homelab training · **D** agent harness & fleet
|
||||
|
||||
---
|
||||
|
||||
## Start here
|
||||
|
||||
| Doc | Role |
|
||||
| --- | ---- |
|
||||
| [`INTENT.md`](INTENT.md) | Why the lab exists (stable) |
|
||||
| [`SCOPE.md`](SCOPE.md) | What is in/out of scope now |
|
||||
| [`research/2026-07-24-baseline-field-survey.md`](research/2026-07-24-baseline-field-survey.md) | **Baseline field survey + model collection recommendations** |
|
||||
| [`docs/daily-brief-playbook.md`](docs/daily-brief-playbook.md) | How to produce a daily brief |
|
||||
| [`history/InitialExploration.md`](history/InitialExploration.md) | Founding conversation |
|
||||
| [`briefs/`](briefs/) | Daily briefs + template |
|
||||
| [`inventory/`](inventory/) | Model catalog & collection policy |
|
||||
| [`workplans/`](workplans/) | Setup workplans (incl. activity-core) |
|
||||
| [`activity-definitions/`](activity-definitions/) | Cron definition for daily brief (disabled until FI-WP-0002) |
|
||||
|
||||
---
|
||||
|
||||
## Workplans
|
||||
|
||||
| ID | Title | Status |
|
||||
| -- | ----- | ------ |
|
||||
| [FI-WP-0001](workplans/FI-WP-0001-lab-operating-foundation.md) | Lab operating foundation | active (storage pin open) |
|
||||
| [FI-WP-0002](workplans/FI-WP-0002-activity-core-daily-research.md) | Activity-core daily research rhythm | active (resolver/ops open) |
|
||||
| [FI-WP-0003](workplans/FI-WP-0003-seed-open-weight-reserve.md) | Seed open-weight reserve | active (catalog seeded) |
|
||||
|
||||
---
|
||||
|
||||
## Collection spine (from baseline)
|
||||
|
||||
**P0 candidates** (approve after backup path pin): Qwen3-8B, Llama-3.2-3B-Instruct,
|
||||
BGE-M3, DeepSeek-R1-Distill-Qwen-14B, nomic-embed-text-v1.5
|
||||
|
||||
Details and P1/P2: see baseline survey §6 and `inventory/catalog/`.
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
freedom-intelligence/
|
||||
├── INTENT.md / SCOPE.md / README.md
|
||||
├── activity-definitions/ # fi-daily-research-brief (enabled: false)
|
||||
├── briefs/ # daily deltas
|
||||
├── research/ # deep surveys (baseline lives here)
|
||||
├── inventory/catalog/ # YAML candidates / collected
|
||||
├── docs/ # playbook, sources, storage, hardware
|
||||
├── workplans/ # FI-WP-*
|
||||
└── history/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick ops
|
||||
|
||||
**New brief**
|
||||
|
||||
```bash
|
||||
mkdir -p briefs/$(date +%Y/%m)
|
||||
cp briefs/_template.md briefs/$(date +%Y/%m)/$(date +%Y-%m-%d).md
|
||||
# follow docs/daily-brief-playbook.md
|
||||
```
|
||||
|
||||
**Do not** bulk-download models until `docs/backup-storage-policy.md` path/quota
|
||||
are pinned (FI-WP-0001-T04).
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT — see [`LICENSE`](LICENSE). Model weights retain **upstream** licenses.
|
||||
|
|
|
|||
143
SCOPE.md
Normal file
143
SCOPE.md
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
# SCOPE
|
||||
|
||||
> This file helps you quickly understand what this repository is about,
|
||||
> when it is relevant, and when it is not.
|
||||
> It is intentionally lightweight and may be incomplete.
|
||||
>
|
||||
> Aspirational charter: `INTENT.md` · Founding record: `history/InitialExploration.md`
|
||||
|
||||
---
|
||||
|
||||
## One-liner
|
||||
|
||||
Daily AI research briefs (models, methods, harness, tooling) plus an open-weight model reserve on lab backup storage — Coulomb/NetKingdom intelligence lab.
|
||||
|
||||
---
|
||||
|
||||
## Core Idea
|
||||
|
||||
Freedom Intelligence is a **sensing and reserve** repository:
|
||||
|
||||
1. Curated **daily research briefs** across axes A–D (frontier, edge/local, training, harness/fleet).
|
||||
2. **Inventory + collection policy** for open-weight models retained on backup-class storage for possible later local training or operations.
|
||||
|
||||
Git holds briefs, schemas, policies, and catalog metadata. Model weight blobs stay off hot cluster disks on designated backup storage.
|
||||
|
||||
---
|
||||
|
||||
## In Scope
|
||||
|
||||
### Research brief
|
||||
|
||||
- Daily brief production using `briefs/_template.md`
|
||||
- Layout under `briefs/YYYY/MM/YYYY-MM-DD.md`
|
||||
- Standing coverage of axes A–D when there is signal
|
||||
- Collection-candidate flagging for the model reserve
|
||||
- Source citation and durable archive of briefs in-repo
|
||||
|
||||
### Model reserve
|
||||
|
||||
- Inventory schema and catalog entries (`inventory/`)
|
||||
- Collection approval policy and license gate
|
||||
- Backup storage target and capacity policy (`docs/backup-storage-policy.md`)
|
||||
- Metadata required to rehydrate a revision (source, hash, license, size, hardware class)
|
||||
- Companion artifacts when justified (tokenizers, adapters, small eval fixtures)
|
||||
|
||||
### Lab documentation
|
||||
|
||||
- INTENT / SCOPE / history
|
||||
- Optional delivery notes for State Hub / operator inbox (`docs/state-hub-delivery.md`)
|
||||
- Hardware envelope notes as they become known
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope
|
||||
|
||||
| Concern | Belongs elsewhere |
|
||||
| ------- | ----------------- |
|
||||
| Public model mirror / CDN | Not this lab |
|
||||
| 24/7 inference serving of the full reserve | Future ops facility / runtime repos |
|
||||
| Frontier-scale pre-training from scratch | Hyperscale; not homelab commitment |
|
||||
| Closed-weight collection or ToS-violating scrapes | Forbidden |
|
||||
| Provider-neutral LLM client library | `llm-connect` |
|
||||
| Console assistant UX | `can-you-assist` |
|
||||
| Sandbox establishment service | `sand-boxer` |
|
||||
| Platform identity / secrets / bootstrap | `net-kingdom`, `key-cape` |
|
||||
| General disaster backup governance | `disaster-control` |
|
||||
| Domain application fine-tune pipelines (end-to-end) | Domain repos; this lab only reserves bases and tracks methods |
|
||||
|
||||
---
|
||||
|
||||
## Relevant When
|
||||
|
||||
- You need the daily or historical AI field picture for Coulomb/NetKingdom decisions
|
||||
- You are evaluating frontier vs open/local options (capability, price, license)
|
||||
- You want to know whether an open-weight model is (or should be) in the lab reserve
|
||||
- You are planning homelab-scale specialization and need method / model candidates
|
||||
- You are designing agent harness or fleet work and want external signal on that axis
|
||||
|
||||
---
|
||||
|
||||
## Not Relevant When
|
||||
|
||||
- You need an LLM API client (use `llm-connect`)
|
||||
- You need to provision agent sandboxes (use `sand-boxer`)
|
||||
- You need platform IAM or secrets (use NetKingdom / KeyCape)
|
||||
- You only need one-off news without archival brief or inventory discipline
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
| Area | Status |
|
||||
| ---- | ------ |
|
||||
| Overall | **Foundation + baseline research** (2026-07-24) |
|
||||
| INTENT / SCOPE | Drafted |
|
||||
| Baseline survey | `research/2026-07-24-baseline-field-survey.md` |
|
||||
| Brief template + playbook | Present; first *daily* delta brief not yet written |
|
||||
| Backup storage policy | Documented; **path/quota TBD** (FI-WP-0001-T04) |
|
||||
| Inventory | Schema + policy + **P0/P1 catalog candidates** seeded |
|
||||
| Activity-core | Definition drafted `enabled: false` (FI-WP-0002) |
|
||||
| State Hub | `fi_daily_brief` contract documented; resolver not implemented |
|
||||
| Workplans | FI-WP-0001 … 0003 active |
|
||||
|
||||
---
|
||||
|
||||
## How It Fits
|
||||
|
||||
| Direction | Systems |
|
||||
| --------- | ------- |
|
||||
| **Upstream** | Public model hubs, papers, pricing pages, release notes, harness projects |
|
||||
| **Peer** | `llm-connect`, `sand-boxer`, `net-kingdom`, `disaster-control`, domain repos |
|
||||
| **Downstream (future)** | Local inference/training facility, specialized domain agents, operator decision-making |
|
||||
| **Distribution (optional)** | State Hub progress/inbox, operator mail/wiki |
|
||||
|
||||
---
|
||||
|
||||
## Near-term deliverables
|
||||
|
||||
1. ✅ `INTENT.md` / `SCOPE.md` / foundation docs
|
||||
2. ✅ Brief template + playbook + sources allowlist
|
||||
3. ✅ Baseline field survey + collection recommendations
|
||||
4. ✅ Catalog candidates (P0/P1)
|
||||
5. ✅ ActivityDefinition draft (disabled)
|
||||
6. ⬜ Pin backup storage path and capacity budget (FI-WP-0001-T04)
|
||||
7. ⬜ activity-core resolver + enable schedule (FI-WP-0002)
|
||||
8. ⬜ Approve/download P0 reserve (FI-WP-0003)
|
||||
9. ⬜ First daily delta brief (manual or after automation)
|
||||
|
||||
---
|
||||
|
||||
## Terminology
|
||||
|
||||
| Term | Meaning |
|
||||
| ---- | ------- |
|
||||
| **Brief** | Dated research digest under `briefs/` |
|
||||
| **Axis A–D** | Standing monitoring lenses (frontier, edge/local, training, harness) |
|
||||
| **Reserve** | Open-weight collection on backup storage |
|
||||
| **Catalog entry** | In-repo inventory metadata for one model revision |
|
||||
| **Collection candidate** | Brief-flagged artifact not yet approved/collected |
|
||||
| **Hardware envelope** | Lab hosts that define “homelab accessible” for B/C |
|
||||
|
||||
Preferred terms: research brief, open-weight reserve, inventory, collection policy.
|
||||
Avoid: “mirror of Hugging Face”, “our AI product”, “always-on model zoo”.
|
||||
61
activity-definitions/fi-daily-research-brief.md
Normal file
61
activity-definitions/fi-daily-research-brief.md
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
---
|
||||
id: fi-daily-research-brief
|
||||
name: Freedom Intelligence Daily Research Brief
|
||||
enabled: false
|
||||
owner: custodian-agent
|
||||
governance: custodian
|
||||
status: proposed
|
||||
trigger:
|
||||
type: cron
|
||||
cron_expression: "30 7 * * 1-5" # weekdays 07:30
|
||||
timezone: Europe/Berlin
|
||||
misfire_policy: skip
|
||||
context_sources:
|
||||
- type: state-hub
|
||||
query: fi_brief_status
|
||||
params:
|
||||
repo: freedom-intelligence
|
||||
bind_to: context.fi_brief
|
||||
# Resolver (FI-WP-0002-T03, activity-core): expected shape
|
||||
# {items: [{kind: "daily_brief", due: bool, date: "YYYY-MM-DD",
|
||||
# last_run_at}]} — due is true when no fi_daily_brief progress event with
|
||||
# detail.repo=freedom-intelligence exists for today (Europe/Berlin).
|
||||
# Executing session must record that event on completion (idempotence).
|
||||
---
|
||||
|
||||
# Freedom Intelligence Daily Research Brief
|
||||
|
||||
Runs weekdays at 07:30 Berlin time. Emits one task when the daily research brief
|
||||
for `freedom-intelligence` is due. Execution is **out of band**: consumer checks
|
||||
out this repo and follows `docs/daily-brief-playbook.md` (axes A–D, collection
|
||||
candidates, template under `briefs/`).
|
||||
|
||||
activity-core only schedules and spawns. It does not call model APIs for the
|
||||
brief body.
|
||||
|
||||
Baseline field memory: `research/2026-07-24-baseline-field-survey.md`. Daily
|
||||
briefs are **deltas** from prior briefs + that baseline — not full resurveys.
|
||||
|
||||
```rule
|
||||
id: emit-fi-daily-brief-task
|
||||
for_each: context.fi_brief.items
|
||||
bind_as: item
|
||||
condition: 'context.item.due'
|
||||
action:
|
||||
task_template: "FI daily research brief ({context.item.kind}) for {context.item.date}"
|
||||
description: >
|
||||
Produce briefs/YYYY/MM/YYYY-MM-DD.md per docs/daily-brief-playbook.md and
|
||||
briefs/_template.md. Cite primary sources. Flag collection candidates.
|
||||
On completion post State Hub progress event_type=fi_daily_brief with
|
||||
detail.repo=freedom-intelligence and detail.date.
|
||||
target_repo: freedom-intelligence
|
||||
priority: medium
|
||||
labels: ["freedom-intelligence", "research-brief", "automated"]
|
||||
```
|
||||
|
||||
## Enable checklist
|
||||
|
||||
1. FI-WP-0002-T03 resolver live
|
||||
2. FI-WP-0002-T04 external definition path registered
|
||||
3. Dry-run spawn + one completed brief with `fi_daily_brief` event
|
||||
4. Set `enabled: true` and `status: active` in this frontmatter
|
||||
63
briefs/README.md
Normal file
63
briefs/README.md
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
# Research briefs
|
||||
|
||||
Daily curated digests of interesting progress in AI **models**, **methods**,
|
||||
**harness**, and **tooling**, filtered through axes A–D.
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
briefs/
|
||||
├── README.md # this file
|
||||
├── _template.md # copy for each new day
|
||||
└── YYYY/
|
||||
└── MM/
|
||||
└── YYYY-MM-DD.md # one brief per calendar day (UTC or lab local — state in front matter)
|
||||
```
|
||||
|
||||
Example path: `briefs/2026/07/2026-07-23.md`
|
||||
|
||||
## Cadence
|
||||
|
||||
| Mode | When |
|
||||
| ---- | ---- |
|
||||
| **Daily** | Preferred operating rhythm when the lab is active |
|
||||
| **Skip empty days** | If there is no meaningful signal, either omit the file or write a short “no material delta” brief |
|
||||
| **Weekly rollup** | Optional later; not required for foundation |
|
||||
|
||||
First operating mode is **manual** (human or agent drafts from the template). Automation is out of scope until the template is validated by at least one real brief.
|
||||
|
||||
## How to write a brief
|
||||
|
||||
1. Copy `briefs/_template.md` to `briefs/YYYY/MM/YYYY-MM-DD.md`.
|
||||
2. Fill only sections with signal; leave others as `*(none)*` rather than inventing news.
|
||||
3. Cite primary sources (papers, model cards, release notes, repos, pricing pages).
|
||||
4. Flag **collection candidates** with enough metadata for inventory review.
|
||||
5. Keep **lab implications** short and actionable (or omit).
|
||||
|
||||
## Design principles
|
||||
|
||||
See `INTENT.md` and `history/InitialExploration.md`:
|
||||
|
||||
- Signal over noise
|
||||
- Actionable for the lab
|
||||
- Price and access aware
|
||||
- Reproducible sources
|
||||
- Just-in-case bias for rare/high-leverage open weights
|
||||
|
||||
## Baseline vs daily
|
||||
|
||||
The **baseline field survey** is not a daily brief:
|
||||
|
||||
- `research/2026-07-24-baseline-field-survey.md`
|
||||
|
||||
Daily files under this tree are **deltas** from that baseline and prior dailies.
|
||||
Playbook: `docs/daily-brief-playbook.md`. Sources: `docs/sources-allowlist.md`.
|
||||
|
||||
## Axes reminder
|
||||
|
||||
| Axis | Section in template |
|
||||
| ---- | ------------------- |
|
||||
| A | Frontier & commercial |
|
||||
| B | Edge / local / open |
|
||||
| C | Training & specialization |
|
||||
| D | Harness & fleet |
|
||||
78
briefs/_template.md
Normal file
78
briefs/_template.md
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
---
|
||||
date: YYYY-MM-DD
|
||||
timezone: Europe/Berlin # or UTC — pick one and keep consistent
|
||||
author: # human or agent name
|
||||
status: draft # draft | final
|
||||
sources_checked: [] # optional list of channels scanned
|
||||
---
|
||||
|
||||
# Freedom Intelligence Daily Brief — YYYY-MM-DD
|
||||
|
||||
## Headline deltas
|
||||
|
||||
What actually moved the needle today (3–7 bullets max). If nothing material: `*No material delta.*`
|
||||
|
||||
-
|
||||
|
||||
## Frontier & commercial (axis A)
|
||||
|
||||
Capabilities, benchmarks, price moves, API access changes.
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| | | | |
|
||||
|
||||
*(none)*
|
||||
|
||||
## Edge / local / open (axis B)
|
||||
|
||||
Runnable intelligence at low cost; quantization; inference runtimes; licenses for local ops.
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| | | | |
|
||||
|
||||
*(none)*
|
||||
|
||||
## Training & specialization (axis C)
|
||||
|
||||
Pre-/mid-/post-training methods, adapters, datasets, homelab-fit recipes, domain eval.
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| | | | |
|
||||
|
||||
*(none)*
|
||||
|
||||
## Harness & fleet (axis D)
|
||||
|
||||
Agent frameworks, orchestration, sandboxing, multi-agent ops, fleet management.
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| | | | |
|
||||
|
||||
*(none)*
|
||||
|
||||
## Collection candidates
|
||||
|
||||
Open-weight models (or justified companions) worth parking on backup storage.
|
||||
Use inventory field names where possible — see `inventory/schema.yaml`.
|
||||
|
||||
| Candidate | Source / revision | License | Est. size | Why reserve | Priority |
|
||||
| --------- | ----------------- | ------- | --------- | ----------- | -------- |
|
||||
| | | | | | low / med / high |
|
||||
|
||||
*(none)*
|
||||
|
||||
## Lab implications
|
||||
|
||||
Optional: concrete next actions for Freedom Intelligence or NetKingdom consumers.
|
||||
|
||||
-
|
||||
|
||||
## Notes / caveats
|
||||
|
||||
Benchmark gaming, unconfirmed rumors, paywalled claims, etc.
|
||||
|
||||
-
|
||||
161
docs/backup-storage-policy.md
Normal file
161
docs/backup-storage-policy.md
Normal file
|
|
@ -0,0 +1,161 @@
|
|||
# Backup storage policy — open-weight model reserve
|
||||
|
||||
**Status:** foundation draft (concrete path and budget **TBD**)
|
||||
**Related:** `INTENT.md`, `SCOPE.md`, `inventory/collection-policy.md`
|
||||
**Adjacent:** `disaster-control` BackupPolicy (platform backups — different concern)
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define where Freedom Intelligence stores **open-weight model blobs** (and justified
|
||||
companions), how much capacity we allow, and how this reserve relates to other
|
||||
lab backup facilities.
|
||||
|
||||
Git never stores weight tensors. Git stores:
|
||||
|
||||
* inventory catalog metadata (`inventory/catalog/`)
|
||||
* policies and schemas
|
||||
* research briefs
|
||||
|
||||
---
|
||||
|
||||
## Storage class
|
||||
|
||||
| Requirement | Policy |
|
||||
| ----------- | ------ |
|
||||
| **Class** | Backup / bulk durable storage — **not** hot cluster PVCs or app disks |
|
||||
| **Durability** | Same or better retention posture as other lab bulk assets; prefer media that survives workstation rebuilds |
|
||||
| **Performance** | Sequential read for restore/training pull is enough; low latency not required |
|
||||
| **Access** | Operator and approved lab hosts only; not a public mirror |
|
||||
| **Separation** | Do not co-mingle with age-encrypted operational backups (Forgejo dumps, k3s state) without a clear subdirectory and different lifecycle rules |
|
||||
|
||||
Platform backup paths such as `/opt/backup/railiance/{infra,cluster}/` and
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. The model
|
||||
reserve may live on the **same physical facility** only if isolated by path and
|
||||
quota so large weight pulls cannot crowd out restore media.
|
||||
|
||||
---
|
||||
|
||||
## Target location (to pin)
|
||||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| **Host / facility** | `TBD` — operator pin (e.g. backup NAS, off-cluster bulk volume, dedicated disk) |
|
||||
| **Base path or bucket** | `TBD` — suggested shape: `…/freedom-intelligence/models/` |
|
||||
| **Layout under base** | See [On-disk layout](#on-disk-layout) |
|
||||
| **Mount on lab hosts** | `TBD` |
|
||||
| **Credentials** | `TBD` — if remote: OpenBao or existing backup credential lane; never commit secrets |
|
||||
|
||||
Until pinned, **do not** bulk-download multi-GB models into this git workspace or
|
||||
into hot root filesystems.
|
||||
|
||||
### Suggested path shape (non-binding)
|
||||
|
||||
```text
|
||||
{BACKUP_ROOT}/freedom-intelligence/
|
||||
├── models/
|
||||
│ └── {org}__{name}/
|
||||
│ └── {revision}/
|
||||
│ ├── blobs/ # weight files, shards
|
||||
│ └── MANIFEST.txt # optional local copy of hashes
|
||||
├── companions/ # adapters, tokenizers only when separate
|
||||
└── staging/ # incomplete downloads; not catalog-ready
|
||||
```
|
||||
|
||||
Map each `{org}__{name}/{revision}` to an inventory catalog entry.
|
||||
|
||||
---
|
||||
|
||||
## On-disk layout
|
||||
|
||||
| Path element | Rule |
|
||||
| ------------ | ---- |
|
||||
| `org__name` | Hugging Face-style id with `/` → `__` |
|
||||
| `revision` | Git commit SHA, tag, or release id used at download time |
|
||||
| `blobs/` | Actual files; prefer original names from source |
|
||||
| `staging/` | Incomplete transfers; purge or resume; never mark collected until complete + verified |
|
||||
|
||||
---
|
||||
|
||||
## Capacity budget
|
||||
|
||||
| Parameter | Policy |
|
||||
| --------- | ------ |
|
||||
| **Soft quota** | `TBD` GiB/TiB — operator pin based on free backup capacity |
|
||||
| **Hard stop** | No new collection when soft quota exceeded unless operator raises budget |
|
||||
| **Per-pull threshold** | See `inventory/collection-policy.md` (size gates approval) |
|
||||
| **Growth review** | Revisit quota when catalog total exceeds 70% of soft quota |
|
||||
| **Eviction** | Prefer archive/delete lowest-priority, easily re-obtainable revisions first; record eviction in catalog history |
|
||||
|
||||
---
|
||||
|
||||
## Integrity and provenance
|
||||
|
||||
For every completed collection:
|
||||
|
||||
1. Record source URL and revision in the catalog entry.
|
||||
2. Store checksums (`sha256` of each blob or upstream manifest digest).
|
||||
3. Record download date (UTC) and downloader identity.
|
||||
4. Prefer official org releases over anonymous re-uploads.
|
||||
5. Keep license text or SPDX id in catalog; refuse unclear licenses.
|
||||
|
||||
Verification command examples belong in tooling later; policy only requires that
|
||||
**catalog claims match on-disk checksums** before status `collected`.
|
||||
|
||||
---
|
||||
|
||||
## Retention
|
||||
|
||||
| Class | Retention |
|
||||
| ----- | --------- |
|
||||
| **Strategic reserve** (high priority, hard to re-obtain) | Keep until explicit deprecation |
|
||||
| **Working set** (common bases for training experiments) | Keep while in active use + one superseded revision optional |
|
||||
| **Staging** | Max 14 days incomplete, then purge |
|
||||
| **Deprecated** | Metadata retained in catalog with status `evicted` or `superseded`; blobs may be deleted |
|
||||
|
||||
---
|
||||
|
||||
## Encryption and offsite
|
||||
|
||||
| Topic | Policy |
|
||||
| ----- | ------ |
|
||||
| **At rest** | Follow host/facility default; extra age/GPG of multi-hundred-GB trees is optional and costly |
|
||||
| **In transit** | HTTPS or trusted lab network only |
|
||||
| **Offsite copy** | Optional later; not required for foundation. If added, coordinate with `disaster-control` so model reserve does not break operational backup SLAs |
|
||||
|
||||
---
|
||||
|
||||
## What must not live here
|
||||
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or training corpora with personal data (domain datasets need their own policy)
|
||||
* Operational backups (databases, k3s state, Forgejo dumps)
|
||||
* Git LFS dumps of full model trees as a substitute for backup storage
|
||||
|
||||
---
|
||||
|
||||
## Operator checklist to pin TBD fields
|
||||
|
||||
- [ ] Choose facility and base path
|
||||
- [ ] Confirm free capacity and set soft quota
|
||||
- [ ] Document mount/access for lab hosts
|
||||
- [ ] Confirm credential path (if any)
|
||||
- [ ] Create empty `models/` and `staging/` directories
|
||||
- [ ] Update this file: replace `TBD` rows with concrete values
|
||||
- [ ] Note pin date and operator in revision history below
|
||||
|
||||
### Pin log
|
||||
|
||||
| Date | Operator | Change |
|
||||
| ---- | -------- | ------ |
|
||||
| 2026-07-23 | foundation | Policy created; path and quota unpinned |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to disaster-control
|
||||
|
||||
`disaster-control` owns **platform resilience** (what to restore after loss).
|
||||
Freedom Intelligence owns **which open weights we choose to retain** and their
|
||||
inventory. Shared facilities are fine; shared lifecycle rules are not automatic —
|
||||
model reserve is large, slow-changing, and rarely needed for emergency restore.
|
||||
111
docs/daily-brief-playbook.md
Normal file
111
docs/daily-brief-playbook.md
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
# Daily research brief — playbook
|
||||
|
||||
**Audience:** operator or agent executing `target_repo: freedom-intelligence`
|
||||
**Template:** `briefs/_template.md`
|
||||
**Baseline memory:** `research/2026-07-24-baseline-field-survey.md`
|
||||
**Sources:** `docs/sources-allowlist.md`
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Produce **one** curated brief per active day covering axes A–D when there is
|
||||
signal. Briefs are **deltas** from the baseline survey and prior briefs — not
|
||||
full resurveys.
|
||||
|
||||
---
|
||||
|
||||
## Idempotence
|
||||
|
||||
| Rule | Detail |
|
||||
| ---- | ------ |
|
||||
| One brief per date | Path `briefs/YYYY/MM/YYYY-MM-DD.md` |
|
||||
| Calendar | Prefer **Europe/Berlin** date in front matter |
|
||||
| Skip empty days | Either omit file or write `*No material delta.*` in Headline deltas |
|
||||
| Automation due bit | Cleared only by State Hub event `fi_daily_brief` (below) |
|
||||
|
||||
---
|
||||
|
||||
## Manual procedure
|
||||
|
||||
1. **Orient**
|
||||
- Read last 1–2 briefs if any; skim baseline § executive snapshot if first week
|
||||
- `git pull` on freedom-intelligence
|
||||
|
||||
2. **Scan** (`docs/sources-allowlist.md`) — 30–60 minutes cap
|
||||
- Capture only items that change capability, cost, access, or operability
|
||||
|
||||
3. **Draft**
|
||||
```bash
|
||||
mkdir -p briefs/$(date +%Y/%m)
|
||||
cp briefs/_template.md briefs/$(date +%Y/%m)/$(date +%Y-%m-%d).md
|
||||
```
|
||||
- Fill front matter (`status: final` when done)
|
||||
- Leave empty sections as `*(none)*`
|
||||
- Cite primary URLs
|
||||
|
||||
4. **Collection candidates**
|
||||
- Use inventory field names; set priority
|
||||
- If nominating: add or update `inventory/catalog/{id}.yaml` with
|
||||
`status: candidate` and `collection.brief_refs`
|
||||
|
||||
5. **Lab implications**
|
||||
- Link open workplans (`FI-WP-*`) only when action is real
|
||||
|
||||
6. **Commit** (operator policy) — brief + any catalog YAML
|
||||
|
||||
7. **Completion evidence** (required for activity-core rhythm)
|
||||
|
||||
Post State Hub progress (HTTP or MCP), shape:
|
||||
|
||||
```yaml
|
||||
event_type: fi_daily_brief
|
||||
author: <agent-or-operator>
|
||||
summary: "FI daily brief YYYY-MM-DD"
|
||||
detail:
|
||||
repo: freedom-intelligence
|
||||
date: "YYYY-MM-DD"
|
||||
path: "briefs/YYYY/MM/YYYY-MM-DD.md"
|
||||
collection_candidates: 0
|
||||
```
|
||||
|
||||
Workstation API often `http://127.0.0.1:8000`; remote tunnel ports per
|
||||
Custodian docs. See `docs/state-hub-delivery.md`.
|
||||
|
||||
---
|
||||
|
||||
## Automation consumer
|
||||
|
||||
When activity-core emits `activity_task_spawn` with
|
||||
`target_repo: freedom-intelligence` and labels including `research-brief`:
|
||||
|
||||
1. Checkout this repo on the execution host
|
||||
2. Run this playbook end-to-end
|
||||
3. Prefer **llm-connect / OpenRouter** for any LLM drafting (Binky lesson: do
|
||||
not install coding-agent CLIs on railiance01 solely for this)
|
||||
4. Always post `fi_daily_brief` even if the brief is “no material delta”
|
||||
5. Idempotency key: do not open a second brief file for the same `date`
|
||||
|
||||
Activity definition: `activity-definitions/fi-daily-research-brief.md`
|
||||
Workplan: `workplans/FI-WP-0002-activity-core-daily-research.md`
|
||||
|
||||
---
|
||||
|
||||
## Quality bar
|
||||
|
||||
| Check | Pass |
|
||||
| ----- | ---- |
|
||||
| Signal | No filler paragraphs |
|
||||
| Sources | At least one primary link per non-empty axis row |
|
||||
| Honesty | Benchmarks marked directional; prices “verify before budget” |
|
||||
| Collection | License noted for any candidate |
|
||||
| Length | Prefer one screen to three; deep dives go under `research/` |
|
||||
|
||||
---
|
||||
|
||||
## First daily after baseline
|
||||
|
||||
The baseline survey (2026-07-24) already covers the field. The **first daily**
|
||||
should only list changes **after** that document’s research window. If nothing
|
||||
moved, a short no-delta brief still completes the rhythm and posts
|
||||
`fi_daily_brief`.
|
||||
65
docs/hardware-envelope.md
Normal file
65
docs/hardware-envelope.md
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
# Hardware envelope — homelab accessible (draft)
|
||||
|
||||
**Status:** assumptions + TBD measurements
|
||||
**Used by:** inventory `hardware_class`, axis B/C prioritization, collection size decisions
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define what Freedom Intelligence means by **homelab accessible** so we do not
|
||||
collect models that only make sense on hyperscale clusters.
|
||||
|
||||
---
|
||||
|
||||
## Working tiers (until hosts are measured)
|
||||
|
||||
| Tier | Assumed capacity | Target model class |
|
||||
| ---- | ---------------- | ------------------ |
|
||||
| **T0 — CPU / edge** | 16–64 GB system RAM, no GPU | ≤3B Q4; embeddings ≤0.5B |
|
||||
| **T1 — consumer GPU** | 8–12 GB VRAM | 7–8B Q4/Q5; 3B fp16 |
|
||||
| **T2 — enthusiast GPU** | 16–24 GB VRAM | 14B Q4; 8B fp16; light 32B Q4 |
|
||||
| **T3 — lab multi-GPU** | 2×24 GB+ or 48 GB+ | 32B fp16 / 70B Q4; small MoE |
|
||||
| **T4 — out of envelope** | multi-node / 8×A100-class | Full DeepSeek-V3 MoE — P2 watch only |
|
||||
|
||||
P0 collection targets **T0–T2**. P2 full MoE is **T4**.
|
||||
|
||||
---
|
||||
|
||||
## Lab hosts (fill in)
|
||||
|
||||
| Host | Role | GPU | VRAM | RAM | Notes |
|
||||
| ---- | ---- | --- | ---- | --- | ----- |
|
||||
| TBD | primary local inference | TBD | TBD | TBD | |
|
||||
| TBD | training experiments | TBD | TBD | TBD | |
|
||||
| railiance01 | cluster (not weight store) | n/a | n/a | n/a | Prefer not to fill hot disks with weights |
|
||||
|
||||
---
|
||||
|
||||
## Inference runtime defaults (intent)
|
||||
|
||||
| Runtime | When |
|
||||
| ------- | ---- |
|
||||
| llama.cpp / GGUF | Desktop & edge |
|
||||
| vLLM | Server multi-request |
|
||||
| MLX | Apple silicon if present |
|
||||
| Ollama | Quick operator UX only |
|
||||
|
||||
---
|
||||
|
||||
## Training defaults (intent)
|
||||
|
||||
| Method | Envelope |
|
||||
| ------ | -------- |
|
||||
| QLoRA 7–8B | T1–T2 |
|
||||
| QLoRA 14B | T2 |
|
||||
| Full FT 7B | T2–T3 |
|
||||
| 70B+ FT | T3+ only with explicit plan |
|
||||
|
||||
---
|
||||
|
||||
## Pin log
|
||||
|
||||
| Date | Change |
|
||||
| ---- | ------ |
|
||||
| 2026-07-24 | Draft tiers; host table empty |
|
||||
62
docs/sources-allowlist.md
Normal file
62
docs/sources-allowlist.md
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
# Sources allowlist — daily research
|
||||
|
||||
Standing channels for Freedom Intelligence briefs (axes A–D). Prefer **primary**
|
||||
sources over aggregators. This is a watchlist, not a scrape mandate.
|
||||
|
||||
---
|
||||
|
||||
## Axis A — Frontier & commercial
|
||||
|
||||
| Channel | What to extract |
|
||||
| ------- | --------------- |
|
||||
| OpenAI / Anthropic / Google / xAI / DeepSeek blogs & release notes | Capability claims, deprecations |
|
||||
| Official pricing pages | $/MTok in/out, cache, batch |
|
||||
| Model cards for API models | Context limits, tool use, policies |
|
||||
| LMSYS / Arena / artificialanalysis-class charts | Directional only; note gaming risk |
|
||||
| SWE-bench Verified / Live leaderboards | Harness+model pairs, not raw IQ |
|
||||
|
||||
## Axis B — Edge / local / open
|
||||
|
||||
| Channel | What to extract |
|
||||
| ------- | --------------- |
|
||||
| Hugging Face org feeds: `Qwen`, `deepseek-ai`, `meta-llama`, `google`, `mistralai`, `BAAI`, `nomic-ai` | New weights, licenses, sizes |
|
||||
| llama.cpp, vLLM, MLX, Ollama release notes | Runtime breakthroughs |
|
||||
| GGUF / official quant announcements | Only after base model known |
|
||||
|
||||
## Axis C — Training & specialization
|
||||
|
||||
| Channel | What to extract |
|
||||
| ------- | --------------- |
|
||||
| arXiv cs.LG, cs.CL, cs.AI (recent) | Methods that fit homelab |
|
||||
| Unsloth, Hugging Face TRL, Axolotl, LLaMA-Factory | Practical recipes |
|
||||
| Preference-optimization papers/releases | DPO/ORPO/RLVR-class |
|
||||
|
||||
## Axis D — Harness & fleet
|
||||
|
||||
| Channel | What to extract |
|
||||
| ------- | --------------- |
|
||||
| OpenHands, Aider, SWE-agent, OpenCode, Cline repos/releases | Harness capability |
|
||||
| sand-boxer / agent-harness / activity-core (internal) | Coulomb adjacency |
|
||||
| MCP / tool-protocol standards | Interop shifts |
|
||||
|
||||
## Internal memory
|
||||
|
||||
| Artifact | Role |
|
||||
| -------- | ---- |
|
||||
| `research/2026-07-24-baseline-field-survey.md` | Baseline; dailies are deltas |
|
||||
| Prior `briefs/YYYY/MM/*.md` | Avoid duplicate headlines |
|
||||
| `inventory/catalog/` | Collection status |
|
||||
|
||||
## Do not
|
||||
|
||||
- Treat Twitter/X rumor as confirmed without primary link
|
||||
- Mirror entire HF orgs
|
||||
- Cite paywalled claims without noting access limit
|
||||
- Collect weights that fail the license gate (`inventory/collection-policy.md`)
|
||||
- Store API keys or customer data in briefs
|
||||
|
||||
## Cadence hint
|
||||
|
||||
For a 30–60 minute manual brief: scan A pricing+releases (10m), B HF/org
|
||||
releases (10m), C one methods thread (10m), D harness releases (10m), write
|
||||
(15m). Skip empty sections.
|
||||
62
docs/state-hub-delivery.md
Normal file
62
docs/state-hub-delivery.md
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
# State Hub / operator delivery
|
||||
|
||||
**Status:** contract defined; optional distribution not fully wired
|
||||
**Related:** `docs/daily-brief-playbook.md`, `workplans/FI-WP-0002-activity-core-daily-research.md`
|
||||
|
||||
---
|
||||
|
||||
## Required: completion evidence for daily rhythm
|
||||
|
||||
Every completed brief day (including “no material delta”) should post:
|
||||
|
||||
```yaml
|
||||
event_type: fi_daily_brief
|
||||
author: <agent-or-operator>
|
||||
summary: "FI daily brief YYYY-MM-DD"
|
||||
detail:
|
||||
repo: freedom-intelligence
|
||||
date: "YYYY-MM-DD"
|
||||
path: "briefs/YYYY/MM/YYYY-MM-DD.md"
|
||||
collection_candidates: 0
|
||||
```
|
||||
|
||||
This is the **idempotence signal** for activity-core context query
|
||||
`fi_brief_status` (FI-WP-0002-T03): `due=false` when an event exists for today
|
||||
(Europe/Berlin) with `detail.repo=freedom-intelligence`.
|
||||
|
||||
Inbound automation uses `activity_task_spawn` from activity-core (state-hub sink).
|
||||
See `activity-core/docs/task-emission-consumer-contract.md`.
|
||||
|
||||
---
|
||||
|
||||
## Optional: operator visibility
|
||||
|
||||
| Channel | Use |
|
||||
| ------- | --- |
|
||||
| **Progress event** (above) | Minimum — always for automation health |
|
||||
| **Inbox message** | Only when high-priority collection candidates appear |
|
||||
| **Wiki / Nextcloud** | Optional human mirror later |
|
||||
|
||||
---
|
||||
|
||||
## Non-goals
|
||||
|
||||
* Freedom Intelligence is **not** required to be a full State Hub domain yet
|
||||
* Do not use `ISSUE_SINK_TYPE=rest` / Forgejo issues for this rhythm
|
||||
* Do not register workplans by hand in the hub (`fix-consistency` after commit)
|
||||
|
||||
---
|
||||
|
||||
## Fallback without MCP
|
||||
|
||||
HTTP against State Hub API (workstation `http://127.0.0.1:8000` or remote
|
||||
tunnel ports). Check health before declaring hub down.
|
||||
|
||||
---
|
||||
|
||||
## Pin log
|
||||
|
||||
| Date | Change |
|
||||
| ---- | ------ |
|
||||
| 2026-07-23 | Stub created |
|
||||
| 2026-07-24 | `fi_daily_brief` contract + activity-core linkage documented |
|
||||
256
history/InitialExploration.md
Normal file
256
history/InitialExploration.md
Normal file
|
|
@ -0,0 +1,256 @@
|
|||
# Freedom Intelligence — Initial Exploration
|
||||
|
||||
**Date:** 2026-07-23
|
||||
**Repo:** `freedom-intelligence`
|
||||
**Scope affiliation:** Coulomb / NetKingdom
|
||||
**Status:** founding conversation recorded
|
||||
**One-liner (repo):** *The just in case open weights model lab*
|
||||
|
||||
---
|
||||
|
||||
## Why this document exists
|
||||
|
||||
This is a greenfield project. There is no INTENT.md, SCOPE.md, or workplan yet —
|
||||
only a README, LICENSE (MIT, © 2026 coulomb), and git scaffolding.
|
||||
|
||||
This file captures the **founding conversation** that defines why the lab exists,
|
||||
what it watches, and what durable assets it should accumulate. Later INTENT/SCOPE
|
||||
and workplans should refine from here, not reinvent the charter.
|
||||
|
||||
---
|
||||
|
||||
## Founding intent
|
||||
|
||||
Establish a standing capability to:
|
||||
|
||||
1. **Produce a daily research brief** on interesting progress in the AI models,
|
||||
methods, harness, and tooling space.
|
||||
2. **Collect and retain open-weight models** on lab backup storage so they remain
|
||||
available for possible future local training or operations — *just in case*.
|
||||
|
||||
The lab sits inside the broader Coulomb NetKingdom landscape: secure, inspectable,
|
||||
homelab- and fleet-accessible infrastructure where agentic systems and open
|
||||
security tooling already grow. Freedom Intelligence is the **intelligence sensing
|
||||
and model-reserve arm** of that landscape — not a product UI, not a cloud SaaS
|
||||
wrapper, and not a commitment to run every frontier model locally.
|
||||
|
||||
---
|
||||
|
||||
## Daily research brief — mandate
|
||||
|
||||
The primary operating rhythm is a **daily research brief**: a curated digest of
|
||||
what moved the needle, not a raw news firehose.
|
||||
|
||||
### What “interesting progress” means here
|
||||
|
||||
Progress that changes (or could soon change) **capability, cost, accessibility,
|
||||
or operability** of intelligence systems we care about — especially where those
|
||||
systems can be run, specialized, orchestrated, or reserved under our own control.
|
||||
|
||||
### Brief design principles (initial)
|
||||
|
||||
| Principle | Meaning |
|
||||
| --- | --- |
|
||||
| **Signal over noise** | Prefer releases, benchmarks, methods, and tooling shifts that matter for A–D below |
|
||||
| **Actionable for the lab** | Note whether something is relevant to monitoring, local/edge use, training, or harness work |
|
||||
| **Price and access aware** | Track not only SOTA quality but cost, license, and where weights can be obtained |
|
||||
| **Reproducible sources** | Cite primary sources (papers, model cards, release notes, repos, pricing pages) |
|
||||
| **Durable archive** | Briefs live in-repo (or linked inventory); they form a longitudinal memory of the field |
|
||||
| **Just-in-case bias** | When an open-weight artifact is rare, high-leverage, or at risk of vanishing, flag for collection |
|
||||
|
||||
### Suggested brief sections (starting shape)
|
||||
|
||||
Not every section every day — only when there is signal:
|
||||
|
||||
1. **Headline deltas** — what actually changed
|
||||
2. **Frontier & commercial** — capabilities, benchmarks, price moves (axis A)
|
||||
3. **Edge / local / open** — runnable intelligence at low cost (axis B)
|
||||
4. **Training & specialization** — methods and tooling for pre-/post-/domain work (axis C)
|
||||
5. **Harness & fleet** — agents, orchestration, sandboxing, multi-agent ops (axis D)
|
||||
6. **Collection candidates** — open-weight models or datasets worth parking on backup storage
|
||||
7. **Lab implications** — optional: what (if anything) the lab should do next
|
||||
|
||||
Format, cadence automation, and distribution (inbox, wiki, State Hub event) remain
|
||||
open design choices.
|
||||
|
||||
---
|
||||
|
||||
## Special interest axes
|
||||
|
||||
The monitoring and reporting charter is multifold. These four axes are the standing
|
||||
lens for both the daily brief and longer research notes.
|
||||
|
||||
### A — Frontier model capabilities, benchmarks, and price trends
|
||||
|
||||
**Watch:**
|
||||
|
||||
- Best-of-class closed and open frontier models (reasoning, code, multimodal, long context, tool use)
|
||||
- Benchmark suites and their gaming/validity debates (e.g. SWE-bench, Arena, GPQA, internal harness scores)
|
||||
- API price-per-token and effective cost-per-useful-task trends
|
||||
- Latency, rate limits, regional availability, and enterprise terms that affect operational use
|
||||
- Distillation / smaller “near-frontier” offerings that shift the price–capability curve
|
||||
|
||||
**Lab value:** know when paid frontier is the right tool, when open alternatives close the gap,
|
||||
and when price collapses make new workflows economical.
|
||||
|
||||
### B — Edge intelligence and low-cost open / local compute
|
||||
|
||||
**Watch:**
|
||||
|
||||
- Small and mid-size open-weight models that run on consumer or lab GPUs, NPUs, and CPUs
|
||||
- Quantization, speculative decoding, KV-cache, and inference runtimes (vLLM, llama.cpp, MLX, TensorRT-LLM, etc.)
|
||||
- Hardware-accessible stacks: single-box homelab, mini-PC, older datacenter cards, Apple silicon, edge SBCs
|
||||
- Specializations: code, RAG, embeddings, rerankers, speech, vision-on-device
|
||||
- Licensing that actually allows local ops and redistribution of derived artifacts
|
||||
|
||||
**Lab value:** independence from continuous cloud spend; privacy; offline and sovereign operation;
|
||||
honest capability floors for NetKingdom-local agents.
|
||||
|
||||
### C — Homelab-accessible training and domain specialization
|
||||
|
||||
**Watch:**
|
||||
|
||||
- Pre-training, continued pre-training, mid-training, SFT, preference optimization (DPO/ORPO/…), RL/verifiers
|
||||
- LoRA / QLoRA / adapters, mixture-of-experts fine-tunes, model merging, unsloth-class efficiency tricks
|
||||
- Datasets and data pipelines that fit homelab budgets (synthetic data, filtering, domain corpora)
|
||||
- Recipes that fit **our** hardware class — not only hyperscale recipes
|
||||
- Evaluation for domain specialization (not only public leaderboards)
|
||||
|
||||
**Lab value:** ability to specialize open bases for Coulomb / NetKingdom domains
|
||||
(security, ops, railiance, tenant workflows, etc.) without assuming a full training cluster.
|
||||
|
||||
### D — Agent harnesses, orchestration, sandboxing, and fleet management
|
||||
|
||||
**Watch:**
|
||||
|
||||
- Agent frameworks and harnesses (tool loops, planning, memory, multi-agent protocols)
|
||||
- Orchestration of long-running and multi-step work (workplans, fleets of workers, queues)
|
||||
- Sandboxing and containment of agent side effects (filesystem, network, credentials, blast radius)
|
||||
- Fleet management: provisioning, identity, observability, cost control, policy for many agents
|
||||
- Adjacent NetKingdom capabilities already in the landscape (e.g. sand-boxer, ops-warden, state-hub, kaizen agents)
|
||||
|
||||
**Lab value:** intelligence is useless without safe, operable delivery. This axis ties model progress
|
||||
to how NetKingdom actually runs agents.
|
||||
|
||||
---
|
||||
|
||||
## Open-weight model collection (backup storage reserve)
|
||||
|
||||
### Purpose
|
||||
|
||||
Use **lab backup storage facilities** to build a **deliberate collection of open-weight models**
|
||||
(and, where justified, companion tokenizers, adapters, and evaluation fixtures) that we might
|
||||
later reuse in a **local training or operations facility**.
|
||||
|
||||
This is a **reserve**, not a mirror of the entire Hub:
|
||||
|
||||
- Prefer models that score high on axes B/C or that are strategically useful even if large
|
||||
- Prefer clear open licenses and stable provenance (Hugging Face, official org releases, mirrors with checksums)
|
||||
- Prefer artifacts that are hard to re-obtain later (takedowns, license flips, silent re-uploads)
|
||||
- Store enough metadata to rehydrate: source URL, revision/hash, license, size, hardware class, brief notes
|
||||
|
||||
### Collection principles (initial)
|
||||
|
||||
| Principle | Detail |
|
||||
| --- | --- |
|
||||
| **Just in case** | Collection is insurance and optionality, not an obligation to serve every model |
|
||||
| **Backup-class storage** | Place weights on facilities already intended for durable bulk retention; do not fill hot cluster disks |
|
||||
| **Catalog in-repo or adjacent** | Git holds inventory/metadata; blobs stay on backup media |
|
||||
| **Integrity** | Checksums, signed manifests where available, recorded download date and source revision |
|
||||
| **License gate** | Only collect what license and terms allow for offline retention and intended reuse |
|
||||
| **Prioritization** | Daily brief feeds candidates; human or workplan approval gates large pulls |
|
||||
| **Reuse paths** | Future consumers: local inference, fine-tuning experiments, offline eval, air-gapped ops |
|
||||
|
||||
### What is explicitly out of scope (for now)
|
||||
|
||||
- Hosting a public model mirror or CDN
|
||||
- Guaranteeing 24/7 inference for the full collection
|
||||
- Training from scratch at frontier scale
|
||||
- Collecting closed weights or scraping in violation of terms
|
||||
|
||||
---
|
||||
|
||||
## Relationship to Coulomb / NetKingdom
|
||||
|
||||
| Concern | Freedom Intelligence role | Adjacent systems (examples) |
|
||||
| --- | --- | --- |
|
||||
| Field sensing | Daily brief + research notes | State Hub progress/inbox for distribution later |
|
||||
| Model reserve | Open-weight collection on backup storage | disaster-control / backup facilities; storage ops |
|
||||
| Local intelligence | Informs what can run on lab hardware | sand-boxer, agent harnesses, local runtimes |
|
||||
| Security posture | Prefers inspectable open stacks | net-kingdom, kings-guard, openbao identity patterns |
|
||||
| Domain specialization | Future fine-tunes for Coulomb domains | domain repos that need specialized assistants |
|
||||
|
||||
Freedom Intelligence does **not** replace NetKingdom security or ops products. It supplies
|
||||
**awareness and optional local intelligence assets** those systems may eventually consume.
|
||||
|
||||
---
|
||||
|
||||
## Current repo state
|
||||
|
||||
**Founding (2026-07-23):** LICENSE, README, this exploration only.
|
||||
|
||||
**Foundation files initiated (same day):**
|
||||
|
||||
```text
|
||||
freedom-intelligence/
|
||||
├── INTENT.md
|
||||
├── SCOPE.md
|
||||
├── README.md
|
||||
├── LICENSE
|
||||
├── briefs/
|
||||
│ ├── README.md
|
||||
│ └── _template.md
|
||||
├── inventory/
|
||||
│ ├── README.md
|
||||
│ ├── schema.yaml
|
||||
│ ├── collection-policy.md
|
||||
│ └── catalog/
|
||||
├── docs/
|
||||
│ ├── backup-storage-policy.md # path/quota TBD
|
||||
│ └── state-hub-delivery.md # stub, not wired
|
||||
└── history/
|
||||
└── InitialExploration.md
|
||||
```
|
||||
|
||||
Still open: first production brief, pinned backup path/quota, catalog entries, automation.
|
||||
|
||||
---
|
||||
|
||||
## Open questions for the next design pass
|
||||
|
||||
1. **Brief cadence and owner** — fully automated scrape + human edit, agent-produced draft, or operator-written?
|
||||
2. **Brief storage layout** — e.g. `briefs/YYYY/MM/YYYY-MM-DD.md` vs weekly rollups?
|
||||
3. **Sources allowlist** — which sites, arXiv categories, HF orgs, pricing pages, and X/GitHub signals matter first?
|
||||
4. **Backup storage location** — exact path/share/bucket, retention, encryption, and capacity budget for weights?
|
||||
5. **Inventory schema** — YAML/JSON catalog fields for each collected model revision?
|
||||
6. **Hardware envelope** — what GPUs/CPU hosts define “homelab accessible” for axes B and C?
|
||||
7. **INTENT/SCOPE split** — formalize aspirational vs near-term deliverables after first brief prototype?
|
||||
8. **State Hub registration** — when (if) this domain/repo becomes a tracked hub domain or remains a satellite under netkingdom/coulomb?
|
||||
|
||||
---
|
||||
|
||||
## Suggested near-term follow-ups
|
||||
|
||||
| Priority | Action | Status |
|
||||
| --- | --- | --- |
|
||||
| 1 | Draft `INTENT.md` / `SCOPE.md` from this charter | **Done** (foundation) |
|
||||
| 2 | Define brief template + layout under `briefs/` | **Done** (template; first filled brief still open) |
|
||||
| 3 | Document backup storage target and capacity policy | **Done** (policy; path/quota TBD pin) |
|
||||
| 4 | Define model inventory schema and collection approval rule of thumb | **Done** |
|
||||
| 5 | Optionally wire brief delivery into operator inbox / State Hub | **Stub only** (`docs/state-hub-delivery.md`) |
|
||||
|
||||
---
|
||||
|
||||
## Central proposition
|
||||
|
||||
> **Freedom Intelligence is the Coulomb NetKingdom lab for watching the AI capability frontier
|
||||
> with a bias toward open, local, and operable intelligence — and for holding a durable
|
||||
> open-weight reserve so useful models remain available when we need to train or run them ourselves.**
|
||||
|
||||
The daily research brief is the **sensing loop**.
|
||||
The open-weight collection on backup storage is the **strategic reserve**.
|
||||
Axes A–D keep both loops pointed at capability, cost, specialization, and safe agent operations.
|
||||
|
||||
---
|
||||
|
||||
*Recorded from the founding project conversation, 2026-07-23.*
|
||||
32
inventory/README.md
Normal file
32
inventory/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# Model inventory
|
||||
|
||||
In-repo **catalog of open-weight revisions** reserved (or nominated) for the lab.
|
||||
Weight blobs live on backup storage — see `docs/backup-storage-policy.md`.
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
inventory/
|
||||
├── README.md # this file
|
||||
├── schema.yaml # field reference + example
|
||||
├── collection-policy.md # approval and eligibility rules
|
||||
└── catalog/
|
||||
└── *.yaml # one file per model revision
|
||||
```
|
||||
|
||||
## Status values
|
||||
|
||||
`candidate` → `approved` → `collecting` → `collected` → `verified`
|
||||
also: `superseded` | `evicted` | `rejected`
|
||||
|
||||
## Adding an entry
|
||||
|
||||
1. Confirm eligibility in `collection-policy.md`.
|
||||
2. Create `catalog/{org}__{name}__{short_revision}.yaml` using fields from `schema.yaml`.
|
||||
3. Set `status: candidate` (or `approved` if already signed off).
|
||||
4. After download and checksums: set `collected` / `verified` and `collection.storage_path`.
|
||||
|
||||
## Empty catalog
|
||||
|
||||
The catalog starts empty on purpose. First entries come from daily brief
|
||||
**collection candidates** after policy checks — not from bulk scraping.
|
||||
0
inventory/catalog/.gitkeep
Normal file
0
inventory/catalog/.gitkeep
Normal file
42
inventory/catalog/BAAI__bge-m3__candidate.yaml
Normal file
42
inventory/catalog/BAAI__bge-m3__candidate.yaml
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
id: BAAI__bge-m3__candidate
|
||||
status: candidate
|
||||
name: bge-m3
|
||||
org: BAAI
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/BAAI/bge-m3
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/BAAI/bge-m3
|
||||
license:
|
||||
spdx: MIT
|
||||
url: https://huggingface.co/BAAI/bge-m3
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: ""
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~2 GB"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 1
|
||||
min_vram_gb_fp16: 2
|
||||
notes: "Runs on CPU comfortably for many workloads"
|
||||
axes: [B]
|
||||
priority: high
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P0 multilingual embedding staple for local RAG (docs, ops notes, DE/EN)."
|
||||
tags: [embedding, multilingual, rag]
|
||||
companions: []
|
||||
notes: "Small download — within agent auto-collect band after license check once storage pinned."
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P0 recommendation from initial deep research."
|
||||
43
inventory/catalog/BAAI__bge-reranker-v2-m3__candidate.yaml
Normal file
43
inventory/catalog/BAAI__bge-reranker-v2-m3__candidate.yaml
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
id: BAAI__bge-reranker-v2-m3__candidate
|
||||
status: candidate
|
||||
name: bge-reranker-v2-m3
|
||||
org: BAAI
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/BAAI/bge-reranker-v2-m3
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/BAAI/bge-reranker-v2-m3
|
||||
license:
|
||||
spdx: Apache-2.0
|
||||
url: https://huggingface.co/BAAI/bge-reranker-v2-m3
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: "Confirm SPDX on card."
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~1–2 GB (estimate)"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 1
|
||||
min_vram_gb_fp16: 2
|
||||
notes: "Companion to bge-m3"
|
||||
axes: [B]
|
||||
priority: medium
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P1 — cheap RAG quality win as companion to BGE-M3."
|
||||
tags: [reranker, rag]
|
||||
companions:
|
||||
- BAAI__bge-m3__candidate
|
||||
notes: ""
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P1 recommendation from initial deep research."
|
||||
42
inventory/catalog/Qwen__Qwen3-14B__candidate.yaml
Normal file
42
inventory/catalog/Qwen__Qwen3-14B__candidate.yaml
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
id: Qwen__Qwen3-14B__candidate
|
||||
status: candidate
|
||||
name: Qwen3-14B
|
||||
org: Qwen
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/Qwen/Qwen3-14B
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/Qwen/Qwen3-14B
|
||||
license:
|
||||
spdx: Apache-2.0
|
||||
url: https://huggingface.co/Qwen/Qwen3-14B
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: "Confirm card at download."
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~28 GB fp16 / ~9 GB Q4 (estimate)"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 10
|
||||
min_vram_gb_fp16: 28
|
||||
notes: "T2 quality step"
|
||||
axes: [B, C]
|
||||
priority: medium
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P1 — stronger single-GPU chat/code when quota allows after P0."
|
||||
tags: [instruct, qwen3]
|
||||
companions: []
|
||||
notes: ""
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P1 recommendation from initial deep research."
|
||||
42
inventory/catalog/Qwen__Qwen3-8B__candidate.yaml
Normal file
42
inventory/catalog/Qwen__Qwen3-8B__candidate.yaml
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
id: Qwen__Qwen3-8B__candidate
|
||||
status: candidate
|
||||
name: Qwen3-8B
|
||||
org: Qwen
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/Qwen/Qwen3-8B
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/Qwen/Qwen3-8B
|
||||
license:
|
||||
spdx: Apache-2.0
|
||||
url: https://huggingface.co/Qwen/Qwen3-8B
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: "Confirm exact card license at download time; Qwen3 line generally Apache-2.0."
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~16 GB fp16 / ~5 GB Q4 (estimate)"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 6
|
||||
min_vram_gb_fp16: 16
|
||||
notes: "Default T1–T2 general instruct and FT base"
|
||||
axes: [B, C]
|
||||
priority: high
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P0 spine — best default open general/tool model for local ops and QLoRA domain specialization."
|
||||
tags: [instruct, text, qwen3, ft-base]
|
||||
companions: []
|
||||
notes: "Prefer Instruct variant on card if separate repo; pin commit SHA at collection."
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P0 recommendation from initial deep research."
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
id: deepseek-ai__DeepSeek-R1-Distill-Qwen-14B__candidate
|
||||
status: candidate
|
||||
name: DeepSeek-R1-Distill-Qwen-14B
|
||||
org: deepseek-ai
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
|
||||
license:
|
||||
spdx: MIT
|
||||
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: "R1 distill series MIT — confirm card at pin time."
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~28 GB fp16 / ~9 GB Q4 (estimate)"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 10
|
||||
min_vram_gb_fp16: 28
|
||||
notes: "T2 Q4 preferred; fall back to 8B distill if VRAM tight"
|
||||
axes: [B, C]
|
||||
priority: high
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P0 local reasoning without full R1 MoE — agent/tool loops and harder offline tasks."
|
||||
tags: [reasoning, distill, deepseek]
|
||||
companions: []
|
||||
notes: "If disk/VRAM constrained, substitute DeepSeek-R1-Distill-Qwen-8B as P0 alternate."
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P0 recommendation from initial deep research."
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
id: deepseek-ai__DeepSeek-R1-Distill-Qwen-32B__candidate
|
||||
status: candidate
|
||||
name: DeepSeek-R1-Distill-Qwen-32B
|
||||
org: deepseek-ai
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
|
||||
license:
|
||||
spdx: MIT
|
||||
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: ""
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~65 GB fp16 / ~20 GB Q4 (estimate)"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 20
|
||||
min_vram_gb_fp16: 64
|
||||
notes: "T2–T3; only if hardware envelope supports"
|
||||
axes: [B, C]
|
||||
priority: medium
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P1 stronger local reasoner — approve only with VRAM + quota headroom."
|
||||
tags: [reasoning, distill, deepseek]
|
||||
companions: []
|
||||
notes: "Do not collect full DeepSeek-V3/R1 MoE under this id."
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P1 recommendation from initial deep research."
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
id: meta-llama__Llama-3.2-3B-Instruct__candidate
|
||||
status: candidate
|
||||
name: Llama-3.2-3B-Instruct
|
||||
org: meta-llama
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
|
||||
license:
|
||||
spdx: custom
|
||||
url: https://ai.meta.com/llama/license/
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: "Llama Community License — not MIT; review terms before commercial redistribution."
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~6 GB fp16 / ~2 GB Q4 (estimate)"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 3
|
||||
min_vram_gb_fp16: 8
|
||||
notes: "T0–T1 edge / always-on"
|
||||
axes: [B]
|
||||
priority: high
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P0 edge instruct — tiny, huge ecosystem, good CPU/GPU floor for agents."
|
||||
tags: [instruct, edge, llama]
|
||||
companions: []
|
||||
notes: "HF gated model — need accepted license on account before download."
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P0 recommendation from initial deep research."
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
id: nomic-ai__nomic-embed-text-v1.5__candidate
|
||||
status: candidate
|
||||
name: nomic-embed-text-v1.5
|
||||
org: nomic-ai
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
|
||||
license:
|
||||
spdx: Apache-2.0
|
||||
url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: ""
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "<1 GB"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 1
|
||||
min_vram_gb_fp16: 1
|
||||
notes: "CPU-friendly"
|
||||
axes: [B]
|
||||
priority: high
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- research/2026-07-24-baseline-field-survey.md
|
||||
reason: "P0 lightweight embed for A/B with BGE-M3; long-context text retrieval."
|
||||
tags: [embedding, rag]
|
||||
companions: []
|
||||
notes: "If v2 text is preferred at collection time, update id/url and supersede this candidate."
|
||||
history:
|
||||
- at: "2026-07-24"
|
||||
event: nominated
|
||||
by: baseline-survey
|
||||
detail: "P0 recommendation from initial deep research."
|
||||
134
inventory/collection-policy.md
Normal file
134
inventory/collection-policy.md
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
# Collection policy — open-weight reserve
|
||||
|
||||
**Status:** foundation
|
||||
**Related:** `schema.yaml`, `docs/backup-storage-policy.md`, `INTENT.md`
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Decide **what** enters the open-weight reserve, **who** may approve it, and
|
||||
**when** a daily-brief candidate becomes a catalog entry with blobs on backup
|
||||
storage.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
* Keep a **small, high-leverage** reserve — not a Hugging Face mirror
|
||||
* Enforce **license and integrity** before download completes
|
||||
* Match capacity to the backup storage soft quota
|
||||
* Prefer models that serve axes **B** and **C**, plus strategic **A** open releases
|
||||
|
||||
---
|
||||
|
||||
## Eligibility (must pass all)
|
||||
|
||||
1. **Open weights** — weights obtainable under terms that allow offline retention for lab use
|
||||
2. **Clear license** — SPDX or linkable license text; `allows_offline_retention: true`
|
||||
3. **Stable provenance** — official org, tagged release, or commit revision (not anonymous drive-by reupload as sole source)
|
||||
4. **Lab rationale** — written `reason` tied to at least one axis A–D (usually B/C)
|
||||
5. **Capacity** — estimated size fits under remaining soft quota (see backup storage policy)
|
||||
|
||||
Fail any gate → status `rejected` with reason, or never enter catalog.
|
||||
|
||||
---
|
||||
|
||||
## Priority rubric
|
||||
|
||||
| Priority | Guidance |
|
||||
| -------- | -------- |
|
||||
| **high** | Rare or strategically important; license/access risk of disappearance; uniquely strong for B/C at our hardware class; hard to re-obtain |
|
||||
| **medium** | Clear lab use within 1–2 quarters; good quality/cost; easy enough to re-download but worth having cold |
|
||||
| **low** | Nice to have; only collect if quota headroom is large and pull is cheap |
|
||||
|
||||
Daily brief **collection candidates** should set a suggested priority; approval may change it.
|
||||
|
||||
---
|
||||
|
||||
## Approval rule of thumb
|
||||
|
||||
| Estimated total size | Approval |
|
||||
| -------------------- | -------- |
|
||||
| **< 5 GiB** | Operator or lab agent may collect after license check; catalog entry required before or immediately after |
|
||||
| **5–40 GiB** | Explicit operator approval (chat, workplan task, or signed catalog `approved_by`) |
|
||||
| **> 40 GiB** | Operator approval **plus** check against soft quota and whether a smaller quant/variant suffices |
|
||||
| **Any size if quota ≥ 70% used** | Operator approval required regardless of size |
|
||||
| **Unclear license or ToS risk** | Do not collect; status `rejected` |
|
||||
|
||||
“Operator” means the human lab owner (or a documented delegate). Agents may
|
||||
**nominate** (`status: candidate`) freely from briefs; they may **collect** only
|
||||
within the < 5 GiB band when licenses are unambiguous — otherwise stop at
|
||||
`candidate` / `approved`.
|
||||
|
||||
---
|
||||
|
||||
## Lifecycle
|
||||
|
||||
```text
|
||||
brief nominates
|
||||
→ candidate (catalog YAML, no blobs required)
|
||||
→ approved (license + size + quota OK)
|
||||
→ collecting (download in staging/)
|
||||
→ collected (blobs complete, checksums recorded, storage_path set)
|
||||
→ verified (optional re-hash / smoke load)
|
||||
→ superseded|evicted (replaced or removed; metadata kept)
|
||||
```
|
||||
|
||||
Rejected candidates stay in catalog only if useful as a decision record; otherwise omit.
|
||||
|
||||
---
|
||||
|
||||
## What we prefer to collect
|
||||
|
||||
* Small/mid instruct and code models that fit the hardware envelope
|
||||
* Strong embedding / rerank models for local RAG
|
||||
* Base models known to fine-tune well under QLoRA/LoRA on lab GPUs
|
||||
* Official quant releases when they are the supported distribution
|
||||
* Adapters and tokenizers that unlock a reserved base (as companions)
|
||||
|
||||
## What we usually skip
|
||||
|
||||
* Duplicate quants of the same revision already reserved
|
||||
* Huge models with no near-term local run/train path and no access-risk story
|
||||
* Merges/repacks without provenance
|
||||
* Datasets larger than model weights unless separately justified (default: out of band)
|
||||
* Anything requiring acceptance flows we cannot satisfy offline
|
||||
|
||||
---
|
||||
|
||||
## Companions
|
||||
|
||||
Tokenizers, LoRA adapters, and small eval fixtures may be collected when:
|
||||
|
||||
* they are required to use a reserved base, or
|
||||
* they are small (< 1 GiB) and high leverage
|
||||
|
||||
Link via `companions` in the catalog schema.
|
||||
|
||||
---
|
||||
|
||||
## Brief integration
|
||||
|
||||
1. Brief section **Collection candidates** nominates items.
|
||||
2. Operator/agent opens `inventory/catalog/{id}.yaml` with `status: candidate`.
|
||||
3. Approval and download follow this policy and `docs/backup-storage-policy.md`.
|
||||
4. Brief `brief_refs` on the entry point back to the nominating day(s).
|
||||
|
||||
---
|
||||
|
||||
## Eviction rule of thumb
|
||||
|
||||
When over quota or cleaning:
|
||||
|
||||
1. `low` priority, easily re-obtainable from still-live official URLs
|
||||
2. Superseded revisions with a newer `verified` replacement
|
||||
3. Never silent-delete: set `status: evicted`, clear or note `storage_path`, append `history`
|
||||
|
||||
---
|
||||
|
||||
## Non-goals
|
||||
|
||||
* Automatic bulk mirrors of entire orgs
|
||||
* Collecting on every brief mention without priority
|
||||
* Bypassing license gates for “research only” convenience
|
||||
118
inventory/schema.yaml
Normal file
118
inventory/schema.yaml
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
# Freedom Intelligence — open-weight inventory entry schema
|
||||
# Each collected (or tracked) model revision is one YAML file under catalog/
|
||||
# Filename suggestion: {org}__{name}__{short_revision}.yaml
|
||||
#
|
||||
# Schema version documents field meaning for humans and future validators.
|
||||
schema_version: "0.1.0"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Example entry (illustrative only — not a real collection claim)
|
||||
# ---------------------------------------------------------------------------
|
||||
# id: meta-llama__Llama-3.2-3B-Instruct__abc1234
|
||||
# status: candidate | approved | collecting | collected | verified | superseded | evicted | rejected
|
||||
# name: Llama-3.2-3B-Instruct
|
||||
# org: meta-llama
|
||||
# source:
|
||||
# kind: huggingface # huggingface | github_release | direct_url | other
|
||||
# url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
|
||||
# revision: abc1234def... # commit sha, tag, or release id
|
||||
# model_card_url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
|
||||
# license:
|
||||
# spdx: llama3.2 # or MIT, Apache-2.0, etc.; use "custom" + notes if needed
|
||||
# url: https://...
|
||||
# allows_offline_retention: true
|
||||
# allows_local_ops: true
|
||||
# allows_fine_tune: true
|
||||
# notes: ""
|
||||
# artifacts:
|
||||
# - path: blobs/model.safetensors
|
||||
# sha256: "..."
|
||||
# bytes: 0
|
||||
# size:
|
||||
# total_bytes: 0
|
||||
# total_human: "0 B"
|
||||
# hardware_class:
|
||||
# # Rough lab guidance — not a guarantee
|
||||
# min_vram_gb_q4: 4
|
||||
# min_vram_gb_fp16: 8
|
||||
# notes: "fits consumer 8GB at Q4"
|
||||
# axes: # why this matters to the lab
|
||||
# - B
|
||||
# - C
|
||||
# priority: medium # low | medium | high
|
||||
# collection:
|
||||
# approved_by: ""
|
||||
# approved_at: null # ISO-8601 date
|
||||
# downloaded_at: null
|
||||
# downloaded_by: ""
|
||||
# storage_path: "" # absolute or facility-relative path on backup storage
|
||||
# brief_refs: # briefs that nominated this candidate
|
||||
# - briefs/2026/07/2026-07-23.md
|
||||
# reason: "Strong small instruct base; clear license; homelab-fit."
|
||||
# tags:
|
||||
# - instruct
|
||||
# - text
|
||||
# companions: [] # optional related catalog ids (adapters, tokenizers)
|
||||
# notes: ""
|
||||
# history:
|
||||
# - at: "2026-07-23"
|
||||
# event: nominated
|
||||
# by: operator
|
||||
# detail: "From daily brief collection candidates."
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Required fields by status (normative for humans; tooling may enforce later)
|
||||
# ---------------------------------------------------------------------------
|
||||
# candidate: id, status, name, org, source.url, source.revision, license, reason, priority
|
||||
# approved: + collection.approved_by, collection.approved_at
|
||||
# collected: + collection.downloaded_at, collection.storage_path, size, artifacts[].sha256
|
||||
# verified: + checksums re-read OK (notes or history event)
|
||||
# rejected: + reason (why rejected)
|
||||
# superseded / evicted: + history event; storage_path may be empty after eviction
|
||||
|
||||
field_reference:
|
||||
id:
|
||||
type: string
|
||||
description: Stable catalog id; prefer {org}__{name}__{short_revision}
|
||||
status:
|
||||
type: enum
|
||||
values: [candidate, approved, collecting, collected, verified, superseded, evicted, rejected]
|
||||
name:
|
||||
type: string
|
||||
org:
|
||||
type: string
|
||||
source:
|
||||
type: object
|
||||
fields: [kind, url, revision, model_card_url]
|
||||
license:
|
||||
type: object
|
||||
fields: [spdx, url, allows_offline_retention, allows_local_ops, allows_fine_tune, notes]
|
||||
artifacts:
|
||||
type: list
|
||||
item_fields: [path, sha256, bytes]
|
||||
size:
|
||||
type: object
|
||||
fields: [total_bytes, total_human]
|
||||
hardware_class:
|
||||
type: object
|
||||
fields: [min_vram_gb_q4, min_vram_gb_fp16, notes]
|
||||
axes:
|
||||
type: list
|
||||
values: [A, B, C, D]
|
||||
priority:
|
||||
type: enum
|
||||
values: [low, medium, high]
|
||||
collection:
|
||||
type: object
|
||||
fields: [approved_by, approved_at, downloaded_at, downloaded_by, storage_path, brief_refs]
|
||||
reason:
|
||||
type: string
|
||||
tags:
|
||||
type: list
|
||||
companions:
|
||||
type: list
|
||||
notes:
|
||||
type: string
|
||||
history:
|
||||
type: list
|
||||
item_fields: [at, event, by, detail]
|
||||
279
research/2026-07-24-baseline-field-survey.md
Normal file
279
research/2026-07-24-baseline-field-survey.md
Normal file
|
|
@ -0,0 +1,279 @@
|
|||
# Baseline field survey — Freedom Intelligence starting point
|
||||
|
||||
**Date:** 2026-07-24
|
||||
**Type:** deep baseline (not a daily delta brief)
|
||||
**Purpose:** Establish longitudinal memory so the **first daily research brief**
|
||||
can be a true delta, not a full survey. Also recommend the **initial open-weight
|
||||
collection** for the lab reserve.
|
||||
**Axes:** A frontier · B edge/local · C training · D harness/fleet
|
||||
**Sources:** public web surveys, HF/model docs, pricing roundups, harness project
|
||||
sites (mid–late 2026 public reporting). Prices and leaderboard claims move weekly —
|
||||
treat numbers as **order-of-magnitude** and re-check on first daily.
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive snapshot
|
||||
|
||||
| Theme | State of the field (mid-2026) | Lab implication |
|
||||
| ----- | ----------------------------- | --------------- |
|
||||
| Frontier | Multi-vendor race: OpenAI GPT-5.x, Anthropic Claude 4.x/Opus, Google Gemini 3.x, xAI Grok 4.x; coding agents dominate product narrative | Keep API access via `llm-connect`; do not try to host frontier closed weights |
|
||||
| Price collapse | DeepSeek-class APIs often **10–100×** cheaper than flagship closed models for many tasks | Route cheap/easy work to low-cost APIs; reserve local for privacy, offline, specialization |
|
||||
| Open weights | Strong mid-size dense + MoE story (Qwen3, DeepSeek distill/V3-class, Gemma/Llama small, Mistral small) | Homelab can be *useful* without 8×H100 |
|
||||
| Local stack | llama.cpp / vLLM / MLX / Ollama mature; Q4–Q5 quants standard | Axis B is operationally ready |
|
||||
| Training | QLoRA/Unsloth/TRL still the homelab path; full pretrain not lab-realistic | Axis C = adapters + domain SFT, not from-scratch |
|
||||
| Harness | OpenHands, Aider, SWE-agent, OpenCode, Cline; closed CLIs (Claude Code, Codex) lead quality | Axis D: watch + integrate with sand-boxer / agent-harness, don’t fork everything |
|
||||
|
||||
**Central proposition for the reserve:** collect a **compact spine** of (1) small
|
||||
general instruct, (2) mid-size code/reason distill, (3) multilingual embed,
|
||||
(4) optional larger “lab ceiling” base — not a mirror of Hugging Face.
|
||||
|
||||
---
|
||||
|
||||
## 2. Axis A — Frontier capabilities, benchmarks, price
|
||||
|
||||
### 2.1 Capability landscape
|
||||
|
||||
Public 2026 reporting groups the closed frontier into:
|
||||
|
||||
- **Flagship general / agentic** — OpenAI GPT-5.x family, Anthropic Claude Opus/Sonnet 4.x+, Google Gemini 3.x Pro
|
||||
- **Fast / cheap tiers** — GPT mini/nano, Claude Haiku, Gemini Flash
|
||||
- **Reasoning-heavy** — “thinking” / extended variants; cost dominated by output tokens
|
||||
- **Coding productization** — agent harnesses matter as much as base model (SWE-bench Verified numbers often quoted 70%+ with strong closed models + good harness; novel-issue sets much lower ~18–20%)
|
||||
|
||||
Benchmark caveats for the lab:
|
||||
|
||||
- Arena and public leaderboards are **gamed and lag** product quality
|
||||
- SWE-bench Verified ≠ “our monorepos + NetKingdom tools”
|
||||
- Prefer **internal harness scores** once we have them; use public numbers only as directional
|
||||
|
||||
### 2.2 Price-point trends (API, ~per 1M tokens)
|
||||
|
||||
Synthesized from 2026 pricing roundups (verify on provider pages before budget decisions):
|
||||
|
||||
| Band | Examples (reported ranges) | Role |
|
||||
| ---- | -------------------------- | ---- |
|
||||
| **Ultra-cheap** | DeepSeek V3.x / V4 Flash-class (~$0.14–0.28 in / ~$0.28–0.42 out; cache hits lower) | Bulk classify, draft, simple tools |
|
||||
| **Budget closed** | Gemini Flash, GPT mini/nano, Haiku-class | Default automation when quality bar is modest |
|
||||
| **Mid flagship** | Claude Sonnet-class, GPT mid-tier (~$2–5 in / ~$10–15 out) | Daily agent coding, serious writing |
|
||||
| **Top flagship** | Opus / GPT pro tiers (~$5+ in / $25–30+ out; pro tiers much higher) | Hard reasoning, high-stakes review |
|
||||
|
||||
**Trend:** continuous compression of mid-tier quality cost; flagship **output**
|
||||
tokens remain expensive. Caching and batch APIs swing real cost 50%+.
|
||||
|
||||
**Lab stance:**
|
||||
|
||||
- Frontier APIs stay first-class for hard work (via llm-connect / OpenRouter-class)
|
||||
- Daily *research* about price moves is high value for routing policy
|
||||
- Local models win on **privacy, offline, predictable unit cost, specialization** — not always raw SOTA
|
||||
|
||||
### 2.3 Open near-frontier (API or weights)
|
||||
|
||||
DeepSeek open-weight + cheap API continues to be the structural disruptor:
|
||||
MIT-leaning releases for R1/V3-class artifacts, distill series for single-GPU
|
||||
reasoning. Track **V3/V4 Flash vs Pro** naming carefully — product lines rename often.
|
||||
|
||||
---
|
||||
|
||||
## 3. Axis B — Edge, local, open compute
|
||||
|
||||
### 3.1 What “homelab useful” means now
|
||||
|
||||
| Class | Params (ballpark) | Typical local use |
|
||||
| ----- | ----------------- | ----------------- |
|
||||
| Edge / CPU-OK | ≤3B Q4 | Chat toys, classification, always-on small agents |
|
||||
| Consumer GPU | 7–14B Q4/Q5 | Daily coding assist, RAG chat, tool loops |
|
||||
| Single enthusiast GPU | 32B Q4 | Stronger reason/code; slower |
|
||||
| Lab multi-GPU | 70B Q4 or sparse MoE | Optional ceiling; not required for v1 reserve |
|
||||
|
||||
Runtimes to track: **llama.cpp**, **vLLM**, **MLX** (Apple), **Ollama** (UX),
|
||||
TensorRT-LLM (NVIDIA optimize). Speculative decoding + good quant matter more
|
||||
than chasing every new 70B.
|
||||
|
||||
### 3.2 Open model families to watch (weights)
|
||||
|
||||
| Family | Why it matters | License caution |
|
||||
| ------ | -------------- | --------------- |
|
||||
| **Qwen3** (Alibaba) | Dense + MoE line; strong multilingual & tool use; embeddings/VL variants | Apache-class for many; check model card |
|
||||
| **DeepSeek** (V3/R1 + distills) | Reasoning + cost; MIT for key R1/distill artifacts | Confirm per-revision card |
|
||||
| **Llama 3.x / 4 text** | Ecosystem, tooling, small 1B–3B instruct | Llama community license (not pure MIT) |
|
||||
| **Gemma 3** | Google small/efficient; EmbeddingGemma | Gemma terms |
|
||||
| **Mistral Small / Ministral** | EU-friendly commercial open options | Apache / research licenses vary |
|
||||
| **Phi / Smol** class | Edge CPU experiments | Check commercial terms |
|
||||
|
||||
### 3.3 Embeddings & retrieval (local RAG spine)
|
||||
|
||||
For NetKingdom docs, code, and ops notes:
|
||||
|
||||
| Model | Role | Notes |
|
||||
| ----- | ---- | ----- |
|
||||
| **BGE-M3** (BAAI) | Default multilingual dense (+ multi-granularity) | MIT; ~0.5B; self-host staple |
|
||||
| **Nomic Embed** text v1/v2 | Strong English/multilingual text | Apache |
|
||||
| **Nomic Embed Code** / code-specialized | Code retrieval | Larger; optional P1 |
|
||||
| **EmbeddingGemma-300M** | Tiny multilingual | Edge RAG |
|
||||
| **Qwen3 embedding / VL-small** | Multimodal retrieval experiments | Watch cards |
|
||||
|
||||
Rerankers (BGE reranker, mini cross-encoders) are high leverage and small — good
|
||||
companions once embed base is reserved.
|
||||
|
||||
---
|
||||
|
||||
## 4. Axis C — Homelab training & domain specialization
|
||||
|
||||
### 4.1 Realistic lab methods
|
||||
|
||||
| Method | Homelab fit | Use |
|
||||
| ------ | ----------- | --- |
|
||||
| QLoRA / LoRA (Unsloth, PEFT, TRL) | Excellent | Domain assistants (security, ops, railiance) |
|
||||
| Preference (DPO/ORPO/KTO) | Good on 7–14B | Style and safety for agents |
|
||||
| Continued pretrain | Possible on small bases | Domain language (logs, policies) |
|
||||
| Full SFT of 70B+ | Poor without multi-GPU budget | Prefer distill/adapter on smaller |
|
||||
| From-scratch pretrain | Out of scope | — |
|
||||
|
||||
### 4.2 Bases worth specializing
|
||||
|
||||
1. **7–8B instruct** (Qwen3-8B or Llama-3.1-8B-Instruct class) — default FT target
|
||||
2. **14B** — quality step if VRAM allows
|
||||
3. **DeepSeek-R1-Distill 14B/32B** — reason-heavy agents after FT discipline
|
||||
4. **Code-specialized mid** — if code assistants dominate workload
|
||||
|
||||
Data: prefer **synthetic + curated NetKingdom docs** over scraping; keep PII out
|
||||
of training sets (separate policy later).
|
||||
|
||||
### 4.3 Tooling to track (not collect as weights)
|
||||
|
||||
Unsloth, Axolotl, TRL, llama-factory, mergekit, preference datasets (UltraFeedback-class),
|
||||
eval harnesses (lm-eval, custom domain suites).
|
||||
|
||||
---
|
||||
|
||||
## 5. Axis D — Agent harnesses, orchestration, sandboxing, fleet
|
||||
|
||||
### 5.1 Open harness landscape (2026)
|
||||
|
||||
| Project | Shape | Why watch |
|
||||
| ------- | ----- | --------- |
|
||||
| **OpenHands** | Full agent platform + Canvas; MIT; multi-backend | Leading open coding-agent platform; multi-agent |
|
||||
| **Aider** | Git-native CLI pair programmer | Atomic commits; simple; great for ops scripts |
|
||||
| **SWE-agent / mini-SWE-agent** | Research harness, clean ACI | Benchmark + security CTF modes |
|
||||
| **OpenCode** | Open Claude Code alternative | High interest community |
|
||||
| **Cline** | IDE agent | Editor-centric workflows |
|
||||
| **Closed CLIs** | Claude Code, Codex, Gemini CLI | Quality bar for comparison |
|
||||
|
||||
SWE-bench: harness + model co-determine scores; Mini-SWE-Agent claims strong
|
||||
results in very little code — architecture lessons for Coulomb harnesses.
|
||||
|
||||
### 5.2 NetKingdom adjacency
|
||||
|
||||
| Coulomb piece | Relation to D |
|
||||
| ------------- | ------------- |
|
||||
| **sand-boxer** | Where agents run safely |
|
||||
| **activity-core** | When work is due (this lab’s daily brief) |
|
||||
| **agent-harness / kaizen** | Who executes with profiles |
|
||||
| **state-hub** | What changed; task spawn completion |
|
||||
| **llm-connect** | Which model backend |
|
||||
|
||||
Freedom Intelligence **reports** on harness tech; it does not replace sand-boxer.
|
||||
|
||||
### 5.3 Sandboxing & fleet themes to track
|
||||
|
||||
- Default-deny network + TTL sandboxes
|
||||
- Credential routing (OpenBao / ops-warden) separate from model weights
|
||||
- Multi-agent kanban / workplan-tied loops (Ralph-workplan class)
|
||||
- Cost and rate-limit fleet management for API models
|
||||
|
||||
---
|
||||
|
||||
## 6. Collection recommendations (open-weight reserve)
|
||||
|
||||
### 6.1 Decision framework (recap)
|
||||
|
||||
- Prefer **B/C** leverage + clear license + stable org provenance
|
||||
- Size gates per `inventory/collection-policy.md`
|
||||
- Backup path must be pinned (FI-WP-0001-T04) before large pulls
|
||||
- Quants: prefer **official or well-known GGUF/AWQ** only after base id is cataloged; v1 recommends **safetensors official** first, quant as companion later
|
||||
|
||||
### 6.2 P0 — seed now (approve after storage pin)
|
||||
|
||||
| # | Model (representative HF id) | Est. size class | Axes | Why |
|
||||
| - | ---------------------------- | --------------- | ---- | --- |
|
||||
| 1 | `Qwen/Qwen3-8B` (or current 8B-Instruct sibling on card) | ~16 GB fp16 / ~5 GB Q4 | B, C | Best default general + FT base; multilingual; tool-friendly |
|
||||
| 2 | `meta-llama/Llama-3.2-3B-Instruct` | ~6 GB fp16 / ~2 GB Q4 | B | Tiny always-on / edge; huge ecosystem |
|
||||
| 3 | `BAAI/bge-m3` | ~2 GB | B | Multilingual RAG staple; MIT |
|
||||
| 4 | `deepseek-ai/DeepSeek-R1-Distill-Qwen-14B` (or 8B if VRAM tight) | ~28 GB fp16 / ~9 GB Q4 | B, C | Local reasoning without full R1; MIT distill story |
|
||||
| 5 | `nomic-ai/nomic-embed-text-v1.5` (or current v2 text) | <1 GB | B | Lightweight embed alternative / A/B with BGE |
|
||||
|
||||
**Rough P0 cold storage (fp16 worst case):** ~50–60 GB.
|
||||
**With Q4 preference for LLMs + fp for embeds:** often **~20–30 GB**. Prefer quants only when catalog notes the exact quant provenance.
|
||||
|
||||
### 6.3 P1 — next wave (quota permitting)
|
||||
|
||||
| # | Model | Why |
|
||||
| - | ----- | --- |
|
||||
| 6 | `Qwen/Qwen3-14B` (instruct) | Quality step for single-GPU chat/code |
|
||||
| 7 | `google/gemma-3-4b-it` or EmbeddingGemma-300M | Efficient Google line; embed edge |
|
||||
| 8 | `mistralai/Mistral-Small-3.x` instruct (current card) | EU-adjacent open option |
|
||||
| 9 | `deepseek-ai/DeepSeek-R1-Distill-Qwen-32B` | Stronger local reason if ≥24 GB VRAM class |
|
||||
| 10 | Code embed: `nomic-ai/nomic-embed-code` or Jina code embed | Code RAG for repos |
|
||||
| 11 | Small reranker: `BAAI/bge-reranker-v2-m3` | Cheap quality win for RAG |
|
||||
|
||||
### 6.4 P2 — watch / large (candidate or docs only)
|
||||
|
||||
| Item | Guidance |
|
||||
| ---- | -------- |
|
||||
| Full **DeepSeek-V3 / R1** MoE (~600B+ class, hundreds of GB even quantized) | **Do not auto-collect.** Separate capacity + multi-GPU decision |
|
||||
| **Llama 70B / Qwen 72B** class | Only if lab gains multi-GPU and clear offline need |
|
||||
| Every new quant repack on HF | Skip; track base official only |
|
||||
|
||||
### 6.5 Explicit non-recommendations (for now)
|
||||
|
||||
- Closed API-only weights
|
||||
- Anonymous “uncensored” reuploads without provenance
|
||||
- Duplicate quants of the same revision
|
||||
- Giant pretraining corpora as “models”
|
||||
|
||||
### 6.6 Catalog seed
|
||||
|
||||
YAML candidates for P0–P1 are under `inventory/catalog/` with
|
||||
`status: candidate` and `brief_refs` pointing at this survey. Approval and
|
||||
download are **FI-WP-0003**.
|
||||
|
||||
---
|
||||
|
||||
## 7. Implications for the first daily brief
|
||||
|
||||
The first `briefs/YYYY/MM/YYYY-MM-DD.md` after this survey should:
|
||||
|
||||
1. **Not** restate this whole document
|
||||
2. Cover **only deltas** since 2026-07-24 (new releases, price changes, harness versions)
|
||||
3. Revisit P0/P1 if a **new open model** clearly displaces a candidate
|
||||
4. Note any **license or access shocks** (takedowns, geo blocks)
|
||||
5. Optionally mark FI-WP-0001-T04 / FI-WP-0003 progress under Lab implications
|
||||
|
||||
Automation: activity-core def `fi-daily-research-brief` (disabled until FI-WP-0002
|
||||
resolver + dry-run). Manual playbook: `docs/daily-brief-playbook.md`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Open questions for later briefs
|
||||
|
||||
1. Exact **hardware envelope** measured VRAM on lab hosts?
|
||||
2. Prefer **GGUF Q5_K_M** vs official safetensors for P0?
|
||||
3. Is **German-first** multilingual quality a hard requirement (affects embed + instruct choice)?
|
||||
4. Will local models serve **air-gapped NetKingdom** scenarios or only cost/privacy convenience?
|
||||
5. Single **code model** vs general instruct for domain FT?
|
||||
|
||||
---
|
||||
|
||||
## 9. Source anchors (non-exhaustive)
|
||||
|
||||
- DeepSeek R1 release / MIT notes — DeepSeek API docs & HF model cards
|
||||
- Pricing roundups 2026 — IntuitionLabs, MorphLLM, provider pricing pages
|
||||
- Embeddings 2026 comparisons — BentoML, Milvus/Zilliz, BGE-M3 cards
|
||||
- Harness landscape — OpenHands, Aider, SWE-agent comparisons (LocalAI Master, vendor blogs)
|
||||
- Open-weight commercial lists — community curations (verify each card)
|
||||
|
||||
Re-verify any number before spend or multi-hundred-GB download.
|
||||
|
||||
---
|
||||
|
||||
*Baseline established 2026-07-24 for Freedom Intelligence. Daily briefs begin as deltas from here.*
|
||||
147
workplans/FI-WP-0001-lab-operating-foundation.md
Normal file
147
workplans/FI-WP-0001-lab-operating-foundation.md
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
---
|
||||
id: FI-WP-0001
|
||||
type: workplan
|
||||
title: "Lab operating foundation: sources, playbook, storage pin, hardware envelope"
|
||||
domain: infotech
|
||||
repo: freedom-intelligence
|
||||
status: active
|
||||
owner: grok
|
||||
topic_slug: freedom-intelligence
|
||||
priority: high
|
||||
created: "2026-07-24"
|
||||
updated: "2026-07-24"
|
||||
---
|
||||
|
||||
# FI-WP-0001 — Lab operating foundation
|
||||
|
||||
## Goal
|
||||
|
||||
Make Freedom Intelligence **operable for daily research and model reserve work**
|
||||
before automation: pin backup storage, document sources and hardware envelope,
|
||||
ship a daily-brief playbook, and close gaps left after the foundation docs
|
||||
(`INTENT.md`, `SCOPE.md`, `briefs/`, `inventory/`).
|
||||
|
||||
Baseline field survey lives in `research/2026-07-24-baseline-field-survey.md`
|
||||
and seeds collection candidates under `inventory/catalog/`.
|
||||
|
||||
## Depends on
|
||||
|
||||
- Foundation docs already present (2026-07-23)
|
||||
- Baseline research (this session) — **source of first collection candidates**
|
||||
|
||||
## Parallel / follow-on
|
||||
|
||||
- **FI-WP-0002** — activity-core daily cadence (after playbook exists)
|
||||
- **FI-WP-0003** — execute reserve seed under collection policy (after storage pin)
|
||||
|
||||
## Tasks
|
||||
|
||||
### T01 — Sources allowlist for daily research
|
||||
|
||||
```task
|
||||
id: FI-WP-0001-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Create `docs/sources-allowlist.md` with standing channels for axes A–D:
|
||||
|
||||
- Frontier: provider blogs, pricing pages, LMSYS/Arena notes, major model cards
|
||||
- Open weights: Hugging Face org watchlist, official GitHub releases
|
||||
- Methods: arXiv categories (cs.LG, cs.CL, cs.AI, cs.SE), Unsloth/axolotl/TRL notes
|
||||
- Harness: OpenHands, Aider, SWE-agent, sand-boxer-adjacent ecosystem
|
||||
- Explicit **do not scrape** / license caution list
|
||||
|
||||
**Done when:** allowlist exists and is linked from `briefs/README.md`.
|
||||
|
||||
---
|
||||
|
||||
### T02 — Daily brief playbook (manual first, automation-ready)
|
||||
|
||||
```task
|
||||
id: FI-WP-0001-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Write `docs/daily-brief-playbook.md`:
|
||||
|
||||
- Step sequence: scan allowlist → draft from template → collection candidates →
|
||||
optional hub progress event
|
||||
- Completion evidence shape: `event_type=fi_daily_brief`, `detail.repo`, `detail.date`
|
||||
- Idempotence: one brief per calendar day (Europe/Berlin)
|
||||
- Quality bar aligned with `briefs/_template.md` and INTENT principles
|
||||
- Pointer that **first automated day** starts from baseline survey, not empty memory
|
||||
|
||||
**Done when:** an operator or agent can produce a brief without reading INTENT end-to-end.
|
||||
|
||||
---
|
||||
|
||||
### T03 — Hardware envelope draft
|
||||
|
||||
```task
|
||||
id: FI-WP-0001-T03
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Write `docs/hardware-envelope.md` with **honest TBD** for unmeasured hosts, plus
|
||||
working assumptions for “homelab accessible” (consumer GPU tiers, CPU-only
|
||||
floor, multi-GPU lab class). Used by inventory `hardware_class` fields and
|
||||
axis B/C prioritization.
|
||||
|
||||
**Done when:** envelope doc exists; operator can fill measured VRAM/host rows later
|
||||
without schema changes.
|
||||
|
||||
---
|
||||
|
||||
### T04 — Pin backup storage path and soft quota
|
||||
|
||||
```task
|
||||
id: FI-WP-0001-T04
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Operator task: fill TBD fields in `docs/backup-storage-policy.md`:
|
||||
|
||||
- facility / host
|
||||
- base path (`…/freedom-intelligence/models/`)
|
||||
- soft quota
|
||||
- access method
|
||||
- pin log entry
|
||||
|
||||
Create empty `models/` and `staging/` on that facility.
|
||||
|
||||
**Done when:** policy has no critical TBD for path/quota; `FI-WP-0003` may download.
|
||||
|
||||
---
|
||||
|
||||
### T05 — Link baseline research into lab navigation
|
||||
|
||||
```task
|
||||
id: FI-WP-0001-T05
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
- Point `README.md` / `SCOPE.md` at `research/2026-07-24-baseline-field-survey.md`
|
||||
- Note that first **daily** brief is a delta from this baseline, not a second full survey
|
||||
- Ensure inventory candidates from the survey are present under `inventory/catalog/`
|
||||
|
||||
**Done when:** navigation and catalog seed are consistent with the survey.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance (workplan-level)
|
||||
|
||||
- [x] Sources allowlist + daily playbook + hardware envelope drafted
|
||||
- [ ] Backup storage path and soft quota pinned (T04 — operator)
|
||||
- [x] Baseline research persisted and linked
|
||||
- [x] Catalog candidates seeded for recommended models
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Enabling activity-core schedule (FI-WP-0002)
|
||||
- Bulk model downloads (FI-WP-0003)
|
||||
- State Hub domain registration for freedom-intelligence as a full domain
|
||||
208
workplans/FI-WP-0002-activity-core-daily-research.md
Normal file
208
workplans/FI-WP-0002-activity-core-daily-research.md
Normal file
|
|
@ -0,0 +1,208 @@
|
|||
---
|
||||
id: FI-WP-0002
|
||||
type: workplan
|
||||
title: "Activity-core daily research brief rhythm"
|
||||
domain: infotech
|
||||
repo: freedom-intelligence
|
||||
status: active
|
||||
owner: grok
|
||||
topic_slug: freedom-intelligence
|
||||
priority: high
|
||||
created: "2026-07-24"
|
||||
updated: "2026-07-24"
|
||||
depends_on:
|
||||
- FI-WP-0001
|
||||
---
|
||||
|
||||
# FI-WP-0002 — Activity-core daily research brief rhythm
|
||||
|
||||
## Goal
|
||||
|
||||
Schedule and emit the **Freedom Intelligence daily research brief** via
|
||||
**activity-core**, following the proven Binky rhythm pattern:
|
||||
|
||||
```text
|
||||
Temporal cron (activity-core)
|
||||
→ context: fi_brief_status (due?)
|
||||
→ rule emit activity_task_spawn (state-hub sink)
|
||||
→ consumer (agent-harness / operator) runs playbook in freedom-intelligence
|
||||
→ progress event fi_daily_brief (idempotence for next day)
|
||||
```
|
||||
|
||||
activity-core answers **when / what / where**. It does **not** write the brief.
|
||||
Execution stays in this repo (or a harness checking out this repo).
|
||||
|
||||
## Architecture
|
||||
|
||||
| Piece | Location | Role |
|
||||
| ----- | -------- | ---- |
|
||||
| ActivityDefinition | `freedom-intelligence/activity-definitions/fi-daily-research-brief.md` | Cron + rules |
|
||||
| External def load | activity-core `ACTIVITY_DEFINITION_DIRS` / external paths | Discover domain def |
|
||||
| Context resolver | activity-core state-hub query `fi_brief_status` (or shell equivalent) | `due` for today |
|
||||
| Task sink | state-hub `activity_task_spawn` | Default safe sink |
|
||||
| Consumer | agent-harness Green/Blue or manual operator session | Run playbook |
|
||||
| Completion | State Hub `fi_daily_brief` progress event | Clears `due` |
|
||||
|
||||
Reference contracts:
|
||||
|
||||
- `activity-core/docs/task-emission-consumer-contract.md`
|
||||
- Binky: `activity-definitions/binky-daily-rhythm.md` + `binky-control/OperatingRhythm.md`
|
||||
- `docs/daily-brief-playbook.md` (this repo)
|
||||
|
||||
## Schedule (proposed)
|
||||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| Cron | `30 7 * * 1-5` (weekdays 07:30) |
|
||||
| Timezone | `Europe/Berlin` |
|
||||
| Misfire | `skip` |
|
||||
| First enable | after dry-run spawn + one manual brief completion event proven |
|
||||
|
||||
Weekends optional later (`0 9 * * 6` optional Saturday scan) — not in v1.
|
||||
|
||||
## Tasks
|
||||
|
||||
### T01 — ActivityDefinition (domain repo, disabled)
|
||||
|
||||
```task
|
||||
id: FI-WP-0002-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Add `activity-definitions/fi-daily-research-brief.md`:
|
||||
|
||||
- `enabled: false` until dry-run passes
|
||||
- cron + timezone as above
|
||||
- context source for due check (stub params documented if resolver not yet live)
|
||||
- rule: emit task when `due`, `target_repo: freedom-intelligence`, labels
|
||||
`["freedom-intelligence", "research-brief", "automated"]`
|
||||
- description points at `docs/daily-brief-playbook.md`
|
||||
|
||||
**Done when:** file exists, frontmatter valid, matches ADR-002 shape used by
|
||||
existing definitions (binky / forgejo prune).
|
||||
|
||||
---
|
||||
|
||||
### T02 — Completion event contract + playbook hook
|
||||
|
||||
```task
|
||||
id: FI-WP-0002-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Document and implement playbook steps for:
|
||||
|
||||
```yaml
|
||||
event_type: fi_daily_brief
|
||||
author: <agent-or-operator>
|
||||
summary: "FI daily brief YYYY-MM-DD"
|
||||
detail:
|
||||
repo: freedom-intelligence
|
||||
date: "YYYY-MM-DD"
|
||||
path: "briefs/YYYY/MM/YYYY-MM-DD.md"
|
||||
collection_candidates: <int>
|
||||
```
|
||||
|
||||
Wire into `docs/daily-brief-playbook.md` and `docs/state-hub-delivery.md`.
|
||||
|
||||
**Done when:** completion evidence shape is unambiguous for a future resolver.
|
||||
|
||||
---
|
||||
|
||||
### T03 — `fi_brief_status` context resolver (activity-core)
|
||||
|
||||
```task
|
||||
id: FI-WP-0002-T03
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
**Cross-repo (activity-core):** implement state-hub (or shell) resolver:
|
||||
|
||||
- Query name: `fi_brief_status`
|
||||
- Returns `{ items: [{ kind: "daily_brief", due: bool, date: "YYYY-MM-DD" }] }`
|
||||
- `due: true` when no `fi_daily_brief` progress event with
|
||||
`detail.repo=freedom-intelligence` exists for today (Europe/Berlin)
|
||||
|
||||
Mirror `binky_rhythm_status` pattern. Add unit tests with mocked hub.
|
||||
|
||||
**Done when:** resolver merged in activity-core and callable from definition params.
|
||||
|
||||
---
|
||||
|
||||
### T04 — Register external activity-definitions path
|
||||
|
||||
```task
|
||||
id: FI-WP-0002-T04
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
**Cross-repo / ops:** ensure railiance activity-core worker loads this repo’s
|
||||
`activity-definitions/` (env `ACTIVITY_DEFINITION_DIRS` or documented external
|
||||
path). Run `make sync-activity-definitions` / schedule reconciliation.
|
||||
|
||||
**Done when:** definition appears in automation inventory; schedule id
|
||||
`activity-schedule-fi-daily-research-brief` (or equivalent) is visible when enabled.
|
||||
|
||||
---
|
||||
|
||||
### T05 — Dry-run spawn, then enable
|
||||
|
||||
```task
|
||||
id: FI-WP-0002-T05
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
1. Manual `POST .../activity-definitions/<id>/trigger` with def still disabled
|
||||
or one-shot test path
|
||||
2. Confirm `activity_task_spawn` in State Hub (no Forgejo spam)
|
||||
3. Manually execute playbook once; post `fi_daily_brief`
|
||||
4. Confirm next resolver `due=false` for that date
|
||||
5. Set `enabled: true` on definition; commit
|
||||
|
||||
**Done when:** one full loop proven; weekday schedule live.
|
||||
|
||||
---
|
||||
|
||||
### T06 — Consumer routing note (agent-harness)
|
||||
|
||||
```task
|
||||
id: FI-WP-0002-T06
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Document how harness/operator picks up `target_repo: freedom-intelligence`
|
||||
tasks (checkout path, Green/Blue profile, LLM via llm-connect if used). Prefer
|
||||
**no** Claude Code install on railiance01 (Binky residual lesson).
|
||||
|
||||
**Done when:** `docs/daily-brief-playbook.md` § Automation consumer is enough for
|
||||
the next agent session.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance (workplan-level)
|
||||
|
||||
- [x] Domain ActivityDefinition drafted (`enabled: false`)
|
||||
- [x] Completion event contract documented
|
||||
- [ ] Resolver + external path + dry-run enable (T03–T05)
|
||||
- [ ] Consumer path documented for harness (T06)
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Writing the LLM brief **inside** activity-core workers
|
||||
- ISSUE_SINK_TYPE=rest / Forgejo issues for this rhythm
|
||||
- Weekend or multi-brief-per-day schedules
|
||||
- Full State Hub domain onboarding for freedom-intelligence
|
||||
|
||||
## Residual handoffs
|
||||
|
||||
| Residual | Owner repo |
|
||||
| -------- | ---------- |
|
||||
| `fi_brief_status` resolver | activity-core |
|
||||
| ACTIVITY_DEFINITION_DIRS pin | activity-core / railiance ops |
|
||||
| Harness consumer profile | agent-harness / the-custodian as applicable |
|
||||
135
workplans/FI-WP-0003-seed-open-weight-reserve.md
Normal file
135
workplans/FI-WP-0003-seed-open-weight-reserve.md
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
---
|
||||
id: FI-WP-0003
|
||||
type: workplan
|
||||
title: "Seed open-weight reserve from baseline recommendations"
|
||||
domain: infotech
|
||||
repo: freedom-intelligence
|
||||
status: active
|
||||
owner: grok
|
||||
topic_slug: freedom-intelligence
|
||||
priority: medium
|
||||
created: "2026-07-24"
|
||||
updated: "2026-07-24"
|
||||
depends_on:
|
||||
- FI-WP-0001
|
||||
---
|
||||
|
||||
# FI-WP-0003 — Seed open-weight reserve from baseline recommendations
|
||||
|
||||
## Goal
|
||||
|
||||
Turn baseline survey recommendations into a **real, policy-compliant reserve**:
|
||||
approve high-priority catalog candidates, download to pinned backup storage,
|
||||
verify checksums, and leave medium/low items as candidates for later briefs.
|
||||
|
||||
Source of truth for *which* models:
|
||||
|
||||
- `research/2026-07-24-baseline-field-survey.md` § Collection recommendations
|
||||
- `inventory/catalog/*.yaml` (seeded as `candidate`)
|
||||
|
||||
Gates: `inventory/collection-policy.md` + `docs/backup-storage-policy.md` (T04 of FI-WP-0001).
|
||||
|
||||
## Priority tiers (from survey)
|
||||
|
||||
| Tier | Action in this workplan |
|
||||
| ---- | ----------------------- |
|
||||
| **P0 — seed now** | Approve + collect after storage pin (small/mid, high leverage) |
|
||||
| **P1 — next wave** | Approve when quota allows; may stay candidate |
|
||||
| **P2 — watch / large** | Keep candidate; do not bulk-pull full 600B+ MoE without explicit operator decision |
|
||||
|
||||
## Tasks
|
||||
|
||||
### T01 — Seed catalog YAML for recommended models
|
||||
|
||||
```task
|
||||
id: FI-WP-0003-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Create `status: candidate` entries under `inventory/catalog/` for every P0/P1
|
||||
recommendation in the baseline survey (and note P2 as candidates or docs-only).
|
||||
|
||||
**Done when:** catalog files exist and match `inventory/schema.yaml` required fields.
|
||||
|
||||
---
|
||||
|
||||
### T02 — Operator approve P0 set
|
||||
|
||||
```task
|
||||
id: FI-WP-0003-T02
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Human review of P0 candidates: license, size vs soft quota, hardware fit.
|
||||
Set `status: approved`, `collection.approved_by`, `collection.approved_at`.
|
||||
|
||||
**Done when:** each P0 entry is `approved` or `rejected` with reason.
|
||||
|
||||
---
|
||||
|
||||
### T03 — Download P0 to backup storage + verify
|
||||
|
||||
```task
|
||||
id: FI-WP-0003-T03
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
For each approved P0:
|
||||
|
||||
1. Download into `{BACKUP}/freedom-intelligence/models/...` (or `staging/` then promote)
|
||||
2. Record `artifacts[].sha256`, `size`, `collection.storage_path`
|
||||
3. Set `status: collected` then `verified` after re-hash or smoke load
|
||||
|
||||
Prefer official HF revisions; use `huggingface-cli` or equivalent with revision pin.
|
||||
|
||||
**Blocked on:** FI-WP-0001-T04 (storage pin).
|
||||
|
||||
**Done when:** all approved P0 are `verified` or explicitly deferred with notes.
|
||||
|
||||
---
|
||||
|
||||
### T04 — P1 decision pass
|
||||
|
||||
```task
|
||||
id: FI-WP-0003-T04
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
After P0, decide which P1 models fit remaining quota. Approve/collect subset or
|
||||
leave as candidates for daily briefs to re-prioritize.
|
||||
|
||||
**Done when:** each P1 has an explicit next status (`approved`, `candidate`, `rejected`).
|
||||
|
||||
---
|
||||
|
||||
### T05 — Reserve index snapshot
|
||||
|
||||
```task
|
||||
id: FI-WP-0003-T05
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
|
||||
Write `inventory/RESERVE-STATUS.md` summarizing collected vs candidate totals,
|
||||
bytes used vs soft quota, and hardware coverage gaps.
|
||||
|
||||
**Done when:** one-page status exists for operators and future daily briefs.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance (workplan-level)
|
||||
|
||||
- [x] Catalog seeded from baseline survey
|
||||
- [ ] Storage pin complete (upstream)
|
||||
- [ ] P0 verified on backup media
|
||||
- [ ] RESERVE-STATUS snapshot after first collections
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Hosting inference endpoints
|
||||
- Full DeepSeek-V3/R1 671B pull without separate capacity decision
|
||||
- Dataset reserve (weights only unless tiny tokenizer/adapters)
|
||||
Loading…
Add table
Add a link
Reference in a new issue