diff --git a/INTENT.md b/INTENT.md new file mode 100644 index 0000000..d196761 --- /dev/null +++ b/INTENT.md @@ -0,0 +1,119 @@ +# INTENT + +> This file captures **why this repository exists**, +> the **direction it is moving toward**, and +> the **kind of system it is meant to become**. +> It is intentionally **aspirational and stable**, not a description of current implementation. +> +> Founding record: `history/InitialExploration.md` + +--- + +## One-liner + +**The just-in-case open-weights model lab** — Coulomb/NetKingdom sensing and reserve for AI capability, cost, and operable local intelligence. + +--- + +## Why This Exists + +AI capability is moving faster than any single product roadmap. Frontier models, open weights, training recipes, and agent harnesses shift monthly. Most organizations either: + +* chase every vendor announcement without durable memory, or +* lock into a single cloud API and lose optionality when prices, licenses, or access change. + +Coulomb NetKingdom needs a third posture: + +1. **Sense** what actually moves capability, cost, accessibility, and operability. +2. **Reserve** open-weight artifacts that may matter later for local training or operations. +3. **Bias** toward intelligence that can be inspected, specialized, and run under our control. + +Freedom Intelligence is that lab — not a chat product, not a public model CDN, and not a commitment to host every frontier model. + +--- + +## Mission + +> *Where we are going.* + +Become the Coulomb NetKingdom **intelligence sensing loop and open-weight strategic reserve**: + +* a **daily research brief** that filters noise into lab-relevant signal across four standing axes; +* a **deliberate collection** of open-weight models (and justified companions) on backup-class storage; +* durable in-repo memory (briefs, inventory metadata, research notes) that outlives any single session or vendor. + +--- + +## Standing axes (the permanent lens) + +| Axis | Focus | +| ---- | ----- | +| **A — Frontier** | Best-of-class capabilities, benchmarks, and price-point trends | +| **B — Edge / local / open** | Low-cost open and local compute intelligence and specializations | +| **C — Homelab training** | Pre-, mid-, post-, and domain specialization recipes that fit lab hardware | +| **D — Harness & fleet** | Agent harnesses, orchestration, sandboxing, and fleet management | + +These axes define *what we watch*, not a promise to implement every technology we report on. + +--- + +## Dual loops + +```text +Daily research brief ──► Sensing loop (awareness, prioritization) +Open-weight reserve ──► Strategic reserve (optionality for train / run) +``` + +The brief feeds collection candidates. Collection never mirrors the entire Hub — only high-leverage, clearly licensed, hard-to-reobtain artifacts. + +--- + +## Design principles + +* **Signal over noise** — report what changes capability, cost, access, or operability. +* **Just in case** — hold optionality without obligation to serve every model. +* **Catalog in git, blobs on backup storage** — metadata is versioned; weights are bulk media. +* **License and integrity first** — no collection that terms forbid; checksums and provenance required. +* **Homelab honesty** — prefer methods and models that fit our real hardware envelope. +* **Operable intelligence** — models without harness, sandbox, and fleet thinking are incomplete for NetKingdom. +* **Inspectable and open-leaning** — bias toward stacks we can audit and specialize. + +--- + +## Strategic role in Coulomb / NetKingdom + +| Freedom Intelligence supplies | Adjacent systems own | +| ----------------------------- | -------------------- | +| Field sensing and briefs | State Hub distribution / inbox (optional consumers) | +| Open-weight reserve + inventory | Backup facilities, storage ops, disaster-control policy | +| Guidance on what can run locally | sand-boxer, agent harnesses, local runtimes, llm-connect | +| Domain-specialization *candidates* | Domain repos that consume specialized models later | + +It does **not** replace NetKingdom security, identity, ops, or application products. + +--- + +## Intended users + +* Operators and agents maintaining Coulomb/NetKingdom infrastructure who need a trusted daily AI field picture +* People deciding when to use frontier APIs vs local open weights +* Future local training / inference facilities that need a curated reserve and inventory + +--- + +## Maturity target + +A mature Freedom Intelligence lab should: + +* produce a reliable **daily brief** with stable structure and cited sources; +* maintain a **living inventory** of reserved model revisions with integrity metadata; +* enforce a **collection policy** (approval thresholds, license gate, capacity budget); +* know its **hardware envelope** for “homelab accessible”; +* optionally distribute briefs into operator workflows (inbox / State Hub) without becoming a general news product; +* feed domain specialization work without owning every consumer pipeline. + +--- + +## Stability note + +Changes to this file represent a **deliberate shift** in why the lab exists or what dual loops it owns. Such changes should be rare. Near-term deliverables and boundaries live in `SCOPE.md`. diff --git a/README.md b/README.md index c93b7ee..c792481 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,82 @@ # freedom-intelligence -The just in case open weights model lab \ No newline at end of file +**The just in case open weights model lab** + +Coulomb / NetKingdom lab for: + +1. **Daily research briefs** on AI models, methods, harness, and tooling +2. An **open-weight model reserve** on backup storage for possible local training or ops + +Standing lenses: **A** frontier & price · **B** edge/local open · **C** homelab training · **D** agent harness & fleet + +--- + +## Start here + +| Doc | Role | +| --- | ---- | +| [`INTENT.md`](INTENT.md) | Why the lab exists (stable) | +| [`SCOPE.md`](SCOPE.md) | What is in/out of scope now | +| [`research/2026-07-24-baseline-field-survey.md`](research/2026-07-24-baseline-field-survey.md) | **Baseline field survey + model collection recommendations** | +| [`docs/daily-brief-playbook.md`](docs/daily-brief-playbook.md) | How to produce a daily brief | +| [`history/InitialExploration.md`](history/InitialExploration.md) | Founding conversation | +| [`briefs/`](briefs/) | Daily briefs + template | +| [`inventory/`](inventory/) | Model catalog & collection policy | +| [`workplans/`](workplans/) | Setup workplans (incl. activity-core) | +| [`activity-definitions/`](activity-definitions/) | Cron definition for daily brief (disabled until FI-WP-0002) | + +--- + +## Workplans + +| ID | Title | Status | +| -- | ----- | ------ | +| [FI-WP-0001](workplans/FI-WP-0001-lab-operating-foundation.md) | Lab operating foundation | active (storage pin open) | +| [FI-WP-0002](workplans/FI-WP-0002-activity-core-daily-research.md) | Activity-core daily research rhythm | active (resolver/ops open) | +| [FI-WP-0003](workplans/FI-WP-0003-seed-open-weight-reserve.md) | Seed open-weight reserve | active (catalog seeded) | + +--- + +## Collection spine (from baseline) + +**P0 candidates** (approve after backup path pin): Qwen3-8B, Llama-3.2-3B-Instruct, +BGE-M3, DeepSeek-R1-Distill-Qwen-14B, nomic-embed-text-v1.5 + +Details and P1/P2: see baseline survey §6 and `inventory/catalog/`. + +--- + +## Layout + +```text +freedom-intelligence/ +├── INTENT.md / SCOPE.md / README.md +├── activity-definitions/ # fi-daily-research-brief (enabled: false) +├── briefs/ # daily deltas +├── research/ # deep surveys (baseline lives here) +├── inventory/catalog/ # YAML candidates / collected +├── docs/ # playbook, sources, storage, hardware +├── workplans/ # FI-WP-* +└── history/ +``` + +--- + +## Quick ops + +**New brief** + +```bash +mkdir -p briefs/$(date +%Y/%m) +cp briefs/_template.md briefs/$(date +%Y/%m)/$(date +%Y-%m-%d).md +# follow docs/daily-brief-playbook.md +``` + +**Do not** bulk-download models until `docs/backup-storage-policy.md` path/quota +are pinned (FI-WP-0001-T04). + +--- + +## License + +MIT — see [`LICENSE`](LICENSE). Model weights retain **upstream** licenses. diff --git a/SCOPE.md b/SCOPE.md new file mode 100644 index 0000000..d926710 --- /dev/null +++ b/SCOPE.md @@ -0,0 +1,143 @@ +# SCOPE + +> This file helps you quickly understand what this repository is about, +> when it is relevant, and when it is not. +> It is intentionally lightweight and may be incomplete. +> +> Aspirational charter: `INTENT.md` · Founding record: `history/InitialExploration.md` + +--- + +## One-liner + +Daily AI research briefs (models, methods, harness, tooling) plus an open-weight model reserve on lab backup storage — Coulomb/NetKingdom intelligence lab. + +--- + +## Core Idea + +Freedom Intelligence is a **sensing and reserve** repository: + +1. Curated **daily research briefs** across axes A–D (frontier, edge/local, training, harness/fleet). +2. **Inventory + collection policy** for open-weight models retained on backup-class storage for possible later local training or operations. + +Git holds briefs, schemas, policies, and catalog metadata. Model weight blobs stay off hot cluster disks on designated backup storage. + +--- + +## In Scope + +### Research brief + +- Daily brief production using `briefs/_template.md` +- Layout under `briefs/YYYY/MM/YYYY-MM-DD.md` +- Standing coverage of axes A–D when there is signal +- Collection-candidate flagging for the model reserve +- Source citation and durable archive of briefs in-repo + +### Model reserve + +- Inventory schema and catalog entries (`inventory/`) +- Collection approval policy and license gate +- Backup storage target and capacity policy (`docs/backup-storage-policy.md`) +- Metadata required to rehydrate a revision (source, hash, license, size, hardware class) +- Companion artifacts when justified (tokenizers, adapters, small eval fixtures) + +### Lab documentation + +- INTENT / SCOPE / history +- Optional delivery notes for State Hub / operator inbox (`docs/state-hub-delivery.md`) +- Hardware envelope notes as they become known + +--- + +## Out of Scope + +| Concern | Belongs elsewhere | +| ------- | ----------------- | +| Public model mirror / CDN | Not this lab | +| 24/7 inference serving of the full reserve | Future ops facility / runtime repos | +| Frontier-scale pre-training from scratch | Hyperscale; not homelab commitment | +| Closed-weight collection or ToS-violating scrapes | Forbidden | +| Provider-neutral LLM client library | `llm-connect` | +| Console assistant UX | `can-you-assist` | +| Sandbox establishment service | `sand-boxer` | +| Platform identity / secrets / bootstrap | `net-kingdom`, `key-cape` | +| General disaster backup governance | `disaster-control` | +| Domain application fine-tune pipelines (end-to-end) | Domain repos; this lab only reserves bases and tracks methods | + +--- + +## Relevant When + +- You need the daily or historical AI field picture for Coulomb/NetKingdom decisions +- You are evaluating frontier vs open/local options (capability, price, license) +- You want to know whether an open-weight model is (or should be) in the lab reserve +- You are planning homelab-scale specialization and need method / model candidates +- You are designing agent harness or fleet work and want external signal on that axis + +--- + +## Not Relevant When + +- You need an LLM API client (use `llm-connect`) +- You need to provision agent sandboxes (use `sand-boxer`) +- You need platform IAM or secrets (use NetKingdom / KeyCape) +- You only need one-off news without archival brief or inventory discipline + +--- + +## Current State + +| Area | Status | +| ---- | ------ | +| Overall | **Foundation + baseline research** (2026-07-24) | +| INTENT / SCOPE | Drafted | +| Baseline survey | `research/2026-07-24-baseline-field-survey.md` | +| Brief template + playbook | Present; first *daily* delta brief not yet written | +| Backup storage policy | Documented; **path/quota TBD** (FI-WP-0001-T04) | +| Inventory | Schema + policy + **P0/P1 catalog candidates** seeded | +| Activity-core | Definition drafted `enabled: false` (FI-WP-0002) | +| State Hub | `fi_daily_brief` contract documented; resolver not implemented | +| Workplans | FI-WP-0001 … 0003 active | + +--- + +## How It Fits + +| Direction | Systems | +| --------- | ------- | +| **Upstream** | Public model hubs, papers, pricing pages, release notes, harness projects | +| **Peer** | `llm-connect`, `sand-boxer`, `net-kingdom`, `disaster-control`, domain repos | +| **Downstream (future)** | Local inference/training facility, specialized domain agents, operator decision-making | +| **Distribution (optional)** | State Hub progress/inbox, operator mail/wiki | + +--- + +## Near-term deliverables + +1. ✅ `INTENT.md` / `SCOPE.md` / foundation docs +2. ✅ Brief template + playbook + sources allowlist +3. ✅ Baseline field survey + collection recommendations +4. ✅ Catalog candidates (P0/P1) +5. ✅ ActivityDefinition draft (disabled) +6. ⬜ Pin backup storage path and capacity budget (FI-WP-0001-T04) +7. ⬜ activity-core resolver + enable schedule (FI-WP-0002) +8. ⬜ Approve/download P0 reserve (FI-WP-0003) +9. ⬜ First daily delta brief (manual or after automation) + +--- + +## Terminology + +| Term | Meaning | +| ---- | ------- | +| **Brief** | Dated research digest under `briefs/` | +| **Axis A–D** | Standing monitoring lenses (frontier, edge/local, training, harness) | +| **Reserve** | Open-weight collection on backup storage | +| **Catalog entry** | In-repo inventory metadata for one model revision | +| **Collection candidate** | Brief-flagged artifact not yet approved/collected | +| **Hardware envelope** | Lab hosts that define “homelab accessible” for B/C | + +Preferred terms: research brief, open-weight reserve, inventory, collection policy. +Avoid: “mirror of Hugging Face”, “our AI product”, “always-on model zoo”. diff --git a/activity-definitions/fi-daily-research-brief.md b/activity-definitions/fi-daily-research-brief.md new file mode 100644 index 0000000..d9daaf5 --- /dev/null +++ b/activity-definitions/fi-daily-research-brief.md @@ -0,0 +1,61 @@ +--- +id: fi-daily-research-brief +name: Freedom Intelligence Daily Research Brief +enabled: false +owner: custodian-agent +governance: custodian +status: proposed +trigger: + type: cron + cron_expression: "30 7 * * 1-5" # weekdays 07:30 + timezone: Europe/Berlin + misfire_policy: skip +context_sources: + - type: state-hub + query: fi_brief_status + params: + repo: freedom-intelligence + bind_to: context.fi_brief +# Resolver (FI-WP-0002-T03, activity-core): expected shape +# {items: [{kind: "daily_brief", due: bool, date: "YYYY-MM-DD", +# last_run_at}]} — due is true when no fi_daily_brief progress event with +# detail.repo=freedom-intelligence exists for today (Europe/Berlin). +# Executing session must record that event on completion (idempotence). +--- + +# Freedom Intelligence Daily Research Brief + +Runs weekdays at 07:30 Berlin time. Emits one task when the daily research brief +for `freedom-intelligence` is due. Execution is **out of band**: consumer checks +out this repo and follows `docs/daily-brief-playbook.md` (axes A–D, collection +candidates, template under `briefs/`). + +activity-core only schedules and spawns. It does not call model APIs for the +brief body. + +Baseline field memory: `research/2026-07-24-baseline-field-survey.md`. Daily +briefs are **deltas** from prior briefs + that baseline — not full resurveys. + +```rule +id: emit-fi-daily-brief-task +for_each: context.fi_brief.items +bind_as: item +condition: 'context.item.due' +action: + task_template: "FI daily research brief ({context.item.kind}) for {context.item.date}" + description: > + Produce briefs/YYYY/MM/YYYY-MM-DD.md per docs/daily-brief-playbook.md and + briefs/_template.md. Cite primary sources. Flag collection candidates. + On completion post State Hub progress event_type=fi_daily_brief with + detail.repo=freedom-intelligence and detail.date. + target_repo: freedom-intelligence + priority: medium + labels: ["freedom-intelligence", "research-brief", "automated"] +``` + +## Enable checklist + +1. FI-WP-0002-T03 resolver live +2. FI-WP-0002-T04 external definition path registered +3. Dry-run spawn + one completed brief with `fi_daily_brief` event +4. Set `enabled: true` and `status: active` in this frontmatter diff --git a/briefs/README.md b/briefs/README.md new file mode 100644 index 0000000..755ee04 --- /dev/null +++ b/briefs/README.md @@ -0,0 +1,63 @@ +# Research briefs + +Daily curated digests of interesting progress in AI **models**, **methods**, +**harness**, and **tooling**, filtered through axes A–D. + +## Layout + +```text +briefs/ +├── README.md # this file +├── _template.md # copy for each new day +└── YYYY/ + └── MM/ + └── YYYY-MM-DD.md # one brief per calendar day (UTC or lab local — state in front matter) +``` + +Example path: `briefs/2026/07/2026-07-23.md` + +## Cadence + +| Mode | When | +| ---- | ---- | +| **Daily** | Preferred operating rhythm when the lab is active | +| **Skip empty days** | If there is no meaningful signal, either omit the file or write a short “no material delta” brief | +| **Weekly rollup** | Optional later; not required for foundation | + +First operating mode is **manual** (human or agent drafts from the template). Automation is out of scope until the template is validated by at least one real brief. + +## How to write a brief + +1. Copy `briefs/_template.md` to `briefs/YYYY/MM/YYYY-MM-DD.md`. +2. Fill only sections with signal; leave others as `*(none)*` rather than inventing news. +3. Cite primary sources (papers, model cards, release notes, repos, pricing pages). +4. Flag **collection candidates** with enough metadata for inventory review. +5. Keep **lab implications** short and actionable (or omit). + +## Design principles + +See `INTENT.md` and `history/InitialExploration.md`: + +- Signal over noise +- Actionable for the lab +- Price and access aware +- Reproducible sources +- Just-in-case bias for rare/high-leverage open weights + +## Baseline vs daily + +The **baseline field survey** is not a daily brief: + +- `research/2026-07-24-baseline-field-survey.md` + +Daily files under this tree are **deltas** from that baseline and prior dailies. +Playbook: `docs/daily-brief-playbook.md`. Sources: `docs/sources-allowlist.md`. + +## Axes reminder + +| Axis | Section in template | +| ---- | ------------------- | +| A | Frontier & commercial | +| B | Edge / local / open | +| C | Training & specialization | +| D | Harness & fleet | diff --git a/briefs/_template.md b/briefs/_template.md new file mode 100644 index 0000000..3267591 --- /dev/null +++ b/briefs/_template.md @@ -0,0 +1,78 @@ +--- +date: YYYY-MM-DD +timezone: Europe/Berlin # or UTC — pick one and keep consistent +author: # human or agent name +status: draft # draft | final +sources_checked: [] # optional list of channels scanned +--- + +# Freedom Intelligence Daily Brief — YYYY-MM-DD + +## Headline deltas + +What actually moved the needle today (3–7 bullets max). If nothing material: `*No material delta.*` + +- + +## Frontier & commercial (axis A) + +Capabilities, benchmarks, price moves, API access changes. + +| Item | Delta | Sources | Lab relevance | +| ---- | ----- | ------- | ------------- | +| | | | | + +*(none)* + +## Edge / local / open (axis B) + +Runnable intelligence at low cost; quantization; inference runtimes; licenses for local ops. + +| Item | Delta | Sources | Lab relevance | +| ---- | ----- | ------- | ------------- | +| | | | | + +*(none)* + +## Training & specialization (axis C) + +Pre-/mid-/post-training methods, adapters, datasets, homelab-fit recipes, domain eval. + +| Item | Delta | Sources | Lab relevance | +| ---- | ----- | ------- | ------------- | +| | | | | + +*(none)* + +## Harness & fleet (axis D) + +Agent frameworks, orchestration, sandboxing, multi-agent ops, fleet management. + +| Item | Delta | Sources | Lab relevance | +| ---- | ----- | ------- | ------------- | +| | | | | + +*(none)* + +## Collection candidates + +Open-weight models (or justified companions) worth parking on backup storage. +Use inventory field names where possible — see `inventory/schema.yaml`. + +| Candidate | Source / revision | License | Est. size | Why reserve | Priority | +| --------- | ----------------- | ------- | --------- | ----------- | -------- | +| | | | | | low / med / high | + +*(none)* + +## Lab implications + +Optional: concrete next actions for Freedom Intelligence or NetKingdom consumers. + +- + +## Notes / caveats + +Benchmark gaming, unconfirmed rumors, paywalled claims, etc. + +- diff --git a/docs/backup-storage-policy.md b/docs/backup-storage-policy.md new file mode 100644 index 0000000..af6f329 --- /dev/null +++ b/docs/backup-storage-policy.md @@ -0,0 +1,161 @@ +# Backup storage policy — open-weight model reserve + +**Status:** foundation draft (concrete path and budget **TBD**) +**Related:** `INTENT.md`, `SCOPE.md`, `inventory/collection-policy.md` +**Adjacent:** `disaster-control` BackupPolicy (platform backups — different concern) + +--- + +## Purpose + +Define where Freedom Intelligence stores **open-weight model blobs** (and justified +companions), how much capacity we allow, and how this reserve relates to other +lab backup facilities. + +Git never stores weight tensors. Git stores: + +* inventory catalog metadata (`inventory/catalog/`) +* policies and schemas +* research briefs + +--- + +## Storage class + +| Requirement | Policy | +| ----------- | ------ | +| **Class** | Backup / bulk durable storage — **not** hot cluster PVCs or app disks | +| **Durability** | Same or better retention posture as other lab bulk assets; prefer media that survives workstation rebuilds | +| **Performance** | Sequential read for restore/training pull is enough; low latency not required | +| **Access** | Operator and approved lab hosts only; not a public mirror | +| **Separation** | Do not co-mingle with age-encrypted operational backups (Forgejo dumps, k3s state) without a clear subdirectory and different lifecycle rules | + +Platform backup paths such as `/opt/backup/railiance/{infra,cluster}/` and +`~/.cache/railiance/backups/` are **operational recovery** lanes. The model +reserve may live on the **same physical facility** only if isolated by path and +quota so large weight pulls cannot crowd out restore media. + +--- + +## Target location (to pin) + +| Field | Value | +| ----- | ----- | +| **Host / facility** | `TBD` — operator pin (e.g. backup NAS, off-cluster bulk volume, dedicated disk) | +| **Base path or bucket** | `TBD` — suggested shape: `…/freedom-intelligence/models/` | +| **Layout under base** | See [On-disk layout](#on-disk-layout) | +| **Mount on lab hosts** | `TBD` | +| **Credentials** | `TBD` — if remote: OpenBao or existing backup credential lane; never commit secrets | + +Until pinned, **do not** bulk-download multi-GB models into this git workspace or +into hot root filesystems. + +### Suggested path shape (non-binding) + +```text +{BACKUP_ROOT}/freedom-intelligence/ +├── models/ +│ └── {org}__{name}/ +│ └── {revision}/ +│ ├── blobs/ # weight files, shards +│ └── MANIFEST.txt # optional local copy of hashes +├── companions/ # adapters, tokenizers only when separate +└── staging/ # incomplete downloads; not catalog-ready +``` + +Map each `{org}__{name}/{revision}` to an inventory catalog entry. + +--- + +## On-disk layout + +| Path element | Rule | +| ------------ | ---- | +| `org__name` | Hugging Face-style id with `/` → `__` | +| `revision` | Git commit SHA, tag, or release id used at download time | +| `blobs/` | Actual files; prefer original names from source | +| `staging/` | Incomplete transfers; purge or resume; never mark collected until complete + verified | + +--- + +## Capacity budget + +| Parameter | Policy | +| --------- | ------ | +| **Soft quota** | `TBD` GiB/TiB — operator pin based on free backup capacity | +| **Hard stop** | No new collection when soft quota exceeded unless operator raises budget | +| **Per-pull threshold** | See `inventory/collection-policy.md` (size gates approval) | +| **Growth review** | Revisit quota when catalog total exceeds 70% of soft quota | +| **Eviction** | Prefer archive/delete lowest-priority, easily re-obtainable revisions first; record eviction in catalog history | + +--- + +## Integrity and provenance + +For every completed collection: + +1. Record source URL and revision in the catalog entry. +2. Store checksums (`sha256` of each blob or upstream manifest digest). +3. Record download date (UTC) and downloader identity. +4. Prefer official org releases over anonymous re-uploads. +5. Keep license text or SPDX id in catalog; refuse unclear licenses. + +Verification command examples belong in tooling later; policy only requires that +**catalog claims match on-disk checksums** before status `collected`. + +--- + +## Retention + +| Class | Retention | +| ----- | --------- | +| **Strategic reserve** (high priority, hard to re-obtain) | Keep until explicit deprecation | +| **Working set** (common bases for training experiments) | Keep while in active use + one superseded revision optional | +| **Staging** | Max 14 days incomplete, then purge | +| **Deprecated** | Metadata retained in catalog with status `evicted` or `superseded`; blobs may be deleted | + +--- + +## Encryption and offsite + +| Topic | Policy | +| ----- | ------ | +| **At rest** | Follow host/facility default; extra age/GPG of multi-hundred-GB trees is optional and costly | +| **In transit** | HTTPS or trusted lab network only | +| **Offsite copy** | Optional later; not required for foundation. If added, coordinate with `disaster-control` so model reserve does not break operational backup SLAs | + +--- + +## What must not live here + +* Closed weights or artifacts whose terms forbid offline retention +* Secrets, API keys, customer data, or training corpora with personal data (domain datasets need their own policy) +* Operational backups (databases, k3s state, Forgejo dumps) +* Git LFS dumps of full model trees as a substitute for backup storage + +--- + +## Operator checklist to pin TBD fields + +- [ ] Choose facility and base path +- [ ] Confirm free capacity and set soft quota +- [ ] Document mount/access for lab hosts +- [ ] Confirm credential path (if any) +- [ ] Create empty `models/` and `staging/` directories +- [ ] Update this file: replace `TBD` rows with concrete values +- [ ] Note pin date and operator in revision history below + +### Pin log + +| Date | Operator | Change | +| ---- | -------- | ------ | +| 2026-07-23 | foundation | Policy created; path and quota unpinned | + +--- + +## Relationship to disaster-control + +`disaster-control` owns **platform resilience** (what to restore after loss). +Freedom Intelligence owns **which open weights we choose to retain** and their +inventory. Shared facilities are fine; shared lifecycle rules are not automatic — +model reserve is large, slow-changing, and rarely needed for emergency restore. diff --git a/docs/daily-brief-playbook.md b/docs/daily-brief-playbook.md new file mode 100644 index 0000000..9ac29ad --- /dev/null +++ b/docs/daily-brief-playbook.md @@ -0,0 +1,111 @@ +# Daily research brief — playbook + +**Audience:** operator or agent executing `target_repo: freedom-intelligence` +**Template:** `briefs/_template.md` +**Baseline memory:** `research/2026-07-24-baseline-field-survey.md` +**Sources:** `docs/sources-allowlist.md` + +--- + +## Purpose + +Produce **one** curated brief per active day covering axes A–D when there is +signal. Briefs are **deltas** from the baseline survey and prior briefs — not +full resurveys. + +--- + +## Idempotence + +| Rule | Detail | +| ---- | ------ | +| One brief per date | Path `briefs/YYYY/MM/YYYY-MM-DD.md` | +| Calendar | Prefer **Europe/Berlin** date in front matter | +| Skip empty days | Either omit file or write `*No material delta.*` in Headline deltas | +| Automation due bit | Cleared only by State Hub event `fi_daily_brief` (below) | + +--- + +## Manual procedure + +1. **Orient** + - Read last 1–2 briefs if any; skim baseline § executive snapshot if first week + - `git pull` on freedom-intelligence + +2. **Scan** (`docs/sources-allowlist.md`) — 30–60 minutes cap + - Capture only items that change capability, cost, access, or operability + +3. **Draft** + ```bash + mkdir -p briefs/$(date +%Y/%m) + cp briefs/_template.md briefs/$(date +%Y/%m)/$(date +%Y-%m-%d).md + ``` + - Fill front matter (`status: final` when done) + - Leave empty sections as `*(none)*` + - Cite primary URLs + +4. **Collection candidates** + - Use inventory field names; set priority + - If nominating: add or update `inventory/catalog/{id}.yaml` with + `status: candidate` and `collection.brief_refs` + +5. **Lab implications** + - Link open workplans (`FI-WP-*`) only when action is real + +6. **Commit** (operator policy) — brief + any catalog YAML + +7. **Completion evidence** (required for activity-core rhythm) + + Post State Hub progress (HTTP or MCP), shape: + + ```yaml + event_type: fi_daily_brief + author: + summary: "FI daily brief YYYY-MM-DD" + detail: + repo: freedom-intelligence + date: "YYYY-MM-DD" + path: "briefs/YYYY/MM/YYYY-MM-DD.md" + collection_candidates: 0 + ``` + + Workstation API often `http://127.0.0.1:8000`; remote tunnel ports per + Custodian docs. See `docs/state-hub-delivery.md`. + +--- + +## Automation consumer + +When activity-core emits `activity_task_spawn` with +`target_repo: freedom-intelligence` and labels including `research-brief`: + +1. Checkout this repo on the execution host +2. Run this playbook end-to-end +3. Prefer **llm-connect / OpenRouter** for any LLM drafting (Binky lesson: do + not install coding-agent CLIs on railiance01 solely for this) +4. Always post `fi_daily_brief` even if the brief is “no material delta” +5. Idempotency key: do not open a second brief file for the same `date` + +Activity definition: `activity-definitions/fi-daily-research-brief.md` +Workplan: `workplans/FI-WP-0002-activity-core-daily-research.md` + +--- + +## Quality bar + +| Check | Pass | +| ----- | ---- | +| Signal | No filler paragraphs | +| Sources | At least one primary link per non-empty axis row | +| Honesty | Benchmarks marked directional; prices “verify before budget” | +| Collection | License noted for any candidate | +| Length | Prefer one screen to three; deep dives go under `research/` | + +--- + +## First daily after baseline + +The baseline survey (2026-07-24) already covers the field. The **first daily** +should only list changes **after** that document’s research window. If nothing +moved, a short no-delta brief still completes the rhythm and posts +`fi_daily_brief`. diff --git a/docs/hardware-envelope.md b/docs/hardware-envelope.md new file mode 100644 index 0000000..174e878 --- /dev/null +++ b/docs/hardware-envelope.md @@ -0,0 +1,65 @@ +# Hardware envelope — homelab accessible (draft) + +**Status:** assumptions + TBD measurements +**Used by:** inventory `hardware_class`, axis B/C prioritization, collection size decisions + +--- + +## Purpose + +Define what Freedom Intelligence means by **homelab accessible** so we do not +collect models that only make sense on hyperscale clusters. + +--- + +## Working tiers (until hosts are measured) + +| Tier | Assumed capacity | Target model class | +| ---- | ---------------- | ------------------ | +| **T0 — CPU / edge** | 16–64 GB system RAM, no GPU | ≤3B Q4; embeddings ≤0.5B | +| **T1 — consumer GPU** | 8–12 GB VRAM | 7–8B Q4/Q5; 3B fp16 | +| **T2 — enthusiast GPU** | 16–24 GB VRAM | 14B Q4; 8B fp16; light 32B Q4 | +| **T3 — lab multi-GPU** | 2×24 GB+ or 48 GB+ | 32B fp16 / 70B Q4; small MoE | +| **T4 — out of envelope** | multi-node / 8×A100-class | Full DeepSeek-V3 MoE — P2 watch only | + +P0 collection targets **T0–T2**. P2 full MoE is **T4**. + +--- + +## Lab hosts (fill in) + +| Host | Role | GPU | VRAM | RAM | Notes | +| ---- | ---- | --- | ---- | --- | ----- | +| TBD | primary local inference | TBD | TBD | TBD | | +| TBD | training experiments | TBD | TBD | TBD | | +| railiance01 | cluster (not weight store) | n/a | n/a | n/a | Prefer not to fill hot disks with weights | + +--- + +## Inference runtime defaults (intent) + +| Runtime | When | +| ------- | ---- | +| llama.cpp / GGUF | Desktop & edge | +| vLLM | Server multi-request | +| MLX | Apple silicon if present | +| Ollama | Quick operator UX only | + +--- + +## Training defaults (intent) + +| Method | Envelope | +| ------ | -------- | +| QLoRA 7–8B | T1–T2 | +| QLoRA 14B | T2 | +| Full FT 7B | T2–T3 | +| 70B+ FT | T3+ only with explicit plan | + +--- + +## Pin log + +| Date | Change | +| ---- | ------ | +| 2026-07-24 | Draft tiers; host table empty | diff --git a/docs/sources-allowlist.md b/docs/sources-allowlist.md new file mode 100644 index 0000000..d365639 --- /dev/null +++ b/docs/sources-allowlist.md @@ -0,0 +1,62 @@ +# Sources allowlist — daily research + +Standing channels for Freedom Intelligence briefs (axes A–D). Prefer **primary** +sources over aggregators. This is a watchlist, not a scrape mandate. + +--- + +## Axis A — Frontier & commercial + +| Channel | What to extract | +| ------- | --------------- | +| OpenAI / Anthropic / Google / xAI / DeepSeek blogs & release notes | Capability claims, deprecations | +| Official pricing pages | $/MTok in/out, cache, batch | +| Model cards for API models | Context limits, tool use, policies | +| LMSYS / Arena / artificialanalysis-class charts | Directional only; note gaming risk | +| SWE-bench Verified / Live leaderboards | Harness+model pairs, not raw IQ | + +## Axis B — Edge / local / open + +| Channel | What to extract | +| ------- | --------------- | +| Hugging Face org feeds: `Qwen`, `deepseek-ai`, `meta-llama`, `google`, `mistralai`, `BAAI`, `nomic-ai` | New weights, licenses, sizes | +| llama.cpp, vLLM, MLX, Ollama release notes | Runtime breakthroughs | +| GGUF / official quant announcements | Only after base model known | + +## Axis C — Training & specialization + +| Channel | What to extract | +| ------- | --------------- | +| arXiv cs.LG, cs.CL, cs.AI (recent) | Methods that fit homelab | +| Unsloth, Hugging Face TRL, Axolotl, LLaMA-Factory | Practical recipes | +| Preference-optimization papers/releases | DPO/ORPO/RLVR-class | + +## Axis D — Harness & fleet + +| Channel | What to extract | +| ------- | --------------- | +| OpenHands, Aider, SWE-agent, OpenCode, Cline repos/releases | Harness capability | +| sand-boxer / agent-harness / activity-core (internal) | Coulomb adjacency | +| MCP / tool-protocol standards | Interop shifts | + +## Internal memory + +| Artifact | Role | +| -------- | ---- | +| `research/2026-07-24-baseline-field-survey.md` | Baseline; dailies are deltas | +| Prior `briefs/YYYY/MM/*.md` | Avoid duplicate headlines | +| `inventory/catalog/` | Collection status | + +## Do not + +- Treat Twitter/X rumor as confirmed without primary link +- Mirror entire HF orgs +- Cite paywalled claims without noting access limit +- Collect weights that fail the license gate (`inventory/collection-policy.md`) +- Store API keys or customer data in briefs + +## Cadence hint + +For a 30–60 minute manual brief: scan A pricing+releases (10m), B HF/org +releases (10m), C one methods thread (10m), D harness releases (10m), write +(15m). Skip empty sections. diff --git a/docs/state-hub-delivery.md b/docs/state-hub-delivery.md new file mode 100644 index 0000000..18e193a --- /dev/null +++ b/docs/state-hub-delivery.md @@ -0,0 +1,62 @@ +# State Hub / operator delivery + +**Status:** contract defined; optional distribution not fully wired +**Related:** `docs/daily-brief-playbook.md`, `workplans/FI-WP-0002-activity-core-daily-research.md` + +--- + +## Required: completion evidence for daily rhythm + +Every completed brief day (including “no material delta”) should post: + +```yaml +event_type: fi_daily_brief +author: +summary: "FI daily brief YYYY-MM-DD" +detail: + repo: freedom-intelligence + date: "YYYY-MM-DD" + path: "briefs/YYYY/MM/YYYY-MM-DD.md" + collection_candidates: 0 +``` + +This is the **idempotence signal** for activity-core context query +`fi_brief_status` (FI-WP-0002-T03): `due=false` when an event exists for today +(Europe/Berlin) with `detail.repo=freedom-intelligence`. + +Inbound automation uses `activity_task_spawn` from activity-core (state-hub sink). +See `activity-core/docs/task-emission-consumer-contract.md`. + +--- + +## Optional: operator visibility + +| Channel | Use | +| ------- | --- | +| **Progress event** (above) | Minimum — always for automation health | +| **Inbox message** | Only when high-priority collection candidates appear | +| **Wiki / Nextcloud** | Optional human mirror later | + +--- + +## Non-goals + +* Freedom Intelligence is **not** required to be a full State Hub domain yet +* Do not use `ISSUE_SINK_TYPE=rest` / Forgejo issues for this rhythm +* Do not register workplans by hand in the hub (`fix-consistency` after commit) + +--- + +## Fallback without MCP + +HTTP against State Hub API (workstation `http://127.0.0.1:8000` or remote +tunnel ports). Check health before declaring hub down. + +--- + +## Pin log + +| Date | Change | +| ---- | ------ | +| 2026-07-23 | Stub created | +| 2026-07-24 | `fi_daily_brief` contract + activity-core linkage documented | diff --git a/history/InitialExploration.md b/history/InitialExploration.md new file mode 100644 index 0000000..80d3acd --- /dev/null +++ b/history/InitialExploration.md @@ -0,0 +1,256 @@ +# Freedom Intelligence — Initial Exploration + +**Date:** 2026-07-23 +**Repo:** `freedom-intelligence` +**Scope affiliation:** Coulomb / NetKingdom +**Status:** founding conversation recorded +**One-liner (repo):** *The just in case open weights model lab* + +--- + +## Why this document exists + +This is a greenfield project. There is no INTENT.md, SCOPE.md, or workplan yet — +only a README, LICENSE (MIT, © 2026 coulomb), and git scaffolding. + +This file captures the **founding conversation** that defines why the lab exists, +what it watches, and what durable assets it should accumulate. Later INTENT/SCOPE +and workplans should refine from here, not reinvent the charter. + +--- + +## Founding intent + +Establish a standing capability to: + +1. **Produce a daily research brief** on interesting progress in the AI models, + methods, harness, and tooling space. +2. **Collect and retain open-weight models** on lab backup storage so they remain + available for possible future local training or operations — *just in case*. + +The lab sits inside the broader Coulomb NetKingdom landscape: secure, inspectable, +homelab- and fleet-accessible infrastructure where agentic systems and open +security tooling already grow. Freedom Intelligence is the **intelligence sensing +and model-reserve arm** of that landscape — not a product UI, not a cloud SaaS +wrapper, and not a commitment to run every frontier model locally. + +--- + +## Daily research brief — mandate + +The primary operating rhythm is a **daily research brief**: a curated digest of +what moved the needle, not a raw news firehose. + +### What “interesting progress” means here + +Progress that changes (or could soon change) **capability, cost, accessibility, +or operability** of intelligence systems we care about — especially where those +systems can be run, specialized, orchestrated, or reserved under our own control. + +### Brief design principles (initial) + +| Principle | Meaning | +| --- | --- | +| **Signal over noise** | Prefer releases, benchmarks, methods, and tooling shifts that matter for A–D below | +| **Actionable for the lab** | Note whether something is relevant to monitoring, local/edge use, training, or harness work | +| **Price and access aware** | Track not only SOTA quality but cost, license, and where weights can be obtained | +| **Reproducible sources** | Cite primary sources (papers, model cards, release notes, repos, pricing pages) | +| **Durable archive** | Briefs live in-repo (or linked inventory); they form a longitudinal memory of the field | +| **Just-in-case bias** | When an open-weight artifact is rare, high-leverage, or at risk of vanishing, flag for collection | + +### Suggested brief sections (starting shape) + +Not every section every day — only when there is signal: + +1. **Headline deltas** — what actually changed +2. **Frontier & commercial** — capabilities, benchmarks, price moves (axis A) +3. **Edge / local / open** — runnable intelligence at low cost (axis B) +4. **Training & specialization** — methods and tooling for pre-/post-/domain work (axis C) +5. **Harness & fleet** — agents, orchestration, sandboxing, multi-agent ops (axis D) +6. **Collection candidates** — open-weight models or datasets worth parking on backup storage +7. **Lab implications** — optional: what (if anything) the lab should do next + +Format, cadence automation, and distribution (inbox, wiki, State Hub event) remain +open design choices. + +--- + +## Special interest axes + +The monitoring and reporting charter is multifold. These four axes are the standing +lens for both the daily brief and longer research notes. + +### A — Frontier model capabilities, benchmarks, and price trends + +**Watch:** + +- Best-of-class closed and open frontier models (reasoning, code, multimodal, long context, tool use) +- Benchmark suites and their gaming/validity debates (e.g. SWE-bench, Arena, GPQA, internal harness scores) +- API price-per-token and effective cost-per-useful-task trends +- Latency, rate limits, regional availability, and enterprise terms that affect operational use +- Distillation / smaller “near-frontier” offerings that shift the price–capability curve + +**Lab value:** know when paid frontier is the right tool, when open alternatives close the gap, +and when price collapses make new workflows economical. + +### B — Edge intelligence and low-cost open / local compute + +**Watch:** + +- Small and mid-size open-weight models that run on consumer or lab GPUs, NPUs, and CPUs +- Quantization, speculative decoding, KV-cache, and inference runtimes (vLLM, llama.cpp, MLX, TensorRT-LLM, etc.) +- Hardware-accessible stacks: single-box homelab, mini-PC, older datacenter cards, Apple silicon, edge SBCs +- Specializations: code, RAG, embeddings, rerankers, speech, vision-on-device +- Licensing that actually allows local ops and redistribution of derived artifacts + +**Lab value:** independence from continuous cloud spend; privacy; offline and sovereign operation; +honest capability floors for NetKingdom-local agents. + +### C — Homelab-accessible training and domain specialization + +**Watch:** + +- Pre-training, continued pre-training, mid-training, SFT, preference optimization (DPO/ORPO/…), RL/verifiers +- LoRA / QLoRA / adapters, mixture-of-experts fine-tunes, model merging, unsloth-class efficiency tricks +- Datasets and data pipelines that fit homelab budgets (synthetic data, filtering, domain corpora) +- Recipes that fit **our** hardware class — not only hyperscale recipes +- Evaluation for domain specialization (not only public leaderboards) + +**Lab value:** ability to specialize open bases for Coulomb / NetKingdom domains +(security, ops, railiance, tenant workflows, etc.) without assuming a full training cluster. + +### D — Agent harnesses, orchestration, sandboxing, and fleet management + +**Watch:** + +- Agent frameworks and harnesses (tool loops, planning, memory, multi-agent protocols) +- Orchestration of long-running and multi-step work (workplans, fleets of workers, queues) +- Sandboxing and containment of agent side effects (filesystem, network, credentials, blast radius) +- Fleet management: provisioning, identity, observability, cost control, policy for many agents +- Adjacent NetKingdom capabilities already in the landscape (e.g. sand-boxer, ops-warden, state-hub, kaizen agents) + +**Lab value:** intelligence is useless without safe, operable delivery. This axis ties model progress +to how NetKingdom actually runs agents. + +--- + +## Open-weight model collection (backup storage reserve) + +### Purpose + +Use **lab backup storage facilities** to build a **deliberate collection of open-weight models** +(and, where justified, companion tokenizers, adapters, and evaluation fixtures) that we might +later reuse in a **local training or operations facility**. + +This is a **reserve**, not a mirror of the entire Hub: + +- Prefer models that score high on axes B/C or that are strategically useful even if large +- Prefer clear open licenses and stable provenance (Hugging Face, official org releases, mirrors with checksums) +- Prefer artifacts that are hard to re-obtain later (takedowns, license flips, silent re-uploads) +- Store enough metadata to rehydrate: source URL, revision/hash, license, size, hardware class, brief notes + +### Collection principles (initial) + +| Principle | Detail | +| --- | --- | +| **Just in case** | Collection is insurance and optionality, not an obligation to serve every model | +| **Backup-class storage** | Place weights on facilities already intended for durable bulk retention; do not fill hot cluster disks | +| **Catalog in-repo or adjacent** | Git holds inventory/metadata; blobs stay on backup media | +| **Integrity** | Checksums, signed manifests where available, recorded download date and source revision | +| **License gate** | Only collect what license and terms allow for offline retention and intended reuse | +| **Prioritization** | Daily brief feeds candidates; human or workplan approval gates large pulls | +| **Reuse paths** | Future consumers: local inference, fine-tuning experiments, offline eval, air-gapped ops | + +### What is explicitly out of scope (for now) + +- Hosting a public model mirror or CDN +- Guaranteeing 24/7 inference for the full collection +- Training from scratch at frontier scale +- Collecting closed weights or scraping in violation of terms + +--- + +## Relationship to Coulomb / NetKingdom + +| Concern | Freedom Intelligence role | Adjacent systems (examples) | +| --- | --- | --- | +| Field sensing | Daily brief + research notes | State Hub progress/inbox for distribution later | +| Model reserve | Open-weight collection on backup storage | disaster-control / backup facilities; storage ops | +| Local intelligence | Informs what can run on lab hardware | sand-boxer, agent harnesses, local runtimes | +| Security posture | Prefers inspectable open stacks | net-kingdom, kings-guard, openbao identity patterns | +| Domain specialization | Future fine-tunes for Coulomb domains | domain repos that need specialized assistants | + +Freedom Intelligence does **not** replace NetKingdom security or ops products. It supplies +**awareness and optional local intelligence assets** those systems may eventually consume. + +--- + +## Current repo state + +**Founding (2026-07-23):** LICENSE, README, this exploration only. + +**Foundation files initiated (same day):** + +```text +freedom-intelligence/ +├── INTENT.md +├── SCOPE.md +├── README.md +├── LICENSE +├── briefs/ +│ ├── README.md +│ └── _template.md +├── inventory/ +│ ├── README.md +│ ├── schema.yaml +│ ├── collection-policy.md +│ └── catalog/ +├── docs/ +│ ├── backup-storage-policy.md # path/quota TBD +│ └── state-hub-delivery.md # stub, not wired +└── history/ + └── InitialExploration.md +``` + +Still open: first production brief, pinned backup path/quota, catalog entries, automation. + +--- + +## Open questions for the next design pass + +1. **Brief cadence and owner** — fully automated scrape + human edit, agent-produced draft, or operator-written? +2. **Brief storage layout** — e.g. `briefs/YYYY/MM/YYYY-MM-DD.md` vs weekly rollups? +3. **Sources allowlist** — which sites, arXiv categories, HF orgs, pricing pages, and X/GitHub signals matter first? +4. **Backup storage location** — exact path/share/bucket, retention, encryption, and capacity budget for weights? +5. **Inventory schema** — YAML/JSON catalog fields for each collected model revision? +6. **Hardware envelope** — what GPUs/CPU hosts define “homelab accessible” for axes B and C? +7. **INTENT/SCOPE split** — formalize aspirational vs near-term deliverables after first brief prototype? +8. **State Hub registration** — when (if) this domain/repo becomes a tracked hub domain or remains a satellite under netkingdom/coulomb? + +--- + +## Suggested near-term follow-ups + +| Priority | Action | Status | +| --- | --- | --- | +| 1 | Draft `INTENT.md` / `SCOPE.md` from this charter | **Done** (foundation) | +| 2 | Define brief template + layout under `briefs/` | **Done** (template; first filled brief still open) | +| 3 | Document backup storage target and capacity policy | **Done** (policy; path/quota TBD pin) | +| 4 | Define model inventory schema and collection approval rule of thumb | **Done** | +| 5 | Optionally wire brief delivery into operator inbox / State Hub | **Stub only** (`docs/state-hub-delivery.md`) | + +--- + +## Central proposition + +> **Freedom Intelligence is the Coulomb NetKingdom lab for watching the AI capability frontier +> with a bias toward open, local, and operable intelligence — and for holding a durable +> open-weight reserve so useful models remain available when we need to train or run them ourselves.** + +The daily research brief is the **sensing loop**. +The open-weight collection on backup storage is the **strategic reserve**. +Axes A–D keep both loops pointed at capability, cost, specialization, and safe agent operations. + +--- + +*Recorded from the founding project conversation, 2026-07-23.* diff --git a/inventory/README.md b/inventory/README.md new file mode 100644 index 0000000..158d068 --- /dev/null +++ b/inventory/README.md @@ -0,0 +1,32 @@ +# Model inventory + +In-repo **catalog of open-weight revisions** reserved (or nominated) for the lab. +Weight blobs live on backup storage — see `docs/backup-storage-policy.md`. + +## Layout + +```text +inventory/ +├── README.md # this file +├── schema.yaml # field reference + example +├── collection-policy.md # approval and eligibility rules +└── catalog/ + └── *.yaml # one file per model revision +``` + +## Status values + +`candidate` → `approved` → `collecting` → `collected` → `verified` +also: `superseded` | `evicted` | `rejected` + +## Adding an entry + +1. Confirm eligibility in `collection-policy.md`. +2. Create `catalog/{org}__{name}__{short_revision}.yaml` using fields from `schema.yaml`. +3. Set `status: candidate` (or `approved` if already signed off). +4. After download and checksums: set `collected` / `verified` and `collection.storage_path`. + +## Empty catalog + +The catalog starts empty on purpose. First entries come from daily brief +**collection candidates** after policy checks — not from bulk scraping. diff --git a/inventory/catalog/.gitkeep b/inventory/catalog/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/inventory/catalog/BAAI__bge-m3__candidate.yaml b/inventory/catalog/BAAI__bge-m3__candidate.yaml new file mode 100644 index 0000000..6daeec9 --- /dev/null +++ b/inventory/catalog/BAAI__bge-m3__candidate.yaml @@ -0,0 +1,42 @@ +id: BAAI__bge-m3__candidate +status: candidate +name: bge-m3 +org: BAAI +source: + kind: huggingface + url: https://huggingface.co/BAAI/bge-m3 + revision: main + model_card_url: https://huggingface.co/BAAI/bge-m3 +license: + spdx: MIT + url: https://huggingface.co/BAAI/bge-m3 + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "" +size: + total_bytes: 0 + total_human: "~2 GB" +hardware_class: + min_vram_gb_q4: 1 + min_vram_gb_fp16: 2 + notes: "Runs on CPU comfortably for many workloads" +axes: [B] +priority: high +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P0 multilingual embedding staple for local RAG (docs, ops notes, DE/EN)." +tags: [embedding, multilingual, rag] +companions: [] +notes: "Small download — within agent auto-collect band after license check once storage pinned." +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P0 recommendation from initial deep research." diff --git a/inventory/catalog/BAAI__bge-reranker-v2-m3__candidate.yaml b/inventory/catalog/BAAI__bge-reranker-v2-m3__candidate.yaml new file mode 100644 index 0000000..b9a975e --- /dev/null +++ b/inventory/catalog/BAAI__bge-reranker-v2-m3__candidate.yaml @@ -0,0 +1,43 @@ +id: BAAI__bge-reranker-v2-m3__candidate +status: candidate +name: bge-reranker-v2-m3 +org: BAAI +source: + kind: huggingface + url: https://huggingface.co/BAAI/bge-reranker-v2-m3 + revision: main + model_card_url: https://huggingface.co/BAAI/bge-reranker-v2-m3 +license: + spdx: Apache-2.0 + url: https://huggingface.co/BAAI/bge-reranker-v2-m3 + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "Confirm SPDX on card." +size: + total_bytes: 0 + total_human: "~1–2 GB (estimate)" +hardware_class: + min_vram_gb_q4: 1 + min_vram_gb_fp16: 2 + notes: "Companion to bge-m3" +axes: [B] +priority: medium +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P1 — cheap RAG quality win as companion to BGE-M3." +tags: [reranker, rag] +companions: + - BAAI__bge-m3__candidate +notes: "" +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P1 recommendation from initial deep research." diff --git a/inventory/catalog/Qwen__Qwen3-14B__candidate.yaml b/inventory/catalog/Qwen__Qwen3-14B__candidate.yaml new file mode 100644 index 0000000..7391f73 --- /dev/null +++ b/inventory/catalog/Qwen__Qwen3-14B__candidate.yaml @@ -0,0 +1,42 @@ +id: Qwen__Qwen3-14B__candidate +status: candidate +name: Qwen3-14B +org: Qwen +source: + kind: huggingface + url: https://huggingface.co/Qwen/Qwen3-14B + revision: main + model_card_url: https://huggingface.co/Qwen/Qwen3-14B +license: + spdx: Apache-2.0 + url: https://huggingface.co/Qwen/Qwen3-14B + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "Confirm card at download." +size: + total_bytes: 0 + total_human: "~28 GB fp16 / ~9 GB Q4 (estimate)" +hardware_class: + min_vram_gb_q4: 10 + min_vram_gb_fp16: 28 + notes: "T2 quality step" +axes: [B, C] +priority: medium +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P1 — stronger single-GPU chat/code when quota allows after P0." +tags: [instruct, qwen3] +companions: [] +notes: "" +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P1 recommendation from initial deep research." diff --git a/inventory/catalog/Qwen__Qwen3-8B__candidate.yaml b/inventory/catalog/Qwen__Qwen3-8B__candidate.yaml new file mode 100644 index 0000000..49e14ae --- /dev/null +++ b/inventory/catalog/Qwen__Qwen3-8B__candidate.yaml @@ -0,0 +1,42 @@ +id: Qwen__Qwen3-8B__candidate +status: candidate +name: Qwen3-8B +org: Qwen +source: + kind: huggingface + url: https://huggingface.co/Qwen/Qwen3-8B + revision: main + model_card_url: https://huggingface.co/Qwen/Qwen3-8B +license: + spdx: Apache-2.0 + url: https://huggingface.co/Qwen/Qwen3-8B + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "Confirm exact card license at download time; Qwen3 line generally Apache-2.0." +size: + total_bytes: 0 + total_human: "~16 GB fp16 / ~5 GB Q4 (estimate)" +hardware_class: + min_vram_gb_q4: 6 + min_vram_gb_fp16: 16 + notes: "Default T1–T2 general instruct and FT base" +axes: [B, C] +priority: high +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P0 spine — best default open general/tool model for local ops and QLoRA domain specialization." +tags: [instruct, text, qwen3, ft-base] +companions: [] +notes: "Prefer Instruct variant on card if separate repo; pin commit SHA at collection." +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P0 recommendation from initial deep research." diff --git a/inventory/catalog/deepseek-ai__DeepSeek-R1-Distill-Qwen-14B__candidate.yaml b/inventory/catalog/deepseek-ai__DeepSeek-R1-Distill-Qwen-14B__candidate.yaml new file mode 100644 index 0000000..a903efb --- /dev/null +++ b/inventory/catalog/deepseek-ai__DeepSeek-R1-Distill-Qwen-14B__candidate.yaml @@ -0,0 +1,42 @@ +id: deepseek-ai__DeepSeek-R1-Distill-Qwen-14B__candidate +status: candidate +name: DeepSeek-R1-Distill-Qwen-14B +org: deepseek-ai +source: + kind: huggingface + url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B + revision: main + model_card_url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B +license: + spdx: MIT + url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "R1 distill series MIT — confirm card at pin time." +size: + total_bytes: 0 + total_human: "~28 GB fp16 / ~9 GB Q4 (estimate)" +hardware_class: + min_vram_gb_q4: 10 + min_vram_gb_fp16: 28 + notes: "T2 Q4 preferred; fall back to 8B distill if VRAM tight" +axes: [B, C] +priority: high +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P0 local reasoning without full R1 MoE — agent/tool loops and harder offline tasks." +tags: [reasoning, distill, deepseek] +companions: [] +notes: "If disk/VRAM constrained, substitute DeepSeek-R1-Distill-Qwen-8B as P0 alternate." +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P0 recommendation from initial deep research." diff --git a/inventory/catalog/deepseek-ai__DeepSeek-R1-Distill-Qwen-32B__candidate.yaml b/inventory/catalog/deepseek-ai__DeepSeek-R1-Distill-Qwen-32B__candidate.yaml new file mode 100644 index 0000000..09b65af --- /dev/null +++ b/inventory/catalog/deepseek-ai__DeepSeek-R1-Distill-Qwen-32B__candidate.yaml @@ -0,0 +1,42 @@ +id: deepseek-ai__DeepSeek-R1-Distill-Qwen-32B__candidate +status: candidate +name: DeepSeek-R1-Distill-Qwen-32B +org: deepseek-ai +source: + kind: huggingface + url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B + revision: main + model_card_url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B +license: + spdx: MIT + url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "" +size: + total_bytes: 0 + total_human: "~65 GB fp16 / ~20 GB Q4 (estimate)" +hardware_class: + min_vram_gb_q4: 20 + min_vram_gb_fp16: 64 + notes: "T2–T3; only if hardware envelope supports" +axes: [B, C] +priority: medium +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P1 stronger local reasoner — approve only with VRAM + quota headroom." +tags: [reasoning, distill, deepseek] +companions: [] +notes: "Do not collect full DeepSeek-V3/R1 MoE under this id." +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P1 recommendation from initial deep research." diff --git a/inventory/catalog/meta-llama__Llama-3.2-3B-Instruct__candidate.yaml b/inventory/catalog/meta-llama__Llama-3.2-3B-Instruct__candidate.yaml new file mode 100644 index 0000000..31061b4 --- /dev/null +++ b/inventory/catalog/meta-llama__Llama-3.2-3B-Instruct__candidate.yaml @@ -0,0 +1,42 @@ +id: meta-llama__Llama-3.2-3B-Instruct__candidate +status: candidate +name: Llama-3.2-3B-Instruct +org: meta-llama +source: + kind: huggingface + url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct + revision: main + model_card_url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct +license: + spdx: custom + url: https://ai.meta.com/llama/license/ + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "Llama Community License — not MIT; review terms before commercial redistribution." +size: + total_bytes: 0 + total_human: "~6 GB fp16 / ~2 GB Q4 (estimate)" +hardware_class: + min_vram_gb_q4: 3 + min_vram_gb_fp16: 8 + notes: "T0–T1 edge / always-on" +axes: [B] +priority: high +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P0 edge instruct — tiny, huge ecosystem, good CPU/GPU floor for agents." +tags: [instruct, edge, llama] +companions: [] +notes: "HF gated model — need accepted license on account before download." +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P0 recommendation from initial deep research." diff --git a/inventory/catalog/nomic-ai__nomic-embed-text-v1.5__candidate.yaml b/inventory/catalog/nomic-ai__nomic-embed-text-v1.5__candidate.yaml new file mode 100644 index 0000000..d06399a --- /dev/null +++ b/inventory/catalog/nomic-ai__nomic-embed-text-v1.5__candidate.yaml @@ -0,0 +1,42 @@ +id: nomic-ai__nomic-embed-text-v1.5__candidate +status: candidate +name: nomic-embed-text-v1.5 +org: nomic-ai +source: + kind: huggingface + url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 + revision: main + model_card_url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 +license: + spdx: Apache-2.0 + url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 + allows_offline_retention: true + allows_local_ops: true + allows_fine_tune: true + notes: "" +size: + total_bytes: 0 + total_human: "<1 GB" +hardware_class: + min_vram_gb_q4: 1 + min_vram_gb_fp16: 1 + notes: "CPU-friendly" +axes: [B] +priority: high +collection: + approved_by: "" + approved_at: null + downloaded_at: null + downloaded_by: "" + storage_path: "" + brief_refs: + - research/2026-07-24-baseline-field-survey.md +reason: "P0 lightweight embed for A/B with BGE-M3; long-context text retrieval." +tags: [embedding, rag] +companions: [] +notes: "If v2 text is preferred at collection time, update id/url and supersede this candidate." +history: + - at: "2026-07-24" + event: nominated + by: baseline-survey + detail: "P0 recommendation from initial deep research." diff --git a/inventory/collection-policy.md b/inventory/collection-policy.md new file mode 100644 index 0000000..d380715 --- /dev/null +++ b/inventory/collection-policy.md @@ -0,0 +1,134 @@ +# Collection policy — open-weight reserve + +**Status:** foundation +**Related:** `schema.yaml`, `docs/backup-storage-policy.md`, `INTENT.md` + +--- + +## Purpose + +Decide **what** enters the open-weight reserve, **who** may approve it, and +**when** a daily-brief candidate becomes a catalog entry with blobs on backup +storage. + +--- + +## Goals + +* Keep a **small, high-leverage** reserve — not a Hugging Face mirror +* Enforce **license and integrity** before download completes +* Match capacity to the backup storage soft quota +* Prefer models that serve axes **B** and **C**, plus strategic **A** open releases + +--- + +## Eligibility (must pass all) + +1. **Open weights** — weights obtainable under terms that allow offline retention for lab use +2. **Clear license** — SPDX or linkable license text; `allows_offline_retention: true` +3. **Stable provenance** — official org, tagged release, or commit revision (not anonymous drive-by reupload as sole source) +4. **Lab rationale** — written `reason` tied to at least one axis A–D (usually B/C) +5. **Capacity** — estimated size fits under remaining soft quota (see backup storage policy) + +Fail any gate → status `rejected` with reason, or never enter catalog. + +--- + +## Priority rubric + +| Priority | Guidance | +| -------- | -------- | +| **high** | Rare or strategically important; license/access risk of disappearance; uniquely strong for B/C at our hardware class; hard to re-obtain | +| **medium** | Clear lab use within 1–2 quarters; good quality/cost; easy enough to re-download but worth having cold | +| **low** | Nice to have; only collect if quota headroom is large and pull is cheap | + +Daily brief **collection candidates** should set a suggested priority; approval may change it. + +--- + +## Approval rule of thumb + +| Estimated total size | Approval | +| -------------------- | -------- | +| **< 5 GiB** | Operator or lab agent may collect after license check; catalog entry required before or immediately after | +| **5–40 GiB** | Explicit operator approval (chat, workplan task, or signed catalog `approved_by`) | +| **> 40 GiB** | Operator approval **plus** check against soft quota and whether a smaller quant/variant suffices | +| **Any size if quota ≥ 70% used** | Operator approval required regardless of size | +| **Unclear license or ToS risk** | Do not collect; status `rejected` | + +“Operator” means the human lab owner (or a documented delegate). Agents may +**nominate** (`status: candidate`) freely from briefs; they may **collect** only +within the < 5 GiB band when licenses are unambiguous — otherwise stop at +`candidate` / `approved`. + +--- + +## Lifecycle + +```text +brief nominates + → candidate (catalog YAML, no blobs required) + → approved (license + size + quota OK) + → collecting (download in staging/) + → collected (blobs complete, checksums recorded, storage_path set) + → verified (optional re-hash / smoke load) + → superseded|evicted (replaced or removed; metadata kept) +``` + +Rejected candidates stay in catalog only if useful as a decision record; otherwise omit. + +--- + +## What we prefer to collect + +* Small/mid instruct and code models that fit the hardware envelope +* Strong embedding / rerank models for local RAG +* Base models known to fine-tune well under QLoRA/LoRA on lab GPUs +* Official quant releases when they are the supported distribution +* Adapters and tokenizers that unlock a reserved base (as companions) + +## What we usually skip + +* Duplicate quants of the same revision already reserved +* Huge models with no near-term local run/train path and no access-risk story +* Merges/repacks without provenance +* Datasets larger than model weights unless separately justified (default: out of band) +* Anything requiring acceptance flows we cannot satisfy offline + +--- + +## Companions + +Tokenizers, LoRA adapters, and small eval fixtures may be collected when: + +* they are required to use a reserved base, or +* they are small (< 1 GiB) and high leverage + +Link via `companions` in the catalog schema. + +--- + +## Brief integration + +1. Brief section **Collection candidates** nominates items. +2. Operator/agent opens `inventory/catalog/{id}.yaml` with `status: candidate`. +3. Approval and download follow this policy and `docs/backup-storage-policy.md`. +4. Brief `brief_refs` on the entry point back to the nominating day(s). + +--- + +## Eviction rule of thumb + +When over quota or cleaning: + +1. `low` priority, easily re-obtainable from still-live official URLs +2. Superseded revisions with a newer `verified` replacement +3. Never silent-delete: set `status: evicted`, clear or note `storage_path`, append `history` + +--- + +## Non-goals + +* Automatic bulk mirrors of entire orgs +* Collecting on every brief mention without priority +* Bypassing license gates for “research only” convenience diff --git a/inventory/schema.yaml b/inventory/schema.yaml new file mode 100644 index 0000000..569b5bd --- /dev/null +++ b/inventory/schema.yaml @@ -0,0 +1,118 @@ +# Freedom Intelligence — open-weight inventory entry schema +# Each collected (or tracked) model revision is one YAML file under catalog/ +# Filename suggestion: {org}__{name}__{short_revision}.yaml +# +# Schema version documents field meaning for humans and future validators. +schema_version: "0.1.0" + +# --------------------------------------------------------------------------- +# Example entry (illustrative only — not a real collection claim) +# --------------------------------------------------------------------------- +# id: meta-llama__Llama-3.2-3B-Instruct__abc1234 +# status: candidate | approved | collecting | collected | verified | superseded | evicted | rejected +# name: Llama-3.2-3B-Instruct +# org: meta-llama +# source: +# kind: huggingface # huggingface | github_release | direct_url | other +# url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct +# revision: abc1234def... # commit sha, tag, or release id +# model_card_url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct +# license: +# spdx: llama3.2 # or MIT, Apache-2.0, etc.; use "custom" + notes if needed +# url: https://... +# allows_offline_retention: true +# allows_local_ops: true +# allows_fine_tune: true +# notes: "" +# artifacts: +# - path: blobs/model.safetensors +# sha256: "..." +# bytes: 0 +# size: +# total_bytes: 0 +# total_human: "0 B" +# hardware_class: +# # Rough lab guidance — not a guarantee +# min_vram_gb_q4: 4 +# min_vram_gb_fp16: 8 +# notes: "fits consumer 8GB at Q4" +# axes: # why this matters to the lab +# - B +# - C +# priority: medium # low | medium | high +# collection: +# approved_by: "" +# approved_at: null # ISO-8601 date +# downloaded_at: null +# downloaded_by: "" +# storage_path: "" # absolute or facility-relative path on backup storage +# brief_refs: # briefs that nominated this candidate +# - briefs/2026/07/2026-07-23.md +# reason: "Strong small instruct base; clear license; homelab-fit." +# tags: +# - instruct +# - text +# companions: [] # optional related catalog ids (adapters, tokenizers) +# notes: "" +# history: +# - at: "2026-07-23" +# event: nominated +# by: operator +# detail: "From daily brief collection candidates." + +# --------------------------------------------------------------------------- +# Required fields by status (normative for humans; tooling may enforce later) +# --------------------------------------------------------------------------- +# candidate: id, status, name, org, source.url, source.revision, license, reason, priority +# approved: + collection.approved_by, collection.approved_at +# collected: + collection.downloaded_at, collection.storage_path, size, artifacts[].sha256 +# verified: + checksums re-read OK (notes or history event) +# rejected: + reason (why rejected) +# superseded / evicted: + history event; storage_path may be empty after eviction + +field_reference: + id: + type: string + description: Stable catalog id; prefer {org}__{name}__{short_revision} + status: + type: enum + values: [candidate, approved, collecting, collected, verified, superseded, evicted, rejected] + name: + type: string + org: + type: string + source: + type: object + fields: [kind, url, revision, model_card_url] + license: + type: object + fields: [spdx, url, allows_offline_retention, allows_local_ops, allows_fine_tune, notes] + artifacts: + type: list + item_fields: [path, sha256, bytes] + size: + type: object + fields: [total_bytes, total_human] + hardware_class: + type: object + fields: [min_vram_gb_q4, min_vram_gb_fp16, notes] + axes: + type: list + values: [A, B, C, D] + priority: + type: enum + values: [low, medium, high] + collection: + type: object + fields: [approved_by, approved_at, downloaded_at, downloaded_by, storage_path, brief_refs] + reason: + type: string + tags: + type: list + companions: + type: list + notes: + type: string + history: + type: list + item_fields: [at, event, by, detail] diff --git a/research/2026-07-24-baseline-field-survey.md b/research/2026-07-24-baseline-field-survey.md new file mode 100644 index 0000000..3e522c6 --- /dev/null +++ b/research/2026-07-24-baseline-field-survey.md @@ -0,0 +1,279 @@ +# Baseline field survey — Freedom Intelligence starting point + +**Date:** 2026-07-24 +**Type:** deep baseline (not a daily delta brief) +**Purpose:** Establish longitudinal memory so the **first daily research brief** +can be a true delta, not a full survey. Also recommend the **initial open-weight +collection** for the lab reserve. +**Axes:** A frontier · B edge/local · C training · D harness/fleet +**Sources:** public web surveys, HF/model docs, pricing roundups, harness project +sites (mid–late 2026 public reporting). Prices and leaderboard claims move weekly — +treat numbers as **order-of-magnitude** and re-check on first daily. + +--- + +## 1. Executive snapshot + +| Theme | State of the field (mid-2026) | Lab implication | +| ----- | ----------------------------- | --------------- | +| Frontier | Multi-vendor race: OpenAI GPT-5.x, Anthropic Claude 4.x/Opus, Google Gemini 3.x, xAI Grok 4.x; coding agents dominate product narrative | Keep API access via `llm-connect`; do not try to host frontier closed weights | +| Price collapse | DeepSeek-class APIs often **10–100×** cheaper than flagship closed models for many tasks | Route cheap/easy work to low-cost APIs; reserve local for privacy, offline, specialization | +| Open weights | Strong mid-size dense + MoE story (Qwen3, DeepSeek distill/V3-class, Gemma/Llama small, Mistral small) | Homelab can be *useful* without 8×H100 | +| Local stack | llama.cpp / vLLM / MLX / Ollama mature; Q4–Q5 quants standard | Axis B is operationally ready | +| Training | QLoRA/Unsloth/TRL still the homelab path; full pretrain not lab-realistic | Axis C = adapters + domain SFT, not from-scratch | +| Harness | OpenHands, Aider, SWE-agent, OpenCode, Cline; closed CLIs (Claude Code, Codex) lead quality | Axis D: watch + integrate with sand-boxer / agent-harness, don’t fork everything | + +**Central proposition for the reserve:** collect a **compact spine** of (1) small +general instruct, (2) mid-size code/reason distill, (3) multilingual embed, +(4) optional larger “lab ceiling” base — not a mirror of Hugging Face. + +--- + +## 2. Axis A — Frontier capabilities, benchmarks, price + +### 2.1 Capability landscape + +Public 2026 reporting groups the closed frontier into: + +- **Flagship general / agentic** — OpenAI GPT-5.x family, Anthropic Claude Opus/Sonnet 4.x+, Google Gemini 3.x Pro +- **Fast / cheap tiers** — GPT mini/nano, Claude Haiku, Gemini Flash +- **Reasoning-heavy** — “thinking” / extended variants; cost dominated by output tokens +- **Coding productization** — agent harnesses matter as much as base model (SWE-bench Verified numbers often quoted 70%+ with strong closed models + good harness; novel-issue sets much lower ~18–20%) + +Benchmark caveats for the lab: + +- Arena and public leaderboards are **gamed and lag** product quality +- SWE-bench Verified ≠ “our monorepos + NetKingdom tools” +- Prefer **internal harness scores** once we have them; use public numbers only as directional + +### 2.2 Price-point trends (API, ~per 1M tokens) + +Synthesized from 2026 pricing roundups (verify on provider pages before budget decisions): + +| Band | Examples (reported ranges) | Role | +| ---- | -------------------------- | ---- | +| **Ultra-cheap** | DeepSeek V3.x / V4 Flash-class (~$0.14–0.28 in / ~$0.28–0.42 out; cache hits lower) | Bulk classify, draft, simple tools | +| **Budget closed** | Gemini Flash, GPT mini/nano, Haiku-class | Default automation when quality bar is modest | +| **Mid flagship** | Claude Sonnet-class, GPT mid-tier (~$2–5 in / ~$10–15 out) | Daily agent coding, serious writing | +| **Top flagship** | Opus / GPT pro tiers (~$5+ in / $25–30+ out; pro tiers much higher) | Hard reasoning, high-stakes review | + +**Trend:** continuous compression of mid-tier quality cost; flagship **output** +tokens remain expensive. Caching and batch APIs swing real cost 50%+. + +**Lab stance:** + +- Frontier APIs stay first-class for hard work (via llm-connect / OpenRouter-class) +- Daily *research* about price moves is high value for routing policy +- Local models win on **privacy, offline, predictable unit cost, specialization** — not always raw SOTA + +### 2.3 Open near-frontier (API or weights) + +DeepSeek open-weight + cheap API continues to be the structural disruptor: +MIT-leaning releases for R1/V3-class artifacts, distill series for single-GPU +reasoning. Track **V3/V4 Flash vs Pro** naming carefully — product lines rename often. + +--- + +## 3. Axis B — Edge, local, open compute + +### 3.1 What “homelab useful” means now + +| Class | Params (ballpark) | Typical local use | +| ----- | ----------------- | ----------------- | +| Edge / CPU-OK | ≤3B Q4 | Chat toys, classification, always-on small agents | +| Consumer GPU | 7–14B Q4/Q5 | Daily coding assist, RAG chat, tool loops | +| Single enthusiast GPU | 32B Q4 | Stronger reason/code; slower | +| Lab multi-GPU | 70B Q4 or sparse MoE | Optional ceiling; not required for v1 reserve | + +Runtimes to track: **llama.cpp**, **vLLM**, **MLX** (Apple), **Ollama** (UX), +TensorRT-LLM (NVIDIA optimize). Speculative decoding + good quant matter more +than chasing every new 70B. + +### 3.2 Open model families to watch (weights) + +| Family | Why it matters | License caution | +| ------ | -------------- | --------------- | +| **Qwen3** (Alibaba) | Dense + MoE line; strong multilingual & tool use; embeddings/VL variants | Apache-class for many; check model card | +| **DeepSeek** (V3/R1 + distills) | Reasoning + cost; MIT for key R1/distill artifacts | Confirm per-revision card | +| **Llama 3.x / 4 text** | Ecosystem, tooling, small 1B–3B instruct | Llama community license (not pure MIT) | +| **Gemma 3** | Google small/efficient; EmbeddingGemma | Gemma terms | +| **Mistral Small / Ministral** | EU-friendly commercial open options | Apache / research licenses vary | +| **Phi / Smol** class | Edge CPU experiments | Check commercial terms | + +### 3.3 Embeddings & retrieval (local RAG spine) + +For NetKingdom docs, code, and ops notes: + +| Model | Role | Notes | +| ----- | ---- | ----- | +| **BGE-M3** (BAAI) | Default multilingual dense (+ multi-granularity) | MIT; ~0.5B; self-host staple | +| **Nomic Embed** text v1/v2 | Strong English/multilingual text | Apache | +| **Nomic Embed Code** / code-specialized | Code retrieval | Larger; optional P1 | +| **EmbeddingGemma-300M** | Tiny multilingual | Edge RAG | +| **Qwen3 embedding / VL-small** | Multimodal retrieval experiments | Watch cards | + +Rerankers (BGE reranker, mini cross-encoders) are high leverage and small — good +companions once embed base is reserved. + +--- + +## 4. Axis C — Homelab training & domain specialization + +### 4.1 Realistic lab methods + +| Method | Homelab fit | Use | +| ------ | ----------- | --- | +| QLoRA / LoRA (Unsloth, PEFT, TRL) | Excellent | Domain assistants (security, ops, railiance) | +| Preference (DPO/ORPO/KTO) | Good on 7–14B | Style and safety for agents | +| Continued pretrain | Possible on small bases | Domain language (logs, policies) | +| Full SFT of 70B+ | Poor without multi-GPU budget | Prefer distill/adapter on smaller | +| From-scratch pretrain | Out of scope | — | + +### 4.2 Bases worth specializing + +1. **7–8B instruct** (Qwen3-8B or Llama-3.1-8B-Instruct class) — default FT target +2. **14B** — quality step if VRAM allows +3. **DeepSeek-R1-Distill 14B/32B** — reason-heavy agents after FT discipline +4. **Code-specialized mid** — if code assistants dominate workload + +Data: prefer **synthetic + curated NetKingdom docs** over scraping; keep PII out +of training sets (separate policy later). + +### 4.3 Tooling to track (not collect as weights) + +Unsloth, Axolotl, TRL, llama-factory, mergekit, preference datasets (UltraFeedback-class), +eval harnesses (lm-eval, custom domain suites). + +--- + +## 5. Axis D — Agent harnesses, orchestration, sandboxing, fleet + +### 5.1 Open harness landscape (2026) + +| Project | Shape | Why watch | +| ------- | ----- | --------- | +| **OpenHands** | Full agent platform + Canvas; MIT; multi-backend | Leading open coding-agent platform; multi-agent | +| **Aider** | Git-native CLI pair programmer | Atomic commits; simple; great for ops scripts | +| **SWE-agent / mini-SWE-agent** | Research harness, clean ACI | Benchmark + security CTF modes | +| **OpenCode** | Open Claude Code alternative | High interest community | +| **Cline** | IDE agent | Editor-centric workflows | +| **Closed CLIs** | Claude Code, Codex, Gemini CLI | Quality bar for comparison | + +SWE-bench: harness + model co-determine scores; Mini-SWE-Agent claims strong +results in very little code — architecture lessons for Coulomb harnesses. + +### 5.2 NetKingdom adjacency + +| Coulomb piece | Relation to D | +| ------------- | ------------- | +| **sand-boxer** | Where agents run safely | +| **activity-core** | When work is due (this lab’s daily brief) | +| **agent-harness / kaizen** | Who executes with profiles | +| **state-hub** | What changed; task spawn completion | +| **llm-connect** | Which model backend | + +Freedom Intelligence **reports** on harness tech; it does not replace sand-boxer. + +### 5.3 Sandboxing & fleet themes to track + +- Default-deny network + TTL sandboxes +- Credential routing (OpenBao / ops-warden) separate from model weights +- Multi-agent kanban / workplan-tied loops (Ralph-workplan class) +- Cost and rate-limit fleet management for API models + +--- + +## 6. Collection recommendations (open-weight reserve) + +### 6.1 Decision framework (recap) + +- Prefer **B/C** leverage + clear license + stable org provenance +- Size gates per `inventory/collection-policy.md` +- Backup path must be pinned (FI-WP-0001-T04) before large pulls +- Quants: prefer **official or well-known GGUF/AWQ** only after base id is cataloged; v1 recommends **safetensors official** first, quant as companion later + +### 6.2 P0 — seed now (approve after storage pin) + +| # | Model (representative HF id) | Est. size class | Axes | Why | +| - | ---------------------------- | --------------- | ---- | --- | +| 1 | `Qwen/Qwen3-8B` (or current 8B-Instruct sibling on card) | ~16 GB fp16 / ~5 GB Q4 | B, C | Best default general + FT base; multilingual; tool-friendly | +| 2 | `meta-llama/Llama-3.2-3B-Instruct` | ~6 GB fp16 / ~2 GB Q4 | B | Tiny always-on / edge; huge ecosystem | +| 3 | `BAAI/bge-m3` | ~2 GB | B | Multilingual RAG staple; MIT | +| 4 | `deepseek-ai/DeepSeek-R1-Distill-Qwen-14B` (or 8B if VRAM tight) | ~28 GB fp16 / ~9 GB Q4 | B, C | Local reasoning without full R1; MIT distill story | +| 5 | `nomic-ai/nomic-embed-text-v1.5` (or current v2 text) | <1 GB | B | Lightweight embed alternative / A/B with BGE | + +**Rough P0 cold storage (fp16 worst case):** ~50–60 GB. +**With Q4 preference for LLMs + fp for embeds:** often **~20–30 GB**. Prefer quants only when catalog notes the exact quant provenance. + +### 6.3 P1 — next wave (quota permitting) + +| # | Model | Why | +| - | ----- | --- | +| 6 | `Qwen/Qwen3-14B` (instruct) | Quality step for single-GPU chat/code | +| 7 | `google/gemma-3-4b-it` or EmbeddingGemma-300M | Efficient Google line; embed edge | +| 8 | `mistralai/Mistral-Small-3.x` instruct (current card) | EU-adjacent open option | +| 9 | `deepseek-ai/DeepSeek-R1-Distill-Qwen-32B` | Stronger local reason if ≥24 GB VRAM class | +| 10 | Code embed: `nomic-ai/nomic-embed-code` or Jina code embed | Code RAG for repos | +| 11 | Small reranker: `BAAI/bge-reranker-v2-m3` | Cheap quality win for RAG | + +### 6.4 P2 — watch / large (candidate or docs only) + +| Item | Guidance | +| ---- | -------- | +| Full **DeepSeek-V3 / R1** MoE (~600B+ class, hundreds of GB even quantized) | **Do not auto-collect.** Separate capacity + multi-GPU decision | +| **Llama 70B / Qwen 72B** class | Only if lab gains multi-GPU and clear offline need | +| Every new quant repack on HF | Skip; track base official only | + +### 6.5 Explicit non-recommendations (for now) + +- Closed API-only weights +- Anonymous “uncensored” reuploads without provenance +- Duplicate quants of the same revision +- Giant pretraining corpora as “models” + +### 6.6 Catalog seed + +YAML candidates for P0–P1 are under `inventory/catalog/` with +`status: candidate` and `brief_refs` pointing at this survey. Approval and +download are **FI-WP-0003**. + +--- + +## 7. Implications for the first daily brief + +The first `briefs/YYYY/MM/YYYY-MM-DD.md` after this survey should: + +1. **Not** restate this whole document +2. Cover **only deltas** since 2026-07-24 (new releases, price changes, harness versions) +3. Revisit P0/P1 if a **new open model** clearly displaces a candidate +4. Note any **license or access shocks** (takedowns, geo blocks) +5. Optionally mark FI-WP-0001-T04 / FI-WP-0003 progress under Lab implications + +Automation: activity-core def `fi-daily-research-brief` (disabled until FI-WP-0002 +resolver + dry-run). Manual playbook: `docs/daily-brief-playbook.md`. + +--- + +## 8. Open questions for later briefs + +1. Exact **hardware envelope** measured VRAM on lab hosts? +2. Prefer **GGUF Q5_K_M** vs official safetensors for P0? +3. Is **German-first** multilingual quality a hard requirement (affects embed + instruct choice)? +4. Will local models serve **air-gapped NetKingdom** scenarios or only cost/privacy convenience? +5. Single **code model** vs general instruct for domain FT? + +--- + +## 9. Source anchors (non-exhaustive) + +- DeepSeek R1 release / MIT notes — DeepSeek API docs & HF model cards +- Pricing roundups 2026 — IntuitionLabs, MorphLLM, provider pricing pages +- Embeddings 2026 comparisons — BentoML, Milvus/Zilliz, BGE-M3 cards +- Harness landscape — OpenHands, Aider, SWE-agent comparisons (LocalAI Master, vendor blogs) +- Open-weight commercial lists — community curations (verify each card) + +Re-verify any number before spend or multi-hundred-GB download. + +--- + +*Baseline established 2026-07-24 for Freedom Intelligence. Daily briefs begin as deltas from here.* diff --git a/workplans/FI-WP-0001-lab-operating-foundation.md b/workplans/FI-WP-0001-lab-operating-foundation.md new file mode 100644 index 0000000..2427951 --- /dev/null +++ b/workplans/FI-WP-0001-lab-operating-foundation.md @@ -0,0 +1,147 @@ +--- +id: FI-WP-0001 +type: workplan +title: "Lab operating foundation: sources, playbook, storage pin, hardware envelope" +domain: infotech +repo: freedom-intelligence +status: active +owner: grok +topic_slug: freedom-intelligence +priority: high +created: "2026-07-24" +updated: "2026-07-24" +--- + +# FI-WP-0001 — Lab operating foundation + +## Goal + +Make Freedom Intelligence **operable for daily research and model reserve work** +before automation: pin backup storage, document sources and hardware envelope, +ship a daily-brief playbook, and close gaps left after the foundation docs +(`INTENT.md`, `SCOPE.md`, `briefs/`, `inventory/`). + +Baseline field survey lives in `research/2026-07-24-baseline-field-survey.md` +and seeds collection candidates under `inventory/catalog/`. + +## Depends on + +- Foundation docs already present (2026-07-23) +- Baseline research (this session) — **source of first collection candidates** + +## Parallel / follow-on + +- **FI-WP-0002** — activity-core daily cadence (after playbook exists) +- **FI-WP-0003** — execute reserve seed under collection policy (after storage pin) + +## Tasks + +### T01 — Sources allowlist for daily research + +```task +id: FI-WP-0001-T01 +status: done +priority: high +``` + +Create `docs/sources-allowlist.md` with standing channels for axes A–D: + +- Frontier: provider blogs, pricing pages, LMSYS/Arena notes, major model cards +- Open weights: Hugging Face org watchlist, official GitHub releases +- Methods: arXiv categories (cs.LG, cs.CL, cs.AI, cs.SE), Unsloth/axolotl/TRL notes +- Harness: OpenHands, Aider, SWE-agent, sand-boxer-adjacent ecosystem +- Explicit **do not scrape** / license caution list + +**Done when:** allowlist exists and is linked from `briefs/README.md`. + +--- + +### T02 — Daily brief playbook (manual first, automation-ready) + +```task +id: FI-WP-0001-T02 +status: done +priority: high +``` + +Write `docs/daily-brief-playbook.md`: + +- Step sequence: scan allowlist → draft from template → collection candidates → + optional hub progress event +- Completion evidence shape: `event_type=fi_daily_brief`, `detail.repo`, `detail.date` +- Idempotence: one brief per calendar day (Europe/Berlin) +- Quality bar aligned with `briefs/_template.md` and INTENT principles +- Pointer that **first automated day** starts from baseline survey, not empty memory + +**Done when:** an operator or agent can produce a brief without reading INTENT end-to-end. + +--- + +### T03 — Hardware envelope draft + +```task +id: FI-WP-0001-T03 +status: done +priority: medium +``` + +Write `docs/hardware-envelope.md` with **honest TBD** for unmeasured hosts, plus +working assumptions for “homelab accessible” (consumer GPU tiers, CPU-only +floor, multi-GPU lab class). Used by inventory `hardware_class` fields and +axis B/C prioritization. + +**Done when:** envelope doc exists; operator can fill measured VRAM/host rows later +without schema changes. + +--- + +### T04 — Pin backup storage path and soft quota + +```task +id: FI-WP-0001-T04 +status: todo +priority: high +``` + +Operator task: fill TBD fields in `docs/backup-storage-policy.md`: + +- facility / host +- base path (`…/freedom-intelligence/models/`) +- soft quota +- access method +- pin log entry + +Create empty `models/` and `staging/` on that facility. + +**Done when:** policy has no critical TBD for path/quota; `FI-WP-0003` may download. + +--- + +### T05 — Link baseline research into lab navigation + +```task +id: FI-WP-0001-T05 +status: done +priority: medium +``` + +- Point `README.md` / `SCOPE.md` at `research/2026-07-24-baseline-field-survey.md` +- Note that first **daily** brief is a delta from this baseline, not a second full survey +- Ensure inventory candidates from the survey are present under `inventory/catalog/` + +**Done when:** navigation and catalog seed are consistent with the survey. + +--- + +## Acceptance (workplan-level) + +- [x] Sources allowlist + daily playbook + hardware envelope drafted +- [ ] Backup storage path and soft quota pinned (T04 — operator) +- [x] Baseline research persisted and linked +- [x] Catalog candidates seeded for recommended models + +## Out of scope + +- Enabling activity-core schedule (FI-WP-0002) +- Bulk model downloads (FI-WP-0003) +- State Hub domain registration for freedom-intelligence as a full domain diff --git a/workplans/FI-WP-0002-activity-core-daily-research.md b/workplans/FI-WP-0002-activity-core-daily-research.md new file mode 100644 index 0000000..b1376b4 --- /dev/null +++ b/workplans/FI-WP-0002-activity-core-daily-research.md @@ -0,0 +1,208 @@ +--- +id: FI-WP-0002 +type: workplan +title: "Activity-core daily research brief rhythm" +domain: infotech +repo: freedom-intelligence +status: active +owner: grok +topic_slug: freedom-intelligence +priority: high +created: "2026-07-24" +updated: "2026-07-24" +depends_on: + - FI-WP-0001 +--- + +# FI-WP-0002 — Activity-core daily research brief rhythm + +## Goal + +Schedule and emit the **Freedom Intelligence daily research brief** via +**activity-core**, following the proven Binky rhythm pattern: + +```text +Temporal cron (activity-core) + → context: fi_brief_status (due?) + → rule emit activity_task_spawn (state-hub sink) + → consumer (agent-harness / operator) runs playbook in freedom-intelligence + → progress event fi_daily_brief (idempotence for next day) +``` + +activity-core answers **when / what / where**. It does **not** write the brief. +Execution stays in this repo (or a harness checking out this repo). + +## Architecture + +| Piece | Location | Role | +| ----- | -------- | ---- | +| ActivityDefinition | `freedom-intelligence/activity-definitions/fi-daily-research-brief.md` | Cron + rules | +| External def load | activity-core `ACTIVITY_DEFINITION_DIRS` / external paths | Discover domain def | +| Context resolver | activity-core state-hub query `fi_brief_status` (or shell equivalent) | `due` for today | +| Task sink | state-hub `activity_task_spawn` | Default safe sink | +| Consumer | agent-harness Green/Blue or manual operator session | Run playbook | +| Completion | State Hub `fi_daily_brief` progress event | Clears `due` | + +Reference contracts: + +- `activity-core/docs/task-emission-consumer-contract.md` +- Binky: `activity-definitions/binky-daily-rhythm.md` + `binky-control/OperatingRhythm.md` +- `docs/daily-brief-playbook.md` (this repo) + +## Schedule (proposed) + +| Field | Value | +| ----- | ----- | +| Cron | `30 7 * * 1-5` (weekdays 07:30) | +| Timezone | `Europe/Berlin` | +| Misfire | `skip` | +| First enable | after dry-run spawn + one manual brief completion event proven | + +Weekends optional later (`0 9 * * 6` optional Saturday scan) — not in v1. + +## Tasks + +### T01 — ActivityDefinition (domain repo, disabled) + +```task +id: FI-WP-0002-T01 +status: done +priority: high +``` + +Add `activity-definitions/fi-daily-research-brief.md`: + +- `enabled: false` until dry-run passes +- cron + timezone as above +- context source for due check (stub params documented if resolver not yet live) +- rule: emit task when `due`, `target_repo: freedom-intelligence`, labels + `["freedom-intelligence", "research-brief", "automated"]` +- description points at `docs/daily-brief-playbook.md` + +**Done when:** file exists, frontmatter valid, matches ADR-002 shape used by +existing definitions (binky / forgejo prune). + +--- + +### T02 — Completion event contract + playbook hook + +```task +id: FI-WP-0002-T02 +status: done +priority: high +``` + +Document and implement playbook steps for: + +```yaml +event_type: fi_daily_brief +author: +summary: "FI daily brief YYYY-MM-DD" +detail: + repo: freedom-intelligence + date: "YYYY-MM-DD" + path: "briefs/YYYY/MM/YYYY-MM-DD.md" + collection_candidates: +``` + +Wire into `docs/daily-brief-playbook.md` and `docs/state-hub-delivery.md`. + +**Done when:** completion evidence shape is unambiguous for a future resolver. + +--- + +### T03 — `fi_brief_status` context resolver (activity-core) + +```task +id: FI-WP-0002-T03 +status: todo +priority: high +``` + +**Cross-repo (activity-core):** implement state-hub (or shell) resolver: + +- Query name: `fi_brief_status` +- Returns `{ items: [{ kind: "daily_brief", due: bool, date: "YYYY-MM-DD" }] }` +- `due: true` when no `fi_daily_brief` progress event with + `detail.repo=freedom-intelligence` exists for today (Europe/Berlin) + +Mirror `binky_rhythm_status` pattern. Add unit tests with mocked hub. + +**Done when:** resolver merged in activity-core and callable from definition params. + +--- + +### T04 — Register external activity-definitions path + +```task +id: FI-WP-0002-T04 +status: todo +priority: high +``` + +**Cross-repo / ops:** ensure railiance activity-core worker loads this repo’s +`activity-definitions/` (env `ACTIVITY_DEFINITION_DIRS` or documented external +path). Run `make sync-activity-definitions` / schedule reconciliation. + +**Done when:** definition appears in automation inventory; schedule id +`activity-schedule-fi-daily-research-brief` (or equivalent) is visible when enabled. + +--- + +### T05 — Dry-run spawn, then enable + +```task +id: FI-WP-0002-T05 +status: todo +priority: medium +``` + +1. Manual `POST .../activity-definitions//trigger` with def still disabled + or one-shot test path +2. Confirm `activity_task_spawn` in State Hub (no Forgejo spam) +3. Manually execute playbook once; post `fi_daily_brief` +4. Confirm next resolver `due=false` for that date +5. Set `enabled: true` on definition; commit + +**Done when:** one full loop proven; weekday schedule live. + +--- + +### T06 — Consumer routing note (agent-harness) + +```task +id: FI-WP-0002-T06 +status: todo +priority: medium +``` + +Document how harness/operator picks up `target_repo: freedom-intelligence` +tasks (checkout path, Green/Blue profile, LLM via llm-connect if used). Prefer +**no** Claude Code install on railiance01 (Binky residual lesson). + +**Done when:** `docs/daily-brief-playbook.md` § Automation consumer is enough for +the next agent session. + +--- + +## Acceptance (workplan-level) + +- [x] Domain ActivityDefinition drafted (`enabled: false`) +- [x] Completion event contract documented +- [ ] Resolver + external path + dry-run enable (T03–T05) +- [ ] Consumer path documented for harness (T06) + +## Out of scope + +- Writing the LLM brief **inside** activity-core workers +- ISSUE_SINK_TYPE=rest / Forgejo issues for this rhythm +- Weekend or multi-brief-per-day schedules +- Full State Hub domain onboarding for freedom-intelligence + +## Residual handoffs + +| Residual | Owner repo | +| -------- | ---------- | +| `fi_brief_status` resolver | activity-core | +| ACTIVITY_DEFINITION_DIRS pin | activity-core / railiance ops | +| Harness consumer profile | agent-harness / the-custodian as applicable | diff --git a/workplans/FI-WP-0003-seed-open-weight-reserve.md b/workplans/FI-WP-0003-seed-open-weight-reserve.md new file mode 100644 index 0000000..1d64332 --- /dev/null +++ b/workplans/FI-WP-0003-seed-open-weight-reserve.md @@ -0,0 +1,135 @@ +--- +id: FI-WP-0003 +type: workplan +title: "Seed open-weight reserve from baseline recommendations" +domain: infotech +repo: freedom-intelligence +status: active +owner: grok +topic_slug: freedom-intelligence +priority: medium +created: "2026-07-24" +updated: "2026-07-24" +depends_on: + - FI-WP-0001 +--- + +# FI-WP-0003 — Seed open-weight reserve from baseline recommendations + +## Goal + +Turn baseline survey recommendations into a **real, policy-compliant reserve**: +approve high-priority catalog candidates, download to pinned backup storage, +verify checksums, and leave medium/low items as candidates for later briefs. + +Source of truth for *which* models: + +- `research/2026-07-24-baseline-field-survey.md` § Collection recommendations +- `inventory/catalog/*.yaml` (seeded as `candidate`) + +Gates: `inventory/collection-policy.md` + `docs/backup-storage-policy.md` (T04 of FI-WP-0001). + +## Priority tiers (from survey) + +| Tier | Action in this workplan | +| ---- | ----------------------- | +| **P0 — seed now** | Approve + collect after storage pin (small/mid, high leverage) | +| **P1 — next wave** | Approve when quota allows; may stay candidate | +| **P2 — watch / large** | Keep candidate; do not bulk-pull full 600B+ MoE without explicit operator decision | + +## Tasks + +### T01 — Seed catalog YAML for recommended models + +```task +id: FI-WP-0003-T01 +status: done +priority: high +``` + +Create `status: candidate` entries under `inventory/catalog/` for every P0/P1 +recommendation in the baseline survey (and note P2 as candidates or docs-only). + +**Done when:** catalog files exist and match `inventory/schema.yaml` required fields. + +--- + +### T02 — Operator approve P0 set + +```task +id: FI-WP-0003-T02 +status: todo +priority: high +``` + +Human review of P0 candidates: license, size vs soft quota, hardware fit. +Set `status: approved`, `collection.approved_by`, `collection.approved_at`. + +**Done when:** each P0 entry is `approved` or `rejected` with reason. + +--- + +### T03 — Download P0 to backup storage + verify + +```task +id: FI-WP-0003-T03 +status: todo +priority: high +``` + +For each approved P0: + +1. Download into `{BACKUP}/freedom-intelligence/models/...` (or `staging/` then promote) +2. Record `artifacts[].sha256`, `size`, `collection.storage_path` +3. Set `status: collected` then `verified` after re-hash or smoke load + +Prefer official HF revisions; use `huggingface-cli` or equivalent with revision pin. + +**Blocked on:** FI-WP-0001-T04 (storage pin). + +**Done when:** all approved P0 are `verified` or explicitly deferred with notes. + +--- + +### T04 — P1 decision pass + +```task +id: FI-WP-0003-T04 +status: todo +priority: medium +``` + +After P0, decide which P1 models fit remaining quota. Approve/collect subset or +leave as candidates for daily briefs to re-prioritize. + +**Done when:** each P1 has an explicit next status (`approved`, `candidate`, `rejected`). + +--- + +### T05 — Reserve index snapshot + +```task +id: FI-WP-0003-T05 +status: todo +priority: low +``` + +Write `inventory/RESERVE-STATUS.md` summarizing collected vs candidate totals, +bytes used vs soft quota, and hardware coverage gaps. + +**Done when:** one-page status exists for operators and future daily briefs. + +--- + +## Acceptance (workplan-level) + +- [x] Catalog seeded from baseline survey +- [ ] Storage pin complete (upstream) +- [ ] P0 verified on backup media +- [ ] RESERVE-STATUS snapshot after first collections + +## Out of scope + +- Hosting inference endpoints +- Full DeepSeek-V3/R1 671B pull without separate capacity decision +- Dataset reserve (weights only unless tiny tokenizer/adapters)