Establish Freedom Intelligence lab foundation and baseline research.

Add INTENT/SCOPE, daily-brief playbook, activity-core definition (disabled),
workplans FI-WP-0001..0003, baseline field survey with open-weight collection
recommendations, and inventory catalog candidates for the model reserve.
This commit is contained in:
tegwick 2026-07-24 00:15:27 +02:00
parent 1dc1e09517
commit b5f911140b
28 changed files with 2651 additions and 1 deletions

32
inventory/README.md Normal file
View file

@ -0,0 +1,32 @@
# Model inventory
In-repo **catalog of open-weight revisions** reserved (or nominated) for the lab.
Weight blobs live on backup storage — see `docs/backup-storage-policy.md`.
## Layout
```text
inventory/
├── README.md # this file
├── schema.yaml # field reference + example
├── collection-policy.md # approval and eligibility rules
└── catalog/
└── *.yaml # one file per model revision
```
## Status values
`candidate``approved``collecting``collected``verified`
also: `superseded` | `evicted` | `rejected`
## Adding an entry
1. Confirm eligibility in `collection-policy.md`.
2. Create `catalog/{org}__{name}__{short_revision}.yaml` using fields from `schema.yaml`.
3. Set `status: candidate` (or `approved` if already signed off).
4. After download and checksums: set `collected` / `verified` and `collection.storage_path`.
## Empty catalog
The catalog starts empty on purpose. First entries come from daily brief
**collection candidates** after policy checks — not from bulk scraping.

View file

View file

@ -0,0 +1,42 @@
id: BAAI__bge-m3__candidate
status: candidate
name: bge-m3
org: BAAI
source:
kind: huggingface
url: https://huggingface.co/BAAI/bge-m3
revision: main
model_card_url: https://huggingface.co/BAAI/bge-m3
license:
spdx: MIT
url: https://huggingface.co/BAAI/bge-m3
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: ""
size:
total_bytes: 0
total_human: "~2 GB"
hardware_class:
min_vram_gb_q4: 1
min_vram_gb_fp16: 2
notes: "Runs on CPU comfortably for many workloads"
axes: [B]
priority: high
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P0 multilingual embedding staple for local RAG (docs, ops notes, DE/EN)."
tags: [embedding, multilingual, rag]
companions: []
notes: "Small download — within agent auto-collect band after license check once storage pinned."
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P0 recommendation from initial deep research."

View file

@ -0,0 +1,43 @@
id: BAAI__bge-reranker-v2-m3__candidate
status: candidate
name: bge-reranker-v2-m3
org: BAAI
source:
kind: huggingface
url: https://huggingface.co/BAAI/bge-reranker-v2-m3
revision: main
model_card_url: https://huggingface.co/BAAI/bge-reranker-v2-m3
license:
spdx: Apache-2.0
url: https://huggingface.co/BAAI/bge-reranker-v2-m3
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: "Confirm SPDX on card."
size:
total_bytes: 0
total_human: "~12 GB (estimate)"
hardware_class:
min_vram_gb_q4: 1
min_vram_gb_fp16: 2
notes: "Companion to bge-m3"
axes: [B]
priority: medium
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P1 — cheap RAG quality win as companion to BGE-M3."
tags: [reranker, rag]
companions:
- BAAI__bge-m3__candidate
notes: ""
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P1 recommendation from initial deep research."

View file

@ -0,0 +1,42 @@
id: Qwen__Qwen3-14B__candidate
status: candidate
name: Qwen3-14B
org: Qwen
source:
kind: huggingface
url: https://huggingface.co/Qwen/Qwen3-14B
revision: main
model_card_url: https://huggingface.co/Qwen/Qwen3-14B
license:
spdx: Apache-2.0
url: https://huggingface.co/Qwen/Qwen3-14B
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: "Confirm card at download."
size:
total_bytes: 0
total_human: "~28 GB fp16 / ~9 GB Q4 (estimate)"
hardware_class:
min_vram_gb_q4: 10
min_vram_gb_fp16: 28
notes: "T2 quality step"
axes: [B, C]
priority: medium
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P1 — stronger single-GPU chat/code when quota allows after P0."
tags: [instruct, qwen3]
companions: []
notes: ""
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P1 recommendation from initial deep research."

View file

@ -0,0 +1,42 @@
id: Qwen__Qwen3-8B__candidate
status: candidate
name: Qwen3-8B
org: Qwen
source:
kind: huggingface
url: https://huggingface.co/Qwen/Qwen3-8B
revision: main
model_card_url: https://huggingface.co/Qwen/Qwen3-8B
license:
spdx: Apache-2.0
url: https://huggingface.co/Qwen/Qwen3-8B
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: "Confirm exact card license at download time; Qwen3 line generally Apache-2.0."
size:
total_bytes: 0
total_human: "~16 GB fp16 / ~5 GB Q4 (estimate)"
hardware_class:
min_vram_gb_q4: 6
min_vram_gb_fp16: 16
notes: "Default T1T2 general instruct and FT base"
axes: [B, C]
priority: high
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P0 spine — best default open general/tool model for local ops and QLoRA domain specialization."
tags: [instruct, text, qwen3, ft-base]
companions: []
notes: "Prefer Instruct variant on card if separate repo; pin commit SHA at collection."
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P0 recommendation from initial deep research."

View file

@ -0,0 +1,42 @@
id: deepseek-ai__DeepSeek-R1-Distill-Qwen-14B__candidate
status: candidate
name: DeepSeek-R1-Distill-Qwen-14B
org: deepseek-ai
source:
kind: huggingface
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
revision: main
model_card_url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
license:
spdx: MIT
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: "R1 distill series MIT — confirm card at pin time."
size:
total_bytes: 0
total_human: "~28 GB fp16 / ~9 GB Q4 (estimate)"
hardware_class:
min_vram_gb_q4: 10
min_vram_gb_fp16: 28
notes: "T2 Q4 preferred; fall back to 8B distill if VRAM tight"
axes: [B, C]
priority: high
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P0 local reasoning without full R1 MoE — agent/tool loops and harder offline tasks."
tags: [reasoning, distill, deepseek]
companions: []
notes: "If disk/VRAM constrained, substitute DeepSeek-R1-Distill-Qwen-8B as P0 alternate."
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P0 recommendation from initial deep research."

View file

@ -0,0 +1,42 @@
id: deepseek-ai__DeepSeek-R1-Distill-Qwen-32B__candidate
status: candidate
name: DeepSeek-R1-Distill-Qwen-32B
org: deepseek-ai
source:
kind: huggingface
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
revision: main
model_card_url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
license:
spdx: MIT
url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: ""
size:
total_bytes: 0
total_human: "~65 GB fp16 / ~20 GB Q4 (estimate)"
hardware_class:
min_vram_gb_q4: 20
min_vram_gb_fp16: 64
notes: "T2T3; only if hardware envelope supports"
axes: [B, C]
priority: medium
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P1 stronger local reasoner — approve only with VRAM + quota headroom."
tags: [reasoning, distill, deepseek]
companions: []
notes: "Do not collect full DeepSeek-V3/R1 MoE under this id."
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P1 recommendation from initial deep research."

View file

@ -0,0 +1,42 @@
id: meta-llama__Llama-3.2-3B-Instruct__candidate
status: candidate
name: Llama-3.2-3B-Instruct
org: meta-llama
source:
kind: huggingface
url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
revision: main
model_card_url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
license:
spdx: custom
url: https://ai.meta.com/llama/license/
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: "Llama Community License — not MIT; review terms before commercial redistribution."
size:
total_bytes: 0
total_human: "~6 GB fp16 / ~2 GB Q4 (estimate)"
hardware_class:
min_vram_gb_q4: 3
min_vram_gb_fp16: 8
notes: "T0T1 edge / always-on"
axes: [B]
priority: high
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P0 edge instruct — tiny, huge ecosystem, good CPU/GPU floor for agents."
tags: [instruct, edge, llama]
companions: []
notes: "HF gated model — need accepted license on account before download."
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P0 recommendation from initial deep research."

View file

@ -0,0 +1,42 @@
id: nomic-ai__nomic-embed-text-v1.5__candidate
status: candidate
name: nomic-embed-text-v1.5
org: nomic-ai
source:
kind: huggingface
url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
revision: main
model_card_url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
license:
spdx: Apache-2.0
url: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
allows_offline_retention: true
allows_local_ops: true
allows_fine_tune: true
notes: ""
size:
total_bytes: 0
total_human: "<1 GB"
hardware_class:
min_vram_gb_q4: 1
min_vram_gb_fp16: 1
notes: "CPU-friendly"
axes: [B]
priority: high
collection:
approved_by: ""
approved_at: null
downloaded_at: null
downloaded_by: ""
storage_path: ""
brief_refs:
- research/2026-07-24-baseline-field-survey.md
reason: "P0 lightweight embed for A/B with BGE-M3; long-context text retrieval."
tags: [embedding, rag]
companions: []
notes: "If v2 text is preferred at collection time, update id/url and supersede this candidate."
history:
- at: "2026-07-24"
event: nominated
by: baseline-survey
detail: "P0 recommendation from initial deep research."

View file

@ -0,0 +1,134 @@
# Collection policy — open-weight reserve
**Status:** foundation
**Related:** `schema.yaml`, `docs/backup-storage-policy.md`, `INTENT.md`
---
## Purpose
Decide **what** enters the open-weight reserve, **who** may approve it, and
**when** a daily-brief candidate becomes a catalog entry with blobs on backup
storage.
---
## Goals
* Keep a **small, high-leverage** reserve — not a Hugging Face mirror
* Enforce **license and integrity** before download completes
* Match capacity to the backup storage soft quota
* Prefer models that serve axes **B** and **C**, plus strategic **A** open releases
---
## Eligibility (must pass all)
1. **Open weights** — weights obtainable under terms that allow offline retention for lab use
2. **Clear license** — SPDX or linkable license text; `allows_offline_retention: true`
3. **Stable provenance** — official org, tagged release, or commit revision (not anonymous drive-by reupload as sole source)
4. **Lab rationale** — written `reason` tied to at least one axis AD (usually B/C)
5. **Capacity** — estimated size fits under remaining soft quota (see backup storage policy)
Fail any gate → status `rejected` with reason, or never enter catalog.
---
## Priority rubric
| Priority | Guidance |
| -------- | -------- |
| **high** | Rare or strategically important; license/access risk of disappearance; uniquely strong for B/C at our hardware class; hard to re-obtain |
| **medium** | Clear lab use within 12 quarters; good quality/cost; easy enough to re-download but worth having cold |
| **low** | Nice to have; only collect if quota headroom is large and pull is cheap |
Daily brief **collection candidates** should set a suggested priority; approval may change it.
---
## Approval rule of thumb
| Estimated total size | Approval |
| -------------------- | -------- |
| **&lt; 5 GiB** | Operator or lab agent may collect after license check; catalog entry required before or immediately after |
| **540 GiB** | Explicit operator approval (chat, workplan task, or signed catalog `approved_by`) |
| **&gt; 40 GiB** | Operator approval **plus** check against soft quota and whether a smaller quant/variant suffices |
| **Any size if quota ≥ 70% used** | Operator approval required regardless of size |
| **Unclear license or ToS risk** | Do not collect; status `rejected` |
“Operator” means the human lab owner (or a documented delegate). Agents may
**nominate** (`status: candidate`) freely from briefs; they may **collect** only
within the &lt; 5 GiB band when licenses are unambiguous — otherwise stop at
`candidate` / `approved`.
---
## Lifecycle
```text
brief nominates
→ candidate (catalog YAML, no blobs required)
→ approved (license + size + quota OK)
→ collecting (download in staging/)
→ collected (blobs complete, checksums recorded, storage_path set)
→ verified (optional re-hash / smoke load)
→ superseded|evicted (replaced or removed; metadata kept)
```
Rejected candidates stay in catalog only if useful as a decision record; otherwise omit.
---
## What we prefer to collect
* Small/mid instruct and code models that fit the hardware envelope
* Strong embedding / rerank models for local RAG
* Base models known to fine-tune well under QLoRA/LoRA on lab GPUs
* Official quant releases when they are the supported distribution
* Adapters and tokenizers that unlock a reserved base (as companions)
## What we usually skip
* Duplicate quants of the same revision already reserved
* Huge models with no near-term local run/train path and no access-risk story
* Merges/repacks without provenance
* Datasets larger than model weights unless separately justified (default: out of band)
* Anything requiring acceptance flows we cannot satisfy offline
---
## Companions
Tokenizers, LoRA adapters, and small eval fixtures may be collected when:
* they are required to use a reserved base, or
* they are small (&lt; 1 GiB) and high leverage
Link via `companions` in the catalog schema.
---
## Brief integration
1. Brief section **Collection candidates** nominates items.
2. Operator/agent opens `inventory/catalog/{id}.yaml` with `status: candidate`.
3. Approval and download follow this policy and `docs/backup-storage-policy.md`.
4. Brief `brief_refs` on the entry point back to the nominating day(s).
---
## Eviction rule of thumb
When over quota or cleaning:
1. `low` priority, easily re-obtainable from still-live official URLs
2. Superseded revisions with a newer `verified` replacement
3. Never silent-delete: set `status: evicted`, clear or note `storage_path`, append `history`
---
## Non-goals
* Automatic bulk mirrors of entire orgs
* Collecting on every brief mention without priority
* Bypassing license gates for “research only” convenience

118
inventory/schema.yaml Normal file
View file

@ -0,0 +1,118 @@
# Freedom Intelligence — open-weight inventory entry schema
# Each collected (or tracked) model revision is one YAML file under catalog/
# Filename suggestion: {org}__{name}__{short_revision}.yaml
#
# Schema version documents field meaning for humans and future validators.
schema_version: "0.1.0"
# ---------------------------------------------------------------------------
# Example entry (illustrative only — not a real collection claim)
# ---------------------------------------------------------------------------
# id: meta-llama__Llama-3.2-3B-Instruct__abc1234
# status: candidate | approved | collecting | collected | verified | superseded | evicted | rejected
# name: Llama-3.2-3B-Instruct
# org: meta-llama
# source:
# kind: huggingface # huggingface | github_release | direct_url | other
# url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
# revision: abc1234def... # commit sha, tag, or release id
# model_card_url: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
# license:
# spdx: llama3.2 # or MIT, Apache-2.0, etc.; use "custom" + notes if needed
# url: https://...
# allows_offline_retention: true
# allows_local_ops: true
# allows_fine_tune: true
# notes: ""
# artifacts:
# - path: blobs/model.safetensors
# sha256: "..."
# bytes: 0
# size:
# total_bytes: 0
# total_human: "0 B"
# hardware_class:
# # Rough lab guidance — not a guarantee
# min_vram_gb_q4: 4
# min_vram_gb_fp16: 8
# notes: "fits consumer 8GB at Q4"
# axes: # why this matters to the lab
# - B
# - C
# priority: medium # low | medium | high
# collection:
# approved_by: ""
# approved_at: null # ISO-8601 date
# downloaded_at: null
# downloaded_by: ""
# storage_path: "" # absolute or facility-relative path on backup storage
# brief_refs: # briefs that nominated this candidate
# - briefs/2026/07/2026-07-23.md
# reason: "Strong small instruct base; clear license; homelab-fit."
# tags:
# - instruct
# - text
# companions: [] # optional related catalog ids (adapters, tokenizers)
# notes: ""
# history:
# - at: "2026-07-23"
# event: nominated
# by: operator
# detail: "From daily brief collection candidates."
# ---------------------------------------------------------------------------
# Required fields by status (normative for humans; tooling may enforce later)
# ---------------------------------------------------------------------------
# candidate: id, status, name, org, source.url, source.revision, license, reason, priority
# approved: + collection.approved_by, collection.approved_at
# collected: + collection.downloaded_at, collection.storage_path, size, artifacts[].sha256
# verified: + checksums re-read OK (notes or history event)
# rejected: + reason (why rejected)
# superseded / evicted: + history event; storage_path may be empty after eviction
field_reference:
id:
type: string
description: Stable catalog id; prefer {org}__{name}__{short_revision}
status:
type: enum
values: [candidate, approved, collecting, collected, verified, superseded, evicted, rejected]
name:
type: string
org:
type: string
source:
type: object
fields: [kind, url, revision, model_card_url]
license:
type: object
fields: [spdx, url, allows_offline_retention, allows_local_ops, allows_fine_tune, notes]
artifacts:
type: list
item_fields: [path, sha256, bytes]
size:
type: object
fields: [total_bytes, total_human]
hardware_class:
type: object
fields: [min_vram_gb_q4, min_vram_gb_fp16, notes]
axes:
type: list
values: [A, B, C, D]
priority:
type: enum
values: [low, medium, high]
collection:
type: object
fields: [approved_by, approved_at, downloaded_at, downloaded_by, storage_path, brief_refs]
reason:
type: string
tags:
type: list
companions:
type: list
notes:
type: string
history:
type: list
item_fields: [at, event, by, detail]