2026-07-24 00:15:27 +02:00
|
|
|
|
# Collection policy — open-weight reserve
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
**Status:** updated for 1 TB NAS + strategic capability reserve (2026-07-24)
|
|
|
|
|
|
**Related:** `schema.yaml`, `docs/backup-storage-policy.md`,
|
|
|
|
|
|
`docs/decisions/2026-07-24-nas-strategic-reserve.md`, `INTENT.md`
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Purpose
|
|
|
|
|
|
|
|
|
|
|
|
Decide **what** enters the open-weight reserve, **who** may approve it, and
|
2026-07-28 00:25:21 +02:00
|
|
|
|
**when** a daily-brief candidate becomes a catalog entry with blobs on the NAS.
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Goals
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
* Hold a **capability-first strategic reserve** of the best open weights we may
|
|
|
|
|
|
need later — **including models too large to run on current lab hardware**
|
|
|
|
|
|
* Also hold a **runnable spine** for day-to-day local ops and fine-tunes
|
2026-07-24 00:15:27 +02:00
|
|
|
|
* Enforce **license and integrity** before download completes
|
2026-07-28 00:25:21 +02:00
|
|
|
|
* Stay within the **1 TB NAS soft quota (850 GiB)** — quality over mirror volume
|
|
|
|
|
|
* Prefer official provenance; not a full Hugging Face scrape
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Two envelopes (do not conflate)
|
|
|
|
|
|
|
|
|
|
|
|
| Envelope | Question | Effect on collection |
|
|
|
|
|
|
| -------- | -------- | -------------------- |
|
|
|
|
|
|
| **Run** | Can we infer / FT this *now*? | Tags `hardware_class`, prioritizes R tier for local work |
|
|
|
|
|
|
| **Reserve** | Is this among the best open artifacts worth keeping *just in case*? | **Not gated by current VRAM** |
|
|
|
|
|
|
|
|
|
|
|
|
**Runnability is not an eligibility gate.** An unrunnable SOTA open MoE can be
|
|
|
|
|
|
priority **high** if license and quota allow.
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Eligibility (must pass all)
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
1. **Open weights** — obtainable under terms that allow offline retention for lab use
|
|
|
|
|
|
2. **Clear license** — SPDX or linkable license text; `allows_offline_retention: true`
|
|
|
|
|
|
3. **Stable provenance** — official org, tagged release, or commit revision
|
|
|
|
|
|
4. **Lab rationale** — written `reason` (capability SOTA, runnable spine, embed, FT base, …)
|
|
|
|
|
|
5. **Capacity** — estimated size fits under remaining soft quota (850 GiB)
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
Fail any gate → `rejected` or never enter catalog.
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
## Collection tiers
|
|
|
|
|
|
|
|
|
|
|
|
| Tier | Code | Meaning |
|
|
|
|
|
|
| ---- | ---- | ------- |
|
|
|
|
|
|
| **Runnable spine** | **R** | Default local chat/code/embed/FT bases; keep resident |
|
|
|
|
|
|
| **Strategic capability** | **S** | Most capable open models (often large); may be T4+ only to *run* |
|
|
|
|
|
|
| **Watch / optional** | **W** | Secondary; collect only with clear headroom |
|
|
|
|
|
|
|
|
|
|
|
|
Catalog field: use `tags` including `tier-r` / `tier-s` / `tier-w` and
|
|
|
|
|
|
`priority: high|medium|low`.
|
|
|
|
|
|
|
|
|
|
|
|
### Priority rubric
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
| Priority | Guidance |
|
|
|
|
|
|
| -------- | -------- |
|
2026-07-28 00:25:21 +02:00
|
|
|
|
| **high** | Top open capability (S) or essential runnable spine (R); hard to re-obtain; license/access risk |
|
|
|
|
|
|
| **medium** | Strong but not unique; mid-size upgrades; companions (rerankers) |
|
|
|
|
|
|
| **low** | Nice-to-have; W tier |
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Approval rule of thumb
|
|
|
|
|
|
|
|
|
|
|
|
| Estimated total size | Approval |
|
|
|
|
|
|
| -------------------- | -------- |
|
2026-07-28 00:25:21 +02:00
|
|
|
|
| **< 5 GiB** | Operator or lab agent after license check |
|
|
|
|
|
|
| **5–40 GiB** | Explicit operator approval |
|
|
|
|
|
|
| **> 40 GiB** | Operator approval + remaining soft-quota check |
|
|
|
|
|
|
| **> 200 GiB (typical S giants)** | Operator approval + written note on which other S models may need to wait |
|
|
|
|
|
|
| **Any size if quota ≥ 70% used** | Operator approval required |
|
|
|
|
|
|
| **Unclear license or ToS risk** | Do not collect; `rejected` |
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
Agents may **nominate** freely; they **collect** only in the < 5 GiB band with
|
|
|
|
|
|
unambiguous licenses unless the operator has approved the catalog entry.
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
## What we prefer to collect
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
### Runnable spine (R)
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
* Small/mid instruct and code models that fit the run envelope
|
|
|
|
|
|
* Strong embedding / rerank models for local RAG
|
|
|
|
|
|
* Bases known to fine-tune well under QLoRA on lab GPUs
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
### Strategic capability (S)
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
* **Best available open general / reasoning / code weights** at the frontier of open
|
|
|
|
|
|
* Large MoE or dense models even if current infra cannot serve them
|
|
|
|
|
|
* Prefer official compressed releases (FP8, published quant) when full precision
|
|
|
|
|
|
would exhaust the 1 TB NAS
|
|
|
|
|
|
* One clear “best open” per capability niche is better than five near-duplicates
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
### Companions
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
Tokenizers, LoRA adapters, small eval fixtures when required to use a reserved base.
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
## What we usually skip
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
* Duplicate quants of the same revision already reserved
|
|
|
|
|
|
* Anonymous merges/repacks without provenance
|
|
|
|
|
|
* Closed weights
|
|
|
|
|
|
* Entire org mirrors
|
|
|
|
|
|
* Giant pretraining corpora (default out of band unless separately justified)
|
|
|
|
|
|
* Anything whose license forbids offline retention
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Lifecycle
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
```text
|
|
|
|
|
|
brief nominates
|
|
|
|
|
|
→ candidate
|
|
|
|
|
|
→ approved
|
|
|
|
|
|
→ collecting (NAS staging/)
|
|
|
|
|
|
→ collected
|
|
|
|
|
|
→ verified
|
|
|
|
|
|
→ superseded|evicted
|
|
|
|
|
|
```
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Brief integration
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
1. Brief **Collection candidates** nominates R/S/W.
|
|
|
|
|
|
2. Catalog YAML under `inventory/catalog/`.
|
|
|
|
|
|
3. Download only after storage path is pinned (`docs/backup-storage-policy.md`) and approval rules pass.
|
|
|
|
|
|
4. `collection.brief_refs` / research refs for provenance of the nomination.
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Eviction rule of thumb
|
|
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
When over soft quota:
|
2026-07-24 00:15:27 +02:00
|
|
|
|
|
2026-07-28 00:25:21 +02:00
|
|
|
|
1. **W** tier and easily re-obtainable duplicates
|
|
|
|
|
|
2. Superseded revisions with a stronger verified successor
|
|
|
|
|
|
3. Never silent-delete **S** SOTA or sole **R** spine without operator note
|
|
|
|
|
|
4. Always set `status: evicted` and append `history`
|