Register reserve under agents hub and pin VAULT storage.
Classify freedom-intelligence for State Hub (agents domain), sync FI-WP-0001..0003 with hub IDs, enrich catalog entries with profile/SWOT, and pin the open-weight reserve to D:\vault\coulomb\freedom-intelligence\ with strategic S-tier policy.
This commit is contained in:
parent
00f469662b
commit
a83ef0a79a
29 changed files with 1343 additions and 279 deletions
|
|
@ -1,7 +1,8 @@
|
|||
# Backup storage policy — open-weight model reserve
|
||||
|
||||
**Status:** foundation draft (concrete path and budget **TBD**)
|
||||
**Related:** `INTENT.md`, `SCOPE.md`, `inventory/collection-policy.md`
|
||||
**Status:** facility + path pinned (workstation VAULT HD)
|
||||
**Related:** `INTENT.md`, `SCOPE.md`, `inventory/collection-policy.md`,
|
||||
`docs/decisions/2026-07-24-nas-strategic-reserve.md`
|
||||
**Adjacent:** `disaster-control` BackupPolicy (platform backups — different concern)
|
||||
|
||||
---
|
||||
|
|
@ -25,35 +26,42 @@ Git never stores weight tensors. Git stores:
|
|||
| Requirement | Policy |
|
||||
| ----------- | ------ |
|
||||
| **Class** | Backup / bulk durable storage — **not** hot cluster PVCs or app disks |
|
||||
| **Durability** | Same or better retention posture as other lab bulk assets; prefer media that survives workstation rebuilds |
|
||||
| **Facility** | Workstation VAULT HD (`D:`) under `D:\vault\coulomb\` (interim pin 2026-07-28) |
|
||||
| **Durability** | Survives WSL rebuilds; weights live on the VAULT volume, not the git/WSL root |
|
||||
| **Performance** | Sequential read for restore/training pull is enough; low latency not required |
|
||||
| **Access** | Operator and approved lab hosts only; not a public mirror |
|
||||
| **Separation** | Do not co-mingle with age-encrypted operational backups (Forgejo dumps, k3s state) without a clear subdirectory and different lifecycle rules |
|
||||
|
||||
Platform backup paths such as `/opt/backup/railiance/{infra,cluster}/` and
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. The model
|
||||
reserve may live on the **same physical facility** only if isolated by path and
|
||||
quota so large weight pulls cannot crowd out restore media.
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. Model weights
|
||||
belong on the **VAULT model tree**, not on railiance hot disks.
|
||||
|
||||
---
|
||||
|
||||
## Target location (to pin)
|
||||
## Target location
|
||||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| **Host / facility** | `TBD` — operator pin (e.g. backup NAS, off-cluster bulk volume, dedicated disk) |
|
||||
| **Base path or bucket** | `TBD` — suggested shape: `…/freedom-intelligence/models/` |
|
||||
| **Host / facility** | Workstation-attached bulk HD **VAULT** (Windows `D:`) |
|
||||
| **Raw capacity** | ~1.86 TB total (~1.76 TB free as of 2026-07-28 pin) |
|
||||
| **Base path (Windows)** | `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| **Base path (WSL, when D: automounted)** | `/mnt/d/vault/coulomb/freedom-intelligence/` |
|
||||
| **Layout under base** | See [On-disk layout](#on-disk-layout) |
|
||||
| **Mount on lab hosts** | `TBD` |
|
||||
| **Credentials** | `TBD` — if remote: OpenBao or existing backup credential lane; never commit secrets |
|
||||
| **Mount on lab hosts** | Local attach on operator workstation; WSL via drvfs `D:` → `/mnt/d` when enabled |
|
||||
| **Credentials** | Local filesystem ACL on the workstation HD; never commit secrets |
|
||||
|
||||
Until pinned, **do not** bulk-download multi-GB models into this git workspace or
|
||||
into hot root filesystems.
|
||||
**Interim note:** Earlier policy assumed a dedicated ~1 TB NAS. Operator chose this
|
||||
VAULT volume under `D:\vault\coulomb\` for now. Soft quota **850 GiB** for the
|
||||
model tree still applies (self-imposed discipline; disk is larger).
|
||||
|
||||
### Suggested path shape (non-binding)
|
||||
Do **not** bulk-download multi-GB models into this git workspace or into hot
|
||||
root filesystems (`/` on WSL, `C:`).
|
||||
|
||||
### Path shape (pinned)
|
||||
|
||||
```text
|
||||
{BACKUP_ROOT}/freedom-intelligence/
|
||||
D:\vault\coulomb\freedom-intelligence\ # Windows
|
||||
/mnt/d/vault/coulomb/freedom-intelligence/ # WSL when D: mounted
|
||||
├── models/
|
||||
│ └── {org}__{name}/
|
||||
│ └── {revision}/
|
||||
|
|
@ -65,6 +73,8 @@ into hot root filesystems.
|
|||
|
||||
Map each `{org}__{name}/{revision}` to an inventory catalog entry.
|
||||
|
||||
Directories `models/`, `staging/`, and `companions/` were created 2026-07-28.
|
||||
|
||||
---
|
||||
|
||||
## On-disk layout
|
||||
|
|
@ -76,17 +86,33 @@ Map each `{org}__{name}/{revision}` to an inventory catalog entry.
|
|||
| `blobs/` | Actual files; prefer original names from source |
|
||||
| `staging/` | Incomplete transfers; purge or resume; never mark collected until complete + verified |
|
||||
|
||||
Optional layout tags (directory name or catalog field):
|
||||
|
||||
| Tag | Meaning |
|
||||
| --- | ------- |
|
||||
| runnable | Fits current run envelope (R tier) |
|
||||
| strategic | Capability reserve; may exceed current hardware (S tier) |
|
||||
|
||||
---
|
||||
|
||||
## Capacity budget
|
||||
|
||||
| Parameter | Policy |
|
||||
| --------- | ------ |
|
||||
| **Soft quota** | `TBD` GiB/TiB — operator pin based on free backup capacity |
|
||||
| **Hard stop** | No new collection when soft quota exceeded unless operator raises budget |
|
||||
| **Per-pull threshold** | See `inventory/collection-policy.md` (size gates approval) |
|
||||
| **Growth review** | Revisit quota when catalog total exceeds 70% of soft quota |
|
||||
| **Eviction** | Prefer archive/delete lowest-priority, easily re-obtainable revisions first; record eviction in catalog history |
|
||||
| **Device** | VAULT HD (`D:`) — model tree under `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| **Soft quota (model reserve)** | **850 GiB** (self-imposed; disk is larger) |
|
||||
| **Hard stop** | **920 GiB** used under the freedom-intelligence base path |
|
||||
| **Growth review** | When catalog total ≥ **70% soft quota (~595 GiB)** |
|
||||
| **Per-pull threshold** | See `inventory/collection-policy.md` |
|
||||
| **Eviction** | Prefer drop **W** (watch) and easily re-obtained quants; protect unique **S** SOTA bases and the **R** spine |
|
||||
|
||||
### Portfolio guidance on 1 TB
|
||||
|
||||
* Always keep the **runnable spine (R / former P0)** resident.
|
||||
* Use remaining space for a **small number of most-capable open models (S)**,
|
||||
preferably well-provenance compressed weights when full precision would exhaust the disk.
|
||||
* Do **not** attempt to mirror entire HF orgs.
|
||||
* Full bf16 of multiple 600B-class models will **not** fit; prioritize top capability identities.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -100,8 +126,7 @@ For every completed collection:
|
|||
4. Prefer official org releases over anonymous re-uploads.
|
||||
5. Keep license text or SPDX id in catalog; refuse unclear licenses.
|
||||
|
||||
Verification command examples belong in tooling later; policy only requires that
|
||||
**catalog claims match on-disk checksums** before status `collected`.
|
||||
Verification: **catalog claims must match on-disk checksums** before status `collected`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -109,10 +134,11 @@ Verification command examples belong in tooling later; policy only requires that
|
|||
|
||||
| Class | Retention |
|
||||
| ----- | --------- |
|
||||
| **Strategic reserve** (high priority, hard to re-obtain) | Keep until explicit deprecation |
|
||||
| **Working set** (common bases for training experiments) | Keep while in active use + one superseded revision optional |
|
||||
| **Strategic capability (S)** | Keep until explicit deprecation or displacement by a clearly stronger open successor |
|
||||
| **Runnable spine (R)** | Keep while still the lab default for local ops / FT |
|
||||
| **Working set** | Active experiment bases + optional one superseded revision |
|
||||
| **Staging** | Max 14 days incomplete, then purge |
|
||||
| **Deprecated** | Metadata retained in catalog with status `evicted` or `superseded`; blobs may be deleted |
|
||||
| **Deprecated / evicted** | Metadata retained in catalog; blobs may be deleted |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -120,42 +146,35 @@ Verification command examples belong in tooling later; policy only requires that
|
|||
|
||||
| Topic | Policy |
|
||||
| ----- | ------ |
|
||||
| **At rest** | Follow host/facility default; extra age/GPG of multi-hundred-GB trees is optional and costly |
|
||||
| **In transit** | HTTPS or trusted lab network only |
|
||||
| **Offsite copy** | Optional later; not required for foundation. If added, coordinate with `disaster-control` so model reserve does not break operational backup SLAs |
|
||||
| **At rest** | NAS default; extra age/GPG of multi-hundred-GB trees optional |
|
||||
| **In transit** | Trusted lab network / local attach |
|
||||
| **Offsite copy** | Optional later; coordinate with disaster-control if added so model bulk does not break ops backup SLAs |
|
||||
|
||||
---
|
||||
|
||||
## What must not live here
|
||||
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or training corpora with personal data (domain datasets need their own policy)
|
||||
* Operational backups (databases, k3s state, Forgejo dumps)
|
||||
* Git LFS dumps of full model trees as a substitute for backup storage
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or ungoverned training corpora with personal data
|
||||
* Operational backups (databases, k3s state, Forgejo dumps) as the primary home
|
||||
* Git LFS dumps of full model trees as a substitute for the NAS
|
||||
|
||||
---
|
||||
|
||||
## Operator checklist to pin TBD fields
|
||||
## Operator checklist
|
||||
|
||||
- [ ] Choose facility and base path
|
||||
- [ ] Confirm free capacity and set soft quota
|
||||
- [ ] Document mount/access for lab hosts
|
||||
- [ ] Confirm credential path (if any)
|
||||
- [ ] Create empty `models/` and `staging/` directories
|
||||
- [ ] Update this file: replace `TBD` rows with concrete values
|
||||
- [ ] Note pin date and operator in revision history below
|
||||
- [x] Choose facility: bulk durable storage for model reserve (2026-07-24 intent)
|
||||
- [x] Set soft quota: **850 GiB** / hard stop **920 GiB**
|
||||
- [x] Pin path: **`D:\vault\coulomb\freedom-intelligence\`** (2026-07-28)
|
||||
- [x] Create `models/`, `staging/`, `companions/`
|
||||
- [x] Document Windows + WSL path forms above
|
||||
- [x] Ensure WSL can see `D:` (`/mnt/d`) when downloads run from Linux (verified 2026-07-28)
|
||||
|
||||
### Pin log
|
||||
|
||||
| Date | Operator | Change |
|
||||
| ---- | -------- | ------ |
|
||||
| 2026-07-23 | foundation | Policy created; path and quota unpinned |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to disaster-control
|
||||
|
||||
`disaster-control` owns **platform resilience** (what to restore after loss).
|
||||
Freedom Intelligence owns **which open weights we choose to retain** and their
|
||||
inventory. Shared facilities are fine; shared lifecycle rules are not automatic —
|
||||
model reserve is large, slow-changing, and rarely needed for emergency restore.
|
||||
| 2026-07-24 | foundation | Policy created; path unpinned |
|
||||
| 2026-07-24 | operator direction | Facility intent = 1 TB local NAS; soft quota 850 GiB; strategic unrunnable models in scope |
|
||||
| 2026-07-28 | operator direction | **Interim pin:** workstation VAULT HD `D:\vault\coulomb\freedom-intelligence\`; layout created; soft quota retained |
|
||||
| 2026-07-28 | operator | WSL mount verified: `D:` → `/mnt/d` (1.9T, writable) |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue