Establish Freedom Intelligence lab foundation and baseline research.
Add INTENT/SCOPE, daily-brief playbook, activity-core definition (disabled), workplans FI-WP-0001..0003, baseline field survey with open-weight collection recommendations, and inventory catalog candidates for the model reserve.
This commit is contained in:
parent
1dc1e09517
commit
b5f911140b
28 changed files with 2651 additions and 1 deletions
161
docs/backup-storage-policy.md
Normal file
161
docs/backup-storage-policy.md
Normal file
|
|
@ -0,0 +1,161 @@
|
|||
# Backup storage policy — open-weight model reserve
|
||||
|
||||
**Status:** foundation draft (concrete path and budget **TBD**)
|
||||
**Related:** `INTENT.md`, `SCOPE.md`, `inventory/collection-policy.md`
|
||||
**Adjacent:** `disaster-control` BackupPolicy (platform backups — different concern)
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define where Freedom Intelligence stores **open-weight model blobs** (and justified
|
||||
companions), how much capacity we allow, and how this reserve relates to other
|
||||
lab backup facilities.
|
||||
|
||||
Git never stores weight tensors. Git stores:
|
||||
|
||||
* inventory catalog metadata (`inventory/catalog/`)
|
||||
* policies and schemas
|
||||
* research briefs
|
||||
|
||||
---
|
||||
|
||||
## Storage class
|
||||
|
||||
| Requirement | Policy |
|
||||
| ----------- | ------ |
|
||||
| **Class** | Backup / bulk durable storage — **not** hot cluster PVCs or app disks |
|
||||
| **Durability** | Same or better retention posture as other lab bulk assets; prefer media that survives workstation rebuilds |
|
||||
| **Performance** | Sequential read for restore/training pull is enough; low latency not required |
|
||||
| **Access** | Operator and approved lab hosts only; not a public mirror |
|
||||
| **Separation** | Do not co-mingle with age-encrypted operational backups (Forgejo dumps, k3s state) without a clear subdirectory and different lifecycle rules |
|
||||
|
||||
Platform backup paths such as `/opt/backup/railiance/{infra,cluster}/` and
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. The model
|
||||
reserve may live on the **same physical facility** only if isolated by path and
|
||||
quota so large weight pulls cannot crowd out restore media.
|
||||
|
||||
---
|
||||
|
||||
## Target location (to pin)
|
||||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| **Host / facility** | `TBD` — operator pin (e.g. backup NAS, off-cluster bulk volume, dedicated disk) |
|
||||
| **Base path or bucket** | `TBD` — suggested shape: `…/freedom-intelligence/models/` |
|
||||
| **Layout under base** | See [On-disk layout](#on-disk-layout) |
|
||||
| **Mount on lab hosts** | `TBD` |
|
||||
| **Credentials** | `TBD` — if remote: OpenBao or existing backup credential lane; never commit secrets |
|
||||
|
||||
Until pinned, **do not** bulk-download multi-GB models into this git workspace or
|
||||
into hot root filesystems.
|
||||
|
||||
### Suggested path shape (non-binding)
|
||||
|
||||
```text
|
||||
{BACKUP_ROOT}/freedom-intelligence/
|
||||
├── models/
|
||||
│ └── {org}__{name}/
|
||||
│ └── {revision}/
|
||||
│ ├── blobs/ # weight files, shards
|
||||
│ └── MANIFEST.txt # optional local copy of hashes
|
||||
├── companions/ # adapters, tokenizers only when separate
|
||||
└── staging/ # incomplete downloads; not catalog-ready
|
||||
```
|
||||
|
||||
Map each `{org}__{name}/{revision}` to an inventory catalog entry.
|
||||
|
||||
---
|
||||
|
||||
## On-disk layout
|
||||
|
||||
| Path element | Rule |
|
||||
| ------------ | ---- |
|
||||
| `org__name` | Hugging Face-style id with `/` → `__` |
|
||||
| `revision` | Git commit SHA, tag, or release id used at download time |
|
||||
| `blobs/` | Actual files; prefer original names from source |
|
||||
| `staging/` | Incomplete transfers; purge or resume; never mark collected until complete + verified |
|
||||
|
||||
---
|
||||
|
||||
## Capacity budget
|
||||
|
||||
| Parameter | Policy |
|
||||
| --------- | ------ |
|
||||
| **Soft quota** | `TBD` GiB/TiB — operator pin based on free backup capacity |
|
||||
| **Hard stop** | No new collection when soft quota exceeded unless operator raises budget |
|
||||
| **Per-pull threshold** | See `inventory/collection-policy.md` (size gates approval) |
|
||||
| **Growth review** | Revisit quota when catalog total exceeds 70% of soft quota |
|
||||
| **Eviction** | Prefer archive/delete lowest-priority, easily re-obtainable revisions first; record eviction in catalog history |
|
||||
|
||||
---
|
||||
|
||||
## Integrity and provenance
|
||||
|
||||
For every completed collection:
|
||||
|
||||
1. Record source URL and revision in the catalog entry.
|
||||
2. Store checksums (`sha256` of each blob or upstream manifest digest).
|
||||
3. Record download date (UTC) and downloader identity.
|
||||
4. Prefer official org releases over anonymous re-uploads.
|
||||
5. Keep license text or SPDX id in catalog; refuse unclear licenses.
|
||||
|
||||
Verification command examples belong in tooling later; policy only requires that
|
||||
**catalog claims match on-disk checksums** before status `collected`.
|
||||
|
||||
---
|
||||
|
||||
## Retention
|
||||
|
||||
| Class | Retention |
|
||||
| ----- | --------- |
|
||||
| **Strategic reserve** (high priority, hard to re-obtain) | Keep until explicit deprecation |
|
||||
| **Working set** (common bases for training experiments) | Keep while in active use + one superseded revision optional |
|
||||
| **Staging** | Max 14 days incomplete, then purge |
|
||||
| **Deprecated** | Metadata retained in catalog with status `evicted` or `superseded`; blobs may be deleted |
|
||||
|
||||
---
|
||||
|
||||
## Encryption and offsite
|
||||
|
||||
| Topic | Policy |
|
||||
| ----- | ------ |
|
||||
| **At rest** | Follow host/facility default; extra age/GPG of multi-hundred-GB trees is optional and costly |
|
||||
| **In transit** | HTTPS or trusted lab network only |
|
||||
| **Offsite copy** | Optional later; not required for foundation. If added, coordinate with `disaster-control` so model reserve does not break operational backup SLAs |
|
||||
|
||||
---
|
||||
|
||||
## What must not live here
|
||||
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or training corpora with personal data (domain datasets need their own policy)
|
||||
* Operational backups (databases, k3s state, Forgejo dumps)
|
||||
* Git LFS dumps of full model trees as a substitute for backup storage
|
||||
|
||||
---
|
||||
|
||||
## Operator checklist to pin TBD fields
|
||||
|
||||
- [ ] Choose facility and base path
|
||||
- [ ] Confirm free capacity and set soft quota
|
||||
- [ ] Document mount/access for lab hosts
|
||||
- [ ] Confirm credential path (if any)
|
||||
- [ ] Create empty `models/` and `staging/` directories
|
||||
- [ ] Update this file: replace `TBD` rows with concrete values
|
||||
- [ ] Note pin date and operator in revision history below
|
||||
|
||||
### Pin log
|
||||
|
||||
| Date | Operator | Change |
|
||||
| ---- | -------- | ------ |
|
||||
| 2026-07-23 | foundation | Policy created; path and quota unpinned |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to disaster-control
|
||||
|
||||
`disaster-control` owns **platform resilience** (what to restore after loss).
|
||||
Freedom Intelligence owns **which open weights we choose to retain** and their
|
||||
inventory. Shared facilities are fine; shared lifecycle rules are not automatic —
|
||||
model reserve is large, slow-changing, and rarely needed for emergency restore.
|
||||
Loading…
Add table
Add a link
Reference in a new issue