freedom-intelligence/docs/backup-storage-policy.md
tegwick b5f911140b Establish Freedom Intelligence lab foundation and baseline research.
Add INTENT/SCOPE, daily-brief playbook, activity-core definition (disabled),
workplans FI-WP-0001..0003, baseline field survey with open-weight collection
recommendations, and inventory catalog candidates for the model reserve.
2026-07-24 00:15:27 +02:00

6.1 KiB

Backup storage policy — open-weight model reserve

Status: foundation draft (concrete path and budget TBD)
Related: INTENT.md, SCOPE.md, inventory/collection-policy.md
Adjacent: disaster-control BackupPolicy (platform backups — different concern)


Purpose

Define where Freedom Intelligence stores open-weight model blobs (and justified companions), how much capacity we allow, and how this reserve relates to other lab backup facilities.

Git never stores weight tensors. Git stores:

  • inventory catalog metadata (inventory/catalog/)
  • policies and schemas
  • research briefs

Storage class

Requirement Policy
Class Backup / bulk durable storage — not hot cluster PVCs or app disks
Durability Same or better retention posture as other lab bulk assets; prefer media that survives workstation rebuilds
Performance Sequential read for restore/training pull is enough; low latency not required
Access Operator and approved lab hosts only; not a public mirror
Separation Do not co-mingle with age-encrypted operational backups (Forgejo dumps, k3s state) without a clear subdirectory and different lifecycle rules

Platform backup paths such as /opt/backup/railiance/{infra,cluster}/ and ~/.cache/railiance/backups/ are operational recovery lanes. The model reserve may live on the same physical facility only if isolated by path and quota so large weight pulls cannot crowd out restore media.


Target location (to pin)

Field Value
Host / facility TBD — operator pin (e.g. backup NAS, off-cluster bulk volume, dedicated disk)
Base path or bucket TBD — suggested shape: …/freedom-intelligence/models/
Layout under base See On-disk layout
Mount on lab hosts TBD
Credentials TBD — if remote: OpenBao or existing backup credential lane; never commit secrets

Until pinned, do not bulk-download multi-GB models into this git workspace or into hot root filesystems.

Suggested path shape (non-binding)

{BACKUP_ROOT}/freedom-intelligence/
├── models/
│   └── {org}__{name}/
│       └── {revision}/
│           ├── blobs/          # weight files, shards
│           └── MANIFEST.txt    # optional local copy of hashes
├── companions/                 # adapters, tokenizers only when separate
└── staging/                    # incomplete downloads; not catalog-ready

Map each {org}__{name}/{revision} to an inventory catalog entry.


On-disk layout

Path element Rule
org__name Hugging Face-style id with /__
revision Git commit SHA, tag, or release id used at download time
blobs/ Actual files; prefer original names from source
staging/ Incomplete transfers; purge or resume; never mark collected until complete + verified

Capacity budget

Parameter Policy
Soft quota TBD GiB/TiB — operator pin based on free backup capacity
Hard stop No new collection when soft quota exceeded unless operator raises budget
Per-pull threshold See inventory/collection-policy.md (size gates approval)
Growth review Revisit quota when catalog total exceeds 70% of soft quota
Eviction Prefer archive/delete lowest-priority, easily re-obtainable revisions first; record eviction in catalog history

Integrity and provenance

For every completed collection:

  1. Record source URL and revision in the catalog entry.
  2. Store checksums (sha256 of each blob or upstream manifest digest).
  3. Record download date (UTC) and downloader identity.
  4. Prefer official org releases over anonymous re-uploads.
  5. Keep license text or SPDX id in catalog; refuse unclear licenses.

Verification command examples belong in tooling later; policy only requires that catalog claims match on-disk checksums before status collected.


Retention

Class Retention
Strategic reserve (high priority, hard to re-obtain) Keep until explicit deprecation
Working set (common bases for training experiments) Keep while in active use + one superseded revision optional
Staging Max 14 days incomplete, then purge
Deprecated Metadata retained in catalog with status evicted or superseded; blobs may be deleted

Encryption and offsite

Topic Policy
At rest Follow host/facility default; extra age/GPG of multi-hundred-GB trees is optional and costly
In transit HTTPS or trusted lab network only
Offsite copy Optional later; not required for foundation. If added, coordinate with disaster-control so model reserve does not break operational backup SLAs

What must not live here

  • Closed weights or artifacts whose terms forbid offline retention
  • Secrets, API keys, customer data, or training corpora with personal data (domain datasets need their own policy)
  • Operational backups (databases, k3s state, Forgejo dumps)
  • Git LFS dumps of full model trees as a substitute for backup storage

Operator checklist to pin TBD fields

  • Choose facility and base path
  • Confirm free capacity and set soft quota
  • Document mount/access for lab hosts
  • Confirm credential path (if any)
  • Create empty models/ and staging/ directories
  • Update this file: replace TBD rows with concrete values
  • Note pin date and operator in revision history below

Pin log

Date Operator Change
2026-07-23 foundation Policy created; path and quota unpinned

Relationship to disaster-control

disaster-control owns platform resilience (what to restore after loss). Freedom Intelligence owns which open weights we choose to retain and their inventory. Shared facilities are fine; shared lifecycle rules are not automatic — model reserve is large, slow-changing, and rarely needed for emergency restore.