Make the sensing loop durable and pin the reserve to Scaleway
The weekday brief only counts when the file is on origin/main. Hub events without git objects are failures. The open-weight SoT moves from the VAULT 850 GiB quota to a dedicated Scaleway bucket (One Zone IA for R, Glacier for S) so V4-Flash and K3 are economically in-scope. Catalogs the real DeepSeek-V4-Flash-0731 (MIT ~167 GiB MoE, not 12B) and nominates Qwen3.8-27B. Bucket create remains FI-WP-0004-T08. Assistant: grok Assistant-Session: 01a09c6a-1cfc-75b1-a78d-c13eaf22241d
This commit is contained in:
parent
8832652ad3
commit
a78187c33e
18 changed files with 1125 additions and 226 deletions
|
|
@ -1,17 +1,19 @@
|
|||
# Backup storage policy — open-weight model reserve
|
||||
|
||||
**Status:** facility + path pinned (workstation VAULT HD)
|
||||
**Status:** facility pin **Scaleway Object Storage** (2026-09-13)
|
||||
**Related:** `INTENT.md`, `SCOPE.md`, `inventory/collection-policy.md`,
|
||||
`docs/decisions/2026-07-24-nas-strategic-reserve.md`
|
||||
**Adjacent:** `disaster-control` BackupPolicy (platform backups — different concern)
|
||||
`docs/decisions/2026-09-13-scaleway-object-reserve.md`
|
||||
**Supersedes storage pin:** 2026-07-28 VAULT HD (now staging only)
|
||||
**Adjacent:** `disaster-control` BackupPolicy and `rapp-postgres` WAL
|
||||
archive — **different buckets, different lifecycles**
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define where Freedom Intelligence stores **open-weight model blobs** (and justified
|
||||
companions), how much capacity we allow, and how this reserve relates to other
|
||||
lab backup facilities.
|
||||
Define where Freedom Intelligence stores **open-weight model blobs** (and
|
||||
justified companions), how much we allow ourselves to spend, and how this
|
||||
reserve relates to other lab backup facilities.
|
||||
|
||||
Git never stores weight tensors. Git stores:
|
||||
|
||||
|
|
@ -25,16 +27,16 @@ Git never stores weight tensors. Git stores:
|
|||
|
||||
| Requirement | Policy |
|
||||
| ----------- | ------ |
|
||||
| **Class** | Backup / bulk durable storage — **not** hot cluster PVCs or app disks |
|
||||
| **Facility** | Workstation VAULT HD (`D:`) under `D:\vault\coulomb\` (interim pin 2026-07-28) |
|
||||
| **Durability** | Survives WSL rebuilds; weights live on the VAULT volume, not the git/WSL root |
|
||||
| **Performance** | Sequential read for restore/training pull is enough; low latency not required |
|
||||
| **Access** | Operator and approved lab hosts only; not a public mirror |
|
||||
| **Separation** | Do not co-mingle with age-encrypted operational backups (Forgejo dumps, k3s state) without a clear subdirectory and different lifecycle rules |
|
||||
| **Class** | Backup / bulk durable object storage — **not** hot cluster PVCs |
|
||||
| **Facility** | Scaleway Object Storage `nl-ams` via `reef-storage` |
|
||||
| **Durability** | Provider 11-nines claim; independent of Host Europe `railiance01` and of the workstation HD |
|
||||
| **Performance** | Sequential restore for training/inference pull is enough |
|
||||
| **Access** | Operator and approved lab hosts; not a public mirror |
|
||||
| **Separation** | **Never** the postgres backup bucket (`railiance-platform-pg-backup` expires at 30 days) |
|
||||
|
||||
Platform backup paths such as `/opt/backup/railiance/{infra,cluster}/` and
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. Model weights
|
||||
belong on the **VAULT model tree**, not on railiance hot disks.
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. Model
|
||||
weights belong in the FI object bucket.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -42,56 +44,33 @@ belong on the **VAULT model tree**, not on railiance hot disks.
|
|||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| **Host / facility** | Workstation-attached bulk HD **VAULT** (Windows `D:`) |
|
||||
| **Raw capacity** | ~1.86 TB total (~1.76 TB free as of 2026-07-28 pin) |
|
||||
| **Base path (Windows)** | `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| **Base path (WSL, when D: automounted)** | `/mnt/d/vault/coulomb/freedom-intelligence/` |
|
||||
| **Layout under base** | See [On-disk layout](#on-disk-layout) |
|
||||
| **Mount on lab hosts** | Local attach on operator workstation; WSL via drvfs `D:` → `/mnt/d` when enabled |
|
||||
| **Credentials** | Local filesystem ACL on the workstation HD; never commit secrets |
|
||||
| **Facility** | Scaleway Object Storage, region `nl-ams` |
|
||||
| **Endpoint** | `https://s3.nl-ams.scw.cloud` |
|
||||
| **Bucket** | `railiance-fi-open-weight-reserve` (planned until FI-WP-0004-T08) |
|
||||
| **Attributes** | `reef:storage/substrate/object-stores/fi-open-weight-reserve.yaml` |
|
||||
| **Credentials** | OpenBao / `railiance-platform` — never git |
|
||||
|
||||
**Interim note:** Earlier policy assumed a dedicated ~1 TB NAS. Operator chose this
|
||||
VAULT volume under `D:\vault\coulomb\` for now. Soft quota **850 GiB** for the
|
||||
model tree still applies (self-imposed discipline; disk is larger).
|
||||
**Local staging (not SoT):** workstation VAULT HD
|
||||
`D:\vault\coulomb\freedom-intelligence\` /
|
||||
`/mnt/d/vault/coulomb/freedom-intelligence/` for `huggingface_hub`
|
||||
downloads and an optional R-spine cache.
|
||||
|
||||
Do **not** bulk-download multi-GB models into this git workspace or into hot
|
||||
root filesystems (`/` on WSL, `C:`).
|
||||
Do **not** bulk-download multi-GB models into this git workspace or into
|
||||
hot root filesystems (`/` on WSL, `C:`, railiance PVCs).
|
||||
|
||||
### Path shape (pinned)
|
||||
### Prefix shape (pinned)
|
||||
|
||||
```text
|
||||
D:\vault\coulomb\freedom-intelligence\ # Windows
|
||||
/mnt/d/vault/coulomb/freedom-intelligence/ # WSL when D: mounted
|
||||
├── models/
|
||||
│ └── {org}__{name}/
|
||||
│ └── {revision}/
|
||||
│ ├── blobs/ # weight files, shards
|
||||
│ └── MANIFEST.txt # optional local copy of hashes
|
||||
├── companions/ # adapters, tokenizers only when separate
|
||||
└── staging/ # incomplete downloads; not catalog-ready
|
||||
s3://railiance-fi-open-weight-reserve/
|
||||
├── models/{org}__{name}/{revision}/ # R / W (One Zone IA)
|
||||
├── strategic/{org}__{name}/{revision}/ # S (Glacier)
|
||||
├── companions/
|
||||
├── staging/
|
||||
└── manifests/{org}__{name}/{revision}/MANIFEST.json
|
||||
```
|
||||
|
||||
Map each `{org}__{name}/{revision}` to an inventory catalog entry.
|
||||
|
||||
Directories `models/`, `staging/`, and `companions/` were created 2026-07-28.
|
||||
|
||||
---
|
||||
|
||||
## On-disk layout
|
||||
|
||||
| Path element | Rule |
|
||||
| ------------ | ---- |
|
||||
| `org__name` | Hugging Face-style id with `/` → `__` |
|
||||
| `revision` | Git commit SHA, tag, or release id used at download time |
|
||||
| `blobs/` | Actual files; prefer original names from source |
|
||||
| `staging/` | Incomplete transfers; purge or resume; never mark collected until complete + verified |
|
||||
|
||||
Optional layout tags (directory name or catalog field):
|
||||
|
||||
| Tag | Meaning |
|
||||
| --- | ------- |
|
||||
| runnable | Fits current run envelope (R tier) |
|
||||
| strategic | Capability reserve; may exceed current hardware (S tier) |
|
||||
`collection.storage_path` is the `s3://` URI.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -99,20 +78,24 @@ Optional layout tags (directory name or catalog field):
|
|||
|
||||
| Parameter | Policy |
|
||||
| --------- | ------ |
|
||||
| **Device** | VAULT HD (`D:`) — model tree under `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| **Soft quota (model reserve)** | **850 GiB** (self-imposed; disk is larger) |
|
||||
| **Hard stop** | **920 GiB** used under the freedom-intelligence base path |
|
||||
| **Growth review** | When catalog total ≥ **70% soft quota (~595 GiB)** |
|
||||
| **Gate** | **Euros per month**, not GiB |
|
||||
| **Soft budget** | **€15 / month** ex VAT for this bucket |
|
||||
| **Hard backstop** | Founder Scaleway project billing alert (no provider euro hard-stop) |
|
||||
| **Growth review** | When the running month is ≥ **70% of soft budget (~€10.50)** |
|
||||
| **Per-pull threshold** | See `inventory/collection-policy.md` |
|
||||
| **Eviction** | Prefer drop **W** (watch) and easily re-obtained quants; protect unique **S** SOTA bases and the **R** spine |
|
||||
| **Eviction** | Prefer drop **W** and easily re-obtained quants; protect unique **S** SOTA bases and the **R** spine |
|
||||
|
||||
### Portfolio guidance on 1 TB
|
||||
Rough cost (2026-09-13 Scaleway list prices):
|
||||
|
||||
* Always keep the **runnable spine (R / former P0)** resident.
|
||||
* Use remaining space for a **small number of most-capable open models (S)**,
|
||||
preferably well-provenance compressed weights when full precision would exhaust the disk.
|
||||
* Do **not** attempt to mirror entire HF orgs.
|
||||
* Full bf16 of multiple 600B-class models will **not** fit; prioritize top capability identities.
|
||||
| Mix | Class | ≈ € / month |
|
||||
| --- | ----- | ----------: |
|
||||
| R-spine ~45 GiB | One Zone IA | 0.54 |
|
||||
| V4-Flash ~167 GiB | Glacier | 0.42 |
|
||||
| Kimi K3 ~1454 GiB | Glacier | 3.69 |
|
||||
| Those three together | | **~4.65** |
|
||||
|
||||
Glacier restore is €0.009/GB — pay it when we actually need the weights
|
||||
on a GPU host.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -121,12 +104,14 @@ Optional layout tags (directory name or catalog field):
|
|||
For every completed collection:
|
||||
|
||||
1. Record source URL and revision in the catalog entry.
|
||||
2. Store checksums (`sha256` of each blob or upstream manifest digest).
|
||||
2. Store checksums (`sha256` of each blob or upstream manifest digest)
|
||||
in `MANIFEST.json` (local staging **and** `manifests/` prefix).
|
||||
3. Record download date (UTC) and downloader identity.
|
||||
4. Prefer official org releases over anonymous re-uploads.
|
||||
5. Keep license text or SPDX id in catalog; refuse unclear licenses.
|
||||
|
||||
Verification: **catalog claims must match on-disk checksums** before status `collected`.
|
||||
Verification: **catalog claims must match object checksums** before
|
||||
status `collected`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -134,11 +119,11 @@ Verification: **catalog claims must match on-disk checksums** before status `col
|
|||
|
||||
| Class | Retention |
|
||||
| ----- | --------- |
|
||||
| **Strategic capability (S)** | Keep until explicit deprecation or displacement by a clearly stronger open successor |
|
||||
| **Runnable spine (R)** | Keep while still the lab default for local ops / FT |
|
||||
| **Strategic capability (S)** | Keep until explicit deprecation; Glacier |
|
||||
| **Runnable spine (R)** | Keep while still the lab default; One Zone IA |
|
||||
| **Working set** | Active experiment bases + optional one superseded revision |
|
||||
| **Staging** | Max 14 days incomplete, then purge |
|
||||
| **Deprecated / evicted** | Metadata retained in catalog; blobs may be deleted |
|
||||
| **Deprecated / evicted** | Metadata retained in catalog; objects may be deleted |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -146,35 +131,37 @@ Verification: **catalog claims must match on-disk checksums** before status `col
|
|||
|
||||
| Topic | Policy |
|
||||
| ----- | ------ |
|
||||
| **At rest** | NAS default; extra age/GPG of multi-hundred-GB trees optional |
|
||||
| **In transit** | Trusted lab network / local attach |
|
||||
| **Offsite copy** | Optional later; coordinate with disaster-control if added so model bulk does not break ops backup SLAs |
|
||||
| **At rest** | Scaleway provider-managed |
|
||||
| **In transit** | TLS to `s3.nl-ams.scw.cloud` |
|
||||
| **Workstation copy** | Optional R-spine on VAULT; not a second SoT |
|
||||
| **Ops backups** | Do not mix with Forgejo/k3s dumps |
|
||||
|
||||
---
|
||||
|
||||
## What must not live here
|
||||
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or ungoverned training corpora with personal data
|
||||
* Operational backups (databases, k3s state, Forgejo dumps) as the primary home
|
||||
* Git LFS dumps of full model trees as a substitute for the NAS
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or ungoverned training corpora
|
||||
* Operational backups (databases, k3s state, Forgejo dumps)
|
||||
* Git LFS dumps of full model trees
|
||||
* Objects in `railiance-platform-pg-backup`
|
||||
|
||||
---
|
||||
|
||||
## Operator checklist
|
||||
|
||||
- [x] Choose facility: bulk durable storage for model reserve (2026-07-24 intent)
|
||||
- [x] Set soft quota: **850 GiB** / hard stop **920 GiB**
|
||||
- [x] Pin path: **`D:\vault\coulomb\freedom-intelligence\`** (2026-07-28)
|
||||
- [x] Create `models/`, `staging/`, `companions/`
|
||||
- [x] Document Windows + WSL path forms above
|
||||
- [x] Ensure WSL can see `D:` (`/mnt/d`) when downloads run from Linux (verified 2026-07-28)
|
||||
- [x] Choose facility: Scaleway object storage (2026-09-13)
|
||||
- [x] Set capacity gate: **€15 / month** soft
|
||||
- [x] Pin prefix layout above
|
||||
- [ ] Create bucket + scoped key (FI-WP-0004-T08)
|
||||
- [ ] Smoke PUT/GET
|
||||
- [ ] First S collect: DeepSeek-V4-Flash-0731 (FI-WP-0004-T09)
|
||||
|
||||
### Pin log
|
||||
|
||||
| Date | Operator | Change |
|
||||
| ---- | -------- | ------ |
|
||||
| 2026-07-24 | foundation | Policy created; path unpinned |
|
||||
| 2026-07-24 | operator direction | Facility intent = 1 TB local NAS; soft quota 850 GiB; strategic unrunnable models in scope |
|
||||
| 2026-07-28 | operator direction | **Interim pin:** workstation VAULT HD `D:\vault\coulomb\freedom-intelligence\`; layout created; soft quota retained |
|
||||
| 2026-07-28 | operator | WSL mount verified: `D:` → `/mnt/d` (1.9T, writable) |
|
||||
| 2026-07-24 | operator direction | Facility intent = 1 TB local NAS; soft quota 850 GiB |
|
||||
| 2026-07-28 | operator direction | **Interim pin:** workstation VAULT HD |
|
||||
| 2026-09-13 | FI-WP-0004 | **SoT pin:** Scaleway `nl-ams`; VAULT demoted to staging; euro gate replaces GiB quota |
|
||||
|
|
|
|||
|
|
@ -22,7 +22,8 @@ full resurveys.
|
|||
| One brief per date | Path `briefs/YYYY/MM/YYYY-MM-DD.md` |
|
||||
| Calendar | Prefer **Europe/Berlin** date in front matter |
|
||||
| Skip empty days | Either omit file or write `*No material delta.*` in Headline deltas |
|
||||
| Automation due bit | Cleared only by State Hub event `fi_daily_brief` (below) |
|
||||
| Automation due bit | Cleared only by State Hub event `fi_daily_brief` **after** the brief exists on `origin/main` |
|
||||
| Hub without git | A `fi_daily_brief` event with no matching file is a **false completion**. Run `scripts/verify_brief_durability.py`. |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -46,17 +47,20 @@ full resurveys.
|
|||
|
||||
4. **Collection candidates**
|
||||
- Use inventory field names; set priority
|
||||
- Do **not** re-nominate a model already in `inventory/catalog/`
|
||||
- Do **not** invent parameter counts; cite the model card or mark unverified
|
||||
- If nominating: add or update `inventory/catalog/{id}.yaml` with
|
||||
`status: candidate` and `collection.brief_refs`
|
||||
|
||||
5. **Lab implications**
|
||||
- Link open workplans (`FI-WP-*`) only when action is real
|
||||
|
||||
6. **Commit** (operator policy) — brief + any catalog YAML
|
||||
6. **Commit and push** — brief + any catalog YAML must land on `origin/main`.
|
||||
A local-only commit does **not** complete the day.
|
||||
|
||||
7. **Completion evidence** (required for activity-core rhythm)
|
||||
|
||||
Post State Hub progress (HTTP or MCP), shape:
|
||||
Post State Hub progress **only after** origin has the commit:
|
||||
|
||||
```yaml
|
||||
event_type: fi_daily_brief
|
||||
|
|
@ -67,8 +71,13 @@ full resurveys.
|
|||
date: "YYYY-MM-DD"
|
||||
path: "briefs/YYYY/MM/YYYY-MM-DD.md"
|
||||
collection_candidates: 0
|
||||
pushed: true
|
||||
origin_sha: "<full sha>"
|
||||
```
|
||||
|
||||
If the brief is written but push fails, post `event_type: executor_run`
|
||||
with `ok: false` so `due` stays true.
|
||||
|
||||
Workstation API often `http://127.0.0.1:8000`; remote tunnel ports per
|
||||
Custodian docs. See `docs/state-hub-delivery.md`.
|
||||
|
||||
|
|
@ -85,8 +94,10 @@ When activity-core emits `activity_task_spawn` with
|
|||
|
||||
1. Railiance host timer (or manual) runs
|
||||
`rein-aharness fi-research-brief --target-repo ~/freedom-intelligence`
|
||||
2. llm-connect drafts the brief; harness commits and posts **`fi_daily_brief`**
|
||||
3. Idempotency: one brief file per date; resolver clears `due` on completion event
|
||||
2. llm-connect drafts the brief; harness **commits, pushes to origin**, then
|
||||
posts **`fi_daily_brief`** with `pushed=true`
|
||||
3. Idempotency: one brief file per date on origin; resolver clears `due` only
|
||||
then. Local commit without push must not clear `due`.
|
||||
|
||||
Do **not** install Claude Code / Codex on railiance01 for this rhythm.
|
||||
|
||||
|
|
@ -116,9 +127,10 @@ make -C ~/activity-core sync-activity-definitions
|
|||
| Check | Pass |
|
||||
| ----- | ---- |
|
||||
| Signal | No filler paragraphs |
|
||||
| Sources | At least one primary link per non-empty axis row |
|
||||
| Honesty | Benchmarks marked directional; prices “verify before budget” |
|
||||
| Collection | License noted for any candidate |
|
||||
| Sources | At least one primary URL per non-empty axis row |
|
||||
| Honesty | Benchmarks marked directional; prices “verify before budget”; sizes from the card |
|
||||
| Collection | License + real size/class; never re-announce a cataloged id as a new release |
|
||||
| Axes | C and D must be scanned; `*(none)*` is allowed only after a look |
|
||||
| Length | Prefer one screen to three; deep dives go under `research/` |
|
||||
|
||||
---
|
||||
|
|
|
|||
136
docs/decisions/2026-09-13-scaleway-object-reserve.md
Normal file
136
docs/decisions/2026-09-13-scaleway-object-reserve.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
# Decision: Scaleway object storage as the open-weight reserve
|
||||
|
||||
**Date:** 2026-09-13
|
||||
**Status:** accepted
|
||||
**Affects:** `docs/backup-storage-policy.md`, `inventory/collection-policy.md`,
|
||||
`docs/hardware-envelope.md`, `scripts/collect_model.py`, FI-WP-0004
|
||||
**Supersedes (storage backend only):**
|
||||
`docs/decisions/2026-07-24-nas-strategic-reserve.md` § storage facility.
|
||||
The **capability-first** rule from that decision stands.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Source of truth for weight blobs** is a dedicated Scaleway Object
|
||||
Storage bucket in `nl-ams`, operated as `reef-storage`, consumed by
|
||||
this repo. Git still holds catalog metadata only.
|
||||
|
||||
2. **VAULT HD** (`D:\vault\coulomb\freedom-intelligence\`) is **staging
|
||||
and optional local R-spine cache**. It is not the strategic envelope.
|
||||
|
||||
3. **Storage class follows tier:**
|
||||
- **R (runnable spine):** One Zone IA — cheap, GET is immediate.
|
||||
- **S (strategic):** Glacier — cheapest just-in-case; restore cost
|
||||
and latency are acceptable.
|
||||
- **W:** only with euro-budget headroom; One Zone IA or Glacier by size.
|
||||
|
||||
4. **Capacity gate is euros/month, not a 850 GiB disk quota.** The 850 /
|
||||
920 GiB numbers were a 1 TB NAS artifact. Usage-based object storage
|
||||
removes that wall. Soft infrastructure budget: **€15 / month**
|
||||
(ex VAT) for this bucket. Founder Scaleway billing alert remains the
|
||||
hard backstop (`RESOURCE-WP-0002`).
|
||||
|
||||
5. **Do not reuse** `railiance-platform-pg-backup`. That bucket expires
|
||||
objects at 30 days. Model weights must not.
|
||||
|
||||
---
|
||||
|
||||
## Why Scaleway (economics)
|
||||
|
||||
Org already selected Scaleway over Hetzner / Host Europe for S3
|
||||
(`resource-control` RESOURCE-WP-0002, 2026-08-14). Same project, same
|
||||
failure domain independence from Host Europe `railiance01`.
|
||||
|
||||
Prices from [Scaleway storage pricing](https://www.scaleway.com/en/pricing/storage/)
|
||||
as of 2026-09-13, ex tax:
|
||||
|
||||
| Class | € / GB-month | 45 GiB (R spine) | 167 GiB (V4-Flash) | 1454 GiB (Kimi K3) |
|
||||
| ----- | -----------: | ---------------: | -----------------: | -----------------: |
|
||||
| Multi-AZ Standard | ~0.0146 | €0.66 | €2.44 | €21.23 |
|
||||
| One Zone IA | ~0.012 | €0.54 | €2.00 | €17.45 |
|
||||
| Glacier | ~0.00254 | €0.11 | €0.42 | **€3.69** |
|
||||
|
||||
Glacier restore to Standard is €0.009/GB (V4-Flash restore ≈ €1.50;
|
||||
K3 restore ≈ €13). Ingress is included. Egress is 75 GB/month free,
|
||||
then €0.01/GB — relevant when we actually pull a giant to a GPU host,
|
||||
not when we park it.
|
||||
|
||||
**Portfolio that the 850 GiB NAS could not hold:**
|
||||
|
||||
| Item | Class | Est. monthly |
|
||||
| ---- | ----- | -----------: |
|
||||
| R-spine resident (One Zone IA) | R | €0.54 |
|
||||
| DeepSeek-V4-Flash-0731 (Glacier) | S | €0.42 |
|
||||
| Kimi K3 official tree (Glacier) | S | €3.69 |
|
||||
| **Total** | | **~€4.65** |
|
||||
|
||||
That is inside the €15 soft budget with room for a second compressed
|
||||
giant or a Qwen3.8-27B R upgrade. Buying another multi-TB disk to
|
||||
fit K3 would spend the same money in a year and stay on the
|
||||
workstation failure domain.
|
||||
|
||||
VAULT remains useful: sequential staging for `huggingface_hub`
|
||||
downloads, and a local copy of the R-spine if the workstation is
|
||||
doing inference. It is no longer the thing INTENT means by “reserve.”
|
||||
|
||||
---
|
||||
|
||||
## Bucket contract
|
||||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| Provider | Scaleway Object Storage |
|
||||
| Region | `nl-ams` |
|
||||
| Endpoint | `https://s3.nl-ams.scw.cloud` |
|
||||
| Bucket (planned) | `railiance-fi-open-weight-reserve` |
|
||||
| Reef attributes | `reef-storage/substrate/object-stores/fi-open-weight-reserve.yaml` |
|
||||
| Public access | disabled |
|
||||
| Versioning | on |
|
||||
| Object expiry | **none** |
|
||||
| Lifecycle | `strategic/` → Glacier after 1 day |
|
||||
| Encryption | provider-managed at rest |
|
||||
| Identity | bucket-scoped key, not the postgres backup key |
|
||||
| Secrets | `railiance-platform` / OpenBao — never this git repo |
|
||||
|
||||
### Prefix layout
|
||||
|
||||
```text
|
||||
s3://railiance-fi-open-weight-reserve/
|
||||
├── models/{org}__{name}/{revision}/ # R / W (One Zone IA at PUT)
|
||||
├── strategic/{org}__{name}/{revision}/ # S (Glacier via lifecycle)
|
||||
├── companions/
|
||||
├── staging/ # incomplete uploads; purge after 14d
|
||||
└── manifests/{org}__{name}/{revision}/MANIFEST.json
|
||||
```
|
||||
|
||||
Each prefix maps to one catalog YAML. `collection.storage_path` is the
|
||||
`s3://` URI, not a VAULT path.
|
||||
|
||||
---
|
||||
|
||||
## What stays from 2026-07-24
|
||||
|
||||
- Run envelope ≠ reserve envelope.
|
||||
- License and integrity first.
|
||||
- Catalog in git, blobs off git.
|
||||
- No public mirror, no closed weights.
|
||||
- R-spine always kept; S is a small number of top open identities.
|
||||
|
||||
## What changes
|
||||
|
||||
| Before | After |
|
||||
| ------ | ----- |
|
||||
| Facility = workstation VAULT HD | Facility = Scaleway `nl-ams` |
|
||||
| Soft quota 850 GiB | Soft budget €15/month |
|
||||
| K3 deferred on capacity | K3 economically in-scope on Glacier |
|
||||
| V4-Flash uncollectable-as-12B | V4-Flash (~167 GiB MIT MoE) is the first S pull |
|
||||
|
||||
---
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- [x] FI-WP-0004-T01 this file
|
||||
- [ ] FI-WP-0004-T08 operator creates the bucket
|
||||
- [ ] FI-WP-0004-T09 first collect: DeepSeek-V4-Flash-0731
|
||||
- [ ] resource-control inventory record (optional; FI consumes, resource-control costs)
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Hardware envelope — run vs reserve
|
||||
|
||||
**Status:** updated 2026-07-24 (1 TB NAS strategic reserve)
|
||||
**Status:** updated 2026-09-13 (Scaleway object reserve; run envelope unchanged)
|
||||
**Used by:** inventory `hardware_class`, axis B/C, collection tiers R/S
|
||||
|
||||
---
|
||||
|
|
@ -10,10 +10,10 @@
|
|||
Separate two questions that used to be conflated:
|
||||
|
||||
1. **What can we run or fine-tune now?** → *run envelope*
|
||||
2. **What should we store on the NAS just in case?** → *reserve envelope* (1 TB)
|
||||
2. **What should we store just in case?** → *reserve envelope* (Scaleway Glacier / One Zone IA)
|
||||
|
||||
Collecting a model **does not** require it to fit the run envelope.
|
||||
See `docs/decisions/2026-07-24-nas-strategic-reserve.md`.
|
||||
See `docs/decisions/2026-09-13-scaleway-object-reserve.md`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -34,13 +34,15 @@ valid for NAS collection.
|
|||
|
||||
---
|
||||
|
||||
## Reserve envelope (NAS)
|
||||
## Reserve envelope (object store)
|
||||
|
||||
| Parameter | Value |
|
||||
| --------- | ----- |
|
||||
| Facility | VAULT HD `D:\vault\coulomb\freedom-intelligence\` (soft quota 850 GiB) |
|
||||
| Soft quota | 850 GiB |
|
||||
| Hard stop | 920 GiB |
|
||||
| Facility | Scaleway Object Storage `nl-ams` (`railiance-fi-open-weight-reserve`) |
|
||||
| Soft budget | **€15 / month** |
|
||||
| R class | One Zone IA |
|
||||
| S class | Glacier |
|
||||
| Staging | VAULT HD `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| Policy | `docs/backup-storage-policy.md` |
|
||||
|
||||
---
|
||||
|
|
@ -51,7 +53,8 @@ valid for NAS collection.
|
|||
| ---- | ---- | --- | ---- | --- | ----- |
|
||||
| TBD | primary local inference | TBD | TBD | TBD | Run envelope |
|
||||
| TBD | training experiments | TBD | TBD | TBD | Run envelope |
|
||||
| **VAULT HD (D:)** | weight store | n/a | n/a | n/a | **Reserve envelope** — soft 850 GiB |
|
||||
| **VAULT HD (D:)** | staging / R cache | n/a | n/a | n/a | **Not SoT** — downloads and optional local R |
|
||||
| **Scaleway nl-ams** | weight SoT | n/a | n/a | n/a | **Reserve envelope** — €15/mo soft |
|
||||
| railiance01 | cluster | n/a | n/a | n/a | Do not store weight trees on hot disks |
|
||||
|
||||
---
|
||||
|
|
@ -76,7 +79,7 @@ Strategic giants may remain **cold storage only** until hardware upgrades.
|
|||
| QLoRA 7–8B | T1–T2 |
|
||||
| QLoRA 14B | T2 |
|
||||
| Full FT 7B | T2–T3 |
|
||||
| 70B+ / large MoE FT | Future facility — weights may already sit on NAS |
|
||||
| 70B+ / large MoE FT | Future facility — weights may already sit on Glacier |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -86,3 +89,4 @@ Strategic giants may remain **cold storage only** until hardware upgrades.
|
|||
| ---- | ------ |
|
||||
| 2026-07-24 | Draft tiers |
|
||||
| 2026-07-24 | Split run vs reserve; NAS 1 TB; S tier may exceed run envelope |
|
||||
| 2026-09-13 | Reserve envelope moved to Scaleway; VAULT is staging; euro gate |
|
||||
|
|
|
|||
|
|
@ -19,7 +19,8 @@ activity-core (when / what / where)
|
|||
rein-aharness on railiance01 (does the work)
|
||||
primary: claim-loop → ops_run → fi-research-brief (REIN-A-0002)
|
||||
break-glass only: host timer 07:35 (disabled 2026-08-05; re-enable if claim loop down)
|
||||
llm-connect drafts brief → commit → push → fi_daily_brief
|
||||
llm-connect drafts brief → commit → git push origin → fi_daily_brief
|
||||
(fi_daily_brief is forbidden until origin has the file)
|
||||
|
||||
Operator morning review (no AI) from this repo:
|
||||
```bash
|
||||
|
|
@ -118,7 +119,19 @@ detail:
|
|||
collection_candidates: <int>
|
||||
```
|
||||
|
||||
Posted automatically by `rein-aharness fi-research-brief` on success.
|
||||
Posted automatically by `rein-aharness fi-research-brief` **only after**
|
||||
`git push origin` succeeds. `detail.pushed` and `detail.origin_sha` are
|
||||
required. A local commit with a failed push is `executor_run` / `ok=false`.
|
||||
|
||||
FI is the definition owner: origin publication is a **granted** capability
|
||||
of this rhythm (FI-WP-0004-T04), not an ungoverned env widening of
|
||||
`FI_RESEARCH_BRIEF_PUSH`.
|
||||
|
||||
Verify drift:
|
||||
|
||||
```bash
|
||||
python3 scripts/verify_brief_durability.py --require-hub
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -127,4 +140,6 @@ Posted automatically by `rein-aharness fi-research-brief` on success.
|
|||
- Workstation `crontab` that writes the brief (bypasses activity-core)
|
||||
- Installing Claude Code / Codex on railiance01 for this rhythm
|
||||
- Treating host timers as the scheduler of record
|
||||
- Skipping the completion event (leaves `due=true` forever)
|
||||
- Skipping the completion event (leaves `due=true` forever)
|
||||
- Posting `fi_daily_brief` for a local-only commit (clears `due`, loses the day)
|
||||
- Reusing `railiance-platform-pg-backup` for weights (30-day expiry)
|
||||
|
|
|
|||
|
|
@ -19,7 +19,8 @@ Do not create workplans via hub `create_workplan` APIs.
|
|||
|
||||
## Required: completion evidence for daily rhythm
|
||||
|
||||
Every completed brief day (including “no material delta”) should post:
|
||||
Every completed brief day (including “no material delta”) should post
|
||||
**after the file is on `origin/main`**:
|
||||
|
||||
```yaml
|
||||
event_type: fi_daily_brief
|
||||
|
|
@ -30,11 +31,15 @@ detail:
|
|||
date: "YYYY-MM-DD"
|
||||
path: "briefs/YYYY/MM/YYYY-MM-DD.md"
|
||||
collection_candidates: 0
|
||||
pushed: true
|
||||
origin_sha: "<full sha>"
|
||||
```
|
||||
|
||||
This is the **idempotence signal** for activity-core context query
|
||||
`fi_brief_status` (FI-WP-0002-T03): `due=false` when an event exists for today
|
||||
(Europe/Berlin) with `detail.repo=freedom-intelligence`.
|
||||
(Europe/Berlin) with `detail.repo=freedom-intelligence` **and**
|
||||
`detail.pushed=true`. A hub event without a git file is a false completion
|
||||
(`scripts/verify_brief_durability.py`).
|
||||
|
||||
Inbound automation uses `activity_task_spawn` from activity-core (state-hub sink).
|
||||
See `activity-core/docs/task-emission-consumer-contract.md`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue