Make the sensing loop durable and pin the reserve to Scaleway
The weekday brief only counts when the file is on origin/main. Hub events without git objects are failures. The open-weight SoT moves from the VAULT 850 GiB quota to a dedicated Scaleway bucket (One Zone IA for R, Glacier for S) so V4-Flash and K3 are economically in-scope. Catalogs the real DeepSeek-V4-Flash-0731 (MIT ~167 GiB MoE, not 12B) and nominates Qwen3.8-27B. Bucket create remains FI-WP-0004-T08. Assistant: grok Assistant-Session: 01a09c6a-1cfc-75b1-a78d-c13eaf22241d
This commit is contained in:
parent
8832652ad3
commit
a78187c33e
18 changed files with 1125 additions and 226 deletions
31
README.md
31
README.md
|
|
@ -23,7 +23,7 @@ Standing lenses: **A** frontier & price · **B** edge/local open · **C** homela
|
|||
| [`briefs/`](briefs/) | Daily briefs + template |
|
||||
| [`inventory/`](inventory/) | Model catalog & collection policy |
|
||||
| [`workplans/`](workplans/) | Setup workplans (incl. activity-core) |
|
||||
| [`activity-definitions/`](activity-definitions/) | Cron definition for daily brief (disabled until FI-WP-0002) |
|
||||
| [`activity-definitions/`](activity-definitions/) | Weekday brief schedule (enabled; completion = origin/main) |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -31,23 +31,26 @@ Standing lenses: **A** frontier & price · **B** edge/local open · **C** homela
|
|||
|
||||
| ID | Title | Status |
|
||||
| -- | ----- | ------ |
|
||||
| [FI-WP-0001](workplans/FI-WP-0001-lab-operating-foundation.md) | Lab operating foundation | active (storage pin open) |
|
||||
| [FI-WP-0002](workplans/FI-WP-0002-activity-core-daily-research.md) | Activity-core daily research rhythm | active (resolver/ops open) |
|
||||
| [FI-WP-0003](workplans/FI-WP-0003-seed-open-weight-reserve.md) | Seed open-weight reserve | active (catalog seeded) |
|
||||
| [FI-WP-0001](workplans/FI-WP-0001-lab-operating-foundation.md) | Lab operating foundation | done |
|
||||
| [FI-WP-0002](workplans/FI-WP-0002-activity-core-daily-research.md) | Activity-core daily research rhythm | finished |
|
||||
| [FI-WP-0003](workplans/FI-WP-0003-seed-open-weight-reserve.md) | Seed open-weight reserve | finished |
|
||||
| [FI-WP-0004](workplans/FI-WP-0004-operational-loop-and-scaleway-reserve.md) | Durable loop + Scaleway reserve | **active** |
|
||||
|
||||
---
|
||||
|
||||
## Collection plan (bulk reserve)
|
||||
|
||||
**Storage:** `D:\vault\coulomb\freedom-intelligence\` (VAULT HD) · soft quota **850 GiB**
|
||||
**SoT:** Scaleway Object Storage `nl-ams` · bucket `railiance-fi-open-weight-reserve`
|
||||
**Gate:** **€15 / month** · R = One Zone IA · S = Glacier
|
||||
**Staging:** VAULT HD `D:\vault\coulomb\freedom-intelligence\`
|
||||
**Policy:** store the **most capable open weights** even if unrunnable today
|
||||
(`docs/decisions/2026-07-24-nas-strategic-reserve.md`).
|
||||
(`docs/decisions/2026-09-13-scaleway-object-reserve.md`).
|
||||
|
||||
| Tier | Examples |
|
||||
| ---- | -------- |
|
||||
| **R — runnable spine** | Qwen3-8B, Llama-3.2-3B-Instruct, BGE-M3, R1-Distill-14B, nomic-embed |
|
||||
| **S — strategic** | DeepSeek-V3 (class), DeepSeek-R1 full, Llama-3.3-70B / Qwen3-72B dense |
|
||||
| **W — optional** | 14B/32B upgrades, reranker, … |
|
||||
| **S — strategic** | **DeepSeek-V4-Flash-0731** (first pull), Kimi K3, V3/R1 full, 70B-class |
|
||||
| **W — optional** | Qwen3.8-27B, 14B/32B upgrades, reranker |
|
||||
|
||||
Plan detail: `research/2026-07-24-nas-strategic-collection-plan.md` · catalog: `inventory/catalog/`.
|
||||
|
||||
|
|
@ -58,7 +61,7 @@ Plan detail: `research/2026-07-24-nas-strategic-collection-plan.md` · catalog:
|
|||
```text
|
||||
freedom-intelligence/
|
||||
├── INTENT.md / SCOPE.md / README.md
|
||||
├── activity-definitions/ # fi-daily-research-brief (enabled: false)
|
||||
├── activity-definitions/ # fi-daily-research-brief (enabled; origin publish required)
|
||||
├── briefs/ # daily deltas
|
||||
├── research/ # deep surveys (baseline lives here)
|
||||
├── inventory/catalog/ # YAML candidates / collected
|
||||
|
|
@ -79,8 +82,14 @@ cp briefs/_template.md briefs/$(date +%Y/%m)/$(date +%Y-%m-%d).md
|
|||
# follow docs/daily-brief-playbook.md
|
||||
```
|
||||
|
||||
**Do not** bulk-download models until `docs/backup-storage-policy.md` path/quota
|
||||
are pinned (FI-WP-0001-T04).
|
||||
**Do not** bulk-download S-tier models until the Scaleway bucket is live
|
||||
(FI-WP-0004-T08). Stage on VAULT; never into this git tree.
|
||||
|
||||
**Durability check** (hub events vs git files):
|
||||
|
||||
```bash
|
||||
python3 scripts/verify_brief_durability.py --require-hub
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
34
SCOPE.md
34
SCOPE.md
|
|
@ -91,17 +91,17 @@ Git holds briefs, schemas, policies, and catalog metadata. Model weight blobs st
|
|||
|
||||
| Area | Status |
|
||||
| ---- | ------ |
|
||||
| Overall | **Operable lab** — foundation + **live** weekday brief schedule + R-spine reserve |
|
||||
| INTENT / SCOPE | Drafted |
|
||||
| Overall | **Repairing the operating loop** — foundation exists; origin publication + Scaleway SoT in FI-WP-0004 |
|
||||
| INTENT / SCOPE | Live; SCOPE current-state refreshed 2026-09-13 |
|
||||
| Baseline survey | `research/2026-07-24-baseline-field-survey.md` |
|
||||
| Briefs | 2026-07-24, 07-28, **08-03** (week catch-up); playbook + recurrence ops |
|
||||
| Backup storage | **VAULT HD** `D:\vault\coulomb\freedom-intelligence\`; soft quota 850 GiB |
|
||||
| Inventory | R verified; **Kimi K3** S1 approved+deferred (~1454 GiB); other S deferred; W candidate |
|
||||
| Reserve policy | Capability-first — not gated by current VRAM |
|
||||
| Activity-core | Definition **enabled on railiance**; Temporal schedule weekdays 07:30 Berlin |
|
||||
| Executor | rein-aharness `fi-research-brief` via railiance user timer 07:35 (not workstation cron) |
|
||||
| State Hub | Repo under **agents**; workplans finished + hub-indexed |
|
||||
| Workplans | FI-WP-0001…0003 **finished** |
|
||||
| Briefs | Origin last 2026-08-14; catch-up **2026-09-13**; weekday schedule still enabled |
|
||||
| Backup storage | **Scaleway `nl-ams`** (bucket planned); VAULT HD is staging; €15/mo gate |
|
||||
| Inventory | R verified on VAULT cache; **V4-Flash** S1 approved (not collected); K3 Glacier-affordable; Qwen3.8-27B candidate |
|
||||
| Reserve policy | Capability-first — not gated by current VRAM; **not** gated by 850 GiB |
|
||||
| Activity-core | Definition **enabled**; completion = brief on `origin/main` |
|
||||
| Executor | rein-aharness `fi-research-brief` **must push** or the day fails |
|
||||
| State Hub | Repo under **agents**; `fi_daily_brief` is not evidence without the git file |
|
||||
| Workplans | FI-WP-0001…0003 finished; **FI-WP-0004 active** |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -121,12 +121,14 @@ Git holds briefs, schemas, policies, and catalog metadata. Model weight blobs st
|
|||
1. ✅ `INTENT.md` / `SCOPE.md` / foundation docs
|
||||
2. ✅ Brief template + playbook + sources allowlist
|
||||
3. ✅ Baseline field survey + collection recommendations
|
||||
4. ✅ Catalog candidates (P0/P1)
|
||||
5. ✅ ActivityDefinition draft (disabled)
|
||||
6. ⬜ Pin backup storage path and capacity budget (FI-WP-0001-T04)
|
||||
7. ⬜ activity-core resolver + enable schedule (FI-WP-0002)
|
||||
8. ⬜ Approve/download P0 reserve (FI-WP-0003)
|
||||
9. ⬜ First daily delta brief (manual or after automation)
|
||||
4. ✅ Catalog candidates (P0/P1) + V4-Flash correction
|
||||
5. ✅ ActivityDefinition enabled
|
||||
6. ✅ Pin backup storage (VAULT staging 2026-07-28; **Scaleway SoT 2026-09-13**)
|
||||
7. ✅ activity-core resolver + weekday schedule
|
||||
8. ✅ R-spine verified on VAULT cache (FI-WP-0003)
|
||||
9. ⬜ Origin publication proven end-to-end on railiance (FI-WP-0004-T04 deploy)
|
||||
10. ⬜ Scaleway bucket live (FI-WP-0004-T08)
|
||||
11. ⬜ First S collect: V4-Flash (FI-WP-0004-T09)
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -18,9 +18,10 @@ context_sources:
|
|||
bind_to: context.fi_brief
|
||||
# Resolver (FI-WP-0002-T03, activity-core): expected shape
|
||||
# {items: [{kind: "daily_brief", due: bool, date: "YYYY-MM-DD",
|
||||
# last_run_at}]} — due is true when no fi_daily_brief progress event with
|
||||
# detail.repo=freedom-intelligence exists for today (Europe/Berlin).
|
||||
# Executing session must record that event on completion (idempotence).
|
||||
# last_run_at}]} — due is true when no *successful origin-published*
|
||||
# fi_daily_brief exists for today (Europe/Berlin).
|
||||
# Completion event is valid only with detail.pushed=true and the file on
|
||||
# origin/main. Local-only commits must not clear due (FI-WP-0004).
|
||||
---
|
||||
|
||||
# Freedom Intelligence Daily Research Brief
|
||||
|
|
@ -46,8 +47,10 @@ action:
|
|||
description: >
|
||||
Produce briefs/YYYY/MM/YYYY-MM-DD.md per docs/daily-brief-playbook.md and
|
||||
briefs/_template.md. Cite primary sources. Flag collection candidates.
|
||||
On completion post State Hub progress event_type=fi_daily_brief with
|
||||
detail.repo=freedom-intelligence and detail.date.
|
||||
Commit AND push to origin/main. Post fi_daily_brief only if
|
||||
detail.pushed=true and detail.origin_sha is set. If push fails, post
|
||||
executor_run ok=false so due stays true. Do not re-announce cataloged
|
||||
models. Do not invent parameter counts.
|
||||
target_repo: freedom-intelligence
|
||||
priority: medium
|
||||
labels: ["freedom-intelligence", "research-brief", "automated"]
|
||||
|
|
|
|||
81
briefs/2026/09/2026-09-13.md
Normal file
81
briefs/2026/09/2026-09-13.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
---
|
||||
date: 2026-09-13
|
||||
timezone: Europe/Berlin
|
||||
author: grok
|
||||
status: final
|
||||
sources_checked:
|
||||
- docs/sources-allowlist.md
|
||||
- inventory/RESERVE-STATUS.md
|
||||
- huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
|
||||
- huggingface.co/Qwen/Qwen3.8-27B (card identity via public trackers)
|
||||
- https://www.scaleway.com/en/pricing/storage/
|
||||
---
|
||||
|
||||
# Freedom Intelligence Daily Brief — 2026-09-13
|
||||
|
||||
> **Month catch-up.** Last file on `origin/main` is 2026-08-14. Hub
|
||||
> `fi_daily_brief` events exist for 2026-09-07…11 with `committed=True`
|
||||
> and no git objects. Those days are **not** recovered here. This file
|
||||
> is a field + ops delta from 2026-08-14, not a resurrection of the
|
||||
> missing executor commits.
|
||||
|
||||
## Headline deltas
|
||||
|
||||
- **Loop failure:** sensing ran locally on railiance and cleared `due` without publishing to origin. Rein-aharness dropped `git push` on 2026-09-04 as an integrity guard; FI now re-grants origin publication as a named completion requirement (FI-WP-0004).
|
||||
- **DeepSeek-V4-Flash-0731 is not 12B.** Official card: MIT, 284B base / ~13B active (304B with DSpark), ~167 GiB HF tree, 2026-07-31. Automated briefs 08-04…14 re-announced it as a new 12B dense model (license flipped MIT/Apache) and 08-07 claimed it was already on disk. Catalog now records the real model as **S, approved, not collected**.
|
||||
- **Qwen3.8-27B** (Apache-2.0 dense multimodal, 2026-08-14) and **Qwen3.8-2.4T-A95B** (custom Max licence, multi-TB, ~08-12) shipped in the hole. 27B is a W/R candidate; Max is S-watch only until euro budget is live.
|
||||
- **Reserve facility:** VAULT HD 850 GiB quota is retired as SoT. Scaleway `nl-ams` object storage is the reserve (One Zone IA for R, Glacier for S). Kimi K3 (~1454 GiB) becomes a few euros/month on Glacier instead of a quota wall.
|
||||
- **Next pull** after the bucket exists: V4-Flash official tree into `strategic/` Glacier — it always fitted even the old 850 GiB quota.
|
||||
|
||||
## Frontier & commercial (axis A)
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| DeepSeek-V4-Flash-0731 | Official open MIT MoE; API `deepseek-v4-flash`. Card scores Terminal Bench 2.1 82.7. | [HF card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731), [arXiv 2606.19348](https://arxiv.org/abs/2606.19348) | Closed-bar comparison via llm-connect; open identity is the S pull |
|
||||
| Qwen3.8-Max family | API GA early Aug; open 2.4T-A95B under **bespoke** licence (revenue threshold), not Apache | Qwen card / trackers (verify licence text at pin) | Do not collect the 2.4T tree until T08 budget + licence read |
|
||||
| GLM-5.3-Flash (late Aug) | 320B-A18B, MIT, 1M ctx, reported $0.15 / $0.50 per MTok | Z.ai release notes (verify) | Watch vs V4-Flash as MIT S alternative; not displacing V4-Flash as first pull |
|
||||
|
||||
## Edge / local / open (axis B)
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| DeepSeek-V4-Flash-0731 size class | ~167 GiB official tree, ~13B active — **S, not R** | HF file listing ~167 GB; card “304B params” | First Scaleway Glacier collect |
|
||||
| Qwen3.8-27B | Dense multimodal Apache-2.0, ~27B, 2026-08-14 — same calendar day as last origin brief | [HF Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) | Candidate R-spine upgrade; cataloged `candidate` |
|
||||
| Muse Glimmer 30B (Meta, ~08-10) | Apache-2.0 dense multimodal, ~24–32 GB class | Meta/HF trackers (verify card) | Watch; do not pull until V4-Flash and 27B decision |
|
||||
| Kimi K3 | Unchanged identity; capacity objection is now euros not GiB | existing catalog | Glacier ~€3.69/mo; still deferred until V4-Flash lands |
|
||||
|
||||
## Training & specialization (axis C)
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| Qwen3.8-27B as FT base | Dense 27B Apache-2.0 is a more serious QLoRA target than 8B if T2–T3 VRAM exists | Qwen3.8-27B card | Hardware envelope hosts still TBD — do not pretend we can FT it today |
|
||||
| No new homelab recipe displacement | Unsloth / TRL / Axolotl still the working set from baseline | allowlist | No stack change |
|
||||
|
||||
## Harness & fleet (axis D)
|
||||
|
||||
| Item | Delta | Sources | Lab relevance |
|
||||
| ---- | ----- | ------- | ------------- |
|
||||
| FI executor publication | 2026-09-04 rein-aharness removed `git push`; hub still cleared `due` | `rein-aharness` REINAH-WP-0003, hub events 2026-09-07…11 | **Broken durable memory.** FI-WP-0004 re-grants origin push and verifier |
|
||||
| V4-Flash agentic stack | Official vLLM DSpark + SGLang recipes on the card | HF card “How to Run” | Axis D watch; serve is not this repo |
|
||||
|
||||
## Collection candidates
|
||||
|
||||
| id | org | name | priority | reason | approx size | license |
|
||||
| -- | --- | ---- | -------- | ------ | ----------- | ------- |
|
||||
| deepseek-ai__DeepSeek-V4-Flash-0731__strategic | deepseek-ai | DeepSeek-V4-Flash-0731 | high | Corrected S1; MIT; ~167 GiB; first Scaleway pull | ~167 GiB | MIT |
|
||||
| Qwen__Qwen3.8-27B__candidate | Qwen | Qwen3.8-27B | medium | Apache-2.0 dense 27B R-upgrade | ~56 GiB class | Apache-2.0 |
|
||||
|
||||
Already in catalog, **not** new: Kimi K3, DeepSeek-V3, Qwen3-8B, R1-Distill-14B, BGE-M3, nomic-embed.
|
||||
|
||||
## Lab implications
|
||||
|
||||
- Finish FI-WP-0004: origin publish in the executor, durability verifier in CI/ops, Scaleway bucket (T08), then collect V4-Flash (T09).
|
||||
- Do not treat hub `fi_daily_brief` as evidence the brief exists until `scripts/verify_brief_durability.py` is green.
|
||||
- Recovering 2026-09-07…11 files from the railiance clone is optional; they were produced by the same failing loop and are not trusted.
|
||||
- Hardware envelope hosts are still TBD — R-upgrade of Qwen3.8-27B waits on that as much as on disk.
|
||||
|
||||
## Notes / caveats
|
||||
|
||||
- Parameter counts and €/GB from primary cards/pricing pages today; re-measure at pull.
|
||||
- Qwen3.8-Max open tree licence is **not** Apache-2.0; do not copy the 27B licence onto it.
|
||||
- `DeepSeek-V4-Flash-0731-12B` is not a real repo id.
|
||||
|
|
@ -1,17 +1,19 @@
|
|||
# Backup storage policy — open-weight model reserve
|
||||
|
||||
**Status:** facility + path pinned (workstation VAULT HD)
|
||||
**Status:** facility pin **Scaleway Object Storage** (2026-09-13)
|
||||
**Related:** `INTENT.md`, `SCOPE.md`, `inventory/collection-policy.md`,
|
||||
`docs/decisions/2026-07-24-nas-strategic-reserve.md`
|
||||
**Adjacent:** `disaster-control` BackupPolicy (platform backups — different concern)
|
||||
`docs/decisions/2026-09-13-scaleway-object-reserve.md`
|
||||
**Supersedes storage pin:** 2026-07-28 VAULT HD (now staging only)
|
||||
**Adjacent:** `disaster-control` BackupPolicy and `rapp-postgres` WAL
|
||||
archive — **different buckets, different lifecycles**
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define where Freedom Intelligence stores **open-weight model blobs** (and justified
|
||||
companions), how much capacity we allow, and how this reserve relates to other
|
||||
lab backup facilities.
|
||||
Define where Freedom Intelligence stores **open-weight model blobs** (and
|
||||
justified companions), how much we allow ourselves to spend, and how this
|
||||
reserve relates to other lab backup facilities.
|
||||
|
||||
Git never stores weight tensors. Git stores:
|
||||
|
||||
|
|
@ -25,16 +27,16 @@ Git never stores weight tensors. Git stores:
|
|||
|
||||
| Requirement | Policy |
|
||||
| ----------- | ------ |
|
||||
| **Class** | Backup / bulk durable storage — **not** hot cluster PVCs or app disks |
|
||||
| **Facility** | Workstation VAULT HD (`D:`) under `D:\vault\coulomb\` (interim pin 2026-07-28) |
|
||||
| **Durability** | Survives WSL rebuilds; weights live on the VAULT volume, not the git/WSL root |
|
||||
| **Performance** | Sequential read for restore/training pull is enough; low latency not required |
|
||||
| **Access** | Operator and approved lab hosts only; not a public mirror |
|
||||
| **Separation** | Do not co-mingle with age-encrypted operational backups (Forgejo dumps, k3s state) without a clear subdirectory and different lifecycle rules |
|
||||
| **Class** | Backup / bulk durable object storage — **not** hot cluster PVCs |
|
||||
| **Facility** | Scaleway Object Storage `nl-ams` via `reef-storage` |
|
||||
| **Durability** | Provider 11-nines claim; independent of Host Europe `railiance01` and of the workstation HD |
|
||||
| **Performance** | Sequential restore for training/inference pull is enough |
|
||||
| **Access** | Operator and approved lab hosts; not a public mirror |
|
||||
| **Separation** | **Never** the postgres backup bucket (`railiance-platform-pg-backup` expires at 30 days) |
|
||||
|
||||
Platform backup paths such as `/opt/backup/railiance/{infra,cluster}/` and
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. Model weights
|
||||
belong on the **VAULT model tree**, not on railiance hot disks.
|
||||
`~/.cache/railiance/backups/` are **operational recovery** lanes. Model
|
||||
weights belong in the FI object bucket.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -42,56 +44,33 @@ belong on the **VAULT model tree**, not on railiance hot disks.
|
|||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| **Host / facility** | Workstation-attached bulk HD **VAULT** (Windows `D:`) |
|
||||
| **Raw capacity** | ~1.86 TB total (~1.76 TB free as of 2026-07-28 pin) |
|
||||
| **Base path (Windows)** | `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| **Base path (WSL, when D: automounted)** | `/mnt/d/vault/coulomb/freedom-intelligence/` |
|
||||
| **Layout under base** | See [On-disk layout](#on-disk-layout) |
|
||||
| **Mount on lab hosts** | Local attach on operator workstation; WSL via drvfs `D:` → `/mnt/d` when enabled |
|
||||
| **Credentials** | Local filesystem ACL on the workstation HD; never commit secrets |
|
||||
| **Facility** | Scaleway Object Storage, region `nl-ams` |
|
||||
| **Endpoint** | `https://s3.nl-ams.scw.cloud` |
|
||||
| **Bucket** | `railiance-fi-open-weight-reserve` (planned until FI-WP-0004-T08) |
|
||||
| **Attributes** | `reef:storage/substrate/object-stores/fi-open-weight-reserve.yaml` |
|
||||
| **Credentials** | OpenBao / `railiance-platform` — never git |
|
||||
|
||||
**Interim note:** Earlier policy assumed a dedicated ~1 TB NAS. Operator chose this
|
||||
VAULT volume under `D:\vault\coulomb\` for now. Soft quota **850 GiB** for the
|
||||
model tree still applies (self-imposed discipline; disk is larger).
|
||||
**Local staging (not SoT):** workstation VAULT HD
|
||||
`D:\vault\coulomb\freedom-intelligence\` /
|
||||
`/mnt/d/vault/coulomb/freedom-intelligence/` for `huggingface_hub`
|
||||
downloads and an optional R-spine cache.
|
||||
|
||||
Do **not** bulk-download multi-GB models into this git workspace or into hot
|
||||
root filesystems (`/` on WSL, `C:`).
|
||||
Do **not** bulk-download multi-GB models into this git workspace or into
|
||||
hot root filesystems (`/` on WSL, `C:`, railiance PVCs).
|
||||
|
||||
### Path shape (pinned)
|
||||
### Prefix shape (pinned)
|
||||
|
||||
```text
|
||||
D:\vault\coulomb\freedom-intelligence\ # Windows
|
||||
/mnt/d/vault/coulomb/freedom-intelligence/ # WSL when D: mounted
|
||||
├── models/
|
||||
│ └── {org}__{name}/
|
||||
│ └── {revision}/
|
||||
│ ├── blobs/ # weight files, shards
|
||||
│ └── MANIFEST.txt # optional local copy of hashes
|
||||
├── companions/ # adapters, tokenizers only when separate
|
||||
└── staging/ # incomplete downloads; not catalog-ready
|
||||
s3://railiance-fi-open-weight-reserve/
|
||||
├── models/{org}__{name}/{revision}/ # R / W (One Zone IA)
|
||||
├── strategic/{org}__{name}/{revision}/ # S (Glacier)
|
||||
├── companions/
|
||||
├── staging/
|
||||
└── manifests/{org}__{name}/{revision}/MANIFEST.json
|
||||
```
|
||||
|
||||
Map each `{org}__{name}/{revision}` to an inventory catalog entry.
|
||||
|
||||
Directories `models/`, `staging/`, and `companions/` were created 2026-07-28.
|
||||
|
||||
---
|
||||
|
||||
## On-disk layout
|
||||
|
||||
| Path element | Rule |
|
||||
| ------------ | ---- |
|
||||
| `org__name` | Hugging Face-style id with `/` → `__` |
|
||||
| `revision` | Git commit SHA, tag, or release id used at download time |
|
||||
| `blobs/` | Actual files; prefer original names from source |
|
||||
| `staging/` | Incomplete transfers; purge or resume; never mark collected until complete + verified |
|
||||
|
||||
Optional layout tags (directory name or catalog field):
|
||||
|
||||
| Tag | Meaning |
|
||||
| --- | ------- |
|
||||
| runnable | Fits current run envelope (R tier) |
|
||||
| strategic | Capability reserve; may exceed current hardware (S tier) |
|
||||
`collection.storage_path` is the `s3://` URI.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -99,20 +78,24 @@ Optional layout tags (directory name or catalog field):
|
|||
|
||||
| Parameter | Policy |
|
||||
| --------- | ------ |
|
||||
| **Device** | VAULT HD (`D:`) — model tree under `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| **Soft quota (model reserve)** | **850 GiB** (self-imposed; disk is larger) |
|
||||
| **Hard stop** | **920 GiB** used under the freedom-intelligence base path |
|
||||
| **Growth review** | When catalog total ≥ **70% soft quota (~595 GiB)** |
|
||||
| **Gate** | **Euros per month**, not GiB |
|
||||
| **Soft budget** | **€15 / month** ex VAT for this bucket |
|
||||
| **Hard backstop** | Founder Scaleway project billing alert (no provider euro hard-stop) |
|
||||
| **Growth review** | When the running month is ≥ **70% of soft budget (~€10.50)** |
|
||||
| **Per-pull threshold** | See `inventory/collection-policy.md` |
|
||||
| **Eviction** | Prefer drop **W** (watch) and easily re-obtained quants; protect unique **S** SOTA bases and the **R** spine |
|
||||
| **Eviction** | Prefer drop **W** and easily re-obtained quants; protect unique **S** SOTA bases and the **R** spine |
|
||||
|
||||
### Portfolio guidance on 1 TB
|
||||
Rough cost (2026-09-13 Scaleway list prices):
|
||||
|
||||
* Always keep the **runnable spine (R / former P0)** resident.
|
||||
* Use remaining space for a **small number of most-capable open models (S)**,
|
||||
preferably well-provenance compressed weights when full precision would exhaust the disk.
|
||||
* Do **not** attempt to mirror entire HF orgs.
|
||||
* Full bf16 of multiple 600B-class models will **not** fit; prioritize top capability identities.
|
||||
| Mix | Class | ≈ € / month |
|
||||
| --- | ----- | ----------: |
|
||||
| R-spine ~45 GiB | One Zone IA | 0.54 |
|
||||
| V4-Flash ~167 GiB | Glacier | 0.42 |
|
||||
| Kimi K3 ~1454 GiB | Glacier | 3.69 |
|
||||
| Those three together | | **~4.65** |
|
||||
|
||||
Glacier restore is €0.009/GB — pay it when we actually need the weights
|
||||
on a GPU host.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -121,12 +104,14 @@ Optional layout tags (directory name or catalog field):
|
|||
For every completed collection:
|
||||
|
||||
1. Record source URL and revision in the catalog entry.
|
||||
2. Store checksums (`sha256` of each blob or upstream manifest digest).
|
||||
2. Store checksums (`sha256` of each blob or upstream manifest digest)
|
||||
in `MANIFEST.json` (local staging **and** `manifests/` prefix).
|
||||
3. Record download date (UTC) and downloader identity.
|
||||
4. Prefer official org releases over anonymous re-uploads.
|
||||
5. Keep license text or SPDX id in catalog; refuse unclear licenses.
|
||||
|
||||
Verification: **catalog claims must match on-disk checksums** before status `collected`.
|
||||
Verification: **catalog claims must match object checksums** before
|
||||
status `collected`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -134,11 +119,11 @@ Verification: **catalog claims must match on-disk checksums** before status `col
|
|||
|
||||
| Class | Retention |
|
||||
| ----- | --------- |
|
||||
| **Strategic capability (S)** | Keep until explicit deprecation or displacement by a clearly stronger open successor |
|
||||
| **Runnable spine (R)** | Keep while still the lab default for local ops / FT |
|
||||
| **Strategic capability (S)** | Keep until explicit deprecation; Glacier |
|
||||
| **Runnable spine (R)** | Keep while still the lab default; One Zone IA |
|
||||
| **Working set** | Active experiment bases + optional one superseded revision |
|
||||
| **Staging** | Max 14 days incomplete, then purge |
|
||||
| **Deprecated / evicted** | Metadata retained in catalog; blobs may be deleted |
|
||||
| **Deprecated / evicted** | Metadata retained in catalog; objects may be deleted |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -146,35 +131,37 @@ Verification: **catalog claims must match on-disk checksums** before status `col
|
|||
|
||||
| Topic | Policy |
|
||||
| ----- | ------ |
|
||||
| **At rest** | NAS default; extra age/GPG of multi-hundred-GB trees optional |
|
||||
| **In transit** | Trusted lab network / local attach |
|
||||
| **Offsite copy** | Optional later; coordinate with disaster-control if added so model bulk does not break ops backup SLAs |
|
||||
| **At rest** | Scaleway provider-managed |
|
||||
| **In transit** | TLS to `s3.nl-ams.scw.cloud` |
|
||||
| **Workstation copy** | Optional R-spine on VAULT; not a second SoT |
|
||||
| **Ops backups** | Do not mix with Forgejo/k3s dumps |
|
||||
|
||||
---
|
||||
|
||||
## What must not live here
|
||||
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or ungoverned training corpora with personal data
|
||||
* Operational backups (databases, k3s state, Forgejo dumps) as the primary home
|
||||
* Git LFS dumps of full model trees as a substitute for the NAS
|
||||
* Closed weights or artifacts whose terms forbid offline retention
|
||||
* Secrets, API keys, customer data, or ungoverned training corpora
|
||||
* Operational backups (databases, k3s state, Forgejo dumps)
|
||||
* Git LFS dumps of full model trees
|
||||
* Objects in `railiance-platform-pg-backup`
|
||||
|
||||
---
|
||||
|
||||
## Operator checklist
|
||||
|
||||
- [x] Choose facility: bulk durable storage for model reserve (2026-07-24 intent)
|
||||
- [x] Set soft quota: **850 GiB** / hard stop **920 GiB**
|
||||
- [x] Pin path: **`D:\vault\coulomb\freedom-intelligence\`** (2026-07-28)
|
||||
- [x] Create `models/`, `staging/`, `companions/`
|
||||
- [x] Document Windows + WSL path forms above
|
||||
- [x] Ensure WSL can see `D:` (`/mnt/d`) when downloads run from Linux (verified 2026-07-28)
|
||||
- [x] Choose facility: Scaleway object storage (2026-09-13)
|
||||
- [x] Set capacity gate: **€15 / month** soft
|
||||
- [x] Pin prefix layout above
|
||||
- [ ] Create bucket + scoped key (FI-WP-0004-T08)
|
||||
- [ ] Smoke PUT/GET
|
||||
- [ ] First S collect: DeepSeek-V4-Flash-0731 (FI-WP-0004-T09)
|
||||
|
||||
### Pin log
|
||||
|
||||
| Date | Operator | Change |
|
||||
| ---- | -------- | ------ |
|
||||
| 2026-07-24 | foundation | Policy created; path unpinned |
|
||||
| 2026-07-24 | operator direction | Facility intent = 1 TB local NAS; soft quota 850 GiB; strategic unrunnable models in scope |
|
||||
| 2026-07-28 | operator direction | **Interim pin:** workstation VAULT HD `D:\vault\coulomb\freedom-intelligence\`; layout created; soft quota retained |
|
||||
| 2026-07-28 | operator | WSL mount verified: `D:` → `/mnt/d` (1.9T, writable) |
|
||||
| 2026-07-24 | operator direction | Facility intent = 1 TB local NAS; soft quota 850 GiB |
|
||||
| 2026-07-28 | operator direction | **Interim pin:** workstation VAULT HD |
|
||||
| 2026-09-13 | FI-WP-0004 | **SoT pin:** Scaleway `nl-ams`; VAULT demoted to staging; euro gate replaces GiB quota |
|
||||
|
|
|
|||
|
|
@ -22,7 +22,8 @@ full resurveys.
|
|||
| One brief per date | Path `briefs/YYYY/MM/YYYY-MM-DD.md` |
|
||||
| Calendar | Prefer **Europe/Berlin** date in front matter |
|
||||
| Skip empty days | Either omit file or write `*No material delta.*` in Headline deltas |
|
||||
| Automation due bit | Cleared only by State Hub event `fi_daily_brief` (below) |
|
||||
| Automation due bit | Cleared only by State Hub event `fi_daily_brief` **after** the brief exists on `origin/main` |
|
||||
| Hub without git | A `fi_daily_brief` event with no matching file is a **false completion**. Run `scripts/verify_brief_durability.py`. |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -46,17 +47,20 @@ full resurveys.
|
|||
|
||||
4. **Collection candidates**
|
||||
- Use inventory field names; set priority
|
||||
- Do **not** re-nominate a model already in `inventory/catalog/`
|
||||
- Do **not** invent parameter counts; cite the model card or mark unverified
|
||||
- If nominating: add or update `inventory/catalog/{id}.yaml` with
|
||||
`status: candidate` and `collection.brief_refs`
|
||||
|
||||
5. **Lab implications**
|
||||
- Link open workplans (`FI-WP-*`) only when action is real
|
||||
|
||||
6. **Commit** (operator policy) — brief + any catalog YAML
|
||||
6. **Commit and push** — brief + any catalog YAML must land on `origin/main`.
|
||||
A local-only commit does **not** complete the day.
|
||||
|
||||
7. **Completion evidence** (required for activity-core rhythm)
|
||||
|
||||
Post State Hub progress (HTTP or MCP), shape:
|
||||
Post State Hub progress **only after** origin has the commit:
|
||||
|
||||
```yaml
|
||||
event_type: fi_daily_brief
|
||||
|
|
@ -67,8 +71,13 @@ full resurveys.
|
|||
date: "YYYY-MM-DD"
|
||||
path: "briefs/YYYY/MM/YYYY-MM-DD.md"
|
||||
collection_candidates: 0
|
||||
pushed: true
|
||||
origin_sha: "<full sha>"
|
||||
```
|
||||
|
||||
If the brief is written but push fails, post `event_type: executor_run`
|
||||
with `ok: false` so `due` stays true.
|
||||
|
||||
Workstation API often `http://127.0.0.1:8000`; remote tunnel ports per
|
||||
Custodian docs. See `docs/state-hub-delivery.md`.
|
||||
|
||||
|
|
@ -85,8 +94,10 @@ When activity-core emits `activity_task_spawn` with
|
|||
|
||||
1. Railiance host timer (or manual) runs
|
||||
`rein-aharness fi-research-brief --target-repo ~/freedom-intelligence`
|
||||
2. llm-connect drafts the brief; harness commits and posts **`fi_daily_brief`**
|
||||
3. Idempotency: one brief file per date; resolver clears `due` on completion event
|
||||
2. llm-connect drafts the brief; harness **commits, pushes to origin**, then
|
||||
posts **`fi_daily_brief`** with `pushed=true`
|
||||
3. Idempotency: one brief file per date on origin; resolver clears `due` only
|
||||
then. Local commit without push must not clear `due`.
|
||||
|
||||
Do **not** install Claude Code / Codex on railiance01 for this rhythm.
|
||||
|
||||
|
|
@ -116,9 +127,10 @@ make -C ~/activity-core sync-activity-definitions
|
|||
| Check | Pass |
|
||||
| ----- | ---- |
|
||||
| Signal | No filler paragraphs |
|
||||
| Sources | At least one primary link per non-empty axis row |
|
||||
| Honesty | Benchmarks marked directional; prices “verify before budget” |
|
||||
| Collection | License noted for any candidate |
|
||||
| Sources | At least one primary URL per non-empty axis row |
|
||||
| Honesty | Benchmarks marked directional; prices “verify before budget”; sizes from the card |
|
||||
| Collection | License + real size/class; never re-announce a cataloged id as a new release |
|
||||
| Axes | C and D must be scanned; `*(none)*` is allowed only after a look |
|
||||
| Length | Prefer one screen to three; deep dives go under `research/` |
|
||||
|
||||
---
|
||||
|
|
|
|||
136
docs/decisions/2026-09-13-scaleway-object-reserve.md
Normal file
136
docs/decisions/2026-09-13-scaleway-object-reserve.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
# Decision: Scaleway object storage as the open-weight reserve
|
||||
|
||||
**Date:** 2026-09-13
|
||||
**Status:** accepted
|
||||
**Affects:** `docs/backup-storage-policy.md`, `inventory/collection-policy.md`,
|
||||
`docs/hardware-envelope.md`, `scripts/collect_model.py`, FI-WP-0004
|
||||
**Supersedes (storage backend only):**
|
||||
`docs/decisions/2026-07-24-nas-strategic-reserve.md` § storage facility.
|
||||
The **capability-first** rule from that decision stands.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Source of truth for weight blobs** is a dedicated Scaleway Object
|
||||
Storage bucket in `nl-ams`, operated as `reef-storage`, consumed by
|
||||
this repo. Git still holds catalog metadata only.
|
||||
|
||||
2. **VAULT HD** (`D:\vault\coulomb\freedom-intelligence\`) is **staging
|
||||
and optional local R-spine cache**. It is not the strategic envelope.
|
||||
|
||||
3. **Storage class follows tier:**
|
||||
- **R (runnable spine):** One Zone IA — cheap, GET is immediate.
|
||||
- **S (strategic):** Glacier — cheapest just-in-case; restore cost
|
||||
and latency are acceptable.
|
||||
- **W:** only with euro-budget headroom; One Zone IA or Glacier by size.
|
||||
|
||||
4. **Capacity gate is euros/month, not a 850 GiB disk quota.** The 850 /
|
||||
920 GiB numbers were a 1 TB NAS artifact. Usage-based object storage
|
||||
removes that wall. Soft infrastructure budget: **€15 / month**
|
||||
(ex VAT) for this bucket. Founder Scaleway billing alert remains the
|
||||
hard backstop (`RESOURCE-WP-0002`).
|
||||
|
||||
5. **Do not reuse** `railiance-platform-pg-backup`. That bucket expires
|
||||
objects at 30 days. Model weights must not.
|
||||
|
||||
---
|
||||
|
||||
## Why Scaleway (economics)
|
||||
|
||||
Org already selected Scaleway over Hetzner / Host Europe for S3
|
||||
(`resource-control` RESOURCE-WP-0002, 2026-08-14). Same project, same
|
||||
failure domain independence from Host Europe `railiance01`.
|
||||
|
||||
Prices from [Scaleway storage pricing](https://www.scaleway.com/en/pricing/storage/)
|
||||
as of 2026-09-13, ex tax:
|
||||
|
||||
| Class | € / GB-month | 45 GiB (R spine) | 167 GiB (V4-Flash) | 1454 GiB (Kimi K3) |
|
||||
| ----- | -----------: | ---------------: | -----------------: | -----------------: |
|
||||
| Multi-AZ Standard | ~0.0146 | €0.66 | €2.44 | €21.23 |
|
||||
| One Zone IA | ~0.012 | €0.54 | €2.00 | €17.45 |
|
||||
| Glacier | ~0.00254 | €0.11 | €0.42 | **€3.69** |
|
||||
|
||||
Glacier restore to Standard is €0.009/GB (V4-Flash restore ≈ €1.50;
|
||||
K3 restore ≈ €13). Ingress is included. Egress is 75 GB/month free,
|
||||
then €0.01/GB — relevant when we actually pull a giant to a GPU host,
|
||||
not when we park it.
|
||||
|
||||
**Portfolio that the 850 GiB NAS could not hold:**
|
||||
|
||||
| Item | Class | Est. monthly |
|
||||
| ---- | ----- | -----------: |
|
||||
| R-spine resident (One Zone IA) | R | €0.54 |
|
||||
| DeepSeek-V4-Flash-0731 (Glacier) | S | €0.42 |
|
||||
| Kimi K3 official tree (Glacier) | S | €3.69 |
|
||||
| **Total** | | **~€4.65** |
|
||||
|
||||
That is inside the €15 soft budget with room for a second compressed
|
||||
giant or a Qwen3.8-27B R upgrade. Buying another multi-TB disk to
|
||||
fit K3 would spend the same money in a year and stay on the
|
||||
workstation failure domain.
|
||||
|
||||
VAULT remains useful: sequential staging for `huggingface_hub`
|
||||
downloads, and a local copy of the R-spine if the workstation is
|
||||
doing inference. It is no longer the thing INTENT means by “reserve.”
|
||||
|
||||
---
|
||||
|
||||
## Bucket contract
|
||||
|
||||
| Field | Value |
|
||||
| ----- | ----- |
|
||||
| Provider | Scaleway Object Storage |
|
||||
| Region | `nl-ams` |
|
||||
| Endpoint | `https://s3.nl-ams.scw.cloud` |
|
||||
| Bucket (planned) | `railiance-fi-open-weight-reserve` |
|
||||
| Reef attributes | `reef-storage/substrate/object-stores/fi-open-weight-reserve.yaml` |
|
||||
| Public access | disabled |
|
||||
| Versioning | on |
|
||||
| Object expiry | **none** |
|
||||
| Lifecycle | `strategic/` → Glacier after 1 day |
|
||||
| Encryption | provider-managed at rest |
|
||||
| Identity | bucket-scoped key, not the postgres backup key |
|
||||
| Secrets | `railiance-platform` / OpenBao — never this git repo |
|
||||
|
||||
### Prefix layout
|
||||
|
||||
```text
|
||||
s3://railiance-fi-open-weight-reserve/
|
||||
├── models/{org}__{name}/{revision}/ # R / W (One Zone IA at PUT)
|
||||
├── strategic/{org}__{name}/{revision}/ # S (Glacier via lifecycle)
|
||||
├── companions/
|
||||
├── staging/ # incomplete uploads; purge after 14d
|
||||
└── manifests/{org}__{name}/{revision}/MANIFEST.json
|
||||
```
|
||||
|
||||
Each prefix maps to one catalog YAML. `collection.storage_path` is the
|
||||
`s3://` URI, not a VAULT path.
|
||||
|
||||
---
|
||||
|
||||
## What stays from 2026-07-24
|
||||
|
||||
- Run envelope ≠ reserve envelope.
|
||||
- License and integrity first.
|
||||
- Catalog in git, blobs off git.
|
||||
- No public mirror, no closed weights.
|
||||
- R-spine always kept; S is a small number of top open identities.
|
||||
|
||||
## What changes
|
||||
|
||||
| Before | After |
|
||||
| ------ | ----- |
|
||||
| Facility = workstation VAULT HD | Facility = Scaleway `nl-ams` |
|
||||
| Soft quota 850 GiB | Soft budget €15/month |
|
||||
| K3 deferred on capacity | K3 economically in-scope on Glacier |
|
||||
| V4-Flash uncollectable-as-12B | V4-Flash (~167 GiB MIT MoE) is the first S pull |
|
||||
|
||||
---
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- [x] FI-WP-0004-T01 this file
|
||||
- [ ] FI-WP-0004-T08 operator creates the bucket
|
||||
- [ ] FI-WP-0004-T09 first collect: DeepSeek-V4-Flash-0731
|
||||
- [ ] resource-control inventory record (optional; FI consumes, resource-control costs)
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Hardware envelope — run vs reserve
|
||||
|
||||
**Status:** updated 2026-07-24 (1 TB NAS strategic reserve)
|
||||
**Status:** updated 2026-09-13 (Scaleway object reserve; run envelope unchanged)
|
||||
**Used by:** inventory `hardware_class`, axis B/C, collection tiers R/S
|
||||
|
||||
---
|
||||
|
|
@ -10,10 +10,10 @@
|
|||
Separate two questions that used to be conflated:
|
||||
|
||||
1. **What can we run or fine-tune now?** → *run envelope*
|
||||
2. **What should we store on the NAS just in case?** → *reserve envelope* (1 TB)
|
||||
2. **What should we store just in case?** → *reserve envelope* (Scaleway Glacier / One Zone IA)
|
||||
|
||||
Collecting a model **does not** require it to fit the run envelope.
|
||||
See `docs/decisions/2026-07-24-nas-strategic-reserve.md`.
|
||||
See `docs/decisions/2026-09-13-scaleway-object-reserve.md`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -34,13 +34,15 @@ valid for NAS collection.
|
|||
|
||||
---
|
||||
|
||||
## Reserve envelope (NAS)
|
||||
## Reserve envelope (object store)
|
||||
|
||||
| Parameter | Value |
|
||||
| --------- | ----- |
|
||||
| Facility | VAULT HD `D:\vault\coulomb\freedom-intelligence\` (soft quota 850 GiB) |
|
||||
| Soft quota | 850 GiB |
|
||||
| Hard stop | 920 GiB |
|
||||
| Facility | Scaleway Object Storage `nl-ams` (`railiance-fi-open-weight-reserve`) |
|
||||
| Soft budget | **€15 / month** |
|
||||
| R class | One Zone IA |
|
||||
| S class | Glacier |
|
||||
| Staging | VAULT HD `D:\vault\coulomb\freedom-intelligence\` |
|
||||
| Policy | `docs/backup-storage-policy.md` |
|
||||
|
||||
---
|
||||
|
|
@ -51,7 +53,8 @@ valid for NAS collection.
|
|||
| ---- | ---- | --- | ---- | --- | ----- |
|
||||
| TBD | primary local inference | TBD | TBD | TBD | Run envelope |
|
||||
| TBD | training experiments | TBD | TBD | TBD | Run envelope |
|
||||
| **VAULT HD (D:)** | weight store | n/a | n/a | n/a | **Reserve envelope** — soft 850 GiB |
|
||||
| **VAULT HD (D:)** | staging / R cache | n/a | n/a | n/a | **Not SoT** — downloads and optional local R |
|
||||
| **Scaleway nl-ams** | weight SoT | n/a | n/a | n/a | **Reserve envelope** — €15/mo soft |
|
||||
| railiance01 | cluster | n/a | n/a | n/a | Do not store weight trees on hot disks |
|
||||
|
||||
---
|
||||
|
|
@ -76,7 +79,7 @@ Strategic giants may remain **cold storage only** until hardware upgrades.
|
|||
| QLoRA 7–8B | T1–T2 |
|
||||
| QLoRA 14B | T2 |
|
||||
| Full FT 7B | T2–T3 |
|
||||
| 70B+ / large MoE FT | Future facility — weights may already sit on NAS |
|
||||
| 70B+ / large MoE FT | Future facility — weights may already sit on Glacier |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -86,3 +89,4 @@ Strategic giants may remain **cold storage only** until hardware upgrades.
|
|||
| ---- | ------ |
|
||||
| 2026-07-24 | Draft tiers |
|
||||
| 2026-07-24 | Split run vs reserve; NAS 1 TB; S tier may exceed run envelope |
|
||||
| 2026-09-13 | Reserve envelope moved to Scaleway; VAULT is staging; euro gate |
|
||||
|
|
|
|||
|
|
@ -19,7 +19,8 @@ activity-core (when / what / where)
|
|||
rein-aharness on railiance01 (does the work)
|
||||
primary: claim-loop → ops_run → fi-research-brief (REIN-A-0002)
|
||||
break-glass only: host timer 07:35 (disabled 2026-08-05; re-enable if claim loop down)
|
||||
llm-connect drafts brief → commit → push → fi_daily_brief
|
||||
llm-connect drafts brief → commit → git push origin → fi_daily_brief
|
||||
(fi_daily_brief is forbidden until origin has the file)
|
||||
|
||||
Operator morning review (no AI) from this repo:
|
||||
```bash
|
||||
|
|
@ -118,7 +119,19 @@ detail:
|
|||
collection_candidates: <int>
|
||||
```
|
||||
|
||||
Posted automatically by `rein-aharness fi-research-brief` on success.
|
||||
Posted automatically by `rein-aharness fi-research-brief` **only after**
|
||||
`git push origin` succeeds. `detail.pushed` and `detail.origin_sha` are
|
||||
required. A local commit with a failed push is `executor_run` / `ok=false`.
|
||||
|
||||
FI is the definition owner: origin publication is a **granted** capability
|
||||
of this rhythm (FI-WP-0004-T04), not an ungoverned env widening of
|
||||
`FI_RESEARCH_BRIEF_PUSH`.
|
||||
|
||||
Verify drift:
|
||||
|
||||
```bash
|
||||
python3 scripts/verify_brief_durability.py --require-hub
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -127,4 +140,6 @@ Posted automatically by `rein-aharness fi-research-brief` on success.
|
|||
- Workstation `crontab` that writes the brief (bypasses activity-core)
|
||||
- Installing Claude Code / Codex on railiance01 for this rhythm
|
||||
- Treating host timers as the scheduler of record
|
||||
- Skipping the completion event (leaves `due=true` forever)
|
||||
- Skipping the completion event (leaves `due=true` forever)
|
||||
- Posting `fi_daily_brief` for a local-only commit (clears `due`, loses the day)
|
||||
- Reusing `railiance-platform-pg-backup` for weights (30-day expiry)
|
||||
|
|
|
|||
|
|
@ -19,7 +19,8 @@ Do not create workplans via hub `create_workplan` APIs.
|
|||
|
||||
## Required: completion evidence for daily rhythm
|
||||
|
||||
Every completed brief day (including “no material delta”) should post:
|
||||
Every completed brief day (including “no material delta”) should post
|
||||
**after the file is on `origin/main`**:
|
||||
|
||||
```yaml
|
||||
event_type: fi_daily_brief
|
||||
|
|
@ -30,11 +31,15 @@ detail:
|
|||
date: "YYYY-MM-DD"
|
||||
path: "briefs/YYYY/MM/YYYY-MM-DD.md"
|
||||
collection_candidates: 0
|
||||
pushed: true
|
||||
origin_sha: "<full sha>"
|
||||
```
|
||||
|
||||
This is the **idempotence signal** for activity-core context query
|
||||
`fi_brief_status` (FI-WP-0002-T03): `due=false` when an event exists for today
|
||||
(Europe/Berlin) with `detail.repo=freedom-intelligence`.
|
||||
(Europe/Berlin) with `detail.repo=freedom-intelligence` **and**
|
||||
`detail.pushed=true`. A hub event without a git file is a false completion
|
||||
(`scripts/verify_brief_durability.py`).
|
||||
|
||||
Inbound automation uses `activity_task_spawn` from activity-core (state-hub sink).
|
||||
See `activity-core/docs/task-emission-consumer-contract.md`.
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
# Model inventory
|
||||
|
||||
In-repo **catalog of open-weight revisions** reserved (or nominated) for the lab.
|
||||
Weight blobs live on the **VAULT HD** at `D:\vault\coulomb\freedom-intelligence\` —
|
||||
see `docs/backup-storage-policy.md`.
|
||||
Weight blobs live on **Scaleway Object Storage** (`s3://railiance-fi-open-weight-reserve/…`).
|
||||
VAULT HD is staging only — see `docs/backup-storage-policy.md`.
|
||||
|
||||
## Layout
|
||||
|
||||
|
|
|
|||
|
|
@ -1,8 +1,10 @@
|
|||
# Open-weight reserve status
|
||||
|
||||
**As of:** 2026-08-03
|
||||
**Store:** `/mnt/d/vault/coulomb/freedom-intelligence/` (`D:\vault\coulomb\freedom-intelligence\`)
|
||||
**Soft quota:** 850 GiB · **Hard stop:** 920 GiB
|
||||
**As of:** 2026-09-13
|
||||
**SoT store:** Scaleway Object Storage `nl-ams` bucket
|
||||
`railiance-fi-open-weight-reserve` (**planned** — FI-WP-0004-T08)
|
||||
**Staging:** `/mnt/d/vault/coulomb/freedom-intelligence/` (VAULT HD)
|
||||
**Capacity gate:** **€15 / month** soft (not 850 GiB)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -10,64 +12,70 @@
|
|||
|
||||
| Class | Count | Status mix |
|
||||
| ----- | ----: | ---------- |
|
||||
| Catalog entries | 13 | + Kimi K3 S1 |
|
||||
| **verified** on disk | 4 | R embeds + Qwen3-8B + R1-Distill-14B |
|
||||
| **approved** (pull deferred) | S giants + Llama edge + **Kimi K3** | capacity / HF gate |
|
||||
| **candidate** (W) | 3 | headroom-gated |
|
||||
| Catalog entries | 15 | + V4-Flash S, + Qwen3.8-27B W/R |
|
||||
| **verified** on VAULT | 4 | R embeds + Qwen3-8B + R1-Distill-14B |
|
||||
| **approved**, blobs missing | S giants + Llama-3.2-3B + **V4-Flash** | HF gate / bucket not live |
|
||||
| **candidate** | 4 | W + Qwen3.8-27B |
|
||||
|
||||
### Disk use (model tree)
|
||||
### Disk / object use
|
||||
|
||||
| Path | Role | Approx |
|
||||
| ---- | ---- | ------ |
|
||||
| `models/nomic-ai__nomic-embed-text-v1.5/main` | R embed | ~0.5 GiB verified |
|
||||
| `models/BAAI__bge-m3/main` | R embed | ~2.1 GiB verified |
|
||||
| `models/Qwen__Qwen3-8B/main` | R instruct | ~15.3 GiB verified |
|
||||
| `models/deepseek-ai__DeepSeek-R1-Distill-Qwen-14B/main` | R reason | ~27.5 GiB verified |
|
||||
| Location | Role | Approx |
|
||||
| -------- | ---- | ------ |
|
||||
| VAULT `models/nomic-ai__nomic-embed-text-v1.5/main` | R embed (staging cache) | ~0.5 GiB verified |
|
||||
| VAULT `models/BAAI__bge-m3/main` | R embed (staging cache) | ~2.1 GiB verified |
|
||||
| VAULT `models/Qwen__Qwen3-8B/main` | R instruct (staging cache) | ~15.3 GiB verified |
|
||||
| VAULT `models/deepseek-ai__DeepSeek-R1-Distill-Qwen-14B/main` | R reason (staging cache) | ~27.5 GiB verified |
|
||||
| Scaleway bucket | SoT | **empty** (not created) |
|
||||
|
||||
**~45 GiB** used → **~800+ GiB** soft-quota headroom (insufficient for full Kimi K3 ~1454 GiB).
|
||||
|
||||
VAULT free (2026-08-03): ~1.7 TB free of 1.9 TB — full K3 would still dominate the volume and **violate soft quota**.
|
||||
VAULT ~45 GiB used remains a **cache**, not the reserve.
|
||||
|
||||
---
|
||||
|
||||
## Decision matrix (2026-08-03)
|
||||
## Decision matrix (2026-09-13)
|
||||
|
||||
| Id | Tier | Decision | Disk |
|
||||
| -- | ---- | -------- | ---- |
|
||||
| Qwen3-8B | R | **verified** | yes |
|
||||
| Id | Tier | Decision | Blobs |
|
||||
| -- | ---- | -------- | ----- |
|
||||
| Qwen3-8B | R | **verified** on VAULT cache | local only |
|
||||
| Llama-3.2-3B-Instruct | R | **approved** | deferred — HF gate |
|
||||
| BGE-M3 | R | **verified** | yes |
|
||||
| R1-Distill-Qwen-14B | R | **verified** | yes |
|
||||
| nomic-embed-text-v1.5 | R | **verified** | yes |
|
||||
| **Kimi-K3** | **S1** | **approved**, **collection deferred** | no — ~1454 GiB > soft quota |
|
||||
| DeepSeek-V3 | S1→secondary | **approved**, deferred | no — K3 is primary open SOTA identity |
|
||||
| BGE-M3 | R | **verified** on VAULT cache | local only |
|
||||
| R1-Distill-Qwen-14B | R | **verified** on VAULT cache | local only |
|
||||
| nomic-embed-text-v1.5 | R | **verified** on VAULT cache | local only |
|
||||
| **DeepSeek-V4-Flash-0731** | **S1** | **approved** — **first Scaleway pull** | no — ~167 GiB Glacier |
|
||||
| Kimi-K3 | S1 secondary | **approved**, deferred until after V4-Flash | no — ~1454 GiB Glacier ≈ €3.69/mo |
|
||||
| DeepSeek-V3 | S superseded identity | **approved**, deferred | no — V4-Flash is the collectable S1 |
|
||||
| DeepSeek-R1 full | S2 | **approved**, deferred | no |
|
||||
| Qwen3-72B | S4 | **approved**, deferred | no |
|
||||
| Llama-3.3-70B-Instruct | S4 | **approved**, deferred | no |
|
||||
| Qwen3.8-27B | W/R | **candidate** | no |
|
||||
| Qwen3-14B | W | candidate | no |
|
||||
| R1-Distill-32B | W | candidate | no |
|
||||
| bge-reranker-v2-m3 | W | candidate | no |
|
||||
|
||||
**Correction:** there is no `DeepSeek-V4-Flash-0731-12B`. Automated briefs
|
||||
that used that id were wrong. Do not collect under that name.
|
||||
|
||||
---
|
||||
|
||||
## Hardware coverage gaps
|
||||
|
||||
| Need | Status |
|
||||
| ---- | ------ |
|
||||
| Default local instruct (8B) | **Covered** (Qwen3-8B) |
|
||||
| Default local instruct (8B) | **Covered** (Qwen3-8B cache) |
|
||||
| Stronger local instruct (27B) | **Candidate** Qwen3.8-27B |
|
||||
| Edge micro-agent | Llama-3.2-3B blocked on HF auth |
|
||||
| Multilingual RAG embed | **Covered** (BGE-M3 + nomic) |
|
||||
| Local reason | **Covered** (R1-Distill-14B) |
|
||||
| Frontier-open general MoE offline | **Cataloged** Kimi K3 — pull blocked on capacity |
|
||||
| Full open reasoner MoE | Deferred S2 |
|
||||
| Frontier-open MIT MoE offline | **Cataloged** V4-Flash — pull blocked on bucket |
|
||||
| Full open giant (K3) | Cataloged; Glacier-affordable; after V4-Flash |
|
||||
|
||||
---
|
||||
|
||||
## Next pulls (operator)
|
||||
|
||||
1. **Capacity decision for Kimi K3** (raise soft quota / dedicate disk / official compressed only).
|
||||
2. Set `HF_TOKEN` and pull Llama-3.2-3B-Instruct.
|
||||
3. Optional S4: one of Qwen3-72B vs Llama-3.3-70B under remaining quota.
|
||||
4. DeepSeek V4-Flash-0731 as mid open value candidate (catalog later if desired).
|
||||
1. **FI-WP-0004-T08** create `railiance-fi-open-weight-reserve` (no 30-day expiry).
|
||||
2. **FI-WP-0004-T09** collect `deepseek-ai/DeepSeek-V4-Flash-0731` → `strategic/` Glacier.
|
||||
3. Set `HF_TOKEN` and pull Llama-3.2-3B-Instruct into `models/` One Zone IA.
|
||||
4. Decide Qwen3.8-27B vs keeping Qwen3-8B as the R instruct default.
|
||||
5. Optional: Kimi K3 on Glacier if the month is still under €15.
|
||||
|
||||
Tool: `scripts/collect_model.py` (weights-only, sequential, writes `MANIFEST.json`).
|
||||
Tool: `scripts/collect_model.py` (weights-only, sequential, MANIFEST, optional `--s3-bucket`).
|
||||
|
|
|
|||
74
inventory/catalog/Qwen__Qwen3.8-27B__candidate.yaml
Normal file
74
inventory/catalog/Qwen__Qwen3.8-27B__candidate.yaml
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
id: Qwen__Qwen3.8-27B__candidate
|
||||
status: candidate
|
||||
name: Qwen3.8-27B
|
||||
org: Qwen
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/Qwen/Qwen3.8-27B
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/Qwen/Qwen3.8-27B
|
||||
project_url: https://qwen.ai/
|
||||
license:
|
||||
spdx: Apache-2.0
|
||||
url: https://huggingface.co/Qwen/Qwen3.8-27B
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: "Apache 2.0 on the 27B dense multimodal card (2026-08-14). Confirm at pin. Not the Qwen3.8-Max bespoke licence."
|
||||
profile:
|
||||
summary: "Dense 27B multimodal Apache-2.0 — candidate R-spine upgrade vs Qwen3-8B."
|
||||
original_source: https://huggingface.co/Qwen/Qwen3.8-27B
|
||||
use_cases:
|
||||
- "Local instruct / vision on T2–T3 (24 GB class at Q4)"
|
||||
- "Coding and office agents that need more than 8B"
|
||||
- "Homelab QLoRA base"
|
||||
sweet_spots:
|
||||
- "Apache-2.0, ungated, actually runnable unlike V4-Flash / K3"
|
||||
- "Native vision; 262k context (YaRN toward 1M)"
|
||||
not_ideal_for:
|
||||
- "Replacing the 8B always-on default until measured"
|
||||
- "Standing in for Qwen3.8-2.4T-A95B (different licence, multi-TB)"
|
||||
capability_notes: >
|
||||
Released 2026-08-14, the day the automated brief last landed on
|
||||
origin — so the sensing loop never recorded it. Dense 27.8B,
|
||||
multimodal, Apache 2.0. Distinct from Qwen3.8-2.4T-A95B (custom
|
||||
Max licence, ~4.89 TB). Nominate as W/R; do not pull until euro
|
||||
budget and R-spine review after V4-Flash lands.
|
||||
swot:
|
||||
strengths:
|
||||
- "Apache-2.0 dense multimodal at a size we can actually serve later"
|
||||
- "Likely better local default than Qwen3-8B if VRAM allows"
|
||||
weaknesses:
|
||||
- "Larger than current verified R instruct (8B)"
|
||||
- "Hardware envelope hosts still TBD"
|
||||
opportunities:
|
||||
- "R-spine upgrade path that does not need Glacier"
|
||||
- "Vision in the local working set"
|
||||
threats:
|
||||
- "Qwen 4.x could supersede before we pull"
|
||||
size:
|
||||
total_bytes: 0
|
||||
total_human: "~56 GiB class (reports); measure at pull — Q4 much smaller"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 20
|
||||
min_vram_gb_fp16: 56
|
||||
notes: "T2 stretch / T3. Not T1."
|
||||
axes: [B, C]
|
||||
priority: medium
|
||||
collection:
|
||||
approved_by: ""
|
||||
approved_at: null
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- briefs/2026/09/2026-09-13.md
|
||||
reason: "Missed by the Aug 14 brief cutoff; Apache-2.0 dense 27B is the obvious R-upgrade candidate."
|
||||
tags: [tier-w, tier-r, instruct, vision, apache]
|
||||
companions: []
|
||||
notes: "Do not confuse with Qwen3.8-2.4T-A95B (S-class, custom licence, multi-TB)."
|
||||
history:
|
||||
- at: "2026-09-13"
|
||||
event: nominated
|
||||
by: grok
|
||||
detail: "Catch-up brief after cadence hole. Candidate only."
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
id: deepseek-ai__DeepSeek-V4-Flash-0731__strategic
|
||||
status: approved
|
||||
name: DeepSeek-V4-Flash-0731
|
||||
org: deepseek-ai
|
||||
source:
|
||||
kind: huggingface
|
||||
url: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
|
||||
revision: main
|
||||
model_card_url: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
|
||||
project_url: https://www.deepseek.com/
|
||||
paper_url: https://arxiv.org/abs/2606.19348
|
||||
license:
|
||||
spdx: MIT
|
||||
url: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
|
||||
allows_offline_retention: true
|
||||
allows_local_ops: true
|
||||
allows_fine_tune: true
|
||||
notes: "Official card: repository and weights MIT. Confirm at pin."
|
||||
profile:
|
||||
summary: "S-tier open MIT MoE — 284B base / ~13B active (304B with DSpark); not a 12B dense model."
|
||||
original_source: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
|
||||
use_cases:
|
||||
- "Primary open-weight agentic / coding identity after Jul 2026"
|
||||
- "Offline optionality if DeepSeek API access changes"
|
||||
- "Future multi-GPU serve (vLLM DSpark / SGLang); not current R-spine"
|
||||
- "Capability bar for daily briefs against closed frontier"
|
||||
sweet_spots:
|
||||
- "Best MIT-licensed open near-frontier that actually fits cheap Glacier (~167 GiB HF tree)"
|
||||
- "Fits the €15/month Scaleway budget with room for K3"
|
||||
- "Displaces DeepSeek-V3 as the collectable S1 identity"
|
||||
not_ideal_for:
|
||||
- "Current single-consumer-GPU daily ops (use R spine / Qwen3.8-27B)"
|
||||
- "Treating 'Flash' as a 12B dense model (that reading was an executor error)"
|
||||
capability_notes: >
|
||||
Official V4-Flash release (2026-07-31). Same structure as
|
||||
DeepSeek-V4-Flash-DSpark (speculative decoding module attached).
|
||||
HF card reports 304B params (284B base + draft). ~13B active.
|
||||
1M context. Terminal Bench 2.1 82.7. HF tree listed ~167 GB.
|
||||
Automated briefs 2026-08-04…14 repeatedly called this a 12B dense
|
||||
consumer-GPU model and once claimed it was already collected. It
|
||||
was not. Size class is S, not R.
|
||||
swot:
|
||||
strengths:
|
||||
- "MIT, official org, actually downloadable"
|
||||
- "Near-frontier agentic scores at ~13B active"
|
||||
- "Footprint (~167 GiB) is cheap on Glacier and was always inside the old 850 GiB quota"
|
||||
weaknesses:
|
||||
- "Serve needs multi-GPU / high unified memory, not T1–T2"
|
||||
- "Executor hallucinated it as 12B for two weeks — catalog must stay the authority"
|
||||
opportunities:
|
||||
- "First S pull onto Scaleway (FI-WP-0004-T09)"
|
||||
- "Compressed / GGUF community packs exist if we want a later R experiment (not a substitute for the official tree)"
|
||||
threats:
|
||||
- "Successor V4.x could land while we still have no object on disk"
|
||||
- "Quota-era deferral already cost a month of optionality"
|
||||
size:
|
||||
total_bytes: 179000000000
|
||||
total_human: "~167 GiB official HF tree (card file listing 2026-09-13); 304B params / ~13B active"
|
||||
hardware_class:
|
||||
min_vram_gb_q4: 0
|
||||
min_vram_gb_fp16: 0
|
||||
notes: "Beyond current lab run envelope (T3–T4). Community 2-bit reports exist for 128 GB unified memory; not an R-spine claim."
|
||||
axes: [A, B, D]
|
||||
priority: high
|
||||
collection:
|
||||
approved_by: "grok"
|
||||
approved_at: "2026-09-13"
|
||||
downloaded_at: null
|
||||
downloaded_by: ""
|
||||
storage_path: ""
|
||||
brief_refs:
|
||||
- briefs/2026/08/2026-08-03.md
|
||||
- briefs/2026/09/2026-09-13.md
|
||||
- docs/decisions/2026-09-13-scaleway-object-reserve.md
|
||||
reason: "S1 collectable — strongest MIT open MoE that fits the Scaleway euro budget; first pull after bucket create."
|
||||
tags: [tier-s, strategic, moe, mit, beyond-run-envelope]
|
||||
companions: []
|
||||
notes: >
|
||||
Do not confuse with a 12B dense checkpoint. There is no
|
||||
deepseek-ai/DeepSeek-V4-Flash-0731-12B. Pull the official card repo.
|
||||
Storage class at upload: strategic/ → Glacier.
|
||||
history:
|
||||
- at: "2026-08-03"
|
||||
event: nominated
|
||||
by: grok
|
||||
detail: "Catch-up brief named V4-Flash-0731 as mid-tier open value leader."
|
||||
- at: "2026-08-04"
|
||||
event: misidentified
|
||||
by: rein-aharness
|
||||
detail: "Automated briefs treated it as 12B dense; never cataloged; 08-07 claimed on-disk collection that did not happen."
|
||||
- at: "2026-09-13"
|
||||
event: approved
|
||||
by: grok
|
||||
detail: "Corrected to 304B-class MoE ~167 GiB MIT. Approved for Scaleway Glacier once FI-WP-0004-T08 bucket exists."
|
||||
|
|
@ -1,15 +1,16 @@
|
|||
# Collection policy — open-weight reserve
|
||||
|
||||
**Status:** updated for 1 TB NAS + strategic capability reserve (2026-07-24)
|
||||
**Status:** updated for Scaleway object reserve (2026-09-13)
|
||||
**Related:** `schema.yaml`, `docs/backup-storage-policy.md`,
|
||||
`docs/decisions/2026-07-24-nas-strategic-reserve.md`, `INTENT.md`
|
||||
`docs/decisions/2026-09-13-scaleway-object-reserve.md`, `INTENT.md`
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Decide **what** enters the open-weight reserve, **who** may approve it, and
|
||||
**when** a daily-brief candidate becomes a catalog entry with blobs on the NAS.
|
||||
**when** a daily-brief candidate becomes a catalog entry with blobs on
|
||||
Scaleway (staged locally first).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -19,7 +20,7 @@ Decide **what** enters the open-weight reserve, **who** may approve it, and
|
|||
need later — **including models too large to run on current lab hardware**
|
||||
* Also hold a **runnable spine** for day-to-day local ops and fine-tunes
|
||||
* Enforce **license and integrity** before download completes
|
||||
* Stay within the **1 TB NAS soft quota (850 GiB)** — quality over mirror volume
|
||||
* Stay within the **€15 / month** soft budget — quality over mirror volume
|
||||
* Prefer official provenance; not a full Hugging Face scrape
|
||||
|
||||
---
|
||||
|
|
@ -32,7 +33,7 @@ Decide **what** enters the open-weight reserve, **who** may approve it, and
|
|||
| **Reserve** | Is this among the best open artifacts worth keeping *just in case*? | **Not gated by current VRAM** |
|
||||
|
||||
**Runnability is not an eligibility gate.** An unrunnable SOTA open MoE can be
|
||||
priority **high** if license and quota allow.
|
||||
priority **high** if license and euro budget allow.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -42,7 +43,8 @@ priority **high** if license and quota allow.
|
|||
2. **Clear license** — SPDX or linkable license text; `allows_offline_retention: true`
|
||||
3. **Stable provenance** — official org, tagged release, or commit revision
|
||||
4. **Lab rationale** — written `reason` (capability SOTA, runnable spine, embed, FT base, …)
|
||||
5. **Capacity** — estimated size fits under remaining soft quota (850 GiB)
|
||||
5. **Capacity** — estimated monthly storage cost fits under remaining
|
||||
€15 soft budget (Glacier for S, One Zone IA for R)
|
||||
|
||||
Fail any gate → `rejected` or never enter catalog.
|
||||
|
||||
|
|
@ -75,9 +77,9 @@ Catalog field: use `tags` including `tier-r` / `tier-s` / `tier-w` and
|
|||
| -------------------- | -------- |
|
||||
| **< 5 GiB** | Operator or lab agent after license check |
|
||||
| **5–40 GiB** | Explicit operator approval |
|
||||
| **> 40 GiB** | Operator approval + remaining soft-quota check |
|
||||
| **> 200 GiB (typical S giants)** | Operator approval + written note on which other S models may need to wait |
|
||||
| **Any size if quota ≥ 70% used** | Operator approval required |
|
||||
| **> 40 GiB** | Operator approval + remaining **euro-budget** check |
|
||||
| **> 200 GiB (typical S giants)** | Operator approval + written note that the object goes to **Glacier** |
|
||||
| **Any size if month ≥ 70% of €15** | Operator approval required |
|
||||
| **Unclear license or ToS risk** | Do not collect; `rejected` |
|
||||
|
||||
Agents may **nominate** freely; they **collect** only in the < 5 GiB band with
|
||||
|
|
@ -97,8 +99,9 @@ unambiguous licenses unless the operator has approved the catalog entry.
|
|||
|
||||
* **Best available open general / reasoning / code weights** at the frontier of open
|
||||
* Large MoE or dense models even if current infra cannot serve them
|
||||
* Prefer official compressed releases (FP8, published quant) when full precision
|
||||
would exhaust the 1 TB NAS
|
||||
* Prefer official compressed releases (FP8, published quant) when full
|
||||
precision would blow the euro budget; Glacier makes full official trees
|
||||
of the *top* S identities affordable (V4-Flash ~€0.42/mo, K3 ~€3.69/mo)
|
||||
* One clear “best open” per capability niche is better than five near-duplicates
|
||||
|
||||
### Companions
|
||||
|
|
@ -124,7 +127,7 @@ Tokenizers, LoRA adapters, small eval fixtures when required to use a reserved b
|
|||
brief nominates
|
||||
→ candidate
|
||||
→ approved
|
||||
→ collecting (NAS staging/)
|
||||
→ collecting (local staging/ then S3 upload)
|
||||
→ collected
|
||||
→ verified
|
||||
→ superseded|evicted
|
||||
|
|
@ -136,14 +139,15 @@ brief nominates
|
|||
|
||||
1. Brief **Collection candidates** nominates R/S/W.
|
||||
2. Catalog YAML under `inventory/catalog/`.
|
||||
3. Download only after storage path is pinned (`docs/backup-storage-policy.md`) and approval rules pass.
|
||||
3. Download only after the Scaleway bucket is live (FI-WP-0004-T08) and
|
||||
approval rules pass. Stage on VAULT; SoT is `s3://`.
|
||||
4. `collection.brief_refs` / research refs for provenance of the nomination.
|
||||
|
||||
---
|
||||
|
||||
## Eviction rule of thumb
|
||||
|
||||
When over soft quota:
|
||||
When over the euro soft budget:
|
||||
|
||||
1. **W** tier and easily re-obtainable duplicates
|
||||
2. Superseded revisions with a stronger verified successor
|
||||
|
|
|
|||
|
|
@ -1,13 +1,27 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Collect one HF model into the VAULT reserve tree (weights-first).
|
||||
"""Collect one HF model into local staging, optionally upload to Scaleway.
|
||||
|
||||
Usage:
|
||||
/tmp/fi-hf/bin/python scripts/collect_model.py \\
|
||||
python3 scripts/collect_model.py \\
|
||||
--repo-id Qwen/Qwen3-8B --local-name Qwen__Qwen3-8B --revision main
|
||||
|
||||
Layout:
|
||||
{base}/models/{local_name}/{revision}/ # HF files (no full onnx dump)
|
||||
{base}/staging/... # incomplete
|
||||
python3 scripts/collect_model.py \\
|
||||
--repo-id deepseek-ai/DeepSeek-V4-Flash-0731 \\
|
||||
--local-name deepseek-ai__DeepSeek-V4-Flash-0731 \\
|
||||
--tier s \\
|
||||
--s3-bucket railiance-fi-open-weight-reserve
|
||||
|
||||
Layout (local staging, still required for huggingface_hub):
|
||||
{base}/models/{local_name}/{revision}/ # R / W
|
||||
{base}/strategic/{local_name}/{revision}/ # S
|
||||
{base}/staging/...
|
||||
|
||||
S3 prefix (SoT once bucket is live):
|
||||
s3://{bucket}/models|strategic/{local_name}/{revision}/
|
||||
s3://{bucket}/manifests/{local_name}/{revision}/MANIFEST.json
|
||||
|
||||
Credentials: AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY, or SCW_ACCESS_KEY +
|
||||
SCW_SECRET_KEY. Never commit keys.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
@ -80,44 +94,62 @@ def main() -> int:
|
|||
ap.add_argument(
|
||||
"--base",
|
||||
default="/mnt/d/vault/coulomb/freedom-intelligence",
|
||||
help="local staging root (VAULT HD)",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--tier",
|
||||
choices=("r", "s", "w"),
|
||||
default="r",
|
||||
help="r/w → models/ (One Zone IA); s → strategic/ (Glacier)",
|
||||
)
|
||||
ap.add_argument("--s3-bucket", default=os.environ.get("FI_S3_BUCKET") or "")
|
||||
ap.add_argument(
|
||||
"--s3-endpoint",
|
||||
default=os.environ.get("FI_S3_ENDPOINT", "https://s3.nl-ams.scw.cloud"),
|
||||
)
|
||||
ap.add_argument(
|
||||
"--s3-region",
|
||||
default=os.environ.get("FI_S3_REGION", "nl-ams"),
|
||||
)
|
||||
ap.add_argument("--token", default=os.environ.get("HF_TOKEN") or None)
|
||||
args = ap.parse_args()
|
||||
|
||||
base = Path(args.base)
|
||||
dest_kind = "strategic" if args.tier == "s" else "models"
|
||||
stage = base / "staging" / args.local_name / args.revision
|
||||
final = base / "models" / args.local_name / args.revision
|
||||
final = base / dest_kind / args.local_name / args.revision
|
||||
|
||||
if final.exists() and any(final.rglob("*.safetensors")) or (
|
||||
final.exists() and any(final.rglob("pytorch_model*.bin"))
|
||||
):
|
||||
print(f"already present with weights: {final}")
|
||||
return 0
|
||||
|
||||
if stage.exists():
|
||||
shutil.rmtree(stage, ignore_errors=True)
|
||||
stage.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f"downloading {args.repo_id}@{args.revision} -> {stage}")
|
||||
snapshot_download(
|
||||
repo_id=args.repo_id,
|
||||
revision=args.revision,
|
||||
local_dir=str(stage),
|
||||
allow_patterns=list(DEFAULT_ALLOW),
|
||||
ignore_patterns=list(IGNORE),
|
||||
token=args.token,
|
||||
max_workers=1,
|
||||
already = final.exists() and (
|
||||
any(final.rglob("*.safetensors")) or any(final.rglob("pytorch_model*.bin"))
|
||||
)
|
||||
if already:
|
||||
print(f"already present with weights: {final}", file=sys.stderr)
|
||||
if not args.s3_bucket:
|
||||
return 0
|
||||
else:
|
||||
if stage.exists():
|
||||
shutil.rmtree(stage, ignore_errors=True)
|
||||
stage.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# strip hub cache dir if present
|
||||
cache = stage / ".cache"
|
||||
if cache.exists():
|
||||
shutil.rmtree(cache, ignore_errors=True)
|
||||
print(f"downloading {args.repo_id}@{args.revision} -> {stage}")
|
||||
snapshot_download(
|
||||
repo_id=args.repo_id,
|
||||
revision=args.revision,
|
||||
local_dir=str(stage),
|
||||
allow_patterns=list(DEFAULT_ALLOW),
|
||||
ignore_patterns=list(IGNORE),
|
||||
token=args.token,
|
||||
max_workers=1,
|
||||
)
|
||||
|
||||
final.parent.mkdir(parents=True, exist_ok=True)
|
||||
if final.exists():
|
||||
shutil.rmtree(final)
|
||||
shutil.move(str(stage), str(final))
|
||||
cache = stage / ".cache"
|
||||
if cache.exists():
|
||||
shutil.rmtree(cache, ignore_errors=True)
|
||||
|
||||
final.parent.mkdir(parents=True, exist_ok=True)
|
||||
if final.exists():
|
||||
shutil.rmtree(final)
|
||||
shutil.move(str(stage), str(final))
|
||||
|
||||
artifacts = []
|
||||
total = 0
|
||||
|
|
@ -145,9 +177,99 @@ def main() -> int:
|
|||
"artifacts": artifacts,
|
||||
}
|
||||
(final / "MANIFEST.json").write_text(json.dumps(manifest, indent=2) + "\n")
|
||||
print(json.dumps({"ok": True, "total_bytes": total, "path": str(final)}, indent=2))
|
||||
result = {"ok": True, "total_bytes": total, "path": str(final), "tier": args.tier}
|
||||
|
||||
if args.s3_bucket:
|
||||
prefix = f"{dest_kind}/{args.local_name}/{args.revision}"
|
||||
storage_class = "GLACIER" if args.tier == "s" else "ONEZONE_IA"
|
||||
uploaded = upload_tree(
|
||||
final,
|
||||
bucket=args.s3_bucket,
|
||||
prefix=prefix,
|
||||
endpoint=args.s3_endpoint,
|
||||
region=args.s3_region,
|
||||
storage_class=storage_class,
|
||||
)
|
||||
man_key = f"manifests/{args.local_name}/{args.revision}/MANIFEST.json"
|
||||
upload_file(
|
||||
final / "MANIFEST.json",
|
||||
bucket=args.s3_bucket,
|
||||
key=man_key,
|
||||
endpoint=args.s3_endpoint,
|
||||
region=args.s3_region,
|
||||
storage_class="STANDARD",
|
||||
)
|
||||
result["s3"] = {
|
||||
"bucket": args.s3_bucket,
|
||||
"prefix": f"s3://{args.s3_bucket}/{prefix}/",
|
||||
"manifest": f"s3://{args.s3_bucket}/{man_key}",
|
||||
"storage_class": storage_class,
|
||||
"objects": uploaded,
|
||||
}
|
||||
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _s3_client(endpoint: str, region: str):
|
||||
try:
|
||||
import boto3
|
||||
except ImportError as exc:
|
||||
raise SystemExit("boto3 required for --s3-bucket (pip install boto3)") from exc
|
||||
access = os.environ.get("AWS_ACCESS_KEY_ID") or os.environ.get("SCW_ACCESS_KEY")
|
||||
secret = os.environ.get("AWS_SECRET_ACCESS_KEY") or os.environ.get("SCW_SECRET_KEY")
|
||||
if not access or not secret:
|
||||
raise SystemExit("set AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY or SCW_ACCESS_KEY/SCW_SECRET_KEY")
|
||||
return boto3.client(
|
||||
"s3",
|
||||
region_name=region,
|
||||
endpoint_url=endpoint,
|
||||
aws_access_key_id=access,
|
||||
aws_secret_access_key=secret,
|
||||
)
|
||||
|
||||
|
||||
def upload_file(
|
||||
path: Path,
|
||||
*,
|
||||
bucket: str,
|
||||
key: str,
|
||||
endpoint: str,
|
||||
region: str,
|
||||
storage_class: str,
|
||||
) -> None:
|
||||
client = _s3_client(endpoint, region)
|
||||
extra = {"StorageClass": storage_class} if storage_class else {}
|
||||
client.upload_file(str(path), bucket, key, ExtraArgs=extra)
|
||||
|
||||
|
||||
def upload_tree(
|
||||
root: Path,
|
||||
*,
|
||||
bucket: str,
|
||||
prefix: str,
|
||||
endpoint: str,
|
||||
region: str,
|
||||
storage_class: str,
|
||||
) -> int:
|
||||
n = 0
|
||||
for p in sorted(root.rglob("*")):
|
||||
if not p.is_file() or p.name.startswith("."):
|
||||
continue
|
||||
rel = p.relative_to(root).as_posix()
|
||||
key = f"{prefix.rstrip('/')}/{rel}"
|
||||
upload_file(
|
||||
p,
|
||||
bucket=bucket,
|
||||
key=key,
|
||||
endpoint=endpoint,
|
||||
region=region,
|
||||
storage_class=storage_class,
|
||||
)
|
||||
n += 1
|
||||
print(f"uploaded {key}", file=sys.stderr)
|
||||
return n
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
|
|||
114
scripts/verify_brief_durability.py
Executable file
114
scripts/verify_brief_durability.py
Executable file
|
|
@ -0,0 +1,114 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Fail if State Hub fi_daily_brief events are not on origin/main.
|
||||
|
||||
A brief day is complete only when briefs/YYYY/MM/YYYY-MM-DD.md exists in
|
||||
this git repository (the clone should be origin/main). Hub events that
|
||||
claim wrote/committed without a matching file are the 2026-08/09 failure
|
||||
mode: local commit on railiance, due-bit cleared, durable memory empty.
|
||||
|
||||
Usage:
|
||||
python3 scripts/verify_brief_durability.py
|
||||
STATE_HUB_URL=http://127.0.0.1:8000 python3 scripts/verify_brief_durability.py
|
||||
|
||||
Exit 0 = every hub success event has a brief file.
|
||||
Exit 1 = mismatch (or hub unreachable when --require-hub is set).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def brief_path(date: str) -> Path:
|
||||
year, month, _ = date.split("-")
|
||||
return ROOT / "briefs" / year / month / f"{date}.md"
|
||||
|
||||
|
||||
def load_events(url: str, timeout: float) -> list[dict]:
|
||||
req = urllib.request.Request(
|
||||
f"{url.rstrip('/')}/progress/?event_type=fi_daily_brief",
|
||||
headers={"Accept": "application/json"},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
data = json.loads(resp.read().decode())
|
||||
if isinstance(data, list):
|
||||
return data
|
||||
if isinstance(data, dict):
|
||||
for key in ("items", "events", "results"):
|
||||
if isinstance(data.get(key), list):
|
||||
return data[key]
|
||||
raise RuntimeError(f"unexpected hub payload type {type(data)}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument(
|
||||
"--hub",
|
||||
default=os.environ.get("STATE_HUB_URL", "http://127.0.0.1:8000"),
|
||||
)
|
||||
ap.add_argument("--timeout", type=float, default=8.0)
|
||||
ap.add_argument(
|
||||
"--require-hub",
|
||||
action="store_true",
|
||||
help="exit 1 if the hub cannot be reached (default: warn and check git only)",
|
||||
)
|
||||
args = ap.parse_args()
|
||||
|
||||
git_dates = sorted(
|
||||
p.stem
|
||||
for p in (ROOT / "briefs").rglob("20*.md")
|
||||
if p.name[0].isdigit()
|
||||
)
|
||||
print(f"git briefs: {len(git_dates)} last={git_dates[-1] if git_dates else '-'}")
|
||||
|
||||
try:
|
||||
events = load_events(args.hub, args.timeout)
|
||||
except (urllib.error.URLError, TimeoutError, json.JSONDecodeError, RuntimeError) as exc:
|
||||
msg = f"hub unreachable or unreadable at {args.hub}: {exc}"
|
||||
if args.require_hub:
|
||||
print(f"FAIL {msg}", file=sys.stderr)
|
||||
return 1
|
||||
print(f"WARN {msg}")
|
||||
return 0
|
||||
|
||||
missing: list[str] = []
|
||||
present: list[str] = []
|
||||
for ev in events:
|
||||
detail = ev.get("detail") or {}
|
||||
date = str(detail.get("date") or "")
|
||||
if not date:
|
||||
continue
|
||||
repo = detail.get("repo")
|
||||
if repo and repo != "freedom-intelligence":
|
||||
continue
|
||||
path = brief_path(date)
|
||||
if path.is_file():
|
||||
present.append(date)
|
||||
else:
|
||||
missing.append(date)
|
||||
|
||||
missing = sorted(set(missing))
|
||||
present = sorted(set(present))
|
||||
print(f"hub fi_daily_brief dates on git: {len(present)}")
|
||||
if missing:
|
||||
print("FAIL hub events with no brief file:")
|
||||
for date in missing:
|
||||
print(f" {date} expected {brief_path(date).relative_to(ROOT)}")
|
||||
print(
|
||||
"A local-only commit must not clear due. "
|
||||
"Push to origin, or treat the hub event as a false completion."
|
||||
)
|
||||
return 1
|
||||
print("ok: every hub fi_daily_brief date has a brief file")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
229
workplans/FI-WP-0004-operational-loop-and-scaleway-reserve.md
Normal file
229
workplans/FI-WP-0004-operational-loop-and-scaleway-reserve.md
Normal file
|
|
@ -0,0 +1,229 @@
|
|||
---
|
||||
id: FI-WP-0004
|
||||
type: workplan
|
||||
title: "Make the sensing loop durable and move the reserve to Scaleway"
|
||||
domain: agents
|
||||
repo: freedom-intelligence
|
||||
status: active
|
||||
owner: grok
|
||||
topic_slug: freedom-intelligence
|
||||
priority: high
|
||||
created: "2026-09-13"
|
||||
updated: "2026-09-13"
|
||||
depends_on:
|
||||
- FI-WP-0001
|
||||
- FI-WP-0002
|
||||
- FI-WP-0003
|
||||
---
|
||||
|
||||
# FI-WP-0004 — Operational loop + Scaleway reserve
|
||||
|
||||
## Goal
|
||||
|
||||
The lab already has INTENT, a brief template, a catalog, and a weekday
|
||||
schedule. It does **not** yet have a trusted daily field picture or a
|
||||
reserve that can hold the models INTENT says to keep.
|
||||
|
||||
This workplan closes that gap:
|
||||
|
||||
1. **Sensing loop is durable.** A brief day is complete only when the file
|
||||
exists on `origin/main`. Hub `fi_daily_brief` events that are not on
|
||||
git are failures, not completions.
|
||||
2. **Sensing quality is gated.** The executor must not re-announce cataloged
|
||||
models, invent sizes, or skip axes C/D by default.
|
||||
3. **Reserve uses Scaleway economics.** VAULT HD is local staging. The
|
||||
source of truth for weight blobs is a dedicated Scaleway bucket on
|
||||
`reef-storage` (One Zone IA for the R-spine, Glacier for S-tier).
|
||||
The 850 GiB NAS quota is retired as the capacity gate; euros/month
|
||||
become the gate.
|
||||
|
||||
## Why now
|
||||
|
||||
Review 2026-09-13: last brief on `origin/main` is 2026-08-14; hub events
|
||||
for 2026-09-07…11 claim `committed=True` with no origin objects. Automated
|
||||
briefs mis-sized DeepSeek-V4-Flash-0731 as a 12B dense model (it is a
|
||||
~304B / ~13B-active MIT MoE, ~167 GiB) and never cataloged it. Kimi K3
|
||||
was deferred only because a 1 TB disk quota could not hold ~1454 GiB;
|
||||
on Scaleway Glacier that footprint is a few euros per month.
|
||||
|
||||
Org storage selection is already Scaleway (`RESOURCE-WP-0002`,
|
||||
`reef-storage`). Do **not** reuse the postgres backup bucket (30-day
|
||||
expiry). FI needs its own long-retention bucket.
|
||||
|
||||
## Tasks
|
||||
|
||||
### T01 — Decision: Scaleway object reserve
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Write `docs/decisions/2026-09-13-scaleway-object-reserve.md`. Pin:
|
||||
|
||||
- SoT for blobs: Scaleway Object Storage `nl-ams` via `reef-storage`
|
||||
- R-spine: One Zone IA (immediate get)
|
||||
- S-tier: Glacier (restore is acceptable for just-in-case)
|
||||
- VAULT HD: staging / optional local R cache, not SoT
|
||||
- Soft euro budget (not GiB quota) as the capacity gate
|
||||
- Dedicated bucket, never `railiance-platform-pg-backup`
|
||||
|
||||
**Done when:** decision file exists and is linked from storage policy.
|
||||
|
||||
### T02 — Policy and operator docs
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Update `docs/backup-storage-policy.md`, `inventory/collection-policy.md`,
|
||||
`docs/hardware-envelope.md`, `README.md`, `SCOPE.md`,
|
||||
`docs/recurrence-ops.md` so they describe the live dual-loop contract
|
||||
and Scaleway pin. Retire “workplans still active” drift from README.
|
||||
|
||||
**Done when:** an operator can find the bucket contract, euro gate, and
|
||||
completion rule without reading INTENT.
|
||||
|
||||
### T03 — Brief durability contract + verifier
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T03
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
- Document: `fi_daily_brief` may fire only after the brief is on
|
||||
`origin/main`.
|
||||
- Add `scripts/verify_brief_durability.py` comparing hub events to git.
|
||||
- Tighten `activity-definitions/fi-daily-research-brief.md` and the
|
||||
playbook so a local-only commit does not clear `due`.
|
||||
|
||||
**Done when:** `python3 scripts/verify_brief_durability.py` fails on the
|
||||
known Sep 2026 hub/git mismatch (or reports it as debt until those
|
||||
events are reconciled).
|
||||
|
||||
### T04 — Executor origin publication (rein-aharness)
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T04
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
FI is the definition owner. Grant origin publication as a named
|
||||
capability of `fi-research-brief` (reverses the 2026-09-04 local-only
|
||||
commit):
|
||||
|
||||
- After commit, `git push origin HEAD:<branch>`
|
||||
- `fi_daily_brief` only if push succeeded (`detail.pushed=true`,
|
||||
`detail.origin_sha`)
|
||||
- Push failure → `executor_run` failure, `due` stays true
|
||||
- Prompt/context: catalog ids, no re-announce, no invented sizes
|
||||
|
||||
**Done when:** rein-aharness tests prove push is attempted and a failed
|
||||
push does not look like a completed brief day.
|
||||
|
||||
### T05 — Catch-up brief + catalog the missed SOTA
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T05
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Write `briefs/2026/09/2026-09-13.md` correcting V4-Flash and covering
|
||||
the Aug 15–Sep 12 gap. Catalog:
|
||||
|
||||
- `deepseek-ai/DeepSeek-V4-Flash-0731` as S (MIT, ~167 GiB) — **primary
|
||||
next pull** once the bucket exists
|
||||
- `Qwen/Qwen3.8-27B` as R/W candidate (Apache-2.0, dense ~27B)
|
||||
|
||||
Update `inventory/RESERVE-STATUS.md`.
|
||||
|
||||
**Done when:** catalog entries exist with correct size/class; catch-up
|
||||
brief cites primary cards.
|
||||
|
||||
### T06 — `collect_model.py` S3 backend
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T06
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Local snapshot still stages on disk. After MANIFEST, optional upload to
|
||||
the FI bucket with an explicit storage class. Credentials from env
|
||||
(`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` or `SCW_*`), never git.
|
||||
|
||||
**Done when:** `--s3-bucket` uploads and prints the object prefix.
|
||||
|
||||
### T07 — reef-storage bucket declaration (planned)
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T07
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Add `reef-storage/substrate/object-stores/fi-open-weight-reserve.yaml`
|
||||
(`status: planned`) and a create script that **must not** copy the
|
||||
postgres 30-day expiry. Bind freedom-intelligence as a potential
|
||||
consumer.
|
||||
|
||||
**Done when:** YAML validates against the reef-storage schema; create
|
||||
script is documented. Bucket create itself is T08 (operator).
|
||||
|
||||
### T08 — Operator: create bucket and scoped key
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T08
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Operator / railiance-platform:
|
||||
|
||||
1. Create private bucket `railiance-fi-open-weight-reserve` in `nl-ams`.
|
||||
2. Versioning on. **No 30-day object expiry.**
|
||||
3. Lifecycle: prefix `strategic/` → Glacier after 1 day.
|
||||
4. Scoped IAM key in OpenBao (not the postgres backup key).
|
||||
5. Flip the reef-storage YAML to `status: active`.
|
||||
6. Confirm a 1-byte PUT/GET.
|
||||
|
||||
**Done when:** attributes are live and a smoke object round-trips.
|
||||
|
||||
### T09 — First Scaleway collect (V4-Flash)
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T09
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
After T08: pull `deepseek-ai/DeepSeek-V4-Flash-0731` via
|
||||
`scripts/collect_model.py` into staging, upload as `strategic/` Glacier
|
||||
(or Standard then lifecycle). Verify MANIFEST vs objects. Update catalog
|
||||
`status: collected` / `verified` and RESERVE-STATUS.
|
||||
|
||||
**Done when:** catalog `storage_path` is the S3 prefix and checksums match.
|
||||
|
||||
## Acceptance (workplan-level)
|
||||
|
||||
- [x] Scaleway reserve decision written
|
||||
- [x] Docs no longer claim VAULT-as-SoT or “workplans still active”
|
||||
- [x] Durability verifier exists
|
||||
- [x] Executor publishes to origin or fails the day
|
||||
- [x] V4-Flash cataloged at the real size
|
||||
- [ ] Bucket live (T08, operator)
|
||||
- [ ] First S-tier object on Scaleway (T09)
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Reusing `railiance-platform-pg-backup`
|
||||
- 24/7 inference of the reserve
|
||||
- Changing Scaleway org billing (resource-control / founder alert)
|
||||
- Recovering the unpushed 2026-09-07…11 railiance-local commits
|
||||
(note in the catch-up brief; do not invent those files)
|
||||
Loading…
Add table
Add a link
Reference in a new issue