coulomb/freedom-intelligence
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Stream model reserve to Scaleway without local weight staging

Browse source 
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a09cbd-43c1-79f3-809e-1ee97b40b64d
This commit is contained in:
tegwick 2026-09-14 00:17:15 +02:00
parent ad88d16a52
commit 8a65297d56
16 changed files with 805 additions and 145 deletions
Download patch file Download diff file Expand all files Collapse all files

21
inventory/RESERVE-STATUS.md
View file

@ -1,9 +1,9 @@
# Open-weight reserve status
**As of:** 2026-09-13
**As of:** 2026-09-14
**SoT store:** Scaleway Object Storage `nl-ams` bucket
`railiance-fi-open-weight-reserve` (**planned** — FI-WP-0004-T08)
**Staging:** `/mnt/d/vault/coulomb/freedom-intelligence/` (VAULT HD)
`railiance-fi-open-weight-reserve` (**live** — FI-WP-0004-T08)
**Transfer:** diskless streaming on Railiance; [operations](../docs/streaming-reserve.md)
**Capacity gate:** **€15 / month** soft (not 850 GiB)
---
@ -14,7 +14,8 @@
| ----- | ----: | ---------- |
| Catalog entries | 15 | + V4-Flash S, + Qwen3.8-27B W/R |
| **verified** on VAULT | 4 | R embeds + Qwen3-8B + R1-Distill-14B |
| **approved**, blobs missing | S giants + Llama-3.2-3B + **V4-Flash** | HF gate / bucket not live |
| **approved**, blobs missing | S giants + Llama-3.2-3B | HF gate / deferred |
| **collecting** | 1 | V4-Flash → Scaleway Glacier, remote stream |
| **candidate** | 4 | W + Qwen3.8-27B |
### Disk / object use
@ -25,7 +26,7 @@
| VAULT `models/BAAI__bge-m3/main` | R embed (staging cache) | ~2.1 GiB verified |
| VAULT `models/Qwen__Qwen3-8B/main` | R instruct (staging cache) | ~15.3 GiB verified |
| VAULT `models/deepseek-ai__DeepSeek-R1-Distill-Qwen-14B/main` | R reason (staging cache) | ~27.5 GiB verified |
| Scaleway bucket | SoT | **empty** (not created) |
| Scaleway bucket | SoT | V4-Flash streaming; final manifest pending |
VAULT ~45 GiB used remains a **cache**, not the reserve.
@ -40,7 +41,7 @@ VAULT ~45 GiB used remains a **cache**, not the reserve.
| BGE-M3 | R | **verified** on VAULT cache | local only |
| R1-Distill-Qwen-14B | R | **verified** on VAULT cache | local only |
| nomic-embed-text-v1.5 | R | **verified** on VAULT cache | local only |
| **DeepSeek-V4-Flash-0731** | **S1** | **approved** — **first Scaleway pull** | no — ~167 GiB Glacier |
| **DeepSeek-V4-Flash-0731** | **S1** | **collecting** — **first Scaleway pull** | partial — 166.9 GB / 155.4 GiB source, Glacier |
| Kimi-K3 | S1 secondary | **approved**, deferred until after V4-Flash | no — ~1454 GiB Glacier ≈ €3.69/mo |
| DeepSeek-V3 | S superseded identity | **approved**, deferred | no — V4-Flash is the collectable S1 |
| DeepSeek-R1 full | S2 | **approved**, deferred | no |
@ -65,17 +66,17 @@ that used that id were wrong. Do not collect under that name.
| Edge micro-agent | Llama-3.2-3B blocked on HF auth |
| Multilingual RAG embed | **Covered** (BGE-M3 + nomic) |
| Local reason | **Covered** (R1-Distill-14B) |
| Frontier-open MIT MoE offline | **Cataloged** V4-Flash — pull blocked on bucket |
| Frontier-open MIT MoE offline | **Collecting** V4-Flash — remote diskless stream |
| Full open giant (K3) | Cataloged; Glacier-affordable; after V4-Flash |
---
## Next pulls (operator)
1. **FI-WP-0004-T08** create `railiance-fi-open-weight-reserve` (no 30-day expiry).
2. **FI-WP-0004-T09** collect `deepseek-ai/DeepSeek-V4-Flash-0731` → `strategic/` Glacier.
1. **FI-WP-0004-T08** bucket live; dedicated IAM key remains outstanding.
2. **FI-WP-0004-T09** monitor the active remote stream; require the complete manifest before marking collected.
3. Set `HF_TOKEN` and pull Llama-3.2-3B-Instruct into `models/` One Zone IA.
4. Decide Qwen3.8-27B vs keeping Qwen3-8B as the R instruct default.
5. Optional: Kimi K3 on Glacier if the month is still under €15.
Tool: `scripts/collect_model.py` (weights-only, sequential, MANIFEST, optional `--s3-bucket`).
Tool: `scripts/collect_model.py` (`--s3-bucket` streams without disk staging; explicit `--local-download` for cache only).