Make the sensing loop durable and pin the reserve to Scaleway
The weekday brief only counts when the file is on origin/main. Hub events without git objects are failures. The open-weight SoT moves from the VAULT 850 GiB quota to a dedicated Scaleway bucket (One Zone IA for R, Glacier for S) so V4-Flash and K3 are economically in-scope. Catalogs the real DeepSeek-V4-Flash-0731 (MIT ~167 GiB MoE, not 12B) and nominates Qwen3.8-27B. Bucket create remains FI-WP-0004-T08. Assistant: grok Assistant-Session: 01a09c6a-1cfc-75b1-a78d-c13eaf22241d
This commit is contained in:
parent
8832652ad3
commit
a78187c33e
18 changed files with 1125 additions and 226 deletions
229
workplans/FI-WP-0004-operational-loop-and-scaleway-reserve.md
Normal file
229
workplans/FI-WP-0004-operational-loop-and-scaleway-reserve.md
Normal file
|
|
@ -0,0 +1,229 @@
|
|||
---
|
||||
id: FI-WP-0004
|
||||
type: workplan
|
||||
title: "Make the sensing loop durable and move the reserve to Scaleway"
|
||||
domain: agents
|
||||
repo: freedom-intelligence
|
||||
status: active
|
||||
owner: grok
|
||||
topic_slug: freedom-intelligence
|
||||
priority: high
|
||||
created: "2026-09-13"
|
||||
updated: "2026-09-13"
|
||||
depends_on:
|
||||
- FI-WP-0001
|
||||
- FI-WP-0002
|
||||
- FI-WP-0003
|
||||
---
|
||||
|
||||
# FI-WP-0004 — Operational loop + Scaleway reserve
|
||||
|
||||
## Goal
|
||||
|
||||
The lab already has INTENT, a brief template, a catalog, and a weekday
|
||||
schedule. It does **not** yet have a trusted daily field picture or a
|
||||
reserve that can hold the models INTENT says to keep.
|
||||
|
||||
This workplan closes that gap:
|
||||
|
||||
1. **Sensing loop is durable.** A brief day is complete only when the file
|
||||
exists on `origin/main`. Hub `fi_daily_brief` events that are not on
|
||||
git are failures, not completions.
|
||||
2. **Sensing quality is gated.** The executor must not re-announce cataloged
|
||||
models, invent sizes, or skip axes C/D by default.
|
||||
3. **Reserve uses Scaleway economics.** VAULT HD is local staging. The
|
||||
source of truth for weight blobs is a dedicated Scaleway bucket on
|
||||
`reef-storage` (One Zone IA for the R-spine, Glacier for S-tier).
|
||||
The 850 GiB NAS quota is retired as the capacity gate; euros/month
|
||||
become the gate.
|
||||
|
||||
## Why now
|
||||
|
||||
Review 2026-09-13: last brief on `origin/main` is 2026-08-14; hub events
|
||||
for 2026-09-07…11 claim `committed=True` with no origin objects. Automated
|
||||
briefs mis-sized DeepSeek-V4-Flash-0731 as a 12B dense model (it is a
|
||||
~304B / ~13B-active MIT MoE, ~167 GiB) and never cataloged it. Kimi K3
|
||||
was deferred only because a 1 TB disk quota could not hold ~1454 GiB;
|
||||
on Scaleway Glacier that footprint is a few euros per month.
|
||||
|
||||
Org storage selection is already Scaleway (`RESOURCE-WP-0002`,
|
||||
`reef-storage`). Do **not** reuse the postgres backup bucket (30-day
|
||||
expiry). FI needs its own long-retention bucket.
|
||||
|
||||
## Tasks
|
||||
|
||||
### T01 — Decision: Scaleway object reserve
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Write `docs/decisions/2026-09-13-scaleway-object-reserve.md`. Pin:
|
||||
|
||||
- SoT for blobs: Scaleway Object Storage `nl-ams` via `reef-storage`
|
||||
- R-spine: One Zone IA (immediate get)
|
||||
- S-tier: Glacier (restore is acceptable for just-in-case)
|
||||
- VAULT HD: staging / optional local R cache, not SoT
|
||||
- Soft euro budget (not GiB quota) as the capacity gate
|
||||
- Dedicated bucket, never `railiance-platform-pg-backup`
|
||||
|
||||
**Done when:** decision file exists and is linked from storage policy.
|
||||
|
||||
### T02 — Policy and operator docs
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Update `docs/backup-storage-policy.md`, `inventory/collection-policy.md`,
|
||||
`docs/hardware-envelope.md`, `README.md`, `SCOPE.md`,
|
||||
`docs/recurrence-ops.md` so they describe the live dual-loop contract
|
||||
and Scaleway pin. Retire “workplans still active” drift from README.
|
||||
|
||||
**Done when:** an operator can find the bucket contract, euro gate, and
|
||||
completion rule without reading INTENT.
|
||||
|
||||
### T03 — Brief durability contract + verifier
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T03
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
- Document: `fi_daily_brief` may fire only after the brief is on
|
||||
`origin/main`.
|
||||
- Add `scripts/verify_brief_durability.py` comparing hub events to git.
|
||||
- Tighten `activity-definitions/fi-daily-research-brief.md` and the
|
||||
playbook so a local-only commit does not clear `due`.
|
||||
|
||||
**Done when:** `python3 scripts/verify_brief_durability.py` fails on the
|
||||
known Sep 2026 hub/git mismatch (or reports it as debt until those
|
||||
events are reconciled).
|
||||
|
||||
### T04 — Executor origin publication (rein-aharness)
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T04
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
FI is the definition owner. Grant origin publication as a named
|
||||
capability of `fi-research-brief` (reverses the 2026-09-04 local-only
|
||||
commit):
|
||||
|
||||
- After commit, `git push origin HEAD:<branch>`
|
||||
- `fi_daily_brief` only if push succeeded (`detail.pushed=true`,
|
||||
`detail.origin_sha`)
|
||||
- Push failure → `executor_run` failure, `due` stays true
|
||||
- Prompt/context: catalog ids, no re-announce, no invented sizes
|
||||
|
||||
**Done when:** rein-aharness tests prove push is attempted and a failed
|
||||
push does not look like a completed brief day.
|
||||
|
||||
### T05 — Catch-up brief + catalog the missed SOTA
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T05
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Write `briefs/2026/09/2026-09-13.md` correcting V4-Flash and covering
|
||||
the Aug 15–Sep 12 gap. Catalog:
|
||||
|
||||
- `deepseek-ai/DeepSeek-V4-Flash-0731` as S (MIT, ~167 GiB) — **primary
|
||||
next pull** once the bucket exists
|
||||
- `Qwen/Qwen3.8-27B` as R/W candidate (Apache-2.0, dense ~27B)
|
||||
|
||||
Update `inventory/RESERVE-STATUS.md`.
|
||||
|
||||
**Done when:** catalog entries exist with correct size/class; catch-up
|
||||
brief cites primary cards.
|
||||
|
||||
### T06 — `collect_model.py` S3 backend
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T06
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Local snapshot still stages on disk. After MANIFEST, optional upload to
|
||||
the FI bucket with an explicit storage class. Credentials from env
|
||||
(`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` or `SCW_*`), never git.
|
||||
|
||||
**Done when:** `--s3-bucket` uploads and prints the object prefix.
|
||||
|
||||
### T07 — reef-storage bucket declaration (planned)
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T07
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Add `reef-storage/substrate/object-stores/fi-open-weight-reserve.yaml`
|
||||
(`status: planned`) and a create script that **must not** copy the
|
||||
postgres 30-day expiry. Bind freedom-intelligence as a potential
|
||||
consumer.
|
||||
|
||||
**Done when:** YAML validates against the reef-storage schema; create
|
||||
script is documented. Bucket create itself is T08 (operator).
|
||||
|
||||
### T08 — Operator: create bucket and scoped key
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T08
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Operator / railiance-platform:
|
||||
|
||||
1. Create private bucket `railiance-fi-open-weight-reserve` in `nl-ams`.
|
||||
2. Versioning on. **No 30-day object expiry.**
|
||||
3. Lifecycle: prefix `strategic/` → Glacier after 1 day.
|
||||
4. Scoped IAM key in OpenBao (not the postgres backup key).
|
||||
5. Flip the reef-storage YAML to `status: active`.
|
||||
6. Confirm a 1-byte PUT/GET.
|
||||
|
||||
**Done when:** attributes are live and a smoke object round-trips.
|
||||
|
||||
### T09 — First Scaleway collect (V4-Flash)
|
||||
|
||||
```task
|
||||
id: FI-WP-0004-T09
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
After T08: pull `deepseek-ai/DeepSeek-V4-Flash-0731` via
|
||||
`scripts/collect_model.py` into staging, upload as `strategic/` Glacier
|
||||
(or Standard then lifecycle). Verify MANIFEST vs objects. Update catalog
|
||||
`status: collected` / `verified` and RESERVE-STATUS.
|
||||
|
||||
**Done when:** catalog `storage_path` is the S3 prefix and checksums match.
|
||||
|
||||
## Acceptance (workplan-level)
|
||||
|
||||
- [x] Scaleway reserve decision written
|
||||
- [x] Docs no longer claim VAULT-as-SoT or “workplans still active”
|
||||
- [x] Durability verifier exists
|
||||
- [x] Executor publishes to origin or fails the day
|
||||
- [x] V4-Flash cataloged at the real size
|
||||
- [ ] Bucket live (T08, operator)
|
||||
- [ ] First S-tier object on Scaleway (T09)
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Reusing `railiance-platform-pg-backup`
|
||||
- 24/7 inference of the reserve
|
||||
- Changing Scaleway org billing (resource-control / founder alert)
|
||||
- Recovering the unpushed 2026-09-07…11 railiance-local commits
|
||||
(note in the catch-up brief; do not invent those files)
|
||||
Loading…
Add table
Add a link
Reference in a new issue