--- id: FI-WP-0004 type: workplan title: "Make the sensing loop durable and move the reserve to Scaleway" domain: agents repo: freedom-intelligence status: finished flavor: implementation owner: grok topic_slug: freedom-intelligence priority: high created: "2026-09-13" updated: "2026-09-22" depends_on: - FI-WP-0001 - FI-WP-0002 - FI-WP-0003 state_hub_workstream_id: "2ca633f0-2938-5f1c-85a2-827ce8014575" --- # FI-WP-0004 — Operational loop + Scaleway reserve ## Goal The lab already has INTENT, a brief template, a catalog, and a weekday schedule. It does **not** yet have a trusted daily field picture or a reserve that can hold the models INTENT says to keep. This workplan closes that gap: 1. **Sensing loop is durable.** A brief day is complete only when the file exists on `origin/main`. Hub `fi_daily_brief` events that are not on git are failures, not completions. 2. **Sensing quality is gated.** The executor must not re-announce cataloged models, invent sizes, or skip axes C/D by default. 3. **Reserve uses Scaleway economics.** VAULT HD is local staging. The source of truth for weight blobs is a dedicated Scaleway bucket on `reef-storage` (One Zone IA for the R-spine, Glacier for S-tier). The 850 GiB NAS quota is retired as the capacity gate; euros/month become the gate. ## Why now Review 2026-09-13: last brief on `origin/main` is 2026-08-14; hub events for 2026-09-07…11 claim `committed=True` with no origin objects. Automated briefs mis-sized DeepSeek-V4-Flash-0731 as a 12B dense model (it is a ~304B / ~13B-active MIT MoE, ~167 GiB) and never cataloged it. Kimi K3 was deferred only because a 1 TB disk quota could not hold ~1454 GiB; on Scaleway Glacier that footprint is a few euros per month. Org storage selection is already Scaleway (`RESOURCE-WP-0002`, `reef-storage`). Do **not** reuse the postgres backup bucket (30-day expiry). FI needs its own long-retention bucket. ## Tasks ### T01 — Decision: Scaleway object reserve ```task id: FI-WP-0004-T01 status: done priority: high state_hub_task_id: "9d8d067d-aac9-5f6e-acb9-8ec89a764ccf" ``` Write `docs/decisions/2026-09-13-scaleway-object-reserve.md`. Pin: - SoT for blobs: Scaleway Object Storage `nl-ams` via `reef-storage` - R-spine: One Zone IA (immediate get) - S-tier: Glacier (restore is acceptable for just-in-case) - VAULT HD: staging / optional local R cache, not SoT - Soft euro budget (not GiB quota) as the capacity gate - Dedicated bucket, never `railiance-platform-pg-backup` **Done when:** decision file exists and is linked from storage policy. ### T02 — Policy and operator docs ```task id: FI-WP-0004-T02 status: done priority: high state_hub_task_id: "4b0a0381-0f3c-5236-abc8-f83a48dc6335" ``` Update `docs/backup-storage-policy.md`, `inventory/collection-policy.md`, `docs/hardware-envelope.md`, `README.md`, `SCOPE.md`, `docs/recurrence-ops.md` so they describe the live dual-loop contract and Scaleway pin. Retire “workplans still active” drift from README. **Done when:** an operator can find the bucket contract, euro gate, and completion rule without reading INTENT. ### T03 — Brief durability contract + verifier ```task id: FI-WP-0004-T03 status: done priority: high state_hub_task_id: "e4fbc634-462c-5f07-8a27-ed6ad7315bfe" ``` - Document: `fi_daily_brief` may fire only after the brief is on `origin/main`. - Add `scripts/verify_brief_durability.py` comparing hub events to git. - Tighten `activity-definitions/fi-daily-research-brief.md` and the playbook so a local-only commit does not clear `due`. **2026-09-14 follow-up:** verifier now resolves live origin/main, reads Git objects, paginates Hub events, and validates pushed/SHA evidence. Exact-event corrections preserve the five missing dates as explicit historical gaps. `--require-date` proves a specific scheduled day; see `docs/brief-durability-audit.md`. **Done when:** `python3 scripts/verify_brief_durability.py` fails on the known Sep 2026 hub/git mismatch (or reports it as debt until those events are reconciled). ### T04 — Executor origin publication (rein-aharness) ```task id: FI-WP-0004-T04 status: done priority: high state_hub_task_id: "f0a8e3e7-d1e9-5c54-966e-abd87effec00" ``` FI is the definition owner. Grant origin publication as a named capability of `fi-research-brief` (reverses the 2026-09-04 local-only commit): - After commit, `git push origin HEAD:` - `fi_daily_brief` only if push succeeded (`detail.pushed=true`, `detail.origin_sha`) - Push failure → `executor_run` failure, `due` stays true - Prompt/context: catalog ids, no re-announce, no invented sizes **Done when:** rein-aharness tests prove push is attempted and a failed push does not look like a completed brief day. ### T05 — Catch-up brief + catalog the missed SOTA ```task id: FI-WP-0004-T05 status: done priority: high state_hub_task_id: "10d027ad-b32e-5ae5-aa6b-5f81a3dbba59" ``` Write `briefs/2026/09/2026-09-13.md` correcting V4-Flash and covering the Aug 15–Sep 12 gap. Catalog: - `deepseek-ai/DeepSeek-V4-Flash-0731` as S (MIT, ~167 GiB) — **primary next pull** once the bucket exists - `Qwen/Qwen3.8-27B` as R/W candidate (Apache-2.0, dense ~27B) Update `inventory/RESERVE-STATUS.md`. **Done when:** catalog entries exist with correct size/class; catch-up brief cites primary cards. ### T06 — `collect_model.py` S3 backend ```task id: FI-WP-0004-T06 status: done priority: medium state_hub_task_id: "05c655a3-3196-5bff-bcc5-11e14e9fc262" ``` Local snapshot still stages on disk. After MANIFEST, optional upload to the FI bucket with an explicit storage class. Credentials from env (`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` or `SCW_*`), never git. **Done when:** `--s3-bucket` uploads and prints the object prefix. ### T07 — reef-storage bucket declaration (planned) ```task id: FI-WP-0004-T07 status: done priority: high state_hub_task_id: "db7e6da1-b261-500e-a55c-ae0c5a4693f1" ``` Add `reef-storage/substrate/object-stores/fi-open-weight-reserve.yaml` (`status: planned`) and a create script that **must not** copy the postgres 30-day expiry. Bind freedom-intelligence as a potential consumer. **Done when:** YAML validates against the reef-storage schema; create script is documented. Bucket create itself is T08 (operator). ### T08 — Operator: create bucket and scoped key ```task id: FI-WP-0004-T08 status: done priority: high state_hub_task_id: "a824a9a1-9cb2-5fbc-9428-708f98392ad6" ``` **2026-09-13:** Bucket `railiance-fi-open-weight-reserve` exists in `nl-ams`, versioning on, no object expiry, `strategic/` → Glacier at 90 days (provider minimum). STANDARD PUT/GET smoked; Glacier PUT smoked (GET needs restore). YAML `status: active`. Residual: bucket-scoped IAM key — bootstrap cannot `iam application list`; collect loads bootstrap from OpenBao at runtime until a founder key is stored at `platform/workloads/railiance/freedom-intelligence/object-storage`. Operator / railiance-platform: 1. Create private bucket `railiance-fi-open-weight-reserve` in `nl-ams`. 2. Versioning on. **No 30-day object expiry.** 3. Lifecycle: prefix `strategic/` → Glacier after 90 days (Scaleway minimum). S-tier PUT still uses `StorageClass=GLACIER` immediately. 4. Scoped IAM key in OpenBao (not the postgres backup key). 5. Flip the reef-storage YAML to `status: active`. 6. Confirm a 1-byte PUT/GET. **Done when:** attributes are live and a smoke object round-trips. ### T09 — First Scaleway collect (V4-Flash) ```task id: FI-WP-0004-T09 status: done priority: medium state_hub_task_id: "304816de-5065-5928-b9bd-da1b5019c3d4" ``` **2026-09-14:** The old WSL download is stopped/inactive. Collection now runs as `fi-reserve-collect.service` on Railiance using diskless streaming (64 MiB parts, 512 MiB memory cap, zero swap). Source pinned to `7872f01b1d1fe23eabc4c98b48bffcef5a386062`; 67 files / 166,898,547,054 bytes. Live multipart tests passed for STANDARD, ONEZONE_IA and GLACIER. Provider ignored wrong Content-MD5, so explicit part/composite ETag validation is used. See `docs/streaming-reserve.md`. Transfer completion and final manifest remain pending. Update catalog `collected` only after the manifest is complete; `verified` requires restore/readback. **2026-09-22:** Done. The service finished 2026-09-13T23:18:26Z and published a complete MANIFEST.json. Re-check: 67/67 objects, 166,898,547,054 bytes, all GLACIER, no open multipart uploads. The 48 LFS SHA256 source digests match the computed SHA256, and HEAD size, ETag and version match the manifest for every object. The catalog is `collected` with the S3 prefix as `storage_path`. Restore readback (`verified`) is a follow-up outside this task. **Done when:** catalog `storage_path` is the S3 prefix and checksums match. ## Acceptance (workplan-level) - [x] Scaleway reserve decision written - [x] Docs no longer claim VAULT-as-SoT or “workplans still active” - [x] Durability verifier exists - [x] Executor publishes to origin or fails the day - [x] V4-Flash cataloged at the real size - [x] Bucket live (T08) - [x] First S-tier object on Scaleway (T09) ## Out of scope - Reusing `railiance-platform-pg-backup` - 24/7 inference of the reserve - Changing Scaleway org billing (resource-control / founder alert) - Recovering the unpushed 2026-09-07…11 railiance-local commits (note in the catch-up brief; do not invent those files)