Stream model reserve to Scaleway without local weight staging
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a09cbd-43c1-79f3-809e-1ee97b40b64d
This commit is contained in:
parent
ad88d16a52
commit
8a65297d56
16 changed files with 805 additions and 145 deletions
|
|
@ -51,13 +51,10 @@ weights belong in the FI object bucket.
|
|||
| **Attributes** | `reef:storage/substrate/object-stores/fi-open-weight-reserve.yaml` |
|
||||
| **Credentials** | OpenBao / `railiance-platform` — never git |
|
||||
|
||||
**Local staging (not SoT):** workstation VAULT HD
|
||||
`D:\vault\coulomb\freedom-intelligence\` /
|
||||
`/mnt/d/vault/coulomb/freedom-intelligence/` for `huggingface_hub`
|
||||
downloads and an optional R-spine cache.
|
||||
|
||||
Do **not** bulk-download multi-GB models into this git workspace or into
|
||||
hot root filesystems (`/` on WSL, `C:`, railiance PVCs).
|
||||
**Bulk transfer:** [diskless streaming on Railiance](streaming-reserve.md),
|
||||
with 64 MiB parts and a 512 MiB process memory limit. Weight files are never
|
||||
staged on WSL or on the remote host's root disk. VAULT is an optional explicit
|
||||
R-spine cache only (`--local-download`), not an intermediate reserve step.
|
||||
|
||||
### Prefix shape (pinned)
|
||||
|
||||
|
|
@ -106,13 +103,14 @@ For every completed collection:
|
|||
|
||||
1. Record source URL and revision in the catalog entry.
|
||||
2. Store checksums (`sha256` of each blob or upstream manifest digest)
|
||||
in `MANIFEST.json` (local staging **and** `manifests/` prefix).
|
||||
in `MANIFEST.json` under the STANDARD `manifests/` prefix.
|
||||
3. Record download date (UTC) and downloader identity.
|
||||
4. Prefer official org releases over anonymous re-uploads.
|
||||
5. Keep license text or SPDX id in catalog; refuse unclear licenses.
|
||||
|
||||
Verification: **catalog claims must match object checksums** before
|
||||
status `collected`.
|
||||
Verification: source hashes, explicit part/composite MD5 ETags and remote object identities
|
||||
must match before `collected`. `verified` requires restore/readback checksums;
|
||||
Glacier HEAD alone is insufficient. See the streaming transfer contract.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue