From 654bbe891bef812ee3bfcec921c8612697606410 Mon Sep 17 00:00:00 2001 From: codex Date: Fri, 14 Aug 2026 15:53:30 +0200 Subject: [PATCH] feat: allow provider-delegated reefs with no rail Object-storage reefs omit primary_rail and hosted_rails. Scaleway operates S3; reef-storage only names the boundary. Finish RMASTER-WP-0022. --- WORK-RECORDS.md | 8 +++--- docs/reef-substrate-model.md | 28 ++++++++++++------- docs/repo-family-bootstrap-contract.md | 7 +++-- schemas/README.md | 8 ++++-- schemas/reef.schema.json | 14 ++++------ .../declarations/reef.yaml | 13 +++++++++ tools/validate-family-declarations.py | 2 +- .../RMASTER-WP-0022-establish-reef-storage.md | 19 ++++++++++--- 8 files changed, 67 insertions(+), 32 deletions(-) create mode 100644 tools/testdata/family-declarations/good/reef-storage-example/declarations/reef.yaml diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 719e557..2f5fd15 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -14,7 +14,7 @@ | workplan | RMASTER-WP-0019 | finished | — | workplans/RMASTER-WP-0019-knative-qonto-runtime-on-reef-railiance.md | | workplan | RMASTER-WP-0020 | backlog | — | workplans/RMASTER-WP-0020-openbao-migration-to-reef-railiance.md | | workplan | RMASTER-WP-0021 | finished | — | workplans/RMASTER-WP-0021-rapp-declaration-schema.md | -| workplan | RMASTER-WP-0022 | proposed | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | +| workplan | RMASTER-WP-0022 | finished | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | | task | ADHOC-2026-07-30-T01 | done | — | workplans/ADHOC-2026-07-30.md | | task | RMASTER-WP-0017-T01 | done | — | workplans/RMASTER-WP-0017-rail-rapp-reef-repo-separation.md | | task | RMASTER-WP-0017-T02 | done | — | workplans/RMASTER-WP-0017-rail-rapp-reef-repo-separation.md | @@ -56,6 +56,6 @@ | task | RMASTER-WP-0021-T07 | done | — | workplans/RMASTER-WP-0021-rapp-declaration-schema.md | | task | RMASTER-WP-0021-T08 | done | — | workplans/RMASTER-WP-0021-rapp-declaration-schema.md | | task | RMASTER-WP-0021-T09 | done | — | workplans/RMASTER-WP-0021-rapp-declaration-schema.md | -| task | RMASTER-WP-0022-T01 | todo | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | -| task | RMASTER-WP-0022-T02 | todo | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | -| task | RMASTER-WP-0022-T03 | todo | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | +| task | RMASTER-WP-0022-T01 | done | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | +| task | RMASTER-WP-0022-T02 | done | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | +| task | RMASTER-WP-0022-T03 | done | — | workplans/RMASTER-WP-0022-establish-reef-storage.md | diff --git a/docs/reef-substrate-model.md b/docs/reef-substrate-model.md index fc93c38..996793d 100644 --- a/docs/reef-substrate-model.md +++ b/docs/reef-substrate-model.md @@ -10,12 +10,16 @@ boundary. A reef is the place where: - infrastructure becomes a named operational reality, -- rails are installed or made available, -- managed workloads are bound, +- rails may be installed or made available, +- managed workloads may be bound, +- a provider-delegated capability may be named as a substrate, - and operators interact with a concrete environment. -More concretely, a reef represents **compute resources organized for a defined -purpose**. +A reef is **one durable operational boundary organized for a defined +purpose**. That purpose is often compute. It can also be a +provider-operated capability we consume as a unit — for example S3 +object storage at Scaleway (`reef-storage`). In that case there is **no +rail**: we do not take care of how the capability is provided. The substrate may be: @@ -24,10 +28,11 @@ The substrate may be: - a workstation, - an edge site, - a lab substrate, -- or a small fleet treated as one unit. +- a small fleet treated as one unit, +- or a provider-delegated service boundary (object storage, later others). The important point is not the shape of the hardware. The important point is -that the compute resources form one recognizable operational boundary. +that the boundary is one recognizable operational reality. ## Why `reef-*` Instead Of `host-*` @@ -41,8 +46,9 @@ Railiance substrates are not always just hosts. They may also be: - grouped node fleets, - or mixed environments with both machines and control surfaces. -`reef-*` is useful because it names the substrate reality that rails attach to, -not merely the hardware object underneath it. +`reef-*` is useful because it names the substrate reality — the place +rails may attach, or the place a delegated capability is consumed — not +merely the hardware object underneath it. ## Core Responsibility Of A Reef @@ -51,8 +57,10 @@ A reef repo should answer: - What is this substrate called? - What components belong to it? - What access paths and operator assumptions apply? -- Which rails exist here? -- Which `rapp`s are allowed or deployed here? +- Which rails exist here, if any? +- Which `rapp`s are allowed or deployed here, if any? +- For a provider-delegated reef: which provider, region, and non-secret + attributes (endpoint, bucket, …) define the boundary? - Which overlays, exceptions, and evidence are specific to this substrate? It should also make the substrate purpose explicit, so the existence of the diff --git a/docs/repo-family-bootstrap-contract.md b/docs/repo-family-bootstrap-contract.md index d387213..5276993 100644 --- a/docs/repo-family-bootstrap-contract.md +++ b/docs/repo-family-bootstrap-contract.md @@ -110,8 +110,11 @@ Schema: [`schemas/reef.schema.json`](../schemas/reef.schema.json) `bound_rapps` is a derived projection of every rapp whose `bound_reefs` lists this reef. It is not a hand-maintained registry. A reef may omit the field; if it keeps the field, the validator requires an exact match against -the projection. `primary_rail` must appear in `hosted_rails`. Hosting a rail -or binding a rapp is topology, not readiness — see ADR-0006. +the projection. A compute reef that hosts rails must set `primary_rail` +and list it in `hosted_rails`. A provider-delegated reef (for example +`reef-storage`, `substrate_kind: object-storage`) omits both: there is no +execution contract to host. Hosting a rail or binding a rapp is topology, +not readiness — see ADR-0006. ## Required Boundary Rules diff --git a/schemas/README.md b/schemas/README.md index 7e362fb..01840b1 100644 --- a/schemas/README.md +++ b/schemas/README.md @@ -93,7 +93,7 @@ owner. Until it is made, treat the permissive rule as provisional. JSON Schema cannot express these; `tools/validate-family-declarations.py` does: -- `primary_rail` must appear in `supported_rails` (rapp) or `hosted_rails` (reef) +- `primary_rail` must appear in `supported_rails` (rapp) or, when set, in `hosted_rails` (reef). A reef may omit both when it hosts no rail. - reef `bound_rapps`, if present, must match the projection of `rapp.bound_reefs` - a rapp `ownership_repo` must not be the rapp repo itself - declared member repos, named rails, and named reefs must resolve on disk @@ -128,8 +128,10 @@ conform. That is the intended contrast with the rapp family. normative change: `bound_rapps` is no longer a hand-maintained required field. Required on every reef: `kind`, `reef_id`, `repo`, `ownership_repo`, -`substrate_kind`, `lifecycle_state`, `criticality`, `primary_rail`, -`hosted_rails`, `current_members`. +`substrate_kind`, `lifecycle_state`, `criticality`, `current_members`. +`primary_rail` and `hosted_rails` are required only when the reef hosts +rails. `current_members` may be empty on a planned provider-delegated +reef. `bound_rapps` is optional. Its source of truth is `rapp.bound_reefs`. The validator inverts that field. If a reef still lists `bound_rapps`, the list diff --git a/schemas/reef.schema.json b/schemas/reef.schema.json index a667250..d7dabfd 100644 --- a/schemas/reef.schema.json +++ b/schemas/reef.schema.json @@ -13,8 +13,6 @@ "substrate_kind", "lifecycle_state", "criticality", - "primary_rail", - "hosted_rails", "current_members" ], "properties": { @@ -38,7 +36,7 @@ "type": "string", "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$", "minLength": 3, - "description": "What kind of substrate this reef is. Live value is server-group. Kept as a slug rather than a closed enum so a new substrate shape does not require a schema change. See docs/reef-substrate-model.md." + "description": "What kind of substrate this reef is. Known values include server-group (reef-railiance) and object-storage (reef-storage). Kept as a slug rather than a closed enum. A provider-delegated kind (object-storage) does not host a rail: the provider operates the capability. See docs/reef-substrate-model.md." }, "lifecycle_state": { "enum": ["planned", "active", "retired"], @@ -50,14 +48,14 @@ }, "primary_rail": { "$ref": "#/$defs/railSlug", - "description": "The default rail on this reef. Must appear in hosted_rails. The validator enforces that sibling check." + "description": "The default rail on this reef, when the reef hosts rails. Omit on a provider-delegated substrate (object-storage): there is no execution contract to host. If present, must also appear in hosted_rails." }, "hosted_rails": { "type": "array", - "minItems": 1, + "minItems": 0, "uniqueItems": true, "items": { "$ref": "#/$defs/railSlug" }, - "description": "Rails this reef hosts or is prepared to host. Topology, not readiness: hosting a rail does not make it production-approved. See ADR-0006." + "description": "Rails this reef hosts or is prepared to host. Empty or omitted when the reef does not host a rail. Topology, not readiness. See ADR-0006." }, "bound_rapps": { "type": "array", @@ -67,10 +65,10 @@ }, "current_members": { "type": "array", - "minItems": 1, + "minItems": 0, "uniqueItems": true, "items": { "type": "string", "minLength": 1 }, - "description": "Named machines or units that currently compose this reef. Not slugs: live value is Railiance01. A reef is a substrate boundary, not a hostname list, but the members must be named so the boundary is inspectable." + "description": "Named machines, buckets, or other units that currently compose this reef. Empty is allowed while a planned provider-delegated reef has no purchased members yet. Live compute example: Railiance01." }, "compatibility_notes": { "$ref": "#/$defs/notes", diff --git a/tools/testdata/family-declarations/good/reef-storage-example/declarations/reef.yaml b/tools/testdata/family-declarations/good/reef-storage-example/declarations/reef.yaml new file mode 100644 index 0000000..58ceb4d --- /dev/null +++ b/tools/testdata/family-declarations/good/reef-storage-example/declarations/reef.yaml @@ -0,0 +1,13 @@ +kind: substrate-reef +reef_id: reef-storage-example +repo: reef-storage-example +ownership_repo: railiance-platform +substrate_kind: object-storage +lifecycle_state: planned +criticality: high +current_members: [] +source_documents: + - repo: railiance-master + path: docs/reef-substrate-model.md +compatibility_notes: + - Provider-delegated S3 fixture. No rail. The provider operates the capability. diff --git a/tools/validate-family-declarations.py b/tools/validate-family-declarations.py index 3c22afa..7c0c52c 100755 --- a/tools/validate-family-declarations.py +++ b/tools/validate-family-declarations.py @@ -190,7 +190,7 @@ def check_reef(loaded: Loaded, report: Report) -> None: report.add(loaded.rel, "reef_id", f"{doc['reef_id']!r} must equal repo {doc['repo']!r}") primary = doc.get("primary_rail") hosted = as_list(doc.get("hosted_rails")) - if primary and hosted and primary not in hosted: + if primary and primary not in hosted: report.add( loaded.rel, "primary_rail", diff --git a/workplans/RMASTER-WP-0022-establish-reef-storage.md b/workplans/RMASTER-WP-0022-establish-reef-storage.md index 10c63ee..d277c10 100644 --- a/workplans/RMASTER-WP-0022-establish-reef-storage.md +++ b/workplans/RMASTER-WP-0022-establish-reef-storage.md @@ -4,7 +4,7 @@ type: workplan title: "Establish reef-storage for Scaleway object storage" domain: financials repo: railiance-master -status: proposed +status: finished owner: grok topic_slug: railiance created: "2026-08-14" @@ -76,7 +76,7 @@ Non-secret attributes live under e.g. ```task id: RMASTER-WP-0022-T01 -status: todo +status: done priority: high state_hub_task_id: "c222e439-795d-45bb-ba81-b94b2a571a1c" ``` @@ -89,11 +89,15 @@ Document `substrate_kind: object-storage`. Keep `reef-railiance` valid. Done when a planned `reef-storage` declaration validates and `reef-railiance` still validates. +Completed 2026-08-14: `primary_rail` / `hosted_rails` optional; empty +`current_members` allowed; substrate model documents provider-delegated +reefs with no rail. Confirmed: no rail — Scaleway operates S3. + ## T02 — Bootstrap the reef-storage repo ```task id: RMASTER-WP-0022-T02 -status: todo +status: done priority: high state_hub_task_id: "61617b38-27b5-4c44-be8a-244a2476f1eb" ``` @@ -107,11 +111,14 @@ Done when the repo exists, declares `lifecycle_state: planned`, and resource-control can name `reef:storage/…` paths that will be filled after purchase. +Completed 2026-08-14: operator provided the repo; declaration, INTENT, +SCOPE, and planned attribute path committed. + ## T03 — Attribute file contract for object stores ```task id: RMASTER-WP-0022-T03 -status: todo +status: done priority: medium state_hub_task_id: "6ea318c4-33e5-4177-9f87-433670636304" ``` @@ -123,3 +130,7 @@ paste. Done when a schema or documented example exists and WP-0002 T03 can cite it as the destination of non-secret attributes after buy. + +Completed 2026-08-14: +`reef-storage/schemas/object-store-attributes.schema.json` and planned +`substrate/object-stores/platform-audit-storage.yaml`.