feat: inventory schema 0.3 and close WP-0006 T01 T02 T04

v0.2 records stay valid. v0.3 requires the five facets. Validators
reject inline Scaleway endpoints and secret-looking strings. The
backup record is the first 0.3 object. Reef views already met T02.
This commit is contained in:
tegwick 2026-08-15 02:40:43 +02:00
parent a9830f3109
commit 34a014a896
8 changed files with 140 additions and 12 deletions

View file

@ -1,5 +1,5 @@
{ {
"schema_version": "0.2", "schema_version": "0.3",
"record_scope": "inventory", "record_scope": "inventory",
"id": "resource:platform:audit-storage", "id": "resource:platform:audit-storage",
"financial_entity_id": "entity:railiance", "financial_entity_id": "entity:railiance",

View file

@ -45,7 +45,7 @@ reef:<reef_id>/<repo-relative-path>#<json_or_yaml_key>
- Fragment is a dotted key into that file (`endpoint`, `bucket.prefix`). - Fragment is a dotted key into that file (`endpoint`, `bucket.prefix`).
- The file is committed, non-secret, and the reef is authoritative for it. - The file is committed, non-secret, and the reef is authoritative for it.
Examples (illustrative until `reef-storage` exists): Examples (live on `reef-storage` and `reef-railiance`):
```text ```text
reef:storage/substrate/object-stores/platform-audit-storage.yaml#endpoint reef:storage/substrate/object-stores/platform-audit-storage.yaml#endpoint

View file

@ -1,12 +1,24 @@
{ {
"$schema": "https://json-schema.org/draft/2020-12/schema", "$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://coulomb.social/resource-control/resource-inventory.schema.json", "$id": "https://coulomb.social/resource-control/resource-inventory.schema.json",
"title": "Railiance managed-infrastructure portfolio resource v0.2", "title": "Railiance managed-infrastructure portfolio resource v0.2/v0.3",
"type": "object", "type": "object",
"additionalProperties": false, "additionalProperties": false,
"required": ["schema_version", "record_scope", "id", "resource_class", "status", "management_model", "provider", "service", "location", "capacity", "ownership", "cost", "lifecycle", "relationships", "requirements", "evidence"], "required": ["schema_version", "record_scope", "id", "resource_class", "status", "management_model", "provider", "service", "location", "capacity", "ownership", "cost", "lifecycle", "relationships", "requirements", "evidence"],
"if": {
"properties": {"schema_version": {"const": "0.3"}}
},
"then": {
"required": [
"schema_version", "record_scope", "id", "resource_class", "status",
"management_model", "provider", "service", "location", "capacity",
"ownership", "cost", "lifecycle", "relationships", "requirements",
"evidence", "description", "decision", "operational_refs",
"credential_handles", "consumers"
]
},
"properties": { "properties": {
"schema_version": {"const": "0.2"}, "schema_version": {"enum": ["0.2", "0.3"]},
"record_scope": {"enum": ["inventory", "example"]}, "record_scope": {"enum": ["inventory", "example"]},
"id": {"type": "string", "pattern": "^resource:[a-z0-9][a-z0-9:_-]+$"}, "id": {"type": "string", "pattern": "^resource:[a-z0-9][a-z0-9:_-]+$"},
"financial_entity_id": {"type": ["string", "null"], "pattern": "^entity:[a-z0-9]+$"}, "financial_entity_id": {"type": ["string", "null"], "pattern": "^entity:[a-z0-9]+$"},

View file

@ -47,6 +47,37 @@ class PortfolioTest(unittest.TestCase):
record["decision"]["approved_on"] = "2026-08-14" record["decision"]["approved_on"] = "2026-08-14"
validate_record(record) validate_record(record)
def test_v03_requires_five_facets(self):
record = deepcopy(next(
r for _, r in self.records() if r["id"] == "resource:platform:audit-storage"
))
self.assertEqual(record["schema_version"], "0.3")
del record["consumers"]
with self.assertRaisesRegex(ValueError, "five facets"):
validate_record(record)
def test_inline_endpoint_is_rejected(self):
record = deepcopy(next(
r for _, r in self.records() if r["id"] == "resource:platform:audit-storage"
))
record["description"] += " https://s3.nl-ams.scw.cloud"
with self.assertRaisesRegex(ValueError, "inline provider endpoint"):
validate_record(record)
def test_inline_secret_is_rejected(self):
record = deepcopy(next(
r for _, r in self.records() if r["id"] == "resource:platform:audit-storage"
))
record["description"] += " AGE-SECRET-KEY-1TEST"
with self.assertRaisesRegex(ValueError, "inline secret"):
validate_record(record)
def test_v02_records_remain_valid(self):
v02 = [r for _, r in self.records() if r.get("schema_version") == "0.2"]
self.assertGreaterEqual(len(v02), 1)
for record in v02:
validate_record(record)
def test_unknown_commission_date_is_preserved(self): def test_unknown_commission_date_is_preserved(self):
record = deepcopy(next(r for _, r in self.records() if r["status"] == "active")) record = deepcopy(next(r for _, r in self.records() if r["status"] == "active"))
record["lifecycle"]["commissioned_on"] = None record["lifecycle"]["commissioned_on"] = None

View file

@ -12,6 +12,24 @@ from pathlib import Path
from entities import association_ok from entities import association_ok
RESOURCE_ID = re.compile(r"^resource:[a-z0-9][a-z0-9:_-]+$") RESOURCE_ID = re.compile(r"^resource:[a-z0-9][a-z0-9:_-]+$")
INLINE_ENDPOINT = re.compile(
r"(https?://\S*s3\S*|https?://\S+\.scw\.cloud|\bs3://)",
re.IGNORECASE,
)
INLINE_SECRET = re.compile(
r"(BEGIN [A-Z ]*PRIVATE KEY|AGE-SECRET-KEY-|AKIA[0-9A-Z]{16}|sk_live_|SCW[A-Z0-9]{17})"
)
V03_FACETS = (
"description", "decision", "operational_refs", "credential_handles", "consumers",
)
# Evidence and decision refs may cite provider HTTPS docs. Those are not
# operating attributes and must not be treated as inline endpoints.
SKIP_INLINE_PATHS = {
("evidence", "ref"),
("decision", "ref"),
("requirements", "ref"),
("cost", "price_evidence"),
}
STATUSES = { STATUSES = {
"proposed", "ordered", "commissioning", "active", "suspended", "proposed", "ordered", "commissioning", "active", "suspended",
"retiring", "retired", "rejected", "retiring", "retired", "rejected",
@ -39,9 +57,27 @@ def validate_transition(current: str, target: str) -> None:
raise ValueError(f"invalid lifecycle transition {current} -> {target}") raise ValueError(f"invalid lifecycle transition {current} -> {target}")
def _walk_strings(node, path=()):
if isinstance(node, dict):
for key, value in node.items():
yield from _walk_strings(value, path + (key,))
elif isinstance(node, list):
for value in node:
yield from _walk_strings(value, path)
elif isinstance(node, str):
yield path, node
def _skip_inline(path: tuple) -> bool:
if path and path[0] in {"operational_refs", "credential_handles"}:
return True
return len(path) >= 2 and (path[0], path[-1]) in SKIP_INLINE_PATHS
def validate_record(record: dict) -> None: def validate_record(record: dict) -> None:
if record.get("schema_version") != "0.2": version = record.get("schema_version")
raise ValueError("portfolio records must use schema_version 0.2") if version not in {"0.2", "0.3"}:
raise ValueError("portfolio records must use schema_version 0.2 or 0.3")
if not RESOURCE_ID.fullmatch(record.get("id", "")): if not RESOURCE_ID.fullmatch(record.get("id", "")):
raise ValueError("invalid resource id") raise ValueError("invalid resource id")
if record.get("status") not in STATUSES: if record.get("status") not in STATUSES:
@ -51,6 +87,14 @@ def validate_record(record: dict) -> None:
association_ok(record) association_ok(record)
if version == "0.3":
missing = [name for name in V03_FACETS if name not in record]
if missing:
raise ValueError("schema 0.3 requires five facets: " + ",".join(missing))
consumers = record.get("consumers") or {}
if "potential" not in consumers or "actual" not in consumers:
raise ValueError("schema 0.3 consumers must list potential and actual")
decision = record.get("decision") decision = record.get("decision")
if record.get("status") in {"ordered", "commissioning"} and ( if record.get("status") in {"ordered", "commissioning"} and (
not decision or decision.get("status") != "approved" not decision or decision.get("status") != "approved"
@ -63,6 +107,20 @@ def validate_record(record: dict) -> None:
if not str(ref).startswith("secret:"): if not str(ref).startswith("secret:"):
raise ValueError(f"credential_handles must be secret: references: {ref}") raise ValueError(f"credential_handles must be secret: references: {ref}")
for path, value in _walk_strings(record):
if _skip_inline(path):
continue
if INLINE_ENDPOINT.search(value):
raise ValueError(
"inline provider endpoint must be a reef: reference, not "
+ ".".join(str(part) for part in path)
)
if INLINE_SECRET.search(value):
raise ValueError(
"inline secret material is forbidden; use a secret: handle at "
+ ".".join(str(part) for part in path)
)
allocation = record["ownership"]["allocation"] allocation = record["ownership"]["allocation"]
if allocation["mode"] == "unattributed": if allocation["mode"] == "unattributed":
if allocation["cost_attribution_key"] is not None: if allocation["cost_attribution_key"] is not None:

View file

@ -41,11 +41,16 @@ def main() -> int:
assert forecast["record_type"] == "forecast" assert forecast["record_type"] == "forecast"
assert len({row["period"] for row in forecast["rows"]}) == len(forecast["rows"]) assert len({row["period"] for row in forecast["rows"]}) == len(forecast["rows"])
assert all(row["total_eur"] == round(row["infrastructure_eur"] + row["internal_labor_eur"], 2) for row in forecast["rows"]) assert all(row["total_eur"] == round(row["infrastructure_eur"] + row["internal_labor_eur"], 2) for row in forecast["rows"])
assert schema["properties"]["schema_version"]["const"] == "0.2" assert set(schema["properties"]["schema_version"]["enum"]) == {"0.2", "0.3"}
required = set(schema["required"]) required = set(schema["required"])
v03 = {"description", "decision", "operational_refs", "credential_handles", "consumers"}
for resource in resources: for resource in resources:
assert not required - resource.keys(), f"missing fields: {required - resource.keys()}" need = set(required)
if resource.get("schema_version") == "0.3":
need |= v03
assert not need - resource.keys(), f"missing fields: {need - resource.keys()}"
validate_record(resource) validate_record(resource)
assert any(resource.get("schema_version") == "0.3" for resource in resources)
operational_records = [load(str(path)) for path in Path("data/control-cycle").glob("*.json")] operational_records = [load(str(path)) for path in Path("data/control-cycle").glob("*.json")]
control_records += operational_records control_records += operational_records
record_ids = {record["record_id"] for record in control_records} record_ids = {record["record_id"] for record in control_records}
@ -126,6 +131,8 @@ def main() -> int:
if view["reef_id"] == "reef-railiance": if view["reef_id"] == "reef-railiance":
assert view["role"] == "compute_substrate" assert view["role"] == "compute_substrate"
assert any("reef-storage" in note for note in view.get("notes") or []) assert any("reef-storage" in note for note in view.get("notes") or [])
if view["reef_id"] == "reef-storage":
assert view["role"] == "storage_substrate"
print("resource-control declarations: valid") print("resource-control declarations: valid")
return 0 return 0

View file

@ -32,6 +32,12 @@ The candidate provider baseline is:
This workplan chooses on evidence. It does not presume that keeping compute and This workplan chooses on evidence. It does not presume that keeping compute and
backup at one provider is cheaper or safer. backup at one provider is cheaper or safer.
Inventory output follows `docs/operational-reference-convention.md`
(`RESOURCE-WP-0006`). Operating attributes are `reef:storage/…` references.
Credentials are `secret:railiance-platform/backup`. Do not write a Scaleway
endpoint or key into `data/resources/`. `rapp-postgres` consumes the
reviewed destination; it does not procure it.
## Context and current evidence ## Context and current evidence
`rapp-postgres` has deployed `platform-pg` on reef-railiance at Host Europe. `rapp-postgres` has deployed `platform-pg` on reef-railiance at Host Europe.

View file

@ -8,7 +8,7 @@ status: active
owner: grok owner: grok
topic_slug: railiance topic_slug: railiance
created: "2026-08-14" created: "2026-08-14"
updated: "2026-08-14" updated: "2026-08-15"
related: related:
- RESOURCE-WP-0002 - RESOURCE-WP-0002
- RESOURCE-WP-0003 - RESOURCE-WP-0003
@ -51,7 +51,7 @@ object storage and own the non-secret attributes this repo will cite.
```task ```task
id: RESOURCE-WP-0006-T01 id: RESOURCE-WP-0006-T01
status: todo status: done
priority: high priority: high
state_hub_task_id: "55d3ebc2-51b7-49d1-8d0d-57ccc6379ae7" state_hub_task_id: "55d3ebc2-51b7-49d1-8d0d-57ccc6379ae7"
``` ```
@ -67,11 +67,16 @@ an inline secret or endpoint that should be a `reef:` / `secret:` ref, and
accept the proposed backup record with an explicit attribute-ref gap until accept the proposed backup record with an explicit attribute-ref gap until
`reef-storage` exists. `reef-storage` exists.
Done 2026-08-15: schema accepts `0.2` and `0.3`. v0.3 requires the five
facets. `portfolio.validate_record` rejects inline `s3://` / Scaleway
endpoints and secret-looking strings. The live backup record is `0.3`.
Other inventory stays `0.2` until T03.
## T02 — Reef views and the reef-railiance projection ## T02 — Reef views and the reef-railiance projection
```task ```task
id: RESOURCE-WP-0006-T02 id: RESOURCE-WP-0006-T02
status: progress status: done
priority: high priority: high
state_hub_task_id: "f2b38140-3d73-45a2-a2ff-fea12733e7b0" state_hub_task_id: "f2b38140-3d73-45a2-a2ff-fea12733e7b0"
``` ```
@ -87,6 +92,11 @@ is out of `reef-railiance`.
Started 2026-08-14: convention published; first reef-railiance view committed. Started 2026-08-14: convention published; first reef-railiance view committed.
Done 2026-08-15: `data/reefs/reef-railiance.json` and `reef-storage.json`
validate. Every cited `resource_id` exists in inventory. railiance view
states S3 backup is out of that reef. `reef-storage` view exists now that
the repo does.
## T03 — Migrate live inventory to the five facets ## T03 — Migrate live inventory to the five facets
```task ```task
@ -110,7 +120,7 @@ only consumer list.
```task ```task
id: RESOURCE-WP-0006-T04 id: RESOURCE-WP-0006-T04
status: todo status: done
priority: medium priority: medium
state_hub_task_id: "a34fa806-ee34-4d3b-a823-2797fe260bbc" state_hub_task_id: "a34fa806-ee34-4d3b-a823-2797fe260bbc"
``` ```
@ -122,3 +132,7 @@ remains a consumer, not the procurer.
Done when WP-0002 cites this convention and will not accept an inventory Done when WP-0002 cites this convention and will not accept an inventory
update that inlines Scaleway endpoint or keys. update that inlines Scaleway endpoint or keys.
Done 2026-08-15: WP-0002 goal cites the convention. The live inventory
record is v0.3 with reef/secret refs; validators reject an inline
Scaleway endpoint on that record.