diff --git a/.repo-classification.yaml b/.repo-classification.yaml deleted file mode 100644 index d5a5941..0000000 --- a/.repo-classification.yaml +++ /dev/null @@ -1,23 +0,0 @@ -repo_classification: - standard: Repo Classification Standard - version: "1.0" - classified_at: "2026-08-14" - classified_by: agent - category: project - domain: financials - secondary_domains: - - infotech - capability_tags: - - platform - - operations - - storage - - governance - business_stake: - - technology - - operations - business_mechanics: - - coordination - - operation - repo_family: reef - ownership_repo: railiance-platform - substrate_kind: object-storage diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index d6283bb..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,17 +0,0 @@ -# reef-storage — Agent Instructions - -## Repo Identity - -**Purpose:** Provider-delegated S3-compatible storage substrate. Owns -non-secret operating attributes. Does not host a rail. - -**Domain:** financials -**Repo slug:** reef-storage -**Workplan prefix:** `REEF-STORAGE-WP-` - -## Standing rules - -- No rail. Scaleway (or a successor provider) operates S3. -- No secrets in this repo. Cite `secret:railiance-platform/`. -- Attribute files are the source for `reef:storage/…#key` references. -- Do not fold this substrate into `reef-railiance`. diff --git a/INTENT.md b/INTENT.md deleted file mode 100644 index d6ac90e..0000000 --- a/INTENT.md +++ /dev/null @@ -1,31 +0,0 @@ -# INTENT - -## Why This Repo Exists - -`reef-storage` is the substrate boundary for **provider-delegated -S3-compatible object storage**. The first instance is Scaleway, used for -postgres backup and other large, slow-tolerant volumes. - -We consume the S3 capability. We do **not** host a rail and we do **not** -operate how Scaleway provides or runs object storage. This reef names the -boundary, holds the committed non-secret attributes, and is the place -operators look for “where is our object store?” - -It is a different reef from `reef-railiance`. Backup is not bolted onto -the home compute substrate. - -## What This Repo Must Protect - -- the split: procure/cost in `resource-control`, operate-by-reference here, - secrets in `railiance-platform`, consume in `rapp-*` -- no rail invented to satisfy a compute-centric schema -- no secret material in git -- independent failure domain from Host Europe / `reef-railiance` - -## What This Repo Is Not - -- a Scaleway account manager or billing ledger -- a Barman/CNPG operator (`rapp-postgres`) -- a secret store -- a generic S3 client library -- an extension of `reef-railiance` diff --git a/README.md b/README.md index 9a73d78..60e7492 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,3 @@ # reef-storage -Low-cost, data-secure, runtime-independent **S3-compatible** storage -substrate for backup and large volumes that can tolerate slow access. - -Scaleway provides the S3 capability. This reef does **not** host a rail -and does not operate the object store. It names the boundary and holds -non-secret attributes. - -- Declaration: `declarations/reef.yaml` -- Attributes: `substrate/object-stores/` -- Join: `reef:storage/#` from `resource-control` \ No newline at end of file +Low cost data secure storage. Runtime independent S3 compatible storage facility for backup and large volume storage that can tolerate rather slow access. \ No newline at end of file diff --git a/SCOPE.md b/SCOPE.md deleted file mode 100644 index 90a9642..0000000 --- a/SCOPE.md +++ /dev/null @@ -1,29 +0,0 @@ -# SCOPE - -## One-liner - -Provider-delegated S3-compatible storage reef: identity and non-secret -attributes, not a rail, not a secret store. - -## In Scope - -- Reef declaration (`substrate_kind: object-storage`, no rail) -- Committed non-secret object-store attributes (endpoint, region, bucket, - prefix, versioning/lifecycle flags, provider project ref) -- Runbooks for “how we use the delegated store,” not how Scaleway runs it -- Binding notes for consumers (`rapp-postgres` first) - -## Out of Scope - -- Purchase approval, forecasts, settlement (`resource-control`) -- Access keys and OpenBao delivery (`railiance-platform`) -- WAL archive, backup schedule, restore (`rapp-postgres`) -- Hosting Kubernetes or any rail -- Implementing S3 - -## Getting Oriented - -- Declaration: `declarations/reef.yaml` -- Attribute contract: `schemas/object-store-attributes.schema.json` -- Attribute files: `substrate/object-stores/` -- Join convention: `resource-control/docs/operational-reference-convention.md` diff --git a/bindings/consumers.yaml b/bindings/consumers.yaml deleted file mode 100644 index bb525b7..0000000 --- a/bindings/consumers.yaml +++ /dev/null @@ -1,19 +0,0 @@ -reef_id: reef-storage -consumers_potential: - - consumer_id: rapp-postgres - resource_id: resource:platform:audit-storage - purpose: WAL archive and physical base backups for platform-pg - readiness_state: declared - evidence: - - repo: resource-control - path: workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md -consumers_actual: - - consumer_id: rapp-postgres - resource_id: resource:platform:audit-storage - readiness_state: wal-archiving - evidence: - - repo: resource-control - path: docs/evidence/RESOURCE-WP-0002-T05-restore-2026-08-14.md -notes: - - Actual consumer is recorded only after WAL flows (RESOURCE-WP-0002-T05). - - rapp-postgres consumes a reviewed destination; it does not procure the bucket. diff --git a/declarations/reef.yaml b/declarations/reef.yaml deleted file mode 100644 index 5827590..0000000 --- a/declarations/reef.yaml +++ /dev/null @@ -1,21 +0,0 @@ -kind: substrate-reef -reef_id: reef-storage -repo: reef-storage -ownership_repo: railiance-platform -substrate_kind: object-storage -lifecycle_state: planned -criticality: high -current_members: - - railiance-platform-pg-backup -source_documents: - - repo: railiance-master - path: docs/reef-substrate-model.md - - repo: railiance-master - path: workplans/RMASTER-WP-0022-establish-reef-storage.md - - repo: resource-control - path: docs/operational-reference-convention.md -compatibility_notes: - - "S3-compatible capability is delegated to Scaleway. This reef does not host a rail and does not operate the object store internals." - - "Non-secret attributes (endpoint, bucket, region, prefix, lifecycle) live under substrate/object-stores/." - - "Credentials live in railiance-platform OpenBao. resource-control cites reef: and secret: references only." - - "This substrate is independent of reef-railiance. Backup is not provided on the home compute reef." diff --git a/docs/post-purchase-seed.md b/docs/post-purchase-seed.md deleted file mode 100644 index 759058d..0000000 --- a/docs/post-purchase-seed.md +++ /dev/null @@ -1,25 +0,0 @@ -# Fill reef-storage after Scaleway purchase - -Run only after human financial authority approves -`resource-control/docs/evidence/RESOURCE-WP-0002-primary-selection-2026-08-14.md`. - -1. Create a Scaleway project on the Railiance (or GmbH) paying account. -2. Create a **private** bucket in `nl-ams`, Standard Multi-AZ. -3. Enable versioning. Set lifecycle to the 30-day recovery window. -4. Create a key that can only list/get/put/delete on that bucket/prefix. -5. Turn on a project cost alert. -6. Write the non-secret facts into - `substrate/object-stores/platform-audit-storage.yaml`: - `endpoint`, `bucket`, `prefix`, `provider_project_ref`, `versioning`, - `lifecycle`. Status → `active`. -7. Append the bucket/project to `declarations/reef.yaml` `current_members` - and `substrate/topology.yaml` `members`. Reef `lifecycle_state` may stay - `planned` until the first restore (T05), or become `active` when the - bucket exists — prefer `active` only after T05. -8. Hand `secret:railiance-platform/backup` to OpenBao (T04). - Do not commit the key. -9. Tell resource-control: set inventory `decision.status: approved`, - `status: ordered`, `ordered_on`, `provider.provider_resource_id`. - -Do not paste endpoint or keys into chat as the source of truth. The YAML -file is the source for `reef:storage/…` references. diff --git a/docs/put-scaleway-bootstrap.md b/docs/put-scaleway-bootstrap.md deleted file mode 100644 index f7badd1..0000000 --- a/docs/put-scaleway-bootstrap.md +++ /dev/null @@ -1,37 +0,0 @@ -# Put the Scaleway bootstrap key (founder, local only) - -Do this on a trusted terminal. **Do not paste ACCESS_KEY or SECRET_KEY -into chat, Git, or State Hub.** - -CCR: `railiance-platform/credential-change-requests/CCR-2026-0011-scaleway-object-storage-bootstrap.yaml` -Path: `platform/workloads/railiance/scaleway/bootstrap` - -1. In [console.scaleway.com](https://console.scaleway.com) create or reuse - the Railiance (or GmbH) project. IAM → API keys: create a key that can - create Object Storage buckets in that project. Copy org id and project id - from the project dashboard (those two are not as sensitive as the secret - key, but still keep them out of git). -2. On this host, with a token that can write the `platform` mount: - -```bash -bao kv put platform/workloads/railiance/scaleway/bootstrap \ - ACCESS_KEY='SCWxxxxxxxx' \ - SECRET_KEY='xxxxxxxx' \ - DEFAULT_ORGANIZATION_ID='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' \ - DEFAULT_PROJECT_ID='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' -``` - -3. Confirm **without printing values**: - -```bash -bao kv metadata get platform/workloads/railiance/scaleway/bootstrap -``` - -You should see a current version. Then tell the agent “bootstrap is in -OpenBao.” They will run `tools/create-platform-audit-bucket.sh`, which -creates the private Multi-AZ bucket, 30-day lifecycle, and writes only -endpoint/bucket/region into `substrate/object-stores/platform-audit-storage.yaml`. - -The scoped backup key is a **different** path -(`platform/workloads/railiance/backup/object-storage`) and is T04. -After the scoped key works, delete or lock down this bootstrap key. diff --git a/schemas/object-store-attributes.schema.json b/schemas/object-store-attributes.schema.json deleted file mode 100644 index 728a4d0..0000000 --- a/schemas/object-store-attributes.schema.json +++ /dev/null @@ -1,24 +0,0 @@ -{ - "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://railiance.dev/reef-storage/object-store-attributes.schema.json", - "title": "Non-secret object-store attributes", - "type": "object", - "additionalProperties": false, - "required": ["schema_version", "resource_id", "provider", "status"], - "properties": { - "schema_version": {"const": "0.1"}, - "resource_id": {"type": "string", "pattern": "^resource:"}, - "provider": {"type": "string", "minLength": 1}, - "product": {"type": ["string", "null"]}, - "status": {"enum": ["planned", "active", "retired"]}, - "endpoint": {"type": ["string", "null"]}, - "region": {"type": ["string", "null"]}, - "bucket": {"type": ["string", "null"]}, - "prefix": {"type": ["string", "null"]}, - "versioning": {"type": ["boolean", "null"]}, - "lifecycle": {"type": ["string", "null"]}, - "provider_project_ref": {"type": ["string", "null"]}, - "intended": {"type": ["object", "null"]}, - "notes": {"type": "array", "items": {"type": "string"}} - } -} diff --git a/substrate/identity.yaml b/substrate/identity.yaml deleted file mode 100644 index a9571e5..0000000 --- a/substrate/identity.yaml +++ /dev/null @@ -1,15 +0,0 @@ -reef_id: reef-storage -display_name: Railiance delegated object storage -substrate_kind: object-storage -ownership_repo: railiance-platform -lifecycle_state: planned -criticality: high -primary_rail: null -current_members: - - railiance-platform-pg-backup -current_role: Provider-delegated S3-compatible store. Scaleway operates the capability; this reef names the boundary and non-secret attributes. -source_of_truth: - repo: reef-storage - paths: - - declarations/reef.yaml - - substrate/object-stores/platform-audit-storage.yaml diff --git a/substrate/object-stores/platform-audit-storage.yaml b/substrate/object-stores/platform-audit-storage.yaml deleted file mode 100644 index 50ef4fb..0000000 --- a/substrate/object-stores/platform-audit-storage.yaml +++ /dev/null @@ -1,25 +0,0 @@ -# Live attributes for RESOURCE-WP-0002. No secrets in this file. -schema_version: "0.1" -resource_id: resource:platform:audit-storage -provider: Scaleway -product: standard-multi-az -status: active -endpoint: https://s3.nl-ams.scw.cloud -region: nl-ams -bucket: railiance-platform-pg-backup -prefix: platform-pg/ -versioning: true -lifecycle: 30-day current and noncurrent expiration -provider_project_ref: e1a0dd0e-04b8-4ea9-8b30-d53f8c35c688 -intended: - public_access: disabled - versioning: true - lifecycle: 30-day recovery window matching data/demand/platform-audit-storage.json - encryption_at_rest: provider-managed - identity: bucket-and-prefix scoped key only -notes: - - Destination for rapp-postgres WAL and base backups once procured. - - resource-control cites reef:storage/substrate/object-stores/platform-audit-storage.yaml# - - "Decision record: resource-control/docs/evidence/RESOURCE-WP-0002-primary-selection-2026-08-14.md" - - Created 2026-08-14. Versioning enabled. ACL owner-only FULL_CONTROL. - - Cost backstop is a founder email billing alert. Scaleway has no euro hard-stop. diff --git a/substrate/topology.yaml b/substrate/topology.yaml deleted file mode 100644 index 7b6d936..0000000 --- a/substrate/topology.yaml +++ /dev/null @@ -1,21 +0,0 @@ -reef_id: reef-storage -membership_policy: Add a member when a purchased object-store (bucket/project) is live. Members are provider resources, not hosts. -members: - - member_id: platform-audit-storage - role: primary-backup-object-store - provider: Scaleway - product: standard-multi-az - region: nl-ams - bucket: railiance-platform-pg-backup - resource_id: resource:platform:audit-storage - status: live - evidence: - - repo: resource-control - path: docs/evidence/RESOURCE-WP-0002-primary-selection-2026-08-14.md - - path: substrate/object-stores/platform-audit-storage.yaml -intended_members: [] -access_surfaces: - - type: s3-https - note: Endpoint and bucket live in substrate/object-stores/platform-audit-storage.yaml. No rail. -unresolved_questions: - - Whether Host Europe Backup Storage or Nextcloud is the T06 second copy diff --git a/tools/apply-backup-bucket-policy.sh b/tools/apply-backup-bucket-policy.sh deleted file mode 100755 index 58c7337..0000000 --- a/tools/apply-backup-bucket-policy.sh +++ /dev/null @@ -1,165 +0,0 @@ -#!/usr/bin/env bash -# Apply a prefix-scoped bucket policy for the resource-control backup key. -# Reads APPLICATION_ID from OpenBao. Never prints secret values. -# Uses the bootstrap key to PutBucketPolicy (org owner can always edit policies). -set -euo pipefail - -ROOT="$(cd "$(dirname "$0")/.." && pwd)" -BAO_ADDR="${BAO_ADDR:-${VAULT_ADDR:-https://bao.coulomb.social}}" -REGION="nl-ams" -BUCKET="${BUCKET:-railiance-platform-pg-backup}" -PREFIX="${PREFIX:-platform-pg/}" - -need() { command -v "$1" >/dev/null || { echo "missing $1" >&2; exit 2; }; } -export PATH="${HOME}/.local/bin:${PATH}" -need python3 -need curl - -TOKEN="${OPENBAO_TOKEN:-${VAULT_TOKEN:-}}" -if [[ -z "$TOKEN" && -f "$HOME/.vault-token" ]]; then - TOKEN="$(cat "$HOME/.vault-token")" -fi -[[ -n "$TOKEN" ]] || { echo "no OpenBao token" >&2; exit 2; } - -read_kv() { - local path="$1" - curl -fsS -H "X-Vault-Token: $TOKEN" \ - "$BAO_ADDR/v1/platform/data/${path#platform/}" \ - | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps(d["data"]["data"]))' -} - -BACKUP_JSON="$(read_kv workloads/railiance/backup/object-storage)" -BOOTSTRAP_JSON="$(read_kv workloads/railiance/scaleway/bootstrap)" - -EXPORTS="$( -BACKUP_JSON="$BACKUP_JSON" BOOTSTRAP_JSON="$BOOTSTRAP_JSON" python3 - <<'PY' -import json, os, sys, re -backup = json.loads(os.environ["BACKUP_JSON"]) -boot = json.loads(os.environ["BOOTSTRAP_JSON"]) -app = (backup.get("APPLICATION_ID") or "").strip() -if not app or app in {"xxx", "redacted", "changeme"}: - sys.stderr.write( - "APPLICATION_ID missing on platform/workloads/railiance/backup/object-storage\n" - "Founder: add that field in the OpenBao UI (IAM application resource-control UUID).\n" - "It is not a secret. Then re-run this script.\n" - ) - sys.exit(6) -if not re.fullmatch(r"[0-9a-fA-F-]{36}", app): - sys.stderr.write("APPLICATION_ID is not a 36-char UUID\n") - sys.exit(7) -def g(data, *ks): - for k in ks: - if data.get(k): - return data[k] - return "" -print("export APPLICATION_ID=" + json.dumps(app)) -print("export SCW_ACCESS_KEY=" + json.dumps(g(boot, "ACCESS_KEY"))) -print("export SCW_SECRET_KEY=" + json.dumps(g(boot, "SECRET_KEY"))) -print("export BACKUP_ACCESS_KEY=" + json.dumps(g(backup, "ACCESS_KEY", "ACCESS_KEY_ID"))) -print("export BACKUP_SECRET_KEY=" + json.dumps(g(backup, "SECRET_KEY", "ACCESS_SECRET_KEY"))) -PY -)" -eval "$EXPORTS" - -export SCW_DEFAULT_REGION="$REGION" -export AWS_ACCESS_KEY_ID="$SCW_ACCESS_KEY" -export AWS_SECRET_ACCESS_KEY="$SCW_SECRET_KEY" - -VENV="${TMPDIR:-/tmp}/reef-storage-boto3" -if [[ ! -x "$VENV/bin/python" ]]; then - python3 -m venv "$VENV" - "$VENV/bin/pip" -q install boto3 -fi - -echo "putting prefix-scoped bucket policy on $BUCKET (application id not printed)" -BUCKET="$BUCKET" PREFIX="$PREFIX" APPLICATION_ID="$APPLICATION_ID" \ - "$VENV/bin/python" - <<'PY' -import json, os -import boto3 -bucket = os.environ["BUCKET"] -prefix = os.environ["PREFIX"] -app = os.environ["APPLICATION_ID"] -policy = { - "Version": "2023-04-17", - "Id": "resource-control-scoped-backup", - "Statement": [ - { - "Sid": "BackupListPrefix", - "Effect": "Allow", - "Principal": {"SCW": f"application_id:{app}"}, - "Action": [ - "s3:ListBucket", - "s3:ListBucketMultipartUploads", - "s3:ListBucketVersions", - "s3:GetBucketLocation", - "s3:GetBucketVersioning", - "s3:GetLifecycleConfiguration", - ], - "Resource": [bucket], - "Condition": {"StringLike": {"s3:prefix": [prefix, prefix + "*"]}}, - }, - { - "Sid": "BackupObjectsUnderPrefix", - "Effect": "Allow", - "Principal": {"SCW": f"application_id:{app}"}, - "Action": [ - "s3:GetObject", - "s3:GetObjectVersion", - "s3:PutObject", - "s3:DeleteObject", - "s3:DeleteObjectVersion", - "s3:AbortMultipartUpload", - "s3:ListMultipartUploadParts", - ], - "Resource": [f"{bucket}/{prefix}*"], - }, - ], -} -client = boto3.client( - "s3", - region_name="nl-ams", - endpoint_url="https://s3.nl-ams.scw.cloud", - aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"], - aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"], -) -client.put_bucket_policy(Bucket=bucket, Policy=json.dumps(policy)) -print("bucket_policy=put") -got = client.get_bucket_policy(Bucket=bucket) -body = json.loads(got["Policy"]) -print("statements", [s.get("Sid") for s in body.get("Statement", [])]) -PY - -echo "verifying backup key: prefix ok, sibling denied" -BUCKET="$BUCKET" PREFIX="$PREFIX" \ -AWS_ACCESS_KEY_ID="$BACKUP_ACCESS_KEY" AWS_SECRET_ACCESS_KEY="$BACKUP_SECRET_KEY" \ - "$VENV/bin/python" - <<'PY' -import os -from botocore.exceptions import ClientError -import boto3 -c = boto3.client( - "s3", - region_name="nl-ams", - endpoint_url="https://s3.nl-ams.scw.cloud", - aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"], - aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"], -) -bucket = os.environ["BUCKET"] -prefix = os.environ["PREFIX"] -ok = prefix + ".policy-probe" -sib = "other-consumer/.policy-probe" -try: - c.put_object(Bucket=bucket, Key=ok, Body=b"ok") - c.delete_object(Bucket=bucket, Key=ok) - print("prefix_write=ok") -except ClientError as e: - print("prefix_write=fail", e.response.get("Error", {}).get("Code")) -try: - c.put_object(Bucket=bucket, Key=sib, Body=b"no") - print("sibling_write=UNEXPECTED_OK") - c.delete_object(Bucket=bucket, Key=sib) -except ClientError as e: - print("sibling_write=denied", e.response.get("Error", {}).get("Code")) -PY - -echo "done" -unset SCW_SECRET_KEY SCW_ACCESS_KEY BACKUP_SECRET_KEY BACKUP_ACCESS_KEY AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY APPLICATION_ID diff --git a/tools/create-platform-audit-bucket.sh b/tools/create-platform-audit-bucket.sh deleted file mode 100755 index eb491ca..0000000 --- a/tools/create-platform-audit-bucket.sh +++ /dev/null @@ -1,139 +0,0 @@ -#!/usr/bin/env bash -# Create the planned Scaleway backup bucket. Reads bootstrap creds from -# OpenBao. Never prints secret values. Writes only non-secret attributes -# into substrate/object-stores/platform-audit-storage.yaml. -set -euo pipefail - -ROOT="$(cd "$(dirname "$0")/.." && pwd)" -ATTR="$ROOT/substrate/object-stores/platform-audit-storage.yaml" -BAO_ADDR="${BAO_ADDR:-${VAULT_ADDR:-https://bao.coulomb.social}}" -BOOTSTRAP_PATH="platform/workloads/railiance/scaleway/bootstrap" -SCOPED_PATH="platform/workloads/railiance/backup/object-storage" -REGION="nl-ams" -BUCKET="${BUCKET:-railiance-platform-pg-backup}" -PREFIX="${PREFIX:-platform-pg/}" -ENDPOINT="https://s3.nl-ams.scw.cloud" - -need() { command -v "$1" >/dev/null || { echo "missing $1" >&2; exit 2; }; } -export PATH="${HOME}/.local/bin:${PATH}" -need python3 -need curl -need scw - -TOKEN="${OPENBAO_TOKEN:-${VAULT_TOKEN:-}}" -if [[ -z "$TOKEN" && -f "$HOME/.vault-token" ]]; then - TOKEN="$(cat "$HOME/.vault-token")" -fi -[[ -n "$TOKEN" ]] || { echo "no OpenBao token" >&2; exit 2; } - -read_kv() { - local path="$1" - curl -fsS -H "X-Vault-Token: $TOKEN" \ - "$BAO_ADDR/v1/platform/data/${path#platform/}" \ - | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps(d["data"]["data"]))' -} - -if ! BOOTSTRAP_JSON="$(read_kv workloads/railiance/scaleway/bootstrap 2>/dev/null)"; then - echo "Bootstrap secret missing at $BOOTSTRAP_PATH" >&2 - echo "Founder: put ACCESS_KEY SECRET_KEY DEFAULT_ORGANIZATION_ID DEFAULT_PROJECT_ID locally (not in chat)." >&2 - echo "See docs/put-scaleway-bootstrap.md" >&2 - exit 3 -fi - -eval "$(BOOTSTRAP_JSON="$BOOTSTRAP_JSON" python3 - <<'PY' -import json, os, sys -data = json.loads(os.environ["BOOTSTRAP_JSON"]) -needed = ("ACCESS_KEY", "SECRET_KEY", "DEFAULT_ORGANIZATION_ID", "DEFAULT_PROJECT_ID") -missing = [k for k in needed if not (data.get(k) or data.get(k.lower()))] -if missing: - sys.stderr.write("bootstrap fields missing: " + ",".join(missing) + "\n") - sys.exit(4) -placeholders = [k for k in needed if (data.get(k) or data.get(k.lower()) or "") in {"xxx", "redacted", "changeme"}] -if placeholders: - sys.stderr.write("bootstrap still has example placeholders; replace them in the OpenBao UI first\n") - sys.exit(5) -def g(k): - return data.get(k) or data.get(k.lower()) -print("export SCW_ACCESS_KEY=" + json.dumps(g("ACCESS_KEY"))) -print("export SCW_SECRET_KEY=" + json.dumps(g("SECRET_KEY"))) -print("export SCW_DEFAULT_ORGANIZATION_ID=" + json.dumps(g("DEFAULT_ORGANIZATION_ID"))) -print("export SCW_DEFAULT_PROJECT_ID=" + json.dumps(g("DEFAULT_PROJECT_ID"))) -PY -)" - -export SCW_DEFAULT_REGION="$REGION" - -if scw object bucket get "$BUCKET" region="$REGION" -o json >/dev/null 2>&1; then - echo "bucket $BUCKET already exists in $REGION; skipping create" -else - echo "creating private bucket $BUCKET in $REGION (versioning on)" - scw object bucket create "$BUCKET" region="$REGION" acl=private enable-versioning=true -o json -fi - -echo "applying 30-day lifecycle (current + noncurrent versions)" -VENV="${TMPDIR:-/tmp}/reef-storage-boto3" -if [[ ! -x "$VENV/bin/python" ]]; then - python3 -m venv "$VENV" - "$VENV/bin/pip" -q install boto3 -fi -BUCKET="$BUCKET" AWS_ACCESS_KEY_ID="$SCW_ACCESS_KEY" AWS_SECRET_ACCESS_KEY="$SCW_SECRET_KEY" \ - "$VENV/bin/python" - <<'PY' -import os -import boto3 -client = boto3.client( - "s3", - region_name="nl-ams", - endpoint_url="https://s3.nl-ams.scw.cloud", - aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"], - aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"], -) -client.put_bucket_lifecycle_configuration( - Bucket=os.environ["BUCKET"], - LifecycleConfiguration={ - "Rules": [ - { - "ID": "retain-30-days", - "Status": "Enabled", - "Filter": {"Prefix": ""}, - "Expiration": {"Days": 30}, - "NoncurrentVersionExpiration": {"NoncurrentDays": 30}, - } - ] - }, -) -print("lifecycle applied") -PY - -echo "creating a 20 EUR monthly budget alert if none exists" -scw billing budget create consumption-limit=20 enabled=true -o json | python3 -c ' -import json,sys -b=json.load(sys.stdin) -print("budget_id", b.get("id") or b.get("budget",{}).get("id")) -print("limit", b.get("consumption_limit") or b.get("budget",{}).get("consumption_limit")) -' || echo "budget create skipped (permissions or API shape)" - -echo "writing non-secret attributes (no keys)" -python3 - <&2 - exit 2 -} -command -v bao >/dev/null || { echo "missing bao" >&2; exit 2; } - -mapfile -t EXPORTS < <(python3 - "$TFVARS" <<'PY' -import re, json, sys -text = open(sys.argv[1], encoding="utf-8").read() -# HCL-ish: key = "value" (ignore comments and the main.tf provider block) -found = {} -for name, dest in ( - ("access_key", "ACCESS_KEY"), - ("secret_key", "SECRET_KEY"), - ("organization_id", "DEFAULT_ORGANIZATION_ID"), - ("project_id", "DEFAULT_PROJECT_ID"), -): - m = re.search(rf'(?m)^\s*{name}\s*=\s*"([^"]*)"\s*$', text) - if not m or not m.group(1) or m.group(1) in {"xxx", "redacted"}: - sys.stderr.write(f"missing or placeholder field: {name}\n") - sys.exit(4) - found[dest] = m.group(1) -for dest, val in found.items(): - print(f"{dest}={json.dumps(val)}") -PY -) - -# Write via env to bao without putting values on argv. -# bao kv put supports @file; we use a mode-0600 temp dir. -umask 077 -tmp="$(mktemp -d)" -trap 'rm -rf "$tmp"' EXIT -for line in "${EXPORTS[@]}"; do - field="${line%%=*}" - python3 -c 'import json,os,sys; open(sys.argv[2],"w").write(json.loads(sys.argv[1]))' \ - "${line#*=}" "$tmp/$field" -done - -bao kv put platform/workloads/railiance/scaleway/bootstrap \ - ACCESS_KEY=@"$tmp/ACCESS_KEY" \ - SECRET_KEY=@"$tmp/SECRET_KEY" \ - DEFAULT_ORGANIZATION_ID=@"$tmp/DEFAULT_ORGANIZATION_ID" \ - DEFAULT_PROJECT_ID=@"$tmp/DEFAULT_PROJECT_ID" - -echo "wrote four fields to platform/workloads/railiance/scaleway/bootstrap" -bao kv metadata get platform/workloads/railiance/scaleway/bootstrap -echo "shred or keep your tfvars; do not commit it" diff --git a/workplans/REEF-STORAGE-WP-0001-bootstrap.md b/workplans/REEF-STORAGE-WP-0001-bootstrap.md deleted file mode 100644 index 5d0eb31..0000000 --- a/workplans/REEF-STORAGE-WP-0001-bootstrap.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -id: REEF-STORAGE-WP-0001 -type: workplan -title: "Bootstrap reef-storage" -domain: financials -repo: reef-storage -status: active -owner: grok -topic_slug: railiance -created: "2026-08-14" -updated: "2026-08-14" -related: - - RMASTER-WP-0022 - - RESOURCE-WP-0002 ---- - -# REEF-STORAGE-WP-0001 — bootstrap - -## Goal - -Stand up the planned storage reef: declaration without a rail, attribute -file layout, and the contract for non-secret object-store facts. - -## T01 — Declare the reef - -```task -id: REEF-STORAGE-WP-0001-T01 -status: done -priority: high -``` - -`declarations/reef.yaml`: `substrate_kind: object-storage`, no -`primary_rail`, no `hosted_rails`, empty `current_members`, planned. - -## T02 — Attribute contract - -```task -id: REEF-STORAGE-WP-0001-T02 -status: done -priority: high -``` - -`schemas/object-store-attributes.schema.json` and a planned -`platform-audit-storage.yaml` with null operating fields until purchase. diff --git a/workplans/REEF-STORAGE-WP-0002-fill-after-purchase.md b/workplans/REEF-STORAGE-WP-0002-fill-after-purchase.md deleted file mode 100644 index 4320070..0000000 --- a/workplans/REEF-STORAGE-WP-0002-fill-after-purchase.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -id: REEF-STORAGE-WP-0002 -type: workplan -title: "Fill Scaleway attributes after purchase" -domain: financials -repo: reef-storage -status: active -owner: grok -topic_slug: railiance -created: "2026-08-14" -updated: "2026-08-14" -related: - - RESOURCE-WP-0002 ---- - -# REEF-STORAGE-WP-0002 — fill after purchase - -Purchase happened 2026-08-14. Follow `docs/post-purchase-seed.md`. - -## T01 — Write live non-secret attributes - -```task -id: REEF-STORAGE-WP-0002-T01 -status: done -priority: high -``` - -Fill `substrate/object-stores/platform-audit-storage.yaml` endpoint, -bucket, prefix, project ref, versioning, lifecycle. Promote topology -members. No secrets. - -Done 2026-08-14: bucket `railiance-platform-pg-backup` in `nl-ams` is -live (versioning on, 30-day current and noncurrent expiration, owner-only -ACL). Attributes, topology member, and declaration `current_members` -written. Reef `lifecycle_state` stays `planned` until RESOURCE-WP-0002-T05. -Cost backstop is a founder email billing alert. Scaleway has no -euro hard-stop; that gap is accepted.