diff --git a/.claude/rules/stack-and-commands.md b/.claude/rules/stack-and-commands.md index 7f7df96..ae64feb 100644 --- a/.claude/rules/stack-and-commands.md +++ b/.claude/rules/stack-and-commands.md @@ -1,18 +1,18 @@ ## Stack - **Language:** Kubernetes manifests + Helm values, Bash Make targets -- **Key deps:** CloudNative-PG (cnpg) clusters, Valkey, OpenBao, SOPS/age; runs against the S2 cluster on railiance01 +- **Key deps:** CloudNative-PG (cnpg) clusters, OpenBao, SOPS/age; runs against the S2 cluster on railiance01. Bitnami postgresql-ha is retired. Valkey is declared, not deployed. ## Dev Commands ```bash -make help # list all targets (db-*, apps-pg-*, pg-*, valkey-*, openbao-*) -make db-status # gitea-db cnpg cluster health (read-only) +make help # list all targets (db-*, apps-pg-*, openbao-*) +make db-status # gitea-db / forgejo-db cnpg cluster health (read-only) make apps-pg-status # shared apps-pg cluster health (read-only) -make pg-status # standalone PostgreSQL HA pod status (read-only) -make valkey-status # Valkey pod status (read-only) -make db-deploy | apps-pg-deploy | pg-deploy | valkey-deploy # production-touching +make pg-status # confirms retired Bitnami postgresql-ha is absent +make valkey-status # confirms Valkey is absent +make db-deploy | apps-pg-deploy # production-touching ``` -`*-deploy` targets change production platform services — operator approval -required. `*-status` / `*-logs` targets are read-only and safe. +`pg-deploy` and `valkey-deploy` are fail-closed. `*-status` / `*-logs` +targets are read-only and safe. diff --git a/.sops.yaml b/.sops.yaml index 7d38f71..13fd78e 100644 --- a/.sops.yaml +++ b/.sops.yaml @@ -1,9 +1,8 @@ # SOPS encryption policy for railiance-platform # Encrypts any file matching *.sops.yaml using the shared age key. -# Decrypt: sops -d helm/postgresql-ha-values.sops.yaml -# Use with helm: helm upgrade postgresql-ha bitnami/postgresql-ha \ -# -n platform -f <(sops -d helm/postgresql-ha-values.sops.yaml) -# Encrypt: sops -e -i helm/postgresql-ha-values.sops.yaml +# Decrypt: sops -d .sops.yaml +# Encrypt: sops -e -i .sops.yaml +# postgresql-ha deploy is retired; valkey-values.sops.yaml is unused until a consumer exists. creation_rules: - path_regex: \.sops\.yaml$ diff --git a/Makefile b/Makefile index 192091b..7d959be 100644 --- a/Makefile +++ b/Makefile @@ -86,41 +86,32 @@ apps-pg-logs: ## Tail apps-pg primary logs net-kingdom-pg-inter-hub-networkpolicy-deploy: ## Allow inter-hub to reach net-kingdom-pg $(KUBECTL) apply -f helm/net-kingdom-pg-inter-hub-networkpolicy.yaml -##@ PostgreSQL HA (legacy — superseded by cnpg above) +##@ PostgreSQL HA (retired — CNPG superseded this) -pg-deploy: ## Deploy / upgrade standalone PostgreSQL HA to platform namespace - $(KUBECTL) create namespace $(NAMESPACE) --dry-run=client -o yaml | $(KUBECTL) apply -f - - $(HELM) repo add bitnami https://charts.bitnami.com/bitnami --force-update - $(HELM) upgrade --install postgresql-ha bitnami/postgresql-ha \ - --version $(PG_CHART_VERSION) \ - --namespace $(NAMESPACE) \ - -f <(sops -d helm/postgresql-ha-values.sops.yaml) \ - --wait --timeout 5m +pg-deploy: ## Retired: refuse to deploy Bitnami postgresql-ha + @echo 'pg-deploy is retired (RAILIANCE-WP-0016 item 14). Use apps-pg / forgejo-db / rapp-postgres.' >&2; exit 2 -pg-status: ## Check PostgreSQL HA pod status - $(KUBECTL) get pods -n $(NAMESPACE) -l app.kubernetes.io/name=postgresql-ha - -pg-pgpool-check: ## Verify pgpool-password secret key is present (see RAIL-BS-WP-0003) - @SECRET=$$($(KUBECTL) get secret -n $(NAMESPACE) postgresql-ha-postgresql \ - -o jsonpath='{.data.pgpool-password}' 2>/dev/null); \ - if [ -z "$$SECRET" ]; then \ - echo "ERROR: pgpool-password key missing from secret — pgpool will CrashLoop on restart"; \ - exit 1; \ +pg-status: ## Confirm Bitnami postgresql-ha is absent + @if $(KUBECTL) get ns $(NAMESPACE) >/dev/null 2>&1; then \ + $(KUBECTL) get pods -n $(NAMESPACE) -l app.kubernetes.io/name=postgresql-ha; \ else \ - echo "OK: pgpool-password key present"; \ + echo 'postgresql-ha retired: namespace $(NAMESPACE) is absent'; \ fi -##@ Valkey (cache) +pg-pgpool-check: ## Retired: no pgpool secret to check + @echo 'pg-pgpool-check is retired with postgresql-ha (RAILIANCE-WP-0016 item 14).' >&2; exit 2 -valkey-deploy: ## Deploy / upgrade Valkey (Redis-compatible) to platform namespace - $(KUBECTL) create namespace $(NAMESPACE) --dry-run=client -o yaml | $(KUBECTL) apply -f - - $(HELM) upgrade --install valkey bitnami/valkey \ - --namespace $(NAMESPACE) \ - -f <(sops -d helm/valkey-values.sops.yaml) \ - --wait --timeout 3m +##@ Valkey (cache) — declared, not deployed -valkey-status: ## Check Valkey pod status - $(KUBECTL) get pods -n $(NAMESPACE) -l app.kubernetes.io/name=valkey +valkey-deploy: ## Gated: do not deploy Valkey until a consumer rapp exists + @echo 'valkey-deploy is gated (RAILIANCE-WP-0016 item 14): no live consumer, nothing to extract from S2.' >&2; exit 2 + +valkey-status: ## Confirm Valkey is absent + @if $(KUBECTL) get ns $(NAMESPACE) >/dev/null 2>&1; then \ + $(KUBECTL) get pods -n $(NAMESPACE) -l app.kubernetes.io/name=valkey; \ + else \ + echo 'valkey undeployed: namespace $(NAMESPACE) is absent'; \ + fi ##@ OpenBao (secrets) diff --git a/SCOPE.md b/SCOPE.md index 24ed9fd..31a605c 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -21,11 +21,10 @@ Stack dimension: the platform services that multiple applications share. Five repos cover S1–S5; other `railiance-*` repos sit on other OAS dimensions rather than on the stack. See `ArchitectureBlueprint.md` for the full backbone. -The active migration is from Bitnami postgresql-ha (repmgr + pgpool, deployed -historically as part of the Gitea subchart in S2 — the forge itself is now -Forgejo) to CloudNative PG (cnpg operator, already deployed in the cnpg-system -namespace) as the canonical database operator. Valkey cluster is also in scope -for S3 extraction from S2. +The database operator is CloudNative PG. Bitnami postgresql-ha is retired +(no live release; `make pg-deploy` fail-closed, `RAILIANCE-WP-0016` item 14). +Valkey is a declared capability with no live instance and nothing left in S2 +to extract; `make valkey-deploy` is gated until a consumer rapp exists. OpenBao is a platform capability in this repo, but not every OpenBao-related file belongs in the long-term S3 ownership home. The deployable package surface @@ -78,18 +77,18 @@ policy, and lane governance. ## Current State - Status: maintained / emerging -- Implementation: CloudNative PG operator (cnpg) deployed; `databases` namespace active; OpenBao is live as the S3 secrets service; Valkey + legacy postgresql-ha extraction from S2 remain in progress -- Stability: emerging — cnpg deployed but database cluster definitions not yet migrated from S2 -- Usage: shared database, cache, and secrets layer; cnpg-system, databases, and openbao namespaces are live +- Implementation: CloudNative PG operator live; `databases` namespace holds the platform clusters; OpenBao is the S3 secrets service. Bitnami postgresql-ha is retired. Valkey is undeployed. +- Stability: emerging — CNPG clusters are live; cache and in-cluster object storage are not +- Usage: shared database and secrets layer; cnpg-system, databases, and openbao namespaces are live - Deploys via ArgoCD: four Applications (`external-secrets`, `issue-core`, `openbao-secretstore`, `target-revenue`) plus AppProjects under `argocd/bootstrap/`; see `docs/argocd-gitops.md` - Emits to `railiance-telemetry` (Q2 Observability) once the evidence plane exists — seeded 2026-08-11, not yet implemented -- Open work: Valkey and legacy postgresql-ha extraction remain active; the - OpenBao package boundary and PAT consumer cutover are now documented and - closed; `rapp-openbao`/`rapp-postgres` declaration conformance - (`RAILIANCE-WP-0015-T02`) is held pending the `railiance-master` schema +- Open work: Valkey remains a declared-but-unbuilt cache; MinIO is likewise + declared, not deployed. OpenBao package boundary and PAT cutover are + closed. Platform rapp declarations conform (`RAILIANCE-WP-0015`). + Versioned consumer interfaces: `docs/s3-consumer-interfaces.md`. - Known drift: State Hub attributes ~11 capabilities to this repo, including S1/S2/S4/S5 concerns it does not own. The four `capability` blocks in this file are authoritative; the hub carries stale pre-split attributions @@ -138,7 +137,7 @@ keywords: [postgresql, postgres, cnpg, cloudnative-pg, operator, database, kuber ```capability type: infrastructure title: Valkey / Redis-compatible cache -description: Shared Redis-compatible cache service (Valkey) for all applications in the Railiance stack. +description: Declared shared Redis-compatible cache. Not deployed on railiance01 as of 2026-08-15; no S2 instance remains to extract. keywords: [valkey, redis, cache, shared, session, queue] ``` diff --git a/docs/evidence/RAILIANCE-WP-0016-item13-restore-2026-08-15.md b/docs/evidence/RAILIANCE-WP-0016-item13-restore-2026-08-15.md new file mode 100644 index 0000000..604ba44 --- /dev/null +++ b/docs/evidence/RAILIANCE-WP-0016-item13-restore-2026-08-15.md @@ -0,0 +1,29 @@ +# Item 13 — restore proven end to end + +Work record: architecture-cleanup `RAILIANCE-WP-0016` item 13 +Date: 2026-08-15 +Status: **done** (unblocked by `RESOURCE-WP-0002` finish) + +The blocker was decision `9c21c0e0` / `RESOURCE-WP-0002`. That workplan is +finished. The restore this item asked for was executed as +`RESOURCE-WP-0002-T05` on 2026-08-14 and is still true on the live cluster. + +## Production path (railiance01 `platform-pg`) + +| Fact | Value | +| --- | --- | +| Destination | `s3://railiance-platform-pg-backup/platform-pg/` | +| Endpoint | `https://s3.nl-ams.scw.cloud` | +| Credential Secret | `databases/platform-pg-backup-s3` (keys present, values not read) | +| ScheduledBackup | `platform-pg-daily` (02:15 UTC) | +| Completed Backups | `platform-pg-daily-20260814181421`, `platform-pg-daily-20260815021500` | + +Full restore 65s (`audit_core.events` 30=30). PITR 65s (`keep` present, +`destroy` absent). Evidence: +`resource-control/docs/evidence/RESOURCE-WP-0002-T05-restore-2026-08-14.md`. +Logical offsite copy: `RESOURCE-WP-0002-T06`. + +This item is the S3-owned *proof*, not a second restore program. `apps-pg` +is still local-path only (RAILIANCE-WP-0016 apps-pg evidence T02). Covering +that cluster is a later consumer of the same bucket, not a re-opening of +item 13. diff --git a/docs/postgresql-ha.md b/docs/postgresql-ha.md index 0dfe978..ea6ef1f 100644 --- a/docs/postgresql-ha.md +++ b/docs/postgresql-ha.md @@ -1,9 +1,16 @@ -# PostgreSQL HA — Platform Service +# PostgreSQL HA — retired -**Chart:** `bitnami/postgresql-ha` -**Namespace:** `platform` -**Managed by:** `railiance-platform` (S3) -**Workplan:** `RAIL-PL-WP-0001` +**Status:** retired 2026-08-15 (`RAILIANCE-WP-0016` item 14) +**Superseded by:** CloudNativePG clusters in `databases` (`apps-pg`, +`forgejo-db`, `platform-pg`, and the other CNPG instances). +**Do not deploy.** `make pg-deploy` is fail-closed. + +The Bitnami `postgresql-ha` (repmgr + pgpool) stack never ran as a live +S3 service on railiance01. There is no `platform` namespace and no +matching pods. Gitea’s historical subchart in S2 was replaced by +`forgejo-db`. Keep this file as the retirement record and the +pgpool-password incident note. New databases go through `docs/apps-pg.md` +or `rapp-postgres`. --- diff --git a/docs/s3-consumer-interfaces.md b/docs/s3-consumer-interfaces.md new file mode 100644 index 0000000..76b5ea4 --- /dev/null +++ b/docs/s3-consumer-interfaces.md @@ -0,0 +1,53 @@ +# S3 consumer service interfaces + +Work record: `RAILIANCE-WP-0016` item 17 +Date: 2026-08-15 +Contract version of this index: `1.0.0` + +Consumers bind to these contracts, not to Helm chart names or pod topology. +Changing an engine behind a versioned interface does not bump the interface +unless the consumer-visible fields change. + +| Interface | Version | Who it is for | Authority | +| --- | --- | --- | --- | +| Shared application database | `apps-pg.v1` | S5 apps that need an isolated database on `apps-pg` | `docs/apps-pg.md` | +| Isolated platform database | `rapp-postgres.v1` | Platform services that need a CNPG consumer on `platform-pg` | `rapp-postgres/declarations/rapp.yaml` `consumer_contract` | +| Standing runtime secret | `ccr-workload-kv.v1` | Workloads that need a KV path via External Secrets | `schemas/credential-change-request.schema.yaml` | +| Short-lived lease | `credential-grant.v1` | Operators/agents and dynamic DB roles | `credential-grants/catalog.yaml` | +| Rapp credential bind | `rapp-lane.v1` | How a rapp names the two lanes above | `docs/rapp-credential-lane-binding.md` | + +## Compatibility rules + +- A `vN` document may add optional fields. Removing or renaming a + required field is `vN+1`. +- Values never appear in an interface document. Paths, grant ids, CCR + ids, and role names are the contract. +- `make pg-deploy` / Bitnami DSN + `postgresql-ha-pgpool.platform.svc.cluster.local` is **not** an + interface. That stack is retired. + +## `apps-pg.v1` + +Consumer supplies: app name, namespace, database name, role name. Platform +provisions the role and CNPG `Database`, labels the namespace +`railiance.io/postgres-client=apps-pg`, and publishes a runtime Secret. +RW endpoint: `apps-pg-rw.databases.svc.cluster.local:5432`. Bootstrap +role `apps_admin` is not a consumer credential. + +## `rapp-postgres.v1` + +Consumer is a `consumers[].declaration`. Boundary unit is +`database-per-consumer`. Credential type is +`openbao-dynamic-database-credential` via grant +`rapp-postgres/-`. Tenant identifiers are opaque. + +## `ccr-workload-kv.v1` and `credential-grant.v1` + +Unchanged semantics from `docs/credential-change-approval.md` and +`docs/credential-broker.md`. The version name exists so a rapp can cite +the lane without restating TTL, delivery, or revocation rules. + +## Not yet an interface + +Valkey and in-cluster MinIO have capability blocks in `SCOPE.md` and no +live service. They do not receive a `v1` until a consumer exists. diff --git a/workplans/RAILIANCE-WP-0016-architecture-cleanup-backlog.md b/workplans/RAILIANCE-WP-0016-architecture-cleanup-backlog.md index 1e42795..c7298da 100644 --- a/workplans/RAILIANCE-WP-0016-architecture-cleanup-backlog.md +++ b/workplans/RAILIANCE-WP-0016-architecture-cleanup-backlog.md @@ -4,11 +4,11 @@ type: workplan title: "Architecture cleanup backlog, WSJF-prioritized" domain: financials repo: railiance-platform -status: active +status: finished owner: codex topic_slug: railiance created: "2026-08-12" -updated: "2026-08-12" +updated: "2026-08-15" related_repos: - railiance-master - railiance-infra @@ -67,11 +67,11 @@ reliable numbers here, because nobody has scoped them yet. | 10b | Schedule verification; route failures where they are seen | `railiance-infra` / Q7 owner | 7 | 6 | 8 | 3 | **7.0** | | 11 | Consolidate the bootstrap path; sort inherited S2 content | `railiance-bootstrap` | 5 | 3 | 4 | 4 | **3.0** | | 12 | Forge migration: `rapp-forgejo` + S4 layer transfer | `railiance-forge` / S4 | 6 | 4 | 4 | 5 | **2.8** | -| 13 | Prove a restore end to end | `railiance-platform` | 9 | 5 | 9 | 8 | **2.9** | -| 14 | Valkey extraction + legacy postgresql-ha retirement | `railiance-platform` | 5 | 4 | 4 | 5 | **2.6** | +| ~~13~~ | ~~Prove a restore end to end~~ **DONE 2026-08-15** | `railiance-platform` | 9 | 5 | 9 | 8 | **2.9** | +| ~~14~~ | ~~Valkey extraction + legacy postgresql-ha retirement~~ **DONE 2026-08-15** | `railiance-platform` | 5 | 4 | 4 | 5 | **2.6** | | 15 | Sweep remaining Gitea prose (~31 files each) | `railiance-forge`, `railiance-apps` | 3 | 2 | 2 | 3 | **2.3** | | 16 | Declare the Q6 cost boundary with `resource-control`/`fin-hub` | `railiance-master` | 4 | 3 | 3 | 4 | **2.5** | -| 17 | Version S3 consumer service interfaces | `railiance-platform` | 6 | 3 | 5 | 6 | **2.3** | +| ~~17~~ | ~~Version S3 consumer service interfaces~~ **DONE 2026-08-15** | `railiance-platform` | 6 | 3 | 5 | 6 | **2.3** | | 18 | Resolve the C4 ecosystem-modelling overlap | `railiance-master` / `railiance-fabric` | 4 | 3 | 4 | 5 | **2.2** | | 19 | S4 build-out: pipeline templates, SDKs, portal | `railiance-enablement` | 7 | 3 | 4 | 9 | **1.6** | @@ -94,9 +94,10 @@ conformance checker already exists in `railiance-infra` and has simply not run since 2026-03-09. **Slice 10a is now the highest-value item after the ratifications** at WSJF 12.0. -**Item 13 is blocked, not deprioritized.** Proving a restore needs the object -storage decision (`9c21c0e0`), which is escalated and awaiting human approval on -commercial terms. Its WSJF understates it; treat the blocker as the real item. +**Item 13 is done, not blocked.** `RESOURCE-WP-0002` finished 2026-08-15. +Barman is live on `platform-pg`; T05 recorded full restore and PITR on +2026-08-14; daily backups have completed since. Pointer: +`docs/evidence/RAILIANCE-WP-0016-item13-restore-2026-08-15.md`. **Item 19 is last and that is deliberate.** S4 build-out is the widest gap in the family, but it is large, and item 12 gives S4 a smaller first @@ -270,7 +271,7 @@ it. ```task id: RAILIANCE-WP-0016-T05 -status: todo +status: done priority: medium state_hub_task_id: "a40faaf1-77db-43f5-a774-6ab3866d8e1d" ``` @@ -278,7 +279,15 @@ state_hub_task_id: "a40faaf1-77db-43f5-a774-6ab3866d8e1d" Execute the S3-owned items in WSJF order — 13 (blocked on `9c21c0e0`), 14, 17 — and keep this table current as items complete or their sizes become known. +**Done 2026-08-15.** Item 13 unblocked by `RESOURCE-WP-0002` finish: restore +already proven as T05 there; live `platform-pg-daily` backups completing. +Item 14: Bitnami postgresql-ha deploy path fail-closed (never live on +railiance01); Valkey has no S2 instance to extract and no consumer, so +deploy is gated. Item 17: `docs/s3-consumer-interfaces.md` v1.0.0. + ## Outcome -Pending. T01-T03 delivered; routed to `railiance-master` for adoption -(message `ac73b172`). T04 rescoping and T05 S3-owned execution remain. +T01–T05 done. Routed to `railiance-master` (message `ac73b172`); master +took items 6 and 8 as WP-0021 and declined the rest. S3-owned items 13, +14 and 17 executed 2026-08-15 after the object-store purchase landed. +Workplan finished. Remaining backlog rows belong to other repos.