Pin apps-pg targets to railiance01 by cluster identity; seed RPF-WP-0020
Two reachable clusters each carry a CNPG Cluster named apps-pg in a namespace named databases. KUBECONFIG is an environment variable, so the Makefile ?= default never applied, and RAILIANCE01_KUBECONFIG pointed at config-hosteurope - a different cluster. Had the environment pointed at the other reachable cluster instead of an unauthorized one, make apps-pg-deploy would have applied RPF-WP-0019 connection limits, role timeouts and backup config to the wrong cluster and reported success. The Unauthorized error was the only thing that prevented it. Filename selection cannot protect against this: both kubeconfigs resolve to a 127.0.0.1 tunnel port and the environment wins either way. railiance01-guard pins identity instead, comparing the live kube-system namespace UID against RAILIANCE01_CLUSTER_UID, and fails closed on mismatch or unreachability. It gates apps-pg deploy, backup-deploy, overflow-dry-run, status and shell. Verified refusing on the wrong cluster, refusing when unreachable, and passing on railiance01. Not global: db-status legitimately targets the other cluster for gitea-db. RPF-WP-0019 blocker note corrected - the cluster was never unreachable, our wiring was wrong. RPF-WP-0020 seeded for the pre-existing CCR test failure, which is two unrelated problems: CCR-2026-0010 is an active lane missing its whole openbao.auth block, and CCR-2026-0011 is an honest in-flight draft the suite has no way to express. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
1cbde550b5
commit
b1f973c2d5
4 changed files with 227 additions and 13 deletions
51
Makefile
51
Makefile
|
|
@ -1,9 +1,25 @@
|
|||
SHELL := /usr/bin/env bash
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
KUBECONFIG ?= $(firstword $(wildcard $(HOME)/.kube/config) $(HOME)/.kube/config-hosteurope)
|
||||
KUBECONFIG ?= $(firstword $(wildcard $(HOME)/.kube/config-railiance01) $(wildcard $(HOME)/.kube/config))
|
||||
KUBECTL_BIN ?= $(firstword $(shell command -v kubectl 2>/dev/null) $(wildcard $(HOME)/.local/bin/kubectl) kubectl)
|
||||
KUBECTL := $(KUBECTL_BIN) --kubeconfig=$(KUBECONFIG)
|
||||
|
||||
# Cluster identity pin (RPF-WP-0019).
|
||||
#
|
||||
# Two reachable clusters each carry a CNPG Cluster named `apps-pg` in a
|
||||
# namespace named `databases`. Selecting by kubeconfig FILENAME does not
|
||||
# protect against applying to the wrong one: KUBECONFIG is an environment
|
||||
# variable, `?=` does not override it, and both files resolve to a
|
||||
# 127.0.0.1 tunnel port. So the guard pins cluster IDENTITY instead —
|
||||
# the kube-system namespace UID, which is unique per cluster and stable
|
||||
# for its lifetime.
|
||||
#
|
||||
# railiance01 (k3s v1.35.1) carries platform-pg, forgejo-db and both
|
||||
# apps-pg consumers. The other cluster carries gitea-db and only one
|
||||
# apps-pg consumer; `db-status` legitimately targets it, which is why the
|
||||
# guard is applied per-target rather than globally.
|
||||
RAILIANCE01_CLUSTER_UID ?= a553c742-0115-43d4-99a4-a5ca56fe0786
|
||||
HELM := helm --kubeconfig=$(KUBECONFIG)
|
||||
NAMESPACE := platform
|
||||
|
||||
|
|
@ -82,10 +98,31 @@ consumption-preflight: ## Refuse a new order that exceeds a restricted entity's
|
|||
|
||||
##@ Shared apps-pg (S5 application databases)
|
||||
|
||||
cluster-id: ## Print the kube-system UID of the cluster KUBECONFIG currently selects
|
||||
@echo "kubeconfig: $(KUBECONFIG)"
|
||||
@echo "cluster-uid: $$($(KUBECTL) get ns kube-system -o jsonpath='{.metadata.uid}' 2>/dev/null || echo UNREACHABLE)"
|
||||
|
||||
railiance01-guard: ## Fail closed unless KUBECONFIG selects railiance01
|
||||
@uid=$$($(KUBECTL) get ns kube-system -o jsonpath='{.metadata.uid}' 2>/dev/null); \
|
||||
if [ -z "$$uid" ]; then \
|
||||
echo "railiance01-guard: cluster unreachable via $(KUBECONFIG)" >&2; \
|
||||
echo " hint: unset KUBECONFIG, or set it to ~/.kube/config-railiance01" >&2; \
|
||||
exit 2; \
|
||||
fi; \
|
||||
if [ "$$uid" != "$(RAILIANCE01_CLUSTER_UID)" ]; then \
|
||||
echo "railiance01-guard: REFUSING - wrong cluster" >&2; \
|
||||
echo " kubeconfig: $(KUBECONFIG)" >&2; \
|
||||
echo " expected: $(RAILIANCE01_CLUSTER_UID)" >&2; \
|
||||
echo " found: $$uid" >&2; \
|
||||
echo " both clusters carry apps-pg in namespace databases; applying here would hit the wrong one." >&2; \
|
||||
exit 2; \
|
||||
fi; \
|
||||
echo "railiance01-guard: ok ($$uid)"
|
||||
|
||||
apps-pg-verify-capacity: ## Verify cell ceiling, role limits and distinct backup prefixes
|
||||
python3 tools/verify_apps_pg_capacity.py helm/apps-pg-cluster.yaml helm/apps-pg-2-cluster.yaml
|
||||
|
||||
apps-pg-deploy: apps-pg-verify-capacity ## Apply shared apps-pg cnpg Cluster + NetworkPolicies
|
||||
apps-pg-deploy: railiance01-guard apps-pg-verify-capacity ## Apply shared apps-pg cnpg Cluster + NetworkPolicies
|
||||
@if [ -n "$(CONSUMING_ENTITY)" ]; then \
|
||||
$(MAKE) consumption-preflight CONSUMING_ENTITY='$(CONSUMING_ENTITY)' \
|
||||
ESTIMATE_EUR='$(ESTIMATE_EUR)' CONSUMPTION_CLASS=new-order; \
|
||||
|
|
@ -93,11 +130,11 @@ apps-pg-deploy: apps-pg-verify-capacity ## Apply shared apps-pg cnpg Cluster + N
|
|||
$(KUBECTL) apply -f helm/apps-pg-cluster.yaml
|
||||
$(KUBECTL) apply -f helm/apps-pg-networkpolicies.yaml
|
||||
|
||||
apps-pg-backup-deploy: ## Apply the daily backup after the governed S3 Secret exists
|
||||
apps-pg-backup-deploy: railiance01-guard ## Apply the daily backup after the governed S3 Secret exists
|
||||
@$(KUBECTL) get secret platform-pg-backup-s3 -n databases >/dev/null || { echo 'missing governed platform-pg-backup-s3 secret' >&2; exit 2; }
|
||||
$(KUBECTL) apply -f helm/apps-pg-backup.yaml
|
||||
|
||||
apps-pg-overflow-dry-run: apps-pg-verify-capacity ## Server-validate the unapplied overflow cell
|
||||
apps-pg-overflow-dry-run: railiance01-guard apps-pg-verify-capacity ## Server-validate the unapplied overflow cell
|
||||
@$(KUBECTL) get secret apps-pg-2-credentials -n databases >/dev/null || { echo 'missing distinct apps-pg-2-credentials secret' >&2; exit 2; }
|
||||
$(KUBECTL) apply --dry-run=server -f helm/apps-pg-2-cluster.yaml
|
||||
$(KUBECTL) apply --dry-run=server -f helm/apps-pg-2-networkpolicies.yaml
|
||||
|
|
@ -108,11 +145,11 @@ apps-pg-overflow-deploy: apps-pg-overflow-dry-run ## Provision only for an appro
|
|||
$(KUBECTL) apply -f helm/apps-pg-2-networkpolicies.yaml
|
||||
$(KUBECTL) apply -f helm/apps-pg-2-backup.yaml
|
||||
|
||||
apps-pg-status: ## Show apps-pg cnpg cluster health
|
||||
apps-pg-status: railiance01-guard ## Show apps-pg cnpg cluster health
|
||||
$(KUBECTL) cnpg status apps-pg -n databases 2>/dev/null || \
|
||||
$(KUBECTL) get cluster apps-pg -n databases -o wide
|
||||
|
||||
apps-pg-shell: ## Open psql shell on apps-pg primary as apps_admin / apps_meta
|
||||
apps-pg-shell: railiance01-guard ## Open psql shell on apps-pg primary as apps_admin / apps_meta
|
||||
$(KUBECTL) cnpg psql apps-pg -n databases -- -U apps_admin apps_meta 2>/dev/null || \
|
||||
$(KUBECTL) exec -it -n databases apps-pg-1 -- psql -U apps_admin apps_meta
|
||||
|
||||
|
|
@ -383,7 +420,7 @@ argocd-status: ## Show Railiance ArgoCD projects, root app, and registered repos
|
|||
|
||||
##@ Backup
|
||||
|
||||
RAILIANCE01_KUBECONFIG ?= $(HOME)/.kube/config-hosteurope
|
||||
RAILIANCE01_KUBECONFIG ?= $(HOME)/.kube/config-railiance01
|
||||
|
||||
backup: ## Backup platform services (PostgreSQL logical dump) — age-encrypted to Nextcloud
|
||||
@test -x tools/cmd/railiance-backup || { echo "tools/cmd/railiance-backup not installed; use forgejo-backup for Forgejo." >&2; exit 1; }
|
||||
|
|
|
|||
|
|
@ -18,6 +18,30 @@ databases. It lives in the `databases` namespace and is owned by
|
|||
into application namespaces, use it in S5 runtime configuration, or treat
|
||||
it as a consumer credential.
|
||||
|
||||
## Which cluster this is
|
||||
|
||||
**Two reachable clusters each carry a CNPG `Cluster` named `apps-pg` in a
|
||||
namespace named `databases`.** The one this document describes is
|
||||
**railiance01** (k3s v1.35.1) — it also carries `platform-pg` and `forgejo-db`,
|
||||
and holds both apps-pg consumers (`vergabe_db`, `coulomb_social_db`). The other
|
||||
cluster carries `gitea-db` and only one apps-pg consumer.
|
||||
|
||||
Selecting the right one by kubeconfig filename is not safe: `KUBECONFIG` is an
|
||||
environment variable that overrides the Makefile default, and both files
|
||||
resolve to a `127.0.0.1` tunnel port. Every `apps-pg-*` target therefore runs
|
||||
`railiance01-guard` first, which compares the live `kube-system` namespace UID
|
||||
against `RAILIANCE01_CLUSTER_UID` and fails closed on mismatch or
|
||||
unreachability.
|
||||
|
||||
```bash
|
||||
make cluster-id # what KUBECONFIG currently selects
|
||||
make railiance01-guard # assert it is railiance01, or refuse
|
||||
```
|
||||
|
||||
If a target refuses, the fix is `KUBECONFIG=~/.kube/config-railiance01`, not
|
||||
`--force` and not editing the pinned UID. `db-status` deliberately targets the
|
||||
other cluster for `gitea-db`, which is why the guard is per-target.
|
||||
|
||||
## Consumer Onboarding
|
||||
|
||||
Each S5 application gets its own role, database, and runtime Secret. The
|
||||
|
|
|
|||
|
|
@ -85,12 +85,36 @@ against a live shared rail, in this order:
|
|||
already.
|
||||
5. T04's probes, in an announced window, after 1–3 have settled.
|
||||
|
||||
**Blocker as of this session:** the cluster is unreachable from the
|
||||
workstation — `kubectl` returns `Unauthorized` against
|
||||
`config-hosteurope`. Credentials or the tunnel need attention before any of
|
||||
the above runs. This is a session-local access problem, not a defect in the
|
||||
manifests: `make apps-pg-verify-capacity` passes and the capacity tests are
|
||||
green.
|
||||
**Blocker resolved 2026-08-18, and it was ours.** The earlier note recorded
|
||||
this as `kubectl` returning `Unauthorized` — an access problem outside the
|
||||
repo. That was wrong. railiance01 is reachable and healthy
|
||||
(`~/.kube/config-railiance01`, k3s v1.35.1, `apps-pg` 9d, both consumers
|
||||
present). What failed was our own wiring: `KUBECONFIG` is an environment
|
||||
variable, so the Makefile's `?=` default never applied, and
|
||||
`RAILIANCE01_KUBECONFIG` pointed at `config-hosteurope` — a different cluster
|
||||
that happens to be unauthorized from here.
|
||||
|
||||
**The near-miss is the finding.** Two reachable clusters each carry a CNPG
|
||||
`Cluster` named `apps-pg` in a namespace named `databases`. The other one
|
||||
(k3s v1.30.3) holds `gitea-db` and only one apps-pg consumer. Had `KUBECONFIG`
|
||||
pointed there instead of at an unauthorized file, `make apps-pg-deploy` would
|
||||
have applied this workplan's connection limits, role timeouts and backup
|
||||
configuration **to the wrong cluster, and reported success.** The
|
||||
`Unauthorized` error was the only thing that prevented it.
|
||||
|
||||
Fixed by pinning cluster *identity* rather than kubeconfig *filename*:
|
||||
`railiance01-guard` compares the live `kube-system` namespace UID against
|
||||
`RAILIANCE01_CLUSTER_UID` and fails closed on mismatch or unreachability. It
|
||||
gates `apps-pg-deploy`, `apps-pg-backup-deploy`, `apps-pg-overflow-dry-run`,
|
||||
`apps-pg-status` and `apps-pg-shell`. Filename selection could not have
|
||||
protected against this: both kubeconfigs resolve to a `127.0.0.1` tunnel port,
|
||||
and the environment overrides the default either way. `make cluster-id` prints
|
||||
what is currently selected.
|
||||
|
||||
The guard is deliberately **not** global. `db-status` legitimately targets the
|
||||
other cluster for `gitea-db`, so a blanket guard would break a working target
|
||||
and teach people to bypass it.
|
||||
|
||||
|
||||
**Do not treat the rollout as evidence.** T04's P1 claim and the R-axis both
|
||||
need artifacts produced *after* application, and `docs/placement-policy.md`
|
||||
|
|
|
|||
129
workplans/RPF-WP-0020-ccr-schema-drift.md
Normal file
129
workplans/RPF-WP-0020-ccr-schema-drift.md
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
---
|
||||
id: RPF-WP-0020
|
||||
type: workplan
|
||||
title: "Close CCR schema drift: one active lane unmigrated, one draft the suite cannot express"
|
||||
domain: financials
|
||||
repo: railiance-platform
|
||||
status: proposed
|
||||
owner: codex
|
||||
topic_slug: railiance
|
||||
created: "2026-08-18"
|
||||
updated: "2026-08-18"
|
||||
related:
|
||||
- RPF-WP-0014
|
||||
origin: residual
|
||||
origin_ref: RPF-WP-0019
|
||||
---
|
||||
|
||||
# RPF-WP-0020 — CCR schema drift
|
||||
|
||||
## Goal
|
||||
|
||||
Make `tests/test_credential_change.py::test_all_repo_ccrs_validate` pass for
|
||||
the right reason: because every credential-change request in the repo is
|
||||
either valid or explicitly declared in-flight — not because the assertion was
|
||||
loosened until it stopped complaining.
|
||||
|
||||
## Why this is its own workplan
|
||||
|
||||
The test has been failing on `main`. It was found while closing
|
||||
`RPF-WP-0019` and confirmed to predate that work (it fails at `HEAD` with the
|
||||
apps-pg changes stashed), so it is neither a regression from that workplan nor
|
||||
something to fix inside it.
|
||||
|
||||
A single red test is hiding **two unrelated problems with opposite remedies**.
|
||||
That is the reason to split them out and name them, rather than fix whichever
|
||||
one makes the suite green first.
|
||||
|
||||
## The two problems
|
||||
|
||||
**P1 — an active credential lane is unmigrated.**
|
||||
`credential-change-requests/CCR-2026-0010-email-connect-transactional.yaml`
|
||||
carries `status: active` and `readiness: ready`, and is missing the entire
|
||||
`openbao.auth` block: `method`, `mount`, `role`, `policies`, `bound_claims`.
|
||||
The validator gained those requirements and this CCR was never brought
|
||||
forward.
|
||||
|
||||
This is the one that matters. A live lane whose declaration does not describe
|
||||
how the workload authenticates is a governance gap, not a lint failure — the
|
||||
document that is supposed to be the authority on the lane cannot answer the
|
||||
first question anyone would ask of it. The lane itself is presumably working,
|
||||
which is exactly what makes it easy to leave.
|
||||
|
||||
**P2 — a genuine in-flight draft the suite cannot express.**
|
||||
`CCR-2026-0011-scaleway-object-storage-bootstrap.yaml` carries
|
||||
`status: apply_pending` and `readiness: waiting-on-ui-replace-of-xxx-placeholders`.
|
||||
It is a founder-bootstrap credential still holding placeholder values, and
|
||||
`ops-warden` already tracks it as a draft lane. Its errors include a
|
||||
`readiness` value outside the permitted enum, which is the file honestly
|
||||
reporting that it is not finished.
|
||||
|
||||
This one is not a defect. The defect is that a suite asserting *all* CCRs
|
||||
validate has no way to say "this one is deliberately incomplete", so an honest
|
||||
draft and a real gap produce identical output.
|
||||
|
||||
## Boundaries
|
||||
|
||||
- **No secret values are read, written or rotated.** This is declaration
|
||||
hygiene. Anything requiring a mint, a rotation or an apply belongs to the
|
||||
credential-change approval flow in `docs/credential-change-approval.md`.
|
||||
- P2 is not closed by filling in placeholders. The Scaleway bootstrap key is
|
||||
a live commercial credential with an owner outside this repo.
|
||||
- The validator's rules are not relaxed to accommodate either file.
|
||||
|
||||
## Tasks
|
||||
|
||||
```task
|
||||
id: RPF-WP-0020-T01
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
**Migrate CCR-2026-0010 to the current schema.** Determine the actual
|
||||
authentication path for the `email-connect` transactional lane — Kubernetes
|
||||
auth mount, role, bound claims, and the policy name that must match
|
||||
`openbao.policy_name` — from the live OpenBao configuration and the existing
|
||||
policy file, not by inventing plausible values. If the live configuration and
|
||||
the declaration disagree, the live configuration is the fact and the
|
||||
disagreement is the finding.
|
||||
|
||||
```task
|
||||
id: RPF-WP-0020-T02
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
**Give the suite a way to express a draft.** Add an explicit in-flight state
|
||||
so `CCR-2026-0011` is skipped *by declaration* rather than by exception list —
|
||||
a `status` the validator recognises as not-yet-complete, with the test
|
||||
asserting that such files are still well-formed in every other respect. An
|
||||
allowlist of filenames would work today and rot on the next draft.
|
||||
|
||||
```task
|
||||
id: RPF-WP-0020-T03
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
**Report the drift rather than only fixing it.** If the `openbao.auth`
|
||||
requirement was added without migrating existing active CCRs, other repos
|
||||
carrying CCRs may have the same gap and no failing test to reveal it. Confirm
|
||||
whether the requirement originated here or upstream, and notify accordingly.
|
||||
|
||||
```task
|
||||
id: RPF-WP-0020-T04
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
**Make the suite green and keep it that way.** With T01 and T02 done, the full
|
||||
suite passes. Record in `docs/credential-change-approval.md` that a new
|
||||
required field obliges a migration pass over existing active CCRs — the
|
||||
omission that produced P1.
|
||||
|
||||
## Risks
|
||||
|
||||
**T01 invents values to make the test pass.** The likeliest failure and the
|
||||
worst one: a declaration that is well-formed and wrong is more dangerous than
|
||||
one that is visibly incomplete, because it stops anyone looking. Mitigation is
|
||||
that T01 reads live configuration and treats disagreement as a finding.
|
||||
|
||||
**T02 becomes a way to silence future failures.** Mitigation is that the
|
||||
in-flight state still requires the file to be well-formed in every other
|
||||
respect, so it suppresses the completeness assertion and nothing else.
|
||||
Loading…
Add table
Add a link
Reference in a new issue