RAILIANCE-WP-0012/0013/0014: Core Hub S5 release, CNPG observability, and follow-up workplans
Deliver the inbox-suggestion implementation from WP-0012: Core Hub Helm chart and Makefile targets, CNPG backup status tooling, and updated backup handoff docs. Archive the finished WP-0012 workplan and register ready follow-ups WP-0013 (CNPG backup wiring + restore drill) and WP-0014 (Core Hub Helm cutover and vergabe-teilnahme image refresh).
This commit is contained in:
parent
85f81714f1
commit
df7225dd3e
19 changed files with 1025 additions and 2 deletions
103
workplans/RAILIANCE-WP-0013-cnpg-backup-wiring-restore-drill.md
Normal file
103
workplans/RAILIANCE-WP-0013-cnpg-backup-wiring-restore-drill.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
---
|
||||
id: RAILIANCE-WP-0013
|
||||
type: workplan
|
||||
title: "CNPG backup wiring and restore drill"
|
||||
domain: financials
|
||||
repo: railiance-apps
|
||||
status: ready
|
||||
owner: codex
|
||||
topic_slug: railiance
|
||||
created: "2026-07-10"
|
||||
updated: "2026-07-10"
|
||||
state_hub_workstream_id: "764c2447-0713-45d5-a14d-810a13027b8b"
|
||||
---
|
||||
|
||||
# CNPG backup wiring and restore drill
|
||||
|
||||
Follow-up to `RAILIANCE-WP-0012-T04` and custodian inbox `a1957cce`. Observability
|
||||
and manifest templates exist; production clusters on railiance01 still report
|
||||
degraded backup posture via `make cnpg-backup-status`.
|
||||
|
||||
Evidence baseline:
|
||||
`the-custodian/docs/evidence/vergabe-teilnahme-delivery-lane-20260710.json`
|
||||
(`s4_2_backup_restore: FAIL`).
|
||||
|
||||
Coordinate with `railiance-platform` for object-store credentials and retention.
|
||||
Phase 1 may use logical `pg_dump` for some clusters per
|
||||
`disaster-control/BackupPickupQueue.md` item 6; barman `ScheduledBackup` remains
|
||||
the longer-term target in `manifests/cnpg-backup-readiness.yaml`.
|
||||
|
||||
## Confirm backup lane and credential path
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0013-T01
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "f2051213-0a9f-418e-82ad-9bd7c79fd0dc"
|
||||
```
|
||||
|
||||
Record the approved backup mechanism per cluster: Phase 1 logical dump vs barman
|
||||
`ObjectStore` + `ScheduledBackup`. Identify OpenBao/operator paths for
|
||||
`cnpg-backup-s3` or equivalent without copying secret values into Git,
|
||||
workplans, or State Hub notes.
|
||||
|
||||
## Wire apps-pg backup coverage
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0013-T02
|
||||
status: wait
|
||||
priority: high
|
||||
state_hub_task_id: "2d78019e-f040-422d-89d0-0d0e8c6af718"
|
||||
```
|
||||
|
||||
Enable backup coverage for `apps-pg` first because `vergabe_db` is
|
||||
production-facing. Apply platform-approved manifests or Makefile-driven operator
|
||||
steps. `make cnpg-backup-status` must show a non-null backup spec or active
|
||||
`ScheduledBackup` for `apps-pg`.
|
||||
|
||||
Blocked on: `RAILIANCE-WP-0013-T01`.
|
||||
|
||||
## Wire remaining production DB clusters
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0013-T03
|
||||
status: wait
|
||||
priority: medium
|
||||
state_hub_task_id: "d3e41204-98a5-4ccb-ae91-526e7a606baa"
|
||||
```
|
||||
|
||||
Extend the same backup lane to `forgejo-db`, `net-kingdom-pg`, and
|
||||
`state-hub-db`. Re-run `make cnpg-backup-status` until all four clusters pass.
|
||||
|
||||
Blocked on: `RAILIANCE-WP-0013-T02`.
|
||||
|
||||
## Run isolated restore drill with evidence
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0013-T04
|
||||
status: wait
|
||||
priority: high
|
||||
state_hub_task_id: "15602bb9-d1f6-456e-9af3-bc7c27ebb36f"
|
||||
```
|
||||
|
||||
Restore the latest `apps-pg` backup into an isolated namespace or test database.
|
||||
Verify row counts, run `vergabe-teilnahme` migrations, and capture non-secret
|
||||
evidence (timestamp, cluster, backup id, restore namespace, post-restore smoke
|
||||
result). Do not store decrypted dumps or credentials in Git.
|
||||
|
||||
Blocked on: `RAILIANCE-WP-0013-T02`.
|
||||
|
||||
## Close vergabe production-trust backup gate
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0013-T05
|
||||
status: wait
|
||||
priority: medium
|
||||
state_hub_task_id: "666c0a9b-4f9b-4ea3-bf0b-14aa76b8ea93"
|
||||
```
|
||||
|
||||
Update `docs/app-data-backup-restore-handoff.md` and `docs/vergabe-teilnahme.md`
|
||||
with backup/restore evidence references. Mark the S5 gate satisfied only after
|
||||
T02 and T04 evidence exists.
|
||||
|
||||
Blocked on: `RAILIANCE-WP-0013-T04`.
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
---
|
||||
id: RAILIANCE-WP-0014
|
||||
type: workplan
|
||||
title: "Core Hub Helm cutover and vergabe-teilnahme image refresh"
|
||||
domain: financials
|
||||
repo: railiance-apps
|
||||
status: ready
|
||||
owner: codex
|
||||
topic_slug: railiance
|
||||
created: "2026-07-10"
|
||||
updated: "2026-07-10"
|
||||
state_hub_workstream_id: "780c8d1d-ea75-479e-9733-10b518dde242"
|
||||
---
|
||||
|
||||
# Core Hub Helm cutover and vergabe-teilnahme image refresh
|
||||
|
||||
Follow-up to `RAILIANCE-WP-0012` and custodian inbox `3ced85c7`. The S5 Helm
|
||||
chart, values, Makefile targets, and runbook exist; the live cluster may still
|
||||
run Core Hub from `core-hub/k8s/railiance-production/` manifests.
|
||||
|
||||
Separately, the 2026-07-10 delivery-lane snapshot shows `vergabe-teilnahme`
|
||||
deployed at `f95de14` while the source repo head is `1061f3e` (14-commit drift).
|
||||
|
||||
Production-touching deploy steps require explicit image tags and operator
|
||||
approval. Do not store secrets in workplans.
|
||||
|
||||
## Core Hub server dry-run on Railiance01
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T01
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "b9dea9f3-f0c0-41c1-b9c4-ec4c44d43d55"
|
||||
```
|
||||
|
||||
Run `CORE_HUB_IMAGE_TAG=<sha> make check-core-hub-image`,
|
||||
`core-hub-server-dry-run`, and `core-hub-dry-run` against Railiance01. Record the
|
||||
chosen image tag and any schema or secret prerequisites before apply.
|
||||
|
||||
## Adopt Helm-managed Core Hub release
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T02
|
||||
status: wait
|
||||
priority: high
|
||||
state_hub_task_id: "0e591104-0123-4e4a-84ff-5deb435e8441"
|
||||
```
|
||||
|
||||
Run `core-hub-migrate` then `core-hub-deploy` with an explicit
|
||||
`CORE_HUB_IMAGE_TAG`. Confirm the Helm release owns `core-hub-api`, runtime
|
||||
ConfigMap, and Service in namespace `core-hub`. Retire duplicate resources from
|
||||
the service-repo `k8s/railiance-production/` apply path once stable.
|
||||
|
||||
Blocked on: `RAILIANCE-WP-0014-T01`.
|
||||
|
||||
## Verify Core Hub post-cutover smoke
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T03
|
||||
status: wait
|
||||
priority: high
|
||||
state_hub_task_id: "eca3ce8b-68a8-44d0-90c7-2cdefa482200"
|
||||
```
|
||||
|
||||
Run `make core-hub-status`, `make core-hub-smoke`, and `make core-hub-release-info`.
|
||||
Ingress for `hub.coulomb.social` must remain healthy. Document rollback steps in
|
||||
`docs/core-hub-on-railiance01.md` if Helm values drift from prior manifests.
|
||||
|
||||
Blocked on: `RAILIANCE-WP-0014-T02`.
|
||||
|
||||
## Refresh vergabe-teilnahme production image
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T04
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "d547f68d-52ec-4ff1-952b-ec70c834e0d3"
|
||||
```
|
||||
|
||||
Identify the current published image tag for `vergabe-teilnahme` in the Forgejo
|
||||
registry, verify the manifest with `tools/check-oci-image.sh`, run
|
||||
`vergabe-dry-run` and `vergabe-server-dry-run` if available, then deploy with an
|
||||
explicit tag. Close the 14-commit drift noted in the delivery-lane evidence.
|
||||
|
||||
## Post-deploy vergabe smoke and runbook evidence
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T05
|
||||
status: wait
|
||||
priority: medium
|
||||
state_hub_task_id: "4eaaa345-4252-46c1-bd8a-49d654ec48f4"
|
||||
```
|
||||
|
||||
After deploy, run `make vergabe-status`, migrations if required, and HTTPS smoke
|
||||
against `vergabe-teilnahme.whywhynot.de`. Update `docs/vergabe-teilnahme.md`
|
||||
with the deployed tag, date, and smoke result (non-secret).
|
||||
|
||||
Blocked on: `RAILIANCE-WP-0014-T04`.
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
---
|
||||
id: RAILIANCE-WP-0012
|
||||
type: workplan
|
||||
title: "Inbox suggestions — Core Hub S5 release and CNPG backup observability"
|
||||
domain: financials
|
||||
repo: railiance-apps
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: railiance
|
||||
created: "2026-07-10"
|
||||
updated: "2026-07-10"
|
||||
state_hub_workstream_id: "25cee4ac-d1e4-441a-bc33-cf8c90ee67a6"
|
||||
---
|
||||
|
||||
# Inbox suggestions — Core Hub S5 release and CNPG backup observability
|
||||
|
||||
Sources:
|
||||
|
||||
- State Hub inbox `a1957cce` (HIGH): no CNPG backups on railiance01 production DB
|
||||
clusters; evidence `the-custodian/docs/evidence/vergabe-teilnahme-delivery-lane-20260710.json`.
|
||||
- State Hub inbox `3ced85c7`: Core Hub release ownership follow-up from
|
||||
`CUST-WP-0052`; cite `the-custodian/docs/core-hub-helixforge-build-alignment.md`
|
||||
and `core-hub/docs/deployment/staging-profile.md`.
|
||||
|
||||
## Core Hub Helm chart and production values
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0012-T01
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "0795f034-eacc-41d5-af8f-e5909f72f79f"
|
||||
```
|
||||
|
||||
Promote the temporary `core-hub/k8s/railiance-production/` profile into
|
||||
`charts/core-hub/` with non-secret production and staging values under `helm/`.
|
||||
Secrets remain operator-owned (`core-hub-prod-env`, `core-hub-staging-env`).
|
||||
|
||||
## Makefile release targets and smoke gate
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0012-T02
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "f975d9d3-e9b4-45ff-8088-03b2a5a855c6"
|
||||
```
|
||||
|
||||
Add `core-hub-dry-run`, `core-hub-server-dry-run`, `core-hub-deploy`,
|
||||
`core-hub-status`, `core-hub-migrate`, `core-hub-smoke`, and staging
|
||||
counterparts. Require explicit `CORE_HUB_IMAGE_TAG=<sha>` for production deploys.
|
||||
|
||||
## Operator runbook for Core Hub on railiance01
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0012-T03
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "41cba5e9-2c19-4b90-9dbe-517a3413eac7"
|
||||
```
|
||||
|
||||
Document the S5 release path in `docs/core-hub-on-railiance01.md`, including
|
||||
migration boundary, ingress cutover notes, rollback, and links to Core Hub
|
||||
service-repo smokes.
|
||||
|
||||
## CNPG backup posture observability
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0012-T04
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "7c9794ca-478c-4da9-ae77-b3aa62924e98"
|
||||
```
|
||||
|
||||
Add `make cnpg-backup-status` and `tools/cnpg-backup-status.sh` to report backup
|
||||
spec and `ScheduledBackup` coverage for `apps-pg`, `forgejo-db`,
|
||||
`net-kingdom-pg`, and `state-hub-db`. Include operator-ready manifest templates
|
||||
for barman `ObjectStore` and `ScheduledBackup` wiring once platform supplies
|
||||
object-store credentials.
|
||||
|
||||
## Update app data backup handoff with delivery-lane evidence
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0012-T05
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "126f8f26-2536-40d6-8166-060f93ada85b"
|
||||
```
|
||||
|
||||
Refresh `docs/app-data-backup-restore-handoff.md` with the 2026-07-10 custodian
|
||||
conformance snapshot, the cluster inventory, and the platform coordination path
|
||||
for backup implementation.
|
||||
Loading…
Add table
Add a link
Reference in a new issue