docs: close railiance01 stabilization
This commit is contained in:
parent
c13834e365
commit
d38ab710c3
4 changed files with 102 additions and 7 deletions
|
|
@ -16,7 +16,7 @@
|
|||
| workplan | CORE-WP-0008 | finished | — | workplans/CORE-WP-0008-api-first-ops-evidence-continuation.md |
|
||||
| workplan | CORE-WP-0009 | finished | — | workplans/CORE-WP-0009-hub-core-adoption-and-integration.md |
|
||||
| workplan | CORE-WP-0010 | proposed | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md |
|
||||
| workplan | CORE-WP-0011 | active | — | workplans/CORE-WP-0011-railiance01-production-relocation.md |
|
||||
| workplan | CORE-WP-0011 | finished | — | workplans/CORE-WP-0011-railiance01-production-relocation.md |
|
||||
| task | CORE-WP-0001-T01 | done | — | workplans/CORE-WP-0001-statehub-bootstrap.md |
|
||||
| task | CORE-WP-0001-T02 | done | — | workplans/CORE-WP-0001-statehub-bootstrap.md |
|
||||
| task | CORE-WP-0001-T03 | done | — | workplans/CORE-WP-0001-statehub-bootstrap.md |
|
||||
|
|
@ -58,4 +58,4 @@
|
|||
| task | CORE-WP-0011-T02 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md |
|
||||
| task | CORE-WP-0011-T03 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md |
|
||||
| task | CORE-WP-0011-T04 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md |
|
||||
| task | CORE-WP-0011-T05 | progress | — | workplans/CORE-WP-0011-railiance01-production-relocation.md |
|
||||
| task | CORE-WP-0011-T05 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Core Hub railiance01 production cutover — 2026-08-21
|
||||
|
||||
Status: production cutover complete; stabilization in progress.
|
||||
Status: production cutover and stabilization complete.
|
||||
|
||||
## Approved boundary
|
||||
|
||||
|
|
@ -109,3 +109,9 @@ After the DNS edit, rollback is:
|
|||
Never allow both databases to accept production writes. During stabilization,
|
||||
CoulombCore remains intact with database-default read-only mode and
|
||||
railiance01 is the sole writer.
|
||||
|
||||
Stabilization closed successfully and the source runtime was retired
|
||||
reversibly. See
|
||||
`docs/evidence/core-hub-railiance01-stabilization-2026-08-21.md` for the
|
||||
observation snapshots, retained backups, zero-replica source state and residual
|
||||
ownership.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,77 @@
|
|||
# Core Hub railiance01 stabilization closeout — 2026-08-21
|
||||
|
||||
Status: complete; CoulombCore runtime retired reversibly.
|
||||
|
||||
## Closeout threshold and observations
|
||||
|
||||
The relocation used a bounded stabilization gate: recursive DNS had to remain
|
||||
fully drained to railiance01 for multiple original TTLs, two independent
|
||||
consumer snapshots had to pass, the target had to remain Ready without
|
||||
restarts or 5xx responses, governed secret delivery and TLS had to remain
|
||||
healthy, and both source and target backups had to be identified before the
|
||||
old runtime stopped.
|
||||
|
||||
- Cloudflare, Google, Quad9 and the workstation resolver continued returning
|
||||
`92.205.62.239`. Normal HTTPS health and readiness returned 200 from that
|
||||
address with successful certificate verification.
|
||||
- Both the cutover snapshot and closeout snapshot passed the ops-hub public
|
||||
gate and activity-core's deployed Core Hub resolver. The closeout resolver
|
||||
passed health, readiness, protected-route denial and the minimum widget-type
|
||||
count of 26.
|
||||
- The target request-log snapshot contained 989 HTTP 200 responses and four
|
||||
expected HTTP 401 responses, with no 5xx response or application
|
||||
error/exception/traceback match.
|
||||
- Helm revision 4 runs chart `0.3.2` at the immutable image digest. The API pod
|
||||
remained Ready with zero restarts; the package live verifier passed after
|
||||
source retirement.
|
||||
- All three ExternalSecrets remained `SecretSynced=True`. Certificate
|
||||
`core-hub-tls` remained Ready through 2026-11-18T22:41:01Z.
|
||||
|
||||
## Retained data and rollback evidence
|
||||
|
||||
- Target CloudNativePG backup `platform-pg-daily-20260821021500` completed at
|
||||
2026-08-21T02:15:11Z after the final restore and cutover write.
|
||||
- CoulombCore logical-backup Job
|
||||
`cnpg-logical-backup-apps-pg-29787990` completed at
|
||||
2026-08-21T02:30:17Z. Its encrypted object-store outputs include
|
||||
`apps-pg/apps-pg-core_hub-20260821T023005Z.dump.age` and the separate staging
|
||||
archive. No credential or archive content was read.
|
||||
- The frozen final-transfer archive SHA-256 remains recorded as
|
||||
`4015ad622303ba206a46627ba0d03513e1054b95b87247d61cce6cb6b94189e6`.
|
||||
The unencrypted mode-0600 workstation copies were removed after the two
|
||||
external backup paths were verified.
|
||||
- Database `core_hub` remains present on CoulombCore with
|
||||
`default_transaction_read_only=on`. It is retained data for rollback, not an
|
||||
active production dependency.
|
||||
|
||||
## Reversible source retirement
|
||||
|
||||
- CoulombCore Deployments `core-hub/core-hub-api` and
|
||||
`core-hub-staging/core-hub-api` now both specify zero replicas. No source API
|
||||
pod remains. Services, Ingress, Secrets and databases were not deleted.
|
||||
- `core-hub-staging-coulombcore` was removed from the operator tunnel map. Its
|
||||
dedicated supervisor ignored graceful TERM, so only its confirmed supervisor
|
||||
and SSH child PIDs were force-stopped; local port 18010 is closed. No other
|
||||
tunnel was changed.
|
||||
- A forced request to the old public address returns 404, while normal public
|
||||
health/readiness and the hardened verifier continue passing on railiance01.
|
||||
|
||||
Rollback remains possible while retained artifacts exist: restore the IONOS A
|
||||
record to `92.205.130.254`, scale `core-hub/core-hub-api` to one, reset database
|
||||
`core_hub`'s `default_transaction_read_only`, restart the Deployment and verify
|
||||
authenticated service before disabling target Ingress. Never enable both
|
||||
writers. The staging Deployment and retired local tunnel are not required for
|
||||
production rollback.
|
||||
|
||||
## Residual ownership
|
||||
|
||||
- `rapp-core-hub` owns the verified railiance01 Helm package, live verifier,
|
||||
NetworkPolicies and rollback surface.
|
||||
- `railiance-platform` / `rapp-postgres` own platform-pg, dynamic database
|
||||
credentials and target backup/restore operations.
|
||||
- The CoulombCore shutdown owner may delete the inert namespaces and retained
|
||||
source database with the host after its retention decision; they no longer
|
||||
block shutdown.
|
||||
- `CORE-WP-0010` remains proposed and independently dependent on
|
||||
`HUB-WP-0004`. This relocation does not authorize repository archival or
|
||||
runtime absorption.
|
||||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Relocate Core Hub production from CoulombCore to railiance01"
|
||||
domain: infotech
|
||||
repo: core-hub
|
||||
status: active
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: infotech
|
||||
created: "2026-08-20"
|
||||
|
|
@ -232,7 +232,7 @@ to T05. Current evidence and exact rollback state are in
|
|||
|
||||
```task
|
||||
id: CORE-WP-0011-T05
|
||||
status: progress
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -251,11 +251,23 @@ CoulombCore Deployment remains Ready for rollback, but database-level
|
|||
`default_transaction_read_only=on` was reverified. Keep the source frozen until
|
||||
the bounded observation window and operator sign-off close this task.
|
||||
|
||||
Completed 2026-08-21 after multiple original DNS TTLs and two independent
|
||||
consumer snapshots. The target recorded 989 successful requests, four expected
|
||||
401s and no 5xx/error match; its pod stayed Ready with zero restarts, all
|
||||
ExternalSecrets and TLS remained Ready, and the post-retirement package
|
||||
verifier passed. Completed target and encrypted source backups were identified
|
||||
before both CoulombCore Core Hub Deployments were scaled to zero. The obsolete
|
||||
staging tunnel was removed from configuration and its exact supervisor/SSH
|
||||
processes were stopped; port 18010 is closed. Source Services, Ingresses,
|
||||
Secrets and the read-only database remain only as reversible retention
|
||||
artifacts and no longer block CoulombCore shutdown. Closeout evidence is in
|
||||
`docs/evidence/core-hub-railiance01-stabilization-2026-08-21.md`.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- [x] Decommission deadline and live source/target inventory are recorded
|
||||
- [x] Governed rapp package renders and passes policy checks
|
||||
- [x] Shadow restore/reconciliation and consumer smokes pass on railiance01
|
||||
- [x] Production traffic and tunnel move with explicit approval and rollback
|
||||
- [ ] Stabilization passes and CoulombCore no longer hosts Core Hub dependencies
|
||||
- [ ] `CORE-WP-0010` remains independently sequenced behind `HUB-WP-0004`
|
||||
- [x] Stabilization passes and CoulombCore no longer hosts Core Hub dependencies
|
||||
- [x] `CORE-WP-0010` remains independently sequenced behind `HUB-WP-0004`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue