117 lines
6 KiB
Markdown
117 lines
6 KiB
Markdown
# Core Hub railiance01 production cutover — 2026-08-21
|
|
|
|
Status: production cutover and stabilization complete.
|
|
|
|
## Approved boundary
|
|
|
|
The operator explicitly approved `CORE-WP-0011-T04`. This gate covers the
|
|
bounded source write freeze, final transfer, public route change, verification
|
|
and rollback readiness. CoulombCore resources remain intact through the later
|
|
stabilization gate.
|
|
|
|
## Freeze and final transfer
|
|
|
|
- Public DNS still resolves `hub.coulomb.social` to CoulombCore
|
|
`92.205.130.254`; its observed TTL is 3600 seconds.
|
|
- CoulombCore database `core_hub` now has
|
|
`default_transaction_read_only=on`. The source Deployment was restarted,
|
|
its application role reports `transaction_read_only=on`, and public health
|
|
and readiness remain HTTP 200.
|
|
- Frozen full-archive SHA-256:
|
|
`4015ad622303ba206a46627ba0d03513e1054b95b87247d61cce6cb6b94189e6`.
|
|
- The data-only transfer contained eight `COPY public.*` streams. Only the
|
|
statement schema qualifiers were mapped to `core_hub`; the source has no
|
|
sequence statements.
|
|
- The target rehearsal state was backed up before its rows were cleared. Its
|
|
archive SHA-256 is
|
|
`92200f21d9f9ea01023ec74dd2e64641ec14544ee5a3a024f911c81e2ed67157`.
|
|
- Restored target counts match the frozen source: `1, 3, 10, 2, 2, 3, 2, 16`
|
|
across `alembic_version`, `api_consumers`, `api_keys`,
|
|
`hub_capability_manifests`, `hubs`, `interaction_events`, `migration_runs`
|
|
and `widgets`.
|
|
- Order-independent content digests match for all eight tables. Alembic is at
|
|
`20260627_0002`; the post-restore migration Job completed.
|
|
|
|
## Target readiness
|
|
|
|
- Helm revision 3 runs chart `0.3.1` at immutable image digest
|
|
`sha256:0db6f7c646c6d4c1fbc844dc266fcf883584637e1c1f47802efe1c6a98558d3c`.
|
|
- The railiance01 Deployment is Ready. The hardened package verifier passes.
|
|
- Direct target resolution returns health/readiness 200, protected hubs 401
|
|
without authentication, and 200 with the governed token.
|
|
- The Ingress is present at `92.205.62.239`.
|
|
|
|
## DNS, TLS and public verification
|
|
|
|
- The operator changed the IONOS A record for `hub.coulomb.social` from
|
|
`92.205.130.254` to `92.205.62.239`. All four authoritative IONOS name
|
|
servers returned the target, as did Google and Quad9. Cloudflare still held
|
|
the source address under the previously observed 3600-second TTL at the
|
|
completion check; source therefore remains available but read-only while
|
|
residual caches drain.
|
|
- The namespace-wide default-deny policy initially denied Traefik access to
|
|
cert-manager's solver. A temporary NetworkPolicy selected only
|
|
`acme.cert-manager.io/http01-solver=true`, allowed only kube-system ingress
|
|
on TCP 8089, and was removed after issuance. Because Cloudflare was still
|
|
cached, a temporary CoulombCore Ingress forwarded only
|
|
`/.well-known/acme-challenge/` to the target solver; it too was removed after
|
|
issuance. No application route or write policy was opened.
|
|
- Certificate `core-hub-tls` is Ready. The Let's Encrypt certificate has SAN
|
|
`hub.coulomb.social`, verifies successfully, and is valid from
|
|
2026-08-20 22:41:02 UTC through 2026-11-18 22:41:01 UTC.
|
|
- Target-forced HTTPS checks returned 200 for `/healthz`, `/readyz`,
|
|
`/api/v2/widget-types`, `/api/v2/event-types`,
|
|
`/api/v2/annotation-categories`, and `/api/v2/policy-scopes`.
|
|
`/api/v2/hubs` and `/console` returned 401 without authentication.
|
|
- Governed-token checks against the target returned 200 for hubs and the HTML
|
|
operator console. The first post-cutover interaction-event write returned
|
|
201, then listed back successfully as
|
|
`e56bca14-51f8-4a73-bdab-f660e360fb14`; the event contains only cutover
|
|
metadata and references a migrated widget.
|
|
- The ops-hub `interhub-gate` probe passed from railiance01: protected hubs
|
|
returned 401 and all required OpenAPI paths were present. The deployed
|
|
activity-core `core-hub` resolver passed health, readiness, protected hubs,
|
|
widget-types status, and the minimum catalog count of 26 through the public
|
|
production hostname.
|
|
- Inspection found no active ops-bridge tunnel in the public request path.
|
|
Public traffic moves by the direct IONOS A record. The workstation's obsolete
|
|
`core-hub-staging-coulombcore` tunnel remains a T05 cleanup item.
|
|
|
|
## Stabilization entry snapshot
|
|
|
|
- Cloudflare (`1.1.1.1`, `1.0.0.1`), Google (`8.8.8.8`), Quad9 (`9.9.9.9`)
|
|
and the workstation resolver now all return `92.205.62.239` with TTL 3600.
|
|
A normal, non-forced HTTPS health request reaches that address with status
|
|
200 and successful certificate verification.
|
|
- Package chart `0.3.2` was deployed as Helm revision 4 after 10 tests, Helm
|
|
lint, deterministic render and server dry-run passed. Its permanent
|
|
`core-hub-acme-http01` NetworkPolicy selects only cert-manager's solver label
|
|
and permits only kube-system ingress on TCP 8089. The hardened live verifier
|
|
and external TLS health check pass after rollout.
|
|
- Certificate `core-hub-tls` remains Ready through
|
|
2026-11-18T22:41:01Z. CoulombCore's Deployment remains Ready as the rollback
|
|
runtime, and the `core_hub` database setting was reverified as
|
|
`default_transaction_read_only=on`.
|
|
|
|
## Current rollback
|
|
|
|
After the DNS edit, rollback is:
|
|
|
|
1. Restore the IONOS A record to `92.205.130.254` and verify authoritative plus
|
|
recursive resolution before enabling source writes.
|
|
2. On CoulombCore, reset database `core_hub`'s
|
|
`default_transaction_read_only` setting and restart Deployment
|
|
`core-hub-api` so the application reconnects writable.
|
|
3. Verify public health, readiness and an authenticated protected operation.
|
|
4. Only after source service is restored, roll railiance01 Helm release
|
|
`core-hub` back to the last private revision to remove target Ingress.
|
|
|
|
Never allow both databases to accept production writes. During stabilization,
|
|
CoulombCore remains intact with database-default read-only mode and
|
|
railiance01 is the sole writer.
|
|
|
|
Stabilization closed successfully and the source runtime was retired
|
|
reversibly. See
|
|
`docs/evidence/core-hub-railiance01-stabilization-2026-08-21.md` for the
|
|
observation snapshots, retained backups, zero-replica source state and residual
|
|
ownership.
|