railiance-platform/docs/evidence/core-hub-private-shadow-2026-08-21.md

80 lines
4.2 KiB
Markdown
Raw Normal View History

# Core Hub private shadow evidence — 2026-08-21
## Scope and safety boundary
This record covers the private Core Hub relocation shadow on railiance01 for
`RPF-WP-0021`. Public Ingress was absent throughout. The CoulombCore writer,
public routing, DNS, and source runtime were not changed or retired. No secret
value is included in this record.
## Credential custody and isolation
- State Hub decision `4f70311b-a4c4-4959-b1a1-cdd1be97d523` was approved by
the platform operator / Core Hub owner at `2026-08-20T22:02:19Z`.
- An attended OpenBao OIDC session carried `platform-admin`; the policy and
Kubernetes-auth role for `CCR-2026-0013` were applied without displaying a
protected value.
- The existing API token moved directly from source Kubernetes custody into
`platform/workloads/core-hub/runtime`; it was neither copied through a
staging file nor printed.
- `openbao-core-hub-runtime` and `openbao-core-hub-database` reported
`Ready=True`, reason `Valid`. The API-token, runtime-database, and
migration-database ExternalSecrets each reported `Ready=True` and used
disjoint target Secrets.
- A disposable outside namespace could use neither ClusterSecretStore and
created no target Secret. A Kubernetes-auth probe could read only the exact
Core Hub KV data/metadata paths and was denied on sibling and parent paths.
- A runtime database lease connected and read but could not create objects or
inherit `core_hub_migrate`. A migration lease created and dropped a bounded
probe table. Probe tokens, pods, Secrets, and namespace were removed.
## Logical shadow restore and reconciliation
- The consistent source archive was captured at `2026-08-20 22:12:26 UTC`
while the source writer remained live.
- Archive size: `22175` bytes.
- SHA-256:
`3770946a1032c4109fea32fc00f4411c2f4d9311a28fb17bfc3d0d2de888db8b`.
- Source and restored target matched at Alembic revision `20260627_0002`,
eight tables, and 39 rows before shadow smoke activity. Per-table counts
were `1, 3, 10, 2, 2, 3, 2, 16`.
- The source archive used `public`; all eight restored tables were moved into
the accepted `core_hub` schema under the migration lease. Reconciliation
confirmed `core_hub_owner` ownership, runtime SELECT grants on every table,
and owner membership for the migration role.
## Package, deployment, and migration
- `rapp-postgres` published the Core Hub database-engine roles and ESO policy
in `cfde57a`; the live provisioner completed successfully.
- `rapp-core-hub` added migration-only database/DNS egress in `1cedab2` and a
fail-closed owned-port verifier in `8d729fd`.
- Live rotation exposed that watcher exit 75 caused scheduled pod
CrashLoopBackOff. The shadow runtime was temporarily stabilized at a
20-minute refresh interval while the source package was corrected.
- `rapp-core-hub` chart `0.3.1`, commit `7cfa8bb`, keeps its PID 1 watcher alive
and restarts only the child with a freshly constructed environment. All nine
package tests, Helm lint, deterministic rendering, and package checks passed.
- Helm release `core-hub` revision 2 reported `deployed`; the private API
Deployment was Ready, the migration Job completed, all three package
NetworkPolicies were present, and no Ingress existed.
## Credential rotation and private smoke
- A forced database lease refresh changed the runtime Secret resourceVersion.
The application child PID changed while the pod name and UID stayed fixed,
container restart count stayed `0 -> 0`, and readiness returned `true`.
- The intended ExternalSecret interval was restored to five minutes. The next
scheduled reconcile at `2026-08-20T22:45:01Z` left the pod Ready with the
same UID and zero container restarts; no CrashLoop behavior recurred.
- The hardened package verifier passed on an owned, unused private port.
- Authenticated deployed-smoke run `20260820224419-43dc05` completed at
`2026-08-20T22:44:20Z`. All ten checks passed: health, readiness, OpenAPI,
four public catalogs, unauthenticated 401 enforcement on protected routes,
and operator-authenticated protected reads/writes. Its temporary token file
was mode 0600 and shredded on exit.
`CCR-2026-0013` is verified. Public promotion, final write freeze/transfer,
DNS changes, and CoulombCore retirement remain separate operator gates under
`CORE-WP-0011`.