hub-core/docs/core-hub-absorption-plan.md

# Core Hub absorption and `/api/v2` cutover plan

**Status:** accepted joint input for `HUB-WP-0004-T06` and
`CORE-WP-0010-T02`
**Date:** 2026-08-21
**Runtime source:** `hub-core`
**Rollback runtime:** `core-hub` on railiance01

## Decision

Absorb Core Hub into hub-core by capability and data boundary. There is no
big-bang route move and no period with two production writers for the same
boundary.

Hub-core owns the surviving wheel, runtime image, extension contracts,
compatibility routes, durable framework store, and migrations. Core Hub stays
deployable as the rollback runtime until every `/api/v2` route group, durable
record family, consumer gate, and rollback exercise has passed. Its Python
package is never imported by hub-core.

The current production baseline is Core Hub chart `0.3.2` on railiance01,
backed by the `core_hub` PostgreSQL schema. Its source database on CoulombCore
is already frozen and is historical rollback evidence, not a participant in
this absorption.

## Non-negotiable invariants

1. A route group has exactly one production writer at any moment.
2. A write freeze, final export, count/content comparison, and explicit
   operator approval precede every production writer change.
3. Shadow traffic consists of reads and synthetic fixture writes only. Do not
   mirror normal writes to both runtimes.
4. Hub-core and Core Hub keep separate SQLAlchemy metadata and migration
   lineages. Hub-core may copy legacy rows but must not import `core_hub`
   models or migrations.
5. IDs, timestamps, relationships, source revision, source table, and import
   bundle hash survive migration as provenance.
6. Only API-key hashes and prefixes move. Raw keys, static operator tokens,
   database credentials, and secret values never enter bundles or evidence.
7. Authorization fails closed when its dependency is unavailable. The
   compatibility API-key validator is temporary; authorization decisions
   remain owned through the identity/policy boundary.
8. Core Hub remains a tested rollback target until the stabilization and
   rollback-retirement gates close.

## Runtime and database shape

The hub-core image selected by ADR-0001 is the candidate runtime. Its current
in-memory `PortStore` is not a migration target. The first absorption slice
must add a PostgreSQL implementation, production readiness checks, an audit
ledger, and migrations owned entirely by hub-core.

Use the existing platform PostgreSQL allocation currently assigned to Core
Hub; do not consume a fifth application slot. Add a distinct target schema
(provisionally `hub_runtime`) beside retained legacy schema `core_hub`, then
transfer the consumer declaration and dynamic credential identity as part of
the final package transition. Separate schemas make source/target counts and
rollback observable without mixing ORM metadata.

The deployment package currently named `rapp-core-hub` owns the verified
railiance01 chart, policies, secret projections, live verifier, and rollback
surface. During dual-run it must deploy both immutable image revisions or a
candidate release beside the legacy release. Before Core Hub is archived, the
package must either be renamed/replaced by a hub-core-owned rapp package or
recorded explicitly as the surviving package with updated ownership and
intent. No application source moves into the deployment package.

## Traffic controls

Implement route-group controls before moving data:

- `HUB_CORE_V2_GROUPS`: compatibility groups served by hub-core;
- `HUB_CORE_V2_WRITE_GROUPS`: subset allowed to mutate hub-core authority;
- `CORE_HUB_V2_WRITE_GROUPS`: legacy groups still allowed to mutate Core Hub;
- an ingress/gateway route map selecting `core-hub` or `hub-core` per group;
- a shadow comparator that calls both read targets, normalizes ordering and
  volatile fields, and records hashes without response bodies containing
  sensitive data.

The exact configuration carrier may be Helm values rather than environment
variables, but the three states are mandatory and independently observable:

| State | Public owner | Candidate behavior | Write rule |
| --- | --- | --- | --- |
| legacy | Core Hub | health plus synthetic fixtures | Core Hub only |
| shadow | Core Hub | normalized read comparison | Core Hub only |
| candidate | hub-core | public read/write for the group | hub-core only |

Direct service access is network-policy restricted. The route map is the
normal traffic authority; application write-group checks are defense in
depth. A configuration assertion must reject overlap between legacy and
candidate write sets.

## Route and module move order

Each slice is independently buildable, reversible, and evidenced. Later
slices do not excuse a failed earlier gate.

### S0 — Durable runtime, compatibility shell, and migration tooling

No public route moves.

- Add hub-core PostgreSQL `PortStore`, session lifecycle, readiness, audit
  ledger, import/export commands, and a migration bundle schema.
- Add `/api/v2` router composition, legacy response serializers, bearer-token
  dependency, per-group write controls, and normalized shadow comparison.
- Import the static operator token through the existing OpenBao/ExternalSecret
  lane and copy API-key hashes into the isolated compatibility auth store.
- Preserve current error/status semantics and unprefixed OpenAPI aliases.
- Extend the conformance suite with durable restart/rebuild, authorization
  allow/deny/unavailable, metadata isolation, and migration idempotence.

Gate: frozen source fixture imports twice without duplicates; all seven source
table counts can be represented; production readiness fails when PostgreSQL or
authorization is unavailable; no traffic has moved.

### S1 — System and read-only discovery

Routes:

- `/healthz`, `/readyz`;
- `/api/v2/widget-types`, `/api/v2/event-types`;
- `/api/v2/annotation-categories`, `/api/v2/policy-scopes`;
- `/api/v2/openapi.json`, `/api/v2/openapi.yaml`, `/api/v2/docs`;
- compatibility aliases published by the current OpenAPI surface.

Move the catalogs, schemas, OpenAPI snapshot/export behavior, and public
consumer fixtures. Static catalog responses are compared as canonical JSON.
Readiness must reflect the dependencies required by enabled groups rather than
reporting healthy from process liveness alone.

Gate: public catalog and documentation smoke passes against both targets;
normalized payload hashes match; ops-hub public gate and activity-core
resolver pass; rollback route-map change is rehearsed.

### S2 — Registry and manifest authority

Routes:

- `GET/POST /api/v2/hubs`;
- `GET/POST /api/v2/hub-capability-manifests`;
- `PATCH /api/v2/hub-capability-manifests/{manifest_id}`;
- `POST /api/v2/hub-capability-manifests/{manifest_id}/activate`;
- `GET /api/v2/hub-registry`.

Modules/data:

- copy `hubs` and `hub_capability_manifests` into hub-core registry authority;
- preserve legacy UUIDs and camel-case compatibility serialization;
- validate new writes against `helixforge.hub-extension`, while a named legacy
  adapter reads migrated 0.x records without silently rewriting them;
- add activation audit records and expose registry projections with
  provenance.

Gate: table counts and order-independent row hashes match; manifest activation
is idempotent and audited; protected allow/deny checks pass; ops-hub bootstrap
can resolve and update its manifest; rollback delta replay is proven.

### S3 — API consumers and compatibility credentials

Routes:

- `GET/POST /api/v2/api-consumers`;
- `POST /api/v2/api-consumers/{consumer_id}/api-keys`;
- `POST /api/v2/token`.

Modules/data:

- copy `api_consumers` and `api_keys`, including status, scopes, quotas, hash,
  and prefix but never raw key material;
- preserve one-time key issuance semantics and ensure generated secrets are
  absent from logs, projections, migration bundles, and evidence;
- keep the static operator credential and dynamic consumer keys in distinct
  custody lanes;
- record a residual owner and follow-on task for replacing compatibility keys
  with the platform identity/authorization ports. The shim may outlive Core
  Hub briefly, but it is not a permanent hub-core authority.

Gate: existing ops-hub and activity-core credentials authenticate against the
candidate; invalid, revoked, missing, and dependency-unavailable cases deny;
rate/quota behavior is either preserved or explicitly residual-owned; reverse
delta replay for newly issued key hashes is rehearsed without exposing a key.

### S4 — Widgets and interaction evidence

Routes:

- `GET/POST /api/v2/widgets`;
- `GET/POST /api/v2/interaction-events`.

Modules/data:

- copy `widgets` into the operator-surface registry/projection boundary;
- copy `interaction_events` into the physically separate interaction-event
  authority while retaining legacy `widgetId`, event type, context, metadata,
  IDs, and timestamps in the compatibility view;
- route new writes through `port.events.interaction` and build `/api/v2` reads
  from the same authority rather than a second compatibility-only store.

Gate: counts and hashes match; projection rebuild fixture and migrated sample
produce provenance-equivalent views; progress events never appear in the
interaction family; ops-hub and activity-core each post and read back unique
correlated evidence; rollback delta replay is proven.

### S5 — Empty compatibility collections and operator console

Routes:

- `GET/POST /api/v2/annotations`;
- `GET/POST /api/v2/requirement-candidates`;
- `GET/POST /api/v2/decision-records`;
- `GET/POST /api/v2/deployment-records`;
- `GET/POST /api/v2/outcome-signals`;
- `/console`.

The five collections have no Core Hub durable models and currently return
empty data. Preserve that behavior only as a compatibility adapter; do not
claim or create authority. Each endpoint needs an explicit retirement or
future-owner record before the adapter is removed. Rebuild the console from
hub-core projections and public APIs; it must not query the retained
`core_hub` schema.

Gate: exact empty/error compatibility passes, the console authorization
boundary and visual smoke pass, and every deferred collection has a residual
disposition.

### S6 — Whole-host cutover and Core Hub retirement

After S1–S5 are in candidate state:

1. Freeze all remaining Core Hub writes and capture a final encrypted backup,
   table counts, content hashes, schema revision, image digest, and correlation
   ID.
2. Import any remaining deltas and require zero unexplained count/hash drift.
3. Run hub-core conformance, the Core Hub deployed smoke, ops-hub bootstrap,
   and activity-core resolver/write-readback against the candidate image.
4. With explicit operator approval, point `hub.coulomb.social` entirely at
   hub-core and set every Core Hub write group empty.
5. Observe at least seven consecutive days with no unexplained 5xx, no failed
   consumer gate, no data drift, and no normal request to Core Hub. Record any
   exception with an owner and expiry.
6. Exercise rollback once during rehearsal. After the stabilization window and
   explicit retirement approval, scale Core Hub to zero while retaining its
   immutable image, database backup, deployment revision, and route-map
   rollback instructions until the agreed expiry.
7. Move the final OpenAPI/contracts/smoke fixtures into hub-core history,
   record residual owners, archive the `core-hub` repository read-only, and
   update the deployment package identity.

## Evidence required for every durable slice

| Evidence | Pass condition |
| --- | --- |
| Source identity | Core Hub image/source revision and schema revision recorded |
| Target identity | hub-core wheel/image version, revision, and contract version recorded |
| Backup | Encrypted location and checksum recorded; no secret value logged |
| Counts | Per-table source/import/target counts with zero unexplained delta |
| Content | Order-independent canonical row hashes match after documented transforms |
| Provenance | Source table/ID/revision, import bundle hash, and target record IDs queryable |
| Contract | Static OpenAPI diff has no unexplained removed route/status/schema behavior |
| Authorization | Missing/invalid/revoked deny; allowed roles pass; dependency outage denies |
| Failure mode | Database, policy/auth, and candidate unavailability behavior recorded |
| Consumers | Owned ops-hub and activity-core gates pass against the candidate |
| Rollback | Reverse delta export/import and route-map reversal exercised |

An unexplained row, response, authorization, or provenance mismatch fails the
slice. A waiver requires a human progress note and a live residual task.

## Smoke and change ownership

| Surface | Owner during absorption |
| --- | --- |
| Hub-core unit, contract, conformance, migration, and image tests | `hub-core` |
| Legacy OpenAPI and deployed-smoke oracle | `core-hub` until archive, then historical fixture in `hub-core` |
| ops-hub public/bootstrap gate | `ops-hub` |
| activity-core resolver and interaction write/readback | `activity-core` |
| Helm render, policy, secret projection, live verification, traffic map, rollback | rapp package (currently `rapp-core-hub`) + platform owners |
| Production writer flip, rollback, scale-to-zero, archive | operator approval required |

## Rollback procedure

For a failed candidate slice:

1. Stop candidate writes for the affected group; do not enable legacy writes
   yet.
2. Export the candidate-only delta with IDs and hashes, validate it against the
   reverse adapter, and import it into the retained legacy schema.
3. Compare counts/content and run the legacy group smoke.
4. Route the group to Core Hub, enable only its legacy writer, and confirm the
   candidate writer remains disabled.
5. Record the failure correlation ID, data interval, revisions, evidence, and
   next owner. Do not continue to the next slice.

If candidate data cannot be replayed safely, keep both writers disabled and
invoke the operator recovery path. Availability pressure does not authorize
discarding acknowledged writes or enabling concurrent writers.

## Completion criteria

`HUB-WP-0004-T06` is complete when this plan is linked to CORE-WP-0010 and its
handoff is recorded. HUB-WP-0004 may then finish; it does not claim production
absorption.

`CORE-WP-0010` completes only when all slices have evidence, production is on
hub-core, the stabilization and rollback gates close, residuals have live
owners, the deployment package names the surviving runtime, and Core Hub is
archived read-only.