coulomb/core-hub
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: define hub-core absorption plan
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
Details
CI Smoke / pytest-smoke (push) Failing after 3s
Details

Browse source
This commit is contained in:
tegwick 2026-08-21 10:57:13 +02:00
parent 5802d6f799
commit 4a9df801c3
7 changed files with 216 additions and 25 deletions
Download patch file Download diff file Expand all files Collapse all files

15
INTENT.md
View file

@ -42,13 +42,14 @@ The framework must be usable by humans and agents. Human operators need a stable
| Repo | Role |
| --- | --- |
| `hub-core` | Shared library — models, router factories, utils (`hub_core.utils`) |
| `hub-core` | Surviving framework/runtime — contracts, ports, package, and primary image |
| `state-hub` | Dev coordination — file-first workplans, dev MCP |
| `core-hub` | Production service — `/api/v2`, hubs, manifests, operator console |
Canon: `/home/worsch/the-custodian/docs/hub-ecosystem-architecture.md`
**Naming:** `core-hub` = core *framework runtime*. `hub-core` = core *primitives library*.
**Transition naming:** `core-hub` remains the current production service and
rollback authority while `hub-core` becomes the surviving framework/runtime.
## Current State
@ -63,7 +64,9 @@ This delivers the practical Gen3 production reset, but not the whole product
intent. Messaging, progress/work projections, several durable evidence and
decision resources, and a single extension runtime remain outside Core Hub or
only partially represented. `CORE-WP-0010` tracks absorption into hub-core and
repository archive; it is blocked on the runtime packaging, minimal-port, and
joint-plan decisions in `HUB-WP-0004`. The healthy service remains the
production and rollback authority until dual-run consumer gates prove a
replacement. See `STATE.md` for current operational truth.
repository archive. Hub-core's packaging, contract, port, and conformance
foundations are complete, and the joint route-group migration design is
recorded. Execution now waits on a published immutable hub-core revision, a
durable PostgreSQL backend, and an internal candidate deployment. The healthy
service remains the production and rollback authority until dual-run consumer
gates prove a replacement. See `STATE.md` for current operational truth.

5
README.md
View file

@ -44,5 +44,6 @@ It carries forward the lessons from State Hub and Inter-Hub: workplans and progr
The contract, FastAPI/Postgres foundation, Inter-Hub migration, Haskell/IHP
retirement, and relocation from CoulombCore to railiance01 are complete.
`CORE-WP-0011` is finished. The only open local workplan is runtime absorption
and eventual archive (`CORE-WP-0010`); it is blocked on hub-core's runtime
packaging, minimal-port, and joint-plan decisions in `HUB-WP-0004`.
and eventual archive (`CORE-WP-0010`). Its dual-run design is recorded; Slice 0
waits on a published hub-core runtime revision, durable PostgreSQL backend, and
internal candidate deployment before any public traffic changes.

12
STATE.md
View file

@ -6,8 +6,8 @@
## One-line posture
**Core Hub is the verified production `/api/v2` runtime on railiance01; the
CoulombCore APIs are stopped, and future absorption into hub-core is blocked on
hub-core's still-proposed runtime contract.**
joint hub-core absorption plan is recorded, and Slice 0 waits on a published
hub-core runtime with durable PostgreSQL storage.**
## Production truth
@ -32,15 +32,15 @@ Deployment packaging and rollout truth lives in the `rapp-core-hub` repository.
| --- | --- | --- |
| `CORE-WP-0001`–`CORE-WP-0009` | finished / archived | Gen3 runtime, contracts, persistence, consumer gates, and hardening delivered |
| `CORE-WP-0011` | finished | Production relocated to railiance01 and active CoulombCore runtime retired |
| `CORE-WP-0010` | blocked | Runtime absorption and archive wait on `HUB-WP-0004` |
| `CORE-WP-0010` | blocked | Dual-run design done; Slice 0 publication/storage gates remain |
### Open work
| Task | Status | Dependency |
| --- | --- | --- |
| `CORE-WP-0010-T01` inventory | done | Inventory recorded in `docs/specs/runtime-absorption-inventory.md` |
| `CORE-WP-0010-T02` dual-run design | wait | `HUB-WP-0004-T03` runtime packaging and `T06` joint plan |
| `CORE-WP-0010-T03` absorption slices | wait | Approved dual-run design and `HUB-WP-0004-T04` ports |
| `CORE-WP-0010-T02` dual-run design | done | Route-group plan recorded in `docs/specs/runtime-absorption-plan.md` |
| `CORE-WP-0010-T03` absorption slices | wait | Published hub-core revision/image, durable backend, candidate deployment |
| `CORE-WP-0010-T04` production cutover | wait | Absorption evidence and operator approval |
| `CORE-WP-0010-T05` archive | wait | Cutover complete and residual ownership recorded |
@ -68,6 +68,8 @@ curl -fsS https://hub.coulomb.social/readyz
- `CORE-WP-0011` is finished but has no State Hub UUID; the registrar warning is
retained rather than inventing an identifier.
- `.custodian-brief.md` is generated and may lag file-backed workplan truth.
- Hub-core T01–T05 are complete in its workplan and State Hub, but Slice 0 must
pin a committed source revision and immutable image before implementation.
- `make lint` currently reports seven baseline style findings in unchanged
Python files (two import-order findings and five long lines); `make test`
passes all 24 tests.

2
WORK-RECORDS.md
View file

@ -50,7 +50,7 @@
| task | CORE-WP-0009-T05 | done | — | workplans/CORE-WP-0009-hub-core-adoption-and-integration.md |
| task | CORE-WP-0009-T06 | done | — | workplans/CORE-WP-0009-hub-core-adoption-and-integration.md |
| task | CORE-WP-0010-T01 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md |
| task | CORE-WP-0010-T02 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md |
| task | CORE-WP-0010-T02 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md |
| task | CORE-WP-0010-T03 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md |
| task | CORE-WP-0010-T04 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md |
| task | CORE-WP-0010-T05 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md |

1
docs/specs/README.md
View file

@ -9,6 +9,7 @@ This directory is the specification map for Core Hub. The specs are intentionall
- [API v2 Compatibility](api-v2-compatibility.md) - Inter-Hub compatibility surface to preserve during transition
- [Inter-Hub Legacy Inventory](interhub-legacy-inventory.md) - initial route/data/consumer inventory for migration
- [Runtime Absorption Inventory](runtime-absorption-inventory.md) - current routes, data, consumers, and ownership that hub-core absorption must preserve
- [Runtime Absorption Plan](runtime-absorption-plan.md) - ordered dual-run route groups, migration gates, traffic control, and rollback
- [Contract Governance](contract-governance.md) - how contract changes are reviewed and recorded
- [Data Model](data-model.md) - initial entity model and migration posture
- [Event and Registry Model](event-and-registry-model.md) - event catalogs, manifests, widgets, and capability registries

168
docs/specs/runtime-absorption-plan.md Normal file
View file

@ -0,0 +1,168 @@
# Core Hub to hub-core absorption plan
**Status:** recorded for `CORE-WP-0010-T02`; joint artifact for `HUB-WP-0004-T06`
**Reviewed:** 2026-08-21
**Strategy:** route-group migration with Core Hub retained as rollback; no
client dual-write and no big-bang cutover
This plan turns the inventory in `runtime-absorption-inventory.md` into ordered,
testable migration slices. Hub-core owns the surviving runtime selected by its
ADR-0001. Core Hub remains the public `/api/v2` authority until each route group
passes its own data, contract, consumer, and rollback gates.
## Decisions
1. **One target runtime.** Hub-core ships the importable wheel and primary OCI
image. Core Hub is a temporary source and rollback runtime, not a permanent
host for hub-core.
2. **Internal candidate first.** A hub-core candidate deployment receives no
public traffic until it has a published source revision, immutable image
digest, durable PostgreSQL backend, migrations, compatible authentication,
and green readiness.
3. **Route groups, not percentages.** Traffic moves by coherent API group.
Random percentage canaries would send related bootstrap calls to different
authorities and are not permitted.
4. **One writer per route group.** Read-only requests may be mirrored for
comparison. Production clients never dual-write. Write cutover uses a short
drain, final delta replay, route switch, and one active writer.
5. **Isolated metadata during dual-run.** Core Hub keeps the `core_hub` schema;
hub-core uses its own migration lineage and schema. Cross-importing either
service's SQLAlchemy metadata is prohibited.
6. **Stable identity and safe key migration.** Durable IDs are preserved.
API-key prefixes, hashes, scopes, and status may move; raw bearer tokens or
one-time full keys must never enter migration bundles or evidence.
7. **Archive last.** Core Hub stays deployable until the final stabilization
gate, reverse-delta requirement, and residual ownership record are closed.
## Entry gates
Absorption implementation does not begin until all of these are evidenced:
- the completed `HUB-WP-0004-T01`–`T05` work is committed and pushed;
- a hub-core wheel and OCI image identify the same revision and version;
- `rapp-core-hub` can deploy a second, cluster-internal candidate without
changing `hub.coulomb.social`;
- hub-core has a production PostgreSQL `PortStore`/compatibility store and
fails readiness when durable storage is unavailable;
- the existing operator token and copied API-key hashes authenticate through
the candidate without exposing credential values;
- Core Hub export, hub-core import, row-count, canonical-digest, and delta
replay commands are repeatable and record non-secret migration metadata;
- the hub-core conformance profile and Core Hub compatibility tests pass
against the candidate.
The current in-memory hub-core backend is a conformance implementation, not an
absorption target. Its deliberate production-readiness failure is a useful
gate and must not be bypassed.
## Ordered slices
| Slice | Surface | Move condition | Consumer gate |
| --- | --- | --- | --- |
| 0 — candidate foundation | image, API/MCP processes, migrations, durable store, auth adapter, health/readiness | All entry gates above; no public route | hub-core conformance plus Core Hub contract suite |
| 1 — stateless compatibility | widget/event/annotation/policy catalogs; empty annotation, requirement, decision, deployment, and outcome collections; compatibility OpenAPI/docs | Response status, headers, and canonical JSON match; unprefixed OpenAPI aliases retained | unauthenticated catalog and OpenAPI probes |
| 2 — protected bootstrap | hubs, capability manifests and activation, API consumers/key issuance, widgets, hub registry | IDs/FKs preserved; source and target counts/digests match; copied key hashes authenticate; isolated create/read fixture passes | complete ops-hub bootstrap/gate probe |
| 3 — interaction evidence | interaction-event append/list and mapping to `port.events.interaction` | Historical IDs/timestamps retained; adapter mapping is reversible; append/readback fixture passes | activity-core resolver and evidence-sink probe |
| 4 — operator surface | `/console`, readiness summary, compatibility redirects and remaining aliases | Console reads target projections; no source-only API calls remain | operator CLI, protected console, visual smoke |
| 5 — public authority | all `/api/v2`, health/readiness, API/MCP entrypoints | Earlier slices stable; rollback and residual-owner evidence current | ops-hub and activity-core gates together |
Slice 2 is intentionally atomic at the traffic boundary. The ops-hub bootstrap
creates records whose identifiers cross hubs, manifests, consumers, keys, and
widgets; splitting those calls across two authorities would manufacture
referential drift. Its code and data migration may still land in smaller
reviewable commits before the route group switches.
## Compatibility adapters
Hub-core's named ports are not themselves the legacy `/api/v2` contract. The
target needs an explicit compatibility composition layer:
- hub and manifest resources map to `port.registry` while preserving legacy
IDs, body fields, activation behavior, pagination envelope, and hub-registry
projection;
- interaction events map to `port.events.interaction` with a stable
correlation ID, original event ID, widget reference, view context, event
type, metadata, and occurrence/recording timestamps;
- widgets and API consumers/keys remain compatibility resources until a named
replacement port is accepted; they must not be forced into registry records;
- health/readiness retain the response shape expected by deployed Core Hub
smoke tooling until consumers adopt the hub-core-native shape;
- `/api/v2/openapi.json`, YAML/docs discovery, and required unprefixed aliases
remain versioned compatibility artifacts through final cutover.
## Data movement and comparison
For each durable slice:
1. Export source records in dependency order with a schema version, source
revision, high-water mark, and bundle SHA-256.
2. Validate without writing; reject unknown fields, missing parents, raw secret
shapes, duplicate identities, or a lower target schema version.
3. Upsert into the target by stable ID. Re-running the same bundle must be
idempotent and recorded as such.
4. Compare table/resource counts, sorted identity sets, and canonical row
digests. Timestamp formatting and JSON key order are normalized before
hashing; semantic field differences are not ignored.
5. Mirror only safe GET requests and compare status plus canonical response.
Exercise writes with isolated migration fixtures, never by duplicating a
production client request.
6. At route cutover, drain writes for that group, capture the final high-water
delta, import/compare it, switch the route group, and enable target writes.
Migration evidence may contain IDs, counts, hashes, versions, timestamps, and
pass/fail results. It must not contain raw API keys, bearer tokens, database
credentials, Authorization headers, or copied Kubernetes Secrets.
## Traffic controls and rollback
`rapp-core-hub` owns one logical switch per slice with the states `source`,
`shadow-read`, and `target`. The concrete gateway or Helm key names are a
deployment implementation detail, but every rendered release must reveal the
selected state as non-secret evidence.
Rollback remains possible while a slice is target-owned:
1. stop new target writes for the affected route group;
2. export and verify the target-only delta by stable ID;
3. replay that delta to Core Hub when the old source must become writable;
4. compare counts/digests and run the consumer gate;
5. switch the whole route group back to `source`.
If reverse replay is not implemented for a write slice, rollback is limited to
the period before target writes are enabled. That limitation blocks public
cutover and must be visible in the evidence report.
## Required evidence per slice
- hub-core source revision, package version, and OCI digest;
- Core Hub source revision and deployed Helm revision;
- source/target schema versions and migration bundle hash;
- before/after counts, identity-set comparison, and canonical digest result;
- compatibility/OpenAPI diff result;
- auth positive and negative checks without credential disclosure;
- named consumer smoke reports and correlation IDs;
- selected route state, switch time, observer, and rollback result or reason it
was not invoked.
Any schema mismatch, unexplained count/digest drift, secret-shaped evidence,
auth broadening, consumer failure, or non-green readiness stops the slice and
keeps public authority on Core Hub.
## Ownership
| Owner | Responsibility |
| --- | --- |
| hub-core | compatibility modules, durable store/migrations, import/delta/reverse-delta, conformance |
| core-hub | source export, legacy fixtures/OpenAPI, canonical response comparison, rollback service |
| rapp-core-hub | candidate deployment, immutable image pin, route switches, rollout/rollback evidence |
| railiance-platform / rapp-postgres | PostgreSQL, credential delivery, backup/restore readiness |
| ops-hub | bootstrap and registry consumer gate |
| activity-core | interaction resolver, append, and readback gate |
| operator | approval for Slice 5 public-authority cutover and later repository archive |
## Next executable work
After hub-core publishes its completed T01–T05 revision, start Slice 0. The
first implementation commit should add the durable PostgreSQL boundary and
legacy auth/health compatibility in hub-core, plus a candidate deployment mode
in `rapp-core-hub`. It must not change public DNS or ingress.

38
workplans/CORE-WP-0010-runtime-absorption-and-archive.md
View file

@ -40,13 +40,24 @@ CoulombCore.
## Readiness and blocker review — 2026-08-21
`CORE-WP-0011` is finished: production now runs on railiance01 and the active
CoulombCore runtimes are stopped. The runtime surface inventory is complete in
`docs/specs/runtime-absorption-inventory.md`. The remaining work is blocked,
not merely unstarted: `HUB-WP-0004` is still proposed and its runtime packaging
(`T03`), minimal ports (`T04`), and joint absorption plan (`T06`) determine the
shape of every subsequent task here. This workplan can return to `active` when
those decisions are owned and ready for joint execution.
At the start of the review, `CORE-WP-0011` was finished and the runtime surface
inventory was complete, but `HUB-WP-0004` was still proposed. Its runtime
packaging (`T03`), minimal ports (`T04`), and joint absorption plan (`T06`) were
therefore recorded as the blockers. The advancement below supersedes that
dependency snapshot.
## Dependency advancement — 2026-08-21
Hub-core has completed `HUB-WP-0004-T01`–`T05` in its file-backed workplan and
State Hub: the primary-image decision, Tier 1 contracts, minimal port runtime,
and initial conformance harness now exist. The paired design is recorded in
`docs/specs/runtime-absorption-plan.md`, completing this workplan's T02 input
and the Core Hub half of `HUB-WP-0004-T06`.
Execution remains blocked on publishing the completed hub-core work at an
immutable source/image revision and replacing its deliberately ephemeral
in-memory backend with a production PostgreSQL boundary. Those are explicit
Slice 0 entry gates, not reasons to reopen the architecture decision.
## Inventory runtime surfaces to absorb
@ -69,7 +80,7 @@ and move constraints.
```task
id: CORE-WP-0010-T02
status: wait
status: done
priority: high
state_hub_task_id: "d2679b41-01e9-477c-9d8a-0b1ddc8847cf"
```
@ -78,7 +89,10 @@ Define how hub-core serves equivalent `/api/v2` (or compatible gateway) while
core-hub remains rollback. Include feature flags, traffic shift, and smoke
ownership.
Waiting on `HUB-WP-0004-T03` and `HUB-WP-0004-T06`.
Completed 2026-08-21. `docs/specs/runtime-absorption-plan.md` defines the
route-group order, one-writer rule, data/delta checks, compatibility adapters,
consumer gates, traffic controls, rollback, evidence, and ownership. Hub-core
must link the joint plan to close `HUB-WP-0004-T06`.
## Execute absorption slices
@ -92,7 +106,9 @@ state_hub_task_id: "c01ddec1-5737-4115-ac73-c402ffb5bdc5"
Migrate in slices with consumer smokes (ops-hub, activity-core) after each.
Record row counts and fixture replays per slice.
Waiting on the approved dual-run design and `HUB-WP-0004-T04`.
Waiting on a committed/published hub-core runtime revision, immutable image,
durable PostgreSQL backend, and candidate deployment. Start with Slice 0; do
not change public traffic.
## Cutover production traffic
@ -125,7 +141,7 @@ Waiting on production cutover and recorded residual ownership.
## Acceptance
- [x] Absorption inventory complete
- [ ] Dual-run design approved/recorded
- [x] Dual-run design approved/recorded
- [ ] Consumer smokes pass on hub-core host
- [ ] Production traffic cut over with evidence
- [ ] core-hub archived read-only; residuals owned elsewhere