From 4a9df801c34dc14f6e041a0cc5c9a1ca6ff12990 Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 21 Aug 2026 10:57:13 +0200 Subject: [PATCH] docs: define hub-core absorption plan --- INTENT.md | 15 +- README.md | 5 +- STATE.md | 12 +- WORK-RECORDS.md | 2 +- docs/specs/README.md | 1 + docs/specs/runtime-absorption-plan.md | 168 ++++++++++++++++++ ...-WP-0010-runtime-absorption-and-archive.md | 38 ++-- 7 files changed, 216 insertions(+), 25 deletions(-) create mode 100644 docs/specs/runtime-absorption-plan.md diff --git a/INTENT.md b/INTENT.md index 4cd2c51..dfa36e0 100644 --- a/INTENT.md +++ b/INTENT.md @@ -42,13 +42,14 @@ The framework must be usable by humans and agents. Human operators need a stable | Repo | Role | | --- | --- | -| `hub-core` | Shared library — models, router factories, utils (`hub_core.utils`) | +| `hub-core` | Surviving framework/runtime — contracts, ports, package, and primary image | | `state-hub` | Dev coordination — file-first workplans, dev MCP | | `core-hub` | Production service — `/api/v2`, hubs, manifests, operator console | Canon: `/home/worsch/the-custodian/docs/hub-ecosystem-architecture.md` -**Naming:** `core-hub` = core *framework runtime*. `hub-core` = core *primitives library*. +**Transition naming:** `core-hub` remains the current production service and +rollback authority while `hub-core` becomes the surviving framework/runtime. ## Current State @@ -63,7 +64,9 @@ This delivers the practical Gen3 production reset, but not the whole product intent. Messaging, progress/work projections, several durable evidence and decision resources, and a single extension runtime remain outside Core Hub or only partially represented. `CORE-WP-0010` tracks absorption into hub-core and -repository archive; it is blocked on the runtime packaging, minimal-port, and -joint-plan decisions in `HUB-WP-0004`. The healthy service remains the -production and rollback authority until dual-run consumer gates prove a -replacement. See `STATE.md` for current operational truth. +repository archive. Hub-core's packaging, contract, port, and conformance +foundations are complete, and the joint route-group migration design is +recorded. Execution now waits on a published immutable hub-core revision, a +durable PostgreSQL backend, and an internal candidate deployment. The healthy +service remains the production and rollback authority until dual-run consumer +gates prove a replacement. See `STATE.md` for current operational truth. diff --git a/README.md b/README.md index 4955116..4590059 100644 --- a/README.md +++ b/README.md @@ -44,5 +44,6 @@ It carries forward the lessons from State Hub and Inter-Hub: workplans and progr The contract, FastAPI/Postgres foundation, Inter-Hub migration, Haskell/IHP retirement, and relocation from CoulombCore to railiance01 are complete. `CORE-WP-0011` is finished. The only open local workplan is runtime absorption -and eventual archive (`CORE-WP-0010`); it is blocked on hub-core's runtime -packaging, minimal-port, and joint-plan decisions in `HUB-WP-0004`. +and eventual archive (`CORE-WP-0010`). Its dual-run design is recorded; Slice 0 +waits on a published hub-core runtime revision, durable PostgreSQL backend, and +internal candidate deployment before any public traffic changes. diff --git a/STATE.md b/STATE.md index 69ab650..60f8198 100644 --- a/STATE.md +++ b/STATE.md @@ -6,8 +6,8 @@ ## One-line posture **Core Hub is the verified production `/api/v2` runtime on railiance01; the -CoulombCore APIs are stopped, and future absorption into hub-core is blocked on -hub-core's still-proposed runtime contract.** +joint hub-core absorption plan is recorded, and Slice 0 waits on a published +hub-core runtime with durable PostgreSQL storage.** ## Production truth @@ -32,15 +32,15 @@ Deployment packaging and rollout truth lives in the `rapp-core-hub` repository. | --- | --- | --- | | `CORE-WP-0001`–`CORE-WP-0009` | finished / archived | Gen3 runtime, contracts, persistence, consumer gates, and hardening delivered | | `CORE-WP-0011` | finished | Production relocated to railiance01 and active CoulombCore runtime retired | -| `CORE-WP-0010` | blocked | Runtime absorption and archive wait on `HUB-WP-0004` | +| `CORE-WP-0010` | blocked | Dual-run design done; Slice 0 publication/storage gates remain | ### Open work | Task | Status | Dependency | | --- | --- | --- | | `CORE-WP-0010-T01` inventory | done | Inventory recorded in `docs/specs/runtime-absorption-inventory.md` | -| `CORE-WP-0010-T02` dual-run design | wait | `HUB-WP-0004-T03` runtime packaging and `T06` joint plan | -| `CORE-WP-0010-T03` absorption slices | wait | Approved dual-run design and `HUB-WP-0004-T04` ports | +| `CORE-WP-0010-T02` dual-run design | done | Route-group plan recorded in `docs/specs/runtime-absorption-plan.md` | +| `CORE-WP-0010-T03` absorption slices | wait | Published hub-core revision/image, durable backend, candidate deployment | | `CORE-WP-0010-T04` production cutover | wait | Absorption evidence and operator approval | | `CORE-WP-0010-T05` archive | wait | Cutover complete and residual ownership recorded | @@ -68,6 +68,8 @@ curl -fsS https://hub.coulomb.social/readyz - `CORE-WP-0011` is finished but has no State Hub UUID; the registrar warning is retained rather than inventing an identifier. - `.custodian-brief.md` is generated and may lag file-backed workplan truth. +- Hub-core T01–T05 are complete in its workplan and State Hub, but Slice 0 must + pin a committed source revision and immutable image before implementation. - `make lint` currently reports seven baseline style findings in unchanged Python files (two import-order findings and five long lines); `make test` passes all 24 tests. diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 4054652..65a6e2e 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -50,7 +50,7 @@ | task | CORE-WP-0009-T05 | done | — | workplans/CORE-WP-0009-hub-core-adoption-and-integration.md | | task | CORE-WP-0009-T06 | done | — | workplans/CORE-WP-0009-hub-core-adoption-and-integration.md | | task | CORE-WP-0010-T01 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | -| task | CORE-WP-0010-T02 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | +| task | CORE-WP-0010-T02 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0010-T03 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0010-T04 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0010-T05 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | diff --git a/docs/specs/README.md b/docs/specs/README.md index 3a14e09..3175838 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -9,6 +9,7 @@ This directory is the specification map for Core Hub. The specs are intentionall - [API v2 Compatibility](api-v2-compatibility.md) - Inter-Hub compatibility surface to preserve during transition - [Inter-Hub Legacy Inventory](interhub-legacy-inventory.md) - initial route/data/consumer inventory for migration - [Runtime Absorption Inventory](runtime-absorption-inventory.md) - current routes, data, consumers, and ownership that hub-core absorption must preserve +- [Runtime Absorption Plan](runtime-absorption-plan.md) - ordered dual-run route groups, migration gates, traffic control, and rollback - [Contract Governance](contract-governance.md) - how contract changes are reviewed and recorded - [Data Model](data-model.md) - initial entity model and migration posture - [Event and Registry Model](event-and-registry-model.md) - event catalogs, manifests, widgets, and capability registries diff --git a/docs/specs/runtime-absorption-plan.md b/docs/specs/runtime-absorption-plan.md new file mode 100644 index 0000000..8324f7b --- /dev/null +++ b/docs/specs/runtime-absorption-plan.md @@ -0,0 +1,168 @@ +# Core Hub to hub-core absorption plan + +**Status:** recorded for `CORE-WP-0010-T02`; joint artifact for `HUB-WP-0004-T06` +**Reviewed:** 2026-08-21 +**Strategy:** route-group migration with Core Hub retained as rollback; no +client dual-write and no big-bang cutover + +This plan turns the inventory in `runtime-absorption-inventory.md` into ordered, +testable migration slices. Hub-core owns the surviving runtime selected by its +ADR-0001. Core Hub remains the public `/api/v2` authority until each route group +passes its own data, contract, consumer, and rollback gates. + +## Decisions + +1. **One target runtime.** Hub-core ships the importable wheel and primary OCI + image. Core Hub is a temporary source and rollback runtime, not a permanent + host for hub-core. +2. **Internal candidate first.** A hub-core candidate deployment receives no + public traffic until it has a published source revision, immutable image + digest, durable PostgreSQL backend, migrations, compatible authentication, + and green readiness. +3. **Route groups, not percentages.** Traffic moves by coherent API group. + Random percentage canaries would send related bootstrap calls to different + authorities and are not permitted. +4. **One writer per route group.** Read-only requests may be mirrored for + comparison. Production clients never dual-write. Write cutover uses a short + drain, final delta replay, route switch, and one active writer. +5. **Isolated metadata during dual-run.** Core Hub keeps the `core_hub` schema; + hub-core uses its own migration lineage and schema. Cross-importing either + service's SQLAlchemy metadata is prohibited. +6. **Stable identity and safe key migration.** Durable IDs are preserved. + API-key prefixes, hashes, scopes, and status may move; raw bearer tokens or + one-time full keys must never enter migration bundles or evidence. +7. **Archive last.** Core Hub stays deployable until the final stabilization + gate, reverse-delta requirement, and residual ownership record are closed. + +## Entry gates + +Absorption implementation does not begin until all of these are evidenced: + +- the completed `HUB-WP-0004-T01`–`T05` work is committed and pushed; +- a hub-core wheel and OCI image identify the same revision and version; +- `rapp-core-hub` can deploy a second, cluster-internal candidate without + changing `hub.coulomb.social`; +- hub-core has a production PostgreSQL `PortStore`/compatibility store and + fails readiness when durable storage is unavailable; +- the existing operator token and copied API-key hashes authenticate through + the candidate without exposing credential values; +- Core Hub export, hub-core import, row-count, canonical-digest, and delta + replay commands are repeatable and record non-secret migration metadata; +- the hub-core conformance profile and Core Hub compatibility tests pass + against the candidate. + +The current in-memory hub-core backend is a conformance implementation, not an +absorption target. Its deliberate production-readiness failure is a useful +gate and must not be bypassed. + +## Ordered slices + +| Slice | Surface | Move condition | Consumer gate | +| --- | --- | --- | --- | +| 0 — candidate foundation | image, API/MCP processes, migrations, durable store, auth adapter, health/readiness | All entry gates above; no public route | hub-core conformance plus Core Hub contract suite | +| 1 — stateless compatibility | widget/event/annotation/policy catalogs; empty annotation, requirement, decision, deployment, and outcome collections; compatibility OpenAPI/docs | Response status, headers, and canonical JSON match; unprefixed OpenAPI aliases retained | unauthenticated catalog and OpenAPI probes | +| 2 — protected bootstrap | hubs, capability manifests and activation, API consumers/key issuance, widgets, hub registry | IDs/FKs preserved; source and target counts/digests match; copied key hashes authenticate; isolated create/read fixture passes | complete ops-hub bootstrap/gate probe | +| 3 — interaction evidence | interaction-event append/list and mapping to `port.events.interaction` | Historical IDs/timestamps retained; adapter mapping is reversible; append/readback fixture passes | activity-core resolver and evidence-sink probe | +| 4 — operator surface | `/console`, readiness summary, compatibility redirects and remaining aliases | Console reads target projections; no source-only API calls remain | operator CLI, protected console, visual smoke | +| 5 — public authority | all `/api/v2`, health/readiness, API/MCP entrypoints | Earlier slices stable; rollback and residual-owner evidence current | ops-hub and activity-core gates together | + +Slice 2 is intentionally atomic at the traffic boundary. The ops-hub bootstrap +creates records whose identifiers cross hubs, manifests, consumers, keys, and +widgets; splitting those calls across two authorities would manufacture +referential drift. Its code and data migration may still land in smaller +reviewable commits before the route group switches. + +## Compatibility adapters + +Hub-core's named ports are not themselves the legacy `/api/v2` contract. The +target needs an explicit compatibility composition layer: + +- hub and manifest resources map to `port.registry` while preserving legacy + IDs, body fields, activation behavior, pagination envelope, and hub-registry + projection; +- interaction events map to `port.events.interaction` with a stable + correlation ID, original event ID, widget reference, view context, event + type, metadata, and occurrence/recording timestamps; +- widgets and API consumers/keys remain compatibility resources until a named + replacement port is accepted; they must not be forced into registry records; +- health/readiness retain the response shape expected by deployed Core Hub + smoke tooling until consumers adopt the hub-core-native shape; +- `/api/v2/openapi.json`, YAML/docs discovery, and required unprefixed aliases + remain versioned compatibility artifacts through final cutover. + +## Data movement and comparison + +For each durable slice: + +1. Export source records in dependency order with a schema version, source + revision, high-water mark, and bundle SHA-256. +2. Validate without writing; reject unknown fields, missing parents, raw secret + shapes, duplicate identities, or a lower target schema version. +3. Upsert into the target by stable ID. Re-running the same bundle must be + idempotent and recorded as such. +4. Compare table/resource counts, sorted identity sets, and canonical row + digests. Timestamp formatting and JSON key order are normalized before + hashing; semantic field differences are not ignored. +5. Mirror only safe GET requests and compare status plus canonical response. + Exercise writes with isolated migration fixtures, never by duplicating a + production client request. +6. At route cutover, drain writes for that group, capture the final high-water + delta, import/compare it, switch the route group, and enable target writes. + +Migration evidence may contain IDs, counts, hashes, versions, timestamps, and +pass/fail results. It must not contain raw API keys, bearer tokens, database +credentials, Authorization headers, or copied Kubernetes Secrets. + +## Traffic controls and rollback + +`rapp-core-hub` owns one logical switch per slice with the states `source`, +`shadow-read`, and `target`. The concrete gateway or Helm key names are a +deployment implementation detail, but every rendered release must reveal the +selected state as non-secret evidence. + +Rollback remains possible while a slice is target-owned: + +1. stop new target writes for the affected route group; +2. export and verify the target-only delta by stable ID; +3. replay that delta to Core Hub when the old source must become writable; +4. compare counts/digests and run the consumer gate; +5. switch the whole route group back to `source`. + +If reverse replay is not implemented for a write slice, rollback is limited to +the period before target writes are enabled. That limitation blocks public +cutover and must be visible in the evidence report. + +## Required evidence per slice + +- hub-core source revision, package version, and OCI digest; +- Core Hub source revision and deployed Helm revision; +- source/target schema versions and migration bundle hash; +- before/after counts, identity-set comparison, and canonical digest result; +- compatibility/OpenAPI diff result; +- auth positive and negative checks without credential disclosure; +- named consumer smoke reports and correlation IDs; +- selected route state, switch time, observer, and rollback result or reason it + was not invoked. + +Any schema mismatch, unexplained count/digest drift, secret-shaped evidence, +auth broadening, consumer failure, or non-green readiness stops the slice and +keeps public authority on Core Hub. + +## Ownership + +| Owner | Responsibility | +| --- | --- | +| hub-core | compatibility modules, durable store/migrations, import/delta/reverse-delta, conformance | +| core-hub | source export, legacy fixtures/OpenAPI, canonical response comparison, rollback service | +| rapp-core-hub | candidate deployment, immutable image pin, route switches, rollout/rollback evidence | +| railiance-platform / rapp-postgres | PostgreSQL, credential delivery, backup/restore readiness | +| ops-hub | bootstrap and registry consumer gate | +| activity-core | interaction resolver, append, and readback gate | +| operator | approval for Slice 5 public-authority cutover and later repository archive | + +## Next executable work + +After hub-core publishes its completed T01–T05 revision, start Slice 0. The +first implementation commit should add the durable PostgreSQL boundary and +legacy auth/health compatibility in hub-core, plus a candidate deployment mode +in `rapp-core-hub`. It must not change public DNS or ingress. diff --git a/workplans/CORE-WP-0010-runtime-absorption-and-archive.md b/workplans/CORE-WP-0010-runtime-absorption-and-archive.md index d75aca4..bdccc51 100644 --- a/workplans/CORE-WP-0010-runtime-absorption-and-archive.md +++ b/workplans/CORE-WP-0010-runtime-absorption-and-archive.md @@ -40,13 +40,24 @@ CoulombCore. ## Readiness and blocker review — 2026-08-21 -`CORE-WP-0011` is finished: production now runs on railiance01 and the active -CoulombCore runtimes are stopped. The runtime surface inventory is complete in -`docs/specs/runtime-absorption-inventory.md`. The remaining work is blocked, -not merely unstarted: `HUB-WP-0004` is still proposed and its runtime packaging -(`T03`), minimal ports (`T04`), and joint absorption plan (`T06`) determine the -shape of every subsequent task here. This workplan can return to `active` when -those decisions are owned and ready for joint execution. +At the start of the review, `CORE-WP-0011` was finished and the runtime surface +inventory was complete, but `HUB-WP-0004` was still proposed. Its runtime +packaging (`T03`), minimal ports (`T04`), and joint absorption plan (`T06`) were +therefore recorded as the blockers. The advancement below supersedes that +dependency snapshot. + +## Dependency advancement — 2026-08-21 + +Hub-core has completed `HUB-WP-0004-T01`–`T05` in its file-backed workplan and +State Hub: the primary-image decision, Tier 1 contracts, minimal port runtime, +and initial conformance harness now exist. The paired design is recorded in +`docs/specs/runtime-absorption-plan.md`, completing this workplan's T02 input +and the Core Hub half of `HUB-WP-0004-T06`. + +Execution remains blocked on publishing the completed hub-core work at an +immutable source/image revision and replacing its deliberately ephemeral +in-memory backend with a production PostgreSQL boundary. Those are explicit +Slice 0 entry gates, not reasons to reopen the architecture decision. ## Inventory runtime surfaces to absorb @@ -69,7 +80,7 @@ and move constraints. ```task id: CORE-WP-0010-T02 -status: wait +status: done priority: high state_hub_task_id: "d2679b41-01e9-477c-9d8a-0b1ddc8847cf" ``` @@ -78,7 +89,10 @@ Define how hub-core serves equivalent `/api/v2` (or compatible gateway) while core-hub remains rollback. Include feature flags, traffic shift, and smoke ownership. -Waiting on `HUB-WP-0004-T03` and `HUB-WP-0004-T06`. +Completed 2026-08-21. `docs/specs/runtime-absorption-plan.md` defines the +route-group order, one-writer rule, data/delta checks, compatibility adapters, +consumer gates, traffic controls, rollback, evidence, and ownership. Hub-core +must link the joint plan to close `HUB-WP-0004-T06`. ## Execute absorption slices @@ -92,7 +106,9 @@ state_hub_task_id: "c01ddec1-5737-4115-ac73-c402ffb5bdc5" Migrate in slices with consumer smokes (ops-hub, activity-core) after each. Record row counts and fixture replays per slice. -Waiting on the approved dual-run design and `HUB-WP-0004-T04`. +Waiting on a committed/published hub-core runtime revision, immutable image, +durable PostgreSQL backend, and candidate deployment. Start with Slice 0; do +not change public traffic. ## Cutover production traffic @@ -125,7 +141,7 @@ Waiting on production cutover and recorded residual ownership. ## Acceptance - [x] Absorption inventory complete -- [ ] Dual-run design approved/recorded +- [x] Dual-run design approved/recorded - [ ] Consumer smokes pass on hub-core host - [ ] Production traffic cut over with evidence - [ ] core-hub archived read-only; residuals owned elsewhere