docs(ecosystem): complete hub stack consolidation (CUST-WP-0057)

Publish ecosystem architecture, event mapping, capability bridge, and async-base decision docs; add cross-repo regression script; update extraction boundary and bootstrap status; archive WP-0058 touch-ups.
2026-07-11 01:26:53 +02:00 · 2026-07-11 01:26:53 +02:00 · aab187937b
commit aab187937b
parent 10d691886d
9 changed files with 717 additions and 11 deletions
--- a/docs/fos-hub-bootstrap-sequence-status.md
+++ b/docs/fos-hub-bootstrap-sequence-status.md
@ -1,6 +1,6 @@
 # FOS Hub Bootstrap Sequence Status

-Updated: 2026-07-08
+Updated: 2026-07-09

 ## Purpose

@ -24,6 +24,7 @@ Do not restart FOS bootstrap at the old `NK-WP-0001` Keycloak path. That workpla
 | Ops hub / Core Hub | T13–T18 done; T16 deployed evidence and T17 cutover coupling closed 2026-07-08. Production serves Core Hub at `hub.coulomb.social` since 2026-07-03. | Monitor `CORE-WP-0007` stabilization window; Haskell retirement is operator-gated rollback cleanup only. |
 | Legacy Inter-Hub | `CUST-WP-0047-T05` and `CUST-WP-0049-T06` superseded 2026-07-03. | Rollback-only; do not request new Inter-Hub operator keys. |
 | Fin hub/business | T20–T23 done (canon + fin-hub scaffold + models). T24–T26 remain open. | Implement cost ingestion, runway calculator, cross-hub coupling, and pricing/legal packaging. |
+| Hub ecosystem alignment | `CUST-WP-0057` active — architecture doc, hub-core utils adoption in core-hub, boundary workplans. | Execute `HUB-WP-0003`, `CORE-WP-0009`, `STATE-WP-0074`; run `scripts/hub-ecosystem-regression.sh`. |

 ## Stable Pickup Order

--- a/docs/hub-core-extraction-boundary.md
+++ b/docs/hub-core-extraction-boundary.md
@ -245,11 +245,20 @@ to import it:
 5. `cd /home/worsch/state-hub && make test` remains green before and after the
   editable `hub-core` dependency is introduced.

+## Ecosystem composition (CUST-WP-0057)
+
+hub-core is the **library layer** only. It does not own:
+
+- core-hub framework tables (`hubs`, `widgets`, `hub_capability_manifests`, …)
+- state-hub dev-hub tables (workplans, tasks, topics, …)
+
+Integration canon: `docs/hub-ecosystem-architecture.md`
+
 ## Next Step

-**CUST-WP-0048** is finished; **CUST-WP-0025-T08** is done. Proceed to Phase 2
-dev-hub rename (**CUST-WP-0025-T09+**): MCP server name, config migration, and
-integration-point renames. Optional follow-up extractions (not blocking rename):
+**CUST-WP-0048** is finished; **CUST-WP-0025-T08** is done. Ecosystem
+consolidation continues under **CUST-WP-0057** (compose hub-core, state-hub,
+core-hub without merging repos). Optional follow-up extractions:

 - map State Hub `ProgressEvent` foreign keys into `subject_refs`
 - map CapabilityRequest workplan/task columns into JSON context fields
--- a/docs/hub-ecosystem-architecture.md
+++ b/docs/hub-ecosystem-architecture.md
@ -0,0 +1,105 @@
+# Hub Ecosystem Architecture
+
+**Updated:** 2026-07-09  
+**Workplan:** `CUST-WP-0057`  
+**Status:** Active decision record
+
+---
+
+## Purpose
+
+Define how `hub-core`, `state-hub`, and `core-hub` compose into one FOS hub
+stack. This document resolves naming confusion and entity ownership without
+merging repositories.
+
+---
+
+## Three-layer model
+
+| Layer | Repo | Role | Production URL |
+| --- | --- | --- | --- |
+| **Library** | `hub-core` | Shared models, schemas, router factories, MCP wrapper, utils | None — pip/uv import only |
+| **Dev coordination** | `state-hub` | File-first workplans, tasks, decisions, consistency sync, dev MCP | Local / operator (`127.0.0.1:8000`) |
+| **Production framework** | `core-hub` | Inter-Hub successor: hubs, manifests, widgets, `/api/v2`, operator console | `hub.coulomb.social` |
+
+```text
+                    ┌─────────────────┐
+                    │    core-hub     │  Production framework (/api/v2)
+                    │  (async service)│
+                    └────────┬────────┘
+                             │ imports utils; HTTP bridge to dev data (future)
+              ┌──────────────┼──────────────┐
+              ▼              ▼              ▼
+       ┌────────────┐  ┌────────────┐  ┌────────────┐
+       │  hub-core  │◄─│ state-hub  │  │  ops-hub   │
+       │  (library) │  │ (dev host) │  │ (consumer) │
+       └────────────┘  └────────────┘  └────────────┘
+```
+
+---
+
+## Naming glossary
+
+| Name | What it is | What it is not |
+| --- | --- | --- |
+| **hub-core** | Python package `hub_core` — library boundary | A running hub service |
+| **core-hub** | Production interaction framework service | The shared library |
+| **state-hub** | Dev coordination host (State Hub / dev-hub) | The Inter-Hub `/api/v2` framework |
+
+Mnemonic: **hub-core** = core *primitives*; **core-hub** = core *framework runtime*.
+
+---
+
+## Canonical entity ownership
+
+| Entity family | Canonical owner | Notes |
+| --- | --- | --- |
+| `domains`, `managed_repos` (registry) | `hub-core` models; `state-hub` host | core-hub uses `hubs` table — see capability bridge doc |
+| `agent_messages` | `hub-core` + `state-hub` | core-hub does not duplicate; reads via State Hub API when needed |
+| `progress_events` | `hub-core` + `state-hub` | Dev coordination audit trail |
+| `interaction_events` | `core-hub` | Widget-scoped ops evidence (`/api/v2/interaction-events`) |
+| `capability_catalog`, `capability_requests` | `hub-core` + `state-hub` | Maturity and workflow lane |
+| `hub_capability_manifests` | `core-hub` | Inter-Hub manifest contract |
+| `hubs`, `widgets`, `api_consumers`, `api_keys` | `core-hub` | Framework registry only |
+| `workplans`, `tasks`, `decisions` | `state-hub` | File-first authority; core-hub may index later |
+| `tpsc_*`, policy documents | `hub-core` + `state-hub` | core-hub references evidence, does not reimplement |
+| Slug/pagination/path utils | `hub-core` | All hosts import; no local duplicates |
+
+---
+
+## Governing decisions
+
+1. **Compose, do not merge** — three repos remain; shared code moves into `hub-core`.
+2. **hub-core stays a library** — no `uvicorn` entrypoint, no K8s manifests.
+3. **core-hub owns `/api/v2`** — Inter-Hub compatibility is non-negotiable for ops-hub and activity-core.
+4. **state-hub owns dev coordination** — workplan files remain source of truth until a recorded ADR says otherwise.
+5. **Separate SQLAlchemy metadata** — each runtime keeps its own `Base`; router factories inject host models.
+6. **Incremental adoption** — core-hub adopts hub-core utilities first; framework models stay local.
+
+---
+
+## Integration contracts
+
+| From | To | Mechanism | Status |
+| --- | --- | --- | --- |
+| `state-hub` | `hub-core` | Editable package import | Done (`CUST-WP-0048`) |
+| `core-hub` | `hub-core` | Editable package import (utils) | In progress (`CORE-WP-0009`) |
+| `core-hub` | `state-hub` | HTTP read APIs for progress/messages | Deferred (`STATE-WP-0070`) |
+| `ops-hub` | `core-hub` | `/api/v2` bootstrap | Production |
+| Agents | `state-hub` | dev-hub MCP | Production |
+| Operators | `core-hub` | Operator CLI + console | Production |
+
+See also:
+
+- `hub-ecosystem-event-mapping.md` — progress vs interaction events
+- `hub-ecosystem-capability-bridge.md` — catalog vs manifest IDs
+- `hub-core-extraction-boundary.md` — library extraction history
+
+---
+
+## Non-goals
+
+- Renaming `hub-core` or `core-hub` repositories
+- Moving State Hub workplan tables into core-hub in this lane
+- Replacing core-hub `/api/v2` with state-hub routes
+- Publishing hub-core to public PyPI (path/editable deps remain fine)
--- a/docs/hub-ecosystem-async-base-decision.md
+++ b/docs/hub-ecosystem-async-base-decision.md
@ -0,0 +1,32 @@
+# Hub Ecosystem — Async SQLAlchemy Base Decision
+
+**Updated:** 2026-07-09  
+**Workplan:** `HUB-WP-0003-T04`  
+**Decision:** Defer shared async base; hosts keep separate metadata
+
+---
+
+## Question
+
+Should `hub_core.models.base` expose an async-compatible declarative base for
+core-hub (async SQLAlchemy) while state-hub remains on sync sessions?
+
+---
+
+## Finding
+
+- `state-hub` uses sync SQLAlchemy with FastAPI dependency-injected sessions.
+- `core-hub` uses `sqlalchemy.ext.asyncio` with a separate `core_hub.db.Base`.
+- Router factories in hub-core accept **host-injected models** — they do not
+  require a shared metadata registry across runtimes.
+- Merging bases would couple migration ownership and risk breaking State Hub's
+  400+ test regression without benefit for the first consolidation slice.
+
+---
+
+## Decision
+
+**Defer.** Each runtime keeps its own `Base` and Alembic history. hub-core
+documents metadata isolation in `docs/metadata-isolation.md`. Revisit only if
+a second async host needs to mount hub-core router factories with shared tables
+in the same database instance.
--- a/docs/hub-ecosystem-capability-bridge.md
+++ b/docs/hub-ecosystem-capability-bridge.md
@ -0,0 +1,91 @@
+# Hub Ecosystem Capability Bridge
+
+**Updated:** 2026-07-09  
+**Workplan:** `CUST-WP-0057-T07`
+
+---
+
+## Purpose
+
+Map reuse-surface capability IDs to State Hub catalog rows and Core Hub
+`hub_capability_manifests` without inventing parallel naming schemes.
+
+---
+
+## Three representations
+
+| Representation | Location | Identifier | Purpose |
+| --- | --- | --- | --- |
+| **Reuse-surface registry** | `registry/capabilities/*.md` + federation index | `capability.<domain>.<name>` | Discovery, maturity, consumer guidance |
+| **State Hub catalog** | `capability_catalog` table | UUID row + `capability_type` + `title` | Dev workflow, requests, maturity tracking |
+| **Core Hub manifest** | `hub_capability_manifests` table | UUID row + `body_json` | Inter-Hub `/api/v2` bootstrap, widget activation |
+
+These are complementary, not duplicates of the same row.
+
+---
+
+## Worked example: hub stack capabilities
+
+### `capability.infotech.hub-core-library`
+
+| Field | Value |
+| --- | --- |
+| Registry owner | `hub-core` |
+| State Hub catalog | Optional row with `capability_type=library`, linked to `hub-core` repo |
+| Core Hub manifest | Not required — library is not a deployed hub |
+
+### `capability.infotech.core-hub`
+
+| Field | Value |
+| --- | --- |
+| Registry owner | `core-hub` |
+| State Hub catalog | Row referencing `core-hub` managed repo |
+| Core Hub manifest | Activated manifest for `hubSlug=core-hub` with widgets for ops evidence |
+
+**Manifest `body_json` should include:**
+
+```json
+{
+  "reuseSurfaceId": "capability.infotech.core-hub",
+  "manifestVersion": "0.1.0",
+  "capabilities": []
+}
+```
+
+### `capability.statehub.workstream-coordinate` (if registered)
+
+| Field | Value |
+| --- | --- |
+| Registry owner | `state-hub` |
+| State Hub catalog | Native — describes the dev coordination service |
+| Core Hub manifest | None — state-hub is not an Inter-Hub framework hub |
+
+---
+
+## Bridge rules
+
+1. **Registry ID is the stable public name** — manifests and catalog rows reference it in JSON metadata.
+2. **Catalog rows are dev-lane** — maturity, disputes, requests, WSJF triage live in State Hub.
+3. **Manifests are ops-lane** — bootstrap, API consumers, widgets, interaction events live in Core Hub.
+4. **Activation does not copy secrets** — API key material stays hash/prefix only in Core Hub.
+5. **One reuse-surface ID per deployed hub** — ops-hub manifest references `capability.infotech.ops-hub` (when registered), not the core-hub library ID.
+
+---
+
+## Registration flow (recommended)
+
+```text
+1. Author capability.infotech.<name> in repo registry/
+2. State Hub: register repo + optional capability_catalog row citing reuseSurfaceId
+3. Core Hub: POST /api/v2/hub-capability-manifests with reuseSurfaceId in body
+4. Core Hub: activate manifest → widgets → interaction-events for ops evidence
+5. reuse-surface federation publish indexes maturity from registry files
+```
+
+---
+
+## Anti-patterns
+
+- Using `hub-core` manifest activation for a library that has no runtime widgets
+- Creating a second capability ID in manifest JSON that disagrees with reuse-surface
+- Storing full API keys in State Hub progress or catalog rows
--- a/docs/hub-ecosystem-event-mapping.md
+++ b/docs/hub-ecosystem-event-mapping.md
@ -0,0 +1,114 @@
+# Hub Ecosystem Event Mapping
+
+**Updated:** 2026-07-09  
+**Workplan:** `CUST-WP-0057-T06`
+
+---
+
+## Purpose
+
+Clarify when to use State Hub `ProgressEvent` vs Core Hub `InteractionEvent`,
+and how downstream consumers (activity-core, ops-hub) should interpret each.
+
+---
+
+## Event families
+
+| Family | Owner | Table | Typical producer | Typical consumer |
+| --- | --- | --- | --- | --- |
+| **Progress** | state-hub | `progress_events` | Agents, consistency sync, custodian operators | State Hub MCP, dashboard, audit |
+| **Interaction** | core-hub | `interaction_events` | ops-hub bootstrap, widget verification, operator CLI | activity-core sink, ops evidence |
+
+Both are append-oriented and must not contain secrets.
+
+---
+
+## ProgressEvent (dev coordination)
+
+**Schema owner:** `hub_core.models.progress_event` (library); State Hub extends with FKs.
+
+**Emit when:**
+
+- A workplan task status changes
+- An agent logs session progress (`POST /progress/`)
+- Consistency sync or reconciliation completes
+- A custodian operator records a milestone
+
+**Key fields:**
+
+- `event_type` — e.g. `note`, `milestone`, `risk`, `alert`
+- `summary` — human-readable one-liner
+- `detail` — JSON payload (non-secret)
+- `subject_refs` — generic JSON linking workplan_id, task_id, topic_id, repo_slug
+
+**Example `subject_refs`:**
+
+```json
+{
+  "workplan_id": "CUST-WP-0057",
+  "task_id": "CUST-WP-0057-T01",
+  "repo": "the-custodian"
+}
+```
+
+---
+
+## InteractionEvent (ops / framework evidence)
+
+**Schema owner:** `core_hub.models.InteractionEvent`
+
+**Emit when:**
+
+- ops-hub bootstrap verifies an endpoint (`ops-endpoint-verified`)
+- Operator CLI deployed smoke records widget activity
+- A widget emits framework-scoped evidence under `/api/v2/interaction-events`
+
+**Key fields:**
+
+- `widget_id` — FK to core-hub `widgets`
+- `event_type` — cataloged in `event-types` seed
+- `view_context`, `metadata_json`, `body_json`
+
+**Example body:**
+
+```json
+{
+  "hubSlug": "ops-hub",
+  "endpoint": "https://hub.coulomb.social/api/v2/hubs",
+  "status": "verified"
+}
+```
+
+---
+
+## Mapping rules
+
+| Situation | Use | Do not |
+| --- | --- | --- |
+| Agent closes a workplan task | `ProgressEvent` via State Hub | Duplicate as `InteractionEvent` |
+| ops-hub bootstrap smoke passes | `InteractionEvent` via Core Hub | POST to State Hub progress |
+| Cutover gate evaluation | `ProgressEvent` (custodian audit) + optional `InteractionEvent` (ops evidence) | Single combined table |
+| activity-core ingestion | `core-hub-interaction-event` sink for interaction family | Assume progress events appear in Core Hub DB |
+
+**Correlation:** When both families describe one operator action, use a shared
+non-secret `correlation_id` in `detail` (progress) and `metadata_json`
+(interaction). Do not dual-write the same semantic event to both tables.
+
+---
+
+## Consumer guidance
+
+### activity-core
+
+- Primary sink: Core Hub interaction events (`core-hub-interaction-event`)
+- State Hub progress remains fallback during transition (see `testing-release-and-migration.md`)
+
+### ops-hub
+
+- Bootstrap and gate probes target Core Hub `/api/v2`
+- Progress notes about bootstrap planning belong in State Hub
+
+### Agents (MCP)
+
+- Use State Hub `append_progress` for coordination work
+- Do not call Core Hub interaction-event APIs unless executing ops-widget workflows