feat: add hub runtime and extension contract

docs/adr/ADR-0001-runtime-packaging.md

@@ -0,0 +1,103 @@
+# ADR-0001: Ship the primary runtime image from hub-core
+
+**Status:** Accepted
+**Date:** 2026-08-21
+**Workplan:** `HUB-WP-0004-T03`
+**Architecture:** `SHR-ARCH-HUB-0001`
+
+## Context
+
+Hub-core currently publishes only the importable `hub_core` Python package.
+Core-hub supplies the deployable FastAPI application, `/api/v2` compatibility,
+PostgreSQL drivers and migrations, OCI build, health endpoints, and operator
+surfaces. Its image vendors a hub-core source checkout through a Docker build
+context, so a production release spans two repositories and two version
+boundaries.
+
+The State Hub retirement architecture makes hub-core the surviving framework
+and runtime and archives core-hub after compatibility cutover. We therefore
+need a durable packaging boundary before implementing ports or designing the
+dual-run.
+
+## Decision
+
+Hub-core will ship the **primary HelixForge hub runtime OCI image** from this
+repository. The `hub_core` Python package remains a supported, independently
+importable SDK/library surface. Both artifacts share one source revision and
+release version.
+
+The packaging contract is:
+
+1. The base wheel keeps reusable models, schemas, router factories, contracts,
+   utilities, and MCP composition importable without starting a service.
+2. Runtime-only dependencies such as the ASGI server, PostgreSQL driver, and
+   migration tooling belong to a declared runtime dependency group/extra and
+   are installed in the image.
+3. Hub-core owns the application factory, health/readiness endpoints, runtime
+   configuration contract, packaged migrations, image definition, and
+   compatibility route composition.
+4. One image may expose separate commands for API serving, MCP serving, and
+   migration jobs. Kubernetes may run those as separate processes or workloads
+   for scaling and privilege isolation; “one primary image” does not require a
+   monolithic process.
+5. Deployment declarations, secret references, rollout policy, and live smoke
+   evidence remain in the rapp/platform repositories. They consume the
+   hub-core image rather than defining a permanent Python host.
+6. Core-hub remains a temporary rollback runtime during dual-run. Its routes,
+   contracts, migrations, and fixtures move in governed slices; hub-core must
+   not retain a steady-state dependency on the `core_hub` package or repository.
+7. The runtime keeps domain data, Git/work-record authority, authorization
+   decisions, credential custody, and schedule execution behind the ports
+   assigned by `SHR-ARCH-HUB-0001`.
+
+The initial command names and module layout are implementation details for
+`HUB-WP-0004-T04`, but they must provide equivalent surfaces for:
+
+- starting the HTTP API;
+- starting or composing the MCP surface;
+- running schema migrations without application auto-create in production;
+- probing liveness and readiness.
+
+## Migration constraints
+
+- `/api/v2` compatibility moves by route/data slice with core-hub retained as
+  rollback until consumer smokes and row/provenance comparisons pass.
+- Hub-core and core-hub SQLAlchemy metadata remain isolated during dual-run;
+  model/migration absorption is explicit rather than cross-imported.
+- The wheel and image report the same semantic version and source revision so
+  evidence can identify the exact contract/runtime pair.
+- Contract fixtures and OpenAPI snapshots move with the implementing slice and
+  remain usable without a live deployment.
+- Production image coordinates and deployment ownership change only through
+  the joint `HUB-WP-0004-T06` / `CORE-WP-0010` cutover plan.
+
+## Alternatives considered
+
+### Keep a permanent thin host repository
+
+Rejected. It preserves two release units, prevents core-hub archival, and makes
+contract/runtime compatibility a cross-repository pinning problem. A temporary
+compatibility host during migration is allowed but has an explicit retirement
+gate.
+
+### Publish a second Python runtime distribution
+
+Rejected for the initial consolidation. One distribution with a runtime extra
+keeps contracts, application composition, and migrations version-aligned. This
+can be revisited only if dependency or release evidence shows a material need.
+
+### Run API and MCP in one mandatory process
+
+Rejected as a packaging requirement. They share the image and contract version,
+but deployments may separate processes to preserve scaling, failure, and access
+boundaries.
+
+## Consequences
+
+- Hub-core gains operational artifact ownership and must test both wheel and
+  image entry surfaces.
+- Library-only consumers keep an import path without runtime startup.
+- Core-hub absorption can converge on one release rather than another
+  permanent wrapper.
+- T04 owns the minimal application/command/image scaffold alongside the core
+  ports; T06 owns dual-run and production transition details.

docs/conformance.md

@@ -0,0 +1,44 @@
+# Hub-extension conformance
+
+`hub_core.conformance` is the reusable Tier 2/3 harness scaffold for contract
+version 0.1.0. It drives only public HTTP ports, so a FastAPI `TestClient`, an
+`httpx.Client`, or another compatible target can be used without importing the
+runtime implementation.
+
+The harness mutates its target. Run it against a disposable instance or a
+dedicated test namespace:
+
+```bash
+hub-core api --host 127.0.0.1 --port 8010
+hub-core conformance --base-url http://127.0.0.1:8010
+hub-core conformance --base-url http://127.0.0.1:8010 --json
+```
+
+## Implemented profile
+
+| ID | Tier | Automated evidence |
+| --- | --- | --- |
+| C1 | 2 | Packaged descriptor, manifest, and catalog validate against Draft 2020-12 schemas |
+| C3 | 2 | Runtime health probe returns healthy |
+| C4 | 2 | Repeated manifest registration is reported as a duplicate |
+| C5 | 2 | Cataloged progress/interaction events are accepted; wrong-family and unknown events are rejected |
+| C6 | 2 | Contract and scenario fixtures reject secret-shaped keys, credentialed database URLs, and private keys |
+| C8 | 2 | Registry response propagates the request correlation identifier |
+| F2 | 3 | Progress and interaction fixture events appear only in their respective projections |
+| F3 | 3 | Authority fixtures appear in projections with declared rebuild sources and provenance hashes |
+
+The projection scenario is shipped in the wheel as
+`fixtures/projection-rebuild.json`. Correlation and time fields are generated
+per run, allowing the harness to identify its own evidence without relying on
+global row counts.
+
+## Deliberately open checks
+
+C2 registry resolution, C7 raw-port configuration policy, C9 dependency-aware
+readiness, C10 version negotiation, F1 registry audit history, F4 `/api/v2`
+consumer smokes, F5 MCP projection binding, F6 policy fail-closed behavior, F7
+telemetry rejection, and F8 migration metadata isolation require ports or
+absorption slices that are not part of the T04 minimal vertical. Tenant
+isolation also remains open because the 0.1 runtime has no tenant identity or
+authorization context yet. These gaps must not be interpreted as passing; the
+harness reports only the implemented profile above.

docs/core-hub-absorption-plan.md

@@ -0,0 +1,307 @@
+# Core Hub absorption and `/api/v2` cutover plan
+
+**Status:** accepted joint input for `HUB-WP-0004-T06` and
+`CORE-WP-0010-T02`
+**Date:** 2026-08-21
+**Runtime source:** `hub-core`
+**Rollback runtime:** `core-hub` on railiance01
+
+## Decision
+
+Absorb Core Hub into hub-core by capability and data boundary. There is no
+big-bang route move and no period with two production writers for the same
+boundary.
+
+Hub-core owns the surviving wheel, runtime image, extension contracts,
+compatibility routes, durable framework store, and migrations. Core Hub stays
+deployable as the rollback runtime until every `/api/v2` route group, durable
+record family, consumer gate, and rollback exercise has passed. Its Python
+package is never imported by hub-core.
+
+The current production baseline is Core Hub chart `0.3.2` on railiance01,
+backed by the `core_hub` PostgreSQL schema. Its source database on CoulombCore
+is already frozen and is historical rollback evidence, not a participant in
+this absorption.
+
+## Non-negotiable invariants
+
+1. A route group has exactly one production writer at any moment.
+2. A write freeze, final export, count/content comparison, and explicit
+   operator approval precede every production writer change.
+3. Shadow traffic consists of reads and synthetic fixture writes only. Do not
+   mirror normal writes to both runtimes.
+4. Hub-core and Core Hub keep separate SQLAlchemy metadata and migration
+   lineages. Hub-core may copy legacy rows but must not import `core_hub`
+   models or migrations.
+5. IDs, timestamps, relationships, source revision, source table, and import
+   bundle hash survive migration as provenance.
+6. Only API-key hashes and prefixes move. Raw keys, static operator tokens,
+   database credentials, and secret values never enter bundles or evidence.
+7. Authorization fails closed when its dependency is unavailable. The
+   compatibility API-key validator is temporary; authorization decisions
+   remain owned through the identity/policy boundary.
+8. Core Hub remains a tested rollback target until the stabilization and
+   rollback-retirement gates close.
+
+## Runtime and database shape
+
+The hub-core image selected by ADR-0001 is the candidate runtime. Its current
+in-memory `PortStore` is not a migration target. The first absorption slice
+must add a PostgreSQL implementation, production readiness checks, an audit
+ledger, and migrations owned entirely by hub-core.
+
+Use the existing platform PostgreSQL allocation currently assigned to Core
+Hub; do not consume a fifth application slot. Add a distinct target schema
+(provisionally `hub_runtime`) beside retained legacy schema `core_hub`, then
+transfer the consumer declaration and dynamic credential identity as part of
+the final package transition. Separate schemas make source/target counts and
+rollback observable without mixing ORM metadata.
+
+The deployment package currently named `rapp-core-hub` owns the verified
+railiance01 chart, policies, secret projections, live verifier, and rollback
+surface. During dual-run it must deploy both immutable image revisions or a
+candidate release beside the legacy release. Before Core Hub is archived, the
+package must either be renamed/replaced by a hub-core-owned rapp package or
+recorded explicitly as the surviving package with updated ownership and
+intent. No application source moves into the deployment package.
+
+## Traffic controls
+
+Implement route-group controls before moving data:
+
+- `HUB_CORE_V2_GROUPS`: compatibility groups served by hub-core;
+- `HUB_CORE_V2_WRITE_GROUPS`: subset allowed to mutate hub-core authority;
+- `CORE_HUB_V2_WRITE_GROUPS`: legacy groups still allowed to mutate Core Hub;
+- an ingress/gateway route map selecting `core-hub` or `hub-core` per group;
+- a shadow comparator that calls both read targets, normalizes ordering and
+  volatile fields, and records hashes without response bodies containing
+  sensitive data.
+
+The exact configuration carrier may be Helm values rather than environment
+variables, but the three states are mandatory and independently observable:
+
+| State | Public owner | Candidate behavior | Write rule |
+| --- | --- | --- | --- |
+| legacy | Core Hub | health plus synthetic fixtures | Core Hub only |
+| shadow | Core Hub | normalized read comparison | Core Hub only |
+| candidate | hub-core | public read/write for the group | hub-core only |
+
+Direct service access is network-policy restricted. The route map is the
+normal traffic authority; application write-group checks are defense in
+depth. A configuration assertion must reject overlap between legacy and
+candidate write sets.
+
+## Route and module move order
+
+Each slice is independently buildable, reversible, and evidenced. Later
+slices do not excuse a failed earlier gate.
+
+### S0 — Durable runtime, compatibility shell, and migration tooling
+
+No public route moves.
+
+- Add hub-core PostgreSQL `PortStore`, session lifecycle, readiness, audit
+  ledger, import/export commands, and a migration bundle schema.
+- Add `/api/v2` router composition, legacy response serializers, bearer-token
+  dependency, per-group write controls, and normalized shadow comparison.
+- Import the static operator token through the existing OpenBao/ExternalSecret
+  lane and copy API-key hashes into the isolated compatibility auth store.
+- Preserve current error/status semantics and unprefixed OpenAPI aliases.
+- Extend the conformance suite with durable restart/rebuild, authorization
+  allow/deny/unavailable, metadata isolation, and migration idempotence.
+
+Gate: frozen source fixture imports twice without duplicates; all seven source
+table counts can be represented; production readiness fails when PostgreSQL or
+authorization is unavailable; no traffic has moved.
+
+### S1 — System and read-only discovery
+
+Routes:
+
+- `/healthz`, `/readyz`;
+- `/api/v2/widget-types`, `/api/v2/event-types`;
+- `/api/v2/annotation-categories`, `/api/v2/policy-scopes`;
+- `/api/v2/openapi.json`, `/api/v2/openapi.yaml`, `/api/v2/docs`;
+- compatibility aliases published by the current OpenAPI surface.
+
+Move the catalogs, schemas, OpenAPI snapshot/export behavior, and public
+consumer fixtures. Static catalog responses are compared as canonical JSON.
+Readiness must reflect the dependencies required by enabled groups rather than
+reporting healthy from process liveness alone.
+
+Gate: public catalog and documentation smoke passes against both targets;
+normalized payload hashes match; ops-hub public gate and activity-core
+resolver pass; rollback route-map change is rehearsed.
+
+### S2 — Registry and manifest authority
+
+Routes:
+
+- `GET/POST /api/v2/hubs`;
+- `GET/POST /api/v2/hub-capability-manifests`;
+- `PATCH /api/v2/hub-capability-manifests/{manifest_id}`;
+- `POST /api/v2/hub-capability-manifests/{manifest_id}/activate`;
+- `GET /api/v2/hub-registry`.
+
+Modules/data:
+
+- copy `hubs` and `hub_capability_manifests` into hub-core registry authority;
+- preserve legacy UUIDs and camel-case compatibility serialization;
+- validate new writes against `helixforge.hub-extension`, while a named legacy
+  adapter reads migrated 0.x records without silently rewriting them;
+- add activation audit records and expose registry projections with
+  provenance.
+
+Gate: table counts and order-independent row hashes match; manifest activation
+is idempotent and audited; protected allow/deny checks pass; ops-hub bootstrap
+can resolve and update its manifest; rollback delta replay is proven.
+
+### S3 — API consumers and compatibility credentials
+
+Routes:
+
+- `GET/POST /api/v2/api-consumers`;
+- `POST /api/v2/api-consumers/{consumer_id}/api-keys`;
+- `POST /api/v2/token`.
+
+Modules/data:
+
+- copy `api_consumers` and `api_keys`, including status, scopes, quotas, hash,
+  and prefix but never raw key material;
+- preserve one-time key issuance semantics and ensure generated secrets are
+  absent from logs, projections, migration bundles, and evidence;
+- keep the static operator credential and dynamic consumer keys in distinct
+  custody lanes;
+- record a residual owner and follow-on task for replacing compatibility keys
+  with the platform identity/authorization ports. The shim may outlive Core
+  Hub briefly, but it is not a permanent hub-core authority.
+
+Gate: existing ops-hub and activity-core credentials authenticate against the
+candidate; invalid, revoked, missing, and dependency-unavailable cases deny;
+rate/quota behavior is either preserved or explicitly residual-owned; reverse
+delta replay for newly issued key hashes is rehearsed without exposing a key.
+
+### S4 — Widgets and interaction evidence
+
+Routes:
+
+- `GET/POST /api/v2/widgets`;
+- `GET/POST /api/v2/interaction-events`.
+
+Modules/data:
+
+- copy `widgets` into the operator-surface registry/projection boundary;
+- copy `interaction_events` into the physically separate interaction-event
+  authority while retaining legacy `widgetId`, event type, context, metadata,
+  IDs, and timestamps in the compatibility view;
+- route new writes through `port.events.interaction` and build `/api/v2` reads
+  from the same authority rather than a second compatibility-only store.
+
+Gate: counts and hashes match; projection rebuild fixture and migrated sample
+produce provenance-equivalent views; progress events never appear in the
+interaction family; ops-hub and activity-core each post and read back unique
+correlated evidence; rollback delta replay is proven.
+
+### S5 — Empty compatibility collections and operator console
+
+Routes:
+
+- `GET/POST /api/v2/annotations`;
+- `GET/POST /api/v2/requirement-candidates`;
+- `GET/POST /api/v2/decision-records`;
+- `GET/POST /api/v2/deployment-records`;
+- `GET/POST /api/v2/outcome-signals`;
+- `/console`.
+
+The five collections have no Core Hub durable models and currently return
+empty data. Preserve that behavior only as a compatibility adapter; do not
+claim or create authority. Each endpoint needs an explicit retirement or
+future-owner record before the adapter is removed. Rebuild the console from
+hub-core projections and public APIs; it must not query the retained
+`core_hub` schema.
+
+Gate: exact empty/error compatibility passes, the console authorization
+boundary and visual smoke pass, and every deferred collection has a residual
+disposition.
+
+### S6 — Whole-host cutover and Core Hub retirement
+
+After S1–S5 are in candidate state:
+
+1. Freeze all remaining Core Hub writes and capture a final encrypted backup,
+   table counts, content hashes, schema revision, image digest, and correlation
+   ID.
+2. Import any remaining deltas and require zero unexplained count/hash drift.
+3. Run hub-core conformance, the Core Hub deployed smoke, ops-hub bootstrap,
+   and activity-core resolver/write-readback against the candidate image.
+4. With explicit operator approval, point `hub.coulomb.social` entirely at
+   hub-core and set every Core Hub write group empty.
+5. Observe at least seven consecutive days with no unexplained 5xx, no failed
+   consumer gate, no data drift, and no normal request to Core Hub. Record any
+   exception with an owner and expiry.
+6. Exercise rollback once during rehearsal. After the stabilization window and
+   explicit retirement approval, scale Core Hub to zero while retaining its
+   immutable image, database backup, deployment revision, and route-map
+   rollback instructions until the agreed expiry.
+7. Move the final OpenAPI/contracts/smoke fixtures into hub-core history,
+   record residual owners, archive the `core-hub` repository read-only, and
+   update the deployment package identity.
+
+## Evidence required for every durable slice
+
+| Evidence | Pass condition |
+| --- | --- |
+| Source identity | Core Hub image/source revision and schema revision recorded |
+| Target identity | hub-core wheel/image version, revision, and contract version recorded |
+| Backup | Encrypted location and checksum recorded; no secret value logged |
+| Counts | Per-table source/import/target counts with zero unexplained delta |
+| Content | Order-independent canonical row hashes match after documented transforms |
+| Provenance | Source table/ID/revision, import bundle hash, and target record IDs queryable |
+| Contract | Static OpenAPI diff has no unexplained removed route/status/schema behavior |
+| Authorization | Missing/invalid/revoked deny; allowed roles pass; dependency outage denies |
+| Failure mode | Database, policy/auth, and candidate unavailability behavior recorded |
+| Consumers | Owned ops-hub and activity-core gates pass against the candidate |
+| Rollback | Reverse delta export/import and route-map reversal exercised |
+
+An unexplained row, response, authorization, or provenance mismatch fails the
+slice. A waiver requires a human progress note and a live residual task.
+
+## Smoke and change ownership
+
+| Surface | Owner during absorption |
+| --- | --- |
+| Hub-core unit, contract, conformance, migration, and image tests | `hub-core` |
+| Legacy OpenAPI and deployed-smoke oracle | `core-hub` until archive, then historical fixture in `hub-core` |
+| ops-hub public/bootstrap gate | `ops-hub` |
+| activity-core resolver and interaction write/readback | `activity-core` |
+| Helm render, policy, secret projection, live verification, traffic map, rollback | rapp package (currently `rapp-core-hub`) + platform owners |
+| Production writer flip, rollback, scale-to-zero, archive | operator approval required |
+
+## Rollback procedure
+
+For a failed candidate slice:
+
+1. Stop candidate writes for the affected group; do not enable legacy writes
+   yet.
+2. Export the candidate-only delta with IDs and hashes, validate it against the
+   reverse adapter, and import it into the retained legacy schema.
+3. Compare counts/content and run the legacy group smoke.
+4. Route the group to Core Hub, enable only its legacy writer, and confirm the
+   candidate writer remains disabled.
+5. Record the failure correlation ID, data interval, revisions, evidence, and
+   next owner. Do not continue to the next slice.
+
+If candidate data cannot be replayed safely, keep both writers disabled and
+invoke the operator recovery path. Availability pressure does not authorize
+discarding acknowledged writes or enabling concurrent writers.
+
+## Completion criteria
+
+`HUB-WP-0004-T06` is complete when this plan is linked to CORE-WP-0010 and its
+handoff is recorded. HUB-WP-0004 may then finish; it does not claim production
+absorption.
+
+`CORE-WP-0010` completes only when all slices have evidence, production is on
+hub-core, the stabilization and rollback gates close, residuals have live
+owners, the deployment package names the surviving runtime, and Core Hub is
+archived read-only.

docs/runtime.md

@@ -0,0 +1,88 @@
+# Hub Core runtime
+
+The initial runtime scaffold implements the packaging decision in ADR-0001 and
+the minimal vertical in `HUB-WP-0004-T04`. It is a conformance and absorption
+base, not yet the production replacement for Core Hub.
+
+## Processes
+
+The `runtime` dependency extra installs one `hub-core` command with three
+process modes:
+
+```bash
+uv sync --extra runtime
+hub-core api --host 127.0.0.1 --port 8010
+hub-core mcp --host 127.0.0.1 --port 8011 --api-base http://127.0.0.1:8010
+hub-core migrate head --database-url postgresql+asyncpg://...
+```
+
+The migration command converts the async PostgreSQL URL for the packaged
+synchronous Alembic environment. Production must run migrations explicitly;
+the API does not auto-create tables.
+
+## Runtime ports
+
+| Port | Initial path | Behavior |
+| --- | --- | --- |
+| `port.registry` | `POST /ports/registry/registrations` | Validates and idempotently records a descriptor/manifest package |
+| `port.messaging` | `GET/POST /ports/messaging/messages` | Addressed messages with optional conversation identity |
+| `port.events.progress` | `POST /ports/events/progress` | Accepts only cataloged progress-family events |
+| `port.events.interaction` | `POST /ports/events/interaction` | Accepts only cataloged interaction-family events |
+| `port.projection.query` | `GET /ports/projections/{id}` | Rebuildable registry/message/event projections with provenance |
+
+Available projection ids are `hub_registry`, `messages`, `progress_events`, and
+`interaction_events`. The two event families use distinct stores and cannot be
+submitted through each other's endpoint.
+
+## Backend boundary and readiness
+
+The app is created with an injected `PortStore`. T04 ships
+`InMemoryPortStore` for deterministic tests and local contract smokes. It is
+ephemeral and is not a production authority.
+
+`GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503
+when the active backend does not match `HUB_CORE_BACKEND`, or when the memory
+backend is used without explicit permission. Development and test permit it by
+default; the OCI image sets production-safe defaults:
+
+```text
+HUB_CORE_ENV=production
+HUB_CORE_BACKEND=memory
+HUB_CORE_ALLOW_EPHEMERAL=0
+```
+
+Therefore the image is deliberately not production-ready until a durable
+backend lands in an absorption slice. A local image smoke may opt in with
+`HUB_CORE_ALLOW_EPHEMERAL=1`.
+
+## OCI image
+
+The `Containerfile` uses `uv.lock` with `uv sync --frozen`, installs the runtime
+extra, runs as UID/GID 10001, includes OCI version/revision labels, and exposes
+API port 8010 plus MCP port 8011.
+
+```bash
+docker build -f Containerfile \
+  --build-arg VERSION=0.2.0 \
+  --build-arg VCS_REF="$(git rev-parse HEAD)" \
+  -t hub-core:dev .
+
+docker run --rm -p 8010:8010 \
+  -e HUB_CORE_ALLOW_EPHEMERAL=1 \
+  hub-core:dev
+```
+
+Deployment charts, secrets, rollout policy, and live evidence remain owned by
+the rapp/platform repositories. `/api/v2` data and compatibility routes remain
+on Core Hub until the T06/CORE-WP-0010 dual-run slices move them.
+
+## Conformance
+
+The public-port Tier 2/3 scaffold is documented in `docs/conformance.md`. Run
+it against an isolated runtime with `hub-core conformance --base-url <url>`.
+
+## Core Hub absorption
+
+`docs/core-hub-absorption-plan.md` defines the capability-sized `/api/v2`
+route and data move order, single-writer dual-run controls, evidence gates,
+rollback, and final cutover criteria shared with `CORE-WP-0010`.