feat: add durable Core Hub absorption runtime
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / pytest-smoke (push) Failing after 2s

This commit is contained in:
tegwick 2026-08-21 16:16:42 +02:00
parent 7e1ec03f0c
commit 8ab1d0c09a
20 changed files with 2423 additions and 15 deletions

View file

@ -52,11 +52,15 @@ uv sync --extra runtime
hub-core api hub-core api
hub-core mcp --api-base http://127.0.0.1:8010 hub-core mcp --api-base http://127.0.0.1:8010
hub-core migrate head --database-url postgresql+asyncpg://... hub-core migrate head --database-url postgresql+asyncpg://...
hub-core migration validate core-hub-export.json
hub-core migration import core-hub-export.json --database-url postgresql+asyncpg://...
``` ```
The initial runtime exposes registry, messaging, progress-event, The runtime exposes registry, messaging, progress-event, interaction-event,
interaction-event, and projection-query ports. Its included memory backend is and projection-query ports, plus the governed Core Hub `/api/v2`
for local/conformance use and fails production readiness unless explicitly compatibility surface. Its PostgreSQL backend includes migrations, audit, and
idempotent seven-table migration tooling. The memory backend remains for
local/conformance use and fails production readiness unless explicitly
enabled. See `docs/runtime.md`. enabled. See `docs/runtime.md`.
The reusable Tier 2/3 scaffold is documented in `docs/conformance.md` and runs The reusable Tier 2/3 scaffold is documented in `docs/conformance.md` and runs

View file

@ -12,6 +12,7 @@
| workplan | HUB-WP-0002 | finished | — | workplans/HUB-WP-0002-import-refactor-adapter-seams.md | | workplan | HUB-WP-0002 | finished | — | workplans/HUB-WP-0002-import-refactor-adapter-seams.md |
| workplan | HUB-WP-0003 | finished | — | workplans/HUB-WP-0003-ecosystem-consolidation-library-lane.md | | workplan | HUB-WP-0003 | finished | — | workplans/HUB-WP-0003-ecosystem-consolidation-library-lane.md |
| workplan | HUB-WP-0004 | finished | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md | | workplan | HUB-WP-0004 | finished | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
| workplan | HUB-WP-0005 | active | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
| task | HUB-WP-0001-T01 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md | | task | HUB-WP-0001-T01 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
| task | HUB-WP-0001-T02 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md | | task | HUB-WP-0001-T02 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
| task | HUB-WP-0001-T03 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md | | task | HUB-WP-0001-T03 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
@ -30,3 +31,9 @@
| task | HUB-WP-0004-T04 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md | | task | HUB-WP-0004-T04 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
| task | HUB-WP-0004-T05 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md | | task | HUB-WP-0004-T05 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
| task | HUB-WP-0004-T06 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md | | task | HUB-WP-0004-T06 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
| task | HUB-WP-0005-T01 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
| task | HUB-WP-0005-T02 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
| task | HUB-WP-0005-T03 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
| task | HUB-WP-0005-T04 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
| task | HUB-WP-0005-T05 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
| task | HUB-WP-0005-T06 | wait | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |

View file

@ -1,8 +1,9 @@
# Hub Core runtime # Hub Core runtime
The initial runtime scaffold implements the packaging decision in ADR-0001 and The runtime implements the packaging decision in ADR-0001, the minimal
the minimal vertical in `HUB-WP-0004-T04`. It is a conformance and absorption vertical in `HUB-WP-0004-T04`, and the durable Core Hub absorption boundary in
base, not yet the production replacement for Core Hub. `HUB-WP-0005`. Public authority still moves only through the gated deployment
slices in the accepted absorption plan.
## Processes ## Processes
@ -14,6 +15,9 @@ uv sync --extra runtime
hub-core api --host 127.0.0.1 --port 8010 hub-core api --host 127.0.0.1 --port 8010
hub-core mcp --host 127.0.0.1 --port 8011 --api-base http://127.0.0.1:8010 hub-core mcp --host 127.0.0.1 --port 8011 --api-base http://127.0.0.1:8010
hub-core migrate head --database-url postgresql+asyncpg://... hub-core migrate head --database-url postgresql+asyncpg://...
hub-core migration validate core-hub-export.json
hub-core migration import core-hub-export.json --database-url postgresql+asyncpg://...
hub-core migration export --database-url postgresql+asyncpg://... --output reverse-delta.json
``` ```
The migration command converts the async PostgreSQL URL for the packaged The migration command converts the async PostgreSQL URL for the packaged
@ -36,9 +40,10 @@ submitted through each other's endpoint.
## Backend boundary and readiness ## Backend boundary and readiness
The app is created with an injected `PortStore`. T04 ships The app is created with an injected `PortStore`. `InMemoryPortStore` remains
`InMemoryPortStore` for deterministic tests and local contract smokes. It is available for deterministic tests and local contract smokes. Production uses
ephemeral and is not a production authority. `PostgresPortStore`, whose registration, messaging, progress, interaction,
compatibility, import-lineage, and audit records are transactionally durable.
`GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503 `GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503
when the active backend does not match `HUB_CORE_BACKEND`, or when the memory when the active backend does not match `HUB_CORE_BACKEND`, or when the memory
@ -51,10 +56,24 @@ HUB_CORE_BACKEND=memory
HUB_CORE_ALLOW_EPHEMERAL=0 HUB_CORE_ALLOW_EPHEMERAL=0
``` ```
Therefore the image is deliberately not production-ready until a durable Set `HUB_CORE_BACKEND=postgresql` and `HUB_CORE_DATABASE_URL` in production.
backend lands in an absorption slice. A local image smoke may opt in with Readiness checks the database and continues to fail closed if the selected
backend is unavailable. A local image smoke may opt in to memory with
`HUB_CORE_ALLOW_EPHEMERAL=1`. `HUB_CORE_ALLOW_EPHEMERAL=1`.
The compatibility adapter is enabled by coherent route groups in
`HUB_CORE_V2_GROUPS`; writes require the corresponding value in
`HUB_CORE_V2_WRITE_GROUPS`. Startup rejects overlap with
`CORE_HUB_V2_WRITE_GROUPS`. Supported groups are `system`, `registry`,
`credentials`, `interaction`, and `deferred`. Static operator authentication
uses `HUB_CORE_API_TOKEN`; imported consumer keys are verified by hash.
Migration bundles cover all seven Core Hub source tables. They preserve stable
IDs and timestamps, reject secret-shaped material and digest drift, record
counts/identity sets/canonical hashes, and return the original import-run ID
when the same bundle is replayed. Reverse export contains prefixes and hashes,
never raw keys.
## OCI image ## OCI image
The `Containerfile` uses `uv.lock` with `uv sync --frozen`, installs the runtime The `Containerfile` uses `uv.lock` with `uv sync --frozen`, installs the runtime

View file

@ -0,0 +1,203 @@
"""hub-core durable runtime ports
Revision ID: 0002_runtime_ports
Revises: 0001_core_schema
Create Date: 2026-08-21
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0002_runtime_ports"
down_revision: Union[str, None] = "0001_core_schema"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"runtime_registrations",
sa.Column("hub_slug", sa.String(120), primary_key=True),
sa.Column("package", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_table(
"runtime_messages",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("schema_version", sa.String(40), nullable=False),
sa.Column("correlation_id", sa.String(36), nullable=False),
sa.Column("conversation_id", sa.String(36), nullable=True),
sa.Column("from_address", sa.String(240), nullable=False),
sa.Column("to_addresses", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("body", sa.Text(), nullable=False),
sa.Column("subject_refs", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_runtime_messages_correlation_id", "runtime_messages", ["correlation_id"])
op.create_index("ix_runtime_messages_conversation_id", "runtime_messages", ["conversation_id"])
op.create_index("ix_runtime_messages_created_at", "runtime_messages", ["created_at"])
for name in ("runtime_progress_events", "runtime_interaction_events"):
op.create_table(
name,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("schema_version", sa.String(40), nullable=False),
sa.Column("correlation_id", sa.String(36), nullable=False),
sa.Column("event_type", sa.String(160), nullable=False),
sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("subject_refs", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("payload", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index(f"ix_{name}_correlation_id", name, ["correlation_id"])
op.create_index(f"ix_{name}_event_type", name, ["event_type"])
op.create_index(f"ix_{name}_recorded_at", name, ["recorded_at"])
op.create_table(
"runtime_audit_ledger",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("action", sa.String(120), nullable=False),
sa.Column("subject_type", sa.String(80), nullable=False),
sa.Column("subject_id", sa.String(240), nullable=False),
sa.Column("correlation_id", sa.String(36), nullable=True),
sa.Column("payload_hash", sa.String(64), nullable=False),
sa.Column("detail", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_runtime_audit_ledger_action", "runtime_audit_ledger", ["action"])
op.create_index("ix_runtime_audit_ledger_subject_id", "runtime_audit_ledger", ["subject_id"])
op.create_index(
"ix_runtime_audit_ledger_correlation_id", "runtime_audit_ledger", ["correlation_id"]
)
op.create_index(
"ix_runtime_audit_ledger_recorded_at", "runtime_audit_ledger", ["recorded_at"]
)
op.create_table(
"compat_hubs",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("slug", sa.String(120), nullable=False, unique=True),
sa.Column("name", sa.String(240), nullable=False),
sa.Column("domain", sa.String(240), nullable=True),
sa.Column("hub_kind", sa.String(80), nullable=True),
sa.Column("hub_family", sa.String(80), nullable=True),
sa.Column("vsm_function", sa.String(80), nullable=True),
sa.Column("vsm_system", sa.String(80), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_compat_hubs_slug", "compat_hubs", ["slug"])
op.create_table(
"compat_manifests",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=True),
sa.Column("hub_slug", sa.String(120), nullable=True),
sa.Column("manifest_version", sa.String(40), nullable=False),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_compat_manifests_hub_id", "compat_manifests", ["hub_id"])
op.create_index("ix_compat_manifests_hub_slug", "compat_manifests", ["hub_slug"])
op.create_table(
"compat_api_consumers",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("slug", sa.String(120), nullable=True, unique=True),
sa.Column("name", sa.String(240), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("hub_capability_manifest_id", sa.String(36), nullable=True),
sa.Column("rate_limit_per_minute", sa.Integer(), nullable=True),
sa.Column("quota_per_day", sa.Integer(), nullable=True),
sa.Column("key_prefix", sa.String(32), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_compat_api_consumers_slug", "compat_api_consumers", ["slug"])
op.create_index("ix_compat_api_consumers_name", "compat_api_consumers", ["name"])
op.create_index(
"ix_compat_api_consumers_manifest", "compat_api_consumers", ["hub_capability_manifest_id"]
)
op.create_table(
"compat_api_keys",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column(
"api_consumer_id",
sa.String(36),
sa.ForeignKey("compat_api_consumers.id"),
nullable=False,
),
sa.Column("key_prefix", sa.String(32), nullable=False),
sa.Column("key_hash", sa.String(64), nullable=False, unique=True),
sa.Column("scopes", sa.Text(), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_compat_api_keys_consumer", "compat_api_keys", ["api_consumer_id"])
op.create_index("ix_compat_api_keys_prefix", "compat_api_keys", ["key_prefix"])
op.create_table(
"compat_widgets",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=False),
sa.Column("name", sa.String(240), nullable=False),
sa.Column("widget_type", sa.String(120), nullable=True),
sa.Column("capability_ref", sa.String(240), nullable=True),
sa.Column("view_context", sa.String(500), nullable=True),
sa.Column("policy_scope", sa.String(120), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_compat_widgets_hub_id", "compat_widgets", ["hub_id"])
op.create_index("ix_compat_widgets_type", "compat_widgets", ["widget_type"])
op.create_index("ix_compat_widgets_capability", "compat_widgets", ["capability_ref"])
op.create_table(
"compat_migration_runs",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("source", sa.String(120), nullable=False),
sa.Column("schema_version", sa.String(80), nullable=False),
sa.Column("bundle_sha256", sa.String(64), nullable=False),
sa.Column("dry_run", sa.Boolean(), nullable=False),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("counts", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("diagnostics", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_compat_migration_runs_source", "compat_migration_runs", ["source"])
op.create_index("ix_compat_migration_runs_bundle", "compat_migration_runs", ["bundle_sha256"])
op.create_table(
"runtime_import_runs",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("source", sa.String(120), nullable=False),
sa.Column("schema_version", sa.String(80), nullable=False),
sa.Column("bundle_sha256", sa.String(64), nullable=False, unique=True),
sa.Column("high_water_mark", sa.String(200), nullable=True),
sa.Column("counts", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("content_hashes", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_runtime_import_runs_source", "runtime_import_runs", ["source"])
op.create_index("ix_runtime_import_runs_bundle", "runtime_import_runs", ["bundle_sha256"])
def downgrade() -> None:
op.drop_table("runtime_import_runs")
op.drop_table("compat_migration_runs")
op.drop_table("compat_widgets")
op.drop_table("compat_api_keys")
op.drop_table("compat_api_consumers")
op.drop_table("compat_manifests")
op.drop_table("compat_hubs")
op.drop_table("runtime_audit_ledger")
op.drop_table("runtime_interaction_events")
op.drop_table("runtime_progress_events")
op.drop_table("runtime_messages")
op.drop_table("runtime_registrations")

View file

@ -1,9 +1,12 @@
from __future__ import annotations from __future__ import annotations
from contextlib import asynccontextmanager
from fastapi import FastAPI, Response, status from fastapi import FastAPI, Response, status
from hub_core import __version__ from hub_core import __version__
from hub_core.runtime.config import RuntimeSettings from hub_core.runtime.config import RuntimeSettings
from hub_core.runtime.compat import SQLCompatibilityStore, create_compatibility_router
from hub_core.runtime.models import HealthResponse, ReadinessResponse from hub_core.runtime.models import HealthResponse, ReadinessResponse
from hub_core.runtime.ports import create_ports_router from hub_core.runtime.ports import create_ports_router
from hub_core.runtime.store import InMemoryPortStore, PortStore from hub_core.runtime.store import InMemoryPortStore, PortStore
@ -17,37 +20,66 @@ def create_app(
) -> FastAPI: ) -> FastAPI:
resolved_settings = settings or RuntimeSettings.from_env() resolved_settings = settings or RuntimeSettings.from_env()
resolved_store = port_store or _create_store(resolved_settings) resolved_store = port_store or _create_store(resolved_settings)
owns_store = port_store is None
@asynccontextmanager
async def lifespan(_: FastAPI):
yield
if owns_store and (closer := getattr(resolved_store, "aclose", None)):
await closer()
app = FastAPI( app = FastAPI(
title="Hub Core Runtime", title="Hub Core Runtime",
version=__version__, version=__version__,
description="HelixForge hub framework and named-port runtime.", description="HelixForge hub framework and named-port runtime.",
lifespan=lifespan,
) )
app.state.settings = resolved_settings app.state.settings = resolved_settings
app.state.port_store = resolved_store app.state.port_store = resolved_store
app.state.compat_store = (
SQLCompatibilityStore(resolved_store)
if resolved_store.backend_name == "postgresql"
else None
)
app.state.contract_validator = ContractValidator() app.state.contract_validator = ContractValidator()
@app.get("/healthz", response_model=HealthResponse, tags=["system"]) @app.get("/healthz", response_model=HealthResponse, tags=["system"])
async def healthz() -> HealthResponse: async def healthz() -> HealthResponse:
return HealthResponse(version=__version__) return HealthResponse(
service="core-hub" if resolved_settings.legacy_health else "hub-core",
version=__version__,
)
@app.get("/readyz", response_model=ReadinessResponse, tags=["system"]) @app.get("/readyz", response_model=ReadinessResponse, tags=["system"])
async def readyz(response: Response) -> ReadinessResponse: async def readyz(response: Response) -> ReadinessResponse:
ready = resolved_settings.is_ready(resolved_store.backend_name) dependency_checks = await resolved_store.readiness_checks()
ready = resolved_settings.is_ready(resolved_store.backend_name) and all(
value in {"ok", "not_applicable"} for value in dependency_checks.values()
)
if not ready: if not ready:
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
return ReadinessResponse( return ReadinessResponse(
status="ok" if ready else "degraded", status="ok" if ready else "degraded",
checks=resolved_settings.readiness_checks(resolved_store.backend_name), checks={
**resolved_settings.readiness_checks(resolved_store.backend_name),
**dependency_checks,
},
) )
app.include_router(create_ports_router()) app.include_router(create_ports_router())
app.include_router(create_compatibility_router())
return app return app
def _create_store(settings: RuntimeSettings) -> PortStore: def _create_store(settings: RuntimeSettings) -> PortStore:
if settings.backend == "memory": if settings.backend == "memory":
return InMemoryPortStore() return InMemoryPortStore()
if settings.backend == "postgresql":
if not settings.database_url:
raise RuntimeError("HUB_CORE_DATABASE_URL is required for PostgreSQL backend")
from hub_core.runtime.postgres_store import PostgresPortStore
return PostgresPortStore.from_url(settings.database_url)
raise RuntimeError(f"Unsupported HUB_CORE_BACKEND '{settings.backend}'") raise RuntimeError(f"Unsupported HUB_CORE_BACKEND '{settings.backend}'")

View file

@ -3,6 +3,7 @@ from __future__ import annotations
import argparse import argparse
import json import json
from importlib.resources import files from importlib.resources import files
from pathlib import Path
from typing import Sequence from typing import Sequence
from hub_core.mcp import HubCoreMCPServer from hub_core.mcp import HubCoreMCPServer
@ -28,6 +29,21 @@ def build_parser(settings: RuntimeSettings | None = None) -> argparse.ArgumentPa
migrate.add_argument("revision", nargs="?", default="head") migrate.add_argument("revision", nargs="?", default="head")
migrate.add_argument("--database-url", default=resolved.database_url) migrate.add_argument("--database-url", default=resolved.database_url)
migration = commands.add_parser("migration", help="Export, validate, or import migration bundles")
migration_commands = migration.add_subparsers(dest="migration_command", required=True)
migration_validate = migration_commands.add_parser("validate")
migration_validate.add_argument("bundle", type=Path)
migration_validate.add_argument("--output", type=Path)
migration_import = migration_commands.add_parser("import")
migration_import.add_argument("bundle", type=Path)
migration_import.add_argument("--database-url", default=resolved.database_url)
migration_import.add_argument("--dry-run", action="store_true")
migration_import.add_argument("--output", type=Path)
migration_export = migration_commands.add_parser("export")
migration_export.add_argument("--database-url", default=resolved.database_url)
migration_export.add_argument("--source-revision")
migration_export.add_argument("--output", type=Path, required=True)
conformance = commands.add_parser( conformance = commands.add_parser(
"conformance", "conformance",
help="Run the implemented Tier 2/3 profile against an HTTP runtime", help="Run the implemented Tier 2/3 profile against an HTTP runtime",
@ -53,6 +69,8 @@ def main(argv: Sequence[str] | None = None) -> int:
raise SystemExit("hub-core migrate requires --database-url or HUB_CORE_DATABASE_URL") raise SystemExit("hub-core migrate requires --database-url or HUB_CORE_DATABASE_URL")
_run_migrations(args.database_url, args.revision) _run_migrations(args.database_url, args.revision)
return 0 return 0
if args.command == "migration":
return _run_migration(args)
if args.command == "conformance": if args.command == "conformance":
return _run_conformance(args.base_url, args.timeout, args.as_json) return _run_conformance(args.base_url, args.timeout, args.as_json)
raise AssertionError(f"Unhandled command {args.command}") raise AssertionError(f"Unhandled command {args.command}")
@ -96,5 +114,38 @@ def _run_conformance(base_url: str, timeout: float, as_json: bool) -> int:
return 0 if report.passed else 1 return 0 if report.passed else 1
def _run_migration(args: argparse.Namespace) -> int:
import asyncio
from hub_core.runtime.migration import (
export_bundle,
import_bundle,
load_bundle,
validate_bundle,
)
from hub_core.runtime.postgres_store import PostgresPortStore
if args.migration_command == "validate":
report = validate_bundle(load_bundle(args.bundle))
else:
if not args.database_url:
raise SystemExit("migration command requires --database-url or HUB_CORE_DATABASE_URL")
store = PostgresPortStore.from_url(args.database_url)
async def run() -> dict:
try:
if args.migration_command == "import":
return await import_bundle(store, load_bundle(args.bundle), dry_run=args.dry_run)
return await export_bundle(store, source_revision=args.source_revision)
finally:
await store.aclose()
report = asyncio.run(run())
rendered = json.dumps(report, indent=2, sort_keys=True)
if args.output:
args.output.write_text(rendered + "\n")
else:
print(rendered)
return 0 if report.get("ok", True) else 1
def _sync_database_url(database_url: str) -> str: def _sync_database_url(database_url: str) -> str:
return database_url.replace("postgresql+asyncpg://", "postgresql+psycopg2://") return database_url.replace("postgresql+asyncpg://", "postgresql+psycopg2://")

605
hub_core/runtime/compat.py Normal file
View file

@ -0,0 +1,605 @@
from __future__ import annotations
import hashlib
import hmac
import secrets
from datetime import datetime, timezone
from typing import Annotated, Any
from uuid import uuid4
import sqlalchemy as sa
from fastapi import APIRouter, Header, HTTPException, Request, status
from fastapi.responses import HTMLResponse, PlainTextResponse, RedirectResponse
from hub_core.runtime.compat_catalogs import (
ANNOTATION_CATEGORIES,
EVENT_TYPES,
POLICY_SCOPES,
WIDGET_TYPES,
)
from hub_core.runtime.models import EventCommand
from hub_core.runtime.postgres_store import PostgresPortStore
from hub_core.runtime.tables import (
compat_api_consumers,
compat_api_keys,
compat_hubs,
compat_manifests,
compat_widgets,
runtime_interaction_events,
)
from hub_core.utils.slugs import slugify_or_default
AuthorizationHeader = Annotated[str | None, Header()]
class SQLCompatibilityStore:
def __init__(self, ports: PostgresPortStore) -> None:
self.ports = ports
self.sessions = ports.sessions
async def validate_token(self, token: str) -> bool:
token_hash = _hash_token(token)
async with self.sessions() as session:
result = await session.execute(
sa.select(compat_api_keys.c.id).where(
compat_api_keys.c.key_hash == token_hash,
compat_api_keys.c.status == "active",
)
)
return result.scalar_one_or_none() is not None
async def list_hubs(self) -> list[dict[str, Any]]:
async with self.sessions() as session:
rows = (
await session.execute(sa.select(compat_hubs).order_by(compat_hubs.c.slug))
).mappings()
return [_hub(row) for row in rows]
async def create_hub(self, body: dict[str, Any]) -> dict[str, Any]:
now = _now()
row = {
"id": str(uuid4()),
"slug": body["slug"],
"name": body["name"],
"domain": body.get("domain"),
"hub_kind": body.get("hubKind"),
"hub_family": body.get("hubFamily"),
"vsm_function": body.get("vsmFunction"),
"vsm_system": body.get("vsmSystem"),
"status": body.get("status", "active"),
"description": body.get("description"),
"body": body,
"created_at": now,
"updated_at": now,
}
async with self.sessions.begin() as session:
await session.execute(compat_hubs.insert().values(**row))
await self.ports._audit(
session,
action="compat.hub.created",
subject_type="hub",
subject_id=row["id"],
correlation_id=None,
value=body,
)
return _hub(row)
async def list_manifests(self, hub_id: str | None = None) -> list[dict[str, Any]]:
statement = sa.select(compat_manifests).order_by(compat_manifests.c.created_at)
if hub_id:
statement = statement.where(compat_manifests.c.hub_id == hub_id)
async with self.sessions() as session:
rows = (await session.execute(statement)).mappings()
return [_manifest(row) for row in rows]
async def create_manifest(self, body: dict[str, Any]) -> dict[str, Any]:
now = _now()
hub_id = body.get("hubId")
hub_slug = None
if hub_id:
async with self.sessions() as session:
hub_slug = (
await session.execute(
sa.select(compat_hubs.c.slug).where(compat_hubs.c.id == hub_id)
)
).scalar_one_or_none()
row = {
"id": str(uuid4()),
"hub_id": hub_id,
"hub_slug": hub_slug,
"manifest_version": body.get("manifestVersion", "0.1.0"),
"status": body.get("status", "draft"),
"body": body,
"created_at": now,
"updated_at": now,
}
async with self.sessions.begin() as session:
await session.execute(compat_manifests.insert().values(**row))
await self.ports._audit(
session,
action="compat.manifest.created",
subject_type="manifest",
subject_id=row["id"],
correlation_id=None,
value=body,
)
return _manifest(row)
async def update_manifest(
self, manifest_id: str, body: dict[str, Any], *, activate: bool = False
) -> dict[str, Any]:
async with self.sessions.begin() as session:
current = (
await session.execute(
sa.select(compat_manifests).where(compat_manifests.c.id == manifest_id)
)
).mappings().one_or_none()
if current is None:
return {"id": manifest_id, "status": "missing"}
payload = dict(current["body"] or {})
payload.update(body)
values = {
"body": payload,
"manifest_version": payload.get(
"manifestVersion", current["manifest_version"]
),
"status": "active" if activate else current["status"],
"updated_at": _now(),
}
await session.execute(
compat_manifests.update()
.where(compat_manifests.c.id == manifest_id)
.values(**values)
)
await self.ports._audit(
session,
action="compat.manifest.activated" if activate else "compat.manifest.updated",
subject_type="manifest",
subject_id=manifest_id,
correlation_id=None,
value=payload,
)
return _manifest({**dict(current), **values})
async def list_consumers(self) -> list[dict[str, Any]]:
async with self.sessions() as session:
rows = (
await session.execute(
sa.select(compat_api_consumers).order_by(compat_api_consumers.c.name)
)
).mappings()
return [_consumer(row) for row in rows]
async def create_consumer(self, body: dict[str, Any]) -> dict[str, Any]:
row = {
"id": str(uuid4()),
"slug": body.get("slug") or slugify_or_default(body["name"]),
"name": body["name"],
"description": body.get("description"),
"hub_capability_manifest_id": body.get("hubCapabilityManifestId"),
"rate_limit_per_minute": body.get("rateLimitPerMinute"),
"quota_per_day": body.get("quotaPerDay"),
"key_prefix": None,
"status": body.get("status", "active"),
"body": body,
"created_at": _now(),
}
async with self.sessions.begin() as session:
await session.execute(compat_api_consumers.insert().values(**row))
await self.ports._audit(
session,
action="compat.consumer.created",
subject_type="api_consumer",
subject_id=row["id"],
correlation_id=None,
value=body,
)
return _consumer(row)
async def create_key(self, consumer_id: str, scopes: str | None) -> dict[str, Any]:
full_key = f"ch_{secrets.token_urlsafe(32)}"
prefix = full_key[:12]
row = {
"id": str(uuid4()),
"api_consumer_id": consumer_id,
"key_prefix": prefix,
"key_hash": _hash_token(full_key),
"scopes": scopes,
"status": "active",
"created_at": _now(),
}
async with self.sessions.begin() as session:
await session.execute(compat_api_keys.insert().values(**row))
await session.execute(
compat_api_consumers.update()
.where(compat_api_consumers.c.id == consumer_id)
.values(key_prefix=prefix)
)
await self.ports._audit(
session,
action="compat.api_key.created",
subject_type="api_key",
subject_id=row["id"],
correlation_id=None,
value={"apiConsumerId": consumer_id, "keyPrefix": prefix, "scopes": scopes},
)
return {
"fullKey": full_key,
"apiKey": {
"id": row["id"],
"apiConsumerId": consumer_id,
"keyPrefix": prefix,
"scopes": scopes,
"status": "active",
},
}
async def list_widgets(self) -> list[dict[str, Any]]:
async with self.sessions() as session:
rows = (
await session.execute(sa.select(compat_widgets).order_by(compat_widgets.c.name))
).mappings()
return [_widget(row) for row in rows]
async def create_widget(self, body: dict[str, Any]) -> dict[str, Any]:
row = {
"id": str(uuid4()),
"hub_id": body["hubId"],
"name": body["name"],
"widget_type": body.get("widgetType"),
"capability_ref": body.get("capabilityRef"),
"view_context": body.get("viewContext"),
"policy_scope": body.get("policyScope"),
"status": body.get("status", "active"),
"body": body,
"created_at": _now(),
}
async with self.sessions.begin() as session:
await session.execute(compat_widgets.insert().values(**row))
await self.ports._audit(
session,
action="compat.widget.created",
subject_type="widget",
subject_id=row["id"],
correlation_id=None,
value=body,
)
return _widget(row)
async def list_interactions(self) -> list[dict[str, Any]]:
async with self.sessions() as session:
rows = (
await session.execute(
sa.select(runtime_interaction_events).order_by(
runtime_interaction_events.c.recorded_at,
runtime_interaction_events.c.id,
)
)
).mappings()
values: list[dict[str, Any]] = []
for row in rows:
legacy = dict((row["payload"] or {}).get("legacy") or {})
if legacy:
legacy["id"] = row["id"]
values.append(legacy)
return values
async def create_interaction(self, body: dict[str, Any]) -> dict[str, Any]:
correlation_id = uuid4()
command = EventCommand(
schema_version="0.1.0",
correlation_id=correlation_id,
event_type=body["eventType"],
occurred_at=_now(),
subject_refs={"widget": body["widgetId"]},
payload={"legacy": body},
)
accepted = await self.ports.append_interaction(command)
return {"id": accepted.id, **body}
def create_compatibility_router() -> APIRouter:
router = APIRouter()
@router.get("/api/v2/widget-types")
async def widget_types(request: Request) -> list[dict[str, str]]:
_enabled(request, "system")
return WIDGET_TYPES
@router.get("/api/v2/event-types")
async def event_types(request: Request) -> list[dict[str, str]]:
_enabled(request, "system")
return EVENT_TYPES
@router.get("/api/v2/annotation-categories")
async def annotation_categories(request: Request) -> list[dict[str, str]]:
_enabled(request, "system")
return ANNOTATION_CATEGORIES
@router.get("/api/v2/policy-scopes")
async def policy_scopes(request: Request) -> list[dict[str, str]]:
_enabled(request, "system")
return POLICY_SCOPES
@router.get("/api/v2/hubs")
async def list_hubs(request: Request, authorization: AuthorizationHeader = None) -> dict:
await _protected(request, "registry", authorization)
return _page(await _store(request).list_hubs())
@router.post("/api/v2/hubs", status_code=status.HTTP_201_CREATED)
async def create_hub(
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "registry", authorization, write=True)
return await _store(request).create_hub(body)
@router.get("/api/v2/hub-capability-manifests")
async def list_manifests(
request: Request,
authorization: AuthorizationHeader = None,
hubId: str | None = None,
) -> dict:
await _protected(request, "registry", authorization)
return _page(await _store(request).list_manifests(hubId))
@router.post("/api/v2/hub-capability-manifests", status_code=status.HTTP_201_CREATED)
async def create_manifest(
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "registry", authorization, write=True)
return await _store(request).create_manifest(body)
@router.patch("/api/v2/hub-capability-manifests/{manifest_id}")
async def patch_manifest(
manifest_id: str,
body: dict[str, Any],
request: Request,
authorization: AuthorizationHeader = None,
) -> dict:
await _protected(request, "registry", authorization, write=True)
return await _store(request).update_manifest(manifest_id, body)
@router.post("/api/v2/hub-capability-manifests/{manifest_id}/activate")
async def activate_manifest(
manifest_id: str, request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "registry", authorization, write=True)
return await _store(request).update_manifest(manifest_id, {}, activate=True)
@router.get("/api/v2/hub-registry")
async def hub_registry(
request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "registry", authorization)
store = _store(request)
return {
"data": {
"hubs": await store.list_hubs(),
"hubCapabilityManifests": await store.list_manifests(),
}
}
@router.get("/api/v2/api-consumers")
async def list_consumers(
request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "credentials", authorization)
return _page(await _store(request).list_consumers())
@router.post("/api/v2/api-consumers", status_code=status.HTTP_201_CREATED)
async def create_consumer(
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "credentials", authorization, write=True)
return await _store(request).create_consumer(body)
@router.post(
"/api/v2/api-consumers/{consumer_id}/api-keys",
status_code=status.HTTP_201_CREATED,
)
async def create_key(
consumer_id: str,
body: dict[str, Any],
request: Request,
authorization: AuthorizationHeader = None,
) -> dict:
await _protected(request, "credentials", authorization, write=True)
return await _store(request).create_key(consumer_id, body.get("scopes"))
@router.post("/api/v2/token")
async def token(request: Request, authorization: AuthorizationHeader = None) -> dict:
await _protected(request, "credentials", authorization)
return {"access_token": "already-authenticated", "token_type": "bearer"}
@router.get("/api/v2/widgets")
async def list_widgets(
request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "interaction", authorization)
return _page(await _store(request).list_widgets())
@router.post("/api/v2/widgets", status_code=status.HTTP_201_CREATED)
async def create_widget(
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "interaction", authorization, write=True)
return await _store(request).create_widget(body)
@router.get("/api/v2/interaction-events")
async def list_interactions(
request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "interaction", authorization)
return _page(await _store(request).list_interactions())
@router.post("/api/v2/interaction-events", status_code=status.HTTP_201_CREATED)
async def create_interaction(
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "interaction", authorization, write=True)
return await _store(request).create_interaction(body)
for index, path in enumerate((
"annotations",
"requirement-candidates",
"decision-records",
"deployment-records",
"outcome-signals",
)):
async def empty_collection(
request: Request, authorization: AuthorizationHeader = None
) -> dict:
await _protected(request, "deferred", authorization)
return {"data": []}
stem = path.replace("-", "_")
router.add_api_route(
f"/api/v2/{path}",
empty_collection,
methods=["GET"],
operation_id=f"compat_list_{stem}_{index}",
)
router.add_api_route(
f"/api/v2/{path}",
empty_collection,
methods=["POST"],
status_code=201,
operation_id=f"compat_create_{stem}_{index}",
)
@router.get("/api/v2/openapi.json", include_in_schema=False)
async def openapi_json(request: Request) -> dict[str, Any]:
_enabled(request, "system")
document = request.app.openapi()
paths = document.setdefault("paths", {})
for path, operations in list(paths.items()):
if path.startswith("/api/v2/"):
paths.setdefault(path.removeprefix("/api/v2"), operations)
return document
@router.get("/api/v2/openapi.yaml", include_in_schema=False)
async def openapi_yaml(request: Request) -> PlainTextResponse:
document = await openapi_json(request)
info = document.get("info", {})
return PlainTextResponse(
"openapi: 3.1.0\ninfo:\n"
f" title: {info.get('title', 'Hub Core Runtime')}\n"
f" version: {info.get('version', '0.2.0')}\n",
media_type="application/yaml",
)
@router.get("/api/v2/docs", include_in_schema=False)
async def docs(request: Request) -> RedirectResponse:
_enabled(request, "system")
return RedirectResponse(url="/docs")
@router.get("/console", response_class=HTMLResponse)
async def console(
request: Request, authorization: AuthorizationHeader = None
) -> HTMLResponse:
await _protected(request, "operator", authorization)
return HTMLResponse(
"<!doctype html><title>Hub Core Operator Console</title>"
"<h1>Hub Core Operator Console</h1>"
"<p>Compatibility runtime candidate; projections are authoritative.</p>"
)
return router
def _enabled(request: Request, group: str) -> None:
if group not in request.app.state.settings.v2_groups:
raise HTTPException(status_code=404, detail="compatibility group is disabled")
async def _protected(
request: Request,
group: str,
authorization: str | None,
*,
write: bool = False,
) -> None:
_enabled(request, group)
if write and group not in request.app.state.settings.v2_write_groups:
raise HTTPException(status_code=503, detail="compatibility group is read-only")
if not authorization or not authorization.startswith("Bearer "):
raise _unauthorized("Missing bearer token")
token = authorization.removeprefix("Bearer ").strip()
configured = request.app.state.settings.api_token
if configured and hmac.compare_digest(token, configured):
return
try:
valid = await _store(request).validate_token(token)
except Exception as exc:
raise HTTPException(status_code=503, detail="authorization dependency unavailable") from exc
if not valid:
raise _unauthorized("Invalid bearer token")
def _store(request: Request) -> SQLCompatibilityStore:
store = request.app.state.compat_store
if store is None:
raise HTTPException(status_code=503, detail="compatibility store unavailable")
return store
def _unauthorized(message: str) -> HTTPException:
return HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail={"code": "unauthorized", "message": message},
)
def _page(values: list[dict[str, Any]]) -> dict[str, Any]:
return {"data": values, "count": len(values)}
def _hub(row: Any) -> dict[str, Any]:
value = dict(row["body"] or {})
value.update(
id=row["id"], slug=row["slug"], name=row["name"], domain=row["domain"],
hubKind=row["hub_kind"], hubFamily=row["hub_family"],
vsmFunction=row["vsm_function"], vsmSystem=row["vsm_system"],
status=row["status"], description=row["description"],
)
return {key: item for key, item in value.items() if item is not None}
def _manifest(row: Any) -> dict[str, Any]:
value = dict(row["body"] or {})
value.update(
id=row["id"], hubId=row["hub_id"], hubSlug=row["hub_slug"],
manifestVersion=row["manifest_version"], status=row["status"],
)
return {key: item for key, item in value.items() if item is not None}
def _consumer(row: Any) -> dict[str, Any]:
value = dict(row["body"] or {})
value.update(
id=row["id"], slug=row["slug"], name=row["name"],
description=row["description"],
hubCapabilityManifestId=row["hub_capability_manifest_id"],
rateLimitPerMinute=row["rate_limit_per_minute"], quotaPerDay=row["quota_per_day"],
keyPrefix=row["key_prefix"], status=row["status"],
)
return {key: item for key, item in value.items() if item is not None}
def _widget(row: Any) -> dict[str, Any]:
value = dict(row["body"] or {})
value.update(
id=row["id"], hubId=row["hub_id"], name=row["name"],
widgetType=row["widget_type"], capabilityRef=row["capability_ref"],
viewContext=row["view_context"], policyScope=row["policy_scope"],
status=row["status"],
)
return {key: item for key, item in value.items() if item is not None}
def _hash_token(token: str) -> str:
return hashlib.sha256(token.encode()).hexdigest()
def _now() -> datetime:
return datetime.now(timezone.utc)

View file

@ -0,0 +1,66 @@
from __future__ import annotations
def item(slug: str, name: str, description: str | None = None) -> dict[str, str]:
value = {"slug": slug, "name": name}
if description is not None:
value["description"] = description
return value
WIDGET_TYPES = [
item("status-summary", "Status Summary"),
item("workplan-board", "Workplan Board"),
item("event-stream", "Event Stream"),
item("action", "Action Control", "Button, link, or trigger widget"),
item("chart", "Chart", "Data visualisation chart widget"),
item("chat", "Chat Region", "Conversational interaction region"),
item("diff", "Diff / Review", "Code diff or change review element"),
item("form", "Form", "Data entry form widget"),
*[item(f"ops-{name}", f"ops-{name}") for name in (
"backup-set", "cluster", "endpoint", "environment", "host", "incident",
"migration-wave", "readiness-gate", "release", "risk", "runbook",
"secret-set", "service", "service-catalog",
)],
item("panel", "Status Panel", "Summary or status information panel"),
item("recommendation", "Recommendation", "AI or system recommendation block"),
item("table", "Table", "Tabular data display widget"),
item("workflow-step", "Workflow Step", "Single step in a multi-step workflow"),
]
EVENT_TYPES = [
item("interaction.event", "Interaction Event"),
item("workplan.progress", "Workplan Progress"),
item("evidence.recorded", "Evidence Recorded"),
*[
item(slug, slug.replace("_", " ").title())
for slug in (
"abandoned", "accepted_recommendation", "blocked_by_policy", "clicked",
"commented", "escalated", "failed", "flagged_confusing", "flagged_helpful",
"focused", "rejected_recommendation", "retracted", "retried", "submitted",
"viewed",
)
],
*[
item(f"ops-{name}", f"ops-{name}")
for name in (
"backup-verified", "drift-detected", "endpoint-verified", "health-checked",
"inventory-registered", "inventory-updated", "migration-gate-failed",
"migration-gate-passed", "readiness-gate-updated", "release-observed",
"restore-tested", "risk-accepted", "risk-raised", "runbook-executed",
"service-discovered",
)
],
]
ANNOTATION_CATEGORIES = [
item("operator-note", "Operator Note"),
item("compatibility-delta", "Compatibility Delta"),
item("migration-evidence", "Migration Evidence"),
]
POLICY_SCOPES = [
item("public-read", "Public Read"),
item("hub-write", "Hub Write"),
item("operator-admin", "Operator Admin"),
]

View file

@ -11,6 +11,10 @@ def _env_bool(name: str, default: bool) -> bool:
return raw.strip().lower() in {"1", "true", "yes", "on"} return raw.strip().lower() in {"1", "true", "yes", "on"}
def _env_set(name: str) -> frozenset[str]:
return frozenset(value.strip() for value in os.getenv(name, "").split(",") if value.strip())
@dataclass(frozen=True, slots=True) @dataclass(frozen=True, slots=True)
class RuntimeSettings: class RuntimeSettings:
environment: str = "development" environment: str = "development"
@ -23,6 +27,21 @@ class RuntimeSettings:
mcp_port: int = 8011 mcp_port: int = 8011
mcp_transport: str = "http" mcp_transport: str = "http"
database_url: str | None = None database_url: str | None = None
api_token: str | None = None
v2_groups: frozenset[str] = frozenset()
v2_write_groups: frozenset[str] = frozenset()
legacy_write_groups: frozenset[str] = frozenset()
legacy_health: bool = False
def __post_init__(self) -> None:
overlap = self.v2_write_groups & self.legacy_write_groups
if overlap:
joined = ", ".join(sorted(overlap))
raise ValueError(f"hub-core and Core Hub write groups overlap: {joined}")
unknown = self.v2_write_groups - self.v2_groups
if unknown:
joined = ", ".join(sorted(unknown))
raise ValueError(f"hub-core write groups are not enabled: {joined}")
@classmethod @classmethod
def from_env(cls) -> RuntimeSettings: def from_env(cls) -> RuntimeSettings:
@ -41,6 +60,11 @@ class RuntimeSettings:
mcp_port=int(os.getenv("HUB_CORE_MCP_PORT", "8011")), mcp_port=int(os.getenv("HUB_CORE_MCP_PORT", "8011")),
mcp_transport=os.getenv("HUB_CORE_MCP_TRANSPORT", "http"), mcp_transport=os.getenv("HUB_CORE_MCP_TRANSPORT", "http"),
database_url=os.getenv("HUB_CORE_DATABASE_URL") or os.getenv("DATABASE_URL"), database_url=os.getenv("HUB_CORE_DATABASE_URL") or os.getenv("DATABASE_URL"),
api_token=os.getenv("HUB_CORE_API_TOKEN") or os.getenv("CORE_HUB_API_TOKEN"),
v2_groups=_env_set("HUB_CORE_V2_GROUPS"),
v2_write_groups=_env_set("HUB_CORE_V2_WRITE_GROUPS"),
legacy_write_groups=_env_set("CORE_HUB_V2_WRITE_GROUPS"),
legacy_health=_env_bool("HUB_CORE_LEGACY_HEALTH", False),
) )
def readiness_checks(self, store_backend: str) -> dict[str, str]: def readiness_checks(self, store_backend: str) -> dict[str, str]:
@ -51,6 +75,9 @@ class RuntimeSettings:
"active_backend": store_backend, "active_backend": store_backend,
"ephemeral_backend": "allowed" if ephemeral_allowed else "not_allowed", "ephemeral_backend": "allowed" if ephemeral_allowed else "not_allowed",
"contract": "helixforge.hub-extension/0.1.0", "contract": "helixforge.hub-extension/0.1.0",
"v2_groups": ",".join(sorted(self.v2_groups)) or "none",
"v2_write_groups": ",".join(sorted(self.v2_write_groups)) or "none",
"legacy_write_groups": ",".join(sorted(self.legacy_write_groups)) or "none",
} }
def is_ready(self, store_backend: str) -> bool: def is_ready(self, store_backend: str) -> bool:

View file

@ -0,0 +1,438 @@
from __future__ import annotations
import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from uuid import NAMESPACE_URL, uuid4, uuid5
import sqlalchemy as sa
from hub_core.runtime.postgres_store import PostgresPortStore
from hub_core.runtime.tables import (
compat_api_consumers,
compat_api_keys,
compat_hubs,
compat_manifests,
compat_migration_runs,
compat_widgets,
runtime_audit_ledger,
runtime_import_runs,
runtime_interaction_events,
)
SCHEMA_VERSION = "core-hub.migration.v1"
COLLECTIONS = (
"hubs",
"hubCapabilityManifests",
"apiConsumers",
"apiKeys",
"widgets",
"interactionEvents",
"migrationRuns",
)
SECRET_FIELDS = {"fullKey", "rawKey", "apiKey", "secret", "token", "authorization"}
def _now() -> datetime:
return datetime.now(timezone.utc)
def _json(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), default=_json_default)
def _json_default(value: Any) -> str:
if isinstance(value, datetime):
return _timestamp(value)
raise TypeError(f"cannot encode {type(value).__name__}")
def _timestamp(value: datetime) -> str:
if value.tzinfo is None:
value = value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
def _datetime(value: Any, default: datetime | None = None) -> datetime:
if isinstance(value, datetime):
return value if value.tzinfo else value.replace(tzinfo=timezone.utc)
if isinstance(value, str) and value:
return datetime.fromisoformat(value.replace("Z", "+00:00"))
return default or _now()
def _records(bundle: dict[str, Any], collection: str) -> list[dict[str, Any]]:
records = bundle.get("records", {})
values = records.get(collection, []) if isinstance(records, dict) else []
return [dict(value) for value in values] if isinstance(values, list) else []
def _contains_secret(value: Any) -> bool:
if isinstance(value, dict):
return any(key in SECRET_FIELDS or _contains_secret(child) for key, child in value.items())
if isinstance(value, list):
return any(_contains_secret(child) for child in value)
return False
def _canonical_record(record: dict[str, Any]) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, value in record.items():
if key in {"createdAt", "updatedAt", "occurredAt", "recordedAt"} and value:
result[key] = _timestamp(_datetime(value))
elif isinstance(value, dict):
result[key] = _canonical_record(value)
elif isinstance(value, list):
result[key] = [
_canonical_record(item) if isinstance(item, dict) else item for item in value
]
else:
result[key] = value
return result
def _identity(record: dict[str, Any]) -> str:
return str(record.get("id") or record.get("slug") or record.get("keyHash") or "")
def content_hashes(bundle: dict[str, Any]) -> dict[str, str]:
hashes: dict[str, str] = {}
for collection in COLLECTIONS:
values = [_canonical_record(value) for value in _records(bundle, collection)]
values.sort(key=_identity)
hashes[collection] = hashlib.sha256(_json(values).encode()).hexdigest()
return hashes
def identity_sets(bundle: dict[str, Any]) -> dict[str, list[str]]:
return {
collection: sorted(_identity(value) for value in _records(bundle, collection))
for collection in COLLECTIONS
}
def bundle_digest(bundle: dict[str, Any]) -> str:
value = {key: child for key, child in bundle.items() if key != "bundleSha256"}
return hashlib.sha256(_json(value).encode()).hexdigest()
def validate_bundle(bundle: dict[str, Any]) -> dict[str, Any]:
errors: list[str] = []
warnings: list[str] = []
unknown = set(bundle) - {
"schemaVersion",
"source",
"sourceRevision",
"exportedAt",
"highWaterMark",
"bundleSha256",
"records",
}
if unknown:
errors.append(f"unknown bundle fields: {', '.join(sorted(unknown))}")
if bundle.get("schemaVersion") != SCHEMA_VERSION:
errors.append(f"schemaVersion must be {SCHEMA_VERSION}")
if not isinstance(bundle.get("records"), dict):
errors.append("records must be an object")
else:
unknown_collections = set(bundle["records"]) - set(COLLECTIONS)
if unknown_collections:
errors.append(f"unknown collections: {', '.join(sorted(unknown_collections))}")
claimed = bundle.get("bundleSha256")
calculated = bundle_digest(bundle)
if claimed and claimed != calculated:
errors.append("bundleSha256 does not match canonical bundle content")
required = {
"hubs": ("slug", "name"),
"hubCapabilityManifests": ("id",),
"apiConsumers": ("name",),
"apiKeys": ("id", "keyPrefix", "keyHash"),
"widgets": ("id", "name"),
"interactionEvents": ("id", "widgetId", "eventType"),
"migrationRuns": ("id", "source", "schemaVersion", "bundleSha256"),
}
for collection in COLLECTIONS:
seen: set[str] = set()
for index, record in enumerate(_records(bundle, collection)):
path = f"records.{collection}[{index}]"
for field in required[collection]:
if record.get(field) in (None, ""):
errors.append(f"{path}.{field} is required")
identity = _identity(record)
if identity and identity in seen:
errors.append(f"duplicate {collection} identity: {identity}")
seen.add(identity)
if _contains_secret(record):
errors.append(f"{path} contains secret-shaped material")
for record in _records(bundle, "hubCapabilityManifests"):
if not record.get("hubId") and not record.get("hubSlug"):
errors.append("manifest must include hubId or hubSlug")
for record in _records(bundle, "apiConsumers"):
if not record.get("id") and not record.get("slug"):
errors.append("API consumer must include id or slug")
for record in _records(bundle, "apiKeys"):
if not record.get("apiConsumerId") and not record.get("apiConsumerSlug"):
errors.append("API key must include apiConsumerId or apiConsumerSlug")
for record in _records(bundle, "widgets"):
if not record.get("hubId") and not record.get("hubSlug"):
errors.append("widget must include hubId or hubSlug")
return {
"ok": not errors,
"schemaVersion": bundle.get("schemaVersion"),
"source": bundle.get("source", "unknown"),
"bundleSha256": calculated,
"highWaterMark": bundle.get("highWaterMark"),
"counts": {name: {"input": len(_records(bundle, name))} for name in COLLECTIONS},
"identitySets": identity_sets(bundle),
"contentHashes": content_hashes(bundle),
"errors": errors,
"warnings": warnings,
}
async def import_bundle(
store: PostgresPortStore, bundle: dict[str, Any], *, dry_run: bool = False
) -> dict[str, Any]:
report = validate_bundle(bundle)
report["dryRun"] = dry_run
report["idempotent"] = False
report["counts"] = {
name: {"input": len(_records(bundle, name)), "created": 0, "updated": 0, "skipped": 0}
for name in COLLECTIONS
}
if not report["ok"]:
return report
async with store.sessions() as session:
prior = (
await session.execute(
sa.select(runtime_import_runs).where(
runtime_import_runs.c.bundle_sha256 == report["bundleSha256"]
)
)
).mappings().first()
if prior and not dry_run:
report["idempotent"] = True
report["migrationRunId"] = prior["id"]
for name in COLLECTIONS:
report["counts"][name]["skipped"] = report["counts"][name]["input"]
return report
async with store.sessions.begin() as session:
hubs = await _maps(session, compat_hubs, "id", "slug")
await _import_hubs(session, bundle, report, hubs, dry_run)
hubs = await _maps(session, compat_hubs, "id", "slug") if not dry_run else hubs
await _import_manifests(session, bundle, report, hubs, dry_run)
consumers = await _maps(session, compat_api_consumers, "id", "slug")
await _import_consumers(session, bundle, report, consumers, dry_run)
consumers = (
await _maps(session, compat_api_consumers, "id", "slug") if not dry_run else consumers
)
await _import_keys(session, bundle, report, consumers, dry_run)
await _import_widgets(session, bundle, report, hubs, dry_run)
await _import_events(session, bundle, report, dry_run)
await _import_source_runs(session, bundle, report, dry_run)
if dry_run:
await session.rollback()
return report
run_id = str(uuid4())
await session.execute(
runtime_import_runs.insert().values(
id=run_id,
source=report["source"],
schema_version=SCHEMA_VERSION,
bundle_sha256=report["bundleSha256"],
high_water_mark=report["highWaterMark"],
counts=report["counts"],
content_hashes=report["contentHashes"],
status="imported",
created_at=_now(),
)
)
await session.execute(
runtime_audit_ledger.insert().values(
id=str(uuid4()),
action="migration.imported",
subject_type="migration_bundle",
subject_id=report["bundleSha256"],
correlation_id=None,
payload_hash=hashlib.sha256(_json(report["contentHashes"]).encode()).hexdigest(),
detail={"source": report["source"], "counts": report["counts"]},
recorded_at=_now(),
)
)
report["migrationRunId"] = run_id
return report
async def _maps(session: Any, table: sa.Table, *keys: str) -> dict[str, dict[str, Any]]:
rows = (await session.execute(sa.select(table))).mappings().all()
return {str(row[key]): dict(row) for row in rows for key in keys if row[key]}
def _mark(report: dict[str, Any], name: str, existing: Any) -> None:
report["counts"][name]["updated" if existing else "created"] += 1
async def _upsert(session: Any, table: sa.Table, row: dict[str, Any], existing: Any) -> None:
if existing:
await session.execute(table.update().where(table.c.id == existing["id"]).values(**row))
else:
await session.execute(table.insert().values(**row))
def _parent_id(record: dict[str, Any], rows: dict[str, dict[str, Any]], kind: str) -> str:
parent = record.get(f"{kind}Id") or record.get(f"{kind}Slug")
if parent in rows:
return str(rows[str(parent)]["id"])
raise ValueError(f"unresolved {kind} reference: {parent}")
async def _import_hubs(session: Any, bundle: dict[str, Any], report: dict, rows: dict, dry: bool) -> None:
for record in _records(bundle, "hubs"):
existing = rows.get(str(record.get("id"))) or rows.get(str(record.get("slug")))
_mark(report, "hubs", existing)
if dry:
continue
now = _now()
row = {
"id": existing["id"] if existing else str(record.get("id") or uuid4()),
"slug": record["slug"], "name": record["name"], "domain": record.get("domain"),
"hub_kind": record.get("hubKind"), "hub_family": record.get("hubFamily"),
"vsm_function": record.get("vsmFunction"), "vsm_system": record.get("vsmSystem"),
"status": record.get("status", "active"), "description": record.get("description"),
"body": record, "created_at": _datetime(record.get("createdAt"), now),
"updated_at": _datetime(record.get("updatedAt"), now),
}
await _upsert(session, compat_hubs, row, existing)
async def _import_manifests(session: Any, bundle: dict, report: dict, hubs: dict, dry: bool) -> None:
current = await _maps(session, compat_manifests, "id")
for record in _records(bundle, "hubCapabilityManifests"):
existing = current.get(str(record["id"])); _mark(report, "hubCapabilityManifests", existing)
if dry: continue
now = _now(); hub_id = _parent_id(record, hubs, "hub")
row = {"id": record["id"], "hub_id": hub_id, "hub_slug": hubs[hub_id]["slug"],
"manifest_version": record.get("manifestVersion", "0.1.0"),
"status": record.get("status", "draft"), "body": record,
"created_at": _datetime(record.get("createdAt"), now),
"updated_at": _datetime(record.get("updatedAt"), now)}
await _upsert(session, compat_manifests, row, existing)
async def _import_consumers(session: Any, bundle: dict, report: dict, rows: dict, dry: bool) -> None:
for record in _records(bundle, "apiConsumers"):
existing = rows.get(str(record.get("id"))) or rows.get(str(record.get("slug")))
_mark(report, "apiConsumers", existing)
if dry: continue
row = {"id": existing["id"] if existing else str(record.get("id") or uuid4()),
"slug": record.get("slug"), "name": record["name"],
"description": record.get("description"),
"hub_capability_manifest_id": record.get("hubCapabilityManifestId"),
"rate_limit_per_minute": record.get("rateLimitPerMinute"),
"quota_per_day": record.get("quotaPerDay"), "key_prefix": record.get("keyPrefix"),
"status": record.get("status", "active"), "body": record,
"created_at": _datetime(record.get("createdAt"))}
await _upsert(session, compat_api_consumers, row, existing)
async def _import_keys(session: Any, bundle: dict, report: dict, consumers: dict, dry: bool) -> None:
current = await _maps(session, compat_api_keys, "id", "key_hash")
for record in _records(bundle, "apiKeys"):
existing = current.get(str(record["id"])) or current.get(str(record["keyHash"]))
_mark(report, "apiKeys", existing)
if dry: continue
consumer_id = _parent_id(record, consumers, "apiConsumer")
row = {"id": record["id"], "api_consumer_id": consumer_id,
"key_prefix": record["keyPrefix"], "key_hash": record["keyHash"],
"scopes": record.get("scopes"), "status": record.get("status", "active"),
"created_at": _datetime(record.get("createdAt"))}
await _upsert(session, compat_api_keys, row, existing)
async def _import_widgets(session: Any, bundle: dict, report: dict, hubs: dict, dry: bool) -> None:
current = await _maps(session, compat_widgets, "id")
for record in _records(bundle, "widgets"):
existing = current.get(str(record["id"])); _mark(report, "widgets", existing)
if dry: continue
row = {"id": record["id"], "hub_id": _parent_id(record, hubs, "hub"),
"name": record["name"], "widget_type": record.get("widgetType"),
"capability_ref": record.get("capabilityRef"),
"view_context": record.get("viewContext"), "policy_scope": record.get("policyScope"),
"status": record.get("status", "active"), "body": record,
"created_at": _datetime(record.get("createdAt"))}
await _upsert(session, compat_widgets, row, existing)
async def _import_events(session: Any, bundle: dict, report: dict, dry: bool) -> None:
current = await _maps(session, runtime_interaction_events, "id")
for record in _records(bundle, "interactionEvents"):
existing = current.get(str(record["id"])); _mark(report, "interactionEvents", existing)
if dry: continue
created = _datetime(record.get("createdAt") or record.get("occurredAt"))
row = {"id": record["id"], "schema_version": "0.1.0",
"correlation_id": str(record.get("correlationId") or uuid5(NAMESPACE_URL, record["id"])),
"event_type": record["eventType"], "occurred_at": created,
"subject_refs": {"widget": record["widgetId"]}, "payload": {"legacy": record},
"recorded_at": _datetime(record.get("recordedAt"), created)}
await _upsert(session, runtime_interaction_events, row, existing)
async def _import_source_runs(session: Any, bundle: dict, report: dict, dry: bool) -> None:
current = await _maps(session, compat_migration_runs, "id")
for record in _records(bundle, "migrationRuns"):
existing = current.get(str(record["id"])); _mark(report, "migrationRuns", existing)
if dry: continue
row = {"id": record["id"], "source": record["source"],
"schema_version": record["schemaVersion"], "bundle_sha256": record["bundleSha256"],
"dry_run": bool(record.get("dryRun", False)), "status": record.get("status", "imported"),
"counts": record.get("counts", {}), "diagnostics": record.get("diagnostics", {}),
"created_at": _datetime(record.get("createdAt"))}
await _upsert(session, compat_migration_runs, row, existing)
async def export_bundle(store: PostgresPortStore, *, source_revision: str | None = None) -> dict[str, Any]:
async with store.sessions() as session:
hubs = (await session.execute(sa.select(compat_hubs).order_by(compat_hubs.c.id))).mappings().all()
manifests = (await session.execute(sa.select(compat_manifests).order_by(compat_manifests.c.id))).mappings().all()
consumers = (await session.execute(sa.select(compat_api_consumers).order_by(compat_api_consumers.c.id))).mappings().all()
keys = (await session.execute(sa.select(compat_api_keys).order_by(compat_api_keys.c.id))).mappings().all()
widgets = (await session.execute(sa.select(compat_widgets).order_by(compat_widgets.c.id))).mappings().all()
events = (await session.execute(sa.select(runtime_interaction_events).order_by(runtime_interaction_events.c.id))).mappings().all()
runs = (await session.execute(sa.select(compat_migration_runs).order_by(compat_migration_runs.c.id))).mappings().all()
records = {
"hubs": [_body(row, created=True, updated=True) for row in hubs],
"hubCapabilityManifests": [_body(row, created=True, updated=True) for row in manifests],
"apiConsumers": [_body(row, created=True) for row in consumers],
"apiKeys": [{"id": row["id"], "apiConsumerId": row["api_consumer_id"], "keyPrefix": row["key_prefix"], "keyHash": row["key_hash"], "scopes": row["scopes"], "status": row["status"], "createdAt": _timestamp(row["created_at"])} for row in keys],
"widgets": [_body(row, created=True) for row in widgets],
"interactionEvents": [{**dict((row["payload"] or {}).get("legacy") or {}), "id": row["id"], "widgetId": (row["subject_refs"] or {}).get("widget"), "eventType": row["event_type"], "occurredAt": _timestamp(row["occurred_at"]), "recordedAt": _timestamp(row["recorded_at"]), "correlationId": row["correlation_id"]} for row in events],
"migrationRuns": [{"id": row["id"], "source": row["source"], "schemaVersion": row["schema_version"], "bundleSha256": row["bundle_sha256"], "dryRun": row["dry_run"], "status": row["status"], "counts": row["counts"], "diagnostics": row["diagnostics"], "createdAt": _timestamp(row["created_at"])} for row in runs],
}
high_water = max((_timestamp(row["recorded_at"]) for row in events), default=None)
bundle: dict[str, Any] = {"schemaVersion": SCHEMA_VERSION, "source": "hub-core", "exportedAt": _timestamp(_now()), "highWaterMark": high_water, "records": records}
if source_revision:
bundle["sourceRevision"] = source_revision
bundle["bundleSha256"] = bundle_digest(bundle)
return bundle
def _body(row: Any, *, created: bool = False, updated: bool = False) -> dict[str, Any]:
value = dict(row["body"] or {})
if created:
value["createdAt"] = _timestamp(row["created_at"])
if updated:
value["updatedAt"] = _timestamp(row["updated_at"])
return value
def load_bundle(path: Path) -> dict[str, Any]:
value = json.loads(path.read_text())
if not isinstance(value, dict):
raise ValueError("migration bundle must be a JSON object")
return value

View file

@ -0,0 +1,321 @@
from __future__ import annotations
import hashlib
import json
from collections.abc import Mapping
from copy import deepcopy
from datetime import datetime, timezone
from typing import Any
from uuid import UUID, uuid4
import sqlalchemy as sa
from sqlalchemy.ext.asyncio import AsyncEngine, async_sessionmaker, create_async_engine
from hub_core.contracts import CONTRACT_VERSION
from hub_core.runtime.models import (
EventCommand,
MessageCommand,
PortAccepted,
PortCollection,
PortRecord,
Provenance,
RegistryRegistration,
)
from hub_core.runtime.tables import (
runtime_audit_ledger,
runtime_interaction_events,
runtime_messages,
runtime_progress_events,
runtime_registrations,
)
class PostgresPortStore:
"""Durable named-port store for PostgreSQL-compatible async engines."""
backend_name = "postgresql"
def __init__(self, engine: AsyncEngine) -> None:
self.engine = engine
self.sessions = async_sessionmaker(engine, expire_on_commit=False)
@classmethod
def from_url(cls, database_url: str) -> PostgresPortStore:
return cls(create_async_engine(database_url, pool_pre_ping=True))
async def readiness_checks(self) -> dict[str, str]:
try:
async with self.engine.connect() as connection:
await connection.execute(sa.text("SELECT 1"))
except Exception:
return {"database": "unavailable"}
return {"database": "ok"}
async def aclose(self) -> None:
await self.engine.dispose()
async def register_extension(
self,
registration: RegistryRegistration,
correlation_id: UUID,
) -> PortAccepted:
hub_slug = str(registration.descriptor["hub_slug"])
value = registration.model_dump(mode="json")
now = _now()
async with self.sessions.begin() as session:
current = (
await session.execute(
sa.select(runtime_registrations.c.package).where(
runtime_registrations.c.hub_slug == hub_slug
)
)
).scalar_one_or_none()
duplicate = current == value
if current is None:
await session.execute(
runtime_registrations.insert().values(
hub_slug=hub_slug,
package=value,
created_at=now,
updated_at=now,
)
)
elif not duplicate:
await session.execute(
runtime_registrations.update()
.where(runtime_registrations.c.hub_slug == hub_slug)
.values(package=value, updated_at=now)
)
await self._audit(
session,
action="registry.duplicate" if duplicate else "registry.accepted",
subject_type="registration",
subject_id=hub_slug,
correlation_id=correlation_id,
value=value,
)
return PortAccepted(
id=hub_slug,
status="duplicate" if duplicate else "accepted",
correlation_id=correlation_id,
)
async def send_message(self, command: MessageCommand) -> PortAccepted:
message_id = uuid4()
created_at = _now()
value = {
"id": str(message_id),
"created_at": created_at.isoformat(),
**command.model_dump(mode="json"),
}
stored = {
**value,
"created_at": created_at,
}
async with self.sessions.begin() as session:
await session.execute(runtime_messages.insert().values(**stored))
await self._audit(
session,
action="message.accepted",
subject_type="message",
subject_id=str(message_id),
correlation_id=command.correlation_id,
value=value,
)
return PortAccepted(
id=str(message_id), status="accepted", correlation_id=command.correlation_id
)
async def list_messages(
self, address: str, conversation_id: UUID | None
) -> PortCollection:
statement = sa.select(runtime_messages).order_by(
runtime_messages.c.created_at, runtime_messages.c.id
)
if conversation_id is not None:
statement = statement.where(
runtime_messages.c.conversation_id == str(conversation_id)
)
async with self.sessions() as session:
rows = (await session.execute(statement)).mappings().all()
values = [
_message_value(row)
for row in rows
if address in list(row["to_addresses"] or [])
]
return PortCollection(items=[self._record("message", value) for value in values])
async def append_progress(self, command: EventCommand) -> PortAccepted:
return await self._append_event(command, runtime_progress_events, "progress")
async def append_interaction(self, command: EventCommand) -> PortAccepted:
return await self._append_event(command, runtime_interaction_events, "interaction")
async def query_projection(self, projection_id: str) -> PortRecord | None:
async with self.sessions() as session:
if projection_id == "hub_registry":
rows = (
await session.execute(
sa.select(runtime_registrations.c.package).order_by(
runtime_registrations.c.hub_slug
)
)
).scalars()
items = [deepcopy(value) for value in rows]
elif projection_id == "messages":
rows = (
await session.execute(
sa.select(runtime_messages).order_by(
runtime_messages.c.created_at, runtime_messages.c.id
)
)
).mappings()
items = [_message_value(row) for row in rows]
elif projection_id in {"progress_events", "interaction_events"}:
table = (
runtime_progress_events
if projection_id == "progress_events"
else runtime_interaction_events
)
family = "progress" if projection_id == "progress_events" else "interaction"
rows = (
await session.execute(
sa.select(table).order_by(table.c.recorded_at, table.c.id)
)
).mappings()
items = [_event_value(row, family) for row in rows]
else:
return None
return self._record(
projection_id,
{
"projection_id": projection_id,
"items": items,
"rebuild_from": _rebuild_sources(projection_id),
},
)
async def _append_event(
self,
command: EventCommand,
table: sa.Table,
family: str,
) -> PortAccepted:
event_id = uuid4()
recorded_at = _now()
value = {
"id": str(event_id),
"family": family,
"recorded_at": recorded_at.isoformat(),
**command.model_dump(mode="json"),
}
stored = {
"id": str(event_id),
"schema_version": command.schema_version,
"correlation_id": str(command.correlation_id),
"event_type": command.event_type,
"occurred_at": command.occurred_at,
"subject_refs": command.subject_refs,
"payload": command.payload,
"recorded_at": recorded_at,
}
async with self.sessions.begin() as session:
await session.execute(table.insert().values(**stored))
await self._audit(
session,
action=f"event.{family}.accepted",
subject_type=f"{family}_event",
subject_id=str(event_id),
correlation_id=command.correlation_id,
value=value,
)
return PortAccepted(
id=str(event_id), status="accepted", correlation_id=command.correlation_id
)
async def _audit(
self,
session: Any,
*,
action: str,
subject_type: str,
subject_id: str,
correlation_id: UUID | None,
value: Mapping[str, Any],
) -> None:
await session.execute(
runtime_audit_ledger.insert().values(
id=str(uuid4()),
action=action,
subject_type=subject_type,
subject_id=subject_id,
correlation_id=str(correlation_id) if correlation_id else None,
payload_hash=_hash(value),
detail={"schema_version": CONTRACT_VERSION},
recorded_at=_now(),
)
)
def _record(self, kind: str, value: dict[str, Any]) -> PortRecord:
record_id = str(value.get("id") or kind)
return PortRecord(
id=record_id,
data=deepcopy(value),
provenance=Provenance(
source_system="hub-core-postgresql",
source_ref=f"postgresql://hub_runtime/{kind}/{record_id}",
schema_version=CONTRACT_VERSION,
content_hash=_hash(value),
indexed_at=_now(),
),
)
def _message_value(row: Mapping[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"created_at": _iso(row["created_at"]),
"schema_version": row["schema_version"],
"correlation_id": str(row["correlation_id"]),
"conversation_id": str(row["conversation_id"]) if row["conversation_id"] else None,
"from_address": row["from_address"],
"to_addresses": list(row["to_addresses"] or []),
"body": row["body"],
"subject_refs": dict(row["subject_refs"] or {}),
}
def _event_value(row: Mapping[str, Any], family: str) -> dict[str, Any]:
return {
"id": str(row["id"]),
"family": family,
"recorded_at": _iso(row["recorded_at"]),
"schema_version": row["schema_version"],
"correlation_id": str(row["correlation_id"]),
"event_type": row["event_type"],
"occurred_at": _iso(row["occurred_at"]),
"subject_refs": dict(row["subject_refs"] or {}),
"payload": dict(row["payload"] or {}),
}
def _hash(value: Mapping[str, Any]) -> str:
encoded = json.dumps(value, sort_keys=True, separators=(",", ":"), default=str).encode()
return hashlib.sha256(encoded).hexdigest()
def _iso(value: Any) -> str:
return value.isoformat() if isinstance(value, datetime) else str(value)
def _now() -> datetime:
return datetime.now(timezone.utc)
def _rebuild_sources(projection_id: str) -> list[str]:
return {
"hub_registry": ["runtime_registrations"],
"messages": ["runtime_messages"],
"progress_events": ["runtime_progress_events"],
"interaction_events": ["runtime_interaction_events"],
}[projection_id]

View file

@ -26,6 +26,8 @@ class PortStore(Protocol):
backend_name: str backend_name: str
async def readiness_checks(self) -> dict[str, str]: ...
async def register_extension( async def register_extension(
self, self,
registration: RegistryRegistration, registration: RegistryRegistration,
@ -59,6 +61,9 @@ class InMemoryPortStore:
self._progress_events: list[dict[str, Any]] = [] self._progress_events: list[dict[str, Any]] = []
self._interaction_events: list[dict[str, Any]] = [] self._interaction_events: list[dict[str, Any]] = []
async def readiness_checks(self) -> dict[str, str]:
return {"database": "not_applicable"}
async def register_extension( async def register_extension(
self, self,
registration: RegistryRegistration, registration: RegistryRegistration,

169
hub_core/runtime/tables.py Normal file
View file

@ -0,0 +1,169 @@
from __future__ import annotations
import sqlalchemy as sa
runtime_metadata = sa.MetaData()
runtime_registrations = sa.Table(
"runtime_registrations",
runtime_metadata,
sa.Column("hub_slug", sa.String(120), primary_key=True),
sa.Column("package", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
runtime_messages = sa.Table(
"runtime_messages",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("schema_version", sa.String(40), nullable=False),
sa.Column("correlation_id", sa.String(36), nullable=False, index=True),
sa.Column("conversation_id", sa.String(36), nullable=True, index=True),
sa.Column("from_address", sa.String(240), nullable=False),
sa.Column("to_addresses", sa.JSON(), nullable=False),
sa.Column("body", sa.Text(), nullable=False),
sa.Column("subject_refs", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, index=True),
)
def _event_table(name: str) -> sa.Table:
return sa.Table(
name,
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("schema_version", sa.String(40), nullable=False),
sa.Column("correlation_id", sa.String(36), nullable=False, index=True),
sa.Column("event_type", sa.String(160), nullable=False, index=True),
sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("subject_refs", sa.JSON(), nullable=False),
sa.Column("payload", sa.JSON(), nullable=False),
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False, index=True),
)
runtime_progress_events = _event_table("runtime_progress_events")
runtime_interaction_events = _event_table("runtime_interaction_events")
runtime_audit_ledger = sa.Table(
"runtime_audit_ledger",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("action", sa.String(120), nullable=False, index=True),
sa.Column("subject_type", sa.String(80), nullable=False),
sa.Column("subject_id", sa.String(240), nullable=False, index=True),
sa.Column("correlation_id", sa.String(36), nullable=True, index=True),
sa.Column("payload_hash", sa.String(64), nullable=False),
sa.Column("detail", sa.JSON(), nullable=False),
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False, index=True),
)
compat_hubs = sa.Table(
"compat_hubs",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("slug", sa.String(120), nullable=False, unique=True, index=True),
sa.Column("name", sa.String(240), nullable=False),
sa.Column("domain", sa.String(240), nullable=True),
sa.Column("hub_kind", sa.String(80), nullable=True),
sa.Column("hub_family", sa.String(80), nullable=True),
sa.Column("vsm_function", sa.String(80), nullable=True),
sa.Column("vsm_system", sa.String(80), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("body", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
compat_manifests = sa.Table(
"compat_manifests",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=True, index=True),
sa.Column("hub_slug", sa.String(120), nullable=True, index=True),
sa.Column("manifest_version", sa.String(40), nullable=False),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("body", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
compat_api_consumers = sa.Table(
"compat_api_consumers",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("slug", sa.String(120), nullable=True, unique=True, index=True),
sa.Column("name", sa.String(240), nullable=False, index=True),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("hub_capability_manifest_id", sa.String(36), nullable=True, index=True),
sa.Column("rate_limit_per_minute", sa.Integer(), nullable=True),
sa.Column("quota_per_day", sa.Integer(), nullable=True),
sa.Column("key_prefix", sa.String(32), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("body", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
compat_api_keys = sa.Table(
"compat_api_keys",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column(
"api_consumer_id",
sa.String(36),
sa.ForeignKey("compat_api_consumers.id"),
nullable=False,
index=True,
),
sa.Column("key_prefix", sa.String(32), nullable=False, index=True),
sa.Column("key_hash", sa.String(64), nullable=False, unique=True),
sa.Column("scopes", sa.Text(), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
compat_widgets = sa.Table(
"compat_widgets",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=False, index=True),
sa.Column("name", sa.String(240), nullable=False),
sa.Column("widget_type", sa.String(120), nullable=True, index=True),
sa.Column("capability_ref", sa.String(240), nullable=True, index=True),
sa.Column("view_context", sa.String(500), nullable=True),
sa.Column("policy_scope", sa.String(120), nullable=True),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("body", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
compat_migration_runs = sa.Table(
"compat_migration_runs",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("source", sa.String(120), nullable=False, index=True),
sa.Column("schema_version", sa.String(80), nullable=False),
sa.Column("bundle_sha256", sa.String(64), nullable=False, index=True),
sa.Column("dry_run", sa.Boolean(), nullable=False),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("counts", sa.JSON(), nullable=False),
sa.Column("diagnostics", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
runtime_import_runs = sa.Table(
"runtime_import_runs",
runtime_metadata,
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("source", sa.String(120), nullable=False, index=True),
sa.Column("schema_version", sa.String(80), nullable=False),
sa.Column("bundle_sha256", sa.String(64), nullable=False, unique=True, index=True),
sa.Column("high_water_mark", sa.String(200), nullable=True),
sa.Column("counts", sa.JSON(), nullable=False),
sa.Column("content_hashes", sa.JSON(), nullable=False),
sa.Column("status", sa.String(40), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)

View file

@ -25,6 +25,7 @@ hub-core = "hub_core.runtime.cli:main"
[dependency-groups] [dependency-groups]
dev = [ dev = [
"aiosqlite>=0.20.0",
"pytest>=8.0.0", "pytest>=8.0.0",
] ]

View file

@ -0,0 +1,16 @@
{
"schemaVersion": "core-hub.migration.v1",
"source": "core-hub-test-fixture",
"sourceRevision": "fixture-revision",
"exportedAt": "2026-08-21T12:00:00Z",
"highWaterMark": "2026-08-21T11:00:00Z",
"records": {
"hubs": [{"id": "11111111-1111-4111-8111-111111111111", "slug": "ops-hub", "name": "Ops Hub", "createdAt": "2026-08-20T10:00:00Z", "updatedAt": "2026-08-20T11:00:00Z"}],
"hubCapabilityManifests": [{"id": "22222222-2222-4222-8222-222222222222", "hubId": "11111111-1111-4111-8111-111111111111", "manifestVersion": "1.0", "status": "active"}],
"apiConsumers": [{"id": "33333333-3333-4333-8333-333333333333", "slug": "ops-hub", "name": "ops-hub", "hubCapabilityManifestId": "22222222-2222-4222-8222-222222222222"}],
"apiKeys": [{"id": "44444444-4444-4444-8444-444444444444", "apiConsumerId": "33333333-3333-4333-8333-333333333333", "keyPrefix": "ch_fixture", "keyHash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "scopes": "framework:read", "status": "active"}],
"widgets": [{"id": "55555555-5555-4555-8555-555555555555", "hubId": "11111111-1111-4111-8111-111111111111", "name": "Readiness", "widgetType": "ops-readiness-gate"}],
"interactionEvents": [{"id": "66666666-6666-4666-8666-666666666666", "widgetId": "55555555-5555-4555-8555-555555555555", "eventType": "ops-endpoint-verified", "metadata": {"expectedStatus": 401}, "createdAt": "2026-08-21T11:00:00Z"}],
"migrationRuns": [{"id": "77777777-7777-4777-8777-777777777777", "source": "inter-hub", "schemaVersion": "core-hub.migration.v1", "bundleSha256": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "dryRun": false, "status": "imported", "counts": {}, "diagnostics": {}, "createdAt": "2026-08-19T09:00:00Z"}]
}
}

126
tests/test_compatibility.py Normal file
View file

@ -0,0 +1,126 @@
from __future__ import annotations
import asyncio
from fastapi.testclient import TestClient
from sqlalchemy.ext.asyncio import create_async_engine
from hub_core.runtime.app import create_app
from hub_core.runtime.config import RuntimeSettings
from hub_core.runtime.postgres_store import PostgresPortStore
from hub_core.runtime.tables import runtime_metadata
GROUPS = frozenset({"system", "registry", "credentials", "interaction", "deferred", "operator"})
HEADERS = {"Authorization": "Bearer operator-token"}
def compatibility_client(tmp_path, *, write_groups: frozenset[str] = GROUPS) -> tuple[TestClient, PostgresPortStore]:
database_url = f"sqlite+aiosqlite:///{tmp_path / 'compat.db'}"
async def create_schema() -> None:
engine = create_async_engine(database_url)
async with engine.begin() as connection:
await connection.run_sync(runtime_metadata.create_all)
await engine.dispose()
asyncio.run(create_schema())
store = PostgresPortStore.from_url(database_url)
settings = RuntimeSettings(
environment="production",
backend="postgresql",
allow_ephemeral=False,
database_url=database_url,
api_token="operator-token",
v2_groups=GROUPS,
v2_write_groups=write_groups,
legacy_health=True,
)
return TestClient(create_app(settings=settings, port_store=store)), store
def test_route_group_configuration_rejects_two_writers() -> None:
try:
RuntimeSettings(
v2_groups=frozenset({"registry"}),
v2_write_groups=frozenset({"registry"}),
legacy_write_groups=frozenset({"registry"}),
)
except ValueError as exc:
assert "write groups overlap" in str(exc)
else:
raise AssertionError("overlapping writers were accepted")
def test_system_compatibility_and_protected_auth(tmp_path) -> None:
client, store = compatibility_client(tmp_path)
assert client.get("/healthz").json()["service"] == "core-hub"
assert client.get("/api/v2/widget-types").status_code == 200
denied = client.get("/api/v2/hubs")
assert denied.status_code == 401
assert denied.json()["detail"]["code"] == "unauthorized"
document = client.get("/api/v2/openapi.json").json()
assert "/api/v2/hubs" in document["paths"]
assert "/hubs" in document["paths"]
asyncio.run(store.aclose())
def test_ops_hub_bootstrap_and_dynamic_key(tmp_path) -> None:
client, store = compatibility_client(tmp_path)
hub = client.post(
"/api/v2/hubs",
headers=HEADERS,
json={"slug": "ops-hub", "name": "Ops Hub", "domain": "ops.coulomb.social"},
).json()
manifest = client.post(
"/api/v2/hub-capability-manifests",
headers=HEADERS,
json={"hubId": hub["id"], "manifestVersion": "1.0"},
).json()
activated = client.post(
f"/api/v2/hub-capability-manifests/{manifest['id']}/activate", headers=HEADERS
)
assert activated.json()["status"] == "active"
consumer = client.post(
"/api/v2/api-consumers",
headers=HEADERS,
json={"name": "ops-hub", "hubCapabilityManifestId": manifest["id"]},
).json()
issued = client.post(
f"/api/v2/api-consumers/{consumer['id']}/api-keys",
headers=HEADERS,
json={"scopes": "framework:read hub:ops-hub:write"},
).json()
runtime_headers = {"Authorization": f"Bearer {issued['fullKey']}"}
widget = client.post(
"/api/v2/widgets",
headers=runtime_headers,
json={"hubId": hub["id"], "name": "Readiness", "widgetType": "ops-readiness-gate"},
).json()
event = client.post(
"/api/v2/interaction-events",
headers=runtime_headers,
json={
"widgetId": widget["id"],
"eventType": "ops-endpoint-verified",
"metadata": {"expectedStatus": 401},
},
)
assert event.status_code == 201
assert client.get("/api/v2/hub-registry", headers=runtime_headers).status_code == 200
assert client.get("/api/v2/interaction-events", headers=runtime_headers).json()["count"] == 1
assert "fullKey" not in str(client.get("/api/v2/api-consumers", headers=HEADERS).json())
asyncio.run(store.aclose())
def test_read_only_group_rejects_write(tmp_path) -> None:
client, store = compatibility_client(tmp_path, write_groups=frozenset())
response = client.post(
"/api/v2/hubs", headers=HEADERS, json={"slug": "ops-hub", "name": "Ops Hub"}
)
assert response.status_code == 503
assert response.json()["detail"] == "compatibility group is read-only"
asyncio.run(store.aclose())

97
tests/test_migration.py Normal file
View file

@ -0,0 +1,97 @@
import json
import asyncio
from sqlalchemy.ext.asyncio import create_async_engine
from hub_core.runtime.migration import (
bundle_digest,
export_bundle,
import_bundle,
validate_bundle,
)
from hub_core.runtime.postgres_store import PostgresPortStore
from hub_core.runtime.tables import runtime_metadata
def bundle() -> dict:
value = {
"schemaVersion": "core-hub.migration.v1",
"source": "core-hub",
"sourceRevision": "abc123",
"exportedAt": "2026-08-21T12:00:00Z",
"highWaterMark": "2026-08-21T11:00:00Z",
"records": {
"hubs": [{"id": "hub-1", "slug": "ops-hub", "name": "Ops Hub", "createdAt": "2026-08-20T10:00:00Z", "updatedAt": "2026-08-20T11:00:00Z"}],
"hubCapabilityManifests": [{"id": "manifest-1", "hubId": "hub-1", "manifestVersion": "1.0", "status": "active"}],
"apiConsumers": [{"id": "consumer-1", "slug": "ops-hub", "name": "ops-hub", "hubCapabilityManifestId": "manifest-1"}],
"apiKeys": [{"id": "key-1", "apiConsumerId": "consumer-1", "keyPrefix": "ch_prefix", "keyHash": "a" * 64, "scopes": "framework:read"}],
"widgets": [{"id": "widget-1", "hubId": "hub-1", "name": "Readiness", "widgetType": "ops-readiness-gate"}],
"interactionEvents": [{"id": "event-1", "widgetId": "widget-1", "eventType": "ops-endpoint-verified", "metadata": {"status": 401}, "createdAt": "2026-08-21T11:00:00Z"}],
"migrationRuns": [{"id": "run-1", "source": "inter-hub", "schemaVersion": "core-hub.migration.v1", "bundleSha256": "b" * 64, "dryRun": False, "status": "imported", "counts": {}, "diagnostics": {}, "createdAt": "2026-08-19T09:00:00Z"}],
},
}
value["bundleSha256"] = bundle_digest(value)
return value
async def make_store(tmp_path) -> PostgresPortStore:
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'migration.db'}")
async with engine.begin() as connection:
await connection.run_sync(runtime_metadata.create_all)
return PostgresPortStore(engine)
def test_validation_rejects_secret_material_and_digest_drift():
value = bundle()
value["records"]["apiKeys"][0]["fullKey"] = "must-not-move"
report = validate_bundle(value)
assert report["ok"] is False
assert any("secret-shaped" in error for error in report["errors"])
assert any("bundleSha256" in error for error in report["errors"])
def test_import_is_idempotent_and_exports_all_seven_collections(tmp_path):
async def run():
store = await make_store(tmp_path)
try:
return (
await import_bundle(store, bundle()),
await import_bundle(store, bundle()),
await export_bundle(store, source_revision="target-revision"),
)
finally:
await store.aclose()
first, second, exported = asyncio.run(run())
assert first["ok"] is True
assert first["counts"]["migrationRuns"]["created"] == 1
assert second["idempotent"] is True
assert second["migrationRunId"] == first["migrationRunId"]
assert set(exported["records"]) == {
"hubs", "hubCapabilityManifests", "apiConsumers", "apiKeys",
"widgets", "interactionEvents", "migrationRuns",
}
assert exported["records"]["apiKeys"][0]["keyHash"] == "a" * 64
assert "fullKey" not in json.dumps(exported)
assert validate_bundle(exported)["ok"] is True
def test_dry_run_does_not_record_import(tmp_path):
async def run():
store = await make_store(tmp_path)
try:
return (
await import_bundle(store, bundle(), dry_run=True),
await import_bundle(store, bundle()),
)
finally:
await store.aclose()
report, followup = asyncio.run(run())
assert report["dryRun"] is True
assert "migrationRunId" not in report
assert followup["idempotent"] is False

View file

@ -0,0 +1,100 @@
from __future__ import annotations
import asyncio
import json
from datetime import datetime, timezone
from uuid import uuid4
from fastapi.testclient import TestClient
from sqlalchemy.ext.asyncio import create_async_engine
from hub_core.contracts import extension_contract_root
from hub_core.runtime.app import create_app
from hub_core.runtime.config import RuntimeSettings
from hub_core.runtime.postgres_store import PostgresPortStore
from hub_core.runtime.tables import runtime_audit_ledger, runtime_metadata
def test_durable_store_survives_reopen_and_keeps_event_families_separate(tmp_path) -> None:
database_url = f"sqlite+aiosqlite:///{tmp_path / 'runtime.db'}"
async def create_schema() -> None:
engine = create_async_engine(database_url)
async with engine.begin() as connection:
await connection.run_sync(runtime_metadata.create_all)
await engine.dispose()
asyncio.run(create_schema())
package = json.loads(
extension_contract_root()
.joinpath("fixtures", "ops-hub.extension.json")
.read_text(encoding="utf-8")
)
settings = RuntimeSettings(
environment="production",
backend="postgresql",
allow_ephemeral=False,
database_url=database_url,
)
first_store = PostgresPortStore.from_url(database_url)
first = TestClient(create_app(settings=settings, port_store=first_store))
correlation_id = str(uuid4())
assert first.get("/readyz").status_code == 200
assert first.post(
"/ports/registry/registrations",
headers={"X-Correlation-ID": correlation_id},
json=package,
).status_code == 202
assert first.post(
"/ports/events/progress", json=_event("hub.progress.recorded")
).status_code == 202
assert first.post(
"/ports/events/interaction", json=_event("hub.interaction.recorded")
).status_code == 202
asyncio.run(first_store.aclose())
second_store = PostgresPortStore.from_url(database_url)
second = TestClient(create_app(settings=settings, port_store=second_store))
registry = second.get("/ports/projections/hub_registry").json()
progress = second.get("/ports/projections/progress_events").json()
interaction = second.get("/ports/projections/interaction_events").json()
assert registry["data"]["items"][0]["descriptor"]["hub_slug"] == "ops-hub"
assert {item["family"] for item in progress["data"]["items"]} == {"progress"}
assert {item["family"] for item in interaction["data"]["items"]} == {"interaction"}
assert progress["provenance"]["source_system"] == "hub-core-postgresql"
async def audit_count() -> int:
async with second_store.sessions() as session:
return len((await session.execute(runtime_audit_ledger.select())).all())
assert asyncio.run(audit_count()) == 3
asyncio.run(second_store.aclose())
def test_postgresql_readiness_fails_when_database_is_unavailable() -> None:
database_url = "sqlite+aiosqlite:////definitely-missing-parent/runtime.db"
store = PostgresPortStore.from_url(database_url)
settings = RuntimeSettings(
environment="production",
backend="postgresql",
allow_ephemeral=False,
database_url=database_url,
)
response = TestClient(create_app(settings=settings, port_store=store)).get("/readyz")
assert response.status_code == 503
assert response.json()["checks"]["database"] == "unavailable"
asyncio.run(store.aclose())
def _event(event_type: str) -> dict:
return {
"schema_version": "0.1.0",
"correlation_id": str(uuid4()),
"event_type": event_type,
"occurred_at": datetime.now(timezone.utc).isoformat(),
"subject_refs": {"hub": "ops-hub"},
"payload": {"result": "ok"},
}

15
uv.lock generated
View file

@ -19,6 +19,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/67/cd/0d76dfc5de72bde52f55f53e925c7d152d9c7906634ec1e0cbc7e8d4ad93/aiofile-3.11.1-py3-none-any.whl", hash = "sha256:ce77d14ac07f77bc2b757834a5c129321f3f705c474593deed5ab209079a52c9", size = 20446 }, { url = "https://files.pythonhosted.org/packages/67/cd/0d76dfc5de72bde52f55f53e925c7d152d9c7906634ec1e0cbc7e8d4ad93/aiofile-3.11.1-py3-none-any.whl", hash = "sha256:ce77d14ac07f77bc2b757834a5c129321f3f705c474593deed5ab209079a52c9", size = 20446 },
] ]
[[package]]
name = "aiosqlite"
version = "0.22.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/4e/8a/64761f4005f17809769d23e518d915db74e6310474e733e3593cfc854ef1/aiosqlite-0.22.1.tar.gz", hash = "sha256:043e0bd78d32888c0a9ca90fc788b38796843360c855a7262a532813133a0650", size = 14821 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/00/b7/e3bf5133d697a08128598c8d0abc5e16377b51465a33756de24fa7dee953/aiosqlite-0.22.1-py3-none-any.whl", hash = "sha256:21c002eb13823fad740196c5a2e9d8e62f6243bd9e7e4a1f87fb5e44ecb4fceb", size = 17405 },
]
[[package]] [[package]]
name = "alembic" name = "alembic"
version = "1.19.1" version = "1.19.1"
@ -648,6 +657,7 @@ runtime = [
[package.dev-dependencies] [package.dev-dependencies]
dev = [ dev = [
{ name = "aiosqlite" },
{ name = "pytest" }, { name = "pytest" },
] ]
@ -666,7 +676,10 @@ requires-dist = [
] ]
[package.metadata.requires-dev] [package.metadata.requires-dev]
dev = [{ name = "pytest", specifier = ">=8.0.0" }] dev = [
{ name = "aiosqlite", specifier = ">=0.20.0" },
{ name = "pytest", specifier = ">=8.0.0" },
]
[[package]] [[package]]
name = "idna" name = "idna"

View file

@ -0,0 +1,108 @@
---
id: HUB-WP-0005
type: workplan
title: "Core Hub absorption execution"
domain: infotech
repo: hub-core
status: active
owner: codex
topic_slug: custodian
created: "2026-08-21"
updated: "2026-08-21"
parent_project: prj-state-hub-retirement
parent_workplan: CORE-WP-0010
related:
- HUB-WP-0004
- CORE-WP-0010
- RAPPCOREHUB-WP-0002
---
# Core Hub absorption execution
## Goal
Implement the accepted S0S6 plan in `docs/core-hub-absorption-plan.md`, move
Core Hub compatibility and durable framework authority into hub-core without
concurrent production writers, and retain a tested rollback until the final
stabilization gate closes.
## S0 durable runtime and audit boundary
```task
id: HUB-WP-0005-T01
status: done
priority: high
```
Add hub-core-owned PostgreSQL models/migrations, a durable `PortStore`, audit
ledger, dependency-aware readiness, and production-safe lifecycle. The memory
backend remains local/conformance only.
## S0 compatibility shell and migration tooling
```task
id: HUB-WP-0005-T02
status: done
priority: high
```
Add legacy bearer authentication, non-overlapping route/write group controls,
versioned import/export/delta bundles, idempotent imports, canonical count/hash
comparison, and safe evidence. Never serialize a raw key or secret.
## S1 system and discovery compatibility
```task
id: HUB-WP-0005-T03
status: done
priority: high
```
Implement the health/readiness, catalog, OpenAPI/docs, and compatibility alias
surface with normalized response comparison against Core Hub.
## S2 and S3 registry and credential compatibility
```task
id: HUB-WP-0005-T04
status: done
priority: high
```
Absorb hubs, manifests/activation, hub registry, API consumers, key issuance,
and token compatibility with stable identities, audit, fail-closed auth, and
reverse-delta support.
## S4 and S5 interaction and operator compatibility
```task
id: HUB-WP-0005-T05
status: done
priority: high
```
Absorb widgets and interaction events through the named interaction port,
preserve explicitly empty deferred collections, and rebuild the operator
surface without reads from the legacy schema.
## S6 production cutover and retirement handoff
```task
id: HUB-WP-0005-T06
status: wait
priority: high
```
Publish immutable artifacts, pass all conformance and consumer gates, support
the operator-approved production writer cutover, exercise rollback, and hand
the seven-day stabilization/archive residual to CORE-WP-0010. Wait until
RAPPCOREHUB-WP-0002 provides the private candidate and route controls.
## Acceptance
- [x] Production readiness requires durable PostgreSQL and auth dependencies
- [x] Import is idempotent; counts, identities, hashes, and provenance compare
- [x] All 22 checked-in Core Hub OpenAPI paths have governed dispositions
- [ ] ops-hub and activity-core gates pass on hub-core
- [ ] Production runs the immutable hub-core image with one writer per group
- [ ] Rollback is exercised and stabilization residual is recorded