feat: add durable Core Hub absorption runtime
This commit is contained in:
parent
7e1ec03f0c
commit
8ab1d0c09a
20 changed files with 2423 additions and 15 deletions
10
README.md
10
README.md
|
|
@ -52,11 +52,15 @@ uv sync --extra runtime
|
||||||
hub-core api
|
hub-core api
|
||||||
hub-core mcp --api-base http://127.0.0.1:8010
|
hub-core mcp --api-base http://127.0.0.1:8010
|
||||||
hub-core migrate head --database-url postgresql+asyncpg://...
|
hub-core migrate head --database-url postgresql+asyncpg://...
|
||||||
|
hub-core migration validate core-hub-export.json
|
||||||
|
hub-core migration import core-hub-export.json --database-url postgresql+asyncpg://...
|
||||||
```
|
```
|
||||||
|
|
||||||
The initial runtime exposes registry, messaging, progress-event,
|
The runtime exposes registry, messaging, progress-event, interaction-event,
|
||||||
interaction-event, and projection-query ports. Its included memory backend is
|
and projection-query ports, plus the governed Core Hub `/api/v2`
|
||||||
for local/conformance use and fails production readiness unless explicitly
|
compatibility surface. Its PostgreSQL backend includes migrations, audit, and
|
||||||
|
idempotent seven-table migration tooling. The memory backend remains for
|
||||||
|
local/conformance use and fails production readiness unless explicitly
|
||||||
enabled. See `docs/runtime.md`.
|
enabled. See `docs/runtime.md`.
|
||||||
|
|
||||||
The reusable Tier 2/3 scaffold is documented in `docs/conformance.md` and runs
|
The reusable Tier 2/3 scaffold is documented in `docs/conformance.md` and runs
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,7 @@
|
||||||
| workplan | HUB-WP-0002 | finished | — | workplans/HUB-WP-0002-import-refactor-adapter-seams.md |
|
| workplan | HUB-WP-0002 | finished | — | workplans/HUB-WP-0002-import-refactor-adapter-seams.md |
|
||||||
| workplan | HUB-WP-0003 | finished | — | workplans/HUB-WP-0003-ecosystem-consolidation-library-lane.md |
|
| workplan | HUB-WP-0003 | finished | — | workplans/HUB-WP-0003-ecosystem-consolidation-library-lane.md |
|
||||||
| workplan | HUB-WP-0004 | finished | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
| workplan | HUB-WP-0004 | finished | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
||||||
|
| workplan | HUB-WP-0005 | active | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
|
||||||
| task | HUB-WP-0001-T01 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
|
| task | HUB-WP-0001-T01 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
|
||||||
| task | HUB-WP-0001-T02 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
|
| task | HUB-WP-0001-T02 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
|
||||||
| task | HUB-WP-0001-T03 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
|
| task | HUB-WP-0001-T03 | done | — | workplans/HUB-WP-0001-statehub-bootstrap.md |
|
||||||
|
|
@ -30,3 +31,9 @@
|
||||||
| task | HUB-WP-0004-T04 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
| task | HUB-WP-0004-T04 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
||||||
| task | HUB-WP-0004-T05 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
| task | HUB-WP-0004-T05 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
||||||
| task | HUB-WP-0004-T06 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
| task | HUB-WP-0004-T06 | done | — | workplans/HUB-WP-0004-runtime-and-extension-contract.md |
|
||||||
|
| task | HUB-WP-0005-T01 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
|
||||||
|
| task | HUB-WP-0005-T02 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
|
||||||
|
| task | HUB-WP-0005-T03 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
|
||||||
|
| task | HUB-WP-0005-T04 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
|
||||||
|
| task | HUB-WP-0005-T05 | done | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
|
||||||
|
| task | HUB-WP-0005-T06 | wait | — | workplans/HUB-WP-0005-core-hub-absorption-execution.md |
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,9 @@
|
||||||
# Hub Core runtime
|
# Hub Core runtime
|
||||||
|
|
||||||
The initial runtime scaffold implements the packaging decision in ADR-0001 and
|
The runtime implements the packaging decision in ADR-0001, the minimal
|
||||||
the minimal vertical in `HUB-WP-0004-T04`. It is a conformance and absorption
|
vertical in `HUB-WP-0004-T04`, and the durable Core Hub absorption boundary in
|
||||||
base, not yet the production replacement for Core Hub.
|
`HUB-WP-0005`. Public authority still moves only through the gated deployment
|
||||||
|
slices in the accepted absorption plan.
|
||||||
|
|
||||||
## Processes
|
## Processes
|
||||||
|
|
||||||
|
|
@ -14,6 +15,9 @@ uv sync --extra runtime
|
||||||
hub-core api --host 127.0.0.1 --port 8010
|
hub-core api --host 127.0.0.1 --port 8010
|
||||||
hub-core mcp --host 127.0.0.1 --port 8011 --api-base http://127.0.0.1:8010
|
hub-core mcp --host 127.0.0.1 --port 8011 --api-base http://127.0.0.1:8010
|
||||||
hub-core migrate head --database-url postgresql+asyncpg://...
|
hub-core migrate head --database-url postgresql+asyncpg://...
|
||||||
|
hub-core migration validate core-hub-export.json
|
||||||
|
hub-core migration import core-hub-export.json --database-url postgresql+asyncpg://...
|
||||||
|
hub-core migration export --database-url postgresql+asyncpg://... --output reverse-delta.json
|
||||||
```
|
```
|
||||||
|
|
||||||
The migration command converts the async PostgreSQL URL for the packaged
|
The migration command converts the async PostgreSQL URL for the packaged
|
||||||
|
|
@ -36,9 +40,10 @@ submitted through each other's endpoint.
|
||||||
|
|
||||||
## Backend boundary and readiness
|
## Backend boundary and readiness
|
||||||
|
|
||||||
The app is created with an injected `PortStore`. T04 ships
|
The app is created with an injected `PortStore`. `InMemoryPortStore` remains
|
||||||
`InMemoryPortStore` for deterministic tests and local contract smokes. It is
|
available for deterministic tests and local contract smokes. Production uses
|
||||||
ephemeral and is not a production authority.
|
`PostgresPortStore`, whose registration, messaging, progress, interaction,
|
||||||
|
compatibility, import-lineage, and audit records are transactionally durable.
|
||||||
|
|
||||||
`GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503
|
`GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503
|
||||||
when the active backend does not match `HUB_CORE_BACKEND`, or when the memory
|
when the active backend does not match `HUB_CORE_BACKEND`, or when the memory
|
||||||
|
|
@ -51,10 +56,24 @@ HUB_CORE_BACKEND=memory
|
||||||
HUB_CORE_ALLOW_EPHEMERAL=0
|
HUB_CORE_ALLOW_EPHEMERAL=0
|
||||||
```
|
```
|
||||||
|
|
||||||
Therefore the image is deliberately not production-ready until a durable
|
Set `HUB_CORE_BACKEND=postgresql` and `HUB_CORE_DATABASE_URL` in production.
|
||||||
backend lands in an absorption slice. A local image smoke may opt in with
|
Readiness checks the database and continues to fail closed if the selected
|
||||||
|
backend is unavailable. A local image smoke may opt in to memory with
|
||||||
`HUB_CORE_ALLOW_EPHEMERAL=1`.
|
`HUB_CORE_ALLOW_EPHEMERAL=1`.
|
||||||
|
|
||||||
|
The compatibility adapter is enabled by coherent route groups in
|
||||||
|
`HUB_CORE_V2_GROUPS`; writes require the corresponding value in
|
||||||
|
`HUB_CORE_V2_WRITE_GROUPS`. Startup rejects overlap with
|
||||||
|
`CORE_HUB_V2_WRITE_GROUPS`. Supported groups are `system`, `registry`,
|
||||||
|
`credentials`, `interaction`, and `deferred`. Static operator authentication
|
||||||
|
uses `HUB_CORE_API_TOKEN`; imported consumer keys are verified by hash.
|
||||||
|
|
||||||
|
Migration bundles cover all seven Core Hub source tables. They preserve stable
|
||||||
|
IDs and timestamps, reject secret-shaped material and digest drift, record
|
||||||
|
counts/identity sets/canonical hashes, and return the original import-run ID
|
||||||
|
when the same bundle is replayed. Reverse export contains prefixes and hashes,
|
||||||
|
never raw keys.
|
||||||
|
|
||||||
## OCI image
|
## OCI image
|
||||||
|
|
||||||
The `Containerfile` uses `uv.lock` with `uv sync --frozen`, installs the runtime
|
The `Containerfile` uses `uv.lock` with `uv sync --frozen`, installs the runtime
|
||||||
|
|
|
||||||
203
hub_core/migrations/versions/0002_runtime_ports.py
Normal file
203
hub_core/migrations/versions/0002_runtime_ports.py
Normal file
|
|
@ -0,0 +1,203 @@
|
||||||
|
"""hub-core durable runtime ports
|
||||||
|
|
||||||
|
Revision ID: 0002_runtime_ports
|
||||||
|
Revises: 0001_core_schema
|
||||||
|
Create Date: 2026-08-21
|
||||||
|
"""
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from sqlalchemy.dialects import postgresql
|
||||||
|
|
||||||
|
revision: str = "0002_runtime_ports"
|
||||||
|
down_revision: Union[str, None] = "0001_core_schema"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"runtime_registrations",
|
||||||
|
sa.Column("hub_slug", sa.String(120), primary_key=True),
|
||||||
|
sa.Column("package", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_table(
|
||||||
|
"runtime_messages",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("schema_version", sa.String(40), nullable=False),
|
||||||
|
sa.Column("correlation_id", sa.String(36), nullable=False),
|
||||||
|
sa.Column("conversation_id", sa.String(36), nullable=True),
|
||||||
|
sa.Column("from_address", sa.String(240), nullable=False),
|
||||||
|
sa.Column("to_addresses", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("body", sa.Text(), nullable=False),
|
||||||
|
sa.Column("subject_refs", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_runtime_messages_correlation_id", "runtime_messages", ["correlation_id"])
|
||||||
|
op.create_index("ix_runtime_messages_conversation_id", "runtime_messages", ["conversation_id"])
|
||||||
|
op.create_index("ix_runtime_messages_created_at", "runtime_messages", ["created_at"])
|
||||||
|
|
||||||
|
for name in ("runtime_progress_events", "runtime_interaction_events"):
|
||||||
|
op.create_table(
|
||||||
|
name,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("schema_version", sa.String(40), nullable=False),
|
||||||
|
sa.Column("correlation_id", sa.String(36), nullable=False),
|
||||||
|
sa.Column("event_type", sa.String(160), nullable=False),
|
||||||
|
sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("subject_refs", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("payload", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index(f"ix_{name}_correlation_id", name, ["correlation_id"])
|
||||||
|
op.create_index(f"ix_{name}_event_type", name, ["event_type"])
|
||||||
|
op.create_index(f"ix_{name}_recorded_at", name, ["recorded_at"])
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"runtime_audit_ledger",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("action", sa.String(120), nullable=False),
|
||||||
|
sa.Column("subject_type", sa.String(80), nullable=False),
|
||||||
|
sa.Column("subject_id", sa.String(240), nullable=False),
|
||||||
|
sa.Column("correlation_id", sa.String(36), nullable=True),
|
||||||
|
sa.Column("payload_hash", sa.String(64), nullable=False),
|
||||||
|
sa.Column("detail", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_runtime_audit_ledger_action", "runtime_audit_ledger", ["action"])
|
||||||
|
op.create_index("ix_runtime_audit_ledger_subject_id", "runtime_audit_ledger", ["subject_id"])
|
||||||
|
op.create_index(
|
||||||
|
"ix_runtime_audit_ledger_correlation_id", "runtime_audit_ledger", ["correlation_id"]
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_runtime_audit_ledger_recorded_at", "runtime_audit_ledger", ["recorded_at"]
|
||||||
|
)
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"compat_hubs",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("slug", sa.String(120), nullable=False, unique=True),
|
||||||
|
sa.Column("name", sa.String(240), nullable=False),
|
||||||
|
sa.Column("domain", sa.String(240), nullable=True),
|
||||||
|
sa.Column("hub_kind", sa.String(80), nullable=True),
|
||||||
|
sa.Column("hub_family", sa.String(80), nullable=True),
|
||||||
|
sa.Column("vsm_function", sa.String(80), nullable=True),
|
||||||
|
sa.Column("vsm_system", sa.String(80), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("description", sa.Text(), nullable=True),
|
||||||
|
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_compat_hubs_slug", "compat_hubs", ["slug"])
|
||||||
|
op.create_table(
|
||||||
|
"compat_manifests",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=True),
|
||||||
|
sa.Column("hub_slug", sa.String(120), nullable=True),
|
||||||
|
sa.Column("manifest_version", sa.String(40), nullable=False),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_compat_manifests_hub_id", "compat_manifests", ["hub_id"])
|
||||||
|
op.create_index("ix_compat_manifests_hub_slug", "compat_manifests", ["hub_slug"])
|
||||||
|
op.create_table(
|
||||||
|
"compat_api_consumers",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("slug", sa.String(120), nullable=True, unique=True),
|
||||||
|
sa.Column("name", sa.String(240), nullable=False),
|
||||||
|
sa.Column("description", sa.Text(), nullable=True),
|
||||||
|
sa.Column("hub_capability_manifest_id", sa.String(36), nullable=True),
|
||||||
|
sa.Column("rate_limit_per_minute", sa.Integer(), nullable=True),
|
||||||
|
sa.Column("quota_per_day", sa.Integer(), nullable=True),
|
||||||
|
sa.Column("key_prefix", sa.String(32), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_compat_api_consumers_slug", "compat_api_consumers", ["slug"])
|
||||||
|
op.create_index("ix_compat_api_consumers_name", "compat_api_consumers", ["name"])
|
||||||
|
op.create_index(
|
||||||
|
"ix_compat_api_consumers_manifest", "compat_api_consumers", ["hub_capability_manifest_id"]
|
||||||
|
)
|
||||||
|
op.create_table(
|
||||||
|
"compat_api_keys",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column(
|
||||||
|
"api_consumer_id",
|
||||||
|
sa.String(36),
|
||||||
|
sa.ForeignKey("compat_api_consumers.id"),
|
||||||
|
nullable=False,
|
||||||
|
),
|
||||||
|
sa.Column("key_prefix", sa.String(32), nullable=False),
|
||||||
|
sa.Column("key_hash", sa.String(64), nullable=False, unique=True),
|
||||||
|
sa.Column("scopes", sa.Text(), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_compat_api_keys_consumer", "compat_api_keys", ["api_consumer_id"])
|
||||||
|
op.create_index("ix_compat_api_keys_prefix", "compat_api_keys", ["key_prefix"])
|
||||||
|
op.create_table(
|
||||||
|
"compat_widgets",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=False),
|
||||||
|
sa.Column("name", sa.String(240), nullable=False),
|
||||||
|
sa.Column("widget_type", sa.String(120), nullable=True),
|
||||||
|
sa.Column("capability_ref", sa.String(240), nullable=True),
|
||||||
|
sa.Column("view_context", sa.String(500), nullable=True),
|
||||||
|
sa.Column("policy_scope", sa.String(120), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("body", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_compat_widgets_hub_id", "compat_widgets", ["hub_id"])
|
||||||
|
op.create_index("ix_compat_widgets_type", "compat_widgets", ["widget_type"])
|
||||||
|
op.create_index("ix_compat_widgets_capability", "compat_widgets", ["capability_ref"])
|
||||||
|
op.create_table(
|
||||||
|
"compat_migration_runs",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("source", sa.String(120), nullable=False),
|
||||||
|
sa.Column("schema_version", sa.String(80), nullable=False),
|
||||||
|
sa.Column("bundle_sha256", sa.String(64), nullable=False),
|
||||||
|
sa.Column("dry_run", sa.Boolean(), nullable=False),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("counts", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("diagnostics", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_compat_migration_runs_source", "compat_migration_runs", ["source"])
|
||||||
|
op.create_index("ix_compat_migration_runs_bundle", "compat_migration_runs", ["bundle_sha256"])
|
||||||
|
op.create_table(
|
||||||
|
"runtime_import_runs",
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("source", sa.String(120), nullable=False),
|
||||||
|
sa.Column("schema_version", sa.String(80), nullable=False),
|
||||||
|
sa.Column("bundle_sha256", sa.String(64), nullable=False, unique=True),
|
||||||
|
sa.Column("high_water_mark", sa.String(200), nullable=True),
|
||||||
|
sa.Column("counts", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("content_hashes", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_runtime_import_runs_source", "runtime_import_runs", ["source"])
|
||||||
|
op.create_index("ix_runtime_import_runs_bundle", "runtime_import_runs", ["bundle_sha256"])
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_table("runtime_import_runs")
|
||||||
|
op.drop_table("compat_migration_runs")
|
||||||
|
op.drop_table("compat_widgets")
|
||||||
|
op.drop_table("compat_api_keys")
|
||||||
|
op.drop_table("compat_api_consumers")
|
||||||
|
op.drop_table("compat_manifests")
|
||||||
|
op.drop_table("compat_hubs")
|
||||||
|
op.drop_table("runtime_audit_ledger")
|
||||||
|
op.drop_table("runtime_interaction_events")
|
||||||
|
op.drop_table("runtime_progress_events")
|
||||||
|
op.drop_table("runtime_messages")
|
||||||
|
op.drop_table("runtime_registrations")
|
||||||
|
|
@ -1,9 +1,12 @@
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
from fastapi import FastAPI, Response, status
|
from fastapi import FastAPI, Response, status
|
||||||
|
|
||||||
from hub_core import __version__
|
from hub_core import __version__
|
||||||
from hub_core.runtime.config import RuntimeSettings
|
from hub_core.runtime.config import RuntimeSettings
|
||||||
|
from hub_core.runtime.compat import SQLCompatibilityStore, create_compatibility_router
|
||||||
from hub_core.runtime.models import HealthResponse, ReadinessResponse
|
from hub_core.runtime.models import HealthResponse, ReadinessResponse
|
||||||
from hub_core.runtime.ports import create_ports_router
|
from hub_core.runtime.ports import create_ports_router
|
||||||
from hub_core.runtime.store import InMemoryPortStore, PortStore
|
from hub_core.runtime.store import InMemoryPortStore, PortStore
|
||||||
|
|
@ -17,37 +20,66 @@ def create_app(
|
||||||
) -> FastAPI:
|
) -> FastAPI:
|
||||||
resolved_settings = settings or RuntimeSettings.from_env()
|
resolved_settings = settings or RuntimeSettings.from_env()
|
||||||
resolved_store = port_store or _create_store(resolved_settings)
|
resolved_store = port_store or _create_store(resolved_settings)
|
||||||
|
owns_store = port_store is None
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(_: FastAPI):
|
||||||
|
yield
|
||||||
|
if owns_store and (closer := getattr(resolved_store, "aclose", None)):
|
||||||
|
await closer()
|
||||||
|
|
||||||
app = FastAPI(
|
app = FastAPI(
|
||||||
title="Hub Core Runtime",
|
title="Hub Core Runtime",
|
||||||
version=__version__,
|
version=__version__,
|
||||||
description="HelixForge hub framework and named-port runtime.",
|
description="HelixForge hub framework and named-port runtime.",
|
||||||
|
lifespan=lifespan,
|
||||||
)
|
)
|
||||||
app.state.settings = resolved_settings
|
app.state.settings = resolved_settings
|
||||||
app.state.port_store = resolved_store
|
app.state.port_store = resolved_store
|
||||||
|
app.state.compat_store = (
|
||||||
|
SQLCompatibilityStore(resolved_store)
|
||||||
|
if resolved_store.backend_name == "postgresql"
|
||||||
|
else None
|
||||||
|
)
|
||||||
app.state.contract_validator = ContractValidator()
|
app.state.contract_validator = ContractValidator()
|
||||||
|
|
||||||
@app.get("/healthz", response_model=HealthResponse, tags=["system"])
|
@app.get("/healthz", response_model=HealthResponse, tags=["system"])
|
||||||
async def healthz() -> HealthResponse:
|
async def healthz() -> HealthResponse:
|
||||||
return HealthResponse(version=__version__)
|
return HealthResponse(
|
||||||
|
service="core-hub" if resolved_settings.legacy_health else "hub-core",
|
||||||
|
version=__version__,
|
||||||
|
)
|
||||||
|
|
||||||
@app.get("/readyz", response_model=ReadinessResponse, tags=["system"])
|
@app.get("/readyz", response_model=ReadinessResponse, tags=["system"])
|
||||||
async def readyz(response: Response) -> ReadinessResponse:
|
async def readyz(response: Response) -> ReadinessResponse:
|
||||||
ready = resolved_settings.is_ready(resolved_store.backend_name)
|
dependency_checks = await resolved_store.readiness_checks()
|
||||||
|
ready = resolved_settings.is_ready(resolved_store.backend_name) and all(
|
||||||
|
value in {"ok", "not_applicable"} for value in dependency_checks.values()
|
||||||
|
)
|
||||||
if not ready:
|
if not ready:
|
||||||
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
|
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
|
||||||
return ReadinessResponse(
|
return ReadinessResponse(
|
||||||
status="ok" if ready else "degraded",
|
status="ok" if ready else "degraded",
|
||||||
checks=resolved_settings.readiness_checks(resolved_store.backend_name),
|
checks={
|
||||||
|
**resolved_settings.readiness_checks(resolved_store.backend_name),
|
||||||
|
**dependency_checks,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
app.include_router(create_ports_router())
|
app.include_router(create_ports_router())
|
||||||
|
app.include_router(create_compatibility_router())
|
||||||
return app
|
return app
|
||||||
|
|
||||||
|
|
||||||
def _create_store(settings: RuntimeSettings) -> PortStore:
|
def _create_store(settings: RuntimeSettings) -> PortStore:
|
||||||
if settings.backend == "memory":
|
if settings.backend == "memory":
|
||||||
return InMemoryPortStore()
|
return InMemoryPortStore()
|
||||||
|
if settings.backend == "postgresql":
|
||||||
|
if not settings.database_url:
|
||||||
|
raise RuntimeError("HUB_CORE_DATABASE_URL is required for PostgreSQL backend")
|
||||||
|
from hub_core.runtime.postgres_store import PostgresPortStore
|
||||||
|
|
||||||
|
return PostgresPortStore.from_url(settings.database_url)
|
||||||
raise RuntimeError(f"Unsupported HUB_CORE_BACKEND '{settings.backend}'")
|
raise RuntimeError(f"Unsupported HUB_CORE_BACKEND '{settings.backend}'")
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,7 @@ from __future__ import annotations
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
from importlib.resources import files
|
from importlib.resources import files
|
||||||
|
from pathlib import Path
|
||||||
from typing import Sequence
|
from typing import Sequence
|
||||||
|
|
||||||
from hub_core.mcp import HubCoreMCPServer
|
from hub_core.mcp import HubCoreMCPServer
|
||||||
|
|
@ -28,6 +29,21 @@ def build_parser(settings: RuntimeSettings | None = None) -> argparse.ArgumentPa
|
||||||
migrate.add_argument("revision", nargs="?", default="head")
|
migrate.add_argument("revision", nargs="?", default="head")
|
||||||
migrate.add_argument("--database-url", default=resolved.database_url)
|
migrate.add_argument("--database-url", default=resolved.database_url)
|
||||||
|
|
||||||
|
migration = commands.add_parser("migration", help="Export, validate, or import migration bundles")
|
||||||
|
migration_commands = migration.add_subparsers(dest="migration_command", required=True)
|
||||||
|
migration_validate = migration_commands.add_parser("validate")
|
||||||
|
migration_validate.add_argument("bundle", type=Path)
|
||||||
|
migration_validate.add_argument("--output", type=Path)
|
||||||
|
migration_import = migration_commands.add_parser("import")
|
||||||
|
migration_import.add_argument("bundle", type=Path)
|
||||||
|
migration_import.add_argument("--database-url", default=resolved.database_url)
|
||||||
|
migration_import.add_argument("--dry-run", action="store_true")
|
||||||
|
migration_import.add_argument("--output", type=Path)
|
||||||
|
migration_export = migration_commands.add_parser("export")
|
||||||
|
migration_export.add_argument("--database-url", default=resolved.database_url)
|
||||||
|
migration_export.add_argument("--source-revision")
|
||||||
|
migration_export.add_argument("--output", type=Path, required=True)
|
||||||
|
|
||||||
conformance = commands.add_parser(
|
conformance = commands.add_parser(
|
||||||
"conformance",
|
"conformance",
|
||||||
help="Run the implemented Tier 2/3 profile against an HTTP runtime",
|
help="Run the implemented Tier 2/3 profile against an HTTP runtime",
|
||||||
|
|
@ -53,6 +69,8 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||||
raise SystemExit("hub-core migrate requires --database-url or HUB_CORE_DATABASE_URL")
|
raise SystemExit("hub-core migrate requires --database-url or HUB_CORE_DATABASE_URL")
|
||||||
_run_migrations(args.database_url, args.revision)
|
_run_migrations(args.database_url, args.revision)
|
||||||
return 0
|
return 0
|
||||||
|
if args.command == "migration":
|
||||||
|
return _run_migration(args)
|
||||||
if args.command == "conformance":
|
if args.command == "conformance":
|
||||||
return _run_conformance(args.base_url, args.timeout, args.as_json)
|
return _run_conformance(args.base_url, args.timeout, args.as_json)
|
||||||
raise AssertionError(f"Unhandled command {args.command}")
|
raise AssertionError(f"Unhandled command {args.command}")
|
||||||
|
|
@ -96,5 +114,38 @@ def _run_conformance(base_url: str, timeout: float, as_json: bool) -> int:
|
||||||
return 0 if report.passed else 1
|
return 0 if report.passed else 1
|
||||||
|
|
||||||
|
|
||||||
|
def _run_migration(args: argparse.Namespace) -> int:
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from hub_core.runtime.migration import (
|
||||||
|
export_bundle,
|
||||||
|
import_bundle,
|
||||||
|
load_bundle,
|
||||||
|
validate_bundle,
|
||||||
|
)
|
||||||
|
from hub_core.runtime.postgres_store import PostgresPortStore
|
||||||
|
|
||||||
|
if args.migration_command == "validate":
|
||||||
|
report = validate_bundle(load_bundle(args.bundle))
|
||||||
|
else:
|
||||||
|
if not args.database_url:
|
||||||
|
raise SystemExit("migration command requires --database-url or HUB_CORE_DATABASE_URL")
|
||||||
|
store = PostgresPortStore.from_url(args.database_url)
|
||||||
|
async def run() -> dict:
|
||||||
|
try:
|
||||||
|
if args.migration_command == "import":
|
||||||
|
return await import_bundle(store, load_bundle(args.bundle), dry_run=args.dry_run)
|
||||||
|
return await export_bundle(store, source_revision=args.source_revision)
|
||||||
|
finally:
|
||||||
|
await store.aclose()
|
||||||
|
report = asyncio.run(run())
|
||||||
|
rendered = json.dumps(report, indent=2, sort_keys=True)
|
||||||
|
if args.output:
|
||||||
|
args.output.write_text(rendered + "\n")
|
||||||
|
else:
|
||||||
|
print(rendered)
|
||||||
|
return 0 if report.get("ok", True) else 1
|
||||||
|
|
||||||
|
|
||||||
def _sync_database_url(database_url: str) -> str:
|
def _sync_database_url(database_url: str) -> str:
|
||||||
return database_url.replace("postgresql+asyncpg://", "postgresql+psycopg2://")
|
return database_url.replace("postgresql+asyncpg://", "postgresql+psycopg2://")
|
||||||
|
|
|
||||||
605
hub_core/runtime/compat.py
Normal file
605
hub_core/runtime/compat.py
Normal file
|
|
@ -0,0 +1,605 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
import secrets
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Annotated, Any
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from fastapi import APIRouter, Header, HTTPException, Request, status
|
||||||
|
from fastapi.responses import HTMLResponse, PlainTextResponse, RedirectResponse
|
||||||
|
|
||||||
|
from hub_core.runtime.compat_catalogs import (
|
||||||
|
ANNOTATION_CATEGORIES,
|
||||||
|
EVENT_TYPES,
|
||||||
|
POLICY_SCOPES,
|
||||||
|
WIDGET_TYPES,
|
||||||
|
)
|
||||||
|
from hub_core.runtime.models import EventCommand
|
||||||
|
from hub_core.runtime.postgres_store import PostgresPortStore
|
||||||
|
from hub_core.runtime.tables import (
|
||||||
|
compat_api_consumers,
|
||||||
|
compat_api_keys,
|
||||||
|
compat_hubs,
|
||||||
|
compat_manifests,
|
||||||
|
compat_widgets,
|
||||||
|
runtime_interaction_events,
|
||||||
|
)
|
||||||
|
from hub_core.utils.slugs import slugify_or_default
|
||||||
|
|
||||||
|
AuthorizationHeader = Annotated[str | None, Header()]
|
||||||
|
|
||||||
|
|
||||||
|
class SQLCompatibilityStore:
|
||||||
|
def __init__(self, ports: PostgresPortStore) -> None:
|
||||||
|
self.ports = ports
|
||||||
|
self.sessions = ports.sessions
|
||||||
|
|
||||||
|
async def validate_token(self, token: str) -> bool:
|
||||||
|
token_hash = _hash_token(token)
|
||||||
|
async with self.sessions() as session:
|
||||||
|
result = await session.execute(
|
||||||
|
sa.select(compat_api_keys.c.id).where(
|
||||||
|
compat_api_keys.c.key_hash == token_hash,
|
||||||
|
compat_api_keys.c.status == "active",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return result.scalar_one_or_none() is not None
|
||||||
|
|
||||||
|
async def list_hubs(self) -> list[dict[str, Any]]:
|
||||||
|
async with self.sessions() as session:
|
||||||
|
rows = (
|
||||||
|
await session.execute(sa.select(compat_hubs).order_by(compat_hubs.c.slug))
|
||||||
|
).mappings()
|
||||||
|
return [_hub(row) for row in rows]
|
||||||
|
|
||||||
|
async def create_hub(self, body: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
now = _now()
|
||||||
|
row = {
|
||||||
|
"id": str(uuid4()),
|
||||||
|
"slug": body["slug"],
|
||||||
|
"name": body["name"],
|
||||||
|
"domain": body.get("domain"),
|
||||||
|
"hub_kind": body.get("hubKind"),
|
||||||
|
"hub_family": body.get("hubFamily"),
|
||||||
|
"vsm_function": body.get("vsmFunction"),
|
||||||
|
"vsm_system": body.get("vsmSystem"),
|
||||||
|
"status": body.get("status", "active"),
|
||||||
|
"description": body.get("description"),
|
||||||
|
"body": body,
|
||||||
|
"created_at": now,
|
||||||
|
"updated_at": now,
|
||||||
|
}
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
await session.execute(compat_hubs.insert().values(**row))
|
||||||
|
await self.ports._audit(
|
||||||
|
session,
|
||||||
|
action="compat.hub.created",
|
||||||
|
subject_type="hub",
|
||||||
|
subject_id=row["id"],
|
||||||
|
correlation_id=None,
|
||||||
|
value=body,
|
||||||
|
)
|
||||||
|
return _hub(row)
|
||||||
|
|
||||||
|
async def list_manifests(self, hub_id: str | None = None) -> list[dict[str, Any]]:
|
||||||
|
statement = sa.select(compat_manifests).order_by(compat_manifests.c.created_at)
|
||||||
|
if hub_id:
|
||||||
|
statement = statement.where(compat_manifests.c.hub_id == hub_id)
|
||||||
|
async with self.sessions() as session:
|
||||||
|
rows = (await session.execute(statement)).mappings()
|
||||||
|
return [_manifest(row) for row in rows]
|
||||||
|
|
||||||
|
async def create_manifest(self, body: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
now = _now()
|
||||||
|
hub_id = body.get("hubId")
|
||||||
|
hub_slug = None
|
||||||
|
if hub_id:
|
||||||
|
async with self.sessions() as session:
|
||||||
|
hub_slug = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(compat_hubs.c.slug).where(compat_hubs.c.id == hub_id)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
row = {
|
||||||
|
"id": str(uuid4()),
|
||||||
|
"hub_id": hub_id,
|
||||||
|
"hub_slug": hub_slug,
|
||||||
|
"manifest_version": body.get("manifestVersion", "0.1.0"),
|
||||||
|
"status": body.get("status", "draft"),
|
||||||
|
"body": body,
|
||||||
|
"created_at": now,
|
||||||
|
"updated_at": now,
|
||||||
|
}
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
await session.execute(compat_manifests.insert().values(**row))
|
||||||
|
await self.ports._audit(
|
||||||
|
session,
|
||||||
|
action="compat.manifest.created",
|
||||||
|
subject_type="manifest",
|
||||||
|
subject_id=row["id"],
|
||||||
|
correlation_id=None,
|
||||||
|
value=body,
|
||||||
|
)
|
||||||
|
return _manifest(row)
|
||||||
|
|
||||||
|
async def update_manifest(
|
||||||
|
self, manifest_id: str, body: dict[str, Any], *, activate: bool = False
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
current = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(compat_manifests).where(compat_manifests.c.id == manifest_id)
|
||||||
|
)
|
||||||
|
).mappings().one_or_none()
|
||||||
|
if current is None:
|
||||||
|
return {"id": manifest_id, "status": "missing"}
|
||||||
|
payload = dict(current["body"] or {})
|
||||||
|
payload.update(body)
|
||||||
|
values = {
|
||||||
|
"body": payload,
|
||||||
|
"manifest_version": payload.get(
|
||||||
|
"manifestVersion", current["manifest_version"]
|
||||||
|
),
|
||||||
|
"status": "active" if activate else current["status"],
|
||||||
|
"updated_at": _now(),
|
||||||
|
}
|
||||||
|
await session.execute(
|
||||||
|
compat_manifests.update()
|
||||||
|
.where(compat_manifests.c.id == manifest_id)
|
||||||
|
.values(**values)
|
||||||
|
)
|
||||||
|
await self.ports._audit(
|
||||||
|
session,
|
||||||
|
action="compat.manifest.activated" if activate else "compat.manifest.updated",
|
||||||
|
subject_type="manifest",
|
||||||
|
subject_id=manifest_id,
|
||||||
|
correlation_id=None,
|
||||||
|
value=payload,
|
||||||
|
)
|
||||||
|
return _manifest({**dict(current), **values})
|
||||||
|
|
||||||
|
async def list_consumers(self) -> list[dict[str, Any]]:
|
||||||
|
async with self.sessions() as session:
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(compat_api_consumers).order_by(compat_api_consumers.c.name)
|
||||||
|
)
|
||||||
|
).mappings()
|
||||||
|
return [_consumer(row) for row in rows]
|
||||||
|
|
||||||
|
async def create_consumer(self, body: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
row = {
|
||||||
|
"id": str(uuid4()),
|
||||||
|
"slug": body.get("slug") or slugify_or_default(body["name"]),
|
||||||
|
"name": body["name"],
|
||||||
|
"description": body.get("description"),
|
||||||
|
"hub_capability_manifest_id": body.get("hubCapabilityManifestId"),
|
||||||
|
"rate_limit_per_minute": body.get("rateLimitPerMinute"),
|
||||||
|
"quota_per_day": body.get("quotaPerDay"),
|
||||||
|
"key_prefix": None,
|
||||||
|
"status": body.get("status", "active"),
|
||||||
|
"body": body,
|
||||||
|
"created_at": _now(),
|
||||||
|
}
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
await session.execute(compat_api_consumers.insert().values(**row))
|
||||||
|
await self.ports._audit(
|
||||||
|
session,
|
||||||
|
action="compat.consumer.created",
|
||||||
|
subject_type="api_consumer",
|
||||||
|
subject_id=row["id"],
|
||||||
|
correlation_id=None,
|
||||||
|
value=body,
|
||||||
|
)
|
||||||
|
return _consumer(row)
|
||||||
|
|
||||||
|
async def create_key(self, consumer_id: str, scopes: str | None) -> dict[str, Any]:
|
||||||
|
full_key = f"ch_{secrets.token_urlsafe(32)}"
|
||||||
|
prefix = full_key[:12]
|
||||||
|
row = {
|
||||||
|
"id": str(uuid4()),
|
||||||
|
"api_consumer_id": consumer_id,
|
||||||
|
"key_prefix": prefix,
|
||||||
|
"key_hash": _hash_token(full_key),
|
||||||
|
"scopes": scopes,
|
||||||
|
"status": "active",
|
||||||
|
"created_at": _now(),
|
||||||
|
}
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
await session.execute(compat_api_keys.insert().values(**row))
|
||||||
|
await session.execute(
|
||||||
|
compat_api_consumers.update()
|
||||||
|
.where(compat_api_consumers.c.id == consumer_id)
|
||||||
|
.values(key_prefix=prefix)
|
||||||
|
)
|
||||||
|
await self.ports._audit(
|
||||||
|
session,
|
||||||
|
action="compat.api_key.created",
|
||||||
|
subject_type="api_key",
|
||||||
|
subject_id=row["id"],
|
||||||
|
correlation_id=None,
|
||||||
|
value={"apiConsumerId": consumer_id, "keyPrefix": prefix, "scopes": scopes},
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"fullKey": full_key,
|
||||||
|
"apiKey": {
|
||||||
|
"id": row["id"],
|
||||||
|
"apiConsumerId": consumer_id,
|
||||||
|
"keyPrefix": prefix,
|
||||||
|
"scopes": scopes,
|
||||||
|
"status": "active",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def list_widgets(self) -> list[dict[str, Any]]:
|
||||||
|
async with self.sessions() as session:
|
||||||
|
rows = (
|
||||||
|
await session.execute(sa.select(compat_widgets).order_by(compat_widgets.c.name))
|
||||||
|
).mappings()
|
||||||
|
return [_widget(row) for row in rows]
|
||||||
|
|
||||||
|
async def create_widget(self, body: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
row = {
|
||||||
|
"id": str(uuid4()),
|
||||||
|
"hub_id": body["hubId"],
|
||||||
|
"name": body["name"],
|
||||||
|
"widget_type": body.get("widgetType"),
|
||||||
|
"capability_ref": body.get("capabilityRef"),
|
||||||
|
"view_context": body.get("viewContext"),
|
||||||
|
"policy_scope": body.get("policyScope"),
|
||||||
|
"status": body.get("status", "active"),
|
||||||
|
"body": body,
|
||||||
|
"created_at": _now(),
|
||||||
|
}
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
await session.execute(compat_widgets.insert().values(**row))
|
||||||
|
await self.ports._audit(
|
||||||
|
session,
|
||||||
|
action="compat.widget.created",
|
||||||
|
subject_type="widget",
|
||||||
|
subject_id=row["id"],
|
||||||
|
correlation_id=None,
|
||||||
|
value=body,
|
||||||
|
)
|
||||||
|
return _widget(row)
|
||||||
|
|
||||||
|
async def list_interactions(self) -> list[dict[str, Any]]:
|
||||||
|
async with self.sessions() as session:
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(runtime_interaction_events).order_by(
|
||||||
|
runtime_interaction_events.c.recorded_at,
|
||||||
|
runtime_interaction_events.c.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).mappings()
|
||||||
|
values: list[dict[str, Any]] = []
|
||||||
|
for row in rows:
|
||||||
|
legacy = dict((row["payload"] or {}).get("legacy") or {})
|
||||||
|
if legacy:
|
||||||
|
legacy["id"] = row["id"]
|
||||||
|
values.append(legacy)
|
||||||
|
return values
|
||||||
|
|
||||||
|
async def create_interaction(self, body: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
correlation_id = uuid4()
|
||||||
|
command = EventCommand(
|
||||||
|
schema_version="0.1.0",
|
||||||
|
correlation_id=correlation_id,
|
||||||
|
event_type=body["eventType"],
|
||||||
|
occurred_at=_now(),
|
||||||
|
subject_refs={"widget": body["widgetId"]},
|
||||||
|
payload={"legacy": body},
|
||||||
|
)
|
||||||
|
accepted = await self.ports.append_interaction(command)
|
||||||
|
return {"id": accepted.id, **body}
|
||||||
|
|
||||||
|
|
||||||
|
def create_compatibility_router() -> APIRouter:
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.get("/api/v2/widget-types")
|
||||||
|
async def widget_types(request: Request) -> list[dict[str, str]]:
|
||||||
|
_enabled(request, "system")
|
||||||
|
return WIDGET_TYPES
|
||||||
|
|
||||||
|
@router.get("/api/v2/event-types")
|
||||||
|
async def event_types(request: Request) -> list[dict[str, str]]:
|
||||||
|
_enabled(request, "system")
|
||||||
|
return EVENT_TYPES
|
||||||
|
|
||||||
|
@router.get("/api/v2/annotation-categories")
|
||||||
|
async def annotation_categories(request: Request) -> list[dict[str, str]]:
|
||||||
|
_enabled(request, "system")
|
||||||
|
return ANNOTATION_CATEGORIES
|
||||||
|
|
||||||
|
@router.get("/api/v2/policy-scopes")
|
||||||
|
async def policy_scopes(request: Request) -> list[dict[str, str]]:
|
||||||
|
_enabled(request, "system")
|
||||||
|
return POLICY_SCOPES
|
||||||
|
|
||||||
|
@router.get("/api/v2/hubs")
|
||||||
|
async def list_hubs(request: Request, authorization: AuthorizationHeader = None) -> dict:
|
||||||
|
await _protected(request, "registry", authorization)
|
||||||
|
return _page(await _store(request).list_hubs())
|
||||||
|
|
||||||
|
@router.post("/api/v2/hubs", status_code=status.HTTP_201_CREATED)
|
||||||
|
async def create_hub(
|
||||||
|
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "registry", authorization, write=True)
|
||||||
|
return await _store(request).create_hub(body)
|
||||||
|
|
||||||
|
@router.get("/api/v2/hub-capability-manifests")
|
||||||
|
async def list_manifests(
|
||||||
|
request: Request,
|
||||||
|
authorization: AuthorizationHeader = None,
|
||||||
|
hubId: str | None = None,
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "registry", authorization)
|
||||||
|
return _page(await _store(request).list_manifests(hubId))
|
||||||
|
|
||||||
|
@router.post("/api/v2/hub-capability-manifests", status_code=status.HTTP_201_CREATED)
|
||||||
|
async def create_manifest(
|
||||||
|
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "registry", authorization, write=True)
|
||||||
|
return await _store(request).create_manifest(body)
|
||||||
|
|
||||||
|
@router.patch("/api/v2/hub-capability-manifests/{manifest_id}")
|
||||||
|
async def patch_manifest(
|
||||||
|
manifest_id: str,
|
||||||
|
body: dict[str, Any],
|
||||||
|
request: Request,
|
||||||
|
authorization: AuthorizationHeader = None,
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "registry", authorization, write=True)
|
||||||
|
return await _store(request).update_manifest(manifest_id, body)
|
||||||
|
|
||||||
|
@router.post("/api/v2/hub-capability-manifests/{manifest_id}/activate")
|
||||||
|
async def activate_manifest(
|
||||||
|
manifest_id: str, request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "registry", authorization, write=True)
|
||||||
|
return await _store(request).update_manifest(manifest_id, {}, activate=True)
|
||||||
|
|
||||||
|
@router.get("/api/v2/hub-registry")
|
||||||
|
async def hub_registry(
|
||||||
|
request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "registry", authorization)
|
||||||
|
store = _store(request)
|
||||||
|
return {
|
||||||
|
"data": {
|
||||||
|
"hubs": await store.list_hubs(),
|
||||||
|
"hubCapabilityManifests": await store.list_manifests(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@router.get("/api/v2/api-consumers")
|
||||||
|
async def list_consumers(
|
||||||
|
request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "credentials", authorization)
|
||||||
|
return _page(await _store(request).list_consumers())
|
||||||
|
|
||||||
|
@router.post("/api/v2/api-consumers", status_code=status.HTTP_201_CREATED)
|
||||||
|
async def create_consumer(
|
||||||
|
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "credentials", authorization, write=True)
|
||||||
|
return await _store(request).create_consumer(body)
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/api/v2/api-consumers/{consumer_id}/api-keys",
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
)
|
||||||
|
async def create_key(
|
||||||
|
consumer_id: str,
|
||||||
|
body: dict[str, Any],
|
||||||
|
request: Request,
|
||||||
|
authorization: AuthorizationHeader = None,
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "credentials", authorization, write=True)
|
||||||
|
return await _store(request).create_key(consumer_id, body.get("scopes"))
|
||||||
|
|
||||||
|
@router.post("/api/v2/token")
|
||||||
|
async def token(request: Request, authorization: AuthorizationHeader = None) -> dict:
|
||||||
|
await _protected(request, "credentials", authorization)
|
||||||
|
return {"access_token": "already-authenticated", "token_type": "bearer"}
|
||||||
|
|
||||||
|
@router.get("/api/v2/widgets")
|
||||||
|
async def list_widgets(
|
||||||
|
request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "interaction", authorization)
|
||||||
|
return _page(await _store(request).list_widgets())
|
||||||
|
|
||||||
|
@router.post("/api/v2/widgets", status_code=status.HTTP_201_CREATED)
|
||||||
|
async def create_widget(
|
||||||
|
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "interaction", authorization, write=True)
|
||||||
|
return await _store(request).create_widget(body)
|
||||||
|
|
||||||
|
@router.get("/api/v2/interaction-events")
|
||||||
|
async def list_interactions(
|
||||||
|
request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "interaction", authorization)
|
||||||
|
return _page(await _store(request).list_interactions())
|
||||||
|
|
||||||
|
@router.post("/api/v2/interaction-events", status_code=status.HTTP_201_CREATED)
|
||||||
|
async def create_interaction(
|
||||||
|
body: dict[str, Any], request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "interaction", authorization, write=True)
|
||||||
|
return await _store(request).create_interaction(body)
|
||||||
|
|
||||||
|
for index, path in enumerate((
|
||||||
|
"annotations",
|
||||||
|
"requirement-candidates",
|
||||||
|
"decision-records",
|
||||||
|
"deployment-records",
|
||||||
|
"outcome-signals",
|
||||||
|
)):
|
||||||
|
async def empty_collection(
|
||||||
|
request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> dict:
|
||||||
|
await _protected(request, "deferred", authorization)
|
||||||
|
return {"data": []}
|
||||||
|
|
||||||
|
stem = path.replace("-", "_")
|
||||||
|
router.add_api_route(
|
||||||
|
f"/api/v2/{path}",
|
||||||
|
empty_collection,
|
||||||
|
methods=["GET"],
|
||||||
|
operation_id=f"compat_list_{stem}_{index}",
|
||||||
|
)
|
||||||
|
router.add_api_route(
|
||||||
|
f"/api/v2/{path}",
|
||||||
|
empty_collection,
|
||||||
|
methods=["POST"],
|
||||||
|
status_code=201,
|
||||||
|
operation_id=f"compat_create_{stem}_{index}",
|
||||||
|
)
|
||||||
|
|
||||||
|
@router.get("/api/v2/openapi.json", include_in_schema=False)
|
||||||
|
async def openapi_json(request: Request) -> dict[str, Any]:
|
||||||
|
_enabled(request, "system")
|
||||||
|
document = request.app.openapi()
|
||||||
|
paths = document.setdefault("paths", {})
|
||||||
|
for path, operations in list(paths.items()):
|
||||||
|
if path.startswith("/api/v2/"):
|
||||||
|
paths.setdefault(path.removeprefix("/api/v2"), operations)
|
||||||
|
return document
|
||||||
|
|
||||||
|
@router.get("/api/v2/openapi.yaml", include_in_schema=False)
|
||||||
|
async def openapi_yaml(request: Request) -> PlainTextResponse:
|
||||||
|
document = await openapi_json(request)
|
||||||
|
info = document.get("info", {})
|
||||||
|
return PlainTextResponse(
|
||||||
|
"openapi: 3.1.0\ninfo:\n"
|
||||||
|
f" title: {info.get('title', 'Hub Core Runtime')}\n"
|
||||||
|
f" version: {info.get('version', '0.2.0')}\n",
|
||||||
|
media_type="application/yaml",
|
||||||
|
)
|
||||||
|
|
||||||
|
@router.get("/api/v2/docs", include_in_schema=False)
|
||||||
|
async def docs(request: Request) -> RedirectResponse:
|
||||||
|
_enabled(request, "system")
|
||||||
|
return RedirectResponse(url="/docs")
|
||||||
|
|
||||||
|
@router.get("/console", response_class=HTMLResponse)
|
||||||
|
async def console(
|
||||||
|
request: Request, authorization: AuthorizationHeader = None
|
||||||
|
) -> HTMLResponse:
|
||||||
|
await _protected(request, "operator", authorization)
|
||||||
|
return HTMLResponse(
|
||||||
|
"<!doctype html><title>Hub Core Operator Console</title>"
|
||||||
|
"<h1>Hub Core Operator Console</h1>"
|
||||||
|
"<p>Compatibility runtime candidate; projections are authoritative.</p>"
|
||||||
|
)
|
||||||
|
|
||||||
|
return router
|
||||||
|
|
||||||
|
|
||||||
|
def _enabled(request: Request, group: str) -> None:
|
||||||
|
if group not in request.app.state.settings.v2_groups:
|
||||||
|
raise HTTPException(status_code=404, detail="compatibility group is disabled")
|
||||||
|
|
||||||
|
|
||||||
|
async def _protected(
|
||||||
|
request: Request,
|
||||||
|
group: str,
|
||||||
|
authorization: str | None,
|
||||||
|
*,
|
||||||
|
write: bool = False,
|
||||||
|
) -> None:
|
||||||
|
_enabled(request, group)
|
||||||
|
if write and group not in request.app.state.settings.v2_write_groups:
|
||||||
|
raise HTTPException(status_code=503, detail="compatibility group is read-only")
|
||||||
|
if not authorization or not authorization.startswith("Bearer "):
|
||||||
|
raise _unauthorized("Missing bearer token")
|
||||||
|
token = authorization.removeprefix("Bearer ").strip()
|
||||||
|
configured = request.app.state.settings.api_token
|
||||||
|
if configured and hmac.compare_digest(token, configured):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
valid = await _store(request).validate_token(token)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=503, detail="authorization dependency unavailable") from exc
|
||||||
|
if not valid:
|
||||||
|
raise _unauthorized("Invalid bearer token")
|
||||||
|
|
||||||
|
|
||||||
|
def _store(request: Request) -> SQLCompatibilityStore:
|
||||||
|
store = request.app.state.compat_store
|
||||||
|
if store is None:
|
||||||
|
raise HTTPException(status_code=503, detail="compatibility store unavailable")
|
||||||
|
return store
|
||||||
|
|
||||||
|
|
||||||
|
def _unauthorized(message: str) -> HTTPException:
|
||||||
|
return HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail={"code": "unauthorized", "message": message},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _page(values: list[dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
return {"data": values, "count": len(values)}
|
||||||
|
|
||||||
|
|
||||||
|
def _hub(row: Any) -> dict[str, Any]:
|
||||||
|
value = dict(row["body"] or {})
|
||||||
|
value.update(
|
||||||
|
id=row["id"], slug=row["slug"], name=row["name"], domain=row["domain"],
|
||||||
|
hubKind=row["hub_kind"], hubFamily=row["hub_family"],
|
||||||
|
vsmFunction=row["vsm_function"], vsmSystem=row["vsm_system"],
|
||||||
|
status=row["status"], description=row["description"],
|
||||||
|
)
|
||||||
|
return {key: item for key, item in value.items() if item is not None}
|
||||||
|
|
||||||
|
|
||||||
|
def _manifest(row: Any) -> dict[str, Any]:
|
||||||
|
value = dict(row["body"] or {})
|
||||||
|
value.update(
|
||||||
|
id=row["id"], hubId=row["hub_id"], hubSlug=row["hub_slug"],
|
||||||
|
manifestVersion=row["manifest_version"], status=row["status"],
|
||||||
|
)
|
||||||
|
return {key: item for key, item in value.items() if item is not None}
|
||||||
|
|
||||||
|
|
||||||
|
def _consumer(row: Any) -> dict[str, Any]:
|
||||||
|
value = dict(row["body"] or {})
|
||||||
|
value.update(
|
||||||
|
id=row["id"], slug=row["slug"], name=row["name"],
|
||||||
|
description=row["description"],
|
||||||
|
hubCapabilityManifestId=row["hub_capability_manifest_id"],
|
||||||
|
rateLimitPerMinute=row["rate_limit_per_minute"], quotaPerDay=row["quota_per_day"],
|
||||||
|
keyPrefix=row["key_prefix"], status=row["status"],
|
||||||
|
)
|
||||||
|
return {key: item for key, item in value.items() if item is not None}
|
||||||
|
|
||||||
|
|
||||||
|
def _widget(row: Any) -> dict[str, Any]:
|
||||||
|
value = dict(row["body"] or {})
|
||||||
|
value.update(
|
||||||
|
id=row["id"], hubId=row["hub_id"], name=row["name"],
|
||||||
|
widgetType=row["widget_type"], capabilityRef=row["capability_ref"],
|
||||||
|
viewContext=row["view_context"], policyScope=row["policy_scope"],
|
||||||
|
status=row["status"],
|
||||||
|
)
|
||||||
|
return {key: item for key, item in value.items() if item is not None}
|
||||||
|
|
||||||
|
|
||||||
|
def _hash_token(token: str) -> str:
|
||||||
|
return hashlib.sha256(token.encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _now() -> datetime:
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
66
hub_core/runtime/compat_catalogs.py
Normal file
66
hub_core/runtime/compat_catalogs.py
Normal file
|
|
@ -0,0 +1,66 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def item(slug: str, name: str, description: str | None = None) -> dict[str, str]:
|
||||||
|
value = {"slug": slug, "name": name}
|
||||||
|
if description is not None:
|
||||||
|
value["description"] = description
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
WIDGET_TYPES = [
|
||||||
|
item("status-summary", "Status Summary"),
|
||||||
|
item("workplan-board", "Workplan Board"),
|
||||||
|
item("event-stream", "Event Stream"),
|
||||||
|
item("action", "Action Control", "Button, link, or trigger widget"),
|
||||||
|
item("chart", "Chart", "Data visualisation chart widget"),
|
||||||
|
item("chat", "Chat Region", "Conversational interaction region"),
|
||||||
|
item("diff", "Diff / Review", "Code diff or change review element"),
|
||||||
|
item("form", "Form", "Data entry form widget"),
|
||||||
|
*[item(f"ops-{name}", f"ops-{name}") for name in (
|
||||||
|
"backup-set", "cluster", "endpoint", "environment", "host", "incident",
|
||||||
|
"migration-wave", "readiness-gate", "release", "risk", "runbook",
|
||||||
|
"secret-set", "service", "service-catalog",
|
||||||
|
)],
|
||||||
|
item("panel", "Status Panel", "Summary or status information panel"),
|
||||||
|
item("recommendation", "Recommendation", "AI or system recommendation block"),
|
||||||
|
item("table", "Table", "Tabular data display widget"),
|
||||||
|
item("workflow-step", "Workflow Step", "Single step in a multi-step workflow"),
|
||||||
|
]
|
||||||
|
|
||||||
|
EVENT_TYPES = [
|
||||||
|
item("interaction.event", "Interaction Event"),
|
||||||
|
item("workplan.progress", "Workplan Progress"),
|
||||||
|
item("evidence.recorded", "Evidence Recorded"),
|
||||||
|
*[
|
||||||
|
item(slug, slug.replace("_", " ").title())
|
||||||
|
for slug in (
|
||||||
|
"abandoned", "accepted_recommendation", "blocked_by_policy", "clicked",
|
||||||
|
"commented", "escalated", "failed", "flagged_confusing", "flagged_helpful",
|
||||||
|
"focused", "rejected_recommendation", "retracted", "retried", "submitted",
|
||||||
|
"viewed",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
*[
|
||||||
|
item(f"ops-{name}", f"ops-{name}")
|
||||||
|
for name in (
|
||||||
|
"backup-verified", "drift-detected", "endpoint-verified", "health-checked",
|
||||||
|
"inventory-registered", "inventory-updated", "migration-gate-failed",
|
||||||
|
"migration-gate-passed", "readiness-gate-updated", "release-observed",
|
||||||
|
"restore-tested", "risk-accepted", "risk-raised", "runbook-executed",
|
||||||
|
"service-discovered",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
]
|
||||||
|
|
||||||
|
ANNOTATION_CATEGORIES = [
|
||||||
|
item("operator-note", "Operator Note"),
|
||||||
|
item("compatibility-delta", "Compatibility Delta"),
|
||||||
|
item("migration-evidence", "Migration Evidence"),
|
||||||
|
]
|
||||||
|
|
||||||
|
POLICY_SCOPES = [
|
||||||
|
item("public-read", "Public Read"),
|
||||||
|
item("hub-write", "Hub Write"),
|
||||||
|
item("operator-admin", "Operator Admin"),
|
||||||
|
]
|
||||||
|
|
@ -11,6 +11,10 @@ def _env_bool(name: str, default: bool) -> bool:
|
||||||
return raw.strip().lower() in {"1", "true", "yes", "on"}
|
return raw.strip().lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
|
def _env_set(name: str) -> frozenset[str]:
|
||||||
|
return frozenset(value.strip() for value in os.getenv(name, "").split(",") if value.strip())
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
@dataclass(frozen=True, slots=True)
|
||||||
class RuntimeSettings:
|
class RuntimeSettings:
|
||||||
environment: str = "development"
|
environment: str = "development"
|
||||||
|
|
@ -23,6 +27,21 @@ class RuntimeSettings:
|
||||||
mcp_port: int = 8011
|
mcp_port: int = 8011
|
||||||
mcp_transport: str = "http"
|
mcp_transport: str = "http"
|
||||||
database_url: str | None = None
|
database_url: str | None = None
|
||||||
|
api_token: str | None = None
|
||||||
|
v2_groups: frozenset[str] = frozenset()
|
||||||
|
v2_write_groups: frozenset[str] = frozenset()
|
||||||
|
legacy_write_groups: frozenset[str] = frozenset()
|
||||||
|
legacy_health: bool = False
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
overlap = self.v2_write_groups & self.legacy_write_groups
|
||||||
|
if overlap:
|
||||||
|
joined = ", ".join(sorted(overlap))
|
||||||
|
raise ValueError(f"hub-core and Core Hub write groups overlap: {joined}")
|
||||||
|
unknown = self.v2_write_groups - self.v2_groups
|
||||||
|
if unknown:
|
||||||
|
joined = ", ".join(sorted(unknown))
|
||||||
|
raise ValueError(f"hub-core write groups are not enabled: {joined}")
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_env(cls) -> RuntimeSettings:
|
def from_env(cls) -> RuntimeSettings:
|
||||||
|
|
@ -41,6 +60,11 @@ class RuntimeSettings:
|
||||||
mcp_port=int(os.getenv("HUB_CORE_MCP_PORT", "8011")),
|
mcp_port=int(os.getenv("HUB_CORE_MCP_PORT", "8011")),
|
||||||
mcp_transport=os.getenv("HUB_CORE_MCP_TRANSPORT", "http"),
|
mcp_transport=os.getenv("HUB_CORE_MCP_TRANSPORT", "http"),
|
||||||
database_url=os.getenv("HUB_CORE_DATABASE_URL") or os.getenv("DATABASE_URL"),
|
database_url=os.getenv("HUB_CORE_DATABASE_URL") or os.getenv("DATABASE_URL"),
|
||||||
|
api_token=os.getenv("HUB_CORE_API_TOKEN") or os.getenv("CORE_HUB_API_TOKEN"),
|
||||||
|
v2_groups=_env_set("HUB_CORE_V2_GROUPS"),
|
||||||
|
v2_write_groups=_env_set("HUB_CORE_V2_WRITE_GROUPS"),
|
||||||
|
legacy_write_groups=_env_set("CORE_HUB_V2_WRITE_GROUPS"),
|
||||||
|
legacy_health=_env_bool("HUB_CORE_LEGACY_HEALTH", False),
|
||||||
)
|
)
|
||||||
|
|
||||||
def readiness_checks(self, store_backend: str) -> dict[str, str]:
|
def readiness_checks(self, store_backend: str) -> dict[str, str]:
|
||||||
|
|
@ -51,6 +75,9 @@ class RuntimeSettings:
|
||||||
"active_backend": store_backend,
|
"active_backend": store_backend,
|
||||||
"ephemeral_backend": "allowed" if ephemeral_allowed else "not_allowed",
|
"ephemeral_backend": "allowed" if ephemeral_allowed else "not_allowed",
|
||||||
"contract": "helixforge.hub-extension/0.1.0",
|
"contract": "helixforge.hub-extension/0.1.0",
|
||||||
|
"v2_groups": ",".join(sorted(self.v2_groups)) or "none",
|
||||||
|
"v2_write_groups": ",".join(sorted(self.v2_write_groups)) or "none",
|
||||||
|
"legacy_write_groups": ",".join(sorted(self.legacy_write_groups)) or "none",
|
||||||
}
|
}
|
||||||
|
|
||||||
def is_ready(self, store_backend: str) -> bool:
|
def is_ready(self, store_backend: str) -> bool:
|
||||||
|
|
|
||||||
438
hub_core/runtime/migration.py
Normal file
438
hub_core/runtime/migration.py
Normal file
|
|
@ -0,0 +1,438 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
from uuid import NAMESPACE_URL, uuid4, uuid5
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
from hub_core.runtime.postgres_store import PostgresPortStore
|
||||||
|
from hub_core.runtime.tables import (
|
||||||
|
compat_api_consumers,
|
||||||
|
compat_api_keys,
|
||||||
|
compat_hubs,
|
||||||
|
compat_manifests,
|
||||||
|
compat_migration_runs,
|
||||||
|
compat_widgets,
|
||||||
|
runtime_audit_ledger,
|
||||||
|
runtime_import_runs,
|
||||||
|
runtime_interaction_events,
|
||||||
|
)
|
||||||
|
|
||||||
|
SCHEMA_VERSION = "core-hub.migration.v1"
|
||||||
|
COLLECTIONS = (
|
||||||
|
"hubs",
|
||||||
|
"hubCapabilityManifests",
|
||||||
|
"apiConsumers",
|
||||||
|
"apiKeys",
|
||||||
|
"widgets",
|
||||||
|
"interactionEvents",
|
||||||
|
"migrationRuns",
|
||||||
|
)
|
||||||
|
SECRET_FIELDS = {"fullKey", "rawKey", "apiKey", "secret", "token", "authorization"}
|
||||||
|
|
||||||
|
|
||||||
|
def _now() -> datetime:
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _json(value: Any) -> str:
|
||||||
|
return json.dumps(value, sort_keys=True, separators=(",", ":"), default=_json_default)
|
||||||
|
|
||||||
|
|
||||||
|
def _json_default(value: Any) -> str:
|
||||||
|
if isinstance(value, datetime):
|
||||||
|
return _timestamp(value)
|
||||||
|
raise TypeError(f"cannot encode {type(value).__name__}")
|
||||||
|
|
||||||
|
|
||||||
|
def _timestamp(value: datetime) -> str:
|
||||||
|
if value.tzinfo is None:
|
||||||
|
value = value.replace(tzinfo=timezone.utc)
|
||||||
|
return value.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _datetime(value: Any, default: datetime | None = None) -> datetime:
|
||||||
|
if isinstance(value, datetime):
|
||||||
|
return value if value.tzinfo else value.replace(tzinfo=timezone.utc)
|
||||||
|
if isinstance(value, str) and value:
|
||||||
|
return datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||||
|
return default or _now()
|
||||||
|
|
||||||
|
|
||||||
|
def _records(bundle: dict[str, Any], collection: str) -> list[dict[str, Any]]:
|
||||||
|
records = bundle.get("records", {})
|
||||||
|
values = records.get(collection, []) if isinstance(records, dict) else []
|
||||||
|
return [dict(value) for value in values] if isinstance(values, list) else []
|
||||||
|
|
||||||
|
|
||||||
|
def _contains_secret(value: Any) -> bool:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return any(key in SECRET_FIELDS or _contains_secret(child) for key, child in value.items())
|
||||||
|
if isinstance(value, list):
|
||||||
|
return any(_contains_secret(child) for child in value)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _canonical_record(record: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
result: dict[str, Any] = {}
|
||||||
|
for key, value in record.items():
|
||||||
|
if key in {"createdAt", "updatedAt", "occurredAt", "recordedAt"} and value:
|
||||||
|
result[key] = _timestamp(_datetime(value))
|
||||||
|
elif isinstance(value, dict):
|
||||||
|
result[key] = _canonical_record(value)
|
||||||
|
elif isinstance(value, list):
|
||||||
|
result[key] = [
|
||||||
|
_canonical_record(item) if isinstance(item, dict) else item for item in value
|
||||||
|
]
|
||||||
|
else:
|
||||||
|
result[key] = value
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def _identity(record: dict[str, Any]) -> str:
|
||||||
|
return str(record.get("id") or record.get("slug") or record.get("keyHash") or "")
|
||||||
|
|
||||||
|
|
||||||
|
def content_hashes(bundle: dict[str, Any]) -> dict[str, str]:
|
||||||
|
hashes: dict[str, str] = {}
|
||||||
|
for collection in COLLECTIONS:
|
||||||
|
values = [_canonical_record(value) for value in _records(bundle, collection)]
|
||||||
|
values.sort(key=_identity)
|
||||||
|
hashes[collection] = hashlib.sha256(_json(values).encode()).hexdigest()
|
||||||
|
return hashes
|
||||||
|
|
||||||
|
|
||||||
|
def identity_sets(bundle: dict[str, Any]) -> dict[str, list[str]]:
|
||||||
|
return {
|
||||||
|
collection: sorted(_identity(value) for value in _records(bundle, collection))
|
||||||
|
for collection in COLLECTIONS
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def bundle_digest(bundle: dict[str, Any]) -> str:
|
||||||
|
value = {key: child for key, child in bundle.items() if key != "bundleSha256"}
|
||||||
|
return hashlib.sha256(_json(value).encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def validate_bundle(bundle: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
errors: list[str] = []
|
||||||
|
warnings: list[str] = []
|
||||||
|
unknown = set(bundle) - {
|
||||||
|
"schemaVersion",
|
||||||
|
"source",
|
||||||
|
"sourceRevision",
|
||||||
|
"exportedAt",
|
||||||
|
"highWaterMark",
|
||||||
|
"bundleSha256",
|
||||||
|
"records",
|
||||||
|
}
|
||||||
|
if unknown:
|
||||||
|
errors.append(f"unknown bundle fields: {', '.join(sorted(unknown))}")
|
||||||
|
if bundle.get("schemaVersion") != SCHEMA_VERSION:
|
||||||
|
errors.append(f"schemaVersion must be {SCHEMA_VERSION}")
|
||||||
|
if not isinstance(bundle.get("records"), dict):
|
||||||
|
errors.append("records must be an object")
|
||||||
|
else:
|
||||||
|
unknown_collections = set(bundle["records"]) - set(COLLECTIONS)
|
||||||
|
if unknown_collections:
|
||||||
|
errors.append(f"unknown collections: {', '.join(sorted(unknown_collections))}")
|
||||||
|
claimed = bundle.get("bundleSha256")
|
||||||
|
calculated = bundle_digest(bundle)
|
||||||
|
if claimed and claimed != calculated:
|
||||||
|
errors.append("bundleSha256 does not match canonical bundle content")
|
||||||
|
|
||||||
|
required = {
|
||||||
|
"hubs": ("slug", "name"),
|
||||||
|
"hubCapabilityManifests": ("id",),
|
||||||
|
"apiConsumers": ("name",),
|
||||||
|
"apiKeys": ("id", "keyPrefix", "keyHash"),
|
||||||
|
"widgets": ("id", "name"),
|
||||||
|
"interactionEvents": ("id", "widgetId", "eventType"),
|
||||||
|
"migrationRuns": ("id", "source", "schemaVersion", "bundleSha256"),
|
||||||
|
}
|
||||||
|
for collection in COLLECTIONS:
|
||||||
|
seen: set[str] = set()
|
||||||
|
for index, record in enumerate(_records(bundle, collection)):
|
||||||
|
path = f"records.{collection}[{index}]"
|
||||||
|
for field in required[collection]:
|
||||||
|
if record.get(field) in (None, ""):
|
||||||
|
errors.append(f"{path}.{field} is required")
|
||||||
|
identity = _identity(record)
|
||||||
|
if identity and identity in seen:
|
||||||
|
errors.append(f"duplicate {collection} identity: {identity}")
|
||||||
|
seen.add(identity)
|
||||||
|
if _contains_secret(record):
|
||||||
|
errors.append(f"{path} contains secret-shaped material")
|
||||||
|
for record in _records(bundle, "hubCapabilityManifests"):
|
||||||
|
if not record.get("hubId") and not record.get("hubSlug"):
|
||||||
|
errors.append("manifest must include hubId or hubSlug")
|
||||||
|
for record in _records(bundle, "apiConsumers"):
|
||||||
|
if not record.get("id") and not record.get("slug"):
|
||||||
|
errors.append("API consumer must include id or slug")
|
||||||
|
for record in _records(bundle, "apiKeys"):
|
||||||
|
if not record.get("apiConsumerId") and not record.get("apiConsumerSlug"):
|
||||||
|
errors.append("API key must include apiConsumerId or apiConsumerSlug")
|
||||||
|
for record in _records(bundle, "widgets"):
|
||||||
|
if not record.get("hubId") and not record.get("hubSlug"):
|
||||||
|
errors.append("widget must include hubId or hubSlug")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"ok": not errors,
|
||||||
|
"schemaVersion": bundle.get("schemaVersion"),
|
||||||
|
"source": bundle.get("source", "unknown"),
|
||||||
|
"bundleSha256": calculated,
|
||||||
|
"highWaterMark": bundle.get("highWaterMark"),
|
||||||
|
"counts": {name: {"input": len(_records(bundle, name))} for name in COLLECTIONS},
|
||||||
|
"identitySets": identity_sets(bundle),
|
||||||
|
"contentHashes": content_hashes(bundle),
|
||||||
|
"errors": errors,
|
||||||
|
"warnings": warnings,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def import_bundle(
|
||||||
|
store: PostgresPortStore, bundle: dict[str, Any], *, dry_run: bool = False
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
report = validate_bundle(bundle)
|
||||||
|
report["dryRun"] = dry_run
|
||||||
|
report["idempotent"] = False
|
||||||
|
report["counts"] = {
|
||||||
|
name: {"input": len(_records(bundle, name)), "created": 0, "updated": 0, "skipped": 0}
|
||||||
|
for name in COLLECTIONS
|
||||||
|
}
|
||||||
|
if not report["ok"]:
|
||||||
|
return report
|
||||||
|
|
||||||
|
async with store.sessions() as session:
|
||||||
|
prior = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(runtime_import_runs).where(
|
||||||
|
runtime_import_runs.c.bundle_sha256 == report["bundleSha256"]
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).mappings().first()
|
||||||
|
if prior and not dry_run:
|
||||||
|
report["idempotent"] = True
|
||||||
|
report["migrationRunId"] = prior["id"]
|
||||||
|
for name in COLLECTIONS:
|
||||||
|
report["counts"][name]["skipped"] = report["counts"][name]["input"]
|
||||||
|
return report
|
||||||
|
|
||||||
|
async with store.sessions.begin() as session:
|
||||||
|
hubs = await _maps(session, compat_hubs, "id", "slug")
|
||||||
|
await _import_hubs(session, bundle, report, hubs, dry_run)
|
||||||
|
hubs = await _maps(session, compat_hubs, "id", "slug") if not dry_run else hubs
|
||||||
|
await _import_manifests(session, bundle, report, hubs, dry_run)
|
||||||
|
consumers = await _maps(session, compat_api_consumers, "id", "slug")
|
||||||
|
await _import_consumers(session, bundle, report, consumers, dry_run)
|
||||||
|
consumers = (
|
||||||
|
await _maps(session, compat_api_consumers, "id", "slug") if not dry_run else consumers
|
||||||
|
)
|
||||||
|
await _import_keys(session, bundle, report, consumers, dry_run)
|
||||||
|
await _import_widgets(session, bundle, report, hubs, dry_run)
|
||||||
|
await _import_events(session, bundle, report, dry_run)
|
||||||
|
await _import_source_runs(session, bundle, report, dry_run)
|
||||||
|
if dry_run:
|
||||||
|
await session.rollback()
|
||||||
|
return report
|
||||||
|
run_id = str(uuid4())
|
||||||
|
await session.execute(
|
||||||
|
runtime_import_runs.insert().values(
|
||||||
|
id=run_id,
|
||||||
|
source=report["source"],
|
||||||
|
schema_version=SCHEMA_VERSION,
|
||||||
|
bundle_sha256=report["bundleSha256"],
|
||||||
|
high_water_mark=report["highWaterMark"],
|
||||||
|
counts=report["counts"],
|
||||||
|
content_hashes=report["contentHashes"],
|
||||||
|
status="imported",
|
||||||
|
created_at=_now(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.execute(
|
||||||
|
runtime_audit_ledger.insert().values(
|
||||||
|
id=str(uuid4()),
|
||||||
|
action="migration.imported",
|
||||||
|
subject_type="migration_bundle",
|
||||||
|
subject_id=report["bundleSha256"],
|
||||||
|
correlation_id=None,
|
||||||
|
payload_hash=hashlib.sha256(_json(report["contentHashes"]).encode()).hexdigest(),
|
||||||
|
detail={"source": report["source"], "counts": report["counts"]},
|
||||||
|
recorded_at=_now(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
report["migrationRunId"] = run_id
|
||||||
|
return report
|
||||||
|
|
||||||
|
|
||||||
|
async def _maps(session: Any, table: sa.Table, *keys: str) -> dict[str, dict[str, Any]]:
|
||||||
|
rows = (await session.execute(sa.select(table))).mappings().all()
|
||||||
|
return {str(row[key]): dict(row) for row in rows for key in keys if row[key]}
|
||||||
|
|
||||||
|
|
||||||
|
def _mark(report: dict[str, Any], name: str, existing: Any) -> None:
|
||||||
|
report["counts"][name]["updated" if existing else "created"] += 1
|
||||||
|
|
||||||
|
|
||||||
|
async def _upsert(session: Any, table: sa.Table, row: dict[str, Any], existing: Any) -> None:
|
||||||
|
if existing:
|
||||||
|
await session.execute(table.update().where(table.c.id == existing["id"]).values(**row))
|
||||||
|
else:
|
||||||
|
await session.execute(table.insert().values(**row))
|
||||||
|
|
||||||
|
|
||||||
|
def _parent_id(record: dict[str, Any], rows: dict[str, dict[str, Any]], kind: str) -> str:
|
||||||
|
parent = record.get(f"{kind}Id") or record.get(f"{kind}Slug")
|
||||||
|
if parent in rows:
|
||||||
|
return str(rows[str(parent)]["id"])
|
||||||
|
raise ValueError(f"unresolved {kind} reference: {parent}")
|
||||||
|
|
||||||
|
|
||||||
|
async def _import_hubs(session: Any, bundle: dict[str, Any], report: dict, rows: dict, dry: bool) -> None:
|
||||||
|
for record in _records(bundle, "hubs"):
|
||||||
|
existing = rows.get(str(record.get("id"))) or rows.get(str(record.get("slug")))
|
||||||
|
_mark(report, "hubs", existing)
|
||||||
|
if dry:
|
||||||
|
continue
|
||||||
|
now = _now()
|
||||||
|
row = {
|
||||||
|
"id": existing["id"] if existing else str(record.get("id") or uuid4()),
|
||||||
|
"slug": record["slug"], "name": record["name"], "domain": record.get("domain"),
|
||||||
|
"hub_kind": record.get("hubKind"), "hub_family": record.get("hubFamily"),
|
||||||
|
"vsm_function": record.get("vsmFunction"), "vsm_system": record.get("vsmSystem"),
|
||||||
|
"status": record.get("status", "active"), "description": record.get("description"),
|
||||||
|
"body": record, "created_at": _datetime(record.get("createdAt"), now),
|
||||||
|
"updated_at": _datetime(record.get("updatedAt"), now),
|
||||||
|
}
|
||||||
|
await _upsert(session, compat_hubs, row, existing)
|
||||||
|
|
||||||
|
|
||||||
|
async def _import_manifests(session: Any, bundle: dict, report: dict, hubs: dict, dry: bool) -> None:
|
||||||
|
current = await _maps(session, compat_manifests, "id")
|
||||||
|
for record in _records(bundle, "hubCapabilityManifests"):
|
||||||
|
existing = current.get(str(record["id"])); _mark(report, "hubCapabilityManifests", existing)
|
||||||
|
if dry: continue
|
||||||
|
now = _now(); hub_id = _parent_id(record, hubs, "hub")
|
||||||
|
row = {"id": record["id"], "hub_id": hub_id, "hub_slug": hubs[hub_id]["slug"],
|
||||||
|
"manifest_version": record.get("manifestVersion", "0.1.0"),
|
||||||
|
"status": record.get("status", "draft"), "body": record,
|
||||||
|
"created_at": _datetime(record.get("createdAt"), now),
|
||||||
|
"updated_at": _datetime(record.get("updatedAt"), now)}
|
||||||
|
await _upsert(session, compat_manifests, row, existing)
|
||||||
|
|
||||||
|
|
||||||
|
async def _import_consumers(session: Any, bundle: dict, report: dict, rows: dict, dry: bool) -> None:
|
||||||
|
for record in _records(bundle, "apiConsumers"):
|
||||||
|
existing = rows.get(str(record.get("id"))) or rows.get(str(record.get("slug")))
|
||||||
|
_mark(report, "apiConsumers", existing)
|
||||||
|
if dry: continue
|
||||||
|
row = {"id": existing["id"] if existing else str(record.get("id") or uuid4()),
|
||||||
|
"slug": record.get("slug"), "name": record["name"],
|
||||||
|
"description": record.get("description"),
|
||||||
|
"hub_capability_manifest_id": record.get("hubCapabilityManifestId"),
|
||||||
|
"rate_limit_per_minute": record.get("rateLimitPerMinute"),
|
||||||
|
"quota_per_day": record.get("quotaPerDay"), "key_prefix": record.get("keyPrefix"),
|
||||||
|
"status": record.get("status", "active"), "body": record,
|
||||||
|
"created_at": _datetime(record.get("createdAt"))}
|
||||||
|
await _upsert(session, compat_api_consumers, row, existing)
|
||||||
|
|
||||||
|
|
||||||
|
async def _import_keys(session: Any, bundle: dict, report: dict, consumers: dict, dry: bool) -> None:
|
||||||
|
current = await _maps(session, compat_api_keys, "id", "key_hash")
|
||||||
|
for record in _records(bundle, "apiKeys"):
|
||||||
|
existing = current.get(str(record["id"])) or current.get(str(record["keyHash"]))
|
||||||
|
_mark(report, "apiKeys", existing)
|
||||||
|
if dry: continue
|
||||||
|
consumer_id = _parent_id(record, consumers, "apiConsumer")
|
||||||
|
row = {"id": record["id"], "api_consumer_id": consumer_id,
|
||||||
|
"key_prefix": record["keyPrefix"], "key_hash": record["keyHash"],
|
||||||
|
"scopes": record.get("scopes"), "status": record.get("status", "active"),
|
||||||
|
"created_at": _datetime(record.get("createdAt"))}
|
||||||
|
await _upsert(session, compat_api_keys, row, existing)
|
||||||
|
|
||||||
|
|
||||||
|
async def _import_widgets(session: Any, bundle: dict, report: dict, hubs: dict, dry: bool) -> None:
|
||||||
|
current = await _maps(session, compat_widgets, "id")
|
||||||
|
for record in _records(bundle, "widgets"):
|
||||||
|
existing = current.get(str(record["id"])); _mark(report, "widgets", existing)
|
||||||
|
if dry: continue
|
||||||
|
row = {"id": record["id"], "hub_id": _parent_id(record, hubs, "hub"),
|
||||||
|
"name": record["name"], "widget_type": record.get("widgetType"),
|
||||||
|
"capability_ref": record.get("capabilityRef"),
|
||||||
|
"view_context": record.get("viewContext"), "policy_scope": record.get("policyScope"),
|
||||||
|
"status": record.get("status", "active"), "body": record,
|
||||||
|
"created_at": _datetime(record.get("createdAt"))}
|
||||||
|
await _upsert(session, compat_widgets, row, existing)
|
||||||
|
|
||||||
|
|
||||||
|
async def _import_events(session: Any, bundle: dict, report: dict, dry: bool) -> None:
|
||||||
|
current = await _maps(session, runtime_interaction_events, "id")
|
||||||
|
for record in _records(bundle, "interactionEvents"):
|
||||||
|
existing = current.get(str(record["id"])); _mark(report, "interactionEvents", existing)
|
||||||
|
if dry: continue
|
||||||
|
created = _datetime(record.get("createdAt") or record.get("occurredAt"))
|
||||||
|
row = {"id": record["id"], "schema_version": "0.1.0",
|
||||||
|
"correlation_id": str(record.get("correlationId") or uuid5(NAMESPACE_URL, record["id"])),
|
||||||
|
"event_type": record["eventType"], "occurred_at": created,
|
||||||
|
"subject_refs": {"widget": record["widgetId"]}, "payload": {"legacy": record},
|
||||||
|
"recorded_at": _datetime(record.get("recordedAt"), created)}
|
||||||
|
await _upsert(session, runtime_interaction_events, row, existing)
|
||||||
|
|
||||||
|
|
||||||
|
async def _import_source_runs(session: Any, bundle: dict, report: dict, dry: bool) -> None:
|
||||||
|
current = await _maps(session, compat_migration_runs, "id")
|
||||||
|
for record in _records(bundle, "migrationRuns"):
|
||||||
|
existing = current.get(str(record["id"])); _mark(report, "migrationRuns", existing)
|
||||||
|
if dry: continue
|
||||||
|
row = {"id": record["id"], "source": record["source"],
|
||||||
|
"schema_version": record["schemaVersion"], "bundle_sha256": record["bundleSha256"],
|
||||||
|
"dry_run": bool(record.get("dryRun", False)), "status": record.get("status", "imported"),
|
||||||
|
"counts": record.get("counts", {}), "diagnostics": record.get("diagnostics", {}),
|
||||||
|
"created_at": _datetime(record.get("createdAt"))}
|
||||||
|
await _upsert(session, compat_migration_runs, row, existing)
|
||||||
|
|
||||||
|
|
||||||
|
async def export_bundle(store: PostgresPortStore, *, source_revision: str | None = None) -> dict[str, Any]:
|
||||||
|
async with store.sessions() as session:
|
||||||
|
hubs = (await session.execute(sa.select(compat_hubs).order_by(compat_hubs.c.id))).mappings().all()
|
||||||
|
manifests = (await session.execute(sa.select(compat_manifests).order_by(compat_manifests.c.id))).mappings().all()
|
||||||
|
consumers = (await session.execute(sa.select(compat_api_consumers).order_by(compat_api_consumers.c.id))).mappings().all()
|
||||||
|
keys = (await session.execute(sa.select(compat_api_keys).order_by(compat_api_keys.c.id))).mappings().all()
|
||||||
|
widgets = (await session.execute(sa.select(compat_widgets).order_by(compat_widgets.c.id))).mappings().all()
|
||||||
|
events = (await session.execute(sa.select(runtime_interaction_events).order_by(runtime_interaction_events.c.id))).mappings().all()
|
||||||
|
runs = (await session.execute(sa.select(compat_migration_runs).order_by(compat_migration_runs.c.id))).mappings().all()
|
||||||
|
records = {
|
||||||
|
"hubs": [_body(row, created=True, updated=True) for row in hubs],
|
||||||
|
"hubCapabilityManifests": [_body(row, created=True, updated=True) for row in manifests],
|
||||||
|
"apiConsumers": [_body(row, created=True) for row in consumers],
|
||||||
|
"apiKeys": [{"id": row["id"], "apiConsumerId": row["api_consumer_id"], "keyPrefix": row["key_prefix"], "keyHash": row["key_hash"], "scopes": row["scopes"], "status": row["status"], "createdAt": _timestamp(row["created_at"])} for row in keys],
|
||||||
|
"widgets": [_body(row, created=True) for row in widgets],
|
||||||
|
"interactionEvents": [{**dict((row["payload"] or {}).get("legacy") or {}), "id": row["id"], "widgetId": (row["subject_refs"] or {}).get("widget"), "eventType": row["event_type"], "occurredAt": _timestamp(row["occurred_at"]), "recordedAt": _timestamp(row["recorded_at"]), "correlationId": row["correlation_id"]} for row in events],
|
||||||
|
"migrationRuns": [{"id": row["id"], "source": row["source"], "schemaVersion": row["schema_version"], "bundleSha256": row["bundle_sha256"], "dryRun": row["dry_run"], "status": row["status"], "counts": row["counts"], "diagnostics": row["diagnostics"], "createdAt": _timestamp(row["created_at"])} for row in runs],
|
||||||
|
}
|
||||||
|
high_water = max((_timestamp(row["recorded_at"]) for row in events), default=None)
|
||||||
|
bundle: dict[str, Any] = {"schemaVersion": SCHEMA_VERSION, "source": "hub-core", "exportedAt": _timestamp(_now()), "highWaterMark": high_water, "records": records}
|
||||||
|
if source_revision:
|
||||||
|
bundle["sourceRevision"] = source_revision
|
||||||
|
bundle["bundleSha256"] = bundle_digest(bundle)
|
||||||
|
return bundle
|
||||||
|
|
||||||
|
|
||||||
|
def _body(row: Any, *, created: bool = False, updated: bool = False) -> dict[str, Any]:
|
||||||
|
value = dict(row["body"] or {})
|
||||||
|
if created:
|
||||||
|
value["createdAt"] = _timestamp(row["created_at"])
|
||||||
|
if updated:
|
||||||
|
value["updatedAt"] = _timestamp(row["updated_at"])
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def load_bundle(path: Path) -> dict[str, Any]:
|
||||||
|
value = json.loads(path.read_text())
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
raise ValueError("migration bundle must be a JSON object")
|
||||||
|
return value
|
||||||
321
hub_core/runtime/postgres_store.py
Normal file
321
hub_core/runtime/postgres_store.py
Normal file
|
|
@ -0,0 +1,321 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from copy import deepcopy
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any
|
||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine, async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
from hub_core.contracts import CONTRACT_VERSION
|
||||||
|
from hub_core.runtime.models import (
|
||||||
|
EventCommand,
|
||||||
|
MessageCommand,
|
||||||
|
PortAccepted,
|
||||||
|
PortCollection,
|
||||||
|
PortRecord,
|
||||||
|
Provenance,
|
||||||
|
RegistryRegistration,
|
||||||
|
)
|
||||||
|
from hub_core.runtime.tables import (
|
||||||
|
runtime_audit_ledger,
|
||||||
|
runtime_interaction_events,
|
||||||
|
runtime_messages,
|
||||||
|
runtime_progress_events,
|
||||||
|
runtime_registrations,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresPortStore:
|
||||||
|
"""Durable named-port store for PostgreSQL-compatible async engines."""
|
||||||
|
|
||||||
|
backend_name = "postgresql"
|
||||||
|
|
||||||
|
def __init__(self, engine: AsyncEngine) -> None:
|
||||||
|
self.engine = engine
|
||||||
|
self.sessions = async_sessionmaker(engine, expire_on_commit=False)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_url(cls, database_url: str) -> PostgresPortStore:
|
||||||
|
return cls(create_async_engine(database_url, pool_pre_ping=True))
|
||||||
|
|
||||||
|
async def readiness_checks(self) -> dict[str, str]:
|
||||||
|
try:
|
||||||
|
async with self.engine.connect() as connection:
|
||||||
|
await connection.execute(sa.text("SELECT 1"))
|
||||||
|
except Exception:
|
||||||
|
return {"database": "unavailable"}
|
||||||
|
return {"database": "ok"}
|
||||||
|
|
||||||
|
async def aclose(self) -> None:
|
||||||
|
await self.engine.dispose()
|
||||||
|
|
||||||
|
async def register_extension(
|
||||||
|
self,
|
||||||
|
registration: RegistryRegistration,
|
||||||
|
correlation_id: UUID,
|
||||||
|
) -> PortAccepted:
|
||||||
|
hub_slug = str(registration.descriptor["hub_slug"])
|
||||||
|
value = registration.model_dump(mode="json")
|
||||||
|
now = _now()
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
current = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(runtime_registrations.c.package).where(
|
||||||
|
runtime_registrations.c.hub_slug == hub_slug
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
duplicate = current == value
|
||||||
|
if current is None:
|
||||||
|
await session.execute(
|
||||||
|
runtime_registrations.insert().values(
|
||||||
|
hub_slug=hub_slug,
|
||||||
|
package=value,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
elif not duplicate:
|
||||||
|
await session.execute(
|
||||||
|
runtime_registrations.update()
|
||||||
|
.where(runtime_registrations.c.hub_slug == hub_slug)
|
||||||
|
.values(package=value, updated_at=now)
|
||||||
|
)
|
||||||
|
await self._audit(
|
||||||
|
session,
|
||||||
|
action="registry.duplicate" if duplicate else "registry.accepted",
|
||||||
|
subject_type="registration",
|
||||||
|
subject_id=hub_slug,
|
||||||
|
correlation_id=correlation_id,
|
||||||
|
value=value,
|
||||||
|
)
|
||||||
|
return PortAccepted(
|
||||||
|
id=hub_slug,
|
||||||
|
status="duplicate" if duplicate else "accepted",
|
||||||
|
correlation_id=correlation_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def send_message(self, command: MessageCommand) -> PortAccepted:
|
||||||
|
message_id = uuid4()
|
||||||
|
created_at = _now()
|
||||||
|
value = {
|
||||||
|
"id": str(message_id),
|
||||||
|
"created_at": created_at.isoformat(),
|
||||||
|
**command.model_dump(mode="json"),
|
||||||
|
}
|
||||||
|
stored = {
|
||||||
|
**value,
|
||||||
|
"created_at": created_at,
|
||||||
|
}
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
await session.execute(runtime_messages.insert().values(**stored))
|
||||||
|
await self._audit(
|
||||||
|
session,
|
||||||
|
action="message.accepted",
|
||||||
|
subject_type="message",
|
||||||
|
subject_id=str(message_id),
|
||||||
|
correlation_id=command.correlation_id,
|
||||||
|
value=value,
|
||||||
|
)
|
||||||
|
return PortAccepted(
|
||||||
|
id=str(message_id), status="accepted", correlation_id=command.correlation_id
|
||||||
|
)
|
||||||
|
|
||||||
|
async def list_messages(
|
||||||
|
self, address: str, conversation_id: UUID | None
|
||||||
|
) -> PortCollection:
|
||||||
|
statement = sa.select(runtime_messages).order_by(
|
||||||
|
runtime_messages.c.created_at, runtime_messages.c.id
|
||||||
|
)
|
||||||
|
if conversation_id is not None:
|
||||||
|
statement = statement.where(
|
||||||
|
runtime_messages.c.conversation_id == str(conversation_id)
|
||||||
|
)
|
||||||
|
async with self.sessions() as session:
|
||||||
|
rows = (await session.execute(statement)).mappings().all()
|
||||||
|
values = [
|
||||||
|
_message_value(row)
|
||||||
|
for row in rows
|
||||||
|
if address in list(row["to_addresses"] or [])
|
||||||
|
]
|
||||||
|
return PortCollection(items=[self._record("message", value) for value in values])
|
||||||
|
|
||||||
|
async def append_progress(self, command: EventCommand) -> PortAccepted:
|
||||||
|
return await self._append_event(command, runtime_progress_events, "progress")
|
||||||
|
|
||||||
|
async def append_interaction(self, command: EventCommand) -> PortAccepted:
|
||||||
|
return await self._append_event(command, runtime_interaction_events, "interaction")
|
||||||
|
|
||||||
|
async def query_projection(self, projection_id: str) -> PortRecord | None:
|
||||||
|
async with self.sessions() as session:
|
||||||
|
if projection_id == "hub_registry":
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(runtime_registrations.c.package).order_by(
|
||||||
|
runtime_registrations.c.hub_slug
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
items = [deepcopy(value) for value in rows]
|
||||||
|
elif projection_id == "messages":
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(runtime_messages).order_by(
|
||||||
|
runtime_messages.c.created_at, runtime_messages.c.id
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).mappings()
|
||||||
|
items = [_message_value(row) for row in rows]
|
||||||
|
elif projection_id in {"progress_events", "interaction_events"}:
|
||||||
|
table = (
|
||||||
|
runtime_progress_events
|
||||||
|
if projection_id == "progress_events"
|
||||||
|
else runtime_interaction_events
|
||||||
|
)
|
||||||
|
family = "progress" if projection_id == "progress_events" else "interaction"
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
sa.select(table).order_by(table.c.recorded_at, table.c.id)
|
||||||
|
)
|
||||||
|
).mappings()
|
||||||
|
items = [_event_value(row, family) for row in rows]
|
||||||
|
else:
|
||||||
|
return None
|
||||||
|
return self._record(
|
||||||
|
projection_id,
|
||||||
|
{
|
||||||
|
"projection_id": projection_id,
|
||||||
|
"items": items,
|
||||||
|
"rebuild_from": _rebuild_sources(projection_id),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _append_event(
|
||||||
|
self,
|
||||||
|
command: EventCommand,
|
||||||
|
table: sa.Table,
|
||||||
|
family: str,
|
||||||
|
) -> PortAccepted:
|
||||||
|
event_id = uuid4()
|
||||||
|
recorded_at = _now()
|
||||||
|
value = {
|
||||||
|
"id": str(event_id),
|
||||||
|
"family": family,
|
||||||
|
"recorded_at": recorded_at.isoformat(),
|
||||||
|
**command.model_dump(mode="json"),
|
||||||
|
}
|
||||||
|
stored = {
|
||||||
|
"id": str(event_id),
|
||||||
|
"schema_version": command.schema_version,
|
||||||
|
"correlation_id": str(command.correlation_id),
|
||||||
|
"event_type": command.event_type,
|
||||||
|
"occurred_at": command.occurred_at,
|
||||||
|
"subject_refs": command.subject_refs,
|
||||||
|
"payload": command.payload,
|
||||||
|
"recorded_at": recorded_at,
|
||||||
|
}
|
||||||
|
async with self.sessions.begin() as session:
|
||||||
|
await session.execute(table.insert().values(**stored))
|
||||||
|
await self._audit(
|
||||||
|
session,
|
||||||
|
action=f"event.{family}.accepted",
|
||||||
|
subject_type=f"{family}_event",
|
||||||
|
subject_id=str(event_id),
|
||||||
|
correlation_id=command.correlation_id,
|
||||||
|
value=value,
|
||||||
|
)
|
||||||
|
return PortAccepted(
|
||||||
|
id=str(event_id), status="accepted", correlation_id=command.correlation_id
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _audit(
|
||||||
|
self,
|
||||||
|
session: Any,
|
||||||
|
*,
|
||||||
|
action: str,
|
||||||
|
subject_type: str,
|
||||||
|
subject_id: str,
|
||||||
|
correlation_id: UUID | None,
|
||||||
|
value: Mapping[str, Any],
|
||||||
|
) -> None:
|
||||||
|
await session.execute(
|
||||||
|
runtime_audit_ledger.insert().values(
|
||||||
|
id=str(uuid4()),
|
||||||
|
action=action,
|
||||||
|
subject_type=subject_type,
|
||||||
|
subject_id=subject_id,
|
||||||
|
correlation_id=str(correlation_id) if correlation_id else None,
|
||||||
|
payload_hash=_hash(value),
|
||||||
|
detail={"schema_version": CONTRACT_VERSION},
|
||||||
|
recorded_at=_now(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
def _record(self, kind: str, value: dict[str, Any]) -> PortRecord:
|
||||||
|
record_id = str(value.get("id") or kind)
|
||||||
|
return PortRecord(
|
||||||
|
id=record_id,
|
||||||
|
data=deepcopy(value),
|
||||||
|
provenance=Provenance(
|
||||||
|
source_system="hub-core-postgresql",
|
||||||
|
source_ref=f"postgresql://hub_runtime/{kind}/{record_id}",
|
||||||
|
schema_version=CONTRACT_VERSION,
|
||||||
|
content_hash=_hash(value),
|
||||||
|
indexed_at=_now(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _message_value(row: Mapping[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": str(row["id"]),
|
||||||
|
"created_at": _iso(row["created_at"]),
|
||||||
|
"schema_version": row["schema_version"],
|
||||||
|
"correlation_id": str(row["correlation_id"]),
|
||||||
|
"conversation_id": str(row["conversation_id"]) if row["conversation_id"] else None,
|
||||||
|
"from_address": row["from_address"],
|
||||||
|
"to_addresses": list(row["to_addresses"] or []),
|
||||||
|
"body": row["body"],
|
||||||
|
"subject_refs": dict(row["subject_refs"] or {}),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _event_value(row: Mapping[str, Any], family: str) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": str(row["id"]),
|
||||||
|
"family": family,
|
||||||
|
"recorded_at": _iso(row["recorded_at"]),
|
||||||
|
"schema_version": row["schema_version"],
|
||||||
|
"correlation_id": str(row["correlation_id"]),
|
||||||
|
"event_type": row["event_type"],
|
||||||
|
"occurred_at": _iso(row["occurred_at"]),
|
||||||
|
"subject_refs": dict(row["subject_refs"] or {}),
|
||||||
|
"payload": dict(row["payload"] or {}),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _hash(value: Mapping[str, Any]) -> str:
|
||||||
|
encoded = json.dumps(value, sort_keys=True, separators=(",", ":"), default=str).encode()
|
||||||
|
return hashlib.sha256(encoded).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _iso(value: Any) -> str:
|
||||||
|
return value.isoformat() if isinstance(value, datetime) else str(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _now() -> datetime:
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _rebuild_sources(projection_id: str) -> list[str]:
|
||||||
|
return {
|
||||||
|
"hub_registry": ["runtime_registrations"],
|
||||||
|
"messages": ["runtime_messages"],
|
||||||
|
"progress_events": ["runtime_progress_events"],
|
||||||
|
"interaction_events": ["runtime_interaction_events"],
|
||||||
|
}[projection_id]
|
||||||
|
|
@ -26,6 +26,8 @@ class PortStore(Protocol):
|
||||||
|
|
||||||
backend_name: str
|
backend_name: str
|
||||||
|
|
||||||
|
async def readiness_checks(self) -> dict[str, str]: ...
|
||||||
|
|
||||||
async def register_extension(
|
async def register_extension(
|
||||||
self,
|
self,
|
||||||
registration: RegistryRegistration,
|
registration: RegistryRegistration,
|
||||||
|
|
@ -59,6 +61,9 @@ class InMemoryPortStore:
|
||||||
self._progress_events: list[dict[str, Any]] = []
|
self._progress_events: list[dict[str, Any]] = []
|
||||||
self._interaction_events: list[dict[str, Any]] = []
|
self._interaction_events: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def readiness_checks(self) -> dict[str, str]:
|
||||||
|
return {"database": "not_applicable"}
|
||||||
|
|
||||||
async def register_extension(
|
async def register_extension(
|
||||||
self,
|
self,
|
||||||
registration: RegistryRegistration,
|
registration: RegistryRegistration,
|
||||||
|
|
|
||||||
169
hub_core/runtime/tables.py
Normal file
169
hub_core/runtime/tables.py
Normal file
|
|
@ -0,0 +1,169 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
runtime_metadata = sa.MetaData()
|
||||||
|
|
||||||
|
runtime_registrations = sa.Table(
|
||||||
|
"runtime_registrations",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("hub_slug", sa.String(120), primary_key=True),
|
||||||
|
sa.Column("package", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
runtime_messages = sa.Table(
|
||||||
|
"runtime_messages",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("schema_version", sa.String(40), nullable=False),
|
||||||
|
sa.Column("correlation_id", sa.String(36), nullable=False, index=True),
|
||||||
|
sa.Column("conversation_id", sa.String(36), nullable=True, index=True),
|
||||||
|
sa.Column("from_address", sa.String(240), nullable=False),
|
||||||
|
sa.Column("to_addresses", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("body", sa.Text(), nullable=False),
|
||||||
|
sa.Column("subject_refs", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, index=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _event_table(name: str) -> sa.Table:
|
||||||
|
return sa.Table(
|
||||||
|
name,
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("schema_version", sa.String(40), nullable=False),
|
||||||
|
sa.Column("correlation_id", sa.String(36), nullable=False, index=True),
|
||||||
|
sa.Column("event_type", sa.String(160), nullable=False, index=True),
|
||||||
|
sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("subject_refs", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("payload", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False, index=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
runtime_progress_events = _event_table("runtime_progress_events")
|
||||||
|
runtime_interaction_events = _event_table("runtime_interaction_events")
|
||||||
|
|
||||||
|
runtime_audit_ledger = sa.Table(
|
||||||
|
"runtime_audit_ledger",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("action", sa.String(120), nullable=False, index=True),
|
||||||
|
sa.Column("subject_type", sa.String(80), nullable=False),
|
||||||
|
sa.Column("subject_id", sa.String(240), nullable=False, index=True),
|
||||||
|
sa.Column("correlation_id", sa.String(36), nullable=True, index=True),
|
||||||
|
sa.Column("payload_hash", sa.String(64), nullable=False),
|
||||||
|
sa.Column("detail", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False, index=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
compat_hubs = sa.Table(
|
||||||
|
"compat_hubs",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("slug", sa.String(120), nullable=False, unique=True, index=True),
|
||||||
|
sa.Column("name", sa.String(240), nullable=False),
|
||||||
|
sa.Column("domain", sa.String(240), nullable=True),
|
||||||
|
sa.Column("hub_kind", sa.String(80), nullable=True),
|
||||||
|
sa.Column("hub_family", sa.String(80), nullable=True),
|
||||||
|
sa.Column("vsm_function", sa.String(80), nullable=True),
|
||||||
|
sa.Column("vsm_system", sa.String(80), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("description", sa.Text(), nullable=True),
|
||||||
|
sa.Column("body", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
compat_manifests = sa.Table(
|
||||||
|
"compat_manifests",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=True, index=True),
|
||||||
|
sa.Column("hub_slug", sa.String(120), nullable=True, index=True),
|
||||||
|
sa.Column("manifest_version", sa.String(40), nullable=False),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("body", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
compat_api_consumers = sa.Table(
|
||||||
|
"compat_api_consumers",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("slug", sa.String(120), nullable=True, unique=True, index=True),
|
||||||
|
sa.Column("name", sa.String(240), nullable=False, index=True),
|
||||||
|
sa.Column("description", sa.Text(), nullable=True),
|
||||||
|
sa.Column("hub_capability_manifest_id", sa.String(36), nullable=True, index=True),
|
||||||
|
sa.Column("rate_limit_per_minute", sa.Integer(), nullable=True),
|
||||||
|
sa.Column("quota_per_day", sa.Integer(), nullable=True),
|
||||||
|
sa.Column("key_prefix", sa.String(32), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("body", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
compat_api_keys = sa.Table(
|
||||||
|
"compat_api_keys",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column(
|
||||||
|
"api_consumer_id",
|
||||||
|
sa.String(36),
|
||||||
|
sa.ForeignKey("compat_api_consumers.id"),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
),
|
||||||
|
sa.Column("key_prefix", sa.String(32), nullable=False, index=True),
|
||||||
|
sa.Column("key_hash", sa.String(64), nullable=False, unique=True),
|
||||||
|
sa.Column("scopes", sa.Text(), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
compat_widgets = sa.Table(
|
||||||
|
"compat_widgets",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("hub_id", sa.String(36), sa.ForeignKey("compat_hubs.id"), nullable=False, index=True),
|
||||||
|
sa.Column("name", sa.String(240), nullable=False),
|
||||||
|
sa.Column("widget_type", sa.String(120), nullable=True, index=True),
|
||||||
|
sa.Column("capability_ref", sa.String(240), nullable=True, index=True),
|
||||||
|
sa.Column("view_context", sa.String(500), nullable=True),
|
||||||
|
sa.Column("policy_scope", sa.String(120), nullable=True),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("body", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
compat_migration_runs = sa.Table(
|
||||||
|
"compat_migration_runs",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("source", sa.String(120), nullable=False, index=True),
|
||||||
|
sa.Column("schema_version", sa.String(80), nullable=False),
|
||||||
|
sa.Column("bundle_sha256", sa.String(64), nullable=False, index=True),
|
||||||
|
sa.Column("dry_run", sa.Boolean(), nullable=False),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("counts", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("diagnostics", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
runtime_import_runs = sa.Table(
|
||||||
|
"runtime_import_runs",
|
||||||
|
runtime_metadata,
|
||||||
|
sa.Column("id", sa.String(36), primary_key=True),
|
||||||
|
sa.Column("source", sa.String(120), nullable=False, index=True),
|
||||||
|
sa.Column("schema_version", sa.String(80), nullable=False),
|
||||||
|
sa.Column("bundle_sha256", sa.String(64), nullable=False, unique=True, index=True),
|
||||||
|
sa.Column("high_water_mark", sa.String(200), nullable=True),
|
||||||
|
sa.Column("counts", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("content_hashes", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("status", sa.String(40), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
)
|
||||||
|
|
@ -25,6 +25,7 @@ hub-core = "hub_core.runtime.cli:main"
|
||||||
|
|
||||||
[dependency-groups]
|
[dependency-groups]
|
||||||
dev = [
|
dev = [
|
||||||
|
"aiosqlite>=0.20.0",
|
||||||
"pytest>=8.0.0",
|
"pytest>=8.0.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
|
||||||
16
tests/fixtures/core-hub-migration.v1.json
vendored
Normal file
16
tests/fixtures/core-hub-migration.v1.json
vendored
Normal file
|
|
@ -0,0 +1,16 @@
|
||||||
|
{
|
||||||
|
"schemaVersion": "core-hub.migration.v1",
|
||||||
|
"source": "core-hub-test-fixture",
|
||||||
|
"sourceRevision": "fixture-revision",
|
||||||
|
"exportedAt": "2026-08-21T12:00:00Z",
|
||||||
|
"highWaterMark": "2026-08-21T11:00:00Z",
|
||||||
|
"records": {
|
||||||
|
"hubs": [{"id": "11111111-1111-4111-8111-111111111111", "slug": "ops-hub", "name": "Ops Hub", "createdAt": "2026-08-20T10:00:00Z", "updatedAt": "2026-08-20T11:00:00Z"}],
|
||||||
|
"hubCapabilityManifests": [{"id": "22222222-2222-4222-8222-222222222222", "hubId": "11111111-1111-4111-8111-111111111111", "manifestVersion": "1.0", "status": "active"}],
|
||||||
|
"apiConsumers": [{"id": "33333333-3333-4333-8333-333333333333", "slug": "ops-hub", "name": "ops-hub", "hubCapabilityManifestId": "22222222-2222-4222-8222-222222222222"}],
|
||||||
|
"apiKeys": [{"id": "44444444-4444-4444-8444-444444444444", "apiConsumerId": "33333333-3333-4333-8333-333333333333", "keyPrefix": "ch_fixture", "keyHash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "scopes": "framework:read", "status": "active"}],
|
||||||
|
"widgets": [{"id": "55555555-5555-4555-8555-555555555555", "hubId": "11111111-1111-4111-8111-111111111111", "name": "Readiness", "widgetType": "ops-readiness-gate"}],
|
||||||
|
"interactionEvents": [{"id": "66666666-6666-4666-8666-666666666666", "widgetId": "55555555-5555-4555-8555-555555555555", "eventType": "ops-endpoint-verified", "metadata": {"expectedStatus": 401}, "createdAt": "2026-08-21T11:00:00Z"}],
|
||||||
|
"migrationRuns": [{"id": "77777777-7777-4777-8777-777777777777", "source": "inter-hub", "schemaVersion": "core-hub.migration.v1", "bundleSha256": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "dryRun": false, "status": "imported", "counts": {}, "diagnostics": {}, "createdAt": "2026-08-19T09:00:00Z"}]
|
||||||
|
}
|
||||||
|
}
|
||||||
126
tests/test_compatibility.py
Normal file
126
tests/test_compatibility.py
Normal file
|
|
@ -0,0 +1,126 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
from hub_core.runtime.app import create_app
|
||||||
|
from hub_core.runtime.config import RuntimeSettings
|
||||||
|
from hub_core.runtime.postgres_store import PostgresPortStore
|
||||||
|
from hub_core.runtime.tables import runtime_metadata
|
||||||
|
|
||||||
|
GROUPS = frozenset({"system", "registry", "credentials", "interaction", "deferred", "operator"})
|
||||||
|
HEADERS = {"Authorization": "Bearer operator-token"}
|
||||||
|
|
||||||
|
|
||||||
|
def compatibility_client(tmp_path, *, write_groups: frozenset[str] = GROUPS) -> tuple[TestClient, PostgresPortStore]:
|
||||||
|
database_url = f"sqlite+aiosqlite:///{tmp_path / 'compat.db'}"
|
||||||
|
|
||||||
|
async def create_schema() -> None:
|
||||||
|
engine = create_async_engine(database_url)
|
||||||
|
async with engine.begin() as connection:
|
||||||
|
await connection.run_sync(runtime_metadata.create_all)
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
asyncio.run(create_schema())
|
||||||
|
store = PostgresPortStore.from_url(database_url)
|
||||||
|
settings = RuntimeSettings(
|
||||||
|
environment="production",
|
||||||
|
backend="postgresql",
|
||||||
|
allow_ephemeral=False,
|
||||||
|
database_url=database_url,
|
||||||
|
api_token="operator-token",
|
||||||
|
v2_groups=GROUPS,
|
||||||
|
v2_write_groups=write_groups,
|
||||||
|
legacy_health=True,
|
||||||
|
)
|
||||||
|
return TestClient(create_app(settings=settings, port_store=store)), store
|
||||||
|
|
||||||
|
|
||||||
|
def test_route_group_configuration_rejects_two_writers() -> None:
|
||||||
|
try:
|
||||||
|
RuntimeSettings(
|
||||||
|
v2_groups=frozenset({"registry"}),
|
||||||
|
v2_write_groups=frozenset({"registry"}),
|
||||||
|
legacy_write_groups=frozenset({"registry"}),
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
assert "write groups overlap" in str(exc)
|
||||||
|
else:
|
||||||
|
raise AssertionError("overlapping writers were accepted")
|
||||||
|
|
||||||
|
|
||||||
|
def test_system_compatibility_and_protected_auth(tmp_path) -> None:
|
||||||
|
client, store = compatibility_client(tmp_path)
|
||||||
|
|
||||||
|
assert client.get("/healthz").json()["service"] == "core-hub"
|
||||||
|
assert client.get("/api/v2/widget-types").status_code == 200
|
||||||
|
denied = client.get("/api/v2/hubs")
|
||||||
|
assert denied.status_code == 401
|
||||||
|
assert denied.json()["detail"]["code"] == "unauthorized"
|
||||||
|
document = client.get("/api/v2/openapi.json").json()
|
||||||
|
assert "/api/v2/hubs" in document["paths"]
|
||||||
|
assert "/hubs" in document["paths"]
|
||||||
|
asyncio.run(store.aclose())
|
||||||
|
|
||||||
|
|
||||||
|
def test_ops_hub_bootstrap_and_dynamic_key(tmp_path) -> None:
|
||||||
|
client, store = compatibility_client(tmp_path)
|
||||||
|
|
||||||
|
hub = client.post(
|
||||||
|
"/api/v2/hubs",
|
||||||
|
headers=HEADERS,
|
||||||
|
json={"slug": "ops-hub", "name": "Ops Hub", "domain": "ops.coulomb.social"},
|
||||||
|
).json()
|
||||||
|
manifest = client.post(
|
||||||
|
"/api/v2/hub-capability-manifests",
|
||||||
|
headers=HEADERS,
|
||||||
|
json={"hubId": hub["id"], "manifestVersion": "1.0"},
|
||||||
|
).json()
|
||||||
|
activated = client.post(
|
||||||
|
f"/api/v2/hub-capability-manifests/{manifest['id']}/activate", headers=HEADERS
|
||||||
|
)
|
||||||
|
assert activated.json()["status"] == "active"
|
||||||
|
consumer = client.post(
|
||||||
|
"/api/v2/api-consumers",
|
||||||
|
headers=HEADERS,
|
||||||
|
json={"name": "ops-hub", "hubCapabilityManifestId": manifest["id"]},
|
||||||
|
).json()
|
||||||
|
issued = client.post(
|
||||||
|
f"/api/v2/api-consumers/{consumer['id']}/api-keys",
|
||||||
|
headers=HEADERS,
|
||||||
|
json={"scopes": "framework:read hub:ops-hub:write"},
|
||||||
|
).json()
|
||||||
|
runtime_headers = {"Authorization": f"Bearer {issued['fullKey']}"}
|
||||||
|
widget = client.post(
|
||||||
|
"/api/v2/widgets",
|
||||||
|
headers=runtime_headers,
|
||||||
|
json={"hubId": hub["id"], "name": "Readiness", "widgetType": "ops-readiness-gate"},
|
||||||
|
).json()
|
||||||
|
event = client.post(
|
||||||
|
"/api/v2/interaction-events",
|
||||||
|
headers=runtime_headers,
|
||||||
|
json={
|
||||||
|
"widgetId": widget["id"],
|
||||||
|
"eventType": "ops-endpoint-verified",
|
||||||
|
"metadata": {"expectedStatus": 401},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert event.status_code == 201
|
||||||
|
assert client.get("/api/v2/hub-registry", headers=runtime_headers).status_code == 200
|
||||||
|
assert client.get("/api/v2/interaction-events", headers=runtime_headers).json()["count"] == 1
|
||||||
|
assert "fullKey" not in str(client.get("/api/v2/api-consumers", headers=HEADERS).json())
|
||||||
|
asyncio.run(store.aclose())
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_only_group_rejects_write(tmp_path) -> None:
|
||||||
|
client, store = compatibility_client(tmp_path, write_groups=frozenset())
|
||||||
|
response = client.post(
|
||||||
|
"/api/v2/hubs", headers=HEADERS, json={"slug": "ops-hub", "name": "Ops Hub"}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 503
|
||||||
|
assert response.json()["detail"] == "compatibility group is read-only"
|
||||||
|
asyncio.run(store.aclose())
|
||||||
97
tests/test_migration.py
Normal file
97
tests/test_migration.py
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
import json
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
from hub_core.runtime.migration import (
|
||||||
|
bundle_digest,
|
||||||
|
export_bundle,
|
||||||
|
import_bundle,
|
||||||
|
validate_bundle,
|
||||||
|
)
|
||||||
|
from hub_core.runtime.postgres_store import PostgresPortStore
|
||||||
|
from hub_core.runtime.tables import runtime_metadata
|
||||||
|
|
||||||
|
|
||||||
|
def bundle() -> dict:
|
||||||
|
value = {
|
||||||
|
"schemaVersion": "core-hub.migration.v1",
|
||||||
|
"source": "core-hub",
|
||||||
|
"sourceRevision": "abc123",
|
||||||
|
"exportedAt": "2026-08-21T12:00:00Z",
|
||||||
|
"highWaterMark": "2026-08-21T11:00:00Z",
|
||||||
|
"records": {
|
||||||
|
"hubs": [{"id": "hub-1", "slug": "ops-hub", "name": "Ops Hub", "createdAt": "2026-08-20T10:00:00Z", "updatedAt": "2026-08-20T11:00:00Z"}],
|
||||||
|
"hubCapabilityManifests": [{"id": "manifest-1", "hubId": "hub-1", "manifestVersion": "1.0", "status": "active"}],
|
||||||
|
"apiConsumers": [{"id": "consumer-1", "slug": "ops-hub", "name": "ops-hub", "hubCapabilityManifestId": "manifest-1"}],
|
||||||
|
"apiKeys": [{"id": "key-1", "apiConsumerId": "consumer-1", "keyPrefix": "ch_prefix", "keyHash": "a" * 64, "scopes": "framework:read"}],
|
||||||
|
"widgets": [{"id": "widget-1", "hubId": "hub-1", "name": "Readiness", "widgetType": "ops-readiness-gate"}],
|
||||||
|
"interactionEvents": [{"id": "event-1", "widgetId": "widget-1", "eventType": "ops-endpoint-verified", "metadata": {"status": 401}, "createdAt": "2026-08-21T11:00:00Z"}],
|
||||||
|
"migrationRuns": [{"id": "run-1", "source": "inter-hub", "schemaVersion": "core-hub.migration.v1", "bundleSha256": "b" * 64, "dryRun": False, "status": "imported", "counts": {}, "diagnostics": {}, "createdAt": "2026-08-19T09:00:00Z"}],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
value["bundleSha256"] = bundle_digest(value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
async def make_store(tmp_path) -> PostgresPortStore:
|
||||||
|
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'migration.db'}")
|
||||||
|
async with engine.begin() as connection:
|
||||||
|
await connection.run_sync(runtime_metadata.create_all)
|
||||||
|
return PostgresPortStore(engine)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validation_rejects_secret_material_and_digest_drift():
|
||||||
|
value = bundle()
|
||||||
|
value["records"]["apiKeys"][0]["fullKey"] = "must-not-move"
|
||||||
|
|
||||||
|
report = validate_bundle(value)
|
||||||
|
|
||||||
|
assert report["ok"] is False
|
||||||
|
assert any("secret-shaped" in error for error in report["errors"])
|
||||||
|
assert any("bundleSha256" in error for error in report["errors"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_import_is_idempotent_and_exports_all_seven_collections(tmp_path):
|
||||||
|
async def run():
|
||||||
|
store = await make_store(tmp_path)
|
||||||
|
try:
|
||||||
|
return (
|
||||||
|
await import_bundle(store, bundle()),
|
||||||
|
await import_bundle(store, bundle()),
|
||||||
|
await export_bundle(store, source_revision="target-revision"),
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
await store.aclose()
|
||||||
|
|
||||||
|
first, second, exported = asyncio.run(run())
|
||||||
|
|
||||||
|
assert first["ok"] is True
|
||||||
|
assert first["counts"]["migrationRuns"]["created"] == 1
|
||||||
|
assert second["idempotent"] is True
|
||||||
|
assert second["migrationRunId"] == first["migrationRunId"]
|
||||||
|
assert set(exported["records"]) == {
|
||||||
|
"hubs", "hubCapabilityManifests", "apiConsumers", "apiKeys",
|
||||||
|
"widgets", "interactionEvents", "migrationRuns",
|
||||||
|
}
|
||||||
|
assert exported["records"]["apiKeys"][0]["keyHash"] == "a" * 64
|
||||||
|
assert "fullKey" not in json.dumps(exported)
|
||||||
|
assert validate_bundle(exported)["ok"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_dry_run_does_not_record_import(tmp_path):
|
||||||
|
async def run():
|
||||||
|
store = await make_store(tmp_path)
|
||||||
|
try:
|
||||||
|
return (
|
||||||
|
await import_bundle(store, bundle(), dry_run=True),
|
||||||
|
await import_bundle(store, bundle()),
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
await store.aclose()
|
||||||
|
|
||||||
|
report, followup = asyncio.run(run())
|
||||||
|
|
||||||
|
assert report["dryRun"] is True
|
||||||
|
assert "migrationRunId" not in report
|
||||||
|
assert followup["idempotent"] is False
|
||||||
100
tests/test_postgres_store.py
Normal file
100
tests/test_postgres_store.py
Normal file
|
|
@ -0,0 +1,100 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
from hub_core.contracts import extension_contract_root
|
||||||
|
from hub_core.runtime.app import create_app
|
||||||
|
from hub_core.runtime.config import RuntimeSettings
|
||||||
|
from hub_core.runtime.postgres_store import PostgresPortStore
|
||||||
|
from hub_core.runtime.tables import runtime_audit_ledger, runtime_metadata
|
||||||
|
|
||||||
|
|
||||||
|
def test_durable_store_survives_reopen_and_keeps_event_families_separate(tmp_path) -> None:
|
||||||
|
database_url = f"sqlite+aiosqlite:///{tmp_path / 'runtime.db'}"
|
||||||
|
|
||||||
|
async def create_schema() -> None:
|
||||||
|
engine = create_async_engine(database_url)
|
||||||
|
async with engine.begin() as connection:
|
||||||
|
await connection.run_sync(runtime_metadata.create_all)
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
asyncio.run(create_schema())
|
||||||
|
package = json.loads(
|
||||||
|
extension_contract_root()
|
||||||
|
.joinpath("fixtures", "ops-hub.extension.json")
|
||||||
|
.read_text(encoding="utf-8")
|
||||||
|
)
|
||||||
|
settings = RuntimeSettings(
|
||||||
|
environment="production",
|
||||||
|
backend="postgresql",
|
||||||
|
allow_ephemeral=False,
|
||||||
|
database_url=database_url,
|
||||||
|
)
|
||||||
|
first_store = PostgresPortStore.from_url(database_url)
|
||||||
|
first = TestClient(create_app(settings=settings, port_store=first_store))
|
||||||
|
|
||||||
|
correlation_id = str(uuid4())
|
||||||
|
assert first.get("/readyz").status_code == 200
|
||||||
|
assert first.post(
|
||||||
|
"/ports/registry/registrations",
|
||||||
|
headers={"X-Correlation-ID": correlation_id},
|
||||||
|
json=package,
|
||||||
|
).status_code == 202
|
||||||
|
assert first.post(
|
||||||
|
"/ports/events/progress", json=_event("hub.progress.recorded")
|
||||||
|
).status_code == 202
|
||||||
|
assert first.post(
|
||||||
|
"/ports/events/interaction", json=_event("hub.interaction.recorded")
|
||||||
|
).status_code == 202
|
||||||
|
asyncio.run(first_store.aclose())
|
||||||
|
|
||||||
|
second_store = PostgresPortStore.from_url(database_url)
|
||||||
|
second = TestClient(create_app(settings=settings, port_store=second_store))
|
||||||
|
registry = second.get("/ports/projections/hub_registry").json()
|
||||||
|
progress = second.get("/ports/projections/progress_events").json()
|
||||||
|
interaction = second.get("/ports/projections/interaction_events").json()
|
||||||
|
|
||||||
|
assert registry["data"]["items"][0]["descriptor"]["hub_slug"] == "ops-hub"
|
||||||
|
assert {item["family"] for item in progress["data"]["items"]} == {"progress"}
|
||||||
|
assert {item["family"] for item in interaction["data"]["items"]} == {"interaction"}
|
||||||
|
assert progress["provenance"]["source_system"] == "hub-core-postgresql"
|
||||||
|
|
||||||
|
async def audit_count() -> int:
|
||||||
|
async with second_store.sessions() as session:
|
||||||
|
return len((await session.execute(runtime_audit_ledger.select())).all())
|
||||||
|
|
||||||
|
assert asyncio.run(audit_count()) == 3
|
||||||
|
asyncio.run(second_store.aclose())
|
||||||
|
|
||||||
|
|
||||||
|
def test_postgresql_readiness_fails_when_database_is_unavailable() -> None:
|
||||||
|
database_url = "sqlite+aiosqlite:////definitely-missing-parent/runtime.db"
|
||||||
|
store = PostgresPortStore.from_url(database_url)
|
||||||
|
settings = RuntimeSettings(
|
||||||
|
environment="production",
|
||||||
|
backend="postgresql",
|
||||||
|
allow_ephemeral=False,
|
||||||
|
database_url=database_url,
|
||||||
|
)
|
||||||
|
response = TestClient(create_app(settings=settings, port_store=store)).get("/readyz")
|
||||||
|
|
||||||
|
assert response.status_code == 503
|
||||||
|
assert response.json()["checks"]["database"] == "unavailable"
|
||||||
|
asyncio.run(store.aclose())
|
||||||
|
|
||||||
|
|
||||||
|
def _event(event_type: str) -> dict:
|
||||||
|
return {
|
||||||
|
"schema_version": "0.1.0",
|
||||||
|
"correlation_id": str(uuid4()),
|
||||||
|
"event_type": event_type,
|
||||||
|
"occurred_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"subject_refs": {"hub": "ops-hub"},
|
||||||
|
"payload": {"result": "ok"},
|
||||||
|
}
|
||||||
15
uv.lock
generated
15
uv.lock
generated
|
|
@ -19,6 +19,15 @@ wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/67/cd/0d76dfc5de72bde52f55f53e925c7d152d9c7906634ec1e0cbc7e8d4ad93/aiofile-3.11.1-py3-none-any.whl", hash = "sha256:ce77d14ac07f77bc2b757834a5c129321f3f705c474593deed5ab209079a52c9", size = 20446 },
|
{ url = "https://files.pythonhosted.org/packages/67/cd/0d76dfc5de72bde52f55f53e925c7d152d9c7906634ec1e0cbc7e8d4ad93/aiofile-3.11.1-py3-none-any.whl", hash = "sha256:ce77d14ac07f77bc2b757834a5c129321f3f705c474593deed5ab209079a52c9", size = 20446 },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "aiosqlite"
|
||||||
|
version = "0.22.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/4e/8a/64761f4005f17809769d23e518d915db74e6310474e733e3593cfc854ef1/aiosqlite-0.22.1.tar.gz", hash = "sha256:043e0bd78d32888c0a9ca90fc788b38796843360c855a7262a532813133a0650", size = 14821 }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/00/b7/e3bf5133d697a08128598c8d0abc5e16377b51465a33756de24fa7dee953/aiosqlite-0.22.1-py3-none-any.whl", hash = "sha256:21c002eb13823fad740196c5a2e9d8e62f6243bd9e7e4a1f87fb5e44ecb4fceb", size = 17405 },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "alembic"
|
name = "alembic"
|
||||||
version = "1.19.1"
|
version = "1.19.1"
|
||||||
|
|
@ -648,6 +657,7 @@ runtime = [
|
||||||
|
|
||||||
[package.dev-dependencies]
|
[package.dev-dependencies]
|
||||||
dev = [
|
dev = [
|
||||||
|
{ name = "aiosqlite" },
|
||||||
{ name = "pytest" },
|
{ name = "pytest" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
@ -666,7 +676,10 @@ requires-dist = [
|
||||||
]
|
]
|
||||||
|
|
||||||
[package.metadata.requires-dev]
|
[package.metadata.requires-dev]
|
||||||
dev = [{ name = "pytest", specifier = ">=8.0.0" }]
|
dev = [
|
||||||
|
{ name = "aiosqlite", specifier = ">=0.20.0" },
|
||||||
|
{ name = "pytest", specifier = ">=8.0.0" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "idna"
|
name = "idna"
|
||||||
|
|
|
||||||
108
workplans/HUB-WP-0005-core-hub-absorption-execution.md
Normal file
108
workplans/HUB-WP-0005-core-hub-absorption-execution.md
Normal file
|
|
@ -0,0 +1,108 @@
|
||||||
|
---
|
||||||
|
id: HUB-WP-0005
|
||||||
|
type: workplan
|
||||||
|
title: "Core Hub absorption execution"
|
||||||
|
domain: infotech
|
||||||
|
repo: hub-core
|
||||||
|
status: active
|
||||||
|
owner: codex
|
||||||
|
topic_slug: custodian
|
||||||
|
created: "2026-08-21"
|
||||||
|
updated: "2026-08-21"
|
||||||
|
parent_project: prj-state-hub-retirement
|
||||||
|
parent_workplan: CORE-WP-0010
|
||||||
|
related:
|
||||||
|
- HUB-WP-0004
|
||||||
|
- CORE-WP-0010
|
||||||
|
- RAPPCOREHUB-WP-0002
|
||||||
|
---
|
||||||
|
|
||||||
|
# Core Hub absorption execution
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Implement the accepted S0–S6 plan in `docs/core-hub-absorption-plan.md`, move
|
||||||
|
Core Hub compatibility and durable framework authority into hub-core without
|
||||||
|
concurrent production writers, and retain a tested rollback until the final
|
||||||
|
stabilization gate closes.
|
||||||
|
|
||||||
|
## S0 durable runtime and audit boundary
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: HUB-WP-0005-T01
|
||||||
|
status: done
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Add hub-core-owned PostgreSQL models/migrations, a durable `PortStore`, audit
|
||||||
|
ledger, dependency-aware readiness, and production-safe lifecycle. The memory
|
||||||
|
backend remains local/conformance only.
|
||||||
|
|
||||||
|
## S0 compatibility shell and migration tooling
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: HUB-WP-0005-T02
|
||||||
|
status: done
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Add legacy bearer authentication, non-overlapping route/write group controls,
|
||||||
|
versioned import/export/delta bundles, idempotent imports, canonical count/hash
|
||||||
|
comparison, and safe evidence. Never serialize a raw key or secret.
|
||||||
|
|
||||||
|
## S1 system and discovery compatibility
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: HUB-WP-0005-T03
|
||||||
|
status: done
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Implement the health/readiness, catalog, OpenAPI/docs, and compatibility alias
|
||||||
|
surface with normalized response comparison against Core Hub.
|
||||||
|
|
||||||
|
## S2 and S3 registry and credential compatibility
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: HUB-WP-0005-T04
|
||||||
|
status: done
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Absorb hubs, manifests/activation, hub registry, API consumers, key issuance,
|
||||||
|
and token compatibility with stable identities, audit, fail-closed auth, and
|
||||||
|
reverse-delta support.
|
||||||
|
|
||||||
|
## S4 and S5 interaction and operator compatibility
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: HUB-WP-0005-T05
|
||||||
|
status: done
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Absorb widgets and interaction events through the named interaction port,
|
||||||
|
preserve explicitly empty deferred collections, and rebuild the operator
|
||||||
|
surface without reads from the legacy schema.
|
||||||
|
|
||||||
|
## S6 production cutover and retirement handoff
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: HUB-WP-0005-T06
|
||||||
|
status: wait
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Publish immutable artifacts, pass all conformance and consumer gates, support
|
||||||
|
the operator-approved production writer cutover, exercise rollback, and hand
|
||||||
|
the seven-day stabilization/archive residual to CORE-WP-0010. Wait until
|
||||||
|
RAPPCOREHUB-WP-0002 provides the private candidate and route controls.
|
||||||
|
|
||||||
|
## Acceptance
|
||||||
|
|
||||||
|
- [x] Production readiness requires durable PostgreSQL and auth dependencies
|
||||||
|
- [x] Import is idempotent; counts, identities, hashes, and provenance compare
|
||||||
|
- [x] All 22 checked-in Core Hub OpenAPI paths have governed dispositions
|
||||||
|
- [ ] ops-hub and activity-core gates pass on hub-core
|
||||||
|
- [ ] Production runs the immutable hub-core image with one writer per group
|
||||||
|
- [ ] Rollback is exercised and stabilization residual is recorded
|
||||||
Loading…
Add table
Add a link
Reference in a new issue