feat: expose repository navigation queries
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / pytest-smoke (push) Failing after 2s

Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a0230c-b06c-7641-808a-e191b6d1da49
This commit is contained in:
tegwick 2026-08-22 00:40:42 +02:00
parent 283bbf048e
commit e3e542f76c
9 changed files with 446 additions and 3 deletions

View file

@ -41,6 +41,6 @@
| task | HUB-WP-0006-T01 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T02 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T03 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T04 | todo | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T04 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T05 | todo | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T06 | wait | — | workplans/HUB-WP-0006-repository-classification-navigation.md |

View file

@ -33,17 +33,39 @@ the API does not auto-create tables.
| `port.events.progress` | `POST /ports/events/progress` | Accepts only cataloged progress-family events |
| `port.events.interaction` | `POST /ports/events/interaction` | Accepts only cataloged interaction-family events |
| `port.projection.query` | `GET /ports/projections/{id}` | Rebuildable registry/message/event projections with provenance |
| `port.projection.query` | `GET /ports/projections/repository-navigation/repositories` | Snapshot-bound cross-repository classification navigation |
Available projection ids are `hub_registry`, `messages`, `progress_events`, and
`interaction_events`. The two event families use distinct stores and cannot be
submitted through each other's endpoint.
Repository navigation consumes an injected `RepoProjectionClient` implementing
the `port.repo` page reader. When supplied to `create_app`, the client is
refreshed at startup; a host scheduler can call the same idempotent
`RepositoryNavigationService.refresh()` for later full or incremental
generations. The client owns transport and credential routing—host paths and
credentials never enter projection payloads. Migration
`0003_repository_navigation` stores active state, normalized repositories, and
derived facets atomically.
The repository query accepts repeated primary/secondary domain, category,
capability-tag, business-stake, and business-mechanic filters. Values within a
family are ORed and families are ANDed. Opaque cursors are bound to the active
content hash and normalized filters; a rebuild or changed filter returns 409.
Every response includes source snapshot/revision, checked/rebuilt times, and a
canonical content hash. MCP exposes the same surface through
`query_repository_navigation` and `get_repository_navigation_facet`. No
classification-write endpoint exists.
## Backend boundary and readiness
The app is created with an injected `PortStore`. `InMemoryPortStore` remains
available for deterministic tests and local contract smokes. Production uses
`PostgresPortStore`, whose registration, messaging, progress, interaction,
compatibility, import-lineage, and audit records are transactionally durable.
Repository navigation readiness is `ok`, `stale`, `unavailable`, or
`not_applicable` depending on whether a `port.repo` client is configured and a
valid generation has been accepted.
`GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503
when the active backend does not match `HUB_CORE_BACKEND`, or when the memory

View file

@ -27,6 +27,8 @@ CORE_TOOL_NAMES = frozenset({
"register_repo",
"update_repo_path",
"list_domain_repos",
"query_repository_navigation",
"get_repository_navigation_facet",
"check_repo_doi",
"get_doi_summary",
"register_service",
@ -298,6 +300,50 @@ class HubCoreMCPServer:
def list_domain_repos(domain_slug: str) -> str:
return self._json(self._get("/repos/", {"domain": domain_slug}))
@register("query_repository_navigation")
def query_repository_navigation(
primary_domain: list[str] | None = None,
secondary_domain: list[str] | None = None,
category: list[str] | None = None,
capability_tag: list[str] | None = None,
business_stake: list[str] | None = None,
business_mechanic: list[str] | None = None,
cursor: str | None = None,
limit: int = 100,
) -> str:
"""Navigate repositories across any combination of classification facets."""
return self._json(
self._get(
"/ports/projections/repository-navigation/repositories",
{
"primary_domain": primary_domain,
"secondary_domain": secondary_domain,
"category": category,
"capability_tag": capability_tag,
"business_stake": business_stake,
"business_mechanic": business_mechanic,
"cursor": cursor,
"limit": limit,
},
)
)
@register("get_repository_navigation_facet")
def get_repository_navigation_facet(
facet_kind: str,
facet_value: str,
cursor: str | None = None,
limit: int = 100,
) -> str:
"""Navigate one domain, category, tag, stake, or mechanic facet."""
return self._json(
self._get(
"/ports/projections/repository-navigation/"
f"facets/{facet_kind}/{facet_value}",
{"cursor": cursor, "limit": limit},
)
)
@register("check_repo_doi")
def check_repo_doi(repo_slug: str, force_refresh: bool = False) -> str:
return self._json(

View file

@ -14,6 +14,9 @@ from hub_core.runtime.repository_navigation import (
RepoProjectionClient,
RepositoryNavigationService,
)
from hub_core.runtime.repository_navigation_routes import (
create_repository_navigation_router,
)
from hub_core.runtime.store import InMemoryPortStore, PortStore
from hub_core.runtime.validation import ContractValidator
@ -88,6 +91,7 @@ def create_app(
)
app.include_router(create_ports_router())
app.include_router(create_repository_navigation_router())
app.include_router(create_compatibility_router())
return app

View file

@ -1,8 +1,10 @@
from __future__ import annotations
import asyncio
import base64
import hashlib
import json
import re
from collections.abc import Mapping
from copy import deepcopy
from dataclasses import dataclass
@ -42,6 +44,10 @@ class ProjectionRejected(ValueError):
"""The upstream transfer cannot safely update the active projection."""
class ProjectionCursorMismatch(ValueError):
"""A cursor does not belong to the active generation and normalized filters."""
@dataclass(frozen=True, slots=True)
class NavigationProjection:
projection_status: Literal["current", "stale"]
@ -134,6 +140,53 @@ class RepositoryNavigationService:
self._last_error = None
return result
async def query(
self,
*,
filters: Mapping[str, list[str] | None],
cursor: str | None = None,
limit: int = 100,
) -> dict[str, Any] | None:
if not 1 <= limit <= 500:
raise ValueError("limit must be between 1 and 500")
normalized_filters = _normalize_filters(filters)
projection = await self.store.get_repository_navigation()
if projection is None:
return None
filter_hash = _filter_hash(normalized_filters)
offset = (
_decode_cursor(
cursor,
content_hash=projection.content_hash,
filter_hash=filter_hash,
)
if cursor
else 0
)
matches = tuple(
repository
for repository in projection.repositories
if _matches(repository, normalized_filters)
)
if offset > len(matches):
raise ProjectionCursorMismatch("cursor offset exceeds the result set")
page = matches[offset : offset + limit]
next_offset = offset + len(page)
result = projection.to_contract()
result["repositories"] = deepcopy(list(page))
result["facets"] = deepcopy(list(_derive_facets(matches)))
result["next_cursor"] = (
_encode_cursor(
offset=next_offset,
content_hash=projection.content_hash,
filter_hash=filter_hash,
)
if next_offset < len(matches)
else None
)
result["total_repository_count"] = len(matches)
return result
async def _fetch_transfer(self) -> list[dict[str, Any]]:
assert self.client is not None
pages: list[dict[str, Any]] = []
@ -349,6 +402,107 @@ def _content_hash(repositories: tuple[dict[str, Any], ...]) -> str:
return hashlib.sha256(encoded).hexdigest()
_FILTER_FIELDS = {
"primary_domain": "primary_domain",
"secondary_domain": "secondary_domains",
"category": "category",
"capability_tag": "capability_tags",
"business_stake": "business_stake",
"business_mechanic": "business_mechanics",
}
def _normalize_filters(
filters: Mapping[str, list[str] | None],
) -> dict[str, tuple[str, ...]]:
unknown = set(filters) - set(_FILTER_FIELDS)
if unknown:
raise ValueError(f"unknown repository navigation filters: {', '.join(sorted(unknown))}")
result: dict[str, tuple[str, ...]] = {}
for name, values in filters.items():
if not values:
continue
normalized = tuple(sorted(set(values)))
if any(not _valid_facet_value(name, value) for value in normalized):
raise ValueError(f"invalid {name} filter value")
result[name] = normalized
return result
def _valid_facet_value(kind: str, value: str) -> bool:
controlled = {
"primary_domain": {
"infotech", "financials", "communication", "consumer", "health",
"industrials", "energy", "utilities", "materials", "realestate",
"crypto", "agents", "space", "government",
},
"secondary_domain": {
"infotech", "financials", "communication", "consumer", "health",
"industrials", "energy", "utilities", "materials", "realestate",
"crypto", "agents", "space", "government",
},
"category": {"experimental", "research", "project", "tooling", "product", "business"},
"business_stake": {
"execution", "intelligence", "finance", "legal", "sales", "experience",
"technology", "operations", "product", "people", "procurement",
"sustainability", "automation",
},
"business_mechanic": {"intention", "control", "coordination", "operation", "adaptation"},
}
if kind == "capability_tag":
return bool(
len(value) <= 120 and re.fullmatch(r"[a-z0-9]+(?:-[a-z0-9]+)*", value)
)
return value in controlled[kind]
def _matches(
repository: Mapping[str, Any], filters: Mapping[str, tuple[str, ...]]
) -> bool:
for name, accepted in filters.items():
value = repository[_FILTER_FIELDS[name]]
actual = {value} if isinstance(value, str) else set(value)
if not actual.intersection(accepted):
return False
return True
def _filter_hash(filters: Mapping[str, tuple[str, ...]]) -> str:
encoded = json.dumps(filters, sort_keys=True, separators=(",", ":")).encode()
return hashlib.sha256(encoded).hexdigest()
def _encode_cursor(*, offset: int, content_hash: str, filter_hash: str) -> str:
payload = json.dumps(
{"content_hash": content_hash, "filter_hash": filter_hash, "offset": offset},
sort_keys=True,
separators=(",", ":"),
).encode()
encoded = base64.urlsafe_b64encode(payload).decode().rstrip("=")
checksum = hashlib.sha256(b"repository-navigation/1.0.0:" + payload).hexdigest()
return f"{encoded}.{checksum}"
def _decode_cursor(cursor: str, *, content_hash: str, filter_hash: str) -> int:
try:
encoded, checksum = cursor.split(".", 1)
payload = base64.urlsafe_b64decode(encoded + "=" * (-len(encoded) % 4))
expected = hashlib.sha256(b"repository-navigation/1.0.0:" + payload).hexdigest()
if checksum != expected:
raise ValueError("checksum")
value = json.loads(payload)
if value["content_hash"] != content_hash or value["filter_hash"] != filter_hash:
raise ValueError("generation or filters")
offset = int(value["offset"])
if offset < 0:
raise ValueError("offset")
return offset
except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
raise ProjectionCursorMismatch(
"cursor_snapshot_mismatch: restart from the first page"
) from exc
def _parse_time(value: str) -> datetime:
return datetime.fromisoformat(value.replace("Z", "+00:00"))

View file

@ -0,0 +1,96 @@
from __future__ import annotations
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from hub_core.runtime.repository_navigation import (
ProjectionCursorMismatch,
RepositoryNavigationService,
)
def get_repository_navigation_service(request: Request) -> RepositoryNavigationService:
return request.app.state.repository_navigation
def create_repository_navigation_router() -> APIRouter:
router = APIRouter(prefix="/ports/projections/repository-navigation")
@router.get(
"/repositories",
response_model=dict[str, Any],
tags=["repository-navigation"],
openapi_extra={"x-port-id": "port.projection.query", "x-direction": "out"},
)
async def query_repositories(
primary_domain: list[str] | None = Query(default=None),
secondary_domain: list[str] | None = Query(default=None),
category: list[str] | None = Query(default=None),
capability_tag: list[str] | None = Query(default=None),
business_stake: list[str] | None = Query(default=None),
business_mechanic: list[str] | None = Query(default=None),
cursor: str | None = None,
limit: int = Query(default=100, ge=1, le=500),
service: RepositoryNavigationService = Depends(
get_repository_navigation_service
),
) -> dict[str, Any]:
return await _query(
service,
filters={
"primary_domain": primary_domain,
"secondary_domain": secondary_domain,
"category": category,
"capability_tag": capability_tag,
"business_stake": business_stake,
"business_mechanic": business_mechanic,
},
cursor=cursor,
limit=limit,
)
@router.get(
"/facets/{facet_kind}/{facet_value}",
response_model=dict[str, Any],
tags=["repository-navigation"],
openapi_extra={"x-port-id": "port.projection.query", "x-direction": "out"},
)
async def query_facet(
facet_kind: str,
facet_value: str,
cursor: str | None = None,
limit: int = Query(default=100, ge=1, le=500),
service: RepositoryNavigationService = Depends(
get_repository_navigation_service
),
) -> dict[str, Any]:
return await _query(
service,
filters={facet_kind: [facet_value]},
cursor=cursor,
limit=limit,
)
return router
async def _query(
service: RepositoryNavigationService,
*,
filters: dict[str, list[str] | None],
cursor: str | None,
limit: int,
) -> dict[str, Any]:
try:
result = await service.query(filters=filters, cursor=cursor, limit=limit)
except ProjectionCursorMismatch as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
if result is None:
raise HTTPException(
status_code=503,
detail="no accepted repository navigation projection is available",
)
return result

View file

@ -30,6 +30,8 @@ def test_mcp_base_server_registers_orientation_doi_and_fos10_tools() -> None:
"get_doi_summary",
"get_risks",
"get_alerts",
"query_repository_navigation",
"get_repository_navigation_facet",
} <= names
assert names == CORE_TOOL_NAMES
@ -49,3 +51,19 @@ def test_attach_to_host_mcp_respects_exclude() -> None:
assert "send_message" not in names
assert "get_domain_summary" in names
assert len(names) == len(CORE_TOOL_NAMES) - 2
def test_repository_navigation_mcp_tool_exposes_all_six_facets() -> None:
server = HubCoreMCPServer(name="test-hub", api_base="http://127.0.0.1:9999")
tools = {tool.name: tool for tool in asyncio.run(server.mcp.list_tools())}
schema = tools["query_repository_navigation"].parameters
assert {
"primary_domain",
"secondary_domain",
"category",
"capability_tag",
"business_stake",
"business_mechanic",
} <= set(schema["properties"])

View file

@ -7,6 +7,7 @@ from typing import Any
import pytest
from fastapi.testclient import TestClient
from jsonschema import Draft202012Validator, FormatChecker
from sqlalchemy.ext.asyncio import create_async_engine
from hub_core.contracts import repository_navigation_contract_root
@ -14,6 +15,7 @@ from hub_core.runtime.app import create_app
from hub_core.runtime.config import RuntimeSettings
from hub_core.runtime.postgres_store import PostgresPortStore
from hub_core.runtime.repository_navigation import (
ProjectionCursorMismatch,
ProjectionRejected,
RepositoryNavigationService,
)
@ -190,6 +192,97 @@ def test_injected_port_repo_client_refreshes_at_startup_and_controls_readiness()
assert response.json()["checks"]["repo_manager_projection"] == "ok"
def test_query_filters_are_ored_within_families_and_anded_across_them() -> None:
async def run() -> None:
store = InMemoryPortStore()
service = RepositoryNavigationService(
client=PageClient([fixture_page()]), store=store
)
await service.refresh()
result = await service.query(
filters={
"primary_domain": ["infotech", "agents"],
"capability_tag": ["repository-navigation"],
}
)
assert result is not None
assert [row["slug"] for row in result["repositories"]] == ["hub-core"]
assert result["source_snapshot"]["source_revision"]
assert result["content_hash"]
assert result["rebuilt_at"]
asyncio.run(run())
def test_query_cursor_is_stable_and_rejects_different_filters() -> None:
async def run() -> None:
store = InMemoryPortStore()
service = RepositoryNavigationService(
client=PageClient([fixture_page()]), store=store
)
await service.refresh()
first = await service.query(filters={}, limit=1)
assert first is not None and first["next_cursor"]
second = await service.query(
filters={}, cursor=first["next_cursor"], limit=1
)
assert second is not None
assert [row["slug"] for row in second["repositories"]] == ["repo-manager"]
with pytest.raises(ProjectionCursorMismatch, match="cursor_snapshot_mismatch"):
await service.query(
filters={"category": ["tooling"]},
cursor=first["next_cursor"],
limit=1,
)
asyncio.run(run())
def test_http_navigation_routes_are_read_only_and_return_contract_provenance() -> None:
settings = RuntimeSettings(
environment="test", backend="memory", allow_ephemeral=True
)
app = create_app(
settings=settings,
port_store=InMemoryPortStore(),
repo_projection_client=PageClient([fixture_page()]),
)
with TestClient(app) as runtime:
response = runtime.get(
"/ports/projections/repository-navigation/repositories",
params={"secondary_domain": "agents"},
)
facet = runtime.get(
"/ports/projections/repository-navigation/facets/business_mechanic/control"
)
document = runtime.get("/openapi.json").json()
assert response.status_code == 200
output_schema = json.loads(
repository_navigation_contract_root()
.joinpath("schemas", "repository-navigation-projection.schema.json")
.read_text(encoding="utf-8")
)
Draft202012Validator(
output_schema, format_checker=FormatChecker()
).validate(response.json())
assert [row["slug"] for row in response.json()["repositories"]] == ["hub-core"]
assert response.json()["source_snapshot"]["source_system"] == "repo-manager"
assert facet.status_code == 200
assert [row["slug"] for row in facet.json()["repositories"]] == ["repo-manager"]
navigation_paths = {
path: path_item
for path, path_item in document["paths"].items()
if path.startswith("/ports/projections/repository-navigation")
}
assert navigation_paths
assert all(set(path_item) == {"get"} for path_item in navigation_paths.values())
def test_failed_startup_refresh_keeps_api_up_for_diagnosis_but_not_ready() -> None:
settings = RuntimeSettings(
environment="test", backend="memory", allow_ephemeral=True

View file

@ -99,7 +99,7 @@ full rebuild, upsert, and deletion.
```task
id: HUB-WP-0006-T04
status: todo
status: done
priority: high
state_hub_task_id: "a1d85610-c3ea-4718-b394-ba4abcecf177"
```
@ -109,6 +109,16 @@ domain, category, capability tag, business stake, and business mechanic.
Responses carry source revision and rebuild provenance. No endpoint writes
classification authority.
Completed 2026-08-22. The read-only projection port now supports cross-facet
HTTP queries plus direct facet resolution for primary/secondary domain,
category, capability tag, business stake, and business mechanic. Filters OR
within a family and AND across families; snapshot/filter-bound cursors reject
reuse after rebuild with 409. Responses validate against the frozen output
schema and carry source revision, observed provenance, rebuild time, status,
and content hash. MCP tools `query_repository_navigation` and
`get_repository_navigation_facet` expose the same navigation fields. Runtime
OpenAPI inspection confirms that every navigation operation is GET-only.
## Prove conformance and failure behavior
```task
@ -142,5 +152,5 @@ expiry. Wait on T02-T05 and the State Hub cutover window.
- [x] Repo Manager v1.0 authority and projection contract accepted
- [x] Hub-side projection and compatibility alias contract versioned
- [ ] Rebuildable durable ingestion passes provenance and failure checks
- [ ] HTTP/MCP navigation reads only the derived projection
- [x] HTTP/MCP navigation reads only the derived projection
- [ ] A5/A4 consumer comparison and rollback evidence recorded