feat: expose repository navigation queries
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / pytest-smoke (push) Failing after 2s

Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a0230c-b06c-7641-808a-e191b6d1da49
This commit is contained in:
tegwick 2026-08-22 00:40:42 +02:00
parent 283bbf048e
commit e3e542f76c
9 changed files with 446 additions and 3 deletions

View file

@ -41,6 +41,6 @@
| task | HUB-WP-0006-T01 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md | | task | HUB-WP-0006-T01 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T02 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md | | task | HUB-WP-0006-T02 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T03 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md | | task | HUB-WP-0006-T03 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T04 | todo | — | workplans/HUB-WP-0006-repository-classification-navigation.md | | task | HUB-WP-0006-T04 | done | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T05 | todo | — | workplans/HUB-WP-0006-repository-classification-navigation.md | | task | HUB-WP-0006-T05 | todo | — | workplans/HUB-WP-0006-repository-classification-navigation.md |
| task | HUB-WP-0006-T06 | wait | — | workplans/HUB-WP-0006-repository-classification-navigation.md | | task | HUB-WP-0006-T06 | wait | — | workplans/HUB-WP-0006-repository-classification-navigation.md |

View file

@ -33,17 +33,39 @@ the API does not auto-create tables.
| `port.events.progress` | `POST /ports/events/progress` | Accepts only cataloged progress-family events | | `port.events.progress` | `POST /ports/events/progress` | Accepts only cataloged progress-family events |
| `port.events.interaction` | `POST /ports/events/interaction` | Accepts only cataloged interaction-family events | | `port.events.interaction` | `POST /ports/events/interaction` | Accepts only cataloged interaction-family events |
| `port.projection.query` | `GET /ports/projections/{id}` | Rebuildable registry/message/event projections with provenance | | `port.projection.query` | `GET /ports/projections/{id}` | Rebuildable registry/message/event projections with provenance |
| `port.projection.query` | `GET /ports/projections/repository-navigation/repositories` | Snapshot-bound cross-repository classification navigation |
Available projection ids are `hub_registry`, `messages`, `progress_events`, and Available projection ids are `hub_registry`, `messages`, `progress_events`, and
`interaction_events`. The two event families use distinct stores and cannot be `interaction_events`. The two event families use distinct stores and cannot be
submitted through each other's endpoint. submitted through each other's endpoint.
Repository navigation consumes an injected `RepoProjectionClient` implementing
the `port.repo` page reader. When supplied to `create_app`, the client is
refreshed at startup; a host scheduler can call the same idempotent
`RepositoryNavigationService.refresh()` for later full or incremental
generations. The client owns transport and credential routing—host paths and
credentials never enter projection payloads. Migration
`0003_repository_navigation` stores active state, normalized repositories, and
derived facets atomically.
The repository query accepts repeated primary/secondary domain, category,
capability-tag, business-stake, and business-mechanic filters. Values within a
family are ORed and families are ANDed. Opaque cursors are bound to the active
content hash and normalized filters; a rebuild or changed filter returns 409.
Every response includes source snapshot/revision, checked/rebuilt times, and a
canonical content hash. MCP exposes the same surface through
`query_repository_navigation` and `get_repository_navigation_facet`. No
classification-write endpoint exists.
## Backend boundary and readiness ## Backend boundary and readiness
The app is created with an injected `PortStore`. `InMemoryPortStore` remains The app is created with an injected `PortStore`. `InMemoryPortStore` remains
available for deterministic tests and local contract smokes. Production uses available for deterministic tests and local contract smokes. Production uses
`PostgresPortStore`, whose registration, messaging, progress, interaction, `PostgresPortStore`, whose registration, messaging, progress, interaction,
compatibility, import-lineage, and audit records are transactionally durable. compatibility, import-lineage, and audit records are transactionally durable.
Repository navigation readiness is `ok`, `stale`, `unavailable`, or
`not_applicable` depending on whether a `port.repo` client is configured and a
valid generation has been accepted.
`GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503 `GET /healthz` proves the process is alive. `GET /readyz` fails with HTTP 503
when the active backend does not match `HUB_CORE_BACKEND`, or when the memory when the active backend does not match `HUB_CORE_BACKEND`, or when the memory

View file

@ -27,6 +27,8 @@ CORE_TOOL_NAMES = frozenset({
"register_repo", "register_repo",
"update_repo_path", "update_repo_path",
"list_domain_repos", "list_domain_repos",
"query_repository_navigation",
"get_repository_navigation_facet",
"check_repo_doi", "check_repo_doi",
"get_doi_summary", "get_doi_summary",
"register_service", "register_service",
@ -298,6 +300,50 @@ class HubCoreMCPServer:
def list_domain_repos(domain_slug: str) -> str: def list_domain_repos(domain_slug: str) -> str:
return self._json(self._get("/repos/", {"domain": domain_slug})) return self._json(self._get("/repos/", {"domain": domain_slug}))
@register("query_repository_navigation")
def query_repository_navigation(
primary_domain: list[str] | None = None,
secondary_domain: list[str] | None = None,
category: list[str] | None = None,
capability_tag: list[str] | None = None,
business_stake: list[str] | None = None,
business_mechanic: list[str] | None = None,
cursor: str | None = None,
limit: int = 100,
) -> str:
"""Navigate repositories across any combination of classification facets."""
return self._json(
self._get(
"/ports/projections/repository-navigation/repositories",
{
"primary_domain": primary_domain,
"secondary_domain": secondary_domain,
"category": category,
"capability_tag": capability_tag,
"business_stake": business_stake,
"business_mechanic": business_mechanic,
"cursor": cursor,
"limit": limit,
},
)
)
@register("get_repository_navigation_facet")
def get_repository_navigation_facet(
facet_kind: str,
facet_value: str,
cursor: str | None = None,
limit: int = 100,
) -> str:
"""Navigate one domain, category, tag, stake, or mechanic facet."""
return self._json(
self._get(
"/ports/projections/repository-navigation/"
f"facets/{facet_kind}/{facet_value}",
{"cursor": cursor, "limit": limit},
)
)
@register("check_repo_doi") @register("check_repo_doi")
def check_repo_doi(repo_slug: str, force_refresh: bool = False) -> str: def check_repo_doi(repo_slug: str, force_refresh: bool = False) -> str:
return self._json( return self._json(

View file

@ -14,6 +14,9 @@ from hub_core.runtime.repository_navigation import (
RepoProjectionClient, RepoProjectionClient,
RepositoryNavigationService, RepositoryNavigationService,
) )
from hub_core.runtime.repository_navigation_routes import (
create_repository_navigation_router,
)
from hub_core.runtime.store import InMemoryPortStore, PortStore from hub_core.runtime.store import InMemoryPortStore, PortStore
from hub_core.runtime.validation import ContractValidator from hub_core.runtime.validation import ContractValidator
@ -88,6 +91,7 @@ def create_app(
) )
app.include_router(create_ports_router()) app.include_router(create_ports_router())
app.include_router(create_repository_navigation_router())
app.include_router(create_compatibility_router()) app.include_router(create_compatibility_router())
return app return app

View file

@ -1,8 +1,10 @@
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import base64
import hashlib import hashlib
import json import json
import re
from collections.abc import Mapping from collections.abc import Mapping
from copy import deepcopy from copy import deepcopy
from dataclasses import dataclass from dataclasses import dataclass
@ -42,6 +44,10 @@ class ProjectionRejected(ValueError):
"""The upstream transfer cannot safely update the active projection.""" """The upstream transfer cannot safely update the active projection."""
class ProjectionCursorMismatch(ValueError):
"""A cursor does not belong to the active generation and normalized filters."""
@dataclass(frozen=True, slots=True) @dataclass(frozen=True, slots=True)
class NavigationProjection: class NavigationProjection:
projection_status: Literal["current", "stale"] projection_status: Literal["current", "stale"]
@ -134,6 +140,53 @@ class RepositoryNavigationService:
self._last_error = None self._last_error = None
return result return result
async def query(
self,
*,
filters: Mapping[str, list[str] | None],
cursor: str | None = None,
limit: int = 100,
) -> dict[str, Any] | None:
if not 1 <= limit <= 500:
raise ValueError("limit must be between 1 and 500")
normalized_filters = _normalize_filters(filters)
projection = await self.store.get_repository_navigation()
if projection is None:
return None
filter_hash = _filter_hash(normalized_filters)
offset = (
_decode_cursor(
cursor,
content_hash=projection.content_hash,
filter_hash=filter_hash,
)
if cursor
else 0
)
matches = tuple(
repository
for repository in projection.repositories
if _matches(repository, normalized_filters)
)
if offset > len(matches):
raise ProjectionCursorMismatch("cursor offset exceeds the result set")
page = matches[offset : offset + limit]
next_offset = offset + len(page)
result = projection.to_contract()
result["repositories"] = deepcopy(list(page))
result["facets"] = deepcopy(list(_derive_facets(matches)))
result["next_cursor"] = (
_encode_cursor(
offset=next_offset,
content_hash=projection.content_hash,
filter_hash=filter_hash,
)
if next_offset < len(matches)
else None
)
result["total_repository_count"] = len(matches)
return result
async def _fetch_transfer(self) -> list[dict[str, Any]]: async def _fetch_transfer(self) -> list[dict[str, Any]]:
assert self.client is not None assert self.client is not None
pages: list[dict[str, Any]] = [] pages: list[dict[str, Any]] = []
@ -349,6 +402,107 @@ def _content_hash(repositories: tuple[dict[str, Any], ...]) -> str:
return hashlib.sha256(encoded).hexdigest() return hashlib.sha256(encoded).hexdigest()
_FILTER_FIELDS = {
"primary_domain": "primary_domain",
"secondary_domain": "secondary_domains",
"category": "category",
"capability_tag": "capability_tags",
"business_stake": "business_stake",
"business_mechanic": "business_mechanics",
}
def _normalize_filters(
filters: Mapping[str, list[str] | None],
) -> dict[str, tuple[str, ...]]:
unknown = set(filters) - set(_FILTER_FIELDS)
if unknown:
raise ValueError(f"unknown repository navigation filters: {', '.join(sorted(unknown))}")
result: dict[str, tuple[str, ...]] = {}
for name, values in filters.items():
if not values:
continue
normalized = tuple(sorted(set(values)))
if any(not _valid_facet_value(name, value) for value in normalized):
raise ValueError(f"invalid {name} filter value")
result[name] = normalized
return result
def _valid_facet_value(kind: str, value: str) -> bool:
controlled = {
"primary_domain": {
"infotech", "financials", "communication", "consumer", "health",
"industrials", "energy", "utilities", "materials", "realestate",
"crypto", "agents", "space", "government",
},
"secondary_domain": {
"infotech", "financials", "communication", "consumer", "health",
"industrials", "energy", "utilities", "materials", "realestate",
"crypto", "agents", "space", "government",
},
"category": {"experimental", "research", "project", "tooling", "product", "business"},
"business_stake": {
"execution", "intelligence", "finance", "legal", "sales", "experience",
"technology", "operations", "product", "people", "procurement",
"sustainability", "automation",
},
"business_mechanic": {"intention", "control", "coordination", "operation", "adaptation"},
}
if kind == "capability_tag":
return bool(
len(value) <= 120 and re.fullmatch(r"[a-z0-9]+(?:-[a-z0-9]+)*", value)
)
return value in controlled[kind]
def _matches(
repository: Mapping[str, Any], filters: Mapping[str, tuple[str, ...]]
) -> bool:
for name, accepted in filters.items():
value = repository[_FILTER_FIELDS[name]]
actual = {value} if isinstance(value, str) else set(value)
if not actual.intersection(accepted):
return False
return True
def _filter_hash(filters: Mapping[str, tuple[str, ...]]) -> str:
encoded = json.dumps(filters, sort_keys=True, separators=(",", ":")).encode()
return hashlib.sha256(encoded).hexdigest()
def _encode_cursor(*, offset: int, content_hash: str, filter_hash: str) -> str:
payload = json.dumps(
{"content_hash": content_hash, "filter_hash": filter_hash, "offset": offset},
sort_keys=True,
separators=(",", ":"),
).encode()
encoded = base64.urlsafe_b64encode(payload).decode().rstrip("=")
checksum = hashlib.sha256(b"repository-navigation/1.0.0:" + payload).hexdigest()
return f"{encoded}.{checksum}"
def _decode_cursor(cursor: str, *, content_hash: str, filter_hash: str) -> int:
try:
encoded, checksum = cursor.split(".", 1)
payload = base64.urlsafe_b64decode(encoded + "=" * (-len(encoded) % 4))
expected = hashlib.sha256(b"repository-navigation/1.0.0:" + payload).hexdigest()
if checksum != expected:
raise ValueError("checksum")
value = json.loads(payload)
if value["content_hash"] != content_hash or value["filter_hash"] != filter_hash:
raise ValueError("generation or filters")
offset = int(value["offset"])
if offset < 0:
raise ValueError("offset")
return offset
except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
raise ProjectionCursorMismatch(
"cursor_snapshot_mismatch: restart from the first page"
) from exc
def _parse_time(value: str) -> datetime: def _parse_time(value: str) -> datetime:
return datetime.fromisoformat(value.replace("Z", "+00:00")) return datetime.fromisoformat(value.replace("Z", "+00:00"))

View file

@ -0,0 +1,96 @@
from __future__ import annotations
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from hub_core.runtime.repository_navigation import (
ProjectionCursorMismatch,
RepositoryNavigationService,
)
def get_repository_navigation_service(request: Request) -> RepositoryNavigationService:
return request.app.state.repository_navigation
def create_repository_navigation_router() -> APIRouter:
router = APIRouter(prefix="/ports/projections/repository-navigation")
@router.get(
"/repositories",
response_model=dict[str, Any],
tags=["repository-navigation"],
openapi_extra={"x-port-id": "port.projection.query", "x-direction": "out"},
)
async def query_repositories(
primary_domain: list[str] | None = Query(default=None),
secondary_domain: list[str] | None = Query(default=None),
category: list[str] | None = Query(default=None),
capability_tag: list[str] | None = Query(default=None),
business_stake: list[str] | None = Query(default=None),
business_mechanic: list[str] | None = Query(default=None),
cursor: str | None = None,
limit: int = Query(default=100, ge=1, le=500),
service: RepositoryNavigationService = Depends(
get_repository_navigation_service
),
) -> dict[str, Any]:
return await _query(
service,
filters={
"primary_domain": primary_domain,
"secondary_domain": secondary_domain,
"category": category,
"capability_tag": capability_tag,
"business_stake": business_stake,
"business_mechanic": business_mechanic,
},
cursor=cursor,
limit=limit,
)
@router.get(
"/facets/{facet_kind}/{facet_value}",
response_model=dict[str, Any],
tags=["repository-navigation"],
openapi_extra={"x-port-id": "port.projection.query", "x-direction": "out"},
)
async def query_facet(
facet_kind: str,
facet_value: str,
cursor: str | None = None,
limit: int = Query(default=100, ge=1, le=500),
service: RepositoryNavigationService = Depends(
get_repository_navigation_service
),
) -> dict[str, Any]:
return await _query(
service,
filters={facet_kind: [facet_value]},
cursor=cursor,
limit=limit,
)
return router
async def _query(
service: RepositoryNavigationService,
*,
filters: dict[str, list[str] | None],
cursor: str | None,
limit: int,
) -> dict[str, Any]:
try:
result = await service.query(filters=filters, cursor=cursor, limit=limit)
except ProjectionCursorMismatch as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
if result is None:
raise HTTPException(
status_code=503,
detail="no accepted repository navigation projection is available",
)
return result

View file

@ -30,6 +30,8 @@ def test_mcp_base_server_registers_orientation_doi_and_fos10_tools() -> None:
"get_doi_summary", "get_doi_summary",
"get_risks", "get_risks",
"get_alerts", "get_alerts",
"query_repository_navigation",
"get_repository_navigation_facet",
} <= names } <= names
assert names == CORE_TOOL_NAMES assert names == CORE_TOOL_NAMES
@ -49,3 +51,19 @@ def test_attach_to_host_mcp_respects_exclude() -> None:
assert "send_message" not in names assert "send_message" not in names
assert "get_domain_summary" in names assert "get_domain_summary" in names
assert len(names) == len(CORE_TOOL_NAMES) - 2 assert len(names) == len(CORE_TOOL_NAMES) - 2
def test_repository_navigation_mcp_tool_exposes_all_six_facets() -> None:
server = HubCoreMCPServer(name="test-hub", api_base="http://127.0.0.1:9999")
tools = {tool.name: tool for tool in asyncio.run(server.mcp.list_tools())}
schema = tools["query_repository_navigation"].parameters
assert {
"primary_domain",
"secondary_domain",
"category",
"capability_tag",
"business_stake",
"business_mechanic",
} <= set(schema["properties"])

View file

@ -7,6 +7,7 @@ from typing import Any
import pytest import pytest
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
from jsonschema import Draft202012Validator, FormatChecker
from sqlalchemy.ext.asyncio import create_async_engine from sqlalchemy.ext.asyncio import create_async_engine
from hub_core.contracts import repository_navigation_contract_root from hub_core.contracts import repository_navigation_contract_root
@ -14,6 +15,7 @@ from hub_core.runtime.app import create_app
from hub_core.runtime.config import RuntimeSettings from hub_core.runtime.config import RuntimeSettings
from hub_core.runtime.postgres_store import PostgresPortStore from hub_core.runtime.postgres_store import PostgresPortStore
from hub_core.runtime.repository_navigation import ( from hub_core.runtime.repository_navigation import (
ProjectionCursorMismatch,
ProjectionRejected, ProjectionRejected,
RepositoryNavigationService, RepositoryNavigationService,
) )
@ -190,6 +192,97 @@ def test_injected_port_repo_client_refreshes_at_startup_and_controls_readiness()
assert response.json()["checks"]["repo_manager_projection"] == "ok" assert response.json()["checks"]["repo_manager_projection"] == "ok"
def test_query_filters_are_ored_within_families_and_anded_across_them() -> None:
async def run() -> None:
store = InMemoryPortStore()
service = RepositoryNavigationService(
client=PageClient([fixture_page()]), store=store
)
await service.refresh()
result = await service.query(
filters={
"primary_domain": ["infotech", "agents"],
"capability_tag": ["repository-navigation"],
}
)
assert result is not None
assert [row["slug"] for row in result["repositories"]] == ["hub-core"]
assert result["source_snapshot"]["source_revision"]
assert result["content_hash"]
assert result["rebuilt_at"]
asyncio.run(run())
def test_query_cursor_is_stable_and_rejects_different_filters() -> None:
async def run() -> None:
store = InMemoryPortStore()
service = RepositoryNavigationService(
client=PageClient([fixture_page()]), store=store
)
await service.refresh()
first = await service.query(filters={}, limit=1)
assert first is not None and first["next_cursor"]
second = await service.query(
filters={}, cursor=first["next_cursor"], limit=1
)
assert second is not None
assert [row["slug"] for row in second["repositories"]] == ["repo-manager"]
with pytest.raises(ProjectionCursorMismatch, match="cursor_snapshot_mismatch"):
await service.query(
filters={"category": ["tooling"]},
cursor=first["next_cursor"],
limit=1,
)
asyncio.run(run())
def test_http_navigation_routes_are_read_only_and_return_contract_provenance() -> None:
settings = RuntimeSettings(
environment="test", backend="memory", allow_ephemeral=True
)
app = create_app(
settings=settings,
port_store=InMemoryPortStore(),
repo_projection_client=PageClient([fixture_page()]),
)
with TestClient(app) as runtime:
response = runtime.get(
"/ports/projections/repository-navigation/repositories",
params={"secondary_domain": "agents"},
)
facet = runtime.get(
"/ports/projections/repository-navigation/facets/business_mechanic/control"
)
document = runtime.get("/openapi.json").json()
assert response.status_code == 200
output_schema = json.loads(
repository_navigation_contract_root()
.joinpath("schemas", "repository-navigation-projection.schema.json")
.read_text(encoding="utf-8")
)
Draft202012Validator(
output_schema, format_checker=FormatChecker()
).validate(response.json())
assert [row["slug"] for row in response.json()["repositories"]] == ["hub-core"]
assert response.json()["source_snapshot"]["source_system"] == "repo-manager"
assert facet.status_code == 200
assert [row["slug"] for row in facet.json()["repositories"]] == ["repo-manager"]
navigation_paths = {
path: path_item
for path, path_item in document["paths"].items()
if path.startswith("/ports/projections/repository-navigation")
}
assert navigation_paths
assert all(set(path_item) == {"get"} for path_item in navigation_paths.values())
def test_failed_startup_refresh_keeps_api_up_for_diagnosis_but_not_ready() -> None: def test_failed_startup_refresh_keeps_api_up_for_diagnosis_but_not_ready() -> None:
settings = RuntimeSettings( settings = RuntimeSettings(
environment="test", backend="memory", allow_ephemeral=True environment="test", backend="memory", allow_ephemeral=True

View file

@ -99,7 +99,7 @@ full rebuild, upsert, and deletion.
```task ```task
id: HUB-WP-0006-T04 id: HUB-WP-0006-T04
status: todo status: done
priority: high priority: high
state_hub_task_id: "a1d85610-c3ea-4718-b394-ba4abcecf177" state_hub_task_id: "a1d85610-c3ea-4718-b394-ba4abcecf177"
``` ```
@ -109,6 +109,16 @@ domain, category, capability tag, business stake, and business mechanic.
Responses carry source revision and rebuild provenance. No endpoint writes Responses carry source revision and rebuild provenance. No endpoint writes
classification authority. classification authority.
Completed 2026-08-22. The read-only projection port now supports cross-facet
HTTP queries plus direct facet resolution for primary/secondary domain,
category, capability tag, business stake, and business mechanic. Filters OR
within a family and AND across families; snapshot/filter-bound cursors reject
reuse after rebuild with 409. Responses validate against the frozen output
schema and carry source revision, observed provenance, rebuild time, status,
and content hash. MCP tools `query_repository_navigation` and
`get_repository_navigation_facet` expose the same navigation fields. Runtime
OpenAPI inspection confirms that every navigation operation is GET-only.
## Prove conformance and failure behavior ## Prove conformance and failure behavior
```task ```task
@ -142,5 +152,5 @@ expiry. Wait on T02-T05 and the State Hub cutover window.
- [x] Repo Manager v1.0 authority and projection contract accepted - [x] Repo Manager v1.0 authority and projection contract accepted
- [x] Hub-side projection and compatibility alias contract versioned - [x] Hub-side projection and compatibility alias contract versioned
- [ ] Rebuildable durable ingestion passes provenance and failure checks - [ ] Rebuildable durable ingestion passes provenance and failure checks
- [ ] HTTP/MCP navigation reads only the derived projection - [x] HTTP/MCP navigation reads only the derived projection
- [ ] A5/A4 consumer comparison and rollback evidence recorded - [ ] A5/A4 consumer comparison and rollback evidence recorded