reuse-surface/specs/FederationHubAPI.md
tegwick d181043717
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 10s
ci / validate-registry (push) Has been cancelled
Build and Publish Container Image / build-and-push (push) Successful in 1m16s
REUSE-WP-0019-T04: reuse telemetry store and recording
Implements the hub side of the shared reuse-event schema (already drafted
in WP-0018-T01, schemas/reuse-event.schema.json): a SQLite reuse_events
table, POST /v1/reuse-events (token-auth), GET /v1/reuse-events?capability_id=
(read-only).

reuse_surface/plan_check.py: refactored record_outcome around a new
shared post_or_fallback_reuse_event() helper -- tries the hub first, falls
back to the local JSONL only on failure/unreachability, never both. New
record_manual_reuse_event() backs a new CLI command, reuse-surface
record-reuse, for retroactive facts recorded outside plan-check.

Privacy/scope (repo slugs and capability ids only, no code, no secrets) is
enforced structurally via the schema's additionalProperties: false, not
just by convention.

21 new pytest cases, 145 total pass. Live-verified against a real running
hub instance: POST/GET /v1/reuse-events directly, record-reuse and
plan-check --record-outcome both posting successfully to the hub, and --
after actually killing the hub process -- confirmed the fallback path
writes correctly to the local JSONL instead of erroring.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 22:32:19 +02:00

12 KiB

Federation Hub API

Repository: reuse-surface
Artifact: specs/FederationHubAPI.md
Status: Draft 0.1 (REUSE-WP-0011-T01)
Schema: schemas/hub-registration.schema.yaml


1. Purpose

The federation hub is a hosted coordination service that records which repositories publish capability indexes and serves a composed federated index for agent discovery. It does not store capability entry Markdown bodies.

Companion deployment workplans: railiance-apps RAILIANCE-WP-0007 (Helm release), RAILIANCE-WP-0008 (browser landing page).

Browser vs API routing

Production ingress (owned by railiance-apps) splits paths:

Path Handler
GET / (HTTPS) Static landing page (reuse-surface-landing) — humans only
GET /health, GET /v1/* Hub API (reuse-surface service)

The landing page does not implement registration or federation; clients and agents should use /health and /v1/* only. See railiance-apps/docs/reuse-surface-on-railiance01.md.


2. Base URL and formats

Item Value
Default production URL https://reuse.coulomb.social (A → 92.205.62.239)
API prefix /v1
Read formats JSON (default), YAML via Accept: application/yaml or ?format=yaml
Write content type application/json

Environment variables for clients:

Variable Purpose
REUSE_SURFACE_URL Service base URL (no trailing slash)
REUSE_SURFACE_TOKEN Bearer token for write operations

3. Authentication

Endpoint class Auth
GET /health, GET /v1/repos, GET /v1/repos/{repo}, GET /v1/federated Public (read)
POST /v1/repos, PATCH /v1/repos/{repo}, DELETE /v1/repos/{repo}, POST /v1/federated/compose Bearer token required

Write requests must include:

Authorization: Bearer <REUSE_SURFACE_TOKEN>

Missing or invalid token → 401 Unauthorized.


4. Registration model

A registration mirrors federation url sources from schemas/federation.schema.yaml, plus hub metadata:

Field Required Notes
repo yes Slug [a-z][a-z0-9-]*; primary key
url yes HTTP(S) URL to capabilities.yaml
enabled yes Include in federated compose when true
domain yes e.g. helix_forge
required no Fail compose if fetch fails and no cache
description no Human-readable note
cache_ttl_seconds no Default 86400
auth_env no Hub container env var for fetch auth (never returned in GET)
auth_header no Default Authorization
registered_at hub ISO-8601 UTC
updated_at hub ISO-8601 UTC
registered_by no Optional client-supplied actor label

Local filesystem index paths are not accepted — registrations must use published raw URLs.


5. Endpoints

5.1 GET /health

Liveness/readiness probe.

Response 200:

{
  "status": "ok",
  "service": "reuse-surface",
  "version": "0.1.0"
}

5.2 GET /v1/repos

List all registrations (including disabled).

Response 200:

{
  "count": 2,
  "repos": [
    {
      "repo": "reuse-surface",
      "url": "https://gitea.coulomb.social/coulomb/reuse-surface/raw/main/registry/indexes/capabilities.yaml",
      "enabled": true,
      "required": true,
      "domain": "helix_forge",
      "description": "Primary registry",
      "cache_ttl_seconds": 86400,
      "registered_at": "2026-06-15T12:00:00Z",
      "updated_at": "2026-06-15T12:00:00Z"
    }
  ]
}

auth_env is omitted from responses.

5.3 POST /v1/repos

Register a new repository. Auth required.

Request body: registration_request from schema.

Response 201: Full registration object.

Errors:

Code Condition
400 Schema validation failure
401 Missing/invalid token
409 repo already registered

5.4 GET /v1/repos/{repo}

Fetch one registration.

Response 200: Registration object.
Response 404: Unknown repo.

5.5 PATCH /v1/repos/{repo}

Update fields on an existing registration. Auth required.

Request body: registration_update from schema (at least one field).

Response 200: Updated registration.

Errors: 400, 401, 404.

5.6 DELETE /v1/repos/{repo}

Remove a registration. Auth required.

Response 204: No content.
Response 404: Unknown repo.

5.7 GET /v1/federated

Return the composed federated index from all enabled registrations.

Reuses WP-0010 remote fetch/cache logic server-side. Output shape matches registry/indexes/federated.yaml:

version: 1
updated: "2026-06-15"
domain: helix_forge
collision_policy: warn
sources:
  - repo: reuse-surface
    url: https://...
    count: 12
capabilities:
  - id: capability.registry.register
    source_repo: reuse-surface
    source_url: https://...
    # ... index fields ...
composed_at: "2026-07-07T16:22:09+00:00"  # REUSE-WP-0019-T02
stale: false                               # REUSE-WP-0019-T02

composed_at/stale (added REUSE-WP-0019-T02) track forced recomposes only (refresh=true, the webhook, or the scheduled fallback) — a plain GET still serves current best-effort data (per-source cache_ttl_seconds still applies) but never silently clears a staleness signal nothing actually refreshed. composed_at is null until the first forced recompose since the hub process's SQLite DB was created. stale: true means a registry/indexes/ change was pushed (via webhook) since the last forced recompose completed.

Query parameters:

Param Default Meaning
format json json or yaml
refresh false Bypass remote cache when true; also updates composed_at and clears stale

Warnings from compose (duplicate IDs, fetch fallbacks) are returned in response header X-Federation-Warnings (semicolon-separated) for MVP; JSON envelope extension is a future option. composed_at is also echoed as response header X-Composed-At when set.

Response 200: Federated index document.
Response 502: Required source unavailable with no cache.

5.8 POST /v1/federated/compose

Trigger federated index refresh (same as GET /v1/federated?refresh=true). Auth required. Useful for operators after bulk registration changes. This is the hub's recompose endpoint referenced elsewhere as "trigger a recompose" — there is no separate /v1/recompose route; this one already does that job.

Response 200: Federated index document (with composed_at updated, stale cleared).

5.9 POST /v1/webhooks/forgejo

Forgejo (or Gitea, during the transition) push-event webhook receiver. Added REUSE-WP-0019-T02. Design: webhook triggers, compose stays pull-based — the webhook never parses pushed file content into registry state; it only decides whether to trigger a pull-based recompose from the already-registered raw URLs.

Signature verification (required, not optional): HMAC-SHA256 over the raw request body, hex-encoded, in X-Forgejo-Signature (or X-Gitea-Signature — both accepted, since Forgejo is Gitea-compatible and repos migrate independently). Secret from REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET (never in code or committed config — route via the standard credential mechanism for this deployment). A missing/wrong signature is 401; an unconfigured secret is 503 (fails closed, not open).

Behavior:

  1. Verify signature (raw body, constant-time compare).
  2. Parse the push-event payload; check every commit's added/modified/removed lists for any path under registry/indexes/.
  3. If none match: 200 {"accepted": false, "reason": "no registry/indexes/ change"} — no-op.
  4. If a match: mark the compose state stale, run a real recompose (refresh=true equivalent) under the same lock used by POST /v1/federated/compose (concurrent triggers coalesce rather than overlap), record composed_at, clear stale.

Response 200: {"accepted": true|false, ...}.
Response 401: Missing or invalid signature.
Response 503: Webhook secret not configured.
Response 502: Recompose failed (stale is deliberately left set so the next check reports the failure honestly, not silently).

Org-level webhook rollout (single config covering all repos) and the Forgejo Actions scheduled fallback both shipped in T03: the org webhook is live on coulomb (id 1, push events only), and .forgejo/workflows/recompose-fallback.yaml in this repo calls POST /v1/federated/compose on a 6-hour cron as a backstop.

5.10 POST /v1/reuse-events

Records one reuse telemetry fact (REUSE-WP-0019-T04). Auth required. Body validated against schemas/reuse-event.schema.json — the same schema plan_check.py's local JSONL fallback uses, so a fact never differs in shape depending on where it landed.

{
  "ts": "2026-07-08T00:00:00Z",
  "consumer_repo": "some-repo",
  "capability_id": "capability.infotech.issue-tracking",
  "verdict": "reuse",
  "outcome": "reused",
  "source": "plan-check"
}

capability_id/outcome may be null (e.g. a new verdict with no existing capability). source is one of plan-check, manual, hub. additionalProperties: false — no code, no secrets, repo slugs and capability ids only (privacy/scope constraint).

Response 201: the recorded event.
Response 400: schema validation failed.
Response 401: missing/invalid bearer token.

5.11 GET /v1/reuse-events

Lists recorded reuse events, oldest first. No auth required (read-only, same posture as GET /v1/federated).

Query parameters:

Param Default Meaning
capability_id (none) Filter to one capability

Response 200: {"count": N, "events": [...]}.


6. Error envelope

Non-2xx responses use:

{
  "error": "validation_error",
  "message": "Human-readable summary",
  "details": ["optional field-level messages"]
}
error code HTTP
validation_error 400
unauthorized 401
not_found 404
conflict 409
misconfigured 503
compose_error 502

7. Hub service configuration

Env var Required Purpose
REUSE_SURFACE_TOKEN yes Write API bearer token
REUSE_SURFACE_DB no SQLite path (default /data/reuse.db)
REUSE_SURFACE_CACHE_DIR no Remote index cache (default /data/cache)
REUSE_SURFACE_DOMAIN no Default federated domain (default helix_forge)
REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET for webhook HMAC secret for POST /v1/webhooks/forgejo (REUSE-WP-0019-T02)
REUSE_SURFACE_FORGE_BASE_URL no Default target host for reuse-surface federation migrate-host (REUSE-WP-0019-T01), e.g. https://forgejo.coulomb.social

8. CLI mapping

CLI command API call
reuse-surface hub status GET /health
reuse-surface hub list GET /v1/repos
reuse-surface hub show --repo X GET /v1/repos/X
reuse-surface hub register ... POST /v1/repos
reuse-surface hub update ... PATCH /v1/repos/{repo}

Run locally: reuse-surface serve. Global client flags: --base-url, env REUSE_SURFACE_URL, REUSE_SURFACE_TOKEN.


9. Deployment reference

  • Image: gitea.coulomb.social/coulomb/reuse-surface:<tag>
  • Public URL: https://reuse.coulomb.social
  • Secret: reuse-surface-env with REUSE_SURFACE_TOKEN
  • Probe path: /health
  • Persistence: PVC at /data (SQLite + fetch cache)
  • Helm release: railiance-apps RAILIANCE-WP-0007
  • Landing page at /: railiance-apps RAILIANCE-WP-0008 (disable via landing.enabled: false in Helm values)