reuse-surface/workplans/REUSE-WP-0019-forgejo-automation-and-telemetry.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id	reuse_check
REUSE-WP-0019	workplan	Forgejo-native federation automation and reuse telemetry	infotech	reuse-surface	active	claude-code	helix-forge	2026-07-06	2026-07-07	569be717-34f8-4039-bb26-497685f60159	new — dogfooded 2026-07-07 via reuse-surface plan-check against the full 61-capability federated index; no existing capability covers Forgejo webhook automation or reuse telemetry

Forgejo-native federation automation and reuse telemetry

Federation compose is on-demand and the hub serves whatever was last composed; roster sweeps are manual; reliability evidence is structural (CI exists) rather than observed (someone reused it and it worked). This workplan makes the registry live (event-driven recompose) and evidence-backed (reuse telemetry feeding the R axis).

Platform constraint: the forge is transitioning Gitea → Forgejo. All new automation attaches to Forgejo (webhooks, Forgejo Actions, API tokens) — nothing new is built against Gitea. Existing raw URLs (https://gitea.coulomb.social/...) and .gitea/workflows/ must migrate or be made host-agnostic. Forgejo is Gitea-API-compatible, so migration is mostly host/path configuration, but every hardcoded gitea. reference is a liability.

Depends on: REUSE-WP-0017 (content worth refreshing), REUSE-WP-0018-T01 telemetry schema (shared). Closes SCOPE "not possible yet" item automatic hub refresh and moves reliability evidence beyond structural.

Design principles

1. Host-agnostic first — a single forge_base_url configuration (env/config + hub setting) replaces hardcoded hosts; the Forgejo cutover becomes a one-line change per surface.
2. Webhook triggers, compose stays pull-based — the webhook only marks the hub's composed index stale and triggers recompose from published raw URLs; no push-parsing of payloads into registry state.
3. Degrade to schedule — if webhooks are unavailable, a scheduled Forgejo Actions job recomposes on an interval; freshness is monitored either way.
4. Telemetry is append-only and low-ceremony — reuse events are JSONL facts (who consumed what, when, outcome); aggregation derives reused_by relations and R-axis evidence citations, never hand-edited.

Dependencies

Dependency	Owner	Notes
Forgejo instance + admin	Bernd / infra	webhook config, org-level token, Actions runners
Gitea→Forgejo cutover plan	infra	final hostname, raw URL scheme, redirect window
Hub deployment (reuse.coulomb.social)	reuse-surface / railiance	new endpoint + config rollout
REUSE-WP-0018-T01	reuse-surface	shared telemetry/outcome schema
plan-check adoption	ecosystem	telemetry volume comes from WP-0018-T05 rollout

---

Forge Host Abstraction And URL Migration Inventory

id: REUSE-WP-0019-T01
status: done
priority: high
state_hub_task_id: "4a187b56-bff9-4097-abd0-b423e7bf9442"

Inventory findings (2026-07-07): cross-referenced sources.yaml (61 entries) against each repo's actual git origin remote. Result: 11 repos already migrated their remote to Forgejo; 50 still on Gitea (correctly, no action needed). Of the 11 Forgejo-origin repos, 9 already had correct sources.yaml entries (from WP-0017 drafting/registration); 2 were stale — activity-core and state-hub — still pointing at their old Gitea raw URL despite having migrated. This was real, live debt, not a hypothetical: both were confirmed reachable on Forgejo (HTTP 200) before being migrated for real, in both sources.yaml and the production hub registration (hub update --url). Verified post-migration against GET /v1/federated — both now show forgejo.coulomb.social.

config-atlas's WP-0017-T06 303 was not a host-transition symptom — already diagnosed there as (a) a redirect the current code already follows fine, and (b) a hub-registration gap unrelated to host. No new finding here.

Implemented:

• reuse_surface/forge_host.py: parse_raw_url/derive_raw_url (handles both the legacy /raw/<branch>/... form and the canonical /raw/branch/<branch>/... form both forges serve without a 303 redirect), rewrite_url_host, forge_base_url (reads REUSE_SURFACE_FORGE_BASE_URL, no hard default since migration is opt-in per repo), migrate_source_host (verifies the new URL resolves via HTTP HEAD before writing — refuses to point a repo at a host it hasn't actually migrated to)
• CLI: reuse-surface federation migrate-host --repo <slug> [--repo ...] --to <base-url> [--from <sanity-check>] [--dry-run] [--no-verify] [--update-hub] — used for real on activity-core/state-hub
• Fixed two host-agnostic code gaps found while inventorying: registry_update.py's SAFE_EVIDENCE_PREFIXES and maintain_llm.py's git-diff pathspec only recognized .gitea/workflows/, missing repos already on .forgejo/workflows/ — both now recognize either
• Fixed stale copy-paste examples in docs/RegistryFederation.md (state-hub's old Gitea URL, now genuinely wrong post-migration) and a pre-existing, unrelated port typo (8088 vs the real llm-connect default 8080) in tools/README.md/registry/README.md, discovered and confirmed live during REUSE-WP-0018-T03
• 17 new pytest cases (tests/test_forge_host.py); 106 total pass
• Recomposed federated.yaml post-migration: still 61 capabilities, no loss

Hub Recompose Endpoint And Webhook Receiver

id: REUSE-WP-0019-T02
status: done
priority: high
state_hub_task_id: "691eb32a-6f20-4a2a-b9ff-0ae427b659aa"

Note: POST /v1/federated/compose (token-auth, triggers a real recompose) already existed from earlier hub work — no separate /v1/recompose route was added; the spec now documents this explicitly rather than duplicating a route that already does the job.

Implemented in reuse_surface/hub/:

• store.py: compose_state table (composed_at, stale), with record_compose()/mark_stale()/get_compose_state(). composed_at updates and stale clears only on a forced recompose (refresh=true, webhook, or future scheduled fallback) — a plain GET still serves current best-effort data but never silently reports itself as freshly composed
• webhooks.py: verify_signature (constant-time HMAC-SHA256, fails closed on empty secret), push_touches_registry_index (path-only inspection of the push payload's added/modified/removed lists — never parses file content, per design principle 2)
• app.py: POST /v1/webhooks/forgejo (accepts both X-Forgejo-Signature and X-Gitea-Signature, since repos migrate independently); GET /v1/federated and POST /v1/federated/compose now share an asyncio.Lock so concurrent recompose triggers (manual, webhook, future scheduled) coalesce instead of overlapping
• specs/FederationHubAPI.md extended (§5.7-5.9, config table, error codes)
• 28 new pytest cases (16 in test_hub.py, 12 in test_webhooks.py); 128 total pass
• Live-verified: ran the actual hub service locally (reuse-surface serve), sent a real HMAC-signed webhook payload over HTTP — confirmed composed_at/stale transitions, signature rejection, irrelevant-path no-op, and the full webhook-to-recompose path end to end

Deployed 2026-07-07 (explicit user sign-off "Deploy to live please"): built and pushed gitea.coulomb.social/coulomb/reuse-surface:e3ae22e, smoke-tested it locally in a standalone container first, then helm upgrade via railiance-apps (make reuse-deploy, pinned in helm/reuse-surface-values.yaml, commits a2c0da1/bcb05f5). Live-verified against https://reuse.coulomb.social: /v1/federated (200, 61 capabilities, composed_at/stale fields present), /v1/repos (200), a real forced recompose via POST /v1/federated/compose (token-auth via warden access --exec, token never printed), and the webhook endpoint correctly failing closed with 503 (REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET not yet configured).

Webhook secret added 2026-07-07 (explicit user sign-off "Go ahead and add the webhook secret"): generated a fresh 32-byte hex secret with openssl rand -hex 32, patched it into the live K8s Secret reuse-surface-env in namespace reuse, restarted the deployment (values injected via envFrom.secretRef, not picked up without a restart) — pod rolled cleanly (1/1 Ready, 0 restarts). The raw secret value was never printed to the transcript: generated and applied in one non-echoing shell step, and re-fetched only inside an exported env var for the live signature-verification test below, then unset. Live-verified against production: a correctly HMAC-signed push payload is now accepted ({"accepted": false, "reason": "no registry/indexes/ change"} for a payload that doesn't touch registry/indexes/, as designed), and a bad signature is still rejected with 401. /v1/federated and /v1/repos unaffected by the restart.

Found (not fixed) while smoke-testing: the public ingress's exact-path /health rule 404s at the Traefik edge (shadowed by the catch-all / rule to the landing page) — confirmed ingress-layer only, not a pod/service problem (/health works via direct port-forward; the Deployment's own readiness/liveness probes pass, pod is 1/1 Ready). /v1/* unaffected. Flagged to railiance-apps via State Hub message and documented in railiance-apps commit bcb05f5 — not fixed here since it's a shared production ingress template edit outside this workplan's scope.

Forgejo Webhook Rollout And Scheduled Fallback

id: REUSE-WP-0019-T03
status: done
priority: medium
state_hub_task_id: "aa9e9f80-b878-490c-832e-515d8cbbbb60"

Org-level Forgejo webhook — done by railiance-apps (2026-07-07), credited not claimed: after I flagged the /health ingress bug to railiance-apps via State Hub message, they picked up this piece too: tools/reuse-forgejo-webhook.sh + make reuse-forgejo-webhook (idempotent, reads the same REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET I put in the K8s Secret directly from the live cluster, never prints it). Verified live: GET /api/v1/orgs/coulomb/hooks shows hook id 1, active, push events only, pointed at https://reuse.coulomb.social/v1/webhooks/forgejo.

Migrated this repo's own CI, following the existing tier-3 playbook (the-custodian/docs/forgejo-repo-migration-pilot-glas-harness.md, railiance-enablement/docs/forgejo-actions-workflow-templates.md — used proven patterns rather than improvising):

• Created coulomb/reuse-surface on Forgejo (private: false, matching every other migrated repo including state-hub/activity-core — confirmed via API before asking, then got explicit user sign-off on the visibility choice specifically, since the auto-mode classifier correctly flagged "public" as an inferred parameter beyond the general migrate instruction)
• .forgejo/workflows/ci.yml: ported .gitea/workflows/ci.yml verbatim in behavior, but using archive checkout (wget+tar + apt install python3) instead of actions/checkout@v4 — this runner's ubuntu-latest label maps to docker://node:20-bookworm with no Python preinstalled and no proven checkout-action support on this substrate
• .forgejo/workflows/ci-smoke.yaml: routing probe, copied from the canonical template (matches sibling convention)
• .forgejo/workflows/image.yaml: container build/push to forgejo.coulomb.social/coulomb/reuse-surface (canonical single-repo template, org secrets REGISTRY_USER/REGISTRY_TOKEN already existed)
• .forgejo/workflows/recompose-fallback.yaml: scheduled (every 6h) POST /v1/federated/compose as a backstop per design principle 3 ("degrade to schedule") — the webhook above is the primary path; no separate /v1/recompose route exists (see T02 note)
• Removed .gitea/workflows/ci.yml (dead once origin moves; matches how state-hub/activity-core/etc. left no .gitea/workflows/ behind)
• origin → forgejo-remote:coulomb/reuse-surface.git, old remote renamed to gitea (kept, read-only, frozen at commit ac81490 — safety contract from the migration playbook: never delete the Gitea copy)
• Live-verified all three workflows green on the actual push: ci.yml (validate-registry, real pytest+validate suite) success; ci-smoke.yaml (host-smoke + container-smoke) success; image.yaml (build-and-push) success — confirmed reuse-surface:latest and reuse-surface:main-09d5b0f both landed in the Forgejo container registry via the packages API

REUSE_SURFACE_TOKEN repo secret added 2026-07-07 (explicit user sign-off "Go ahead and add the repo secret"): fetched the token via warden access reuse-surface-hub-write-token --exec (never printed) and PUT it to Forgejo's repo Actions secrets API (201). Live-verified by manually dispatching recompose-fallback.yaml (POST .../actions/workflows/recompose-fallback.yaml/dispatches) — run completed success, and GET /v1/federated on the live hub showed composed_at advance to the exact dispatch time, confirming the full scheduled-fallback path works end to end, not just that the workflow didn't error.

T03 is now fully complete: org webhook live (railiance-apps), this repo's CI on Forgejo Actions (all three workflows verified green), and the scheduled fallback verified working end to end.

Not done — moved out of this workplan's scope: updating docs/deploy/reuse-kubernetes.md/railiance-apps to make forgejo.coulomb.social the canonical production deploy image registry (vs. the current live pod, which still runs the gitea.coulomb.social image built in T02 and is working fine). Didn't force another production redeploy in this same pass; the Forgejo image pipeline is proven and ready whenever a future deploy wants to switch registries.

Reuse Telemetry Store And Recording

id: REUSE-WP-0019-T04
status: done
priority: medium
state_hub_task_id: "c8e9064e-5c39-4c84-80e9-8b255f8edaec"

The shared schema (schemas/reuse-event.schema.json) already existed from WP-0018-T01 drafting — this task implemented the hub side and wired plan-check/manual recording against it, rather than designing it fresh.

Implemented:

• reuse_surface/hub/store.py: reuse_events SQLite table (append-only), record_reuse_event() (schema-validated, raises ValueError on a bad shape rather than silently accepting drift), list_reuse_events(capability_id=None)
• reuse_surface/hub/app.py: POST /v1/reuse-events (token-auth, 201/400), GET /v1/reuse-events?capability_id= (no auth, read-only, same posture as GET /v1/federated)
• reuse_surface/hub_client.py: hub_record_reuse_event/hub_list_reuse_events
• reuse_surface/plan_check.py: refactored record_outcome around a new shared post_or_fallback_reuse_event() — tries POST /v1/reuse-events first, falls back to the local JSONL only on failure/unreachability (never both, per the design: "+ local JSONL fallback when hub unreachable", not a dual-write). New record_manual_reuse_event() for retroactive facts, sharing the same post-or-fallback path
• New CLI command reuse-surface record-reuse --consumer-repo --capability-id --verdict --outcome [--hub-url] [--format]
• plan-check --record-outcome's help text and JSON output updated (recorded_reuse_event field); schemas/plan-check-result.schema.json extended for the new field
• Privacy/scope enforced structurally: the schema's additionalProperties: false means a caller literally cannot attach code or secrets to an event, only the declared fields (repo slug, capability id, verdict, outcome, source)
• 21 new pytest cases (hub store/API, plan_check dual-path, CLI); 145 total pass
• Live-verified against a real locally-running hub instance: POST/GET /v1/reuse-events directly, record-reuse CLI posting to the hub, plan-check --record-outcome posting to the hub, and — after actually killing the hub process — confirmed the fallback path writes correctly to the local JSONL instead of erroring

Telemetry Aggregation Into R-Axis Evidence

id: REUSE-WP-0019-T05
status: done
priority: medium
state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7"

"Blocked on T04 plus initial event volume" — T04 is done, and the tooling doesn't actually need to wait for real ecosystem adoption volume to be built and tested correctly (an empty/near-empty dataset is itself a valid, tested case: report reuse prints "No reuse events recorded yet" rather than erroring). Real cross-repo adoption volume ramping up is an ecosystem-timing question, not a coding blocker — implemented now so the tooling is ready the moment volume exists.

Implemented:

• reuse_surface/reports.py: collect_reuse_events() (merges the hub's GET /v1/reuse-events, if reachable, with this repo's local JSONL fallback, deduped by (ts, consumer_repo, capability_id, source)), collect_reuse_report() (per-capability consumer counts, outcome breakdown, last-used), format_reuse_report_markdown/json
• collect_reused_by_suggestions(): evidence-gated relation_add suggestions — only for capabilities this repo actually owns (checked against the local index, not invented), only for consumer repos not already listed. Reuses the existing patches.py:apply_patches mechanism (which already handled relation_add, not in SAFE_DETERMINISTIC_KINDS, so never auto-applied by maintain --auto) rather than building a new apply path
• New CLI: reuse-surface report reuse [--capability-id] [--format] [--suggest-relations] [--apply] — --apply requires --suggest-relations and is the only thing that writes; nothing happens automatically from telemetry alone
• Schema: added relations.reused_by to schemas/capability.schema.yaml (a new repoSlugList $defs type — distinct from the existing capabilityIdList relations, since reused-by targets are consumer repo slugs, not capability ids)
• specs/CapabilityMaturityStandard.md §8.9 amended: what observed-reuse evidence counts toward R2→R3 (single corroborating consumer) vs R3→R4+ (multiple independent consumers, report reuse's consumer_count), and explicitly what it never substitutes for (bug reports, incidents, explicit promotion_history citations — maturity_promote stays LLM-suggested and review-gated)
• 19 new pytest cases (test_reports.py); 162 total pass
• Live-verified with synthetic local-JSONL events against a real capability entry: report reuse --suggest-relations --apply correctly wrote relations.reused_by into the entry's front matter via the real apply_patches path (confirmed via git diff, then reverted since it was a smoke test, not a real fact)

Deliberately deferred, not silently dropped: "Catalog + graph surface consumer counts." graph.py's RELATION_TYPES graph is capability-to- capability edges; reused_by targets are repo slugs, a different namespace entirely — forcing it into that edge model would either error or produce meaningless nodes. catalog.py doesn't currently parse full entry front matter per capability at all (works off the index). Surfacing consumer counts in either artifact is a real, separate rendering-layer task, not a natural extension of what T05 already built — left for a follow-up rather than rushed in.

Freshness Monitoring, Docs, SCOPE

id: REUSE-WP-0019-T06
status: todo
priority: low
state_hub_task_id: "a9f44d45-91e2-4b43-909f-30a5f906cf3b"

• reuse-surface stats: hub composed_at age + stale flag; CI informational check warns when the hub index is older than N days
• docs/RegistryFederation.md + docs/deploy/reuse-kubernetes.md: webhook setup, recompose endpoint, Forgejo token handling (route credentials per credential-routing rules — no secrets in repo)
• SCOPE.md: flip "automatic hub refresh" to possible; update federation posture

---

Acceptance

• No hardcoded forge host in code or sources.yaml; migrate-host tested (T01, 2026-07-07)
• Push to a sibling repo's registry/indexes/ recomposes the hub index without manual action (webhook), with scheduled fallback in place (T02/T03, 2026-07-07 — both live-verified end to end)
• This repo's CI runs on Forgejo Actions (.forgejo/workflows/) (T03, 2026-07-07 — ci.yml/ci-smoke.yaml/image.yaml all verified green on the live push)
• Reuse events recordable via hub API and CLI (T04, 2026-07-08 — live-verified); report reuse aggregation done in T05
• R-axis evidence rules for observed reuse documented in the maturity standard (T05, 2026-07-08 — specs/CapabilityMaturityStandard.md §8.9)
• Hub freshness visible (composed_at, stale flag) in API and stats (T02, 2026-07-07)

Out of scope

• Operating the Forgejo instance or the Gitea→Forgejo data migration itself
• Multi-replica/Postgres hub posture (separate managed-platform track)
• Blocking CI gates on registry freshness in sibling repos
• ActivityPub/Forgejo-native federation features (our federation layer stays raw-URL based)