coulomb/reuse-surface
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore(workplans): archive finished REUSE-WP-0017/0018/0019
All checks were successful
CI Smoke / host-smoke (push) Successful in 1s
Details
CI Smoke / container-smoke (push) Successful in 3s
Details
ci / validate-registry (push) Successful in 4m44s
Details

Browse source 
Move completed workplans to workplans/archived/ with 260707 date prefix
and sync state_hub workstream/task IDs from fix-consistency.
This commit is contained in:
tegwick 2026-07-08 00:32:14 +02:00
parent 3e11d9785e
commit 8f10f9fc6f
3 changed files with 22 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

287
workplans/archived/260707-REUSE-WP-0017-capability-coverage-campaign.md Normal file
View file

@ -0,0 +1,287 @@
---
id: REUSE-WP-0017
type: workplan
title: "Capability coverage campaign: seed or explicitly close all empty scaffolds"
domain: infotech
repo: reuse-surface
status: finished
owner: claude-code
topic_slug: helix-forge
created: "2026-07-06"
updated: "2026-07-07"
state_hub_workstream_id: "e8737366-d477-468f-bca8-7f900a9146d0"
---
# Capability coverage campaign: seed or explicitly close all empty scaffolds
The federation infrastructure is complete (61/61 repos established and
hub-registered, WP-0014/0015), but **51 of 61 roster repos publish zero
capabilities**. The federated index carries 24 capabilities, concentrated in a
handful of custodian-core repos. A registry that answers most queries with
"nothing found" trains agents to stop querying — content coverage is the
prerequisite for the consumption loop (REUSE-WP-0018) and the automation loop
(REUSE-WP-0019).
**Goal:** every roster repo either publishes ≥1 capability entry or carries an
explicit, machine-readable `no-reusable-capability` marker with a rationale.
"Empty" becomes informative instead of ambiguous.
**Baseline:** 10/61 repos with ≥1 capability; 51 empty scaffolds; 7 seed-ready
(gap report 2026-07-06).
**Target:** 0 ambiguous scaffolds; coverage ratio surfaced in `report gaps`
and CI.
## Approach
1. **Classify before drafting.** Sweep the 51 empty scaffolds and bucket each:
`has-capability` (draft entries) vs `no-capability` (mark explicitly —
experiments, forks, canon/docs-only repos, probes).
2. **Draft with existing tooling.** `reuse-surface establish --discover` +
llm-connect per repo produces first-pass entries at honest low maturity
(typically D1–D2 / A0–A1); no invented evidence.
3. **Human review gate.** Drafts land as branches/commits flagged for review;
nothing publishes to the federated index without a human pass (same pattern
as CUST-WP-0050 repo classification).
4. **Publish + recompose.** After review: publish-check, hub state refresh,
`federation compose`, catalog/graph regeneration.
## Dependencies
| Dependency | Owner | Notes |
|---|---|---|
| llm-connect | llm-connect | drafting backend for `establish --discover` |
| local-repo-roster.yaml | reuse-surface | 61-repo roster, sweep source of truth |
| .repo-classification.yaml | the-custodian | classification signals for the no-capability bucket |
| config-atlas publish fix | config-atlas | 303 publish blocker (T06) |
| Sibling repo write access | Bernd | drafts commit into sibling checkouts at `~/<slug>/` |
## Design decisions
- **`no-capability` marker format:** a `registry/NO_CAPABILITIES.md` file with
frontmatter (`reason`, `reviewed`, `revisit`) in the sibling repo, plus a
`capability_status: none` field in the roster row. `report gaps` treats
marked repos as resolved, not empty.
- **Honest maturity floor:** first-pass entries never claim above D2/A2/C2/R1
without cited on-disk evidence (WP-0016 evidence gates apply).
- **Batch execution:** repos are processed in cohorts of ~10 via
`/ralph-workplan`; each cohort ends with validate + gap-report delta.
---
## Extend Gap Report And Roster For Explicit No-Capability Status
```task
id: REUSE-WP-0017-T01
status: done
priority: high
state_hub_task_id: "40236c01-58f0-43a2-a4d6-5bf5bd80bdc7"
```
- Add `capability_status: has | none | pending` to roster schema and
`local-repo-roster.yaml` handling (default `pending` for empty scaffolds)
- `report gaps`: split "Empty scaffolds" into "Unclassified" and
"Explicitly none (n)"; add a coverage ratio headline
(`repos with capabilities or explicit none / total`)
- Define `registry/NO_CAPABILITIES.md` template in `templates/`
- Pytest coverage for both report paths
## Classification Sweep Of The 51 Empty Scaffolds
```task
id: REUSE-WP-0017-T02
status: done
priority: high
state_hub_task_id: "8f608a69-4bd1-4018-b51d-bb19d0ca887c"
```
- For each empty-scaffold repo: read INTENT/SCOPE/README + top-level layout,
bucket `has-capability` vs `no-capability` with a one-line rationale
- Output: `history/2026-07-06-coverage-classification.md` table (repo, bucket,
rationale, candidate capability ids for the has bucket) — done, 37
has-capability / 13 no-capability / 0 missing
- Cross-check against `.repo-classification.yaml` where available
- **Human review checkpoint:** Bernd confirms the bucket split before T03/T04
## Mark No-Capability Repos
```task
id: REUSE-WP-0017-T03
status: done
priority: medium
state_hub_task_id: "2e0b5b06-b000-45a8-8d8c-31fee5eb6678"
```
For each confirmed `no-capability` repo:
- Write `registry/NO_CAPABILITIES.md` from template into the sibling checkout — done, 13/13
- Set `capability_status: none` in the roster — done
- Commit in sibling repo with a uniform message referencing this workplan — done,
13/13 on origin (confirmed at T05 closeout 2026-07-07)
## Draft Capability Entries For Has-Capability Repos
```task
id: REUSE-WP-0017-T04
status: done
priority: high
state_hub_task_id: "89a30f80-d0bd-432f-81ae-005fd1ec1356"
```
**Note:** `establish --discover` requires a running `llm-connect` backend,
which isn't up on this workstation; spinning up a new inference service was
out of scope for this task. Entries are instead drafted directly by the
implementing agent, grounded in each repo's actual README/docs/tests (no
invented evidence), which better matches the "no invented evidence" design
principle anyway.
**Cohort 1 (10/37 done):** artifact-store, can-you-assist, citation-engine,
citation-evidence, email-connect, guide-board, hub-core, info-tech-canon,
infospace-bench, inter-hub. All validate; no overlap with the existing 24
federated capabilities. Committed locally in each sibling repo (push held
for T05). Coverage 24/61 -> 34/61.
**Cohort 2 (20/37 done):** issue-core (migrated its existing
`CAPABILITY-issue-tracking.yaml` into the standard registry location rather
than drafting fresh), kaizen-agentic, key-cape, kontextual-engine,
llm-connect, markitect-filter, markitect-main (registered honestly as a
superseded legacy platform pointing at its three successor repos, not a
forward-looking target), markitect-quarkdown, markitect-tool, net-kingdom.
All validate; no overlap with existing federated capabilities. Committed
locally (push held for T05). Coverage 34/61 -> 44/61. Also fixed a stale
`empty_scaffold_count >= 40` test threshold in `tests/test_reports.py` —
no longer meaningful once the coverage campaign started shrinking that
number by design.
**Cohort 3 / final (17/17 done):** open-cmis-tck, open-reuse, ops-bridge,
phase-memory, railiance-apps, railiance-cluster, railiance-enablement,
railiance-fabric, railiance-forge, railiance-infra, railiance-platform,
repo-scoping, the-custodian, user-engine, vantage-point, vergabe-teilnahme,
whynot-design. All validate; no overlap with existing federated
capabilities. Committed locally (push held for T05). Coverage 44/61 ->
**61/61 — full coverage, T04 complete**.
Notable calls in this cohort:
- **the-custodian**: repo root is confidential/proprietary (NDA notice in
its README). Entry is deliberately scoped only to the non-confidential
`runtime/` agent framework and `tools/` repo-classification scripts;
canon/memory content is explicitly excluded from the entry's `includes`
and called out in `consumer_guidance.not_recommended_for`.
- **vergabe-teilnahme**: its own `SCOPE.md` is an unfilled template, so
discovery is honestly D1/C0 — flagged in `known_limitations` that filling
in SCOPE.md is the natural next step before further promotion.
- **markitect-main** (cohort 2) and **vantage-point**/**railiance-forge**
patterns repeat here: several repos (open-reuse, vantage-point) have a
stale repo-seed-template README; SCOPE.md/INTENT.md were used as the
authoritative source instead, noted per-entry.
- **railiance-enablement**, **railiance-apps**, **the-custodian**: each
surfaced tooling directly relevant to REUSE-WP-0019 (Forgejo promotion
script, forgejo-smoke.sh, patch-forgejo-remote-urls.sh) — worth cross-
referencing when that workplan starts its host-migration inventory (T01).
- `reuse-surface establish --discover` with llm-connect per repo
- Manual tightening: id namespace (`capability.<domain>.<verb>`), scope
boundary, honest vectors, relations to existing federated capabilities
- `reuse-surface validate` + `overlaps` check per cohort (no duplicate
semantics vs the existing 24)
- Drafts committed in sibling repos, flagged for human review; roster
`capability_status: has`
## Human Review And Publish Pass
```task
id: REUSE-WP-0017-T05
status: done
priority: high
state_hub_task_id: "efa64aab-dde5-499e-a10b-11ce81ef6941"
```
- Bernd reviews drafted entries per cohort (maturity honesty, scope, ids)
- After approval: sibling repos push; `establish --publish-check` per repo;
hub registrations refreshed where raw URLs changed
- `reuse-surface federation compose` + catalog + graph regeneration in this repo
**Done 2026-07-07.** Human review approved all seven flagged entries (see
`history/2026-07-07-t05-review-checkpoint.md`). Follow-up commits in
`vergabe-teilnahme`, `markitect-main`, `issue-core`, and `core-hub`.
Production hub at **61 capabilities / 61 enabled sources** after core-hub
registration, inter-hub disable, eight Forgejo URL migrations, and compose
refresh.
## Fix config-atlas Publish Blocker
```task
id: REUSE-WP-0017-T06
status: done
priority: medium
state_hub_task_id: "e94a792d-9b6d-4a4b-b9a6-6171a52557d2"
```
**Diagnosis complete — two separate issues, conflated in the original note:**
1. **The 303 itself was not actually a live blocker.** Gitea's raw-URL
scheme is `/raw/main/<path>` -> 303 redirect -> `/raw/branch/main/<path>`
-> 200. `establish.py`'s `_probe_raw_url` uses `urllib.request.urlopen`,
which already follows redirects transparently (verified: HEAD request to
the `/raw/main/...` URL resolves to 200 today). Re-running
`establish --publish-check` for config-atlas now returns **PASS**. The
roster's `publish_check: fail` was stale data from the 2026-06-16 sweep,
not a reproducible current failure — corrected in the roster (publish
pass now 61/61).
2. **config-atlas was never actually registered with the production hub.**
Diffing `https://reuse.coulomb.social/v1/federated` (60 sources) against
the local roster (61 repos) shows config-atlas is the only mismatch —
present in `sources.yaml` and locally claimed `hub_registered: true`
(from the config-atlas registration commit), but genuinely absent from
the hub. Corrected the roster's `hub_registered` to `false` for
config-atlas (summary count 61 -> 60) to reflect reality.
**Resolved 2026-07-07.** A routing catalog entry
(`reuse-surface-hub-write-token`) appeared in
`~/ops-warden/registry/routing/catalog.yaml` pointing at the token's actual
home: a Kubernetes Secret (`reuse-surface-env` in namespace `reuse` on the
Railiance01 cluster), not an OpenBao/Vault lane. Registration was completed
through that channel (by a separate concurrent session, not this one — this
session verified the *result*, not the credential itself, per the
credential-routing rule against exploring live cluster secrets directly).
config-atlas is now present among the 61 sources at
`https://reuse.coulomb.social/v1/federated`. Roster corrected:
`hub_registered: true`, publish_sweep 2026-07-07.
- [x] Diagnose the 303 (resolved — not a current blocker, code already follows the redirect)
- [x] Local publish-check target: 61/61 pass
- [x] Complete hub registration for config-atlas — done 2026-07-07, 61/61 hub-registered
## Closeout: Coverage Metrics, Docs, SCOPE Update
```task
id: REUSE-WP-0017-T07
status: done
priority: low
state_hub_task_id: "c8cf18f5-3140-42f1-a065-9aa6f03de1f0"
```
- `report gaps` shows 0 unclassified scaffolds; record final coverage ratio
- Update `SCOPE.md` Current State (capability count, coverage), regenerate
`docs/CapabilityCatalog.md`, graph, search catalog
- `history/` milestone note; progress event + fix-consistency
**Done 2026-07-07.** Coverage **62/62** roster repos; federated index **61**
capabilities on production hub; `history/2026-07-07-wp0017-coverage-campaign-complete.md`.
---
## Acceptance
- [x] Every roster repo is `has` or `none` — zero `pending` (62/62)
- [x] All `none` repos carry a reviewed `registry/NO_CAPABILITIES.md` (13)
- [x] Federated index capability count reflects the seeded entries (61 live; inter-hub disabled on hub)
- [x] 62/62 publish pass
- [x] Coverage ratio visible in `report gaps` and CI output
- [x] No entry claims maturity without on-disk evidence citations (T05 human review)
## Out of scope
- Consumption/plan-check tooling (REUSE-WP-0018)
- Forgejo webhooks/automation (REUSE-WP-0019)
- Maturity promotions beyond honest first-pass levels
- Multi-domain federation (all entries remain `helix_forge`)

257
workplans/archived/260707-REUSE-WP-0018-plan-check-consumption-loop.md Normal file
View file

@ -0,0 +1,257 @@
---
id: REUSE-WP-0018
type: workplan
title: "plan-check: close the consumption loop for capability reuse"
domain: infotech
repo: reuse-surface
status: finished
owner: claude-code
topic_slug: helix-forge
created: "2026-07-06"
updated: "2026-07-07"
reuse_check: "new — dogfooded 2026-07-07 via reuse-surface plan-check against the full 61-capability federated index; no existing capability covers query-before-build matching"
state_hub_workstream_id: "a5351af2-2de8-40d6-9f3c-027807e86ee1"
---
# plan-check: close the consumption loop for capability reuse
The registry is write-mostly: nothing nudges an agent planning work in a
sibling repo to query the federated index before building. The State Hub has a
`request_capability` / `list_capability_requests` flow, but it is not bridged
to reuse-surface data. This workplan adds the **query-before-build** step to
the ecosystem: a `plan-check` command that matches a draft workplan or intent
text against federated capabilities, wired into every repo's session protocol,
with a two-way bridge to State Hub capability requests.
This is the highest-leverage coherence mechanism: it is what stops 61 repos
from independently reinventing the same thing.
**Depends on REUSE-WP-0017** for meaningful match rates — plan-check against a
mostly-empty index produces noise and erodes trust. T01–T03 (design +
implementation) can proceed in parallel with the coverage campaign; the
ecosystem rollout (T05) waits for coverage.
## Proposed CLI surface
```bash
# Match a draft workplan against the federated index
reuse-surface plan-check workplans/XXX-WP-0042-something.md
# Free-text intent
reuse-surface plan-check --intent "parse invoices from email and file evidence"
# Agent-consumable output
reuse-surface plan-check --format json workplans/XXX-WP-0042.md
# Record the outcome (feeds R-axis evidence, WP-0019)
reuse-surface plan-check ... --record-outcome reused|extended|new
```
**Verdict classes per match:** `reuse` (capability covers the need — link it),
`extend` (close scope — extend instead of duplicating), `new` (no match —
proceed, optionally file a capability request). Output includes capability id,
maturity vector, owning repo, and consumer guidance.
## Design principles
1. **Deterministic core, LLM assist optional** — keyword/scope/relation
matching against index rows works without llm-connect; semantic matching
via llm-connect improves recall when available (same optional-backend
pattern as WP-0016).
2. **Advisory, not blocking** — plan-check informs; it never vetoes a
workplan. Adoption comes from usefulness plus protocol convention, not
gates.
3. **One line in the session protocol** — sibling integration is a single
instruction in the shared `.claude/rules/` template: run plan-check before
creating a workplan. No per-repo bespoke wiring.
4. **Every check is a data point** — plan-check invocations and recorded
outcomes are the raw material for reuse telemetry (WP-0019-T04).
## Dependencies
| Dependency | Owner | Notes |
|---|---|---|
| REUSE-WP-0017 coverage | reuse-surface | rollout gate for T05 |
| federated.yaml freshness | reuse-surface | improved by WP-0019 auto-recompose |
| llm-connect | llm-connect | optional semantic matching backend |
| State Hub capability-request API | state-hub | `request_capability`, `list_capability_requests` |
| .claude/rules template propagation | the-custodian / repo-seed | mechanism for T05 rollout |
---
## Design Matching Semantics And Verdict Model
```task
id: REUSE-WP-0018-T01
status: done
priority: high
state_hub_task_id: "2507e1d0-5c67-491e-a5d1-6ac32a59ed7f"
```
Design doc `specs/PlanCheck.md`:
- Input model: workplan file (frontmatter + body) vs free intent text
- Match pipeline: token/keyword scoring over id, name, scope, guidance →
relation expansion → optional llm-connect semantic rerank
- Verdict thresholds and tie-breaking (maturity vector as rank signal:
prefer higher D/A when scopes tie)
- JSON output schema (`schemas/plan-check-result.schema.json`)
- Outcome-recording format (append-only JSONL under `registry/telemetry/`,
schema shared with WP-0019-T04)
## Implement Deterministic plan-check
```task
id: REUSE-WP-0018-T02
status: done
priority: high
state_hub_task_id: "33bbe7c8-461b-43e9-9b51-1a10f6413e6a"
```
Implemented in `reuse_surface/plan_check.py` + `plan-check` CLI command:
- Reuses `overlaps.py`'s `TOKEN_RE`/Jaccard approach rather than a second
scoring method (per spec §4)
- Reads `registry/indexes/federated.yaml`, falls back to the local index,
warns when the composed index is >14 days stale
- Markdown (TTY default) and `--format json` (validated against
`schemas/plan-check-result.schema.json`)
- 11 pytest cases: verdict thresholds, workplan-file parsing, intent-text
parsing, tie-window vector ranking, staleness warning, CLI wiring
- `--record-outcome` appends to `registry/telemetry/plan-check-events.jsonl`
per the shared `schemas/reuse-event.schema.json` (WP-0019-T04 will post the
same shape to the hub)
## Add llm-connect Semantic Rerank
```task
id: REUSE-WP-0018-T03
status: done
priority: medium
state_hub_task_id: "cf3f1923-a7b7-4ccc-8b81-6b21ba040835"
```
Unblocked 2026-07-07 — `llm-connect` running locally (mock provider,
`127.0.0.1:8080`). Implemented in `reuse_surface/plan_check.py`:
- `build_rerank_prompt`/`request_rerank`/`apply_rerank`, reusing
`llm_bridge.execute_prompt`/`extract_json_object` (same pattern as
`maintain_llm.py`'s `request_maintain_patches`)
- New schema `schemas/plan-check-rerank.schema.json`; malformed responses
(missing fields, non-JSON, invented candidate ids not in the input set)
are rejected via schema validation + an id-membership filter, never
guessed at
- `apply_rerank` **appends** LLM-scored entries after the deterministic
list (`kind: "llm"`) rather than reordering it — deterministic matches
keep their original order/scores untouched, per design principle 3
- Graceful skip (`ValueError` → a `notes` field in the result, mirroring
`maintain.py`'s `no_llm`/skip pattern) when `LLM_CONNECT_URL` is unset
or the LLM response is malformed; `--no-llm` skips without attempting a
call at all
- New CLI flags: `--llm-url`, `--no-llm`
- `schemas/plan-check-result.schema.json` extended for `notes` and the
(pre-existing, previously unschema'd) `filed_capability_request` field
- 9 new pytest cases (valid, malformed, non-JSON, invented-id-filtering,
append-not-reorder, graceful skip, `--no-llm` short-circuit, successful
integration) — 89 total pass
- **Live-verified**, not just mocked: real HTTP round-trip against the
running mock `llm-connect` instance correctly rejected its non-JSON mock
response and surfaced the skip note, while the deterministic verdict and
match order stayed completely unaffected
## Bridge State Hub Capability Requests
```task
id: REUSE-WP-0018-T04
status: done
priority: medium
state_hub_task_id: "ef94baac-57ed-405c-a9f1-9752683bfc71"
```
Implemented in `reuse_surface/statehub_bridge.py` against the live local
State Hub HTTP API (`http://127.0.0.1:8000`, confirmed reachable):
- **new → request:** `plan-check --file-request` files a State Hub
capability request (`POST /capability-requests/`) on a `new` verdict
- **request → gap:** `report gaps --check-capability-requests` lists open
requests (`GET /capability-requests/`, filtered on `status` — a request
can carry a `catalog_entry_id` while still `requested`/routed, so status
is the correct open/closed signal, not `catalog_entry_id` presence).
Opt-in flag so `report gaps` stays fast and offline-safe by default
- Both degrade to `None`/graceful-skip when the hub is unreachable; neither
ever blocks the primary command
- List endpoint observed taking ~7s in production with only 5 rows; uses a
20s timeout distinct from the 5s health-check timeout
- 5 pytest cases with the hub client fully mocked (never hits the network
in the default test run)
- **Not done:** marking matched requests resolved with a pointer to the
capability id — deferred, no current caller needs it yet
## Ecosystem Rollout: Session-Protocol Integration
```task
id: REUSE-WP-0018-T05
status: done
priority: high
state_hub_task_id: "572bd9ed-1e2d-46bc-8ae5-e8b5f8ea03ea"
```
Gate cleared 2026-07-07: WP-0017 finished, `federated.yaml` composed with
61 real capabilities (up from the original 24). Dogfooded immediately —
`plan-check` against "unified interface for issue tracking across Gitea
GitHub GitLab" now correctly returns `extend` pointing at
`capability.infotech.issue-tracking`, where before WP-0017 published it
would have returned `new`. That's the whole point of sequencing T05 after
WP-0017 landed.
- [x] Drafted the convention in `.claude/rules/workplan-convention.md`
("Query before building" section): run `plan-check`, record
`reuse_check: reuse|extend|new — <capability-id?>` in frontmatter,
advisory only
- [x] Sent State Hub message to `custodian-agent` proposing propagation
via the fix-consistency/repo-seed shared rules template (message
`13cecd08-14e8-46d9-880c-65b8fef857cb`) — propagation itself depends on
the-custodian's own template mechanism, not something reuse-surface
can push directly into sibling repos
- [x] Added `reuse_check` frontmatter to this repo's own REUSE-WP-0018 and
REUSE-WP-0019 as the reference implementation, both dogfooded for real
(both correctly verdict `new`)
## Docs, CI, SCOPE
```task
id: REUSE-WP-0018-T06
status: done
priority: low
state_hub_task_id: "6c00931f-5ff6-4bde-b04d-9146a999047d"
```
- `tools/README.md` command reference for `plan-check` and
`report gaps --check-capability-requests`; `docs/RegistryFederation.md`
"Query before building" consumer section; `SCOPE.md` "What Is Possible Now"
- CI: informational `plan-check --intent "smoke test" --format json` run
added to `.gitea/workflows/ci.yml`
- `docs/IntentScopeGapAnalysis.md`: priority 29 added, marked **Partial**
(T02/T04 shipped; T03/T05 remain open — not flipped fully to "shipped"
since the ecosystem rollout hasn't happened yet)
---
## Acceptance
- [x] `plan-check` returns reuse/extend/new verdicts for workplan files and intent text, with and without llm-connect (T03 shipped 2026-07-07, live-verified against a running instance)
- [x] JSON output validates against the published schema
- [x] `new` verdicts can file State Hub capability requests; `report gaps` lists unmatched open requests
- [x] Session-protocol convention drafted; propagation *proposed* to custodian-agent
2026-07-07 (message `13cecd08-14e8-46d9-880c-65b8fef857cb`) — actual
adoption into the shared rules template is the-custodian's call, not
something reuse-surface can complete unilaterally
- [x] reuse-surface itself records `reuse_check` in its own workplans
(REUSE-WP-0018, REUSE-WP-0019) as the reference implementation
## Out of scope
- Blocking/gating workplan creation on plan-check results
- Reuse telemetry aggregation and R-axis evidence (WP-0019)
- Embedding-based matching (llm-connect rerank only)
- Editing sibling repos' rules files directly (owned by template propagation)

435
workplans/archived/260707-REUSE-WP-0019-forgejo-automation-and-telemetry.md Normal file
View file

@ -0,0 +1,435 @@
---
id: REUSE-WP-0019
type: workplan
title: "Forgejo-native federation automation and reuse telemetry"
domain: infotech
repo: reuse-surface
status: finished
owner: claude-code
topic_slug: helix-forge
created: "2026-07-06"
updated: "2026-07-08"
reuse_check: "new — dogfooded 2026-07-07 via reuse-surface plan-check against the full 61-capability federated index; no existing capability covers Forgejo webhook automation or reuse telemetry"
state_hub_workstream_id: "34749ac3-ceec-4890-b001-e25ee41ea278"
---
# Forgejo-native federation automation and reuse telemetry
Federation compose is on-demand and the hub serves whatever was last composed;
roster sweeps are manual; reliability evidence is structural (CI exists) rather
than observed (someone reused it and it worked). This workplan makes the
registry **live** (event-driven recompose) and **evidence-backed** (reuse
telemetry feeding the R axis).
**Platform constraint:** the forge is transitioning **Gitea → Forgejo**. All
new automation attaches to Forgejo (webhooks, Forgejo Actions, API tokens) —
nothing new is built against Gitea. Existing raw URLs
(`https://gitea.coulomb.social/...`) and `.gitea/workflows/` must migrate or
be made host-agnostic. Forgejo is Gitea-API-compatible, so migration is mostly
host/path configuration, but every hardcoded `gitea.` reference is a liability.
**Depends on:** REUSE-WP-0017 (content worth refreshing), REUSE-WP-0018-T01
telemetry schema (shared). Closes SCOPE "not possible yet" item *automatic hub
refresh* and moves reliability evidence beyond structural.
## Design principles
1. **Host-agnostic first** — a single `forge_base_url` configuration
(env/config + hub setting) replaces hardcoded hosts; the Forgejo cutover
becomes a one-line change per surface.
2. **Webhook triggers, compose stays pull-based** — the webhook only marks the
hub's composed index stale and triggers recompose from published raw URLs;
no push-parsing of payloads into registry state.
3. **Degrade to schedule** — if webhooks are unavailable, a scheduled Forgejo
Actions job recomposes on an interval; freshness is monitored either way.
4. **Telemetry is append-only and low-ceremony** — reuse events are JSONL
facts (who consumed what, when, outcome); aggregation derives `reused_by`
relations and R-axis evidence citations, never hand-edited.
## Dependencies
| Dependency | Owner | Notes |
|---|---|---|
| Forgejo instance + admin | Bernd / infra | webhook config, org-level token, Actions runners |
| Gitea→Forgejo cutover plan | infra | final hostname, raw URL scheme, redirect window |
| Hub deployment (reuse.coulomb.social) | reuse-surface / railiance | new endpoint + config rollout |
| REUSE-WP-0018-T01 | reuse-surface | shared telemetry/outcome schema |
| plan-check adoption | ecosystem | telemetry volume comes from WP-0018-T05 rollout |
---
## Forge Host Abstraction And URL Migration Inventory
```task
id: REUSE-WP-0019-T01
status: done
priority: high
state_hub_task_id: "337866d5-9952-40b8-a338-8ac16a699be0"
```
**Inventory findings (2026-07-07):** cross-referenced `sources.yaml` (61
entries) against each repo's actual git `origin` remote. Result: 11 repos
already migrated their remote to Forgejo; 50 still on Gitea (correctly, no
action needed). Of the 11 Forgejo-origin repos, 9 already had correct
`sources.yaml` entries (from WP-0017 drafting/registration); **2 were
stale** — `activity-core` and `state-hub` — still pointing at their old
Gitea raw URL despite having migrated. This was real, live debt, not a
hypothetical: both were confirmed reachable on Forgejo (HTTP 200) before
being migrated for real, in both `sources.yaml` and the production hub
registration (`hub update --url`). Verified post-migration against
`GET /v1/federated` — both now show `forgejo.coulomb.social`.
`config-atlas`'s WP-0017-T06 303 was **not** a host-transition symptom —
already diagnosed there as (a) a redirect the current code already follows
fine, and (b) a hub-registration gap unrelated to host. No new finding here.
Implemented:
- `reuse_surface/forge_host.py`: `parse_raw_url`/`derive_raw_url` (handles
both the legacy `/raw/<branch>/...` form and the canonical
`/raw/branch/<branch>/...` form both forges serve without a 303
redirect), `rewrite_url_host`, `forge_base_url` (reads
`REUSE_SURFACE_FORGE_BASE_URL`, no hard default since migration is
opt-in per repo), `migrate_source_host` (verifies the new URL resolves
via HTTP HEAD **before** writing — refuses to point a repo at a host it
hasn't actually migrated to)
- CLI: `reuse-surface federation migrate-host --repo <slug> [--repo ...]
--to <base-url> [--from <sanity-check>] [--dry-run] [--no-verify]
[--update-hub]` — used for real on `activity-core`/`state-hub`
- Fixed two host-agnostic code gaps found while inventorying:
`registry_update.py`'s `SAFE_EVIDENCE_PREFIXES` and `maintain_llm.py`'s
git-diff pathspec only recognized `.gitea/workflows/`, missing repos
already on `.forgejo/workflows/` — both now recognize either
- Fixed stale copy-paste examples in `docs/RegistryFederation.md`
(state-hub's old Gitea URL, now genuinely wrong post-migration) and a
pre-existing, unrelated port typo (`8088` vs the real llm-connect
default `8080`) in `tools/README.md`/`registry/README.md`, discovered
and confirmed live during REUSE-WP-0018-T03
- 17 new pytest cases (`tests/test_forge_host.py`); 106 total pass
- Recomposed `federated.yaml` post-migration: still 61 capabilities, no
loss
## Hub Recompose Endpoint And Webhook Receiver
```task
id: REUSE-WP-0019-T02
status: done
priority: high
state_hub_task_id: "212352fe-3caf-4999-b5a2-58fa2441ca56"
```
**Note:** `POST /v1/federated/compose` (token-auth, triggers a real
recompose) already existed from earlier hub work — no separate
`/v1/recompose` route was added; the spec now documents this explicitly
rather than duplicating a route that already does the job.
Implemented in `reuse_surface/hub/`:
- `store.py`: `compose_state` table (`composed_at`, `stale`), with
`record_compose()`/`mark_stale()`/`get_compose_state()`. `composed_at`
updates and `stale` clears only on a *forced* recompose (`refresh=true`,
webhook, or future scheduled fallback) — a plain `GET` still serves
current best-effort data but never silently reports itself as freshly
composed
- `webhooks.py`: `verify_signature` (constant-time HMAC-SHA256, fails
closed on empty secret), `push_touches_registry_index` (path-only
inspection of the push payload's `added`/`modified`/`removed` lists —
never parses file content, per design principle 2)
- `app.py`: `POST /v1/webhooks/forgejo` (accepts both `X-Forgejo-Signature`
and `X-Gitea-Signature`, since repos migrate independently);
`GET /v1/federated` and `POST /v1/federated/compose` now share an
`asyncio.Lock` so concurrent recompose triggers (manual, webhook, future
scheduled) coalesce instead of overlapping
- `specs/FederationHubAPI.md` extended (§5.7-5.9, config table, error
codes)
- 28 new pytest cases (16 in `test_hub.py`, 12 in `test_webhooks.py`); 128
total pass
- **Live-verified**: ran the actual hub service locally
(`reuse-surface serve`), sent a real HMAC-signed webhook payload over
HTTP — confirmed `composed_at`/`stale` transitions, signature rejection,
irrelevant-path no-op, and the full webhook-to-recompose path end to end
**Deployed 2026-07-07 (explicit user sign-off "Deploy to live please"):**
built and pushed `gitea.coulomb.social/coulomb/reuse-surface:e3ae22e`,
smoke-tested it locally in a standalone container first, then
`helm upgrade` via `railiance-apps` (`make reuse-deploy`, pinned in
`helm/reuse-surface-values.yaml`, commits `a2c0da1`/`bcb05f5`). Live-verified
against `https://reuse.coulomb.social`: `/v1/federated` (200, 61
capabilities, `composed_at`/`stale` fields present), `/v1/repos` (200), a
real forced recompose via `POST /v1/federated/compose` (token-auth via
`warden access --exec`, token never printed), and the webhook endpoint
correctly failing closed with 503 (`REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET`
not yet configured).
**Webhook secret added 2026-07-07 (explicit user sign-off "Go ahead and
add the webhook secret"):** generated a fresh 32-byte hex secret with
`openssl rand -hex 32`, patched it into the live K8s Secret
`reuse-surface-env` in namespace `reuse`, restarted the deployment (values
injected via `envFrom.secretRef`, not picked up without a restart) — pod
rolled cleanly (`1/1 Ready`, 0 restarts). The raw secret value was never
printed to the transcript: generated and applied in one non-echoing shell
step, and re-fetched only inside an exported env var for the live
signature-verification test below, then unset. Live-verified against
production: a correctly HMAC-signed push payload is now accepted
(`{"accepted": false, "reason": "no registry/indexes/ change"}` for a
payload that doesn't touch `registry/indexes/`, as designed), and a bad
signature is still rejected with 401. `/v1/federated` and `/v1/repos`
unaffected by the restart.
**Found (not fixed) while smoke-testing:** the public ingress's exact-path
`/health` rule 404s at the Traefik edge (shadowed by the catch-all `/`
rule to the landing page) — confirmed ingress-layer only, not a pod/service
problem (`/health` works via direct port-forward; the Deployment's own
readiness/liveness probes pass, pod is `1/1 Ready`). `/v1/*` unaffected.
Flagged to `railiance-apps` via State Hub message and documented in
`railiance-apps` commit `bcb05f5` — not fixed here since it's a shared
production ingress template edit outside this workplan's scope.
## Forgejo Webhook Rollout And Scheduled Fallback
```task
id: REUSE-WP-0019-T03
status: done
priority: medium
state_hub_task_id: "a8e0bdc6-0a92-4a87-a7b2-19cf0a46db70"
```
**Org-level Forgejo webhook — done by `railiance-apps` (2026-07-07),
credited not claimed:** after I flagged the `/health` ingress bug to
`railiance-apps` via State Hub message, they picked up this piece too:
`tools/reuse-forgejo-webhook.sh` + `make reuse-forgejo-webhook`
(idempotent, reads the same `REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET` I put
in the K8s Secret directly from the live cluster, never prints it).
Verified live: `GET /api/v1/orgs/coulomb/hooks` shows hook id `1`, active,
`push` events only, pointed at `https://reuse.coulomb.social/v1/webhooks/forgejo`.
**Migrated this repo's own CI, following the existing tier-3 playbook
(`the-custodian/docs/forgejo-repo-migration-pilot-glas-harness.md`,
`railiance-enablement/docs/forgejo-actions-workflow-templates.md` — used
proven patterns rather than improvising):**
- Created `coulomb/reuse-surface` on Forgejo (`private: false`, matching
every other migrated repo including `state-hub`/`activity-core` —
confirmed via API before asking, then got explicit user sign-off on the
visibility choice specifically, since the auto-mode classifier correctly
flagged "public" as an inferred parameter beyond the general migrate
instruction)
- `.forgejo/workflows/ci.yml`: ported `.gitea/workflows/ci.yml` verbatim
in behavior, but using archive checkout (`wget`+`tar` + `apt install
python3`) instead of `actions/checkout@v4` — this runner's
`ubuntu-latest` label maps to `docker://node:20-bookworm` with no Python
preinstalled and no proven checkout-action support on this substrate
- `.forgejo/workflows/ci-smoke.yaml`: routing probe, copied from the
canonical template (matches sibling convention)
- `.forgejo/workflows/image.yaml`: container build/push to
`forgejo.coulomb.social/coulomb/reuse-surface` (canonical single-repo
template, org secrets `REGISTRY_USER`/`REGISTRY_TOKEN` already existed)
- `.forgejo/workflows/recompose-fallback.yaml`: scheduled (every 6h) `POST
/v1/federated/compose` as a backstop per design principle 3 ("degrade to
schedule") — the webhook above is the primary path; no separate
`/v1/recompose` route exists (see T02 note)
- Removed `.gitea/workflows/ci.yml` (dead once `origin` moves; matches how
`state-hub`/`activity-core`/etc. left no `.gitea/workflows/` behind)
- `origin` → `forgejo-remote:coulomb/reuse-surface.git`, old remote
renamed to `gitea` (kept, read-only, frozen at commit `ac81490` — safety
contract from the migration playbook: never delete the Gitea copy)
- **Live-verified all three workflows green** on the actual push: `ci.yml`
(`validate-registry`, real pytest+validate suite) success; `ci-smoke.yaml`
(`host-smoke` + `container-smoke`) success; `image.yaml`
(`build-and-push`) success — confirmed `reuse-surface:latest` and
`reuse-surface:main-09d5b0f` both landed in the Forgejo container
registry via the packages API
**`REUSE_SURFACE_TOKEN` repo secret added 2026-07-07 (explicit user
sign-off "Go ahead and add the repo secret"):** fetched the token via
`warden access reuse-surface-hub-write-token --exec` (never printed) and
`PUT` it to Forgejo's repo Actions secrets API (`201`). Live-verified by
manually dispatching `recompose-fallback.yaml`
(`POST .../actions/workflows/recompose-fallback.yaml/dispatches`) — run
completed `success`, and `GET /v1/federated` on the live hub showed
`composed_at` advance to the exact dispatch time, confirming the full
scheduled-fallback path works end to end, not just that the workflow
didn't error.
T03 is now fully complete: org webhook live (railiance-apps), this repo's
CI on Forgejo Actions (all three workflows verified green), and the
scheduled fallback verified working end to end.
**Not done — moved out of this workplan's scope:** updating
`docs/deploy/reuse-kubernetes.md`/`railiance-apps` to make
`forgejo.coulomb.social` the canonical *production deploy* image registry
(vs. the current live pod, which still runs the `gitea.coulomb.social`
image built in T02 and is working fine). Didn't force another production
redeploy in this same pass; the Forgejo image pipeline is proven and ready
whenever a future deploy wants to switch registries.
## Reuse Telemetry Store And Recording
```task
id: REUSE-WP-0019-T04
status: done
priority: medium
state_hub_task_id: "88e240d1-0b85-42d8-9d9e-44fb46937894"
```
The shared schema (`schemas/reuse-event.schema.json`) already existed from
WP-0018-T01 drafting — this task implemented the hub side and wired
`plan-check`/manual recording against it, rather than designing it fresh.
Implemented:
- `reuse_surface/hub/store.py`: `reuse_events` SQLite table (append-only),
`record_reuse_event()` (schema-validated, raises `ValueError` on a bad
shape rather than silently accepting drift), `list_reuse_events(capability_id=None)`
- `reuse_surface/hub/app.py`: `POST /v1/reuse-events` (token-auth, 201/400),
`GET /v1/reuse-events?capability_id=` (no auth, read-only, same posture
as `GET /v1/federated`)
- `reuse_surface/hub_client.py`: `hub_record_reuse_event`/`hub_list_reuse_events`
- `reuse_surface/plan_check.py`: refactored `record_outcome` around a new
shared `post_or_fallback_reuse_event()` — tries `POST /v1/reuse-events`
first, falls back to the local JSONL only on failure/unreachability
(never both, per the design: "+ local JSONL fallback when hub
unreachable", not a dual-write). New `record_manual_reuse_event()` for
retroactive facts, sharing the same post-or-fallback path
- New CLI command `reuse-surface record-reuse --consumer-repo --capability-id
--verdict --outcome [--hub-url] [--format]`
- `plan-check --record-outcome`'s help text and JSON output updated
(`recorded_reuse_event` field); `schemas/plan-check-result.schema.json`
extended for the new field
- Privacy/scope enforced structurally: the schema's `additionalProperties: false`
means a caller literally cannot attach code or secrets to an event, only
the declared fields (repo slug, capability id, verdict, outcome, source)
- 21 new pytest cases (hub store/API, plan_check dual-path, CLI); 145 total pass
- **Live-verified** against a real locally-running hub instance: `POST`/`GET
/v1/reuse-events` directly, `record-reuse` CLI posting to the hub,
`plan-check --record-outcome` posting to the hub, and — after actually
killing the hub process — confirmed the fallback path writes correctly
to the local JSONL instead of erroring
## Telemetry Aggregation Into R-Axis Evidence
```task
id: REUSE-WP-0019-T05
status: done
priority: medium
state_hub_task_id: "5694b776-f516-47ec-a848-714060dd2261"
```
"Blocked on T04 plus initial event volume" — T04 is done, and the
*tooling* doesn't actually need to wait for real ecosystem adoption volume
to be built and tested correctly (an empty/near-empty dataset is itself a
valid, tested case: `report reuse` prints "No reuse events recorded yet"
rather than erroring). Real cross-repo adoption volume ramping up is an
ecosystem-timing question, not a coding blocker — implemented now so the
tooling is ready the moment volume exists.
Implemented:
- `reuse_surface/reports.py`: `collect_reuse_events()` (merges the hub's
`GET /v1/reuse-events`, if reachable, with this repo's local JSONL
fallback, deduped by `(ts, consumer_repo, capability_id, source)`),
`collect_reuse_report()` (per-capability consumer counts, outcome
breakdown, last-used), `format_reuse_report_markdown/json`
- `collect_reused_by_suggestions()`: evidence-gated `relation_add`
suggestions — only for capabilities this repo actually owns (checked
against the local index, not invented), only for consumer repos not
already listed. Reuses the *existing* `patches.py:apply_patches`
mechanism (which already handled `relation_add`, not in
`SAFE_DETERMINISTIC_KINDS`, so never auto-applied by `maintain --auto`)
rather than building a new apply path
- New CLI: `reuse-surface report reuse [--capability-id] [--format]
[--suggest-relations] [--apply]` — `--apply` requires
`--suggest-relations` and is the only thing that writes; nothing happens
automatically from telemetry alone
- Schema: added `relations.reused_by` to `schemas/capability.schema.yaml`
(a new `repoSlugList` `$defs` type — distinct from the existing
`capabilityIdList` relations, since reused-by targets are consumer repo
slugs, not capability ids)
- `specs/CapabilityMaturityStandard.md` §8.9 amended: what observed-reuse
evidence counts toward R2→R3 (single corroborating consumer) vs R3→R4+
(multiple *independent* consumers, `report reuse`'s `consumer_count`),
and explicitly what it never substitutes for (bug reports, incidents,
explicit `promotion_history` citations — `maturity_promote` stays
LLM-suggested and review-gated)
- 19 new pytest cases (`test_reports.py`); 162 total pass
- **Live-verified** with synthetic local-JSONL events against a real
capability entry: `report reuse --suggest-relations --apply` correctly
wrote `relations.reused_by` into the entry's front matter via the real
`apply_patches` path (confirmed via `git diff`, then reverted since it
was a smoke test, not a real fact)
**Deliberately deferred, not silently dropped:** "Catalog + graph surface
consumer counts." `graph.py`'s `RELATION_TYPES` graph is capability-to-
capability edges; `reused_by` targets are repo slugs, a different
namespace entirely — forcing it into that edge model would either error or
produce meaningless nodes. `catalog.py` doesn't currently parse full entry
front matter per capability at all (works off the index). Surfacing
consumer counts in either artifact is a real, separate rendering-layer
task, not a natural extension of what T05 already built — left for a
follow-up rather than rushed in.
## Freshness Monitoring, Docs, SCOPE
```task
id: REUSE-WP-0019-T06
status: done
priority: low
state_hub_task_id: "b5b4cfaf-16fa-45ce-b1f2-6ad48fb090ab"
```
SCOPE.md's "automatic hub refresh" flip was already done incidentally in
T02/T03 (moved from "not possible yet" to "possible now"); no further
change needed there.
Implemented:
- `reuse_surface/hub_client.py`: new `hub_federated()` (`GET /v1/federated`)
- `reuse_surface/stats.py`: `_hub_summary()` now also reports
`composed_at`, `stale`, `age_days` (computed from `composed_at`),
`freshness_threshold_days` (`REUSE_SURFACE_FRESHNESS_DAYS` env, default
7), and a computed `stale_warning` (age beyond threshold OR the hub's own
`stale` flag). `format_stats_markdown` surfaces these with a `⚠ STALE`
marker when triggered; `format_stats_json` picks them up automatically
(no format-specific code needed there)
- `.forgejo/workflows/ci.yml`: new informational (non-failing) "Hub
freshness check" step against the live production hub — prints a
`::warning::` annotation when stale, never fails the build
- `docs/RegistryFederation.md`: new "Automatic recompose and freshness"
section tying together the webhook (T02/T03), scheduled fallback (T03),
and freshness visibility (T06) into one coherent explanation, pointing
at the authoritative operator runbook (`railiance-apps`) for actual
secret/webhook setup rather than duplicating operational steps here
- `docs/deploy/reuse-kubernetes.md`: image section updated to reflect the
T03 Forgejo migration (repo canonical remote moved; production image
still built from the pre-migration Gitea registry, deliberately not
switched over in this task); image promotion checklist updated for the
now-automated `.forgejo/workflows/image.yaml` build and the known
`/health` ingress bug (use `/v1/repos`/`/v1/federated` for verification
instead)
- 14 new pytest cases (`test_stats.py`); 173 total pass
- **Live-verified** against the real production hub: `reuse-surface stats`
correctly showed `composed_at`/`age_days` for the actual federated index
(0.08 days old, no stale warning); separately discovered and confirmed
(via a live signed webhook test) that an external secrets-management
change — `reuse-surface-env` is now ExternalSecret-managed from OpenBao
(`railiance-apps` commit `706f6c7`, found while updating these same
docs) — did **not** break the T02/T03 webhook: the synced value still
matches what the hub actually uses
---
## Acceptance
- [x] No hardcoded forge host in code or sources.yaml; `migrate-host` tested (T01, 2026-07-07)
- [x] Push to a sibling repo's `registry/indexes/` recomposes the hub index without manual action (webhook), with scheduled fallback in place (T02/T03, 2026-07-07 — both live-verified end to end)
- [x] This repo's CI runs on Forgejo Actions (`.forgejo/workflows/`) (T03, 2026-07-07 — `ci.yml`/`ci-smoke.yaml`/`image.yaml` all verified green on the live push)
- [x] Reuse events recordable via hub API and CLI (T04, 2026-07-08 — live-verified); `report reuse` aggregation done in T05
- [x] R-axis evidence rules for observed reuse documented in the maturity standard (T05, 2026-07-08 — `specs/CapabilityMaturityStandard.md` §8.9)
- [x] Hub freshness visible (`composed_at`, stale flag) in API and stats (T02 API, T06 `stats`/CI, 2026-07-07/08)
## Out of scope
- Operating the Forgejo instance or the Gitea→Forgejo data migration itself
- Multi-replica/Postgres hub posture (separate managed-platform track)
- Blocking CI gates on registry freshness in sibling repos
- ActivityPub/Forgejo-native federation features (our federation layer stays raw-URL based)