reuse-surface/SCOPE.md

197 lines
9.8 KiB
Markdown
Raw Normal View History

2026-06-15 00:04:09 +02:00
# SCOPE
## One-liner
Capability registry for planning and implementation reuse based on discovery and delivery maturity.
## Core Idea
2026-06-15 01:04:30 +02:00
`reuse-surface` provides a registry-centric reuse layer for capabilities. It
makes capabilities visible, comparable, assessable, and reusable for planning,
implementation, and operation. A capability that is not registered is invisible
for reuse within this product boundary.
2026-06-15 00:04:09 +02:00
## In Scope
2026-06-15 01:04:30 +02:00
- Maintain the capability maturity model, standards, schemas, registry formats,
sample entries, indexes, validation guidance, CLI tooling, hub service, and
agent instructions.
2026-06-15 01:04:30 +02:00
- Keep `INTENT.md`, `specs/`, registry artifacts, and State Hub workplans
aligned on the registry-first reuse boundary.
- Support humans and agents as registry consumers through Markdown-first
authoring and machine-readable metadata.
- Record decisions, progress, and workplan status through State Hub.
- Verify changes with `reuse-surface validate`, `git diff --check`, and ADR-001
2026-06-15 01:04:30 +02:00
consistency checks.
2026-06-15 00:04:09 +02:00
## Out of Scope
- Host or operate the registered capabilities themselves (except the federation
hub coordinator, which stores repo metadata and index URLs only).
2026-06-15 01:04:30 +02:00
- Replace package registries, service catalogs, issue trackers, or project
management systems.
- Judge internal code quality as capability maturity.
- Own unrelated adjacent systems or make irreversible operational decisions
without human approval.
2026-06-15 00:04:09 +02:00
## What Is Possible Now
The MVP registry foundation, CLI tooling (REUSE-WP-0003), federation stack
(WP-0005/0010), and hosted hub (WP-0011) are in place. Humans and agents can:
- **Discover capabilities** via `registry/indexes/capabilities.yaml`,
`reuse-surface query`, or `GET https://reuse.coulomb.social/v1/federated`
- **Add a new capability** at D0/A0/C0/R0 using
`templates/capability-entry.template.md`
- **Promote capabilities** with evidence, `promotion_history`, and index vector
updates
- **Compare candidates** using maturity vectors, scope, relations, and consumer
guidance
- **Record expectations** through `external_evidence.completeness` and
`external_evidence.reliability`
- **Validate entries automatically** with `reuse-surface validate`
- **Export a machine-readable bundle** with `reuse-surface export`
- **Detect overlap candidates** with `reuse-surface overlaps`
- **Generate a human-readable catalog** with `reuse-surface catalog`
- **Browse a searchable catalog** at `docs/catalog/search.html`
- **Compose federated indexes** with `reuse-surface federation compose`
(local paths and remote HTTP URLs with cache)
- **Register federation sources on the hosted hub** with `reuse-surface hub`
against `https://reuse.coulomb.social`
- **Sync local federation manifest from hub** with `reuse-surface hub sync`
- **Export planning cohorts** with `reuse-surface report cohorts`
- **Report registry gaps** with `reuse-surface report gaps` (roster blockers,
empty scaffolds, dedup stubs)
- **Bootstrap a sibling registry** with `reuse-surface establish --scaffold`
- **Verify index publish readiness** with `reuse-surface establish --publish-check`
- **View registry stats** with `reuse-surface stats` (per-repo or
`--roster registry/federation/local-repo-roster.yaml --federation-ready`)
- **Draft or refresh entries** with `reuse-surface establish --discover` and
`reuse-surface update` (optional llm-connect backend)
- **Maintain registry interactively or automatically** with `reuse-surface maintain`
(TTY prompts, `--auto`, optional llm-connect, `--publish` chain)
- **Run the hub locally or in a container** with `reuse-surface serve`
- **Generate relation graphs** with `reuse-surface graph`
- **Explore relations interactively** at `docs/graph/index.html`
- **Avoid duplicates** by querying the index and checking overlaps before adding
entries
REUSE-WP-0018 T01/T02/T04/T06: plan-check deterministic matching + State Hub bridge T01: specs/PlanCheck.md design doc, plan-check-result.schema.json and reuse-event.schema.json (the latter shared with WP-0019's reuse telemetry). T02: reuse_surface/plan_check.py + 'reuse-surface plan-check' CLI command. Deterministic token-Jaccard matching against registry/indexes/federated.yaml (reuses overlaps.py's TOKEN_RE rather than a second scoring method), with reuse/extend/new verdicts, markdown and --format json output, staleness warning, and --record-outcome JSONL telemetry. T04: reuse_surface/statehub_bridge.py bridges plan-check 'new' verdicts to State Hub capability requests (--file-request) and surfaces open requests with no matching capability (report gaps --check-capability-requests, opt-in to stay offline-safe). Verified against the live local State Hub API; status field (not catalog_entry_id presence) is the correct open/closed signal, and the list endpoint needs a longer timeout than the health check (~7s observed with 5 rows). T06: docs (tools/README.md, RegistryFederation.md, SCOPE.md, IntentScopeGapAnalysis.md priority 29) and an informational CI smoke step. T03 (LLM rerank) not started -- llm-connect isn't running on this workstation. T05 (ecosystem rollout) remains blocked: WP-0017 has drafted entries for all 61 repos but they're still local-only pending its own T05 push/publish pass, so the federated index isn't yet worth rolling out plan-check as ecosystem convention. 16 new tests, all mocked -- no network calls in the default test run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 00:57:18 +02:00
- **Check before building** with `reuse-surface plan-check` (REUSE-WP-0018) —
match a draft workplan or free-text intent against the federated index and
get a reuse/extend/new verdict; `--file-request` bridges a `new` verdict to
a State Hub capability request; `report gaps --check-capability-requests`
surfaces open requests with no matching capability
- **Get automatic hub refresh** (REUSE-WP-0019-T02/T03) — a Forgejo push
webhook (`POST /v1/webhooks/forgejo`, HMAC-signed) recomposes the hub's
federated index when a registered repo's `registry/indexes/` changes,
with `composed_at`/`stale` visibility on `GET /v1/federated` and a
scheduled fallback recompose if a webhook delivery is ever missed
- **Record reuse telemetry** (REUSE-WP-0019-T04) — `plan-check
--record-outcome` and `reuse-surface record-reuse` post facts to
`POST /v1/reuse-events` when the hub is reachable, falling back to
`registry/telemetry/plan-check-events.jsonl` otherwise (same schema
either way); `GET /v1/reuse-events?capability_id=` aggregates them
REUSE-WP-0019-T05: reuse telemetry aggregation into R-axis evidence reuse_surface/reports.py: collect_reuse_events() merges the hub's GET /v1/reuse-events (if reachable) with this repo's local JSONL fallback, deduped. collect_reuse_report() aggregates per-capability consumer counts, outcome breakdown, and last-used. collect_reused_by_suggestions() proposes evidence-gated relation_add patches -- only for capabilities this repo owns, only for consumer repos not already listed -- reusing the existing patches.py:apply_patches mechanism (relation_add already isn't in SAFE_DETERMINISTIC_KINDS, so it was already never auto-applied by maintain --auto). New CLI: reuse-surface report reuse [--capability-id] [--format] [--suggest-relations] [--apply]. --apply requires --suggest-relations and is the only thing that writes -- nothing happens automatically from telemetry alone. schemas/capability.schema.yaml: added relations.reused_by as a new repoSlugList type, distinct from the existing capability-id relations, since reused-by targets are consumer repo slugs. specs/CapabilityMaturityStandard.md Sec8.9: what observed-reuse evidence counts toward R2->R3 (single corroborating consumer) vs R3->R4+ (multiple independent consumers) and what it never substitutes for. 19 new pytest cases, 162 total pass. Live-verified with synthetic local events against a real capability entry: --suggest-relations --apply correctly wrote relations.reused_by via the real apply_patches path (reverted after, since it was a smoke test). Deliberately deferred: surfacing consumer counts in the catalog/graph -- graph.py's relation model is capability-to-capability edges, a different namespace than repo-slug reused_by targets; catalog.py doesn't currently parse full front matter per entry. Left for a follow-up. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 22:47:51 +02:00
- **Aggregate reuse telemetry into R-axis evidence** (REUSE-WP-0019-T05) —
`reuse-surface report reuse` shows per-capability consumer counts,
outcomes, and last-used; `--suggest-relations` proposes evidence-gated
`relations.reused_by` patches (via the same `apply_patches` mechanism
`maintain` uses), applied only with an explicit `--apply` — never
automatic. `specs/CapabilityMaturityStandard.md` §8.9 documents what
observed-reuse evidence does (and doesn't) count toward R2/R3+
Registry **tooling** availability is **A4** (CLI plus hosted hub HTTP API).
Registry **authoring** remains Markdown-first; consumption combines entries, the
index, CLI automation, and the production hub.
## What Is Not Possible Yet
- **Multi-domain federation** — all indexed capabilities remain `helix_forge`
- **Planning analytics breadth** — `report gaps` shipped (REUSE-WP-0015-T03);
no roadmap views or standardization tracker beyond `overlaps` and compose
collision warnings
- **Managed platform posture** — hub runs as a container (A5 artifact) without
implemented SLO, multi-replica, or Postgres backing (criteria documented)
- **Formal consumer feedback loop** for registry workflows (reliability evidence
is mostly structural: CI/tests, not production telemetry)
See `tools/README.md` for command reference.
2026-06-15 00:04:09 +02:00
## Current State
- **Status:** active registry with CLI, federation, production hub, and
workstation-wide coverage campaign complete (REUSE-WP-0017 finished 2026-07-07).
- **Capabilities (reuse-surface):** 2 helix_forge meta-registry entries in
`registry/capabilities/`.
- **Workstation roster:** 62 local git repos at `~/<slug>/` tracked in
`registry/federation/local-repo-roster.yaml` — all **established**, **62/62**
hub-registered (inter-hub registration disabled on hub), **62/62**
publish-check pass, **62/62** capability coverage (`has` or explicit `none`).
- **Federation:** `registry/federation/sources.yaml`**61** enabled sources;
`registry/indexes/federated.yaml`**61** composed capability rows
(inter-hub excluded; core-hub included; 0 duplicate-ID warnings at last compose).
- **CLI / service:** `reuse_surface/` — validate, query, export, overlaps,
catalog, federation, graph, hub client, establish/update/stats, `serve`
(FastAPI hub).
- **Production hub:** `https://reuse.coulomb.social`**61** enabled repo
registrations; `GET /v1/federated` serves **61** capabilities from published
raw URLs (compose refreshed 2026-07-07).
- **Specs:** `specs/FederationHubAPI.md`, `schemas/hub-registration.schema.yaml`.
- **Docs:** `docs/CapabilityRegistryConcept.md`, `docs/RegistryFederation.md`,
`docs/IntentScopeGapAnalysis.md`, deploy guide `docs/deploy/reuse-kubernetes.md`.
- **CI:** `.forgejo/workflows/ci.yml` — validate, federation compose, catalog,
graph, pytest, informational `report cohorts`, `stats --roster`, `report gaps`
(migrated from Gitea 2026-07-07, REUSE-WP-0019-T03). Also
`.forgejo/workflows/image.yaml` (container build/push) and
`recompose-fallback.yaml` (scheduled hub recompose, webhook backstop).
- **Relation graph:** `docs/graph/capability-graph.mmd`, `docs/graph/index.html`.
- **Searchable catalog:** `docs/catalog/search.html`.
- **Workplans:** REUSE-WP-0001 through REUSE-WP-0015 finished (archived);
**REUSE-WP-0017** capability coverage campaign finished (2026-07-07);
**REUSE-WP-0018** plan-check consumption loop active next.
- **Assessment history:** `history/` — intent/scope assessments, rollout
milestone, dedup plan, per-repo follow-up.
- **Self-assessed vector:** `D5 / A4 / C5 / R3` (see `docs/IntentScopeGapAnalysis.md`).
## Repository Layout
```text
reuse-surface/
├── INTENT.md
├── SCOPE.md
├── AGENTS.md
├── pyproject.toml
├── Dockerfile
├── reuse_surface/ # CLI, hub service, federation, graph, catalog
├── specs/
├── schemas/
├── templates/
├── registry/
│ ├── capabilities/ # per-entry Markdown
│ ├── indexes/ # capabilities.yaml, federated.yaml
│ └── federation/ # sources.yaml, local-repo-roster.yaml, cache/
├── docs/
├── tools/
└── workplans/
└── archived/
```
2026-06-15 00:04:09 +02:00
## Getting Oriented
- Start with: INTENT.md
- Registry concept: docs/CapabilityRegistryConcept.md
- Intent vs scope gaps: docs/IntentScopeGapAnalysis.md
- Assessment snapshots: history/
2026-06-15 01:04:30 +02:00
- Product requirements: specs/ProductRequirementsDocument.md
- Use cases: specs/UseCaseCatalog.md
- Maturity standard: specs/CapabilityMaturityStandard.md
- Hub API: specs/FederationHubAPI.md
- Registry index: registry/indexes/capabilities.yaml
- Registry guidance: registry/README.md
- Federation guide: docs/RegistryFederation.md
- Hub deployment: docs/deploy/reuse-kubernetes.md
- Generated catalog: docs/CapabilityCatalog.md
- Searchable catalog: docs/catalog/search.html
- Relation graph: docs/graph/capability-graph.mmd
- Graph explorer: docs/graph/index.html
- CLI reference: tools/README.md
2026-06-15 00:04:09 +02:00
- Agent instructions: AGENTS.md
2026-06-15 01:04:30 +02:00
- Workplans: workplans/