REUSE-WP-0019-T05: reuse telemetry aggregation into R-axis evidence
reuse_surface/reports.py: collect_reuse_events() merges the hub's GET /v1/reuse-events (if reachable) with this repo's local JSONL fallback, deduped. collect_reuse_report() aggregates per-capability consumer counts, outcome breakdown, and last-used. collect_reused_by_suggestions() proposes evidence-gated relation_add patches -- only for capabilities this repo owns, only for consumer repos not already listed -- reusing the existing patches.py:apply_patches mechanism (relation_add already isn't in SAFE_DETERMINISTIC_KINDS, so it was already never auto-applied by maintain --auto). New CLI: reuse-surface report reuse [--capability-id] [--format] [--suggest-relations] [--apply]. --apply requires --suggest-relations and is the only thing that writes -- nothing happens automatically from telemetry alone. schemas/capability.schema.yaml: added relations.reused_by as a new repoSlugList type, distinct from the existing capability-id relations, since reused-by targets are consumer repo slugs. specs/CapabilityMaturityStandard.md Sec8.9: what observed-reuse evidence counts toward R2->R3 (single corroborating consumer) vs R3->R4+ (multiple independent consumers) and what it never substitutes for. 19 new pytest cases, 162 total pass. Live-verified with synthetic local events against a real capability entry: --suggest-relations --apply correctly wrote relations.reused_by via the real apply_patches path (reverted after, since it was a smoke test). Deliberately deferred: surfacing consumer counts in the catalog/graph -- graph.py's relation model is capability-to-capability edges, a different namespace than repo-slug reused_by targets; catalog.py doesn't currently parse full front matter per entry. Left for a follow-up. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
e6e275ce79
commit
bca7165e02
8 changed files with 576 additions and 13 deletions
7
SCOPE.md
7
SCOPE.md
|
|
@ -90,6 +90,13 @@ The MVP registry foundation, CLI tooling (REUSE-WP-0003), federation stack
|
||||||
`POST /v1/reuse-events` when the hub is reachable, falling back to
|
`POST /v1/reuse-events` when the hub is reachable, falling back to
|
||||||
`registry/telemetry/plan-check-events.jsonl` otherwise (same schema
|
`registry/telemetry/plan-check-events.jsonl` otherwise (same schema
|
||||||
either way); `GET /v1/reuse-events?capability_id=` aggregates them
|
either way); `GET /v1/reuse-events?capability_id=` aggregates them
|
||||||
|
- **Aggregate reuse telemetry into R-axis evidence** (REUSE-WP-0019-T05) —
|
||||||
|
`reuse-surface report reuse` shows per-capability consumer counts,
|
||||||
|
outcomes, and last-used; `--suggest-relations` proposes evidence-gated
|
||||||
|
`relations.reused_by` patches (via the same `apply_patches` mechanism
|
||||||
|
`maintain` uses), applied only with an explicit `--apply` — never
|
||||||
|
automatic. `specs/CapabilityMaturityStandard.md` §8.9 documents what
|
||||||
|
observed-reuse evidence does (and doesn't) count toward R2/R3+
|
||||||
|
|
||||||
Registry **tooling** availability is **A4** (CLI plus hosted hub HTTP API).
|
Registry **tooling** availability is **A4** (CLI plus hosted hub HTTP API).
|
||||||
Registry **authoring** remains Markdown-first; consumption combines entries, the
|
Registry **authoring** remains Markdown-first; consumption combines entries, the
|
||||||
|
|
|
||||||
|
|
@ -35,11 +35,16 @@ from reuse_surface.plan_check import (
|
||||||
from reuse_surface.reports import (
|
from reuse_surface.reports import (
|
||||||
cohort_filters_from_args,
|
cohort_filters_from_args,
|
||||||
collect_gap_report,
|
collect_gap_report,
|
||||||
|
collect_reuse_events,
|
||||||
|
collect_reuse_report,
|
||||||
|
collect_reused_by_suggestions,
|
||||||
default_roster_path,
|
default_roster_path,
|
||||||
format_cohort_json,
|
format_cohort_json,
|
||||||
format_cohort_markdown,
|
format_cohort_markdown,
|
||||||
format_gap_json,
|
format_gap_json,
|
||||||
format_gap_markdown,
|
format_gap_markdown,
|
||||||
|
format_reuse_report_json,
|
||||||
|
format_reuse_report_markdown,
|
||||||
select_cohort,
|
select_cohort,
|
||||||
)
|
)
|
||||||
from reuse_surface.establish import (
|
from reuse_surface.establish import (
|
||||||
|
|
@ -753,6 +758,46 @@ def cmd_report_gaps(args: argparse.Namespace) -> int:
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_report_reuse(args: argparse.Namespace) -> int:
|
||||||
|
if args.apply and not args.suggest_relations:
|
||||||
|
print("error: --apply requires --suggest-relations", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
events = collect_reuse_events(hub_url=args.hub_url)
|
||||||
|
report = collect_reuse_report(events, capability_id=args.capability_id)
|
||||||
|
|
||||||
|
suggestions: list[dict[str, Any]] = []
|
||||||
|
if args.suggest_relations:
|
||||||
|
suggestions = collect_reused_by_suggestions(report)
|
||||||
|
|
||||||
|
applied: list[str] = []
|
||||||
|
if args.apply and suggestions:
|
||||||
|
from reuse_surface.patches import apply_patches
|
||||||
|
from reuse_surface.registry import ROOT
|
||||||
|
|
||||||
|
applied = apply_patches(ROOT, suggestions)
|
||||||
|
|
||||||
|
if args.format == "json":
|
||||||
|
payload = dict(report)
|
||||||
|
if args.suggest_relations:
|
||||||
|
payload["reused_by_suggestions"] = suggestions
|
||||||
|
if args.apply:
|
||||||
|
payload["applied"] = applied
|
||||||
|
print(format_reuse_report_json(payload))
|
||||||
|
else:
|
||||||
|
print(format_reuse_report_markdown(report), end="")
|
||||||
|
if args.suggest_relations:
|
||||||
|
print(f"\n## relations.reused_by suggestions ({len(suggestions)})\n")
|
||||||
|
if not suggestions:
|
||||||
|
print("_None — all observed reuse already reflected, or no capabilities owned here._")
|
||||||
|
else:
|
||||||
|
for s in suggestions:
|
||||||
|
print(f"- `{s['capability_id']}`: {s['detail']}")
|
||||||
|
if args.apply:
|
||||||
|
print(f"\nApplied {len(applied)} patch(es).")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def cmd_export(args: argparse.Namespace) -> int:
|
def cmd_export(args: argparse.Namespace) -> int:
|
||||||
index = load_index()
|
index = load_index()
|
||||||
bundle: dict[str, Any] = {
|
bundle: dict[str, Any] = {
|
||||||
|
|
@ -1073,6 +1118,27 @@ def main(argv: list[str] | None = None) -> int:
|
||||||
)
|
)
|
||||||
gaps.set_defaults(func=cmd_report_gaps)
|
gaps.set_defaults(func=cmd_report_gaps)
|
||||||
|
|
||||||
|
reuse = report_sub.add_parser(
|
||||||
|
"reuse",
|
||||||
|
help="reuse telemetry aggregation: consumer counts, outcomes, last-used (REUSE-WP-0019-T05)",
|
||||||
|
)
|
||||||
|
reuse.add_argument("--capability-id", help="filter to one capability")
|
||||||
|
reuse.add_argument("--hub-url", help="hub base URL (or REUSE_SURFACE_URL)")
|
||||||
|
reuse.add_argument("--format", choices=["markdown", "json"], default="markdown")
|
||||||
|
reuse.add_argument(
|
||||||
|
"--suggest-relations",
|
||||||
|
action="store_true",
|
||||||
|
help="also list evidence-gated relations.reused_by suggestions for "
|
||||||
|
"capabilities this repo owns",
|
||||||
|
)
|
||||||
|
reuse.add_argument(
|
||||||
|
"--apply",
|
||||||
|
action="store_true",
|
||||||
|
help="apply --suggest-relations suggestions (requires --suggest-relations; "
|
||||||
|
"never applies without this explicit flag)",
|
||||||
|
)
|
||||||
|
reuse.set_defaults(func=cmd_report_reuse)
|
||||||
|
|
||||||
stats = subparsers.add_parser("stats", help="registry maturity and federation stats")
|
stats = subparsers.add_parser("stats", help="registry maturity and federation stats")
|
||||||
stats.add_argument("--path", help="repo root (default: cwd)")
|
stats.add_argument("--path", help="repo root (default: cwd)")
|
||||||
stats.add_argument(
|
stats.add_argument(
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,15 @@ from typing import Any
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
from reuse_surface.registry import ROOT, level_at_least, load_index, parse_vector
|
from reuse_surface.registry import (
|
||||||
|
ROOT,
|
||||||
|
level_at_least,
|
||||||
|
load_index,
|
||||||
|
load_index_at,
|
||||||
|
parse_front_matter,
|
||||||
|
parse_vector,
|
||||||
|
registry_paths,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _availability_at_most(current: str, maximum: str) -> bool:
|
def _availability_at_most(current: str, maximum: str) -> bool:
|
||||||
|
|
@ -229,3 +237,126 @@ def format_gap_json(report: dict[str, Any]) -> str:
|
||||||
|
|
||||||
def default_roster_path() -> Path:
|
def default_roster_path() -> Path:
|
||||||
return ROOT / "registry/federation/local-repo-roster.yaml"
|
return ROOT / "registry/federation/local-repo-roster.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
def collect_reuse_events(
|
||||||
|
*, repo_root: Path = ROOT, hub_url: str | None = None
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Merges the hub's reuse_events (ecosystem-wide, if reachable) with this
|
||||||
|
repo's local JSONL fallback (covers facts recorded while the hub was
|
||||||
|
down), deduped by (ts, consumer_repo, capability_id, source)."""
|
||||||
|
import urllib.error
|
||||||
|
|
||||||
|
from reuse_surface import hub_client
|
||||||
|
|
||||||
|
events: list[dict[str, Any]] = []
|
||||||
|
try:
|
||||||
|
status, payload = hub_client.hub_list_reuse_events(base_url=hub_url)
|
||||||
|
if status == 200:
|
||||||
|
events.extend(payload.get("events", []))
|
||||||
|
except (ValueError, urllib.error.URLError, TimeoutError, OSError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
telemetry_path = repo_root / "registry" / "telemetry" / "plan-check-events.jsonl"
|
||||||
|
if telemetry_path.exists():
|
||||||
|
for line in telemetry_path.read_text(encoding="utf-8").splitlines():
|
||||||
|
if line.strip():
|
||||||
|
events.append(json.loads(line))
|
||||||
|
|
||||||
|
seen: set[tuple[Any, ...]] = set()
|
||||||
|
deduped: list[dict[str, Any]] = []
|
||||||
|
for event in events:
|
||||||
|
key = (event.get("ts"), event.get("consumer_repo"), event.get("capability_id"), event.get("source"))
|
||||||
|
if key not in seen:
|
||||||
|
seen.add(key)
|
||||||
|
deduped.append(event)
|
||||||
|
return deduped
|
||||||
|
|
||||||
|
|
||||||
|
def collect_reuse_report(
|
||||||
|
events: list[dict[str, Any]], *, capability_id: str | None = None
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Per-capability consumer counts, outcome breakdown, last-used."""
|
||||||
|
by_capability: dict[str, dict[str, Any]] = {}
|
||||||
|
for event in events:
|
||||||
|
cap_id = event.get("capability_id")
|
||||||
|
if not cap_id or (capability_id and cap_id != capability_id):
|
||||||
|
continue
|
||||||
|
bucket = by_capability.setdefault(
|
||||||
|
cap_id,
|
||||||
|
{"consumer_repos": set(), "outcomes": {}, "last_used": None},
|
||||||
|
)
|
||||||
|
bucket["consumer_repos"].add(event["consumer_repo"])
|
||||||
|
outcome = event.get("outcome") or "unspecified"
|
||||||
|
bucket["outcomes"][outcome] = bucket["outcomes"].get(outcome, 0) + 1
|
||||||
|
ts = event.get("ts")
|
||||||
|
if ts and (bucket["last_used"] is None or ts > bucket["last_used"]):
|
||||||
|
bucket["last_used"] = ts
|
||||||
|
|
||||||
|
rows = [
|
||||||
|
{
|
||||||
|
"capability_id": cap_id,
|
||||||
|
"consumer_count": len(bucket["consumer_repos"]),
|
||||||
|
"consumer_repos": sorted(bucket["consumer_repos"]),
|
||||||
|
"outcomes": bucket["outcomes"],
|
||||||
|
"last_used": bucket["last_used"],
|
||||||
|
}
|
||||||
|
for cap_id, bucket in sorted(by_capability.items())
|
||||||
|
]
|
||||||
|
return {"capabilities": rows, "total_events": len(events)}
|
||||||
|
|
||||||
|
|
||||||
|
def format_reuse_report_markdown(report: dict[str, Any]) -> str:
|
||||||
|
lines = ["# Reuse telemetry report", ""]
|
||||||
|
if not report["capabilities"]:
|
||||||
|
lines.append("_No reuse events recorded yet._")
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
for row in report["capabilities"]:
|
||||||
|
outcomes = ", ".join(f"{k}: {v}" for k, v in sorted(row["outcomes"].items()))
|
||||||
|
lines.append(f"## `{row['capability_id']}`")
|
||||||
|
lines.append(f"- Consumers ({row['consumer_count']}): {', '.join(row['consumer_repos'])}")
|
||||||
|
lines.append(f"- Outcomes: {outcomes}")
|
||||||
|
lines.append(f"- Last used: {row['last_used']}")
|
||||||
|
lines.append("")
|
||||||
|
lines.append(f"**{report['total_events']}** total event(s) across **{len(report['capabilities'])}** capability(ies).")
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def format_reuse_report_json(report: dict[str, Any]) -> str:
|
||||||
|
return json.dumps(report, indent=2, sort_keys=True)
|
||||||
|
|
||||||
|
|
||||||
|
def collect_reused_by_suggestions(
|
||||||
|
report: dict[str, Any], *, repo_root: Path = ROOT
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Evidence-gated relation_add suggestions (REUSE-WP-0019-T05): only for
|
||||||
|
capabilities this repo actually owns (in its own index), and only for
|
||||||
|
consumer repos not already listed in relations.reused_by. Suggestions
|
||||||
|
only -- applying them is a separate, explicit step
|
||||||
|
(`report reuse --suggest-relations --apply`), never automatic."""
|
||||||
|
paths = registry_paths(repo_root)
|
||||||
|
if not paths["index"].exists():
|
||||||
|
return []
|
||||||
|
index = load_index_at(paths["index"])
|
||||||
|
index_by_id = {row["id"]: row for row in index.get("capabilities", [])}
|
||||||
|
|
||||||
|
suggestions: list[dict[str, Any]] = []
|
||||||
|
for row in report["capabilities"]:
|
||||||
|
cap_id = row["capability_id"]
|
||||||
|
entry_row = index_by_id.get(cap_id)
|
||||||
|
if not entry_row:
|
||||||
|
continue
|
||||||
|
front_matter = parse_front_matter(repo_root / entry_row["path"])
|
||||||
|
existing = set(front_matter.get("relations", {}).get("reused_by", []))
|
||||||
|
for consumer_repo in row["consumer_repos"]:
|
||||||
|
if consumer_repo in existing:
|
||||||
|
continue
|
||||||
|
suggestions.append(
|
||||||
|
{
|
||||||
|
"capability_id": cap_id,
|
||||||
|
"kind": "relation_add",
|
||||||
|
"detail": f"observed reuse by {consumer_repo} (outcomes: {row['outcomes']})",
|
||||||
|
"value": {"type": "reused_by", "target": consumer_repo},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return suggestions
|
||||||
|
|
@ -122,6 +122,12 @@ properties:
|
||||||
$ref: '#/$defs/capabilityIdList'
|
$ref: '#/$defs/capabilityIdList'
|
||||||
wraps:
|
wraps:
|
||||||
$ref: '#/$defs/capabilityIdList'
|
$ref: '#/$defs/capabilityIdList'
|
||||||
|
reused_by:
|
||||||
|
description: >
|
||||||
|
Consumer repo slugs with observed reuse-event evidence
|
||||||
|
(REUSE-WP-0019-T05), not capability ids. Evidence-gated: only
|
||||||
|
`reuse-surface maintain` proposes/applies these, never silent.
|
||||||
|
$ref: '#/$defs/repoSlugList'
|
||||||
evidence:
|
evidence:
|
||||||
type: object
|
type: object
|
||||||
additionalProperties: false
|
additionalProperties: false
|
||||||
|
|
@ -187,6 +193,11 @@ $defs:
|
||||||
items:
|
items:
|
||||||
type: string
|
type: string
|
||||||
pattern: '^capability(\.[a-z][a-z0-9-]*)+$'
|
pattern: '^capability(\.[a-z][a-z0-9-]*)+$'
|
||||||
|
repoSlugList:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: string
|
||||||
|
minLength: 1
|
||||||
internalMaturityDimension:
|
internalMaturityDimension:
|
||||||
type: object
|
type: object
|
||||||
additionalProperties: false
|
additionalProperties: false
|
||||||
|
|
|
||||||
|
|
@ -807,6 +807,38 @@ reliability_evidence:
|
||||||
- consumer workarounds
|
- consumer workarounds
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Observed-reuse evidence (REUSE-WP-0019-T05)
|
||||||
|
|
||||||
|
`reuse-surface`'s own reuse telemetry (`registry/telemetry/plan-check-events.jsonl`
|
||||||
|
locally, `GET /v1/reuse-events` on the hub — see `specs/PlanCheck.md` and
|
||||||
|
`specs/FederationHubAPI.md`) is a form of `integration_evidence` /
|
||||||
|
`consumer_feedback`: a `reuse` verdict acted on (`outcome: reused` or
|
||||||
|
`extended`) is a real consumer choosing to build on the capability rather
|
||||||
|
than duplicate it.
|
||||||
|
|
||||||
|
What it counts toward, and what it doesn't:
|
||||||
|
|
||||||
|
- **Toward R2 → R3:** a single distinct consumer repo with a recorded
|
||||||
|
`reused`/`extended` outcome is corroborating evidence that the capability
|
||||||
|
"works reliably for normal use" for at least one real consumer — combine
|
||||||
|
with existing test/CI evidence, don't promote on telemetry alone.
|
||||||
|
- **Toward R3 → R4+:** requires **multiple independent consumer repos**
|
||||||
|
(`reuse-surface report reuse` shows `consumer_count`), not just repeated
|
||||||
|
events from the same one, plus no contradicting `bug_reports`/`incidents`
|
||||||
|
evidence for the same period. One repo reusing a capability five times
|
||||||
|
is still one data point, not five.
|
||||||
|
- **Never sufficient alone:** telemetry records that a decision was made
|
||||||
|
and (optionally) an outcome — it is not a substitute for `bug_reports`,
|
||||||
|
`operational_evidence`, or `incident` evidence, and a `reused_by` relation
|
||||||
|
(`report reuse --suggest-relations --apply`) documents *who* reused a
|
||||||
|
capability, not that the reuse *went well*. A capability can be reused
|
||||||
|
by five repos and still be R2 if the outcome evidence includes
|
||||||
|
`outcome: skipped` or recorded incidents.
|
||||||
|
- Promotion must still cite evidence explicitly in `promotion_history`
|
||||||
|
(§ maturity promotion convention) — `reuse-surface maintain`'s
|
||||||
|
`maturity_promote` patch kind is LLM-suggested and review-gated, never
|
||||||
|
auto-applied from telemetry counts alone.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 9. Relationship Between Dimensions
|
## 9. Relationship Between Dimensions
|
||||||
|
|
|
||||||
|
|
@ -5,12 +5,19 @@ import json
|
||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
from reuse_surface.reports import (
|
from reuse_surface.reports import (
|
||||||
cohort_filters_from_args,
|
cohort_filters_from_args,
|
||||||
collect_gap_report,
|
collect_gap_report,
|
||||||
|
collect_reuse_events,
|
||||||
|
collect_reuse_report,
|
||||||
|
collect_reused_by_suggestions,
|
||||||
format_cohort_json,
|
format_cohort_json,
|
||||||
format_cohort_markdown,
|
format_cohort_markdown,
|
||||||
format_gap_markdown,
|
format_gap_markdown,
|
||||||
|
format_reuse_report_json,
|
||||||
|
format_reuse_report_markdown,
|
||||||
select_cohort,
|
select_cohort,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -190,3 +197,243 @@ def test_cmd_report_cohorts_markdown(monkeypatch):
|
||||||
)
|
)
|
||||||
exit_code = main(["report", "cohorts", "--planning-min", "D5"])
|
exit_code = main(["report", "cohorts", "--planning-min", "D5"])
|
||||||
assert exit_code == 0
|
assert exit_code == 0
|
||||||
|
|
||||||
|
# --- T05: reuse telemetry aggregation ---
|
||||||
|
|
||||||
|
SAMPLE_REUSE_EVENTS = [
|
||||||
|
{
|
||||||
|
"ts": "2026-07-01T00:00:00Z",
|
||||||
|
"consumer_repo": "repo-a",
|
||||||
|
"capability_id": "capability.infotech.issue-tracking",
|
||||||
|
"verdict": "reuse",
|
||||||
|
"outcome": "reused",
|
||||||
|
"source": "plan-check",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ts": "2026-07-02T00:00:00Z",
|
||||||
|
"consumer_repo": "repo-b",
|
||||||
|
"capability_id": "capability.infotech.issue-tracking",
|
||||||
|
"verdict": "reuse",
|
||||||
|
"outcome": "reused",
|
||||||
|
"source": "manual",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ts": "2026-07-03T00:00:00Z",
|
||||||
|
"consumer_repo": "repo-a",
|
||||||
|
"capability_id": "capability.audit.event-retain",
|
||||||
|
"verdict": "extend",
|
||||||
|
"outcome": "extended",
|
||||||
|
"source": "plan-check",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reuse_report_aggregates_per_capability():
|
||||||
|
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||||
|
assert report["total_events"] == 3
|
||||||
|
by_id = {row["capability_id"]: row for row in report["capabilities"]}
|
||||||
|
assert by_id["capability.infotech.issue-tracking"]["consumer_count"] == 2
|
||||||
|
assert by_id["capability.infotech.issue-tracking"]["consumer_repos"] == ["repo-a", "repo-b"]
|
||||||
|
assert by_id["capability.infotech.issue-tracking"]["outcomes"] == {"reused": 2}
|
||||||
|
assert by_id["capability.infotech.issue-tracking"]["last_used"] == "2026-07-02T00:00:00Z"
|
||||||
|
assert by_id["capability.audit.event-retain"]["consumer_count"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reuse_report_filters_by_capability_id():
|
||||||
|
report = collect_reuse_report(SAMPLE_REUSE_EVENTS, capability_id="capability.audit.event-retain")
|
||||||
|
assert len(report["capabilities"]) == 1
|
||||||
|
assert report["capabilities"][0]["capability_id"] == "capability.audit.event-retain"
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reuse_report_ignores_events_without_capability_id():
|
||||||
|
events = SAMPLE_REUSE_EVENTS + [
|
||||||
|
{"ts": "2026-07-04T00:00:00Z", "consumer_repo": "repo-c", "capability_id": None, "verdict": "new", "outcome": "new", "source": "manual"}
|
||||||
|
]
|
||||||
|
report = collect_reuse_report(events)
|
||||||
|
assert report["total_events"] == 4
|
||||||
|
assert len(report["capabilities"]) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_format_reuse_report_markdown_empty():
|
||||||
|
output = format_reuse_report_markdown({"capabilities": [], "total_events": 0})
|
||||||
|
assert "No reuse events recorded yet" in output
|
||||||
|
|
||||||
|
|
||||||
|
def test_format_reuse_report_markdown_nonempty():
|
||||||
|
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||||
|
output = format_reuse_report_markdown(report)
|
||||||
|
assert "capability.infotech.issue-tracking" in output
|
||||||
|
assert "3" in output
|
||||||
|
|
||||||
|
|
||||||
|
def test_format_reuse_report_json_roundtrips():
|
||||||
|
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||||
|
payload = json.loads(format_reuse_report_json(report))
|
||||||
|
assert payload["total_events"] == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reuse_events_merges_hub_and_local(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"reuse_surface.hub_client.hub_list_reuse_events",
|
||||||
|
lambda capability_id=None, base_url=None: (200, {"events": [SAMPLE_REUSE_EVENTS[0]]}),
|
||||||
|
)
|
||||||
|
telemetry_dir = tmp_path / "registry" / "telemetry"
|
||||||
|
telemetry_dir.mkdir(parents=True)
|
||||||
|
(telemetry_dir / "plan-check-events.jsonl").write_text(
|
||||||
|
json.dumps(SAMPLE_REUSE_EVENTS[1], sort_keys=True) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
events = collect_reuse_events(repo_root=tmp_path)
|
||||||
|
assert len(events) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reuse_events_dedupes_identical_facts(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"reuse_surface.hub_client.hub_list_reuse_events",
|
||||||
|
lambda capability_id=None, base_url=None: (200, {"events": [SAMPLE_REUSE_EVENTS[0]]}),
|
||||||
|
)
|
||||||
|
telemetry_dir = tmp_path / "registry" / "telemetry"
|
||||||
|
telemetry_dir.mkdir(parents=True)
|
||||||
|
(telemetry_dir / "plan-check-events.jsonl").write_text(
|
||||||
|
json.dumps(SAMPLE_REUSE_EVENTS[0], sort_keys=True) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
events = collect_reuse_events(repo_root=tmp_path)
|
||||||
|
assert len(events) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reuse_events_degrades_gracefully_when_hub_unreachable(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.delenv("REUSE_SURFACE_URL", raising=False)
|
||||||
|
events = collect_reuse_events(repo_root=tmp_path)
|
||||||
|
assert events == []
|
||||||
|
|
||||||
|
|
||||||
|
def _write_entry_with_relations(tmp_path: Path, cap_id: str, reused_by: list[str]) -> str:
|
||||||
|
rel = "registry/capabilities/capability-demo-sample.md"
|
||||||
|
front_matter = {
|
||||||
|
"id": cap_id,
|
||||||
|
"name": "Sample",
|
||||||
|
"summary": "Sample",
|
||||||
|
"owner": "demo",
|
||||||
|
"status": "draft",
|
||||||
|
"domain": "helix_forge",
|
||||||
|
"tags": ["demo"],
|
||||||
|
"maturity": {
|
||||||
|
"discovery": {"current": "D3", "target": "D5", "confidence": "low"},
|
||||||
|
"availability": {"current": "A3", "target": "A4", "confidence": "low"},
|
||||||
|
},
|
||||||
|
"external_evidence": {
|
||||||
|
"completeness": {"level": "C2", "confidence": "low"},
|
||||||
|
"reliability": {"level": "R2", "confidence": "low"},
|
||||||
|
},
|
||||||
|
"discovery": {"intent": "demo", "includes": [], "excludes": []},
|
||||||
|
"availability": {
|
||||||
|
"current_level": "A3",
|
||||||
|
"target_level": "A4",
|
||||||
|
"current_artifacts": [],
|
||||||
|
"consumption_modes": ["informational"],
|
||||||
|
},
|
||||||
|
"relations": {"depends_on": [], "supports": [], "related_to": [], "reused_by": reused_by},
|
||||||
|
"evidence": {"documentation": [], "tests": []},
|
||||||
|
"consumer_guidance": {
|
||||||
|
"recommended_for": [],
|
||||||
|
"not_recommended_for": [],
|
||||||
|
"known_limitations": [],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
path = tmp_path / rel
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text("---\n" + yaml.safe_dump(front_matter, sort_keys=False) + "---\n", encoding="utf-8")
|
||||||
|
return rel
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reused_by_suggestions_skips_already_listed_consumers(tmp_path):
|
||||||
|
rel = _write_entry_with_relations(tmp_path, "capability.infotech.issue-tracking", reused_by=["repo-a"])
|
||||||
|
index_path = tmp_path / "registry" / "indexes" / "capabilities.yaml"
|
||||||
|
index_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
index_path.write_text(
|
||||||
|
yaml.safe_dump({
|
||||||
|
"version": 1, "domain": "helix_forge", "updated": "2026-07-01",
|
||||||
|
"capabilities": [{"id": "capability.infotech.issue-tracking", "path": rel, "vector": "D3/A3/C2/R2"}],
|
||||||
|
}),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||||
|
suggestions = collect_reused_by_suggestions(report, repo_root=tmp_path)
|
||||||
|
# repo-a already listed -> only repo-b should be suggested
|
||||||
|
assert len(suggestions) == 1
|
||||||
|
assert suggestions[0]["value"] == {"type": "reused_by", "target": "repo-b"}
|
||||||
|
assert suggestions[0]["kind"] == "relation_add"
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reused_by_suggestions_skips_capabilities_not_owned_locally(tmp_path):
|
||||||
|
# empty local index -- no capabilities owned here at all
|
||||||
|
index_path = tmp_path / "registry" / "indexes" / "capabilities.yaml"
|
||||||
|
index_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
index_path.write_text(
|
||||||
|
yaml.safe_dump({"version": 1, "domain": "helix_forge", "updated": "2026-07-01", "capabilities": []}),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||||
|
suggestions = collect_reused_by_suggestions(report, repo_root=tmp_path)
|
||||||
|
assert suggestions == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_reused_by_suggestions_no_index_returns_empty(tmp_path):
|
||||||
|
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||||
|
suggestions = collect_reused_by_suggestions(report, repo_root=tmp_path)
|
||||||
|
assert suggestions == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_report_reuse_markdown(monkeypatch):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||||
|
exit_code = main(["report", "reuse"])
|
||||||
|
assert exit_code == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_report_reuse_json(monkeypatch, capsys):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||||
|
exit_code = main(["report", "reuse", "--format", "json"])
|
||||||
|
assert exit_code == 0
|
||||||
|
payload = json.loads(capsys.readouterr().out)
|
||||||
|
assert payload["total_events"] == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_report_reuse_apply_without_suggest_relations_errors(monkeypatch):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: [])
|
||||||
|
exit_code = main(["report", "reuse", "--apply"])
|
||||||
|
assert exit_code == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_report_reuse_suggest_relations(monkeypatch, capsys):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"reuse_surface.cli.collect_reused_by_suggestions",
|
||||||
|
lambda report: [{"capability_id": "capability.infotech.issue-tracking", "kind": "relation_add", "detail": "observed reuse by repo-a", "value": {"type": "reused_by", "target": "repo-a"}}],
|
||||||
|
)
|
||||||
|
exit_code = main(["report", "reuse", "--suggest-relations"])
|
||||||
|
assert exit_code == 0
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "repo-a" in out
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_report_reuse_apply_calls_apply_patches(monkeypatch):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||||
|
suggestion = {"capability_id": "capability.infotech.issue-tracking", "kind": "relation_add", "detail": "x", "value": {"type": "reused_by", "target": "repo-a"}}
|
||||||
|
monkeypatch.setattr("reuse_surface.cli.collect_reused_by_suggestions", lambda report: [suggestion])
|
||||||
|
applied_calls = []
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"reuse_surface.patches.apply_patches",
|
||||||
|
lambda repo_root, patches: applied_calls.append(patches) or ["applied one"],
|
||||||
|
)
|
||||||
|
exit_code = main(["report", "reuse", "--suggest-relations", "--apply"])
|
||||||
|
assert exit_code == 0
|
||||||
|
assert applied_calls == [[suggestion]]
|
||||||
|
|
|
||||||
|
|
@ -170,6 +170,30 @@ it isn't. Off by default so `report gaps` stays fast and offline-safe.
|
||||||
Workstation roster report: publish blockers, empty scaffolds, seed-ready repos,
|
Workstation roster report: publish blockers, empty scaffolds, seed-ready repos,
|
||||||
and local index owner stubs pending dedup.
|
and local index owner stubs pending dedup.
|
||||||
|
|
||||||
|
### report reuse
|
||||||
|
|
||||||
|
Reuse telemetry aggregation (REUSE-WP-0019-T05): per-capability consumer
|
||||||
|
counts, outcome breakdown, last-used, merged from the hub's
|
||||||
|
`GET /v1/reuse-events` (if reachable) and this repo's local
|
||||||
|
`registry/telemetry/plan-check-events.jsonl` fallback, deduped.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
reuse-surface report reuse
|
||||||
|
reuse-surface report reuse --capability-id capability.infotech.issue-tracking
|
||||||
|
reuse-surface report reuse --format json
|
||||||
|
reuse-surface report reuse --suggest-relations
|
||||||
|
reuse-surface report reuse --suggest-relations --apply
|
||||||
|
```
|
||||||
|
|
||||||
|
`--suggest-relations` lists evidence-gated `relations.reused_by` suggestions
|
||||||
|
for capabilities this repo owns (skips consumer repos already listed).
|
||||||
|
`--apply` applies them via the same `apply_patches` mechanism `maintain`
|
||||||
|
uses for `relation_add` patches — requires `--suggest-relations`, and never
|
||||||
|
runs without it: nothing is written unless real observed-reuse events exist
|
||||||
|
*and* an operator explicitly passes `--apply`. See
|
||||||
|
`specs/CapabilityMaturityStandard.md` §8.9 for what observed-reuse evidence
|
||||||
|
does (and doesn't) count toward R-axis promotion.
|
||||||
|
|
||||||
### stats
|
### stats
|
||||||
|
|
||||||
Registry maturity aggregates and federation readiness.
|
Registry maturity aggregates and federation readiness.
|
||||||
|
|
@ -280,6 +304,7 @@ Stable IDs and maturity fields are preserved for agent consumption (UC-RS-019).
|
||||||
| Relation graph | `reuse-surface graph` |
|
| Relation graph | `reuse-surface graph` |
|
||||||
| Query before building | `reuse-surface plan-check --intent "..."` |
|
| Query before building | `reuse-surface plan-check --intent "..."` |
|
||||||
| Record a reuse fact retroactively | `reuse-surface record-reuse` |
|
| Record a reuse fact retroactively | `reuse-surface record-reuse` |
|
||||||
|
| Aggregate reuse telemetry into evidence | `reuse-surface report reuse --suggest-relations` |
|
||||||
|
|
||||||
## Related use cases
|
## Related use cases
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -310,19 +310,63 @@ Implemented:
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: REUSE-WP-0019-T05
|
id: REUSE-WP-0019-T05
|
||||||
status: wait
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7"
|
state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7"
|
||||||
```
|
```
|
||||||
|
|
||||||
Blocked on T04 plus initial event volume.
|
"Blocked on T04 plus initial event volume" — T04 is done, and the
|
||||||
|
*tooling* doesn't actually need to wait for real ecosystem adoption volume
|
||||||
|
to be built and tested correctly (an empty/near-empty dataset is itself a
|
||||||
|
valid, tested case: `report reuse` prints "No reuse events recorded yet"
|
||||||
|
rather than erroring). Real cross-repo adoption volume ramping up is an
|
||||||
|
ecosystem-timing question, not a coding blocker — implemented now so the
|
||||||
|
tooling is ready the moment volume exists.
|
||||||
|
|
||||||
- `reuse-surface report reuse`: per-capability consumer counts, outcomes,
|
Implemented:
|
||||||
last-used; feeds `reused_by` relation suggestions via the WP-0016
|
|
||||||
maintain/patch pipeline (evidence-gated, never silent promotion)
|
- `reuse_surface/reports.py`: `collect_reuse_events()` (merges the hub's
|
||||||
- Maturity standard note: what observed-reuse evidence counts toward R2/R3+
|
`GET /v1/reuse-events`, if reachable, with this repo's local JSONL
|
||||||
(`specs/CapabilityMaturityStandard.md` amendment)
|
fallback, deduped by `(ts, consumer_repo, capability_id, source)`),
|
||||||
- Catalog + graph surface consumer counts
|
`collect_reuse_report()` (per-capability consumer counts, outcome
|
||||||
|
breakdown, last-used), `format_reuse_report_markdown/json`
|
||||||
|
- `collect_reused_by_suggestions()`: evidence-gated `relation_add`
|
||||||
|
suggestions — only for capabilities this repo actually owns (checked
|
||||||
|
against the local index, not invented), only for consumer repos not
|
||||||
|
already listed. Reuses the *existing* `patches.py:apply_patches`
|
||||||
|
mechanism (which already handled `relation_add`, not in
|
||||||
|
`SAFE_DETERMINISTIC_KINDS`, so never auto-applied by `maintain --auto`)
|
||||||
|
rather than building a new apply path
|
||||||
|
- New CLI: `reuse-surface report reuse [--capability-id] [--format]
|
||||||
|
[--suggest-relations] [--apply]` — `--apply` requires
|
||||||
|
`--suggest-relations` and is the only thing that writes; nothing happens
|
||||||
|
automatically from telemetry alone
|
||||||
|
- Schema: added `relations.reused_by` to `schemas/capability.schema.yaml`
|
||||||
|
(a new `repoSlugList` `$defs` type — distinct from the existing
|
||||||
|
`capabilityIdList` relations, since reused-by targets are consumer repo
|
||||||
|
slugs, not capability ids)
|
||||||
|
- `specs/CapabilityMaturityStandard.md` §8.9 amended: what observed-reuse
|
||||||
|
evidence counts toward R2→R3 (single corroborating consumer) vs R3→R4+
|
||||||
|
(multiple *independent* consumers, `report reuse`'s `consumer_count`),
|
||||||
|
and explicitly what it never substitutes for (bug reports, incidents,
|
||||||
|
explicit `promotion_history` citations — `maturity_promote` stays
|
||||||
|
LLM-suggested and review-gated)
|
||||||
|
- 19 new pytest cases (`test_reports.py`); 162 total pass
|
||||||
|
- **Live-verified** with synthetic local-JSONL events against a real
|
||||||
|
capability entry: `report reuse --suggest-relations --apply` correctly
|
||||||
|
wrote `relations.reused_by` into the entry's front matter via the real
|
||||||
|
`apply_patches` path (confirmed via `git diff`, then reverted since it
|
||||||
|
was a smoke test, not a real fact)
|
||||||
|
|
||||||
|
**Deliberately deferred, not silently dropped:** "Catalog + graph surface
|
||||||
|
consumer counts." `graph.py`'s `RELATION_TYPES` graph is capability-to-
|
||||||
|
capability edges; `reused_by` targets are repo slugs, a different
|
||||||
|
namespace entirely — forcing it into that edge model would either error or
|
||||||
|
produce meaningless nodes. `catalog.py` doesn't currently parse full entry
|
||||||
|
front matter per capability at all (works off the index). Surfacing
|
||||||
|
consumer counts in either artifact is a real, separate rendering-layer
|
||||||
|
task, not a natural extension of what T05 already built — left for a
|
||||||
|
follow-up rather than rushed in.
|
||||||
|
|
||||||
## Freshness Monitoring, Docs, SCOPE
|
## Freshness Monitoring, Docs, SCOPE
|
||||||
|
|
||||||
|
|
@ -348,8 +392,8 @@ state_hub_task_id: "a9f44d45-91e2-4b43-909f-30a5f906cf3b"
|
||||||
- [x] No hardcoded forge host in code or sources.yaml; `migrate-host` tested (T01, 2026-07-07)
|
- [x] No hardcoded forge host in code or sources.yaml; `migrate-host` tested (T01, 2026-07-07)
|
||||||
- [x] Push to a sibling repo's `registry/indexes/` recomposes the hub index without manual action (webhook), with scheduled fallback in place (T02/T03, 2026-07-07 — both live-verified end to end)
|
- [x] Push to a sibling repo's `registry/indexes/` recomposes the hub index without manual action (webhook), with scheduled fallback in place (T02/T03, 2026-07-07 — both live-verified end to end)
|
||||||
- [x] This repo's CI runs on Forgejo Actions (`.forgejo/workflows/`) (T03, 2026-07-07 — `ci.yml`/`ci-smoke.yaml`/`image.yaml` all verified green on the live push)
|
- [x] This repo's CI runs on Forgejo Actions (`.forgejo/workflows/`) (T03, 2026-07-07 — `ci.yml`/`ci-smoke.yaml`/`image.yaml` all verified green on the live push)
|
||||||
- [x] Reuse events recordable via hub API and CLI (T04, 2026-07-08 — live-verified); `report reuse` aggregation is T05
|
- [x] Reuse events recordable via hub API and CLI (T04, 2026-07-08 — live-verified); `report reuse` aggregation done in T05
|
||||||
- [ ] R-axis evidence rules for observed reuse documented in the maturity standard
|
- [x] R-axis evidence rules for observed reuse documented in the maturity standard (T05, 2026-07-08 — `specs/CapabilityMaturityStandard.md` §8.9)
|
||||||
- [x] Hub freshness visible (`composed_at`, stale flag) in API and stats (T02, 2026-07-07)
|
- [x] Hub freshness visible (`composed_at`, stale flag) in API and stats (T02, 2026-07-07)
|
||||||
|
|
||||||
## Out of scope
|
## Out of scope
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue