REUSE-WP-0019-T05: reuse telemetry aggregation into R-axis evidence
reuse_surface/reports.py: collect_reuse_events() merges the hub's GET /v1/reuse-events (if reachable) with this repo's local JSONL fallback, deduped. collect_reuse_report() aggregates per-capability consumer counts, outcome breakdown, and last-used. collect_reused_by_suggestions() proposes evidence-gated relation_add patches -- only for capabilities this repo owns, only for consumer repos not already listed -- reusing the existing patches.py:apply_patches mechanism (relation_add already isn't in SAFE_DETERMINISTIC_KINDS, so it was already never auto-applied by maintain --auto). New CLI: reuse-surface report reuse [--capability-id] [--format] [--suggest-relations] [--apply]. --apply requires --suggest-relations and is the only thing that writes -- nothing happens automatically from telemetry alone. schemas/capability.schema.yaml: added relations.reused_by as a new repoSlugList type, distinct from the existing capability-id relations, since reused-by targets are consumer repo slugs. specs/CapabilityMaturityStandard.md Sec8.9: what observed-reuse evidence counts toward R2->R3 (single corroborating consumer) vs R3->R4+ (multiple independent consumers) and what it never substitutes for. 19 new pytest cases, 162 total pass. Live-verified with synthetic local events against a real capability entry: --suggest-relations --apply correctly wrote relations.reused_by via the real apply_patches path (reverted after, since it was a smoke test). Deliberately deferred: surfacing consumer counts in the catalog/graph -- graph.py's relation model is capability-to-capability edges, a different namespace than repo-slug reused_by targets; catalog.py doesn't currently parse full front matter per entry. Left for a follow-up. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
e6e275ce79
commit
bca7165e02
8 changed files with 576 additions and 13 deletions
7
SCOPE.md
7
SCOPE.md
|
|
@ -90,6 +90,13 @@ The MVP registry foundation, CLI tooling (REUSE-WP-0003), federation stack
|
|||
`POST /v1/reuse-events` when the hub is reachable, falling back to
|
||||
`registry/telemetry/plan-check-events.jsonl` otherwise (same schema
|
||||
either way); `GET /v1/reuse-events?capability_id=` aggregates them
|
||||
- **Aggregate reuse telemetry into R-axis evidence** (REUSE-WP-0019-T05) —
|
||||
`reuse-surface report reuse` shows per-capability consumer counts,
|
||||
outcomes, and last-used; `--suggest-relations` proposes evidence-gated
|
||||
`relations.reused_by` patches (via the same `apply_patches` mechanism
|
||||
`maintain` uses), applied only with an explicit `--apply` — never
|
||||
automatic. `specs/CapabilityMaturityStandard.md` §8.9 documents what
|
||||
observed-reuse evidence does (and doesn't) count toward R2/R3+
|
||||
|
||||
Registry **tooling** availability is **A4** (CLI plus hosted hub HTTP API).
|
||||
Registry **authoring** remains Markdown-first; consumption combines entries, the
|
||||
|
|
|
|||
|
|
@ -35,11 +35,16 @@ from reuse_surface.plan_check import (
|
|||
from reuse_surface.reports import (
|
||||
cohort_filters_from_args,
|
||||
collect_gap_report,
|
||||
collect_reuse_events,
|
||||
collect_reuse_report,
|
||||
collect_reused_by_suggestions,
|
||||
default_roster_path,
|
||||
format_cohort_json,
|
||||
format_cohort_markdown,
|
||||
format_gap_json,
|
||||
format_gap_markdown,
|
||||
format_reuse_report_json,
|
||||
format_reuse_report_markdown,
|
||||
select_cohort,
|
||||
)
|
||||
from reuse_surface.establish import (
|
||||
|
|
@ -753,6 +758,46 @@ def cmd_report_gaps(args: argparse.Namespace) -> int:
|
|||
return 0
|
||||
|
||||
|
||||
def cmd_report_reuse(args: argparse.Namespace) -> int:
|
||||
if args.apply and not args.suggest_relations:
|
||||
print("error: --apply requires --suggest-relations", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
events = collect_reuse_events(hub_url=args.hub_url)
|
||||
report = collect_reuse_report(events, capability_id=args.capability_id)
|
||||
|
||||
suggestions: list[dict[str, Any]] = []
|
||||
if args.suggest_relations:
|
||||
suggestions = collect_reused_by_suggestions(report)
|
||||
|
||||
applied: list[str] = []
|
||||
if args.apply and suggestions:
|
||||
from reuse_surface.patches import apply_patches
|
||||
from reuse_surface.registry import ROOT
|
||||
|
||||
applied = apply_patches(ROOT, suggestions)
|
||||
|
||||
if args.format == "json":
|
||||
payload = dict(report)
|
||||
if args.suggest_relations:
|
||||
payload["reused_by_suggestions"] = suggestions
|
||||
if args.apply:
|
||||
payload["applied"] = applied
|
||||
print(format_reuse_report_json(payload))
|
||||
else:
|
||||
print(format_reuse_report_markdown(report), end="")
|
||||
if args.suggest_relations:
|
||||
print(f"\n## relations.reused_by suggestions ({len(suggestions)})\n")
|
||||
if not suggestions:
|
||||
print("_None — all observed reuse already reflected, or no capabilities owned here._")
|
||||
else:
|
||||
for s in suggestions:
|
||||
print(f"- `{s['capability_id']}`: {s['detail']}")
|
||||
if args.apply:
|
||||
print(f"\nApplied {len(applied)} patch(es).")
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_export(args: argparse.Namespace) -> int:
|
||||
index = load_index()
|
||||
bundle: dict[str, Any] = {
|
||||
|
|
@ -1073,6 +1118,27 @@ def main(argv: list[str] | None = None) -> int:
|
|||
)
|
||||
gaps.set_defaults(func=cmd_report_gaps)
|
||||
|
||||
reuse = report_sub.add_parser(
|
||||
"reuse",
|
||||
help="reuse telemetry aggregation: consumer counts, outcomes, last-used (REUSE-WP-0019-T05)",
|
||||
)
|
||||
reuse.add_argument("--capability-id", help="filter to one capability")
|
||||
reuse.add_argument("--hub-url", help="hub base URL (or REUSE_SURFACE_URL)")
|
||||
reuse.add_argument("--format", choices=["markdown", "json"], default="markdown")
|
||||
reuse.add_argument(
|
||||
"--suggest-relations",
|
||||
action="store_true",
|
||||
help="also list evidence-gated relations.reused_by suggestions for "
|
||||
"capabilities this repo owns",
|
||||
)
|
||||
reuse.add_argument(
|
||||
"--apply",
|
||||
action="store_true",
|
||||
help="apply --suggest-relations suggestions (requires --suggest-relations; "
|
||||
"never applies without this explicit flag)",
|
||||
)
|
||||
reuse.set_defaults(func=cmd_report_reuse)
|
||||
|
||||
stats = subparsers.add_parser("stats", help="registry maturity and federation stats")
|
||||
stats.add_argument("--path", help="repo root (default: cwd)")
|
||||
stats.add_argument(
|
||||
|
|
|
|||
|
|
@ -6,7 +6,15 @@ from typing import Any
|
|||
|
||||
import yaml
|
||||
|
||||
from reuse_surface.registry import ROOT, level_at_least, load_index, parse_vector
|
||||
from reuse_surface.registry import (
|
||||
ROOT,
|
||||
level_at_least,
|
||||
load_index,
|
||||
load_index_at,
|
||||
parse_front_matter,
|
||||
parse_vector,
|
||||
registry_paths,
|
||||
)
|
||||
|
||||
|
||||
def _availability_at_most(current: str, maximum: str) -> bool:
|
||||
|
|
@ -228,4 +236,127 @@ def format_gap_json(report: dict[str, Any]) -> str:
|
|||
|
||||
|
||||
def default_roster_path() -> Path:
|
||||
return ROOT / "registry/federation/local-repo-roster.yaml"
|
||||
return ROOT / "registry/federation/local-repo-roster.yaml"
|
||||
|
||||
|
||||
def collect_reuse_events(
|
||||
*, repo_root: Path = ROOT, hub_url: str | None = None
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Merges the hub's reuse_events (ecosystem-wide, if reachable) with this
|
||||
repo's local JSONL fallback (covers facts recorded while the hub was
|
||||
down), deduped by (ts, consumer_repo, capability_id, source)."""
|
||||
import urllib.error
|
||||
|
||||
from reuse_surface import hub_client
|
||||
|
||||
events: list[dict[str, Any]] = []
|
||||
try:
|
||||
status, payload = hub_client.hub_list_reuse_events(base_url=hub_url)
|
||||
if status == 200:
|
||||
events.extend(payload.get("events", []))
|
||||
except (ValueError, urllib.error.URLError, TimeoutError, OSError):
|
||||
pass
|
||||
|
||||
telemetry_path = repo_root / "registry" / "telemetry" / "plan-check-events.jsonl"
|
||||
if telemetry_path.exists():
|
||||
for line in telemetry_path.read_text(encoding="utf-8").splitlines():
|
||||
if line.strip():
|
||||
events.append(json.loads(line))
|
||||
|
||||
seen: set[tuple[Any, ...]] = set()
|
||||
deduped: list[dict[str, Any]] = []
|
||||
for event in events:
|
||||
key = (event.get("ts"), event.get("consumer_repo"), event.get("capability_id"), event.get("source"))
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
deduped.append(event)
|
||||
return deduped
|
||||
|
||||
|
||||
def collect_reuse_report(
|
||||
events: list[dict[str, Any]], *, capability_id: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
"""Per-capability consumer counts, outcome breakdown, last-used."""
|
||||
by_capability: dict[str, dict[str, Any]] = {}
|
||||
for event in events:
|
||||
cap_id = event.get("capability_id")
|
||||
if not cap_id or (capability_id and cap_id != capability_id):
|
||||
continue
|
||||
bucket = by_capability.setdefault(
|
||||
cap_id,
|
||||
{"consumer_repos": set(), "outcomes": {}, "last_used": None},
|
||||
)
|
||||
bucket["consumer_repos"].add(event["consumer_repo"])
|
||||
outcome = event.get("outcome") or "unspecified"
|
||||
bucket["outcomes"][outcome] = bucket["outcomes"].get(outcome, 0) + 1
|
||||
ts = event.get("ts")
|
||||
if ts and (bucket["last_used"] is None or ts > bucket["last_used"]):
|
||||
bucket["last_used"] = ts
|
||||
|
||||
rows = [
|
||||
{
|
||||
"capability_id": cap_id,
|
||||
"consumer_count": len(bucket["consumer_repos"]),
|
||||
"consumer_repos": sorted(bucket["consumer_repos"]),
|
||||
"outcomes": bucket["outcomes"],
|
||||
"last_used": bucket["last_used"],
|
||||
}
|
||||
for cap_id, bucket in sorted(by_capability.items())
|
||||
]
|
||||
return {"capabilities": rows, "total_events": len(events)}
|
||||
|
||||
|
||||
def format_reuse_report_markdown(report: dict[str, Any]) -> str:
|
||||
lines = ["# Reuse telemetry report", ""]
|
||||
if not report["capabilities"]:
|
||||
lines.append("_No reuse events recorded yet._")
|
||||
return "\n".join(lines) + "\n"
|
||||
for row in report["capabilities"]:
|
||||
outcomes = ", ".join(f"{k}: {v}" for k, v in sorted(row["outcomes"].items()))
|
||||
lines.append(f"## `{row['capability_id']}`")
|
||||
lines.append(f"- Consumers ({row['consumer_count']}): {', '.join(row['consumer_repos'])}")
|
||||
lines.append(f"- Outcomes: {outcomes}")
|
||||
lines.append(f"- Last used: {row['last_used']}")
|
||||
lines.append("")
|
||||
lines.append(f"**{report['total_events']}** total event(s) across **{len(report['capabilities'])}** capability(ies).")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def format_reuse_report_json(report: dict[str, Any]) -> str:
|
||||
return json.dumps(report, indent=2, sort_keys=True)
|
||||
|
||||
|
||||
def collect_reused_by_suggestions(
|
||||
report: dict[str, Any], *, repo_root: Path = ROOT
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Evidence-gated relation_add suggestions (REUSE-WP-0019-T05): only for
|
||||
capabilities this repo actually owns (in its own index), and only for
|
||||
consumer repos not already listed in relations.reused_by. Suggestions
|
||||
only -- applying them is a separate, explicit step
|
||||
(`report reuse --suggest-relations --apply`), never automatic."""
|
||||
paths = registry_paths(repo_root)
|
||||
if not paths["index"].exists():
|
||||
return []
|
||||
index = load_index_at(paths["index"])
|
||||
index_by_id = {row["id"]: row for row in index.get("capabilities", [])}
|
||||
|
||||
suggestions: list[dict[str, Any]] = []
|
||||
for row in report["capabilities"]:
|
||||
cap_id = row["capability_id"]
|
||||
entry_row = index_by_id.get(cap_id)
|
||||
if not entry_row:
|
||||
continue
|
||||
front_matter = parse_front_matter(repo_root / entry_row["path"])
|
||||
existing = set(front_matter.get("relations", {}).get("reused_by", []))
|
||||
for consumer_repo in row["consumer_repos"]:
|
||||
if consumer_repo in existing:
|
||||
continue
|
||||
suggestions.append(
|
||||
{
|
||||
"capability_id": cap_id,
|
||||
"kind": "relation_add",
|
||||
"detail": f"observed reuse by {consumer_repo} (outcomes: {row['outcomes']})",
|
||||
"value": {"type": "reused_by", "target": consumer_repo},
|
||||
}
|
||||
)
|
||||
return suggestions
|
||||
|
|
@ -122,6 +122,12 @@ properties:
|
|||
$ref: '#/$defs/capabilityIdList'
|
||||
wraps:
|
||||
$ref: '#/$defs/capabilityIdList'
|
||||
reused_by:
|
||||
description: >
|
||||
Consumer repo slugs with observed reuse-event evidence
|
||||
(REUSE-WP-0019-T05), not capability ids. Evidence-gated: only
|
||||
`reuse-surface maintain` proposes/applies these, never silent.
|
||||
$ref: '#/$defs/repoSlugList'
|
||||
evidence:
|
||||
type: object
|
||||
additionalProperties: false
|
||||
|
|
@ -187,6 +193,11 @@ $defs:
|
|||
items:
|
||||
type: string
|
||||
pattern: '^capability(\.[a-z][a-z0-9-]*)+$'
|
||||
repoSlugList:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
minLength: 1
|
||||
internalMaturityDimension:
|
||||
type: object
|
||||
additionalProperties: false
|
||||
|
|
|
|||
|
|
@ -807,6 +807,38 @@ reliability_evidence:
|
|||
- consumer workarounds
|
||||
```
|
||||
|
||||
#### Observed-reuse evidence (REUSE-WP-0019-T05)
|
||||
|
||||
`reuse-surface`'s own reuse telemetry (`registry/telemetry/plan-check-events.jsonl`
|
||||
locally, `GET /v1/reuse-events` on the hub — see `specs/PlanCheck.md` and
|
||||
`specs/FederationHubAPI.md`) is a form of `integration_evidence` /
|
||||
`consumer_feedback`: a `reuse` verdict acted on (`outcome: reused` or
|
||||
`extended`) is a real consumer choosing to build on the capability rather
|
||||
than duplicate it.
|
||||
|
||||
What it counts toward, and what it doesn't:
|
||||
|
||||
- **Toward R2 → R3:** a single distinct consumer repo with a recorded
|
||||
`reused`/`extended` outcome is corroborating evidence that the capability
|
||||
"works reliably for normal use" for at least one real consumer — combine
|
||||
with existing test/CI evidence, don't promote on telemetry alone.
|
||||
- **Toward R3 → R4+:** requires **multiple independent consumer repos**
|
||||
(`reuse-surface report reuse` shows `consumer_count`), not just repeated
|
||||
events from the same one, plus no contradicting `bug_reports`/`incidents`
|
||||
evidence for the same period. One repo reusing a capability five times
|
||||
is still one data point, not five.
|
||||
- **Never sufficient alone:** telemetry records that a decision was made
|
||||
and (optionally) an outcome — it is not a substitute for `bug_reports`,
|
||||
`operational_evidence`, or `incident` evidence, and a `reused_by` relation
|
||||
(`report reuse --suggest-relations --apply`) documents *who* reused a
|
||||
capability, not that the reuse *went well*. A capability can be reused
|
||||
by five repos and still be R2 if the outcome evidence includes
|
||||
`outcome: skipped` or recorded incidents.
|
||||
- Promotion must still cite evidence explicitly in `promotion_history`
|
||||
(§ maturity promotion convention) — `reuse-surface maintain`'s
|
||||
`maturity_promote` patch kind is LLM-suggested and review-gated, never
|
||||
auto-applied from telemetry counts alone.
|
||||
|
||||
---
|
||||
|
||||
## 9. Relationship Between Dimensions
|
||||
|
|
|
|||
|
|
@ -5,12 +5,19 @@ import json
|
|||
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from reuse_surface.reports import (
|
||||
cohort_filters_from_args,
|
||||
collect_gap_report,
|
||||
collect_reuse_events,
|
||||
collect_reuse_report,
|
||||
collect_reused_by_suggestions,
|
||||
format_cohort_json,
|
||||
format_cohort_markdown,
|
||||
format_gap_markdown,
|
||||
format_reuse_report_json,
|
||||
format_reuse_report_markdown,
|
||||
select_cohort,
|
||||
)
|
||||
|
||||
|
|
@ -189,4 +196,244 @@ def test_cmd_report_cohorts_markdown(monkeypatch):
|
|||
lambda: SAMPLE_INDEX,
|
||||
)
|
||||
exit_code = main(["report", "cohorts", "--planning-min", "D5"])
|
||||
assert exit_code == 0
|
||||
assert exit_code == 0
|
||||
|
||||
# --- T05: reuse telemetry aggregation ---
|
||||
|
||||
SAMPLE_REUSE_EVENTS = [
|
||||
{
|
||||
"ts": "2026-07-01T00:00:00Z",
|
||||
"consumer_repo": "repo-a",
|
||||
"capability_id": "capability.infotech.issue-tracking",
|
||||
"verdict": "reuse",
|
||||
"outcome": "reused",
|
||||
"source": "plan-check",
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-02T00:00:00Z",
|
||||
"consumer_repo": "repo-b",
|
||||
"capability_id": "capability.infotech.issue-tracking",
|
||||
"verdict": "reuse",
|
||||
"outcome": "reused",
|
||||
"source": "manual",
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-03T00:00:00Z",
|
||||
"consumer_repo": "repo-a",
|
||||
"capability_id": "capability.audit.event-retain",
|
||||
"verdict": "extend",
|
||||
"outcome": "extended",
|
||||
"source": "plan-check",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_collect_reuse_report_aggregates_per_capability():
|
||||
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||
assert report["total_events"] == 3
|
||||
by_id = {row["capability_id"]: row for row in report["capabilities"]}
|
||||
assert by_id["capability.infotech.issue-tracking"]["consumer_count"] == 2
|
||||
assert by_id["capability.infotech.issue-tracking"]["consumer_repos"] == ["repo-a", "repo-b"]
|
||||
assert by_id["capability.infotech.issue-tracking"]["outcomes"] == {"reused": 2}
|
||||
assert by_id["capability.infotech.issue-tracking"]["last_used"] == "2026-07-02T00:00:00Z"
|
||||
assert by_id["capability.audit.event-retain"]["consumer_count"] == 1
|
||||
|
||||
|
||||
def test_collect_reuse_report_filters_by_capability_id():
|
||||
report = collect_reuse_report(SAMPLE_REUSE_EVENTS, capability_id="capability.audit.event-retain")
|
||||
assert len(report["capabilities"]) == 1
|
||||
assert report["capabilities"][0]["capability_id"] == "capability.audit.event-retain"
|
||||
|
||||
|
||||
def test_collect_reuse_report_ignores_events_without_capability_id():
|
||||
events = SAMPLE_REUSE_EVENTS + [
|
||||
{"ts": "2026-07-04T00:00:00Z", "consumer_repo": "repo-c", "capability_id": None, "verdict": "new", "outcome": "new", "source": "manual"}
|
||||
]
|
||||
report = collect_reuse_report(events)
|
||||
assert report["total_events"] == 4
|
||||
assert len(report["capabilities"]) == 2
|
||||
|
||||
|
||||
def test_format_reuse_report_markdown_empty():
|
||||
output = format_reuse_report_markdown({"capabilities": [], "total_events": 0})
|
||||
assert "No reuse events recorded yet" in output
|
||||
|
||||
|
||||
def test_format_reuse_report_markdown_nonempty():
|
||||
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||
output = format_reuse_report_markdown(report)
|
||||
assert "capability.infotech.issue-tracking" in output
|
||||
assert "3" in output
|
||||
|
||||
|
||||
def test_format_reuse_report_json_roundtrips():
|
||||
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||
payload = json.loads(format_reuse_report_json(report))
|
||||
assert payload["total_events"] == 3
|
||||
|
||||
|
||||
def test_collect_reuse_events_merges_hub_and_local(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
"reuse_surface.hub_client.hub_list_reuse_events",
|
||||
lambda capability_id=None, base_url=None: (200, {"events": [SAMPLE_REUSE_EVENTS[0]]}),
|
||||
)
|
||||
telemetry_dir = tmp_path / "registry" / "telemetry"
|
||||
telemetry_dir.mkdir(parents=True)
|
||||
(telemetry_dir / "plan-check-events.jsonl").write_text(
|
||||
json.dumps(SAMPLE_REUSE_EVENTS[1], sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
events = collect_reuse_events(repo_root=tmp_path)
|
||||
assert len(events) == 2
|
||||
|
||||
|
||||
def test_collect_reuse_events_dedupes_identical_facts(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
"reuse_surface.hub_client.hub_list_reuse_events",
|
||||
lambda capability_id=None, base_url=None: (200, {"events": [SAMPLE_REUSE_EVENTS[0]]}),
|
||||
)
|
||||
telemetry_dir = tmp_path / "registry" / "telemetry"
|
||||
telemetry_dir.mkdir(parents=True)
|
||||
(telemetry_dir / "plan-check-events.jsonl").write_text(
|
||||
json.dumps(SAMPLE_REUSE_EVENTS[0], sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
events = collect_reuse_events(repo_root=tmp_path)
|
||||
assert len(events) == 1
|
||||
|
||||
|
||||
def test_collect_reuse_events_degrades_gracefully_when_hub_unreachable(tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("REUSE_SURFACE_URL", raising=False)
|
||||
events = collect_reuse_events(repo_root=tmp_path)
|
||||
assert events == []
|
||||
|
||||
|
||||
def _write_entry_with_relations(tmp_path: Path, cap_id: str, reused_by: list[str]) -> str:
|
||||
rel = "registry/capabilities/capability-demo-sample.md"
|
||||
front_matter = {
|
||||
"id": cap_id,
|
||||
"name": "Sample",
|
||||
"summary": "Sample",
|
||||
"owner": "demo",
|
||||
"status": "draft",
|
||||
"domain": "helix_forge",
|
||||
"tags": ["demo"],
|
||||
"maturity": {
|
||||
"discovery": {"current": "D3", "target": "D5", "confidence": "low"},
|
||||
"availability": {"current": "A3", "target": "A4", "confidence": "low"},
|
||||
},
|
||||
"external_evidence": {
|
||||
"completeness": {"level": "C2", "confidence": "low"},
|
||||
"reliability": {"level": "R2", "confidence": "low"},
|
||||
},
|
||||
"discovery": {"intent": "demo", "includes": [], "excludes": []},
|
||||
"availability": {
|
||||
"current_level": "A3",
|
||||
"target_level": "A4",
|
||||
"current_artifacts": [],
|
||||
"consumption_modes": ["informational"],
|
||||
},
|
||||
"relations": {"depends_on": [], "supports": [], "related_to": [], "reused_by": reused_by},
|
||||
"evidence": {"documentation": [], "tests": []},
|
||||
"consumer_guidance": {
|
||||
"recommended_for": [],
|
||||
"not_recommended_for": [],
|
||||
"known_limitations": [],
|
||||
},
|
||||
}
|
||||
path = tmp_path / rel
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text("---\n" + yaml.safe_dump(front_matter, sort_keys=False) + "---\n", encoding="utf-8")
|
||||
return rel
|
||||
|
||||
|
||||
def test_collect_reused_by_suggestions_skips_already_listed_consumers(tmp_path):
|
||||
rel = _write_entry_with_relations(tmp_path, "capability.infotech.issue-tracking", reused_by=["repo-a"])
|
||||
index_path = tmp_path / "registry" / "indexes" / "capabilities.yaml"
|
||||
index_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
index_path.write_text(
|
||||
yaml.safe_dump({
|
||||
"version": 1, "domain": "helix_forge", "updated": "2026-07-01",
|
||||
"capabilities": [{"id": "capability.infotech.issue-tracking", "path": rel, "vector": "D3/A3/C2/R2"}],
|
||||
}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||
suggestions = collect_reused_by_suggestions(report, repo_root=tmp_path)
|
||||
# repo-a already listed -> only repo-b should be suggested
|
||||
assert len(suggestions) == 1
|
||||
assert suggestions[0]["value"] == {"type": "reused_by", "target": "repo-b"}
|
||||
assert suggestions[0]["kind"] == "relation_add"
|
||||
|
||||
|
||||
def test_collect_reused_by_suggestions_skips_capabilities_not_owned_locally(tmp_path):
|
||||
# empty local index -- no capabilities owned here at all
|
||||
index_path = tmp_path / "registry" / "indexes" / "capabilities.yaml"
|
||||
index_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
index_path.write_text(
|
||||
yaml.safe_dump({"version": 1, "domain": "helix_forge", "updated": "2026-07-01", "capabilities": []}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||
suggestions = collect_reused_by_suggestions(report, repo_root=tmp_path)
|
||||
assert suggestions == []
|
||||
|
||||
|
||||
def test_collect_reused_by_suggestions_no_index_returns_empty(tmp_path):
|
||||
report = collect_reuse_report(SAMPLE_REUSE_EVENTS)
|
||||
suggestions = collect_reused_by_suggestions(report, repo_root=tmp_path)
|
||||
assert suggestions == []
|
||||
|
||||
|
||||
def test_cmd_report_reuse_markdown(monkeypatch):
|
||||
from reuse_surface.cli import main
|
||||
|
||||
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||
exit_code = main(["report", "reuse"])
|
||||
assert exit_code == 0
|
||||
|
||||
|
||||
def test_cmd_report_reuse_json(monkeypatch, capsys):
|
||||
from reuse_surface.cli import main
|
||||
|
||||
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||
exit_code = main(["report", "reuse", "--format", "json"])
|
||||
assert exit_code == 0
|
||||
payload = json.loads(capsys.readouterr().out)
|
||||
assert payload["total_events"] == 3
|
||||
|
||||
|
||||
def test_cmd_report_reuse_apply_without_suggest_relations_errors(monkeypatch):
|
||||
from reuse_surface.cli import main
|
||||
|
||||
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: [])
|
||||
exit_code = main(["report", "reuse", "--apply"])
|
||||
assert exit_code == 1
|
||||
|
||||
|
||||
def test_cmd_report_reuse_suggest_relations(monkeypatch, capsys):
|
||||
from reuse_surface.cli import main
|
||||
|
||||
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||
monkeypatch.setattr(
|
||||
"reuse_surface.cli.collect_reused_by_suggestions",
|
||||
lambda report: [{"capability_id": "capability.infotech.issue-tracking", "kind": "relation_add", "detail": "observed reuse by repo-a", "value": {"type": "reused_by", "target": "repo-a"}}],
|
||||
)
|
||||
exit_code = main(["report", "reuse", "--suggest-relations"])
|
||||
assert exit_code == 0
|
||||
out = capsys.readouterr().out
|
||||
assert "repo-a" in out
|
||||
|
||||
|
||||
def test_cmd_report_reuse_apply_calls_apply_patches(monkeypatch):
|
||||
from reuse_surface.cli import main
|
||||
|
||||
monkeypatch.setattr("reuse_surface.cli.collect_reuse_events", lambda hub_url=None: SAMPLE_REUSE_EVENTS)
|
||||
suggestion = {"capability_id": "capability.infotech.issue-tracking", "kind": "relation_add", "detail": "x", "value": {"type": "reused_by", "target": "repo-a"}}
|
||||
monkeypatch.setattr("reuse_surface.cli.collect_reused_by_suggestions", lambda report: [suggestion])
|
||||
applied_calls = []
|
||||
monkeypatch.setattr(
|
||||
"reuse_surface.patches.apply_patches",
|
||||
lambda repo_root, patches: applied_calls.append(patches) or ["applied one"],
|
||||
)
|
||||
exit_code = main(["report", "reuse", "--suggest-relations", "--apply"])
|
||||
assert exit_code == 0
|
||||
assert applied_calls == [[suggestion]]
|
||||
|
|
|
|||
|
|
@ -170,6 +170,30 @@ it isn't. Off by default so `report gaps` stays fast and offline-safe.
|
|||
Workstation roster report: publish blockers, empty scaffolds, seed-ready repos,
|
||||
and local index owner stubs pending dedup.
|
||||
|
||||
### report reuse
|
||||
|
||||
Reuse telemetry aggregation (REUSE-WP-0019-T05): per-capability consumer
|
||||
counts, outcome breakdown, last-used, merged from the hub's
|
||||
`GET /v1/reuse-events` (if reachable) and this repo's local
|
||||
`registry/telemetry/plan-check-events.jsonl` fallback, deduped.
|
||||
|
||||
```bash
|
||||
reuse-surface report reuse
|
||||
reuse-surface report reuse --capability-id capability.infotech.issue-tracking
|
||||
reuse-surface report reuse --format json
|
||||
reuse-surface report reuse --suggest-relations
|
||||
reuse-surface report reuse --suggest-relations --apply
|
||||
```
|
||||
|
||||
`--suggest-relations` lists evidence-gated `relations.reused_by` suggestions
|
||||
for capabilities this repo owns (skips consumer repos already listed).
|
||||
`--apply` applies them via the same `apply_patches` mechanism `maintain`
|
||||
uses for `relation_add` patches — requires `--suggest-relations`, and never
|
||||
runs without it: nothing is written unless real observed-reuse events exist
|
||||
*and* an operator explicitly passes `--apply`. See
|
||||
`specs/CapabilityMaturityStandard.md` §8.9 for what observed-reuse evidence
|
||||
does (and doesn't) count toward R-axis promotion.
|
||||
|
||||
### stats
|
||||
|
||||
Registry maturity aggregates and federation readiness.
|
||||
|
|
@ -280,6 +304,7 @@ Stable IDs and maturity fields are preserved for agent consumption (UC-RS-019).
|
|||
| Relation graph | `reuse-surface graph` |
|
||||
| Query before building | `reuse-surface plan-check --intent "..."` |
|
||||
| Record a reuse fact retroactively | `reuse-surface record-reuse` |
|
||||
| Aggregate reuse telemetry into evidence | `reuse-surface report reuse --suggest-relations` |
|
||||
|
||||
## Related use cases
|
||||
|
||||
|
|
|
|||
|
|
@ -310,19 +310,63 @@ Implemented:
|
|||
|
||||
```task
|
||||
id: REUSE-WP-0019-T05
|
||||
status: wait
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7"
|
||||
```
|
||||
|
||||
Blocked on T04 plus initial event volume.
|
||||
"Blocked on T04 plus initial event volume" — T04 is done, and the
|
||||
*tooling* doesn't actually need to wait for real ecosystem adoption volume
|
||||
to be built and tested correctly (an empty/near-empty dataset is itself a
|
||||
valid, tested case: `report reuse` prints "No reuse events recorded yet"
|
||||
rather than erroring). Real cross-repo adoption volume ramping up is an
|
||||
ecosystem-timing question, not a coding blocker — implemented now so the
|
||||
tooling is ready the moment volume exists.
|
||||
|
||||
- `reuse-surface report reuse`: per-capability consumer counts, outcomes,
|
||||
last-used; feeds `reused_by` relation suggestions via the WP-0016
|
||||
maintain/patch pipeline (evidence-gated, never silent promotion)
|
||||
- Maturity standard note: what observed-reuse evidence counts toward R2/R3+
|
||||
(`specs/CapabilityMaturityStandard.md` amendment)
|
||||
- Catalog + graph surface consumer counts
|
||||
Implemented:
|
||||
|
||||
- `reuse_surface/reports.py`: `collect_reuse_events()` (merges the hub's
|
||||
`GET /v1/reuse-events`, if reachable, with this repo's local JSONL
|
||||
fallback, deduped by `(ts, consumer_repo, capability_id, source)`),
|
||||
`collect_reuse_report()` (per-capability consumer counts, outcome
|
||||
breakdown, last-used), `format_reuse_report_markdown/json`
|
||||
- `collect_reused_by_suggestions()`: evidence-gated `relation_add`
|
||||
suggestions — only for capabilities this repo actually owns (checked
|
||||
against the local index, not invented), only for consumer repos not
|
||||
already listed. Reuses the *existing* `patches.py:apply_patches`
|
||||
mechanism (which already handled `relation_add`, not in
|
||||
`SAFE_DETERMINISTIC_KINDS`, so never auto-applied by `maintain --auto`)
|
||||
rather than building a new apply path
|
||||
- New CLI: `reuse-surface report reuse [--capability-id] [--format]
|
||||
[--suggest-relations] [--apply]` — `--apply` requires
|
||||
`--suggest-relations` and is the only thing that writes; nothing happens
|
||||
automatically from telemetry alone
|
||||
- Schema: added `relations.reused_by` to `schemas/capability.schema.yaml`
|
||||
(a new `repoSlugList` `$defs` type — distinct from the existing
|
||||
`capabilityIdList` relations, since reused-by targets are consumer repo
|
||||
slugs, not capability ids)
|
||||
- `specs/CapabilityMaturityStandard.md` §8.9 amended: what observed-reuse
|
||||
evidence counts toward R2→R3 (single corroborating consumer) vs R3→R4+
|
||||
(multiple *independent* consumers, `report reuse`'s `consumer_count`),
|
||||
and explicitly what it never substitutes for (bug reports, incidents,
|
||||
explicit `promotion_history` citations — `maturity_promote` stays
|
||||
LLM-suggested and review-gated)
|
||||
- 19 new pytest cases (`test_reports.py`); 162 total pass
|
||||
- **Live-verified** with synthetic local-JSONL events against a real
|
||||
capability entry: `report reuse --suggest-relations --apply` correctly
|
||||
wrote `relations.reused_by` into the entry's front matter via the real
|
||||
`apply_patches` path (confirmed via `git diff`, then reverted since it
|
||||
was a smoke test, not a real fact)
|
||||
|
||||
**Deliberately deferred, not silently dropped:** "Catalog + graph surface
|
||||
consumer counts." `graph.py`'s `RELATION_TYPES` graph is capability-to-
|
||||
capability edges; `reused_by` targets are repo slugs, a different
|
||||
namespace entirely — forcing it into that edge model would either error or
|
||||
produce meaningless nodes. `catalog.py` doesn't currently parse full entry
|
||||
front matter per capability at all (works off the index). Surfacing
|
||||
consumer counts in either artifact is a real, separate rendering-layer
|
||||
task, not a natural extension of what T05 already built — left for a
|
||||
follow-up rather than rushed in.
|
||||
|
||||
## Freshness Monitoring, Docs, SCOPE
|
||||
|
||||
|
|
@ -348,8 +392,8 @@ state_hub_task_id: "a9f44d45-91e2-4b43-909f-30a5f906cf3b"
|
|||
- [x] No hardcoded forge host in code or sources.yaml; `migrate-host` tested (T01, 2026-07-07)
|
||||
- [x] Push to a sibling repo's `registry/indexes/` recomposes the hub index without manual action (webhook), with scheduled fallback in place (T02/T03, 2026-07-07 — both live-verified end to end)
|
||||
- [x] This repo's CI runs on Forgejo Actions (`.forgejo/workflows/`) (T03, 2026-07-07 — `ci.yml`/`ci-smoke.yaml`/`image.yaml` all verified green on the live push)
|
||||
- [x] Reuse events recordable via hub API and CLI (T04, 2026-07-08 — live-verified); `report reuse` aggregation is T05
|
||||
- [ ] R-axis evidence rules for observed reuse documented in the maturity standard
|
||||
- [x] Reuse events recordable via hub API and CLI (T04, 2026-07-08 — live-verified); `report reuse` aggregation done in T05
|
||||
- [x] R-axis evidence rules for observed reuse documented in the maturity standard (T05, 2026-07-08 — `specs/CapabilityMaturityStandard.md` §8.9)
|
||||
- [x] Hub freshness visible (`composed_at`, stale flag) in API and stats (T02, 2026-07-07)
|
||||
|
||||
## Out of scope
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue