feat: automate classification convergence

Makefile

@@ -20,6 +20,13 @@ AGENT_VARS_FILE   := $(RAILIANCE_INFRA)/ansible/inventory/group_vars/all.yaml
 ops-inventory-view: ## Render the ops-hub service catalog now view
 	python3 ops/render_service_inventory.py
 
+.PHONY: classification-status classification-check
+classification-status: ## Report active-fleet Repo Classification convergence
+	python3 tools/repo_classification_convergence.py
+
+classification-check: ## Fail until every active repo has a valid source and projection
+	python3 tools/repo_classification_convergence.py --require-converged
+
 .PHONY: custodian-keygen
 custodian-keygen: ## Generate custodian agent SSH keypair (one-time setup)
 	@if [ -f "$(CUSTODIAN_KEY)" ]; then \

SCOPE.md

@@ -1,7 +1,7 @@
 ---
 domain: custodian
 repo: the-custodian
-updated: "2026-07-08"
+updated: "2026-08-23"
 ---
 
 # SCOPE
@@ -87,9 +87,12 @@ their own repositories and are referenced here only as integration pointers.
 
 ## Current State
 
-- Status: **stable maintenance** — governance substrate in daily ecosystem use.
+- Status: **stable maintenance with two active coordination workplans** —
-  No `active`/`ready`/`blocked` CUST-WP workplans; one `proposed` plan
+  `CUST-WP-0064` awaits the first unassisted weekday controlled-source SBOM
-  (`CUST-WP-0055` terminology refactor) awaits review before pickup.
+  fire; `CUST-WP-0065` awaits 13 repo-owner classification decisions while its
+  live convergence check is automated in `make classification-status`. The
+  active registry contains 116 repositories after archiving the already-retired
+  Inter-Hub row.
 - Business platform enablement (`CUST-WP-0058`, finished 2026-07-10):
   DR-1/2/3 resolved (instance-per-client tenancy; coulomb.social as
   standalone app; app-local identity), business-app service contract

WORK-RECORDS.md

@@ -409,7 +409,7 @@
 | task | CUST-WP-0063-T05 | done | — | workplans/CUST-WP-0063-inbox-governance-packets.md |
 | task | CUST-WP-0064-T01 | done | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
 | task | CUST-WP-0064-T02 | done | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
-| task | CUST-WP-0064-T03 | progress | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
+| task | CUST-WP-0064-T03 | done | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
 | task | CUST-WP-0064-T04 | progress | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
 | task | CUST-WP-0065-T01 | done | — | workplans/CUST-WP-0065-reclassify-repos-and-guidance-docs-to-the-new-sector-domain.md |
 | task | CUST-WP-0065-T02 | done | — | workplans/CUST-WP-0065-reclassify-repos-and-guidance-docs-to-the-new-sector-domain.md |

docs/repo-classification-sector-migration-baseline-2026-08-23.md

@@ -59,9 +59,12 @@ This table is a triage aid, not an automatic rewrite map.
 
 ## Reproduction
 
-Read active repository records from `GET /repos/`, select records whose
+Run `make classification-status`. The command reads active repository records
-`status` is `active` and whose `category` is null, then test the registered
+from `GET /repos/`, compares every registered `local_path` with its
-`local_path` for `.repo-classification.yaml`. The observed counts are:
+`.repo-classification.yaml`, validates present source files against the canon,
+and reports owner-source gaps separately from projection gaps. Use
+`make classification-check` as the eventual fail-closed convergence gate.
+The initial observed counts were:
 
 ```text
 registered: 121
@@ -105,3 +108,12 @@ digest verification, and API projection passed. The current checkpoint is
 104/117 classified, 13 missing-source/null-category repos, and zero present
 source files left unprojected. This is governed pending work rather than an
 unexplained projection failure.
+
+The first automated convergence run additionally found `inter-hub` still
+marked active with a projected classification even though its registered
+checkout had been removed. Custodian retirement evidence already records Core
+Hub as the sole production surface from 2026-07-08. The supported State Hub
+archive transition corrected the stale row while preserving its history. The
+current active denominator is therefore 116: 103 classified, 13
+missing-source/null-category, zero present-but-unprojected, and zero invalid
+source records.

tests/test_repo_classification_convergence.py

@@ -0,0 +1,83 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+from tools.repo_classification_convergence import analyze_repositories, render_text
+
+
+def _record(slug: str, path: Path | None, *, category: str | None) -> dict:
+    return {
+        "slug": slug,
+        "status": "active",
+        "category": category,
+        "local_path": str(path) if path else None,
+    }
+
+
+def _write_valid(path: Path) -> None:
+    path.mkdir()
+    (path / ".repo-classification.yaml").write_text(
+        """repo_classification:
+  category: project
+  domain: infotech
+  secondary_domains: []
+  capability_tags: []
+  business_stake: []
+  business_mechanics: []
+""",
+        encoding="utf-8",
+    )
+
+
+def test_report_separates_owner_gap_from_projection_gap(tmp_path: Path) -> None:
+    classified = tmp_path / "classified"
+    unprojected = tmp_path / "unprojected"
+    _write_valid(classified)
+    _write_valid(unprojected)
+
+    report = analyze_repositories(
+        [
+            _record("classified", classified, category="project"),
+            _record("owner-gap", tmp_path / "owner-gap", category=None),
+            _record("projection-gap", unprojected, category=None),
+            {"slug": "archived", "status": "archived", "category": None},
+        ]
+    )
+
+    assert report["registered"] == 4
+    assert report["active"] == 3
+    assert report["active_classified"] == 1
+    assert report["missing_source"] == ["owner-gap"]
+    assert report["present_unprojected"] == ["projection-gap"]
+    assert report["converged"] is False
+
+
+def test_report_flags_stale_projection_and_invalid_source(tmp_path: Path) -> None:
+    invalid = tmp_path / "invalid"
+    invalid.mkdir()
+    (invalid / ".repo-classification.yaml").write_text(
+        "repo_classification:\n  category: made-up\n  domain: custodian\n",
+        encoding="utf-8",
+    )
+
+    report = analyze_repositories(
+        [
+            _record("stale", tmp_path / "stale", category="project"),
+            _record("invalid", invalid, category="project"),
+        ]
+    )
+
+    assert report["projected_without_source"] == ["stale"]
+    assert "invalid" in report["invalid_source"]
+    assert report["converged"] is False
+
+
+def test_clean_active_fleet_converges_and_renders(tmp_path: Path) -> None:
+    repo = tmp_path / "repo"
+    _write_valid(repo)
+    report = analyze_repositories([_record("repo", repo, category="project")])
+
+    assert report["converged"] is True
+    assert report["source_warning_count"] == 0
+    assert "active classified: 1" in render_text(report)
+    assert "converged: yes" in render_text(report)

tools/repo_classification_convergence.py

@@ -0,0 +1,204 @@
+#!/usr/bin/env python3
+"""Report active-fleet Repo Classification convergence from State Hub.
+
+The repository file remains authoritative.  This command compares the live
+State Hub projection with each active record's registered local checkout and
+separates missing owner decisions from projection failures.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+import urllib.error
+import urllib.request
+from pathlib import Path
+from typing import Any, Callable
+
+import yaml
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from tools.validate_repo_classification import load_allowed, validate  # noqa: E402
+
+DEFAULT_API_BASE = "http://127.0.0.1:8000"
+UrlOpen = Callable[..., Any]
+
+
+class FleetReadError(RuntimeError):
+    """State Hub did not return a usable repository projection."""
+
+
+def fetch_repositories(
+    api_base: str,
+    *,
+    timeout: float = 15.0,
+    opener: UrlOpen = urllib.request.urlopen,
+) -> list[dict[str, Any]]:
+    url = f"{api_base.rstrip('/')}/repos/?limit=500"
+    request = urllib.request.Request(url, headers={"Accept": "application/json"})
+    try:
+        with opener(request, timeout=timeout) as response:
+            payload = json.load(response)
+    except (OSError, urllib.error.URLError, json.JSONDecodeError) as exc:
+        raise FleetReadError(f"cannot read State Hub repositories: {exc}") from exc
+    if not isinstance(payload, list) or any(not isinstance(item, dict) for item in payload):
+        raise FleetReadError("State Hub /repos response is not a list of objects")
+    return payload
+
+
+def _source_path(record: dict[str, Any]) -> Path | None:
+    local_path = record.get("local_path")
+    if not isinstance(local_path, str) or not local_path.strip():
+        return None
+    return Path(local_path) / ".repo-classification.yaml"
+
+
+def analyze_repositories(
+    records: list[dict[str, Any]],
+    *,
+    validate_sources: bool = True,
+) -> dict[str, Any]:
+    active = [record for record in records if record.get("status") == "active"]
+    classified = [record for record in active if record.get("category") is not None]
+    null_records = [record for record in active if record.get("category") is None]
+    missing_source: list[str] = []
+    present_unprojected: list[str] = []
+    projected_without_source: list[str] = []
+    invalid_source: dict[str, list[str]] = {}
+    source_warnings: dict[str, list[str]] = {}
+    duplicate_slugs: list[str] = []
+
+    slug_counts: dict[str, int] = {}
+    allowed = load_allowed() if validate_sources else None
+    for record in active:
+        slug = str(record.get("slug") or "<missing-slug>")
+        slug_counts[slug] = slug_counts.get(slug, 0) + 1
+        path = _source_path(record)
+        source_present = path is not None and path.is_file()
+
+        if record.get("category") is None:
+            (present_unprojected if source_present else missing_source).append(slug)
+        elif not source_present:
+            projected_without_source.append(slug)
+
+        if validate_sources and source_present and path is not None:
+            try:
+                document = yaml.safe_load(path.read_text(encoding="utf-8"))
+                errors, warnings = validate(document, allowed)
+            except (OSError, yaml.YAMLError) as exc:
+                errors, warnings = [f"cannot read/parse source: {exc}"], []
+            if errors:
+                invalid_source[slug] = errors
+            if warnings:
+                source_warnings[slug] = warnings
+
+    duplicate_slugs = sorted(slug for slug, count in slug_counts.items() if count > 1)
+    missing_source.sort()
+    present_unprojected.sort()
+    projected_without_source.sort()
+
+    blocking = bool(
+        missing_source
+        or present_unprojected
+        or projected_without_source
+        or invalid_source
+        or duplicate_slugs
+    )
+    return {
+        "registered": len(records),
+        "active": len(active),
+        "active_classified": len(classified),
+        "active_null_category": len(null_records),
+        "missing_source": missing_source,
+        "present_unprojected": present_unprojected,
+        "projected_without_source": projected_without_source,
+        "invalid_source": dict(sorted(invalid_source.items())),
+        "source_warnings": dict(sorted(source_warnings.items())),
+        "source_warning_count": sum(len(items) for items in source_warnings.values()),
+        "duplicate_active_slugs": duplicate_slugs,
+        "converged": not blocking,
+    }
+
+
+def render_text(report: dict[str, Any], *, show_warnings: bool = False) -> str:
+    lines = [
+        f"registered: {report['registered']}",
+        f"active: {report['active']}",
+        f"active classified: {report['active_classified']}",
+        f"active null category: {report['active_null_category']}",
+        f"missing source file: {len(report['missing_source'])}",
+        f"source present but unprojected: {len(report['present_unprojected'])}",
+        f"projected without source file: {len(report['projected_without_source'])}",
+        f"invalid source file: {len(report['invalid_source'])}",
+        f"source validation warnings: {report['source_warning_count']}",
+        f"duplicate active slug: {len(report['duplicate_active_slugs'])}",
+        f"converged: {'yes' if report['converged'] else 'no'}",
+    ]
+    for key in (
+        "missing_source",
+        "present_unprojected",
+        "projected_without_source",
+        "duplicate_active_slugs",
+    ):
+        if report[key]:
+            lines.append(f"{key}: {', '.join(report[key])}")
+    for slug, errors in report["invalid_source"].items():
+        lines.append(f"invalid_source[{slug}]: {'; '.join(errors)}")
+    if show_warnings:
+        for slug, warnings in report["source_warnings"].items():
+            lines.append(f"source_warnings[{slug}]: {'; '.join(warnings)}")
+    return "\n".join(lines)
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--api-base",
+        default=os.environ.get("STATE_HUB_API_BASE", DEFAULT_API_BASE),
+        help="State Hub REST base URL (default: %(default)s)",
+    )
+    parser.add_argument("--timeout", type=float, default=15.0)
+    parser.add_argument("--json", action="store_true", help="emit machine-readable JSON")
+    parser.add_argument(
+        "--show-warnings",
+        action="store_true",
+        help="include individual non-blocking source validation warnings",
+    )
+    parser.add_argument(
+        "--skip-source-validation",
+        action="store_true",
+        help="only compare file presence and projection state",
+    )
+    parser.add_argument(
+        "--require-converged",
+        action="store_true",
+        help="exit 1 while any active classification gap remains",
+    )
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    try:
+        records = fetch_repositories(args.api_base, timeout=args.timeout)
+        report = analyze_repositories(
+            records,
+            validate_sources=not args.skip_source_validation,
+        )
+    except FleetReadError as exc:
+        print(f"ERROR: {exc}", file=sys.stderr)
+        return 2
+
+    if args.json:
+        print(json.dumps(report, indent=2, sort_keys=True))
+    else:
+        print(render_text(report, show_warnings=args.show_warnings))
+    return 1 if args.require_converged and not report["converged"] else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md

@@ -124,7 +124,7 @@ Process/database health, repository reads, and zero-restarter startup all pass.
 
 ```task
 id: CUST-WP-0064-T03
-status: progress
+status: done
 priority: high
 state_hub_task_id: "6c645778-be59-57b1-bf44-d974a3a1e49f"
 ```
@@ -143,8 +143,16 @@ It then projected the live oldest-three target set (`can-you-assist`,
 The existing unpaused Temporal schedule was reconciled unchanged at limit 3
 and operator-triggered: it froze exactly those targets, spawned zero tasks,
 created three provenance-bearing terminal `no-manifest` snapshots, and moved
-`never_count` 94 to 91. Keep this task open for sustainable projection as each
+`never_count` 94 to 91.
-new oldest-N batch is exposed.
+
+**Done (2026-08-23):** after that controlled batch advanced, the next oldest
+three (`citation-work`, `clay-borg`, and `config-atlas`) were selected with
+matching `forgejo-archive-v1` references and full 40-character revisions. An
+in-worker read-only report probe on deployed Activity Core digest
+`sha256:9c611a394c117c8ccfe2fd813c0dda1943ea8a7444aa4bd4742d83712b2cf559`
+returned `selected_count=3` and `controlled_source_count=3`. This proves
+source-reference projection follows each newly exposed bounded batch rather
+than the attended target set; T04 retains only the first unassisted fire.
 
 ## Prove real daily freshness improvement
 

workplans/CUST-WP-0065-reclassify-repos-and-guidance-docs-to-the-new-sector-domain.md

@@ -141,6 +141,24 @@ unprojected. Project names and repository identities were not rewritten.
 After the first owner delivery, the live projection is 104/117 classified with
 13 active null-category records and again zero present-but-unprojected files.
 
+**Repeatable convergence gate (2026-08-23):**
+`tools/repo_classification_convergence.py` now derives those counts directly
+from the State Hub registry, checks each active record's registered checkout,
+validates every present source file against the canonical vocabulary, and
+separates missing owner records, present-but-unprojected records, stale
+projections without a source file, invalid sources, and duplicate active slugs.
+`make classification-status` reports progress; `make classification-check`
+fails closed until all active gaps are resolved.
+
+Its first run exposed a stale active registry row hidden by the former
+null-category-only check: retired `inter-hub` retained a projected category but
+its registered checkout no longer exists. Existing Custodian retirement
+evidence records Core Hub as the sole production surface since 2026-07-08, so
+the supported State Hub archive endpoint corrected that row without altering
+its historical classification. The active fleet is now 103/116 classified;
+the same 13 owner-source gaps remain, with no projection or source-validity
+failure.
+
 ## Indexing note
 
 **Resolved (2026-08-23):** Repo Manager revisions `7a15f1d` and `7b9fdaa`