--- id: CUST-WP-0062 type: workplan title: "Extract sbom-nexus and bound daily SBOM catch-up" domain: infotech repo: the-custodian status: finished owner: codex topic_slug: custodian planning_priority: high planning_order: 62 created: "2026-08-18" updated: "2026-08-22" related: - CUST-WP-0002 - CUST-WP-0013 - CUST-WP-0043 - ACTIVITY-WP-0030 - STATE-WP-0079 state_hub_workstream_id: "ce062024-b221-54b1-8c1d-f14360d0354e" --- # Extract sbom-nexus and bound daily SBOM catch-up ## Goal Move SBOM capture, storage, and evaluation out of State Hub into a dedicated application **`sbom-nexus`**, then replace the weekly fleet-wide staleness flood with a daily, bounded catch-up: evaluate efficiently, then update the **N oldest-stale** repo SBOMs. **N defaults to 3**. This is a coordination workplan (ADR-005). Implementation lands in `sbom-nexus` (new), `state-hub` (extract + strangler), and `activity-core` (schedule). Child plan for the schedule: **ACTIVITY-WP-0030**. ## Why now The Monday `weekly-sbom-staleness` check on railiance01 (2026-08-17) spawned **75 tasks**. A live State Hub scan on 2026-08-18 shows **111 / 111** registered repos stale (>30 days or never scanned); **93 have `last_sbom_at` null**. `activity-core` itself is at `2026-04-26`. The current rule emits one task per stale repo, so the check cannot catch up — it only reports the backlog. Current evaluation (`repo_sbom_status` with `repos: all`) pulls `GET /repos/` and computes age in activity-core. There is no oldest-N or ingest-queue surface. Ingest, snapshots, licence report, MCP tools, and dashboard pages still live in State Hub (`api/routers/sbom.py`, `api/models/sbom_*.py`, `scripts/ingest_sbom.py`, `scripts/capture_sbom_tools.py`, `prompts/sbom-capture-agent.md`, `dashboard/src/sbom.md`). ## Design 1. **`sbom-nexus` owns SBOM.** Snapshots, entries, ingest parsers, licence report, and staleness ranking are its product surface — not a State Hub subsystem. 2. **State Hub keeps a thin projection.** `managed_repos.last_sbom_at` / `sbom_source` remain readable on `/repos/` via a write-back or query adapter so existing callers do not break during cutover. 3. **Efficient evaluation.** One ranked query, not a per-repo `/sbom/{slug}` walk and not a `for_each` over every stale repo: `GET /sbom/catch-up?limit=N` (default N=3) returns the N repos with the oldest `last_sbom_at` (null first), plus a fleet summary (`stale_count`, `never_count`, `total_count`). 4. **Daily update, not weekly task flood.** activity-core runs the catch-up daily. For those N repos it **performs ingest** (declared side-effect) when a reachable checkout exists; otherwise it records a skip with reason and must not retry the same impossible repo forever. 5. **Weekly check is retired at cutover.** `weekly-sbom-staleness` / `flag-stale-sbom` stop spawning one task per stale repo. A compact progress event remains acceptable as evidence. ### Catch-up selection Order: never scanned first, then oldest `last_sbom_at`. Eligible repos are active (or otherwise in-scope) registered repos. Repos with no lockfile / no checkout are not left at the head of the queue after a recorded empty-or-skipped snapshot (`source` distinguishes `ingest` vs `no-checkout` vs `no-manifest`). N is a definition/runtime parameter (`catch_up_limit`, default 3), not a hard-coded constant in the nexus API. ## Non-goals - Rebuilding licence policy or adding new ecosystems (already in CUST-WP-0013). - Scanning every stale repo on every fire. - Using coding-assistant schedulers as the recurrence or evidence source. - Moving State Hub work-index / consistency / inbox into sbom-nexus. ## Inventory current State Hub SBOM surface ```task id: CUST-WP-0062-T01 status: done priority: high state_hub_task_id: "9b65928b-cc69-53a8-803e-daae1109f939" ``` List every SBOM caller and artefact that must move or adapt: models (`sbom_entries`, `sbom_snapshots`), router `/sbom/*`, ingest and tool-capture scripts, MCP resources/tools, dashboard pages, `ManagedRepo.last_sbom_at`, activity-core `repo_sbom_status`, `make ingest-sbom`, onboarding tasks. Tag each as *move to sbom-nexus*, *strangle in state-hub*, or *retarget caller*. Open the state-hub child workplan (next `STATE-WP`) from this inventory. **Done (2026-08-22):** `SBOM-WP-0001-T03` produced the complete disposition in `sbom-nexus/docs/state-hub-sbom-extraction-review.md`, covering State Hub models/routes/scripts/MCP/dashboard, Repo Manager, Activity Core, repository freshness projection, and migration. State Hub implementation joined its existing retirement workplan `STATE-WP-0079` rather than creating a competing strangler. ## Stand up sbom-nexus and extract ```task id: CUST-WP-0062-T02 status: done priority: high state_hub_task_id: "cbbbd680-2b17-540f-a339-30436b4ebf86" ``` Create `/home/worsch/sbom-nexus` as a real repo (`category: tooling`), register it, and move the SBOM product into it: store, ingest, HTTP API, operator docs. State Hub routes keep working via proxy or dual-write until T04. Preserve snapshot history; do not start from an empty store. **Done (2026-08-22):** `sbom-nexus` is registered as the product repository. `SBOM-WP-0001` delivered the service and `SBOM-WP-0002` deployed it on managed PostgreSQL, imported 22 snapshots / 3,123 entries exactly, and reconciled the second import idempotently. The private production deployment is Ready on immutable digest `sha256:62929abc…`; backup and a 51-second scratch restore are recorded in the child evidence. ## Efficient catch-up API ```task id: CUST-WP-0062-T03 status: done priority: high state_hub_task_id: "d1c0c3ed-fc55-5656-a504-5562fd848dae" ``` Add the ranked catch-up surface on sbom-nexus (State Hub may proxy during strangle): - `GET /sbom/catch-up?limit=N` — N oldest-stale repos + fleet counts - ingest that updates those repos and advances `last_sbom_at` - skip contract for missing checkout / no manifest, so the same repo does not block the queue Prove with fixtures that limit=3 returns the three oldest (null first) and that a skip advances queue position. **Done (2026-08-22):** Nexus implements one ranked `GET /sbom/catch-up?limit=N` call, bounded ingest/skip routes, null-first ordering, and terminal `no-checkout` / `no-manifest` semantics. Production projection reconciled 120 repositories and selected exactly the oldest three of 116 active repositories without ingest. Two bounded fires advanced `never_count` 101 → 98; a live read at 2026-08-22T20:51Z returned exactly three targets and `never_count: 95`, demonstrating continued queue advance. ## Strangle State Hub SBOM ```task id: CUST-WP-0062-T04 status: done priority: medium state_hub_task_id: "5e3132ff-4000-5eac-9d29-017c2dcec5e4" ``` Depends on T02/T03. State Hub `/sbom/*` becomes a compatibility façade or is documented 410-strangled with callers already on sbom-nexus. Dashboard and MCP read sbom-nexus (or the façade). `last_sbom_at` on `/repos/` stays correct. Align with STATE-WP-0079 retirement strangler, do not invent a second retirement path. **Done (2026-08-22):** `SBOM-WP-0002-T04` and `STATE-WP-0079-T04` moved State Hub reads and writes behind independently reversible Nexus façade flags. Dashboard, summary, DoI, MCP/CLI compatibility, and repository freshness read Nexus authority. State Hub retains its 22 historical snapshots and façade as read-only rollback evidence; new reads and writes remain Nexus-owned. ## Replace weekly evaluation in activity-core ```task id: CUST-WP-0062-T05 status: done priority: high state_hub_task_id: "27172afa-072f-5b47-a7be-5b857ba74463" ``` Depends on T03. Implement in **ACTIVITY-WP-0030**: daily ActivityDefinition, resolver against the catch-up API, ingest of N repos (default 3), deterministic progress evidence, retire `weekly-sbom-staleness` task flood. Do not enable the daily job against the old per-repo `/sbom/{slug}` walk. **Done (2026-08-22):** `ACTIVITY-WP-0030` is finished. The weekday 09:15 Europe/Berlin definition performs one Nexus selection and at most three terminal operations, emits deterministic `sbom_catchup` progress, and has no task-spawning rule. `weekly-sbom-staleness` is disabled and its Temporal schedule paused. `ACTIVITY-WP-0033` subsequently fixed the retry boundary so one workflow retains one target set across retries; Nexus-side durable operation replay remains live as `CUST-IN-0013`. ## Cutover and catch-up proof ```task id: CUST-WP-0062-T06 status: done priority: medium state_hub_task_id: "a15dc52b-43b5-56b1-90a7-49d5ab84c67a" ``` Depends on T05. On railiance01: daily job fires, updates ≤N SBOMs, posts progress (`stale_count`, `updated`, `skipped`), weekly spawn count goes to zero. After several successful days, `never_count` declines. Document the operator knobs (`catch_up_limit`, disable/enable). **Done (2026-08-22):** two operator-confirmed Railiance01 fires processed six distinct repositories as terminal `no-checkout` skips, spawned zero tasks, emitted linked progress events `a9e8acea` and `f8b59633`, and advanced the queue. Production has the daily definition enabled with limit 3 and the legacy weekly definition disabled/paused. The SBOM product owner explicitly waived waiting for the next Monday window; this closure does not claim that a natural Monday fire occurred. The remaining reason all observed outcomes were skips—no controlled checkout inside the Nexus plane—is successor `CUST-WP-0064`, not hidden closure prose. ## Acceptance - [x] SBOM product lives in `sbom-nexus`; State Hub is not the permanent store - [x] Catch-up API returns oldest-N in one call (default N=3) - [x] Daily activity-core job updates those N instead of spawning one task per stale repo - [x] Weekly SBOM task flood is off - [x] Skipped repos do not permanently head the queue - [x] `/repos/` last_sbom_at remains truthful during and after cutover ## Closure Review — 2026-08-22 The coordination goal is achieved by the finished child workplans `SBOM-WP-0001`, `SBOM-WP-0002`, `ACTIVITY-WP-0030`, and `ACTIVITY-WP-0033`, plus the SBOM disposition slice of `STATE-WP-0079`. Production evidence shows a single Nexus authority, reversible State Hub compatibility, bounded queue advancement, zero task flood, and truthful attempt/success projections. Actionable leftovers are live before closure: - `CUST-WP-0064` coordinates a controlled scan-input topology so daily runs create real authoritative SBOMs rather than only terminal `no-checkout` outcomes; - `CUST-IN-0013` routes durable operation-id replay to SBOM Nexus; - `CUST-IN-0014` routes the observed 30-minute database-lease restart cycle to SBOM Nexus for credential reread/pool-rotation repair and live proof; - `RMGR-WP-0011` remains ready for the authoritative Repo Manager HTTP client and explicit non-authoritative preview semantics. No legacy State Hub rows were deleted. Their retention and façade rollback role remain governed by `SBOM-WP-0002-T07`.