diff --git a/workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md b/workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md new file mode 100644 index 0000000..4ddec4e --- /dev/null +++ b/workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md @@ -0,0 +1,185 @@ +--- +id: CUST-WP-0062 +type: workplan +title: "Extract sbom-nexus and bound daily SBOM catch-up" +domain: infotech +repo: the-custodian +status: proposed +owner: grok +topic_slug: custodian +planning_priority: high +planning_order: 62 +created: "2026-08-18" +updated: "2026-08-18" +related: + - CUST-WP-0002 + - CUST-WP-0013 + - CUST-WP-0043 + - ACTIVITY-WP-0030 + - STATE-WP-0079 +--- + +# Extract sbom-nexus and bound daily SBOM catch-up + +## Goal + +Move SBOM capture, storage, and evaluation out of State Hub into a dedicated +application **`sbom-nexus`**, then replace the weekly fleet-wide staleness +flood with a daily, bounded catch-up: evaluate efficiently, then update the +**N oldest-stale** repo SBOMs. **N defaults to 3**. + +This is a coordination workplan (ADR-005). Implementation lands in +`sbom-nexus` (new), `state-hub` (extract + strangler), and `activity-core` +(schedule). Child plan for the schedule: **ACTIVITY-WP-0030**. + +## Why now + +The Monday `weekly-sbom-staleness` check on railiance01 (2026-08-17) spawned +**75 tasks**. A live State Hub scan on 2026-08-18 shows **111 / 111** +registered repos stale (>30 days or never scanned); **93 have `last_sbom_at` +null**. `activity-core` itself is at `2026-04-26`. The current rule emits one +task per stale repo, so the check cannot catch up — it only reports the +backlog. + +Current evaluation (`repo_sbom_status` with `repos: all`) pulls `GET /repos/` +and computes age in activity-core. There is no oldest-N or ingest-queue +surface. Ingest, snapshots, licence report, MCP tools, and dashboard pages +still live in State Hub (`api/routers/sbom.py`, `api/models/sbom_*.py`, +`scripts/ingest_sbom.py`, `scripts/capture_sbom_tools.py`, +`prompts/sbom-capture-agent.md`, `dashboard/src/sbom.md`). + +## Design + +1. **`sbom-nexus` owns SBOM.** Snapshots, entries, ingest parsers, licence + report, and staleness ranking are its product surface — not a State Hub + subsystem. +2. **State Hub keeps a thin projection.** `managed_repos.last_sbom_at` / + `sbom_source` remain readable on `/repos/` via a write-back or query + adapter so existing callers do not break during cutover. +3. **Efficient evaluation.** One ranked query, not a per-repo `/sbom/{slug}` + walk and not a `for_each` over every stale repo: + `GET /sbom/catch-up?limit=N` (default N=3) returns the N repos with the + oldest `last_sbom_at` (null first), plus a fleet summary + (`stale_count`, `never_count`, `total_count`). +4. **Daily update, not weekly task flood.** activity-core runs the catch-up + daily. For those N repos it **performs ingest** (declared side-effect) + when a reachable checkout exists; otherwise it records a skip with + reason and must not retry the same impossible repo forever. +5. **Weekly check is retired at cutover.** `weekly-sbom-staleness` / + `flag-stale-sbom` stop spawning one task per stale repo. A compact + progress event remains acceptable as evidence. + +### Catch-up selection + +Order: never scanned first, then oldest `last_sbom_at`. Eligible repos are +active (or otherwise in-scope) registered repos. Repos with no lockfile / +no checkout are not left at the head of the queue after a recorded +empty-or-skipped snapshot (`source` distinguishes `ingest` vs `no-checkout` +vs `no-manifest`). + +N is a definition/runtime parameter (`catch_up_limit`, default 3), not a +hard-coded constant in the nexus API. + +## Non-goals + +- Rebuilding licence policy or adding new ecosystems (already in CUST-WP-0013). +- Scanning every stale repo on every fire. +- Using coding-assistant schedulers as the recurrence or evidence source. +- Moving State Hub work-index / consistency / inbox into sbom-nexus. + +## Inventory current State Hub SBOM surface + +```task +id: CUST-WP-0062-T01 +status: todo +priority: high +``` + +List every SBOM caller and artefact that must move or adapt: models +(`sbom_entries`, `sbom_snapshots`), router `/sbom/*`, ingest and tool-capture +scripts, MCP resources/tools, dashboard pages, `ManagedRepo.last_sbom_at`, +activity-core `repo_sbom_status`, `make ingest-sbom`, onboarding tasks. +Tag each as *move to sbom-nexus*, *strangle in state-hub*, or *retarget +caller*. Open the state-hub child workplan (next `STATE-WP`) from this +inventory. + +## Stand up sbom-nexus and extract + +```task +id: CUST-WP-0062-T02 +status: todo +priority: high +``` + +Create `/home/worsch/sbom-nexus` as a real repo (`category: tooling`), +register it, and move the SBOM product into it: store, ingest, HTTP API, +operator docs. State Hub routes keep working via proxy or dual-write until +T04. Preserve snapshot history; do not start from an empty store. + +## Efficient catch-up API + +```task +id: CUST-WP-0062-T03 +status: todo +priority: high +``` + +Add the ranked catch-up surface on sbom-nexus (State Hub may proxy during +strangle): + +- `GET /sbom/catch-up?limit=N` — N oldest-stale repos + fleet counts +- ingest that updates those repos and advances `last_sbom_at` +- skip contract for missing checkout / no manifest, so the same repo does + not block the queue + +Prove with fixtures that limit=3 returns the three oldest (null first) and +that a skip advances queue position. + +## Strangle State Hub SBOM + +```task +id: CUST-WP-0062-T04 +status: wait +priority: medium +``` + +Depends on T02/T03. State Hub `/sbom/*` becomes a compatibility façade or is +documented 410-strangled with callers already on sbom-nexus. Dashboard and +MCP read sbom-nexus (or the façade). `last_sbom_at` on `/repos/` stays +correct. Align with STATE-WP-0079 retirement strangler, do not invent a +second retirement path. + +## Replace weekly evaluation in activity-core + +```task +id: CUST-WP-0062-T05 +status: wait +priority: high +``` + +Depends on T03. Implement in **ACTIVITY-WP-0030**: daily ActivityDefinition, +resolver against the catch-up API, ingest of N repos (default 3), +deterministic progress evidence, retire `weekly-sbom-staleness` task flood. +Do not enable the daily job against the old per-repo `/sbom/{slug}` walk. + +## Cutover and catch-up proof + +```task +id: CUST-WP-0062-T06 +status: wait +priority: medium +``` + +Depends on T05. On railiance01: daily job fires, updates ≤N SBOMs, posts +progress (`stale_count`, `updated`, `skipped`), weekly spawn count goes to +zero. After several successful days, `never_count` declines. Document the +operator knobs (`catch_up_limit`, disable/enable). + +## Acceptance + +- [ ] SBOM product lives in `sbom-nexus`; State Hub is not the permanent store +- [ ] Catch-up API returns oldest-N in one call (default N=3) +- [ ] Daily activity-core job updates those N instead of spawning one task per stale repo +- [ ] Weekly SBOM task flood is off +- [ ] Skipped repos do not permanently head the queue +- [ ] `/repos/` last_sbom_at remains truthful during and after cutover