the-custodian/workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md

id	type	title	domain	repo	status	owner	topic_slug	planning_priority	planning_order	created	updated	related	state_hub_workstream_id
CUST-WP-0062	workplan	Extract sbom-nexus and bound daily SBOM catch-up	infotech	the-custodian	finished	codex	custodian	high	62	2026-08-18	2026-08-22	CUST-WP-0002 CUST-WP-0013 CUST-WP-0043 ACTIVITY-WP-0030 STATE-WP-0079	ce062024-b221-54b1-8c1d-f14360d0354e

Extract sbom-nexus and bound daily SBOM catch-up

Goal

Move SBOM capture, storage, and evaluation out of State Hub into a dedicated application sbom-nexus, then replace the weekly fleet-wide staleness flood with a daily, bounded catch-up: evaluate efficiently, then update the N oldest-stale repo SBOMs. N defaults to 3.

This is a coordination workplan (ADR-005). Implementation lands in sbom-nexus (new), state-hub (extract + strangler), and activity-core (schedule). Child plan for the schedule: ACTIVITY-WP-0030.

Why now

The Monday weekly-sbom-staleness check on railiance01 (2026-08-17) spawned 75 tasks. A live State Hub scan on 2026-08-18 shows 111 / 111 registered repos stale (>30 days or never scanned); 93 have last_sbom_at null. activity-core itself is at 2026-04-26. The current rule emits one task per stale repo, so the check cannot catch up — it only reports the backlog.

Current evaluation (repo_sbom_status with repos: all) pulls GET /repos/ and computes age in activity-core. There is no oldest-N or ingest-queue surface. Ingest, snapshots, licence report, MCP tools, and dashboard pages still live in State Hub (api/routers/sbom.py, api/models/sbom_*.py, scripts/ingest_sbom.py, scripts/capture_sbom_tools.py, prompts/sbom-capture-agent.md, dashboard/src/sbom.md).

Design

1. sbom-nexus owns SBOM. Snapshots, entries, ingest parsers, licence report, and staleness ranking are its product surface — not a State Hub subsystem.
2. State Hub keeps a thin projection. managed_repos.last_sbom_at / sbom_source remain readable on /repos/ via a write-back or query adapter so existing callers do not break during cutover.
3. Efficient evaluation. One ranked query, not a per-repo /sbom/{slug} walk and not a for_each over every stale repo: GET /sbom/catch-up?limit=N (default N=3) returns the N repos with the oldest last_sbom_at (null first), plus a fleet summary (stale_count, never_count, total_count).
4. Daily update, not weekly task flood. activity-core runs the catch-up daily. For those N repos it performs ingest (declared side-effect) when a reachable checkout exists; otherwise it records a skip with reason and must not retry the same impossible repo forever.
5. Weekly check is retired at cutover. weekly-sbom-staleness / flag-stale-sbom stop spawning one task per stale repo. A compact progress event remains acceptable as evidence.

Catch-up selection

Order: never scanned first, then oldest last_sbom_at. Eligible repos are active (or otherwise in-scope) registered repos. Repos with no lockfile / no checkout are not left at the head of the queue after a recorded empty-or-skipped snapshot (source distinguishes ingest vs no-checkout vs no-manifest).

N is a definition/runtime parameter (catch_up_limit, default 3), not a hard-coded constant in the nexus API.

Non-goals

• Rebuilding licence policy or adding new ecosystems (already in CUST-WP-0013).
• Scanning every stale repo on every fire.
• Using coding-assistant schedulers as the recurrence or evidence source.
• Moving State Hub work-index / consistency / inbox into sbom-nexus.

Inventory current State Hub SBOM surface

id: CUST-WP-0062-T01
status: done
priority: high
state_hub_task_id: "9b65928b-cc69-53a8-803e-daae1109f939"

List every SBOM caller and artefact that must move or adapt: models (sbom_entries, sbom_snapshots), router /sbom/*, ingest and tool-capture scripts, MCP resources/tools, dashboard pages, ManagedRepo.last_sbom_at, activity-core repo_sbom_status, make ingest-sbom, onboarding tasks. Tag each as move to sbom-nexus, strangle in state-hub, or retarget caller. Open the state-hub child workplan (next STATE-WP) from this inventory.

Done (2026-08-22): SBOM-WP-0001-T03 produced the complete disposition in sbom-nexus/docs/state-hub-sbom-extraction-review.md, covering State Hub models/routes/scripts/MCP/dashboard, Repo Manager, Activity Core, repository freshness projection, and migration. State Hub implementation joined its existing retirement workplan STATE-WP-0079 rather than creating a competing strangler.

Stand up sbom-nexus and extract

id: CUST-WP-0062-T02
status: done
priority: high
state_hub_task_id: "cbbbd680-2b17-540f-a339-30436b4ebf86"

Create /home/worsch/sbom-nexus as a real repo (category: tooling), register it, and move the SBOM product into it: store, ingest, HTTP API, operator docs. State Hub routes keep working via proxy or dual-write until T04. Preserve snapshot history; do not start from an empty store.

Done (2026-08-22): sbom-nexus is registered as the product repository. SBOM-WP-0001 delivered the service and SBOM-WP-0002 deployed it on managed PostgreSQL, imported 22 snapshots / 3,123 entries exactly, and reconciled the second import idempotently. The private production deployment is Ready on immutable digest sha256:62929abc…; backup and a 51-second scratch restore are recorded in the child evidence.

Efficient catch-up API

id: CUST-WP-0062-T03
status: done
priority: high
state_hub_task_id: "d1c0c3ed-fc55-5656-a504-5562fd848dae"

Add the ranked catch-up surface on sbom-nexus (State Hub may proxy during strangle):

• GET /sbom/catch-up?limit=N — N oldest-stale repos + fleet counts
• ingest that updates those repos and advances last_sbom_at
• skip contract for missing checkout / no manifest, so the same repo does not block the queue

Prove with fixtures that limit=3 returns the three oldest (null first) and that a skip advances queue position.

Done (2026-08-22): Nexus implements one ranked GET /sbom/catch-up?limit=N call, bounded ingest/skip routes, null-first ordering, and terminal no-checkout / no-manifest semantics. Production projection reconciled 120 repositories and selected exactly the oldest three of 116 active repositories without ingest. Two bounded fires advanced never_count 101 → 98; a live read at 2026-08-22T20:51Z returned exactly three targets and never_count: 95, demonstrating continued queue advance.

Strangle State Hub SBOM

id: CUST-WP-0062-T04
status: done
priority: medium
state_hub_task_id: "5e3132ff-4000-5eac-9d29-017c2dcec5e4"

Depends on T02/T03. State Hub /sbom/* becomes a compatibility façade or is documented 410-strangled with callers already on sbom-nexus. Dashboard and MCP read sbom-nexus (or the façade). last_sbom_at on /repos/ stays correct. Align with STATE-WP-0079 retirement strangler, do not invent a second retirement path.

Done (2026-08-22): SBOM-WP-0002-T04 and STATE-WP-0079-T04 moved State Hub reads and writes behind independently reversible Nexus façade flags. Dashboard, summary, DoI, MCP/CLI compatibility, and repository freshness read Nexus authority. State Hub retains its 22 historical snapshots and façade as read-only rollback evidence; new reads and writes remain Nexus-owned.

Replace weekly evaluation in activity-core

id: CUST-WP-0062-T05
status: done
priority: high
state_hub_task_id: "27172afa-072f-5b47-a7be-5b857ba74463"

Depends on T03. Implement in ACTIVITY-WP-0030: daily ActivityDefinition, resolver against the catch-up API, ingest of N repos (default 3), deterministic progress evidence, retire weekly-sbom-staleness task flood. Do not enable the daily job against the old per-repo /sbom/{slug} walk.

Done (2026-08-22): ACTIVITY-WP-0030 is finished. The weekday 09:15 Europe/Berlin definition performs one Nexus selection and at most three terminal operations, emits deterministic sbom_catchup progress, and has no task-spawning rule. weekly-sbom-staleness is disabled and its Temporal schedule paused. ACTIVITY-WP-0033 subsequently fixed the retry boundary so one workflow retains one target set across retries; Nexus-side durable operation replay remains live as CUST-IN-0013.

Cutover and catch-up proof

id: CUST-WP-0062-T06
status: done
priority: medium
state_hub_task_id: "a15dc52b-43b5-56b1-90a7-49d5ab84c67a"

Depends on T05. On railiance01: daily job fires, updates ≤N SBOMs, posts progress (stale_count, updated, skipped), weekly spawn count goes to zero. After several successful days, never_count declines. Document the operator knobs (catch_up_limit, disable/enable).

Done (2026-08-22): two operator-confirmed Railiance01 fires processed six distinct repositories as terminal no-checkout skips, spawned zero tasks, emitted linked progress events a9e8acea and f8b59633, and advanced the queue. Production has the daily definition enabled with limit 3 and the legacy weekly definition disabled/paused. The SBOM product owner explicitly waived waiting for the next Monday window; this closure does not claim that a natural Monday fire occurred. The remaining reason all observed outcomes were skips—no controlled checkout inside the Nexus plane—is successor CUST-WP-0064, not hidden closure prose.

Acceptance

• SBOM product lives in sbom-nexus; State Hub is not the permanent store
• Catch-up API returns oldest-N in one call (default N=3)
• Daily activity-core job updates those N instead of spawning one task per stale repo
• Weekly SBOM task flood is off
• Skipped repos do not permanently head the queue
• /repos/ last_sbom_at remains truthful during and after cutover

Closure Review — 2026-08-22

The coordination goal is achieved by the finished child workplans SBOM-WP-0001, SBOM-WP-0002, ACTIVITY-WP-0030, and ACTIVITY-WP-0033, plus the SBOM disposition slice of STATE-WP-0079. Production evidence shows a single Nexus authority, reversible State Hub compatibility, bounded queue advancement, zero task flood, and truthful attempt/success projections.

Actionable leftovers are live before closure:

• CUST-WP-0064 coordinates a controlled scan-input topology so daily runs create real authoritative SBOMs rather than only terminal no-checkout outcomes;
• CUST-IN-0013 routes durable operation-id replay to SBOM Nexus;
• CUST-IN-0014 routes the observed 30-minute database-lease restart cycle to SBOM Nexus for credential reread/pool-rotation repair and live proof;
• RMGR-WP-0011 remains ready for the authoritative Repo Manager HTTP client and explicit non-authoritative preview semantics.

No legacy State Hub rows were deleted. Their retention and façade rollback role remain governed by SBOM-WP-0002-T07.