5331f9af68
Write back the workplan and task UUIDs from fix-consistency so the sbom-nexus extraction plan stays indexed in WORK-RECORDS.md.
192 lines
6.9 KiB
Markdown
192 lines
6.9 KiB
Markdown
---
|
|
id: CUST-WP-0062
|
|
type: workplan
|
|
title: "Extract sbom-nexus and bound daily SBOM catch-up"
|
|
domain: infotech
|
|
repo: the-custodian
|
|
status: proposed
|
|
owner: grok
|
|
topic_slug: custodian
|
|
planning_priority: high
|
|
planning_order: 62
|
|
created: "2026-08-18"
|
|
updated: "2026-08-18"
|
|
related:
|
|
- CUST-WP-0002
|
|
- CUST-WP-0013
|
|
- CUST-WP-0043
|
|
- ACTIVITY-WP-0030
|
|
- STATE-WP-0079
|
|
state_hub_workstream_id: "6d9e5d3e-e779-471c-aaf4-74a33f9dec7f"
|
|
---
|
|
|
|
# Extract sbom-nexus and bound daily SBOM catch-up
|
|
|
|
## Goal
|
|
|
|
Move SBOM capture, storage, and evaluation out of State Hub into a dedicated
|
|
application **`sbom-nexus`**, then replace the weekly fleet-wide staleness
|
|
flood with a daily, bounded catch-up: evaluate efficiently, then update the
|
|
**N oldest-stale** repo SBOMs. **N defaults to 3**.
|
|
|
|
This is a coordination workplan (ADR-005). Implementation lands in
|
|
`sbom-nexus` (new), `state-hub` (extract + strangler), and `activity-core`
|
|
(schedule). Child plan for the schedule: **ACTIVITY-WP-0030**.
|
|
|
|
## Why now
|
|
|
|
The Monday `weekly-sbom-staleness` check on railiance01 (2026-08-17) spawned
|
|
**75 tasks**. A live State Hub scan on 2026-08-18 shows **111 / 111**
|
|
registered repos stale (>30 days or never scanned); **93 have `last_sbom_at`
|
|
null**. `activity-core` itself is at `2026-04-26`. The current rule emits one
|
|
task per stale repo, so the check cannot catch up — it only reports the
|
|
backlog.
|
|
|
|
Current evaluation (`repo_sbom_status` with `repos: all`) pulls `GET /repos/`
|
|
and computes age in activity-core. There is no oldest-N or ingest-queue
|
|
surface. Ingest, snapshots, licence report, MCP tools, and dashboard pages
|
|
still live in State Hub (`api/routers/sbom.py`, `api/models/sbom_*.py`,
|
|
`scripts/ingest_sbom.py`, `scripts/capture_sbom_tools.py`,
|
|
`prompts/sbom-capture-agent.md`, `dashboard/src/sbom.md`).
|
|
|
|
## Design
|
|
|
|
1. **`sbom-nexus` owns SBOM.** Snapshots, entries, ingest parsers, licence
|
|
report, and staleness ranking are its product surface — not a State Hub
|
|
subsystem.
|
|
2. **State Hub keeps a thin projection.** `managed_repos.last_sbom_at` /
|
|
`sbom_source` remain readable on `/repos/` via a write-back or query
|
|
adapter so existing callers do not break during cutover.
|
|
3. **Efficient evaluation.** One ranked query, not a per-repo `/sbom/{slug}`
|
|
walk and not a `for_each` over every stale repo:
|
|
`GET /sbom/catch-up?limit=N` (default N=3) returns the N repos with the
|
|
oldest `last_sbom_at` (null first), plus a fleet summary
|
|
(`stale_count`, `never_count`, `total_count`).
|
|
4. **Daily update, not weekly task flood.** activity-core runs the catch-up
|
|
daily. For those N repos it **performs ingest** (declared side-effect)
|
|
when a reachable checkout exists; otherwise it records a skip with
|
|
reason and must not retry the same impossible repo forever.
|
|
5. **Weekly check is retired at cutover.** `weekly-sbom-staleness` /
|
|
`flag-stale-sbom` stop spawning one task per stale repo. A compact
|
|
progress event remains acceptable as evidence.
|
|
|
|
### Catch-up selection
|
|
|
|
Order: never scanned first, then oldest `last_sbom_at`. Eligible repos are
|
|
active (or otherwise in-scope) registered repos. Repos with no lockfile /
|
|
no checkout are not left at the head of the queue after a recorded
|
|
empty-or-skipped snapshot (`source` distinguishes `ingest` vs `no-checkout`
|
|
vs `no-manifest`).
|
|
|
|
N is a definition/runtime parameter (`catch_up_limit`, default 3), not a
|
|
hard-coded constant in the nexus API.
|
|
|
|
## Non-goals
|
|
|
|
- Rebuilding licence policy or adding new ecosystems (already in CUST-WP-0013).
|
|
- Scanning every stale repo on every fire.
|
|
- Using coding-assistant schedulers as the recurrence or evidence source.
|
|
- Moving State Hub work-index / consistency / inbox into sbom-nexus.
|
|
|
|
## Inventory current State Hub SBOM surface
|
|
|
|
```task
|
|
id: CUST-WP-0062-T01
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "b593aeab-c435-47d1-9775-bbfbcf4cf4ab"
|
|
```
|
|
|
|
List every SBOM caller and artefact that must move or adapt: models
|
|
(`sbom_entries`, `sbom_snapshots`), router `/sbom/*`, ingest and tool-capture
|
|
scripts, MCP resources/tools, dashboard pages, `ManagedRepo.last_sbom_at`,
|
|
activity-core `repo_sbom_status`, `make ingest-sbom`, onboarding tasks.
|
|
Tag each as *move to sbom-nexus*, *strangle in state-hub*, or *retarget
|
|
caller*. Open the state-hub child workplan (next `STATE-WP`) from this
|
|
inventory.
|
|
|
|
## Stand up sbom-nexus and extract
|
|
|
|
```task
|
|
id: CUST-WP-0062-T02
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "eaf08a83-9b25-4e5e-beeb-d24fa696a626"
|
|
```
|
|
|
|
Create `/home/worsch/sbom-nexus` as a real repo (`category: tooling`),
|
|
register it, and move the SBOM product into it: store, ingest, HTTP API,
|
|
operator docs. State Hub routes keep working via proxy or dual-write until
|
|
T04. Preserve snapshot history; do not start from an empty store.
|
|
|
|
## Efficient catch-up API
|
|
|
|
```task
|
|
id: CUST-WP-0062-T03
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "936d6554-dbd9-4a24-a223-3858b49d30c3"
|
|
```
|
|
|
|
Add the ranked catch-up surface on sbom-nexus (State Hub may proxy during
|
|
strangle):
|
|
|
|
- `GET /sbom/catch-up?limit=N` — N oldest-stale repos + fleet counts
|
|
- ingest that updates those repos and advances `last_sbom_at`
|
|
- skip contract for missing checkout / no manifest, so the same repo does
|
|
not block the queue
|
|
|
|
Prove with fixtures that limit=3 returns the three oldest (null first) and
|
|
that a skip advances queue position.
|
|
|
|
## Strangle State Hub SBOM
|
|
|
|
```task
|
|
id: CUST-WP-0062-T04
|
|
status: wait
|
|
priority: medium
|
|
state_hub_task_id: "afd45c7f-55d7-4312-a6cf-df511913c998"
|
|
```
|
|
|
|
Depends on T02/T03. State Hub `/sbom/*` becomes a compatibility façade or is
|
|
documented 410-strangled with callers already on sbom-nexus. Dashboard and
|
|
MCP read sbom-nexus (or the façade). `last_sbom_at` on `/repos/` stays
|
|
correct. Align with STATE-WP-0079 retirement strangler, do not invent a
|
|
second retirement path.
|
|
|
|
## Replace weekly evaluation in activity-core
|
|
|
|
```task
|
|
id: CUST-WP-0062-T05
|
|
status: wait
|
|
priority: high
|
|
state_hub_task_id: "0bdfbc56-3293-4578-a266-e4ca9313aabf"
|
|
```
|
|
|
|
Depends on T03. Implement in **ACTIVITY-WP-0030**: daily ActivityDefinition,
|
|
resolver against the catch-up API, ingest of N repos (default 3),
|
|
deterministic progress evidence, retire `weekly-sbom-staleness` task flood.
|
|
Do not enable the daily job against the old per-repo `/sbom/{slug}` walk.
|
|
|
|
## Cutover and catch-up proof
|
|
|
|
```task
|
|
id: CUST-WP-0062-T06
|
|
status: wait
|
|
priority: medium
|
|
state_hub_task_id: "ceb75c3d-945f-4fc2-9399-5df8b5baa10d"
|
|
```
|
|
|
|
Depends on T05. On railiance01: daily job fires, updates ≤N SBOMs, posts
|
|
progress (`stale_count`, `updated`, `skipped`), weekly spawn count goes to
|
|
zero. After several successful days, `never_count` declines. Document the
|
|
operator knobs (`catch_up_limit`, disable/enable).
|
|
|
|
## Acceptance
|
|
|
|
- [ ] SBOM product lives in `sbom-nexus`; State Hub is not the permanent store
|
|
- [ ] Catch-up API returns oldest-N in one call (default N=3)
|
|
- [ ] Daily activity-core job updates those N instead of spawning one task per stale repo
|
|
- [ ] Weekly SBOM task flood is off
|
|
- [ ] Skipped repos do not permanently head the queue
|
|
- [ ] `/repos/` last_sbom_at remains truthful during and after cutover
|