sbom-nexus/docs/state-hub-sbom-extraction-review.md
tegwick d61698ea51 feat: establish sbom nexus extraction slice
Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a028f0-a42f-7582-89a8-ebaad7343834
2026-08-22 12:50:08 +02:00

9.4 KiB

State Hub SBOM extraction review

Reviewed: 2026-08-22

Coordination: CUST-WP-0062-T01, STATE-WP-0079, RMGR-WP-0008, ACTIVITY-WP-0030

Outcome

The SBOM capability is a coherent product embedded in State Hub, with a newer derived-scanner copy in Repo Manager and an already-prepared Activity Core consumer contract. The target ownership is:

  • SBOM Nexus: scanning, ingest, snapshots, entries, history, reports, freshness, catch-up, import/export;
  • Repo Manager: repository identity, active status, and checkout/path representation; invoke Nexus rather than retain a second scanner;
  • State Hub: temporary compatibility façade and repository freshness projection only;
  • Activity Core: recurrence and the at-most-N side effect;
  • dashboard/MCP/CLI callers: retarget to Nexus or the temporary façade.

This corrects a timing overlap in the retirement work: RMGR-WP-0008 built a good repository-derived scanner before CUST-WP-0062 established the dedicated Nexus. That scanner is the best extraction source, but Repo Manager is not the durable SBOM product owner.

Current behavior

State Hub stores an immutable snapshot container and entries keyed to a managed repository. Manual ingest creates a snapshot, stores entries, and updates ManagedRepo.last_sbom_at. Reads return snapshot history, snapshot detail, latest entries per repository, a repository view, and a licence grouping with a simple copyleft-family signal.

Its ingest script recursively recognizes eight source families and posts the merged result to the API. MCP, the statehub CLI, Make targets, onboarding, repository health, summary caches, and dashboard pages depend on these routes.

Repo Manager subsequently improved the derivation step with source-path and SHA-256 provenance, Git revision, canonical generation time, structured parser errors, and direct-production copyleft findings. Its snapshot schema is repo-manager.sbom-snapshot.v1.

Activity Core already contains a disabled/test-double consumer for the desired Nexus endpoint. It expects GET /sbom/catch-up?limit=N, defaults to 3, bounds N to 1..25, and requires the Nexus to return an ordered repos list plus fleet counts. The old weekly task-flood definition is disabled.

Artifact and caller disposition

Existing surface Location Disposition Target/action
SBOMSnapshot model/table state-hub/api/models/sbom_snapshot.py, migrations move/import Nexus snapshot store; retain legacy id/timestamp/source provenance
SBOMEntry model/table and ecosystem enum state-hub/api/models/sbom_entry.py, migrations move/import Nexus entries; retain compatibility field names
manual ingest POST /sbom/ingest/ move + proxy Implement compat route in Nexus; State Hub proxies during cutover
snapshot list/detail GET /sbom/snapshots/* move + proxy Nexus owns history and response contract
latest entry filters GET /sbom/ move + proxy Nexus owns current projections
repository view GET /sbom/{slug} move + proxy Nexus owns inventory; join/sync minimal repo projection
licence report GET /sbom/report/licences/ move + proxy Nexus owns qualified signal and later SPDX policy integration
parser/detection script state-hub/scripts/ingest_sbom.py move, then retire Extract improved Repo Manager variant into independent Nexus scanner
derived scanner repo-manager/src/repo_manager/sbom.py move/replace Make Nexus the implementation; Repo Manager becomes caller/adapter
scanner CLI rmgr sbom scan, licence-report retarget Preserve CLI UX by calling Nexus/local Nexus library during transition
State Hub Make target make ingest-sbom retarget Invoke Nexus CLI/API; retain temporary alias with deprecation notice
statehub ingest-sbom custodian_cli.py retarget Invoke Nexus, not State Hub-local script
tool capture script/prompt scripts/capture_sbom_tools.py, prompts/sbom-capture-agent.md move later Nexus operator workflow; keep reviewed manifest as repository authority
MCP resources state://sbom/* retarget/proxy Fetch Nexus; later publish Nexus-native MCP if useful
MCP ingest/report tools ingest_sbom_tool, get_licence_report retarget Call Nexus contracts
dashboard SBOM page/data loader dashboard/src/sbom.md, data/sbom.json.py retarget Read Nexus or façade; no new dashboard in extraction release
repo/dashboard coverage cards repos/overview pages projection Consume Nexus freshness projection
ManagedRepo.last_sbom_at, sbom_source State Hub repo model strangle/projection Keep compatible fields, written/synchronized from Nexus during cutover
summary cache revision api/services/summary_cache.py adapt Revision from Nexus projection/event, not local SBOM tables
State summary licence scan api/routers/state.py adapt Consume Nexus report/projection
DoI criterion C8 api/doi_engine.py, policies/repo-doi.md retarget Query precise Nexus attempt/success state
edge cache /sbom/ api/edge/read_cache.py retire/retarget Cache façade only while compatibility route exists
repo registration optional ingest scripts/register_project.sh and onboarding docs retarget Register projection then invoke Nexus ingest
repository synchronization view dashboard/src/repo-sync.md retarget Use Nexus status
Activity Core bulk status resolver old repo_sbom_status retire Replace with one Nexus catch-up query
daily bounded consumer activity_core/context_resolvers/sbom_nexus.py retain/activate later Contract is accepted input for Nexus implementation
weekly task flood weekly-sbom-staleness retire Keep disabled; no one-task-per-stale-repo behavior
historical State Hub rows 22 snapshots/18 repos in RMGR evidence import + retain Import history; reconcile counts; delete only after retention gate

Extracted source coverage

The first Nexus scanner preserves the proven coverage common to State Hub and Repo Manager:

Source Ecosystem Important behavior
uv.lock Python all locked packages
requirements.txt Python declared lines marked direct
package-lock.json Node package metadata, dev/direct hints, licence when present
yarn.lock Node locked package/version extraction
Cargo.lock Rust all locked packages
go.sum + go.mod Go deduplicated modules; direct hint from go.mod
.terraform.lock.hcl Terraform provider/version entries
ansible/requirements.y[a]ml Ansible collections and roles under ansible directories
root sbom-tools.yaml Tool/declared reviewed escape hatch including licence and direct/dev flags

Excluded directories remain .git, VCS metadata, virtual environments, node_modules, caches, distribution/build outputs, Rust target, tox, and nox environments.

Contract differences and improvements

  1. Nexus records last_attempt_at separately from last_success_at. State Hub's single last_sbom_at cannot truthfully express a skipped scan.
  2. Every attempt has a terminal status. no-checkout and no-manifest are successful control-loop outcomes but not successful inventories.
  3. Parser errors are stored with the snapshot; they are not silently reduced to empty entries.
  4. Source paths, digests, and Git revision are retained from the Repo Manager implementation.
  5. Copyleft substring matching remains compatibility triage. It is explicitly not an SPDX expression evaluator or legal conclusion.
  6. Catch-up uses last attempt for queue fairness and last success for inventory truth. Consumers can display both rather than infer one from the other.

Historical migration requirements

The migration must enumerate State Hub snapshots by repository and oldest first, fetch each snapshot detail, upsert the repository projection, and insert the snapshot with its legacy UUID and timestamp. It must be idempotent on the legacy UUID and compare:

  • repository count;
  • snapshot count per repository and total;
  • entry count per snapshot and total;
  • earliest/latest timestamps;
  • sampled current-repository views;
  • licence group and direct-production copyleft counts.

Fresh repository scans do not replace imported history. They become later snapshots based on current source evidence.

Cutover gates

  • Nexus production store, migrations, backup, and restore evidence exist.
  • Historical import reconciles before State Hub deletion is considered.
  • State Hub compatibility tests pass through the façade.
  • Repo Manager no longer presents its scanner as independent SBOM authority.
  • Dashboard, MCP, CLI, DoI, summary, and onboarding callers are inventoried and retargeted.
  • Activity Core's ingest side effect is integrated but remains bounded by the resolver's effective limit.
  • Two production daily fires record at most N terminal outcomes and zero spawned per-repository tasks.
  • Weekly fleet task-flood behavior remains disabled.

Review conclusion

Extraction is feasible without changing supported ecosystems. The principal risk is not parser migration; it is authority and history cutover across three temporarily overlapping implementations. The first Nexus slice therefore pins the compatibility and catch-up contracts, while production ownership remains a separate gated task requiring State Hub and Repo Manager adapter changes.