codex
947f95e42a
These workplans exist only in the retired local hub. Their random pre-ADR-007 identifiers are refused by C-06 as stale references, so they cannot be registered. Deriving from the canonical record id takes no identity from anything: central does not hold them and the old ids die with the cache. Records central already holds were deliberately left untouched. Refs CUST-WP-0068-T06
261 lines
11 KiB
Markdown
261 lines
11 KiB
Markdown
---
|
|
id: CUST-WP-0062
|
|
type: workplan
|
|
title: "Extract sbom-nexus and bound daily SBOM catch-up"
|
|
domain: infotech
|
|
repo: the-custodian
|
|
status: finished
|
|
owner: codex
|
|
topic_slug: custodian
|
|
planning_priority: high
|
|
planning_order: 62
|
|
created: "2026-08-18"
|
|
updated: "2026-08-22"
|
|
related:
|
|
- CUST-WP-0002
|
|
- CUST-WP-0013
|
|
- CUST-WP-0043
|
|
- ACTIVITY-WP-0030
|
|
- STATE-WP-0079
|
|
state_hub_workstream_id: "ce062024-b221-54b1-8c1d-f14360d0354e"
|
|
---
|
|
|
|
# Extract sbom-nexus and bound daily SBOM catch-up
|
|
|
|
## Goal
|
|
|
|
Move SBOM capture, storage, and evaluation out of State Hub into a dedicated
|
|
application **`sbom-nexus`**, then replace the weekly fleet-wide staleness
|
|
flood with a daily, bounded catch-up: evaluate efficiently, then update the
|
|
**N oldest-stale** repo SBOMs. **N defaults to 3**.
|
|
|
|
This is a coordination workplan (ADR-005). Implementation lands in
|
|
`sbom-nexus` (new), `state-hub` (extract + strangler), and `activity-core`
|
|
(schedule). Child plan for the schedule: **ACTIVITY-WP-0030**.
|
|
|
|
## Why now
|
|
|
|
The Monday `weekly-sbom-staleness` check on railiance01 (2026-08-17) spawned
|
|
**75 tasks**. A live State Hub scan on 2026-08-18 shows **111 / 111**
|
|
registered repos stale (>30 days or never scanned); **93 have `last_sbom_at`
|
|
null**. `activity-core` itself is at `2026-04-26`. The current rule emits one
|
|
task per stale repo, so the check cannot catch up — it only reports the
|
|
backlog.
|
|
|
|
Current evaluation (`repo_sbom_status` with `repos: all`) pulls `GET /repos/`
|
|
and computes age in activity-core. There is no oldest-N or ingest-queue
|
|
surface. Ingest, snapshots, licence report, MCP tools, and dashboard pages
|
|
still live in State Hub (`api/routers/sbom.py`, `api/models/sbom_*.py`,
|
|
`scripts/ingest_sbom.py`, `scripts/capture_sbom_tools.py`,
|
|
`prompts/sbom-capture-agent.md`, `dashboard/src/sbom.md`).
|
|
|
|
## Design
|
|
|
|
1. **`sbom-nexus` owns SBOM.** Snapshots, entries, ingest parsers, licence
|
|
report, and staleness ranking are its product surface — not a State Hub
|
|
subsystem.
|
|
2. **State Hub keeps a thin projection.** `managed_repos.last_sbom_at` /
|
|
`sbom_source` remain readable on `/repos/` via a write-back or query
|
|
adapter so existing callers do not break during cutover.
|
|
3. **Efficient evaluation.** One ranked query, not a per-repo `/sbom/{slug}`
|
|
walk and not a `for_each` over every stale repo:
|
|
`GET /sbom/catch-up?limit=N` (default N=3) returns the N repos with the
|
|
oldest `last_sbom_at` (null first), plus a fleet summary
|
|
(`stale_count`, `never_count`, `total_count`).
|
|
4. **Daily update, not weekly task flood.** activity-core runs the catch-up
|
|
daily. For those N repos it **performs ingest** (declared side-effect)
|
|
when a reachable checkout exists; otherwise it records a skip with
|
|
reason and must not retry the same impossible repo forever.
|
|
5. **Weekly check is retired at cutover.** `weekly-sbom-staleness` /
|
|
`flag-stale-sbom` stop spawning one task per stale repo. A compact
|
|
progress event remains acceptable as evidence.
|
|
|
|
### Catch-up selection
|
|
|
|
Order: never scanned first, then oldest `last_sbom_at`. Eligible repos are
|
|
active (or otherwise in-scope) registered repos. Repos with no lockfile /
|
|
no checkout are not left at the head of the queue after a recorded
|
|
empty-or-skipped snapshot (`source` distinguishes `ingest` vs `no-checkout`
|
|
vs `no-manifest`).
|
|
|
|
N is a definition/runtime parameter (`catch_up_limit`, default 3), not a
|
|
hard-coded constant in the nexus API.
|
|
|
|
## Non-goals
|
|
|
|
- Rebuilding licence policy or adding new ecosystems (already in CUST-WP-0013).
|
|
- Scanning every stale repo on every fire.
|
|
- Using coding-assistant schedulers as the recurrence or evidence source.
|
|
- Moving State Hub work-index / consistency / inbox into sbom-nexus.
|
|
|
|
## Inventory current State Hub SBOM surface
|
|
|
|
```task
|
|
id: CUST-WP-0062-T01
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "9b65928b-cc69-53a8-803e-daae1109f939"
|
|
```
|
|
|
|
List every SBOM caller and artefact that must move or adapt: models
|
|
(`sbom_entries`, `sbom_snapshots`), router `/sbom/*`, ingest and tool-capture
|
|
scripts, MCP resources/tools, dashboard pages, `ManagedRepo.last_sbom_at`,
|
|
activity-core `repo_sbom_status`, `make ingest-sbom`, onboarding tasks.
|
|
Tag each as *move to sbom-nexus*, *strangle in state-hub*, or *retarget
|
|
caller*. Open the state-hub child workplan (next `STATE-WP`) from this
|
|
inventory.
|
|
|
|
**Done (2026-08-22):** `SBOM-WP-0001-T03` produced the complete disposition
|
|
in `sbom-nexus/docs/state-hub-sbom-extraction-review.md`, covering State Hub
|
|
models/routes/scripts/MCP/dashboard, Repo Manager, Activity Core, repository
|
|
freshness projection, and migration. State Hub implementation joined its
|
|
existing retirement workplan `STATE-WP-0079` rather than creating a competing
|
|
strangler.
|
|
|
|
## Stand up sbom-nexus and extract
|
|
|
|
```task
|
|
id: CUST-WP-0062-T02
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "cbbbd680-2b17-540f-a339-30436b4ebf86"
|
|
```
|
|
|
|
Create `/home/worsch/sbom-nexus` as a real repo (`category: tooling`),
|
|
register it, and move the SBOM product into it: store, ingest, HTTP API,
|
|
operator docs. State Hub routes keep working via proxy or dual-write until
|
|
T04. Preserve snapshot history; do not start from an empty store.
|
|
|
|
**Done (2026-08-22):** `sbom-nexus` is registered as the product repository.
|
|
`SBOM-WP-0001` delivered the service and `SBOM-WP-0002` deployed it on managed
|
|
PostgreSQL, imported 22 snapshots / 3,123 entries exactly, and reconciled the
|
|
second import idempotently. The private production deployment is Ready on
|
|
immutable digest `sha256:62929abc…`; backup and a 51-second scratch restore are
|
|
recorded in the child evidence.
|
|
|
|
## Efficient catch-up API
|
|
|
|
```task
|
|
id: CUST-WP-0062-T03
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "d1c0c3ed-fc55-5656-a504-5562fd848dae"
|
|
```
|
|
|
|
Add the ranked catch-up surface on sbom-nexus (State Hub may proxy during
|
|
strangle):
|
|
|
|
- `GET /sbom/catch-up?limit=N` — N oldest-stale repos + fleet counts
|
|
- ingest that updates those repos and advances `last_sbom_at`
|
|
- skip contract for missing checkout / no manifest, so the same repo does
|
|
not block the queue
|
|
|
|
Prove with fixtures that limit=3 returns the three oldest (null first) and
|
|
that a skip advances queue position.
|
|
|
|
**Done (2026-08-22):** Nexus implements one ranked
|
|
`GET /sbom/catch-up?limit=N` call, bounded ingest/skip routes, null-first
|
|
ordering, and terminal `no-checkout` / `no-manifest` semantics. Production
|
|
projection reconciled 120 repositories and selected exactly the oldest three
|
|
of 116 active repositories without ingest. Two bounded fires advanced
|
|
`never_count` 101 → 98; a live read at 2026-08-22T20:51Z returned exactly
|
|
three targets and `never_count: 95`, demonstrating continued queue advance.
|
|
|
|
## Strangle State Hub SBOM
|
|
|
|
```task
|
|
id: CUST-WP-0062-T04
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "5e3132ff-4000-5eac-9d29-017c2dcec5e4"
|
|
```
|
|
|
|
Depends on T02/T03. State Hub `/sbom/*` becomes a compatibility façade or is
|
|
documented 410-strangled with callers already on sbom-nexus. Dashboard and
|
|
MCP read sbom-nexus (or the façade). `last_sbom_at` on `/repos/` stays
|
|
correct. Align with STATE-WP-0079 retirement strangler, do not invent a
|
|
second retirement path.
|
|
|
|
**Done (2026-08-22):** `SBOM-WP-0002-T04` and `STATE-WP-0079-T04` moved State
|
|
Hub reads and writes behind independently reversible Nexus façade flags.
|
|
Dashboard, summary, DoI, MCP/CLI compatibility, and repository freshness read
|
|
Nexus authority. State Hub retains its 22 historical snapshots and façade as
|
|
read-only rollback evidence; new reads and writes remain Nexus-owned.
|
|
|
|
## Replace weekly evaluation in activity-core
|
|
|
|
```task
|
|
id: CUST-WP-0062-T05
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "27172afa-072f-5b47-a7be-5b857ba74463"
|
|
```
|
|
|
|
Depends on T03. Implement in **ACTIVITY-WP-0030**: daily ActivityDefinition,
|
|
resolver against the catch-up API, ingest of N repos (default 3),
|
|
deterministic progress evidence, retire `weekly-sbom-staleness` task flood.
|
|
Do not enable the daily job against the old per-repo `/sbom/{slug}` walk.
|
|
|
|
**Done (2026-08-22):** `ACTIVITY-WP-0030` is finished. The weekday 09:15
|
|
Europe/Berlin definition performs one Nexus selection and at most three
|
|
terminal operations, emits deterministic `sbom_catchup` progress, and has no
|
|
task-spawning rule. `weekly-sbom-staleness` is disabled and its Temporal
|
|
schedule paused. `ACTIVITY-WP-0033` subsequently fixed the retry boundary so
|
|
one workflow retains one target set across retries; Nexus-side durable
|
|
operation replay remains live as `CUST-IN-0013`.
|
|
|
|
## Cutover and catch-up proof
|
|
|
|
```task
|
|
id: CUST-WP-0062-T06
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "a15dc52b-43b5-56b1-90a7-49d5ab84c67a"
|
|
```
|
|
|
|
Depends on T05. On railiance01: daily job fires, updates ≤N SBOMs, posts
|
|
progress (`stale_count`, `updated`, `skipped`), weekly spawn count goes to
|
|
zero. After several successful days, `never_count` declines. Document the
|
|
operator knobs (`catch_up_limit`, disable/enable).
|
|
|
|
**Done (2026-08-22):** two operator-confirmed Railiance01 fires processed six
|
|
distinct repositories as terminal `no-checkout` skips, spawned zero tasks,
|
|
emitted linked progress events `a9e8acea` and `f8b59633`, and advanced the
|
|
queue. Production has the daily definition enabled with limit 3 and the
|
|
legacy weekly definition disabled/paused. The SBOM product owner explicitly
|
|
waived waiting for the next Monday window; this closure does not claim that a
|
|
natural Monday fire occurred. The remaining reason all observed outcomes were
|
|
skips—no controlled checkout inside the Nexus plane—is successor
|
|
`CUST-WP-0064`, not hidden closure prose.
|
|
|
|
## Acceptance
|
|
|
|
- [x] SBOM product lives in `sbom-nexus`; State Hub is not the permanent store
|
|
- [x] Catch-up API returns oldest-N in one call (default N=3)
|
|
- [x] Daily activity-core job updates those N instead of spawning one task per stale repo
|
|
- [x] Weekly SBOM task flood is off
|
|
- [x] Skipped repos do not permanently head the queue
|
|
- [x] `/repos/` last_sbom_at remains truthful during and after cutover
|
|
|
|
## Closure Review — 2026-08-22
|
|
|
|
The coordination goal is achieved by the finished child workplans
|
|
`SBOM-WP-0001`, `SBOM-WP-0002`, `ACTIVITY-WP-0030`, and
|
|
`ACTIVITY-WP-0033`, plus the SBOM disposition slice of `STATE-WP-0079`.
|
|
Production evidence shows a single Nexus authority, reversible State Hub
|
|
compatibility, bounded queue advancement, zero task flood, and truthful
|
|
attempt/success projections.
|
|
|
|
Actionable leftovers are live before closure:
|
|
|
|
- `CUST-WP-0064` coordinates a controlled scan-input topology so daily runs
|
|
create real authoritative SBOMs rather than only terminal `no-checkout`
|
|
outcomes;
|
|
- `CUST-IN-0013` routes durable operation-id replay to SBOM Nexus;
|
|
- `CUST-IN-0014` routes the observed 30-minute database-lease restart cycle to
|
|
SBOM Nexus for credential reread/pool-rotation repair and live proof;
|
|
- `RMGR-WP-0011` remains ready for the authoritative Repo Manager HTTP client
|
|
and explicit non-authoritative preview semantics.
|
|
|
|
No legacy State Hub rows were deleted. Their retention and façade rollback
|
|
role remain governed by `SBOM-WP-0002-T07`.
|