workplans: close sbom extraction coordination

2026-08-22 22:56:52 +02:00 · 2026-08-22 22:56:52 +02:00 · c1756de260
commit c1756de260
parent 01e7c7b4e4
4 changed files with 220 additions and 27 deletions
--- a/WORK-RECORDS.md
+++ b/WORK-RECORDS.md
@ -60,8 +60,9 @@
 | workplan | CUST-WP-0059 | finished | — | workplans/CUST-WP-0059-forgejo-ci-publish-cutover-tail.md |
 | workplan | CUST-WP-0060 | finished | — | workplans/CUST-WP-0060-work-record-canonization.md |
 | workplan | CUST-WP-0061 | finished | — | workplans/CUST-WP-0061-work-record-stage3-intake-promotion-index.md |
-| workplan | CUST-WP-0062 | proposed | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
+| workplan | CUST-WP-0062 | finished | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
 | workplan | CUST-WP-0063 | finished | — | workplans/CUST-WP-0063-inbox-governance-packets.md |
+| workplan | CUST-WP-0064 | ready | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
 | task | ADHOC-2026-05-02-T01 | done | — | workplans/ADHOC-2026-05-02.md |
 | task | ADHOC-2026-07-02-T01 | done | — | workplans/ADHOC-2026-07-02.md |
 | task | CUST-WP-0001-T01 | done | — | workplans/CUST-WP-0001-custodian-agent-runtime.md |
@ -394,17 +395,21 @@
 | task | CUST-WP-0061-T04 | done | — | workplans/CUST-WP-0061-work-record-stage3-intake-promotion-index.md |
 | task | CUST-WP-0061-T05 | done | — | workplans/CUST-WP-0061-work-record-stage3-intake-promotion-index.md |
 | task | CUST-WP-0061-T06 | done | — | workplans/CUST-WP-0061-work-record-stage3-intake-promotion-index.md |
-| task | CUST-WP-0062-T01 | todo | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
-| task | CUST-WP-0062-T02 | todo | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
-| task | CUST-WP-0062-T03 | todo | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
-| task | CUST-WP-0062-T04 | wait | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
-| task | CUST-WP-0062-T05 | wait | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
-| task | CUST-WP-0062-T06 | wait | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
+| task | CUST-WP-0062-T01 | done | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
+| task | CUST-WP-0062-T02 | done | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
+| task | CUST-WP-0062-T03 | done | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
+| task | CUST-WP-0062-T04 | done | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
+| task | CUST-WP-0062-T05 | done | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
+| task | CUST-WP-0062-T06 | done | — | workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md |
 | task | CUST-WP-0063-T01 | done | — | workplans/CUST-WP-0063-inbox-governance-packets.md |
 | task | CUST-WP-0063-T02 | done | — | workplans/CUST-WP-0063-inbox-governance-packets.md |
 | task | CUST-WP-0063-T03 | done | — | workplans/CUST-WP-0063-inbox-governance-packets.md |
 | task | CUST-WP-0063-T04 | done | — | workplans/CUST-WP-0063-inbox-governance-packets.md |
 | task | CUST-WP-0063-T05 | done | — | workplans/CUST-WP-0063-inbox-governance-packets.md |
+| task | CUST-WP-0064-T01 | todo | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
+| task | CUST-WP-0064-T02 | wait | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
+| task | CUST-WP-0064-T03 | wait | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
+| task | CUST-WP-0064-T04 | wait | — | workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md |
 | intake | CUST-IN-0001 | closed | green | intake-legacy-suggestions-migration.md |
 | intake | CUST-IN-0002 | closed | green | intake-legacy-suggestions-migration.md |
 | intake | CUST-IN-0003 | closed | green | intake-legacy-suggestions-migration.md |
@ -415,5 +420,6 @@
 | intake | CUST-IN-0008 | open | blue | intake-legacy-suggestions-migration.md |
 | intake | CUST-IN-0009 | open | blue | intake-legacy-suggestions-migration.md |
 | intake | CUST-IN-0010 | open | green | intake-legacy-suggestions-migration.md |
-| intake | CUST-IN-0011 | open | red | intakes.md |
+| intake | CUST-IN-0011 | routed | red | intakes.md |
 | intake | CUST-IN-0012 | open | green | intakes.md |
+| intake | CUST-IN-0013 | open | blue | intakes.md |
--- a/intakes.md
+++ b/intakes.md
@ -6,14 +6,16 @@
 id: CUST-IN-0011
 kind: intake
 title: "Provision a monitored external security-report Contact URI"
-status: open
+status: routed
 lane: red
 priority: high
-owner: the-custodian
-tags: [needs-human, compliance-relevant]
+owner: policy-nexus
+tags: [compliance-relevant]
 origin: residual
 origin_ref: CUST-WP-0063
-notes: "The minimal RFC 9116 security.txt route is approved for policy.coulomb.social/.well-known/security.txt, with reports routed privately to risk-nexus and no bounty, response-time, or safe-harbour promise. Completion requires the operator to select and provision a real monitored Contact URI; no address may be invented or published before its receiving path is verified."
+selected_contact_uri: "https://security.coulomb.social/"
+updated: "2026-08-22"
+notes: "The operator selected https://security.coulomb.social/ as the RFC 9116 Contact URI. Policy Nexus owns provisioning and receipt testing before policy.coulomb.social/.well-known/security.txt may publish it. Reports route privately to risk-nexus; the route creates no bounty, response-time, or safe-harbour promise. Close only after the HTTPS endpoint is reachable and a private test report reaches Risk Nexus."
 state_hub_intake_id: "01a02b31-f4b0-75e4-a15c-a78e1c276689"
 ```

@ -32,3 +34,18 @@ origin_ref: CUST-WP-0063
 notes: "State Hub returns unread risk-nexus message id 0b8dd0bf-41d-47da-96ac-40e443c32e47, whose second UUID group has only three characters. PATCH /messages/{id}/read rejects it during UUID path parsing, so the already-handled superseded request cannot be marked read through the supported API. Repair must preserve the message body and chronology, assign or map a valid stable identity, and then apply the read transition without direct ad hoc database mutation from this repo."
 state_hub_intake_id: "01a02b32-009b-71bd-a7bf-2ce888164d6a"
 ```
+
+## CUST-IN-0013 — Enforce durable SBOM catch-up operation idempotency
+
+```yaml
+id: CUST-IN-0013
+kind: intake
+title: "Enforce durable SBOM catch-up operation idempotency"
+status: open
+lane: blue
+priority: high
+owner: sbom-nexus
+origin: residual
+origin_ref: CUST-WP-0062
+notes: "Activity Core completed ACTIVITY-WP-0033 and now sends a stable Idempotency-Key plus X-Activity-Core-Operation-ID for each workflow-run/repository pair. SBOM Nexus must durably enforce that identity on both POST /sbom/{slug}/ingest and POST /sbom/{slug}/skip and replay the original terminal response. This closes the crash window where Nexus commits but the worker dies before its heartbeat. Source handoff: State Hub message bc5caa49-25eb-4942-9deb-411b6080d0bb."
+```
--- a/workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md
+++ b/workplans/CUST-WP-0062-sbom-nexus-daily-catchup.md
@ -4,13 +4,13 @@ type: workplan
 title: "Extract sbom-nexus and bound daily SBOM catch-up"
 domain: infotech
 repo: the-custodian
-status: proposed
-owner: grok
+status: finished
+owner: codex
 topic_slug: custodian
 planning_priority: high
 planning_order: 62
 created: "2026-08-18"
-updated: "2026-08-18"
+updated: "2026-08-22"
 related:
  - CUST-WP-0002
  - CUST-WP-0013
@ -92,7 +92,7 @@ hard-coded constant in the nexus API.

 ```task
 id: CUST-WP-0062-T01
-status: todo
+status: done
 priority: high
 state_hub_task_id: "b593aeab-c435-47d1-9775-bbfbcf4cf4ab"
 ```
@ -105,11 +105,18 @@ Tag each as *move to sbom-nexus*, *strangle in state-hub*, or *retarget
 caller*. Open the state-hub child workplan (next `STATE-WP`) from this
 inventory.

+**Done (2026-08-22):** `SBOM-WP-0001-T03` produced the complete disposition
+in `sbom-nexus/docs/state-hub-sbom-extraction-review.md`, covering State Hub
+models/routes/scripts/MCP/dashboard, Repo Manager, Activity Core, repository
+freshness projection, and migration. State Hub implementation joined its
+existing retirement workplan `STATE-WP-0079` rather than creating a competing
+strangler.
+
 ## Stand up sbom-nexus and extract

 ```task
 id: CUST-WP-0062-T02
-status: todo
+status: done
 priority: high
 state_hub_task_id: "eaf08a83-9b25-4e5e-beeb-d24fa696a626"
 ```
@ -119,11 +126,18 @@ register it, and move the SBOM product into it: store, ingest, HTTP API,
 operator docs. State Hub routes keep working via proxy or dual-write until
 T04. Preserve snapshot history; do not start from an empty store.

+**Done (2026-08-22):** `sbom-nexus` is registered as the product repository.
+`SBOM-WP-0001` delivered the service and `SBOM-WP-0002` deployed it on managed
+PostgreSQL, imported 22 snapshots / 3,123 entries exactly, and reconciled the
+second import idempotently. The private production deployment is Ready on
+immutable digest `sha256:62929abc…`; backup and a 51-second scratch restore are
+recorded in the child evidence.
+
 ## Efficient catch-up API

 ```task
 id: CUST-WP-0062-T03
-status: todo
+status: done
 priority: high
 state_hub_task_id: "936d6554-dbd9-4a24-a223-3858b49d30c3"
 ```
@ -139,11 +153,19 @@ strangle):
 Prove with fixtures that limit=3 returns the three oldest (null first) and
 that a skip advances queue position.

+**Done (2026-08-22):** Nexus implements one ranked
+`GET /sbom/catch-up?limit=N` call, bounded ingest/skip routes, null-first
+ordering, and terminal `no-checkout` / `no-manifest` semantics. Production
+projection reconciled 120 repositories and selected exactly the oldest three
+of 116 active repositories without ingest. Two bounded fires advanced
+`never_count` 101 → 98; a live read at 2026-08-22T20:51Z returned exactly
+three targets and `never_count: 95`, demonstrating continued queue advance.
+
 ## Strangle State Hub SBOM

 ```task
 id: CUST-WP-0062-T04
-status: wait
+status: done
 priority: medium
 state_hub_task_id: "afd45c7f-55d7-4312-a6cf-df511913c998"
 ```
@ -154,11 +176,17 @@ MCP read sbom-nexus (or the façade). `last_sbom_at` on `/repos/` stays
 correct. Align with STATE-WP-0079 retirement strangler, do not invent a
 second retirement path.

+**Done (2026-08-22):** `SBOM-WP-0002-T04` and `STATE-WP-0079-T04` moved State
+Hub reads and writes behind independently reversible Nexus façade flags.
+Dashboard, summary, DoI, MCP/CLI compatibility, and repository freshness read
+Nexus authority. State Hub retains its 22 historical snapshots and façade as
+read-only rollback evidence; new reads and writes remain Nexus-owned.
+
 ## Replace weekly evaluation in activity-core

 ```task
 id: CUST-WP-0062-T05
-status: wait
+status: done
 priority: high
 state_hub_task_id: "0bdfbc56-3293-4578-a266-e4ca9313aabf"
 ```
@ -168,11 +196,19 @@ resolver against the catch-up API, ingest of N repos (default 3),
 deterministic progress evidence, retire `weekly-sbom-staleness` task flood.
 Do not enable the daily job against the old per-repo `/sbom/{slug}` walk.

+**Done (2026-08-22):** `ACTIVITY-WP-0030` is finished. The weekday 09:15
+Europe/Berlin definition performs one Nexus selection and at most three
+terminal operations, emits deterministic `sbom_catchup` progress, and has no
+task-spawning rule. `weekly-sbom-staleness` is disabled and its Temporal
+schedule paused. `ACTIVITY-WP-0033` subsequently fixed the retry boundary so
+one workflow retains one target set across retries; Nexus-side durable
+operation replay remains live as `CUST-IN-0013`.
+
 ## Cutover and catch-up proof

 ```task
 id: CUST-WP-0062-T06
-status: wait
+status: done
 priority: medium
 state_hub_task_id: "ceb75c3d-945f-4fc2-9399-5df8b5baa10d"
 ```
@ -182,11 +218,42 @@ progress (`stale_count`, `updated`, `skipped`), weekly spawn count goes to
 zero. After several successful days, `never_count` declines. Document the
 operator knobs (`catch_up_limit`, disable/enable).

+**Done (2026-08-22):** two operator-confirmed Railiance01 fires processed six
+distinct repositories as terminal `no-checkout` skips, spawned zero tasks,
+emitted linked progress events `a9e8acea` and `f8b59633`, and advanced the
+queue. Production has the daily definition enabled with limit 3 and the
+legacy weekly definition disabled/paused. The SBOM product owner explicitly
+waived waiting for the next Monday window; this closure does not claim that a
+natural Monday fire occurred. The remaining reason all observed outcomes were
+skips—no controlled checkout inside the Nexus plane—is successor
+`CUST-WP-0064`, not hidden closure prose.
+
 ## Acceptance

- [ ] SBOM product lives in `sbom-nexus`; State Hub is not the permanent store
- [ ] Catch-up API returns oldest-N in one call (default N=3)
- [ ] Daily activity-core job updates those N instead of spawning one task per stale repo
- [ ] Weekly SBOM task flood is off
- [ ] Skipped repos do not permanently head the queue
- [ ] `/repos/` last_sbom_at remains truthful during and after cutover
+- [x] SBOM product lives in `sbom-nexus`; State Hub is not the permanent store
+- [x] Catch-up API returns oldest-N in one call (default N=3)
+- [x] Daily activity-core job updates those N instead of spawning one task per stale repo
+- [x] Weekly SBOM task flood is off
+- [x] Skipped repos do not permanently head the queue
+- [x] `/repos/` last_sbom_at remains truthful during and after cutover
+
+## Closure Review — 2026-08-22
+
+The coordination goal is achieved by the finished child workplans
+`SBOM-WP-0001`, `SBOM-WP-0002`, `ACTIVITY-WP-0030`, and
+`ACTIVITY-WP-0033`, plus the SBOM disposition slice of `STATE-WP-0079`.
+Production evidence shows a single Nexus authority, reversible State Hub
+compatibility, bounded queue advancement, zero task flood, and truthful
+attempt/success projections.
+
+Actionable leftovers are live before closure:
+
+- `CUST-WP-0064` coordinates a controlled scan-input topology so daily runs
+  create real authoritative SBOMs rather than only terminal `no-checkout`
+  outcomes;
+- `CUST-IN-0013` routes durable operation-id replay to SBOM Nexus;
+- `RMGR-WP-0011` remains ready for the authoritative Repo Manager HTTP client
+  and explicit non-authoritative preview semantics.
+
+No legacy State Hub rows were deleted. Their retention and façade rollback
+role remain governed by `SBOM-WP-0002-T07`.
--- a/workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md
+++ b/workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md
@ -0,0 +1,103 @@
+---
+id: CUST-WP-0064
+type: workplan
+title: "Controlled scan inputs for authoritative daily SBOM catch-up"
+domain: infotech
+repo: the-custodian
+status: ready
+owner: codex
+topic_slug: custodian
+planning_priority: high
+planning_order: 64
+created: "2026-08-22"
+updated: "2026-08-22"
+origin: residual
+origin_ref: CUST-WP-0062
+related:
+  - SBOM-WP-0002
+  - ACTIVITY-WP-0030
+  - ACTIVITY-WP-0033
+  - RMGR-WP-0011
+---
+
+# Controlled scan inputs for authoritative daily SBOM catch-up
+
+## Goal
+
+Give the private SBOM Nexus production plane a controlled, revision-pinned
+source input so bounded daily catch-up can produce authoritative ingested
+snapshots. Current scheduling, ranking, fairness, and zero-task behavior are
+proven, but production attempts are `no-checkout` because workstation paths
+are not reachable inside the cluster.
+
+This is a coordination workplan. SBOM Nexus owns scan semantics and durable
+history; Repo Manager owns repository identity, active status, and source-path
+projection; Activity Core owns recurrence and the at-most-N workflow bound;
+the deployment package owns the runtime/network boundary.
+
+## Select the source-transfer and trust-boundary contract
+
+```task
+id: CUST-WP-0064-T01
+status: todo
+priority: high
+```
+
+Choose one controlled input shape—such as a revision-pinned Forgejo clone in a
+short-lived scanner job or a content-addressed source artifact—without mounting
+operator workstations into the cluster. Define repository/revision identity,
+authentication custody, size/time limits, egress, provenance, unsupported repo
+behavior, cleanup, and the boundary between preview and authoritative ingest.
+
+Done when the four owning repos have one reviewable contract and rollback; do
+not enable source transfer from a prose-only assumption.
+
+## Implement the Nexus-owned authoritative scan path
+
+```task
+id: CUST-WP-0064-T02
+status: wait
+priority: high
+```
+
+Depends on T01. Open and execute the SBOM Nexus/package child work needed to
+consume the selected input, scan at a pinned revision, persist provenance, and
+remove temporary source material. Preserve Nexus as the only snapshot writer
+and enforce `CUST-IN-0013` operation idempotency on the mutation boundary.
+
+## Retarget bounded catch-up without widening it
+
+```task
+id: CUST-WP-0064-T03
+status: wait
+priority: high
+```
+
+Depends on T02 and `RMGR-WP-0011`. Supply the controlled input reference for
+the already-fixed oldest-N target set. Activity Core must still process no more
+than `catch_up_limit`, reuse the same targets and operation ids across retries,
+and record terminal unsupported/failed inputs without advancing into a second
+batch.
+
+## Prove real daily freshness improvement
+
+```task
+id: CUST-WP-0064-T04
+status: wait
+priority: medium
+```
+
+Run an attended bounded proof, then observe a normal scheduled fire. Require at
+least one `ingested` outcome with repository slug, immutable revision, snapshot
+id, and licence summary; zero spawned tasks; cleanup of transient source; and
+truthful `last_success_at` / State Hub compatibility projection. Record the
+remaining `never_count` and operator disable/rollback controls.
+
+## Acceptance
+
+- [ ] Production scans consume a controlled, revision-pinned source input
+- [ ] No workstation filesystem is mounted or implicitly trusted
+- [ ] Nexus remains the only authoritative snapshot writer
+- [ ] One fire remains bounded to its original N targets across retries
+- [ ] At least one normal scheduled fire produces real ingested snapshots
+- [ ] Source cleanup, provenance, failure evidence, and rollback are verified