the-custodian/workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md
codex 93b8174abd
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Python Tests / pytest (push) Successful in 21s
finish three custodian workplans from live hub evidence
Close CUST-WP-0064 after the 2026-08-24 unassisted fire ingested
clay-borg, close CUST-WP-0065 now that all 120 active repos project a
classification, and close ADHOC-2026-08-25. Mark CUST-WP-0067 T02/T10
done (reverse relays already gone; work-record recovery lives on 0068).
Park the later no-checkout SBOM regression as CUST-IN-0015. Teach the
classification gate to use this host's checkout path.
2026-08-28 20:27:05 +02:00

208 lines
9.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
id: CUST-WP-0064
type: workplan
title: "Controlled scan inputs for authoritative daily SBOM catch-up"
domain: infotech
repo: the-custodian
status: finished
owner: codex
topic_slug: custodian
planning_priority: high
planning_order: 64
created: "2026-08-22"
updated: "2026-08-28"
quality_dor: DoR-Ok
quality_dor_at: "2026-08-22"
quality_dor_by: codex
quality_dor_note: "Current no-checkout production evidence, owner boundaries, trust decisions, dependencies, acceptance evidence, and rollback requirements were reviewed against SBOM-WP-0002 and ACTIVITY-WP-0030/0033."
origin: residual
origin_ref: CUST-WP-0062
related:
- SBOM-WP-0002
- ACTIVITY-WP-0030
- ACTIVITY-WP-0033
- RMGR-WP-0011
state_hub_workstream_id: "c06ca8e9-8240-5cdf-8013-4e3a9ba8f1d8"
---
# Controlled scan inputs for authoritative daily SBOM catch-up
## Goal
Give the private SBOM Nexus production plane a controlled, revision-pinned
source input so bounded daily catch-up can produce authoritative ingested
snapshots. Current scheduling, ranking, fairness, and zero-task behavior are
proven, but production attempts are `no-checkout` because workstation paths
are not reachable inside the cluster.
This is a coordination workplan. SBOM Nexus owns scan semantics and durable
history; Repo Manager owns repository identity, active status, and source-path
projection; Activity Core owns recurrence and the at-most-N workflow bound;
the deployment package owns the runtime/network boundary.
## Select the source-transfer and trust-boundary contract
```task
id: CUST-WP-0064-T01
status: done
priority: high
state_hub_task_id: "02ac7278-8536-5ce8-9027-39345aab0539"
```
Choose one controlled input shape—such as a revision-pinned Forgejo clone in a
short-lived scanner job or a content-addressed source artifact—without mounting
operator workstations into the cluster. Define repository/revision identity,
authentication custody, size/time limits, egress, provenance, unsupported repo
behavior, cleanup, and the boundary between preview and authoritative ingest.
Done when the four owning repos have one reviewable contract and rollback; do
not enable source transfer from a prose-only assumption.
**Done (2026-08-22):** live verification confirmed the private Nexus pod
cannot reach projected workstation checkout paths and that queue fairness is
therefore advancing through `no-checkout` outcomes. The Custodian fixed the
non-negotiable boundary—revision-pinned input, no workstation mount, bounded
credentials/egress/time/size, provenance, cleanup, and rollback—and routed
owner participation to SBOM Nexus (`95c1b226`), Repo Manager (`075e21de`),
Activity Core (`7233d2d1`), and the package owner (`e0af24b1`). Selection of
the concrete transfer mechanism is recorded in
`docs/sbom-controlled-scan-input-contract-v1.md`: a public Coulomb Forgejo
archive addressed by a full commit SHA, fetched and safely scanned by Nexus
from a bounded ephemeral volume. A live full-SHA archive fetch succeeded; v1
therefore needs no source credential and explicitly forbids reuse of the
Forgejo administrator PAT.
## Implement the Nexus-owned authoritative scan path
```task
id: CUST-WP-0064-T02
status: done
priority: high
state_hub_task_id: "2029e525-0573-5fea-881c-d3a418b91c9d"
```
Depends on T01. Open and execute the SBOM Nexus/package child work needed to
consume the selected input, scan at a pinned revision, persist provenance, and
remove temporary source material. Preserve Nexus as the only snapshot writer
and enforce `CUST-IN-0013` operation idempotency on the mutation boundary.
**Started (2026-08-22):** the selected v1 contract fixes the application and
package changes, additive skip reasons, safety limits, durable replay rule,
and rollback sequence. Implementation handoffs now carry one shared contract
instead of separate mechanism proposals.
**Implementation checkpoint (2026-08-23):** three indexed child workplans are
now committed and pushed. `SBOM-WP-0003` (`dc07cc14`) implements migration
`0002`, durable operation receipts/replay, controlled full-SHA fetch and safe
extraction, provenance, source projection, and additive outcomes at application
revision `879012c` (25 tests passed, one conditional PostgreSQL skip; Ruff
clean; real archive scan returned 33 entries and zero errors).
`RAPP-SBOM-NEXUS-WP-0002` (`63ba3813`) stages the feature dark with a 768 MiB
ephemeral volume and Forgejo-only egress at package revision `7d8f76e` (6 tests
passed; server-side cluster admission dry-run passed). `ACTIVITY-WP-0034`
(`409c06c0`) freezes and submits the exact source reference across retries and
accepts the new terminal outcomes at revision `e690170` (434 tests passed, one
conditional integration skip; 28 focused tests passed). No production digest,
migration, feature flag, or schedule was changed. Remaining T02 work is the
reviewed image build/pin, migration `0002`, dark verification, and attended
one-repository canary. Repo Manager subsequently reported its direct producer
interface ready at revision `e6cc18b`: `rmgr sbom source-ref` resolves and
projects the full-SHA reference, verifies Nexus returns it unchanged, and
projects no workstation path. T03 therefore waits only on the Nexus/package
dark canary rather than new Repo Manager implementation.
**Done (2026-08-23):** corrected application digest
`sha256:1da0f4f008643a0dec3f00bbad15f287103aa4b469577b78cfe1d67f8b3cbe31`
is live at migration `0002 (head)`. The package enforces migration-before-
runtime ordering, Forgejo-only egress, bounded ephemeral storage, and a dark
feature default. The first promotion exposed an Alembic `SET ROLE` outer-
transaction rollback; the runtime was restored immediately, the defect gained
a regression test, and the corrected migration committed before retry rollout.
Process/database health, repository reads, and zero-restarter startup all pass.
## Retarget bounded catch-up without widening it
```task
id: CUST-WP-0064-T03
status: done
priority: high
state_hub_task_id: "6c645778-be59-57b1-bf44-d974a3a1e49f"
```
Depends on T02 and `RMGR-WP-0011`. Supply the controlled input reference for
the already-fixed oldest-N target set. Activity Core must still process no more
than `catch_up_limit`, reuse the same targets and operation ids across retries,
and record terminal unsupported/failed inputs without advancing into a second
batch.
Repo Manager projected `sbom-nexus` with `checkout_path: null` and exact
`forgejo-archive-v1` revision
`b1fd3ec131666e5300aa98abcdddd46219303edb`; Nexus returned it unchanged.
It then projected the live oldest-three target set (`can-you-assist`,
`citation-engine`, `citation-evidence`) with exact public full-SHA references.
The existing unpaused Temporal schedule was reconciled unchanged at limit 3
and operator-triggered: it froze exactly those targets, spawned zero tasks,
created three provenance-bearing terminal `no-manifest` snapshots, and moved
`never_count` 94 to 91.
**Done (2026-08-23):** after that controlled batch advanced, the next oldest
three (`citation-work`, `clay-borg`, and `config-atlas`) were selected with
matching `forgejo-archive-v1` references and full 40-character revisions. An
in-worker read-only report probe on deployed Activity Core digest
`sha256:9c611a394c117c8ccfe2fd813c0dda1943ea8a7444aa4bd4742d83712b2cf559`
returned `selected_count=3` and `controlled_source_count=3`. This proves
source-reference projection follows each newly exposed bounded batch rather
than the attended target set; T04 retains only the first unassisted fire.
## Prove real daily freshness improvement
```task
id: CUST-WP-0064-T04
status: done
priority: medium
state_hub_task_id: "664d90b0-1169-58fa-9a77-df53e739f957"
```
Run an attended bounded proof, then observe a normal scheduled fire. Require at
least one `ingested` outcome with repository slug, immutable revision, snapshot
id, and licence summary; zero spawned tasks; cleanup of transient source; and
truthful `last_success_at` / State Hub compatibility projection. Record the
remaining `never_count` and operator disable/rollback controls.
The attended operation created snapshot
`04f5c0ba-d073-4577-ba2d-0854346ac7be` with 33 entries, exact revision and
archive provenance, zero errors, identical replay for the same durable key,
and an empty transient directory before and after. The feature was returned
dark without deleting the snapshot. A normal scheduled fire and fleet summary
remain the final proof.
An operator-trigger through the existing Temporal schedule subsequently
proved the production scheduled path and fleet summary without changing its
weekday cadence. The first unassisted 09:15 Europe/Berlin fire remains.
**Done (2026-08-28):** the first unassisted weekday fire ran at 2026-08-24
07:15 UTC (09:15 Europe/Berlin) against the next oldest three. `clay-borg`
ingested from `forgejo-archive-v1`:
| Field | Value |
|---|---|
| snapshot | `63abb22f-0701-4d56-ba16-75bdff9362c4` |
| repo | `clay-borg` |
| revision | `18c57f2e9dce0589c1f0e24dd4d939aea50cee86` |
| entries | 77 (`Cargo.lock`) |
| provenance | archive sha256 `df9e5fac…`, 1_033_378 bytes |
| siblings | `citation-work` and `config-atlas` terminal `no-manifest` at pinned SHAs |
`never_count` is 76 (was 94 on 2026-08-23). Daily 07:15 UTC fires continued
2428 August. Residual: from 2026-08-25 the same schedule writes `no-checkout`
again, so source-ref projection is not following later batches —
`CUST-IN-0015`.
## Acceptance
- [x] Production scans consume a controlled, revision-pinned source input
- [x] No workstation filesystem is mounted or implicitly trusted
- [x] Nexus remains the only authoritative snapshot writer
- [x] One fire remains bounded to its original N targets across retries
- [x] At least one normal scheduled fire produces real ingested snapshots
- [x] Source cleanup, provenance, failure evidence, and rollback are verified