the-custodian/workplans/CUST-WP-0064-sbom-controlled-scan-inputs.md
codex 93b8174abd
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Python Tests / pytest (push) Successful in 21s
finish three custodian workplans from live hub evidence
Close CUST-WP-0064 after the 2026-08-24 unassisted fire ingested
clay-borg, close CUST-WP-0065 now that all 120 active repos project a
classification, and close ADHOC-2026-08-25. Mark CUST-WP-0067 T02/T10
done (reverse relays already gone; work-record recovery lives on 0068).
Park the later no-checkout SBOM regression as CUST-IN-0015. Teach the
classification gate to use this host's checkout path.
2026-08-28 20:27:05 +02:00

9.7 KiB
Raw Permalink Blame History

id type title domain repo status owner topic_slug planning_priority planning_order created updated quality_dor quality_dor_at quality_dor_by quality_dor_note origin origin_ref related state_hub_workstream_id
CUST-WP-0064 workplan Controlled scan inputs for authoritative daily SBOM catch-up infotech the-custodian finished codex custodian high 64 2026-08-22 2026-08-28 DoR-Ok 2026-08-22 codex Current no-checkout production evidence, owner boundaries, trust decisions, dependencies, acceptance evidence, and rollback requirements were reviewed against SBOM-WP-0002 and ACTIVITY-WP-0030/0033. residual CUST-WP-0062
SBOM-WP-0002
ACTIVITY-WP-0030
ACTIVITY-WP-0033
RMGR-WP-0011
c06ca8e9-8240-5cdf-8013-4e3a9ba8f1d8

Controlled scan inputs for authoritative daily SBOM catch-up

Goal

Give the private SBOM Nexus production plane a controlled, revision-pinned source input so bounded daily catch-up can produce authoritative ingested snapshots. Current scheduling, ranking, fairness, and zero-task behavior are proven, but production attempts are no-checkout because workstation paths are not reachable inside the cluster.

This is a coordination workplan. SBOM Nexus owns scan semantics and durable history; Repo Manager owns repository identity, active status, and source-path projection; Activity Core owns recurrence and the at-most-N workflow bound; the deployment package owns the runtime/network boundary.

Select the source-transfer and trust-boundary contract

id: CUST-WP-0064-T01
status: done
priority: high
state_hub_task_id: "02ac7278-8536-5ce8-9027-39345aab0539"

Choose one controlled input shape—such as a revision-pinned Forgejo clone in a short-lived scanner job or a content-addressed source artifact—without mounting operator workstations into the cluster. Define repository/revision identity, authentication custody, size/time limits, egress, provenance, unsupported repo behavior, cleanup, and the boundary between preview and authoritative ingest.

Done when the four owning repos have one reviewable contract and rollback; do not enable source transfer from a prose-only assumption.

Done (2026-08-22): live verification confirmed the private Nexus pod cannot reach projected workstation checkout paths and that queue fairness is therefore advancing through no-checkout outcomes. The Custodian fixed the non-negotiable boundary—revision-pinned input, no workstation mount, bounded credentials/egress/time/size, provenance, cleanup, and rollback—and routed owner participation to SBOM Nexus (95c1b226), Repo Manager (075e21de), Activity Core (7233d2d1), and the package owner (e0af24b1). Selection of the concrete transfer mechanism is recorded in docs/sbom-controlled-scan-input-contract-v1.md: a public Coulomb Forgejo archive addressed by a full commit SHA, fetched and safely scanned by Nexus from a bounded ephemeral volume. A live full-SHA archive fetch succeeded; v1 therefore needs no source credential and explicitly forbids reuse of the Forgejo administrator PAT.

Implement the Nexus-owned authoritative scan path

id: CUST-WP-0064-T02
status: done
priority: high
state_hub_task_id: "2029e525-0573-5fea-881c-d3a418b91c9d"

Depends on T01. Open and execute the SBOM Nexus/package child work needed to consume the selected input, scan at a pinned revision, persist provenance, and remove temporary source material. Preserve Nexus as the only snapshot writer and enforce CUST-IN-0013 operation idempotency on the mutation boundary.

Started (2026-08-22): the selected v1 contract fixes the application and package changes, additive skip reasons, safety limits, durable replay rule, and rollback sequence. Implementation handoffs now carry one shared contract instead of separate mechanism proposals.

Implementation checkpoint (2026-08-23): three indexed child workplans are now committed and pushed. SBOM-WP-0003 (dc07cc14) implements migration 0002, durable operation receipts/replay, controlled full-SHA fetch and safe extraction, provenance, source projection, and additive outcomes at application revision 879012c (25 tests passed, one conditional PostgreSQL skip; Ruff clean; real archive scan returned 33 entries and zero errors). RAPP-SBOM-NEXUS-WP-0002 (63ba3813) stages the feature dark with a 768 MiB ephemeral volume and Forgejo-only egress at package revision 7d8f76e (6 tests passed; server-side cluster admission dry-run passed). ACTIVITY-WP-0034 (409c06c0) freezes and submits the exact source reference across retries and accepts the new terminal outcomes at revision e690170 (434 tests passed, one conditional integration skip; 28 focused tests passed). No production digest, migration, feature flag, or schedule was changed. Remaining T02 work is the reviewed image build/pin, migration 0002, dark verification, and attended one-repository canary. Repo Manager subsequently reported its direct producer interface ready at revision e6cc18b: rmgr sbom source-ref resolves and projects the full-SHA reference, verifies Nexus returns it unchanged, and projects no workstation path. T03 therefore waits only on the Nexus/package dark canary rather than new Repo Manager implementation.

Done (2026-08-23): corrected application digest sha256:1da0f4f008643a0dec3f00bbad15f287103aa4b469577b78cfe1d67f8b3cbe31 is live at migration 0002 (head). The package enforces migration-before- runtime ordering, Forgejo-only egress, bounded ephemeral storage, and a dark feature default. The first promotion exposed an Alembic SET ROLE outer- transaction rollback; the runtime was restored immediately, the defect gained a regression test, and the corrected migration committed before retry rollout. Process/database health, repository reads, and zero-restarter startup all pass.

Retarget bounded catch-up without widening it

id: CUST-WP-0064-T03
status: done
priority: high
state_hub_task_id: "6c645778-be59-57b1-bf44-d974a3a1e49f"

Depends on T02 and RMGR-WP-0011. Supply the controlled input reference for the already-fixed oldest-N target set. Activity Core must still process no more than catch_up_limit, reuse the same targets and operation ids across retries, and record terminal unsupported/failed inputs without advancing into a second batch.

Repo Manager projected sbom-nexus with checkout_path: null and exact forgejo-archive-v1 revision b1fd3ec131666e5300aa98abcdddd46219303edb; Nexus returned it unchanged. It then projected the live oldest-three target set (can-you-assist, citation-engine, citation-evidence) with exact public full-SHA references. The existing unpaused Temporal schedule was reconciled unchanged at limit 3 and operator-triggered: it froze exactly those targets, spawned zero tasks, created three provenance-bearing terminal no-manifest snapshots, and moved never_count 94 to 91.

Done (2026-08-23): after that controlled batch advanced, the next oldest three (citation-work, clay-borg, and config-atlas) were selected with matching forgejo-archive-v1 references and full 40-character revisions. An in-worker read-only report probe on deployed Activity Core digest sha256:9c611a394c117c8ccfe2fd813c0dda1943ea8a7444aa4bd4742d83712b2cf559 returned selected_count=3 and controlled_source_count=3. This proves source-reference projection follows each newly exposed bounded batch rather than the attended target set; T04 retains only the first unassisted fire.

Prove real daily freshness improvement

id: CUST-WP-0064-T04
status: done
priority: medium
state_hub_task_id: "664d90b0-1169-58fa-9a77-df53e739f957"

Run an attended bounded proof, then observe a normal scheduled fire. Require at least one ingested outcome with repository slug, immutable revision, snapshot id, and licence summary; zero spawned tasks; cleanup of transient source; and truthful last_success_at / State Hub compatibility projection. Record the remaining never_count and operator disable/rollback controls.

The attended operation created snapshot 04f5c0ba-d073-4577-ba2d-0854346ac7be with 33 entries, exact revision and archive provenance, zero errors, identical replay for the same durable key, and an empty transient directory before and after. The feature was returned dark without deleting the snapshot. A normal scheduled fire and fleet summary remain the final proof.

An operator-trigger through the existing Temporal schedule subsequently proved the production scheduled path and fleet summary without changing its weekday cadence. The first unassisted 09:15 Europe/Berlin fire remains.

Done (2026-08-28): the first unassisted weekday fire ran at 2026-08-24 07:15 UTC (09:15 Europe/Berlin) against the next oldest three. clay-borg ingested from forgejo-archive-v1:

Field Value
snapshot 63abb22f-0701-4d56-ba16-75bdff9362c4
repo clay-borg
revision 18c57f2e9dce0589c1f0e24dd4d939aea50cee86
entries 77 (Cargo.lock)
provenance archive sha256 df9e5fac…, 1_033_378 bytes
siblings citation-work and config-atlas terminal no-manifest at pinned SHAs

never_count is 76 (was 94 on 2026-08-23). Daily 07:15 UTC fires continued 2428 August. Residual: from 2026-08-25 the same schedule writes no-checkout again, so source-ref projection is not following later batches — CUST-IN-0015.

Acceptance

  • Production scans consume a controlled, revision-pinned source input
  • No workstation filesystem is mounted or implicitly trusted
  • Nexus remains the only authoritative snapshot writer
  • One fire remains bounded to its original N targets across retries
  • At least one normal scheduled fire produces real ingested snapshots
  • Source cleanup, provenance, failure evidence, and rollback are verified