Close CUST-WP-0064 after the 2026-08-24 unassisted fire ingested clay-borg, close CUST-WP-0065 now that all 120 active repos project a classification, and close ADHOC-2026-08-25. Mark CUST-WP-0067 T02/T10 done (reverse relays already gone; work-record recovery lives on 0068). Park the later no-checkout SBOM regression as CUST-IN-0015. Teach the classification gate to use this host's checkout path.
208 lines
9.7 KiB
Markdown
208 lines
9.7 KiB
Markdown
---
|
||
id: CUST-WP-0064
|
||
type: workplan
|
||
title: "Controlled scan inputs for authoritative daily SBOM catch-up"
|
||
domain: infotech
|
||
repo: the-custodian
|
||
status: finished
|
||
owner: codex
|
||
topic_slug: custodian
|
||
planning_priority: high
|
||
planning_order: 64
|
||
created: "2026-08-22"
|
||
updated: "2026-08-28"
|
||
quality_dor: DoR-Ok
|
||
quality_dor_at: "2026-08-22"
|
||
quality_dor_by: codex
|
||
quality_dor_note: "Current no-checkout production evidence, owner boundaries, trust decisions, dependencies, acceptance evidence, and rollback requirements were reviewed against SBOM-WP-0002 and ACTIVITY-WP-0030/0033."
|
||
origin: residual
|
||
origin_ref: CUST-WP-0062
|
||
related:
|
||
- SBOM-WP-0002
|
||
- ACTIVITY-WP-0030
|
||
- ACTIVITY-WP-0033
|
||
- RMGR-WP-0011
|
||
state_hub_workstream_id: "c06ca8e9-8240-5cdf-8013-4e3a9ba8f1d8"
|
||
---
|
||
|
||
# Controlled scan inputs for authoritative daily SBOM catch-up
|
||
|
||
## Goal
|
||
|
||
Give the private SBOM Nexus production plane a controlled, revision-pinned
|
||
source input so bounded daily catch-up can produce authoritative ingested
|
||
snapshots. Current scheduling, ranking, fairness, and zero-task behavior are
|
||
proven, but production attempts are `no-checkout` because workstation paths
|
||
are not reachable inside the cluster.
|
||
|
||
This is a coordination workplan. SBOM Nexus owns scan semantics and durable
|
||
history; Repo Manager owns repository identity, active status, and source-path
|
||
projection; Activity Core owns recurrence and the at-most-N workflow bound;
|
||
the deployment package owns the runtime/network boundary.
|
||
|
||
## Select the source-transfer and trust-boundary contract
|
||
|
||
```task
|
||
id: CUST-WP-0064-T01
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "02ac7278-8536-5ce8-9027-39345aab0539"
|
||
```
|
||
|
||
Choose one controlled input shape—such as a revision-pinned Forgejo clone in a
|
||
short-lived scanner job or a content-addressed source artifact—without mounting
|
||
operator workstations into the cluster. Define repository/revision identity,
|
||
authentication custody, size/time limits, egress, provenance, unsupported repo
|
||
behavior, cleanup, and the boundary between preview and authoritative ingest.
|
||
|
||
Done when the four owning repos have one reviewable contract and rollback; do
|
||
not enable source transfer from a prose-only assumption.
|
||
|
||
**Done (2026-08-22):** live verification confirmed the private Nexus pod
|
||
cannot reach projected workstation checkout paths and that queue fairness is
|
||
therefore advancing through `no-checkout` outcomes. The Custodian fixed the
|
||
non-negotiable boundary—revision-pinned input, no workstation mount, bounded
|
||
credentials/egress/time/size, provenance, cleanup, and rollback—and routed
|
||
owner participation to SBOM Nexus (`95c1b226`), Repo Manager (`075e21de`),
|
||
Activity Core (`7233d2d1`), and the package owner (`e0af24b1`). Selection of
|
||
the concrete transfer mechanism is recorded in
|
||
`docs/sbom-controlled-scan-input-contract-v1.md`: a public Coulomb Forgejo
|
||
archive addressed by a full commit SHA, fetched and safely scanned by Nexus
|
||
from a bounded ephemeral volume. A live full-SHA archive fetch succeeded; v1
|
||
therefore needs no source credential and explicitly forbids reuse of the
|
||
Forgejo administrator PAT.
|
||
|
||
## Implement the Nexus-owned authoritative scan path
|
||
|
||
```task
|
||
id: CUST-WP-0064-T02
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "2029e525-0573-5fea-881c-d3a418b91c9d"
|
||
```
|
||
|
||
Depends on T01. Open and execute the SBOM Nexus/package child work needed to
|
||
consume the selected input, scan at a pinned revision, persist provenance, and
|
||
remove temporary source material. Preserve Nexus as the only snapshot writer
|
||
and enforce `CUST-IN-0013` operation idempotency on the mutation boundary.
|
||
|
||
**Started (2026-08-22):** the selected v1 contract fixes the application and
|
||
package changes, additive skip reasons, safety limits, durable replay rule,
|
||
and rollback sequence. Implementation handoffs now carry one shared contract
|
||
instead of separate mechanism proposals.
|
||
|
||
**Implementation checkpoint (2026-08-23):** three indexed child workplans are
|
||
now committed and pushed. `SBOM-WP-0003` (`dc07cc14`) implements migration
|
||
`0002`, durable operation receipts/replay, controlled full-SHA fetch and safe
|
||
extraction, provenance, source projection, and additive outcomes at application
|
||
revision `879012c` (25 tests passed, one conditional PostgreSQL skip; Ruff
|
||
clean; real archive scan returned 33 entries and zero errors).
|
||
`RAPP-SBOM-NEXUS-WP-0002` (`63ba3813`) stages the feature dark with a 768 MiB
|
||
ephemeral volume and Forgejo-only egress at package revision `7d8f76e` (6 tests
|
||
passed; server-side cluster admission dry-run passed). `ACTIVITY-WP-0034`
|
||
(`409c06c0`) freezes and submits the exact source reference across retries and
|
||
accepts the new terminal outcomes at revision `e690170` (434 tests passed, one
|
||
conditional integration skip; 28 focused tests passed). No production digest,
|
||
migration, feature flag, or schedule was changed. Remaining T02 work is the
|
||
reviewed image build/pin, migration `0002`, dark verification, and attended
|
||
one-repository canary. Repo Manager subsequently reported its direct producer
|
||
interface ready at revision `e6cc18b`: `rmgr sbom source-ref` resolves and
|
||
projects the full-SHA reference, verifies Nexus returns it unchanged, and
|
||
projects no workstation path. T03 therefore waits only on the Nexus/package
|
||
dark canary rather than new Repo Manager implementation.
|
||
|
||
**Done (2026-08-23):** corrected application digest
|
||
`sha256:1da0f4f008643a0dec3f00bbad15f287103aa4b469577b78cfe1d67f8b3cbe31`
|
||
is live at migration `0002 (head)`. The package enforces migration-before-
|
||
runtime ordering, Forgejo-only egress, bounded ephemeral storage, and a dark
|
||
feature default. The first promotion exposed an Alembic `SET ROLE` outer-
|
||
transaction rollback; the runtime was restored immediately, the defect gained
|
||
a regression test, and the corrected migration committed before retry rollout.
|
||
Process/database health, repository reads, and zero-restarter startup all pass.
|
||
|
||
## Retarget bounded catch-up without widening it
|
||
|
||
```task
|
||
id: CUST-WP-0064-T03
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "6c645778-be59-57b1-bf44-d974a3a1e49f"
|
||
```
|
||
|
||
Depends on T02 and `RMGR-WP-0011`. Supply the controlled input reference for
|
||
the already-fixed oldest-N target set. Activity Core must still process no more
|
||
than `catch_up_limit`, reuse the same targets and operation ids across retries,
|
||
and record terminal unsupported/failed inputs without advancing into a second
|
||
batch.
|
||
|
||
Repo Manager projected `sbom-nexus` with `checkout_path: null` and exact
|
||
`forgejo-archive-v1` revision
|
||
`b1fd3ec131666e5300aa98abcdddd46219303edb`; Nexus returned it unchanged.
|
||
It then projected the live oldest-three target set (`can-you-assist`,
|
||
`citation-engine`, `citation-evidence`) with exact public full-SHA references.
|
||
The existing unpaused Temporal schedule was reconciled unchanged at limit 3
|
||
and operator-triggered: it froze exactly those targets, spawned zero tasks,
|
||
created three provenance-bearing terminal `no-manifest` snapshots, and moved
|
||
`never_count` 94 to 91.
|
||
|
||
**Done (2026-08-23):** after that controlled batch advanced, the next oldest
|
||
three (`citation-work`, `clay-borg`, and `config-atlas`) were selected with
|
||
matching `forgejo-archive-v1` references and full 40-character revisions. An
|
||
in-worker read-only report probe on deployed Activity Core digest
|
||
`sha256:9c611a394c117c8ccfe2fd813c0dda1943ea8a7444aa4bd4742d83712b2cf559`
|
||
returned `selected_count=3` and `controlled_source_count=3`. This proves
|
||
source-reference projection follows each newly exposed bounded batch rather
|
||
than the attended target set; T04 retains only the first unassisted fire.
|
||
|
||
## Prove real daily freshness improvement
|
||
|
||
```task
|
||
id: CUST-WP-0064-T04
|
||
status: done
|
||
priority: medium
|
||
state_hub_task_id: "664d90b0-1169-58fa-9a77-df53e739f957"
|
||
```
|
||
|
||
Run an attended bounded proof, then observe a normal scheduled fire. Require at
|
||
least one `ingested` outcome with repository slug, immutable revision, snapshot
|
||
id, and licence summary; zero spawned tasks; cleanup of transient source; and
|
||
truthful `last_success_at` / State Hub compatibility projection. Record the
|
||
remaining `never_count` and operator disable/rollback controls.
|
||
|
||
The attended operation created snapshot
|
||
`04f5c0ba-d073-4577-ba2d-0854346ac7be` with 33 entries, exact revision and
|
||
archive provenance, zero errors, identical replay for the same durable key,
|
||
and an empty transient directory before and after. The feature was returned
|
||
dark without deleting the snapshot. A normal scheduled fire and fleet summary
|
||
remain the final proof.
|
||
|
||
An operator-trigger through the existing Temporal schedule subsequently
|
||
proved the production scheduled path and fleet summary without changing its
|
||
weekday cadence. The first unassisted 09:15 Europe/Berlin fire remains.
|
||
|
||
**Done (2026-08-28):** the first unassisted weekday fire ran at 2026-08-24
|
||
07:15 UTC (09:15 Europe/Berlin) against the next oldest three. `clay-borg`
|
||
ingested from `forgejo-archive-v1`:
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| snapshot | `63abb22f-0701-4d56-ba16-75bdff9362c4` |
|
||
| repo | `clay-borg` |
|
||
| revision | `18c57f2e9dce0589c1f0e24dd4d939aea50cee86` |
|
||
| entries | 77 (`Cargo.lock`) |
|
||
| provenance | archive sha256 `df9e5fac…`, 1_033_378 bytes |
|
||
| siblings | `citation-work` and `config-atlas` terminal `no-manifest` at pinned SHAs |
|
||
|
||
`never_count` is 76 (was 94 on 2026-08-23). Daily 07:15 UTC fires continued
|
||
24–28 August. Residual: from 2026-08-25 the same schedule writes `no-checkout`
|
||
again, so source-ref projection is not following later batches —
|
||
`CUST-IN-0015`.
|
||
|
||
## Acceptance
|
||
|
||
- [x] Production scans consume a controlled, revision-pinned source input
|
||
- [x] No workstation filesystem is mounted or implicitly trusted
|
||
- [x] Nexus remains the only authoritative snapshot writer
|
||
- [x] One fire remains bounded to its original N targets across retries
|
||
- [x] At least one normal scheduled fire produces real ingested snapshots
|
||
- [x] Source cleanup, provenance, failure evidence, and rollback are verified
|