feat: carry controlled SBOM source refs
Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a02b22-9638-76d2-bbff-b7ea1770b118
This commit is contained in:
parent
04c0535a61
commit
e6901705be
5 changed files with 153 additions and 11 deletions
|
|
@ -37,6 +37,7 @@
|
||||||
| workplan | ACTIVITY-WP-0031 | active | — | workplans/ACTIVITY-WP-0031-production-execution-reliability-cleanup.md |
|
| workplan | ACTIVITY-WP-0031 | active | — | workplans/ACTIVITY-WP-0031-production-execution-reliability-cleanup.md |
|
||||||
| workplan | ACTIVITY-WP-0032 | active | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
| workplan | ACTIVITY-WP-0032 | active | — | workplans/ACTIVITY-WP-0032-glas-profile-execution-contract.md |
|
||||||
| workplan | ACTIVITY-WP-0033 | finished | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
| workplan | ACTIVITY-WP-0033 | finished | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
||||||
|
| workplan | ACTIVITY-WP-0034 | active | — | workplans/ACTIVITY-WP-0034-sbom-controlled-source-reference.md |
|
||||||
| workplan | ADHOC-2026-06-01 | finished | — | workplans/ADHOC-2026-06-01.md |
|
| workplan | ADHOC-2026-06-01 | finished | — | workplans/ADHOC-2026-06-01.md |
|
||||||
| workplan | ADHOC-2026-08-20 | finished | — | workplans/ADHOC-2026-08-20.md |
|
| workplan | ADHOC-2026-08-20 | finished | — | workplans/ADHOC-2026-08-20.md |
|
||||||
| workplan | custodian-WP-0001 | done | — | workplans/custodian-WP-0001-temporal-backbone.md |
|
| workplan | custodian-WP-0001 | done | — | workplans/custodian-WP-0001-temporal-backbone.md |
|
||||||
|
|
@ -203,6 +204,8 @@
|
||||||
| task | ACTIVITY-WP-0033-T01 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
| task | ACTIVITY-WP-0033-T01 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
||||||
| task | ACTIVITY-WP-0033-T02 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
| task | ACTIVITY-WP-0033-T02 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
||||||
| task | ACTIVITY-WP-0033-T03 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
| task | ACTIVITY-WP-0033-T03 | done | — | workplans/ACTIVITY-WP-0033-sbom-catchup-retry-boundary.md |
|
||||||
|
| task | ACTIVITY-WP-0034-T01 | done | — | workplans/ACTIVITY-WP-0034-sbom-controlled-source-reference.md |
|
||||||
|
| task | ACTIVITY-WP-0034-T02 | wait | — | workplans/ACTIVITY-WP-0034-sbom-controlled-source-reference.md |
|
||||||
| task | ADHOC-2026-06-01-T01 | done | — | workplans/ADHOC-2026-06-01.md |
|
| task | ADHOC-2026-06-01-T01 | done | — | workplans/ADHOC-2026-06-01.md |
|
||||||
| task | ADHOC-2026-06-01-T02 | done | — | workplans/ADHOC-2026-06-01.md |
|
| task | ADHOC-2026-06-01-T02 | done | — | workplans/ADHOC-2026-06-01.md |
|
||||||
| task | ADHOC-2026-06-01-T03 | done | — | workplans/ADHOC-2026-06-01.md |
|
| task | ADHOC-2026-06-01-T03 | done | — | workplans/ADHOC-2026-06-01.md |
|
||||||
|
|
|
||||||
|
|
@ -44,7 +44,7 @@ Context source `sbom-nexus / catch_up` (`context_resolvers/sbom_nexus.py`):
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| input | `{"limit": 3}` |
|
| input | `{"limit": 3}` |
|
||||||
| output | `{repos, selected_count, stale_count, never_count, total_count, limit}` |
|
| output | `{repos, selected_count, stale_count, never_count, total_count, limit}` |
|
||||||
| each repo | `repo_slug`, `last_sbom_at`, `sbom_age_days`, `has_sbom`, `checkout_available` |
|
| each repo | `repo_slug`, `last_sbom_at`, `sbom_age_days`, `has_sbom`, `checkout_available`, optional immutable `source_ref` |
|
||||||
|
|
||||||
Ranking (never-scanned first, then oldest `last_sbom_at`) belongs to
|
Ranking (never-scanned first, then oldest `last_sbom_at`) belongs to
|
||||||
sbom-nexus. The adapter truncates to `limit` so an over-long response can never
|
sbom-nexus. The adapter truncates to `limit` so an over-long response can never
|
||||||
|
|
@ -79,6 +79,10 @@ The progress event names the repos selected, updated, and skipped with a
|
||||||
reason (`no-checkout`, `no-manifest`, `ingest-error`), plus the fleet counters
|
reason (`no-checkout`, `no-manifest`, `ingest-error`), plus the fleet counters
|
||||||
so `never_count` can be watched declining day over day.
|
so `never_count` can be watched declining day over day.
|
||||||
|
|
||||||
|
Controlled-source outcomes additionally include `source-unavailable` and
|
||||||
|
`source-rejected`. They remain terminal for the selected repository and never
|
||||||
|
cause the same fire to select a replacement target.
|
||||||
|
|
||||||
## Bounded side-effect (ACTIVITY-WP-0030-T02 / ACTIVITY-WP-0033)
|
## Bounded side-effect (ACTIVITY-WP-0030-T02 / ACTIVITY-WP-0033)
|
||||||
|
|
||||||
`params.apply: true` declares the write, but context resolution remains
|
`params.apply: true` declares the write, but context resolution remains
|
||||||
|
|
@ -98,6 +102,11 @@ and Activity Core's next heartbeat. The resulting `updated` and `skipped`
|
||||||
arrays are part of the run context and progress report. There is no task or
|
arrays are part of the run context and progress report. There is no task or
|
||||||
issue emission.
|
issue emission.
|
||||||
|
|
||||||
|
When Nexus returns a `forgejo-archive-v1` `source_ref`, the workflow freezes it
|
||||||
|
with the original target set and sends that exact object in the ingest body.
|
||||||
|
Retries reuse the same full commit SHA even if the repository default branch
|
||||||
|
has advanced.
|
||||||
|
|
||||||
## Enable checklist
|
## Enable checklist
|
||||||
|
|
||||||
1. CUST-WP-0062-T02/T03 done: `sbom-nexus` stood up, `GET /sbom/catch-up`
|
1. CUST-WP-0062-T02/T03 done: `sbom-nexus` stood up, `GET /sbom/catch-up`
|
||||||
|
|
|
||||||
|
|
@ -19,6 +19,11 @@ first), plus fleet counts for the evidence report.
|
||||||
"sbom_age_days": int,
|
"sbom_age_days": int,
|
||||||
"has_sbom": bool,
|
"has_sbom": bool,
|
||||||
"checkout_available": bool | None,
|
"checkout_available": bool | None,
|
||||||
|
"source_ref": {
|
||||||
|
"kind": "forgejo-archive-v1",
|
||||||
|
"repository": str,
|
||||||
|
"revision": str,
|
||||||
|
} | None,
|
||||||
},
|
},
|
||||||
...
|
...
|
||||||
],
|
],
|
||||||
|
|
@ -43,6 +48,7 @@ Config: SBOM_NEXUS_URL env var (default: http://127.0.0.1:8010).
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
import os
|
||||||
|
import re
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from urllib.parse import quote
|
from urllib.parse import quote
|
||||||
|
|
@ -50,7 +56,10 @@ from uuid import NAMESPACE_URL, uuid5
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
from activity_core.context_resolvers.base import CONTEXT_RESOLVER_REGISTRY, ContextResolver
|
from activity_core.context_resolvers.base import (
|
||||||
|
CONTEXT_RESOLVER_REGISTRY,
|
||||||
|
ContextResolver,
|
||||||
|
)
|
||||||
|
|
||||||
_DEFAULT_SBOM_NEXUS_URL = "http://127.0.0.1:8010"
|
_DEFAULT_SBOM_NEXUS_URL = "http://127.0.0.1:8010"
|
||||||
_TIMEOUT_SECONDS = 15.0
|
_TIMEOUT_SECONDS = 15.0
|
||||||
|
|
@ -62,6 +71,13 @@ _MAX_CATCH_UP_LIMIT = 25
|
||||||
# Mirrors state_hub._NEVER_SCANNED_AGE_DAYS: a repo that was never scanned
|
# Mirrors state_hub._NEVER_SCANNED_AGE_DAYS: a repo that was never scanned
|
||||||
# sorts ahead of every real age without needing a null-aware comparison.
|
# sorts ahead of every real age without needing a null-aware comparison.
|
||||||
_NEVER_SCANNED_AGE_DAYS = 9999
|
_NEVER_SCANNED_AGE_DAYS = 9999
|
||||||
|
_TERMINAL_SKIP_REASONS = {
|
||||||
|
"no-checkout",
|
||||||
|
"no-manifest",
|
||||||
|
"ingest-error",
|
||||||
|
"source-unavailable",
|
||||||
|
"source-rejected",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _base_url() -> str:
|
def _base_url() -> str:
|
||||||
|
|
@ -135,12 +151,24 @@ def _normalise_entry(raw: Any) -> dict[str, Any] | None:
|
||||||
if not isinstance(checkout_available, bool):
|
if not isinstance(checkout_available, bool):
|
||||||
checkout_available = None
|
checkout_available = None
|
||||||
|
|
||||||
|
source_ref = raw.get("source_ref")
|
||||||
|
if not isinstance(source_ref, dict) or (
|
||||||
|
source_ref.get("kind") != "forgejo-archive-v1"
|
||||||
|
or not isinstance(source_ref.get("repository"), str)
|
||||||
|
or not isinstance(source_ref.get("revision"), str)
|
||||||
|
or re.fullmatch(r"[0-9a-f]{40}", source_ref["revision"]) is None
|
||||||
|
):
|
||||||
|
source_ref = None
|
||||||
|
else:
|
||||||
|
source_ref = dict(source_ref)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"repo_slug": repo_slug,
|
"repo_slug": repo_slug,
|
||||||
"last_sbom_at": last_sbom_at,
|
"last_sbom_at": last_sbom_at,
|
||||||
"sbom_age_days": max(0, age_days),
|
"sbom_age_days": max(0, age_days),
|
||||||
"has_sbom": has_sbom,
|
"has_sbom": has_sbom,
|
||||||
"checkout_available": checkout_available,
|
"checkout_available": checkout_available,
|
||||||
|
"source_ref": source_ref,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -206,25 +234,27 @@ def _skip(
|
||||||
if (
|
if (
|
||||||
not isinstance(raw, dict)
|
not isinstance(raw, dict)
|
||||||
or raw.get("status") != "skipped"
|
or raw.get("status") != "skipped"
|
||||||
or raw.get("reason") not in {"no-checkout", "no-manifest", "ingest-error"}
|
or raw.get("reason") not in _TERMINAL_SKIP_REASONS
|
||||||
):
|
):
|
||||||
raise RuntimeError(f"sbom-nexus skip returned an invalid outcome for {repo_slug}")
|
raise RuntimeError(f"sbom-nexus skip returned an invalid outcome for {repo_slug}")
|
||||||
return raw
|
return raw
|
||||||
|
|
||||||
|
|
||||||
def _ingest(repo_slug: str, *, operation_id: str) -> dict[str, Any]:
|
def _ingest(
|
||||||
|
repo_slug: str,
|
||||||
|
*,
|
||||||
|
operation_id: str,
|
||||||
|
source_ref: dict[str, Any] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
raw = _post_json(
|
raw = _post_json(
|
||||||
f"/sbom/{quote(repo_slug, safe='')}/ingest",
|
f"/sbom/{quote(repo_slug, safe='')}/ingest",
|
||||||
|
{"source_ref": source_ref} if source_ref else None,
|
||||||
idempotency_key=_operation_key(operation_id, repo_slug),
|
idempotency_key=_operation_key(operation_id, repo_slug),
|
||||||
)
|
)
|
||||||
valid = isinstance(raw, dict) and raw.get("status") in {"ingested", "skipped"}
|
valid = isinstance(raw, dict) and raw.get("status") in {"ingested", "skipped"}
|
||||||
if not valid:
|
if not valid:
|
||||||
raise RuntimeError(f"sbom-nexus ingest returned an invalid outcome for {repo_slug}")
|
raise RuntimeError(f"sbom-nexus ingest returned an invalid outcome for {repo_slug}")
|
||||||
if raw.get("status") == "skipped" and raw.get("reason") not in {
|
if raw.get("status") == "skipped" and raw.get("reason") not in _TERMINAL_SKIP_REASONS:
|
||||||
"no-checkout",
|
|
||||||
"no-manifest",
|
|
||||||
"ingest-error",
|
|
||||||
}:
|
|
||||||
raise RuntimeError(f"sbom-nexus ingest returned an invalid skip for {repo_slug}")
|
raise RuntimeError(f"sbom-nexus ingest returned an invalid skip for {repo_slug}")
|
||||||
return raw
|
return raw
|
||||||
|
|
||||||
|
|
@ -282,14 +312,19 @@ def apply_bounded_ingest(
|
||||||
continue
|
continue
|
||||||
if on_progress:
|
if on_progress:
|
||||||
on_progress(list(outcomes_by_slug.values()))
|
on_progress(list(outcomes_by_slug.values()))
|
||||||
if repo.get("checkout_available") is False:
|
source_ref = repo.get("source_ref")
|
||||||
|
if repo.get("checkout_available") is False and not source_ref:
|
||||||
outcome = _skip(
|
outcome = _skip(
|
||||||
repo_slug,
|
repo_slug,
|
||||||
"no-checkout",
|
"no-checkout",
|
||||||
operation_id=operation_id,
|
operation_id=operation_id,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
outcome = _ingest(repo_slug, operation_id=operation_id)
|
outcome = _ingest(
|
||||||
|
repo_slug,
|
||||||
|
operation_id=operation_id,
|
||||||
|
source_ref=source_ref if isinstance(source_ref, dict) else None,
|
||||||
|
)
|
||||||
outcomes_by_slug[repo_slug] = _compact_outcome(repo_slug, outcome)
|
outcomes_by_slug[repo_slug] = _compact_outcome(repo_slug, outcome)
|
||||||
if on_progress:
|
if on_progress:
|
||||||
on_progress(list(outcomes_by_slug.values()))
|
on_progress(list(outcomes_by_slug.values()))
|
||||||
|
|
|
||||||
|
|
@ -93,6 +93,12 @@ def test_catch_up_returns_ranked_targets_and_fleet_counts(monkeypatch) -> None:
|
||||||
"sbom_age_days": 9999,
|
"sbom_age_days": 9999,
|
||||||
"has_sbom": False,
|
"has_sbom": False,
|
||||||
"checkout_available": True,
|
"checkout_available": True,
|
||||||
|
"source_ref": {
|
||||||
|
"kind": "forgejo-archive-v1",
|
||||||
|
"repository": "coulomb/never-scanned",
|
||||||
|
"revision": "a" * 40,
|
||||||
|
"observed_ref": "refs/heads/main",
|
||||||
|
},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"repo_slug": "activity-core",
|
"repo_slug": "activity-core",
|
||||||
|
|
@ -131,6 +137,7 @@ def test_catch_up_returns_ranked_targets_and_fleet_counts(monkeypatch) -> None:
|
||||||
assert result["never_count"] == 93
|
assert result["never_count"] == 93
|
||||||
assert result["total_count"] == 111
|
assert result["total_count"] == 111
|
||||||
assert result["repos"][0]["has_sbom"] is False
|
assert result["repos"][0]["has_sbom"] is False
|
||||||
|
assert result["repos"][0]["source_ref"]["revision"] == "a" * 40
|
||||||
assert result["repos"][2]["checkout_available"] is False
|
assert result["repos"][2]["checkout_available"] is False
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -266,6 +273,43 @@ def test_apply_processes_only_fixed_targets_and_records_terminal_outcomes(monkey
|
||||||
assert first_key != client.posts[1][2]["Idempotency-Key"]
|
assert first_key != client.posts[1][2]["Idempotency-Key"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_posts_frozen_source_ref_and_accepts_source_outcome(monkeypatch) -> None:
|
||||||
|
source_ref = {
|
||||||
|
"kind": "forgejo-archive-v1",
|
||||||
|
"repository": "coulomb/scan-me",
|
||||||
|
"revision": "b" * 40,
|
||||||
|
"observed_ref": "refs/heads/main",
|
||||||
|
}
|
||||||
|
client = _install(monkeypatch, _payload([]))
|
||||||
|
client.post_results = {
|
||||||
|
"/sbom/scan-me/ingest": [
|
||||||
|
DummyResponse(
|
||||||
|
{
|
||||||
|
"repo_slug": "scan-me",
|
||||||
|
"status": "skipped",
|
||||||
|
"reason": "source-unavailable",
|
||||||
|
"snapshot_id": "source-skip-1",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
result = apply_bounded_ingest(
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"repo_slug": "scan-me",
|
||||||
|
"checkout_available": False,
|
||||||
|
"source_ref": source_ref,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
operation_id="run-controlled-source",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["attempted_count"] == 1
|
||||||
|
assert result["skipped"][0]["reason"] == "source-unavailable"
|
||||||
|
assert client.posts[0][1] == {"source_ref": source_ref}
|
||||||
|
|
||||||
|
|
||||||
def test_ambiguous_ingest_failure_does_not_create_synthetic_skip(monkeypatch) -> None:
|
def test_ambiguous_ingest_failure_does_not_create_synthetic_skip(monkeypatch) -> None:
|
||||||
client = _install(
|
client = _install(
|
||||||
monkeypatch,
|
monkeypatch,
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,51 @@
|
||||||
|
---
|
||||||
|
id: ACTIVITY-WP-0034
|
||||||
|
type: workplan
|
||||||
|
title: "Freeze controlled SBOM source references across retries"
|
||||||
|
domain: infotech
|
||||||
|
repo: activity-core
|
||||||
|
status: active
|
||||||
|
owner: codex
|
||||||
|
topic_slug: infotech
|
||||||
|
created: "2026-08-22"
|
||||||
|
updated: "2026-08-22"
|
||||||
|
quality_dor: DoR-Ok
|
||||||
|
quality_dor_at: "2026-08-22"
|
||||||
|
quality_dor_by: codex
|
||||||
|
quality_dor_note: "CUST-WP-0064 selected a reviewable full-SHA source_ref contract; Nexus owns fetch/scan, Activity Core retains the existing bounded target and stable operation identity, and rollout waits on the new Nexus/package digest."
|
||||||
|
parent_workplan: CUST-WP-0064
|
||||||
|
related:
|
||||||
|
- SBOM-WP-0003
|
||||||
|
- ACTIVITY-WP-0033
|
||||||
|
---
|
||||||
|
|
||||||
|
# Freeze controlled SBOM source references across retries
|
||||||
|
|
||||||
|
## Carry the immutable source reference
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: ACTIVITY-WP-0034-T01
|
||||||
|
status: done
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Validate and retain Nexus's `forgejo-archive-v1` source reference in the
|
||||||
|
ranked selection, send it on ingest, reuse it with the existing stable
|
||||||
|
operation key, and report additive source failure outcomes without selecting a
|
||||||
|
replacement target.
|
||||||
|
|
||||||
|
Completed in the resolver/apply adapter and daily definition with focused
|
||||||
|
tests for normalization, POST payload, retry identity, and terminal reasons.
|
||||||
|
|
||||||
|
## Promote after the Nexus dark canary
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: ACTIVITY-WP-0034-T02
|
||||||
|
status: wait
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
After `SBOM-WP-0003` and `RAPP-SBOM-NEXUS-WP-0002` migrate and pass the
|
||||||
|
attended one-repository canary, project this revision to Railiance, sync the
|
||||||
|
existing schedule without widening its limit, and capture the first normal
|
||||||
|
scheduled controlled-source result.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue