feat(sbom): draft bounded daily SBOM catch-up (ACTIVITY-WP-0030-T01)
All checks were successful
CI Smoke / host-smoke (push) Successful in 1s
CI Smoke / container-smoke (push) Successful in 2s
Build and Publish Container Image / build-and-push (push) Successful in 23s

Add the sbom-nexus catch_up resolver contract and the daily replacement for
weekly-sbom-staleness, both disabled until CUST-WP-0062-T03 lands.

The weekly check used `for_each: context.repos.repos` and emitted one task per
stale repo — 75 tasks on 2026-08-17 against 111/111 stale repos. The
replacement asks sbom-nexus for only the N oldest-stale repos in one ranked
call and carries no rule block at all, so tasks_spawned is 0 by construction.

- context_resolvers/sbom_nexus.py: source type `sbom-nexus`, query `catch_up`,
  GET /sbom/catch-up?limit=N. Read-only; ingest is T02. Limit bounded 1..25 and
  the response truncated to it so an over-long reply cannot widen T02's
  side-effect.
- activity-definitions/daily-sbom-catchup.md: weekdays 09:15 Berlin, enabled:
  false, deterministic sbom_catchup progress sink.
- rules/executor.py: the deterministic report builder only special-cased
  context.repos, which would have emitted a contentless progress event for this
  definition. _sbom_catchup_report names the selected repos and reads
  updated/skipped from context when T02 populates them.
- 17 tests against a test double; no live nexus exists yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-21 08:45:40 +02:00
parent a446de1c45
commit e64af4102d
6 changed files with 667 additions and 2 deletions

View file

@ -0,0 +1,100 @@
---
id: daily-sbom-catchup
name: Daily SBOM Catch-up
enabled: false
owner: custodian-agent
governance: custodian
status: paused
trigger:
type: cron
cron_expression: "15 9 * * 1-5" # weekdays 09:15, after the daily triage window
timezone: Europe/Berlin
misfire_policy: skip
context_sources:
- type: sbom-nexus
query: catch_up
required: true
params:
limit: 3 # catch_up_limit — operator knob, not a nexus constant
bind_to: context.catchup
# One ranked call returns only the N repos that have lacked a current SBOM the
# longest (never-scanned first) plus fleet counts. There is deliberately no
# `for_each` over the stale set: the 2026-08-17 fire emitted 75 tasks that way.
---
# Daily SBOM Catch-up
> **Disabled until CUST-WP-0062-T03 lands.** The `sbom-nexus` catch-up API does
> not exist yet; the resolver contract is exercised against a test double in
> `tests/test_sbom_nexus_context_resolver.py`. Do not enable this schedule
> against the old per-repo `/sbom/{slug}` walk (CUST-WP-0062-T05).
Replaces `weekly-sbom-staleness` / `flag-stale-sbom` (ACTIVITY-WP-0030).
The weekly check reported the backlog — 111 / 111 repos stale on 2026-08-18,
93 never scanned — and spawned one task per stale repo, so it could never
close it. This definition **updates N instead of reporting N**.
Runs weekdays at 09:15 Berlin, after the daily triage window.
## Contract
Context source `sbom-nexus / catch_up` (`context_resolvers/sbom_nexus.py`):
| Direction | Shape |
| --- | --- |
| input | `{"limit": 3}` |
| output | `{repos, selected_count, stale_count, never_count, total_count, limit}` |
| each repo | `repo_slug`, `last_sbom_at`, `sbom_age_days`, `has_sbom`, `checkout_available` |
Ranking (never-scanned first, then oldest `last_sbom_at`) belongs to
sbom-nexus. The adapter truncates to `limit` so an over-long response can never
widen the bounded side-effect below.
## Task emission
None. This definition carries **no `rule` block** — that is the point of the
replacement. `tasks_spawned` must stay 0 on every fire, and no Forgejo issues
are emitted (ACTIVITY-WP-0022).
## Evidence
```instruction
id: daily-sbom-catchup-report
trusted_fields: []
model: deterministic
temperature: 0
max_tokens: 1
prompt: |
Deterministic SBOM catch-up report from context.catchup (no LLM).
output_schema: ""
review_required: false
report_sinks:
- type: state-hub-progress
event_type: sbom_catchup
author: activity-core
topic_id: cee7bedf-2b48-46ef-8601-006474f2ad7a
```
The progress event names the repos selected, updated, and skipped with a
reason (`no-checkout`, `no-manifest`, `ingest-error`), plus the fleet counters
so `never_count` can be watched declining day over day.
## Not yet implemented (ACTIVITY-WP-0030-T02)
The bounded ingest side-effect — calling sbom-nexus ingest for each of the N
selected repos using the registered checkout — waits on CUST-WP-0062-T03. Until
then this definition resolves and reports only. When T02 lands, a recorded skip
must advance the queue position so an impossible repo does not permanently head
it.
## Enable checklist
1. CUST-WP-0062-T02/T03 done: `sbom-nexus` stood up, `GET /sbom/catch-up`
returns oldest-N in one call.
2. `SBOM_NEXUS_URL` reachable from the railiance01 worker.
3. ACTIVITY-WP-0030-T02 ingest side-effect implemented and dry-run proven.
4. `weekly-sbom-staleness` confirmed off in source **and** production
(ACTIVITY-WP-0030-T03).
5. Project into `k8s/railiance/20-runtime.yaml`, `enabled: true`, sync
schedules, capture evidence via `./scripts/prod_automation_status.sh`
(ACTIVITY-WP-0030-T04).

View file

@ -4,6 +4,7 @@ from activity_core.context_resolvers import ( # noqa: F401
kaizen,
ops_inventory,
repo_scoping,
sbom_nexus,
state_hub,
reuse_surface,
)

View file

@ -0,0 +1,169 @@
"""sbom-nexus context adapter (ACTIVITY-WP-0030-T01).
Registered as source type ``sbom-nexus``.
Supported queries:
- catch_up: GET {SBOM_NEXUS_URL}/sbom/catch-up?limit=N
Contract (CUST-WP-0062-T03). One ranked call replaces the fleet-wide
``for_each`` walk that produced the 2026-08-17 task flood: the nexus returns
only the N repos that have lacked a current SBOM the longest (never-scanned
first), plus fleet counts for the evidence report.
input : {"limit": 3}
output: {
"repos": [
{
"repo_slug": str,
"last_sbom_at": str | None,
"sbom_age_days": int,
"has_sbom": bool,
"checkout_available": bool | None,
},
...
],
"stale_count": int,
"never_count": int,
"total_count": int,
"limit": int,
}
Ordering is the nexus's responsibility (never-scanned first, then oldest
``last_sbom_at``); this adapter validates the shape and normalises the entries
so the deterministic report can render them without comprehensions.
Until CUST-WP-0062-T03 lands there is no live endpoint the query is exercised
against a test double (``tests/test_sbom_nexus_context_resolver.py``) and the
daily definition stays ``enabled: false``.
This adapter is read-only. Ingest of the selected repos is a declared bounded
side-effect owned by ACTIVITY-WP-0030-T02 and is deliberately not implemented
here.
Config: SBOM_NEXUS_URL env var (default: http://127.0.0.1:8010).
"""
from __future__ import annotations
import os
from typing import Any
import httpx
from activity_core.context_resolvers.base import CONTEXT_RESOLVER_REGISTRY, ContextResolver
_DEFAULT_SBOM_NEXUS_URL = "http://127.0.0.1:8010"
_TIMEOUT_SECONDS = 15.0
_DEFAULT_CATCH_UP_LIMIT = 3
_MIN_CATCH_UP_LIMIT = 1
_MAX_CATCH_UP_LIMIT = 25
# Mirrors state_hub._NEVER_SCANNED_AGE_DAYS: a repo that was never scanned
# sorts ahead of every real age without needing a null-aware comparison.
_NEVER_SCANNED_AGE_DAYS = 9999
def _base_url() -> str:
return os.getenv("SBOM_NEXUS_URL", _DEFAULT_SBOM_NEXUS_URL).rstrip("/")
def _bounded_limit(value: Any) -> int:
try:
parsed = int(value)
except (TypeError, ValueError):
return _DEFAULT_CATCH_UP_LIMIT
return max(_MIN_CATCH_UP_LIMIT, min(_MAX_CATCH_UP_LIMIT, parsed))
def _fetch_json(path: str, params: dict[str, Any] | None = None) -> Any:
url = f"{_base_url()}{path}"
with httpx.Client(timeout=_TIMEOUT_SECONDS) as client:
response = client.get(url, params=params)
response.raise_for_status()
return response.json()
def _int_or(value: Any, default: int) -> int:
try:
return int(value)
except (TypeError, ValueError):
return default
def _normalise_entry(raw: Any) -> dict[str, Any] | None:
if not isinstance(raw, dict):
return None
repo_slug = raw.get("repo_slug")
if not isinstance(repo_slug, str) or not repo_slug:
return None
last_sbom_at = raw.get("last_sbom_at")
if not isinstance(last_sbom_at, str) or not last_sbom_at:
last_sbom_at = None
has_sbom = raw.get("has_sbom")
if not isinstance(has_sbom, bool):
has_sbom = last_sbom_at is not None
age_days = _int_or(
raw.get("sbom_age_days"),
0 if has_sbom else _NEVER_SCANNED_AGE_DAYS,
)
checkout_available = raw.get("checkout_available")
if not isinstance(checkout_available, bool):
checkout_available = None
return {
"repo_slug": repo_slug,
"last_sbom_at": last_sbom_at,
"sbom_age_days": max(0, age_days),
"has_sbom": has_sbom,
"checkout_available": checkout_available,
}
def _catch_up(params: dict[str, Any]) -> dict[str, Any]:
limit = _bounded_limit(params.get("limit", _DEFAULT_CATCH_UP_LIMIT))
payload = _fetch_json("/sbom/catch-up", {"limit": limit})
if not isinstance(payload, dict):
raise RuntimeError("sbom-nexus catch_up returned a non-object response")
raw_repos = payload.get("repos")
if not isinstance(raw_repos, list):
raise RuntimeError("sbom-nexus catch_up response missing required key: repos")
repos: list[dict[str, Any]] = []
for raw in raw_repos:
entry = _normalise_entry(raw)
if entry is not None:
repos.append(entry)
# The nexus owns ranking, but the definition promises "at most N": never let
# an over-long response widen the bounded side-effect in T02.
repos = repos[:limit]
total_count = _int_or(payload.get("total_count"), len(repos))
never_count = _int_or(payload.get("never_count"), 0)
stale_count = _int_or(payload.get("stale_count"), len(repos))
return {
"repos": repos,
"selected_count": len(repos),
"stale_count": stale_count,
"never_count": never_count,
"total_count": total_count,
"limit": limit,
}
class SbomNexusContextResolver(ContextResolver):
"""Fetches the ranked SBOM catch-up queue from sbom-nexus."""
def resolve(self, query: str, event: Any, params: dict[str, Any]) -> Any:
if query == "catch_up":
return _catch_up(params)
return {}
CONTEXT_RESOLVER_REGISTRY["sbom-nexus"] = SbomNexusContextResolver

View file

@ -692,6 +692,10 @@ def _execution_failure_report(instr: Any, error: str) -> dict[str, Any] | None:
def _deterministic_context_report(instr: Any, context: dict) -> InstructionResult:
"""Build a report from resolved context without calling an LLM."""
catchup = context.get("catchup") if isinstance(context, dict) else None
if isinstance(catchup, dict) and isinstance(catchup.get("repos"), list):
return _sbom_catchup_report(instr, catchup)
repos = context.get("repos") if isinstance(context, dict) else None
if isinstance(repos, dict):
repo_list = repos.get("repos") if isinstance(repos.get("repos"), list) else []
@ -746,6 +750,59 @@ def _deterministic_context_report(instr: Any, context: dict) -> InstructionResul
)
def _sbom_catchup_report(instr: Any, catchup: dict) -> InstructionResult:
"""Deterministic SBOM catch-up evidence (ACTIVITY-WP-0030).
Names the repos selected by the ranked sbom-nexus call, plus fleet counters
so ``never_count`` can be watched declining day over day. ``updated`` and
``skipped`` are read from the context when the bounded ingest side-effect
(ACTIVITY-WP-0030-T02) has populated them, and stay empty until then.
"""
selected = [r for r in catchup["repos"] if isinstance(r, dict)]
updated = [r for r in catchup.get("updated", []) if isinstance(r, dict)]
skipped = [r for r in catchup.get("skipped", []) if isinstance(r, dict)]
limit = catchup.get("limit")
report: dict[str, Any] = {
"summary": (
f"SBOM catch-up: {len(selected)} selected (limit {limit}), "
f"{len(updated)} updated, {len(skipped)} skipped; "
f"{catchup.get('never_count')} never scanned of "
f"{catchup.get('total_count')} repos."
),
"status": "deterministic",
"deterministic": True,
"limit": limit,
"selected_count": len(selected),
"stale_count": catchup.get("stale_count"),
"never_count": catchup.get("never_count"),
"total_count": catchup.get("total_count"),
"selected_repos": [
{
"repo_slug": r.get("repo_slug"),
"sbom_age_days": r.get("sbom_age_days"),
"has_sbom": r.get("has_sbom"),
"last_sbom_at": r.get("last_sbom_at"),
"checkout_available": r.get("checkout_available"),
}
for r in selected
],
"updated_repos": [r.get("repo_slug") for r in updated],
"skipped_repos": [
{"repo_slug": r.get("repo_slug"), "reason": r.get("reason")}
for r in skipped
],
}
return InstructionResult(
tasks=[],
report=report,
prompt_hash=None,
model=getattr(instr, "model", None),
output_validated=True,
review_required=bool(getattr(instr, "review_required", False)),
condition_matched=getattr(instr, "condition", "") or None,
)
def _validate_output(
raw_output: Any,
instr: Any,

View file

@ -0,0 +1,318 @@
"""sbom-nexus catch_up resolver (ACTIVITY-WP-0030-T01).
CUST-WP-0062-T03 has not landed, so the contract is exercised against a test
double rather than a live nexus.
"""
from __future__ import annotations
from typing import Any
import httpx
import pytest
from activity_core.context_resolvers.base import CONTEXT_RESOLVER_REGISTRY
from activity_core.context_resolvers.sbom_nexus import SbomNexusContextResolver
class DummyResponse:
def __init__(self, payload: Any, status_code: int = 200) -> None:
self._payload = payload
self.status_code = status_code
self.text = str(payload)
def raise_for_status(self) -> None:
if self.status_code >= 400:
raise httpx.HTTPStatusError(
f"HTTP {self.status_code}",
request=httpx.Request("GET", "http://sbom-nexus.test"),
response=None, # type: ignore[arg-type]
)
def json(self) -> Any:
return self._payload
class DummyClient:
"""Records the single ranked call the contract allows."""
def __init__(self, payload: Any, status_code: int = 200) -> None:
self._payload = payload
self._status_code = status_code
self.calls: list[tuple[str, dict[str, Any] | None]] = []
def __enter__(self) -> "DummyClient":
return self
def __exit__(self, *args: Any) -> None:
return None
def get(self, url: str, params: dict[str, Any] | None = None) -> DummyResponse:
self.calls.append((url, params))
return DummyResponse(self._payload, self._status_code)
def _install(monkeypatch, payload: Any, status_code: int = 200) -> DummyClient:
client = DummyClient(payload, status_code)
monkeypatch.setattr(httpx, "Client", lambda **kwargs: client)
return client
def _payload(repos: list[dict[str, Any]], **counts: Any) -> dict[str, Any]:
base = {"stale_count": 111, "never_count": 93, "total_count": 111}
base.update(counts)
base["repos"] = repos
return base
def test_registered_as_sbom_nexus_source_type() -> None:
assert CONTEXT_RESOLVER_REGISTRY["sbom-nexus"] is SbomNexusContextResolver
def test_catch_up_returns_ranked_targets_and_fleet_counts(monkeypatch) -> None:
client = _install(
monkeypatch,
_payload(
[
{
"repo_slug": "never-scanned",
"last_sbom_at": None,
"sbom_age_days": 9999,
"has_sbom": False,
"checkout_available": True,
},
{
"repo_slug": "activity-core",
"last_sbom_at": "2026-04-26T00:00:00Z",
"sbom_age_days": 117,
"has_sbom": True,
"checkout_available": True,
},
{
"repo_slug": "no-checkout",
"last_sbom_at": "2026-05-01T00:00:00Z",
"sbom_age_days": 112,
"has_sbom": True,
"checkout_available": False,
},
]
),
)
result = SbomNexusContextResolver().resolve("catch_up", None, {"limit": 3})
# One ranked call, not a per-repo walk.
assert len(client.calls) == 1
url, params = client.calls[0]
assert url.endswith("/sbom/catch-up")
assert params == {"limit": 3}
assert [repo["repo_slug"] for repo in result["repos"]] == [
"never-scanned",
"activity-core",
"no-checkout",
]
assert result["selected_count"] == 3
assert result["limit"] == 3
assert result["stale_count"] == 111
assert result["never_count"] == 93
assert result["total_count"] == 111
assert result["repos"][0]["has_sbom"] is False
assert result["repos"][2]["checkout_available"] is False
def test_catch_up_defaults_to_three(monkeypatch) -> None:
client = _install(monkeypatch, _payload([]))
result = SbomNexusContextResolver().resolve("catch_up", None, {})
assert client.calls[0][1] == {"limit": 3}
assert result["limit"] == 3
assert result["repos"] == []
assert result["selected_count"] == 0
@pytest.mark.parametrize("requested", [0, -5, 500, "seven", None])
def test_catch_up_limit_stays_bounded(monkeypatch, requested: Any) -> None:
_install(monkeypatch, _payload([]))
result = SbomNexusContextResolver().resolve("catch_up", None, {"limit": requested})
assert 1 <= result["limit"] <= 25
def test_catch_up_truncates_an_over_long_response(monkeypatch) -> None:
"""The bounded side-effect in T02 must never exceed the declared N."""
repos = [
{"repo_slug": f"repo-{index}", "last_sbom_at": None, "has_sbom": False}
for index in range(10)
]
_install(monkeypatch, _payload(repos))
result = SbomNexusContextResolver().resolve("catch_up", None, {"limit": 2})
assert len(result["repos"]) == 2
assert result["selected_count"] == 2
def test_catch_up_normalises_partial_entries(monkeypatch) -> None:
_install(
monkeypatch,
_payload(
[
{"repo_slug": "sparse"},
{"repo_slug": ""},
"not-an-object",
{"last_sbom_at": "2026-01-01T00:00:00Z"},
]
),
)
result = SbomNexusContextResolver().resolve("catch_up", None, {"limit": 10})
assert [repo["repo_slug"] for repo in result["repos"]] == ["sparse"]
sparse = result["repos"][0]
assert sparse["has_sbom"] is False
assert sparse["last_sbom_at"] is None
assert sparse["sbom_age_days"] == 9999
assert sparse["checkout_available"] is None
def test_catch_up_rejects_a_response_without_repos(monkeypatch) -> None:
_install(monkeypatch, {"stale_count": 111})
with pytest.raises(RuntimeError, match="missing required key: repos"):
SbomNexusContextResolver().resolve("catch_up", None, {})
def test_catch_up_rejects_a_non_object_response(monkeypatch) -> None:
_install(monkeypatch, ["activity-core"])
with pytest.raises(RuntimeError, match="non-object response"):
SbomNexusContextResolver().resolve("catch_up", None, {})
def test_unknown_query_returns_empty(monkeypatch) -> None:
_install(monkeypatch, _payload([]))
assert SbomNexusContextResolver().resolve("ingest", None, {}) == {}
def test_base_url_is_configurable(monkeypatch) -> None:
client = _install(monkeypatch, _payload([]))
monkeypatch.setenv("SBOM_NEXUS_URL", "http://sbom-nexus.test:9000/")
SbomNexusContextResolver().resolve("catch_up", None, {})
assert client.calls[0][0] == "http://sbom-nexus.test:9000/sbom/catch-up"
def _instruction():
from activity_core.models import InstructionDef
return InstructionDef.model_validate(
{
"id": "daily-sbom-catchup-report",
"trusted_fields": [],
"model": "deterministic",
"temperature": 0,
"max_tokens": 1,
"prompt": "Deterministic SBOM catch-up report.",
"output_schema": "",
"review_required": False,
"report_sinks": [
{
"type": "state-hub-progress",
"event_type": "sbom_catchup",
"author": "activity-core",
}
],
}
)
def test_deterministic_report_names_selected_repos() -> None:
from activity_core.rules.executor import _deterministic_context_report
context = {
"catchup": {
"repos": [
{
"repo_slug": "never-scanned",
"last_sbom_at": None,
"sbom_age_days": 9999,
"has_sbom": False,
"checkout_available": True,
},
{
"repo_slug": "no-checkout",
"last_sbom_at": "2026-05-01T00:00:00Z",
"sbom_age_days": 112,
"has_sbom": True,
"checkout_available": False,
},
],
"selected_count": 2,
"stale_count": 111,
"never_count": 93,
"total_count": 111,
"limit": 3,
}
}
result = _deterministic_context_report(_instruction(), context)
assert result.tasks == []
report = result.report
assert report["deterministic"] is True
assert report["never_count"] == 93
assert [r["repo_slug"] for r in report["selected_repos"]] == [
"never-scanned",
"no-checkout",
]
assert "93 never scanned of 111 repos" in report["summary"]
assert report["updated_repos"] == []
assert report["skipped_repos"] == []
def test_deterministic_report_carries_t02_ingest_outcomes() -> None:
"""Forward-compatible with the bounded ingest side-effect (T02)."""
from activity_core.rules.executor import _deterministic_context_report
context = {
"catchup": {
"repos": [{"repo_slug": "a"}, {"repo_slug": "b"}],
"updated": [{"repo_slug": "a"}],
"skipped": [{"repo_slug": "b", "reason": "no-checkout"}],
"stale_count": 110,
"never_count": 92,
"total_count": 111,
"limit": 3,
}
}
report = _deterministic_context_report(_instruction(), context).report
assert report["updated_repos"] == ["a"]
assert report["skipped_repos"] == [{"repo_slug": "b", "reason": "no-checkout"}]
assert "1 updated, 1 skipped" in report["summary"]
def test_daily_definition_is_bounded_and_disabled() -> None:
from pathlib import Path
from activity_core.definition_parser import parse_file
definition = parse_file(Path("activity-definitions/daily-sbom-catchup.md"))
assert definition.enabled is False
# No rule block: the weekly flood came from `for_each` over every stale repo.
assert definition.rules == []
source = definition.context_sources[0]
assert source["type"] == "sbom-nexus"
assert source["query"] == "catch_up"
assert source["params"]["limit"] == 3
assert source["bind_to"] == "context.catchup"
instruction = definition.instructions[0]
assert instruction["model"] == "deterministic"
assert instruction["report_sinks"][0]["event_type"] == "sbom_catchup"

View file

@ -8,7 +8,7 @@ status: active
owner: grok
topic_slug: infotech
created: "2026-08-18"
updated: "2026-08-20"
updated: "2026-08-21"
parent_workplan: CUST-WP-0062
related:
- CUST-WP-0062
@ -53,7 +53,7 @@ each. The replacement must ask sbom-nexus for **only N targets** and then
```task
id: ACTIVITY-WP-0030-T01
status: wait
status: done
priority: high
state_hub_task_id: "b216976e-5402-4e40-b282-78c5d021df72"
```
@ -72,6 +72,26 @@ lands):
- each repo: `repo_slug`, `last_sbom_at`, `sbom_age_days`, `has_sbom`,
`checkout_available` if known
Done 2026-08-21 against a test double — CUST-WP-0062-T03 has not landed, so
there is no live nexus yet and the definition stays `enabled: false`:
- `activity-definitions/daily-sbom-catchup.md` — weekdays 09:15 Berlin, one
`sbom-nexus / catch_up` source bound to `context.catchup`, **no rule block**
(`tasks_spawned` stays 0 by construction), deterministic `sbom_catchup`
progress sink.
- `src/activity_core/context_resolvers/sbom_nexus.py` — source type
`sbom-nexus`, query `catch_up`, `GET /sbom/catch-up?limit=N` against
`SBOM_NEXUS_URL`. Read-only; ingest is T02. Limit is bounded 1..25 and the
response is truncated to it so an over-long reply cannot widen T02's
side-effect.
- `_sbom_catchup_report` in `rules/executor.py` — the existing deterministic
builder only special-cased `context.repos`, which would have emitted a
contentless progress event. The new branch names selected repos and reads
`updated` / `skipped` from context when T02 populates them.
- `tests/test_sbom_nexus_context_resolver.py` — 17 tests: contract shape,
default N=3, bounding, truncation, partial-entry normalisation, malformed
responses, progress content, and definition boundedness.
### Implement ingest side-effect for N targets
```task