diff --git a/docs/evidence/SBOM-WP-0002-T01-dark-deployment-preflight-2026-08-22.md b/docs/evidence/SBOM-WP-0002-T01-dark-deployment-preflight-2026-08-22.md index 5ed360b..538cd7a 100644 --- a/docs/evidence/SBOM-WP-0002-T01-dark-deployment-preflight-2026-08-22.md +++ b/docs/evidence/SBOM-WP-0002-T01-dark-deployment-preflight-2026-08-22.md @@ -72,3 +72,26 @@ written to a repository. State Hub handoffs: `T01` remains in progress until those gates are satisfied, the dark workload is healthy on PostgreSQL, and a restore is verified against the deployed topology. + +## Post-preflight status + +Later on 2026-08-22, the Forgejo package gate was completed and the package was +registered. A namespace-first apply correction passed five package tests and a +server-side dry-run, then was published as `rapp-sbom-nexus` commit `dd110d1`. + +The database owner accepted the declaration under `RAPP-POSTGRES-WP-0005` +(State Hub workplan `d6d36add-27d9-5b5d-b6fe-ce80eb7b6dab`). The distinct +`platform-pg-2` cell and bootstrap secret were created and the cell reached 1/1 +Ready. Acceptance remains blocked because: + +- Warden still requires the attended founder act for the OpenBao connection, + dynamic runtime/migration roles, exact two-path policy, and renewable parent + token; +- continuous archiving is `False` and the first base backup is + `walArchivingFailing`; the value-safe pod log reports S3 `HeadBucket` 403 for + the governed `platform-pg-2` prefix. + +The backup diagnosis and request for a successful base backup plus scratch +restore were sent to `rapp-postgres` as message +`31b72f4b-fcf8-4454-82b7-2dbf325c573e`. The SBOM Nexus workload remains +undeployed and no caller ingress is enabled. diff --git a/docs/evidence/SBOM-WP-0002-T02-repository-projection-rehearsal-2026-08-22.md b/docs/evidence/SBOM-WP-0002-T02-repository-projection-rehearsal-2026-08-22.md new file mode 100644 index 0000000..abaa871 --- /dev/null +++ b/docs/evidence/SBOM-WP-0002-T02-repository-projection-rehearsal-2026-08-22.md @@ -0,0 +1,48 @@ +# SBOM-WP-0002-T02 repository projection rehearsal — 2026-08-22 + +## Result + +The projection-only synchronization path is implemented and rehearsed against +an isolated SBOM Nexus database. It does not scan repositories, invoke an +ingest route, or create snapshots. + +## Source and selection + +The current Repo Manager compatibility projection was read from State Hub's +`GET /repos/` boundary using explicit host id `bnt-lap001`: + +| Measure | Result | +| --- | ---: | +| Source repositories | 120 | +| Active | 116 | +| Inactive | 4 | +| Selected checkout path | 120 | + +The dry-run used an intentionally unreachable target URL and still completed, +proving that dry-run does not contact or mutate the Nexus target. + +## Apply rehearsal + +An ephemeral SQLite-backed Nexus on `127.0.0.1:18010` received the projection: + +- 120 upserts; +- 120 exact active/path matches on read-back; +- zero missing, mismatched, or extra target repositories; +- zero SBOM ingest calls and zero snapshots. + +The bounded catch-up query with `limit=3` returned exactly three of 116 active, +never-attempted repositories, ordered deterministically by slug: +`activity-core`, `adaptive-pricing`, and `agent-harness`. The response reported +`selected_count=3`, `never_count=116`, `stale_count=116`, and +`total_count=116`. + +Production apply remains gated on the dark PostgreSQL runtime in +`SBOM-WP-0002-T01`. + +## Verification + +- `uv run ruff check src tests scripts`: pass +- `uv run pytest -q`: 15 passed, 1 conditional PostgreSQL skip +- projection dry-run: 120 would-upsert, target not contacted +- projection apply/read-back: 120/120 matched +- bounded catch-up: 3/3 selected in deterministic order diff --git a/docs/operator-guide.md b/docs/operator-guide.md index eb4275f..66becc4 100644 --- a/docs/operator-guide.md +++ b/docs/operator-guide.md @@ -52,6 +52,21 @@ operation. ## Import State Hub history +Synchronize the minimum repository identity/path projection first. This command +does not call an ingest endpoint or create an SBOM snapshot: + +```bash +uv run python scripts/sync_repository_projections.py \ + --source-url http://127.0.0.1:8000 \ + --target-url http://127.0.0.1:8010 \ + --host-id bnt-lap001 \ + --dry-run +``` + +Remove `--dry-run` only after the target is deployed. The apply path upserts all +active and inactive records, then reads them back and fails on a missing or +mismatched active/path projection. Extra target rows are reported but retained. + Run a read-only preview first: ```bash diff --git a/scripts/sync_repository_projections.py b/scripts/sync_repository_projections.py new file mode 100644 index 0000000..6e27e7e --- /dev/null +++ b/scripts/sync_repository_projections.py @@ -0,0 +1,35 @@ +#!/usr/bin/env python3 +"""Synchronize Repo Manager repository projections into SBOM Nexus.""" + +from __future__ import annotations + +import argparse +import json +import urllib.error + +from sbom_nexus.projection import sync_repository_projections + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--source-url", default="http://127.0.0.1:8000") + parser.add_argument("--target-url", default="http://127.0.0.1:8010") + parser.add_argument("--host-id") + parser.add_argument("--dry-run", action="store_true") + args = parser.parse_args() + try: + result = sync_repository_projections( + args.source_url, + args.target_url, + dry_run=args.dry_run, + host_id=args.host_id, + ) + except (urllib.error.URLError, json.JSONDecodeError) as exc: + raise SystemExit(f"Projection sync failed: {exc}") from exc + print(json.dumps(result, indent=2, sort_keys=True)) + if not result["ok"]: + raise SystemExit(1) + + +if __name__ == "__main__": + main() diff --git a/src/sbom_nexus/projection.py b/src/sbom_nexus/projection.py new file mode 100644 index 0000000..deca0dd --- /dev/null +++ b/src/sbom_nexus/projection.py @@ -0,0 +1,102 @@ +"""Synchronize the minimum repository projection used by SBOM Nexus.""" + +from __future__ import annotations + +import socket +import urllib.parse +from collections import Counter +from typing import Any + +from sbom_nexus.importer import request_json + + +def _checkout_path(repo: dict[str, Any], host_id: str | None) -> str | None: + host_paths = repo.get("host_paths") or {} + selected_host = host_id or socket.gethostname() + return host_paths.get(selected_host) or repo.get("local_path") + + +def sync_repository_projections( + source_url: str, + target_url: str, + *, + dry_run: bool, + host_id: str | None = None, +) -> dict[str, Any]: + """Upsert repository identity/path projections without scanning or ingesting.""" + repositories = request_json(source_url, "/repos/") + projections: dict[str, dict[str, Any]] = {} + statuses = Counter() + for repo in repositories: + slug = repo["slug"] + if slug in projections: + return { + "ok": False, + "dry_run": dry_run, + "source_repo_count": len(repositories), + "error": f"duplicate repository slug: {slug}", + } + active = repo.get("status", "active") == "active" + checkout_path = _checkout_path(repo, host_id) + projections[slug] = { + "slug": slug, + "active": active, + "checkout_path": checkout_path, + } + statuses["active" if active else "inactive"] += 1 + statuses["with_checkout_path" if checkout_path else "without_checkout_path"] += 1 + + if dry_run: + return { + "ok": True, + "dry_run": True, + "host_id": host_id or socket.gethostname(), + "source_repo_count": len(projections), + "counts": dict(statuses), + "would_upsert": len(projections), + "reconciliation": None, + } + + for slug, projection in sorted(projections.items()): + request_json( + target_url, + f"/repositories/{urllib.parse.quote(slug, safe='')}", + method="PUT", + body={ + "active": projection["active"], + "checkout_path": projection["checkout_path"], + }, + ) + + target_repositories = { + repo["slug"]: repo for repo in request_json(target_url, "/repositories/") + } + missing = sorted(set(projections) - set(target_repositories)) + mismatched = [] + for slug in sorted(set(projections) & set(target_repositories)): + expected = projections[slug] + actual = target_repositories[slug] + differences = { + field: {"source": expected[field], "target": actual.get(field)} + for field in ("active", "checkout_path") + if expected[field] != actual.get(field) + } + if differences: + mismatched.append({"repo_slug": slug, "differences": differences}) + + reconciliation = { + "ok": not missing and not mismatched, + "matched_repo_count": len(projections) - len(missing) - len(mismatched), + "missing_repo_slugs": missing, + "mismatched_repositories": mismatched, + "extra_target_repo_slugs": sorted(set(target_repositories) - set(projections)), + } + return { + "ok": reconciliation["ok"], + "dry_run": False, + "host_id": host_id or socket.gethostname(), + "source_repo_count": len(projections), + "counts": dict(statuses), + "upserted": len(projections), + "reconciliation": reconciliation, + } diff --git a/tests/test_projection.py b/tests/test_projection.py new file mode 100644 index 0000000..874664a --- /dev/null +++ b/tests/test_projection.py @@ -0,0 +1,82 @@ +from __future__ import annotations + +from typing import Any + +from sbom_nexus import projection + + +def test_projection_sync_is_projection_only_and_reconciles(monkeypatch) -> None: + source = [ + { + "slug": "active-repo", + "status": "active", + "local_path": "/fallback/active", + "host_paths": {"build-host": "/repos/active"}, + }, + { + "slug": "retired-repo", + "status": "retired", + "local_path": "/repos/retired", + }, + ] + target: dict[str, dict[str, Any]] = {} + calls: list[tuple[str, str, str]] = [] + + def fake_request( + base_url: str, + path: str, + *, + method: str = "GET", + body: dict[str, Any] | None = None, + ) -> Any: + calls.append((base_url, method, path)) + if base_url == "source" and path == "/repos/": + return source + if base_url == "target" and method == "PUT": + assert body is not None + slug = path.rsplit("/", 1)[1] + target[slug] = {"slug": slug, **body} + return target[slug] + if base_url == "target" and path == "/repositories/": + return list(target.values()) + raise AssertionError((base_url, method, path)) + + monkeypatch.setattr(projection, "request_json", fake_request) + result = projection.sync_repository_projections( + "source", "target", dry_run=False, host_id="build-host" + ) + + assert result["ok"] is True + assert result["source_repo_count"] == 2 + assert result["counts"] == { + "active": 1, + "with_checkout_path": 2, + "inactive": 1, + } + assert result["reconciliation"]["matched_repo_count"] == 2 + assert target["active-repo"]["checkout_path"] == "/repos/active" + assert target["retired-repo"]["active"] is False + assert all("/sbom/" not in path for _, _, path in calls) + + +def test_projection_dry_run_does_not_contact_target(monkeypatch) -> None: + def fake_request( + base_url: str, + path: str, + *, + method: str = "GET", + body: dict[str, Any] | None = None, + ) -> Any: + assert base_url == "source" + assert method == "GET" + assert path == "/repos/" + return [{"slug": "demo", "status": "active", "local_path": "/repos/demo"}] + + monkeypatch.setattr(projection, "request_json", fake_request) + result = projection.sync_repository_projections( + "source", "unreachable-target", dry_run=True, host_id="build-host" + ) + + assert result["ok"] is True + assert result["would_upsert"] == 1 + assert result["reconciliation"] is None diff --git a/workplans/SBOM-WP-0002-production-cutover.md b/workplans/SBOM-WP-0002-production-cutover.md index 61d4369..fb7753c 100644 --- a/workplans/SBOM-WP-0002-production-cutover.md +++ b/workplans/SBOM-WP-0002-production-cutover.md @@ -38,7 +38,7 @@ id: SBOM-WP-0002-T01 status: progress priority: high needs_human: true -intervention_note: "Warden requires operator OIDC/MFA for first rapp-sbom-nexus Forgejo repository creation and one-time secret provisioning for platform-pg-2/OpenBao." +intervention_note: "Warden requires an attended founder act for the first platform-pg-2 OpenBao database connection, SBOM Nexus dynamic roles/policies, and renewable External Secrets parent token. The database owner must also repair the governed backup credential/policy after a live S3 HeadBucket 403." state_hub_task_id: "95a520d4-30c2-5c87-8054-6bfe549c2686" ``` @@ -47,12 +47,15 @@ deploy the API without callers, and capture health plus backup/restore evidence. Image publication, package rendering, family validation, and server-side dry-run are complete; see `docs/evidence/SBOM-WP-0002-T01-dark-deployment-preflight-2026-08-22.md`. +The overflow cell is now 1/1 Ready, but continuous archiving and its first base +backup fail closed on an S3 `HeadBucket` 403. Database-owner work is tracked by +`RAPP-POSTGRES-WP-0005`; no caller or Nexus runtime has been enabled. ## Synchronize repository projections ```task id: SBOM-WP-0002-T02 -status: todo +status: progress priority: high state_hub_task_id: "22cbb75f-d82f-5b47-9fef-27bde3b410d5" ``` @@ -60,6 +63,13 @@ state_hub_task_id: "22cbb75f-d82f-5b47-9fef-27bde3b410d5" Populate active repository identity and host checkout paths from Repo Manager. Verify fleet totals and catch-up ordering without performing ingest. +The projection-only synchronizer and reconciliation contract are implemented. +Its dry-run never contacts the Nexus target and no code path calls an SBOM +ingest route. Production apply and catch-up ordering proof wait for the dark +runtime from T01. The isolated rehearsal reconciled all 120 source projections +and selected exactly the oldest three of 116 active repositories; see +`docs/evidence/SBOM-WP-0002-T02-repository-projection-rehearsal-2026-08-22.md`. + ## Import and reconcile State Hub history ```task