approval-engine/tests/test_deploy_manifest.py
tegwick 119359c33d Reconcile the release record; guard the image pin with tests
The publication at b51d174 pinned the manifest and appended a note, but left
three documents asserting the opposite. docs/image-scan-2026-09-06.md said "It
has not been pushed" in its Outcome block at the top while recording the
successful push at the bottom; its "Release status" section still said no
release digest existed and the manifest carried REPLACE_WITH_RELEASE_DIGEST;
and deploy/README.md still instructed an operator to replace a placeholder that
is now a real digest -- an instruction to undo the pin.

This is the derived-artifact staleness this repo argued for a rule about in
v0.8 §12 and then produced in its own release evidence, in the file another
repository would read to confirm what shipped. Superseded sections are now
marked in place with what replaced them rather than deleted, since they record
why the base changed and why the release was held. The scan doc carries a
status marker naming which parts state current state, including that a clean
scan is point-in-time and ages as advisories are published.

The workplan gains the publication entry it never got, and a re-check of T03's
full acceptance: a published image reads like a finished task, but the digest
was one of five requirements. Nothing is deployed -- the namespace is empty and
no secrets exist -- and production serve refuses to start without authenticated
audit delivery, so a rollout today would fail closed and prove nothing. T03
stays wait on T01's registrations and the audit credential.

Also corrects an earlier caution in the workplan that the digest was "a local
image id, not a release digest". That was wrong: with the containerd/OCI store
the build id is the manifest digest, and the registry resolves it.

tests/test_deploy_manifest.py holds the pin mechanically instead of by review:
both image references exist, are digest-pinned rather than tagged, are
identical, and no placeholder remains. The identity check matters because the
init container migrates the database the server then serves -- two different
digests there is a split-brain migration. Verified the guard fails on the drift
it targets, not only that it passes today. 116 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PM5HnEAhokxdfcPqBNpT7D

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 715850@bnt-lap001
Assistant-Session: eb557e93-7cb1-45d0-9e57-7d15b3edc60e
2026-09-07 09:04:34 +02:00

69 lines
2.6 KiB
Python

"""Mechanical checks on the deployment manifest's image pinning.
These exist because the manifest drifted in a way review did not catch: the
release was published and pinned, while three documents went on instructing an
operator to replace a placeholder that no longer existed. A pin is exactly the
kind of property a test can hold and prose cannot.
"""
from __future__ import annotations
import re
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
MANIFEST = ROOT / "deploy" / "approval-engine.yaml"
IMAGE_LINE = re.compile(r"^\s*image:\s*(\S+)\s*$", re.MULTILINE)
DIGEST_PIN = re.compile(r"^[^\s@]+@sha256:[0-9a-f]{64}$")
def image_refs() -> list[str]:
return IMAGE_LINE.findall(MANIFEST.read_text(encoding="utf-8"))
def test_manifest_declares_both_images():
"""The init container and the server are two references, not one."""
refs = image_refs()
assert len(refs) == 2, f"expected 2 image references, found {len(refs)}: {refs}"
def test_images_are_digest_pinned_not_tagged():
"""A tag would let the migration and the server run different code.
They share one database: the init container migrates it and the server then
serves it. A floating tag makes 'the schema the migration wrote' and 'the
schema the server expects' independently resolvable, which is the failure
the digest pin exists to prevent.
"""
for ref in image_refs():
assert DIGEST_PIN.match(ref), f"image is not digest-pinned: {ref!r}"
def test_both_images_are_identical():
"""Two different digests here is a split-brain migration waiting to happen."""
refs = image_refs()
assert len(set(refs)) == 1, f"image references differ: {sorted(set(refs))}"
def test_no_release_placeholder_remains():
"""Catches the drift that motivated this file.
A placeholder left in the manifest is caught by the digest-pin check above;
this asserts the specific token is gone so a half-applied release cannot
reintroduce it silently.
"""
text = MANIFEST.read_text(encoding="utf-8")
assert "REPLACE_WITH_RELEASE_DIGEST" not in text
def test_deploy_readme_does_not_instruct_replacing_a_pinned_placeholder():
"""The runbook and the manifest must not disagree about what is left to do.
While the manifest carries a real digest, an operator instruction to replace
a placeholder is an instruction to undo the pin.
"""
manifest = MANIFEST.read_text(encoding="utf-8")
readme = (ROOT / "deploy" / "README.md").read_text(encoding="utf-8")
if "REPLACE_WITH_RELEASE_DIGEST" not in manifest:
assert "replace `REPLACE_WITH_RELEASE_DIGEST`" not in readme