ops-warden/tests/test_layer_conformance.py

134 lines
5.5 KiB
Python
Raw Permalink Normal View History

Implement §5.3 machine-readably — layer.yaml, checker, conformance tests The security layer model moved v0.1 -> v0.4 (accepted) after ops-warden's assent. Both §5 asks from ADR-0010 were adopted: §5.2 now sanctions the conduit shape on the supplied-authority property, and §5.3 is the declared engine gap amendment, carrying the four fields verbatim and crediting ops-warden's delegation machinery as prior art. Which creates an obligation. §5.3 requires those fields MACHINE-READABLY, and §11 makes "every direct Tooling client maps to a declared §5.1/§5.2/§5.3 entry" a mechanical check. ops-warden's declaration was prose in INTENT.md — the repo that proposed the shape was not implementing it. layer.yaml is the map: 5 contacts (2 declared gaps, 1 read-only observation, 2 conduits) plus the non-Tooling clients recorded explicitly so the check is total rather than silently selective. scripts/check_layer_conformance.py enforces it and found three undeclared modules on its first run — all false positives (help text, a docstring, and the doubles library that SIMULATES bao rather than calling it), which is why the scan now matches invocation shapes instead of the word: an httpx call built against the configured OpenBao address, or an argv whose first element is the bao binary. tests/test_layer_conformance.py adds the §5.2 test the standard says SHOULD exist: _caller_env() returns the caller's environment unchanged, and proxy.py is asserted not to reference X-Vault-Token, approle login, or token create — a conduit that presents its own token is not a conduit. No assertion on review dates, deliberately: a date-triggered failure breaks the build on a calendar day with no code change, the same reasoning WP-0033-T05 recorded for blocker staleness. 398 tests pass, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YWBMovyFoy9RRrfL7zKvPJ Assistant: claude-code Assistant-Model: opus Assistant-Process: 4014535@bnt-lap001 Assistant-Session: d0036016-73e8-4da1-8e47-563e3ab39a3c
2026-08-29 02:45:29 +02:00
"""Layer-model conformance (security-layer-model_v0.4 §5, §11).
Two things are checked here. §11 makes one of them mechanical: every direct
Tooling client maps to a declared shape. §5.2 asks for the other: the conduit's
supplied-authority property covered by a test.
Deliberately absent: any assertion on a §5.3 review date. A date-triggered
failure breaks the build on a calendar day with no code change, punishing
whoever commits next rather than whoever owns the gap the same reasoning
recorded in WARDEN-WP-0033-T05 for blocker staleness.
"""
from __future__ import annotations
import os
import subprocess
import sys
from pathlib import Path
import yaml
ROOT = Path(__file__).resolve().parents[1]
def _decl() -> dict:
return yaml.safe_load((ROOT / "layer.yaml").read_text())
class TestDeclaration:
def test_declares_staff_layer_in_its_own_voice(self):
d = _decl()
assert d["repository"] == "ops-warden"
assert d["layer"] == "staff"
# §11: "only the repository's own file, in its own voice, conforms."
assert d["declared_by"] == "docs/adr/ADR-0010"
def test_every_tooling_contact_maps_to_a_declared_shape(self):
"""§11 mechanical check — the guard against a new undeclared client."""
result = subprocess.run(
[sys.executable, str(ROOT / "scripts" / "check_layer_conformance.py")],
capture_output=True,
text=True,
)
assert result.returncode == 0, (
f"undeclared Tooling contact — a finding under §11, not a tracked gap:\n"
f"{result.stdout}{result.stderr}"
)
def test_declared_gaps_carry_all_four_fields(self):
"""§5.3 is machine-readable or it is prose wearing a schema."""
for c in _decl()["tooling_contacts"]:
if c["shape"] == "5.3":
for field in ("capability", "intended_owner", "blocked_on", "review"):
assert c.get(field), f"{c['id']} missing {field}"
def test_gaps_are_not_counted_as_conformance(self):
"""§11: a declared gap is tracked non-conformance. Keep that visible."""
text = (ROOT / "layer.yaml").read_text()
assert "TRACKED NON-CONFORMANCE" in text.upper()
class TestConduitSuppliesNoAuthority:
"""§5.2: 'MUST NOT present its own credential, MUST NOT widen what the
caller could already do.' The standard says this SHOULD be covered by a
test; this is that test."""
def test_conduit_supplies_no_authority_of_its_own(self, monkeypatch):
from warden import proxy
monkeypatch.setenv("VAULT_TOKEN", "caller-own-token")
monkeypatch.setenv("HOME", "/home/nobody")
before = dict(os.environ)
env = proxy._caller_env()
# The child environment IS the caller's environment — nothing added,
# nothing removed, no ops-warden credential injected.
assert env == before, (
"conduit altered the caller's environment; §5.2 requires it to "
"supply no authority of its own"
)
assert env["VAULT_TOKEN"] == "caller-own-token"
def test_conduit_declares_supplied_authority_none(self):
conduits = [c for c in _decl()["tooling_contacts"] if c["shape"] == "5.2"]
assert conduits, "no §5.2 conduit declared — proxy.py is one"
for c in conduits:
assert c["supplied_authority"] == "none"
def test_proxy_holds_no_credential_constant(self):
"""A conduit that presents its own token is not a conduit (§5.2)."""
src = (ROOT / "src" / "warden" / "proxy.py").read_text()
# It may name token ENV VARS to detect caller auth; it must not carry a
# token value or mint one.
for forbidden in ("X-Vault-Token", "auth/approle/login", "token create"):
assert forbidden not in src, (
f"proxy.py references {forbidden!r} — that is presenting or "
f"minting authority, not conducting the caller's"
)
Review layer model v0.6; publish the PEP stance map §6.4 requires All three v0.4 findings were acted on — §9.1 split into pending/declared-gap and §5's scope rule adopted as recommended and credited, and §9.6 ruled via the load-bearing/attributive distinction with ops-warden's `# audit must not block signing` named as the estate's live example. Checked the favourable ruling rather than accepting it. §9.6's test is "no control branches on its presence": the only consumer of audit.jsonl is `warden activity`, which displays. Nothing gates on a signing record, so the lane is genuinely attributive. AuditTrail.md now records the ruling instead of the open question, and states that the trade must be revisited if a control ever gates on the trail. CONFORMANCE ACTION. §6.4 obligation 3 requires a stance map "published rather than held in code", and requires every PEP-shaped consumer to publish one so the maps can be inventoried — naming ADR-0009 as the reference shape. ops-warden was not doing it: the map lived in PolicyConfig.failure_modes, a dataclass default. Not a code comment, but not published either. pep-stance.yaml publishes it, and the test asserts the published map EQUALS the shipped default. A published map that may drift from the code is worse than no map, because it invites reliance it cannot support. Two findings sent to gate-house, in history/2026-08-29-layer-model-v06-review.md: §6.4 obligation 1 (no side effect without a decision record) contradicts obligation 3 and §9.3, with ops-warden's blessed fail-open stance as the instance; and §6.4 mandates a stance-map inventory in §13 that §13 does not implement — where ops-warden is currently the only PEP to have published one. 402 tests pass, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YWBMovyFoy9RRrfL7zKvPJ Assistant: claude-code Assistant-Model: opus Assistant-Process: 4014535@bnt-lap001 Assistant-Session: d0036016-73e8-4da1-8e47-563e3ab39a3c
2026-08-29 10:20:49 +02:00
class TestPepStanceMap:
"""§6.4: every PEP-shaped consumer MUST publish its unreachable-engine
stance map, total and per zone, 'published rather than held in code'.
ADR-0009 is named as the reference shape, so it should actually hold."""
def _stance(self) -> dict:
return yaml.safe_load((ROOT / "pep-stance.yaml").read_text())
def test_published_map_equals_shipped_behaviour(self):
"""The whole point. A published map that may drift from the code is
worse than none, because it invites reliance it cannot support."""
from warden.config import PolicyConfig
assert self._stance()["stance"] == PolicyConfig().failure_modes
def test_stance_is_total_over_the_zone_model(self):
"""§6.4 obligation 3: total, no implicit default."""
stance = self._stance()["stance"]
required = {
"z0-experimental", "z1-operational", "z2-protected",
"z2-continuity", "z3-critical", "unknown", "not-applicable",
}
assert required <= set(stance), f"stance not total; missing {required - set(stance)}"
assert set(stance.values()) <= {"fail_open", "fail_closed"}
def test_critical_zone_fails_closed(self):
"""ADR-0009's one non-negotiable row."""
assert self._stance()["stance"]["z3-critical"] == "fail_closed"
def test_verdict_is_never_cached(self):
"""§6.4 obligation 2: caching an input claim is permitted; caching the
answer is a second decision point deciding early (§6.1)."""
assert self._stance()["verdict_caching"] == "none"