docs: mark the unknown cell, measure the coverage we asked to publish
gate-house ruled the v0.8 assent round (GH-DEC-2026-011, net-kingdom@64394e9): ask 1 declined, ask 2 adopted. Ask 1's refusal is accepted without reservation and the reason is better than the ask -- a sanctioned transitional fail_open is indistinguishable at runtime from the stance the rule forbids, and would make the rule optional at the only moment it costs anything. Ask 2 gave §13.1 a Coverage column with this repo's figures as its first entries. Since we asked for the column, we owe it accuracy: scripts/report_coverage.py measures both populations from the artifacts the runtime uses (reusing the workload-join build rather than re-deriving it), and a test asserts pep-stance.yaml's published block equals what it measures. A hand-counted number in a register that explicitly does not recompute it decays silently, and a stale figure beside a marked cell is worse than the blank the other four rows carry. pep-stance.yaml marks the unknown cell inline as a declared gap -- assent, the measured reason for not flipping, the declined ask, WARDEN-WP-0040 as route -- and a second test keeps it marked while it is fail_open, failing when it is flipped. standard_version stays 0.7 because that is what binds; v0.8 is proposed, so it gains standard_version_reviewed rather than pre-adopting. Separately, gate-house corrected GH-DEC-2026-008: the claim/decision digest comparison it originally required is unimplementable and a fail-closed consumer obeying it would have denied permanently. We had never copied the wording, so nothing to unwind -- but everything they have sent about this lane was living in an inbox thread, a bad home for a correction that only matters when someone finally wires the consume. Now wiki/ApprovalConsumption.md, leading with "nothing is wired", carrying the corrected target and the attribution gap that digest matching does not discharge. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013EPuTc18FjU5WFqoSEKH3C Assistant: claude-code Assistant-Model: opus Assistant-Process: 1276224@bnt-lap001 Assistant-Session: 426ec497-e1c4-4dd3-b417-dfce1ca1dbc3
This commit is contained in:
parent
a942ce805d
commit
5b1a508610
8 changed files with 388 additions and 2 deletions
129
scripts/report_coverage.py
Normal file
129
scripts/report_coverage.py
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Measure the classification coverage published in `pep-stance.yaml`.
|
||||
|
||||
Read-only. No network, no OpenBao, no secret material.
|
||||
|
||||
Why this exists. security-layer-model v0.8 §6.4 obligation 3 requires a dated
|
||||
classification-coverage figure beside each stance, and §13.1 carries a Coverage
|
||||
column whose first entries are ops-warden's — because ops-warden asked for the
|
||||
column after measuring that 0 of 3 signing targets resolve to a zone.
|
||||
|
||||
Having asked for it, we own the figure's accuracy. A hand-counted number in a
|
||||
published register decays silently: the register explicitly does not compute
|
||||
anyone's coverage, and a stale figure beside a marked cell is worse than a blank,
|
||||
which at least reads as "not reported". So the figure is measured from the same
|
||||
two artifacts the runtime uses, and `tests/test_layer_conformance.py` asserts the
|
||||
published block equals what this reports — the same property that makes the stance
|
||||
map worth publishing (`pep-stance.yaml` equals `PolicyConfig.failure_modes` by
|
||||
test), applied one level up.
|
||||
|
||||
Two populations, deliberately not summed. They answer different questions and
|
||||
share no denominator:
|
||||
|
||||
signing targets — actor resources in the flex-auth registry snapshot. This is
|
||||
the population the stance map actually governs: `warden sign` resolves a zone
|
||||
per actor resource, and an unresolved one takes the `unknown` cell.
|
||||
routing lanes — catalog entries with an explicit workload reference. Wider than
|
||||
the stance map's reach, and the figure that shows *why* coverage is low: an
|
||||
unknown lane is almost always another repository's undeclared workload
|
||||
identity, which `ADR-0009` rule 3 forbids closing by inference here.
|
||||
|
||||
Usage:
|
||||
python scripts/report_coverage.py [--json]
|
||||
|
||||
Exit: 0 always. This reports; it does not gate. Coverage is disclosure, never a
|
||||
transitional licence, and a script that failed on low coverage would be arguing
|
||||
the case v0.8 declined.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(REPO / "src"))
|
||||
|
||||
SNAPSHOT = REPO / "registry" / "flex-auth" / "production_registry_snapshot.json"
|
||||
NOT_APPLICABLE = "not-applicable"
|
||||
|
||||
|
||||
def signing_target_coverage(snapshot_path: Path = SNAPSHOT) -> dict[str, int]:
|
||||
"""Zone resolution across the actor resources `warden sign` can name."""
|
||||
registry = json.loads(snapshot_path.read_text())
|
||||
resolved = unknown = not_applicable = 0
|
||||
for manifest in registry.get("resource_manifests") or []:
|
||||
for resource in manifest.get("resources") or []:
|
||||
if str(resource.get("type")) != "ssh-certificate":
|
||||
continue
|
||||
attributes = resource.get("attributes") or {}
|
||||
admission = str(attributes.get("security_zone_admission") or "unknown")
|
||||
zone = str(attributes.get("security_zone") or "unknown")
|
||||
if admission == NOT_APPLICABLE:
|
||||
not_applicable += 1
|
||||
elif zone == "unknown":
|
||||
unknown += 1
|
||||
else:
|
||||
resolved += 1
|
||||
return {"resolved": resolved, "unknown": unknown, "not_applicable": not_applicable}
|
||||
|
||||
|
||||
def routing_lane_coverage(estate_root: Path | None = None) -> dict[str, int]:
|
||||
"""Workload resolution across catalog lanes.
|
||||
|
||||
Delegates to `report_workload_join.build` rather than re-deriving the join: two
|
||||
implementations of "is this lane resolved" would drift, and the published figure
|
||||
should be the one the join report shows.
|
||||
"""
|
||||
import importlib.util
|
||||
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"report_workload_join", Path(__file__).resolve().parent / "report_workload_join.py"
|
||||
)
|
||||
join = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(join)
|
||||
|
||||
report = join.build(
|
||||
REPO / "registry" / "routing" / "catalog.yaml",
|
||||
estate_root if estate_root is not None else Path.home(),
|
||||
)
|
||||
return {
|
||||
"resolved": len(report["resolved"]),
|
||||
"unknown": len(report["unknown"]),
|
||||
"not_applicable": len(report["not_applicable"]),
|
||||
}
|
||||
|
||||
|
||||
def measure(estate_root: Path | None = None) -> dict[str, Any]:
|
||||
return {
|
||||
"signing_targets": signing_target_coverage(),
|
||||
"routing_lanes": routing_lane_coverage(estate_root),
|
||||
}
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--json", action="store_true", dest="as_json")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
report = measure()
|
||||
if args.as_json:
|
||||
print(json.dumps(report, indent=2))
|
||||
return 0
|
||||
|
||||
for name, counts in report.items():
|
||||
total = sum(counts.values())
|
||||
scoped = counts["resolved"] + counts["unknown"]
|
||||
print(f"{name.replace('_', ' ')}: {counts['resolved']}/{scoped} resolved "
|
||||
f"({counts['unknown']} unknown, {counts['not_applicable']} not-applicable, "
|
||||
f"{total} total)")
|
||||
print()
|
||||
print("Coverage is disclosure, not a transitional licence (v0.8 §6.4 obligation 3).")
|
||||
print("Publish in pep-stance.yaml; §13.1 does not compute it for you.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Loading…
Add table
Add a link
Reference in a new issue