Add value-safe verification and audit reporting
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 2s

Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a0217e-8c4c-7383-be6b-f50a6e485306
This commit is contained in:
tegwick 2026-08-23 12:33:38 +02:00
parent 491e706a70
commit c4504c6de9
19 changed files with 598 additions and 50 deletions

View file

@ -201,7 +201,7 @@ secrets-engine plan <decision-or-ccr-id>
secrets-engine apply <decision-or-ccr-id> --stage <build|test|prod>
secrets-engine provision <catalog-id> --stage <stage> --from-file <path>
secrets-engine verify <catalog-id> --positive
secrets-engine verify <catalog-id> --negative
secrets-engine verify <catalog-id> --negative --negative-token-file <path>
secrets-engine exec --catalog <catalog-id> -- <command...>
secrets-engine revoke <catalog-id-or-lease>
```

View file

@ -7,7 +7,8 @@ and production stages.
OpenBao remains the custody and enforcement backend. `secrets-engine` owns the
operator and agent interaction model: catalog, decision checks, plan/apply,
guarded provisioning, verification, delivery, evidence, lifecycle metadata, and
native-access deactivation.
native-access deactivation. Local evidence can be inspected through an
allowlisted per-lane `audit` summary without exposing record detail.
## Start Here

View file

@ -58,14 +58,17 @@ provider, credential broker, or general secrets API.
- KV positive verification logs in through the lane AppRole and reports field
presence without printing values. With no `--field`, it checks every declared
field; an explicit `--field` narrows the check.
- KV negative verification checks that a fixed invalid token cannot read the
path.
- KV negative verification requires a real unrelated token supplied through a
mode-0600 file outside Git and checks that it cannot read the path. Missing
unrelated identity material fails closed without a backend probe.
- Auth-capability verification checks the AppRole token's capabilities on exact
allowlisted and denial-probe paths.
These are bounded policy/presence probes. They do not yet prove denial for a
real unrelated workload identity, exercise a provider operation, or produce
OpenBao audit-request correlation.
These are bounded policy/presence probes. A throwaway OpenBao integration test
proves an overlapping unrelated policy is detected. Production still needs a
reviewed owner/source for each unrelated identity; the engine does not mint or
select that identity itself. Verification does not yet exercise a provider
operation or produce OpenBao audit-request correlation.
### Exec-time delivery
@ -104,12 +107,19 @@ cannot be recovered through that AppRole.
status, metadata/value-presence booleans, missing declared field names,
readiness, and a safe next command. Every declared field must be present.
- Records scrubbed local JSONL evidence and posts a minimal State Hub progress
event on a best-effort basis.
event on a best-effort basis. Each requested State Hub delivery receives an
append-only local `delivered`, `failed`, or `skipped-no-topic` companion
record.
- `audit` summarizes allowlisted local lane evidence: action/result counts,
canonical decision references, session cleanup outcomes, and State Hub
delivery outcomes. It never re-emits arbitrary evidence detail.
- Keeps OpenBao audit logs as the backend source of truth.
State Hub evidence delivery is not durable or transactional, and route does not
replace authorization or unrelated-identity denial evidence. Treat route output
as operational guidance, not complete attestation for high-risk lanes.
State Hub evidence delivery is not queued or transactional; the local receipt
makes failure visible but does not replay it. Route and audit do not replace
authorization, OpenBao audit logs, or unrelated-identity denial evidence.
Treat their output as operational guidance, not complete attestation for
high-risk lanes.
### Revocation currently available
@ -145,6 +155,7 @@ secrets-engine policy publication
secrets-engine route
secrets-engine revoke [--dry-run]
secrets-engine lifecycle suspend|deactivate|destroy
secrets-engine audit <catalog-id> [--json]
```
The implemented exec adapters are `exec-env` and `npm-config`. `read-check` is
@ -254,7 +265,9 @@ keywords: [openbao, approle, auth-capability, handoff, least-privilege]
type: governance
title: Non-secret routing and evidence pointers
description: Reports decision/readiness metadata and records scrubbed local and best-effort State Hub
evidence without returning secret values. Readiness covers every declared field, but this is not a
durable audit store or a real-unrelated-identity denial attestation.
evidence without returning secret values. Append-only delivery receipts expose State Hub failures,
and an allowlisted audit command summarizes lane operations and cleanup. KV verification can attest a
supplied real unrelated identity, but the engine does not own identity selection. This is not a durable
remote audit store.
keywords: [routing, evidence, state-hub, audit, secrets]
```

View file

@ -48,7 +48,7 @@ secrets-engine decision inspect <decision-or-ccr-id>
secrets-engine plan <ref> --stage <build|test|prod>
secrets-engine apply <ref> --stage <stage> [--dry-run] [--bootstrap-token-file F]
secrets-engine provision <catalog-id> --stage <stage> --field NAME (--from-file F | --generate)
secrets-engine verify <catalog-id> [--field NAME] [--positive] [--negative]
secrets-engine verify <catalog-id> [--field NAME] [--positive] [--negative] [--negative-token-file F]
secrets-engine handoff <catalog-id> --stage <stage> --role-id-file F --secret-id-file F
secrets-engine exec --catalog <catalog-id> [--field NAME] [--mode auto|npm-config|exec-env] -- CMD...
secrets-engine policy publication <catalog-id>
@ -57,6 +57,7 @@ secrets-engine revoke <catalog-id> [--dry-run]
secrets-engine lifecycle suspend <catalog-id> [--dry-run]
secrets-engine lifecycle deactivate <catalog-id> [--dry-run]
secrets-engine lifecycle destroy <catalog-id> [--dry-run] [--confirm-destroy <catalog-id>]
secrets-engine audit <catalog-id> [--json]
```
`policy publication` resolves a lane's effective publication scope and the env
@ -81,7 +82,9 @@ CAS-aware create/patch behavior with a strict temporary input reference: values
are absent from argv, sibling fields are preserved, and stale writes fail.
With no `--field`, `verify` checks every declared KV field positively and runs
one path-level negative probe. `route` likewise requires every declared field
one path-level negative probe. KV denial requires `--negative-token-file` with a
real unrelated identity's mode-0600 token file outside Git; absence fails the
check without calling OpenBao. `route` likewise requires every declared field
and reports only missing field names. An explicit `--field` narrows positive
verification; production use will bind such subsets to the action approval.
@ -102,6 +105,13 @@ Exec and verification AppRole logins are scoped sessions. The issued token
self-revokes on every exit path before exec starts (or when verification ends),
and evidence stores only a short accessor fingerprint plus cleanup outcome.
`audit` is a read-only, local evidence summary. It reports action and result
counts, canonical decision references, session cleanup outcomes, and State Hub
delivery outcomes for one cataloged lane. Its parser allowlists those fields and
does not echo arbitrary JSONL detail. State Hub failures are recorded locally as
append-only companion receipts; they are visible but are not automatically
replayed.
## Exit codes
| Code | Meaning |
@ -177,7 +187,8 @@ secrets-engine apply whynot-design-npm-publish --stage prod
secrets-engine provision whynot-design-npm-publish --stage prod \
--field npm_token --from-file ~/.secrets-engine/whynot.token
# 5. prove access without printing the value
secrets-engine verify whynot-design-npm-publish --field npm_token --positive --negative
secrets-engine verify whynot-design-npm-publish --field npm_token --positive --negative \
--negative-token-file /secure/path/unrelated.token
# 6. publish with the token injected into the child only
secrets-engine exec --catalog whynot-design-npm-publish -- npm publish
```

View file

@ -57,8 +57,10 @@ contents in this repo.
## H5 — Audit report command
- `secrets-engine audit <catalog-id>` summarizing non-secret evidence (decision,
applies, provisions, verifications, execs, revokes) for a lane.
- Implemented: `secrets-engine audit <catalog-id>` summarizes allowlisted local
action/result, decision, session-cleanup, and State Hub delivery evidence.
- Remaining: durable/queued State Hub delivery and OpenBao audit request-id
correlation.
## H6 — API service mode

View file

@ -70,9 +70,14 @@ BAO_ADDR=https://bao.coulomb.social \
BAO_ADDR=https://bao.coulomb.social \
secrets-engine verify whynot-design-npm-publish \
--field npm_token --positive --negative \
--negative-token-file /secure/path/reviewed-unrelated.token \
--bootstrap-token-file ~/.secrets-engine/bootstrap/prod-whynot-design.token
```
The unrelated token must represent a reviewed identity outside the lane policy;
it is not the bootstrap token and must be held in a separate mode-0600 file
outside Git.
6. In the `whynot-design` repo, coordinate a real version bump. Run the publish
through child-only injection:

View file

@ -21,6 +21,7 @@ export BAO_ADDR="http://127.0.0.1:8270"
export BAO_TOKEN="se-demo-root"
# Keep secret material OUTSIDE the repo worktree (engine enforces this).
TOKENFILE="$WORK/whynot.token"
NEGATIVE_TOKENFILE="$WORK/unrelated.token"
cleanup() {
set +e
@ -59,7 +60,15 @@ secrets-engine provision whynot-design-npm-publish --stage prod \
echo
echo "### 4. verify positive (approved consumer can read) + negative (others denied)"
secrets-engine verify whynot-design-npm-publish --field npm_token --positive --negative | sed 's/^/ /'
"$BAO_BIN" policy write se-demo-unrelated - <<'EOF' >/dev/null
path "secret/data/prod/coulomb/whynot-design/npm" {
capabilities = ["deny"]
}
EOF
umask 077
"$BAO_BIN" token create -field=token -policy=se-demo-unrelated > "$NEGATIVE_TOKENFILE"
secrets-engine verify whynot-design-npm-publish --field npm_token \
--positive --negative --negative-token-file "$NEGATIVE_TOKENFILE" | sed 's/^/ /'
echo
echo "### 5. exec-time delivery: child sees the token via a temp npmrc; parent never does"

167
src/secrets_engine/audit.py Normal file
View file

@ -0,0 +1,167 @@
"""Allowlisted, non-secret summaries over local append-only evidence."""
from __future__ import annotations
import json
import re
import uuid
from collections import Counter
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
_LABEL = re.compile(r"^[a-z][a-z0-9-]{0,79}$")
_DECISION_REF = re.compile(r"^[A-Z][A-Z0-9-]{2,80}$")
_DELIVERY_RESULTS = {"delivered", "failed", "skipped-no-topic"}
_VERIFY_RESULT = re.compile(r"^(positive|negative):(pass|fail)$")
_ERROR_RESULT = re.compile(
r"^failed-(Catalog|Decision|PolicyGuard|Backend|Provisioning|Verification|Delivery)Error$"
)
def _safe_label(value: object) -> str:
text = value if isinstance(value, str) else ""
return text if _LABEL.fullmatch(text) else "invalid-label"
def _safe_result(value: object) -> str:
text = value if isinstance(value, str) else ""
if (
_LABEL.fullmatch(text)
or _VERIFY_RESULT.fullmatch(text)
or _ERROR_RESULT.fullmatch(text)
):
return text
return "invalid-label"
def _safe_decision_ref(value: object) -> str:
text = value if isinstance(value, str) else ""
if not text:
return ""
try:
return str(uuid.UUID(text))
except ValueError:
return text if _DECISION_REF.fullmatch(text) else ""
def _safe_timestamp(value: object) -> str:
if not isinstance(value, str):
return ""
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
if parsed.tzinfo is None:
return ""
return parsed.astimezone(timezone.utc).isoformat()
except ValueError:
return ""
@dataclass(frozen=True)
class LaneAuditSummary:
catalog_id: str
operation_records: int
malformed_records: int
first_ts: str
last_ts: str
actions: dict[str, int]
results: dict[str, int]
decision_refs: list[str]
session_cleanup: dict[str, int]
hub_delivery: dict[str, int]
def to_json(self) -> dict[str, Any]:
return asdict(self)
def render(self) -> str:
lines = [
f"Lane audit summary for '{self.catalog_id}'",
f" operation records: {self.operation_records}",
f" malformed records: {self.malformed_records}",
f" first: {self.first_ts or 'n/a'}",
f" last: {self.last_ts or 'n/a'}",
" actions: " + _render_counts(self.actions),
" results: " + _render_counts(self.results),
" decisions: " + (", ".join(self.decision_refs) or "none"),
" session cleanup: " + _render_counts(self.session_cleanup),
" hub delivery: " + _render_counts(self.hub_delivery),
]
return "\n".join(lines)
def _render_counts(counts: dict[str, int]) -> str:
return ", ".join(f"{key}={value}" for key, value in sorted(counts.items())) or "none"
def summarize_lane_evidence(evidence_dir: Path, catalog_id: str) -> LaneAuditSummary:
"""Summarize one lane without returning arbitrary record fields or detail."""
actions: Counter[str] = Counter()
results: Counter[str] = Counter()
cleanup: Counter[str] = Counter()
hub_delivery: Counter[str] = Counter()
decisions: set[str] = set()
timestamps: list[str] = []
malformed = 0
operation_records = 0
for path in sorted(Path(evidence_dir).glob("evidence-*.jsonl")):
try:
lines = path.read_text(encoding="utf-8").splitlines()
except OSError:
malformed += 1
continue
for line in lines:
try:
record = json.loads(line)
except (json.JSONDecodeError, TypeError):
malformed += 1
continue
if not isinstance(record, dict):
malformed += 1
continue
if record.get("catalog_id") != catalog_id:
continue
action = _safe_label(record.get("action"))
result = _safe_result(record.get("result"))
if action == "evidence-delivery":
if result in _DELIVERY_RESULTS:
hub_delivery[result] += 1
else:
hub_delivery["invalid"] += 1
continue
operation_records += 1
actions[action] += 1
results[result] += 1
timestamp = _safe_timestamp(record.get("ts"))
if timestamp:
timestamps.append(timestamp)
decision = _safe_decision_ref(record.get("decision_id"))
if decision:
decisions.add(decision)
detail = record.get("detail")
session = detail.get("session") if isinstance(detail, dict) else None
if isinstance(session, dict):
attempted = session.get("revocation_attempted") is True
succeeded = session.get("revocation_succeeded") is True
if succeeded:
cleanup["succeeded"] += 1
elif attempted:
cleanup["failed"] += 1
else:
cleanup["not-attempted"] += 1
timestamps.sort()
return LaneAuditSummary(
catalog_id=catalog_id,
operation_records=operation_records,
malformed_records=malformed,
first_ts=timestamps[0] if timestamps else "",
last_ts=timestamps[-1] if timestamps else "",
actions=dict(sorted(actions.items())),
results=dict(sorted(results.items())),
decision_refs=sorted(decisions),
session_cleanup=dict(sorted(cleanup.items())),
hub_delivery=dict(sorted(hub_delivery.items())),
)

View file

@ -7,12 +7,13 @@ Command surface (FR7):
plan <decision-or-ref> --stage <stage>
apply <decision-or-ref> --stage <stage> [--dry-run] [--bootstrap-token-file F]
provision <catalog-id> --stage <stage> (--from-file F | --generate) --field NAME
verify <catalog-id> [--positive] [--negative] [--field NAME]
verify <catalog-id> [--positive] [--negative] [--field NAME] [--negative-token-file F]
handoff <catalog-id> --stage <stage> --role-id-file F --secret-id-file F
exec --catalog <catalog-id> [--field NAME] [--mode auto|npm-config|exec-env] -- CMD...
route <catalog-id> [--json]
revoke <catalog-id>
lifecycle suspend|deactivate|destroy <catalog-id>
audit <catalog-id> [--json]
Every privileged action is decision-gated and writes non-secret evidence.
`plan` and `apply --dry-run` never mutate OpenBao.
@ -221,6 +222,13 @@ def cmd_verify(cfg: Config, args) -> int:
raise VerificationError(f"lane '{entry.id}' has no field to verify")
positive = args.positive or not args.negative
negative = args.negative or not args.positive
unrelated_token = None
if entry.stores_kv_value() and negative and args.negative_token_file:
from secrets_engine.openbao import read_strict_token_file
unrelated_token = read_strict_token_file(
Path(args.negative_token_file), purpose="negative verification token"
)
if entry.stores_kv_value():
results = []
if positive:
@ -234,7 +242,12 @@ def cmd_verify(cfg: Config, args) -> int:
# Denial is path-scoped, so one probe covers every field on this path.
results.extend(
run_verification(
client, entry, fields[0], positive=False, negative=True
client,
entry,
fields[0],
positive=False,
negative=True,
unrelated_token=unrelated_token,
)
)
else:
@ -460,6 +473,21 @@ def cmd_lifecycle(cfg: Config, args) -> int:
return 0
def cmd_audit(cfg: Config, args) -> int:
"""Summarize allowlisted non-secret evidence for one cataloged lane."""
import json
from secrets_engine.audit import summarize_lane_evidence
entry = get_entry(cfg.catalog_dir, args.catalog_id)
summary = summarize_lane_evidence(cfg.evidence_dir, entry.id)
if args.json:
print(json.dumps(summary.to_json(), indent=2, sort_keys=True))
else:
print(summary.render())
return 0
# -- parser ----------------------------------------------------------------
@ -512,6 +540,11 @@ def build_parser() -> argparse.ArgumentParser:
ve.add_argument("--field", default=None)
ve.add_argument("--positive", action="store_true")
ve.add_argument("--negative", action="store_true")
ve.add_argument(
"--negative-token-file",
default=None,
help="mode-0600 out-of-repo token for a real unrelated identity",
)
add_token_arg(ve)
ve.set_defaults(func=cmd_verify)
@ -571,6 +604,11 @@ def build_parser() -> argparse.ArgumentParser:
add_token_arg(lp)
lp.set_defaults(func=cmd_lifecycle)
au = sub.add_parser("audit", help="summarize non-secret local lane evidence")
au.add_argument("catalog_id")
au.add_argument("--json", action="store_true")
au.set_defaults(func=cmd_audit)
return p

View file

@ -10,6 +10,7 @@ import json
import os
import urllib.error
import urllib.request
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from pathlib import Path
@ -55,6 +56,12 @@ class EvidenceWriter:
day = datetime.now(timezone.utc).strftime("%Y-%m-%d")
return self.evidence_dir / f"evidence-{day}.jsonl"
def _append_local(self, record: dict[str, Any]) -> None:
"""Append one record before any best-effort external delivery."""
path = self._log_path()
with path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(record, sort_keys=True) + "\n")
def record(
self,
action: str,
@ -67,7 +74,10 @@ class EvidenceWriter:
hub: bool = True,
) -> dict[str, Any]:
"""Append one non-secret evidence record. Returns the stored record."""
record_id = str(uuid.uuid4())
hub_requested = bool(hub and self.hub_url)
record = {
"record_id": record_id,
"ts": datetime.now(timezone.utc).isoformat(),
"action": action,
"result": result,
@ -76,20 +86,38 @@ class EvidenceWriter:
"stage": stage,
"decision_id": decision_id,
"detail": _scrub(detail or {}),
"hub_delivery_requested": hub_requested,
}
path = self._log_path()
with path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(record, sort_keys=True) + "\n")
if hub and self.hub_url:
self._post_hub(action, result, catalog_id, stage, decision_id)
self._append_local(record)
if hub_requested:
delivery_result = self._post_hub(
action, result, catalog_id, stage, decision_id
)
# Append-only companion evidence makes an unavailable State Hub
# visible without rewriting or delaying the primary local record.
self._append_local(
{
"record_id": str(uuid.uuid4()),
"related_record_id": record_id,
"ts": datetime.now(timezone.utc).isoformat(),
"action": "evidence-delivery",
"result": delivery_result,
"actor": self.actor,
"catalog_id": catalog_id,
"stage": stage,
"decision_id": decision_id,
"detail": {},
"hub_delivery_requested": False,
}
)
return record
def _post_hub(
self, action: str, result: str, catalog_id: str, stage: str, decision_id: str
) -> None:
"""Best-effort progress note to State Hub. Never raises; never sends values."""
) -> str:
"""Best-effort progress note; return a non-secret delivery outcome."""
if not self.topic_id:
return
return "skipped-no-topic"
summary = f"secrets-engine {action}: {result}"
if catalog_id:
summary += f" [{catalog_id}{'/' + stage if stage else ''}]"
@ -111,6 +139,7 @@ class EvidenceWriter:
method="POST",
)
urllib.request.urlopen(req, timeout=3).read()
return "delivered"
except (urllib.error.URLError, OSError, ValueError):
# Hub being offline must never block secret work or leak anything.
pass
return "failed"

View file

@ -28,29 +28,34 @@ from secrets_engine.errors import BackendError, ProvisioningError
from secrets_engine.safe_paths import containing_git_worktree
def _check_token_file(path: Path) -> str:
"""Read a bootstrap token file after enforcing mode-0600 and out-of-repo."""
def read_strict_token_file(path: Path, *, purpose: str = "token") -> str:
"""Read token material only from a mode-0600 path outside Git worktrees."""
if not path.exists():
raise ProvisioningError(f"bootstrap token file not found: {path}")
raise ProvisioningError(f"{purpose} file not found: {path}")
st = path.stat()
if st.st_mode & 0o077:
raise ProvisioningError(
f"bootstrap token file {path} is group/other-accessible "
f"{purpose} file {path} is group/other-accessible "
f"(mode {oct(st.st_mode & 0o777)}); must be 0600"
)
# Refuse a token file living inside a Git worktree.
worktree = containing_git_worktree(path)
if worktree is not None:
raise ProvisioningError(
f"bootstrap token file {path} is inside a Git worktree ({worktree}); "
f"{purpose} file {path} is inside a Git worktree ({worktree}); "
"store it outside any repo"
)
token = path.read_text(encoding="utf-8").strip()
if not token:
raise ProvisioningError(f"bootstrap token file {path} is empty")
raise ProvisioningError(f"{purpose} file {path} is empty")
return token
def _check_token_file(path: Path) -> str:
"""Compatibility wrapper for bootstrap authentication input."""
return read_strict_token_file(path, purpose="bootstrap token")
@dataclass
class ScopedTokenSession:
"""One AppRole login token that revokes itself on close."""

View file

@ -86,7 +86,8 @@ def route_lane(
if entry.kind == "kv" and entry.delivery_auth_management == "existing":
missing = "externally managed OpenBao policy/AppRole readiness"
next_command = (
f"secrets-engine verify {entry.id} --positive --negative"
f"secrets-engine verify {entry.id} --positive --negative "
"--negative-token-file <unrelated-token-path>"
)
elif entry.kind == "kv" and not entry.has_delivery_auth:
missing = "native delivery auth declaration"
@ -114,7 +115,8 @@ def route_lane(
next_command = f"secrets-engine exec --catalog {entry.id} -- <command...>"
else:
next_command = (
f"secrets-engine verify {entry.id} --positive --negative"
f"secrets-engine verify {entry.id} --positive --negative "
"--negative-token-file <unrelated-token-path>"
)
return RouteResult(

View file

@ -74,10 +74,25 @@ def verify_positive(client: OpenBaoClient, entry: CatalogEntry, field: str) -> V
)
def verify_negative(client: OpenBaoClient, entry: CatalogEntry) -> VerifyResult:
"""An unrelated token must be denied. Uses an empty (invalid) token."""
# An empty/garbage token stands in for an unrelated consumer.
denied = not client.kv_can_read(entry.mount, entry.path, token="se-unrelated-denied")
def verify_negative(
client: OpenBaoClient,
entry: CatalogEntry,
*,
unrelated_token: str | None,
) -> VerifyResult:
"""A real unrelated token must be denied the cataloged path."""
if not unrelated_token:
return VerifyResult(
"negative",
False,
{
"reason": "no real unrelated token supplied; denial not proven",
"path": entry.path,
},
)
denied = not client.kv_can_read(
entry.mount, entry.path, token=unrelated_token
)
return VerifyResult(
"negative",
denied,
@ -167,7 +182,13 @@ def verify_auth_capability_negative(client: OpenBaoClient, entry: CatalogEntry)
def run_verification(
client: OpenBaoClient, entry: CatalogEntry, field: str, *, positive: bool, negative: bool
client: OpenBaoClient,
entry: CatalogEntry,
field: str,
*,
positive: bool,
negative: bool,
unrelated_token: str | None = None,
) -> list[VerifyResult]:
if entry.kind == "kv" and field not in entry.fields:
raise VerificationError(
@ -183,7 +204,9 @@ def run_verification(
if positive:
results.append(verify_positive(client, entry, field))
if negative:
results.append(verify_negative(client, entry))
results.append(
verify_negative(client, entry, unrelated_token=unrelated_token)
)
if not results:
raise VerificationError("no verification check selected (use --positive/--negative)")
return results

102
tests/test_audit.py Normal file
View file

@ -0,0 +1,102 @@
import json
import urllib.error
from pathlib import Path
from types import SimpleNamespace
from secrets_engine.audit import summarize_lane_evidence
from secrets_engine.cli import build_parser
from secrets_engine.evidence import EvidenceWriter
DECISION_ID = "e6381a56-3e55-4fac-b22c-63ee1c152ce8"
def test_lane_audit_summarizes_allowlisted_fields_and_cleanup(tmp_path, monkeypatch):
writer = EvidenceWriter(
evidence_dir=tmp_path,
hub_url="http://hub.invalid",
topic_id="topic-id",
)
monkeypatch.setattr(
"urllib.request.urlopen", lambda *_args, **_kwargs: SimpleNamespace(read=lambda: b"{}")
)
writer.record(
"exec",
result="exit-0",
catalog_id="test-lane",
stage="test",
decision_id=DECISION_ID,
detail={
"session": {
"session_handle": "safe-fingerprint",
"revocation_attempted": True,
"revocation_succeeded": True,
}
},
)
def offline(*_args, **_kwargs):
raise urllib.error.URLError("offline")
monkeypatch.setattr("urllib.request.urlopen", offline)
writer.record(
"verify",
result="positive-pass",
catalog_id="test-lane",
stage="test",
decision_id=DECISION_ID,
detail={
"session": {
"session_handle": "another-safe-fingerprint",
"revocation_attempted": True,
"revocation_succeeded": False,
}
},
)
path = next(tmp_path.glob("evidence-*.jsonl"))
with path.open("a", encoding="utf-8") as fh:
fh.write("not-json\n")
fh.write(
json.dumps(
{
"catalog_id": "test-lane",
"action": "fake-SUPER-SECRET-value",
"result": "fake-SUPER-SECRET-value",
"decision_id": "fake-SUPER-SECRET-value",
"ts": "not-a-time",
"detail": {"value": "fake-SUPER-SECRET-value"},
}
)
+ "\n"
)
summary = summarize_lane_evidence(tmp_path, "test-lane")
rendered = json.dumps(summary.to_json()) + summary.render()
assert summary.operation_records == 3
assert summary.malformed_records == 1
assert summary.actions == {"exec": 1, "invalid-label": 1, "verify": 1}
assert summary.results == {
"exit-0": 1,
"invalid-label": 1,
"positive-pass": 1,
}
assert summary.decision_refs == [DECISION_ID]
assert summary.session_cleanup == {"failed": 1, "succeeded": 1}
assert summary.hub_delivery == {"delivered": 1, "failed": 1}
assert "fake-SUPER-SECRET-value" not in rendered
assert "safe-fingerprint" not in rendered
def test_lane_audit_empty_directory_is_a_valid_empty_summary(tmp_path):
summary = summarize_lane_evidence(tmp_path, "test-lane")
assert summary.operation_records == 0
assert summary.actions == {}
assert summary.render().startswith("Lane audit summary for 'test-lane'")
def test_cli_parser_exposes_audit_json_command():
args = build_parser().parse_args(["audit", "test-lane", "--json"])
assert args.catalog_id == "test-lane"
assert args.json is True

View file

@ -90,9 +90,32 @@ def test_full_chain(bao_dev, tmp_path):
pos = verify_positive(client, entry, "npm_token")
assert pos.passed, pos.detail
neg = verify_negative(client, entry)
client.write_policy(
"se-unrelated-denied",
f'path "{entry.mount}/data/{entry.path}" {{ capabilities = ["deny"] }}\n',
)
unrelated = json.loads(
client._run_ok(
["token", "create", "-format=json", "-policy=se-unrelated-denied"]
)
)["auth"]["client_token"]
neg = verify_negative(client, entry, unrelated_token=unrelated)
assert neg.passed, neg.detail
# A real unrelated identity with accidental policy overlap must make the
# negative check fail; an invalid/garbage token could not detect this.
client.write_policy(
"se-unrelated-overlap",
f'path "{entry.mount}/data/{entry.path}" {{ capabilities = ["read"] }}\n',
)
overlapping = json.loads(
client._run_ok(
["token", "create", "-format=json", "-policy=se-unrelated-overlap"]
)
)["auth"]["client_token"]
leaked = verify_negative(client, entry, unrelated_token=overlapping)
assert not leaked.passed, leaked.detail
# exec delivery: child can resolve token via npmrc; assert via a probe script
probe = tmp_path / "probe.sh"
probe.write_text(

View file

@ -83,8 +83,11 @@ def test_verify_defaults_to_every_declared_field_and_one_path_denial(
monkeypatch.setattr(cli, "_require_lane_approval", lambda *_args: None)
monkeypatch.setattr(cli.OpenBaoClient, "resolve", lambda *_args, **_kwargs: object())
def fake_verify(_client, _entry, field, *, positive, negative):
def fake_verify(
_client, _entry, field, *, positive, negative, unrelated_token=None
):
calls.append((field, positive, negative))
assert unrelated_token is None
check = "positive" if positive else "negative"
return [VerifyResult(check, True, {"field": field, "reason": "test"})]
@ -100,6 +103,7 @@ def test_verify_defaults_to_every_declared_field_and_one_path_denial(
field=None,
positive=False,
negative=False,
negative_token_file=None,
)
assert cli.cmd_verify(_config(tmp_path), args) == 0

View file

@ -1,4 +1,6 @@
import json
import urllib.error
from types import SimpleNamespace
from secrets_engine.evidence import EvidenceWriter, _scrub
from secrets_engine.redact import looks_secret, redact_text
@ -39,3 +41,48 @@ def test_evidence_record_has_no_value(tmp_path):
# written to disk too
files = list(tmp_path.glob("evidence-*.jsonl"))
assert files and "npm_shouldnotappear123" not in files[0].read_text()
def test_evidence_records_append_only_hub_delivery_success(tmp_path, monkeypatch):
monkeypatch.setattr(
"urllib.request.urlopen",
lambda *_args, **_kwargs: SimpleNamespace(read=lambda: b"{}"),
)
writer = EvidenceWriter(
evidence_dir=tmp_path,
hub_url="http://hub.invalid",
topic_id="topic-id",
)
primary = writer.record("verify", result="pass", catalog_id="lane")
lines = [
json.loads(line)
for line in next(tmp_path.glob("evidence-*.jsonl")).read_text().splitlines()
]
assert len(lines) == 2
assert lines[0] == primary
assert lines[1]["action"] == "evidence-delivery"
assert lines[1]["result"] == "delivered"
assert lines[1]["related_record_id"] == primary["record_id"]
def test_evidence_records_hub_failure_without_raising(tmp_path, monkeypatch):
def offline(*_args, **_kwargs):
raise urllib.error.URLError("offline")
monkeypatch.setattr("urllib.request.urlopen", offline)
writer = EvidenceWriter(
evidence_dir=tmp_path,
hub_url="http://hub.invalid",
topic_id="topic-id",
)
writer.record("verify", result="pass", catalog_id="lane")
lines = [
json.loads(line)
for line in next(tmp_path.glob("evidence-*.jsonl")).read_text().splitlines()
]
assert lines[-1]["action"] == "evidence-delivery"
assert lines[-1]["result"] == "failed"

View file

@ -0,0 +1,61 @@
import copy
import os
import pytest
from secrets_engine.catalog import validate_entry
from secrets_engine.errors import ProvisioningError
from secrets_engine.openbao import read_strict_token_file
from secrets_engine.verify import verify_negative
from tests.test_catalog import VALID
def _entry():
return validate_entry(copy.deepcopy(VALID))
class NegativeClient:
def __init__(self, can_read):
self.can_read = can_read
self.tokens = []
def kv_can_read(self, _mount, _path, *, token):
self.tokens.append(token)
return self.can_read
def test_negative_check_fails_closed_without_real_unrelated_token():
client = NegativeClient(can_read=False)
result = verify_negative(client, _entry(), unrelated_token=None)
assert result.passed is False
assert "not proven" in result.detail["reason"]
assert client.tokens == []
def test_negative_check_uses_real_token_and_detects_policy_overlap():
denied = NegativeClient(can_read=False)
assert verify_negative(
denied, _entry(), unrelated_token="test-unrelated-token"
).passed
assert denied.tokens == ["test-unrelated-token"]
overlapping = NegativeClient(can_read=True)
result = verify_negative(
overlapping, _entry(), unrelated_token="test-overlapping-token"
)
assert result.passed is False
assert "LEAK RISK" in result.detail["reason"]
def test_negative_token_file_requires_mode_0600(tmp_path):
path = tmp_path / "unrelated.token"
path.write_text("test-unrelated-token", encoding="utf-8")
os.chmod(path, 0o644)
with pytest.raises(ProvisioningError, match="must be 0600"):
read_strict_token_file(path, purpose="negative verification token")
os.chmod(path, 0o600)
assert (
read_strict_token_file(path, purpose="negative verification token")
== "test-unrelated-token"
)

View file

@ -290,11 +290,17 @@ priority: high
Progress 2026-08-23. KV verification now checks every declared field by default,
with one path-level negative probe, and route readiness reads once and requires
all declared fields. Missing readiness evidence contains field names only. Tests
prove a missing sibling makes readiness false. The real unrelated-identity
negative contract, audit request correlation, durable evidence delivery, and
lane audit summary remain outstanding.
prove a missing sibling makes readiness false. Evidence delivery attempts now
receive append-only local success/failure receipts, and `audit` produces an
allowlisted per-lane summary of actions, results, decisions, session cleanup,
and delivery outcomes without echoing arbitrary detail. The real
unrelated-identity input is now enforced as a strict mode-0600 out-of-repo token
file; absence fails closed, and throwaway OpenBao integration deliberately adds
an overlapping unrelated read policy and proves the check fails. Production
identity selection/ownership, audit request correlation, queued/replayed State
Hub delivery, and complete privileged failure-path evidence remain outstanding.
The complete repository suite passes with 103 tests after these changes,
The complete repository suite passes with 111 tests after these changes,
including throwaway OpenBao integration coverage.
Make verification and routing truthful for multi-field and high-risk lanes: