Admit fixture-asm for ASM T-06 and calibrate known-bad replay

Reassess the T-06 blocker against the secrets-engine consume client.
Gate House cited 3cd9955; approval_consume.py first appears at 4b4d556.
An in-process CAS disables different-digest conflict for known-bad and
drives consume_approval through a mock opener. No network or OpenBao.
Live asm-t06 stays pending.

Assistant: grok
Assistant-Session: 01a05e32-c776-72a3-86ec-c490e027aca9
This commit is contained in:
tegwick 2026-09-02 10:10:53 +02:00
parent 75b79a0167
commit 75deaf073f
13 changed files with 424 additions and 21 deletions

View file

@ -12,3 +12,4 @@ fixture-evidence:
PYTHONPATH=src python3 -m whitehat_security.cli fixtures --output evidence/offline-calibration.json
PYTHONPATH=src python3 -m whitehat_security.cli e3-fixtures --output evidence/offline-e3-calibration.json
PYTHONPATH=src python3 -m whitehat_security.cli capacity-fixture --output evidence/offline-capacity-calibration.json
PYTHONPATH=src python3 -m whitehat_security.cli asm-fixtures --test-id T-06 --output evidence/offline-asm-t06-calibration.json

View file

@ -70,8 +70,9 @@ boundary always holds.
- Finished: `WHITEHAT-WP-0001` through `WHITEHAT-WP-0005`. Residual live
Tenancy Posture evidence is owned by `WHITEHAT-WP-0006` and is blocked on
authorization.
- Active: `WHITEHAT-WP-0007` triages Gate House ASM T-01…T-10. All ten
registrations are `pending`. That work authorizes no probe.
- Active: `WHITEHAT-WP-0007` triages Gate House ASM T-01…T-10. Live
registrations remain `pending`. `fixture-asm-t06` is the first in-process
ASM calibration. That work authorizes no live probe.
- `WHITEHAT-WP-0001` T01T08 are complete for every applicable target.
`audit-core` has dated E2 pass `WH-ENG-20260822-AUDIT-E2-03`.
`tenant-engine` E2 and `platform-pg` E3 stay `not_applicable`. `flex-auth`

View file

@ -1,10 +1,10 @@
# Evidence
This directory stores sanitized run artifacts. `offline-calibration.json`,
`offline-e3-calibration.json` and `offline-capacity-calibration.json` are
generated from repository-created fixtures and prove only that the harness
distinguishes known-good from known-bad behavior. They are not target
assurance. `WH-ENG-20260822-AUDIT-E2-02-abort.json`
`offline-e3-calibration.json`, `offline-capacity-calibration.json`, and
`offline-asm-t06-calibration.json` are generated from repository-created
fixtures and prove only that the harness distinguishes known-good from
known-bad behavior. They are not target assurance. `WH-ENG-20260822-AUDIT-E2-02-abort.json`
is an abort record (`evidence_class: abort`), not an E2 pass or finding.
`WH-ENG-20260822-AUDIT-E2-03.json` is the first authorized target pass; SHA-256
`2d5a21141b78024a5334881e2b7fd62a69c46931057f77515a6c6f18ec497593`. A pass

View file

@ -0,0 +1,50 @@
{
"ended_at": "2026-09-02T08:10:37.894382Z",
"evidence_class": "fixture",
"known_bad": [
{
"openbao_calls": 2,
"outcome": "pass",
"probe_id": "t06-exact-consume",
"reason": "first and same-digest retry consumed"
},
{
"openbao_calls": 1,
"outcome": "finding",
"probe_id": "t06-different-digest-replay",
"reason": "known-bad CAS accepted a different-digest replay and the PEP called OpenBao"
}
],
"known_good": [
{
"openbao_calls": 2,
"outcome": "pass",
"probe_id": "t06-exact-consume",
"reason": "first and same-digest retry consumed"
},
{
"openbao_calls": 0,
"outcome": "pass",
"probe_id": "t06-different-digest-replay",
"reason": "different-digest replay rejected; OpenBao not called"
}
],
"limitations": [
"Offline ASM T-06 calibration evaluates the harness against an in-process CAS.",
"No network, OpenBao, live approval-engine, or reusable credential was used.",
"Gate House cited secrets-engine 3cd9955; approval_consume.py first appears at 4b4d556.",
"This is fixture evidence, not live target assurance."
],
"outcome": "pass",
"run_id": "asm-t06-calibration-2026-09-02T08:10:37.814145Z",
"schema_version": "whitehat-asm-calibration/v1",
"specification": "asm-assurance-targets.v1",
"started_at": "2026-09-02T08:10:37.814145Z",
"target": {
"claimed_revision": "3cd9955ac935be398f0fbb6aa7263660a9888721",
"component": "secrets-engine",
"consume_module_revision": "4b4d556d6264a5da933f7e14aae0fe27ba1f4300",
"module": "secrets_engine.approval_consume"
},
"test_id": "T-06"
}

View file

@ -0,0 +1,232 @@
"""In-process ASM fixtures. No network, OpenBao, or live approval-engine."""
from __future__ import annotations
import json
import os
import stat
import sys
import tempfile
from dataclasses import dataclass
from io import BytesIO
from pathlib import Path
from typing import Any, Callable
from urllib.error import HTTPError
from urllib.request import Request
from .model import Outcome, utc_now
from .plane import repo_root
DIGEST_A = "sha256:" + ("ab" * 32)
DIGEST_B = "sha256:" + ("cd" * 32)
CANDIDATE_CLAIMED = "3cd9955ac935be398f0fbb6aa7263660a9888721"
CONSUME_MODULE_REV = "4b4d556d6264a5da933f7e14aae0fe27ba1f4300"
SYNTHETIC_TOKEN = "whitehat-t06-synthetic-not-for-use"
@dataclass(frozen=True)
class AsmResult:
probe_id: str
outcome: Outcome
reason: str
openbao_calls: int
class _Response:
def __init__(self, status: int, payload: dict[str, Any]) -> None:
self._status = status
self._raw = json.dumps(payload).encode("utf-8")
def getcode(self) -> int:
return self._status
def read(self, _n: int = -1) -> bytes:
return self._raw
def close(self) -> None:
return None
class ConsumeCAS:
"""In-process consume store. Known-bad disables different-digest conflict."""
def __init__(self, *, enforce: bool) -> None:
self.enforce = enforce
self._digest: dict[str, str] = {}
self.openbao_calls = 0
def opener(self, request: Request, timeout: float) -> _Response:
del timeout
body = json.loads(request.data.decode("utf-8"))
approval_id = request.full_url.rstrip("/").rsplit("/", 2)[1]
digest = body["request_digest"]
prior = self._digest.get(approval_id)
if prior is None:
self._digest[approval_id] = digest
return _Response(200, _consumed(approval_id, digest, idempotent=False))
if prior == digest:
return _Response(200, _consumed(approval_id, digest, idempotent=True))
if self.enforce:
raise HTTPError(
request.full_url, 409, "conflict", hdrs=None, fp=BytesIO(b'{"error":"conflict"}')
)
self._digest[approval_id] = digest
return _Response(200, _consumed(approval_id, digest, idempotent=False))
def _consumed(approval_id: str, digest: str, *, idempotent: bool) -> dict[str, Any]:
return {
"approval_id": approval_id,
"status": "consumed",
"request_digest": digest,
"decision_id": "decision:whitehat-t06",
"consumed_at": "2026-09-02T00:00:00Z",
"idempotent": idempotent,
}
def secrets_engine_src() -> Path | None:
env = os.environ.get("WHITEHAT_SECRETS_ENGINE_SRC")
if env:
path = Path(env)
return path if path.exists() else None
sibling = repo_root().parent / "secrets-engine" / "src"
return sibling if sibling.exists() else None
def _load_consume() -> tuple[Callable, Any, type]:
src = secrets_engine_src()
if src is None:
raise RuntimeError("secrets-engine src is not available")
root = str(src)
if root not in sys.path:
sys.path.insert(0, root)
from secrets_engine.approval_consume import ConsumeBinding, consume_approval
from secrets_engine.errors import DecisionError
return consume_approval, ConsumeBinding, DecisionError
def _token_file() -> Path:
handle = tempfile.NamedTemporaryFile("w", delete=False, encoding="utf-8")
try:
handle.write(SYNTHETIC_TOKEN)
handle.flush()
os.fchmod(handle.fileno(), stat.S_IRUSR | stat.S_IWUSR)
finally:
handle.close()
return Path(handle.name)
def _pep_act(cas: ConsumeCAS, digest: str) -> str:
consume_approval, ConsumeBinding, DecisionError = _load_consume()
token = _token_file()
try:
consume_approval(
base_url="http://approval.invalid",
token_file=token,
binding=ConsumeBinding(
approval_id="appr_whitehat-t06",
request_digest=digest,
decision_id="decision:whitehat-t06",
),
opener=cas.opener,
)
except DecisionError:
return "rejected"
finally:
token.unlink(missing_ok=True)
cas.openbao_calls += 1
return "acted"
def t06_results(*, enforce: bool) -> list[AsmResult]:
"""Drive secrets-engine consume_approval through an in-process CAS opener."""
cas = ConsumeCAS(enforce=enforce)
first = _pep_act(cas, DIGEST_A)
same = _pep_act(cas, DIGEST_A)
before_replay = cas.openbao_calls
replay = _pep_act(cas, DIGEST_B)
extra = cas.openbao_calls - before_replay
first_ok = first == "acted" and same == "acted"
if enforce:
replay_ok = replay == "rejected" and extra == 0
return [
AsmResult(
"t06-exact-consume",
"pass" if first_ok else "finding",
"first and same-digest retry consumed",
before_replay,
),
AsmResult(
"t06-different-digest-replay",
"pass" if replay_ok and first_ok else "finding",
"different-digest replay rejected; OpenBao not called"
if replay_ok else "enforcing fixture failed T-06 oracle",
extra,
),
]
replay_bad = replay == "acted" and extra == 1
return [
AsmResult(
"t06-exact-consume",
"pass" if first_ok else "finding",
"first and same-digest retry consumed",
before_replay,
),
AsmResult(
"t06-different-digest-replay",
"finding" if replay_bad else "pass",
"known-bad CAS accepted a different-digest replay and the PEP called OpenBao"
if replay_bad else "known-bad fixture did not lose the T-06 oracle",
extra,
),
]
def t06_calibration() -> dict:
started = utc_now()
try:
good = t06_results(enforce=True)
bad = t06_results(enforce=False)
ok = (
all(item.outcome == "pass" for item in good)
and any(item.outcome == "finding" and item.probe_id.endswith("replay")
for item in bad)
)
limitations = [
"Offline ASM T-06 calibration evaluates the harness against an in-process CAS.",
"No network, OpenBao, live approval-engine, or reusable credential was used.",
"Gate House cited secrets-engine 3cd9955; approval_consume.py first appears at 4b4d556.",
"This is fixture evidence, not live target assurance.",
]
except Exception as error:
return {
"schema_version": "whitehat-asm-calibration/v1",
"evidence_class": "fixture",
"run_id": f"asm-t06-calibration-{started}",
"started_at": started,
"ended_at": utc_now(),
"outcome": "inconclusive",
"specification": "asm-assurance-targets.v1",
"test_id": "T-06",
"limitations": [f"T-06 fixture could not import the candidate: {error}"],
}
return {
"schema_version": "whitehat-asm-calibration/v1",
"evidence_class": "fixture",
"run_id": f"asm-t06-calibration-{started}",
"started_at": started,
"ended_at": utc_now(),
"outcome": "pass" if ok else "finding",
"specification": "asm-assurance-targets.v1",
"test_id": "T-06",
"target": {
"component": "secrets-engine",
"claimed_revision": CANDIDATE_CLAIMED,
"consume_module_revision": CONSUME_MODULE_REV,
"module": "secrets_engine.approval_consume",
},
"known_good": [item.__dict__ for item in good],
"known_bad": [item.__dict__ for item in bad],
"limitations": limitations,
}

View file

@ -9,6 +9,7 @@ from pathlib import Path
from .audit_fixtures import AuditFixture, audit_probe_suite
from .capacity import capacity_calibration
from .differential import execute
from .asm import t06_calibration
from .e3 import CADENCE, PROBES, e3_calibration
from .engagement import AuthorizationError, Engagement
from .fixtures import FixtureService, probe_suite
@ -102,6 +103,11 @@ def main(argv: list[str] | None = None) -> None:
"capacity-fixture", help="calibrate P1/P2 evaluator offline"
)
capacity_fix.add_argument("--output")
asm_fix = commands.add_parser(
"asm-fixtures", help="calibrate ASM probes offline"
)
asm_fix.add_argument("--test-id", default="T-06")
asm_fix.add_argument("--output")
message = commands.add_parser("risk-message")
message.add_argument("report")
conformance = commands.add_parser(
@ -215,6 +221,17 @@ def main(argv: list[str] | None = None) -> None:
else:
print(rendered, end="")
raise SystemExit(0 if result["outcome"] == "pass" else 1)
if args.command == "asm-fixtures":
if args.test_id != "T-06":
print(f"not authorized: no in-process fixture for {args.test_id}", file=sys.stderr)
raise SystemExit(2)
result = t06_calibration()
rendered = json.dumps(result, indent=2, sort_keys=True) + "\n"
if args.output:
Path(args.output).write_text(rendered, encoding="utf-8")
else:
print(rendered, end="")
raise SystemExit(0 if result["outcome"] == "pass" else 1)
if args.command == "capacity-fixture":
result = capacity_calibration()
rendered = json.dumps(result, indent=2, sort_keys=True) + "\n"

View file

@ -13,7 +13,8 @@ honest applicability record the test plane admits against.
| `platform-pg` | not_applicable | No ordinary runtime identity can read the conformance view. |
| `fixture-capacity` | applicable | In-process P1/P2 evaluator. Generates no load. |
| `shared-substrate` | pending | Live capacity needs an operator window and aggressor ceiling. |
| `asm-t01``asm-t10` | pending | Gate House ASM T-01…T-10. Separate `asm` class; not E2/E3/capacity. See `WHITEHAT-WP-0007`. |
| `asm-t01``asm-t10` | pending | Gate House ASM T-01…T-10 live registrations. Separate `asm` class; not E2/E3/capacity. See `WHITEHAT-WP-0007`. |
| `fixture-asm-t06` | applicable | In-process T-06 known-bad/known-good CAS. Offline only. |
`not_applicable` is a completed artifact, not a deferral. Do not relabel it to
close a workplan. Do not reuse cancelled engagement IDs from `engagements/`.

View file

@ -4,7 +4,7 @@
"posture_claim": "ASM T-06",
"attacker_model": "replay, mutation, race, or post-revocation reuse of a bound approval",
"applicability": "pending",
"applicability_reason": "No synthetic approval fixture and no admitted consumption contract in this plane. Known-bad design: disable one binding dimension or atomic consumption so a prohibited replay is accepted. This registration does not authorize a probe.",
"applicability_reason": "Live T-06 still needs a named approval-engine and consuming PEP under a dated engagement. In-process known-bad calibration exists as fixture-asm-t06 against secrets-engine.approval_consume (module first appears at 4b4d556; Gate House cited 3cd9955). This live registration does not authorize a probe.",
"approval_classes": [
"asm"
],

View file

@ -0,0 +1,44 @@
{
"schema_version": "whitehat-target/v1",
"target_id": "fixture-asm-t06",
"posture_claim": "ASM T-06",
"attacker_model": "replay, mutation, race, or post-revocation reuse of a bound approval",
"applicability": "applicable",
"applicability_reason": "In-process T-06 evaluator owned by this repository. It drives secrets-engine.approval_consume through a mock HTTP opener and an in-process CAS. Known-bad disables different-digest conflict. No network, OpenBao, or live approval-engine.",
"approval_classes": ["fixture-asm"],
"specification": "asm-assurance-targets.v1",
"test_id": "T-06",
"title": "Approval Replay Test",
"claims": ["A-04", "A-07", "A-13", "A-15"],
"oracle": "only-live-exactly-bound-unconsumed-request-succeeds",
"adapter": "src/whitehat_security/asm.py",
"probe_pack": "src/whitehat_security/asm.py",
"known_bad_calibration": "src/whitehat_security/asm.py",
"fixture_lifecycle": {
"create": "in-process CAS and temporary 0600 synthetic token file",
"delete": "drop process state and unlink token file",
"ids_are_synthetic": true
},
"egress": "in-process",
"surface": ["in-process-cas", "secrets-engine.approval_consume"],
"result_route": {
"conformance": "gate-house",
"implementation_finding": "risk-nexus",
"specification_finding": "gate-house",
"harness_gap": "whitehat-security"
},
"routes": [
"t06-exact-consume",
"t06-different-digest-replay"
],
"identities": {
"count": 0,
"role": "none; fixture generates no live identity",
"broker_audience": "whitehat-asm/fixture-t06"
},
"abort_telemetry": [
"kill_switch",
"secret_value_observed",
"openbao_called_on_conflict"
]
}

47
tests/test_asm.py Normal file
View file

@ -0,0 +1,47 @@
import json
import pytest
from whitehat_security.asm import t06_calibration, t06_results
from whitehat_security.cli import main
def test_t06_enforcing_rejects_different_digest_without_openbao():
results = {item.probe_id: item for item in t06_results(enforce=True)}
assert results["t06-exact-consume"].outcome == "pass"
assert results["t06-different-digest-replay"].outcome == "pass"
assert results["t06-different-digest-replay"].openbao_calls == 0
def test_t06_known_bad_accepts_different_digest_and_calls_openbao():
results = {item.probe_id: item for item in t06_results(enforce=False)}
assert results["t06-different-digest-replay"].outcome == "finding"
assert results["t06-different-digest-replay"].openbao_calls == 1
def test_t06_calibration_detects_known_bad_and_keeps_token_out():
report = t06_calibration()
assert report["outcome"] == "pass"
assert report["evidence_class"] == "fixture"
assert report["test_id"] == "T-06"
rendered = json.dumps(report)
assert "whitehat-t06-synthetic" not in rendered
assert "Bearer" not in rendered
bad = {item["probe_id"]: item for item in report["known_bad"]}
assert bad["t06-different-digest-replay"]["outcome"] == "finding"
def test_asm_fixtures_cli_writes_t06(tmp_path):
output = tmp_path / "offline-asm-t06-calibration.json"
with pytest.raises(SystemExit) as stopped:
main(["asm-fixtures", "--test-id", "T-06", "--output", str(output)])
assert stopped.value.code == 0
report = json.loads(output.read_text(encoding="utf-8"))
assert report["outcome"] == "pass"
assert report["test_id"] == "T-06"
def test_asm_fixtures_cli_refuses_unterminated_test():
with pytest.raises(SystemExit) as stopped:
main(["asm-fixtures", "--test-id", "T-01"])
assert stopped.value.code == 2

View file

@ -18,7 +18,7 @@ def test_validate_engagement_reports_clean_denial(tmp_path, capsys):
def test_validate_targets_accepts_catalog(capsys):
main(["validate-targets", "targets"])
assert capsys.readouterr().out.startswith("validated 18 target registrations")
assert capsys.readouterr().out.startswith("validated 19 target registrations")
def test_kill_switch_is_clear_by_default(capsys):

View file

@ -70,6 +70,9 @@ def test_catalog_loads_honest_applicability():
assert "asm" in catalog["asm-t06"]["approval_classes"]
assert "live-e2" not in catalog["asm-t01"]["approval_classes"]
assert "capacity" not in catalog["asm-t07"]["approval_classes"]
assert catalog["fixture-asm-t06"]["applicability"] == "applicable"
assert catalog["fixture-asm-t06"]["test_id"] == "T-06"
assert catalog["asm-t06"]["applicability"] == "pending"
def test_retired_ids_include_cancelled_records():

View file

@ -181,14 +181,20 @@ priority: high
state_hub_task_id: "dfe0ff4e-cafe-5f4e-bfe7-6238412632f3"
```
- **Applicability:** `pending`
- **Surface:** approval-engine, `access-engine`, consuming PEP, approval
consumption contract
- **Known-bad design:** one binding dimension or atomic consumption disabled;
a prohibited replay is accepted
- **Blocker:** no synthetic approval fixture and no admitted consumption
contract in this plane
- **Route:** `[GH-CONFORMANCE] T-06 …`; finding → `risk-nexus` (approval-engine)
- **Applicability:** `fixture-asm-t06` **applicable** (in-process). Live
`asm-t06` remains `pending`.
- **Surface:** secrets-engine `approval_consume` PEP client driven through an
in-process CAS opener. Live surface is still approval-engine plus a consuming
PEP.
- **Known-bad design:** CAS disables different-digest conflict; the PEP then
calls OpenBao. The same probe against the enforcing CAS rejects the replay
and does not call OpenBao.
- **Reassessment:** Gate House candidate `3cd9955` does **not** contain
`approval_consume.py`. That module first appears at `4b4d556`. The in-process
fixture is sufficient to admit `fixture-asm` only. It is not a live
approval-engine.
- **Route:** `[GH-CONFORMANCE] T-06 …`; finding → `risk-nexus` (secrets-engine /
approval-engine)
### T-07 — Circuit Breaker
@ -275,10 +281,11 @@ priority: medium
state_hub_task_id: "13518801-015f-51ef-9d07-8cf989e08710"
```
Build in-process known-bad fixtures and, only after that, any live ASM run.
Blocked until a target owner names the surface, identities, and window for a
specific test. Do not relabel a pending target to finish this plan. Do not
send a packet.
In-process T-06 known-bad calibration exists (`fixture-asm-t06`,
`evidence/offline-asm-t06-calibration.json`). Remaining tests still need
fixtures. Any live ASM run is blocked until a target owner names the surface,
identities, and window for that test. Do not relabel a pending live target to
finish this plan. Do not send a packet.
## Sequencing