Classify I and A as mechanical and calibrate V observations offline.
WHITEHAT-WP-0012 records that identity and authorization evidence belongs in the declaring service's CI, and adds an in-process V2/V3/V4 observation judge that injects no failure. Live E4, R4, and V stay wait on an unnamed owner or operator, so the plan is blocked. No packet is authorized. Assistant: grok Assistant-Session: 01a10450-dda4-7253-91a9-b8dccb2f0173
This commit is contained in:
parent
3cd8a81d77
commit
eebcc39a12
18 changed files with 678 additions and 25 deletions
1
Makefile
1
Makefile
|
|
@ -13,6 +13,7 @@ fixture-evidence:
|
|||
PYTHONPATH=src python3 -m whitehat_security.cli e3-fixtures --output evidence/offline-e3-calibration.json
|
||||
PYTHONPATH=src python3 -m whitehat_security.cli e4-fixtures --output evidence/offline-e4-calibration.json
|
||||
PYTHONPATH=src python3 -m whitehat_security.cli r4-fixtures --output evidence/offline-r4-calibration.json
|
||||
PYTHONPATH=src python3 -m whitehat_security.cli v-fixtures --output evidence/offline-v-calibration.json
|
||||
PYTHONPATH=src python3 -m whitehat_security.cli cadence > evidence/probe-cadence.json
|
||||
PYTHONPATH=src python3 -m whitehat_security.cli capacity-fixture --output evidence/offline-capacity-calibration.json
|
||||
PYTHONPATH=src python3 -m whitehat_security.cli asm-fixtures --test-id T-01 --output evidence/offline-asm-t01-calibration.json
|
||||
|
|
|
|||
25
SCOPE.md
25
SCOPE.md
|
|
@ -17,7 +17,12 @@ a complete engagement record already exists. No live target is authorized.
|
|||
- Calibrate E4 credential confinement and R4 erasure recovery in process.
|
||||
`whitehat e4-fixtures` and `whitehat r4-fixtures` project no credential,
|
||||
open no store, and are not target assurance. No live E4 or R4 target is
|
||||
registered.
|
||||
registered. Tenancy Posture §13 classifies the live E4 artifact as
|
||||
Mechanical. The fixture does not perform that demonstration.
|
||||
- Judge a supplied V2, V3, or V4 observation in process.
|
||||
`whitehat v-fixtures` injects no failure and makes no cluster call. A pass
|
||||
is a complete observation shape. V1 stays inconclusive because it is a
|
||||
mechanical exercise of the declaring service.
|
||||
- Calibrate P1/P2 capacity judgments from supplied samples.
|
||||
`whitehat capacity-fixture` generates no load.
|
||||
- Calibrate Active Secrets Management Canon T-01 through T-10 in process.
|
||||
|
|
@ -51,8 +56,11 @@ a complete engagement record already exists. No live target is authorized.
|
|||
- Hold an OpenBao client, a database driver, or a cluster-mutation API.
|
||||
- Produce E4 or R4 evidence about a real substrate. Those families stop at
|
||||
the in-process harness.
|
||||
- Produce identity, authorization, or availability evidence. The attacker
|
||||
model does not define those axes.
|
||||
- Produce identity or authorization evidence. Tenancy Posture §13 classifies
|
||||
I2, I3, A2, A3, and A4 as Mechanical, and that evidence belongs in the
|
||||
declaring service's CI. This repository has no I or A probe.
|
||||
- Inject a failure, or produce live V2, V3, or V4 evidence. The in-process
|
||||
judge only checks a supplied observation. A restart is not V2.
|
||||
- Admit `tenant-engine` for E2 or `platform-pg` for E3. Both are
|
||||
`not_applicable`.
|
||||
|
||||
|
|
@ -106,6 +114,12 @@ a complete engagement record already exists. No live target is authorized.
|
|||
[`history/2026-10-04T0649Z-scope-against-intent.md`](history/2026-10-04T0649Z-scope-against-intent.md).
|
||||
`WHITEHAT-WP-0011` added the E4 and R4 calibrations and the cadence
|
||||
register. It authorizes no packet.
|
||||
- `WHITEHAT-WP-0012` records that I and A evidence is mechanical and adds
|
||||
the offline V observation judge. Live E4, live R4, and a live V2–V4
|
||||
exercise stay `wait` on an owner or operator this repository cannot name.
|
||||
The plan stays `blocked`. It does not add a `risk-nexus` writer and it
|
||||
does not change `conformance_state`. It authorizes no packet and no
|
||||
failure injection.
|
||||
- Finished: `WHITEHAT-WP-0001` through `WHITEHAT-WP-0005`,
|
||||
`WHITEHAT-WP-0007`, `WHITEHAT-WP-0009`, `WHITEHAT-WP-0010`, and
|
||||
`WHITEHAT-WP-0011`.
|
||||
|
|
@ -167,4 +181,7 @@ a complete engagement record already exists. No live target is authorized.
|
|||
[`WHITEHAT-WP-0008`](workplans/WHITEHAT-WP-0008-live-asm-residuals.md)
|
||||
for unauthorized live ASM residuals, and
|
||||
[`WHITEHAT-WP-0011`](workplans/WHITEHAT-WP-0011-scope-capability-gaps.md)
|
||||
for the offline E4, R4, and cadence work.
|
||||
for the offline E4, R4, and cadence work, and
|
||||
[`WHITEHAT-WP-0012`](workplans/WHITEHAT-WP-0012-ia-classification-and-v-observations.md)
|
||||
for the I/A classification, the offline V judge, and the blocked live
|
||||
E4, R4, and V requirements.
|
||||
|
|
|
|||
|
|
@ -167,6 +167,9 @@ credential issuer; or permission to alter routing/provisioning state.
|
|||
|
||||
Executable calibration is `whitehat e4-fixtures`. It is in-process, projects
|
||||
no credential, and is not target assurance. No live E4 target is registered.
|
||||
Tenancy Posture §13 classifies the live E4 artifact as Mechanical. This
|
||||
calibration does not perform that demonstration. A live demonstration waits
|
||||
on `WHITEHAT-WP-0012-T03`.
|
||||
|
||||
## 6. Shared P1/P2 capacity — bounded co-resident
|
||||
|
||||
|
|
@ -229,7 +232,8 @@ alone is not a pass.
|
|||
|
||||
An undeclared backup, escrow or recovery key invented for the test. If the
|
||||
evidence inventory cannot show that every declared copy was exercised, the
|
||||
outcome is inconclusive rather than pass.
|
||||
outcome is inconclusive rather than pass. A live exercise waits on
|
||||
`WHITEHAT-WP-0012-T04`.
|
||||
|
||||
`audit-core` is not an R4 target under its current integrity-chain design; it
|
||||
targets R2. The facility does not erase the evidence service's retained facts
|
||||
|
|
@ -239,12 +243,45 @@ Executable calibration is `whitehat r4-fixtures`. It is in-process, opens no
|
|||
store, and is not target assurance. An incomplete copy inventory stays
|
||||
inconclusive. No live R4 target is registered.
|
||||
|
||||
## 8. Axes outside this first model
|
||||
## 8. Axes outside this adversarial facility
|
||||
|
||||
Identity (`I`), authorization (`A`) and availability (`V`) have evidence
|
||||
requirements in Tenancy Posture, but `WHITEHAT-WP-0001` does not yet define
|
||||
their adversarial facilities. A run under this document must not claim evidence
|
||||
for those axes merely because it observed related behavior.
|
||||
Tenancy Posture v0.1 §13 classifies the evidence, and this section follows
|
||||
that classification. A run under this document must not claim an axis merely
|
||||
because it observed related behavior.
|
||||
|
||||
### Identity (I) and authorization (A)
|
||||
|
||||
I2, I3, A2, A3, and A4 evidence is Mechanical. It belongs in the declaring
|
||||
service's CI: identifier validation, a rejection test, a verified-token
|
||||
binding, a live re-query on an `aal2`-class path, a choke point that refuses
|
||||
an unbound request, a denial observed at the endpoint, and a second PDP
|
||||
substituted without a PEP change. This repository does not build an
|
||||
adversarial probe for those artifacts. `whitehat cadence` has no I or A
|
||||
family.
|
||||
|
||||
### Availability (V)
|
||||
|
||||
V1 is a Mechanical exercise: enumerate critical dependencies, restart or
|
||||
recreate, and record the interruption and recovery time. A restart is not
|
||||
V2. V2, V3, and V4 are Adversarial and failure-injected. Decision 4.6 says V
|
||||
is end-to-end and names the operation; the delivering service owns the
|
||||
declaration.
|
||||
|
||||
`whitehat v-fixtures` judges a supplied observation. It does not terminate an
|
||||
instance, remove a failure domain, or make a region unavailable. A pass means
|
||||
the observation record has the shape §13 requires. It does not mean a service
|
||||
recovered. A V2 claim that is only a restart, a claim with no failure
|
||||
injected, and traffic that did not continue or recover are findings. A claim
|
||||
that omits the operation, measured RTO/RPO, or the remaining shared failure
|
||||
domain stays inconclusive. V1 stays inconclusive here. A live exercise waits
|
||||
on `WHITEHAT-WP-0012-T05` and needs the separate operator approval required
|
||||
by the rules of engagement §1.
|
||||
|
||||
### R4
|
||||
|
||||
R4 remains Adversarial. The in-process calibration is `whitehat r4-fixtures`.
|
||||
No live R4 target is registered. A live exercise waits on
|
||||
`WHITEHAT-WP-0012-T04`. `audit-core` targets R2 and is not that owner.
|
||||
|
||||
## 9. Model selection and review
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,8 @@
|
|||
|
||||
This directory stores sanitized run artifacts. `offline-calibration.json`,
|
||||
`offline-e3-calibration.json`, `offline-e4-calibration.json`,
|
||||
`offline-r4-calibration.json`, `offline-capacity-calibration.json`, and
|
||||
`offline-r4-calibration.json`, `offline-v-calibration.json`,
|
||||
`offline-capacity-calibration.json`, and
|
||||
`offline-asm-t01-calibration.json` … `offline-asm-t10-calibration.json` are
|
||||
generated from repository-created fixtures and prove only that the harness
|
||||
distinguishes known-good from known-bad behavior. They are not target
|
||||
|
|
|
|||
82
evidence/offline-v-calibration.json
Normal file
82
evidence/offline-v-calibration.json
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
{
|
||||
"ended_at": "2026-10-04T09:13:43.644466Z",
|
||||
"evidence_class": "fixture",
|
||||
"failure_injected_by_harness": false,
|
||||
"known_bad": [
|
||||
{
|
||||
"outcome": "finding",
|
||||
"probe_id": "restart-as-v2",
|
||||
"reason": "a restart or recreate is not V2, V3, or V4"
|
||||
},
|
||||
{
|
||||
"outcome": "finding",
|
||||
"probe_id": "no-injection",
|
||||
"reason": "no failure was injected"
|
||||
},
|
||||
{
|
||||
"outcome": "inconclusive",
|
||||
"probe_id": "missing-rto-rpo",
|
||||
"reason": "measured RTO and RPO are absent"
|
||||
},
|
||||
{
|
||||
"outcome": "finding",
|
||||
"probe_id": "traffic-stopped",
|
||||
"reason": "traffic did not continue or recover"
|
||||
},
|
||||
{
|
||||
"outcome": "inconclusive",
|
||||
"probe_id": "missing-failure-domain",
|
||||
"reason": "remaining shared failure domain was not recorded"
|
||||
},
|
||||
{
|
||||
"outcome": "inconclusive",
|
||||
"probe_id": "unnamed-operation",
|
||||
"reason": "the claim does not name the operation"
|
||||
},
|
||||
{
|
||||
"outcome": "inconclusive",
|
||||
"probe_id": "v1-restart",
|
||||
"reason": "V1 is a mechanical restart exercise and is not evidence from this harness"
|
||||
},
|
||||
{
|
||||
"outcome": "finding",
|
||||
"probe_id": "v3-wrong-kind",
|
||||
"reason": "failure kind does not match the claimed level"
|
||||
},
|
||||
{
|
||||
"outcome": "finding",
|
||||
"probe_id": "v4-no-alternate",
|
||||
"reason": "traffic and state did not recover in the alternate region"
|
||||
}
|
||||
],
|
||||
"known_bad_failed_first": true,
|
||||
"known_good": [
|
||||
{
|
||||
"outcome": "pass",
|
||||
"probe_id": "v2-shape",
|
||||
"reason": "observation shape matches V2; this harness injected no failure"
|
||||
},
|
||||
{
|
||||
"outcome": "pass",
|
||||
"probe_id": "v3-shape",
|
||||
"reason": "observation shape matches V3; this harness injected no failure"
|
||||
},
|
||||
{
|
||||
"outcome": "pass",
|
||||
"probe_id": "v4-shape",
|
||||
"reason": "observation shape matches V4; this harness injected no failure"
|
||||
}
|
||||
],
|
||||
"limitations": [
|
||||
"Offline V calibration evaluates the harness; it is not target assurance.",
|
||||
"No instance was terminated, no failure domain was removed, and no region was made unavailable.",
|
||||
"No cluster call, packet, database connection, or load was made.",
|
||||
"A pass is a complete observation shape. It is not availability of a service.",
|
||||
"V1 is a mechanical exercise and stays inconclusive here.",
|
||||
"Identity and authorization evidence is not judged by this command."
|
||||
],
|
||||
"outcome": "pass",
|
||||
"run_id": "v-calibration-2026-10-04T09:13:43.644222Z",
|
||||
"schema_version": "whitehat-v-calibration/v1",
|
||||
"started_at": "2026-10-04T09:13:43.644222Z"
|
||||
}
|
||||
|
|
@ -21,7 +21,7 @@
|
|||
"executable": "whitehat e4-fixtures",
|
||||
"exposure_window": "No live target and no schedule, so no detection window.",
|
||||
"family": "E4",
|
||||
"live_owner": null,
|
||||
"live_owner": "WHITEHAT-WP-0012-T03",
|
||||
"scheduled": false
|
||||
},
|
||||
{
|
||||
|
|
@ -37,7 +37,15 @@
|
|||
"executable": "whitehat r4-fixtures",
|
||||
"exposure_window": "No live target and no schedule, so no detection window.",
|
||||
"family": "R4",
|
||||
"live_owner": null,
|
||||
"live_owner": "WHITEHAT-WP-0012-T04",
|
||||
"scheduled": false
|
||||
},
|
||||
{
|
||||
"declared_interval": null,
|
||||
"executable": "whitehat v-fixtures",
|
||||
"exposure_window": "An operator-approved failure-injection window only. The harness injects no failure. V1 is not this family.",
|
||||
"family": "V2/V3/V4",
|
||||
"live_owner": "WHITEHAT-WP-0012-T05",
|
||||
"scheduled": false
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@
|
|||
"items": {"enum": [
|
||||
"fixture-e2", "live-e2", "fixture-e3", "e3",
|
||||
"fixture-capacity", "capacity", "fixture-asm", "asm",
|
||||
"fixture-e4", "fixture-r4"
|
||||
"fixture-e4", "fixture-r4", "fixture-v"
|
||||
]}
|
||||
},
|
||||
"routes": {"type": "array", "items": {"type": "string"}},
|
||||
|
|
|
|||
|
|
@ -42,7 +42,7 @@ def register() -> dict:
|
|||
"declared_interval": None,
|
||||
"scheduled": False,
|
||||
"exposure_window": "No live target and no schedule, so no detection window.",
|
||||
"live_owner": None,
|
||||
"live_owner": "WHITEHAT-WP-0012-T03",
|
||||
},
|
||||
{
|
||||
"family": "P1/P2",
|
||||
|
|
@ -58,7 +58,18 @@ def register() -> dict:
|
|||
"declared_interval": None,
|
||||
"scheduled": False,
|
||||
"exposure_window": "No live target and no schedule, so no detection window.",
|
||||
"live_owner": None,
|
||||
"live_owner": "WHITEHAT-WP-0012-T04",
|
||||
},
|
||||
{
|
||||
"family": "V2/V3/V4",
|
||||
"executable": "whitehat v-fixtures",
|
||||
"declared_interval": None,
|
||||
"scheduled": False,
|
||||
"exposure_window": (
|
||||
"An operator-approved failure-injection window only. "
|
||||
"The harness injects no failure. V1 is not this family."
|
||||
),
|
||||
"live_owner": "WHITEHAT-WP-0012-T05",
|
||||
},
|
||||
{
|
||||
"family": "ASM T-01..T-10",
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ from .cadence import register as cadence_register
|
|||
from .e3 import CADENCE, PROBES, e3_calibration
|
||||
from .e4 import e4_calibration
|
||||
from .r4 import r4_calibration
|
||||
from .v import v_calibration
|
||||
from .engagement import AuthorizationError, Engagement
|
||||
from .fixtures import FixtureService, probe_suite
|
||||
from .model import RunReport, utc_now
|
||||
|
|
@ -107,6 +108,10 @@ def main(argv: list[str] | None = None) -> None:
|
|||
e4_fix.add_argument("--output")
|
||||
r4_fix = commands.add_parser("r4-fixtures", help="calibrate R4 probes offline")
|
||||
r4_fix.add_argument("--output")
|
||||
v_fix = commands.add_parser(
|
||||
"v-fixtures", help="calibrate V observation shapes offline"
|
||||
)
|
||||
v_fix.add_argument("--output")
|
||||
capacity_fix = commands.add_parser(
|
||||
"capacity-fixture", help="calibrate P1/P2 evaluator offline"
|
||||
)
|
||||
|
|
@ -240,6 +245,14 @@ def main(argv: list[str] | None = None) -> None:
|
|||
else:
|
||||
print(rendered, end="")
|
||||
raise SystemExit(0 if result["outcome"] == "pass" else 1)
|
||||
if args.command == "v-fixtures":
|
||||
result = v_calibration()
|
||||
rendered = json.dumps(result, indent=2, sort_keys=True) + "\n"
|
||||
if args.output:
|
||||
Path(args.output).write_text(rendered, encoding="utf-8")
|
||||
else:
|
||||
print(rendered, end="")
|
||||
raise SystemExit(0 if result["outcome"] == "pass" else 1)
|
||||
if args.command == "e3-fixtures":
|
||||
result = e3_calibration()
|
||||
rendered = json.dumps(result, indent=2, sort_keys=True) + "\n"
|
||||
|
|
|
|||
|
|
@ -30,10 +30,11 @@ APPROVAL_CLASSES = {
|
|||
"asm": frozenset({"asm-assurance"}),
|
||||
"fixture-e4": frozenset({"e4-confinement"}),
|
||||
"fixture-r4": frozenset({"r4-erasure"}),
|
||||
"fixture-v": frozenset({"v-observation"}),
|
||||
}
|
||||
FIXTURE_CLASSES = frozenset({
|
||||
"fixture-e2", "fixture-e3", "fixture-capacity", "fixture-asm",
|
||||
"fixture-e4", "fixture-r4",
|
||||
"fixture-e4", "fixture-r4", "fixture-v",
|
||||
})
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -45,12 +45,14 @@ def load_registration(path: str | Path) -> dict[str, Any]:
|
|||
raise AuthorizationError(f"{path}: E3 registration must project one runtime identity")
|
||||
if classes & {"fixture-capacity", "capacity"} and count not in {0, 1}:
|
||||
raise AuthorizationError(f"{path}: capacity registration projects at most one aggressor identity")
|
||||
if classes & {"fixture-e4", "fixture-r4"} and count != 0:
|
||||
raise AuthorizationError(f"{path}: E4 and R4 fixtures project no credential")
|
||||
if (classes & {"fixture-e4", "fixture-r4"}) and (
|
||||
if classes & {"fixture-e4", "fixture-r4", "fixture-v"} and count != 0:
|
||||
raise AuthorizationError(f"{path}: E4, R4, and V fixtures project no credential")
|
||||
if (classes & {"fixture-e4", "fixture-r4", "fixture-v"}) and (
|
||||
classes & {"live-e2", "e3", "capacity", "asm"}
|
||||
):
|
||||
raise AuthorizationError(f"{path}: E4 and R4 fixtures cannot carry a live approval class")
|
||||
raise AuthorizationError(
|
||||
f"{path}: E4, R4, and V fixtures cannot carry a live approval class"
|
||||
)
|
||||
if classes & {"fixture-asm", "asm"} and count not in {0, 1, 2}:
|
||||
raise AuthorizationError(f"{path}: ASM registration projects at most two identities")
|
||||
if classes & {"fixture-asm", "asm"} and not data.get("test_id"):
|
||||
|
|
|
|||
203
src/whitehat_security/v.py
Normal file
203
src/whitehat_security/v.py
Normal file
|
|
@ -0,0 +1,203 @@
|
|||
"""Judge a supplied availability observation. Injects no failure."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .model import Outcome, utc_now
|
||||
|
||||
_LEVELS = frozenset({"V1", "V2", "V3", "V4"})
|
||||
_KIND = {
|
||||
"V2": "instance-terminated",
|
||||
"V3": "failure-domain-removed",
|
||||
"V4": "region-unavailable",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VObservation:
|
||||
"""Supplied record. This process never sets a failure in a cluster."""
|
||||
|
||||
probe_id: str
|
||||
claimed_level: str
|
||||
operation: str = ""
|
||||
failure_injected: bool = False
|
||||
failure_kind: str = ""
|
||||
traffic_continued_or_recovered: bool = False
|
||||
rto_recorded: bool = False
|
||||
rpo_recorded: bool = False
|
||||
shared_failure_domain_recorded: bool = False
|
||||
critical_path_observed: bool = False
|
||||
degraded_modes_observed: bool = False
|
||||
alternate_region_recovered: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VResult:
|
||||
probe_id: str
|
||||
outcome: Outcome
|
||||
reason: str
|
||||
|
||||
|
||||
def evaluate(observation: VObservation) -> VResult:
|
||||
"""Judge one supplied observation against Tenancy Posture §13.
|
||||
|
||||
V1 is a mechanical restart exercise and cannot pass here. V2, V3, and V4
|
||||
require a recorded failure injection of the matching kind. A missing
|
||||
measurement stays inconclusive. A pass is a complete observation shape.
|
||||
"""
|
||||
if observation.claimed_level not in _LEVELS:
|
||||
return VResult(observation.probe_id, "aborted", "unknown availability claim")
|
||||
if observation.claimed_level == "V1":
|
||||
return VResult(
|
||||
observation.probe_id, "inconclusive",
|
||||
"V1 is a mechanical restart exercise and is not evidence from this harness",
|
||||
)
|
||||
if observation.failure_kind == "restart":
|
||||
return VResult(
|
||||
observation.probe_id, "finding",
|
||||
"a restart or recreate is not V2, V3, or V4",
|
||||
)
|
||||
if not observation.failure_injected:
|
||||
return VResult(
|
||||
observation.probe_id, "finding",
|
||||
"no failure was injected",
|
||||
)
|
||||
required = _KIND[observation.claimed_level]
|
||||
if observation.failure_kind != required:
|
||||
return VResult(
|
||||
observation.probe_id, "finding",
|
||||
"failure kind does not match the claimed level",
|
||||
)
|
||||
if not observation.operation.strip():
|
||||
return VResult(
|
||||
observation.probe_id, "inconclusive",
|
||||
"the claim does not name the operation",
|
||||
)
|
||||
if not observation.rto_recorded or not observation.rpo_recorded:
|
||||
return VResult(
|
||||
observation.probe_id, "inconclusive",
|
||||
"measured RTO and RPO are absent",
|
||||
)
|
||||
if observation.claimed_level == "V2":
|
||||
if not observation.traffic_continued_or_recovered:
|
||||
return VResult(
|
||||
observation.probe_id, "finding",
|
||||
"traffic did not continue or recover",
|
||||
)
|
||||
if not observation.shared_failure_domain_recorded:
|
||||
return VResult(
|
||||
observation.probe_id, "inconclusive",
|
||||
"remaining shared failure domain was not recorded",
|
||||
)
|
||||
return VResult(
|
||||
observation.probe_id, "pass",
|
||||
"observation shape matches V2; this harness injected no failure",
|
||||
)
|
||||
if observation.claimed_level == "V3":
|
||||
if not observation.critical_path_observed or not observation.degraded_modes_observed:
|
||||
return VResult(
|
||||
observation.probe_id, "inconclusive",
|
||||
"critical path or degraded modes were not observed",
|
||||
)
|
||||
return VResult(
|
||||
observation.probe_id, "pass",
|
||||
"observation shape matches V3; this harness injected no failure",
|
||||
)
|
||||
if not observation.alternate_region_recovered:
|
||||
return VResult(
|
||||
observation.probe_id, "finding",
|
||||
"traffic and state did not recover in the alternate region",
|
||||
)
|
||||
return VResult(
|
||||
observation.probe_id, "pass",
|
||||
"observation shape matches V4; this harness injected no failure",
|
||||
)
|
||||
|
||||
|
||||
def _observation(probe_id: str, level: str, **overrides: object) -> VObservation:
|
||||
kind = {"V1": "restart", "V2": "instance-terminated",
|
||||
"V3": "failure-domain-removed", "V4": "region-unavailable"}[level]
|
||||
fields: dict[str, object] = {
|
||||
"probe_id": probe_id,
|
||||
"claimed_level": level,
|
||||
"operation": "read",
|
||||
"failure_injected": level != "V1",
|
||||
"failure_kind": kind,
|
||||
"traffic_continued_or_recovered": True,
|
||||
"rto_recorded": True,
|
||||
"rpo_recorded": True,
|
||||
"shared_failure_domain_recorded": True,
|
||||
"critical_path_observed": True,
|
||||
"degraded_modes_observed": True,
|
||||
"alternate_region_recovered": level == "V4",
|
||||
}
|
||||
fields.update(overrides)
|
||||
return VObservation(
|
||||
probe_id=str(fields["probe_id"]),
|
||||
claimed_level=str(fields["claimed_level"]),
|
||||
operation=str(fields["operation"]),
|
||||
failure_injected=bool(fields["failure_injected"]),
|
||||
failure_kind=str(fields["failure_kind"]),
|
||||
traffic_continued_or_recovered=bool(fields["traffic_continued_or_recovered"]),
|
||||
rto_recorded=bool(fields["rto_recorded"]),
|
||||
rpo_recorded=bool(fields["rpo_recorded"]),
|
||||
shared_failure_domain_recorded=bool(fields["shared_failure_domain_recorded"]),
|
||||
critical_path_observed=bool(fields["critical_path_observed"]),
|
||||
degraded_modes_observed=bool(fields["degraded_modes_observed"]),
|
||||
alternate_region_recovered=bool(fields["alternate_region_recovered"]),
|
||||
)
|
||||
|
||||
|
||||
def v_calibration() -> dict:
|
||||
"""Known-bad must fail before a known-good pass is trusted. No injection."""
|
||||
started = utc_now()
|
||||
known_bad = [
|
||||
evaluate(_observation("restart-as-v2", "V2", failure_kind="restart")),
|
||||
evaluate(_observation("no-injection", "V2", failure_injected=False)),
|
||||
evaluate(_observation("missing-rto-rpo", "V2", rto_recorded=False, rpo_recorded=False)),
|
||||
evaluate(_observation("traffic-stopped", "V2", traffic_continued_or_recovered=False)),
|
||||
evaluate(_observation("missing-failure-domain", "V2", shared_failure_domain_recorded=False)),
|
||||
evaluate(_observation("unnamed-operation", "V2", operation="")),
|
||||
evaluate(_observation("v1-restart", "V1")),
|
||||
evaluate(_observation("v3-wrong-kind", "V3", failure_kind="instance-terminated")),
|
||||
evaluate(_observation("v4-no-alternate", "V4", alternate_region_recovered=False)),
|
||||
]
|
||||
expected = {
|
||||
"restart-as-v2": "finding",
|
||||
"no-injection": "finding",
|
||||
"missing-rto-rpo": "inconclusive",
|
||||
"traffic-stopped": "finding",
|
||||
"missing-failure-domain": "inconclusive",
|
||||
"unnamed-operation": "inconclusive",
|
||||
"v1-restart": "inconclusive",
|
||||
"v3-wrong-kind": "finding",
|
||||
"v4-no-alternate": "finding",
|
||||
}
|
||||
bad_held = all(result.outcome == expected[result.probe_id] for result in known_bad)
|
||||
known_good = [
|
||||
evaluate(_observation("v2-shape", "V2")),
|
||||
evaluate(_observation("v3-shape", "V3")),
|
||||
evaluate(_observation("v4-shape", "V4")),
|
||||
] if bad_held else []
|
||||
good_held = bool(known_good) and all(result.outcome == "pass" for result in known_good)
|
||||
return {
|
||||
"schema_version": "whitehat-v-calibration/v1",
|
||||
"evidence_class": "fixture",
|
||||
"failure_injected_by_harness": False,
|
||||
"run_id": f"v-calibration-{started}",
|
||||
"started_at": started,
|
||||
"ended_at": utc_now(),
|
||||
"outcome": "pass" if bad_held and good_held else "finding",
|
||||
"known_bad_failed_first": bad_held,
|
||||
"known_good": [result.__dict__ for result in known_good],
|
||||
"known_bad": [result.__dict__ for result in known_bad],
|
||||
"limitations": [
|
||||
"Offline V calibration evaluates the harness; it is not target assurance.",
|
||||
"No instance was terminated, no failure domain was removed, and no region was made unavailable.",
|
||||
"No cluster call, packet, database connection, or load was made.",
|
||||
"A pass is a complete observation shape. It is not availability of a service.",
|
||||
"V1 is a mechanical exercise and stays inconclusive here.",
|
||||
"Identity and authorization evidence is not judged by this command.",
|
||||
],
|
||||
}
|
||||
34
targets/fixture-v.json
Normal file
34
targets/fixture-v.json
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
{
|
||||
"schema_version": "whitehat-target/v1",
|
||||
"target_id": "fixture-v",
|
||||
"posture_claim": "V2",
|
||||
"attacker_model": "supplied V2/V3/V4 observation; the harness injects no failure",
|
||||
"applicability": "applicable",
|
||||
"applicability_reason": "In-process evaluator created by this repository. It injects no failure, projects no credential, and makes no cluster call. No live availability target is registered. V1 stays a mechanical exercise of the declaring service.",
|
||||
"approval_classes": ["fixture-v"],
|
||||
"adapter": "src/whitehat_security/v.py",
|
||||
"probe_pack": "src/whitehat_security/v.py",
|
||||
"known_bad_calibration": "src/whitehat_security/v.py",
|
||||
"fixture_lifecycle": {
|
||||
"create": "in-process observations",
|
||||
"delete": "drop process state",
|
||||
"ids_are_synthetic": true
|
||||
},
|
||||
"egress": "in-process",
|
||||
"routes": [
|
||||
"instance-terminated",
|
||||
"failure-domain-removed",
|
||||
"region-unavailable",
|
||||
"restart"
|
||||
],
|
||||
"identities": {
|
||||
"count": 0,
|
||||
"role": "none; fixture projects no credential and injects no failure",
|
||||
"broker_audience": "whitehat-v/fixture"
|
||||
},
|
||||
"abort_telemetry": [
|
||||
"failure_injection_attempt",
|
||||
"cluster_mutation",
|
||||
"missing_measurement"
|
||||
]
|
||||
}
|
||||
|
|
@ -18,7 +18,7 @@ def test_validate_engagement_reports_clean_denial(tmp_path, capsys):
|
|||
|
||||
def test_validate_targets_accepts_catalog(capsys):
|
||||
main(["validate-targets", "targets"])
|
||||
assert capsys.readouterr().out.startswith("validated 30 target registrations")
|
||||
assert capsys.readouterr().out.startswith("validated 31 target registrations")
|
||||
|
||||
|
||||
def test_kill_switch_is_clear_by_default(capsys):
|
||||
|
|
|
|||
|
|
@ -40,8 +40,13 @@ def test_cadence_register_schedules_nothing():
|
|||
record = register()
|
||||
assert record["scheduled_live_runs"] is False
|
||||
families = {item["family"]: item for item in record["families"]}
|
||||
assert set(families) == {"E2", "E3", "E4", "P1/P2", "R4", "ASM T-01..T-10"}
|
||||
assert set(families) == {"E2", "E3", "E4", "P1/P2", "R4", "V2/V3/V4", "ASM T-01..T-10"}
|
||||
assert "I" not in families
|
||||
assert "A" not in families
|
||||
assert all(item["scheduled"] is False for item in families.values())
|
||||
assert families["E3"]["declared_interval"] == "24h"
|
||||
assert families["E4"]["declared_interval"] is None
|
||||
assert families["R4"]["live_owner"] is None
|
||||
assert families["E4"]["live_owner"] == "WHITEHAT-WP-0012-T03"
|
||||
assert families["R4"]["live_owner"] == "WHITEHAT-WP-0012-T04"
|
||||
assert families["V2/V3/V4"]["scheduled"] is False
|
||||
assert families["V2/V3/V4"]["live_owner"] == "WHITEHAT-WP-0012-T05"
|
||||
|
|
|
|||
|
|
@ -263,10 +263,11 @@ def test_fixture_e3_projects_one_runtime_identity(tmp_path):
|
|||
assert cleanup(lease, broker)["credential_revocation"] == "revoked"
|
||||
|
||||
|
||||
def test_fixture_e4_and_r4_project_no_credential_and_refuse_a_live_environment(tmp_path):
|
||||
def test_fixture_e4_r4_and_v_project_no_credential_and_refuse_a_live_environment(tmp_path):
|
||||
cases = (
|
||||
("fixture-e4", "fixture-e4", "e4-confinement", "other-substrate"),
|
||||
("fixture-r4", "fixture-r4", "r4-erasure", "horizon"),
|
||||
("fixture-v", "fixture-v", "v-observation", "instance-terminated"),
|
||||
)
|
||||
for target_id, approval_class, technique, route in cases:
|
||||
broker = LocalBroker()
|
||||
|
|
|
|||
93
tests/test_v_observation.py
Normal file
93
tests/test_v_observation.py
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from whitehat_security.engagement import AuthorizationError
|
||||
from whitehat_security.targets import load_registration
|
||||
from whitehat_security.v import VObservation, evaluate, v_calibration
|
||||
|
||||
|
||||
def test_restart_and_missing_injection_are_not_a_pass():
|
||||
restart = evaluate(VObservation(
|
||||
"restart", claimed_level="V2", operation="read", failure_injected=True,
|
||||
failure_kind="restart", traffic_continued_or_recovered=True,
|
||||
rto_recorded=True, rpo_recorded=True, shared_failure_domain_recorded=True,
|
||||
))
|
||||
assert restart.outcome == "finding"
|
||||
absent = evaluate(VObservation(
|
||||
"absent", claimed_level="V2", operation="read", failure_injected=False,
|
||||
failure_kind="instance-terminated", traffic_continued_or_recovered=True,
|
||||
rto_recorded=True, rpo_recorded=True, shared_failure_domain_recorded=True,
|
||||
))
|
||||
assert absent.outcome == "finding"
|
||||
|
||||
|
||||
def test_missing_measurements_stay_inconclusive():
|
||||
missing = evaluate(VObservation(
|
||||
"rto", claimed_level="V2", operation="read", failure_injected=True,
|
||||
failure_kind="instance-terminated", traffic_continued_or_recovered=True,
|
||||
rto_recorded=False, rpo_recorded=True, shared_failure_domain_recorded=True,
|
||||
))
|
||||
assert missing.outcome == "inconclusive"
|
||||
unnamed = evaluate(VObservation(
|
||||
"operation", claimed_level="V2", operation=" ", failure_injected=True,
|
||||
failure_kind="instance-terminated", traffic_continued_or_recovered=True,
|
||||
rto_recorded=True, rpo_recorded=True, shared_failure_domain_recorded=True,
|
||||
))
|
||||
assert unnamed.outcome == "inconclusive"
|
||||
|
||||
|
||||
def test_v1_is_not_adversarial_evidence_from_this_harness():
|
||||
result = evaluate(VObservation(
|
||||
"v1", claimed_level="V1", operation="read", failure_injected=True,
|
||||
failure_kind="restart", rto_recorded=True,
|
||||
))
|
||||
assert result.outcome == "inconclusive"
|
||||
|
||||
|
||||
def test_v3_and_v4_reject_the_wrong_shape():
|
||||
wrong = evaluate(VObservation(
|
||||
"v3", claimed_level="V3", operation="read", failure_injected=True,
|
||||
failure_kind="instance-terminated", rto_recorded=True, rpo_recorded=True,
|
||||
critical_path_observed=True, degraded_modes_observed=True,
|
||||
))
|
||||
assert wrong.outcome == "finding"
|
||||
region = evaluate(VObservation(
|
||||
"v4", claimed_level="V4", operation="read", failure_injected=True,
|
||||
failure_kind="region-unavailable", rto_recorded=True, rpo_recorded=True,
|
||||
alternate_region_recovered=False,
|
||||
))
|
||||
assert region.outcome == "finding"
|
||||
|
||||
|
||||
def test_unknown_claim_aborts():
|
||||
assert evaluate(VObservation("nope", claimed_level="V9")).outcome == "aborted"
|
||||
|
||||
|
||||
def test_calibration_fails_known_bad_before_trusting_a_pass():
|
||||
report = v_calibration()
|
||||
assert report["known_bad_failed_first"] is True
|
||||
assert report["failure_injected_by_harness"] is False
|
||||
assert report["evidence_class"] == "fixture"
|
||||
assert report["outcome"] == "pass"
|
||||
assert "pass" not in {item["outcome"] for item in report["known_bad"]}
|
||||
assert {item["outcome"] for item in report["known_good"]} == {"pass"}
|
||||
assert any("no cluster" in item.lower() or "no instance" in item.lower()
|
||||
for item in report["limitations"])
|
||||
|
||||
|
||||
def test_fixture_v_projects_no_credential_and_no_live_class(tmp_path: Path):
|
||||
registration = json.loads(Path("targets/fixture-v.json").read_text(encoding="utf-8"))
|
||||
assert registration["identities"]["count"] == 0
|
||||
assert registration["approval_classes"] == ["fixture-v"]
|
||||
registration["identities"]["count"] = 1
|
||||
path = tmp_path / "counted.json"
|
||||
path.write_text(json.dumps(registration), encoding="utf-8")
|
||||
with pytest.raises(AuthorizationError, match="project no credential"):
|
||||
load_registration(path)
|
||||
registration["identities"]["count"] = 0
|
||||
registration["approval_classes"] = ["fixture-v", "capacity"]
|
||||
path.write_text(json.dumps(registration), encoding="utf-8")
|
||||
with pytest.raises(AuthorizationError, match="cannot carry a live approval class"):
|
||||
load_registration(path)
|
||||
|
|
@ -0,0 +1,144 @@
|
|||
---
|
||||
id: WHITEHAT-WP-0012
|
||||
type: workplan
|
||||
title: "Classify I and A, calibrate V observations, and name the live E4, R4, and V requirements"
|
||||
domain: infotech
|
||||
repo: whitehat-security
|
||||
status: blocked
|
||||
owner: whitehat-security
|
||||
topic_slug: whitehat-security
|
||||
created: "2026-10-04"
|
||||
updated: "2026-10-04"
|
||||
related:
|
||||
- WHITEHAT-WP-0006
|
||||
- WHITEHAT-WP-0008
|
||||
- WHITEHAT-WP-0011
|
||||
---
|
||||
|
||||
# Classify I and A, calibrate V observations, and name the live residuals
|
||||
|
||||
`history/2026-10-04T0649Z-scope-against-intent.md` left four items off a
|
||||
workplan: the identity, authorization, and availability axes; a live E4 or
|
||||
R4 target; a `risk-nexus` filing client; and any change to
|
||||
`conformance_state`. Live E2, E3, P1/P2, and ASM stay on blocked
|
||||
`WHITEHAT-WP-0006` and `WHITEHAT-WP-0008`. This plan does not reopen them.
|
||||
Offline E4 and R4 calibration stays on finished `WHITEHAT-WP-0011`.
|
||||
|
||||
The local work is the classification record and an in-process availability
|
||||
judge. The live work cannot start. No scanned service declares current E: 4.
|
||||
No owner has named an R4 copy inventory. No operator has approved a
|
||||
failure-injection exercise. This plan authorizes no packet, database
|
||||
connection, load, or failure injection.
|
||||
|
||||
## Refused as tasks
|
||||
|
||||
These gaps stay closed as boundaries. They are not wait tasks.
|
||||
|
||||
- A `risk-nexus` filing client. Delivery stays the local outbox.
|
||||
`whitehat deliver` does not file findings or assign severity. A writer in
|
||||
this repository would cross that ownership boundary.
|
||||
- `conformance_state`. It stays `blocked-clean` in `INTENT.md`. The v0.8
|
||||
flip remains held under gate-house GH-WP-0004. This plan does not edit
|
||||
`INTENT.md`.
|
||||
|
||||
## T01 — Record that I and A evidence is mechanical
|
||||
|
||||
```task
|
||||
id: WHITEHAT-WP-0012-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Tenancy Posture v0.1 §13 classifies I2, I3, A2, A3, and A4 evidence as
|
||||
Mechanical. It belongs in the declaring service's CI. `docs/attacker-model.md`
|
||||
§8 now says this repository does not build an adversarial I or A probe, and
|
||||
`whitehat cadence` has no I or A family. A run under that model must not
|
||||
claim I or A evidence. No probe was added.
|
||||
|
||||
The same section records that §13 classifies the live E4 artifact as
|
||||
Mechanical, and that V1 is a mechanical exercise. Those sentences do not
|
||||
authorize a demonstration.
|
||||
|
||||
## T02 — Calibrate V2, V3, and V4 observation shapes offline
|
||||
|
||||
```task
|
||||
id: WHITEHAT-WP-0012-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
`whitehat v-fixtures` judges a supplied observation. A V2 claim that is only
|
||||
a restart, a claim with no failure injected, and a claim whose failure kind
|
||||
does not match the level are findings. Missing RTO/RPO, a missing operation
|
||||
name, and a missing shared failure domain stay inconclusive. V1 stays
|
||||
inconclusive. Known-good shapes are trusted only after those known-bad cases
|
||||
hold. `targets/fixture-v.json` is applicable only as `fixture-v` and projects
|
||||
no credential. The plane rejects that class outside `environment=fixture`.
|
||||
The command injects no failure and makes no cluster call.
|
||||
|
||||
## T03 — Live E4 demonstration
|
||||
|
||||
```task
|
||||
id: WHITEHAT-WP-0012-T03
|
||||
status: wait
|
||||
priority: medium
|
||||
needs_human: true
|
||||
blocking_reason: "No scanned service declares current E: 4, and apps-pg is recorded as E4 unreachable. An owner must name a service that has a per-tenant credential and a separate per-tenant substrate, the credential class (never the value), the other tenant's substrate, the window, the abort contact, and a complete engagement. Tenancy Posture §13 classifies that demonstration as Mechanical. This task authorizes no connection attempt."
|
||||
```
|
||||
|
||||
Blocked on an owner this repository cannot name. Tenancy Posture Decision
|
||||
5.5 records `apps-pg` as E4 unreachable because the consumer shares one
|
||||
credential. E4 is reachable only at P3 or above. The offline calibration on
|
||||
`WHITEHAT-WP-0011` is not that demonstration. Do not open a connection to
|
||||
finish this task, and do not invent the declaring service.
|
||||
|
||||
## T04 — Live R4 exercise
|
||||
|
||||
```task
|
||||
id: WHITEHAT-WP-0012-T04
|
||||
status: wait
|
||||
priority: medium
|
||||
needs_human: true
|
||||
blocking_reason: "R4 evidence is Adversarial. An owner must name the route (horizon-elapsed or key-destroyed), a complete copy inventory covering live data, backups, and derived copies, the erasure horizon, the window, the abort contact, and a complete engagement. audit-core targets R2 and is not this owner. This task authorizes no store access."
|
||||
```
|
||||
|
||||
Blocked on an owner this repository cannot name. `audit-core` remains an R2
|
||||
target. The offline calibration on `WHITEHAT-WP-0011` does not erase a
|
||||
retained copy. Do not open a store to finish this task.
|
||||
|
||||
## T05 — Live V2, V3, or V4 exercise
|
||||
|
||||
```task
|
||||
id: WHITEHAT-WP-0012-T05
|
||||
status: wait
|
||||
priority: medium
|
||||
needs_human: true
|
||||
blocking_reason: "V2, V3, and V4 are Adversarial and failure-injected. An operator must name the delivering service, the operation the claim covers, the failure domain, the RTO and RPO, the window, the abort contact, and give the separate approval the rules of engagement §1 require for a recovery probe. V1 stays a mechanical exercise in the declaring service. This task authorizes no failure injection."
|
||||
```
|
||||
|
||||
Blocked on an operator record. Decision 4.6 says V is end-to-end and names
|
||||
the operation; the delivering service owns the declaration. Rules of
|
||||
engagement §1 require separate operator approval for a recovery probe, and
|
||||
§4 prohibits denial-of-service outside an approved window. No repository has
|
||||
published that approval. The in-process judge does not substitute for it.
|
||||
Do not terminate an instance, remove a failure domain, or make a region
|
||||
unavailable to finish this task.
|
||||
|
||||
## Dependencies owned elsewhere
|
||||
|
||||
Nothing further is local. T03, T04, and T05 stay `wait`, and this plan stays
|
||||
`blocked`. Whitehat does not edit the owner repositories and does not message
|
||||
a guessed owner.
|
||||
|
||||
1. An owner publishes a service that actually has a per-tenant credential and
|
||||
a separate per-tenant substrate before T03 can start. The mechanical E4
|
||||
demonstration then still needs a complete engagement. No such service is
|
||||
declared in the scanned standard.
|
||||
2. An owner publishes the R4 route and a complete copy inventory before T04
|
||||
can start. `audit-core` is not that owner.
|
||||
3. An operator publishes the failure-injection approval before T05 can start.
|
||||
The approval names the delivering service, operation, failure domain,
|
||||
RTO, RPO, window, and abort contact.
|
||||
|
||||
`WHITEHAT-WP-0006` still owns live E3, P1/P2, and the remaining E2.
|
||||
`WHITEHAT-WP-0008` still owns live ASM. Those requirements are unchanged.
|
||||
Loading…
Add table
Add a link
Reference in a new issue