T05: the lab and its labelled mutation catalogue
lab/app.py (users, tenants, auth, resources, sharing, read/write, revoke, audit), lab/http_api.py (JSON API + browser UI, stdlib only), 20 labelled composable version-stamped mutations, ground-truth matrix. 48 tests pass. Detection against the reference scenario: MECHANICAL 0/10 flagged (correct), DEFECT 6/6, SEMANTIC 2/4 with both inert cases declared. - F-0002: M16 and M18 initially escaped detection entirely. A use case protects exactly what it asserts. Resolved by adding two claims already stated as intent in INTENT.md; the six-mutation catalogue would never have surfaced this. - test-id axis added: stable selectors survive most UI mutations, which would make H-001 trivially false. Mutations now vary on preserves_test_ids so the hypothesis is analysed split by that axis rather than rigged. - M12 (semantic deferred revoke) and M19 (defect race) are behaviourally identical and asserted as such - the discrimination problem as a test. lab/minimal.py removed; superseded by lab/app.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
parent
4d3421ca57
commit
4ddb2f896c
22 changed files with 1091 additions and 207 deletions
|
|
@ -11,12 +11,12 @@
|
|||
| workplan | TD-WP-0001 | active | — | workplans/TD-WP-0001-statehub-bootstrap.md |
|
||||
| workplan | TD-WP-0002 | active | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0001-T01 | done | — | workplans/TD-WP-0001-statehub-bootstrap.md |
|
||||
| task | TD-WP-0001-T02 | wait | — | workplans/TD-WP-0001-statehub-bootstrap.md |
|
||||
| task | TD-WP-0001-T02 | done | — | workplans/TD-WP-0001-statehub-bootstrap.md |
|
||||
| task | TD-WP-0001-T03 | done | — | workplans/TD-WP-0001-statehub-bootstrap.md |
|
||||
| task | TD-WP-0002-T01 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0002-T02 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0002-T03 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0002-T04 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0002-T04 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0002-T05 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0002-T06 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
| task | TD-WP-0002-T07 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md |
|
||||
|
|
|
|||
61
lab/GROUND-TRUTH.md
Normal file
61
lab/GROUND-TRUTH.md
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
# Lab Ground Truth
|
||||
|
||||
**Lab version base:** `lab-0.2.0` · **Catalogue:** 20 mutations · **Scenario:**
|
||||
`scenarios/alice_bob_carol.py`
|
||||
|
||||
Labels are decided by a human from the use case and recorded **before** any run.
|
||||
They are never inferred from behaviour — that is the whole point, and M12/M19
|
||||
below show why.
|
||||
|
||||
| ID | Mutation | Label | Test ids | Reference scenario |
|
||||
|---|---|---|---|---|
|
||||
| M01 | Sharing control moves into a modal | MECHANICAL | preserved | PASS |
|
||||
| M02 | DOM rewritten, test ids not carried forward | MECHANICAL | **dropped** | PASS |
|
||||
| M03 | API renames `resource_id` to `id` | MECHANICAL | preserved | PASS |
|
||||
| M04 | UI labels reworded | MECHANICAL | preserved | PASS |
|
||||
| M05 | Form field order reversed | MECHANICAL | preserved | PASS |
|
||||
| M06 | API paths shortened | MECHANICAL | preserved | PASS |
|
||||
| M07 | Buttons become anchors | MECHANICAL | preserved | PASS |
|
||||
| M08 | Revoke gains a confirmation step | MECHANICAL | preserved | PASS |
|
||||
| M09 | Responses are slower | MECHANICAL | preserved | PASS |
|
||||
| M10 | Denials return 401 instead of 403 | MECHANICAL | preserved | PASS |
|
||||
| M11 | A share must be accepted first | SEMANTIC | preserved | FAIL |
|
||||
| M12 | Revocation is deferred by decision | SEMANTIC | preserved | FAIL |
|
||||
| M13 | Grants default to WRITE | SEMANTIC | preserved | PASS *(inert)* |
|
||||
| M14 | Cross-tenant sharing declared prohibited | SEMANTIC | preserved | PASS *(inert)* |
|
||||
| M15 | Revocation updates record but not enforcement | DEFECT | preserved | FAIL |
|
||||
| M16 | A READ grant confers WRITE | DEFECT | preserved | FAIL |
|
||||
| M17 | Any authenticated user can read anything | DEFECT | preserved | FAIL |
|
||||
| M18 | Revocation is not audited | DEFECT | preserved | FAIL |
|
||||
| M19 | Revocation propagates after a delay | DEFECT | preserved | FAIL |
|
||||
| M20 | Tenant isolation leaks | DEFECT | preserved | FAIL |
|
||||
|
||||
Baseline: PASS. Detection: **MECHANICAL 0/10 flagged** (correct — semantics
|
||||
preserved), **DEFECT 6/6 flagged**, **SEMANTIC 2/4 flagged**.
|
||||
|
||||
## The two inert mutations
|
||||
|
||||
Recorded rather than hidden. `test_inert_semantic_mutations_are_declared` fails
|
||||
if an invisible mutation is ever left undeclared.
|
||||
|
||||
- **M13** only affects grants that omit a permission; the reference scenario
|
||||
passes `READ` explicitly, so nothing changes.
|
||||
- **M14** is a change of *intent* with no change of code — cross-tenant sharing
|
||||
was already enforced, and the mutation declares it deliberate. Nothing
|
||||
observable moves. This is the sharpest available demonstration that
|
||||
classification cannot be a diff.
|
||||
|
||||
## M12 vs M19 — the discrimination problem in one row
|
||||
|
||||
Both produce an identical failure: `c-bob-revoked`, same step, same evidence.
|
||||
One is a deliberate product decision that revocation batches; the other is a
|
||||
propagation race. **No observation distinguishes them.** Only intent does.
|
||||
|
||||
This is why claims require independent provenance (D-06), why `AMBIGUOUS`
|
||||
escalates to a human rather than resolving itself, and why T08's classifier is
|
||||
not permitted to guess.
|
||||
|
||||
## Regenerating
|
||||
|
||||
The matrix is asserted in `tests/test_lab_ground_truth.py`. A moved cell fails
|
||||
the suite: changing the measuring instrument must be a deliberate, reviewed act.
|
||||
BIN
lab/__pycache__/app.cpython-312.pyc
Normal file
BIN
lab/__pycache__/app.cpython-312.pyc
Normal file
Binary file not shown.
BIN
lab/__pycache__/http_api.cpython-312.pyc
Normal file
BIN
lab/__pycache__/http_api.cpython-312.pyc
Normal file
Binary file not shown.
BIN
lab/__pycache__/mutations.cpython-312.pyc
Normal file
BIN
lab/__pycache__/mutations.cpython-312.pyc
Normal file
Binary file not shown.
309
lab/app.py
Normal file
309
lab/app.py
Normal file
|
|
@ -0,0 +1,309 @@
|
|||
"""The test-driver lab — the system under test.
|
||||
|
||||
Grown from the T04 seed (`lab/minimal.py`, now removed). Users belong to
|
||||
tenants, own resources, share them with READ or WRITE permission, and revoke
|
||||
that sharing. Every state change is audited.
|
||||
|
||||
Two access paths exist, and the distinction is the whole point:
|
||||
|
||||
* the **enforcement path** (`request`) is what an actor uses. It checks
|
||||
authorization and can therefore be *wrong* — that is where a seeded
|
||||
authorization defect lives.
|
||||
* the **observation channel** (`ObservationChannel`) reads stored state directly
|
||||
and probes enforcement out-of-band. It is the independent channel required by
|
||||
decision D-07.
|
||||
|
||||
An oracle consulting only stored state would verify test-driver's own
|
||||
reimplementation of the rules rather than the system's enforcement of them. An
|
||||
oracle consulting only enforcement could not notice that record and enforcement
|
||||
disagree. test-driver observes both and treats disagreement as meaningful in
|
||||
itself — that disagreement is the precise signature of an authorization defect
|
||||
which leaves the audit trail looking correct.
|
||||
|
||||
Mutations are applied by `lab.mutations.build_lab`, never by editing this file.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Callable, Literal
|
||||
|
||||
Permission = Literal["READ", "WRITE"]
|
||||
|
||||
BASE_VERSION = "lab-0.2.0"
|
||||
|
||||
|
||||
class Denied(Exception):
|
||||
"""The enforcement path refused the request."""
|
||||
|
||||
def __init__(self, reason: str, status: int = 403) -> None:
|
||||
super().__init__(reason)
|
||||
self.status = status
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class AuditRecord:
|
||||
sequence: int
|
||||
event: str
|
||||
actor_id: str
|
||||
resource_id: str | None = None
|
||||
subject_id: str | None = None
|
||||
permission: str | None = None
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class User:
|
||||
id: str
|
||||
tenant_id: str
|
||||
token: str
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class LabApp:
|
||||
"""In-process resource-sharing service.
|
||||
|
||||
Mutation hooks are plain attributes holding callables or flags. A mutation
|
||||
replaces one, so that every mutated build differs from baseline in exactly
|
||||
one named way and the difference is inspectable at runtime.
|
||||
"""
|
||||
|
||||
version: str = f"{BASE_VERSION}-baseline"
|
||||
applied_mutations: tuple[str, ...] = ()
|
||||
|
||||
users: dict[str, User] = field(default_factory=dict)
|
||||
_tokens: dict[str, str] = field(default_factory=dict)
|
||||
resources: dict[str, dict[str, Any]] = field(default_factory=dict)
|
||||
grants: dict[tuple[str, str], Permission] = field(default_factory=dict)
|
||||
audit: list[AuditRecord] = field(default_factory=list)
|
||||
_seq: int = 0
|
||||
|
||||
# --- mutation hooks (baseline behaviour) ----------------------------
|
||||
may_read: Callable[["LabApp", str, str], bool] | None = None
|
||||
grant_permission_for: Callable[[str | None], Permission] | None = None
|
||||
audit_revoke: bool = True
|
||||
revoke_delay_seconds: float = 0.0
|
||||
denied_status: int = 403
|
||||
require_share_acceptance: bool = False
|
||||
enforce_tenant_isolation: bool = True
|
||||
response_id_field: str = "resource_id"
|
||||
latency_seconds: float = 0.0
|
||||
|
||||
# --- presentation hooks (surface only; never affect domain semantics) ---
|
||||
ui_share_control: str = "inline"
|
||||
ui_dom_style: str = "flat"
|
||||
ui_labels: str = "plain"
|
||||
ui_field_order: str = "natural"
|
||||
ui_button_element: str = "button"
|
||||
ui_test_ids: str = "stable"
|
||||
ui_confirm_revoke: bool = False
|
||||
api_path_style: str = "long"
|
||||
tenant_sharing_announced: bool = False
|
||||
|
||||
# -- setup -----------------------------------------------------------
|
||||
|
||||
def add_user(self, user_id: str, tenant_id: str = "t-acme") -> str:
|
||||
token = f"tok-{user_id}"
|
||||
self.users[user_id] = User(user_id, tenant_id, token)
|
||||
self._tokens[token] = user_id
|
||||
return token
|
||||
|
||||
def _audit(self, event: str, actor_id: str, **kw: Any) -> None:
|
||||
self._seq += 1
|
||||
self.audit.append(AuditRecord(self._seq, event, actor_id, **kw))
|
||||
|
||||
def _whoami(self, token: str) -> str:
|
||||
if token not in self._tokens:
|
||||
raise Denied("unknown token", status=401)
|
||||
return self._tokens[token]
|
||||
|
||||
def tenant_of(self, user_id: str) -> str | None:
|
||||
user = self.users.get(user_id)
|
||||
return user.tenant_id if user else None
|
||||
|
||||
# -- enforcement path (what actors use) ------------------------------
|
||||
|
||||
def request(self, token: str, op: str, **args: Any) -> Any:
|
||||
"""The single entry point actors go through. Authorization is enforced here."""
|
||||
if self.latency_seconds:
|
||||
time.sleep(self.latency_seconds)
|
||||
user_id = self._whoami(token)
|
||||
handler = getattr(self, f"_op_{op}", None)
|
||||
if handler is None:
|
||||
raise Denied(f"unknown operation {op!r}", status=404)
|
||||
return handler(user_id, **args)
|
||||
|
||||
def _op_create_resource(self, user_id: str, resource_id: str, content: str) -> dict:
|
||||
self.resources[resource_id] = {"owner": user_id, "content": content}
|
||||
self._audit("create", user_id, resource_id=resource_id)
|
||||
return {self.response_id_field: resource_id}
|
||||
|
||||
def _op_read_resource(self, user_id: str, resource_id: str) -> dict:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None:
|
||||
raise Denied("no such resource", status=404)
|
||||
if not self._may_read(user_id, resource_id):
|
||||
raise Denied("not authorized to read", status=self.denied_status)
|
||||
return {self.response_id_field: resource_id, "content": resource["content"]}
|
||||
|
||||
def _op_write_resource(self, user_id: str, resource_id: str, content: str) -> dict:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None:
|
||||
raise Denied("no such resource", status=404)
|
||||
if not self._may_write(user_id, resource_id):
|
||||
raise Denied("not authorized to write", status=self.denied_status)
|
||||
resource["content"] = content
|
||||
self._audit("write", user_id, resource_id=resource_id)
|
||||
return {"written": True}
|
||||
|
||||
def _op_grant(
|
||||
self,
|
||||
user_id: str,
|
||||
resource_id: str,
|
||||
subject_id: str,
|
||||
permission: Permission | None = None,
|
||||
) -> dict:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None or resource["owner"] != user_id:
|
||||
raise Denied("only the owner may grant")
|
||||
if self.enforce_tenant_isolation:
|
||||
if self.tenant_of(subject_id) != self.tenant_of(user_id):
|
||||
raise Denied("cross-tenant sharing is not permitted")
|
||||
effective = (
|
||||
self.grant_permission_for(permission)
|
||||
if self.grant_permission_for
|
||||
else (permission or "READ")
|
||||
)
|
||||
self.grants[(resource_id, subject_id)] = effective
|
||||
if self.require_share_acceptance:
|
||||
self.resources[resource_id].setdefault("pending", set()).add(subject_id)
|
||||
self._audit(
|
||||
"grant", user_id, resource_id=resource_id,
|
||||
subject_id=subject_id, permission=effective,
|
||||
)
|
||||
return {"granted": effective}
|
||||
|
||||
def _op_accept_share(self, user_id: str, resource_id: str) -> dict:
|
||||
pending = self.resources.get(resource_id, {}).get("pending")
|
||||
if pending:
|
||||
pending.discard(user_id)
|
||||
self._audit("accept", user_id, resource_id=resource_id, subject_id=user_id)
|
||||
return {"accepted": True}
|
||||
|
||||
def _op_revoke(self, user_id: str, resource_id: str, subject_id: str) -> dict:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None or resource["owner"] != user_id:
|
||||
raise Denied("only the owner may revoke")
|
||||
if self.revoke_delay_seconds:
|
||||
self.resources[resource_id].setdefault("revoke_after", {})[subject_id] = (
|
||||
time.monotonic() + self.revoke_delay_seconds
|
||||
)
|
||||
else:
|
||||
self.grants.pop((resource_id, subject_id), None)
|
||||
if self.audit_revoke:
|
||||
self._audit("revoke", user_id, resource_id=resource_id, subject_id=subject_id)
|
||||
return {"revoked": True}
|
||||
|
||||
def _may_read(self, user_id: str, resource_id: str) -> bool:
|
||||
"""Authorization as the system actually enforces it."""
|
||||
if self.may_read is not None:
|
||||
return self.may_read(self, user_id, resource_id)
|
||||
return self.baseline_may_read(user_id, resource_id)
|
||||
|
||||
def _may_write(self, user_id: str, resource_id: str) -> bool:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None:
|
||||
return False
|
||||
if resource["owner"] == user_id:
|
||||
return True
|
||||
return self.grants.get((resource_id, user_id)) == "WRITE"
|
||||
|
||||
def baseline_may_read(self, user_id: str, resource_id: str) -> bool:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None:
|
||||
return False
|
||||
if resource["owner"] == user_id:
|
||||
return True
|
||||
deadline = resource.get("revoke_after", {}).get(user_id)
|
||||
if deadline is not None and time.monotonic() < deadline:
|
||||
return True # revocation is scheduled but not yet effective
|
||||
if user_id in resource.get("pending", set()):
|
||||
return False
|
||||
return (resource_id, user_id) in self.grants
|
||||
|
||||
|
||||
class ObservationChannel:
|
||||
"""Independent read access to lab state — decision D-07.
|
||||
|
||||
Bypasses authorization deliberately. This is the channel test-driver requires
|
||||
of any system under test, and the main integration burden the framework
|
||||
imposes on an adopter.
|
||||
"""
|
||||
|
||||
def __init__(self, app: LabApp) -> None:
|
||||
self._app = app
|
||||
|
||||
@property
|
||||
def version(self) -> str:
|
||||
return self._app.version
|
||||
|
||||
def state_permission(self, user_id: str, resource_id: str) -> str | None:
|
||||
"""What the stored record says, independent of any enforcement decision."""
|
||||
resource = self._app.resources.get(resource_id)
|
||||
if resource is None:
|
||||
return None
|
||||
if resource["owner"] == user_id:
|
||||
return "OWNER"
|
||||
return self._app.grants.get((resource_id, user_id))
|
||||
|
||||
def probe_read(self, user_id: str, resource_id: str) -> bool:
|
||||
"""Exercise the enforcement path out-of-band and report what it did.
|
||||
|
||||
This uses the subject's own credentials, which can look like a violation
|
||||
of actor isolation but is not: independence means the *actor's report* is
|
||||
never the evidence. The observer issues its own request and records the
|
||||
raw outcome. No actor is ever asked whether it succeeded.
|
||||
"""
|
||||
user = self._app.users.get(user_id)
|
||||
if user is None:
|
||||
return False
|
||||
try:
|
||||
self._app.request(user.token, "read_resource", resource_id=resource_id)
|
||||
except Denied:
|
||||
return False
|
||||
return True
|
||||
|
||||
def probe_write(self, user_id: str, resource_id: str) -> bool:
|
||||
"""Out-of-band probe of the write path, non-destructive on refusal."""
|
||||
user = self._app.users.get(user_id)
|
||||
if user is None:
|
||||
return False
|
||||
original = self._app.resources.get(resource_id, {}).get("content")
|
||||
try:
|
||||
self._app.request(
|
||||
user.token, "write_resource", resource_id=resource_id, content=original
|
||||
)
|
||||
except Denied:
|
||||
return False
|
||||
return True
|
||||
|
||||
def audit_events(self, resource_id: str) -> list[dict[str, Any]]:
|
||||
return [
|
||||
{
|
||||
"sequence": r.sequence,
|
||||
"event": r.event,
|
||||
"actor_id": r.actor_id,
|
||||
"subject_id": r.subject_id,
|
||||
"permission": r.permission,
|
||||
}
|
||||
for r in self._app.audit
|
||||
if r.resource_id == resource_id
|
||||
]
|
||||
|
||||
|
||||
def build_baseline() -> tuple[LabApp, dict[str, str]]:
|
||||
"""Known initial state, so that runs replay from the same starting point."""
|
||||
app = LabApp()
|
||||
tokens = {u: app.add_user(u) for u in ("alice", "bob", "carol")}
|
||||
tokens["mallory"] = app.add_user("mallory", tenant_id="t-other")
|
||||
return app, tokens
|
||||
215
lab/http_api.py
Normal file
215
lab/http_api.py
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
"""HTTP surface for the lab: a JSON API and a minimal browser UI.
|
||||
|
||||
Stdlib only. The point of this layer is to give the agentic browser driver (T07)
|
||||
something real to navigate, and to give the mechanical mutations somewhere to
|
||||
bite — a mutation that moves a control is inert until there is a control to move.
|
||||
|
||||
The domain is not reimplemented here. Every request funnels into
|
||||
`LabApp.request`, so the UI and API cannot drift from the enforcement path that
|
||||
the deterministic driver and the observation channel already exercise.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from typing import Any
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
from .app import Denied, LabApp
|
||||
|
||||
|
||||
def _resource_path(app: LabApp, resource_id: str) -> str:
|
||||
return (
|
||||
f"/r/{resource_id}" if app.api_path_style == "short"
|
||||
else f"/resources/{resource_id}"
|
||||
)
|
||||
|
||||
|
||||
def render_resource_page(app: LabApp, user_id: str, resource_id: str) -> str:
|
||||
"""The browser surface. Every mechanical UI mutation shows up here."""
|
||||
html = _render(app, user_id, resource_id)
|
||||
if app.ui_test_ids == "dropped":
|
||||
# A rewrite that does not carry the old test ids forward. See the
|
||||
# "test-id axis" note in lab/mutations.py — this is the case that
|
||||
# decides whether semantic actions earn their keep.
|
||||
html = _TEST_ID_ATTR.sub("", html)
|
||||
return html
|
||||
|
||||
|
||||
_TEST_ID_ATTR = re.compile(r'\s*data-td="[^"]*"')
|
||||
|
||||
|
||||
def _render(app: LabApp, user_id: str, resource_id: str) -> str:
|
||||
verbose = app.ui_labels == "verbose"
|
||||
share_label = "Give access" if verbose else "Share"
|
||||
revoke_label = "Withdraw access" if verbose else "Revoke"
|
||||
|
||||
tag = "a" if app.ui_button_element == "anchor" else "button"
|
||||
role = ' role="button"' if tag == "a" else ""
|
||||
href = ' href="#"' if tag == "a" else ""
|
||||
|
||||
subject_field = (
|
||||
'<label for="subject">Person</label>'
|
||||
'<input id="subject" name="subject_id" data-td="subject">'
|
||||
)
|
||||
permission_field = (
|
||||
'<label for="permission">Permission</label>'
|
||||
'<select id="permission" name="permission" data-td="permission">'
|
||||
'<option>READ</option><option>WRITE</option></select>'
|
||||
)
|
||||
fields = (
|
||||
permission_field + subject_field
|
||||
if app.ui_field_order == "reversed"
|
||||
else subject_field + permission_field
|
||||
)
|
||||
|
||||
share_form = (
|
||||
f'<form id="share" data-td="share-form" method="post" '
|
||||
f'action="{_resource_path(app, resource_id)}/grant">'
|
||||
f"{fields}"
|
||||
f'<{tag}{href}{role} id="share-submit" data-td="share-submit">'
|
||||
f"{share_label}</{tag}></form>"
|
||||
)
|
||||
if app.ui_share_control == "modal":
|
||||
share_form = (
|
||||
f'<{tag}{href}{role} id="open-share" data-td="open-share">{share_label}…'
|
||||
f"</{tag}>"
|
||||
f'<dialog id="share-modal" data-td="share-modal" open>{share_form}</dialog>'
|
||||
)
|
||||
|
||||
confirm = (
|
||||
'<p data-td="confirm-note">This cannot be undone.</p>'
|
||||
f'<{tag}{href}{role} data-td="revoke-confirm">Yes, {revoke_label.lower()}</{tag}>'
|
||||
if app.ui_confirm_revoke else ""
|
||||
)
|
||||
revoke_form = (
|
||||
f'<form id="revoke" data-td="revoke-form" method="post" '
|
||||
f'action="{_resource_path(app, resource_id)}/revoke">'
|
||||
f'<input name="subject_id" data-td="revoke-subject">'
|
||||
f'<{tag}{href}{role} data-td="revoke-submit">{revoke_label}</{tag}>'
|
||||
f"{confirm}</form>"
|
||||
)
|
||||
|
||||
grants = "".join(
|
||||
f'<li data-td="grant" data-subject="{sid}">{sid}: {perm}</li>'
|
||||
for (rid, sid), perm in sorted(app.grants.items())
|
||||
if rid == resource_id
|
||||
)
|
||||
|
||||
body = (
|
||||
f'<h1 data-td="resource-title">Resource {resource_id}</h1>'
|
||||
f'<ul data-td="grant-list">{grants}</ul>{share_form}{revoke_form}'
|
||||
)
|
||||
if app.ui_dom_style == "nested":
|
||||
body = (
|
||||
'<div class="shell"><section class="panel"><div class="panel-inner">'
|
||||
f"{body}</div></section></div>"
|
||||
)
|
||||
|
||||
return (
|
||||
"<!doctype html><html><head><title>Lab</title></head>"
|
||||
f'<body data-td-version="{app.version}" data-td-user="{user_id}">{body}</body></html>'
|
||||
)
|
||||
|
||||
|
||||
class LabHandler(BaseHTTPRequestHandler):
|
||||
app: LabApp
|
||||
|
||||
def log_message(self, *args: Any) -> None: # keep the test output quiet
|
||||
pass
|
||||
|
||||
# -- helpers ---------------------------------------------------------
|
||||
|
||||
def _token(self) -> str:
|
||||
header = self.headers.get("Authorization", "")
|
||||
return header.removeprefix("Bearer ").strip()
|
||||
|
||||
def _send(self, status: int, payload: Any, content_type: str = "application/json") -> None:
|
||||
body = (
|
||||
json.dumps(payload).encode()
|
||||
if content_type == "application/json"
|
||||
else payload.encode()
|
||||
)
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", content_type)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def _dispatch(self, op: str, **args: Any) -> None:
|
||||
try:
|
||||
self._send(200, self.app.request(self._token(), op, **args))
|
||||
except Denied as denied:
|
||||
self._send(denied.status, {"error": str(denied)})
|
||||
|
||||
def _resource_id(self, path: str) -> str | None:
|
||||
parts = [p for p in path.split("/") if p]
|
||||
if parts and parts[0] in ("resources", "r") and len(parts) >= 2:
|
||||
return parts[1]
|
||||
return None
|
||||
|
||||
# -- routes ----------------------------------------------------------
|
||||
|
||||
def do_GET(self) -> None: # noqa: N802 — stdlib naming
|
||||
url = urlparse(self.path)
|
||||
resource_id = self._resource_id(url.path)
|
||||
if resource_id is None:
|
||||
self._send(404, {"error": "not found"})
|
||||
return
|
||||
if url.path.endswith("/view"):
|
||||
try:
|
||||
user_id = self.app._whoami(self._token())
|
||||
except Denied as denied:
|
||||
self._send(denied.status, {"error": str(denied)})
|
||||
return
|
||||
self._send(200, render_resource_page(self.app, user_id, resource_id), "text/html")
|
||||
return
|
||||
self._dispatch("read_resource", resource_id=resource_id)
|
||||
|
||||
def do_POST(self) -> None: # noqa: N802
|
||||
url = urlparse(self.path)
|
||||
resource_id = self._resource_id(url.path)
|
||||
length = int(self.headers.get("Content-Length", 0))
|
||||
raw = self.rfile.read(length).decode() if length else ""
|
||||
try:
|
||||
body = json.loads(raw) if raw.startswith("{") else {
|
||||
k: v[0] for k, v in parse_qs(raw).items()
|
||||
}
|
||||
except json.JSONDecodeError:
|
||||
self._send(400, {"error": "malformed body"})
|
||||
return
|
||||
|
||||
if resource_id is None:
|
||||
if url.path.rstrip("/") in ("/resources", "/r"):
|
||||
self._dispatch("create_resource", **body)
|
||||
return
|
||||
self._send(404, {"error": "not found"})
|
||||
return
|
||||
|
||||
if url.path.endswith("/grant"):
|
||||
self._dispatch("grant", resource_id=resource_id, **body)
|
||||
elif url.path.endswith("/revoke"):
|
||||
self._dispatch("revoke", resource_id=resource_id, **body)
|
||||
elif url.path.endswith("/accept"):
|
||||
self._dispatch("accept_share", resource_id=resource_id)
|
||||
else:
|
||||
self._send(404, {"error": "not found"})
|
||||
|
||||
|
||||
def serve(app: LabApp, port: int = 0) -> ThreadingHTTPServer:
|
||||
"""Start a server on `port` (0 picks a free one). Caller owns shutdown."""
|
||||
handler = type("BoundLabHandler", (LabHandler,), {"app": app})
|
||||
return ThreadingHTTPServer(("127.0.0.1", port), handler)
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover - manual use
|
||||
import sys
|
||||
from .mutations import build_lab
|
||||
|
||||
lab, tokens = build_lab(*sys.argv[1:])
|
||||
lab.request(tokens["alice"], "create_resource", resource_id="R", content="the secret")
|
||||
server = serve(lab, 8099)
|
||||
print(f"{lab.version} on http://127.0.0.1:8099 tokens={tokens}")
|
||||
server.serve_forever()
|
||||
191
lab/minimal.py
191
lab/minimal.py
|
|
@ -1,191 +0,0 @@
|
|||
"""Minimal lab: the seed of the system under test.
|
||||
|
||||
Deliberately small. Users own resources, share them with read or write
|
||||
permission, and revoke that sharing. Every state change is audited.
|
||||
|
||||
Two access paths exist, and the distinction is the whole point:
|
||||
|
||||
* the **enforcement path** (`request`) is what an actor uses. It checks
|
||||
authorization and can therefore be *wrong* — that is where a seeded
|
||||
authorization defect lives.
|
||||
* the **observation channel** (`ObservationChannel`) reads stored state directly,
|
||||
without authorization. It is the independent channel required by decision
|
||||
D-07.
|
||||
|
||||
An oracle that consulted only stored state would verify the framework's own
|
||||
reimplementation of the rules rather than the system's enforcement of them. An
|
||||
oracle that consulted only the enforcement path would have no way to notice that
|
||||
enforcement and record disagree. test-driver observes both, and treats
|
||||
disagreement between them as meaningful in its own right — that disagreement is
|
||||
the precise signature of the M05 authorization defect.
|
||||
|
||||
TD-WP-0002-T05 grows this into the full lab with an HTTP API, a browser UI and
|
||||
the labelled mutation catalogue. It is kept in-process here so that T04 can prove
|
||||
the kernel without dragging in a web stack.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Literal
|
||||
|
||||
Permission = Literal["READ", "WRITE"]
|
||||
|
||||
|
||||
class Denied(Exception):
|
||||
"""The enforcement path refused the request."""
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class AuditRecord:
|
||||
sequence: int
|
||||
event: str
|
||||
actor_id: str
|
||||
resource_id: str | None = None
|
||||
subject_id: str | None = None
|
||||
permission: str | None = None
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class MinimalLab:
|
||||
"""In-process resource-sharing service."""
|
||||
|
||||
version: str = "lab-0.1.0-baseline"
|
||||
users: dict[str, str] = field(default_factory=dict) # user_id -> token
|
||||
_tokens: dict[str, str] = field(default_factory=dict) # token -> user_id
|
||||
resources: dict[str, dict[str, Any]] = field(default_factory=dict)
|
||||
grants: dict[tuple[str, str], Permission] = field(default_factory=dict)
|
||||
audit: list[AuditRecord] = field(default_factory=list)
|
||||
_seq: int = 0
|
||||
|
||||
# -- setup -----------------------------------------------------------
|
||||
|
||||
def add_user(self, user_id: str) -> str:
|
||||
token = f"tok-{user_id}"
|
||||
self.users[user_id] = token
|
||||
self._tokens[token] = user_id
|
||||
return token
|
||||
|
||||
def _audit(self, event: str, actor_id: str, **kw: Any) -> None:
|
||||
self._seq += 1
|
||||
self.audit.append(AuditRecord(self._seq, event, actor_id, **kw))
|
||||
|
||||
def _whoami(self, token: str) -> str:
|
||||
if token not in self._tokens:
|
||||
raise Denied("unknown token")
|
||||
return self._tokens[token]
|
||||
|
||||
# -- enforcement path (what actors use) ------------------------------
|
||||
|
||||
def request(self, token: str, op: str, **args: Any) -> Any:
|
||||
"""The single entry point actors go through. Authorization is enforced here."""
|
||||
user_id = self._whoami(token)
|
||||
handler = getattr(self, f"_op_{op}", None)
|
||||
if handler is None:
|
||||
raise Denied(f"unknown operation {op!r}")
|
||||
return handler(user_id, **args)
|
||||
|
||||
def _op_create_resource(self, user_id: str, resource_id: str, content: str) -> dict:
|
||||
self.resources[resource_id] = {"owner": user_id, "content": content}
|
||||
self._audit("create", user_id, resource_id=resource_id)
|
||||
return {"resource_id": resource_id}
|
||||
|
||||
def _op_read_resource(self, user_id: str, resource_id: str) -> dict:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None:
|
||||
raise Denied("no such resource")
|
||||
if not self._may_read(user_id, resource_id):
|
||||
raise Denied("not authorized to read")
|
||||
return {"resource_id": resource_id, "content": resource["content"]}
|
||||
|
||||
def _op_grant(
|
||||
self, user_id: str, resource_id: str, subject_id: str, permission: Permission
|
||||
) -> dict:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None or resource["owner"] != user_id:
|
||||
raise Denied("only the owner may grant")
|
||||
self.grants[(resource_id, subject_id)] = permission
|
||||
self._audit(
|
||||
"grant", user_id, resource_id=resource_id,
|
||||
subject_id=subject_id, permission=permission,
|
||||
)
|
||||
return {"granted": permission}
|
||||
|
||||
def _op_revoke(self, user_id: str, resource_id: str, subject_id: str) -> dict:
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None or resource["owner"] != user_id:
|
||||
raise Denied("only the owner may revoke")
|
||||
self.grants.pop((resource_id, subject_id), None)
|
||||
self._audit("revoke", user_id, resource_id=resource_id, subject_id=subject_id)
|
||||
return {"revoked": True}
|
||||
|
||||
def _may_read(self, user_id: str, resource_id: str) -> bool:
|
||||
"""The authorization rule as the system actually enforces it."""
|
||||
resource = self.resources.get(resource_id)
|
||||
if resource is None:
|
||||
return False
|
||||
if resource["owner"] == user_id:
|
||||
return True
|
||||
return (resource_id, user_id) in self.grants
|
||||
|
||||
|
||||
class ObservationChannel:
|
||||
"""Independent read access to lab state — decision D-07.
|
||||
|
||||
Bypasses authorization deliberately. This is the channel test-driver requires
|
||||
of any system under test, and the main integration burden the framework
|
||||
imposes on an adopter.
|
||||
"""
|
||||
|
||||
def __init__(self, lab: MinimalLab) -> None:
|
||||
self._lab = lab
|
||||
|
||||
@property
|
||||
def version(self) -> str:
|
||||
return self._lab.version
|
||||
|
||||
def state_permission(self, user_id: str, resource_id: str) -> str | None:
|
||||
"""What the stored record says, independent of any enforcement decision."""
|
||||
resource = self._lab.resources.get(resource_id)
|
||||
if resource is None:
|
||||
return None
|
||||
if resource["owner"] == user_id:
|
||||
return "OWNER"
|
||||
return self._lab.grants.get((resource_id, user_id))
|
||||
|
||||
def probe_read(self, user_id: str, resource_id: str) -> bool:
|
||||
"""Exercise the enforcement path out-of-band and report what it did.
|
||||
|
||||
This uses the subject's own credentials, which can look like a violation
|
||||
of actor isolation but is not: independence means the *actor's report* is
|
||||
never the evidence. The observer issues its own request and records the
|
||||
raw outcome. No actor is ever asked whether it succeeded.
|
||||
"""
|
||||
token = self._lab.users.get(user_id)
|
||||
if token is None:
|
||||
return False
|
||||
try:
|
||||
self._lab.request(token, "read_resource", resource_id=resource_id)
|
||||
except Denied:
|
||||
return False
|
||||
return True
|
||||
|
||||
def audit_events(self, resource_id: str) -> list[dict[str, Any]]:
|
||||
return [
|
||||
{
|
||||
"sequence": r.sequence,
|
||||
"event": r.event,
|
||||
"actor_id": r.actor_id,
|
||||
"subject_id": r.subject_id,
|
||||
"permission": r.permission,
|
||||
}
|
||||
for r in self._lab.audit
|
||||
if r.resource_id == resource_id
|
||||
]
|
||||
|
||||
|
||||
def build_baseline() -> tuple[MinimalLab, dict[str, str]]:
|
||||
"""Known initial state, so that runs replay from the same starting point."""
|
||||
lab = MinimalLab()
|
||||
tokens = {user: lab.add_user(user) for user in ("alice", "bob", "carol")}
|
||||
return lab, tokens
|
||||
205
lab/mutations.py
Normal file
205
lab/mutations.py
Normal file
|
|
@ -0,0 +1,205 @@
|
|||
"""The labelled mutation catalogue — the project's measuring instrument.
|
||||
|
||||
Every claim test-driver makes is measured against this catalogue, so its quality
|
||||
caps the credibility of every downstream result. Six mutations, as the milestones
|
||||
document originally sketched, cannot support any statement about precision or
|
||||
recall; there are twenty here.
|
||||
|
||||
Each mutation carries a **ground-truth label**, decided by a human from the use
|
||||
case and recorded before any run:
|
||||
|
||||
MECHANICAL the surface changed; protected semantics are identical.
|
||||
test-driver should recover and report an adaptation.
|
||||
SEMANTIC intended behaviour genuinely changed. test-driver must escalate
|
||||
to a human and must never rewrite a claim by itself.
|
||||
DEFECT the system violates unchanged intent. test-driver must report a
|
||||
Product Finding and must never adapt to it.
|
||||
|
||||
The distinction between SEMANTIC and DEFECT is deliberately *not* inferable from
|
||||
the code — both change behaviour. It is a statement about intent, which is why
|
||||
claims must have independent provenance (D-06) and why ambiguity escalates
|
||||
rather than resolving itself.
|
||||
|
||||
Mutations compose: `build_lab("M01", "M15")` applies both and records both in the
|
||||
version string, so a mechanical change shipping alongside a defect is
|
||||
reproducible. That combination is decision-table row 3 and the case a naive
|
||||
self-healing tool gets wrong.
|
||||
|
||||
## The test-id axis
|
||||
|
||||
The UI carries stable `data-td` attributes, as a well-instrumented application
|
||||
would. Most mechanical mutations preserve them, and a recorded selector sequence
|
||||
keyed on those attributes survives such a mutation untouched — which would make
|
||||
H-001 (semantic actions outlast recorded sequences) trivially *false*.
|
||||
|
||||
That is not a flaw to be rigged away. It is the honest shape of the question:
|
||||
**a semantic action earns its keep exactly when stable identifiers are absent or
|
||||
not carried forward.** Mutations therefore vary along `preserves_test_ids`, and
|
||||
H-001 must be analysed split by that axis rather than as a single rate. A
|
||||
catalogue whose mutations all broke naive selectors would flatter the thesis and
|
||||
tell us nothing.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable, Literal
|
||||
|
||||
from .app import BASE_VERSION, LabApp, ObservationChannel, build_baseline
|
||||
|
||||
__all__ = [
|
||||
"CATALOGUE", "BY_ID", "Mutation", "ObservationChannel", "build_lab",
|
||||
"expected_classification",
|
||||
]
|
||||
|
||||
Label = Literal["MECHANICAL", "SEMANTIC", "DEFECT"]
|
||||
Layer = Literal["ui", "api", "domain"]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Mutation:
|
||||
id: str
|
||||
title: str
|
||||
label: Label
|
||||
layer: Layer
|
||||
rationale: str
|
||||
apply: Callable[[LabApp], None]
|
||||
preserves_test_ids: bool = True
|
||||
|
||||
|
||||
def _m(app: LabApp, **flags) -> None:
|
||||
for key, value in flags.items():
|
||||
setattr(app, key, value)
|
||||
|
||||
|
||||
# --- MECHANICAL -----------------------------------------------------------
|
||||
# The surface moves. What the system means does not.
|
||||
|
||||
def _m01(app): _m(app, ui_share_control="modal")
|
||||
def _m02(app): _m(app, ui_dom_style="nested", ui_test_ids="dropped")
|
||||
def _m03(app): _m(app, response_id_field="id")
|
||||
def _m04(app): _m(app, ui_labels="verbose")
|
||||
def _m05(app): _m(app, ui_field_order="reversed")
|
||||
def _m06(app): _m(app, api_path_style="short")
|
||||
def _m07(app): _m(app, ui_button_element="anchor")
|
||||
def _m08(app): _m(app, ui_confirm_revoke=True)
|
||||
def _m09(app): _m(app, latency_seconds=0.05)
|
||||
def _m10(app): _m(app, denied_status=401)
|
||||
|
||||
|
||||
# --- SEMANTIC -------------------------------------------------------------
|
||||
# Intended behaviour changed. A human must decide; test-driver must not.
|
||||
|
||||
def _m11(app): _m(app, require_share_acceptance=True)
|
||||
def _m12(app): _m(app, revoke_delay_seconds=3600.0)
|
||||
def _m13(app): _m(app, grant_permission_for=lambda p: p or "WRITE")
|
||||
def _m14(app): _m(app, enforce_tenant_isolation=True, tenant_sharing_announced=True)
|
||||
|
||||
|
||||
# --- DEFECT ---------------------------------------------------------------
|
||||
# Unchanged intent, violated.
|
||||
|
||||
def _revoke_is_cosmetic(app: LabApp, user_id: str, resource_id: str) -> bool:
|
||||
"""Record and audit say revoked; enforcement still allows the read."""
|
||||
resource = app.resources.get(resource_id)
|
||||
if resource is None:
|
||||
return False
|
||||
if resource["owner"] == user_id:
|
||||
return True
|
||||
return any(
|
||||
r.event == "grant" and r.subject_id == user_id and r.resource_id == resource_id
|
||||
for r in app.audit
|
||||
)
|
||||
|
||||
|
||||
def _read_permits_everyone(app: LabApp, user_id: str, resource_id: str) -> bool:
|
||||
return resource_id in app.resources
|
||||
|
||||
|
||||
def _ignores_tenant(app: LabApp, user_id: str, resource_id: str) -> bool:
|
||||
return app.baseline_may_read(user_id, resource_id) or resource_id in app.resources
|
||||
|
||||
|
||||
def _m15(app): _m(app, may_read=_revoke_is_cosmetic)
|
||||
def _m16(app): _m(app, grant_permission_for=lambda p: "WRITE")
|
||||
def _m17(app): _m(app, may_read=_read_permits_everyone)
|
||||
def _m18(app): _m(app, audit_revoke=False)
|
||||
def _m19(app): _m(app, revoke_delay_seconds=2.0)
|
||||
def _m20(app): _m(app, enforce_tenant_isolation=False, may_read=_ignores_tenant)
|
||||
|
||||
|
||||
CATALOGUE: tuple[Mutation, ...] = (
|
||||
Mutation("M01", "Sharing control moves into a modal", "MECHANICAL", "ui",
|
||||
"Same action, different place. The canonical mechanical change.", _m01),
|
||||
Mutation("M02", "DOM rewritten, test ids not carried forward", "MECHANICAL", "ui",
|
||||
"Selectors break; nothing the user can do changes. Unlike M01 this "
|
||||
"drops the stable test ids, which is what a real rewrite usually "
|
||||
"does.", _m02, preserves_test_ids=False),
|
||||
Mutation("M03", "API renames resource_id to id", "MECHANICAL", "api",
|
||||
"Compatible representation change of the same field.", _m03),
|
||||
Mutation("M04", "UI labels reworded", "MECHANICAL", "ui",
|
||||
"'Share' becomes 'Give access'. Wording, not meaning.", _m04),
|
||||
Mutation("M05", "Form field order reversed", "MECHANICAL", "ui",
|
||||
"Permission precedes subject. Same form, same result.", _m05),
|
||||
Mutation("M06", "API paths shortened", "MECHANICAL", "api",
|
||||
"/resources/{id} becomes /r/{id}.", _m06),
|
||||
Mutation("M07", "Buttons become anchors", "MECHANICAL", "ui",
|
||||
"Element type changes; affordance does not.", _m07),
|
||||
Mutation("M08", "Revoke gains a confirmation step", "MECHANICAL", "ui",
|
||||
"An extra legitimate click. The postcondition is unchanged, so this "
|
||||
"is mechanical — but it is the closest mechanical case to a semantic "
|
||||
"one, and a good test of where the boundary really sits.", _m08),
|
||||
Mutation("M09", "Responses are slower", "MECHANICAL", "api",
|
||||
"Latency only. Tests patience, not semantics.", _m09),
|
||||
Mutation("M10", "Denials return 401 instead of 403", "MECHANICAL", "api",
|
||||
"Both mean refused. A driver keying on the exact code breaks.", _m10),
|
||||
|
||||
Mutation("M11", "A share must be accepted before it takes effect", "SEMANTIC", "domain",
|
||||
"Bob genuinely cannot read until he accepts. The old claim 'Bob can "
|
||||
"read after the grant' is now wrong, and only a human may say so.", _m11),
|
||||
Mutation("M12", "Revocation is deferred, not immediate", "SEMANTIC", "domain",
|
||||
"A deliberate product decision that revocation batches. Looks exactly "
|
||||
"like M19 from the outside; the difference is intent.", _m12),
|
||||
Mutation("M13", "Grants default to WRITE", "SEMANTIC", "domain",
|
||||
"Only affects grants that omit a permission. Inert for the reference "
|
||||
"scenario, which passes READ explicitly — recorded as such rather "
|
||||
"than quietly dropped.", _m13),
|
||||
Mutation("M14", "Cross-tenant sharing is announced as prohibited", "SEMANTIC", "domain",
|
||||
"Codifies existing enforcement as intended behaviour. A change of "
|
||||
"intent with no change of code — the case that shows classification "
|
||||
"cannot be a diff.", _m14),
|
||||
|
||||
Mutation("M15", "Revocation updates the record but not enforcement", "DEFECT", "domain",
|
||||
"Audit trail and stored grant both say revoked; the read still "
|
||||
"succeeds. The defect that looks correct to an operator.", _m15),
|
||||
Mutation("M16", "A READ grant confers WRITE", "DEFECT", "domain",
|
||||
"Privilege escalation through the ordinary sharing path.", _m16),
|
||||
Mutation("M17", "Any authenticated user can read any resource", "DEFECT", "domain",
|
||||
"Authorization effectively removed. Carol reads R.", _m17),
|
||||
Mutation("M18", "Revocation is not audited", "DEFECT", "domain",
|
||||
"Enforcement is correct; the evidence trail is not. Detectable only "
|
||||
"because audit is observed, not assumed.", _m18),
|
||||
Mutation("M19", "Revocation propagates after a delay", "DEFECT", "domain",
|
||||
"Not a decision — a race. Indistinguishable from M12 by behaviour "
|
||||
"alone, which is exactly the point.", _m19),
|
||||
Mutation("M20", "Tenant isolation leaks", "DEFECT", "domain",
|
||||
"A user of another tenant reads the resource.", _m20),
|
||||
)
|
||||
|
||||
BY_ID: dict[str, Mutation] = {m.id: m for m in CATALOGUE}
|
||||
|
||||
|
||||
def expected_classification(mutation_id: str) -> Label:
|
||||
"""Ground truth. Recorded by a human before any run — never inferred."""
|
||||
return BY_ID[mutation_id].label
|
||||
|
||||
|
||||
def build_lab(*mutation_ids: str) -> tuple[LabApp, dict[str, str]]:
|
||||
"""Build a lab with the named mutations applied, versioned by what it carries."""
|
||||
app, tokens = build_baseline()
|
||||
for mutation_id in mutation_ids:
|
||||
BY_ID[mutation_id].apply(app)
|
||||
app.applied_mutations = tuple(mutation_ids)
|
||||
suffix = "+".join(mutation_ids) if mutation_ids else "baseline"
|
||||
app.version = f"{BASE_VERSION}-{suffix}"
|
||||
return app, tokens
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Concept ↔ Implementation Fitness Map
|
||||
|
||||
**Updated:** 2026-08-22 (TD-WP-0002-T04)
|
||||
**Updated:** 2026-08-22 (TD-WP-0002-T05)
|
||||
|
||||
Traces each important concept to the implementation, experiment and evidence that
|
||||
support it. **Unsupported entries are the point of this map** — a concept with no
|
||||
|
|
@ -29,7 +29,7 @@ were aspirational, not evidenced.
|
|||
| `C-semantic-action` | C1 | `actions.py` | E-001 | — | Does identity survive restructuring better than a recorded sequence? (H-001) |
|
||||
| `C-oracle-independence` | C1 | `runner.py`, `oracles.py` | E-001, E-003 | — | Independence of components ≠ independence of belief. (H-004) |
|
||||
| `C-evidence-pack` | C1 | `evidence.py` | — | — | What is the minimum sufficient for replay? |
|
||||
| `C-observation-channel` | C1 | `lab/minimal.py` | — | — | **D-07** — required of every system under test. Adoption cost unknown. |
|
||||
| `C-observation-channel` | C1 | `lab/app.py` | — | — | **D-07** — required of every system under test. Adoption cost unknown. |
|
||||
| `C-adaptation` | C1 | — (T08) | E-001 | — | (H-002) |
|
||||
| `C-classification` | C1 | — (T08) | E-001, E-003 | — | Decision table is total on paper; unexercised. |
|
||||
| `C-crystallization` | C1 | — (T09) | E-002 | — | (H-003) |
|
||||
|
|
@ -41,7 +41,7 @@ were aspirational, not evidenced.
|
|||
| `C-campaign` | C0 | — | — | — | Deferred. |
|
||||
| `C-metabolism` | C0 | — | — | — | Deferred. Depends on C-energy. |
|
||||
| `C-retirement` | C0 | — | — | — | Deferred. Depends on C-energy. |
|
||||
| `C-security-mutation` | C1 | — | E-003 | — | Catalogue exists; no derivation mechanism. |
|
||||
| `C-security-mutation` | C1 | `lab/mutations.py` | E-003 | `lab/GROUND-TRUTH.md` | Catalogue is hand-written; no derivation mechanism from use cases yet. |
|
||||
|
||||
## Orphan check
|
||||
|
||||
|
|
|
|||
74
research/findings/F-0002-scenario-coverage-gap.md
Normal file
74
research/findings/F-0002-scenario-coverage-gap.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
---
|
||||
id: F-0002
|
||||
type: framework-finding
|
||||
class: FRAMEWORK_LIMITATION
|
||||
status: resolved
|
||||
discovered: "2026-08-22"
|
||||
resolved: "2026-08-22"
|
||||
discovered_by: TD-WP-0002-T05
|
||||
workplan: TD-WP-0002
|
||||
task: TD-WP-0002-T05
|
||||
---
|
||||
|
||||
# F-0002 — Two seeded defects were invisible to the reference scenario
|
||||
|
||||
## Observation
|
||||
|
||||
On first running the reference scenario against the twenty labelled mutations,
|
||||
**six of six DEFECT mutations should have failed; only four did.**
|
||||
|
||||
| Mutation | Defect | Verdict before | Why it escaped |
|
||||
|---|---|---|---|
|
||||
| M16 | A READ grant confers WRITE | `PASS` | No claim mentioned writing. The scenario never attempted one. |
|
||||
| M18 | Revocation is not audited | `PASS` | `i-audit-append-only` checks *ordering*, not *completeness*. A trail missing an entry is still ordered. |
|
||||
|
||||
Both passed cleanly. Nothing was flaky, nothing was ambiguous, and no oracle
|
||||
reported `INCONCLUSIVE` — the framework simply had nothing to say, confidently.
|
||||
|
||||
## Why it matters
|
||||
|
||||
This is the failure mode most likely to be mistaken for success. A green run
|
||||
against a lab carrying a seeded privilege escalation looks exactly like a green
|
||||
run against a correct system. Had the catalogue been the six mutations the
|
||||
milestones document originally sketched, this would not have surfaced at all —
|
||||
which is the concrete argument for the larger catalogue, now made from evidence
|
||||
rather than from assertion.
|
||||
|
||||
It also sharpens what a verification asset is: **a use case protects exactly what
|
||||
it asserts, and not one thing more.** Coverage is a property of the claim set, not
|
||||
of the framework. No amount of adaptation, crystallization or energy scoring
|
||||
compensates for an assertion nobody wrote.
|
||||
|
||||
## Resolution
|
||||
|
||||
Path 1 — the implementation changes to match the concept.
|
||||
|
||||
Two claims were added to the reference use case, both `Provenance.HUMAN` and both
|
||||
derivable from `INTENT.md` rather than from watching the lab:
|
||||
|
||||
- `c-bob-cannot-write` — "A READ grant does not let Bob write R".
|
||||
`INTENT.md` § Security by Use-Case Mutation already derives this exact question
|
||||
from the reference use case: *"Can Bob write when only read permission was
|
||||
granted?"* It was always part of what sharing means; it had simply never been
|
||||
written down as an assertion.
|
||||
- `c-revoke-audited` — "Revocation is recorded in the audit trail". Enforcement
|
||||
being correct is not sufficient: an access change nobody can later evidence is
|
||||
a compliance failure even when the access itself is right.
|
||||
|
||||
Supporting changes: the lab gained a write path and the observation channel a
|
||||
non-destructive `probe_write`.
|
||||
|
||||
Both defects are now detected. DEFECT detection is 6/6, and
|
||||
`test_every_defect_is_detected` fails the suite if that ever regresses.
|
||||
|
||||
## Note on provenance
|
||||
|
||||
These claims were added *after* observing that mutations escaped, which is
|
||||
uncomfortably close to fitting assertions to the lab. They are admissible because
|
||||
both were already stated as intent in `INTENT.md` before any lab existed — the
|
||||
finding revealed a transcription gap, not a new requirement. Had the intended
|
||||
behaviour not already been on record, the correct resolution would have been to
|
||||
escalate to a human, not to write the claim.
|
||||
|
||||
That distinction is exactly what `Provenance` exists to make checkable, and this
|
||||
finding is the first case where it did real work.
|
||||
Binary file not shown.
|
|
@ -19,7 +19,7 @@ from testdriver import (
|
|||
Scenario, SemanticAction, StateObserver, Step, UseCase,
|
||||
VerificationAsset, Watch, World,
|
||||
)
|
||||
from lab.minimal import ObservationChannel, build_baseline
|
||||
from lab.mutations import ObservationChannel, build_lab
|
||||
|
||||
RESOURCE = "R"
|
||||
API = frozenset({"api"})
|
||||
|
|
@ -40,6 +40,25 @@ def _carol_cannot_read(obs) -> bool:
|
|||
return obs["probe_read:carol:R"] is False
|
||||
|
||||
|
||||
def _bob_cannot_write(obs) -> bool:
|
||||
"""A READ grant must not confer write.
|
||||
|
||||
INTENT.md derives exactly this question from the reference use case:
|
||||
"Can Bob write when only read permission was granted?" It is part of what
|
||||
sharing means, not an extra.
|
||||
"""
|
||||
return obs["probe_write:bob:R"] is False
|
||||
|
||||
|
||||
def _revocation_is_audited(obs) -> bool:
|
||||
"""Revocation must leave a trace.
|
||||
|
||||
Enforcement being correct is not sufficient. An access change nobody can
|
||||
later evidence is a compliance failure even when the access itself is right.
|
||||
"""
|
||||
return any(event["event"] == "revoke" for event in obs["audit:R"])
|
||||
|
||||
|
||||
def _enforcement_matches_record(obs) -> bool:
|
||||
"""Enforcement and stored record must agree about every watched subject.
|
||||
|
||||
|
|
@ -77,8 +96,13 @@ USE_CASE = UseCase(
|
|||
Provenance.HUMAN, _bob_can_read, after_step="s2-grant"),
|
||||
Claim("c-carol-denied", "Carol can never read R",
|
||||
Provenance.HUMAN, _carol_cannot_read, after_step="s2-grant"),
|
||||
Claim("c-bob-cannot-write", "A READ grant does not let Bob write R",
|
||||
Provenance.HUMAN, _bob_cannot_write, after_step="s2-grant",
|
||||
source_ref="INTENT.md#security-by-use-case-mutation"),
|
||||
Claim("c-bob-revoked", "Bob cannot read R after revocation",
|
||||
Provenance.HUMAN, _bob_cannot_read, after_step="s3-revoke"),
|
||||
Claim("c-revoke-audited", "Revocation is recorded in the audit trail",
|
||||
Provenance.HUMAN, _revocation_is_audited, after_step="s3-revoke"),
|
||||
),
|
||||
invariants=(
|
||||
Invariant("i-enforcement-matches-record",
|
||||
|
|
@ -90,9 +114,15 @@ USE_CASE = UseCase(
|
|||
)
|
||||
|
||||
|
||||
def build(variant: str = "baseline"):
|
||||
"""Assemble world, driver, observer and asset from a known initial state."""
|
||||
lab, tokens = build_baseline()
|
||||
def build(*mutations: str):
|
||||
"""Assemble world, driver, observer and asset from a known initial state.
|
||||
|
||||
`mutations` names entries from the lab catalogue. The same scenario runs
|
||||
unchanged against every lab version — that is the point: the use case does
|
||||
not know the implementation moved.
|
||||
"""
|
||||
lab, tokens = build_lab(*mutations)
|
||||
variant = "+".join(mutations) if mutations else "baseline"
|
||||
cast = Cast()
|
||||
for name in ("alice", "bob", "carol"):
|
||||
cast.add(Actor(id=name, display_name=name.title(),
|
||||
|
|
|
|||
Binary file not shown.
|
|
@ -41,6 +41,9 @@ class StateObserver:
|
|||
out[f"state_permission:{key}"] = self.channel.state_permission(
|
||||
watch.subject_id, watch.resource_id
|
||||
)
|
||||
out[f"probe_write:{key}"] = self.channel.probe_write(
|
||||
watch.subject_id, watch.resource_id
|
||||
)
|
||||
resources.add(watch.resource_id)
|
||||
for resource_id in sorted(resources):
|
||||
out[f"audit:{resource_id}"] = self.channel.audit_events(resource_id)
|
||||
|
|
|
|||
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -15,7 +15,7 @@ from testdriver import (
|
|||
Claim, InadmissibleProvenance, Invariant, Oracle, Provenance, Runner,
|
||||
SemanticAction, StateObserver, SurfaceNotPermitted, Stratum, Verdict,
|
||||
)
|
||||
from lab.minimal import Denied, MinimalLab, ObservationChannel, build_baseline
|
||||
from lab.app import Denied, LabApp, ObservationChannel, build_baseline
|
||||
from scenarios.alice_bob_carol import build
|
||||
|
||||
|
||||
|
|
@ -83,7 +83,7 @@ def test_actors_cannot_be_recorded_as_judgment_collectors():
|
|||
# --- preview of the M05 authorization defect (built properly in T05) --------
|
||||
|
||||
|
||||
class RevokeIsCosmetic(MinimalLab):
|
||||
class RevokeIsCosmetic(LabApp):
|
||||
"""Revocation updates the record and the audit trail but not enforcement.
|
||||
|
||||
This is the shape of a real authorization defect: everything an operator
|
||||
|
|
@ -101,6 +101,10 @@ class RevokeIsCosmetic(MinimalLab):
|
|||
for r in self.audit
|
||||
)
|
||||
|
||||
# NOTE: this predates the mutation catalogue, where the same defect is M15.
|
||||
# Kept as a direct subclass so the kernel test does not depend on the lab
|
||||
# catalogue's wiring being correct.
|
||||
|
||||
|
||||
def test_seeded_authorization_defect_fails_the_run():
|
||||
"""The kernel must report FAIL, not adapt, when revocation does not revoke."""
|
||||
|
|
@ -124,7 +128,8 @@ def test_seeded_authorization_defect_fails_the_run():
|
|||
assert result.verdict is Verdict.FAIL
|
||||
assert result.judgment("c-bob-revoked").verdict is Verdict.FAIL
|
||||
# The claim set is untouched by the failure — there is no path to adapt it.
|
||||
assert USE_CASE.claims[2].text == "Bob cannot read R after revocation"
|
||||
by_id = {c.id: c for c in USE_CASE.claims}
|
||||
assert by_id["c-bob-revoked"].text == "Bob cannot read R after revocation"
|
||||
|
||||
|
||||
def test_defect_run_emits_an_energy_event():
|
||||
|
|
|
|||
148
tests/test_lab_ground_truth.py
Normal file
148
tests/test_lab_ground_truth.py
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
"""The lab is the measuring instrument. These tests keep it honest.
|
||||
|
||||
Nothing here tests the kernel's cleverness — the kernel has no classifier yet.
|
||||
What is established is the **ground truth** every later measurement is taken
|
||||
against: which mutations the reference scenario responds to, and how.
|
||||
|
||||
The response matrix below is an asserted fact, not a snapshot. If a change to the
|
||||
lab or the scenario moves a cell, that is a change to the measuring instrument
|
||||
and must be a deliberate, reviewed act.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from testdriver import Runner, Verdict
|
||||
from lab.mutations import BY_ID, CATALOGUE, build_lab, expected_classification
|
||||
from scenarios.alice_bob_carol import build
|
||||
|
||||
|
||||
def run_against(*mutations: str):
|
||||
world, driver, observer, asset, oracle = build(*mutations)
|
||||
return Runner(world, driver, observer, oracle).run(asset)
|
||||
|
||||
|
||||
# --- the catalogue itself -------------------------------------------------
|
||||
|
||||
|
||||
def test_catalogue_is_large_enough_to_support_a_rate():
|
||||
"""Six mutations cannot support precision or recall. Twenty can begin to."""
|
||||
assert len(CATALOGUE) >= 20
|
||||
|
||||
|
||||
def test_every_mutation_is_labelled_and_reasoned():
|
||||
for mutation in CATALOGUE:
|
||||
assert mutation.label in ("MECHANICAL", "SEMANTIC", "DEFECT")
|
||||
assert mutation.rationale.strip(), f"{mutation.id} has no rationale"
|
||||
|
||||
|
||||
def test_labels_cover_all_three_classes_with_useful_weight():
|
||||
counts = {label: 0 for label in ("MECHANICAL", "SEMANTIC", "DEFECT")}
|
||||
for mutation in CATALOGUE:
|
||||
counts[mutation.label] += 1
|
||||
assert all(count >= 4 for count in counts.values()), counts
|
||||
|
||||
|
||||
def test_every_mutation_is_reproducible_and_version_stamped():
|
||||
for mutation in CATALOGUE:
|
||||
first, _ = build_lab(mutation.id)
|
||||
second, _ = build_lab(mutation.id)
|
||||
assert first.version == second.version == f"lab-0.2.0-{mutation.id}"
|
||||
assert first.applied_mutations == (mutation.id,)
|
||||
|
||||
|
||||
def test_mutations_compose_and_record_both():
|
||||
"""Row 3 of the decision table needs a lab carrying both at once."""
|
||||
app, _ = build_lab("M01", "M15")
|
||||
assert app.version == "lab-0.2.0-M01+M15"
|
||||
assert app.applied_mutations == ("M01", "M15")
|
||||
|
||||
|
||||
def test_test_id_axis_is_represented():
|
||||
"""H-001 must be analysable split by whether stable selectors survived."""
|
||||
preserved = [m.id for m in CATALOGUE if m.preserves_test_ids]
|
||||
dropped = [m.id for m in CATALOGUE if not m.preserves_test_ids]
|
||||
assert preserved and dropped, "both sides of the test-id axis must exist"
|
||||
|
||||
|
||||
# --- the response matrix --------------------------------------------------
|
||||
|
||||
# Ground truth: what the reference scenario reports for each lab version.
|
||||
# `None` means "no assertion in this scenario covers this mutation" — recorded
|
||||
# honestly rather than papered over.
|
||||
EXPECTED_VERDICT = {
|
||||
"M01": Verdict.PASS, "M02": Verdict.PASS, "M03": Verdict.PASS,
|
||||
"M04": Verdict.PASS, "M05": Verdict.PASS, "M06": Verdict.PASS,
|
||||
"M07": Verdict.PASS, "M08": Verdict.PASS, "M09": Verdict.PASS,
|
||||
"M10": Verdict.PASS,
|
||||
"M11": Verdict.FAIL, "M12": Verdict.FAIL,
|
||||
"M13": Verdict.PASS, "M14": Verdict.PASS,
|
||||
"M15": Verdict.FAIL, "M16": Verdict.FAIL, "M17": Verdict.FAIL,
|
||||
"M18": Verdict.FAIL, "M19": Verdict.FAIL, "M20": Verdict.FAIL,
|
||||
}
|
||||
|
||||
# The two SEMANTIC mutations the reference scenario cannot see, and why.
|
||||
KNOWN_INERT = {
|
||||
"M13": "only affects grants that omit a permission; the scenario passes READ explicitly",
|
||||
"M14": "a change of intent with no change of code; nothing observable moved",
|
||||
}
|
||||
|
||||
|
||||
def test_baseline_passes():
|
||||
assert run_against().verdict is Verdict.PASS
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mutation_id", sorted(EXPECTED_VERDICT))
|
||||
def test_response_matrix_is_stable(mutation_id):
|
||||
assert run_against(mutation_id).verdict is EXPECTED_VERDICT[mutation_id]
|
||||
|
||||
|
||||
def test_no_mechanical_mutation_changes_the_verdict():
|
||||
"""Semantics are preserved, so the use case must not notice."""
|
||||
for mutation in CATALOGUE:
|
||||
if mutation.label == "MECHANICAL":
|
||||
assert run_against(mutation.id).verdict is Verdict.PASS, mutation.id
|
||||
|
||||
|
||||
def test_every_defect_is_detected():
|
||||
"""The floor of the whole project. A defect the lab cannot surface is a
|
||||
defect no later classifier can be measured against."""
|
||||
missed = [
|
||||
m.id for m in CATALOGUE
|
||||
if m.label == "DEFECT" and run_against(m.id).verdict is Verdict.PASS
|
||||
]
|
||||
assert missed == [], f"undetected seeded defects: {missed}"
|
||||
|
||||
|
||||
def test_inert_semantic_mutations_are_declared():
|
||||
"""A mutation the scenario cannot see must be named, not silently ignored."""
|
||||
for mutation in CATALOGUE:
|
||||
if mutation.label != "SEMANTIC":
|
||||
continue
|
||||
if run_against(mutation.id).verdict is Verdict.PASS:
|
||||
assert mutation.id in KNOWN_INERT, (
|
||||
f"{mutation.id} is invisible to the reference scenario and "
|
||||
"undeclared — either cover it or record why not"
|
||||
)
|
||||
|
||||
|
||||
def test_deferred_revoke_and_revoke_race_are_behaviourally_identical():
|
||||
"""M12 (SEMANTIC) and M19 (DEFECT) must be indistinguishable from evidence.
|
||||
|
||||
This is the discrimination problem in its sharpest form, and the reason
|
||||
classification cannot be a diff over observed behaviour. Both produce the
|
||||
same failure; only intent separates them, which is why claims need
|
||||
independent provenance and why ambiguity escalates to a human.
|
||||
"""
|
||||
semantic = run_against("M12")
|
||||
defect = run_against("M19")
|
||||
failed = lambda r: sorted({j.assertion_id for j in r.judgments
|
||||
if j.verdict is not Verdict.PASS})
|
||||
assert failed(semantic) == failed(defect) == ["c-bob-revoked"]
|
||||
assert expected_classification("M12") != expected_classification("M19")
|
||||
|
||||
|
||||
def test_a_mechanical_change_shipping_with_a_defect_still_fails():
|
||||
"""Decision-table row 3: coincidence is not exoneration."""
|
||||
assert run_against("M01", "M15").verdict is Verdict.FAIL
|
||||
|
|
@ -10,8 +10,8 @@ from testdriver import Runner, Stratum, Verdict
|
|||
from scenarios.alice_bob_carol import build
|
||||
|
||||
|
||||
def run_once(variant: str = "baseline"):
|
||||
world, driver, observer, asset, oracle = build(variant)
|
||||
def run_once(*mutations: str):
|
||||
world, driver, observer, asset, oracle = build(*mutations)
|
||||
return Runner(world, driver, observer, oracle).run(asset), world
|
||||
|
||||
|
||||
|
|
@ -52,7 +52,7 @@ def test_evidence_is_stratified_and_serializable():
|
|||
assert pack.of_stratum(Stratum.JUDGMENT)
|
||||
parsed = json.loads(pack.to_json())
|
||||
assert parsed["run_id"] == result.run_id
|
||||
assert parsed["sut_version"] == "lab-0.1.0-baseline"
|
||||
assert parsed["sut_version"] == "lab-0.2.0-baseline"
|
||||
|
||||
|
||||
def test_evidence_records_claim_provenance():
|
||||
|
|
|
|||
|
|
@ -222,7 +222,7 @@ Three things that came out of building it rather than designing it:
|
|||
|
||||
```task
|
||||
id: TD-WP-0002-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "595a89c2-1462-57fc-8b08-a5a6b875fd48"
|
||||
```
|
||||
|
|
@ -240,6 +240,31 @@ The lab is the measuring instrument for every claim the framework makes — a we
|
|||
lab caps the credibility of all downstream results. It is also potentially the
|
||||
project's first publishable artefact in its own right.
|
||||
|
||||
**Done 2026-08-22.** `lab/app.py` (users, tenants, auth, resources, sharing,
|
||||
read/write, revoke, audit), `lab/http_api.py` (JSON API + browser UI, stdlib
|
||||
only), `lab/mutations.py` (20 labelled, composable, version-stamped mutations),
|
||||
`lab/GROUND-TRUTH.md`. 48 tests pass. Detection: MECHANICAL 0/10 flagged,
|
||||
DEFECT 6/6, SEMANTIC 2/4.
|
||||
|
||||
Three results worth carrying:
|
||||
|
||||
- **F-0002 — two seeded defects were initially invisible.** M16 (READ grant
|
||||
confers WRITE) and M18 (revocation unaudited) both passed cleanly: nothing
|
||||
flaky, nothing `INCONCLUSIVE`, the framework simply had nothing to say. A use
|
||||
case protects exactly what it asserts and not one thing more. Resolved by
|
||||
adding two claims already stated as intent in `INTENT.md`. Had the six-mutation
|
||||
catalogue from the milestones doc been used, this would never have surfaced —
|
||||
the argument for the larger catalogue is now evidenced rather than asserted.
|
||||
- **The test-id axis.** Stable `data-td` selectors survive most UI mutations,
|
||||
which would make H-001 trivially *false*. Rather than rig the catalogue,
|
||||
mutations now vary on `preserves_test_ids`, and H-001 must be analysed split by
|
||||
that axis. A semantic action earns its keep exactly where stable identifiers
|
||||
are absent or not carried forward — that is the honest shape of the claim.
|
||||
- **M12 vs M19 are behaviourally identical.** A deliberate deferred-revocation
|
||||
decision and a revocation race produce the same failure, same step, same
|
||||
evidence. Only intent separates them. This is the discrimination problem in its
|
||||
sharpest form and is now a test, not a paragraph.
|
||||
|
||||
## Out-of-band ground truth for self-verification
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue