T04: deterministic semantic kernel
Alice/Bob/Carol runs end to end, deterministically, replayable from seed. 16 tests pass, no third-party dependencies. - src/testdriver: intent, provenance, world, actions, drivers, observers, oracles, evidence, energy, scenario, runner - lab/minimal.py: the SUT, exposing the independent observation channel required by D-07 - evidence is stratified S1/S2/S3; Runner refuses to attribute S2/S3 to an actor; claims are frozen and provenance-checked at construction - missing evidence yields INCONCLUSIVE, which outranks PASS in the run verdict - EnergyEvents captured, no scoring (H-005 dormant) The observation channel records both stored state and an out-of-band enforcement probe; their disagreement is an invariant and is what detects an authorization defect that leaves the audit trail intact. A seeded RevokeIsCosmetic lab fails the run via both the claim and that invariant. Also closes TD-WP-0001-T02 (stack and commands now exist). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
parent
8da4c5bf7a
commit
04e9573b5a
42 changed files with 1533 additions and 20 deletions
86
src/testdriver/evidence.py
Normal file
86
src/testdriver/evidence.py
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
"""Stratified evidence — decision D-01.
|
||||
|
||||
S1 Surface how an action was performed (selectors, routes, payloads)
|
||||
S2 Realization whether it happened, and through which surface
|
||||
S3 Judgment whether that was correct
|
||||
|
||||
Each stratum has a different authority. Models may write S1. Nothing but an
|
||||
independent Observer writes S2 or S3.
|
||||
|
||||
See docs/TestDriverClassificationDesign.md, Part A.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field, asdict
|
||||
from datetime import datetime, timezone
|
||||
from enum import Enum
|
||||
from typing import Any
|
||||
|
||||
|
||||
class Stratum(str, Enum):
|
||||
SURFACE = "S1"
|
||||
REALIZATION = "S2"
|
||||
JUDGMENT = "S3"
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Observation:
|
||||
"""One recorded fact, attributed to a stratum and a collector.
|
||||
|
||||
`collector` is never an actor for S2/S3 observations. The runner enforces
|
||||
this; see runner._assert_collector_independence.
|
||||
"""
|
||||
|
||||
id: str
|
||||
stratum: Stratum
|
||||
collector: str
|
||||
step_id: str | None
|
||||
kind: str
|
||||
data: dict[str, Any]
|
||||
at: str = field(default_factory=_now)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class EvidencePack:
|
||||
"""Everything retained from one run.
|
||||
|
||||
The pack must be sufficient to replay the run and to diagnose a finding
|
||||
without the original process. An assertion that cannot be supported from the
|
||||
pack is an EVIDENCE_FAILURE, not a defect in the system under test.
|
||||
"""
|
||||
|
||||
run_id: str
|
||||
scenario_id: str
|
||||
use_case_id: str
|
||||
sut_version: str
|
||||
started_at: str = field(default_factory=_now)
|
||||
finished_at: str | None = None
|
||||
observations: list[Observation] = field(default_factory=list)
|
||||
verdicts: list[dict[str, Any]] = field(default_factory=list)
|
||||
energy_events: list[dict[str, Any]] = field(default_factory=list)
|
||||
provenance_index: dict[str, str] = field(default_factory=dict)
|
||||
|
||||
def record(self, observation: Observation) -> None:
|
||||
self.observations.append(observation)
|
||||
|
||||
def of_stratum(self, stratum: Stratum) -> list[Observation]:
|
||||
return [o for o in self.observations if o.stratum is stratum]
|
||||
|
||||
def latest(self, kind: str) -> Observation | None:
|
||||
for observation in reversed(self.observations):
|
||||
if observation.kind == kind:
|
||||
return observation
|
||||
return None
|
||||
|
||||
def to_json(self) -> str:
|
||||
payload = asdict(self)
|
||||
payload["observations"] = [
|
||||
{**asdict(o), "stratum": o.stratum.value} for o in self.observations
|
||||
]
|
||||
return json.dumps(payload, indent=2, sort_keys=True, default=str)
|
||||
Loading…
Add table
Add a link
Reference in a new issue