test-driver/research/experiments/E-001-mechanical-vs-defect.md
tegwick 84848e9a0e T08: the classifier, measured and attacked
False Adaptation Rate = 0/7 across the labelled catalogue and the three E-003
attacks. 11 of 12 mechanical mutations absorbed without a human, so the safety
result is not bought by escalating everything.

- classification.py: total function over three signals, rule order chosen so
  every rule that could excuse a regression sits after the rule that reports
  one. SAFE_TO_ACCEPT is a two-element closed set, asserted.
- CompositeDriver plus scenarios/full_journey.py: one asset crossing both
  surfaces, so UI mutations are visible as surface differences while the
  claims they do not touch stay green.
- E-003: surface substitution (new M23), concurrent mechanical+defect,
  evidence starvation, provenance laundering. All held.

F-0006 (CONCEPT_DRIFT, resolved): the T02 design listed SEMANTIC_CHANGE as an
outcome the table could produce. It cannot - M12 and M19 are behaviourally
identical, as the lab has asserted since T05. PRODUCT_DEFECT and
SEMANTIC_CHANGE collapse into one escalating outcome, BEHAVIOUR_CHANGED, and
the distinction becomes a human adjudication. INTENT_CHANGED survives but is
detected by the claim fingerprint moving, not inferred from behaviour.

Two classifier defects found and fixed rather than reported: claims downstream
of a failed realization now yield INCONCLUSIVE rather than FAIL (a false
accusation is the mirror image of a false adaptation), and the browser driver
records a page signature so surface change is detectable when the interaction
path is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-23 00:02:58 +02:00

1.2 KiB

id title status hypotheses task created
E-001 Mechanical recovery and defect discrimination over the labelled mutation set PLANNED
H-001
H-002
H-004
TD-WP-0002-T08 2026-08-22

E-001 — Mechanical recovery and defect discrimination

Design

Run the same verification asset against every labelled lab mutation (T05), in two arms:

  • arm A — semantic action realized by the agentic driver;
  • arm B — a recorded selector-based sequence captured against the baseline, built with the most robust selector strategy reasonably available.

Arm B is the control for H-001 and must actually be run.

Outcomes recorded per run

surface diff (S1) · postcondition met and surface used (S2) · oracle verdicts (S3) · classification emitted · claim/invariant diff (expected empty) · tokens · wall time · retries.

Decision rules

  • False Adaptation Rate > 0 → stop. H-004 falsified; redesign, do not tune.
  • Arm B recovery ≥ arm A recovery → H-001 falsified.
  • Recovery achieved with a non-empty claim diff → H-002 falsified and a framework defect, since D-02 forbids the write path.

Status

EXECUTED 2026-08-22 (T08). Arm A run over 23 mutations; arm B over the 12 mechanical ones. Results in H-001 and H-004. FAR 0/7.