test-driver/research/hypotheses/H-001-semantic-action-stability.md

---
id: H-001
title: Semantic Action Stability
status: EXPERIMENTING
created: "2026-08-22"
experiments: [E-001]
concepts: [C-semantic-action]
---

# H-001 — Semantic Action Stability

## Claim

A semantic action survives implementation restructuring better than a recorded UI
interaction sequence.

## Falsification condition

Across the labelled mechanical mutations in the lab (T05), a recorded interaction
sequence survives **at least as many** mutations as the semantic action does.

If mechanics-free identity buys no measurable durability, the central abstraction
is decorative and `SemanticAction` should be reduced to a naming convention.

## Measurement

Mechanical Recovery Rate for each of two arms over the same mutation set:
- **arm A** — semantic action realized by an agentic driver;
- **arm B** — a recorded selector-based sequence captured against the baseline.

Arm B is a genuine control and must be run, not assumed to fail.

## Threats to validity

The comparison is unfair if arm B is built naively — a brittle straw man makes
H-001 trivially true and worthless. Arm B uses the most robust selector strategy
reasonably available (roles, labels, test ids where the lab provides them).

## Result so far (TD-WP-0002-T07)

Twelve mechanical mutations, both arms, same semantic action:

| | Discovery (agentic) | Recorded selectors (control) |
|---|---|---|
| test ids preserved (9) | 9/9 | **9/9** |
| test ids dropped (3) | 2/3 | 0/3 |

**Not falsified, but substantially narrowed.** Where stable identifiers survive,
the control arm matches the agentic arm exactly — the semantic action buys
nothing. It earns its keep only where identifiers are absent or not carried
forward.

Three mutations on the deciding side is directionally clear and statistically
nothing. See `research/findings/F-0005-...` — the concept model overstates this
and should be revised to match the evidence.

Scope caveat (F-0004): the driver tests **structural** durability only. Visual
relayout is untested.

## Status log

- 2026-08-22 `PROPOSED`. No evidence.
- 2026-08-22 `EXPERIMENTING`. Partial evidence from T07; awaiting E-001 in full.