T07: agentic realization over a stdlib browser surface
Two decisions taken with the operator: stdlib HTML driver instead of Playwright (F-0004), and a deterministic discovery runtime instead of a live model. Both sit behind interfaces so the alternatives drop in later. - html.py: stdlib DOM parse and query - agentic.py: DiscoveryRuntime (agentic arm, ignores data-td by construction) and RecordedSelectorRuntime (control arm, uses the strongest identifier the page offers) - browser.py: per-actor sessions over real HTTP, constructed per call so no actor inherits another's connection state - cost/nondeterminism metrics recorded from the first run F-0005 (CONCEPT_DRIFT): the H-001 result is a narrowing. Where test ids are preserved, discovery 9/9 and recorded selectors 9/9 - the semantic action buys nothing. Where they are dropped, discovery 2/3 and recorded 0/3. The concept model presents semantic actions as generally superior; the evidence says conditionally superior. M21 and M22 added mid-task: the deciding side of the axis was N=1. M22 (field names renamed) defeats the heuristic and is the first concrete evidence that a live model would add capability, not just cost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
parent
925ff2dd91
commit
44faf3de8e
23 changed files with 1008 additions and 9 deletions
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
id: F-0004
|
||||
type: framework-finding
|
||||
class: FRAMEWORK_LIMITATION
|
||||
status: open
|
||||
discovered: "2026-08-22"
|
||||
discovered_by: TD-WP-0002-T07
|
||||
workplan: TD-WP-0002
|
||||
task: TD-WP-0002-T07
|
||||
carried_to: TD-WP-0002-T10
|
||||
---
|
||||
|
||||
# F-0004 — The browser surface is driven without a browser
|
||||
|
||||
## Decision
|
||||
|
||||
`TD-WP-0002-T07` names Playwright as the browser driver. It was not used.
|
||||
Instead, `src/testdriver/html.py` parses the lab's server-rendered HTML with
|
||||
`html.parser`, and `src/testdriver/browser.py` drives it over real HTTP.
|
||||
|
||||
Reasons, in order of weight:
|
||||
|
||||
1. **The lab's surface has no JavaScript.** It is server-rendered forms. A
|
||||
browser engine would not change what H-001 or H-002 can be measured against —
|
||||
the mutations that matter are DOM restructuring, rewording and field renaming,
|
||||
all of which a parser faces in full.
|
||||
2. **Establishing Playwright is a real cost to the operator** — installation plus
|
||||
a licence review, and the usual snag is the Chromium binaries the installer
|
||||
pulls rather than the library licence itself. Blocking the vertical spike on
|
||||
that was not worth it for a surface that gains nothing from it.
|
||||
3. **The stack is deliberately boring.** Zero dependencies remains true.
|
||||
|
||||
`Driver` is a Protocol, so a Playwright implementation sits alongside this one
|
||||
without touching the kernel, the oracles or the evidence format.
|
||||
|
||||
## What this costs — stated, not buried
|
||||
|
||||
The driver cannot:
|
||||
|
||||
- execute JavaScript, so **no single-page-application surface can be driven**;
|
||||
- take screenshots, so that evidence type listed in `INTENT.md` is unavailable;
|
||||
- read an accessibility tree, or observe visual layout at all.
|
||||
|
||||
The third is the most consequential for the thesis. A real mechanical change
|
||||
often moves a control *visually* while leaving the DOM largely intact, and this
|
||||
driver is blind to that entire class. H-001's mutation set is therefore narrower
|
||||
than the hypothesis's wording implies: it tests **structural** durability, not
|
||||
**visual** durability.
|
||||
|
||||
That should be said plainly whenever the H-001 result is quoted.
|
||||
|
||||
## Carried to T10
|
||||
|
||||
Two questions, neither answerable now:
|
||||
|
||||
1. Does a Playwright driver actually change the H-001 result, or merely widen the
|
||||
surface it can reach? Worth one experiment, not a rewrite.
|
||||
2. Is the screenshot evidence type in `INTENT.md` load-bearing, or was it listed
|
||||
because screenshots are conventional in this space? If nothing has needed one
|
||||
by T10, that is a candidate for compression.
|
||||
Loading…
Add table
Add a link
Reference in a new issue