test-driver/workplans/TD-WP-0002-vertical-spike-crystallization.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated
TD-WP-0002	workplan	Vertical spike: falsify the crystallization thesis	infotech	test-driver	proposed	codex	custodian	2026-08-22	2026-08-22

Vertical spike: falsify the crystallization thesis

Why this workplan exists

The concept corpus (INTENT.md, docs/) describes eleven milestones that build four complete layers — research control plane, deterministic kernel, lab, self-verification — before the central thesis is exercised even once. The repo currently holds ~3,000 lines of theory and zero lines of executable code.

This workplan inverts that order. It drives one thin thread end to end through every layer of the model, so that the thesis can be supported or falsified cheaply and early:

one use case → deterministic kernel → labelled mutation lab
  → agentic realisation of one semantic action
  → adaptation classification → one crystallization

Reasoning and the full assessment behind this sequencing: history/2026-08-22-concept-assessment-swot.md.

Success gate

The spike succeeds when, in one coherent run:

1. agentic realisation recovers from lab mutations M01 (moved/renamed control) and M02 (restructured DOM) with zero changes to claims or invariants;
2. lab mutation M05 (authorization defect) is reported as a Product Finding, not adapted to;
3. one verification asset crystallizes into a deterministic test that runs with no model involvement and preserves the relevant oracles;
4. False Adaptation Rate = 0 across the labelled mutation set.

Failing this gate early and cheaply is a valid and valuable outcome. Passing it is worth more than any other ten items on the M0–M10 roadmap.

Explicitly deferred

Not in this workplan, by decision rather than omission — each is cheap to build, satisfying to build, and impossible to validate at this stage:

• Energy scoring, decay, and campaign selection (raw immutable EnergyEvents are recorded from the first run; no scoring logic is implemented);
• Temperature and Confidence as computed values;
• Test Metabolism, Campaigns, Retirement floors;
• Lineage graph storage beyond a parent pointer;
• the five agentic roles (Builder / Experimenter / Critic / Auditor / Maintainer);
• any surface beyond one HTTP API and one browser UI.

Constraints

• Stack: Python, pytest, Playwright, Pydantic/dataclasses, YAML, SQLite. One process, one database, one browser engine, one application under test.
• Novelty belongs in the verification model, never in the infrastructure.
• No LLM judgment where a deterministic oracle is available (Concept Model §2.3).

---

Reconcile milestone numbering and commit the concept corpus

id: TD-WP-0002-T01
status: todo
priority: high

Three documents carry three different milestone sequences: INTENT.md (M0–M3), docs/TestDriverInitialMilestones.md (M0–M10), and docs/TestDriverConceptModel.md §15 (a third M0–M3). This is a CONCEPT_DRIFT finding by the project's own taxonomy, present before any code exists.

Pick one canonical sequence, mark the others superseded in place, and commit INTENT.md and docs/ to git (currently untracked). Record the drift as the first entry in the framework findings log — the project should catch this on itself before it catches anything else.

Decide intent provenance and the classification mechanism on paper

id: TD-WP-0002-T02
status: todo
priority: high

Two design questions gate everything downstream. Answer both in a written design note plus a State Hub decision record, before writing kernel code.

(a) The classifier. What deterministic evidence separates "the button moved" from "Bob can still read after revoke"? Specify the signal, not the intent. If the honest answer is "a model decides", that violates Concept Model §2.3 and must be resolved now rather than discovered at M6.

(b) Intent provenance. If an agent writes the implementation, an agent writes the use case, and an agent realises the test, then "oracles independent from actors" is procedurally true but epistemically thin. Record whether use cases are human-authored, spec-derived, or agent-generated, and what independence guarantee survives in each case.

Deliverable: docs/TestDriverClassificationDesign.md + one recorded decision.

Minimal research control plane

id: TD-WP-0002-T03
status: todo
priority: medium

Timebox: half a day. Five hypothesis files (H-001…H-005) each with an explicit falsification condition, one Concept ↔ Implementation Fitness Map, and a stable identifier convention. Plain files under research/.

No CLI, no register tooling, no schema. The control plane is overhead until there are readings to record.

Deterministic semantic kernel

id: TD-WP-0002-T04
status: todo
priority: high

Minimal executable representations of UseCase, Actor, World, Scenario, SemanticAction, Observation, Evidence, Oracle, Verdict, VerificationAsset, Run. One deterministic driver (HTTP or direct adapter).

Run the reference use case end to end:

Alice owns resource R. Alice grants Bob READ. Bob can read R.
Carol cannot read R. Alice revokes Bob. Bob can no longer read R.

Exit: Alice, Bob and Carol hold genuinely isolated identities and sessions; all outcomes are judged by independent deterministic oracles; a run emits a structured Evidence Pack; the scenario replays from known initial state.

Emit raw EnergyEvent records from this point onward. Implement no scoring.

Test-driver lab with labelled ground truth

id: TD-WP-0002-T05
status: todo
priority: high

A deliberately small application: users, tenants, auth, resources, sharing, read/write permissions, revoke, audit history, HTTP API, minimal browser UI.

Build 15–20 labelled mutations, not the six sketched in the milestones doc. Six cannot support any statement about precision or recall. Each mutation is a reproducible toggle carrying a recorded expected classification (MECHANICAL / SEMANTIC / DEFECT), with baseline and mutated versions carrying explicit version identifiers.

The lab is the measuring instrument for every claim the framework makes — a weak lab caps the credibility of all downstream results. It is also potentially the project's first publishable artefact in its own right.

Out-of-band ground truth for self-verification

id: TD-WP-0002-T06
status: todo
priority: medium

Write the actor-isolation and oracle-independence checks as plain pytest against observable behaviour, outside the framework. Using test-driver to verify that test-driver's oracles are independent is a system certifying itself.

Exit: deliberately breaking actor isolation fails the out-of-band check; allowing an actor to determine its own verdict fails oracle-independence; each failure produces a Framework Finding.

Agentic realisation of one semantic action, fully instrumented

id: TD-WP-0002-T07
status: todo
priority: high

Browser driver plus one agentic actor runtime, realising exactly one semantic action — grant_access(Bob, R, READ) — from intent, against the lab UI. Strict per-actor context isolation; bounded navigation and tool permissions; full action and evidence recording; agent/model/configuration identity captured in evidence. Oracles stay deterministic.

Instrument cost and nondeterminism from the very first run: tokens, wall time, retry count, and run-to-run variance. If agentic realisation costs more per run than simply asking an agent to rewrite the broken test, the crystallization argument is an aesthetic preference rather than a value proposition. This data is free to collect from run one and impossible to backfill.

Adaptation detection and the defect-vs-adaptation classifier

id: TD-WP-0002-T08
status: todo
priority: high

The centrepiece. Implement the mechanism designed in T02 and run it across the full labelled mutation set from T05.

Classification path: IMPLEMENTATION CHANGE / INTENT CHANGE / PRODUCT DEFECT / AMBIGUOUS, with escalation for semantic changes and ambiguity. Preserve semantic action identity across adaptation; record adaptation history on the verification asset; never modify claims or invariants to accommodate an observed behaviour.

Measure: Mechanical Recovery Rate, classification precision/recall, and False Adaptation Rate. The last is the project's existential safety metric — one publicly normalised authorization defect kills the thesis permanently. Target is zero across the set, and a non-zero result is a stop-and-redesign signal, not a tuning exercise.

Crystallize one asset into deterministic regression

id: TD-WP-0002-T09
status: todo
priority: high

Semantic action trajectory capture, stable-realisation detection, deterministic candidate generation, candidate comparison against the existing oracle set, and lineage from the agentic ancestor to the deterministic descendant. Manual acceptance step for now.

Emit ordinary pytest/Playwright code. Output that drops into a CI system which already exists lets a user adopt the result without adopting the framework.

Exit: the generated test runs with zero agentic involvement, preserves the relevant claims and oracles, retains visible lineage, and measurably costs less to execute than the agentic ancestor.

Gate review and first compression pass

id: TD-WP-0002-T10
status: todo
priority: medium

Evaluate the four success-gate criteria against collected evidence and write the result up in history/ regardless of outcome.

Then run the compression questions from Milestone 10 while the spike is still small: which concepts have no supporting evidence, which abstractions have no conceptual justification, which metadata never informed a decision, what did the experiments falsify. Mark rejected concepts as rejected rather than silently retaining them, and update the Concept ↔ Implementation Fitness Map.

Convert at least one framework finding discovered during T04–T09 into a permanent self-verification or deterministic regression — that closes the first genuine Concept–Implementation Fitness Loop.