test-driver/workplans/TD-WP-0002-vertical-spike-crystallization.md

---
id: TD-WP-0002
type: workplan
title: "Vertical spike: falsify the crystallization thesis"
domain: infotech
repo: test-driver
status: active
owner: codex
topic_slug: custodian
created: "2026-08-22"
updated: "2026-08-22"
state_hub_workstream_id: "68b252b8-03e7-55c7-8dec-ac13f6ca0aa1"
---

# Vertical spike: falsify the crystallization thesis

## Why this workplan exists

The concept corpus (`INTENT.md`, `docs/`) describes eleven milestones that build
four complete layers — research control plane, deterministic kernel, lab,
self-verification — before the central thesis is exercised even once. The repo
currently holds ~3,000 lines of theory and zero lines of executable code.

This workplan inverts that order. It drives **one thin thread end to end**
through every layer of the model, so that the thesis can be supported or
falsified cheaply and early:

```
one use case → deterministic kernel → labelled mutation lab
  → agentic realisation of one semantic action
  → adaptation classification → one crystallization
```

Reasoning and the full assessment behind this sequencing:
`history/2026-08-22-concept-assessment-swot.md`.

## Success gate

The spike succeeds when, in one coherent run:

1. agentic realisation recovers from lab mutations **M01** (moved/renamed
   control) and **M02** (restructured DOM) with **zero** changes to claims or
   invariants;
2. lab mutation **M05** (authorization defect) is reported as a **Product
   Finding**, not adapted to;
3. one verification asset crystallizes into a deterministic test that runs with
   **no model involvement** and preserves the relevant oracles;
4. **False Adaptation Rate = 0** across the labelled mutation set.

Failing this gate early and cheaply is a valid and valuable outcome. Passing it
is worth more than any other ten items on the M0–M10 roadmap.

## Explicitly deferred

Not in this workplan, by decision rather than omission — each is cheap to build,
satisfying to build, and impossible to validate at this stage:

- Energy scoring, decay, and campaign selection (raw immutable `EnergyEvent`s are
  recorded from the first run; **no** scoring logic is implemented);
- Temperature and Confidence as computed values;
- Test Metabolism, Campaigns, Retirement floors;
- Lineage graph storage beyond a parent pointer;
- the five agentic roles (Builder / Experimenter / Critic / Auditor / Maintainer);
- any surface beyond one HTTP API and one browser UI.

## Constraints

- Stack: Python, pytest, Playwright, Pydantic/dataclasses, YAML, SQLite.
  One process, one database, one browser engine, one application under test.
- Novelty belongs in the verification model, never in the infrastructure.
- No LLM judgment where a deterministic oracle is available (Concept Model §2.3).
- Evidence stratification S1/S2/S3 and claim provenance per
  `docs/TestDriverClassificationDesign.md` (decision
  `fef5213f-ce9b-44c2-b327-a0b0ba4b6270`). Adaptation writes only to S1.

---

## Reconcile milestone numbering and commit the concept corpus

```task
id: TD-WP-0002-T01
status: done
priority: high
state_hub_task_id: "b1755e17-279a-56cf-86eb-359e9739070a"
```

Three documents carry three different milestone sequences: `INTENT.md` (M0–M3),
`docs/TestDriverInitialMilestones.md` (M0–M10), and
`docs/TestDriverConceptModel.md` §15 (a third M0–M3). This is a CONCEPT_DRIFT
finding by the project's own taxonomy, present before any code exists.

Pick one canonical sequence, mark the others superseded in place, and commit
`INTENT.md` and `docs/` to git (currently untracked). Record the drift as the
first entry in the framework findings log — the project should catch this on
itself before it catches anything else.

**Done 2026-08-22.** `docs/TestDriverInitialMilestones.md` (M0–M10) is canonical;
the `INTENT.md` and Concept Model § 15 sequences carry superseded banners with a
mapping table. Recorded as `research/findings/F-0001-milestone-sequence-drift.md`.
`INTENT.md` and `docs/` are now tracked. One residual carried to T10: the finding
taxonomy has no class for document-to-document drift.

## Decide intent provenance and the classification mechanism on paper

```task
id: TD-WP-0002-T02
status: done
priority: high
state_hub_task_id: "e5776cf5-0475-5dc6-90c3-db772ce24d63"
```

Two design questions gate everything downstream. Answer both in a written design
note plus a State Hub decision record, **before** writing kernel code.

**(a) The classifier.** What *deterministic evidence* separates "the button
moved" from "Bob can still read after revoke"? Specify the signal, not the
intent. If the honest answer is "a model decides", that violates Concept Model
§2.3 and must be resolved now rather than discovered at M6.

**(b) Intent provenance.** If an agent writes the implementation, an agent writes
the use case, and an agent realises the test, then "oracles independent from
actors" is procedurally true but epistemically thin. Record whether use cases are
human-authored, spec-derived, or agent-generated, and what independence guarantee
survives in each case.

Deliverable: `docs/TestDriverClassificationDesign.md` + one recorded decision.

**Done 2026-08-22.** Design note written; decision
`fef5213f-ce9b-44c2-b327-a0b0ba4b6270` recorded in the hub.

Outcome in one line: **adaptation may write only to surface evidence (S1); claims
and invariants are run inputs with no adaptation write path.** False Adaptation
Rate = 0 therefore becomes an architectural property rather than a tuning target —
the system cannot express "accept a defect as an adaptation". Classification is a
total function over three signals defaulting to `AMBIGUOUS`; a surface change
concurrent with a verdict change classifies as `PRODUCT_DEFECT`, never as
adaptation. Claims require causally independent provenance.

Two consequences that change later tasks:
- **D-07** — an independent observation channel into the system under test is now
  an explicit adoption precondition, not an implementation detail. T05 must expose
  one in the lab; without it S2/S3 collapse into S1.
- A fourth self-test, `td://self/intent-independence`, joins the T06 set.

## Minimal research control plane

```task
id: TD-WP-0002-T03
status: done
priority: medium
state_hub_task_id: "31b88522-decd-5f4c-b0e2-05b6981f33b8"
```

Timebox: half a day. Five hypothesis files (H-001…H-005) each with an explicit
falsification condition, one Concept ↔ Implementation Fitness Map, and a stable
identifier convention. Plain files under `research/`.

No CLI, no register tooling, no schema. The control plane is overhead until there
are readings to record.

**Done 2026-08-22.** `research/` holds H-001…H-005 (each with an explicit
falsification condition), E-001…E-003, the fitness map, the findings log and the
ID convention.

Three things worth carrying forward:
- The fitness map corrects the Improvement Loop § 13 starting classifications
  **downward**. Nothing can honestly exceed `C1` with no implementation; `C2` was
  aspirational.
- **H-005 (Energy) is dormant by decision** — untestable at this scale, so events
  are captured and no scoring is written.
- **H-001 requires a real control arm** (a robust recorded selector sequence). A
  straw-man control would make the semantic-action thesis trivially true and
  worthless.

## Deterministic semantic kernel

```task
id: TD-WP-0002-T04
status: done
priority: high
state_hub_task_id: "ffbcc9e8-c1bd-5c50-b62d-495ad9e135a4"
```

Minimal executable representations of UseCase, Actor, World, Scenario,
SemanticAction, Observation, Evidence, Oracle, Verdict, VerificationAsset, Run.
One deterministic driver (HTTP or direct adapter).

Run the reference use case end to end:

```
Alice owns resource R. Alice grants Bob READ. Bob can read R.
Carol cannot read R. Alice revokes Bob. Bob can no longer read R.
```

Exit: Alice, Bob and Carol hold genuinely isolated identities and sessions; all
outcomes are judged by independent deterministic oracles; a run emits a
structured Evidence Pack; the scenario replays from known initial state.

Emit raw `EnergyEvent` records from this point onward. Implement no scoring.

**Done 2026-08-22.** `src/testdriver/` (11 modules), `lab/minimal.py`,
`scenarios/alice_bob_carol.py`, 16 passing tests. The reference scenario runs
end to end and replays identically from the same seed; evidence comes out
stratified 3/3/3 across S1/S2/S3.

Three things that came out of building it rather than designing it:

- **The observation channel needs two probes, not one.** Reading stored state
  alone verifies test-driver's reimplementation of the rules rather than the
  system's enforcement of them; probing enforcement alone cannot notice that
  record and enforcement disagree. The lab exposes both, and their disagreement
  is now an invariant (`i-enforcement-matches-record`). That invariant is what
  catches an authorization defect which leaves the audit trail looking correct.
- **A seeded `RevokeIsCosmetic` lab already fails the run** — both the claim and
  the independent invariant fire, and the claim set is provably untouched. Early
  evidence for H-004, though not yet the experiment.
- **Scenarios are Python, not YAML.** Claims are predicates over observations; a
  YAML dialect able to express them would be a programming language with worse
  tooling. Revisit once we know which predicates actually recur.

## Test-driver lab with labelled ground truth

```task
id: TD-WP-0002-T05
status: done
priority: high
state_hub_task_id: "595a89c2-1462-57fc-8b08-a5a6b875fd48"
```

A deliberately small application: users, tenants, auth, resources, sharing,
read/write permissions, revoke, audit history, HTTP API, minimal browser UI.

Build **15–20 labelled mutations**, not the six sketched in the milestones doc.
Six cannot support any statement about precision or recall. Each mutation is a
reproducible toggle carrying a recorded expected classification
(`MECHANICAL` / `SEMANTIC` / `DEFECT`), with baseline and mutated versions
carrying explicit version identifiers.

The lab is the measuring instrument for every claim the framework makes — a weak
lab caps the credibility of all downstream results. It is also potentially the
project's first publishable artefact in its own right.

**Done 2026-08-22.** `lab/app.py` (users, tenants, auth, resources, sharing,
read/write, revoke, audit), `lab/http_api.py` (JSON API + browser UI, stdlib
only), `lab/mutations.py` (20 labelled, composable, version-stamped mutations),
`lab/GROUND-TRUTH.md`. 48 tests pass. Detection: MECHANICAL 0/10 flagged,
DEFECT 6/6, SEMANTIC 2/4.

Three results worth carrying:

- **F-0002 — two seeded defects were initially invisible.** M16 (READ grant
  confers WRITE) and M18 (revocation unaudited) both passed cleanly: nothing
  flaky, nothing `INCONCLUSIVE`, the framework simply had nothing to say. A use
  case protects exactly what it asserts and not one thing more. Resolved by
  adding two claims already stated as intent in `INTENT.md`. Had the six-mutation
  catalogue from the milestones doc been used, this would never have surfaced —
  the argument for the larger catalogue is now evidenced rather than asserted.
- **The test-id axis.** Stable `data-td` selectors survive most UI mutations,
  which would make H-001 trivially *false*. Rather than rig the catalogue,
  mutations now vary on `preserves_test_ids`, and H-001 must be analysed split by
  that axis. A semantic action earns its keep exactly where stable identifiers
  are absent or not carried forward — that is the honest shape of the claim.
- **M12 vs M19 are behaviourally identical.** A deliberate deferred-revocation
  decision and a revocation race produce the same failure, same step, same
  evidence. Only intent separates them. This is the discrimination problem in its
  sharpest form and is now a test, not a paragraph.

## Out-of-band ground truth for self-verification

```task
id: TD-WP-0002-T06
status: todo
priority: medium
state_hub_task_id: "4fb31485-8650-556c-bee4-b4e067a89d04"
```

Write the actor-isolation and oracle-independence checks as **plain pytest**
against observable behaviour, outside the framework. Using test-driver to verify
that test-driver's oracles are independent is a system certifying itself.

Exit: deliberately breaking actor isolation fails the out-of-band check;
allowing an actor to determine its own verdict fails oracle-independence;
each failure produces a Framework Finding.

## Agentic realisation of one semantic action, fully instrumented

```task
id: TD-WP-0002-T07
status: todo
priority: high
state_hub_task_id: "b58cf7ce-dc18-5218-8eac-179291626467"
```

Browser driver plus one agentic actor runtime, realising exactly one semantic
action — `grant_access(Bob, R, READ)` — from intent, against the lab UI. Strict
per-actor context isolation; bounded navigation and tool permissions; full
action and evidence recording; agent/model/configuration identity captured in
evidence. Oracles stay deterministic.

**Instrument cost and nondeterminism from the very first run:** tokens, wall
time, retry count, and run-to-run variance. If agentic realisation costs more per
run than simply asking an agent to rewrite the broken test, the crystallization
argument is an aesthetic preference rather than a value proposition. This data is
free to collect from run one and impossible to backfill.

## Adaptation detection and the defect-vs-adaptation classifier

```task
id: TD-WP-0002-T08
status: todo
priority: high
state_hub_task_id: "e2403d3c-bc35-5dd3-b1c8-2474663e06d0"
```

The centrepiece. Implement the mechanism designed in T02 and run it across the
full labelled mutation set from T05.

Classification path: `IMPLEMENTATION CHANGE` / `INTENT CHANGE` /
`PRODUCT DEFECT` / `AMBIGUOUS`, with escalation for semantic changes and
ambiguity. Preserve semantic action identity across adaptation; record adaptation
history on the verification asset; never modify claims or invariants to
accommodate an observed behaviour.

Measure: Mechanical Recovery Rate, classification precision/recall, and
**False Adaptation Rate**. The last is the project's existential safety metric —
one publicly normalised authorization defect kills the thesis permanently. Target
is zero across the set, and a non-zero result is a stop-and-redesign signal, not
a tuning exercise.

## Crystallize one asset into deterministic regression

```task
id: TD-WP-0002-T09
status: todo
priority: high
state_hub_task_id: "855a1f41-b839-57fc-87b8-198ce2a9f6b1"
```

Semantic action trajectory capture, stable-realisation detection, deterministic
candidate generation, candidate comparison against the existing oracle set, and
lineage from the agentic ancestor to the deterministic descendant. Manual
acceptance step for now.

Emit ordinary pytest/Playwright code. Output that drops into a CI system which
already exists lets a user adopt the result without adopting the framework.

Exit: the generated test runs with zero agentic involvement, preserves the
relevant claims and oracles, retains visible lineage, and measurably costs less
to execute than the agentic ancestor.

## Gate review and first compression pass

```task
id: TD-WP-0002-T10
status: todo
priority: medium
state_hub_task_id: "884c2e91-b56d-5b5e-a2d3-52d2a2768cae"
```

Evaluate the four success-gate criteria against collected evidence and write the
result up in `history/` regardless of outcome.

Then run the compression questions from Milestone 10 while the spike is still
small: which concepts have no supporting evidence, which abstractions have no
conceptual justification, which metadata never informed a decision, what did the
experiments falsify. Mark rejected concepts as rejected rather than silently
retaining them, and update the Concept ↔ Implementation Fitness Map.

Convert at least one framework finding discovered during T04–T09 into a permanent
self-verification or deterministic regression — that closes the first genuine
Concept–Implementation Fitness Loop.