coulomb/test-driver
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
test-driver/research/findings
History
tegwick eee7722714 T09: crystallization 
A stable agentic realization becomes deterministic code. All four exit
criteria met; 163 tests pass.

- crystallization.py: trajectory capture, stability assessment requiring the
  same path across several runs, CrystallizedDriver, pytest codegen
- crystallized/test_grant_access.py: generated, runs with no model, carries
  its lineage in the docstring
- descendant preserves the ancestor's oracle set, agrees with it across five
  lab versions, and still catches a seeded defect
- reversibility shown both ways via new M24 (grant endpoint renamed): the
  frozen descendant fails loudly rather than searching, and the agentic
  ancestor recovers from the same mutation

F-0007 (open): the 54% cost reduction must not be quoted in support of the
thesis. The T07 runtime is token-free, so the measured saving is one page
fetch, one parse and a two-candidate scoring pass. The saving the concept
actually claims - tokens, latency, retry variance - is unmeasured. Together
with F-0005 this makes a bounded live-model experiment the highest-value next
investment.

Assertions in the generated test are imported rather than restated, so it is
not fully standalone. Deliberate: paraphrased claims would be a second
unverified statement of intent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-23 00:21:48 +02:00
..
F-0001-milestone-sequence-drift.md T01: establish canonical milestone sequence, record F-0001 2026-08-22 23:07:31 +02:00
F-0002-scenario-coverage-gap.md T05: the lab and its labelled mutation catalogue 2026-08-22 23:31:22 +02:00
F-0003-isolation-needs-canaries.md T06: out-of-band self-verification 2026-08-22 23:38:23 +02:00
F-0004-stdlib-driver-instead-of-playwright.md T07: agentic realization over a stdlib browser surface 2026-08-22 23:50:29 +02:00
F-0005-semantic-actions-earn-their-keep-narrowly.md T07: agentic realization over a stdlib browser surface 2026-08-22 23:50:29 +02:00
F-0006-classifier-cannot-infer-intent.md T08: the classifier, measured and attacked 2026-08-23 00:02:58 +02:00
F-0007-crystallization-cost-unmeasurable.md T09: crystallization 2026-08-23 00:21:48 +02:00
README.md T01: establish canonical milestone sequence, record F-0001 2026-08-22 23:07:31 +02:00

README.md

Framework Findings

Findings about test-driver itself, kept strictly separate from findings about any system under test. Classes are defined in docs/TestDriverImprovementLoop.md § 9:

PRODUCT_DEFECT · TEST_DEFECT · MECHANICAL_ADAPTATION · SEMANTIC_CHANGE · CONCEPT_DRIFT · FRAMEWORK_LIMITATION · EVIDENCE_FAILURE · UNNECESSARY_COMPLEXITY

File naming: F-NNNN-<slug>.md. Identifiers are stable and never reused.

Each finding carries frontmatter (id, class, status, dates, originating workplan/task) and, when resolved, states which of the three §5 resolution paths was taken. A finding that leaves an open question records it under "Residual" and names the task that will answer it — residuals are not parked in prose.

The wider research control plane (hypotheses/, experiments/, concepts/, decisions/) is built in TD-WP-0002-T03.