tegwick
eee7722714
A stable agentic realization becomes deterministic code. All four exit criteria met; 163 tests pass. - crystallization.py: trajectory capture, stability assessment requiring the same path across several runs, CrystallizedDriver, pytest codegen - crystallized/test_grant_access.py: generated, runs with no model, carries its lineage in the docstring - descendant preserves the ancestor's oracle set, agrees with it across five lab versions, and still catches a seeded defect - reversibility shown both ways via new M24 (grant endpoint renamed): the frozen descendant fails loudly rather than searching, and the agentic ancestor recovers from the same mutation F-0007 (open): the 54% cost reduction must not be quoted in support of the thesis. The T07 runtime is token-free, so the measured saving is one page fetch, one parse and a two-candidate scoring pass. The saving the concept actually claims - tokens, latency, retry variance - is unmeasured. Together with F-0005 this makes a bounded live-model experiment the highest-value next investment. Assertions in the generated test are imported rather than restated, so it is not fully standalone. Deliberate: paraphrased claims would be a second unverified statement of intent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
65 lines
2.4 KiB
Markdown
65 lines
2.4 KiB
Markdown
---
|
|
id: H-003
|
|
title: Crystallization
|
|
status: EXPERIMENTING
|
|
created: "2026-08-22"
|
|
experiments: [E-002]
|
|
concepts: [C-crystallization]
|
|
---
|
|
|
|
# H-003 — Crystallization
|
|
|
|
## Claim
|
|
|
|
A sufficiently stable agentic execution can be converted into deterministic test
|
|
code without losing relevant oracle coverage.
|
|
|
|
## Falsification condition
|
|
|
|
Any of:
|
|
- the generated deterministic test cannot be produced without human authorship of
|
|
the assertions;
|
|
- it drops one or more oracles present in the agentic ancestor;
|
|
- it passes where the ancestor would have failed, or vice versa, on the same lab
|
|
version;
|
|
- it costs no less to execute than the agentic ancestor.
|
|
|
|
## Measurement
|
|
|
|
- Crystallization success rate.
|
|
- Oracle set of descendant vs ancestor — must be a superset or equal.
|
|
- Verdict agreement between ancestor and descendant across lab versions.
|
|
- Execution cost ratio (tokens, wall time) descendant : ancestor.
|
|
|
|
## Note on the cost criterion
|
|
|
|
The cost criterion is included deliberately. If crystallization preserves
|
|
semantics but saves nothing, the thesis is intact but the product rationale is
|
|
not — and that distinction should be visible in the record rather than discovered
|
|
by a user.
|
|
|
|
## Result (TD-WP-0002-T09)
|
|
|
|
| Criterion | Outcome |
|
|
|---|---|
|
|
| produced without human authorship of assertions | **met** — assertions are imported, not restated |
|
|
| drops no oracle from the ancestor | **met** — descendant's set is a superset |
|
|
| agrees with the ancestor across lab versions | **met** — baseline, M15, M17, M16, M13 |
|
|
| costs less to execute | **met, but see F-0007** |
|
|
|
|
Also demonstrated: the descendant still catches a seeded defect (M17), it fails
|
|
*loudly* rather than searching when its frozen path breaks (M24), and the agentic
|
|
ancestor recovers from that same M24 where the descendant cannot — the
|
|
reversibility case from `INTENT.md`, shown in both directions.
|
|
|
|
**Not falsified. Not yet supported on the criterion that matters most.** The
|
|
measured 54 % cost reduction comes entirely from removing local discovery work,
|
|
because the T07 runtime consumes no tokens. The saving crystallization actually
|
|
claims — model tokens, latency, retry variance — is unmeasured and unmeasurable
|
|
with this runtime. See `research/findings/F-0007-...`.
|
|
|
|
## Status log
|
|
|
|
- 2026-08-22 `PROPOSED`. No evidence.
|
|
- 2026-08-23 `EXPERIMENTING`. Fidelity criteria met; the economic criterion is
|
|
met only in a sense too weak to support the thesis.
|