T09: crystallization

A stable agentic realization becomes deterministic code. All four exit
criteria met; 163 tests pass.

- crystallization.py: trajectory capture, stability assessment requiring the
  same path across several runs, CrystallizedDriver, pytest codegen
- crystallized/test_grant_access.py: generated, runs with no model, carries
  its lineage in the docstring
- descendant preserves the ancestor's oracle set, agrees with it across five
  lab versions, and still catches a seeded defect
- reversibility shown both ways via new M24 (grant endpoint renamed): the
  frozen descendant fails loudly rather than searching, and the agentic
  ancestor recovers from the same mutation

F-0007 (open): the 54% cost reduction must not be quoted in support of the
thesis. The T07 runtime is token-free, so the measured saving is one page
fetch, one parse and a two-candidate scoring pass. The saving the concept
actually claims - tokens, latency, retry variance - is unmeasured. Together
with F-0005 this makes a bounded live-model experiment the highest-value next
investment.

Assertions in the generated test are imported rather than restated, so it is
not fully standalone. Deliberate: paraphrased claims would be a second
unverified statement of intent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
tegwick 2026-08-23 00:21:48 +02:00
parent b9ad259468
commit eee7722714
28 changed files with 794 additions and 10 deletions

View file

@ -0,0 +1,87 @@
---
id: F-0007
type: framework-finding
class: FRAMEWORK_LIMITATION
status: open
discovered: "2026-08-23"
discovered_by: TD-WP-0002-T09
workplan: TD-WP-0002
task: TD-WP-0002-T09
hypotheses: [H-003]
carried_to: TD-WP-0002-T10
---
# F-0007 — Crystallization's economic case cannot be measured yet
## The criterion
H-003 includes a cost clause, deliberately:
> - it costs no less to execute than the agentic ancestor.
with the note: *if crystallization preserves semantics but saves nothing, the
thesis is intact but the product rationale is not.*
## What was measured
| | median per run |
|---|---|
| agentic ancestor | 6.87 ms |
| crystallized descendant | 3.17 ms |
| **reduction** | **53.9 %** |
The criterion is met — the descendant is measurably cheaper. But the number is
close to meaningless as evidence for the thesis.
## Why it is close to meaningless
The T07 runtime is a **deterministic heuristic**, chosen with the operator to
avoid API cost and nondeterminism. It consumes zero tokens. So the entire
measured saving is one page fetch, one HTML parse and a two-candidate scoring
pass — a few milliseconds of local work.
The saving crystallization actually claims is of a different kind and two or
three orders of magnitude larger: **model tokens, model latency, and the
variance that forces retries.** None of those exist in this measurement, because
none of those exist in this runtime.
So the honest statement is:
> Crystallization is measurably cheaper than the ancestor it was frozen from.
> The measured 54 % is a **floor** produced by removing local discovery work, and
> it says nothing about the saving that motivates the concept.
Quoting "54 % cheaper" as support for the crystallization thesis would be
misleading, and this finding exists so that nobody does.
## What would make it measurable
A live-model runtime behind the same `ActorRuntime` interface. The
`RealizationMetrics` fields (`tokens_in`, `tokens_out`, `model`, `retries`) were
populated from the first run precisely so this comparison becomes a subtraction
rather than a re-run of everything — see T07.
**F-0005 already gives an independent reason to want one:** M22 defeats the
heuristic runtime while remaining solvable by reading a visible label. So a live
model would settle two open questions at once — whether it adds *capability*
(F-0005) and whether crystallization has an economic case (this finding).
That makes a bounded live-model experiment the highest-value next investment,
above any further framework feature.
## A second, smaller limitation
The generated test is **not fully standalone**. Its realization is plain
`urllib` with no framework dependency, but its assertions are *imported* from the
originating scenario module rather than restated.
That was the right call — a generated test that paraphrases its claims creates a
second, unverified statement of intent, and drift between them would be silent.
But it qualifies the adoption story in the workplan ("output that drops into a CI
system which already exists"): what drops in is the realization, while the claims
still require the use-case module on the path.
Fully standalone generation would need claims expressible in a serializable form
rather than as Python predicates. That is a real design question — it is the same
question as "should scenarios be YAML", deferred at T04 — and both should be
answered together at T10, with evidence about which predicates actually recur.