T09: crystallization

A stable agentic realization becomes deterministic code. All four exit
criteria met; 163 tests pass.

- crystallization.py: trajectory capture, stability assessment requiring the
  same path across several runs, CrystallizedDriver, pytest codegen
- crystallized/test_grant_access.py: generated, runs with no model, carries
  its lineage in the docstring
- descendant preserves the ancestor's oracle set, agrees with it across five
  lab versions, and still catches a seeded defect
- reversibility shown both ways via new M24 (grant endpoint renamed): the
  frozen descendant fails loudly rather than searching, and the agentic
  ancestor recovers from the same mutation

F-0007 (open): the 54% cost reduction must not be quoted in support of the
thesis. The T07 runtime is token-free, so the measured saving is one page
fetch, one parse and a two-candidate scoring pass. The saving the concept
actually claims - tokens, latency, retry variance - is unmeasured. Together
with F-0005 this makes a bounded live-model experiment the highest-value next
investment.

Assertions in the generated test are imported rather than restated, so it is
not fully standalone. Deliberate: paraphrased claims would be a second
unverified statement of intent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
tegwick 2026-08-23 00:21:48 +02:00
parent b9ad259468
commit eee7722714
28 changed files with 794 additions and 10 deletions

View file

@ -1,6 +1,6 @@
# Concept ↔ Implementation Fitness Map
**Updated:** 2026-08-22 (TD-WP-0002-T08)
**Updated:** 2026-08-23 (TD-WP-0002-T09)
Traces each important concept to the implementation, experiment and evidence that
support it. **Unsupported entries are the point of this map** — a concept with no
@ -36,9 +36,9 @@ were aspirational, not evidenced.
| `C-observation-channel` | C1 | `lab/app.py` | — | — | **D-07** — required of every system under test. Adoption cost unknown. |
| `C-adaptation` | **C2** | `classification.py`, `agentic.py` | E-001 | T08 matrix | 11/12 mechanical absorbed without a human. |
| `C-classification` | **C2** | `classification.py` | E-001, E-003 | T08 matrix, FAR 0/7 | **F-0006** — cannot infer SEMANTIC vs DEFECT; collapses to one escalating outcome. |
| `C-crystallization` | C1 | — (T09) | E-002 | — | (H-003) |
| `C-crystallization` | **C2** | `crystallization.py` | E-002 | T09 agreement matrix | Fidelity supported; **F-0007** — the economic case is unmeasurable with a token-free runtime. |
| `C-intent-provenance` | C1 | `provenance.py` | E-003 | `td://self/intent-independence` | Constrains provenance, not quality. Accepted residual. |
| `C-lineage` | C0 | — | — | — | Parent pointer only in the spike. |
| `C-lineage` | C1 | `scenario.py`, generated headers | E-002 | generated module docstring | Parent pointer plus provenance in the artefact; no graph. |
| `C-energy` | C0 | `energy.py`, capture only | — | — | Dormant by decision. (H-005) |
| `C-temperature` | C0 | — | — | — | Deferred. No implementation planned in TD-WP-0002. |
| `C-confidence` | C0 | — | — | — | Deferred. |

View file

@ -30,4 +30,5 @@ was required.
## Status
`PLANNED`. Blocked on T07.
`EXECUTED` 2026-08-23 (T09). Fidelity criteria met across five lab versions;
the cost criterion is met but uninformative — F-0007.

View file

@ -0,0 +1,87 @@
---
id: F-0007
type: framework-finding
class: FRAMEWORK_LIMITATION
status: open
discovered: "2026-08-23"
discovered_by: TD-WP-0002-T09
workplan: TD-WP-0002
task: TD-WP-0002-T09
hypotheses: [H-003]
carried_to: TD-WP-0002-T10
---
# F-0007 — Crystallization's economic case cannot be measured yet
## The criterion
H-003 includes a cost clause, deliberately:
> - it costs no less to execute than the agentic ancestor.
with the note: *if crystallization preserves semantics but saves nothing, the
thesis is intact but the product rationale is not.*
## What was measured
| | median per run |
|---|---|
| agentic ancestor | 6.87 ms |
| crystallized descendant | 3.17 ms |
| **reduction** | **53.9 %** |
The criterion is met — the descendant is measurably cheaper. But the number is
close to meaningless as evidence for the thesis.
## Why it is close to meaningless
The T07 runtime is a **deterministic heuristic**, chosen with the operator to
avoid API cost and nondeterminism. It consumes zero tokens. So the entire
measured saving is one page fetch, one HTML parse and a two-candidate scoring
pass — a few milliseconds of local work.
The saving crystallization actually claims is of a different kind and two or
three orders of magnitude larger: **model tokens, model latency, and the
variance that forces retries.** None of those exist in this measurement, because
none of those exist in this runtime.
So the honest statement is:
> Crystallization is measurably cheaper than the ancestor it was frozen from.
> The measured 54 % is a **floor** produced by removing local discovery work, and
> it says nothing about the saving that motivates the concept.
Quoting "54 % cheaper" as support for the crystallization thesis would be
misleading, and this finding exists so that nobody does.
## What would make it measurable
A live-model runtime behind the same `ActorRuntime` interface. The
`RealizationMetrics` fields (`tokens_in`, `tokens_out`, `model`, `retries`) were
populated from the first run precisely so this comparison becomes a subtraction
rather than a re-run of everything — see T07.
**F-0005 already gives an independent reason to want one:** M22 defeats the
heuristic runtime while remaining solvable by reading a visible label. So a live
model would settle two open questions at once — whether it adds *capability*
(F-0005) and whether crystallization has an economic case (this finding).
That makes a bounded live-model experiment the highest-value next investment,
above any further framework feature.
## A second, smaller limitation
The generated test is **not fully standalone**. Its realization is plain
`urllib` with no framework dependency, but its assertions are *imported* from the
originating scenario module rather than restated.
That was the right call — a generated test that paraphrases its claims creates a
second, unverified statement of intent, and drift between them would be silent.
But it qualifies the adoption story in the workplan ("output that drops into a CI
system which already exists"): what drops in is the realization, while the claims
still require the use-case module on the path.
Fully standalone generation would need claims expressible in a serializable form
rather than as Python predicates. That is a real design question — it is the same
question as "should scenarios be YAML", deferred at T04 — and both should be
answered together at T10, with evidence about which predicates actually recur.

View file

@ -1,7 +1,7 @@
---
id: H-003
title: Crystallization
status: PROPOSED
status: EXPERIMENTING
created: "2026-08-22"
experiments: [E-002]
concepts: [C-crystallization]
@ -38,6 +38,28 @@ semantics but saves nothing, the thesis is intact but the product rationale is
not — and that distinction should be visible in the record rather than discovered
by a user.
## Result (TD-WP-0002-T09)
| Criterion | Outcome |
|---|---|
| produced without human authorship of assertions | **met** — assertions are imported, not restated |
| drops no oracle from the ancestor | **met** — descendant's set is a superset |
| agrees with the ancestor across lab versions | **met** — baseline, M15, M17, M16, M13 |
| costs less to execute | **met, but see F-0007** |
Also demonstrated: the descendant still catches a seeded defect (M17), it fails
*loudly* rather than searching when its frozen path breaks (M24), and the agentic
ancestor recovers from that same M24 where the descendant cannot — the
reversibility case from `INTENT.md`, shown in both directions.
**Not falsified. Not yet supported on the criterion that matters most.** The
measured 54 % cost reduction comes entirely from removing local discovery work,
because the T07 runtime consumes no tokens. The saving crystallization actually
claims — model tokens, latency, retry variance — is unmeasured and unmeasurable
with this runtime. See `research/findings/F-0007-...`.
## Status log
- 2026-08-22 `PROPOSED`. No evidence.
- 2026-08-23 `EXPERIMENTING`. Fidelity criteria met; the economic criterion is
met only in a sense too weak to support the thesis.