T09: crystallization
A stable agentic realization becomes deterministic code. All four exit criteria met; 163 tests pass. - crystallization.py: trajectory capture, stability assessment requiring the same path across several runs, CrystallizedDriver, pytest codegen - crystallized/test_grant_access.py: generated, runs with no model, carries its lineage in the docstring - descendant preserves the ancestor's oracle set, agrees with it across five lab versions, and still catches a seeded defect - reversibility shown both ways via new M24 (grant endpoint renamed): the frozen descendant fails loudly rather than searching, and the agentic ancestor recovers from the same mutation F-0007 (open): the 54% cost reduction must not be quoted in support of the thesis. The T07 runtime is token-free, so the measured saving is one page fetch, one parse and a two-candidate scoring pass. The saving the concept actually claims - tokens, latency, retry variance - is unmeasured. Together with F-0005 this makes a bounded live-model experiment the highest-value next investment. Assertions in the generated test are imported rather than restated, so it is not fully standalone. Deliberate: paraphrased claims would be a second unverified statement of intent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
parent
b9ad259468
commit
eee7722714
28 changed files with 794 additions and 10 deletions
6
crystallized/__init__.py
Normal file
6
crystallized/__init__.py
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
"""Generated deterministic regression tests. Do not edit by hand.
|
||||
|
||||
Each file here is the frozen descendant of an agentic verification asset. Its
|
||||
provenance, ancestor and the number of identical realizations it was frozen from
|
||||
are recorded in its own docstring.
|
||||
"""
|
||||
BIN
crystallized/__pycache__/__init__.cpython-312.pyc
Normal file
BIN
crystallized/__pycache__/__init__.cpython-312.pyc
Normal file
Binary file not shown.
BIN
crystallized/__pycache__/conftest.cpython-312-pytest-7.4.4.pyc
Normal file
BIN
crystallized/__pycache__/conftest.cpython-312-pytest-7.4.4.pyc
Normal file
Binary file not shown.
Binary file not shown.
39
crystallized/conftest.py
Normal file
39
crystallized/conftest.py
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
"""Fixtures for crystallized regression tests.
|
||||
|
||||
The only framework code a crystallized test touches is the observation channel —
|
||||
because judging still requires observing the system independently, whether or not
|
||||
a model was involved in getting there (D-07). Nothing here discovers anything.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
|
||||
from lab.http_api import serve
|
||||
from lab.mutations import ObservationChannel, build_lab
|
||||
from testdriver.observers import StateObserver, Watch
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def crystallized_world(request):
|
||||
"""A live lab, the owner's token, and an independent observation function."""
|
||||
mutations = getattr(request, "param", ())
|
||||
app, tokens = build_lab(*mutations)
|
||||
app.request(tokens["alice"], "create_resource",
|
||||
resource_id="R", content="the secret")
|
||||
server = serve(app)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
observer = StateObserver(
|
||||
ObservationChannel(app), (Watch("bob", "R"), Watch("carol", "R"))
|
||||
)
|
||||
try:
|
||||
yield (
|
||||
f"http://127.0.0.1:{server.server_address[1]}",
|
||||
tokens["alice"],
|
||||
observer.snapshot,
|
||||
)
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
73
crystallized/test_grant_access.py
Normal file
73
crystallized/test_grant_access.py
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
"""Crystallized regression test — generated, do not edit by hand.
|
||||
|
||||
Lineage
|
||||
-------
|
||||
ancestor asset : va-grant-via-browser
|
||||
ancestor maturity: T1
|
||||
descendant : va-grant-crystallized (T5 Deterministic)
|
||||
frozen from : 4 identical realizations
|
||||
surface version: lab-0.2.0-baseline
|
||||
generated : 2026-08-22
|
||||
|
||||
Why this file exists
|
||||
--------------------
|
||||
An agent discovered this path 4 times running and it did not change. The
|
||||
search is now waste, so it has been frozen. **No model is involved in running
|
||||
this test.**
|
||||
|
||||
The realization below is plain HTTP with no framework dependency. The assertions
|
||||
are imported from the originating use case rather than restated — a generated
|
||||
test that paraphrases its assertions creates a second, unverified statement of
|
||||
intent, and drift between the two would be silent. See F-0007 for what that
|
||||
costs.
|
||||
|
||||
If this test starts failing, the correct first response is **not** to update the
|
||||
selectors. Re-run the agentic ancestor: if it recovers, the surface moved and
|
||||
this file should be regenerated; if it does not, the behaviour changed and that
|
||||
is a finding.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from scenarios.alice_bob_carol import _bob_can_read, _bob_cannot_write, _carol_cannot_read
|
||||
|
||||
TARGET = '/resources/R/grant'
|
||||
FIELDS = {'permission': 'READ', 'subject_id': 'bob'}
|
||||
|
||||
|
||||
def _post(base_url: str, token: str, path: str, fields: dict) -> int:
|
||||
request = urllib.request.Request(
|
||||
urllib.parse.urljoin(base_url, path),
|
||||
data=urllib.parse.urlencode(fields).encode(),
|
||||
method="POST",
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/x-www-form-urlencoded",
|
||||
},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=10) as response:
|
||||
return response.status
|
||||
except urllib.error.HTTPError as error:
|
||||
return error.code
|
||||
|
||||
|
||||
def realize(base_url: str, token: str) -> int:
|
||||
"""Perform grant_access deterministically, exactly as the agent learned to."""
|
||||
return _post(base_url, token, TARGET, FIELDS)
|
||||
|
||||
|
||||
def test_grant_access(crystallized_world):
|
||||
"""grant_access still works, and the claims it protects still hold."""
|
||||
base_url, token, observe = crystallized_world
|
||||
|
||||
assert realize(base_url, token) < 400, "the frozen realization no longer works"
|
||||
|
||||
snapshot = observe()
|
||||
assert _bob_can_read(snapshot), 'Bob can read R after the grant'
|
||||
assert _carol_cannot_read(snapshot), 'Carol can never read R'
|
||||
assert _bob_cannot_write(snapshot), 'A READ grant does not let Bob write R'
|
||||
Loading…
Add table
Add a link
Reference in a new issue