T10: gate review and first compression pass
All four gate criteria met. False Adaptation Rate 0/7 with 12 of 13 mechanical mutations absorbed. 178 tests pass. TD-WP-0002 finished. Fitness loop closed via F-0003: actor isolation was a property of scenarios written to expose it, not of runs. Actors now carry an automatic private marker and the runner examines all of them on every scenario, with two permanent regressions behind it. Compression - six abstractions removed, each declared and never used: Verdict.SUSPICIOUS (a verdict no oracle could emit), Step.expect_refusal, ActorIsolationError, World.seed, EvidencePack.latest, Trajectory.method. F-0008: Temperature may be redundant. Crystallization was built without it ever being consulted; measured stability of realization did the work, and is observed rather than declared. Gated for removal alongside energy.py. INTENT_CHANGED and REALIZATION_FAILED had never run. Both now have purpose-built cases and a test that fails if a seventh outcome is added without one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
parent
4f4219d8f7
commit
1b9860a8ee
40 changed files with 1074 additions and 46 deletions
212
history/2026-08-23-td-wp-0002-gate-review.md
Normal file
212
history/2026-08-23-td-wp-0002-gate-review.md
Normal file
|
|
@ -0,0 +1,212 @@
|
|||
# TD-WP-0002 — Gate Review and First Compression Pass
|
||||
|
||||
**Date:** 2026-08-23
|
||||
**Workplan:** `TD-WP-0002` — Vertical spike: falsify the crystallization thesis
|
||||
**Tests:** 172 passing · **Lab catalogue:** 24 labelled mutations
|
||||
**Assessment that set the direction:** `history/2026-08-22-concept-assessment-swot.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. The gate
|
||||
|
||||
The workplan fixed four criteria in advance, before any code existed.
|
||||
|
||||
### Criterion 1 — recover from M01 and M02 with zero claim or invariant changes
|
||||
|
||||
**Met.**
|
||||
|
||||
| Mutation | Classification | Claim set unchanged | Accepted without a human |
|
||||
|---|---|---|---|
|
||||
| M01 sharing control moved into a modal | `MECHANICAL_ADAPTATION` | yes | yes |
|
||||
| M02 DOM rewritten, test ids dropped | `MECHANICAL_ADAPTATION` | yes | yes |
|
||||
|
||||
The claim set is unchanged *by construction*, not by good behaviour: claims are
|
||||
run inputs and the adaptation machinery has no write path to them (D-02).
|
||||
|
||||
### Criterion 2 — the authorization defect is reported as a Product Finding
|
||||
|
||||
**Met**, with a naming correction. The workplan inherited "M05" from the original
|
||||
milestones document; in the catalogue as built, the authorization defect is
|
||||
**M15** (revocation updates the record but not enforcement). All three
|
||||
authorization defects behave correctly:
|
||||
|
||||
| Mutation | Classification | Regressions reported |
|
||||
|---|---|---|
|
||||
| M15 revoke is cosmetic | `BEHAVIOUR_CHANGED` | `c-bob-revoked`, `i-enforcement-matches-record` |
|
||||
| M17 anyone can read anything | `BEHAVIOUR_CHANGED` | 5 assertions across 3 steps |
|
||||
| M20 tenant isolation leaks | `BEHAVIOUR_CHANGED` | 5 assertions across 3 steps |
|
||||
|
||||
None is safe to accept. The classifier collapses `PRODUCT_DEFECT` and
|
||||
`SEMANTIC_CHANGE` into one escalating outcome — see F-0006 — so "Product Finding"
|
||||
in the original wording is now "escalated to a human with the regressions named".
|
||||
|
||||
### Criterion 3 — one asset crystallizes into a deterministic test
|
||||
|
||||
**Met.** `crystallized/test_grant_access.py` is generated, runs with **no model
|
||||
involvement**, preserves its ancestor's oracle set as a superset, agrees with the
|
||||
ancestor across five lab versions, still catches a seeded defect, and carries its
|
||||
lineage in its own docstring.
|
||||
|
||||
### Criterion 4 — False Adaptation Rate = 0
|
||||
|
||||
**Met. 0 / 7.** No `DEFECT`-labelled mutation was classified as safe to accept,
|
||||
including the three E-003 attacks (surface substitution, concurrent
|
||||
mechanical+defect, evidence starvation) and provenance laundering.
|
||||
|
||||
The other side of the trade is asserted too: **12 of 13** mechanical mutations are
|
||||
absorbed without a human. A classifier that escalated everything would score a
|
||||
perfect FAR and be worthless.
|
||||
|
||||
### Gate verdict
|
||||
|
||||
**Passed, on all four criteria.** With one qualification that belongs in the same
|
||||
sentence: FAR = 0 follows largely from *architecture* — claims are run inputs,
|
||||
and `SAFE_TO_ACCEPT` is a closed two-element set — not from classifier accuracy.
|
||||
The experiment shows the architecture behaves as designed over 24 hand-written
|
||||
mutations. It says nothing about mutations nobody thought of.
|
||||
|
||||
---
|
||||
|
||||
## 2. What the evidence changed
|
||||
|
||||
Three of the project's own claims were narrowed or corrected by building them.
|
||||
This is the part worth more than the green gate.
|
||||
|
||||
**H-001 was narrowed (F-0005).** Semantic actions do not generally outlast
|
||||
recorded interaction sequences. Where an application keeps stable identifiers,
|
||||
the two arms tie exactly — 9/9 against 9/9 — and the conventional approach is
|
||||
cheaper and deterministic. Semantic actions earn their keep only where
|
||||
identifiers are absent or not carried forward (2/3 against 0/3). The concept
|
||||
model presents this as a general superiority; it is conditional.
|
||||
|
||||
**The classifier cannot infer intent (F-0006).** The T02 design listed
|
||||
`SEMANTIC_CHANGE` as an outcome the decision table could produce. It cannot: a
|
||||
deliberate product decision and a defect are behaviourally identical, as M12 and
|
||||
M19 have asserted since T05. The two collapse into `BEHAVIOUR_CHANGED` and the
|
||||
distinction becomes a recorded human adjudication. Erring toward false alarm
|
||||
costs one review; erring toward false adaptation ends the project.
|
||||
|
||||
**Crystallization's economic case is unmeasured (F-0007).** The descendant is 54%
|
||||
cheaper than its ancestor, and that number must not be quoted in support of the
|
||||
thesis. The runtime is token-free by design, so the entire saving is one page
|
||||
fetch, one parse and a two-candidate scoring pass. The saving the concept
|
||||
actually claims — model tokens, latency, retry variance — is absent from the
|
||||
measurement.
|
||||
|
||||
---
|
||||
|
||||
## 3. The compression pass
|
||||
|
||||
### Which concepts have no supporting evidence?
|
||||
|
||||
`Temperature`, `Confidence`, `Campaign`, `Metabolism`, `Retirement` — declared,
|
||||
unimplemented, never consulted by any decision.
|
||||
|
||||
`Temperature` is the decidable one and is now **F-0008**. Crystallization was
|
||||
built and demonstrated without it: what triggered freezing was *measured
|
||||
stability of realization*, which is observed rather than declared. A declared
|
||||
temperature is a claim nobody verifies, and the project's own first heuristic
|
||||
argues against it. Not deleted yet — two unevidenced cases for keeping it are
|
||||
recorded — but gated: if the next workplan completes without a decision
|
||||
consulting it, it goes.
|
||||
|
||||
### Which implementation abstractions have no conceptual justification?
|
||||
|
||||
Removed at T10:
|
||||
|
||||
| Removed | Why |
|
||||
|---|---|
|
||||
| `Verdict.SUSPICIOUS` | No oracle could produce it. A verdict nothing can emit is a promise not kept. |
|
||||
| `Step.expect_refusal` | Never set by any scenario. Speculative generality. |
|
||||
| `ActorIsolationError` | Declared, never raised, never caught. |
|
||||
| `World.seed` | Written at T04, never read. Replay comes from rebuilding the lab. |
|
||||
| `EvidencePack.latest()` | Never called. |
|
||||
| `Trajectory.method` | Computed on every capture, never read. |
|
||||
|
||||
### Which metadata has not informed a decision?
|
||||
|
||||
**`EnergyEvent` history.** Events have been recorded since T04 and have informed
|
||||
nothing — as designed, since H-005 is dormant. Capture is retained because
|
||||
history cannot be reconstructed later while scores always can, but this is now on
|
||||
the same gate as Temperature: if the next workplan ends with no decision having
|
||||
used it, the module goes.
|
||||
|
||||
### Which agentic behaviour can now be deterministic?
|
||||
|
||||
The `grant_access` realization, and it has been — that is T09. Nothing else in
|
||||
the spike is agentic.
|
||||
|
||||
### What have the experiments falsified?
|
||||
|
||||
Nothing outright. Two claims were **narrowed** (H-001, and the classifier's
|
||||
output space), and one criterion was met in a sense **too weak to support the
|
||||
thesis** (H-003's cost clause). No hypothesis was rejected, and none was promoted
|
||||
past `EXPERIMENTING`: 24 hand-written mutations is a small, self-chosen sample.
|
||||
|
||||
### Two outcomes that had never run
|
||||
|
||||
`INTENT_CHANGED` and `REALIZATION_FAILED` were declared at T08 and produced by no
|
||||
test. Code that has never run is code nobody has checked. Rather than delete
|
||||
meaningful outcomes or trust them untested, both now have purpose-built cases,
|
||||
and `test_no_classification_is_unreachable` fails if a seventh is ever added
|
||||
without one.
|
||||
|
||||
Building the `REALIZATION_FAILED` case clarified a boundary worth keeping: a run
|
||||
that asserts *nothing at all* is `AMBIGUOUS`, not `REALIZATION_FAILED` — a use
|
||||
case with no claims cannot conclude anything, however well its steps ran.
|
||||
|
||||
---
|
||||
|
||||
## 4. The fitness loop, closed
|
||||
|
||||
The milestone asks that a framework finding become a permanent guarantee rather
|
||||
than a note. **F-0003 is resolved.**
|
||||
|
||||
The finding: actor isolation left no trace in ordinary evidence. The self-test
|
||||
caught a shared memory store only because the harness planted per-actor secrets
|
||||
first. Isolation was a property of *scenarios written to expose it*, not of runs.
|
||||
|
||||
The fix: every `Actor` is seeded at construction with an automatic private
|
||||
marker, and the runner examines all actors on **every** scenario, recording the
|
||||
verdict as S3 evidence. Two regressions now stand behind it —
|
||||
`test_every_run_records_a_verdict_on_isolation`, and a self-check that catches a
|
||||
leak with the harness planting nothing at all.
|
||||
|
||||
```
|
||||
Framework Finding (F-0003)
|
||||
→ classified FRAMEWORK_LIMITATION
|
||||
→ improvement hypothesis: automatic canaries make isolation observable
|
||||
→ implemented at T10
|
||||
→ permanent self-verification + deterministic regression
|
||||
→ fitness map updated
|
||||
```
|
||||
|
||||
That is the first complete Concept–Implementation Fitness Loop, which the
|
||||
milestones document calls the point at which the self-improvement system
|
||||
genuinely exists.
|
||||
|
||||
---
|
||||
|
||||
## 5. Where this leaves the project
|
||||
|
||||
**The thesis survived contact with its own evidence, in narrowed form.** The
|
||||
distinctive claim — that verification can adapt to a changing surface without
|
||||
adapting to a defect — is demonstrated, and the demonstration is architectural
|
||||
rather than statistical, which is the stronger kind.
|
||||
|
||||
**The single highest-value next step is a bounded live-model experiment.** Two
|
||||
independent findings converge on it: M22 defeats the heuristic runtime while
|
||||
remaining solvable by reading a visible label (F-0005), and crystallization's
|
||||
economic case cannot be measured without token costs (F-0007). One experiment
|
||||
settles whether a model adds capability *and* whether crystallization has a
|
||||
product rationale. The `RealizationMetrics` fields were populated from the first
|
||||
run so that comparison is a subtraction rather than a re-run.
|
||||
|
||||
That should come before any further framework feature.
|
||||
|
||||
**Open findings carried forward:** F-0003 resolved · F-0004 (structural
|
||||
durability only; visual relayout untested) · F-0005 (H-001 narrowed) · F-0007
|
||||
(cost unmeasurable; generated tests not fully standalone) · F-0008 (Temperature
|
||||
gated). The F-0001 residual — the taxonomy has no class for document-to-document
|
||||
drift — was considered and deliberately not resolved: adding a finding class to a
|
||||
taxonomy already larger than its evidence would be the opposite of compression.
|
||||
Loading…
Add table
Add a link
Reference in a new issue