2026-08-22 23:11:21 +02:00
# Concept ↔ Implementation Fitness Map
2026-08-23 00:39:36 +02:00
**Updated:** 2026-08-23 (TD-WP-0002-T10)
2026-08-22 23:11:21 +02:00
Traces each important concept to the implementation, experiment and evidence that
support it. **Unsupported entries are the point of this map** — a concept with no
implementation and no evidence is not a gap to be embarrassed about, it is the
current honest state, and hiding it defeats the map's purpose.
Support levels follow `TestDriverImprovementLoop.md` §13:
`C0 Idea` · `C1 Hypothesis` · `C2 Experimentally Supported` ·
`C3 Practically Validated` · `C4 Architectural Invariant`
## Current state
2026-08-22 23:50:29 +02:00
`C-semantic-action` is the first concept to reach `C2` : it has an experiment
behind it (the T07 two-arm comparison), and that experiment narrowed the claim
rather than confirming it. Everything else still rests on unit tests.
T04: deterministic semantic kernel
Alice/Bob/Carol runs end to end, deterministically, replayable from seed.
16 tests pass, no third-party dependencies.
- src/testdriver: intent, provenance, world, actions, drivers, observers,
oracles, evidence, energy, scenario, runner
- lab/minimal.py: the SUT, exposing the independent observation channel
required by D-07
- evidence is stratified S1/S2/S3; Runner refuses to attribute S2/S3 to an
actor; claims are frozen and provenance-checked at construction
- missing evidence yields INCONCLUSIVE, which outranks PASS in the run verdict
- EnergyEvents captured, no scoring (H-005 dormant)
The observation channel records both stored state and an out-of-band
enforcement probe; their disagreement is an invariant and is what detects an
authorization defect that leaves the audit trail intact. A seeded
RevokeIsCosmetic lab fails the run via both the claim and that invariant.
Also closes TD-WP-0001-T02 (stack and commands now exist).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:21:07 +02:00
The deterministic kernel exists (T04) and its guarantees are covered by unit
2026-08-22 23:50:29 +02:00
tests. **Levels do not move for those.** A passing unit test is not an experiment: it
T04: deterministic semantic kernel
Alice/Bob/Carol runs end to end, deterministically, replayable from seed.
16 tests pass, no third-party dependencies.
- src/testdriver: intent, provenance, world, actions, drivers, observers,
oracles, evidence, energy, scenario, runner
- lab/minimal.py: the SUT, exposing the independent observation channel
required by D-07
- evidence is stratified S1/S2/S3; Runner refuses to attribute S2/S3 to an
actor; claims are frozen and provenance-checked at construction
- missing evidence yields INCONCLUSIVE, which outranks PASS in the run verdict
- EnergyEvents captured, no scoring (H-005 dormant)
The observation channel records both stored state and an out-of-band
enforcement probe; their disagreement is an invariant and is what detects an
authorization defect that leaves the audit trail intact. A seeded
RevokeIsCosmetic lab fails the run via both the claim and that invariant.
Also closes TD-WP-0001-T02 (stack and commands now exist).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:21:07 +02:00
shows the code does what its author intended, not that the concept holds under
the mutations it claims to survive. Levels rise when E-001/E-002/E-003 produce
evidence, not before. The implementation column below moves; the level column
does not. The initial classifications in §13 of the Improvement Loop
2026-08-22 23:11:21 +02:00
(`Actor Isolation C2` , `Independent Oracles C2` ) are corrected downward here: they
were aspirational, not evidenced.
| Concept | Level | Implementation | Experiment | Evidence | Open question |
|---|---|---|---|---|---|
T04: deterministic semantic kernel
Alice/Bob/Carol runs end to end, deterministically, replayable from seed.
16 tests pass, no third-party dependencies.
- src/testdriver: intent, provenance, world, actions, drivers, observers,
oracles, evidence, energy, scenario, runner
- lab/minimal.py: the SUT, exposing the independent observation channel
required by D-07
- evidence is stratified S1/S2/S3; Runner refuses to attribute S2/S3 to an
actor; claims are frozen and provenance-checked at construction
- missing evidence yields INCONCLUSIVE, which outranks PASS in the run verdict
- EnergyEvents captured, no scoring (H-005 dormant)
The observation channel records both stored state and an out-of-band
enforcement probe; their disagreement is an invariant and is what detects an
authorization defect that leaves the audit trail intact. A seeded
RevokeIsCosmetic lab fails the run via both the claim and that invariant.
Also closes TD-WP-0001-T02 (stack and commands now exist).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:21:07 +02:00
| `C-use-case` | C1 | `intent.py` | — | — | Is a use case expressible without leaking mechanics? |
2026-08-23 00:39:36 +02:00
| `C-actor-isolation` | **C2** | `world.py` , `runner.py` | E-001 | `td://self/actor-isolation` , examined every run | **F-0003 resolved** — automatic canaries; observable on every scenario. |
2026-08-22 23:50:29 +02:00
| `C-semantic-action` | **C2** | `actions.py` , `agentic.py` | E-001 (partial) | T07 arm comparison | **F-0005** — supported only where stable identifiers are absent. Narrower than the concept model claims. |
T08: the classifier, measured and attacked
False Adaptation Rate = 0/7 across the labelled catalogue and the three E-003
attacks. 11 of 12 mechanical mutations absorbed without a human, so the safety
result is not bought by escalating everything.
- classification.py: total function over three signals, rule order chosen so
every rule that could excuse a regression sits after the rule that reports
one. SAFE_TO_ACCEPT is a two-element closed set, asserted.
- CompositeDriver plus scenarios/full_journey.py: one asset crossing both
surfaces, so UI mutations are visible as surface differences while the
claims they do not touch stay green.
- E-003: surface substitution (new M23), concurrent mechanical+defect,
evidence starvation, provenance laundering. All held.
F-0006 (CONCEPT_DRIFT, resolved): the T02 design listed SEMANTIC_CHANGE as an
outcome the table could produce. It cannot - M12 and M19 are behaviourally
identical, as the lab has asserted since T05. PRODUCT_DEFECT and
SEMANTIC_CHANGE collapse into one escalating outcome, BEHAVIOUR_CHANGED, and
the distinction becomes a human adjudication. INTENT_CHANGED survives but is
detected by the claim fingerprint moving, not inferred from behaviour.
Two classifier defects found and fixed rather than reported: claims downstream
of a failed realization now yield INCONCLUSIVE rather than FAIL (a false
accusation is the mirror image of a false adaptation), and the browser driver
records a page signature so surface change is detectable when the interaction
path is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-23 00:02:58 +02:00
| `C-oracle-independence` | **C2** | `runner.py` , `oracles.py` | E-001, E-003 | — | Independence of components ≠ independence of belief. (H-004) |
2026-08-22 23:38:23 +02:00
| `C-evidence-pack` | C1 | `evidence.py` | — | `td://self/evidence-reproducibility` | Verdicts are reproducible from S3 alone, on passing and failing runs. |
T05: the lab and its labelled mutation catalogue
lab/app.py (users, tenants, auth, resources, sharing, read/write, revoke,
audit), lab/http_api.py (JSON API + browser UI, stdlib only), 20 labelled
composable version-stamped mutations, ground-truth matrix. 48 tests pass.
Detection against the reference scenario: MECHANICAL 0/10 flagged (correct),
DEFECT 6/6, SEMANTIC 2/4 with both inert cases declared.
- F-0002: M16 and M18 initially escaped detection entirely. A use case
protects exactly what it asserts. Resolved by adding two claims already
stated as intent in INTENT.md; the six-mutation catalogue would never have
surfaced this.
- test-id axis added: stable selectors survive most UI mutations, which would
make H-001 trivially false. Mutations now vary on preserves_test_ids so the
hypothesis is analysed split by that axis rather than rigged.
- M12 (semantic deferred revoke) and M19 (defect race) are behaviourally
identical and asserted as such - the discrimination problem as a test.
lab/minimal.py removed; superseded by lab/app.py.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:31:22 +02:00
| `C-observation-channel` | C1 | `lab/app.py` | — | — | **D-07** — required of every system under test. Adoption cost unknown. |
T08: the classifier, measured and attacked
False Adaptation Rate = 0/7 across the labelled catalogue and the three E-003
attacks. 11 of 12 mechanical mutations absorbed without a human, so the safety
result is not bought by escalating everything.
- classification.py: total function over three signals, rule order chosen so
every rule that could excuse a regression sits after the rule that reports
one. SAFE_TO_ACCEPT is a two-element closed set, asserted.
- CompositeDriver plus scenarios/full_journey.py: one asset crossing both
surfaces, so UI mutations are visible as surface differences while the
claims they do not touch stay green.
- E-003: surface substitution (new M23), concurrent mechanical+defect,
evidence starvation, provenance laundering. All held.
F-0006 (CONCEPT_DRIFT, resolved): the T02 design listed SEMANTIC_CHANGE as an
outcome the table could produce. It cannot - M12 and M19 are behaviourally
identical, as the lab has asserted since T05. PRODUCT_DEFECT and
SEMANTIC_CHANGE collapse into one escalating outcome, BEHAVIOUR_CHANGED, and
the distinction becomes a human adjudication. INTENT_CHANGED survives but is
detected by the claim fingerprint moving, not inferred from behaviour.
Two classifier defects found and fixed rather than reported: claims downstream
of a failed realization now yield INCONCLUSIVE rather than FAIL (a false
accusation is the mirror image of a false adaptation), and the browser driver
records a page signature so surface change is detectable when the interaction
path is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-23 00:02:58 +02:00
| `C-adaptation` | **C2** | `classification.py` , `agentic.py` | E-001 | T08 matrix | 11/12 mechanical absorbed without a human. |
| `C-classification` | **C2** | `classification.py` | E-001, E-003 | T08 matrix, FAR 0/7 | **F-0006** — cannot infer SEMANTIC vs DEFECT; collapses to one escalating outcome. |
T09: crystallization
A stable agentic realization becomes deterministic code. All four exit
criteria met; 163 tests pass.
- crystallization.py: trajectory capture, stability assessment requiring the
same path across several runs, CrystallizedDriver, pytest codegen
- crystallized/test_grant_access.py: generated, runs with no model, carries
its lineage in the docstring
- descendant preserves the ancestor's oracle set, agrees with it across five
lab versions, and still catches a seeded defect
- reversibility shown both ways via new M24 (grant endpoint renamed): the
frozen descendant fails loudly rather than searching, and the agentic
ancestor recovers from the same mutation
F-0007 (open): the 54% cost reduction must not be quoted in support of the
thesis. The T07 runtime is token-free, so the measured saving is one page
fetch, one parse and a two-candidate scoring pass. The saving the concept
actually claims - tokens, latency, retry variance - is unmeasured. Together
with F-0005 this makes a bounded live-model experiment the highest-value next
investment.
Assertions in the generated test are imported rather than restated, so it is
not fully standalone. Deliberate: paraphrased claims would be a second
unverified statement of intent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-23 00:21:48 +02:00
| `C-crystallization` | **C2** | `crystallization.py` | E-002 | T09 agreement matrix | Fidelity supported; **F-0007** — the economic case is unmeasurable with a token-free runtime. |
2026-08-22 23:38:23 +02:00
| `C-intent-provenance` | C1 | `provenance.py` | E-003 | `td://self/intent-independence` | Constrains provenance, not quality. Accepted residual. |
T09: crystallization
A stable agentic realization becomes deterministic code. All four exit
criteria met; 163 tests pass.
- crystallization.py: trajectory capture, stability assessment requiring the
same path across several runs, CrystallizedDriver, pytest codegen
- crystallized/test_grant_access.py: generated, runs with no model, carries
its lineage in the docstring
- descendant preserves the ancestor's oracle set, agrees with it across five
lab versions, and still catches a seeded defect
- reversibility shown both ways via new M24 (grant endpoint renamed): the
frozen descendant fails loudly rather than searching, and the agentic
ancestor recovers from the same mutation
F-0007 (open): the 54% cost reduction must not be quoted in support of the
thesis. The T07 runtime is token-free, so the measured saving is one page
fetch, one parse and a two-candidate scoring pass. The saving the concept
actually claims - tokens, latency, retry variance - is unmeasured. Together
with F-0005 this makes a bounded live-model experiment the highest-value next
investment.
Assertions in the generated test are imported rather than restated, so it is
not fully standalone. Deliberate: paraphrased claims would be a second
unverified statement of intent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-23 00:21:48 +02:00
| `C-lineage` | C1 | `scenario.py` , generated headers | E-002 | generated module docstring | Parent pointer plus provenance in the artefact; no graph. |
2026-08-23 00:39:36 +02:00
| `C-energy` | C0 | `energy.py` , capture only | — | none | Dormant. **Gated** : if the next workplan ends with no decision having used the history, remove. |
| `C-temperature` | **C0 — under review** | — | — | none | **F-0008** — crystallization was built without it; measured stability did the work. Gated for removal. |
| `C-confidence` | C0 | — | — | none | Declared, unimplemented, never consulted. Same position as `C-temperature` without a built alternative. |
2026-08-22 23:11:21 +02:00
| `C-campaign` | C0 | — | — | — | Deferred. |
| `C-metabolism` | C0 | — | — | — | Deferred. Depends on C-energy. |
| `C-retirement` | C0 | — | — | — | Deferred. Depends on C-energy. |
T05: the lab and its labelled mutation catalogue
lab/app.py (users, tenants, auth, resources, sharing, read/write, revoke,
audit), lab/http_api.py (JSON API + browser UI, stdlib only), 20 labelled
composable version-stamped mutations, ground-truth matrix. 48 tests pass.
Detection against the reference scenario: MECHANICAL 0/10 flagged (correct),
DEFECT 6/6, SEMANTIC 2/4 with both inert cases declared.
- F-0002: M16 and M18 initially escaped detection entirely. A use case
protects exactly what it asserts. Resolved by adding two claims already
stated as intent in INTENT.md; the six-mutation catalogue would never have
surfaced this.
- test-id axis added: stable selectors survive most UI mutations, which would
make H-001 trivially false. Mutations now vary on preserves_test_ids so the
hypothesis is analysed split by that axis rather than rigged.
- M12 (semantic deferred revoke) and M19 (defect race) are behaviourally
identical and asserted as such - the discrimination problem as a test.
lab/minimal.py removed; superseded by lab/app.py.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:31:22 +02:00
| `C-security-mutation` | C1 | `lab/mutations.py` | E-003 | `lab/GROUND-TRUTH.md` | Catalogue is hand-written; no derivation mechanism from use cases yet. |
2026-08-22 23:11:21 +02:00
## Orphan check
**Conceptual orphans** — concepts with no planned implementation in `TD-WP-0002` :
`C-temperature` , `C-confidence` , `C-campaign` , `C-metabolism` , `C-retirement` .
All five are deferred *by explicit decision* , not oversight. They are the group
most at risk of being built because they are easy and satisfying, and never
validated. They are revisited at T10, where the question is not "when do we build
these" but "does the evidence justify keeping them in the model at all".
T04: deterministic semantic kernel
Alice/Bob/Carol runs end to end, deterministically, replayable from seed.
16 tests pass, no third-party dependencies.
- src/testdriver: intent, provenance, world, actions, drivers, observers,
oracles, evidence, energy, scenario, runner
- lab/minimal.py: the SUT, exposing the independent observation channel
required by D-07
- evidence is stratified S1/S2/S3; Runner refuses to attribute S2/S3 to an
actor; claims are frozen and provenance-checked at construction
- missing evidence yields INCONCLUSIVE, which outranks PASS in the run verdict
- EnergyEvents captured, no scoring (H-005 dormant)
The observation channel records both stored state and an out-of-band
enforcement probe; their disagreement is an invariant and is what detects an
authorization defect that leaves the audit trail intact. A seeded
RevokeIsCosmetic lab fails the run via both the claim and that invariant.
Also closes TD-WP-0001-T02 (stack and commands now exist).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:21:07 +02:00
**Implementation orphans** — none. Every module in `src/testdriver/` traces to a
2026-08-23 00:39:36 +02:00
concept above.
**Removed at T10** (compression pass — see the gate review § 3):
`Verdict.SUSPICIOUS` , `Step.expect_refusal` , `ActorIsolationError` ,
`World.seed` , `EvidencePack.latest()` , `Trajectory.method` . Each was declared and
never used; `SUSPICIOUS` was additionally a verdict no oracle could emit.
**Gated for removal**: `C-temperature` (F-0008) and `energy.py` . Both survive the
current review on the strength of being cheap, not of being used. If the next
workplan closes without a decision consulting either, they go.
