tegwick
84848e9a0e
False Adaptation Rate = 0/7 across the labelled catalogue and the three E-003 attacks. 11 of 12 mechanical mutations absorbed without a human, so the safety result is not bought by escalating everything. - classification.py: total function over three signals, rule order chosen so every rule that could excuse a regression sits after the rule that reports one. SAFE_TO_ACCEPT is a two-element closed set, asserted. - CompositeDriver plus scenarios/full_journey.py: one asset crossing both surfaces, so UI mutations are visible as surface differences while the claims they do not touch stay green. - E-003: surface substitution (new M23), concurrent mechanical+defect, evidence starvation, provenance laundering. All held. F-0006 (CONCEPT_DRIFT, resolved): the T02 design listed SEMANTIC_CHANGE as an outcome the table could produce. It cannot - M12 and M19 are behaviourally identical, as the lab has asserted since T05. PRODUCT_DEFECT and SEMANTIC_CHANGE collapse into one escalating outcome, BEHAVIOUR_CHANGED, and the distinction becomes a human adjudication. INTENT_CHANGED survives but is detected by the claim fingerprint moving, not inferred from behaviour. Two classifier defects found and fixed rather than reported: claims downstream of a failed realization now yield INCONCLUSIVE rather than FAIL (a false accusation is the mirror image of a false adaptation), and the browser driver records a page signature so surface change is detectable when the interaction path is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
63 lines
4.3 KiB
Markdown
63 lines
4.3 KiB
Markdown
# Concept ↔ Implementation Fitness Map
|
|
|
|
**Updated:** 2026-08-22 (TD-WP-0002-T08)
|
|
|
|
Traces each important concept to the implementation, experiment and evidence that
|
|
support it. **Unsupported entries are the point of this map** — a concept with no
|
|
implementation and no evidence is not a gap to be embarrassed about, it is the
|
|
current honest state, and hiding it defeats the map's purpose.
|
|
|
|
Support levels follow `TestDriverImprovementLoop.md` §13:
|
|
`C0 Idea` · `C1 Hypothesis` · `C2 Experimentally Supported` ·
|
|
`C3 Practically Validated` · `C4 Architectural Invariant`
|
|
|
|
## Current state
|
|
|
|
`C-semantic-action` is the first concept to reach `C2`: it has an experiment
|
|
behind it (the T07 two-arm comparison), and that experiment narrowed the claim
|
|
rather than confirming it. Everything else still rests on unit tests.
|
|
|
|
The deterministic kernel exists (T04) and its guarantees are covered by unit
|
|
tests. **Levels do not move for those.** A passing unit test is not an experiment: it
|
|
shows the code does what its author intended, not that the concept holds under
|
|
the mutations it claims to survive. Levels rise when E-001/E-002/E-003 produce
|
|
evidence, not before. The implementation column below moves; the level column
|
|
does not. The initial classifications in §13 of the Improvement Loop
|
|
(`Actor Isolation C2`, `Independent Oracles C2`) are corrected downward here: they
|
|
were aspirational, not evidenced.
|
|
|
|
| Concept | Level | Implementation | Experiment | Evidence | Open question |
|
|
|---|---|---|---|---|---|
|
|
| `C-use-case` | C1 | `intent.py` | — | — | Is a use case expressible without leaking mechanics? |
|
|
| `C-actor-isolation` | C1 | `world.py` | E-001 | `td://self/actor-isolation` | **F-0003** — only observable when the scenario plants canaries. |
|
|
| `C-semantic-action` | **C2** | `actions.py`, `agentic.py` | E-001 (partial) | T07 arm comparison | **F-0005** — supported only where stable identifiers are absent. Narrower than the concept model claims. |
|
|
| `C-oracle-independence` | **C2** | `runner.py`, `oracles.py` | E-001, E-003 | — | Independence of components ≠ independence of belief. (H-004) |
|
|
| `C-evidence-pack` | C1 | `evidence.py` | — | `td://self/evidence-reproducibility` | Verdicts are reproducible from S3 alone, on passing and failing runs. |
|
|
| `C-observation-channel` | C1 | `lab/app.py` | — | — | **D-07** — required of every system under test. Adoption cost unknown. |
|
|
| `C-adaptation` | **C2** | `classification.py`, `agentic.py` | E-001 | T08 matrix | 11/12 mechanical absorbed without a human. |
|
|
| `C-classification` | **C2** | `classification.py` | E-001, E-003 | T08 matrix, FAR 0/7 | **F-0006** — cannot infer SEMANTIC vs DEFECT; collapses to one escalating outcome. |
|
|
| `C-crystallization` | C1 | — (T09) | E-002 | — | (H-003) |
|
|
| `C-intent-provenance` | C1 | `provenance.py` | E-003 | `td://self/intent-independence` | Constrains provenance, not quality. Accepted residual. |
|
|
| `C-lineage` | C0 | — | — | — | Parent pointer only in the spike. |
|
|
| `C-energy` | C0 | `energy.py`, capture only | — | — | Dormant by decision. (H-005) |
|
|
| `C-temperature` | C0 | — | — | — | Deferred. No implementation planned in TD-WP-0002. |
|
|
| `C-confidence` | C0 | — | — | — | Deferred. |
|
|
| `C-campaign` | C0 | — | — | — | Deferred. |
|
|
| `C-metabolism` | C0 | — | — | — | Deferred. Depends on C-energy. |
|
|
| `C-retirement` | C0 | — | — | — | Deferred. Depends on C-energy. |
|
|
| `C-security-mutation` | C1 | `lab/mutations.py` | E-003 | `lab/GROUND-TRUTH.md` | Catalogue is hand-written; no derivation mechanism from use cases yet. |
|
|
|
|
## Orphan check
|
|
|
|
**Conceptual orphans** — concepts with no planned implementation in `TD-WP-0002`:
|
|
`C-temperature`, `C-confidence`, `C-campaign`, `C-metabolism`, `C-retirement`.
|
|
|
|
All five are deferred *by explicit decision*, not oversight. They are the group
|
|
most at risk of being built because they are easy and satisfying, and never
|
|
validated. They are revisited at T10, where the question is not "when do we build
|
|
these" but "does the evidence justify keeping them in the model at all".
|
|
|
|
**Implementation orphans** — none. Every module in `src/testdriver/` traces to a
|
|
concept above. `energy.py` is the one to watch: it exists solely to capture
|
|
events for a dormant hypothesis, and if T10 finds no use for the history it
|
|
should be removed rather than kept out of sentiment.
|