tegwick 84848e9a0e T08: the classifier, measured and attacked

False Adaptation Rate = 0/7 across the labelled catalogue and the three E-003
attacks. 11 of 12 mechanical mutations absorbed without a human, so the safety
result is not bought by escalating everything.

- classification.py: total function over three signals, rule order chosen so
  every rule that could excuse a regression sits after the rule that reports
  one. SAFE_TO_ACCEPT is a two-element closed set, asserted.
- CompositeDriver plus scenarios/full_journey.py: one asset crossing both
  surfaces, so UI mutations are visible as surface differences while the
  claims they do not touch stay green.
- E-003: surface substitution (new M23), concurrent mechanical+defect,
  evidence starvation, provenance laundering. All held.

F-0006 (CONCEPT_DRIFT, resolved): the T02 design listed SEMANTIC_CHANGE as an
outcome the table could produce. It cannot - M12 and M19 are behaviourally
identical, as the lab has asserted since T05. PRODUCT_DEFECT and
SEMANTIC_CHANGE collapse into one escalating outcome, BEHAVIOUR_CHANGED, and
the distinction becomes a human adjudication. INTENT_CHANGED survives but is
detected by the claim fingerprint moving, not inferred from behaviour.

Two classifier defects found and fixed rather than reported: claims downstream
of a failed realization now yield INCONCLUSIVE rather than FAIL (a false
accusation is the mirror image of a false adaptation), and the browser driver
records a page signature so surface change is detectable when the interaction
path is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39

2026-08-23 00:02:58 +02:00

3.8 KiB

Raw Blame History

type

class

status

discovered

resolved

discovered_by

workplan

task

hypotheses

F-0006

framework-finding

CONCEPT_DRIFT

resolved

2026-08-22

TD-WP-0002-T08

TD-WP-0002

TD-WP-0002-T08

H-004

F-0006 — The classifier cannot infer a semantic change, only escalate

What the design said

docs/TestDriverClassificationDesign.md (T02) specifies a decision table with four outcomes, one of which is SEMANTIC_CHANGE:

| any | yes | unchanged, but the asset's declared claim set differs from the use case | SEMANTIC_CHANGE → human |

TestDriverInitialMilestones.md M6 states the same four-way path: IMPLEMENTATION CHANGE / INTENT CHANGE / PRODUCT DEFECT / AMBIGUOUS.

Both read as though a classifier could look at a run and determine that intended behaviour had changed.

What building it showed

It cannot, and the lab already contained the proof before the classifier existed.

M12 (revocation deferred by product decision, SEMANTIC) and M19 (revocation propagates after a delay, DEFECT) produce identical evidence: same failing assertion, same step, same snapshot, same audit trail. test_deferred_revoke_and_revoke_race_are_behaviourally_identical has asserted this since T05.

No quantity of evidence distinguishes them, because the difference is not in the system. It is in whether someone decided the new behaviour was wanted. A classifier that emitted SEMANTIC_CHANGE from evidence would be guessing, and guessing in the one direction the project cannot afford: SEMANTIC_CHANGE sounds benign, and "the team must have meant it" is exactly the reasoning that normalizes a defect.

Resolution

Path 2 — the concept is deliberately revised.

The classifier's output space is now:

Outcome	Safe to accept	Meaning
`UNCHANGED`	yes	no observable difference
`MECHANICAL_ADAPTATION`	yes	surface moved, every claim still holds
`BEHAVIOUR_CHANGED`	no	a claim that held no longer holds
`INTENT_CHANGED`	no	the recorded claim set itself moved
`REALIZATION_FAILED`	no	the action could not be performed
`AMBIGUOUS`	no	the evidence does not support a conclusion

BEHAVIOUR_CHANGED replaces both PRODUCT_DEFECT and SEMANTIC_CHANGE. Which of the two it is remains a real and important question — it is simply a question for a human, recorded as an adjudication, not inferred from a run.

INTENT_CHANGED is detectable, but note what makes it so: the claim fingerprint changed, meaning a human already edited what is being asserted. That is a fact about the recorded use case, not an inference about behaviour. This is what the T02 table's fourth row was really describing; it was filed under the wrong heading.

Why this makes the framework better, not weaker

Calling a semantic change a defect is a false alarm: a human looks, says "we meant that", and updates the claim — which then registers as INTENT_CHANGED with full provenance. Cost: one review.

Calling a defect a semantic change is a false adaptation: the thesis dies.

Collapsing the two into one escalating outcome makes the framework err only in the direction it can afford. The measured consequence is visible in the T08 matrix: 2 of 4 SEMANTIC mutations are escalated as BEHAVIOUR_CHANGED, which looks like imprecision and is in fact the design working.

Consequences

docs/TestDriverClassificationDesign.md decision table updated in place, with the original row preserved and marked.
TestDriverInitialMilestones.md M6's four-way path is superseded by this finding; the milestone's intent (do not normalize defects) is unchanged and is met.
H-004's falsification condition is unaffected: False Adaptation Rate over DEFECT-labelled mutations. Measured 0/7 at T08.

3.8 KiB Raw Blame History