test-driver/research/hypotheses/H-004-independent-judgment.md

---
id: H-004
title: Independent Judgment
status: EXPERIMENTING
created: "2026-08-22"
experiments: [E-001, E-003]
concepts: [C-oracle-independence, C-intent-provenance]
---

# H-004 — Independent Judgment

## Claim

Separating actor execution from deterministic oracles reduces false-positive
adaptation to defective behaviour.

## Falsification condition

The framework classifies any seeded `DEFECT`-labelled mutation as
`MECHANICAL_ADAPTATION` — i.e. **False Adaptation Rate > 0** over the labelled set.

This is the project's existential safety metric. A non-zero result is a
stop-and-redesign signal, not a tuning exercise.

## Measurement

- False Adaptation Rate over `DEFECT`-labelled mutations. Target: exactly 0.
- Classification precision/recall over the full labelled set.
- Rate of `AMBIGUOUS` escalation — tracked as a cost, not as a failure. A system
  that escalates everything is safe and useless.

## Relationship to the design

`docs/TestDriverClassificationDesign.md` argues that FAR = 0 follows
architecturally from D-02 (claims are run inputs with no adaptation write path)
rather than from classifier accuracy. This hypothesis therefore tests the
**architecture**, not a model.

The most plausible route to falsification is not a misclassification but
**surface-substitution**: an actor routing around a broken authorization control
via another surface and scoring as a successful recovery. D-05 exists to close
that route; E-003 exists to attack it deliberately.

## Result (TD-WP-0002-T08)

**False Adaptation Rate = 0/7.** No `DEFECT`-labelled mutation was classified as
safe to accept, including the three deliberate attacks in E-003.

| Ground truth | Accepted without a human |
|---|---|
| MECHANICAL (12) | 11 |
| SEMANTIC (4) | 2 — both genuinely inert for this scenario |
| DEFECT (7) | **0** |

E-003 attacks, all held:

- **surface substitution** (M23, UI control removed, API left open) → `AMBIGUOUS`.
  The driver did not route around; discovery failed loudly.
- **concurrent mechanical + defect** (M01+M15, M02+M17, M21+M20) →
  `BEHAVIOUR_CHANGED`, with the reason explicitly noting that the coincident
  surface change does not excuse it.
- **evidence starvation** → `AMBIGUOUS`, never a pass.
- **provenance laundering** → rejected at authoring, and caught in the record.

Note what this does and does not establish. FAR = 0 follows largely from
*architecture* — claims are run inputs with no adaptation write path (D-02), and
`SAFE_TO_ACCEPT` is a two-element closed set. The experiment confirms the
architecture behaves as designed over 23 mutations; it does not establish that
the architecture is correct for mutations nobody thought of. That distinction
should survive into any external claim.

The other side of the trade is asserted too: a classifier that escalated
everything would score a perfect FAR and be useless.
`test_mechanical_changes_are_mostly_absorbed` pins 11 of 12.

## Status log

- 2026-08-22 `PROPOSED`. No evidence.
- 2026-08-22 `EXPERIMENTING`. FAR 0/7 over the labelled set plus E-003 attacks.
  Not promoted to SUPPORTED: 23 hand-written mutations is a small, self-chosen
  sample.