T03: research control plane
H-001..H-005 with explicit falsification conditions, E-001..E-003, the Concept-Implementation Fitness Map, findings log and ID convention. Plain files, no tooling. - fitness map corrects Improvement Loop section 13 levels downward: nothing exceeds C1 without implementation - H-005 (Energy) dormant by decision - events captured, no scoring written - H-001 gets a genuine control arm so the semantic-action thesis is not trivially true - E-003 added: deliberately attacks the safety boundary rather than only sampling benign cases Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
parent
15b55c807a
commit
bec7fdc465
11 changed files with 430 additions and 0 deletions
41
research/hypotheses/H-001-semantic-action-stability.md
Normal file
41
research/hypotheses/H-001-semantic-action-stability.md
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
id: H-001
|
||||
title: Semantic Action Stability
|
||||
status: PROPOSED
|
||||
created: "2026-08-22"
|
||||
experiments: [E-001]
|
||||
concepts: [C-semantic-action]
|
||||
---
|
||||
|
||||
# H-001 — Semantic Action Stability
|
||||
|
||||
## Claim
|
||||
|
||||
A semantic action survives implementation restructuring better than a recorded UI
|
||||
interaction sequence.
|
||||
|
||||
## Falsification condition
|
||||
|
||||
Across the labelled mechanical mutations in the lab (T05), a recorded interaction
|
||||
sequence survives **at least as many** mutations as the semantic action does.
|
||||
|
||||
If mechanics-free identity buys no measurable durability, the central abstraction
|
||||
is decorative and `SemanticAction` should be reduced to a naming convention.
|
||||
|
||||
## Measurement
|
||||
|
||||
Mechanical Recovery Rate for each of two arms over the same mutation set:
|
||||
- **arm A** — semantic action realized by an agentic driver;
|
||||
- **arm B** — a recorded selector-based sequence captured against the baseline.
|
||||
|
||||
Arm B is a genuine control and must be run, not assumed to fail.
|
||||
|
||||
## Threats to validity
|
||||
|
||||
The comparison is unfair if arm B is built naively — a brittle straw man makes
|
||||
H-001 trivially true and worthless. Arm B uses the most robust selector strategy
|
||||
reasonably available (roles, labels, test ids where the lab provides them).
|
||||
|
||||
## Status log
|
||||
|
||||
- 2026-08-22 `PROPOSED`. No evidence.
|
||||
40
research/hypotheses/H-002-mechanical-adaptation.md
Normal file
40
research/hypotheses/H-002-mechanical-adaptation.md
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
id: H-002
|
||||
title: Mechanical Adaptation
|
||||
status: PROPOSED
|
||||
created: "2026-08-22"
|
||||
experiments: [E-001]
|
||||
concepts: [C-adaptation]
|
||||
---
|
||||
|
||||
# H-002 — Mechanical Adaptation
|
||||
|
||||
## Claim
|
||||
|
||||
An agentic driver can recover from a mechanical implementation change without
|
||||
modifying the semantics of the protected use case.
|
||||
|
||||
## Falsification condition
|
||||
|
||||
Either:
|
||||
- the driver fails to recover from a majority of labelled `MECHANICAL` mutations
|
||||
(recovery is not achievable in practice); **or**
|
||||
- recovery is achieved only in runs where claims or invariants were altered
|
||||
(recovery is achievable but not semantics-preserving).
|
||||
|
||||
The second branch matters more than the first. A high recovery rate purchased by
|
||||
weakening assertions falsifies this hypothesis just as decisively as no recovery
|
||||
at all.
|
||||
|
||||
## Measurement
|
||||
|
||||
- Mechanical Recovery Rate over `MECHANICAL`-labelled mutations.
|
||||
- Claim/invariant diff per run — expected to be empty by construction
|
||||
(`docs/TestDriverClassificationDesign.md` D-02). Any non-empty diff is both a
|
||||
falsification signal **and** a framework defect, since no write path should exist.
|
||||
|
||||
## Status log
|
||||
|
||||
- 2026-08-22 `PROPOSED`. Design decision D-02 makes the second falsification
|
||||
branch structurally unreachable; the measurement is retained anyway, as an
|
||||
assertion that the architecture is what we believe it is.
|
||||
43
research/hypotheses/H-003-crystallization.md
Normal file
43
research/hypotheses/H-003-crystallization.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
id: H-003
|
||||
title: Crystallization
|
||||
status: PROPOSED
|
||||
created: "2026-08-22"
|
||||
experiments: [E-002]
|
||||
concepts: [C-crystallization]
|
||||
---
|
||||
|
||||
# H-003 — Crystallization
|
||||
|
||||
## Claim
|
||||
|
||||
A sufficiently stable agentic execution can be converted into deterministic test
|
||||
code without losing relevant oracle coverage.
|
||||
|
||||
## Falsification condition
|
||||
|
||||
Any of:
|
||||
- the generated deterministic test cannot be produced without human authorship of
|
||||
the assertions;
|
||||
- it drops one or more oracles present in the agentic ancestor;
|
||||
- it passes where the ancestor would have failed, or vice versa, on the same lab
|
||||
version;
|
||||
- it costs no less to execute than the agentic ancestor.
|
||||
|
||||
## Measurement
|
||||
|
||||
- Crystallization success rate.
|
||||
- Oracle set of descendant vs ancestor — must be a superset or equal.
|
||||
- Verdict agreement between ancestor and descendant across lab versions.
|
||||
- Execution cost ratio (tokens, wall time) descendant : ancestor.
|
||||
|
||||
## Note on the cost criterion
|
||||
|
||||
The cost criterion is included deliberately. If crystallization preserves
|
||||
semantics but saves nothing, the thesis is intact but the product rationale is
|
||||
not — and that distinction should be visible in the record rather than discovered
|
||||
by a user.
|
||||
|
||||
## Status log
|
||||
|
||||
- 2026-08-22 `PROPOSED`. No evidence.
|
||||
46
research/hypotheses/H-004-independent-judgment.md
Normal file
46
research/hypotheses/H-004-independent-judgment.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
id: H-004
|
||||
title: Independent Judgment
|
||||
status: PROPOSED
|
||||
created: "2026-08-22"
|
||||
experiments: [E-001, E-003]
|
||||
concepts: [C-oracle-independence, C-intent-provenance]
|
||||
---
|
||||
|
||||
# H-004 — Independent Judgment
|
||||
|
||||
## Claim
|
||||
|
||||
Separating actor execution from deterministic oracles reduces false-positive
|
||||
adaptation to defective behaviour.
|
||||
|
||||
## Falsification condition
|
||||
|
||||
The framework classifies any seeded `DEFECT`-labelled mutation as
|
||||
`MECHANICAL_ADAPTATION` — i.e. **False Adaptation Rate > 0** over the labelled set.
|
||||
|
||||
This is the project's existential safety metric. A non-zero result is a
|
||||
stop-and-redesign signal, not a tuning exercise.
|
||||
|
||||
## Measurement
|
||||
|
||||
- False Adaptation Rate over `DEFECT`-labelled mutations. Target: exactly 0.
|
||||
- Classification precision/recall over the full labelled set.
|
||||
- Rate of `AMBIGUOUS` escalation — tracked as a cost, not as a failure. A system
|
||||
that escalates everything is safe and useless.
|
||||
|
||||
## Relationship to the design
|
||||
|
||||
`docs/TestDriverClassificationDesign.md` argues that FAR = 0 follows
|
||||
architecturally from D-02 (claims are run inputs with no adaptation write path)
|
||||
rather than from classifier accuracy. This hypothesis therefore tests the
|
||||
**architecture**, not a model.
|
||||
|
||||
The most plausible route to falsification is not a misclassification but
|
||||
**surface-substitution**: an actor routing around a broken authorization control
|
||||
via another surface and scoring as a successful recovery. D-05 exists to close
|
||||
that route; E-003 exists to attack it deliberately.
|
||||
|
||||
## Status log
|
||||
|
||||
- 2026-08-22 `PROPOSED`. No evidence.
|
||||
40
research/hypotheses/H-005-verification-energy.md
Normal file
40
research/hypotheses/H-005-verification-energy.md
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
id: H-005
|
||||
title: Verification Energy
|
||||
status: PROPOSED
|
||||
created: "2026-08-22"
|
||||
experiments: []
|
||||
concepts: [C-energy]
|
||||
---
|
||||
|
||||
# H-005 — Verification Energy
|
||||
|
||||
## Claim
|
||||
|
||||
Historical evidence about defects caught, adaptations required, false positives
|
||||
and duplication can identify verification assets whose continued execution is
|
||||
more valuable than others.
|
||||
|
||||
## Falsification condition
|
||||
|
||||
Energy scores computed from event history fail to rank assets better than a
|
||||
trivial baseline (e.g. recency, or uniform priority) when predicting which assets
|
||||
subsequently catch a defect.
|
||||
|
||||
## Why no experiment is planned yet
|
||||
|
||||
**This hypothesis is not testable at the current scale and is deliberately
|
||||
dormant.** Validating it requires event history across many assets over months —
|
||||
history the spike will not accumulate. Implementing a scoring function now would
|
||||
produce a number that cannot be checked, which is worse than no number.
|
||||
|
||||
`TD-WP-0002` therefore records **raw immutable `EnergyEvent`s from the first run**
|
||||
and implements no scoring, decay, or selection logic. Events cannot be
|
||||
reconstructed later; scores can always be computed later.
|
||||
|
||||
This is the hypothesis most likely to be **cheaply built and never validated**,
|
||||
which is precisely why it is fenced off.
|
||||
|
||||
## Status log
|
||||
|
||||
- 2026-08-22 `PROPOSED`, dormant by decision. Event capture only.
|
||||
Loading…
Add table
Add a link
Reference in a new issue