coulomb/test-driver
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Complete local generalisation tasks and document remaining experiment blockers

Browse source 
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a0e76f-be98-7ae3-965d-e0b31290a4c4
This commit is contained in:
tegwick 2026-09-28 12:06:24 +02:00
parent d54af628df
commit cf682281d7
36 changed files with 5394 additions and 279 deletions
Download patch file Download diff file Expand all files Collapse all files

17
research/experiments/E-001-mechanical-vs-defect.md
View file

@ -1,7 +1,7 @@
---
id: E-001
title: Mechanical recovery and defect discrimination over the labelled mutation set
status: PLANNED
status: EXECUTED
hypotheses: [H-001, H-002, H-004]
task: TD-WP-0002-T08
created: "2026-08-22"
@ -37,3 +37,18 @@ tokens · wall time · retries.
`EXECUTED` 2026-08-22 (T08). Arm A run over 23 mutations; arm B over
the 12 mechanical ones. Results in H-001 and H-004. FAR 0/7.
## Expanded measurement — 2026-09-28 (TD-WP-0003-T03)
38 mutations, including ten that drop test ids and seven new paired structural
controls. Discovery and recorded selectors both pass 17/17 preserved-id cases;
on dropped ids discovery passes 9/10 and recorded selectors 0/10. M22 remains
unrealizable. The full journey accepts 26/27 mechanical cases; FAR remains 0/7
on the unchanged defect set. Claim/provenance indices remain unchanged.
Reproduce: `PYTHONPATH=src:. python3 tools/measure_e001.py /tmp/e001.json`.
[Raw result](../evidence/2026-09-28-e001.json).
These selected and correlated synthetic HTML cases do not establish population
reliability, visual/browser coverage, model capability or economics. See
[review](../../docs/TestDriverGeneralisationReview.md) for the mechanism list and
limits. H-001 remains supported only in the narrowed identifier-loss setting.