coulomb/test-driver
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
test-driver/research/findings
History
tegwick 4ddb2f896c T05: the lab and its labelled mutation catalogue 
lab/app.py (users, tenants, auth, resources, sharing, read/write, revoke,
audit), lab/http_api.py (JSON API + browser UI, stdlib only), 20 labelled
composable version-stamped mutations, ground-truth matrix. 48 tests pass.

Detection against the reference scenario: MECHANICAL 0/10 flagged (correct),
DEFECT 6/6, SEMANTIC 2/4 with both inert cases declared.

- F-0002: M16 and M18 initially escaped detection entirely. A use case
  protects exactly what it asserts. Resolved by adding two claims already
  stated as intent in INTENT.md; the six-mutation catalogue would never have
  surfaced this.
- test-id axis added: stable selectors survive most UI mutations, which would
  make H-001 trivially false. Mutations now vary on preserves_test_ids so the
  hypothesis is analysed split by that axis rather than rigged.
- M12 (semantic deferred revoke) and M19 (defect race) are behaviourally
  identical and asserted as such - the discrimination problem as a test.

lab/minimal.py removed; superseded by lab/app.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
2026-08-22 23:31:22 +02:00
..
F-0001-milestone-sequence-drift.md T01: establish canonical milestone sequence, record F-0001 2026-08-22 23:07:31 +02:00
F-0002-scenario-coverage-gap.md T05: the lab and its labelled mutation catalogue 2026-08-22 23:31:22 +02:00
README.md T01: establish canonical milestone sequence, record F-0001 2026-08-22 23:07:31 +02:00

README.md

Framework Findings

Findings about test-driver itself, kept strictly separate from findings about any system under test. Classes are defined in docs/TestDriverImprovementLoop.md § 9:

PRODUCT_DEFECT · TEST_DEFECT · MECHANICAL_ADAPTATION · SEMANTIC_CHANGE · CONCEPT_DRIFT · FRAMEWORK_LIMITATION · EVIDENCE_FAILURE · UNNECESSARY_COMPLEXITY

File naming: F-NNNN-<slug>.md. Identifiers are stable and never reused.

Each finding carries frontmatter (id, class, status, dates, originating workplan/task) and, when resolved, states which of the three §5 resolution paths was taken. A finding that leaves an open question records it under "Residual" and names the task that will answer it — residuals are not parked in prose.

The wider research control plane (hypotheses/, experiments/, concepts/, decisions/) is built in TD-WP-0002-T03.