coulomb/test-driver
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

T05: the lab and its labelled mutation catalogue

Browse source 
lab/app.py (users, tenants, auth, resources, sharing, read/write, revoke,
audit), lab/http_api.py (JSON API + browser UI, stdlib only), 20 labelled
composable version-stamped mutations, ground-truth matrix. 48 tests pass.

Detection against the reference scenario: MECHANICAL 0/10 flagged (correct),
DEFECT 6/6, SEMANTIC 2/4 with both inert cases declared.

- F-0002: M16 and M18 initially escaped detection entirely. A use case
  protects exactly what it asserts. Resolved by adding two claims already
  stated as intent in INTENT.md; the six-mutation catalogue would never have
  surfaced this.
- test-id axis added: stable selectors survive most UI mutations, which would
  make H-001 trivially false. Mutations now vary on preserves_test_ids so the
  hypothesis is analysed split by that axis rather than rigged.
- M12 (semantic deferred revoke) and M19 (defect race) are behaviourally
  identical and asserted as such - the discrimination problem as a test.

lab/minimal.py removed; superseded by lab/app.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 1629012@bnt-lap001
Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
This commit is contained in:
tegwick 2026-08-22 23:31:22 +02:00
parent 4d3421ca57
commit 4ddb2f896c
22 changed files with 1091 additions and 207 deletions
Download patch file Download diff file Expand all files Collapse all files

27
workplans/TD-WP-0002-vertical-spike-crystallization.md
View file

@ -222,7 +222,7 @@ Three things that came out of building it rather than designing it:
```task
id: TD-WP-0002-T05
status: todo
status: done
priority: high
state_hub_task_id: "595a89c2-1462-57fc-8b08-a5a6b875fd48"
```
@ -240,6 +240,31 @@ The lab is the measuring instrument for every claim the framework makes — a we
lab caps the credibility of all downstream results. It is also potentially the
project's first publishable artefact in its own right.
**Done 2026-08-22.** `lab/app.py` (users, tenants, auth, resources, sharing,
read/write, revoke, audit), `lab/http_api.py` (JSON API + browser UI, stdlib
only), `lab/mutations.py` (20 labelled, composable, version-stamped mutations),
`lab/GROUND-TRUTH.md`. 48 tests pass. Detection: MECHANICAL 0/10 flagged,
DEFECT 6/6, SEMANTIC 2/4.
Three results worth carrying:
- **F-0002 — two seeded defects were initially invisible.** M16 (READ grant
confers WRITE) and M18 (revocation unaudited) both passed cleanly: nothing
flaky, nothing `INCONCLUSIVE`, the framework simply had nothing to say. A use
case protects exactly what it asserts and not one thing more. Resolved by
adding two claims already stated as intent in `INTENT.md`. Had the six-mutation
catalogue from the milestones doc been used, this would never have surfaced —
the argument for the larger catalogue is now evidenced rather than asserted.
- **The test-id axis.** Stable `data-td` selectors survive most UI mutations,
which would make H-001 trivially *false*. Rather than rig the catalogue,
mutations now vary on `preserves_test_ids`, and H-001 must be analysed split by
that axis. A semantic action earns its keep exactly where stable identifiers
are absent or not carried forward — that is the honest shape of the claim.
- **M12 vs M19 are behaviourally identical.** A deliberate deferred-revocation
decision and a revocation race produce the same failure, same step, same
evidence. Only intent separates them. This is the discrimination problem in its
sharpest form and is now a test, not a paragraph.
## Out-of-band ground truth for self-verification
```task