tegwick
4ddb2f896c
lab/app.py (users, tenants, auth, resources, sharing, read/write, revoke, audit), lab/http_api.py (JSON API + browser UI, stdlib only), 20 labelled composable version-stamped mutations, ground-truth matrix. 48 tests pass. Detection against the reference scenario: MECHANICAL 0/10 flagged (correct), DEFECT 6/6, SEMANTIC 2/4 with both inert cases declared. - F-0002: M16 and M18 initially escaped detection entirely. A use case protects exactly what it asserts. Resolved by adding two claims already stated as intent in INTENT.md; the six-mutation catalogue would never have surfaced this. - test-id axis added: stable selectors survive most UI mutations, which would make H-001 trivially false. Mutations now vary on preserves_test_ids so the hypothesis is analysed split by that axis rather than rigged. - M12 (semantic deferred revoke) and M19 (defect race) are behaviourally identical and asserted as such - the discrimination problem as a test. lab/minimal.py removed; superseded by lab/app.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39
3.1 KiB
Lab Ground Truth
Lab version base: lab-0.2.0 · Catalogue: 20 mutations · Scenario:
scenarios/alice_bob_carol.py
Labels are decided by a human from the use case and recorded before any run. They are never inferred from behaviour — that is the whole point, and M12/M19 below show why.
| ID | Mutation | Label | Test ids | Reference scenario |
|---|---|---|---|---|
| M01 | Sharing control moves into a modal | MECHANICAL | preserved | PASS |
| M02 | DOM rewritten, test ids not carried forward | MECHANICAL | dropped | PASS |
| M03 | API renames resource_id to id |
MECHANICAL | preserved | PASS |
| M04 | UI labels reworded | MECHANICAL | preserved | PASS |
| M05 | Form field order reversed | MECHANICAL | preserved | PASS |
| M06 | API paths shortened | MECHANICAL | preserved | PASS |
| M07 | Buttons become anchors | MECHANICAL | preserved | PASS |
| M08 | Revoke gains a confirmation step | MECHANICAL | preserved | PASS |
| M09 | Responses are slower | MECHANICAL | preserved | PASS |
| M10 | Denials return 401 instead of 403 | MECHANICAL | preserved | PASS |
| M11 | A share must be accepted first | SEMANTIC | preserved | FAIL |
| M12 | Revocation is deferred by decision | SEMANTIC | preserved | FAIL |
| M13 | Grants default to WRITE | SEMANTIC | preserved | PASS (inert) |
| M14 | Cross-tenant sharing declared prohibited | SEMANTIC | preserved | PASS (inert) |
| M15 | Revocation updates record but not enforcement | DEFECT | preserved | FAIL |
| M16 | A READ grant confers WRITE | DEFECT | preserved | FAIL |
| M17 | Any authenticated user can read anything | DEFECT | preserved | FAIL |
| M18 | Revocation is not audited | DEFECT | preserved | FAIL |
| M19 | Revocation propagates after a delay | DEFECT | preserved | FAIL |
| M20 | Tenant isolation leaks | DEFECT | preserved | FAIL |
Baseline: PASS. Detection: MECHANICAL 0/10 flagged (correct — semantics preserved), DEFECT 6/6 flagged, SEMANTIC 2/4 flagged.
The two inert mutations
Recorded rather than hidden. test_inert_semantic_mutations_are_declared fails
if an invisible mutation is ever left undeclared.
- M13 only affects grants that omit a permission; the reference scenario
passes
READexplicitly, so nothing changes. - M14 is a change of intent with no change of code — cross-tenant sharing was already enforced, and the mutation declares it deliberate. Nothing observable moves. This is the sharpest available demonstration that classification cannot be a diff.
M12 vs M19 — the discrimination problem in one row
Both produce an identical failure: c-bob-revoked, same step, same evidence.
One is a deliberate product decision that revocation batches; the other is a
propagation race. No observation distinguishes them. Only intent does.
This is why claims require independent provenance (D-06), why AMBIGUOUS
escalates to a human rather than resolving itself, and why T08's classifier is
not permitted to guess.
Regenerating
The matrix is asserted in tests/test_lab_ground_truth.py. A moved cell fails
the suite: changing the measuring instrument must be a deliberate, reviewed act.