From 4ddb2f896cea4fa8743739bf2e3edb2f01a0493e Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 22 Aug 2026 23:31:22 +0200 Subject: [PATCH] T05: the lab and its labelled mutation catalogue lab/app.py (users, tenants, auth, resources, sharing, read/write, revoke, audit), lab/http_api.py (JSON API + browser UI, stdlib only), 20 labelled composable version-stamped mutations, ground-truth matrix. 48 tests pass. Detection against the reference scenario: MECHANICAL 0/10 flagged (correct), DEFECT 6/6, SEMANTIC 2/4 with both inert cases declared. - F-0002: M16 and M18 initially escaped detection entirely. A use case protects exactly what it asserts. Resolved by adding two claims already stated as intent in INTENT.md; the six-mutation catalogue would never have surfaced this. - test-id axis added: stable selectors survive most UI mutations, which would make H-001 trivially false. Mutations now vary on preserves_test_ids so the hypothesis is analysed split by that axis rather than rigged. - M12 (semantic deferred revoke) and M19 (defect race) are behaviourally identical and asserted as such - the discrimination problem as a test. lab/minimal.py removed; superseded by lab/app.py. Co-Authored-By: Claude Opus 5 Assistant: claude-code Assistant-Model: opus Assistant-Process: 1629012@bnt-lap001 Assistant-Session: 78d4fb13-8a1e-474b-87a3-9b9261c49a39 --- WORK-RECORDS.md | 4 +- lab/GROUND-TRUTH.md | 61 ++++ lab/__pycache__/app.cpython-312.pyc | Bin 0 -> 16581 bytes lab/__pycache__/http_api.cpython-312.pyc | Bin 0 -> 11471 bytes lab/__pycache__/mutations.cpython-312.pyc | Bin 0 -> 12948 bytes lab/app.py | 309 ++++++++++++++++++ lab/http_api.py | 215 ++++++++++++ lab/minimal.py | 191 ----------- lab/mutations.py | 205 ++++++++++++ research/concepts/fitness-map.md | 6 +- .../findings/F-0002-scenario-coverage-gap.md | 74 +++++ .../alice_bob_carol.cpython-312.pyc | Bin 6248 -> 7840 bytes scenarios/alice_bob_carol.py | 38 ++- .../__pycache__/observers.cpython-312.pyc | Bin 2724 -> 2910 bytes src/testdriver/observers.py | 3 + ...el_guarantees.cpython-312-pytest-7.4.4.pyc | Bin 16008 -> 16114 bytes ..._ground_truth.cpython-312-pytest-7.4.4.pyc | Bin 0 -> 23264 bytes ...ence_scenario.cpython-312-pytest-7.4.4.pyc | Bin 13455 -> 13438 bytes tests/test_kernel_guarantees.py | 11 +- tests/test_lab_ground_truth.py | 148 +++++++++ tests/test_reference_scenario.py | 6 +- ...-WP-0002-vertical-spike-crystallization.md | 27 +- 22 files changed, 1091 insertions(+), 207 deletions(-) create mode 100644 lab/GROUND-TRUTH.md create mode 100644 lab/__pycache__/app.cpython-312.pyc create mode 100644 lab/__pycache__/http_api.cpython-312.pyc create mode 100644 lab/__pycache__/mutations.cpython-312.pyc create mode 100644 lab/app.py create mode 100644 lab/http_api.py delete mode 100644 lab/minimal.py create mode 100644 lab/mutations.py create mode 100644 research/findings/F-0002-scenario-coverage-gap.md create mode 100644 tests/__pycache__/test_lab_ground_truth.cpython-312-pytest-7.4.4.pyc create mode 100644 tests/test_lab_ground_truth.py diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index eeeaba5..0ce0894 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -11,12 +11,12 @@ | workplan | TD-WP-0001 | active | — | workplans/TD-WP-0001-statehub-bootstrap.md | | workplan | TD-WP-0002 | active | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | | task | TD-WP-0001-T01 | done | — | workplans/TD-WP-0001-statehub-bootstrap.md | -| task | TD-WP-0001-T02 | wait | — | workplans/TD-WP-0001-statehub-bootstrap.md | +| task | TD-WP-0001-T02 | done | — | workplans/TD-WP-0001-statehub-bootstrap.md | | task | TD-WP-0001-T03 | done | — | workplans/TD-WP-0001-statehub-bootstrap.md | | task | TD-WP-0002-T01 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | | task | TD-WP-0002-T02 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | | task | TD-WP-0002-T03 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | -| task | TD-WP-0002-T04 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | +| task | TD-WP-0002-T04 | done | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | | task | TD-WP-0002-T05 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | | task | TD-WP-0002-T06 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | | task | TD-WP-0002-T07 | todo | — | workplans/TD-WP-0002-vertical-spike-crystallization.md | diff --git a/lab/GROUND-TRUTH.md b/lab/GROUND-TRUTH.md new file mode 100644 index 0000000..e2f4067 --- /dev/null +++ b/lab/GROUND-TRUTH.md @@ -0,0 +1,61 @@ +# Lab Ground Truth + +**Lab version base:** `lab-0.2.0` · **Catalogue:** 20 mutations · **Scenario:** +`scenarios/alice_bob_carol.py` + +Labels are decided by a human from the use case and recorded **before** any run. +They are never inferred from behaviour — that is the whole point, and M12/M19 +below show why. + +| ID | Mutation | Label | Test ids | Reference scenario | +|---|---|---|---|---| +| M01 | Sharing control moves into a modal | MECHANICAL | preserved | PASS | +| M02 | DOM rewritten, test ids not carried forward | MECHANICAL | **dropped** | PASS | +| M03 | API renames `resource_id` to `id` | MECHANICAL | preserved | PASS | +| M04 | UI labels reworded | MECHANICAL | preserved | PASS | +| M05 | Form field order reversed | MECHANICAL | preserved | PASS | +| M06 | API paths shortened | MECHANICAL | preserved | PASS | +| M07 | Buttons become anchors | MECHANICAL | preserved | PASS | +| M08 | Revoke gains a confirmation step | MECHANICAL | preserved | PASS | +| M09 | Responses are slower | MECHANICAL | preserved | PASS | +| M10 | Denials return 401 instead of 403 | MECHANICAL | preserved | PASS | +| M11 | A share must be accepted first | SEMANTIC | preserved | FAIL | +| M12 | Revocation is deferred by decision | SEMANTIC | preserved | FAIL | +| M13 | Grants default to WRITE | SEMANTIC | preserved | PASS *(inert)* | +| M14 | Cross-tenant sharing declared prohibited | SEMANTIC | preserved | PASS *(inert)* | +| M15 | Revocation updates record but not enforcement | DEFECT | preserved | FAIL | +| M16 | A READ grant confers WRITE | DEFECT | preserved | FAIL | +| M17 | Any authenticated user can read anything | DEFECT | preserved | FAIL | +| M18 | Revocation is not audited | DEFECT | preserved | FAIL | +| M19 | Revocation propagates after a delay | DEFECT | preserved | FAIL | +| M20 | Tenant isolation leaks | DEFECT | preserved | FAIL | + +Baseline: PASS. Detection: **MECHANICAL 0/10 flagged** (correct — semantics +preserved), **DEFECT 6/6 flagged**, **SEMANTIC 2/4 flagged**. + +## The two inert mutations + +Recorded rather than hidden. `test_inert_semantic_mutations_are_declared` fails +if an invisible mutation is ever left undeclared. + +- **M13** only affects grants that omit a permission; the reference scenario + passes `READ` explicitly, so nothing changes. +- **M14** is a change of *intent* with no change of code — cross-tenant sharing + was already enforced, and the mutation declares it deliberate. Nothing + observable moves. This is the sharpest available demonstration that + classification cannot be a diff. + +## M12 vs M19 — the discrimination problem in one row + +Both produce an identical failure: `c-bob-revoked`, same step, same evidence. +One is a deliberate product decision that revocation batches; the other is a +propagation race. **No observation distinguishes them.** Only intent does. + +This is why claims require independent provenance (D-06), why `AMBIGUOUS` +escalates to a human rather than resolving itself, and why T08's classifier is +not permitted to guess. + +## Regenerating + +The matrix is asserted in `tests/test_lab_ground_truth.py`. A moved cell fails +the suite: changing the measuring instrument must be a deliberate, reviewed act. diff --git a/lab/__pycache__/app.cpython-312.pyc b/lab/__pycache__/app.cpython-312.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d0d6b766bda7f5bc219c7c723603a2ab2712f6bc GIT binary patch literal 16581 zcmch8ZEzE3o@Yzy*1IiRGJeA^4YmOpYzToNyd^Q%5FnVD;7k^!h(=bojX;(>-4e!% znDK1bTYGNiFgv@2Q@eBVZq44pk*amwexdHRu9&I&kgBV?laTi;x^lI-+5IwgU!0hm znazE<-~Z`uNtQ+GGJACme)PP*{@N$_}A1>spS+^c_ zeOu}_)U4Zpx{X`vHnK7OC<~ZnA-}XuYP!RR0<*SYb9#yAxUp-pn3C1hfTYB3%8D3| zjEldV`(rVMe05e$$x~uFA)$aO4Z7WDmE=r9oKTWetjgGdL!v6nlDOwPXt{qXmWWM7 z;)Bz(-`OW7k~5+rPbF{4QgBdwTa^`69GBzC#H5%?x>Ir@l1QohMCwnG)nrRKelTE-HS zgW?DlGApX7NJ0673EtoHKlh*>z0s; zC88;eLJcWv=2|i?i_^(iB6ZB|?q_-Z{c>U=iCIs{i4^OpzaRUc$lpz4uf7wcwlg%b zNCLy9u!giMtApaX6lNtyZ>Vk&lS*AnDzQw24Mr=9qBcfUWOOZKzr_9-1zXl{oC>;Q z-J*gRVOuXtTl$gY2|1b)<2X@uur%A`IF8_^J`!z9LXNWuUn=LH*0V8pMUF@+_7Ibh zjIESn3VMyt(r}pQw2~Z`v3C`FkW8lrk`n{tC}ztvHZKMMnwAk@n~Z_c|98{qPZGze zKS_?p=upMsfdfZyLPH50c_fN0k0ujpI*#L*6qAWKoodC5W|HZ+B;vGV6SI|2yjvxx zaw~FdYC6u2M_+Yvf*qWaj$`5qj-KEeOJ=z`w4>>!6|ojmgW^!a{el@)R*$j?C6Xxs ze<~IwIL3it+medJHm#zv8(01WxQ?8t8@-1fCIOPEW^U>~KY*3xiW;>EL)e7MH@v zBLX24#TlO!WndE!p!-)u9)lLm12|0@U5^H>noEkL02mQf)r5(d9G8MN z%{G*n)tsjzadaD(HOHI4b0rc-ZJwxyCNSEOaiMtNz~I5b15tzavL6sZ`Yao}5dI-} zgUp32;15>G0)AkV_<3#uoZ?Ph2(eq1kZ_p`3YvXbPGE(Z;25Yw#i=qnfD@kpuH)!X zXwZYfsKFfAXcWhvW#b?se`2}MrC2rv3$`rxW3a2N<&nWWEL_%_<-!Kz;J8`tp=A{7 z7SuH8q2>&SL3UH&ux2NkmsW$k#uGGXc0i7rOel@$R&#~JY~R9Rm6j}u;@p=#`>!RZ zAMU7tDU%?0VW3Tq(6hXnv+JfM>bKIxSx&=qx*}Bxc>J*F3 z9rOH;JsT*(`TgoASVPdR1d!5fYD&@PVgnte*#QU>8b6v$$U!@;BN3UBu?}}QJe8Dy zrHFgO;qRs+aihc$4ok^sILs!c)fr2Q%JFzO9JDA+7)GUCdtKRvQ1O~U9~IC-etU;| z#=ra<-YRr{!68tZ%_?JnHbIRiQ|c!0Fiy>~@oiKr1IODC8DyJeJ%L=wh8SOp2`pw0 z@sb1iP9r6-l*>rjS;|eRS$oix{vMTzEC)8p5Hl`LLnm_diqfu5933c9FIUpQQWUD#AQR3%73KZ1vx zzBz0sdnlKJ0sN`^5&Qx7S$*RN7mJ?y56%}^tZV1og~j#)zq8o6WA6N-yTI=-VypZP zy)Wt^ioeol(_@y6eTckZdugL>Y~v`#HhNY7!0;qf=-Xq(x?Q?RLCY3WkO~r`LB49q z%3h43&`=digQ&D>``H)TK|CE!g zKedJgz=z#{4_|CiIS}TYvg>1~iRU3Vgh;!@pkhmzNAjS|TeF>4szX_Q&9XYl zhq8uJS$)NLKB*C9ekzj!p$4f5sckIP9BPzWkZLtjeyI(qc9z;63P>GDb+S}fs7dNZ zN@S@WAqbdA?PRH*P&2F9YqV=&sXmt46>4SsvYVy4LT#*O<{So zx{M1Frrx!V{0_{7q(_F>8DA<5q3>N#$9>}9;NVrQ(U3;M=Au+kdLP7`2n5UU1f!_4 z8J`r3ru0@sR<3FSIJlx_>dX1)@~Y+tGuonN+R6px7$lLaS2dS;K5AwME323ib+S+V zW=W5^s@W$M(u6XNap)cIRxkq0feBp-aekdE@L{lp(ac^`*m?Ke-8zKq#=+2v_3p=$ z$@o>1IXI^xvtg3wGr@8*)HBLA&&F)j#3mIkvyU+Jkui;@o;5F<7WyRRYZ8-V9|W>^ zGLq7o^!%_S$1$)9o|ceQg=7V-4mtp&rm!xjoY7hhH6YBizA!0e)2RqxKx;K*s<27a z)nuFrn2bqjjo9I7NSiXKeVECCnh)gwpVEk$J(eKw(;Ct-qkqgKrC|O%;5?;ykuN2u zFqv6MjRM0E&4oJrPr;j6KtMQiHYn$aE}BDyT7sR%5XSRx|FL#%@TPWJfY(fNfef#u0%j z!;-c9Dfd(BZClEa7fE)6#Aq|k5n1j!K~U9~wGg#KP42oO0#{bZGKQPALD;p9W_ZB- zK+wjxq(W>`c?AL3s~Bbght@$eBSw{}uQ{&h zZ(f{PjO7}{dEwKpoy$iaoLfm3x=zmXciju_#Upv=_F`8bQr-pc;@gz+HO+e&XWq0u z4)o@s?O8e7iD zuoFQ8CxI&~n#HID}!0uVUl)l&h3WB+g1CPO_RL?Y58=<{DC% zr~XMc)PS%p@gECi__K0ZAh=RDo&Dkqa@Rl z*B}RNGK@ehF_@Ahl7VheW=}*N(7UG@_QCP8czZYDVVZ6 ztq|qb95`(f$aE05mqA>!CZ{z%qD-m^0{{n1R8kzmh#k!U1Nd|>a2U3Q68IbG@1dSb zq;-z_%;Q`0bU*cUf9h?xdw${kVrs2rZ@y*ks&^jDnK$_h4fd}IczMX43^!W0se<Xi|h94 zOuI>UwwBGKwaMBJ9C>m!zw1pfulj}h#rN~h9-_*g1<&FeOOq?d=RK>=*NP2IYYn^d z4Z9wAaz3Euq1vd)_VmA??-!wFtv$?IP1L(>%i8AZQgk}(t3k4BA|`q;*n`?q(VEA3 zI?cYyzg0}YT`nH%A`YnSbfNv(;TOYzc$iIQ}5+-Zml@yO(j1@&*}sM zP}@Mu{a5e3x)$im2l|$86aok5okdR|=h;pu)V6o!^s4`roc9%?%8Vi}zrX5xnU%ft zxOeT?oB3mJuKF+Lyca7#Vdcsb=O=CX*DvRMV>##8SK!$%F^S(1CipD>{~0E1Szcmx zGQdxXh+_f9C)pX!JIXjOIVBgvZ}+G|9IgU1TDF45j6x=0f(=d3<7tXJC=P-!wiROB z3}W=U3}lEn1cfpiF4G0&eH5vLK|~x3QtT&~Va>}x(^dgAn~VO0dZ6ijJ*fHq!uOYE z9(Y!LhZxixe0*r__{IG3i>v<8oOhI@W9K5IyzM4&IlJ`E%3dNauNE8HmJa27V$Lb* zAXFj%m;@m_BP+iK)A}7j2>kd5&a5CY*)D6j0eMF8R1k_10b)`iF|w*p83ZVlcz(GM zY7AyY-wVo1^7WQ6OD)-$HHeVXSR4GZyg6n9^205QN;ky(ZF|;^`b1!=K2o+OOcq80 zUDc<21MRBvD4Deq`MXXg$Er^fQUNpTkr6Qo@O3iCR(<7NV(gKW?W#{lsjMw)|Jd=! zIDM?jnYA;ta*ZkqT4Y+;EgRiiFcvC;j*83_km-vmNmU)tja@9Z#S2dRm+Y;_`ag{}|&d}CGTo)m(zCu%!F6DfL%Ig0{FvqReb4{%; zAn!}J9ys#d2UgBL?LN*B^W}p7P|ka(NV33H6|d(C?=?>)G3^lzj8K`wZQ*Sv4!y>EQd zzUmz_C`8d4SbXQf-eSv+2eEa5b2UBZY%ce@lWS~WI-2wK<(z$6r4T}1MlMzm`Fi9c z#+85=Jyd^E36)7o0dm@EDc@x!;YDhME-_{;kV1HU3_@Q?V1RBPRX#u_lx}8hdT1>} z4Bf7f0ZNv}AQ6^C3eADb)M!L8fUR1LVBJ+o5Df%EdS|-nDbrHH3n~l#J?5f*3jsI* z`9L-<{4wQ~;gl{~2`tR9UmTUV|o^)D9H02A9ZS zAzw*LVbiJpBv@ihu*)*DR-y}i)S%O~D6vl4rBV+J=!o1@eabfUS%Wf5Jc%+{8}!mL zFyaD;0i)BPx@|S6u5LxuyoNP3G69K(?o7$%s*u>JLH(lypw&}`Rwe)|r%;7@{~Jy)idaq9UVU<~;63vs^VItmXjWb0 z-3tpB7<#<>pg-4fcqO&!J;A!|cCO$>^Mf z$cyOQ4+1!LLqI{b(wAj6SW|wzOh8-Nw^5Hk!vsy}K$gr)n@tXI^B`7iIAY&QMWSRqDmD7A$W;#_V5Wn+Cl^lU zJiVmZoV<5(xoLT3#Zw5J$YEQ3fi+)W-q*K0`QZJ6?|9C6d@H9QfM+{snqq4SK{fZK zBrY<2)iO)g112m%w1g<3SZ3iRQOodDySJ=mqST|rQxDi}Ve|}>dBzkXL#9!H4qKNx zh}LYT>&hIm%5YCQCX?&`j%vP%;H%)i7-)qyxOg`2=~#9=_3Xx`l3eiq!uxk2*Td%W zPOkfK!S{{FyYs%GoO5XFc9!V3_`B0@QvYjjr>)l3Nqm`h)+#SE3IJLC&Re(Cbf@XQ6WW4 z`9lhbE^gKLhz4p-3AO;bN~%)&ln+swF--$o@bAB)R3$?Fb)h1#{r-u2Czk71tcAeK zYk?#Az>&v}Lg017oY4KV;lCRF^U*@@p_N-tdtW0s^7t38-H+dkFCYCl`baJ`AIf_U z5xg~QUwQ{(+=HgS-u`fVg;fcZ#-198M5o|)qg%aNF;D&ehQ-<`NwJ_)zlvbFyUbkW zJ&F-LDy-&qs>w6^JZq)rvi5vM|E%tAs_U$%J4M~F{u2}{_ewF;yizJ>N(8G(x^sCX zji+1IO-m_PWrO~d>J!Hgx@-2w7#{1nAC)qV<@;IVLR4Q7b@PH5m3t@+_^@(st0wds z_4HD;drCKQHSejR%Ol}$kGysH+@(?F5VCZ)mBSPxrlhddl&;})-hYgm@^ppFE|v7_ zrIS>f=~1s!j7bHjDK<<2Q?<@e>?{T6D7ZkuMG7uaK$59)nSwD2=rTxohk_6VBs?%% zwyv;EQ}QMSSqgqY!4D}Q@~_;dfS3-Wq6~McJ#k*9QVM9w>VE{4`vdNCUJ#tnzFK+~ z-&;PK3k=Mioxfb*2a3VHIey#X_Y3^q|IxW~<Gdb@-fxqyqyLV-5wforINY2wy;Exp> z8s~=Rdp@`beeC3rW$uk!{m}yddhyKp9N(I2KV9I@7u!1L-dyZ0@SVk-eaKzvEAV~A zwvM@ri!T@Wj$&Vbj_+8K3VeUDFK9%A#ZKgITU05xQ=GfB_;!I8F-9kPXj+UG_}*e` z7rN~!@Lk1i9dqaAQMIGk-ZMA4h(UXbRFSpsE4FmQ(<)HlyNg{tl$oOPc2Tb>R*e7L z$qU;yyjJ0yWy5I`dN$lv;lPI1DeT#3vI&Pb+;-t@%SMA!*sn@79xox{xt#vQwuHUFb-o`fsVdqA@SJ<)9!3!rgymsM~<#~fg zILufWJJ2W&wDge}g3q#X9%0#Siuqf*;AzQ+c=Dc?YP?G~7+$ z{$iS+^(!8@H^j#2zDJJF4vJ&sx{CiKu9-g0757zK4 z#YlI7G>gkwLSOxxHHf1HS6aA?Htt(tVVRy*si9#k(Op5eZ{6gQTj&NVtHim%^O{=V z2B3aw@0*gh>8*p2# z>ogd&HeGid zJl0@Nc<}N|Dk=TNN-PCeK6)xc)QUbv-r$E|=%c!WvHr|L#GNQeQz{mT1EBP01N4Z6 z-U#3c$_;v_ax-RlZR0bteswHurW&gDUli4qbMBY z2jOL1ny$!REkslpUa(pj1EOoZK1#!Oh>gIm>02g>XSBLacZEtC=S6o<411XT#;}LU zP2@JEdjf1RFv=*ZLM%6XeZwv(Go3Y1s$2}v;8ddYuwggAz)ZM73g?<{Cmdgvj}&}+ z=Xn@O=1)LjShw0-ZDc4pw{ULF+nM)vE=eC<|M2>Q{ZHP{c{^9Vmzk0Jx6cKxq4^8W z;%a+_MmGF@snRXRzuhww;C>!(4Rzao-ep0!$uo=Oc}7eRU{3$<@eH~k2B`l2?@4Ec zrb`r|>VxkIF)n9(P% zCUBFy@bm8ADHrz(m+RCv+b^0d2sa5>boLCoe~HM~f$m}Mt~F|%gfHS%XkX+qR?Dzu z7`~37l<=tPL$0$b3rXbGfIZ}v;H=yRxuhRgeQ?f5{S<2t)OI{zWGx%Q7*K5Qv= zbh1=e_eb#$<3;^v!(*?j|H4M?Ik(IGPhW6|GC!;jhZ=_V@$|&xFKLZTb#12@vxQhI zEFfQD1t439@~;t~8@5Z#1VP1vDW*$g_x-~88l2x*Stdi1#4_-#bwLBbh%is$^DJW#lTwtE3^mY!i zi>?HgpEuyi0TVOe!(M*wtU~jmiVAzK$ovKe=^i|8Z9F-y3AibQi@5{OpyNqp zHsipPSdtN-sT{zJKx`SPgeY!fu%lVhnq7ZXuo?R6^~aK?VNv;8v?121eiM;7?jPL^ z|8mdV>0N+75H8w8M0ESAr$Ef)TN9po%M|G;_v zp4;>HTrkfCKdWn64Cm|m=G>q8+ULB_?SiFq{>pO>Kc6>nR?lM7s-^iiR=Z{AI)?zL zU~O5a-; Fe*hPi5RL!< literal 0 HcmV?d00001 diff --git a/lab/__pycache__/http_api.cpython-312.pyc b/lab/__pycache__/http_api.cpython-312.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e9bfbb7d47de37250d5abe3e7ec21fede64eadd7 GIT binary patch literal 11471 zcmcgSYfxO*dH1>RCods{#MOckD+w@qSg&Pe86?@p#Y@(>&Vx^D_dV${f9!*j6lt>LxA}ulz zI?4<&G?v*2JH(P-Zis_lF2Y9*Lx!kv$VmG6h$(6wGLy0)Vu@OZtWn#L4eCbG6tPDg zLyo9($O-kWqWMj3$R#$27Wj2Dl*Em-6c8!>iq@Ne3#OrJ7;A&EH4Jr{676qNqC+y= zWb?hX;x@5SbiTqS?F4e2)F^Jb3G*=xHNjc}KzTy1 zm*?+NV!gEeUFtep*dI+vO=Bm^$LQso`6im0WG7kg_Hi44e{yi}c|jSMM}i?q7>Ub* zIw}c~;P6o)C_FvzT)%MQ`KN?nOoVDQ91BN-5n))4PbiWsy!cdy#WJ9Zk?^n(k3}Xs zgh7~_h=*gU5FY_RVFe&f!bm8nap6+q94Z15W$2NGw!udac`ZsjD#6s) zB|(;8F_;$%UI||cs?uJ9M>(LV6dDc2!XX$J9an>DI38031TZlw$*@-3G8|SV;WyK7 zBl`IffI?6eqVX$|0Zx02)gpU}2mDd&X^w@n|p{!)dX& z3aEyoiHH=1eMzE#b325Su%}7rd1YKuRAFR17Ly{l2w-*bnc(n=M4}_#d2ufwO(^rC zPHUujC>TS?K${VmEytr|fE0r>3Xz3`L{J?io3Y@TiIOVGcti@4HcwJKIP8oM!+~8P z1QL*BfQk@|0D;9xLAol*p|AoQ`wSVxOKVoLv`)@`FQe5x5mY2(lr#Efo&>NFN!F?d zM-c>^6?P853X%-d*%Przm}N=?Wkm|SqG+aZIYKI*;SGc?L$4GIOQL4b=@81ZIpI$% zvlK;d9Du?+m7>N-^Mjd4lWWO~Oo|59X9g&dEtI@mzs4)0aam2WopNW=+9^v)d>p8z zbOLscU?Lnq(ghTCG9m#sY%r0~tc5{=u&5vof^fgBa};=~b0RJ)q0vqij*YbRWg z&QVoO1OQk^Vp4MiiqjFaTm=BOz`xQC*)(;x$+Ozjm22u+JbSyzmu^m9PQRQrZChtq z-f-91kU5rV&)VD8d8lr{1S$jsC?+0Cgdj+Wc}mSoF$zeNqEk#kZcVX3s&Yc+0pv+l zr{<*#MU5GYu!{vCBwLU+0Er!-ylnDuWc0#q&&!AcUzc4yVA|r{0=iEq!gxZz1Da8k zPj0^eB2!vUlwl3KF|22TzQ2A=8-oERKDqp`~Pu$PmafSG0-2d;%d zvI}e6s2YtZxC;T2?8del9b!54FG=!t=-L7Q%0bAcsoz@bL&%q^h%H<_-a=;Ar)&UY z>vb!EK`e^Wd@RX3#D)jf-j?FWkn)>SLAw?4YP_z~$u4ex=-zfb0J$l}mu9EzP!?cs z4$(4(?6@hVnGsst`KC!lMkEPnfcJh=8mryZ0M!Dn1&!VNrIsnD=$T;vgXWYI@)o30 z$^oLzIOT$M>b?w9e1y>2AhxDlFk{zscFHBT!PxpQkA;3Oic4`f8N27Y5$^lme$5C1 zd^oO1N$Y9U)xi+ROhw}dMuW1Hbe;{XqoN#~D6|{SNLS*QCCw0wg+SbDbWk%4kE?1t zmUIZ9Q>AW?UiHX`Z_+G)x)|y9G<%ZgeIP+W2Z1u{_P~uFlR~P;|GXqC@t6;#s}%A3 z!m-4-Dum(g6-Iz55Cy3DDo6&8P`saoG5+NBa^MM6ii!f30fC<{7uQgv2rNw&DI1wz zr@W=OPFbVh7f+zg;XiZoM6a(iUnw-6J@eGy$x>^lzLZ}x$r6xRk;J6W2m6oeJ46zf z?*{Ab*zFO}9*P4M5^+WK0P%6jZcozGc}Wh&fNb|^UvhT|N;^;@8itWYlobLj-|SB^ zzRsk{@9RY9evOaD#b6}aN&v(YQmmZ}FPmQ0=pRRI^qc9w1`r4RJ~14O#4nXCO<9x?CaF0`I85Ii4u}vh>e8hXge4^lE*)Y#$(sKwd77r)+Rjmv+~2Yjw3knMe35Vo9i3(LdLH3!&wa)NTya!L@^CN0{{9ar zFRw8jk7}$O7C|FJ#v$7iJ**%Hpaz~MD)_$H2iClFH6iw* zsr44WVmkGE7H0f`sQ+*5Y@w5_zR|AIwhGo)yBby_aA;@p7N3yZ;p-gj^83akrNM*- z?GZ4PVcW3f@rYkD#K0bbE7=1)eMJaGf{FriW>ktqfCCf|5pd@U&50nG&(K^pk~Fo4 z!LF78W*xHl6}VyfiBQ6Nvd_0ujEB_8ge0IXfxYw?k!P+7cRngo~)J1XtMB5W?BHBYj= zP*D2dMgkkJIo#KxGttas&e1k~`cCuix%Nflbl;t}&RliN;<@Q(?z(GM-K{xy>y5VC z?tL@OOQz|bwIb=wUY%+ZGJ3)SD>J#*}aXS#RYK%0B$b(Z3*)-8(cGBa;7;`n;Bg#BF2iU59g)O^?qG&YsE~$h2mKQ@38d6FK|FZA-?6DWJFAQV{&t?bCWzW6z zdzLnzr|-75;g)Rcw#{2wrlsUf>S{IZbCEri3?mCp)b!d6lk!9iNJsx_X+hLeK1vIdBEpA)1!`}QY zYc0DM4&OMq*sy3@I=}SnI$+mF&d0oNBDC6)Gj7Qo23O*)?S(aGb$TRQvl~X)_aYta zfWO(Z=6U2td*0u(usz%1Uz%77-;zIW`MB=m*AQ3$fPvFT+>H=0Rx#xrKzid0G|f{I zHAF)Yjd>9yzQl@*4_Za5$U+c{6Kx^~zr1J{9U>25D}(414bWl~U7`_wO`=;g!LJ#9 zF;r%mw0NsD3y7M$BQIM49J!JYXeC4joC10SeTjK$5<&z0@*XII&&+`}Goo=p`I3_N zmUV~B3id!C3RWY;c#?IcB|6I5uL207Mc@LNZ2I2IwIMD?+$tRSFsEuPPazmmOc10i zIxbUmiXKB%wJC|kFj*xxxL}xrISf$*`-yR|yyS3_ghF&9*7$@3mYFP|Ka27iQ1bP8XE0E8-s6JADs7S`7c`H>(ssDkTi2&f=}Vo7%yk@Ti17^Uok4Cv#o zoo}3LTy^ivxpyvfF78`)KbAJEIcry)o}AOOaAeupzUthcbM9Yu9?Y5!>clMf{m`@^ z#5@Ni24X_Yl^77sr8q*Rr~$+k@<&pMhy^mmoGIq3`oYhw7;Q+=1?V&ec+7y~rc@q} zc<2~bMaCsbEqa70t^m9Ei)4pH;8gJB3cDfw7;4y*#xNMLDV0`%Xeu(xeHTQLA`ps1 zOQCZN{FtPz2V#q$W7`KoX}mZdO(>c{iiP5$1d0t}BpA;~ zKrB-NorqPPeZUe^WsQwUG0j?P1zkHa5(dS{PspHjy?mbK6o`k8E1EG8oQ%YSqGki- zhu8$r{*a%7{S+;^d@zL%;9^NvIYVO08#wk`kWEwT2Flf#ZR)!1?7GKNcK5ZXXP%y& zShh8-+FEnA))iaZU3dNbj=3Gn?wugPK>}JJTQxQ0ObwaGR!kna@T;c!oT)yu<3{6( z>9JxxyVd*Q{tuPK12-pc`d3UxiybSb1}I;%%vkahYd$gU(D^)nuHeq%!vw_$5V;Fkvvme@K6uF2qZa8=5J#7z=5k8` zqfpBS)AGS)`7Ac!!m@=jC`R*Q-VBB=RO7P-q8fV4b}&iskz|0C>kw)hK(}HL+IfZ5Gbk~#9ID zene#*-vM|E-hWUpjlz3Rzw`8h^6qp0;p&=wOsYNMh~kq3xmsc>&A z#Uufjuh2I|yfJcLjJ}pOC-N$K9VEes3Tvj=Qv6>mzF?vZg7L_Bo04cuF&~;X$*Q8p zBeLpY8k?&W+XPuvjBb`RVmXzGtm0ndh^&I&GFfHr|LkqJFY;wT^1d10x<=v<=SlLN zSHjYSJO<54vnpLxi6^Vs@u{iE;2+S$arD|{WHI>)W|NqWLZ+EOYbg^DuFQ+Rz{F@g z7!6kvedMt~j>D{i=49cuO7}{Me;R|X&6?66#sjBM4nAPJ=}qtA=uaS>UAUZW+_z%ezhD@m||IZL1Bta}B$19Qg6E502gJUT!!tYXpZRuaA|L z#zz-x{-OTw>X#1P>RWDnK6~cn72DSkC@50LjAQoo#lE!T6Vu~g+_O?O4WCmC0Cv^P zTjng8$KLCHr~AgCpY|*ne$toS{rHOO#HX&hKcaKD<-UTkMUzfnhiK6&UMfl9tj# zeUj;vklxT&3 zq#kE6g1UMBxq-oCRhgBSZ^J9EAQ$~9I_Oh~;|(Pj6s zwBgRa!)fu_*v#0q_)I)Avg~+d)v-6{*t_EBSQ<{V>x{u%z1HgeQSANL;=oGlk+%=b zADugz3B7&%)0VwA27d7DYRjQq%c13#BmcuPkJFAjjY8%`=9O7`wrlofq-TS1mN{=Jc4CgAn?j~Fp}?a1_EWD z83h9JbGSNjPSJ+buL&{fzK+%JVzvh}jK=6O`93URHV*#^cK#W)W_Es~Z-b+G`-X+# zAG>F=@J$=_9Pi(-aeU2&%fX+Z?+M%Z13Da>{eZ`ajiJP3?*PgSAt>NySP(sg$E1r1crm#sv^^>$!lBFf zq1lKm!IxTaj2%KxFajS~3GoU1o^5no6`|CjIiElSxpd))b(=JI?K{-*DExJKS&bvn z9YPqy$jj;$Fo%mlz*7HkOF^_jp6H^4*Kv9u+Wrmx6)$A)(Tt-iTix=*H^ZzAWb>RK)Yfui*j#kgew7@i4q5MS{Z`AB2FIB>y=8 z)aWxw@|hYui2=(3rxJmOFESXE3jY9aEgu9+m2Tit zG{f|WHG(MP;)_rPelPW-Ii&Yj8cFLOcsaoLa00n1vfcS+bn~gMRwbx%dW0z`-b5ZO&ji=VP2r= z16lX3dlVM#jWR6Vy>6p8TRM50-TDj8@k_4d*Hzmx1Mi)C=iEYYVK{RkTebU*zUiKH z-5O{7X5Slq@SZg>=gjWhcZ>b_!gSvXH+092pL%i2_U+TB?r`Sxz_s%;=hM$-w;o$M fwRAXZ>0RMYu5p%Cu0F@rf5J7ayD6@L@bLcv1+U(_ literal 0 HcmV?d00001 diff --git a/lab/__pycache__/mutations.cpython-312.pyc b/lab/__pycache__/mutations.cpython-312.pyc new file mode 100644 index 0000000000000000000000000000000000000000..000d0db55881285ea55d98f3ca7a0da830b74b60 GIT binary patch literal 12948 zcmb_iYj6}texKQgwCqYO5AigZ0r8TMRuTdP26GmYcpM-w62>PVw~V%1(xBa)bCEQzaRY%_4RQL*Xs?xeevv8P5XCx2(IVUjXr-ltZ6s2vl`Qe zG^VpqR?mfoLOR`tv*BE1C?fA8*=Vk2s3uoCREy^*tI5W4i-s2E;zRLVVknWT8>-W_ zQH|C9i9S@%Ygud!78qKLyG8OY8Pa%!#aaBr#0Md@c1Sy-vAQ=kRv+|y;6pdGguTud zzZn@?%93wtL(A9_w9DC2v<++-+7)a$+Lf#U?JBkc?P|6X?HaZU?OL`P?Q?7m+I4I# z+V$)?w9m73Xg4sy)?;o1|Ig!pBmOtwe-ktCwvlZ_yP0i5yM;BP-O4tj-Nv?{-Hu%> zVO!ZYww*PxX1?r$nqV*J-@#gBe=A?kQXjT`P+Re?ops>dPS(jAs=nF9cFS+}@D)}4 zU2L!H-^W*0^}oRO%Xtp)RaN~jvV(H`H~8wR{+C#{>_5cURQ30;UO7)6Ut86Gm>rS* zNBMJA{m0lh*>QG)o#gAP`uo`_*?*d^uj+r9ossL3UR5zrr_E^`B#}vTtGj zA>LTk{~G(Y>_5*pSM`5~rDgvGzNM<)WW%yQ!?#xTGsb2A2;WxKKgz~rzs0u){UZ%S zP3$6XX5$AyK;G1=y`yD97JlcGns!B-(k{N@Git@(l=j2W+e;%DbwwL$VOe~iqwkr+ z^5Cvk<}w@qsmVeH(nX^tw5DRQ7}K=DF>YkdVV=!$X5@;l=~{N)$e6B~wMUEG__yib zH(ZP<2>T+>xZ4~f$4#dwto*29$>(j? zaEgV3EnLIQ<0a6+X@7<}Y!_W1foCj-5M+xu?lHrD0#R_o#Lu0;&`E=y2_a@?tPBrk z7s3J}OryDZRM^ElYX$DyF@^l*<`#otZZL{5Jc&VL#hjTpMuZJy0{caW10N2S$usOX zV+%QwkJz*{AQi>@3-P#t>ttWg(eD0ZJ>4e^`6*~MVrICJ88h>vob4w{x}YRhayS-o ztqh>TG+3M-%xp@|LvsaJB|V)nyO>o#5P20qKE@1_nFYl{YLbDzlimG;$9iZYEAR5W z0(;E7WZ6YwjPiWZ%5#vxhb~pkIz?~^cK{N|@FY-n8BYX6jt1w6Ie?%EVYxtz&yK(- z7>j2|6i~f=hx>X4)!qoX&ZGk>8vu#cag1W#hem-6&Z^wMTt{%V<0pVREo@fIxW-{C z&qzcbgG}PbaEZ~uA%j?)S&nPvGg9h?IXH;tjWX8gYad#O3gcO8m4~-7#D_%R+goqowjc}+1|0K#n^PRWA~<}3x-)JWJ#Fh78M7Q zNcbIAo)|w4PLQGsErFG!dNHBA2qVPh0cxBvt5A@n%G&u+2T}m!Fg^ki0Tv+T*d|D7 z0m&(2aLjZGz+q?)Nis*h%l&1wXc^S(~9yxogtUjQTh}stz6m+Z0fC?Ae8!qA_sqsu>0;;@q zK{J!SC2T>@LT|+& zALo-;0vgdB1JNqLfg%-`uxkf5U?`wIFfPTsYoiOFk#SM{CgGB{B7U^By}iTO9w;Zn zl)~W{P{3IeV3*Zj0E5$H00);Wsn457%&fzkrF4N-pbsf^(-_H`6C`-U+<@|jA~ehi zb21PIBGnk#2)#AM5pE0mcOd zEZ(BB((zXD%LQpD(-J0Y#22EbuoqvjvR*7u*Itd%)p#E&T3*;JSY8dYbMSTsv7s7R z^AgW{Q73D=PKHnrTw!&i&%cVB8yeS!bfyi35E$t^%tF$Yqr0%Yi{LIQ?`m*YTYejp zcZ<*$mv@Pw2&=Zc2*9xbebJh+fEIUevW*pf?Hx0#{KWoBmFilU&df{Vf7d zfc?!ZFRc#f@!D$&n#a?pzmL#27kp5)N}rl=Q7r}Jixr_)|Moz6j{ zX6e2zohEnBf0In7D;Of37MlU@``SEO?aQXNF+0cGCT!tk#@Z?@avK0|EAwcoFe$ew zoBjCEX+iUS?Xy_ojU#i3#czCbF0tf|6RJrrd!v6Y(fA?O|g!6Iy}3@3BYpZF?cS{ zCOe)Po+x4+;Sw-V<>>+(Sj9t62WF2T5)C{C%;^6InQGu@103QTpavcvV%jw9FNLR1 z2dGO70vP&LQ0vy}+2qb=#m=A)2}FLx61g;J2gl&)kn9!b0L5ASL@za@WuHO&jQ-(! zbiMx#fRUuQEuWc8JBW<)>}!~JpBTcjltHOL$4z%O+48LLSU-T411$9jjV#W?8R!E32s|EaX3kq4OWiylEh|A%xKb1Zm9pVCjA)qP%x*eHK z?tWH)MlmN0z=toruX_!OP11;E&0kHgj-7(x)9>I{0o1L-v&qh{0m{Je$4$kX+7tGx ziuoulyqruQxk#|?+GX95iRc0C{SZvuxiiNG`$Q97ViS6y!d!eAJCH@Z%glqNmFZkZ zXX-%KMtJEQ#Et4t<12@p%W3Vl`RCc0u4%>ngpN^x3{1MfMb2_4T1_Jqcsh+bMIO`0 z`Re3+WI6A*9GXq`Jc*C5H=Sh9%WjCY=j5ZV=6L{+;joYh>=9ldPM02_x7zO^hJp_- z9W^9M_|t7x$1WiNQnHWDfxd*AKwmbqfdriLWsPshOX)(Ar^Tvh86rIdhm|^d$qui z3uYbAnxdtb&d9`@iiwL1VU=Sd^CtBk@@hWnWV#SRBSIboF!qGXLLyPgQir`LvX)#7 zRI=Ciu`hhc3|{GHlswb#4u$JTxnJNNGEKbd-O>ej^l4SVo3)A3g? z{>6)TkKf;a?m_I_=V5&GO9{#w`w!{>bx{C0d}*Xz;9Te7P@6=@rik=Y#T3?5dx0mS zwy8FA;esJI<1PP+9f&vZ*$BZ*F6}R;wa=2v-nnw^$|p(Ve$u%4+=FE6^wGK608^=6W3Yc!9A%X>;S*)+XjeBN_<}67G;YGeU4?NI<9L>K zZBK=u`gV=P#i8AQPuVk(#r@Nb*B)^l@8w4e8@%80;%xHZ*KF|DN4WSMOdp_`822mD z+$G4qT*0pj&3k5(U0(ymuN4p<2EFT~Lh-f4({Zs^{0I<`U{?4wH~VIj&Cd#ia<(+> zB{7WIpANx3aRcvvvl@b1JzpP$!(cqd{Y87dSiZ$Plf;$&&pHX;g zGLSs5c*&=%!HdrQU{ z=`I?w1P=Q7(X;fiQ8VC$;vJBpJC9<5D@-HHM=jUNA-qK?)5?r1jzxLQr9c<2P13o}d0~)n$rZ^v5fD#AQ zyD<;y;(12(3F1WXo`Z6T-0Qwl!x=xnrAi-2drxptYB`~Tb<~WqsJ{#X2vnVb04gUb zZU~l_6LqwgHufT>Mo5J<+@i=EyC5i3;ero9g_>R9?*oUZ+DUALK_ied6uG(~Nb(2* zQm0@pRixsoOasKD&?f~{t!#WdwpFg7$fy}Llwnnd0rFgVn1Mr!WL&WfloBgR za^5&(4_6kmWfq$-!ay2?0yq*N8NUGBNS$M&g07BAY@<&}A;{6@NKihJPWU!eznV}b zLPZHe8yH6&5P)|fgAm6ioOkRjZ6W?rwLbcm>X-t@Clp?-Tn;BGfd0}Xa8s@(Qz2It zt?2}UESbt$5d}8>Ov5IfkpS!`Ye#n%P60JKs-Vh45oK+gLD^U;Xs8nj zW@l;D5sG^yd?qB6w2+Lji+==ko{~G1ICcz)2Vz;MJ39vwn8rf1jPf0vRF~>!k{LmI zQtw#M$w1c$%MLVV0YxPU;+ekgUVu;aP}DvqWdX8WPed(DmMSajW5}X#8t7+}CHT8a zZ9T$voL1#tO0FUP!X}CEGAgFiCqU~M&aEIDT_juEW5fJV)taLShnoLrRm6z#PacpU z9`-Bv;B!@>Hzq7_dOk28Bmm`-zZ7h$I7@Uv%OU!#u3ZtUQRR_JgpxpVN4rZM6?{`H zpbiH(X*Pu%sXOSiiWpZcC(bW+%Rm^q3S3UbVjQ&rzbUe*`g~Q8tmcGVs0bPO&T!6r zsqm}r68Nm-G*tBgf#eT}hz-h{m|w!Yr?j?P(NnUB)TC5pm3&m{I!zT}s87hdIzU3I z(jCLeo6#}3GiiQEarqeyX&U6rU8R<89H}vJItLs}ML%q)Y`B%2(vchAik3RMF%n0L zN=l0>?Eu9Cwh^4X6X>EI6J-%8<6uI$%HGnN3Y>#?2*1*Mgju@Sr}8i|nSggb9+X`1 zOH@?`lPj#NPg6!Or(-?xc!1OqFjXd?R&x?_+Oj|zC<6!t4nr{s%f3=$1uWp(0?ud= z3n+a==OK`;D{@f0pPpw{}-cO*wscM@UyenT3+||DKE3#5S>I~VXbrH?Q8tR(Tf;UronpT(BlqQxk z{bC|`Uie=apT`N|D-oASC$ArCEIbwYvG&Jd%(O5+vI`?2P#{t|euxebfj{A!Dz!;0 zMn3SuQAhB&JWK===^q#(gHMMzU~ph=lzZ=_zSMVku^-ZuMymR5y_)zmR)2LIsj^RE#{HOabNNqW zTONk-=4b0R-fVfWu5G&K=+UdieuGo0J|K`SyJX)^~iq@>YCp8AKJ!$ z(Ki35cI<(6>`^3~2u;^MjBAUQ+*`8qeynpkIu}dcTe9VTZ0mIN^BOIbxW4&a_Oagh zKcQNE^Fs~Id{_%@en^k=4IzEwqa~W&;c2~|mh!Y6o`!qe?38Vbr!_sQ)AT)_*6C?I zo;EmN7uRd%*XsKA`IxSs(;vk&eYd9_^t3KdYxcBuPdoj2OpDfEJ$tWV?@Y_x$Xxx& zyY4)A_^ke~kvi(`zH{K-%V!^kLW@^?wsP&Q;up2y=$`UNowvVzXZKwFp*y|w(KmyS zw%=-*!CXCe2C4ga(A{=>-HiKWchju{Gb`rm+io||=q`WZmRs4G-nsgHGrOp_JD6k7 zOx+zu;2hCEJ9=Us(DdtNaGGzOnAtK{zkf!q+Y@}#HM8zcajyROUD-XL1J!EJeRoRt z23~s@(iflB=UzTL9}Pv%Exg@7Gjw+~z3kQJ`i{}d;|niaZ9?|CjKPCnI0X*%ux6d4%t3P}oN0L^r;Jkh?Hy?0)vS3BoAyXe)v@~f6xV>7V|96ikG zdmBU3i|!>h%xin~&_kR`Y2P=ye&4N)@0veh@3EWR?~Q)6e&2NTUhUck+P?XkkiPkg ZcuZgQ%STJK=U;G0%>Qm#!{M&T{{nia#JB(e literal 0 HcmV?d00001 diff --git a/lab/app.py b/lab/app.py new file mode 100644 index 0000000..04f5161 --- /dev/null +++ b/lab/app.py @@ -0,0 +1,309 @@ +"""The test-driver lab — the system under test. + +Grown from the T04 seed (`lab/minimal.py`, now removed). Users belong to +tenants, own resources, share them with READ or WRITE permission, and revoke +that sharing. Every state change is audited. + +Two access paths exist, and the distinction is the whole point: + +* the **enforcement path** (`request`) is what an actor uses. It checks + authorization and can therefore be *wrong* — that is where a seeded + authorization defect lives. +* the **observation channel** (`ObservationChannel`) reads stored state directly + and probes enforcement out-of-band. It is the independent channel required by + decision D-07. + +An oracle consulting only stored state would verify test-driver's own +reimplementation of the rules rather than the system's enforcement of them. An +oracle consulting only enforcement could not notice that record and enforcement +disagree. test-driver observes both and treats disagreement as meaningful in +itself — that disagreement is the precise signature of an authorization defect +which leaves the audit trail looking correct. + +Mutations are applied by `lab.mutations.build_lab`, never by editing this file. +""" + +from __future__ import annotations + +import time +from dataclasses import dataclass, field +from typing import Any, Callable, Literal + +Permission = Literal["READ", "WRITE"] + +BASE_VERSION = "lab-0.2.0" + + +class Denied(Exception): + """The enforcement path refused the request.""" + + def __init__(self, reason: str, status: int = 403) -> None: + super().__init__(reason) + self.status = status + + +@dataclass(slots=True) +class AuditRecord: + sequence: int + event: str + actor_id: str + resource_id: str | None = None + subject_id: str | None = None + permission: str | None = None + + +@dataclass(slots=True) +class User: + id: str + tenant_id: str + token: str + + +@dataclass(slots=True) +class LabApp: + """In-process resource-sharing service. + + Mutation hooks are plain attributes holding callables or flags. A mutation + replaces one, so that every mutated build differs from baseline in exactly + one named way and the difference is inspectable at runtime. + """ + + version: str = f"{BASE_VERSION}-baseline" + applied_mutations: tuple[str, ...] = () + + users: dict[str, User] = field(default_factory=dict) + _tokens: dict[str, str] = field(default_factory=dict) + resources: dict[str, dict[str, Any]] = field(default_factory=dict) + grants: dict[tuple[str, str], Permission] = field(default_factory=dict) + audit: list[AuditRecord] = field(default_factory=list) + _seq: int = 0 + + # --- mutation hooks (baseline behaviour) ---------------------------- + may_read: Callable[["LabApp", str, str], bool] | None = None + grant_permission_for: Callable[[str | None], Permission] | None = None + audit_revoke: bool = True + revoke_delay_seconds: float = 0.0 + denied_status: int = 403 + require_share_acceptance: bool = False + enforce_tenant_isolation: bool = True + response_id_field: str = "resource_id" + latency_seconds: float = 0.0 + + # --- presentation hooks (surface only; never affect domain semantics) --- + ui_share_control: str = "inline" + ui_dom_style: str = "flat" + ui_labels: str = "plain" + ui_field_order: str = "natural" + ui_button_element: str = "button" + ui_test_ids: str = "stable" + ui_confirm_revoke: bool = False + api_path_style: str = "long" + tenant_sharing_announced: bool = False + + # -- setup ----------------------------------------------------------- + + def add_user(self, user_id: str, tenant_id: str = "t-acme") -> str: + token = f"tok-{user_id}" + self.users[user_id] = User(user_id, tenant_id, token) + self._tokens[token] = user_id + return token + + def _audit(self, event: str, actor_id: str, **kw: Any) -> None: + self._seq += 1 + self.audit.append(AuditRecord(self._seq, event, actor_id, **kw)) + + def _whoami(self, token: str) -> str: + if token not in self._tokens: + raise Denied("unknown token", status=401) + return self._tokens[token] + + def tenant_of(self, user_id: str) -> str | None: + user = self.users.get(user_id) + return user.tenant_id if user else None + + # -- enforcement path (what actors use) ------------------------------ + + def request(self, token: str, op: str, **args: Any) -> Any: + """The single entry point actors go through. Authorization is enforced here.""" + if self.latency_seconds: + time.sleep(self.latency_seconds) + user_id = self._whoami(token) + handler = getattr(self, f"_op_{op}", None) + if handler is None: + raise Denied(f"unknown operation {op!r}", status=404) + return handler(user_id, **args) + + def _op_create_resource(self, user_id: str, resource_id: str, content: str) -> dict: + self.resources[resource_id] = {"owner": user_id, "content": content} + self._audit("create", user_id, resource_id=resource_id) + return {self.response_id_field: resource_id} + + def _op_read_resource(self, user_id: str, resource_id: str) -> dict: + resource = self.resources.get(resource_id) + if resource is None: + raise Denied("no such resource", status=404) + if not self._may_read(user_id, resource_id): + raise Denied("not authorized to read", status=self.denied_status) + return {self.response_id_field: resource_id, "content": resource["content"]} + + def _op_write_resource(self, user_id: str, resource_id: str, content: str) -> dict: + resource = self.resources.get(resource_id) + if resource is None: + raise Denied("no such resource", status=404) + if not self._may_write(user_id, resource_id): + raise Denied("not authorized to write", status=self.denied_status) + resource["content"] = content + self._audit("write", user_id, resource_id=resource_id) + return {"written": True} + + def _op_grant( + self, + user_id: str, + resource_id: str, + subject_id: str, + permission: Permission | None = None, + ) -> dict: + resource = self.resources.get(resource_id) + if resource is None or resource["owner"] != user_id: + raise Denied("only the owner may grant") + if self.enforce_tenant_isolation: + if self.tenant_of(subject_id) != self.tenant_of(user_id): + raise Denied("cross-tenant sharing is not permitted") + effective = ( + self.grant_permission_for(permission) + if self.grant_permission_for + else (permission or "READ") + ) + self.grants[(resource_id, subject_id)] = effective + if self.require_share_acceptance: + self.resources[resource_id].setdefault("pending", set()).add(subject_id) + self._audit( + "grant", user_id, resource_id=resource_id, + subject_id=subject_id, permission=effective, + ) + return {"granted": effective} + + def _op_accept_share(self, user_id: str, resource_id: str) -> dict: + pending = self.resources.get(resource_id, {}).get("pending") + if pending: + pending.discard(user_id) + self._audit("accept", user_id, resource_id=resource_id, subject_id=user_id) + return {"accepted": True} + + def _op_revoke(self, user_id: str, resource_id: str, subject_id: str) -> dict: + resource = self.resources.get(resource_id) + if resource is None or resource["owner"] != user_id: + raise Denied("only the owner may revoke") + if self.revoke_delay_seconds: + self.resources[resource_id].setdefault("revoke_after", {})[subject_id] = ( + time.monotonic() + self.revoke_delay_seconds + ) + else: + self.grants.pop((resource_id, subject_id), None) + if self.audit_revoke: + self._audit("revoke", user_id, resource_id=resource_id, subject_id=subject_id) + return {"revoked": True} + + def _may_read(self, user_id: str, resource_id: str) -> bool: + """Authorization as the system actually enforces it.""" + if self.may_read is not None: + return self.may_read(self, user_id, resource_id) + return self.baseline_may_read(user_id, resource_id) + + def _may_write(self, user_id: str, resource_id: str) -> bool: + resource = self.resources.get(resource_id) + if resource is None: + return False + if resource["owner"] == user_id: + return True + return self.grants.get((resource_id, user_id)) == "WRITE" + + def baseline_may_read(self, user_id: str, resource_id: str) -> bool: + resource = self.resources.get(resource_id) + if resource is None: + return False + if resource["owner"] == user_id: + return True + deadline = resource.get("revoke_after", {}).get(user_id) + if deadline is not None and time.monotonic() < deadline: + return True # revocation is scheduled but not yet effective + if user_id in resource.get("pending", set()): + return False + return (resource_id, user_id) in self.grants + + +class ObservationChannel: + """Independent read access to lab state — decision D-07. + + Bypasses authorization deliberately. This is the channel test-driver requires + of any system under test, and the main integration burden the framework + imposes on an adopter. + """ + + def __init__(self, app: LabApp) -> None: + self._app = app + + @property + def version(self) -> str: + return self._app.version + + def state_permission(self, user_id: str, resource_id: str) -> str | None: + """What the stored record says, independent of any enforcement decision.""" + resource = self._app.resources.get(resource_id) + if resource is None: + return None + if resource["owner"] == user_id: + return "OWNER" + return self._app.grants.get((resource_id, user_id)) + + def probe_read(self, user_id: str, resource_id: str) -> bool: + """Exercise the enforcement path out-of-band and report what it did. + + This uses the subject's own credentials, which can look like a violation + of actor isolation but is not: independence means the *actor's report* is + never the evidence. The observer issues its own request and records the + raw outcome. No actor is ever asked whether it succeeded. + """ + user = self._app.users.get(user_id) + if user is None: + return False + try: + self._app.request(user.token, "read_resource", resource_id=resource_id) + except Denied: + return False + return True + + def probe_write(self, user_id: str, resource_id: str) -> bool: + """Out-of-band probe of the write path, non-destructive on refusal.""" + user = self._app.users.get(user_id) + if user is None: + return False + original = self._app.resources.get(resource_id, {}).get("content") + try: + self._app.request( + user.token, "write_resource", resource_id=resource_id, content=original + ) + except Denied: + return False + return True + + def audit_events(self, resource_id: str) -> list[dict[str, Any]]: + return [ + { + "sequence": r.sequence, + "event": r.event, + "actor_id": r.actor_id, + "subject_id": r.subject_id, + "permission": r.permission, + } + for r in self._app.audit + if r.resource_id == resource_id + ] + + +def build_baseline() -> tuple[LabApp, dict[str, str]]: + """Known initial state, so that runs replay from the same starting point.""" + app = LabApp() + tokens = {u: app.add_user(u) for u in ("alice", "bob", "carol")} + tokens["mallory"] = app.add_user("mallory", tenant_id="t-other") + return app, tokens diff --git a/lab/http_api.py b/lab/http_api.py new file mode 100644 index 0000000..f9f1bbf --- /dev/null +++ b/lab/http_api.py @@ -0,0 +1,215 @@ +"""HTTP surface for the lab: a JSON API and a minimal browser UI. + +Stdlib only. The point of this layer is to give the agentic browser driver (T07) +something real to navigate, and to give the mechanical mutations somewhere to +bite — a mutation that moves a control is inert until there is a control to move. + +The domain is not reimplemented here. Every request funnels into +`LabApp.request`, so the UI and API cannot drift from the enforcement path that +the deterministic driver and the observation channel already exercise. +""" + +from __future__ import annotations + +import json +import re +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from typing import Any +from urllib.parse import parse_qs, urlparse + +from .app import Denied, LabApp + + +def _resource_path(app: LabApp, resource_id: str) -> str: + return ( + f"/r/{resource_id}" if app.api_path_style == "short" + else f"/resources/{resource_id}" + ) + + +def render_resource_page(app: LabApp, user_id: str, resource_id: str) -> str: + """The browser surface. Every mechanical UI mutation shows up here.""" + html = _render(app, user_id, resource_id) + if app.ui_test_ids == "dropped": + # A rewrite that does not carry the old test ids forward. See the + # "test-id axis" note in lab/mutations.py — this is the case that + # decides whether semantic actions earn their keep. + html = _TEST_ID_ATTR.sub("", html) + return html + + +_TEST_ID_ATTR = re.compile(r'\s*data-td="[^"]*"') + + +def _render(app: LabApp, user_id: str, resource_id: str) -> str: + verbose = app.ui_labels == "verbose" + share_label = "Give access" if verbose else "Share" + revoke_label = "Withdraw access" if verbose else "Revoke" + + tag = "a" if app.ui_button_element == "anchor" else "button" + role = ' role="button"' if tag == "a" else "" + href = ' href="#"' if tag == "a" else "" + + subject_field = ( + '' + '' + ) + permission_field = ( + '' + '' + ) + fields = ( + permission_field + subject_field + if app.ui_field_order == "reversed" + else subject_field + permission_field + ) + + share_form = ( + f'
' + f"{fields}" + f'<{tag}{href}{role} id="share-submit" data-td="share-submit">' + f"{share_label}
" + ) + if app.ui_share_control == "modal": + share_form = ( + f'<{tag}{href}{role} id="open-share" data-td="open-share">{share_label}…' + f"" + f'{share_form}' + ) + + confirm = ( + '

This cannot be undone.

' + f'<{tag}{href}{role} data-td="revoke-confirm">Yes, {revoke_label.lower()}' + if app.ui_confirm_revoke else "" + ) + revoke_form = ( + f'
' + f'' + f'<{tag}{href}{role} data-td="revoke-submit">{revoke_label}' + f"{confirm}
" + ) + + grants = "".join( + f'
  • {sid}: {perm}
    • ' + for (rid, sid), perm in sorted(app.grants.items()) + if rid == resource_id + ) + + body = ( + f'

    Resource {resource_id}

    ' + f'
      {grants}
    {share_form}{revoke_form}' + ) + if app.ui_dom_style == "nested": + body = ( + '
    ' + f"{body}
    " + ) + + return ( + "Lab" + f'{body}' + ) + + +class LabHandler(BaseHTTPRequestHandler): + app: LabApp + + def log_message(self, *args: Any) -> None: # keep the test output quiet + pass + + # -- helpers --------------------------------------------------------- + + def _token(self) -> str: + header = self.headers.get("Authorization", "") + return header.removeprefix("Bearer ").strip() + + def _send(self, status: int, payload: Any, content_type: str = "application/json") -> None: + body = ( + json.dumps(payload).encode() + if content_type == "application/json" + else payload.encode() + ) + self.send_response(status) + self.send_header("Content-Type", content_type) + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _dispatch(self, op: str, **args: Any) -> None: + try: + self._send(200, self.app.request(self._token(), op, **args)) + except Denied as denied: + self._send(denied.status, {"error": str(denied)}) + + def _resource_id(self, path: str) -> str | None: + parts = [p for p in path.split("/") if p] + if parts and parts[0] in ("resources", "r") and len(parts) >= 2: + return parts[1] + return None + + # -- routes ---------------------------------------------------------- + + def do_GET(self) -> None: # noqa: N802 — stdlib naming + url = urlparse(self.path) + resource_id = self._resource_id(url.path) + if resource_id is None: + self._send(404, {"error": "not found"}) + return + if url.path.endswith("/view"): + try: + user_id = self.app._whoami(self._token()) + except Denied as denied: + self._send(denied.status, {"error": str(denied)}) + return + self._send(200, render_resource_page(self.app, user_id, resource_id), "text/html") + return + self._dispatch("read_resource", resource_id=resource_id) + + def do_POST(self) -> None: # noqa: N802 + url = urlparse(self.path) + resource_id = self._resource_id(url.path) + length = int(self.headers.get("Content-Length", 0)) + raw = self.rfile.read(length).decode() if length else "" + try: + body = json.loads(raw) if raw.startswith("{") else { + k: v[0] for k, v in parse_qs(raw).items() + } + except json.JSONDecodeError: + self._send(400, {"error": "malformed body"}) + return + + if resource_id is None: + if url.path.rstrip("/") in ("/resources", "/r"): + self._dispatch("create_resource", **body) + return + self._send(404, {"error": "not found"}) + return + + if url.path.endswith("/grant"): + self._dispatch("grant", resource_id=resource_id, **body) + elif url.path.endswith("/revoke"): + self._dispatch("revoke", resource_id=resource_id, **body) + elif url.path.endswith("/accept"): + self._dispatch("accept_share", resource_id=resource_id) + else: + self._send(404, {"error": "not found"}) + + +def serve(app: LabApp, port: int = 0) -> ThreadingHTTPServer: + """Start a server on `port` (0 picks a free one). Caller owns shutdown.""" + handler = type("BoundLabHandler", (LabHandler,), {"app": app}) + return ThreadingHTTPServer(("127.0.0.1", port), handler) + + +if __name__ == "__main__": # pragma: no cover - manual use + import sys + from .mutations import build_lab + + lab, tokens = build_lab(*sys.argv[1:]) + lab.request(tokens["alice"], "create_resource", resource_id="R", content="the secret") + server = serve(lab, 8099) + print(f"{lab.version} on http://127.0.0.1:8099 tokens={tokens}") + server.serve_forever() diff --git a/lab/minimal.py b/lab/minimal.py deleted file mode 100644 index 9864964..0000000 --- a/lab/minimal.py +++ /dev/null @@ -1,191 +0,0 @@ -"""Minimal lab: the seed of the system under test. - -Deliberately small. Users own resources, share them with read or write -permission, and revoke that sharing. Every state change is audited. - -Two access paths exist, and the distinction is the whole point: - -* the **enforcement path** (`request`) is what an actor uses. It checks - authorization and can therefore be *wrong* — that is where a seeded - authorization defect lives. -* the **observation channel** (`ObservationChannel`) reads stored state directly, - without authorization. It is the independent channel required by decision - D-07. - -An oracle that consulted only stored state would verify the framework's own -reimplementation of the rules rather than the system's enforcement of them. An -oracle that consulted only the enforcement path would have no way to notice that -enforcement and record disagree. test-driver observes both, and treats -disagreement between them as meaningful in its own right — that disagreement is -the precise signature of the M05 authorization defect. - -TD-WP-0002-T05 grows this into the full lab with an HTTP API, a browser UI and -the labelled mutation catalogue. It is kept in-process here so that T04 can prove -the kernel without dragging in a web stack. -""" - -from __future__ import annotations - -from dataclasses import dataclass, field -from typing import Any, Literal - -Permission = Literal["READ", "WRITE"] - - -class Denied(Exception): - """The enforcement path refused the request.""" - - -@dataclass(slots=True) -class AuditRecord: - sequence: int - event: str - actor_id: str - resource_id: str | None = None - subject_id: str | None = None - permission: str | None = None - - -@dataclass(slots=True) -class MinimalLab: - """In-process resource-sharing service.""" - - version: str = "lab-0.1.0-baseline" - users: dict[str, str] = field(default_factory=dict) # user_id -> token - _tokens: dict[str, str] = field(default_factory=dict) # token -> user_id - resources: dict[str, dict[str, Any]] = field(default_factory=dict) - grants: dict[tuple[str, str], Permission] = field(default_factory=dict) - audit: list[AuditRecord] = field(default_factory=list) - _seq: int = 0 - - # -- setup ----------------------------------------------------------- - - def add_user(self, user_id: str) -> str: - token = f"tok-{user_id}" - self.users[user_id] = token - self._tokens[token] = user_id - return token - - def _audit(self, event: str, actor_id: str, **kw: Any) -> None: - self._seq += 1 - self.audit.append(AuditRecord(self._seq, event, actor_id, **kw)) - - def _whoami(self, token: str) -> str: - if token not in self._tokens: - raise Denied("unknown token") - return self._tokens[token] - - # -- enforcement path (what actors use) ------------------------------ - - def request(self, token: str, op: str, **args: Any) -> Any: - """The single entry point actors go through. Authorization is enforced here.""" - user_id = self._whoami(token) - handler = getattr(self, f"_op_{op}", None) - if handler is None: - raise Denied(f"unknown operation {op!r}") - return handler(user_id, **args) - - def _op_create_resource(self, user_id: str, resource_id: str, content: str) -> dict: - self.resources[resource_id] = {"owner": user_id, "content": content} - self._audit("create", user_id, resource_id=resource_id) - return {"resource_id": resource_id} - - def _op_read_resource(self, user_id: str, resource_id: str) -> dict: - resource = self.resources.get(resource_id) - if resource is None: - raise Denied("no such resource") - if not self._may_read(user_id, resource_id): - raise Denied("not authorized to read") - return {"resource_id": resource_id, "content": resource["content"]} - - def _op_grant( - self, user_id: str, resource_id: str, subject_id: str, permission: Permission - ) -> dict: - resource = self.resources.get(resource_id) - if resource is None or resource["owner"] != user_id: - raise Denied("only the owner may grant") - self.grants[(resource_id, subject_id)] = permission - self._audit( - "grant", user_id, resource_id=resource_id, - subject_id=subject_id, permission=permission, - ) - return {"granted": permission} - - def _op_revoke(self, user_id: str, resource_id: str, subject_id: str) -> dict: - resource = self.resources.get(resource_id) - if resource is None or resource["owner"] != user_id: - raise Denied("only the owner may revoke") - self.grants.pop((resource_id, subject_id), None) - self._audit("revoke", user_id, resource_id=resource_id, subject_id=subject_id) - return {"revoked": True} - - def _may_read(self, user_id: str, resource_id: str) -> bool: - """The authorization rule as the system actually enforces it.""" - resource = self.resources.get(resource_id) - if resource is None: - return False - if resource["owner"] == user_id: - return True - return (resource_id, user_id) in self.grants - - -class ObservationChannel: - """Independent read access to lab state — decision D-07. - - Bypasses authorization deliberately. This is the channel test-driver requires - of any system under test, and the main integration burden the framework - imposes on an adopter. - """ - - def __init__(self, lab: MinimalLab) -> None: - self._lab = lab - - @property - def version(self) -> str: - return self._lab.version - - def state_permission(self, user_id: str, resource_id: str) -> str | None: - """What the stored record says, independent of any enforcement decision.""" - resource = self._lab.resources.get(resource_id) - if resource is None: - return None - if resource["owner"] == user_id: - return "OWNER" - return self._lab.grants.get((resource_id, user_id)) - - def probe_read(self, user_id: str, resource_id: str) -> bool: - """Exercise the enforcement path out-of-band and report what it did. - - This uses the subject's own credentials, which can look like a violation - of actor isolation but is not: independence means the *actor's report* is - never the evidence. The observer issues its own request and records the - raw outcome. No actor is ever asked whether it succeeded. - """ - token = self._lab.users.get(user_id) - if token is None: - return False - try: - self._lab.request(token, "read_resource", resource_id=resource_id) - except Denied: - return False - return True - - def audit_events(self, resource_id: str) -> list[dict[str, Any]]: - return [ - { - "sequence": r.sequence, - "event": r.event, - "actor_id": r.actor_id, - "subject_id": r.subject_id, - "permission": r.permission, - } - for r in self._lab.audit - if r.resource_id == resource_id - ] - - -def build_baseline() -> tuple[MinimalLab, dict[str, str]]: - """Known initial state, so that runs replay from the same starting point.""" - lab = MinimalLab() - tokens = {user: lab.add_user(user) for user in ("alice", "bob", "carol")} - return lab, tokens diff --git a/lab/mutations.py b/lab/mutations.py new file mode 100644 index 0000000..9a2c897 --- /dev/null +++ b/lab/mutations.py @@ -0,0 +1,205 @@ +"""The labelled mutation catalogue — the project's measuring instrument. + +Every claim test-driver makes is measured against this catalogue, so its quality +caps the credibility of every downstream result. Six mutations, as the milestones +document originally sketched, cannot support any statement about precision or +recall; there are twenty here. + +Each mutation carries a **ground-truth label**, decided by a human from the use +case and recorded before any run: + + MECHANICAL the surface changed; protected semantics are identical. + test-driver should recover and report an adaptation. + SEMANTIC intended behaviour genuinely changed. test-driver must escalate + to a human and must never rewrite a claim by itself. + DEFECT the system violates unchanged intent. test-driver must report a + Product Finding and must never adapt to it. + +The distinction between SEMANTIC and DEFECT is deliberately *not* inferable from +the code — both change behaviour. It is a statement about intent, which is why +claims must have independent provenance (D-06) and why ambiguity escalates +rather than resolving itself. + +Mutations compose: `build_lab("M01", "M15")` applies both and records both in the +version string, so a mechanical change shipping alongside a defect is +reproducible. That combination is decision-table row 3 and the case a naive +self-healing tool gets wrong. + +## The test-id axis + +The UI carries stable `data-td` attributes, as a well-instrumented application +would. Most mechanical mutations preserve them, and a recorded selector sequence +keyed on those attributes survives such a mutation untouched — which would make +H-001 (semantic actions outlast recorded sequences) trivially *false*. + +That is not a flaw to be rigged away. It is the honest shape of the question: +**a semantic action earns its keep exactly when stable identifiers are absent or +not carried forward.** Mutations therefore vary along `preserves_test_ids`, and +H-001 must be analysed split by that axis rather than as a single rate. A +catalogue whose mutations all broke naive selectors would flatter the thesis and +tell us nothing. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Callable, Literal + +from .app import BASE_VERSION, LabApp, ObservationChannel, build_baseline + +__all__ = [ + "CATALOGUE", "BY_ID", "Mutation", "ObservationChannel", "build_lab", + "expected_classification", +] + +Label = Literal["MECHANICAL", "SEMANTIC", "DEFECT"] +Layer = Literal["ui", "api", "domain"] + + +@dataclass(frozen=True, slots=True) +class Mutation: + id: str + title: str + label: Label + layer: Layer + rationale: str + apply: Callable[[LabApp], None] + preserves_test_ids: bool = True + + +def _m(app: LabApp, **flags) -> None: + for key, value in flags.items(): + setattr(app, key, value) + + +# --- MECHANICAL ----------------------------------------------------------- +# The surface moves. What the system means does not. + +def _m01(app): _m(app, ui_share_control="modal") +def _m02(app): _m(app, ui_dom_style="nested", ui_test_ids="dropped") +def _m03(app): _m(app, response_id_field="id") +def _m04(app): _m(app, ui_labels="verbose") +def _m05(app): _m(app, ui_field_order="reversed") +def _m06(app): _m(app, api_path_style="short") +def _m07(app): _m(app, ui_button_element="anchor") +def _m08(app): _m(app, ui_confirm_revoke=True) +def _m09(app): _m(app, latency_seconds=0.05) +def _m10(app): _m(app, denied_status=401) + + +# --- SEMANTIC ------------------------------------------------------------- +# Intended behaviour changed. A human must decide; test-driver must not. + +def _m11(app): _m(app, require_share_acceptance=True) +def _m12(app): _m(app, revoke_delay_seconds=3600.0) +def _m13(app): _m(app, grant_permission_for=lambda p: p or "WRITE") +def _m14(app): _m(app, enforce_tenant_isolation=True, tenant_sharing_announced=True) + + +# --- DEFECT --------------------------------------------------------------- +# Unchanged intent, violated. + +def _revoke_is_cosmetic(app: LabApp, user_id: str, resource_id: str) -> bool: + """Record and audit say revoked; enforcement still allows the read.""" + resource = app.resources.get(resource_id) + if resource is None: + return False + if resource["owner"] == user_id: + return True + return any( + r.event == "grant" and r.subject_id == user_id and r.resource_id == resource_id + for r in app.audit + ) + + +def _read_permits_everyone(app: LabApp, user_id: str, resource_id: str) -> bool: + return resource_id in app.resources + + +def _ignores_tenant(app: LabApp, user_id: str, resource_id: str) -> bool: + return app.baseline_may_read(user_id, resource_id) or resource_id in app.resources + + +def _m15(app): _m(app, may_read=_revoke_is_cosmetic) +def _m16(app): _m(app, grant_permission_for=lambda p: "WRITE") +def _m17(app): _m(app, may_read=_read_permits_everyone) +def _m18(app): _m(app, audit_revoke=False) +def _m19(app): _m(app, revoke_delay_seconds=2.0) +def _m20(app): _m(app, enforce_tenant_isolation=False, may_read=_ignores_tenant) + + +CATALOGUE: tuple[Mutation, ...] = ( + Mutation("M01", "Sharing control moves into a modal", "MECHANICAL", "ui", + "Same action, different place. The canonical mechanical change.", _m01), + Mutation("M02", "DOM rewritten, test ids not carried forward", "MECHANICAL", "ui", + "Selectors break; nothing the user can do changes. Unlike M01 this " + "drops the stable test ids, which is what a real rewrite usually " + "does.", _m02, preserves_test_ids=False), + Mutation("M03", "API renames resource_id to id", "MECHANICAL", "api", + "Compatible representation change of the same field.", _m03), + Mutation("M04", "UI labels reworded", "MECHANICAL", "ui", + "'Share' becomes 'Give access'. Wording, not meaning.", _m04), + Mutation("M05", "Form field order reversed", "MECHANICAL", "ui", + "Permission precedes subject. Same form, same result.", _m05), + Mutation("M06", "API paths shortened", "MECHANICAL", "api", + "/resources/{id} becomes /r/{id}.", _m06), + Mutation("M07", "Buttons become anchors", "MECHANICAL", "ui", + "Element type changes; affordance does not.", _m07), + Mutation("M08", "Revoke gains a confirmation step", "MECHANICAL", "ui", + "An extra legitimate click. The postcondition is unchanged, so this " + "is mechanical — but it is the closest mechanical case to a semantic " + "one, and a good test of where the boundary really sits.", _m08), + Mutation("M09", "Responses are slower", "MECHANICAL", "api", + "Latency only. Tests patience, not semantics.", _m09), + Mutation("M10", "Denials return 401 instead of 403", "MECHANICAL", "api", + "Both mean refused. A driver keying on the exact code breaks.", _m10), + + Mutation("M11", "A share must be accepted before it takes effect", "SEMANTIC", "domain", + "Bob genuinely cannot read until he accepts. The old claim 'Bob can " + "read after the grant' is now wrong, and only a human may say so.", _m11), + Mutation("M12", "Revocation is deferred, not immediate", "SEMANTIC", "domain", + "A deliberate product decision that revocation batches. Looks exactly " + "like M19 from the outside; the difference is intent.", _m12), + Mutation("M13", "Grants default to WRITE", "SEMANTIC", "domain", + "Only affects grants that omit a permission. Inert for the reference " + "scenario, which passes READ explicitly — recorded as such rather " + "than quietly dropped.", _m13), + Mutation("M14", "Cross-tenant sharing is announced as prohibited", "SEMANTIC", "domain", + "Codifies existing enforcement as intended behaviour. A change of " + "intent with no change of code — the case that shows classification " + "cannot be a diff.", _m14), + + Mutation("M15", "Revocation updates the record but not enforcement", "DEFECT", "domain", + "Audit trail and stored grant both say revoked; the read still " + "succeeds. The defect that looks correct to an operator.", _m15), + Mutation("M16", "A READ grant confers WRITE", "DEFECT", "domain", + "Privilege escalation through the ordinary sharing path.", _m16), + Mutation("M17", "Any authenticated user can read any resource", "DEFECT", "domain", + "Authorization effectively removed. Carol reads R.", _m17), + Mutation("M18", "Revocation is not audited", "DEFECT", "domain", + "Enforcement is correct; the evidence trail is not. Detectable only " + "because audit is observed, not assumed.", _m18), + Mutation("M19", "Revocation propagates after a delay", "DEFECT", "domain", + "Not a decision — a race. Indistinguishable from M12 by behaviour " + "alone, which is exactly the point.", _m19), + Mutation("M20", "Tenant isolation leaks", "DEFECT", "domain", + "A user of another tenant reads the resource.", _m20), +) + +BY_ID: dict[str, Mutation] = {m.id: m for m in CATALOGUE} + + +def expected_classification(mutation_id: str) -> Label: + """Ground truth. Recorded by a human before any run — never inferred.""" + return BY_ID[mutation_id].label + + +def build_lab(*mutation_ids: str) -> tuple[LabApp, dict[str, str]]: + """Build a lab with the named mutations applied, versioned by what it carries.""" + app, tokens = build_baseline() + for mutation_id in mutation_ids: + BY_ID[mutation_id].apply(app) + app.applied_mutations = tuple(mutation_ids) + suffix = "+".join(mutation_ids) if mutation_ids else "baseline" + app.version = f"{BASE_VERSION}-{suffix}" + return app, tokens diff --git a/research/concepts/fitness-map.md b/research/concepts/fitness-map.md index ea72475..88151cf 100644 --- a/research/concepts/fitness-map.md +++ b/research/concepts/fitness-map.md @@ -1,6 +1,6 @@ # Concept ↔ Implementation Fitness Map -**Updated:** 2026-08-22 (TD-WP-0002-T04) +**Updated:** 2026-08-22 (TD-WP-0002-T05) Traces each important concept to the implementation, experiment and evidence that support it. **Unsupported entries are the point of this map** — a concept with no @@ -29,7 +29,7 @@ were aspirational, not evidenced. | `C-semantic-action` | C1 | `actions.py` | E-001 | — | Does identity survive restructuring better than a recorded sequence? (H-001) | | `C-oracle-independence` | C1 | `runner.py`, `oracles.py` | E-001, E-003 | — | Independence of components ≠ independence of belief. (H-004) | | `C-evidence-pack` | C1 | `evidence.py` | — | — | What is the minimum sufficient for replay? | -| `C-observation-channel` | C1 | `lab/minimal.py` | — | — | **D-07** — required of every system under test. Adoption cost unknown. | +| `C-observation-channel` | C1 | `lab/app.py` | — | — | **D-07** — required of every system under test. Adoption cost unknown. | | `C-adaptation` | C1 | — (T08) | E-001 | — | (H-002) | | `C-classification` | C1 | — (T08) | E-001, E-003 | — | Decision table is total on paper; unexercised. | | `C-crystallization` | C1 | — (T09) | E-002 | — | (H-003) | @@ -41,7 +41,7 @@ were aspirational, not evidenced. | `C-campaign` | C0 | — | — | — | Deferred. | | `C-metabolism` | C0 | — | — | — | Deferred. Depends on C-energy. | | `C-retirement` | C0 | — | — | — | Deferred. Depends on C-energy. | -| `C-security-mutation` | C1 | — | E-003 | — | Catalogue exists; no derivation mechanism. | +| `C-security-mutation` | C1 | `lab/mutations.py` | E-003 | `lab/GROUND-TRUTH.md` | Catalogue is hand-written; no derivation mechanism from use cases yet. | ## Orphan check diff --git a/research/findings/F-0002-scenario-coverage-gap.md b/research/findings/F-0002-scenario-coverage-gap.md new file mode 100644 index 0000000..fd972e5 --- /dev/null +++ b/research/findings/F-0002-scenario-coverage-gap.md @@ -0,0 +1,74 @@ +--- +id: F-0002 +type: framework-finding +class: FRAMEWORK_LIMITATION +status: resolved +discovered: "2026-08-22" +resolved: "2026-08-22" +discovered_by: TD-WP-0002-T05 +workplan: TD-WP-0002 +task: TD-WP-0002-T05 +--- + +# F-0002 — Two seeded defects were invisible to the reference scenario + +## Observation + +On first running the reference scenario against the twenty labelled mutations, +**six of six DEFECT mutations should have failed; only four did.** + +| Mutation | Defect | Verdict before | Why it escaped | +|---|---|---|---| +| M16 | A READ grant confers WRITE | `PASS` | No claim mentioned writing. The scenario never attempted one. | +| M18 | Revocation is not audited | `PASS` | `i-audit-append-only` checks *ordering*, not *completeness*. A trail missing an entry is still ordered. | + +Both passed cleanly. Nothing was flaky, nothing was ambiguous, and no oracle +reported `INCONCLUSIVE` — the framework simply had nothing to say, confidently. + +## Why it matters + +This is the failure mode most likely to be mistaken for success. A green run +against a lab carrying a seeded privilege escalation looks exactly like a green +run against a correct system. Had the catalogue been the six mutations the +milestones document originally sketched, this would not have surfaced at all — +which is the concrete argument for the larger catalogue, now made from evidence +rather than from assertion. + +It also sharpens what a verification asset is: **a use case protects exactly what +it asserts, and not one thing more.** Coverage is a property of the claim set, not +of the framework. No amount of adaptation, crystallization or energy scoring +compensates for an assertion nobody wrote. + +## Resolution + +Path 1 — the implementation changes to match the concept. + +Two claims were added to the reference use case, both `Provenance.HUMAN` and both +derivable from `INTENT.md` rather than from watching the lab: + +- `c-bob-cannot-write` — "A READ grant does not let Bob write R". + `INTENT.md` § Security by Use-Case Mutation already derives this exact question + from the reference use case: *"Can Bob write when only read permission was + granted?"* It was always part of what sharing means; it had simply never been + written down as an assertion. +- `c-revoke-audited` — "Revocation is recorded in the audit trail". Enforcement + being correct is not sufficient: an access change nobody can later evidence is + a compliance failure even when the access itself is right. + +Supporting changes: the lab gained a write path and the observation channel a +non-destructive `probe_write`. + +Both defects are now detected. DEFECT detection is 6/6, and +`test_every_defect_is_detected` fails the suite if that ever regresses. + +## Note on provenance + +These claims were added *after* observing that mutations escaped, which is +uncomfortably close to fitting assertions to the lab. They are admissible because +both were already stated as intent in `INTENT.md` before any lab existed — the +finding revealed a transcription gap, not a new requirement. Had the intended +behaviour not already been on record, the correct resolution would have been to +escalate to a human, not to write the claim. + +That distinction is exactly what `Provenance` exists to make checkable, and this +finding is the first case where it did real work. diff --git a/scenarios/__pycache__/alice_bob_carol.cpython-312.pyc b/scenarios/__pycache__/alice_bob_carol.cpython-312.pyc index a6b11c15875654ea58a26ce5a93134406dff4ed3..9053e9f2bf8653b87f2d25000d3a4ec49519cd5e 100644 GIT binary patch delta 3136 zcmah~TWnlM8J^jj_ipdSUf(@lCyke8w+WC4+$4>2aa%$xFo9lZu+N^EwU50%$DK24 zXWezzR6$A+qOA@h2m*>gNFW<2U?KW|1mcAUUN+E*vO>^=mx`xW&O-?g@Xy&jn{|oB zNIU=RnQy-N=fA#p(i5c-Gw>pZ;V%~SJK?Xn$fDD2h1Q`N31TqYA7-R(Ghzqe!8l|1Iiv<@?JKPT>jRC?(<23Q`ebz^l z+ud8~poh0SEml7rqX!;_djugi;s6V-?Soq#jYH3m(hPkBOr0(t%6x3+F79^+ z1No9xq_agOzZ3G_!4^q)tiO8)s_SWSJ?tdDzw-_}LY~m7EP&kF9Vc&t~n%OyrIH#LN_j*hakH^7}fE#b)8# ztg7hn@CJwfqDXWYq1k_a>Q=iuh23OfK1DE8Pe4J`< z&Q#j~(miIWP|(z>27nxy)QO_1%ruF*py=}qOprHd87!{b?=LC>T*+27YC8jLr~u#c zN>Nj!a|()9EO7<^aD(j&>AWVRCQMc=NV{`wenB)0sx}|g$Urwd^AiZ+U1^OA2!ll` z=yDjL4P3z#Nn237G>*`>@pX*RH=-U?cKyt~hBNNES6(XWJ`P4%V$-%)GOoHScZ45; zbEXR>Liiwv`+*blOlMbC_^I{GtURDrZ8Oel=B!ep5J@z5q6oGvnx}Ft?BkL~>hK== za1;G8I`tEFt@QdcufBY<_wCNi?da6MOqpx1#FO}bhfOmvlfiYjqL+EUJS*SY(4gMa zxVCj`u6Zrb0_Kj(GbC^aDUWDpdYXr__OqM7wWtqdJA znOcI>EobxPEF>S$!L(cww(1!de*!M72Th{{Q3j7HtVbHxa!Z7dpXgGv4Tof#4_wc* zMXihl2JuK4C7Nz$x($;NtQD=FhJR7zrzO8nEh^fwSr2OZDx@gX7gL4rts&b9lq|ga zH*j-%8dcCY(F^cx18=x0h{q^iltK29Q9({(>6p4U+@e{D5t?u^Z^J{~=g{Sk1C_Vp zS!~wFb{-LM{#Cpc?_$XMkFP&UJr!5QD}AZqZLc*HO;sZfjQT1N){9Z+{{uOXsQZUl zbU9yW#|>A-v+Z{}#D-5KT6+PJbk-HONI7d7d~kkC20skB+w65T^uOB$mynv;gNA8D z47C15AE0Hsz2Oovk?cWZSJ8Id=^k56^R_W1_AphGZB#+pO?z&*+sIP?hpcxGS>GP6 z{yl09JdCN}j&2XO%-A7DtlxBYNBm;cd5sve?sO*RIu>Q!{>X65xDgPtq}YQFRXU&{ zk2vq%2v&R*e}?H7z1>TiUB_63N@>IH<45cMJk*Aw zrZek$*9bmi{i^FU%pZ0g>-j90f?tGbJzyTqs?eOFk6f_M#^yYB<-)#h>({X{{D$>z z>=Zs_jl}!$v(`jh#k1BA;(hqb)(eS2eBF9GJ~B87H1$ZMs?0We3zVT}Shtd?qc6d#1V7UN$3sbv zqQu~Lo~y+Vef7fT*=s$2IhebqzP9k{!t3R?qNBGyv-#xC-DDm|T-=n@Zd35gn02{_ ze*cZTFd=SkGQ(wF|In7^` zA6~LXQ$xY8!4kAYrfNNvdJKQcx{^wVI&gM3>BCRr-MIDLl#;GR)3pN^-ih}9IXZkh zI$R@1-&}mB@60`~D{u~f;6s6^_3u>Qfjj<4ElO(Xi*3l`zkU3k8wWmz-|r5GeIE>3 Y1HC6>j}3=ioBn%e@Dcn;J`A`27xlB`;{X5v delta 1736 zcmaJ>&2Jl35PxsiYp*}zBu-*`{hgpfO%z%nEmcF(L@iYnN|91fMVcyI`>oL$e@Jg% z0*%&kKsi*1OL+$*6eK|E$-baSs2Bc%9we5Mxgw%%QdiM7!Y1ELPigd8ybnU$68AFMh81iCX7JyFft?g z1UO1!M-V4Sd_Y{1fm$URNgRPrwLJY4RFWj~sUv`C7g&6Wj1X*EkG$rQ=#D66?! z+5A&`%Thuf4i`4e3Ry4cmQgWl#$HW%i1&QKJCZUD*QWVrL%;YR`}7TBva>7vgYY%w zML3_^`#F3U^T#6#d}H*hrW3I08?MZR*t2Ys@O*ZVDY80WBkh zAKjA>`UJ6ltknap!FdBWBnGIixRHiT@C9_^1&DiWx!h|AO1e0(tXC~Uw#W9A{ruth z7@rCHP7}$NY>x&I6Sw77kGs|Tb&Nim^&l2>nTW^6HZF(wiLr?${t~lrKcehPw%qo* z>V^YZ&8Q+!naZU8NnjQ8-$f5_xVh70L2h!3GORFTe6a828 zvb0+Syu9STQp3lxIXA#2UC+jD<7{tv3VNJ8r?O*ilq790Nx|gzn%wZ)itV#i+s~eH zG44o}l&dJ8*}BLVCNA=|XcoW3?dT%tZ=$E7i-4v}ptk(h^D{+i=**z!xEi~OXSf-g z!xq0ATS(0U!U?@iVbs_4V$rZH$FuUv(uKTCFY|w6#pG4NBN|9Wrz|mZIhqpRU+0rr z3g6HJ$F1Xx;XI?YHk|TQR@ZEpu$sh%QBcH_IIu?okAU@vu9`(LO0-Jy#yFkIGql>2d?Of73ZhYC-59M(o_87w4S?N ze~>-@Q#k%(IP+aN^B{Nf-u1s_>|e%z27Mv#UlG2NnIAc(g`{oupEJ+mS(*Ws{{Sju BJjDP2 diff --git a/scenarios/alice_bob_carol.py b/scenarios/alice_bob_carol.py index 4d478d6..70bcfa7 100644 --- a/scenarios/alice_bob_carol.py +++ b/scenarios/alice_bob_carol.py @@ -19,7 +19,7 @@ from testdriver import ( Scenario, SemanticAction, StateObserver, Step, UseCase, VerificationAsset, Watch, World, ) -from lab.minimal import ObservationChannel, build_baseline +from lab.mutations import ObservationChannel, build_lab RESOURCE = "R" API = frozenset({"api"}) @@ -40,6 +40,25 @@ def _carol_cannot_read(obs) -> bool: return obs["probe_read:carol:R"] is False +def _bob_cannot_write(obs) -> bool: + """A READ grant must not confer write. + + INTENT.md derives exactly this question from the reference use case: + "Can Bob write when only read permission was granted?" It is part of what + sharing means, not an extra. + """ + return obs["probe_write:bob:R"] is False + + +def _revocation_is_audited(obs) -> bool: + """Revocation must leave a trace. + + Enforcement being correct is not sufficient. An access change nobody can + later evidence is a compliance failure even when the access itself is right. + """ + return any(event["event"] == "revoke" for event in obs["audit:R"]) + + def _enforcement_matches_record(obs) -> bool: """Enforcement and stored record must agree about every watched subject. @@ -77,8 +96,13 @@ USE_CASE = UseCase( Provenance.HUMAN, _bob_can_read, after_step="s2-grant"), Claim("c-carol-denied", "Carol can never read R", Provenance.HUMAN, _carol_cannot_read, after_step="s2-grant"), + Claim("c-bob-cannot-write", "A READ grant does not let Bob write R", + Provenance.HUMAN, _bob_cannot_write, after_step="s2-grant", + source_ref="INTENT.md#security-by-use-case-mutation"), Claim("c-bob-revoked", "Bob cannot read R after revocation", Provenance.HUMAN, _bob_cannot_read, after_step="s3-revoke"), + Claim("c-revoke-audited", "Revocation is recorded in the audit trail", + Provenance.HUMAN, _revocation_is_audited, after_step="s3-revoke"), ), invariants=( Invariant("i-enforcement-matches-record", @@ -90,9 +114,15 @@ USE_CASE = UseCase( ) -def build(variant: str = "baseline"): - """Assemble world, driver, observer and asset from a known initial state.""" - lab, tokens = build_baseline() +def build(*mutations: str): + """Assemble world, driver, observer and asset from a known initial state. + + `mutations` names entries from the lab catalogue. The same scenario runs + unchanged against every lab version — that is the point: the use case does + not know the implementation moved. + """ + lab, tokens = build_lab(*mutations) + variant = "+".join(mutations) if mutations else "baseline" cast = Cast() for name in ("alice", "bob", "carol"): cast.add(Actor(id=name, display_name=name.title(), diff --git a/src/testdriver/__pycache__/observers.cpython-312.pyc b/src/testdriver/__pycache__/observers.cpython-312.pyc index 193b6e8158d3ab4701dff7a3eb5c06d30072392b..033be7dd72170073546fa9d17ad8c122de543334 100644 GIT binary patch delta 333 zcmZ1?dQXh^G%qg~0}w0`>B{=EkvEo=DTrxu4y!5St<8&Ag&8M%vWYSBPflR-n=t zX0Mjf0Gimqa6?X^gZYk#_+-}(&KshV(_<#ad}8399K_MZqp7=|XC=>d4Tp;w4wD-> zM6@;eI-Doy%qX6bdO<+tB9H0^HU?g?e&0^t8LHQ%v@c3&U**yHAkNLH`$b~%7mi*= q?a57?UW|g1&vF(q8cufL@)eh06#l>fBtA1UFo}O;0Ux$Q#Sb^o((G4y!5a5(b7^=E?Wi#3t`y{mIOf#XmWL zO?0w6o5*Beb}rUsEDR-rlVjKo8HFeJvCE6H)UeKGn9F1XG^t0Gg`tKqm_d_u@*#Fr zPPW9-l*|$XWthC5Lz0nW@=cB|30|>&-%j5ds@J8oFG^`&<d z$ib=mMQHLm&R#~X$u?YGjC_;Zxr!JKCja8{6&Ga`{=fhvJ~J~giGO4QvWi53b^-t= CQ8v>6 diff --git a/src/testdriver/observers.py b/src/testdriver/observers.py index 40de7bb..7783820 100644 --- a/src/testdriver/observers.py +++ b/src/testdriver/observers.py @@ -41,6 +41,9 @@ class StateObserver: out[f"state_permission:{key}"] = self.channel.state_permission( watch.subject_id, watch.resource_id ) + out[f"probe_write:{key}"] = self.channel.probe_write( + watch.subject_id, watch.resource_id + ) resources.add(watch.resource_id) for resource_id in sorted(resources): out[f"audit:{resource_id}"] = self.channel.audit_events(resource_id) diff --git a/tests/__pycache__/test_kernel_guarantees.cpython-312-pytest-7.4.4.pyc b/tests/__pycache__/test_kernel_guarantees.cpython-312-pytest-7.4.4.pyc index cfc3f095cb9bd8b3dad55a3e2c0e3b7835b39829..0fcf25b8e6b396f926051b044f1d70c6b9b0715a 100644 GIT binary patch delta 595 zcmeCE{Zz|$nwOW00SI1-bY;bgZsfbk#LniEnB-Vcu=x|y6g9>Ro7*+r*+pC!`lXYZ zG8vMYV)+>uYNczWv)m>xWEHlRp3KxE$ih%70~C`<5s?HMRV!N~o8?nL z7F8zcP5~$3Q4CFFQR^u1kY-qWkiR}cCS9E;x zH)~C9FcajTqQjH5YYc0o~lbEEJSWr*|j5&rP0T3YwB7{K1 zy2-0;4OKwiECT5(5(g2oAVLaA{Nk|5%}*)KNwq7wJju?E)t&J&gWF_hJAJXTFAM^V TT%YvB8T}bQo3JoyfHeUCl}e_~ delta 429 zcmexV+fmDRnwOW00SK~$y0Y>`HuBwM;^y+r%*)J8%<)M~+Wd}biW=jj&CQzb>^v!w zl0b#E(lydqZj%o(3k%AmNYx6}$kYgBYXHS%Yh)+u87lHi*T|+ZrpSQA)0kGXOcu~r zWt5)Gr?1W^10pBK>T)s4PQGUtE-78XSStg>vb92$!pTgG49QHj?Bxuaa+@QJte99| zG6AidJjo=7QDyRflV|Kj-ax*l+~g0YTPJTc6K9bFNnbRRp8V3xlIt*#lL^GdnwxdZ zA2S9&XYsmD0lLEC_4AWF1D`8nF)Ps2m6CxhERzd(#5T{j+|R^z94J(DY;uZ?CO4RA z3*sM|Jj2F^^)!&#H2I~?CB}7=_t;wVaOWf@>E%KKbn<^&QO5O?MePjPK+Y{XJ=w=j smhsZ$EIT_^f5yuUev{YQ>5El;VGv;C`lKet=*{@qh=oz3NEWCV043ajzyJUM diff --git a/tests/__pycache__/test_lab_ground_truth.cpython-312-pytest-7.4.4.pyc b/tests/__pycache__/test_lab_ground_truth.cpython-312-pytest-7.4.4.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e7f430471ab203796b41093d59177f0db8776513 GIT binary patch literal 23264 zcmeHPYmgkpeV^H#eebRA^Z*HvG(rNjdaUmB7Dz%T>11T2;~{MD8M3T)=I-{ikE3T# z(k;8>AWD#I#gdI<rSjMhC50zUP|;OO3A(?zaA>Kl~R4F z(wx3IrS`t|Qb%70zaB2mEzRqjhvU^+OTk=#5$ebfK06U8J>v z-c*pZl%5>gY**ObKPPK*roGcX?VXNk@64U{&b(>w%%Aqof@$yEFzuZir@gZfcec81 zw~x>Vi>AGE)3l}CJnfxZrY&vpw0D+FcgO$wN4cskeKGjwa^GT2)s|_uz8L6RqAka$ zSgOZ{cG-R4?uYic$M%u7A|UCv>Wg2M#g$hBVz1qHLVDFo)VEx_9i^_&?f|__TM2r* zmIl2;TLrpuG?-p9as>rEG^ne^++np~s8t-6^qeum3grQ{P&TS;q@&NK0NZ>2g&83uJ`r=gOL=&Y<4?=aMSQ9nw@hN1rR*w^_}#65K| zXQ<^0666e{&|lD*I;x|@hX-?2st=0K9WEA(L0z+}w`$b@tBjO26;-Pau3Dv1(W7WU z6`4idb*eH?pF5(L#C90%Q;r4s7yD8 zD`gG$D{Isoo-|lFhDK61iCgH1Qm)Dh$JN7nv2u)_lq;)ffzGO^Xn!tWU4vFt)tqXS zbHm191s%A*AE)`jTzLRpRG~hIQ>RxL?+)kTrOHtq#i@C{SX_f!q8}#Z^nFKhs%mM( zsG?RVK+}tb!#YE^tU`Wu2Lk&+AMp z8wx|1{?gRII=5sw}?C?mTsAW-g^G5ynu%1VYv-a!D^TJSch^NY1 zBa!b2lOvKOUwZ@x&r4Nnx*bmwVGeMvQ?`4l~F`%IUBfbKbcbI#yvt%?xX-fN^6+D~Ivc=qN<*uWE)X zEQjH0P<2(leYj=aV5Ow5Lq;P%xQ;%7wIVk^HaMjY$%=uQw=14iW zor9@4%HYyk=(^QtO0Tu7(UcshQOMJ7pH|eJchVxAZ1;;6eo?964K;#VM2psg1(}~I z)egIBJSEMwuep2o`>;!VM%H4-<%WcRpdpXO((&59LqIdM+yu z1p}B#aE0Ssu?UWE3@AHFNdY(mg)&er^X_~+H!Xh{*tsJu*E(-oIXt={ZN#6n4%8hg z9d)LSm9+0l8|ipM-6^haP8(*hsFxXyYhX_h35j$O*+67-EhRvfm$H@8507r`O~=_y z$inlP@m<5C^oeG3%C4`UT(&dc^!82CUIFj&l$1@wi?1U*4ibG`Ru zllNq!I1vSG@t$lm=(FU&!>BXOPTm?IYpz%s7}2u@16acb^sHX4j0_BBtCg(vnzFep z!981xY>V+X${=IXJId0D-^lMxWhuJ>k8A>|C@yf2dIl6Ag;TNZmCnkK{Sd<%3}DFl zknFp}4ggsJVgN@m^y@+6HP0~j2dp3mEadoDfVIF=z7JrzA1?EzobeMUfbx{;m)7~r zTciuvAFmdAF(?2)3u}?OV!=UZ)^HHAOS1QX1A+n>{~&r?8lrJGX#?|+hv?-_+8V)t zR1ey)-w4;k#!o#hSvV;HOQaaIlK4``d_UyT*yLbPd?gM7@P|v~c-iP+@ zF~fU%_Vw&}=)7zO3T43eTEgh{Hhkm2h3^c+9lZ$b2JBO+O~B%)%|6Zt+0A%;1kUS; zZ2Hg(+gk)|Z>e?I!aqO@pqeY{wOo9s2jIaF1n0nk_2`ZEB^+3nrm3J)XuOeu7E>6G zH&%k#q!nH_w#BO8Mz#b;>6Bm|A>J&e;u2d%r*awM2b{|+1Sf&PN8T%)k@A58{*Qs55|7aW&jn6L6jC#qxX@#ZaG&!T zKQ44Ua6Etnk`}DBfwKkkf$caueX{~|IswtxPa(0923J;)I)4TJ{V#@n+xEssNQbparae+huZalxa;Qd5hSPP0DlY5oX2f# zi*$|>r$sFO7OIlE#5Ye~A07nt(CSfpZ($#l$x(~|8UKcKOiD+4**cWRNZNRpvgECN z{2>@W5_B{98hzmRN;1Y#U3v^XLcC9tNOAgx{JbnnrxQR@`L*CNIURUcLE_%?64d8F zrqc}NVVbBK^Z@Jo{4lnCKrcf?WIJp7`KQV;#>nQO=(8-zr`18G>q3XtjqI^Pbuc?( z==~$b>@mGCFj&oO7h>wpj8oJaJis7E_wUkwk4Y2C-9K0E{=lHRJTs?5u4S5V)CV+e z2^`=FP&9+6hK1T38QEPZDV^k?$Y{Wu;iI|Yh;Fc6oCtipn~n%drp3o}*vcz9g<~M= zrX0ILya>0+qizdS+x+pmiXB49EAcm;0~wPpt+?l{w*BWeoz%}haBA>E+y1eAAV9sV zk{nubGI}YRI(hueQ>UI9k1lz6*~RGcH=|3g1aa#fy8r$7 z-iVS%1)Yt)JC0x9#XFoNB?lh+EFqZ3C?XBfo1V1UWB*YMJ~E|zr1T{B$$n-4U{uie zdZ@0{Lr2injj#s8f?thZkiHdYMCze>gr|$v!z@^jo|TOjMg*81!u4o>0LBMAqaNuG zddv^AMZX0e%juMW2_(fFiN6g zBtrXm&C?zq1CsCne~~@wdVsL26pUVp9eOj1GNJ)C=g(NWKe<%u|t@D zpce>f9|x9b53M5Oq^qr?(psgmOWOu5Bh+7D1}wUUo`*R<9dadqwiOj*cY=_9vaL49 zqjhw_DrG7#J9V9xc|*?2wQVyrBsV-GcWXv$(7G@L2Y}6v1!ya7~LyLkTge-6j&)V4H7-~G=7zg_b z;IE+1{Y_{AE%CA5NAmyQhv&`uvAq`WV>F&_tL=Ocp4ts+rJws-HQS*<{$*V)>pJ|s zIqn9}=NKC$Z}wq$wN?0EEAS3y%;3RJcp?sVZK`#+{0`Sv;}ZKKpe!v57B!7Q%3)+S ziz1s@5MVxuMZgow`cWJkAOfWmHao`o#bb0qCDNKru-~DK5jIH4vqXp?aKz_~zys|B zH5RT5P`{mny2H<)sg{N-hQM(+e=1DNQrEBoGLjw5AOskb{_0noXIkRhckp43J|4h*OstytQObf^r3QOat<}2B#q!Jd?I2%ud)hhx|`rwktz$rb)RTqz+q z(rCgpgSkiPWNq&~Ydbr;+^`Xt;YSS)i3Y@4gp!ORYDPz}3R8I~4#H&gB7!~;3jyyi zJBlC*EmMnF^06?h$KWpJK^2;~0s9j-D47wBRfb9W!&fHN(i~rnf`)h(3zZ3Y))o+^ za=c)OI_Ws7*4mrwSPc8ehm{dY!wh8FiNEm!5Mb)km2b6opIi1y_r#iA<7;-E?Rb9vbMr5>caQA{d3*8g zZ>9E}lV8bAWOk2dcAtFWO!dpv3#mP0p8>h79Gv*%e?M0aesDD;b?lZuFjIRBGTEN$ z=vl42kOKLuj-F2CC!rK5=V`=5;SxI733vX%&W%Ji5!p;+3z4lvwt+ww0pT2%a&%FS zu8o{2(h=q8+L8{^j`DO-p7ouz`zqz)sG95V#~3x#0hWU`Rzy)%N*K;Lby#Pm0%FZ7 z<-52b-~?U`lU8J?5B7BLRmr1Y$QP=`(b}Gz9khipTEII(WEq)gdDNPS=E9A9Mbq!1 zkS`tvCt`~DS481l8aVpzqF>k;NFIM@BV^`$=WBxRKo=65jnC6@oQ7Og?6kh-IquLH z;f{0ggt;EGfsX_a10K1~c^;44ADp!b%j-DsF=4q*e;b)vx*F`pIngq}=b)z*eyHoD zZIRC5XmBvk*6h=&h0fcUbqlBDn1y-K;Z_I&5XLxVQV_te>oV~K0>D`u?d8s31ETHY z1o1we1b=ClT&-lfp)f-75*v2Jjxg~cbjLFBz_pvXzx&Xkc!LMXF_OfZEH&hb;67Tm z6h|L6S!&?L;Fg+BxGJdO+?Ya-p$e46l>74kKeq8J|6SiEREoe>dF9(Z`PQoHL@P!9~ z5bvI}!P(%9=DI}gQlaMpe}qgLpINd4{*c`PGHjFGA?T5Jd?ZsFBe|>7l^lH}Q(ydX zbIb~{2!DVVt^_=DjTU%X>{BDRPa&bsSQ9rT$A_8II>^?dOb16=q;oAMHr_x=N6?XL zF_~u%1y4yn@@?Iboo#&dgr`_m6OK3RSriDkIY@8=4O!p?4lKNA8BOSmIs#kHQqh(b z_!b;93X&71V8Rx5ipZBhpl}nMurGs{?LCj&kEocQz1h$AJh*rNo`+_QC=o_aWGjiR zBJzhs9w$O@!oC7x&;MsBG8<~L?M5(=aAuX%1Pk>6y0}V;y3=gs2rH8 zMAN6ig_$=&o0Vup`u*ed>a!-3ff7pee19bx!j3cST_qD9%YdJnh*%3p#PL&T0ZH&H z0-8eg5JaPAt8-9qz76>R0YE7PXhQrUAj%sC?F@XsXt^e6a?AILU6>kE0ny0K;iKkF z%|XC`u;ba*Xo07EpBlM<18ADDCT>d9Ak-qQ3r#bZ=|a=Y>0D?E&${K_p7m+DJ8o={ zYWXglk!I&WPlQ|agzF=Yycpre0dJ0*?9u4%{l5zPiCUD$1;MXr84My-!Zp5mYC>Ud zleW6!+qKIAD3ckRB0&k1gZPIUFe;t?C;0l}=|pYYA-x2Zs*s1V6jOa{y<1zO8hlEN ziog{_gwq5WnutPkVDK-%e#$!WbQ|5DCUp0AO3=H&LCq$QcpN!AWd^JcR5EMEXE#9ZhD4E>ww&pBsA9?QWtN^!OD}hWOjBArRw+_KLddsLc|F^fH=WRiFs#sp4xft(eXt3 za`OI(|0BOQllN2b-~&FvgT!1oV45hN+C&jp=!0qT(hfxRcxoHw?1}bv~moPGX^z3{xb< z1V?+O9qh4U;*CA!5k^xn%`wB)W6aqeskwx-z);)iM3@yO0~ohT!KWdl;M&cYQK?r^ zBTN!B;0(h~JleQZon(HcsD`b`*o=A{@ODvoBsG6r)C;p=%KgVgxZF%Sx!20NksDa2 z=-!LD&@6(WMZHO4wp+wrL@7o5jTb=1q+dp*8`bxN!BqHC$J|Sa>nogljJ^~Wc&7*rko z7ZZ6Pv_t5Qxt{hRA4?ut0=Ura$=f7+(_rJ9-;@-O@V&tE93N5I6<~o;j+e@BdWxHp zW7N!P9Xx|jJlmQs@;u+CrfwhF*aOjY##&6xK|(r$_{8L)peHv8J(D*7w~0wQ5f75u zf~W24Q>vs!^UYJbYF+Z|d}kzb!LxzUaT>9tT-f|fw=*NL*;EhYBgqmtGg`=LwFz0w#J#=uJ=~k!-`n%x zL+n0W8F>_h5V}x4ib;_aJ|O0EV`k)(x#Tn_o1hiAjjHh3?>f8(1m6_@O|$65luCHj z;b|MC7EHohm`J$+v^U-1kus;n+do0=KANZ*niQEW99gST&g8j>tAu+ElfhO@_?fAXtOjwhB+BvxNc ztp4t?@72%O-$>kbB@k>&{VFUaQ>WJdDniE+2%xtu^`GH)CHK-V=@Jq~Q%@h|SFbju z;b*uVNtoF;0YirCgKHLoZ=JnAQ3vLxY>hChapWFK*%jZV8i?z z_B;e;M}W(J;0BHR1#}}ajKZ-4kf|s%JM0O}j<_ZoKl!!^lhI1C`OP4&JM97{yD5`) z9(V7i1hQdf^mVI&<40;)11BYF5N%Og*I@DK_f_n=^_4TeTK1Kb5`7hG(N~U%q(wT% zs*DMf!jux#5b#`&TC#o2)eSr)`DjD+nEjpDd&u@}wh09L2>uF9b!fz0b5q?3X>TO7 zKqKk7G?SZ=5VlUy}Ht#-J9ks*&t3r zT`8vdrXAU-P}UG+gNVlwEVRR;7W!GGgmrZgnnR(F@i=Ls4-JyYBNoj;`aCO?3S!X% zQfnavl}L0dAlw3luwg{TBt;!gw=2Lu(gcUDz1LiYPYDLYd zgCj7BW_S_oDST+`1W1!>l?FDZuwqWwz`7s0X2ym_v;kU9$(W?q zn@M4YL?8`3$(a5qGgBTis5$n`$LRJVI!(*%tUbu(MpDcfq+kR*9jzR(<4O4;*4)84 z#7E~{V@BdA3dWPkSm~JpX_a{z4>ERvj7eWV`uY<;dh+{EzAwp3yXDuHUAlGochcWZ z|Miw1ZU6rEi?`kl-CbSw(o-)yg-L%%a7~gI?v}@-55y^#algK+BUGj#4OCtIB;Ve_;M#Ocq3u0i^^-6%r1{lM+wvD z#@mS6LO0eg;`Sp%_;MSLYquslKaY;{*sYs_CeRZ0M@ZC5%d{+`Wm@j13i*U~*bNku zMAcDa#a0&)t&cHcY7?up;--$iBjW#!(9Pw)4Ja_95M-sHJ_^rwXS$fO^^LA(5V^eaV z$GkJ9{CdpMZ!iGntmJ$hKL1DGM))3&!Se~83yqTpuOM(P4Rr& z-(~BV<+^ZWF;8MQnSI^+4>UyxT5NAR-on0%S~fEnJd`t-jlEW|pexUq3YFi&X6A4l zEau{}4$f0J|KQUoEzfKhdkxQLe+kmWMK<|(X7A$MBjDC{SZ(18vX_miUry}@muq|sBGsTjNoT-0GL6nkdscEk!)70oJ(fJy2jt085UX0ICZL6 zqmi!zDdIuf;AqR(;g1F*7p-SDF5#lN$PNcLGC4$mLmsUh#v*d8QWT3ya?xZhW$Ynx zh=@j{K*S(If+_m}5qc?%n`6)5$W%%>cEn7OMpDwTB2i6ee@ZEyBSQXm?xx~)Q!Wwk zx63|DH|Wdf!p4Wu?Ip~sE$Tbj-{2ady74nmtXUz;?*(GA@?SSd^73~i>4fxqZSP38 zUzU2urQUy#R$mJT}Wel>E#|NoQAJeT`eC zM#T9|ORaLP^K~1olKDAYYj*k=rH{Mk332Y^=jTcHejD9SO3rJ|7V+t2%ZfQwkNoIat@#LF~O?(zWsRo8Sx<(s3 lFX-BB&S0ulWHj3ROxcfrvV-v##@5NECR&W2CnuQb0|1kACQ|?a delta 179 zcmeyD(VwY)nwOW00SK&wy0ZRqFfcp@abSQM!eIP7X`*@(D}M@G3+u!leMW(adm8l^ zVTyn>8xRB4GJZ}1GNv<>z~yQfRzuj6QyEh@*vk@&G86MkCSPW3;xz{gYqY_9 vb=>A;rdmZt!_5zs{rI^VZ*e3g7N_Q9=A};7GyclhHd)0)i}A~3e-nKGcA+U* diff --git a/tests/test_kernel_guarantees.py b/tests/test_kernel_guarantees.py index 44a2547..d56280b 100644 --- a/tests/test_kernel_guarantees.py +++ b/tests/test_kernel_guarantees.py @@ -15,7 +15,7 @@ from testdriver import ( Claim, InadmissibleProvenance, Invariant, Oracle, Provenance, Runner, SemanticAction, StateObserver, SurfaceNotPermitted, Stratum, Verdict, ) -from lab.minimal import Denied, MinimalLab, ObservationChannel, build_baseline +from lab.app import Denied, LabApp, ObservationChannel, build_baseline from scenarios.alice_bob_carol import build @@ -83,7 +83,7 @@ def test_actors_cannot_be_recorded_as_judgment_collectors(): # --- preview of the M05 authorization defect (built properly in T05) -------- -class RevokeIsCosmetic(MinimalLab): +class RevokeIsCosmetic(LabApp): """Revocation updates the record and the audit trail but not enforcement. This is the shape of a real authorization defect: everything an operator @@ -101,6 +101,10 @@ class RevokeIsCosmetic(MinimalLab): for r in self.audit ) + # NOTE: this predates the mutation catalogue, where the same defect is M15. + # Kept as a direct subclass so the kernel test does not depend on the lab + # catalogue's wiring being correct. + def test_seeded_authorization_defect_fails_the_run(): """The kernel must report FAIL, not adapt, when revocation does not revoke.""" @@ -124,7 +128,8 @@ def test_seeded_authorization_defect_fails_the_run(): assert result.verdict is Verdict.FAIL assert result.judgment("c-bob-revoked").verdict is Verdict.FAIL # The claim set is untouched by the failure — there is no path to adapt it. - assert USE_CASE.claims[2].text == "Bob cannot read R after revocation" + by_id = {c.id: c for c in USE_CASE.claims} + assert by_id["c-bob-revoked"].text == "Bob cannot read R after revocation" def test_defect_run_emits_an_energy_event(): diff --git a/tests/test_lab_ground_truth.py b/tests/test_lab_ground_truth.py new file mode 100644 index 0000000..d2fcbc2 --- /dev/null +++ b/tests/test_lab_ground_truth.py @@ -0,0 +1,148 @@ +"""The lab is the measuring instrument. These tests keep it honest. + +Nothing here tests the kernel's cleverness — the kernel has no classifier yet. +What is established is the **ground truth** every later measurement is taken +against: which mutations the reference scenario responds to, and how. + +The response matrix below is an asserted fact, not a snapshot. If a change to the +lab or the scenario moves a cell, that is a change to the measuring instrument +and must be a deliberate, reviewed act. +""" + +from __future__ import annotations + +import pytest + +from testdriver import Runner, Verdict +from lab.mutations import BY_ID, CATALOGUE, build_lab, expected_classification +from scenarios.alice_bob_carol import build + + +def run_against(*mutations: str): + world, driver, observer, asset, oracle = build(*mutations) + return Runner(world, driver, observer, oracle).run(asset) + + +# --- the catalogue itself ------------------------------------------------- + + +def test_catalogue_is_large_enough_to_support_a_rate(): + """Six mutations cannot support precision or recall. Twenty can begin to.""" + assert len(CATALOGUE) >= 20 + + +def test_every_mutation_is_labelled_and_reasoned(): + for mutation in CATALOGUE: + assert mutation.label in ("MECHANICAL", "SEMANTIC", "DEFECT") + assert mutation.rationale.strip(), f"{mutation.id} has no rationale" + + +def test_labels_cover_all_three_classes_with_useful_weight(): + counts = {label: 0 for label in ("MECHANICAL", "SEMANTIC", "DEFECT")} + for mutation in CATALOGUE: + counts[mutation.label] += 1 + assert all(count >= 4 for count in counts.values()), counts + + +def test_every_mutation_is_reproducible_and_version_stamped(): + for mutation in CATALOGUE: + first, _ = build_lab(mutation.id) + second, _ = build_lab(mutation.id) + assert first.version == second.version == f"lab-0.2.0-{mutation.id}" + assert first.applied_mutations == (mutation.id,) + + +def test_mutations_compose_and_record_both(): + """Row 3 of the decision table needs a lab carrying both at once.""" + app, _ = build_lab("M01", "M15") + assert app.version == "lab-0.2.0-M01+M15" + assert app.applied_mutations == ("M01", "M15") + + +def test_test_id_axis_is_represented(): + """H-001 must be analysable split by whether stable selectors survived.""" + preserved = [m.id for m in CATALOGUE if m.preserves_test_ids] + dropped = [m.id for m in CATALOGUE if not m.preserves_test_ids] + assert preserved and dropped, "both sides of the test-id axis must exist" + + +# --- the response matrix -------------------------------------------------- + +# Ground truth: what the reference scenario reports for each lab version. +# `None` means "no assertion in this scenario covers this mutation" — recorded +# honestly rather than papered over. +EXPECTED_VERDICT = { + "M01": Verdict.PASS, "M02": Verdict.PASS, "M03": Verdict.PASS, + "M04": Verdict.PASS, "M05": Verdict.PASS, "M06": Verdict.PASS, + "M07": Verdict.PASS, "M08": Verdict.PASS, "M09": Verdict.PASS, + "M10": Verdict.PASS, + "M11": Verdict.FAIL, "M12": Verdict.FAIL, + "M13": Verdict.PASS, "M14": Verdict.PASS, + "M15": Verdict.FAIL, "M16": Verdict.FAIL, "M17": Verdict.FAIL, + "M18": Verdict.FAIL, "M19": Verdict.FAIL, "M20": Verdict.FAIL, +} + +# The two SEMANTIC mutations the reference scenario cannot see, and why. +KNOWN_INERT = { + "M13": "only affects grants that omit a permission; the scenario passes READ explicitly", + "M14": "a change of intent with no change of code; nothing observable moved", +} + + +def test_baseline_passes(): + assert run_against().verdict is Verdict.PASS + + +@pytest.mark.parametrize("mutation_id", sorted(EXPECTED_VERDICT)) +def test_response_matrix_is_stable(mutation_id): + assert run_against(mutation_id).verdict is EXPECTED_VERDICT[mutation_id] + + +def test_no_mechanical_mutation_changes_the_verdict(): + """Semantics are preserved, so the use case must not notice.""" + for mutation in CATALOGUE: + if mutation.label == "MECHANICAL": + assert run_against(mutation.id).verdict is Verdict.PASS, mutation.id + + +def test_every_defect_is_detected(): + """The floor of the whole project. A defect the lab cannot surface is a + defect no later classifier can be measured against.""" + missed = [ + m.id for m in CATALOGUE + if m.label == "DEFECT" and run_against(m.id).verdict is Verdict.PASS + ] + assert missed == [], f"undetected seeded defects: {missed}" + + +def test_inert_semantic_mutations_are_declared(): + """A mutation the scenario cannot see must be named, not silently ignored.""" + for mutation in CATALOGUE: + if mutation.label != "SEMANTIC": + continue + if run_against(mutation.id).verdict is Verdict.PASS: + assert mutation.id in KNOWN_INERT, ( + f"{mutation.id} is invisible to the reference scenario and " + "undeclared — either cover it or record why not" + ) + + +def test_deferred_revoke_and_revoke_race_are_behaviourally_identical(): + """M12 (SEMANTIC) and M19 (DEFECT) must be indistinguishable from evidence. + + This is the discrimination problem in its sharpest form, and the reason + classification cannot be a diff over observed behaviour. Both produce the + same failure; only intent separates them, which is why claims need + independent provenance and why ambiguity escalates to a human. + """ + semantic = run_against("M12") + defect = run_against("M19") + failed = lambda r: sorted({j.assertion_id for j in r.judgments + if j.verdict is not Verdict.PASS}) + assert failed(semantic) == failed(defect) == ["c-bob-revoked"] + assert expected_classification("M12") != expected_classification("M19") + + +def test_a_mechanical_change_shipping_with_a_defect_still_fails(): + """Decision-table row 3: coincidence is not exoneration.""" + assert run_against("M01", "M15").verdict is Verdict.FAIL diff --git a/tests/test_reference_scenario.py b/tests/test_reference_scenario.py index 090e807..5739969 100644 --- a/tests/test_reference_scenario.py +++ b/tests/test_reference_scenario.py @@ -10,8 +10,8 @@ from testdriver import Runner, Stratum, Verdict from scenarios.alice_bob_carol import build -def run_once(variant: str = "baseline"): - world, driver, observer, asset, oracle = build(variant) +def run_once(*mutations: str): + world, driver, observer, asset, oracle = build(*mutations) return Runner(world, driver, observer, oracle).run(asset), world @@ -52,7 +52,7 @@ def test_evidence_is_stratified_and_serializable(): assert pack.of_stratum(Stratum.JUDGMENT) parsed = json.loads(pack.to_json()) assert parsed["run_id"] == result.run_id - assert parsed["sut_version"] == "lab-0.1.0-baseline" + assert parsed["sut_version"] == "lab-0.2.0-baseline" def test_evidence_records_claim_provenance(): diff --git a/workplans/TD-WP-0002-vertical-spike-crystallization.md b/workplans/TD-WP-0002-vertical-spike-crystallization.md index c558ae1..9cc9215 100644 --- a/workplans/TD-WP-0002-vertical-spike-crystallization.md +++ b/workplans/TD-WP-0002-vertical-spike-crystallization.md @@ -222,7 +222,7 @@ Three things that came out of building it rather than designing it: ```task id: TD-WP-0002-T05 -status: todo +status: done priority: high state_hub_task_id: "595a89c2-1462-57fc-8b08-a5a6b875fd48" ``` @@ -240,6 +240,31 @@ The lab is the measuring instrument for every claim the framework makes — a we lab caps the credibility of all downstream results. It is also potentially the project's first publishable artefact in its own right. +**Done 2026-08-22.** `lab/app.py` (users, tenants, auth, resources, sharing, +read/write, revoke, audit), `lab/http_api.py` (JSON API + browser UI, stdlib +only), `lab/mutations.py` (20 labelled, composable, version-stamped mutations), +`lab/GROUND-TRUTH.md`. 48 tests pass. Detection: MECHANICAL 0/10 flagged, +DEFECT 6/6, SEMANTIC 2/4. + +Three results worth carrying: + +- **F-0002 — two seeded defects were initially invisible.** M16 (READ grant + confers WRITE) and M18 (revocation unaudited) both passed cleanly: nothing + flaky, nothing `INCONCLUSIVE`, the framework simply had nothing to say. A use + case protects exactly what it asserts and not one thing more. Resolved by + adding two claims already stated as intent in `INTENT.md`. Had the six-mutation + catalogue from the milestones doc been used, this would never have surfaced — + the argument for the larger catalogue is now evidenced rather than asserted. +- **The test-id axis.** Stable `data-td` selectors survive most UI mutations, + which would make H-001 trivially *false*. Rather than rig the catalogue, + mutations now vary on `preserves_test_ids`, and H-001 must be analysed split by + that axis. A semantic action earns its keep exactly where stable identifiers + are absent or not carried forward — that is the honest shape of the claim. +- **M12 vs M19 are behaviourally identical.** A deliberate deferred-revocation + decision and a revocation race produce the same failure, same step, same + evidence. Only intent separates them. This is the discrimination problem in its + sharpest form and is now a test, not a paragraph. + ## Out-of-band ground truth for self-verification ```task