From b52a9ec88ae8b448f5322207699ea04e44c84581 Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 31 Jul 2026 10:17:51 +0200 Subject: [PATCH] =?UTF-8?q?CB-WP-0004=20T02:=20make=20task-done=20?= =?UTF-8?q?=E2=80=94=20close=20a=20task=20on=20measured=20numbers?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the three hand-done steps of a task close (46 turns, $11.52 per CB-RES-0003): the heredoc flipping status in the workplan file, the hand-written hub call, and the hand-typed token counts. The third is the reason this task exists. Every update_task_status this repo produced carried estimated tokens_in/tokens_out — in a project whose central finding is that estimated token counts are worthless. task-done reads the measured figure from the transcripts, or refuses; there is no path through it that emits an estimate. cb-cost gains by_task_detail: cost, response count, model histogram and token components per task. task-done imports cb-cost rather than parsing its printed table, so the hub figure is not a copy that can drift from its source. The positive control found a real defect before the tool ran once. Attribution keyed on a bare T\d\d from the commit subject, so CB-WP-0002 T01, CB-WP-0003 T01 and CB-WP-0004 T01 shared a bucket: the self-test reported $12.10 for "T01" where the qualified figure is $2.33. That 5.2x overstatement would have been pushed to the hub as a *measured* number — the same fiction in a new form. task_label() now keys qualified subjects on the full id and leaves unqualified ones bare rather than retro-assigning them to a workplan. The pinned $93.15 benchmark is unchanged, so historical attribution was not disturbed. Fourth instance of trusted arithmetic: a number believed because a program produced it rather than a hand. Refusals, all exercised by --self-test: unknown id, typo'd id, already-done task, missing state_hub_task_id, no measured spend, and a status flip that produced no change. Co-Authored-By: Claude Opus 5 --- Makefile | 11 +- tools/__pycache__/cb-cost.cpython-312.pyc | Bin 30154 -> 36204 bytes tools/cb-cost.py | 67 ++++- tools/task-done.py | 322 ++++++++++++++++++++++ workplans/CB-WP-0004-mechanical-work.md | 22 ++ 5 files changed, 419 insertions(+), 3 deletions(-) create mode 100644 tools/task-done.py diff --git a/Makefile b/Makefile index 98cf81e..4bddd14 100644 --- a/Makefile +++ b/Makefile @@ -24,7 +24,7 @@ TOOLS := $(REPO)/tools # Every cargo recipe runs at the repo root; the shell does not persist cd. IN_REPO := cd $(REPO) && -.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test loc all +.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done loc all ## fmt + clippy (deny warnings) + HashMap deny-lint check: @@ -61,6 +61,7 @@ self-tests: $(PY) $(TOOLS)/rule-coverage.py --self-test $(PY) $(TOOLS)/dep-weight.py --self-test $(PY) $(TOOLS)/repo.py --self-test + $(PY) $(TOOLS)/task-done.py --self-test # T01 positive control: prove the environment fix, do not assume it. Runs # every tool from a foreign working directory with a PATH that has no @@ -79,6 +80,14 @@ env-test: @$(MAKE) -C $(REPO) coverage >/dev/null \ && echo " [ok ] make -C works from any directory" +# T02: close a task — flip the workplan file, read the *measured* cost +# from the transcripts, push the hub event with real numbers. Refuses on an +# unknown or already-done task, and refuses to report an estimate. +# make task-done T=CB-WP-0004-T02 +task-done: + @test -n "$(T)" || { echo "usage: make task-done T=CB-WP-0004-T02" >&2; exit 2; } + $(PY) $(TOOLS)/task-done.py $(T) $(ARGS) + # CB-01/CB-02: live spend since the last commit. cost-budget: cost-test $(PY) $(TOOLS)/cb-cost.py --budget diff --git a/tools/__pycache__/cb-cost.cpython-312.pyc b/tools/__pycache__/cb-cost.cpython-312.pyc index 2154d3e99d9424db26b64d12df06a6c661f712b9..21da165bb72e3e4fff7ca5ba472e308f61a5d417 100644 GIT binary patch delta 12938 zcma)i3sfA}m0(r%zZ<$~x*KS|1>)D@+eiq60SSS=Adm!-teCpmb?nQ?YJ8&90&#))+ z1qW~!2ql6Oay3w-j#iN zpuBxo)$v1Br-cg7DZc7VdEaT_wBR{C5H+11py|^C{=`^PaXNrFLic_I;0mLBrmjF~ z$u$|LSeYdyGOKtyoJviBS^pvDdz>RCXp^1jWdvO> z z@C5^d0a4%sg2b0U9`9joZS7XRuh#pxhaV830+C-Q#I{x;rJCR6BO;cf^^`vj$R@tJ zx*Ezz5#J;~faDLSUr01S;>U?7_#%PO5FZ&1NPrHV6+^VSLO|d{;Rqk{jfu3j$gmh7 z{JAh09S{0Kye}m1V-pfE08?y0Lt-f{KKklh=eD`RRiTXvfIO{hQyeu z?3jcV@ce_anqlQHjU_SF9Pvq`{dAm(8)yyiFP#D~&D^&#rtDire#ywsWiA^%(`_FZ zj4SDmD`l65a!kvA= z`+UR2h8gd3O^e#%IW2&DP(7oj9PMsXer$N#L$V=DvJm7Y!Cpv6*z;!?Rl7gVk*EB# z@!Ofmr&@wxpU^)}0)9~ugo0=9OzL4Z6f$>QHxGyp+k$&L5 zqP&$M3+=+6gyL| zJjg0!GZk-v!vLbD(&cpJ^$MyqVnP9q4Gj*X_lX38NX&Kead_eYK1h^e2($tDR1T; z%ijVus-{R_Oa$dIHm*X3h)+yFZofcUl-j&AX>_7-juW%1yp;FjTwHS9qzE7^T&}uS zWg}~gR(U?(%l=ULhy2om$fg=VVn-9PFQv#D}!C0eG+(_s#?8WLVJHGIIu2 z&lDS3)9NxiqQO%$l}Yw9fpIEZ&1rqFC1(0d>=l+}E^XDqG4pi|oG2Pqi`tD~C%U1j zss13Ut-;BDYztkZuvCYEs74H?;w$3yJ+F??qjIH6EjCP7HT{0LB~T3IIR_#`!&@7qezMaDHDfD1kbFtvwVb0Z~$o$*MdK z<*)ccwnX_6Uj#e+4xiVI0m?q|Fr4p9^}JqV&3-yRv38lv5YXw{5OF98PXw*?kTC z{#K*OG{fQXL1h(`O|{;|yy+gJNwZwgt{Rn^;=YpnqLp0F^}fYi?;X1-)%shF#bjPR z$a;0=Re@zvY|{sRe?O-D=i<#HAK%LYVISYsLnTrT^y{mxPA~ViJEzN{|DH9gD^35s z9VuNd<2&}Iu59D?J(e!s{I1v0Wz@ZAU;#d)$WVzrIW`9#)q)dhWhY&6&RE6<&P*XI zQImU;@nm;L`7!@^f4?A}ts0&f=(2hEWZTje$u+L}*MjmNGsW84icG@KtZ> znRrnML{!~~6b=Q+AXG+h*F4Df301>kKfit`fdS1)TRw!A zf(L9gGxUx`q;rADFx88yP6Eqz9Dc@wfrv*#!YEJ^2#tfhF+svLhz1`X#bi7QehCTb zEdbNZiovpI^WHFcS8P0_pN%~gTg)uGX((T@Wv|2xmJ4S;`{atvv0|}Oi);I`CGUo1 z(_H#3^X4V<=4Er`bo&PuD{>fIvyLT0?ux}d+q-1p=ep+GrP z?Mrzz^9^tKUvFJ;)U4P_Rt)B6pM2`c#fe|5>&!#F7%xTU|1`UkRffxR*-_5vT<6})WkUghMW&H_B((}Wf$J?9MtH@ zb;u~@s>|#i^2rWqWIkV}vv&(dr_3ae~CL8VP@BcQ!P>lGG2{EO@H zl(^T^4=YhTPd)^OVs_z?>hE%CH?SCrBtJm#u440M#@@r^5&%!CY7|34eDN(K_9g;6 zeaVjzd=0@K1Ro$!5&TaCv^oC^Q~v}2PIwIw2gy$$y>Zl43!E+C;90SMJP;xu0ks*F zr}7o=wWdEZ6pcWo`qfGG>ku==3G9?cW;>WOl?-zlOpPh+l#Z$~XH3?}+6c1KPu3;1 z8faHN%S6xupx>Y-NnsgTJEgf)tp%M-j4~^5R6A-AhFB1#ml+V%24W1dLC_+8N-OJ$ zO$IpH0bMmma8=@Wm;n|Yn%9@y>%x*anS+XQpv}<0DdUuB$}AgYt)Dy3jY2PoUN((F z_a*j{BLqjmr3~3D8L$L=E@3;B-ngRS|oi^zs6M%9AJ6`@N86j)g0h6AdOdT@+CZK zk=z6NA(#c}JCK-WR#O}yFfCMW`xdJ#G&rn+@tHMy zbIPo!4Q0xf#x2VA2CpX-)&(4Yf{9L#P1bcVGwc~$8%EhE7!W(1E?Y+ml-I``%FB%@ zh8)=h6tl9@$U9-6A&E@1c$%CB=kV>u+})6{oQL^Aw2%@7f6A}%b2z_1NkuFaq)XyI ze>>9&Y&vA$u|mMoQqE&sHvNx14fKWyluUm294z@lYpHHd178V3PH1B8DVQaLSp2>qV`Q z@h9ssW>aFOoJD`LUNR)B`Ya2}pcI#YJ$uTH(|!ibcAWH)Ds)g-H#&{sO@Pah-EvNH z96m9b+3OUTDY{$|_E`2qBT)kWT5t)&FWsFZ4Y&w;*P^&WIa1?rgSZ5DQs_gJhqf)$OY`x?&FjT)<1rwWzV3<`H?9>Novsr)Gy3|7^Kwx=fh1NiC<__atk;A=ME z*G38OwX$`?5{I=y^(bgMTU*fd6}s0=NQhmB2+fNfTfMp&yk%cdYK26c^c)_6#ZY9LnRQ(E9(mJptFd#bv2(fc;B9yDEqCRT zyK>oGeMiIEYo`;HA3_OejMFC3s4V#%W)CAcf&e!p4b~Bpa-zB4K+YlJZ3)hb=p2TJAbREx z1Vy1wHG&Zv5a8y(lS`w>*BAttD5d;?P4Hv|tfqM8rxZr|6|y}!M! zi_ScG8ksF)FnImw2!unP0y>HiE((MoyfYXN1XCYF7mwJ|4a1_B8Sw>0uo9ghREJ%+ zY#4Cmqt>Ef6JSXS0=l>_VDtJA97S*pW$S3%TV+Sd0^-1^g%ysKnP534=GllzI>8DS z(c-j&M?Ep1vS+0NKtWt8KPO>~LePM2N>T!~jKRP3K&O0W$3N%ZH#3g>Tj`#qbPoiM zOttqkoWncad0)>s3lw2zi{>7ap?rDgwo@sYv+Qi?Rnt;R(FJzJXu44PoN4Cp%-PE) zmW=t&nGjR)oOQNv$(Zw;6?5emEi-L1LzfT0jl`UH$yf}(wA@8g!p61FZoBHfY22iI zuyY~hu8uQ#S3Maz@BLb3wk;a7GC8HUva6P|tCq8CzL|5&yKl+6?>*PDx97rnFrHI0 z?lKx(CA7t!e#Ld!H6t(Ei*MLBU+2J{HdS6|TS;-v_@3*UZCgsod#-CG#W{2Mxz1Vd z?6IrcZ{$>5*DR$}TQ9$twIU!=^OP6yCO_&&0e}>Mz&N<}N!*Z#XKi z7cJYWzG=Q;YocK^h`psWPj}EC7%Gc(DYb1ba5J@S+1xhWzRFdl=vR!H*Yf5jm)+h~ z4P<{}Ouy}N&w7;p+s>@nET#K4ziF=HRnzT)BG}$cFd$RChG-h)q!azV{*RGJL^^+TXkVRgE3{iAkEe*(bY`MXECO<8P}>C)7gy4 zJ-cZ&2h+KX$u%R+`Y(^J=3zdcF{Qt-b=IJ?tQKIN2j10E1{86YXDQ1QTh6NZU70h- ze5aJLyJpxK?`jDI5LdF?xC&PC@@GGD^|4hmlbti$cs1viyLQQ4J8xfhZ<%jfa&Mb| zV#&Szc6R>k6LVQtKeL=&aXSmIta8Xy=q*1d1ux18U2 zr_i2ey~Eg3t@kw0pVdO_?E#iCIbP_%nON;)5z_@2`wPXej?#-E{F#rqZouVU>su_^ zvYfYdwTJS7BWH4~D8XF?vBMN`3=pmvVf8r8J&NR+eXA#Ej;Ye=KfY^%)$;MZauELU z-CE9NmK?A>-z%-|t=7I@w`)u97V~m?dhd3lANpg8D{BSsi`rz3GIi;MI&es~0B&oH+LUo* z{fpXl_XE82dT@(C5yd}Y8?ZM}8#7tBk>;S3uwBRlD$P@O4&DzSEphIqqw^<}0bkR` z`eB3~apPsvlzD@gW*n7@7TGMc(&~3Wv7KwVHVh*`Z+Ag&+o5+IFX<@%Zs6GS5Pv+k z_Y?g4;{4wILLQVjfF-cOLMKS+N@9eAqo*H|(j6}q>j92KSOz2?hV+p{x);)YiS*G= zbnzJCN%Mvt9*@`nML!q8oO=^aPEj@jRTN8-pdE zQ~qvmn~Q~|5KYD4N%Hc9hX9OyUnZwzmCgGK_G^&A&hvdeeQjN9<%{Xr7`2*td?zp- zmI4?sj+==VC_ydTs22I7nr3euF|*1S_if7iYiMay*IHlg9elu+MDa1hz$A^s!=bI* z*tcaH+P840&5fRwaFr(F!yjY0ECSYvNdDNs<_?7{tdtcnn54{ zFTZ#=wCQO0c(m5a@ErpT6TW8{_K~6ZBLwlRNGAEj{n#WAvF@!{dID~=g-TuuLz&>j zSV-c-z!>z6Km5P}-Zwxrbu1%i%oV~(agcu!8m`sgW3j>}2*B!V0 zXx}-NJr9>^f%EYIO*m!ib{StTh{6N}@NrZG4-#Ri{5wb@3@!|q z;TAq>D&fO}a4#WAF6c!hTrKeA zO~j_Guf0cj^7n|cBXt1pIdEpE)t$mByF0hCElRX=Qw(jgsHS?`Ab9}naKE903xBvF zHn;a2Jk-;@ySuM9{(baCjwdX|4NF+nM8XkLfP84)#iQ8h z2dUgd2#CHRm6IBVRd#|@1CxYK5FL~R*clH(>f=k5JOVtwLcT67OwK}E*KgRE#d=}n zV$F0r+#OtK_^f=}n7U}~zG>{CiH@7b-M8(TvzpnxS5uZZx8JmPfW2j^0lO+U@5|X& zdoGvGXIyB%onoKqn03tR@J-5`V|HlHf4%IL;Jo*h@ch7H$#$@inle^0z-G8SI#+l( zG$Js+Zg~ORk!E$Ncfvb8mQey=7l=wZGNA=-l^?2|}Z`UiLm1fPL($ zaiPBZ91FYg>pgDvrt&{}c=pV~zw~U=#5f39gJ+M<2K|JZ*I;b$6Ke4e@;poO;lNy{ zd)Mf72|CRD2KtsYIxU|(PFaRZ+PZ$`(PZ zo_-MBY!tR9pU+LDeS*@oMzPZdJOV-2=7coxNwz`Av<0(r8XO6&vK=3cC5ymSsz$H{ z)8HA%rgd?0I)suSB2>0R*eN)ZyvgQGIsO!HY=_V$?D`{KCv8%@>=2-XFw$DMNN_@$ z6*`i<(1zW@9%1h_ZVFx+!d&eWH1I}bKdP%_)E`P3p`T@`C}5BBtG@oiLoz&)!C_dB9m;E`Bc8!Z=xBrP$W&&;qkQdXSzfQ~ zhI?f8l2LX;5XS;>M~Ix#=JiF&lwTcnyVt^i>)}GE=O|G5V=nixm$d60To3OG$Cd75 zg(Vfq!4@8s^S}l<0XE1>|RW3Nfo9sB0?~HOW-<*348>6 z@$7)|{S!60c;dKF zlO?gn_O`0pM*fMlXNL5u9N!uWVfvg8{n~&a@;*NaOVZk#KPj5|@KeMleh^MxxR;Jv z*9rI%F`BWKufsQsO>laWv*IM>i@|eEc+6Nw8MD_I=@Z5#et2?x7~Z15ffP-&b0W0f z(!}dOY5h54C2cW%d{~uybQsN{GSLdC^7;K3T*D`h@iwS#4YttX9}W|-2};Lb3cyobr%GT4}tfV}BTB@!^nioKn zJHt=l%^Ocm5f4kB62RM;%{V#75ja?-_)KB!X-KQOb0h$<6q%-liLr5suBv!=26ZGk zfh7GzC>RKh#;?@c5KZqnY3ODzW}iaPkDv}3;o;*!l=l(Q3jl8Q!*Kh5&Ijiwhzy5@ zNbD#u61XS8Q_l!JSsTY5AP7*!2vMTqBV3zw{Y#T7A|%th0&Q8%^kPRaL83J zn`;)gv@e<4Ay{GQWN%xu7Tw-uYu)0uJxkU-cQuIoIU?(qt=krx4lG#@+&5@+UF;p> zhHC1T&0FRtmdwrYT*%T$E2&<#*355Rv^Ly_Tf*(&pPSRBcim2PE&S~9dX8(?DB1pl z+&;L5_m{?cGBs~!xO$42?>gI34`ph;x4Gg_G4pO}!=YBqd#S~TEzA!yO@~XgKP=P& zd_PmyQ)qasyl2nELk!S z7ik`HY>hB%s8beFh5xRph4UzGrEk1d2P*kFfT*>q3MO*Q7ilRO=1ZhL^4@}Gc$|%u zU@5xxa4kXDYf^%jxDl|*N!;N2SFBw_sIUu@+WT&Eau) z8wPJpApH16;FQp5PMA~)i)vs(7&5{G;&CYQUltw&j_TMx{1K;9i)7S;3EY05YSOZb9%!>;JA={q!uBd2P>Ypa4Q0F@rj_=N-h8k{4vsTfM5W# zN+42k_%~*T&HstX@_gpS%;$42=FSMSUDqoX8xAa+I;Zt3se7i)u!UU(GrO)FxO`x- zpn2KZvdGw1%+3o%&u_lCdFJ%8xoFyOS8riUAn=!#uiT3??cU9@w!6o;EH>><6JyN; zyFH78Cx2GWwE1p|oz>r~cCl%9Yaw;FJd<_ZYkHCeJRCv4oO z$Rtbx32C6WU!jG}km;68nobh8)1=dUnI>tcLo-=Q7xxX*$+Z32rp@=&K-w=&mzi^( zEaT+MeBV6q(YfcGyPtFJfA8C$5ikFSG=10QatQFX_9jNv1A_1~EEGQLJagl|vrPv~ zvSoA$e+3!Gf~1j+Ls{2lZb+T>@qd8=eEwNNXyJz>h55BjKm z&@T$Ib+m;?@Ac*!g$4%8sdF$$iv}x1VYfhCCj?q7S3V)l*Q=6!!-Tp|kilwNazYqf zLPOL8r5gAxh2IeTdf~U0mVveoN1hR?~Bs92;*3wE) zhG`Xum(yyXE9erSx5&-3hF>PN|fHTNyU=%Zu6M}B)Vtzx;Kkj@X5CB$pCQ?xv zNwauNR#**4Hq6i>BARW$KF&vrEB%I6~RoDtpZQ&P+8S&2)yT40Zb`UW1 zjIhX-@n4j*ml<_zEUL;%B&xD?pzGl)J;yBAeN5yYU)juyo(&>d#{cYXY-z&=HUrV6 z2vym75wtV=)?3y4C-#{zs$S*3Xg2oCae*##Ixk#9fAEewJ=vAT)LQOb&d zWiB8%GURb2VLLNA#Ff!aYHCbjYf#sYWr@X;P&t)R*)Bt$Nyb$L8_#zSl;vkE?NtEB7obx$iH3FN>1>1YnF=PO1?I9CkgY1LRDln|A)|_5@XeH!RD8W2Wo#< zgWD!l`q6T`PHxZ>(Fo8qH$!5ecjypqJsI-na)yk3fWTf+ie- zCC?I#P$3Au>2djaw!&UINfeO?M}!^1p$gE8n#kvh{JFx=sWb-L#Z1)nDRPbw;dGTr zIKAE^95X*DK?GRXe&|8B#!^WYT$242mTYP&m55{%nc+D47_MJfVuw*jm`-8}hIL{8 z*j-SVszon}Fj)cLR%C_mulEi;4+`aWAiomcoffVO=Ah%v63?tZaNd44@gx7zH~dR4 z9lPRh{hoi_Q+uylK|N;?Jf*Yll7};=GH335Fnh`IJ@@i=tb(g}P6G98k?R`%{z1Wf zbWg_?{&amshgEkl`J)+GQ6q86aM{=(5ZwynP~xd1>%x-5m{cs9NGQ5Z$qYd(B-L;k zFKehJ6?|<&-6*=XZi%MTa+2!ixGIk;x|EQU`3%sH?N+0SZ78b9$>_Lj#IJ6m@tDeR znz{s$rb|j@JnYSzN@<7z%Ml2Iw4$35@uaL1%EJ8lhQLlHV-<%Y%gM-iipmMynwD9F zr6%$NHV?<;1Z5GUOBz?ycrvEyrV)(BF%T+VApE%5D}pa=Z7g%%59J3@@iUP}8eb$U zXX=(7A(ih~1b^_FvG~5=J7!qE8-m&IIO&)x;+|y#byeZV241gfz2>&LihnOT?2fq} zveN9ByNd`#?&+N1{ zz8oe%XWqNzh#1BtH_}Jm`yUq#?=hyl5CCX$7k^-7qxb`XKfQ7}8RJ)1_Eh8C^0n9l zzRunRV$%T;;Czvt$$hyY_8|6wpC-n=+ai9#`3M;Fx9<<4Q^=5pb`} z(5MVs|4*pG&HS+BRs`vOJPw|jr1B)gi1?K-vtjMaV#$}V+Gmk`4$0?v@%ljS3&=hJ zB<#{{;D^dYTpeLgqV&s1{t?MjKyV|xD4)RJ?q$De81oF#nI4B@{WldSL zD)23~YL~f?X0jH@JO6!$&namp&B8pIWz>oQ!6goN%%VK&HP667&~ zib#_Lda-Jl#X6w}n`WaXlxIzvmARo!>lmC<%%NhcG(NB}J~+gve*biC(+GDbhBdQi~INN>;sG5&aS>4JNJf**ad#~ zrX{(%vDI&&1sR7l5QVKXf?rIeba5oEn-kF?IicVg3t4sH%O8*Ss0uqhlu>1N1q{ky zUzG0wGcC*&2_Els*R0buy<;}8wbPP}ie*w30jzTrqWGJ5z8Z z8Kt84m0X1qExu^ZmZBq|WVD_~@2sFOn8tbE^=8X7hvuCGUqAz`Yzjb4Op1x>U0wvpsM zWC6c28$a3=;ot1`^WSxKHYHH`10)#r|M$7m(o;hcJlow~4*(c@)h&Q`<)ktYb{i)N z+lP(dfdN7PbWcalHgqh1uCrM*_u$YXAX~uyx-FA5pf5!L0|+b@PsmiaL0XQ}Of(S= z7}p3RH5*{p03y0Yo&=~y86Nj6g9I|8U|?q>K^TV&8_%II2}Cy!!_7q1t=m%>0C~(v zTLye*D%u=F8_UsX0uU5j5r&lfbS6L?E;nWcz$BgoY&V=o`^KSY&SUTkSec~pG>Rr< zBdf9hL|GBPuQ#;yC&02dkr;c@ja(0s+koh1<65JWG4>G3bR<7T1wg1F1rjx&+`jUP zd>RCZnn>Vrdky4mm;@UozPs-i(pB#g-qqhBT^Bt3RR3CDx#yXpYi7~jI#=T{x4yHK z|9H>TphIvLzv1e<6#u@f^NOQ$di&3Vh&_X?vn6G-?$WEi^7BD{_-dfyd^?|-_4sEi zs%QO`=VNEbu6hIKt-RxES^4>L-gmXse@44f+HkeH_Vv2eSE|>%$=}{rv&kX^Drd{8 zuGN}XNv*i!hD@32exNKcsRoY{(ied6Be30Gp$@J?cL715s_`O)1wW~0v{6|;t zvje^rw`g|Sj0uRgKrUFhP+Em&7ZYhK9GYzx1iJbWvthpmEZW~^pD(Tbfc?6>ed|YP zHPl#NNO5!n^s=!ap_|4IzOR>#e68GOu-Jlipy#c?ch2+MfZsmP@3@(g2wYs~PL#7v zi)OfMQDT|b7cu~}yZaHTP`X(kk~lO|m31s?#-};xt@E8$X%1d9SYOix9d@IP_P~U0 zTinz!A>7NOgQ0Sa)v1zD2>g2Cw1qp_6#w*Kr|-w8bb)-~HWPx7{_S8uyee?pftrf{ zKm$)GG|)fLdD}w$25?~Bd7wP-U!Z1Z(8h$yDQQz{8{`0bcp;N`90gFM09+JMmO_6AE*d?3f&(w+O+WO z(7$+|lb^kFT3~+Oa`3a{QQmj>R`Lmc?(km1`L7SRlTY$cWZm*H?AaZ{TTsZj1%+Vl z>{BQ$x;fL0A$E=*j|9Xw1^$`H8o1lO7OBr+P)I5_FJ?YMsgepW^G-}By~|4 z(Md+ZNC`DAtnvJdfrUW=giI#EI^2V6F$&Q&^VnU#BQtl$j*?0Py+F-{57){-FLm54 zhseH}Rq|RXrwLiNidQ=01KiOA~+%Zwqc*Syr7!2y_H0A;`vx}jhToo=L?Xvak{>ocr3Qwd;&5xRLKYz*B7HGOm| z#>*JAo)qzjrJV)K61ok7XnVngZsSKs$E`c!5VIFWe(6X#7sqzi@6x0r0?z&>{LmI6 z*obG5|73VsskQ)mppRPti8S%gjWv-j{>oS|*1f25MIrp*`2tS257R)iJ&WY!g&H8g z?Ge+>Rk#VcXfMAtQQNox!DydW4QaF=(&!$jb-Pvtn`Y9MFf-lDKc3hwiKl1y_Y=+V z`&Dil?~a2hvRZte@b4s-l%S&;z$1R?J|X}#`F-;Cs&_#M(Y9t7!gX2eNN6IJ zNzl-c9BN=c{$grn6=D~L zYXJ86+QT5t=O0Yie(p+l1icVYZ@mP<_*OWtb&<8@0hWWu(rZW=Kc8;oKltb*aq>R4 zmbCD{VY}eU_zLT?R6tEu$t#rxi_!9azEkNVBm9$!(((uF|u8iej z4MO=mDC$7Yz({u>_eUV%P#(oRj)E@!g6hvL!3LK=88S~R{IxNN0YD)`QL0f0=RYx#&eFbG>D6kfuR(Yg7BeIQft#x;MoV)B{FJ4+c_hjx3u3|9+>Be*j9Z18Q3YNg*I;d27UM;gxv>SJ zTkuf`9w9~JaAbcG`1+G=xu4@`9pG(>9Eq|cim~6ufIc`cW1SF<-&`!#jyOGL6Ob7^-ewZ= zX0{hh;J={s^FKY<7MXJhHpl%F$0zR39nYO%=O)fhoXeffU1DFHcy!{6xkqzPOHqUiPfmDM&AGAwd5ZDuw8x diff --git a/tools/cb-cost.py b/tools/cb-cost.py index dd328c4..e0af2bf 100644 --- a/tools/cb-cost.py +++ b/tools/cb-cost.py @@ -44,7 +44,26 @@ from repo import ROOT as REPO # noqa: E402 (single source of fact, T01) PRICES = os.path.join(REPO, "benchmarks", "baselines", "model-prices.toml") COMPONENTS = ("input", "output", "cache_read", "write_5m", "write_1h") +# CA-08 attribution key. Qualified first: a bare `T\d\d` collides across +# workplans — CB-WP-0002 T01, CB-WP-0003 T01 and CB-WP-0004 T01 all matched +# the same bucket, so a per-task figure summed three unrelated tasks. Found +# by T02's positive control, which reported $12.10 for "T01". +TASK_QUALIFIED_RE = re.compile(r"\b(CB-WP-\d+)[ -](T\d\d)\b") TASK_RE = re.compile(r"\bT\d\d\b") + + +def task_label(subject): + """Attribution label for a commit subject, or None. + + Qualified ids (`CB-WP-0004 T01`) become `CB-WP-0004-T01`. Bare ids + (`T01: ...`) stay bare — commits predating this convention did not name + their workplan and must not be retroactively assigned to one. + """ + m = TASK_QUALIFIED_RE.search(subject) + if m: + return f"{m.group(1)}-{m.group(2)}" + m = TASK_RE.search(subject) + return m.group(0) if m else None UNATTRIBUTED = "UNATTRIBUTED" OPEN_REMAINDER = "OPEN (uncommitted)" @@ -323,8 +342,7 @@ def attribute(responses, commits): label_for = [] prev = "" for ts, subject in commits: - m = TASK_RE.search(subject) - label_for.append((prev, ts, m.group(0) if m else UNATTRIBUTED)) + label_for.append((prev, ts, task_label(subject) or UNATTRIBUTED)) prev = ts last = commits[-1][0] if commits else "" @@ -369,6 +387,13 @@ def collect(slug, pin_ref=None): attribute(responses, commit_index(pin)) by_task = collections.defaultdict(float) + # T02: per-task token detail, so a task close can report *measured* + # tokens. Every update_task_status in this project before T02 carried + # hand-typed estimates — in a repo whose finding is that estimated + # token counts are worthless. + task_detail = collections.defaultdict( + lambda: {"cost": 0.0, "responses": 0, "models": collections.Counter(), + "toks": collections.Counter()}) by_component = collections.Counter() by_component_cost = collections.defaultdict(float) by_model = collections.defaultdict(float) @@ -379,6 +404,11 @@ def collect(slug, pin_ref=None): unpriced.append(r) continue by_task[r["task"]] += r["cost"] + d = task_detail[r["task"]] + d["cost"] += r["cost"] + d["responses"] += 1 + d["models"][r["model"]] += 1 + d["toks"].update(r["toks"]) by_model[r["model"]] += r["cost"] rin, rout = rates_at(prices, r["model"], r["timestamp"]) unit = rin / 1e6 @@ -429,6 +459,27 @@ def collect(slug, pin_ref=None): "subagent_total": sub, "main_total": total - sub, "by_task": dict(by_task), + "by_task_detail": { + k: { + "cost": v["cost"], + "responses": v["responses"], + # The model that ran the most responses for this task. The + # hub schema has one model field per event; a task split + # across models cannot be represented faithfully, so the + # full count travels in the note. + "model": v["models"].most_common(1)[0][0], + "models": dict(v["models"]), + "toks": dict(v["toks"]), + # CA-03/CA-04: the hub has no cache fields. tokens_in is + # the honest sum of everything billed on the input side. + "tokens_in": (v["toks"].get("input", 0) + + v["toks"].get("cache_read", 0) + + v["toks"].get("write_5m", 0) + + v["toks"].get("write_1h", 0)), + "tokens_out": v["toks"].get("output", 0), + } + for k, v in task_detail.items() + }, "by_model": dict(by_model), "tokens": dict(by_component), "cost_by_component": dict(by_component_cost), @@ -566,6 +617,18 @@ def self_test(): finally: os.unlink(partial) + # CA-08 label collision (found by T02's positive control): three + # workplans each had a T01, and a bare-label bucket summed all three + # into one figure — $12.10 for a task that cost $2.33. Qualified + # subjects must produce distinct buckets. + check("CA-08 qualified task ids do not collide across workplans", + task_label("CB-WP-0004 T01: fix env") == "CB-WP-0004-T01" + and task_label("CB-WP-0002 T01: survey") == "CB-WP-0002-T01" + and task_label("CB-WP-0004-T01: hyphenated") == "CB-WP-0004-T01") + check("CA-08 unqualified subjects stay bare, not retro-assigned", + task_label("T01: cost-accounting survey") == "T01" + and task_label("chore: no task here") is None) + # CA-16: a dated promo rate must apply before its expiry and lapse after. pr = prices before = rates_at(pr, "claude-sonnet-5", "2026-07-31T00:00:00Z") diff --git a/tools/task-done.py b/tools/task-done.py new file mode 100644 index 0000000..83508ea --- /dev/null +++ b/tools/task-done.py @@ -0,0 +1,322 @@ +#!/usr/bin/env python3 +"""Close a workplan task: flip the file, read the measured cost, tell the hub. + +CB-WP-0004 T02. Replaces three things that were done by hand on every task +close, at a measured 46 turns and $11.52 per pass (CB-RES-0003 candidates +3 and 5): + + 1. a heredoc doing `s.replace("status: todo", "status: done")` on the + workplan file — which silently did nothing on a typo'd task id or an + already-closed task; + 2. a hand-written `update_task_status` hub call; + 3. **hand-typed `tokens_in` / `tokens_out`**, in a project whose central + finding is that estimated token counts are worthless. Every hub token + event this repo produced before T02 was an estimate. This tool reads + the measured figure from the transcripts via cb-cost, or refuses. + +Ordering matters and is enforced, not assumed. Attribution is by commit +subject (CA-08), so the work commit naming the task must exist *before* +the close: otherwise the spend still sits in OPEN (uncommitted) and there +is nothing measured to report. The usual sequence is + + + git commit -m "CB-WP-0004 T02: ..." # subject names the task + make task-done T=CB-WP-0004-T02 # measures, flips, pushes + git commit -m "chore: mark T02 done (measured: ...)" + +Usage: + python3 tools/task-done.py CB-WP-0004-T02 + python3 tools/task-done.py CB-WP-0004-T02 --dry-run # measure, change nothing + python3 tools/task-done.py CB-WP-0004-T02 --no-hub # file only + python3 tools/task-done.py --self-test +""" +import glob +import json +import os +import re +import subprocess +import sys +import urllib.error +import urllib.request + +from repo import ROOT, enter_root + +HUB = os.environ.get("CUSTODIAN_HUB", "http://127.0.0.1:8000") +AGENT = "custodian" +TASK_ID_RE = re.compile(r"^(?PCB-WP-\d+)-(?P