From 53c460c992c805dce3968d046c749ce91739373b Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 31 Jul 2026 10:24:39 +0200 Subject: [PATCH] =?UTF-8?q?CB-WP-0004=20T04:=20fact=20registry=20and=20mak?= =?UTF-8?q?e=20facts-check=20=E2=80=94=20DFD=20gets=20a=20gate?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Duplicated-fact drift is the fourth error class and the only one with no executable rule. No positive control catches it (both copies are internally consistent) and re-derivation does not either (the copy reproduces whatever it was copied from). It is caught only by reading a copy against its source, which nothing in the loop required. facts.toml holds 15 facts and is GENERATED by `make facts-gen` from cb-cost, dep-weight and rule-coverage. The trap this task named — a hand-maintained registry that becomes another drifting copy — is closed by facts-check re-running the instruments and failing when the committed registry disagrees with them. A stale registry cannot certify stale artifacts. An artifact quoting a fact tags it: **$93.15** with the key. 17 occurrences across 5 artifacts are now checked. Falsified before being believed: changing CostAccounting.md line 158 from $93.15 to $92.87 — the exact historical drift — produced exit 1 naming the file, the line and the expected value. Tested against the class it exists to catch, on a real artifact, not only in its self-test. It then caught a live tag inside its own documentation example in InnerLoop.md within the hour. Third time a gate has failed on its own pass's work. What it does not close is stated rather than implied: 22 untagged literal copies remain and are reported, not failed. Tagging is opt-in, a number can legitimately recur, and a gate that fires on coincidence gets routed around. Naming the uncovered surface beats claiming the class is closed. InnerLoop single-source-of-fact moves from prose to executable — v1.3. Co-Authored-By: Claude Opus 5 --- Makefile | 14 +- decisions/ADR-0004-am4-ratification.md | 4 +- evidence/CB-EV-0001-game-kernel.md | 8 +- evidence/CB-EV-0002-cost-accounting.md | 2 +- facts.toml | 100 +++++ specs/CostAccounting.md | 4 +- specs/InnerLoop.md | 19 +- specs/MetricsAndScenarios.md | 2 +- tools/__pycache__/dep-weight.cpython-312.pyc | Bin 0 -> 9295 bytes .../__pycache__/rule-coverage.cpython-312.pyc | Bin 0 -> 10007 bytes tools/facts.py | 369 ++++++++++++++++++ workplans/CB-WP-0004-mechanical-work.md | 30 +- 12 files changed, 536 insertions(+), 16 deletions(-) create mode 100644 facts.toml create mode 100644 tools/__pycache__/dep-weight.cpython-312.pyc create mode 100644 tools/__pycache__/rule-coverage.cpython-312.pyc create mode 100644 tools/facts.py diff --git a/Makefile b/Makefile index 89a7113..d786a54 100644 --- a/Makefile +++ b/Makefile @@ -24,7 +24,7 @@ TOOLS := $(REPO)/tools # Every cargo recipe runs at the repo root; the shell does not persist cd. IN_REPO := cd $(REPO) && -.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done status loc all +.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin cost-budget cost-mix loop-lint self-tests env-test task-done status facts-check facts-gen loc all ## fmt + clippy (deny warnings) + HashMap deny-lint check: @@ -63,6 +63,7 @@ self-tests: $(PY) $(TOOLS)/repo.py --self-test $(PY) $(TOOLS)/task-done.py --self-test $(PY) $(TOOLS)/status.py --self-test + $(PY) $(TOOLS)/facts.py --self-test # T01 positive control: prove the environment fix, do not assume it. Runs # every tool from a foreign working directory with a PATH that has no @@ -81,6 +82,15 @@ env-test: @$(MAKE) -C $(REPO) coverage >/dev/null \ && echo " [ok ] make -C works from any directory" +# T04: single source of fact (InnerLoop v1.2) — the DFD gate. +# facts.toml is GENERATED; facts-check fails if it disagrees with the +# instruments, or if a tagged artifact disagrees with it. +facts-check: + $(PY) $(TOOLS)/facts.py --check + +facts-gen: + $(PY) $(TOOLS)/facts.py --gen + # T03: one-shot orientation — workplans, next task, spend, fast gates. # Cheap by design: no build. Start a session with this instead of grepping. status: @@ -124,4 +134,4 @@ loc: printf '%-28s %s\n' $$d "$$(find $$d/src -name '*.rs' | xargs cat | grep -vcE '^\s*(//|$$)')"; \ done -all: check test sim coverage dep-weight self-tests env-test loop-lint bench-test +all: check test sim coverage dep-weight self-tests env-test facts-check loop-lint bench-test diff --git a/decisions/ADR-0004-am4-ratification.md b/decisions/ADR-0004-am4-ratification.md index 1223b9a..fde8be8 100644 --- a/decisions/ADR-0004-am4-ratification.md +++ b/decisions/ADR-0004-am4-ratification.md @@ -49,8 +49,8 @@ At the time of the retarget, `make dep-weight`: | configuration | crates | third-party LOC | |---|---|---| -| shipped-runtime (`--no-default-features`) | 23 | **246,250** | -| dev-toolchain (default features) | 29 | **317,021** | +| shipped-runtime (`--no-default-features`) | 23 | **246,250** | +| dev-toolchain (default features) | 29 | **317,021** | | own source | — | 3,443 | ## Why these targets bind on future work rather than merely passing present work diff --git a/evidence/CB-EV-0001-game-kernel.md b/evidence/CB-EV-0001-game-kernel.md index 8bc157f..2b923a8 100644 --- a/evidence/CB-EV-0001-game-kernel.md +++ b/evidence/CB-EV-0001-game-kernel.md @@ -17,8 +17,8 @@ Machine: WSL2, Linux 6.18.33.2-microsoft-standard-WSL2, rustc 1.97.1, | Metric | Target | Measured | Verdict | |---|---|---|---| | AM-1 rule coverage | 100% of GR-rules | 58/58 (100%) | **met** | -| AM-4a dep weight, shipped runtime | ≤250,000 third-party lines | 246,250 (23 crates) | **met** | -| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** | +| AM-4a dep weight, shipped runtime | ≤250,000 third-party lines | 246,250 (23 crates) | **met** | +| AM-4b dep weight, dev toolchain | ≤350,000 third-party lines | 317,021 (29 crates) | **met** | | AM-6 throughput | ≥100,000 events/s | 1,651,400 events/s | **met, 16.5×** | | AM-7 scaling | ≥0.9× at 20× workload | 1.08× | **met** | | AM-7 replay | 100k events ≤5s | 2.18 ms (CI 2.14–2.23) | **met, 2,290×** | @@ -152,8 +152,8 @@ retargeted onto third-party source under audit. | Configuration | Crates | Third-party LOC | Target | Verdict | |---|---|---|---|---| -| Shipped runtime (`--no-default-features`) | 23 | 246,250 | ≤250,000 | **met** | -| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** | +| Shipped runtime (`--no-default-features`) | 23 | 246,250 | ≤250,000 | **met** | +| Dev toolchain (default features) | 29 | 317,021 | ≤350,000 | **met** | | Our own source | — | 3,408 | — | — | Scenario tooling costs **70,771 lines that a shipped game never diff --git a/evidence/CB-EV-0002-cost-accounting.md b/evidence/CB-EV-0002-cost-accounting.md index e193449..4800d4c 100644 --- a/evidence/CB-EV-0002-cost-accounting.md +++ b/evidence/CB-EV-0002-cost-accounting.md @@ -17,7 +17,7 @@ transcribed by hand (CA-15). | ID | Metric | Target | Measured | Verdict | |---|---|---|---|---| -| AC-1 | pinned total, as two components | $93.15 = $92.03 + $1.11 | **$92.03 main + $1.11 subagent = $93.15** | **met** | +| AC-1 | pinned total, as two components | $93.15 = $92.03 + $1.11 | **$92.03 main + $1.11 subagent = $93.15** | **met** | | AC-2 | reconciliation residual | $0.00 | **$0.000000** | **met** | | AC-3 | unattributed share reported | present, 33% | **32.4%, own line** | **met** | | AC-4 | composition reported | 5 components | **5 of 5** | **met** | diff --git a/facts.toml b/facts.toml new file mode 100644 index 0000000..4081f65 --- /dev/null +++ b/facts.toml @@ -0,0 +1,100 @@ +# GENERATED — do not edit. `make facts-gen` rewrites this file. +# +# The single source of fact for numbers that appear in more than +# one artifact (InnerLoop v1.2). Every value here is produced by +# the instrument named in its `by` field, on the current tree. +# `make facts-check` fails if this file disagrees with the +# instruments, or if a tagged artifact disagrees with this file. + +generated = "2026-07-31" +pin = "fc76445" + +[am4a_loc] +value = 246250 +text = "246,250" +fmt = "{:,}" +by = "tools/dep-weight.py" + +[am4a_target] +value = 250000 +text = "250,000" +fmt = "{:,}" +by = "tools/dep-weight.py TARGETS" + +[am4b_loc] +value = 317021 +text = "317,021" +fmt = "{:,}" +by = "tools/dep-weight.py" + +[am4b_target] +value = 350000 +text = "350,000" +fmt = "{:,}" +by = "tools/dep-weight.py TARGETS" + +[gr_covered] +value = 58 +text = "58" +fmt = "{:,}" +by = "tools/rule-coverage.py" + +[gr_linked] +value = 49 +text = "49" +fmt = "{:,}" +by = "tools/rule-coverage.py" + +[gr_rules] +value = 58 +text = "58" +fmt = "{:,}" +by = "tools/rule-coverage.py" + +[gr_scenarios] +value = 21 +text = "21" +fmt = "{:,}" +by = "tools/rule-coverage.py" + +[pinned_main] +value = 92.03371920000004 +text = "$92.03" +fmt = "${:,.2f}" +by = "tools/cb-cost.py --pin fc76445" + +[pinned_mechanical_cost] +value = 33.3506319 +text = "$33.35" +fmt = "${:,.2f}" +by = "tools/cb-cost.py --pin fc76445" + +[pinned_mechanical_share] +value = 36 +text = "36%" +fmt = "{:d}%" +by = "tools/cb-cost.py --pin fc76445" + +[pinned_mechanical_turns] +value = 167 +text = "167" +fmt = "{:,}" +by = "tools/cb-cost.py --pin fc76445" + +[pinned_responses] +value = 346 +text = "346" +fmt = "{:,}" +by = "tools/cb-cost.py --pin fc76445" + +[pinned_subagent] +value = 1.1137235 +text = "$1.11" +fmt = "${:,.2f}" +by = "tools/cb-cost.py --pin fc76445" + +[pinned_total] +value = 93.14744270000004 +text = "$93.15" +fmt = "${:,.2f}" +by = "tools/cb-cost.py --pin fc76445" diff --git a/specs/CostAccounting.md b/specs/CostAccounting.md index aa19d7a..1cb3cfc 100644 --- a/specs/CostAccounting.md +++ b/specs/CostAccounting.md @@ -135,7 +135,7 @@ Each row names the command that produces its number, per InnerLoop §Step 4. | ID | Metric | Target | Instrument | |---|---|---|---| -| **AC-1** | reproduces the pinned CB-WP-0001 total | **$93.15** = $92.03 main + $1.11 subagent | `make cost-pin` | +| **AC-1** | reproduces the pinned CB-WP-0001 total | **$93.15** = $92.03 main + $1.11 subagent | `make cost-pin` | | **AC-2** | reconciliation residual (CA-14) | **$0.00** exactly | same command, `reconciled: ok` line | | **AC-3** | unattributed share reported (CA-10) | present, and **32.4%** on the pinned run | `cb-cost --pin fc76445 --by-task` | | **AC-4** | composition reported (CA-13) | all five components present | `cb-cost --pin fc76445 --composition` | @@ -155,7 +155,7 @@ assertion against a fixture whose expected value is known and fails loudly; Per InnerLoop §Step 4, the acceptance table is checked against the contracts in this same spec: -- AC-1's $93.15 is reachable only if CA-06 holds (both trees enumerated). +- AC-1's $93.15 is reachable only if CA-06 holds (both trees enumerated). Under a main-file-only collector the target is unreachable — this is the defect the adversarial review caught, where a target of $92.21 would have been hit *only* by a broken collector. diff --git a/specs/InnerLoop.md b/specs/InnerLoop.md index 3e1ccee..b3e505e 100644 --- a/specs/InnerLoop.md +++ b/specs/InnerLoop.md @@ -1,7 +1,11 @@ # The Inner Loop — Assimilate and Surpass -Status: **v1.2** — corrected from CB-WP-0003 (loop hardening) on -2026-07-31. Changes from v1.1: single source of fact; review targets the +Status: **v1.3** — corrected from CB-WP-0004 (mechanical work) on +2026-07-31. Change from v1.2: single source of fact is now executable +(`make facts-check`, CB-WP-0004 T04), giving the duplicated-fact-drift +class its first gate. + +v1.2 changes from v1.1: single source of fact; review targets the harness and states its sampling limit; correction vs retarget; the chaos roll's calibration window; the live cost budget. The design goal is now stated: **optimize for cheap correction, not for exhaustive prevention.** @@ -39,7 +43,16 @@ command so re-running is free. > acceptance figure had to be chased across three artifacts each time it > moved. No positive control catches this — both copies are internally > consistent — and re-derivation does not either, because the copy -> reproduces whatever it was copied from.)* The loop's own optimization target is **agentic efficiency**: +> reproduces whatever it was copied from.)* +> +> **Now executable (v1.3, CB-WP-0004 T04):** `make facts-check`. `facts.toml` +> is generated from the instruments, never edited; an artifact quoting a +> registry value tags it `` and the gate fails when the +> two disagree. Untagged literal copies are reported, not failed — that is +> the drift surface still uncovered, and naming it is more useful than +> pretending it is closed. + +The loop's own optimization target is **agentic efficiency**: every artifact it produces must be small enough to load whole, structured enough to act on without interpretation, and falsifiable enough that an agent can judge its own work without a human in the iteration. diff --git a/specs/MetricsAndScenarios.md b/specs/MetricsAndScenarios.md index 9ae2846..9d523f8 100644 --- a/specs/MetricsAndScenarios.md +++ b/specs/MetricsAndScenarios.md @@ -56,7 +56,7 @@ rule 4). a rule a scenario claims should also appear in the aggregate source, or the claim rests on a tag and nothing else. -Measured 2026-07-31: **49 of 58** claimed rules are named in +Measured 2026-07-31: **49 of 58** claimed rules are named in `games/ground/src/lib.rs`. **Unmet**, target 58. The nine unlinked: ```text diff --git a/tools/__pycache__/dep-weight.cpython-312.pyc b/tools/__pycache__/dep-weight.cpython-312.pyc new file mode 100644 index 0000000000000000000000000000000000000000..912597bf8720249c0e4b853e9d854580774f5732 GIT binary patch literal 9295 zcma)Cdu&_Rc|Z5X$K^w!M7?auy0UCZltoF7Y&muvCyFIM;@D1XIZo6D0>yhpk)lZU z+)K&gE*Eu3Q7RE?2}vs&wqY3^UR$eaBJ zWfQIMq9MD;L+B7~5IRLWgk@5>=ooK92-ng`;zGNPK^0@jFT0!nd6&v1C^St^(;jF1 z?-+#o&Uz0V5tOkY7QHbaR;C3}ib)Ysiuk96DJd8jQ#ynR$tTCLBnm!RkfU+zmxMS} z!GbR?29-{pKaPEhB>1E8h#~}K!6(F`Q7rgzRF=JxKPpeliZmg_aP-Qw@GmnT3K!zC zBFM3DP?4b)w3UTX9~LH}SmH;65h*+kL-Iw0NNl15dK7l|^ceLfWnuqw-K0c_`TXO) zfF$o0M#DZuk+3YpWg!?5q7jJ~gIMw_!DvMAMPT4UI69RR;mr+G@!^1^2(fr1D33w^ zxt7Y9Bt#*=ox&@1HbtcSgn$nk5CO6WB#F|b zSBXZ${xKhP2Vbcq@_U=F%_qVF$bKo}!@;QVR^QoIJv{$fR1PY^NwWJ91xLd|+v!L| z!mmc7F=29lXSeXvI~NrxCLHt}5u`~8Pv>?)pCv4I@wD*)?2C=TlETp#kzl|4a4tmH zOIR9tT;D}FKw=yYpHGNHBi@9Bqe2)?OYY!x3Vk`zyuO}(UnCOMC5^(d6v>WZDHg>< zCOHvgr(qoh=?Xc71$nUCu=`|7BD@?7OA!UmqJG%Dx#q!0KyRx=6<=6C%+HoWPeizM z#phQ_jv?q6*>eR4{bazfV=zmyDQ*nOEp{yd84SVC}-fxjm|f!sx;6v_;kG@Oieo;u_jQo(rNEKs|DdL%8Gx9@rBUOx5N@PV&aT+*|z@nlA zWQsibch7b=KeAA{QL`8)=^#onDjljU<{2qtuKnokHxWv*DjTXY%8Id3E6FJ}My|>k za}e$0FmKtdaw1a)#UTDNh^)P%2ykc~B-$1V=d%#S?JY+lZFHvG9~=t)OddXZ#fqHgMs> z`3pz#P(qY1r+J{kuk{U{5(sQYgg@=-)DLPG98_I>h+=}y$74_mAR7)#;)@?q8aoQw z_giwHG*-cql%Tz_gxTwrL_mDalp9jQ;`PR`G#b33u`oDk(8Fj>zb{6xZ#WuPVsQmp zOIH+)^N&gX@!yierBRcZ#0k2s%j3l4Y-(H#2P2Bcz)hu@WCiXvtg!?dG+vI60PgrD zS=JZ;49)yn-|;hjCkOO;&yDy#sF|YKHM>iJQY;NG*egx#gBO`pqp0| z>>CE)hT8f{Xbh3U>5HYYL=Fsxw1hc1LFcKnWG zto&08vf8g&W-arsIomq#zS=q4nYB9SO=FAap{!MpYIbHF zuB@}-sm)|FeQ8H#$G1i|9Eh`QIFX}lW*}Q$x#(PQu9XX$$inSPvDtF>BEP_YU{7)D zHfJjS{*g5G-m#^srElJ@{Z-R})usb07aukq&f4AC+J?06nq_J4GIhJ{PS@?OmFBhG zy{SNoePk<}KQMcg)TwSraVhaV+lC9IZ&V<=Yv$D_Fj3^sLkdZw0%Q$BZYgAi^y04u zN}*+_09(ZxA#*Wpq(sV4QX@{%Dh<%^OMnKZ2$0+LrZU%&imoG(Rw)RXn+16If3{<9 z7C?~z8}xdO0FuZ7=oBD$J1|vQW3AzP15~$zWzk}cECFLma-x;| zGo(^zF3h62DS$r?M*cmO^V1ZZ6bfBLke_AIG}nywBbl0lytfbq|K@gFVctShR5McC z#t8Nx%#iZA52jGA)WauwUL{~4L`QRmZXQJ^!K?-YS2Rp$Vn+gI<&2ijL~Q{Xu@ndb zTugV#*x%LOwMBUobbD8V@wRvAz_rD6Vp*f{$P)sO{5g+JV*}ynh(<>NGh$%D5R{35 zl@pgGz_BS`cwDnU?>Q`DqhR1`Y$ytr0*fV|2slJ6cg=kM;sC}`?6Kh%5R6+P&^Tg6 zfLIb~6x7uXAI%K*s@@Yd4A2EoEo;=MMvsokYJ`N*BX?ZBPz?eNN>`+aLI#~!e2O#>Twehy zQRDrpHP4Hy!i!7Xmkg=!`^RPwyumj-shJb_ zUtct~Y$B?z>9cxagMoNUZimJCsi(bvr{!mr&HZJzpY>WG{qr)ezqah>ZZk>mwDz~! ze%?Ys=!e>ck6ao=_n&OQT7Sp1hyv(6lUSQFjRH-5*P28$V2)UVOvG3+v_v!!dTIOu zlZ+fV{Uaunk)Nb=8CEc~OyjV(5cAhr(RP!*n@1fg*NeS~LPg`w&|yPc|FGl=C4&xv zG4oYvPNjgq3Yn$;6{Hl)_-D(=oUTxK56q{4R$|$^s3;}04LTX)>_!>(J zrGQsrxlt2l;Y!l8CcsqKQXOTPBn#+VtXrY7m=-G|l<0;#po-iHH(8a#oLH@Lkfx!8 zBIsU}snRN&0P{9U&vgKv(a_#>5s5Wm*46p;!j=vAfTsFn00W&6$H34Me89MaVX%|H z!Xu0Y7)OqKiIM`u88``1;z6+f^2Cm?8eozWI@1<_Bmob@z*&H%$0Ish^Nob_XqJxJKgN-)a7c(pFdyvc_HM382e8;A^0A(h-R5k;q>b@^gxBvuF|80dl#H5 z$U~j;P@2gHTs3xWIqOWb>5-+DOpWJLOFJeQkvOo;R^*&FvY@Yb8+cQQ zNgs&&uqcdzlSK9;8o_N4j06=}5pW_XULSB>U)ZZ#jovHMi3E4J6aML!5160E!JHMgrW@DAb~f^oyunRh0XdP%;*P00rKz~7TsAUFfp_aXq@+rW~N zR1wBhs%WMda0bx8Q3S*g0kiK@ixgb_4PX#Zf5A*q9#Ah3NEkOCly%Ay@k<&5^Z~Ny z&Td=*#WJarzx=kWM*~s>E@XTxQNPXZ7D~TG8j@}N4cTM30=iC6*{<%nP}+6l_?lDr z#M!>|`m*cxr8Q?enA}kMpZW>wZnFoN*)EhaiHXzYo; z)2}Azw}m*N;(|`cFb_!&g-?*ks$bO_I|AsCBT5Wxj%FV0yKr(~@SJ`FiJH88~p4vCm-3O)mYOn_R@zn5~hg0-+lco@oj zq|!c6Jy?^Ca#ZG;Ir%%5;`Tk}5yxlvm)7~}OwIH6Ti5uPANW@JlTR6nJ54?1k-au; zUaDNP?VaiWjJ0O?o|VD-eLs2=NXmRu>eT#+48N1qZFua|1s(V+@h6A>=4)%cC)fGP z_xF9m?@8W zg{^slsE!lV7Y=vo^iwm#)xvD+g9uGS0)FwRFt)rd!grtFGNEyJ6eQdY>Z7dgzgbPmRvnXXy=)UiH|5 zT$QQ)8MZD>L7;uyzOt1hE$@xtVu}CR+d5ZL@ z)VH&p59irYgZIXCgMmwg*1a*`Wr6%9t)dxbvjQu`D+RAksJ<}uLPF(BX8--XL19bU zlMd0M+Cq(miiL!-!$^h%BUOwIX}XulqdH&|K^TQob%vUYwTu*ysr=d#>Zj~A@=N+4 zG~$^rM4S-Me5y@#8t5zXMhb{unQB$bLi?Vn4nK``40->sOtcx}(uto{gjX4{#E6|6 z2ZEP|zrO@y$qL4=ovgj+DBcAB%js`Yx{ThG{YEPEoRKQVAk%rT%$SEU8VW_eU6fF2 z6qe+=RI6A4#&czn%7d9&&@9-Imx$(&6GOaz$#Ug@(QiJ-rIw3s-81MKhusEaunJ7Z z>SRSpFEuJH)?R0!W*w<1?tmV58twG{>(vT4{SB&1+y!NgHwz~X?!uR0-H<5gEpt5~ zv8nhg0XImpvZS}&YNal-xpYn{g*!^>>=E~Fdol{{l#<=)HD+9l@3j;S@9Ea!n*3kL z(T}duS6zN40JNXj1$j(z8EacuMX&;}4UFvknj86}%mPj9f}|Ue%%!?vZ@{^x8})Ru zD(NbjU7L<29<@qe&Gy`?-ZcHuIpy6?u;!hx=5E-_Rxs@8i|AR`#5^_k%bfc`cVjR~ zpczxhv+rEOR`Obq^Bzz_f)?7gD36EHJ%U;p*w$Da)}Q}|!RFMgF?c=DjX9$Vr@tZ) z@(&Lanki?GVWNDP*vFVChGvdVMM(REDfd)?39BKNaFDzwDv4_H*jVa2&OL4<_6P#B zG~6ONTi_4E1as_g2R=_aBNrRKM1nUUfXN66=IH+C^~ogYV?7CL&Zh#eUx})m-APDJ z!MQDbJz){@L;;@z5nI&Tm{>rV*ow$@)u|Zoc7Dq)7v*!pQLn{0;Jx;1?#) zfyuoB1yT&d{fRJ9E=({DA0ol?Bsfe0V%34y2%excPK-~)Wc(J%CYPY6S~KNd$29Xq zPzI|wpmAZ}h!oZbDPwYbV37pmM#wvsbYBoSzo)$+Q5axYD-xw}#W*)lhH zxD(m(8sZ3F+Ot%>^v&g(W$Q}C$^bZ)nlgW`||c6P6pb!W?KvsF95 zEOqqGQ`t)Q;*o_TY3b9-J?nK1H%4v+t_5y|uZ5Qnt@N$!>b=jd)x7{7tE%1y71`RI zX?$zy+SK*wTkl+ZXF0yscyP7$V449BSXHlH=`jaV!4qcf1y2~)3zmFc{mhBf;Z?Tk zk-6-{p2g=Eo?kWZ1S@z>yc(JfrK^{zhtBp-E&IN(I5N(bhn7|-%ap^L!2QaV)A!%V zIQt)32G*>4AlFkW=4~+10gL-@P*MN&Vr4gZC#>{p)sD zw!%GsDcv)FW{FyAyKP&3In&s+JPp&$RJE*Dwk|W7iVhgE*PV6Og6UmLULBU|mYX&> z+U5Nn!#bQBR_JlVjU1KpFRt2~GkXrM+IzE3U4XMS({^~(c?2wV$JaI(WOY7;hZH9n z_@($}GW$<{%zf;8 zFrBf#o_`7Y5AqTOf;b-NmaC31jK;JNf-hgEU}u5)RQ3M%lcUgsOla@lyPMB6S^v>2 zoar?EqKP}R&!Jg{hl$b+4|^&!^YE}3^$!o@Zm5f6B+zk)Jc#BoO@|RsOOyjTVpZtq zmIE7NJLWKLn52jzlMcu;8Av7C{}Ix;kFM@pmZKB!9|pqGG5l>PBF3sb4FNb+De5s> zL9tKT5M}=y**-_s&ynkM#Q!^LdBoSwm><=4Woo*!RW)#IZS;&8j%#gWwyHk=bJMwEjx0L^EG)a5nOPym zv&rVlRe?xXg0t?xCplSHR3&`&RdJP6om_QSv2(d{>T*?Cyf{V^{_x%Ts?Pt(l2VoE zpS#yHJF_c-VyCO<>Ftl#uV24@{eJ!URZ$U3Kw5b@GV$RKg7|Oz!$?f7@bHIFSR^=N zh~P+$ijYI3_N9g>?Mn~Q@TDWnlx4_55*#yR1-77@pXB2dera1@DgJ|wpE3b90#8vr_D^Av}IW5IA#l<+9_ zTcH?CD0X}cMZ%&aYGZN}h%IBvdw;A=D-42QCJ#qz>e@R_h=+20plYxKt-B~ z$M$k4CP4jVK8oveON5mVN?ONAcq%NR`o3tC7tY3FaTK14NBAk|9hAbcs6L=A&;d&g zi)akSz|@H-tU?NoiOr}_V&eh`05hm}hHiUfVqj+mI-a+(GGNy;so#aPj0BF+qYZXxILh{kzbA zrEaje*LYBf@}k&;x4sFsejIlJDT-nvRxp#0G#-?4L_`!z2xy!SauHAx)>IJvAb`AH z6hU7)ScKv;(s(R-P-8`G!F$qV^lpyNd}RmJ)Fkqe(I$x(CH4z^BKrfR%3=i*hJ{#6 zl8Y}kTx_U^gZFA@)6k`T7rA`_*_S&e=;EvO%?-~6E?$x=#_&!WXG9c2Es^j@vmky^ z12kn<;rLMQkDFS?V^e&~ zbW9LK<1MgpGfg8gVXOu3{vV56b%GHR`C+^jqTmJ!d*M$6+f1a04X5vh|HjDT#KOew ziseAIye(UNDC<0&VGcip4#X3p!Hd;Ia~cX;i)O+w@py19Nv4qgSRjK$fRqmpU=xaM zBR+ljT+e{C&vHF6-QzVfG3hXnaBfrgHoc zc4CnjBa);-bQ72UjUbX#UR4yz*yssUX@FEN85d5H{?m%YkqRxD@|-Z$%Ai9K@8+nB z?fqs^+?l(#s zqe$Y~M+~8mpjTG8t3Do-#v4&=8t$J)Bm}|Qa{&ZaH+PL2#u)KY&lgS8_Xlh$ z6N~dvl@Y+EYAOuIfDa0xag}^twFcwR$f*{5#m56y)q+wN{R594Kt!cOS2AO6JKkfn&vSXZiR-COp6uv-r_~af+Mr-$lcw)pgv+gWU}6#S<8m6 zWU*?YDzm-gZuc*CeYE?--C1AntaaV)yw1+C^WFFA_RaS!KDY2(n!M4!QhIN1)BLH$ zvkPa_z8mLO+U{GZ()QmGl%wN*5pZ+Am~eSgXa4vQV(Y7!A+L4_^T%!?XKkU~O!)(( z5RzHDmEATD6C7z$WF>4>v}7FDK~Sr$)R`j<<=awGO;gZKT&F_RI1!>Q`LVfBs0nOe zHYH;=IMStmy2oEr*Y+?#ElCYRPHLrTB0%>GWkB;W*%+|vrV*Ps)dAjX&%hvumm4T2*(3-POYYb5>~{Jh~*3lrK^+6FrwHc6&j%FS@;uQf(&jFm-~9( zT;Gjwy7abiM|xYzx*AhG_nh7hPw~~ntZTh?_m5tC^R?yHo8S4g_TXym!IiFT?UCO& zPc7}d+xfHJ_j~`PZ?^M#?_BR+^?mF-m0?c(0Y0J&Xn$B)c9Ld3Y%Yd81i#H>gOa8Q zmto_-Z^QQZ2R07P+C~9XyzWR66rm6tomY^&br~ZtFyJi4rjvSKYe8R|v8|k)bBs~A zzj2-Cdb4?#fODF?0iKL+)!S9jyU6IBx3$K`Z!wQs!YkDh=P^cdPn*YE(A#J9{;GMn zVy+~@BuUA%*OFZ0u{w66+?q!mx-ue6R3Mmwmgyb{maOJNP3e`;* zt2by#_=46_e(X(xt9Xmb@0f)3TuWS~F%H0DRg#{w!JQ2Hc@eZD$s{dFYeC;_aH~+< zcDP@vf#ME@xkP*()(E!?4daUk39c4ynVoW5 zPcSr|^ZhZXi34~;K{Of+M-qm6uleOu2>^8*FYzI%StUYPbB%;e8eZ|kX2|}ATqNe` z34Bj%N(z`Dg(|%5W(WZr{QxO!!yiUvunVP9u}RemQvnaE4G<{YWTJ>&2{_9s{V-cX z<9ujRuEgFpIG3W-d@K?R1ta2#X07E6^cKq@`+bU7w|lO;=iE!o(nz{4QxW*Y-k_Of z8EHsU^DJ{K!|q$b^5vo&auWeO2)ulVpNdN}I<(P1jS20 z#EGGyT!*KCcCbpkfWYyc)Y{|OABJ6$Yje;`2g)MKtt7WLUeZ>bqdCdX z$bRg1HibEJ0_^1jPQ-tJ^E9Vcu5W3<697LQC<0Ca8Vw78fkvZfv_9}!66wvDZ2(kT zkptwfxy*^^R8W|NjmQx$(@CJ$fUJbBpyR0ZC`tvyn9Q_cHTa#$r}Dk&IBtlvnF9mmG_D1Rj`oRF)x zP-L?;1PqAPko_-K=Z~w~->F#kEq5+|d+D2h_fop`=1Z&9?SIf{|7B@8DVL(n%dQs# zasyUE6U-tLrlR`zrpJQ7d6`D&Rhf=WB9%GS*>_f^E}?|(o&gF++7%6lwh>qgI5kkF z-KYA+j4gr6j74H2n!6@o>#s5+fbLWpV?#Ouj-w?mgrgGN@dt1#H9Q7>Avmf+3;YzY z8Bwiq{2;*zSo%UMWFm$TU;%LYhHeEMrqqOPXgq~`rNHulls|0yY4PuvS*5Reo2$gU5{wW z;{VdFbwRfqGcA4DU1th<_;)=bNQ?hJ8ApcQ_ufUI-*o0bLciiyn+DgI^ICPw23wh_ z>dvw~1>+3L`sD3em;G6`^{0bBd2yBPz^om62#w_%&gxqi(kJiqzTKN??8)vvmEF;s zb)HUj-@{XOZm{lI>1xBrZ2PSe(1Dfiwd(e4Rr^MfcfR!ME3?mKD)zy`A6Y0%^CNu~ zhu4^A)*VHulj~^j1A?}ct=g++nc30#(G7diJpJZ{g`VZ|Ov(NgYNc(ZF5_;$SJyay zc&Towa@DtMWfw@ra~Rg_c;+)ZJ3Bh(nxpR9fC|XSS2Ek0VXBr$$aJq4cEe2A6T(-< z!=HmX6oPN#>V&J0luQqJDV;!%f6iE`!XRp_2fT6;T!+2* zs#hqf%4pBaMKSsRI&zVL|0H!Sz!3Sl;1MKuwFA`7yGV>e$xv^$S@6Tcb!&(k!-xSK zu%uP7D%52XkbzyXE>PE;43V^fYm=t|z*>ff5R-;o!aly1@+*+A9n_N1@9V52=_r^d z$E)IaVm7v*hhq~bPR=gX8uKa^V`do~pMrXma~bqF=_+W+&sZpp_JXY|T{Cj$AhQ8Le)w}9XBf}L_3tHTbtQ6>T4!IN8Y(xO5=2+rU2;Yn*aFht*F}YBV8Qp`>0y7$p z@R~E4=-)hv2@K%D6vwbIQrR4O{FC&*;DDrDs)bNB{H7ry9411K$rhPH^|F0W0O0^9 zlG!|QxdIOv(H7Kn0%IKrenuxdki6rGSRde!2m~wjm;rDnpF#*CeLWq8LByl^04H{j zc@#7!fE+hIUpFj{~#8U!>syCvGTF}!F07U6VM2Jc0lE(ovGe!~Uz`j4M z5G6Wb{RriEP7?y4K;Ttdl@?#agSQIAc4z=iuF(RQx##CikJBO+;MP?tK?`T9M^;6dAZ_9 zG#1Gu4>8J!?7~~2=?%|yBruXG6sWm32?*SpA#6>+hFCkkz%FrMhk)-gh_lUzDiajO zE^BZ{2VVG*L%^<}2DF4@xbzLk;6YyEctOxX89XNuTG&W+boTZR^z?R~?-B6(r^-yk z!ch&WsdkKkr7*m}Fu0^zq!gHb6qrPA?`q+0b8Ae_X!oyXY)jaC8^ zhnOWi83)05V6RR&f8uS}oD}g0@S5lKv+K6f zTlA0YZ`xOFXv0%^^|g$B+vl!|TTEuh!E9yQrR1Y9hmk+?$eS-8fZX7uDPYwUr@H?7xocPS+#qZHuw|wsQUFB1~S}cUL z9Gqp~(d)*s+f4fWkJ&q(w>??Znk_l7YCkY%yASB{AaDyH(Q@?+Fu{7~%a$q@YP0N4 zh^B0-{f)bA{+sF6<=&6|2Y>MGjJqw>3)zOJY(W-4aXF3*TL?UoAh7^&EsMcj^4gY0GNao~*kr)%zJ+bc3Gndc!`~ ze5)&Mxp`)ltxI)b%(`c3??OxZ-7=8Rv&9?5W%K+S4fEAYL$EAo0K@Ty&+#~a`c(SJa_f78D<$u~ynF_FKU2O@ zUOWFP?x(|c+>aetV+QZpJRsJLyMN7o4hpmU{NUot3okE?XUq1k+3WAw+*&grf8X`- zx$*hwth@Gh=+4C36L@0|z{Y&;tX_IC>-49(Rh#p-_KJ0HOZmSKXLbR)_-bs`>9X2on75_cE0Prl_;%T>{#en3cc}c`oL;Q z-E8-U%Qt^=URvs!pGe!5Er0J>DZfi+_8z<2^MU_^vJZlp;?rxczI!E=>)wh*_kw## zTJ!GS@Ki1NZ;79H{LA!Ob62(=ht?Kd3$8S?bU8h+Os5meCo?-*KJo6qUj^KLxs7m@ zz)YFq_PYmvcI5pdAM|FAo?UZ2kJS{aOWW2CoyoL4_sheX;sIFEV*Nt>(xEq6;CZ9S z|GBQmc*h(Kp!0UT1+&qm(2vI798XVYYg*Rq`;CqH#s~h7>OZXi<&o^k^BX1Qc(GXW zwfIPsZpZHqyV5?${_r3{5}rf9e{`B8DtA2~ND$t(9dC5Op8IMZi`cyU>tV+^miYz2 zK>qLU`g0!Y*Bmof5Y8bMYeBiDlw>=mj&y%G8ev~2@*R1?8UA4|Y ZH1+$Zzju1>{M8GeFs1itg4w3A_5U>oQ_uha literal 0 HcmV?d00001 diff --git a/tools/facts.py b/tools/facts.py new file mode 100644 index 0000000..00dd687 --- /dev/null +++ b/tools/facts.py @@ -0,0 +1,369 @@ +#!/usr/bin/env python3 +"""The fact registry: a number lives in one place, and copies are checked. + +CB-WP-0004 T04. Duplicated-fact drift (DFD) is the fourth error class on +record and the only one with no executable gate. Two instances: + + * `specs/MetricsAndScenarios.md` §1a inlined a copy of the price sheet; + the real sheet changed and the copy went stale within the hour. + * the acceptance figure moved $92.21 -> $92.87 -> $93.32 -> $93.15 and + each move had to be chased by hand across a survey, a workplan and an + evidence file. + +No positive control catches DFD — both copies are internally consistent — +and re-derivation does not either, because the copy reproduces whatever it +was copied from. It is caught only by reading a copy against its source. +That is what this does. + +## The trap, and how it is avoided + +A hand-maintained registry is just another copy that drifts. So the +registry is **generated from the instruments** (`make facts-gen`), never +edited: every value here is produced by cb-cost, dep-weight or +rule-coverage on the current tree. `--check` re-runs the instruments and +fails if the committed registry disagrees with them, so a stale registry +cannot silently certify stale artifacts. + +## How an artifact quotes a fact + +Tag the occurrence with an HTML comment naming the registry key: + + The benchmark to beat is **$93.15**. + +`make facts-check` re-reads every tagged occurrence and fails when the +text disagrees with the registry. Tagging is opt-in, so the gate also +reports **untagged copies** — literal occurrences of a registry value in +artifacts that did not declare them — which is the drift surface that is +not yet covered. + +Usage: + python3 tools/facts.py --gen # regenerate facts.toml from instruments + python3 tools/facts.py --check # gate: registry vs instruments vs artifacts + python3 tools/facts.py --self-test +""" +import datetime +import glob +import importlib.util +import os +import re +import subprocess +import sys + +from repo import ROOT, enter_root + +REGISTRY = os.path.join(ROOT, "facts.toml") +PIN = "fc76445" # CB-WP-0001 acceptance pin (specs/CostAccounting.md §7) +TAG_RE = re.compile(r"") +# Artifact kinds that quote measured numbers. `history/` is excluded from +# the untagged sweep: it narrates corrections and legitimately states +# superseded values. +ARTIFACT_DIRS = ("specs", "evidence", "research", "decisions", "workplans") + +try: + import tomllib +except ModuleNotFoundError: # pragma: no cover + print("ERROR: needs Python 3.11+ for tomllib", file=sys.stderr) + sys.exit(1) + + +def _load(script): + spec = importlib.util.spec_from_file_location( + script.replace("-", "_").replace(".py", ""), + os.path.join(ROOT, "tools", script)) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +# --------------------------------------------------------------- generate + + +def measure(): + """Every registry value, produced by the instrument that owns it. + + Each entry is (value, format, instrument). The format string is how the + value appears in prose, so the check compares like with like rather + than re-deriving a rendering in two places. + """ + facts = {} + + cb = _load("cb-cost.py") + rep = cb.collect("-home-worsch-clay-borg", PIN) + facts["pinned_total"] = (rep["total"], "${:,.2f}", + f"tools/cb-cost.py --pin {PIN}") + facts["pinned_main"] = (rep["main_total"], "${:,.2f}", + f"tools/cb-cost.py --pin {PIN}") + facts["pinned_subagent"] = (rep["subagent_total"], "${:,.2f}", + f"tools/cb-cost.py --pin {PIN}") + facts["pinned_responses"] = (rep["responses"], "{:,}", + f"tools/cb-cost.py --pin {PIN}") + mix = rep["tool_mix"] + facts["pinned_mechanical_cost"] = (mix["mechanical_cost"], "${:,.2f}", + f"tools/cb-cost.py --pin {PIN}") + facts["pinned_mechanical_turns"] = (mix["mechanical_turns"], "{:,}", + f"tools/cb-cost.py --pin {PIN}") + facts["pinned_mechanical_share"] = ( + round(100 * mix["mechanical_cost"] / rep["total"]), "{:d}%", + f"tools/cb-cost.py --pin {PIN}") + + dw = _load("dep-weight.py") + for label, key in (("shipped-runtime", "am4a_loc"), ("dev-toolchain", "am4b_loc")): + found = dw.crates(dw.CONFIGS[label]) + total = sum(dw.source_lines(n, v) for n, v in sorted(found.items())) + facts[key] = (total, "{:,}", "tools/dep-weight.py") + facts["am4a_target"] = (dw.TARGETS["shipped-runtime"], "{:,}", + "tools/dep-weight.py TARGETS") + facts["am4b_target"] = (dw.TARGETS["dev-toolchain"], "{:,}", + "tools/dep-weight.py TARGETS") + + rc = _load("rule-coverage.py") + rules = rc.parse_rules(open(os.path.join(ROOT, "specs/GroundRules.md")).read()) + paths = sorted(glob.glob(os.path.join(ROOT, "scenarios/ground/*.yaml"))) + covered = set() + for p in paths: + covered |= rc.parse_covers(open(p).read()) + code_ids = rc.parse_code_ids(open(os.path.join(ROOT, rc.AGGREGATE)).read()) + hit = set(rules) & covered + facts["gr_rules"] = (len(rules), "{:,}", "tools/rule-coverage.py") + facts["gr_covered"] = (len(hit), "{:,}", "tools/rule-coverage.py") + facts["gr_linked"] = (len(hit & code_ids), "{:,}", "tools/rule-coverage.py") + facts["gr_scenarios"] = (len(paths), "{:,}", "tools/rule-coverage.py") + return facts + + +def render(value, fmt): + return fmt.format(value) + + +def generate(): + facts = measure() + lines = [ + "# GENERATED — do not edit. `make facts-gen` rewrites this file.", + "#", + "# The single source of fact for numbers that appear in more than", + "# one artifact (InnerLoop v1.2). Every value here is produced by", + "# the instrument named in its `by` field, on the current tree.", + "# `make facts-check` fails if this file disagrees with the", + "# instruments, or if a tagged artifact disagrees with this file.", + "", + f'generated = "{datetime.date.today().isoformat()}"', + f'pin = "{PIN}"', + "", + ] + for key in sorted(facts): + value, fmt, by = facts[key] + lines += [ + f"[{key}]", + f"value = {value!r}", + f'text = "{render(value, fmt)}"', + f'fmt = "{fmt}"', + f'by = "{by}"', + "", + ] + with open(REGISTRY, "w") as fh: + fh.write("\n".join(lines)) + print(f"wrote {os.path.relpath(REGISTRY, ROOT)} — {len(facts)} facts") + for key in sorted(facts): + print(f" {key:<26} {facts[key][1].format(facts[key][0]):>12}" + f" {facts[key][2]}") + return 0 + + +# ------------------------------------------------------------------ check + + +def load_registry(): + if not os.path.isfile(REGISTRY): + return None + with open(REGISTRY, "rb") as fh: + return tomllib.load(fh) + + +def artifacts(): + for d in ARTIFACT_DIRS: + base = os.path.join(ROOT, d) + if not os.path.isdir(base): + continue + for path in sorted(glob.glob(os.path.join(base, "**", "*.md"), + recursive=True)): + yield os.path.relpath(path, ROOT) + + +def tagged_occurrences(text): + """(key, line_no, the text of that line) for each fact tag.""" + out = [] + for i, line in enumerate(text.splitlines(), 1): + for m in TAG_RE.finditer(line): + out.append((m.group(1), i, line)) + return out + + +def check(): + reg = load_registry() + if reg is None: + print(f"ERROR — {os.path.relpath(REGISTRY, ROOT)} missing; " + f"run `make facts-gen`", file=sys.stderr) + return 1 + keys = {k: v for k, v in reg.items() if isinstance(v, dict)} + + findings = [] + + # 1. The registry must still agree with the instruments. Without this + # a stale registry would happily certify stale artifacts — the + # trap this task was warned about. + live = measure() + for key, (value, fmt, _by) in sorted(live.items()): + if key not in keys: + findings.append(f"registry missing {key} (instrument says " + f"{render(value, fmt)}) — run `make facts-gen`") + continue + if render(value, fmt) != keys[key]["text"]: + findings.append( + f"registry stale: {key} = {keys[key]['text']} but " + f"{keys[key]['by']} now measures {render(value, fmt)} " + f"— run `make facts-gen`") + for key in sorted(set(keys) - set(live)): + findings.append(f"registry has {key}, no instrument produces it " + f"— hand-edited?") + + # 2. Every tagged occurrence must state the registry value. + tagged = 0 + for rel in artifacts(): + text = open(os.path.join(ROOT, rel)).read() + for key, lineno, line in tagged_occurrences(text): + tagged += 1 + if key not in keys: + findings.append(f"{rel}:{lineno} tags unknown fact {key!r}") + continue + if keys[key]["text"] not in line: + findings.append( + f"{rel}:{lineno} claims fact:{key} but does not state " + f"{keys[key]['text']} | {line.strip()[:70]}") + + # 3. Positive control. A gate that checked nothing would pass silently + # — the harness-does-nothing class, in the tool meant to close DFD. + if tagged == 0: + print("ERROR — no fact tags found in any artifact; the check verified " + "nothing. Tag at least one occurrence, or delete this gate.", + file=sys.stderr) + return 1 + + # 4. Report untagged copies: the drift surface still uncovered. This + # reports and does not fail — a number can legitimately recur (a + # round figure, a year), and failing on that would make the gate + # something people route around. + untagged = {} + for rel in artifacts(): + text = open(os.path.join(ROOT, rel)).read() + declared = {k for k, _, _ in tagged_occurrences(text)} + for i, line in enumerate(text.splitlines(), 1): + if TAG_RE.search(line): + continue + for key, spec in keys.items(): + if key in declared: + continue + # Only distinctive values: a bare "58" is everywhere. + if len(spec["text"]) < 5: + continue + if spec["text"] in line: + untagged.setdefault(key, []).append(f"{rel}:{i}") + + print("facts-check — single source of fact (DFD gate)") + print(f" registry {len(keys)} facts, generated {reg.get('generated')}, " + f"pin {reg.get('pin')}") + print(f" tagged {tagged} occurrence(s) checked against the registry") + if untagged: + n = sum(len(v) for v in untagged.values()) + print(f" untagged {n} literal copy/copies, not covered by the gate:") + for key in sorted(untagged): + print(f" {key:<26} {' '.join(untagged[key][:6])}" + + (" …" if len(untagged[key]) > 6 else "")) + print(" NOTE: reported, not failed — tag them to bring them under " + "the gate") + + if findings: + print(f"\n{len(findings)} finding(s):", file=sys.stderr) + for f in findings: + print(f" [facts] {f}", file=sys.stderr) + return 1 + print(" no findings") + return 0 + + +# --------------------------------------------------------------- self-test + + +def self_test(): + """Each assertion pins a failure this gate must detect. + + The class being closed is DFD, so the assertions are about *copies*: + a copy that disagrees must fail, a copy that agrees must pass, and a + run that checked no copies at all must not report success. + """ + results = [] + + def check_(name, ok, detail=""): + results.append((name, ok, detail)) + + check_("tag is recognised in prose", + tagged_occurrences("cost was **$93.15** ") + == [("pinned_total", 1, "cost was **$93.15** ")]) + check_("untagged prose yields no occurrence", + tagged_occurrences("cost was $93.15") == []) + check_("tag with spaces is recognised", + [k for k, _, _ in tagged_occurrences("x ")] + == ["am4a_loc"]) + check_("a malformed tag is not silently accepted", + tagged_occurrences("") == []) + + # The DFD detection itself: agreeing and disagreeing copies. + spec = {"text": "$93.15"} + agree = "the benchmark is **$93.15** " + drift = "the benchmark is **$92.87** " + check_("an agreeing copy passes", spec["text"] in agree) + check_("a drifted copy is detected", spec["text"] not in drift, + "this is the whole class: both lines are internally consistent") + + # Registry must exist, be complete, and match the instruments. + reg = load_registry() + check_("registry exists", reg is not None) + if reg: + keys = {k: v for k, v in reg.items() if isinstance(v, dict)} + check_("registry is non-empty", len(keys) >= 8, f"{len(keys)} facts") + check_("every fact records the instrument that produced it", + all(v.get("by") for v in keys.values())) + check_("every fact records both a value and its rendered text", + all("value" in v and v.get("text") for v in keys.values())) + check_("registry is generated, not hand-written", + open(REGISTRY).read().startswith("# GENERATED")) + + # At least one artifact must actually be tagged, or the gate is inert. + total = sum(len(tagged_occurrences(open(os.path.join(ROOT, r)).read())) + for r in artifacts()) + check_("at least one artifact is tagged (gate is not inert)", total > 0, + f"{total} tagged occurrence(s)") + + print("facts self-test (positive control)") + ok = True + for name, passed, det in results: + print(f" [{'ok ' if passed else 'FAIL'}] {name}" + + (f" — {det}" if det else "")) + ok &= passed + return 0 if ok else 1 + + +def main(): + enter_root() + if "--self-test" in sys.argv: + return self_test() + if "--gen" in sys.argv: + return generate() + return check() + + +if __name__ == "__main__": + try: + sys.exit(main()) + except subprocess.CalledProcessError as e: + print(f"ERROR — {e}", file=sys.stderr) + sys.exit(1) diff --git a/workplans/CB-WP-0004-mechanical-work.md b/workplans/CB-WP-0004-mechanical-work.md index affa88a..03a9eac 100644 --- a/workplans/CB-WP-0004-mechanical-work.md +++ b/workplans/CB-WP-0004-mechanical-work.md @@ -136,7 +136,7 @@ believed because it was produced by a program rather than by hand. ```task id: CB-WP-0004-T03 -status: todo +status: done priority: medium state_hub_task_id: "5466a510-37a5-4491-b93e-509cd400cc23" ``` @@ -207,6 +207,34 @@ and say so — a gate with no generator still closes the class. **Predicted:** **$6–9** recovered, plus DFD's first executable gate. Confidence medium; this is the hardest task here and the most valuable. +**Delivered — both halves, not just the check.** `facts.toml` holds 15 +facts and is **generated** by `make facts-gen` from cb-cost, dep-weight +and rule-coverage; the file opens with `# GENERATED — do not edit` and +the self-test asserts that line is still there. The trap named in this +task — a hand-maintained registry that becomes another drifting copy — +is closed by `facts-check` re-running the instruments and failing if the +committed registry disagrees with them. A stale registry cannot certify +stale artifacts. + +An artifact quoting a fact tags it: `**$93.15** `. +17 occurrences across 5 artifacts are now under the gate. + +**Falsified before being believed.** Changing `specs/CostAccounting.md` +line 158 from $93.15 to $92.87 — the exact historical drift — produced +exit 1 and `specs/CostAccounting.md:158 claims fact:pinned_total but does +not state $93.15`. The gate was tested against the class it exists to +catch, on a real artifact, not only in its self-test. + +**What it does not close, stated rather than implied.** 22 untagged +literal copies remain, across `specs/InnerLoop.md`, `specs/GameKernel.md`, +`research/CB-RES-0002` and the older workplans. They are **reported, not +failed**: tagging is opt-in, a number can legitimately recur, and a gate +that fires on coincidence gets routed around. Naming the uncovered +surface is more useful than claiming the class is closed. + +InnerLoop's single-source-of-fact rule moves from prose to executable — +**v1.3**. + ## Phase C — Prove it, or withdraw the claim ## Task: Control loop — measure recovery and test for relocation