CB-WP-0022-T01: survey — how rule systems record the ambiguity they find

CB-RES-0007 plus a runnable baseline harness. Tier L invokes the
runnable-baseline option; the external candidates are practices rather
than software, so their rows are directional and cap at parity, and the
row that CAN be run is our own.

Measured: 6 findings across 11 files with no index, 2 of 6 (33%) with a
runnable reproduction, U1-U10 raised 2026-07-30 and first READ
2026-08-03 -- 4 days, 0 of 10 ruled.

The uncomfortable number is stated before the review can find it: the
proposed 'no finding without its reproduction' rule would reject four of
our six existing findings. The survey answers rather than routes around
it -- none of the four is expensive to reproduce, so 33% is evidence
nobody was ever asked for one.

Magic corrected an assumption this pass was about to build on. Rulings
are NOT authoritative -- they are 'reminder information with no actual
weight or rules meaning' -- and the authoritative fix folds into the
Oracle card text. So a finding closes when the SOURCE changes, not when
an annotation is added, and the register must be a queue that empties
rather than an archive that grows. That is now a constraint on the ADR's
lifecycle.

Model checkers supply the reproduction rule independently: a
counterexample trace IS the finding. W3C's implementation-defined mark is
the machinery we already have in provisional: scenarios and must reuse.

The loop-lint gate caught the new tool with no --self-test; it has one,
pinning the 2-of-6 baseline so a later edit cannot move it silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

tools/design-baseline.py

@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+"""Baseline harness: how findable, reproducible and answered are the
+design findings this project has already produced?
+
+The comparator is US, today. The external candidates (rulings databases,
+model-checker traces, W3C provisional marks) are practices rather than
+runnable software, so per InnerLoop Step 1 their rows are DIRECTIONAL and
+cap at `parity`. This is the row that can be measured.
+"""
+import os, re, subprocess, sys, datetime
+
+ROOT = "/home/worsch/clay-borg"
+os.chdir(ROOT)
+
+# The findings this project has actually produced, and where each lives.
+FINDINGS = {
+    "U1..U10 underdetermined points": ["specs/GroundRules.md"],
+    "SOLVE on a face-down Problem": ["workplans/CB-WP-0018-the-browser-is-a-client.md",
+                                     "evidence/CB-EV-0016-the-browser-is-a-client.md"],
+    "GR-A13 wasted SOLVE": ["evidence/CB-EV-0007-stage-0.md"],
+    "GR-E01 unreachable below 5 seats": ["evidence/CB-EV-0007-stage-0.md",
+                                         "scenarios/ground/gr-e01-threshold-unreachable-2p.yaml",
+                                         "workplans/CB-WP-0021-import-the-edition.md"],
+    "six provisional defaults": sorted(
+        os.path.join("scenarios/ground", f)
+        for f in os.listdir("scenarios/ground")
+        if f.endswith(".yaml")
+        and "provisional: true" in open(os.path.join("scenarios/ground", f)).read()),
+    "GR-E03/GR-E04 never played": ["evidence/CB-EV-0007-stage-0.md"],
+}
+
+def has_reproduction(paths):
+    """A runnable thing: a scenario file, or a named test/command."""
+    for p in paths:
+        if p.startswith("scenarios/"):
+            return True
+    return False
+
+def self_test():
+    """The control that matters: a harness that read nothing must not
+    report a clean baseline. Every path this survey cites must exist, and
+    the reproduction test must be able to say NO — one that answered yes
+    for everything would report 100% and look excellent."""
+    results = []
+
+    def check(name, ok, detail=""):
+        results.append((name, ok, detail))
+
+    missing = [p for paths in FINDINGS.values() for p in paths
+               if not os.path.exists(p)]
+    check("every cited location exists", not missing, ", ".join(missing[:3]))
+    check("the finding set is not empty", len(FINDINGS) >= 6, f"{len(FINDINGS)}")
+    check("reproduction detection can say NO",
+          not has_reproduction(["evidence/CB-EV-0007-stage-0.md"]),
+          "a detector that always says yes would report 100%")
+    check("reproduction detection can say YES",
+          has_reproduction(["scenarios/ground/gr-e01-threshold-unreachable-2p.yaml"]))
+    # The number this survey turns on, pinned so a later edit cannot move
+    # it silently: 2 of 6 today.
+    repro_now = sum(has_reproduction(v) for v in FINDINGS.values())
+    check("the measured baseline is 2 of 6", repro_now == 2, f"{repro_now}/6")
+
+    print("design-baseline self-test (positive control)")
+    ok = True
+    for name, passed, det in results:
+        print(f"  [{'ok ' if passed else 'FAIL'}] {name}" + (f"  — {det}" if det else ""))
+        ok &= passed
+    return 0 if ok else 1
+
+
+if "--self-test" in sys.argv:
+    raise SystemExit(self_test())
+
+print("BASELINE — design findings as they stand, 2026-08-03\n")
+places = set()
+repro = 0
+for name, paths in FINDINGS.items():
+    places.update(paths)
+    r = has_reproduction(paths)
+    repro += r
+    print(f"  {'repro' if r else '  -  '}  {len(paths)} location(s)  {name}")
+
+n = len(FINDINGS)
+print(f"\n  findings                      {n}")
+print(f"  with a runnable reproduction  {repro}/{n} = {100*repro//n}%")
+print(f"  distinct files holding them   {len(places)}")
+print(f"  single register?              NO — {len(places)} files, no index")
+
+# Time from raised to READ, for the one finding with a timestamp trail.
+raised = datetime.date(2026, 7, 30)   # hub message from clay-borg-custodian
+read   = datetime.date(2026, 8, 3)    # marked read this session
+print(f"\n  U1..U10: raised {raised}, first READ {read} — {(read-raised).days} days")
+print(f"  U1..U10: answered? NO — {(read-raised).days}+ days open, 0 of 10 ruled")