tegwick
a86efba4c3
CB-RES-0007 plus a runnable baseline harness. Tier L invokes the runnable-baseline option; the external candidates are practices rather than software, so their rows are directional and cap at parity, and the row that CAN be run is our own. Measured: 6 findings across 11 files with no index, 2 of 6 (33%) with a runnable reproduction, U1-U10 raised 2026-07-30 and first READ 2026-08-03 -- 4 days, 0 of 10 ruled. The uncomfortable number is stated before the review can find it: the proposed 'no finding without its reproduction' rule would reject four of our six existing findings. The survey answers rather than routes around it -- none of the four is expensive to reproduce, so 33% is evidence nobody was ever asked for one. Magic corrected an assumption this pass was about to build on. Rulings are NOT authoritative -- they are 'reminder information with no actual weight or rules meaning' -- and the authoritative fix folds into the Oracle card text. So a finding closes when the SOURCE changes, not when an annotation is added, and the register must be a queue that empties rather than an archive that grows. That is now a constraint on the ADR's lifecycle. Model checkers supply the reproduction rule independently: a counterexample trace IS the finding. W3C's implementation-defined mark is the machinery we already have in provisional: scenarios and must reuse. The loop-lint gate caught the new tool with no --self-test; it has one, pinning the 2-of-6 baseline so a later edit cannot move it silently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
93 lines
4.1 KiB
Python
Executable file
93 lines
4.1 KiB
Python
Executable file
#!/usr/bin/env python3
|
|
"""Baseline harness: how findable, reproducible and answered are the
|
|
design findings this project has already produced?
|
|
|
|
The comparator is US, today. The external candidates (rulings databases,
|
|
model-checker traces, W3C provisional marks) are practices rather than
|
|
runnable software, so per InnerLoop Step 1 their rows are DIRECTIONAL and
|
|
cap at `parity`. This is the row that can be measured.
|
|
"""
|
|
import os, re, subprocess, sys, datetime
|
|
|
|
ROOT = "/home/worsch/clay-borg"
|
|
os.chdir(ROOT)
|
|
|
|
# The findings this project has actually produced, and where each lives.
|
|
FINDINGS = {
|
|
"U1..U10 underdetermined points": ["specs/GroundRules.md"],
|
|
"SOLVE on a face-down Problem": ["workplans/CB-WP-0018-the-browser-is-a-client.md",
|
|
"evidence/CB-EV-0016-the-browser-is-a-client.md"],
|
|
"GR-A13 wasted SOLVE": ["evidence/CB-EV-0007-stage-0.md"],
|
|
"GR-E01 unreachable below 5 seats": ["evidence/CB-EV-0007-stage-0.md",
|
|
"scenarios/ground/gr-e01-threshold-unreachable-2p.yaml",
|
|
"workplans/CB-WP-0021-import-the-edition.md"],
|
|
"six provisional defaults": sorted(
|
|
os.path.join("scenarios/ground", f)
|
|
for f in os.listdir("scenarios/ground")
|
|
if f.endswith(".yaml")
|
|
and "provisional: true" in open(os.path.join("scenarios/ground", f)).read()),
|
|
"GR-E03/GR-E04 never played": ["evidence/CB-EV-0007-stage-0.md"],
|
|
}
|
|
|
|
def has_reproduction(paths):
|
|
"""A runnable thing: a scenario file, or a named test/command."""
|
|
for p in paths:
|
|
if p.startswith("scenarios/"):
|
|
return True
|
|
return False
|
|
|
|
def self_test():
|
|
"""The control that matters: a harness that read nothing must not
|
|
report a clean baseline. Every path this survey cites must exist, and
|
|
the reproduction test must be able to say NO — one that answered yes
|
|
for everything would report 100% and look excellent."""
|
|
results = []
|
|
|
|
def check(name, ok, detail=""):
|
|
results.append((name, ok, detail))
|
|
|
|
missing = [p for paths in FINDINGS.values() for p in paths
|
|
if not os.path.exists(p)]
|
|
check("every cited location exists", not missing, ", ".join(missing[:3]))
|
|
check("the finding set is not empty", len(FINDINGS) >= 6, f"{len(FINDINGS)}")
|
|
check("reproduction detection can say NO",
|
|
not has_reproduction(["evidence/CB-EV-0007-stage-0.md"]),
|
|
"a detector that always says yes would report 100%")
|
|
check("reproduction detection can say YES",
|
|
has_reproduction(["scenarios/ground/gr-e01-threshold-unreachable-2p.yaml"]))
|
|
# The number this survey turns on, pinned so a later edit cannot move
|
|
# it silently: 2 of 6 today.
|
|
repro_now = sum(has_reproduction(v) for v in FINDINGS.values())
|
|
check("the measured baseline is 2 of 6", repro_now == 2, f"{repro_now}/6")
|
|
|
|
print("design-baseline self-test (positive control)")
|
|
ok = True
|
|
for name, passed, det in results:
|
|
print(f" [{'ok ' if passed else 'FAIL'}] {name}" + (f" — {det}" if det else ""))
|
|
ok &= passed
|
|
return 0 if ok else 1
|
|
|
|
|
|
if "--self-test" in sys.argv:
|
|
raise SystemExit(self_test())
|
|
|
|
print("BASELINE — design findings as they stand, 2026-08-03\n")
|
|
places = set()
|
|
repro = 0
|
|
for name, paths in FINDINGS.items():
|
|
places.update(paths)
|
|
r = has_reproduction(paths)
|
|
repro += r
|
|
print(f" {'repro' if r else ' - '} {len(paths)} location(s) {name}")
|
|
|
|
n = len(FINDINGS)
|
|
print(f"\n findings {n}")
|
|
print(f" with a runnable reproduction {repro}/{n} = {100*repro//n}%")
|
|
print(f" distinct files holding them {len(places)}")
|
|
print(f" single register? NO — {len(places)} files, no index")
|
|
|
|
# Time from raised to READ, for the one finding with a timestamp trail.
|
|
raised = datetime.date(2026, 7, 30) # hub message from clay-borg-custodian
|
|
read = datetime.date(2026, 8, 3) # marked read this session
|
|
print(f"\n U1..U10: raised {raised}, first READ {read} — {(read-raised).days} days")
|
|
print(f" U1..U10: answered? NO — {(read-raised).days}+ days open, 0 of 10 ruled")
|