coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CB-WP-0022-T01: survey — how rule systems record the ambiguity they find

Browse source 
CB-RES-0007 plus a runnable baseline harness. Tier L invokes the
runnable-baseline option; the external candidates are practices rather
than software, so their rows are directional and cap at parity, and the
row that CAN be run is our own.

Measured: 6 findings across 11 files with no index, 2 of 6 (33%) with a
runnable reproduction, U1-U10 raised 2026-07-30 and first READ
2026-08-03 -- 4 days, 0 of 10 ruled.

The uncomfortable number is stated before the review can find it: the
proposed 'no finding without its reproduction' rule would reject four of
our six existing findings. The survey answers rather than routes around
it -- none of the four is expensive to reproduce, so 33% is evidence
nobody was ever asked for one.

Magic corrected an assumption this pass was about to build on. Rulings
are NOT authoritative -- they are 'reminder information with no actual
weight or rules meaning' -- and the authoritative fix folds into the
Oracle card text. So a finding closes when the SOURCE changes, not when
an annotation is added, and the register must be a queue that empties
rather than an archive that grows. That is now a constraint on the ADR's
lifecycle.

Model checkers supply the reproduction rule independently: a
counterexample trace IS the finding. W3C's implementation-defined mark is
the machinery we already have in provisional: scenarios and must reuse.

The loop-lint gate caught the new tool with no --self-test; it has one,
pinning the 2-of-6 baseline so a later edit cannot move it silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-03 21:19:12 +02:00
parent bb426b3e9b
commit a86efba4c3
4 changed files with 283 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

31
workplans/CB-WP-0022-the-design-instrument.md
View file

@ -2,7 +2,7 @@
id: CB-WP-0022
kind: product
title: "The design instrument: findings about the game, with their reproductions"
status: ready
status: active
---
# Purpose
@ -88,7 +88,7 @@ tier L exists to surface.
```task
id: CB-WP-0022-T01
status: todo
status: done
priority: high
```
@ -114,6 +114,33 @@ found*? Candidates worth a benchmark-to-beat:
Name, per dimension, the property to beat — findability, reproducibility,
and whether a ruling can *close* a finding mechanically.
**Done 2026-08-03.**
[CB-RES-0007](../research/CB-RES-0007-design-instrument.md), with a
runnable baseline (`tools/design-baseline.py`).
**The baseline is us, and it is measured**: 6 findings across **11 files**
with no index, **2 of 6 (33%) with a runnable reproduction**, and U1–U10
raised 2026-07-30, first *read* 2026-08-03 — **4 days**, 0 of 10 ruled.
**The uncomfortable number is stated up front rather than left for the
review to find: the reproduction rule would reject four of our six
existing findings.** The survey answers it — none of the four is
expensive to reproduce, so the 33% is evidence that nobody was ever asked
for one, not that the rule is unaffordable.
**Magic corrected an assumption this pass was about to build on.** I
expected a ruling to be the authoritative resolution. It is not: rulings
are *"reminder information with no actual weight or rules meaning"*, and
the authoritative fix folds into the **Oracle** card text. **A finding
closes when the source changes, not when an annotation is added** — so
the register must be a queue that empties, not an archive that grows.
That is now a constraint on T03's lifecycle.
Model checkers supplied the reproduction rule independently (a
counterexample trace *is* the finding), and W3C's *implementation-defined*
mark is the one piece of machinery we already have and must reuse rather
than duplicate.
## Task: adversarial review
```task