CB-REV-0003: round 3, and three of four FATAL came from round 2's fixes
Some checks failed
ci / check (push) Failing after 3s
Some checks failed
ci / check (push) Failing after 3s
The pattern is now measured over three rounds: 5 fatal, then 3 (2 from the previous round's corrections), then 4 (3 from them). The corrections are not getting safer. FATAL 1: round 2's short-cell assertion went into regulation.rs only. attack-value.rs — which produced every number in CB-EV-0030's DARVO table — still just warned, and the gate registered to close the finding claimed the property for both. FATAL 2, the sharpest of the three rounds: counting games proves they STARTED. Stopping the engine after one round gives 200 games, all-zero columns and exit 0 — byte for byte the signature CB-EV-0030 says the instrumentation distinguishes from a real result. Both harnesses now require every counted game to have reached an outcome over five rounds. FATAL 3: round 2's `.csv` filter was applied to all three loops, so catalog.yaml and rules_delta.yaml — whose missing digests were round 1's finding — were recorded and then never compared, and never checked against upstream at all. Only the parser loop filters now. FATAL 4: five of six tiebreak comparators had no coverage. GR-E04's tiebreak never executes in any scenario. All four are now covered and mutation-verified; the Blame key needed compensating claims to be reachable at all, since Blame also lowers the coalition score. SERIOUS: "peak held" computed the same number as "peak assigned" for every possible input — the real gap was that START_STRESS was an unchecked constant, now read off the dealt state; cadence="none" was a pure loophole, removed; sibling discovery swapped a hand-written list for hand-written globs and missed metadata.json and VARIANT.md, both named in the package's own changed_files — now walked, and it found them immediately; and "~72,000 games" was unsourced, make panels runs 17,600. Also separated two kinds of number that were presented alike: seats×games is invariant, 363 and 29 vary 7.1%-11.5% across samples. Round 4 owed. The conclusion is not that the work is nearly right — it is that author-made corrections to measurement work should be assumed defective until a fresh reader has attacked them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
feb68027f9
commit
c4a8a227c0
9 changed files with 386 additions and 57 deletions
|
|
@ -66,18 +66,27 @@ SIBLINGS = "../"
|
|||
# raised FileNotFoundError from `digest` instead of failing with the
|
||||
# designed message, and "the sibling packages are covered" counted lines
|
||||
# in a Markdown file -- it passed with all three files deleted.
|
||||
SIBLING_GLOBS = ("catalog.yaml", "experiments/*/rules_delta.yaml", "experiments/*/*.csv")
|
||||
# Everything under `editions/` that is not the edition directory itself.
|
||||
# **Walked, not globbed** (CB-REV-0003 #8): the first version listed three
|
||||
# hand-written glob patterns, which is the same self-certifying shape as
|
||||
# reading the list out of PROVENANCE -- one hand-written list swapped for
|
||||
# another. It missed `metadata.json` and `VARIANT.md`, both named in the
|
||||
# package's OWN `changed_files` manifest, and anything a directory deeper.
|
||||
SIBLING_SKIP = {".DS_Store"}
|
||||
|
||||
|
||||
def sibling_files():
|
||||
"""Sibling packages that are on disk, as `../`-relative paths."""
|
||||
import glob as _glob
|
||||
|
||||
root = os.path.join(ROOT, EDITION, SIBLINGS)
|
||||
"""Every file beside the edition, as `../`-relative paths."""
|
||||
base = os.path.join(ROOT, "editions")
|
||||
edition_dir = os.path.join(ROOT, EDITION)
|
||||
out = []
|
||||
for pattern in SIBLING_GLOBS:
|
||||
for hit in _glob.glob(os.path.join(root, pattern)):
|
||||
rel = os.path.relpath(hit, os.path.join(ROOT, EDITION))
|
||||
for dirpath, _dirs, files in os.walk(base):
|
||||
if os.path.abspath(dirpath).startswith(os.path.abspath(edition_dir)):
|
||||
continue
|
||||
for name in files:
|
||||
if name in SIBLING_SKIP:
|
||||
continue
|
||||
rel = os.path.relpath(os.path.join(dirpath, name), edition_dir)
|
||||
out.append(rel.replace(os.sep, "/"))
|
||||
return sorted(out)
|
||||
|
||||
|
|
@ -98,7 +107,7 @@ def check():
|
|||
print(f" [FAIL] a digest is recorded for a file that is not here: {', '.join(missing)}")
|
||||
rc = 1
|
||||
|
||||
for name in [f for f in present if f.endswith(".csv")]:
|
||||
for name in present:
|
||||
if name not in want:
|
||||
continue
|
||||
path = os.path.join(ROOT, EDITION, name)
|
||||
|
|
@ -118,6 +127,17 @@ def check():
|
|||
# ADR-0015 D3's falsifier, checked rather than asserted: the hand
|
||||
# reader handles commas inside quotes and NOTHING ELSE. A doubled
|
||||
# quote or an embedded newline means `csv` is the answer after all.
|
||||
# ONLY this loop filters: it is ADR-0015 D3's falsifier about the
|
||||
# hand-rolled CSV reader, and running it over YAML made a valid
|
||||
# `catalog.yaml` line with an odd quote count fail with a message
|
||||
# about a parser that never reads it (CB-REV-0002 #9).
|
||||
#
|
||||
# **The digest and freshness loops must NOT filter** — CB-REV-0003 #3:
|
||||
# the round-2 correction applied this filter to all three, so the two
|
||||
# sibling YAMLs whose missing digests were round 1's finding were
|
||||
# recorded and then never compared, and never checked against
|
||||
# upstream at all. `make edition-check` answered its own headline
|
||||
# question with [ok] when the answer was no.
|
||||
for name in [f for f in present if f.endswith(".csv")]:
|
||||
raw = open(os.path.join(ROOT, EDITION, name), encoding="utf-8-sig").read()
|
||||
if '""' in raw:
|
||||
|
|
@ -138,7 +158,7 @@ def check():
|
|||
print(" [----] upstream not checked out — freshness UNVERIFIED")
|
||||
print(f" expected {UPSTREAM_DIR}")
|
||||
return rc
|
||||
for name in [f for f in present if f.endswith(".csv")]:
|
||||
for name in present:
|
||||
up = os.path.join(UPSTREAM_DIR, name)
|
||||
if not os.path.exists(up):
|
||||
print(f" [FAIL] {name} is not in upstream — where did it come from?")
|
||||
|
|
|
|||
|
|
@ -316,11 +316,13 @@ def check_gate_registry(root=REPO):
|
|||
if not target:
|
||||
continue
|
||||
cadence = g.get("cadence")
|
||||
if cadence not in ("all", "manual", "none"):
|
||||
if cadence not in ("all", "manual"):
|
||||
out.append(Finding(
|
||||
"gates", "gates.toml",
|
||||
f"{target!r} declares no cadence — say whether `make all` runs "
|
||||
f"it, or a gate can exist without ever running"))
|
||||
f"{target!r} declares no cadence — say `all` or `manual`. "
|
||||
f"`none` was a pure loophole: a real target that runs "
|
||||
f"nowhere and passes, which is the condition this rule "
|
||||
f"exists to prevent (CB-REV-0003 #7)"))
|
||||
elif cadence == "all" and target not in deps:
|
||||
out.append(Finding(
|
||||
"gates", "gates.toml",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue