CB-REV-0002: round 2, and the corrections were not approvable either
Some checks failed
ci / check (push) Failing after 4s
Some checks failed
ci / check (push) Failing after 4s
Three FATAL, five SERIOUS. The substance of round 1's corrections held — Reactive is genuinely one arm different, the five replacement controls are non-inert, the inert metric is right, the numbers reproduce. What failed were the CLAIMS about them, and two defects the corrections introduced. FATAL 1: the fix for round 1's #11 did not fix it. The assertion was `games + setup_fails == 200`, and a refused setup increments setup_fails while skipping games — so the sum is invariant under exactly the failure it claimed to catch. Injecting setup failures gave exit 0 over 196-game columns. Now asserts games == GAMES, verified to exit 101. FATAL 2: the correction to the selective-column FATAL was itself selective. "81-1000 per cell, baseline AND H1" and "31-1000" twelve lines apart, both taken from the baseline row; under H1 rank-75 arms are 59/0/0/0. Every cell is now printed rather than summarised, and the corrected verdict is the opposite of the one it replaced: under rank-75, H1 REDUCES DARVO arms to zero at 3p and above. FATAL 3: "DARVO arms 2 per seat per game" is 1 per seat per game, exactly, at every band. SERIOUS: the tiebreak oracle asserted only that the winner set CHANGED, so reversing the tiebreak left it green; the #13 defect's impact was claimed and never measured (72,000 games: zero divergences — real in principle, witnessed only by a constructed board); a 29-of-363 citation pointed at a file that did not contain it (round 1's reviewer did report it, and it was never transcribed — the record was wrong, not the number); the harnesses were run by NO GATE, so every published figure came from a manual run of an ungated binary, including the assertion added for #1; and edition-check's sibling handling — added by the last correction — was self-certifying, crashed instead of failing, and counted Markdown lines as coverage. Now discovered on disk, and it found a real gap on its first run: Rules_Text.csv vendored with no digest. Also: "peak Stress held" was dead code kept quiet by `let _ = held;` — the numbers were right by coincidence. make panels is now a registered gate. Round 3 is owed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
84aa09264c
commit
da58e78e4a
10 changed files with 285 additions and 56 deletions
|
|
@ -59,10 +59,27 @@ def vendored_files():
|
|||
# control unverified — correctly. Recorded relative to `editions/`.
|
||||
SIBLINGS = "../"
|
||||
|
||||
# Discovered ON DISK, not read out of PROVENANCE (CB-REV-0002 #8). The
|
||||
# first version listed siblings by reading the digest block, which made
|
||||
# three controls vacuous at once: a recorded-but-absent file could never
|
||||
# be reported missing (it was in `present` by construction), an absent one
|
||||
# raised FileNotFoundError from `digest` instead of failing with the
|
||||
# designed message, and "the sibling packages are covered" counted lines
|
||||
# in a Markdown file -- it passed with all three files deleted.
|
||||
SIBLING_GLOBS = ("catalog.yaml", "experiments/*/rules_delta.yaml", "experiments/*/*.csv")
|
||||
|
||||
|
||||
def sibling_files():
|
||||
"""Recorded paths that live beside the edition rather than in it."""
|
||||
return sorted(f for f in recorded() if f.startswith(SIBLINGS))
|
||||
"""Sibling packages that are on disk, as `../`-relative paths."""
|
||||
import glob as _glob
|
||||
|
||||
root = os.path.join(ROOT, EDITION, SIBLINGS)
|
||||
out = []
|
||||
for pattern in SIBLING_GLOBS:
|
||||
for hit in _glob.glob(os.path.join(root, pattern)):
|
||||
rel = os.path.relpath(hit, os.path.join(ROOT, EDITION))
|
||||
out.append(rel.replace(os.sep, "/"))
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def check():
|
||||
|
|
@ -72,7 +89,7 @@ def check():
|
|||
|
||||
siblings = sibling_files()
|
||||
present = vendored_files() + siblings
|
||||
undocumented = [f for f in vendored_files() if f not in want]
|
||||
undocumented = [f for f in vendored_files() + siblings if f not in want]
|
||||
if undocumented:
|
||||
print(f" [FAIL] vendored with no recorded digest: {', '.join(undocumented)}")
|
||||
rc = 1
|
||||
|
|
@ -81,10 +98,15 @@ def check():
|
|||
print(f" [FAIL] a digest is recorded for a file that is not here: {', '.join(missing)}")
|
||||
rc = 1
|
||||
|
||||
for name in present:
|
||||
for name in [f for f in present if f.endswith(".csv")]:
|
||||
if name not in want:
|
||||
continue
|
||||
have = digest(os.path.join(ROOT, EDITION, name))
|
||||
path = os.path.join(ROOT, EDITION, name)
|
||||
if not os.path.exists(path):
|
||||
print(f" [FAIL] {name} is recorded but not on disk")
|
||||
rc = 1
|
||||
continue
|
||||
have = digest(path)
|
||||
# `../x` resolves out of the edition dir, which is the point.
|
||||
if have != want[name]:
|
||||
print(f" [FAIL] {name} does not match its recorded digest")
|
||||
|
|
@ -96,7 +118,7 @@ def check():
|
|||
# ADR-0015 D3's falsifier, checked rather than asserted: the hand
|
||||
# reader handles commas inside quotes and NOTHING ELSE. A doubled
|
||||
# quote or an embedded newline means `csv` is the answer after all.
|
||||
for name in present:
|
||||
for name in [f for f in present if f.endswith(".csv")]:
|
||||
raw = open(os.path.join(ROOT, EDITION, name), encoding="utf-8-sig").read()
|
||||
if '""' in raw:
|
||||
print(f" [FAIL] {name} contains a doubled quote — ADR-0011's revisit")
|
||||
|
|
@ -116,7 +138,7 @@ def check():
|
|||
print(" [----] upstream not checked out — freshness UNVERIFIED")
|
||||
print(f" expected {UPSTREAM_DIR}")
|
||||
return rc
|
||||
for name in present:
|
||||
for name in [f for f in present if f.endswith(".csv")]:
|
||||
up = os.path.join(UPSTREAM_DIR, name)
|
||||
if not os.path.exists(up):
|
||||
print(f" [FAIL] {name} is not in upstream — where did it come from?")
|
||||
|
|
@ -151,9 +173,9 @@ def self_test():
|
|||
chk("digests match the real files",
|
||||
all(digest(os.path.join(ROOT, EDITION, f)) == want[f] for f in everything))
|
||||
chk("the sibling packages are covered",
|
||||
len(sibling_files()) >= 3,
|
||||
"catalog.yaml and rules_delta.yaml decide WHAT WE MEASURED; "
|
||||
"CB-WP-0038 claimed digests for them and recorded none")
|
||||
all(f in want for f in sibling_files()) and len(sibling_files()) >= 3,
|
||||
"catalog.yaml and rules_delta.yaml decide WHAT WE MEASURED; the first "
|
||||
"version counted lines in PROVENANCE and passed with the files deleted")
|
||||
|
||||
# The control that matters: a changed byte must be detected.
|
||||
import tempfile
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue