coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
clay-borg/facts.toml

131 lines
2.2 KiB
TOML
Raw Normal View History

CB-WP-0004 T04: fact registry and make facts-check — DFD gets a gate Duplicated-fact drift is the fourth error class and the only one with no executable rule. No positive control catches it (both copies are internally consistent) and re-derivation does not either (the copy reproduces whatever it was copied from). It is caught only by reading a copy against its source, which nothing in the loop required. facts.toml holds 15 facts and is GENERATED by `make facts-gen` from cb-cost, dep-weight and rule-coverage. The trap this task named — a hand-maintained registry that becomes another drifting copy — is closed by facts-check re-running the instruments and failing when the committed registry disagrees with them. A stale registry cannot certify stale artifacts. An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key. 17 occurrences across 5 artifacts are now checked. Falsified before being believed: changing CostAccounting.md line 158 from $93.15 to $92.87 — the exact historical drift — produced exit 1 naming the file, the line and the expected value. Tested against the class it exists to catch, on a real artifact, not only in its self-test. It then caught a live tag inside its own documentation example in InnerLoop.md within the hour. Third time a gate has failed on its own pass's work. What it does not close is stated rather than implied: 22 untagged literal copies remain and are reported, not failed. Tagging is opt-in, a number can legitimately recur, and a gate that fires on coincidence gets routed around. Naming the uncovered surface beats claiming the class is closed. InnerLoop single-source-of-fact moves from prose to executable — v1.3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:24:39 +02:00
# GENERATED — do not edit. `make facts-gen` rewrites this file.
#
# The single source of fact for numbers that appear in more than
# one artifact (InnerLoop v1.2). Every value here is produced by
# the instrument named in its `by` field, on the current tree.
# `make facts-check` fails if this file disagrees with the
# instruments, or if a tagged artifact disagrees with this file.
generated = "2026-07-31"
pin = "fc76445"
[am4a_loc]
value = 246250
text = "246,250"
fmt = "{:,}"
by = "tools/dep-weight.py"
[am4a_target]
value = 250000
text = "250,000"
fmt = "{:,}"
by = "tools/dep-weight.py TARGETS"
[am4b_loc]
value = 317021
text = "317,021"
fmt = "{:,}"
by = "tools/dep-weight.py"
[am4b_target]
value = 350000
text = "350,000"
fmt = "{:,}"
by = "tools/dep-weight.py TARGETS"
CB-WP-0005 T02: M-D1-MUT — 4 of 14 acceptance rows are enforced For each acceptance row in GameKernel §5, invert the property and require the verifying command to go red. adapted:mutation-testing, with the denominator changed from source lines to acceptance rows. M-D1-MUT: 4/14 rows enforced PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert) unmutatable 8 (no property to invert, reason stated per row) SURVIVED 0 Two corrections to our own numbers. There are 14 rows, not the twelve ADR-0005 and CB-WP-0005 both asserted — AM-4 splits into a/b/c. And the prediction of 9-of-12 (75%) becomes >=10 of 14; measured 4 (29%), badly unmet. No target moved in this commit. The second correction matters more. My first run reported two SURVIVED rows and both were my own no-op mutations: `pub struct NullRng;` -> `pub struct NullRng {}` is semantically identical, and renaming max_age_days does nothing because CA-17 reads it with a default of 90. Both would have been published as "this row asserts nothing" — a false accusation against code that is fine. Replaced with real inversions (a per-construction counter in the ChaCha seed; reverting AC-9's output resolution to the first-wins bug it was fixed for), after which both go red. T08 asks whether writing a weak mutation is the new grep. It is, demonstrably, on the first attempt. The finding is larger than the workplan assumed. 8 of 14 rows are unmutatable — AM-2, AM-3, AM-4c, AM-5, AM-6, AM-9, AM-10, AM-11 have no instrument at all. AM-6 is the sharpest: nothing in the workspace compares any number to 100,000 events/s, the headline throughput claim. The problem is not three unimplemented rules, it is that more than half the acceptance table has nothing behind it. Harness controls: a stale find-string reports HARNESS-BROKEN rather than scoring the baseline as the mutant; a red baseline reports inconclusive rather than red; the tree is restored in a finally and the restoration is verified. Not in `make all` — it rebuilds the workspace once per row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:27:23 +02:00
[am_rows]
value = 14
text = "14"
fmt = "{:,}"
by = "tools/mutation-check.py"
[am_unmutatable]
CB-WP-0006 T03: measure AM-5 and AM-9; AM-5 breaches AM-9 is met and gated: 13.4 MB peak RSS against a 64 MB target, 4.8x headroom, in `make all` via --fast. CB-EV-0001's "very unlikely to bind" was right, but it is now measured rather than assumed, and verified red by a property mutation (a 300 MB allocation in the workload). AM-5 is BREACHED on both readings, on the machine the spec names: dev toolchain (default features) 87.0 s [FAIL target <= 60 s] shipped runtime (--no-default-features) 61.3 s [FAIL target <= 60 s] bnt-lap001, 8 cores — a direct comparison, not a directional one. A row declared "recorded not gated" and never recorded fails its own target by 45% on first measurement. The tool reports and exits 0 because the spec says the row is ungated. Gating it is a spec change needing an ADR; a tool that promotes itself is how a target starts binding without anyone deciding it should. So AM-5 stays unmutatable — for the accurate reason now — and the breach is raised as a maintainer decision: speed the build, move the target by ADR (arguing why 60 s was wrong rather than why 87 s is convenient), or withdraw the row. The measurement itself had a real bug, found only by cross-validation. getrusage(RUSAGE_CHILDREN) is a high-water mark across every reaped child, so it attributed cargo's memory to the workload and reported 38.2 MB for a run that used 12.3 MB — a 3x over-report that was plausible, passed its target, and would have been published. Fixed with os.wait4, which returns that specific child's rusage, and the self-test now cross-checks against /usr/bin/time -v. That is the false-accusation shape in the measurement layer rather than the mutation layer: an instrument confidently reporting a number it had not earned. Also: the clean build measures into a throwaway CARGO_TARGET_DIR rather than running `cargo clean`, so measuring the metric does not cost several minutes of rebuild afterwards. A metric that punishes its own measurement gets measured once and never again. M-D1-MUT: 6 -> 7 of 14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 18:59:32 +02:00
value = 5
text = "5"
CB-WP-0005 T02: M-D1-MUT — 4 of 14 acceptance rows are enforced For each acceptance row in GameKernel §5, invert the property and require the verifying command to go red. adapted:mutation-testing, with the denominator changed from source lines to acceptance rows. M-D1-MUT: 4/14 rows enforced PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert) unmutatable 8 (no property to invert, reason stated per row) SURVIVED 0 Two corrections to our own numbers. There are 14 rows, not the twelve ADR-0005 and CB-WP-0005 both asserted — AM-4 splits into a/b/c. And the prediction of 9-of-12 (75%) becomes >=10 of 14; measured 4 (29%), badly unmet. No target moved in this commit. The second correction matters more. My first run reported two SURVIVED rows and both were my own no-op mutations: `pub struct NullRng;` -> `pub struct NullRng {}` is semantically identical, and renaming max_age_days does nothing because CA-17 reads it with a default of 90. Both would have been published as "this row asserts nothing" — a false accusation against code that is fine. Replaced with real inversions (a per-construction counter in the ChaCha seed; reverting AC-9's output resolution to the first-wins bug it was fixed for), after which both go red. T08 asks whether writing a weak mutation is the new grep. It is, demonstrably, on the first attempt. The finding is larger than the workplan assumed. 8 of 14 rows are unmutatable — AM-2, AM-3, AM-4c, AM-5, AM-6, AM-9, AM-10, AM-11 have no instrument at all. AM-6 is the sharpest: nothing in the workspace compares any number to 100,000 events/s, the headline throughput claim. The problem is not three unimplemented rules, it is that more than half the acceptance table has nothing behind it. Harness controls: a stale find-string reports HARNESS-BROKEN rather than scoring the baseline as the mutant; a red baseline reports inconclusive rather than red; the tree is restored in a finally and the restoration is verified. Not in `make all` — it rebuilds the workspace once per row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 17:27:23 +02:00
fmt = "{:,}"
by = "tools/mutation-check.py"
CB-WP-0004 T04: fact registry and make facts-check — DFD gets a gate Duplicated-fact drift is the fourth error class and the only one with no executable rule. No positive control catches it (both copies are internally consistent) and re-derivation does not either (the copy reproduces whatever it was copied from). It is caught only by reading a copy against its source, which nothing in the loop required. facts.toml holds 15 facts and is GENERATED by `make facts-gen` from cb-cost, dep-weight and rule-coverage. The trap this task named — a hand-maintained registry that becomes another drifting copy — is closed by facts-check re-running the instruments and failing when the committed registry disagrees with them. A stale registry cannot certify stale artifacts. An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key. 17 occurrences across 5 artifacts are now checked. Falsified before being believed: changing CostAccounting.md line 158 from $93.15 to $92.87 — the exact historical drift — produced exit 1 naming the file, the line and the expected value. Tested against the class it exists to catch, on a real artifact, not only in its self-test. It then caught a live tag inside its own documentation example in InnerLoop.md within the hour. Third time a gate has failed on its own pass's work. What it does not close is stated rather than implied: 22 untagged literal copies remain and are reported, not failed. Tagging is opt-in, a number can legitimately recur, and a gate that fires on coincidence gets routed around. Naming the uncovered surface beats claiming the class is closed. InnerLoop single-source-of-fact moves from prose to executable — v1.3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:24:39 +02:00
[gr_covered]
value = 58
text = "58"
fmt = "{:,}"
by = "tools/rule-coverage.py"
[gr_linked]
value = 49
text = "49"
fmt = "{:,}"
by = "tools/rule-coverage.py"
[gr_rules]
value = 58
text = "58"
fmt = "{:,}"
by = "tools/rule-coverage.py"
[gr_scenarios]
value = 21
text = "21"
fmt = "{:,}"
by = "tools/rule-coverage.py"
CB-WP-0005 T01: spec->code link over every numbered spec and every crate AGGREGATE becomes a list of source roots and rule patterns become per-spec, so the link runs over every numbered spec x every crate rather than GroundRules.md x games/ground/src/lib.rs. The prediction held on the first run: AM-1b kernel spec->code link: 15/18 (83%) across 10 source files unlinked: K10 K14 K18 Kernel rules are link-only by design, and the output says so: they are kernel invariants with no aggregate, setup preset or command vocabulary, so scenarios/kernel/*.yaml with covers: [K11] would be a tag in a directory the runner cannot dispatch. Claiming scenario coverage for them is the inflation this gate exists to prevent. Per ADR-0005 §5 the kernel arm reports without feeding the exit code until 2026-08-31, then binds — the date in the tool, not in prose, with days remaining printed every run, because open-ended "gate it later" is how AM-4's targets went unratified for four workplans. The self-test asserts the gate returns 0 before that date and 2 after. The zero-rules positive control is replicated on the new denominator: a kernel regex that stops matching aborts rather than printing 0/0 as though it were 100%. The self-test passed while the tool was completely broken. A print( inside say() became say(), so every real `make coverage` died with RecursionError while --self-test reported all-ok — it only ever called kernel_arm(quiet=True) and never executed the reporting path. The control named the behaviour and did not assert it, which is precisely what this workplan is about. Fixed by exercising the loud path and asserting it prints, then verified by re-breaking say() and confirming both new checks go red. Seventh instance of the harness-does-nothing shape, in the tool written to find that shape. Also caught by its own gate: a self-test label that printed "0 K-ids" beside a passing ">5" assertion, because the detail string rebuilt the pattern with different escaping. A label that contradicts its own check is worse than no label. k_rules, k_linked and k_unlinked are registered facts under facts-check. A limit of that checker is recorded rather than patched: it is line-based, so a tagged value that prose-wraps fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 16:54:07 +02:00
[k_linked]
value = 15
text = "15"
fmt = "{:,}"
by = "tools/rule-coverage.py"
[k_rules]
value = 18
text = "18"
fmt = "{:,}"
by = "tools/rule-coverage.py"
[k_unlinked]
value = 'K10 K14 K18'
text = "K10 K14 K18"
fmt = "{}"
by = "tools/rule-coverage.py"
CB-WP-0004 T04: fact registry and make facts-check — DFD gets a gate Duplicated-fact drift is the fourth error class and the only one with no executable rule. No positive control catches it (both copies are internally consistent) and re-derivation does not either (the copy reproduces whatever it was copied from). It is caught only by reading a copy against its source, which nothing in the loop required. facts.toml holds 15 facts and is GENERATED by `make facts-gen` from cb-cost, dep-weight and rule-coverage. The trap this task named — a hand-maintained registry that becomes another drifting copy — is closed by facts-check re-running the instruments and failing when the committed registry disagrees with them. A stale registry cannot certify stale artifacts. An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key. 17 occurrences across 5 artifacts are now checked. Falsified before being believed: changing CostAccounting.md line 158 from $93.15 to $92.87 — the exact historical drift — produced exit 1 naming the file, the line and the expected value. Tested against the class it exists to catch, on a real artifact, not only in its self-test. It then caught a live tag inside its own documentation example in InnerLoop.md within the hour. Third time a gate has failed on its own pass's work. What it does not close is stated rather than implied: 22 untagged literal copies remain and are reported, not failed. Tagging is opt-in, a number can legitimately recur, and a gate that fires on coincidence gets routed around. Naming the uncovered surface beats claiming the class is closed. InnerLoop single-source-of-fact moves from prose to executable — v1.3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:24:39 +02:00
[pinned_main]
value = 92.03371920000004
text = "$92.03"
fmt = "${:,.2f}"
by = "tools/cb-cost.py --pin fc76445"
[pinned_mechanical_cost]
value = 33.3506319
text = "$33.35"
fmt = "${:,.2f}"
by = "tools/cb-cost.py --pin fc76445"
[pinned_mechanical_share]
value = 36
text = "36%"
fmt = "{:d}%"
by = "tools/cb-cost.py --pin fc76445"
[pinned_mechanical_turns]
value = 167
text = "167"
fmt = "{:,}"
by = "tools/cb-cost.py --pin fc76445"
[pinned_responses]
value = 346
text = "346"
fmt = "{:,}"
by = "tools/cb-cost.py --pin fc76445"
[pinned_subagent]
value = 1.1137235
text = "$1.11"
fmt = "${:,.2f}"
by = "tools/cb-cost.py --pin fc76445"
[pinned_total]
value = 93.14744270000004
text = "$93.15"
fmt = "${:,.2f}"
by = "tools/cb-cost.py --pin fc76445"
Reference in a new issue Copy permalink