CB-WP-0015: the two inert clauses, AM-7 scaling and AM-8 N=10
Some checks failed
ci / check (push) Failing after 3s
Some checks failed
ci / check (push) Failing after 3s
Provenance (tier S, one paragraph in lieu of survey and ADR): the two clauses mutation-check has reported inert since CB-WP-0005. AM-7's scaling ratio was held up by a test literally named replay_100k_events_is_linear_and_fast that computed both throughputs, printed both, and never divided one by the other. AM-8's N=10 was held up by a runner that does two. Both are now red. AM-7 3/3, AM-8 2/2, M-D1-MUT 10/14, and ADR-0005's >=10-of-14 prediction MET for the first time. Neither was closed by amending the question away, which was the live risk: the denominator is unchanged and the four unenforced rows are the four already unenforceable. AM-7 needed three estimators. Best-of-N per leg then divide (AM-6's, correct for a floor on one number) gave 0.581-1.085 on an unchanged binary; legs back-to-back gave medians 0.931-1.004; legs interleaved at fold granularity give 0.987/0.991/0.989, and 0.989 under 8-way CPU contention while absolute throughput fell 4x. The INDETERMINATE guard demanded unanimity and failed a good measurement over one sample 0.001 under the floor; it now requires a two-thirds majority. The control that matters: AM-6's constant-cost mutation halves throughput and leaves this ratio at 0.999x green, so AM-7 is not a second AM-6. AM-8 kept N=10 because the measurement said so. Perturbing the RNG only from its fourth construction on: --runs 2 PASSES, --runs 10 fails. A late-onset divergence is deterministic, not flaky, so it is a control rather than a coin flip. Ten runs live on one scenario (make am8, ~2s) rather than all 25 (47s a build). GameKernel 5b records it. The full run also found AM-4a's own mutation stale since ADR-0008 D3 moved the target 250,000 -> 161,000 in CB-WP-0013 -- reported HARNESS-BROKEN, no score published. The build-free half of that check is now a --self-test assertion, so make all catches the next one. mutation-check clauses may now carry their own verify and mutation, and then the enforced flag is measured rather than declared; a declaration disagreeing with its measurement is refused. make all exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
7e9ab221a7
commit
ee37b82675
9 changed files with 802 additions and 51 deletions
|
|
@ -2,7 +2,7 @@
|
|||
id: CB-WP-0015
|
||||
kind: product
|
||||
title: "The two inert clauses: AM-7 scaling and AM-8 N=10"
|
||||
status: todo
|
||||
status: done
|
||||
---
|
||||
|
||||
# Purpose
|
||||
|
|
@ -68,7 +68,7 @@ what the mutation says, not on which outcome closes more rows.
|
|||
|
||||
```task
|
||||
id: CB-WP-0015-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -103,11 +103,41 @@ write that up and amend the row, following the precedent AM-4c set in
|
|||
`specs/GameKernel.md` §5a — the argument goes in the spec, at tier S,
|
||||
with the row retained in the denominator.
|
||||
|
||||
**Done 2026-08-02.** `make am7`; AM-7 is now **red, 3/3 clauses**.
|
||||
|
||||
The clause is live and no amendment was needed. Measured ratio **0.987 /
|
||||
0.991 / 0.989** across three runs against the 0.9 floor, and **0.989 under
|
||||
eight-way CPU contention** — the design is flat, as the bounded-state
|
||||
analysis predicted, with a few percent of cache cost on the longer log.
|
||||
|
||||
**Two estimators were wrong before the third, and both looked
|
||||
reasonable.** Best-of-N per leg then divide — AM-6's estimator, correct for
|
||||
a floor on one number — gave **0.581 to 1.085** on an unchanged binary.
|
||||
Running the legs back-to-back inside one sample gave medians 1.004 /
|
||||
0.931 / 0.956: closer, but a 50 ms leg samples a *point* on this machine's
|
||||
drift rather than averaging it. Interleaving at fold granularity with 10 M
|
||||
events a leg makes the drift common-mode, and it divides out. Absolute
|
||||
throughput still swings **11 M – 55 M ev/s**; the ratio does not.
|
||||
|
||||
**The INDETERMINATE guard was wrong for the same reason the estimator
|
||||
was.** It demanded unanimity. Under contention the median read 0.971 and
|
||||
one sample read 0.899 — a thousandth under — and it failed the build; it
|
||||
also fired intermittently inside `mutation-check`, which runs this test
|
||||
straight after a 50-second rebuild. It now requires a two-thirds majority
|
||||
agreeing with the median. A gate that fails when the machine is busy is a
|
||||
flake, and a flake gets suppressed rather than fixed.
|
||||
|
||||
**The mutation, and the control that matters more.** Making fold cost grow
|
||||
with history drives the ratio to **0.751** — red, tight, no straddle.
|
||||
AM-6's *constant*-cost mutation halves throughput (28M → 15M ev/s) and
|
||||
leaves this ratio at **0.999× — green**. Without that control, "AM-7 goes
|
||||
red" would have been consistent with AM-7 being a redundant copy of AM-6.
|
||||
|
||||
## Task: settle AM-8's N, at 10 or at 2, with the argument
|
||||
|
||||
```task
|
||||
id: CB-WP-0015-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -136,11 +166,44 @@ does not change what that mutation catches, that is itself the finding.
|
|||
Do not amend the spec silently. If N becomes 2, GameKernel §5 says so and
|
||||
says why, in the same shape as §5a.
|
||||
|
||||
**Done 2026-08-02.** `make am8`; AM-8 is now **red, 2/2 clauses**.
|
||||
`GameKernel.md` §5b records where the ten runs live.
|
||||
|
||||
**N stays at 10, and the measurement is why.** The argument above is
|
||||
correct as far as it goes — a deterministic divergence shows on run 2
|
||||
exactly as reliably as on run 10 — but it is not the whole population.
|
||||
Perturbing the RNG only from its **fourth** construction onward, on
|
||||
`gr-r06-round-resolve`:
|
||||
|
||||
| | result |
|
||||
|---|---|
|
||||
| `cb-sim --runs 2` | **PASS** |
|
||||
| `cb-sim --runs 10` | FAIL — *"run 1 hash … != run 4 hash … (of 10)"* |
|
||||
|
||||
A **late-onset** divergence is a real class the double-run structurally
|
||||
cannot see, and — the part that makes it usable — it is *deterministic,
|
||||
not probabilistic*. So it can be a control rather than a coin flip, and
|
||||
the `N=10` clause is now measured by it rather than declared.
|
||||
|
||||
The cost objection stands and is answered by placement, not by amendment:
|
||||
ten runs across all 25 scenarios would cost **47 s per build** to
|
||||
re-answer an answered question, so `make sim` keeps K8's double-run over
|
||||
everything and `make am8` runs ten on one scenario in ~2 s. The primary
|
||||
defence against the *probabilistic* class is still the `HashMap`/`HashSet`
|
||||
deny lint — this row's other clause, already live — and the ten runs are
|
||||
defence in depth against that exclusion failing.
|
||||
|
||||
Also here: `scenario::run_n`, `cb-sim --runs <n>` (which refuses `n < 2`
|
||||
rather than falling back to 2 — a flag that silently degraded would report
|
||||
an N=10 result after an N=2 check), and later runs compared to the **first**
|
||||
rather than to their predecessor, so a divergence appearing at run 5 and
|
||||
persisting cannot hide after run 6.
|
||||
|
||||
## Task: evidence, and what the acceptance table now claims
|
||||
|
||||
```task
|
||||
id: CB-WP-0015-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
|
|
@ -164,3 +227,29 @@ Also due here:
|
|||
- **Whether the acceptance table still has a PARTIAL row**, and if the
|
||||
answer is no, whether that is because both clauses became live or
|
||||
because one became a spec amendment.
|
||||
|
||||
**Done 2026-08-02.**
|
||||
[CB-EV-0013](../evidence/CB-EV-0013-the-inert-clauses.md). `make all`
|
||||
exits 0.
|
||||
|
||||
- **No PARTIAL rows remain, and neither clause was amended away.** AM-7
|
||||
red 3/3, AM-8 red 2/2. **M-D1-MUT 10/14**, measured, and ADR-0005's
|
||||
standing prediction (≥10 of 14) is **MET for the first time**.
|
||||
- **The denominator did not move**, which was the live risk. This pass
|
||||
could have improved its score by deleting a question — amending AM-8's
|
||||
N down to 2, or splitting AM-7's scaling into a fifteenth row — and both
|
||||
were considered and refused. The four unenforced rows are the four that
|
||||
were already unenforceable.
|
||||
- **The full run found a third defect: AM-4a's mutation was stale**, still
|
||||
pointing at the 250,000 target ADR-0008 D3 replaced with 161,000. It
|
||||
reported HARNESS-BROKEN and refused to publish a score. The build-free
|
||||
half of that check is now a `--self-test` assertion, so `make all`
|
||||
catches the next one instead of only a full mutation run.
|
||||
- **`mutation-check` now measures a claim it used to assert.** A clause
|
||||
may carry its own verify and mutation; then its `enforced` flag is
|
||||
measured and a declaration disagreeing with its measurement is refused.
|
||||
Two clauses carry one, and print as `red*`.
|
||||
- **CB-EV-0009's prediction came due**: CB-WP-0014 opened above the SH-1
|
||||
hard line and cost 0.220 $/response against the predicted floor of
|
||||
0.123. Unfalsified, **not confirmed** — there is no control, because no
|
||||
pass has opened below the line since it was made.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue