Declare CB-WP-0015: the two inert clauses, tier S
Provenance (tier S, one paragraph in lieu of survey and ADR): the two clauses tools/mutation-check.py has reported inert since CB-WP-0005 — AM-7's scaling ratio (nothing relates the two throughput numbers Criterion prints) and AM-8's N=10 (the runner does two). They are the last two PARTIAL rows in the acceptance table. Structural tier S: acceptance rows measure the product, and bench-test is in gates.toml's not_control_gates list, so the M trigger about the loop's own constraints does not fire. Chaos d4=2, no override. Declaration 10 of 12. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
2513d17704
commit
7e9ab221a7
1 changed files with 166 additions and 0 deletions
166
workplans/CB-WP-0015-the-inert-clauses.md
Normal file
166
workplans/CB-WP-0015-the-inert-clauses.md
Normal file
|
|
@ -0,0 +1,166 @@
|
|||
---
|
||||
id: CB-WP-0015
|
||||
kind: product
|
||||
title: "The two inert clauses: AM-7 scaling and AM-8 N=10"
|
||||
status: todo
|
||||
---
|
||||
|
||||
# Purpose
|
||||
|
||||
```
|
||||
structural tier S (acceptance rows measure the product, not how we
|
||||
work — `bench-test` is in gates.toml's
|
||||
not_control_gates list, so the M trigger about the
|
||||
loop's own constraints does not fire)
|
||||
chaos d4 = 2 → no override
|
||||
declared tier S
|
||||
```
|
||||
|
||||
Declaration 10 of 12. Tier S: one provenance paragraph in the commit
|
||||
message, no survey and no ADR. Steps 4–5 are not skipped.
|
||||
|
||||
## The two clauses nobody has touched in seven passes
|
||||
|
||||
`tools/mutation-check.py` has carried these two verdicts since CB-WP-0005,
|
||||
in its own words:
|
||||
|
||||
> **AM-7 `scaling >= 0.9x`** — *"no code computes the ratio of throughput
|
||||
> @100k to @5k or compares it to 0.9; Criterion reports both and nothing
|
||||
> relates them."*
|
||||
>
|
||||
> **AM-8 `N=10 same-seed replays`** — *"the runner executes each scenario
|
||||
> TWICE (K8), not ten times; the spec says N=10. The double-run is live
|
||||
> and the count is not."*
|
||||
|
||||
Both have been reported honestly every run and neither has been worked.
|
||||
AM-7 moved 1/3 → 2/3 in CB-WP-0006; AM-8 has been 1/2 since it was
|
||||
instrumented. They are the last two PARTIAL rows in the acceptance table.
|
||||
|
||||
**And the test is named for the property it does not check.**
|
||||
`replay_100k_events_is_linear_and_fast` folds a log at 10,000 and at
|
||||
100,000 events, prints a throughput for each, and asserts each is under
|
||||
five seconds *independently*. The two numbers are computed, printed, and
|
||||
never compared. At 10,000 the five-second assertion is close to vacuous on
|
||||
its own.
|
||||
|
||||
This is the same family as the last four passes' central findings, one
|
||||
step further along: not a claim nobody checked, but **a number measured
|
||||
and then not used**. The name asserts linearity; the body does not.
|
||||
|
||||
## The risk this pass has to face rather than route around
|
||||
|
||||
AM-7's clause was written against a measured boardgame.io defect —
|
||||
throughput halving as history doubled. Our aggregate may not be able to
|
||||
exhibit that defect at all: if GROUND state is bounded and `fold` does not
|
||||
scan history, replay is O(n) by construction and the ratio is ~1.0 no
|
||||
matter what.
|
||||
|
||||
**If so, the honest result is not a green row.** It is that AM-7's scaling
|
||||
clause measures something our design makes structurally impossible to
|
||||
violate, and a clause that cannot fail is the `unmutatable` verdict this
|
||||
project already counts *against* itself (ADR-0005 §1). Making it green by
|
||||
asserting a tautology would be the worse outcome of the two.
|
||||
|
||||
So this task may end in an amendment rather than an assertion. Decide it on
|
||||
what the mutation says, not on which outcome closes more rows.
|
||||
|
||||
## Task: make AM-7's scaling clause live, or show it cannot be
|
||||
|
||||
```task
|
||||
id: CB-WP-0015-T01
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Compute the ratio the spec names — throughput folding a growing log at
|
||||
100k events against the same at 5k — and assert it against 0.9.
|
||||
|
||||
**The measurement has a known trap, and it is documented in this repo
|
||||
already.** AM-6's assertion first ran inside `make all` and read 38,753
|
||||
ev/s against 341,280 in isolation, a 9× error, because `cargo test` runs
|
||||
binaries and threads concurrently: *"a throughput assertion inside a
|
||||
parallel harness measures contention, not throughput."* A **ratio** of two
|
||||
timings taken under varying contention is worse than a single reading, not
|
||||
better — the noise does not cancel, it multiplies. Whatever this lands in
|
||||
must run where AM-6 runs (release, `--test-threads=1`, its own target) or
|
||||
not be gated at all.
|
||||
|
||||
Required, and the pass fails without them:
|
||||
|
||||
- **The mutation must go red for the stated reason.** Slowing `fold` in
|
||||
proportion to log length — not by a constant, which AM-6 already
|
||||
catches — must break the ratio and nothing else.
|
||||
- **A positive control on the ratio itself.** A harness that measured the
|
||||
same size twice would report 1.0 and look excellent. Prove the two legs
|
||||
are different sizes and that the denominator is the small one.
|
||||
- **State the variance.** One reading of a ratio is not a measurement.
|
||||
Report the spread across samples, and if the spread crosses 0.9 then
|
||||
the threshold is not gateable at this sample count — say that instead of
|
||||
taking the best reading.
|
||||
|
||||
If the ratio is ~1.0 because nothing in the design can make it otherwise,
|
||||
write that up and amend the row, following the precedent AM-4c set in
|
||||
`specs/GameKernel.md` §5a — the argument goes in the spec, at tier S,
|
||||
with the row retained in the denominator.
|
||||
|
||||
## Task: settle AM-8's N, at 10 or at 2, with the argument
|
||||
|
||||
```task
|
||||
id: CB-WP-0015-T02
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
The spec says ten same-seed replays; the runner does two. Close the gap in
|
||||
whichever direction survives the argument — and the argument is the
|
||||
deliverable here, not the number.
|
||||
|
||||
**Ten is doing real work only if the failure it hunts is probabilistic.**
|
||||
A deterministic divergence — a seeded RNG threaded wrong, a fold that
|
||||
depends on insertion order — fails on run two exactly as reliably as on
|
||||
run ten. A divergence from ambient nondeterminism — hash iteration order,
|
||||
pointer-dependent hashing, thread scheduling — appears with some
|
||||
probability per run, and there ten runs are eight more chances than two.
|
||||
|
||||
So: which does AM-8 exist to catch, and what does the repo already do
|
||||
about the other? `clippy.toml` denies `HashMap`/`HashSet` and `make check`
|
||||
runs `-D warnings`, which is AM-8's other clause and already live. If the
|
||||
lint removes the probabilistic class structurally, ten runs are buying
|
||||
repetitions of a check that cannot fail — and that costs time on every
|
||||
`make all`.
|
||||
|
||||
Whichever way it goes, the mutation must still go red: the existing
|
||||
seed-perturbation mutation is the control, and if raising or lowering N
|
||||
does not change what that mutation catches, that is itself the finding.
|
||||
|
||||
Do not amend the spec silently. If N becomes 2, GameKernel §5 says so and
|
||||
says why, in the same shape as §5a.
|
||||
|
||||
## Task: evidence, and what the acceptance table now claims
|
||||
|
||||
```task
|
||||
id: CB-WP-0015-T03
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
`evidence/CB-EV-0013-*.md`.
|
||||
|
||||
Report the mutation table before and after, and be exact about the
|
||||
denominator — CB-EV-0005 established that a score improved by deleting the
|
||||
question is not an improvement, and this pass can improve its score two
|
||||
ways, one of which is deleting a question.
|
||||
|
||||
Also due here:
|
||||
|
||||
- **Quote CB-WP-0014's final cost, not this pass's.** The rule adopted in
|
||||
CB-EV-0012. CB-WP-0014 closed at **$7.47 / 34 responses** = 0.220.
|
||||
- **CB-EV-0009's standing prediction is now testable for the first
|
||||
time.** It predicted a pass opening above the SH-1 hard line would cost
|
||||
more than 0.123 per response. CB-WP-0014 opened above the line and cost
|
||||
0.220. Report it as one data point, and say plainly that one point past
|
||||
a threshold is not a confirmation — the prediction has a direction and
|
||||
no control.
|
||||
- **Whether the acceptance table still has a PARTIAL row**, and if the
|
||||
answer is no, whether that is because both clauses became live or
|
||||
because one became a spec amendment.
|
||||
Loading…
Add table
Add a link
Reference in a new issue