CB-WP-0021 T06: fix AM-7's measurement, not its floor
Some checks failed
ci / check (push) Has been cancelled

The row folded a 5,000-event log against a 100,000-event log and compared
throughputs, which confounds 'does cost per event grow with history'
(the property it claims) with 'does streaming a 20x longer Vec cost more
per element' (a memory-hierarchy fact true of any program). It measured
the second and reported it as the first: importing the edition enlarged
the aggregate and the ratio fell to 0.845 with the state bounded.

Corrected to time the SAME 5,000 events on a state at depth 0 and on a
state at depth 100,000. Equal windows, equal event mix, so the only
difference left is history depth.

  corrected: clean 1.004, mutated 0.589 (red)
  old:       clean 0.845 (red on healthy code), mutated 0.751

It also runs in 8.5s instead of timing out: the first version re-walked
the 100k prefix every repetition, 200M untimed folds per sample, which
under the mutation never finished. A control that cannot be run is not a
control. It now advances to depth once per sample and clones.

Two of my own measurements here were wrong and both were caught by
measuring again. A 2-minute timeout killed the shell line before its
restoring cp ran, so three readings were taken on MUTATED code -- I
diagnosed an event-mix confound that did not exist and 'fixed' it. The
fix is kept on its merits; the justification was fiction. And the probe
that proved state was bounded had checked four of eleven collections.

make all exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-04 01:10:38 +02:00
parent 2da19a49b7
commit 503966bce9
5 changed files with 155 additions and 82 deletions

View file

@ -2,7 +2,7 @@
id: CB-WP-0021
kind: product
title: "Import the edition: the game plays its own data"
status: ready
status: active
state_hub_workstream_id: "782b1c37-f3a7-469b-87a3-fa73ebe758d2"
---
@ -114,7 +114,7 @@ alternatives to take, and that is the ADR.
```task
id: CB-WP-0021-T01
status: todo
status: done
priority: high
state_hub_task_id: "68e4fe63-eec6-4fb8-a84f-32c7edee19af"
```
@ -155,7 +155,7 @@ replay determinism, is not a data-loading change.
```task
id: CB-WP-0021-T05
status: todo
status: done
priority: high
state_hub_task_id: "c77c0b39-0841-40bf-8078-135486d7ed55"
```
@ -182,7 +182,7 @@ numbers nobody ruled on.
```task
id: CB-WP-0021-T02
status: todo
status: done
priority: high
state_hub_task_id: "28c3ff2c-16ae-47b5-9474-10e754936c60"
```
@ -207,7 +207,7 @@ Hidden → face down) and `hidden_priority`.
```task
id: CB-WP-0021-T03
status: todo
status: done
priority: medium
state_hub_task_id: "ff3bd923-9066-49ce-aadd-a3552e4964ff"
```
@ -228,6 +228,51 @@ ruled. Message `ground-game` with the outcome.
Do **not** quietly delete a failing-in-fact scenario. CB-EV-0005: *a score
improved by deleting the question is not an improvement.*
## Task: fix AM-7's measurement, not its floor
```task
id: CB-WP-0021-T06
status: done
priority: high
```
T05 turned AM-7 red: median **0.845** against a 0.9 floor, all nine
samples below. The maintainer chose to **fix the measurement** rather than
ADR the floor or optimise the fold.
**The row was measuring the wrong thing.** It folded a 5,000-event log and
a 100,000-event log and compared throughputs, which confounds:
1. does cost per event grow with how many events have been folded? — the
property AM-7 claims; and
2. does streaming a 20× longer `Vec` cost more per element? — a
memory-hierarchy fact true of any program.
It measured (2) and reported it as (1). Importing the edition enlarged the
aggregate and the ratio fell, **with the state bounded**.
**Corrected:** time a 5,000-event window on a state at depth 0, and *the
same events* on a state at depth 100,000. Equal windows, equal event mix;
the only difference left is history depth.
| | clean | history-proportional mutation |
|---|---|---|
| corrected | **1.004** | **0.589 — red** |
| old | 0.845 (red on healthy code) | 0.751 |
Renamed `am7_cost_per_event_does_not_grow_with_history`, because the old
name described the confounded measurement.
**Two of my own measurements in this task were wrong, and both were caught
by measuring again.** A 2-minute timeout killed the shell line before its
restoring `cp` ran, so the next three readings were taken on **mutated
code** — I diagnosed a "second confound" from event-mix that did not
exist, and "fixed" it by folding identical events on both legs. That
change is kept, on its own merits: identical events remove a real
potential confound. But the justification I gave for it was fiction, and
the probe that proved state was bounded had only checked four of eleven
collections.
## Task: evidence
```task