CB-WP-0011-T03: evidence — the chaos roll fired, and it was right
Some checks failed
ci / check (push) Has been cancelled
Some checks failed
ci / check (push) Has been cancelled
CB-EV-0009. Tier S did not produce a worse outcome than tier L would have. The roll deleted a survey that would have opened on 2D toolkit selection; the decomposition it forced instead found that the existing text renderer was showing 24 of 41 view fields. The structural trigger fires on a property of the plan, not of the code, so nothing in the tier derivation could have said the port was the wrong first question. Recorded honestly in both directions: the pass also made an interface change with no review, which is the cost side. One favourable fire is not a calibration; the window stays open to 2026-09-30. Corrects a number this pass asserted: the T01 commit message says '42 of 43, up from 24'. Measured by splicing the old renderer back in, it is 41 of 42 with 1 declared omitted, up from 24 -- and 16 fields were genuinely absent, not 17, because outcome.winners was rendered in a different format. Both original figures were counted by hand. CHAOS gains its first caught entry. Cheapest pass per response yet recorded (0.094 vs a previous best of 0.123).
This commit is contained in:
parent
b11fc91fd4
commit
0d2ab2286d
3 changed files with 216 additions and 6 deletions
191
evidence/CB-EV-0009-inspectable-table.md
Normal file
191
evidence/CB-EV-0009-inspectable-table.md
Normal file
|
|
@ -0,0 +1,191 @@
|
||||||
|
# CB-EV-0009 — the chaos roll fired, and what it cost
|
||||||
|
|
||||||
|
CB-WP-0011 T03. Measured 2026-08-02 at `HEAD` after T02. Pass kind
|
||||||
|
`product`.
|
||||||
|
|
||||||
|
This is the first pass in which the chaos mechanism changed a tier, so
|
||||||
|
the first question is the one the calibration window exists to answer.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Did tier S produce a worse outcome than tier L would have?
|
||||||
|
|
||||||
|
**No — and the reason is not "tier L was unnecessary". It is that the
|
||||||
|
roll forced a decomposition that was better than the one I had planned.**
|
||||||
|
|
||||||
|
The declaration:
|
||||||
|
|
||||||
|
```
|
||||||
|
structural tier L (INTENT stage 1 creates a new capability port)
|
||||||
|
chaos d4 = 4 → override fires
|
||||||
|
override roll shuf -e S M L → S
|
||||||
|
```
|
||||||
|
|
||||||
|
Before the roll I had the shape of the tier-L pass in hand: a survey
|
||||||
|
(`CB-RES-0006`) whose leading constraint would have been **AM-4a's 3,750
|
||||||
|
lines of headroom** — 246,250 of a 250,000 target, measured — followed by
|
||||||
|
an ADR choosing a 2D toolkit and standing up `cb-render-api` /
|
||||||
|
`cb-render-null`.
|
||||||
|
|
||||||
|
Tier S has no survey and no ADR, and the hard gate — *no implementation
|
||||||
|
code for a capability exists before its ADR is committed* — is not a tier
|
||||||
|
weight and does not roll down. So the two rules met head-on, and the
|
||||||
|
resolution was to **split the declaration**: take the part of stage 1
|
||||||
|
that creates no port and adds no dependency, and leave the port to its
|
||||||
|
own declaration and its own roll.
|
||||||
|
|
||||||
|
What that split surfaced is the finding this pass acted on:
|
||||||
|
|
||||||
|
> `cb-play` already had a renderer. It had had one since stage 0. It was
|
||||||
|
> showing **24 of the 41** leaf paths a populated `GroundView` carries.
|
||||||
|
|
||||||
|
A tier-L pass would have opened by surveying 2D toolkits. It would have
|
||||||
|
been surveying how to *draw* a table whose text renderer was silently
|
||||||
|
dropping the DARVO state machine, the GROUND practice, the scoring mode,
|
||||||
|
the Focus tokens, the discard pile, per-seat protection, and every part
|
||||||
|
of the outcome except the headline. **The port was the wrong first
|
||||||
|
question**, and nothing in the structural tier derivation could have said
|
||||||
|
so, because the trigger fires on "creates a capability port" — a property
|
||||||
|
of the *plan*, not of the code.
|
||||||
|
|
||||||
|
**Stated against my own prior.** In the turn before the roll I recommended
|
||||||
|
tier L and said the AM-4a headroom "should lead the survey". The roll
|
||||||
|
deleted the survey and the pass was better for it. One data point, and it
|
||||||
|
is one data point — but it is the *first* evidence the CHAOS gate has
|
||||||
|
produced in six declarations, and it points the way the mechanism's
|
||||||
|
defenders hoped and I did not expect.
|
||||||
|
|
||||||
|
**What tier L would have caught that this pass did not.** Being honest in
|
||||||
|
the other direction: this pass made a real interface change —
|
||||||
|
`cb_game_runtime::replay::open`, extracted so the inspector and the replay
|
||||||
|
gate share one bundle reader — with no review and no ADR. It is small,
|
||||||
|
dev-only, and flagged in its commit per the chaos limits, and I believe it
|
||||||
|
is right. But "I believe it is right" is exactly what a tier-L review
|
||||||
|
exists not to accept, and that is the cost side of the ledger.
|
||||||
|
|
||||||
|
**Carried, not concluded.** Six of twelve declarations used, one override.
|
||||||
|
The retire condition in `gates.toml` asks whether an overridden tier ever
|
||||||
|
produces *a different outcome than the argued one*. It just did. That is
|
||||||
|
recorded as the gate's first `caught` entry, and the window stays open —
|
||||||
|
one favourable fire is not a calibration.
|
||||||
|
|
||||||
|
## 2. The field-coverage gap, before and after
|
||||||
|
|
||||||
|
| | leaf paths shown |
|
||||||
|
|---|---|
|
||||||
|
| before (stage-0 renderer) | **24** of 41 |
|
||||||
|
| after | **41** of 41, plus 1 declared omitted |
|
||||||
|
|
||||||
|
**Measured, not counted.** The old renderer was spliced back in beside the
|
||||||
|
new fixture and run. That matters, because the first two figures I wrote
|
||||||
|
for this — "42 of 43, up from 24", in the T01 commit message — were
|
||||||
|
counted by hand and both were wrong. The trusted-arithmetic class, in a
|
||||||
|
pass whose entire subject is a gate against unverified claims.
|
||||||
|
|
||||||
|
The measurement also corrects itself in the honest direction:
|
||||||
|
`outcome.winners.*` shows as missing under the new token but *was* being
|
||||||
|
rendered, in a different format. So **16 fields were genuinely absent**,
|
||||||
|
not 17. That is the number to quote.
|
||||||
|
|
||||||
|
The one deliberate omission is `players.*.hand` — `null` for every seat
|
||||||
|
but the viewer, where the absence is what `hand N card(s)` renders.
|
||||||
|
|
||||||
|
**The gate caught a field its own author had missed**, on its first run,
|
||||||
|
before it had ever been committed: `players.*.hand` was in neither list.
|
||||||
|
That is the entry worth having in `M-D1-MUT`'s ledger, more than the three
|
||||||
|
mutations I designed on purpose.
|
||||||
|
|
||||||
|
## 3. What no test could have caught, and why
|
||||||
|
|
||||||
|
Worth writing down as a class, because it is new here:
|
||||||
|
|
||||||
|
> Every assertion a renderer test naturally makes — *the output mentions
|
||||||
|
> the round*, *the output is non-empty*, *P2's hand does not appear* — is
|
||||||
|
> satisfied by a renderer that shows a third of the state.
|
||||||
|
|
||||||
|
This is the harness-does-nothing class in **presentation** form. The
|
||||||
|
harness runs, the assertions are real, and they are all satisfied by the
|
||||||
|
defect. What breaks it is asserting over the *shape of the input* rather
|
||||||
|
than the content of the output: walk the serialized view, require every
|
||||||
|
leaf path to be classified, and make silence cost a build.
|
||||||
|
|
||||||
|
Paths, not keys. `problem` occurs under a DARVO target, a GROUND choice
|
||||||
|
and a Selection; a key-set walk would let one of the three vouch for the
|
||||||
|
other two.
|
||||||
|
|
||||||
|
## 4. Cost, shape, and the meta budget
|
||||||
|
|
||||||
|
| pass | kind | responses | cost | $/response |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| CB-WP-0006 | meta | 158 | $57.22 | 0.362 |
|
||||||
|
| CB-WP-0007 | meta | 21 | $7.95 | 0.298 |
|
||||||
|
| CB-WP-0008 | product | 134 | $17.38 | 0.123 |
|
||||||
|
| CB-WP-0009 | meta | 46 | $11.31 | 0.246 |
|
||||||
|
| CB-WP-0010 | product | 26 | $4.08 | 0.157 |
|
||||||
|
| **CB-WP-0011** | **product** | **45** | **$4.23** | **0.094** |
|
||||||
|
|
||||||
|
**The cheapest pass per response yet recorded**, against a previous best
|
||||||
|
of 0.123. Two mechanical causes, both boring and both worth keeping:
|
||||||
|
|
||||||
|
- it opened immediately after a compaction (§ below), and
|
||||||
|
- every task had a `cargo test` between it and being wrong.
|
||||||
|
|
||||||
|
The CB-WP-0009 figure moved from $6.73 (as reported in CB-EV-0008 §4) to
|
||||||
|
$11.31. That is not a correction: CB-EV-0008 was written *during*
|
||||||
|
CB-WP-0009, and the responses after it are attributed to the pass they
|
||||||
|
belong to. A pass cannot measure its own final cost, and reporting one
|
||||||
|
mid-pass will always read low.
|
||||||
|
|
||||||
|
### Session shape
|
||||||
|
|
||||||
|
| metric | this pass | previous pass opened at | target |
|
||||||
|
|---|---|---|---|
|
||||||
|
| SH-1 mean context | **147,808** | 338,852 `[HARD]` | ≤ 200,000 |
|
||||||
|
| SH-2 p90 context | **149,262** | 339,342 `[SOFT]` | ≤ 300,000 |
|
||||||
|
| SH-3 batching | 0.0% `[SOFT]` | 0.0% | ≥ 20% |
|
||||||
|
|
||||||
|
Both context metrics went from breach to comfortably inside, and the
|
||||||
|
lever was one `/compact` before the pass opened. That is now the second
|
||||||
|
time the same lever has produced the cheapest pass on record
|
||||||
|
(CB-WP-0008 was the first). Two observations are not a law, but the
|
||||||
|
prediction is cheap and falsifiable: **the next pass opened above the
|
||||||
|
SH-1 hard line will cost more per response than 0.123.**
|
||||||
|
|
||||||
|
SH-3 stays at 0.0% against a 20% floor and remains unfalsified and
|
||||||
|
unremedied. It has now read 0.0% for five consecutive passes.
|
||||||
|
|
||||||
|
### Meta budget
|
||||||
|
|
||||||
|
**58% of the trailing three, against a soft 25%** — up from 45%, during
|
||||||
|
two consecutive `product` passes.
|
||||||
|
|
||||||
|
That is not an instrument defect, but it is a property worth naming: the
|
||||||
|
budget is a **cost share**, so two cheap product passes move it less than
|
||||||
|
one expensive meta pass moved it up. CB-WP-0009's $11.31 is 58% of the
|
||||||
|
$19.62 the window holds, and it leaves the window on the next pass.
|
||||||
|
|
||||||
|
**Falsifiable prediction:** if the next pass is `product`, the trailing-3
|
||||||
|
meta share drops to **0%**, because CB-WP-0009 will be the pass that
|
||||||
|
rolled off. If it does not, the windowing is wrong in a way neither
|
||||||
|
CB-EV-0007 §3 nor CB-EV-0008 §1 found.
|
||||||
|
|
||||||
|
No product work was displaced by meta work in this pass — nothing meta
|
||||||
|
was opened. The number is over the line and reported under the rule that
|
||||||
|
requires reporting it.
|
||||||
|
|
||||||
|
## 5. Open
|
||||||
|
|
||||||
|
- **Stage 1 is not shipped.** This is its inspectable half. INTENT's
|
||||||
|
stage-1 line stays open; the port, the visualization and
|
||||||
|
drag-to-propose are untouched.
|
||||||
|
- **The port declaration is still owed**, structurally tier L, with its
|
||||||
|
own chaos roll, and with AM-4a's 3,750-line headroom as its leading
|
||||||
|
constraint. That number is unchanged by this pass — nothing was added
|
||||||
|
to the shipped runtime.
|
||||||
|
- **CHAOS has its first `caught` entry** and stays open to 2026-09-30.
|
||||||
|
- **GATE-REVIEW still has none**, one pass older.
|
||||||
|
- **SH-3 at 0.0% for five passes.** Either the floor is wrong or the
|
||||||
|
behaviour is, and neither has been argued.
|
||||||
|
- **`cb-play` is now two tools in one binary.** Play and inspect share a
|
||||||
|
renderer and nothing else. If a third mode arrives, that is the second
|
||||||
|
use, and the split should be reconsidered then rather than now.
|
||||||
|
|
@ -120,7 +120,9 @@ target = ""
|
||||||
checks = "d4 on each tier declaration, 12-declaration calibration window"
|
checks = "d4 on each tier declaration, 12-declaration calibration window"
|
||||||
added = "2026-07-30"
|
added = "2026-07-30"
|
||||||
review_by = "2026-09-30"
|
review_by = "2026-09-30"
|
||||||
caught = []
|
caught = [
|
||||||
|
"CB-WP-0011: first fire in 6 declarations — d4=4 rolled stage 1 from structural L to S; the deleted survey would have opened on 2D toolkits while the existing text renderer was showing 24 of 41 view fields (CB-EV-0009 §1)",
|
||||||
|
]
|
||||||
retire_if = "the window closes with no overridden tier producing a different outcome than the argued one — the evaluation this window exists to make possible"
|
retire_if = "the window closes with no overridden tier producing a different outcome than the argued one — the evaluation this window exists to make possible"
|
||||||
|
|
||||||
[[gate]]
|
[[gate]]
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
id: CB-WP-0011
|
id: CB-WP-0011
|
||||||
kind: product
|
kind: product
|
||||||
title: "Stage 1, first slice: an inspector that shows everything"
|
title: "Stage 1, first slice: an inspector that shows everything"
|
||||||
status: todo
|
status: done
|
||||||
state_hub_workstream_id: "1142442a-65f1-483a-800d-6ca8490c5f1a"
|
state_hub_workstream_id: "1142442a-65f1-483a-800d-6ca8490c5f1a"
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -110,9 +110,17 @@ test run against a state where the fields are absent is the same lie in a
|
||||||
different costume.
|
different costume.
|
||||||
|
|
||||||
**Done 2026-08-02.** The renderer moved to `tools/cb-play/src/inspect.rs`
|
**Done 2026-08-02.** The renderer moved to `tools/cb-play/src/inspect.rs`
|
||||||
and now covers **42 of the 43 leaf paths** in a populated `GroundView`,
|
and now covers **41 of the 42 leaf paths** in a populated `GroundView`.
|
||||||
up from 24. The one omission is `players.*.hand` — `null` for a
|
The one omission is `players.*.hand` — `null` for a non-viewer seat,
|
||||||
non-viewer seat, where the absence is what `hand N card(s)` renders.
|
where the absence is what `hand N card(s)` renders.
|
||||||
|
|
||||||
|
**Measured, not counted** (T03): the old renderer produced 24 of the 41
|
||||||
|
tokens. One of the 17 it missed, `outcome.winners.*`, it did in fact
|
||||||
|
show — in a different format (`["P1", "P2"]` rather than `winners P1,
|
||||||
|
P2`), so the token misses it. **16 fields were genuinely absent**, and
|
||||||
|
that is the number to quote. The commit message for this task says
|
||||||
|
"42 of 43, up from 24"; both figures were counted by hand before the
|
||||||
|
measurement and both were wrong.
|
||||||
|
|
||||||
`every_view_field_is_classified` walks the serialized view for leaf
|
`every_view_field_is_classified` walks the serialized view for leaf
|
||||||
*paths* (not keys — `problem` occurs under a DARVO target, a GROUND
|
*paths* (not keys — `problem` occurs under a DARVO target, a GROUND
|
||||||
|
|
@ -193,7 +201,7 @@ nothing at all would still have reported a matching hash and passed.
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: CB-WP-0011-T03
|
id: CB-WP-0011-T03
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "db722865-1233-4918-952d-0bf4761199e4"
|
state_hub_task_id: "db722865-1233-4918-952d-0bf4761199e4"
|
||||||
```
|
```
|
||||||
|
|
@ -217,3 +225,12 @@ Write `evidence/CB-EV-0009-inspectable-table.md` covering:
|
||||||
lands; the port half is untouched and INTENT's stage-1 line stays open.
|
lands; the port half is untouched and INTENT's stage-1 line stays open.
|
||||||
Marking a stage complete because part of it works is the failure mode
|
Marking a stage complete because part of it works is the failure mode
|
||||||
stage 0 avoided by leaving the CLI player open for three passes.
|
stage 0 avoided by leaving the CLI player open for three passes.
|
||||||
|
|
||||||
|
**Done 2026-08-02.** [CB-EV-0009](../evidence/CB-EV-0009-inspectable-table.md).
|
||||||
|
The headline: tier S did **not** produce a worse outcome, and the reason
|
||||||
|
is not that tier L was unnecessary — the roll forced a decomposition that
|
||||||
|
found a defect the survey would have walked past. Recorded as the CHAOS
|
||||||
|
gate's first `caught` entry, with the window left open.
|
||||||
|
|
||||||
|
Also on record, against my own prior: I recommended tier L in the turn
|
||||||
|
before the roll, and the roll was right.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue