CB-WP-0030: a control for the wrong-subject family
Some checks failed
ci / check (push) Failing after 4s
Some checks failed
ci / check (push) Failing after 4s
Seven claims in this project have been wrong the same way — the arithmetic right, the subject wrong. Four reached a document or another repo. ADR-0018 states the rule: a measured claim names the variable it depends on and shows the measurement moving with it. GameDesign §1.4 adds it as a fourth admissibility clause; the existing three are untouched. Honest yield is 3 clear + 1 partial of 7. It misses a count never made and a claim that is not a number, and D3 names those so nobody stops looking. The tempting fix — closing facts-check's 62 untagged literals — would have caught 1 of 7 and is deferred as separate work rather than bundled. loop-lint gains check_sensitivity_stated over arithmetic register rows, mutation-proven red on F17 and green when restored, with four self-test controls. It checks presence, never adequacy, and says so in its output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ffd2279e16
commit
d7279483f6
6 changed files with 503 additions and 6 deletions
|
|
@ -37,6 +37,7 @@
|
||||||
| workplan | CB-WP-0027 | done | — | workplans/CB-WP-0027-the-commentary-track.md |
|
| workplan | CB-WP-0027 | done | — | workplans/CB-WP-0027-the-commentary-track.md |
|
||||||
| workplan | CB-WP-0028 | done | — | workplans/CB-WP-0028-the-table-you-sit-at.md |
|
| workplan | CB-WP-0028 | done | — | workplans/CB-WP-0028-the-table-you-sit-at.md |
|
||||||
| workplan | CB-WP-0029 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
| workplan | CB-WP-0029 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
||||||
|
| workplan | CB-WP-0030 | ready | — | workplans/CB-WP-0030-a-number-that-does-not-move.md |
|
||||||
| task | CB-WP-0001-T01 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
| task | CB-WP-0001-T01 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||||
| task | CB-WP-0001-T02 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
| task | CB-WP-0001-T02 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||||
| task | CB-WP-0001-T03 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
| task | CB-WP-0001-T03 | done | — | workplans/CB-WP-0001-inner-loop.md |
|
||||||
|
|
@ -194,3 +195,7 @@
|
||||||
| task | CB-WP-0029-T02 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
| task | CB-WP-0029-T02 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
||||||
| task | CB-WP-0029-T03 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
| task | CB-WP-0029-T03 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
||||||
| task | CB-WP-0029-T04 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
| task | CB-WP-0029-T04 | done | — | workplans/CB-WP-0029-the-tokens-on-the-table.md |
|
||||||
|
| task | CB-WP-0030-T01 | todo | — | workplans/CB-WP-0030-a-number-that-does-not-move.md |
|
||||||
|
| task | CB-WP-0030-T02 | todo | — | workplans/CB-WP-0030-a-number-that-does-not-move.md |
|
||||||
|
| task | CB-WP-0030-T03 | todo | — | workplans/CB-WP-0030-a-number-that-does-not-move.md |
|
||||||
|
| task | CB-WP-0030-T04 | todo | — | workplans/CB-WP-0030-a-number-that-does-not-move.md |
|
||||||
|
|
|
||||||
126
decisions/ADR-0018-a-number-that-does-not-move.md
Normal file
126
decisions/ADR-0018-a-number-that-does-not-move.md
Normal file
|
|
@ -0,0 +1,126 @@
|
||||||
|
# ADR-0018: a number that does not move when its subject moves is measuring something else
|
||||||
|
|
||||||
|
status: accepted
|
||||||
|
date: 2026-08-07
|
||||||
|
decided by: agent, under the standing loop authorization
|
||||||
|
tier: M (structural M — adds an admissibility requirement, changing how
|
||||||
|
the loop constrains its own operation; ADR-0006 D5). chaos d8 = 7 → no
|
||||||
|
override. **Declaration 1 of chaos window 3.**
|
||||||
|
references: [CB-WP-0030](../workplans/CB-WP-0030-a-number-that-does-not-move.md),
|
||||||
|
CB-EV-0019 §1 (where the family was named),
|
||||||
|
[GameDesign.md](../specs/GameDesign.md) §1,
|
||||||
|
[RetrospectiveAnalysis.md](../specs/RetrospectiveAnalysis.md) §4.1,
|
||||||
|
[ADR-0017](ADR-0017-chaos-window-2-verdict.md) (instance 7)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Seven claims in this project have been wrong in the same way: the
|
||||||
|
arithmetic was right and the **subject** was not. Four reached a document
|
||||||
|
or another repository.
|
||||||
|
|
||||||
|
## D1 — what this does not do, stated first
|
||||||
|
|
||||||
|
**`facts-check`'s untagged surface is not this family's control.** It
|
||||||
|
reports **62** untagged literal copies across 7 registry keys and fails on
|
||||||
|
none. That is a real gap, and closing it would have caught **at most one**
|
||||||
|
of the seven:
|
||||||
|
|
||||||
|
- Instance 1's denominator **was** a registry fact and **was correct** —
|
||||||
|
`am4a_target` = 3,798 is right; AM-4a simply does not cover the code
|
||||||
|
being priced.
|
||||||
|
- Instances 2, 4, 5 and 6 were **fresh measurements** with no registry key
|
||||||
|
at all.
|
||||||
|
|
||||||
|
> **A tag proves a number came from somewhere. It cannot prove the
|
||||||
|
> somewhere was the right place.**
|
||||||
|
|
||||||
|
The untagged surface is worth closing on its own merits and is **named as
|
||||||
|
separate work**. Bundling it here would let this decision claim a victory
|
||||||
|
it did not win — which is the family's own error, applied to itself.
|
||||||
|
|
||||||
|
## D2 — the rule
|
||||||
|
|
||||||
|
> **A measured claim must name the variable it depends on, and show the
|
||||||
|
> measurement moving with it.**
|
||||||
|
>
|
||||||
|
> **A number that does not move when its subject moves is measuring
|
||||||
|
> something else.**
|
||||||
|
|
||||||
|
This is not new machinery. It is the generalisation of two things this
|
||||||
|
project already built, each in response to one instance:
|
||||||
|
|
||||||
|
- `RetrospectiveAnalysis` §4.1's **policy panel** — born from instance 5,
|
||||||
|
which is why a single policy's win rate may not be reported as a
|
||||||
|
difficulty;
|
||||||
|
- `attack-value.rs`'s **one-varying-number sweep** — built for F17, which
|
||||||
|
varies ATTACK's rank and holds everything else fixed.
|
||||||
|
|
||||||
|
**Both were built after an instance. The rule requires them before.**
|
||||||
|
|
||||||
|
## D3 — the yield, instance by instance, including the misses
|
||||||
|
|
||||||
|
| # | the claim | would D2 have caught it? |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | `csv` against AM-4a's headroom | **yes** — vary which crates are in scope and the budget moves; that *is* the error |
|
||||||
|
| 2 | *"12 in the file"* | **no** — but §1.2's row-level rule, born from it, does |
|
||||||
|
| 3 | SOLVE's wrong condition | **no** — not a number |
|
||||||
|
| 4 | 112–161 µs per node | **yes** — cost must rise with branching; it fell, and the tell was already printed |
|
||||||
|
| 5 | too easy at 5–6 seats | **yes** — vary the policy and the rate collapses to 0% |
|
||||||
|
| 6 | 2 Focus/Blame conflicts | **partly** — varying seat count would scale a real conflict; it is the weakest row |
|
||||||
|
| 7 | window 2 had zero overrides | **no** — a count that was never made has no sensitivity to state |
|
||||||
|
|
||||||
|
**Three clear, one partial, three missed.**
|
||||||
|
|
||||||
|
**The two it catches outright are the two that travelled furthest**, and
|
||||||
|
both were bound for `ground-game`: instance 4 sat in a survey, instance 5
|
||||||
|
was queued for a repo blocked on that exact number. A control that catches
|
||||||
|
the consequential half is worth having even at 3/7.
|
||||||
|
|
||||||
|
**What it misses is stated so nobody stops looking.** Instance 7 needs
|
||||||
|
counting, not sensitivity. Instance 3 was not a number. **No syntactic gate
|
||||||
|
detects a wrong subject in general** — it is a semantic error, and
|
||||||
|
claiming otherwise would build a gate that cannot cash out (ADR-0006 D3,
|
||||||
|
ADR-0017 D2).
|
||||||
|
|
||||||
|
## D4 — where it binds
|
||||||
|
|
||||||
|
- **Register findings whose claim is arithmetic** — the same scope
|
||||||
|
GameDesign §1.2 already uses.
|
||||||
|
- **Anything leaving the repo.** Four of the seven were outbound or nearly
|
||||||
|
so, and the recipient caught one we did not.
|
||||||
|
|
||||||
|
**Not** every number in every artifact. A rule that binds everywhere is
|
||||||
|
obeyed nowhere, and this project has 62 untagged literals as evidence of
|
||||||
|
what a broad, unenforced requirement produces.
|
||||||
|
|
||||||
|
## D5 — half of it is mechanical, and only half
|
||||||
|
|
||||||
|
`loop-lint` can check that a sensitivity line **is present**. It cannot
|
||||||
|
check that the sensitivity is the **right one**, because that is the same
|
||||||
|
judgement the rule exists to force.
|
||||||
|
|
||||||
|
> **The lint checks presence, not adequacy, and must say so in its own
|
||||||
|
> output** — otherwise a green lint reads as *"the claim was verified"*,
|
||||||
|
> and the control becomes a way of not looking.
|
||||||
|
|
||||||
|
That is the honest split: the mechanical half stops the requirement being
|
||||||
|
forgotten; the judgement half stays with the author and the reviewer, who
|
||||||
|
between them caught five of the seven.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- `GameDesign.md` §1 gains a fourth clause. The existing three are
|
||||||
|
untouched — each was bought by an earlier instance.
|
||||||
|
- `loop-lint` gains a check over the register's arithmetic rows.
|
||||||
|
- Existing rows are brought under it or exempted **with a reason**.
|
||||||
|
- The untagged-facts surface is recorded as separate, unclosed work.
|
||||||
|
|
||||||
|
## What was rejected
|
||||||
|
|
||||||
|
| rejected | why |
|
||||||
|
|---|---|
|
||||||
|
| closing the untagged-facts surface as *the* control | 1 of 7; a tag reproduces a wrong subject faithfully |
|
||||||
|
| requiring a regenerating command on every claim | catches transcription, and six of seven were not transcription errors |
|
||||||
|
| a rule binding every number everywhere | obeyed nowhere; the 62 untagged literals are the evidence |
|
||||||
|
| a lint that claims to verify the claim | it can only see presence, and pretending otherwise trains the eye to stop at green |
|
||||||
|
| stating a yield of "the family is closed" | the family's own error, applied to itself |
|
||||||
168
evidence/CB-EV-0028-a-number-that-does-not-move.md
Normal file
168
evidence/CB-EV-0028-a-number-that-does-not-move.md
Normal file
|
|
@ -0,0 +1,168 @@
|
||||||
|
# CB-EV-0028 — a number that does not move
|
||||||
|
|
||||||
|
CB-WP-0030 T04. Tier M (structural M — adds an admissibility requirement;
|
||||||
|
chaos d8 = 7 → no override). **Declaration 1 of chaos window 3**, opened
|
||||||
|
2026-08-07 by ADR-0017. Closed 2026-08-07.
|
||||||
|
|
||||||
|
**Delivered:** [ADR-0018](../decisions/ADR-0018-a-number-that-does-not-move.md),
|
||||||
|
`GameDesign.md` §1.4, `check_sensitivity_stated` in `loop-lint` with four
|
||||||
|
self-test controls, and a mutation proof that it goes red on a real row.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The seven instances, re-run against the rule
|
||||||
|
|
||||||
|
The workplan's own falsifier: **if fewer than 3 of 7, the rule is not the
|
||||||
|
one.**
|
||||||
|
|
||||||
|
| # | the claim | caught? | by what, exactly |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | `csv` costs 21,613 against 3,798 of headroom | **yes** | vary which crates AM-4a scopes and the headroom moves — that *is* the error |
|
||||||
|
| 2 | *"12 in the file"* | no | §1.2's row-level rule, born from it, does |
|
||||||
|
| 3 | SOLVE offered on a face-down Problem | no | **not a number** |
|
||||||
|
| 4 | 112–161 µs per node | **yes** | cost must rise with branching; it fell |
|
||||||
|
| 5 | too easy at 5–6 seats | **yes** | vary the policy, the rate goes 100% → 0% on the same deals |
|
||||||
|
| 6 | 2 Focus/Blame conflicts | **partly** | varying seat count scales a real conflict; a phantom one need not |
|
||||||
|
| 7 | window 2 had zero overrides | no | **a count never made has no sensitivity** |
|
||||||
|
|
||||||
|
**Three clear, one partial, three missed.** Above the falsifier, and not
|
||||||
|
by much.
|
||||||
|
|
||||||
|
**The distribution matters more than the count.** The two it catches
|
||||||
|
outright, 4 and 5, are the two that travelled furthest — instance 4 sat in
|
||||||
|
a published survey, instance 5 was queued for `ground-game`, which was
|
||||||
|
blocked on that exact number. **The three it misses were each caught by a
|
||||||
|
human re-reading**, which is the mechanism the rule generalises and cannot
|
||||||
|
replace.
|
||||||
|
|
||||||
|
**Row 6 is honestly "partly" and stays that way.** A phantom conflict from
|
||||||
|
comparing the wrong disc set *might* scale with seats too. The variation
|
||||||
|
would have prompted the question *what am I comparing*, which is what
|
||||||
|
actually found it — but that is the rule working as a **prompt**, not as a
|
||||||
|
test, and writing it down as a catch would inflate the yield.
|
||||||
|
|
||||||
|
## 2. What the untagged surface is worth, since it was measured
|
||||||
|
|
||||||
|
`facts-check` reports **62 untagged literal copies** across 7 registry
|
||||||
|
keys and fails on none.
|
||||||
|
|
||||||
|
**Against this family it is worth 1 of 7, and arguably 0.** Instance 1's
|
||||||
|
denominator was a registry fact and was **correct** — 3,798 *is* AM-4a's
|
||||||
|
target. The error was that AM-4a does not cover the code being priced. A
|
||||||
|
provenance tag would have reproduced the wrong subject faithfully and with
|
||||||
|
a citation.
|
||||||
|
|
||||||
|
> **A tag proves a number came from somewhere. It cannot prove the
|
||||||
|
> somewhere was the right place.**
|
||||||
|
|
||||||
|
Instances 2, 4, 5 and 6 were **fresh measurements** — there was no
|
||||||
|
registry key to tag.
|
||||||
|
|
||||||
|
**It is still worth closing, for a different failure mode**: a literal
|
||||||
|
that drifts from its registry key is a copy that disagrees, and
|
||||||
|
`facts-check` exists for exactly that. **Named as separate work and not
|
||||||
|
done here** — bundling it would have let this pass claim a victory it did
|
||||||
|
not win, which is this family's own error applied to itself.
|
||||||
|
|
||||||
|
## 3. Is a presence-only lint worth having
|
||||||
|
|
||||||
|
The strongest argument against: **a green lint reads as "the claim was
|
||||||
|
verified."** If it trains the eye to stop at green, it is worse than
|
||||||
|
nothing, because five of the seven instances were caught by an eye that
|
||||||
|
did not stop.
|
||||||
|
|
||||||
|
Three things decide it in favour.
|
||||||
|
|
||||||
|
**It says what it is, in its own output.** The finding text ends *"NOTE:
|
||||||
|
this checks presence, not adequacy."* A reader who stops at green is
|
||||||
|
stopping against an explicit statement rather than an implied one.
|
||||||
|
|
||||||
|
**Its cost is bounded by scope.** It binds arithmetic register rows, not
|
||||||
|
every number in every artifact — 62 untagged literals are this project's
|
||||||
|
own evidence of what an unenforced everywhere-rule produces (ADR-0018 D4).
|
||||||
|
|
||||||
|
**The failure it prevents is forgetting, and forgetting is what happened.**
|
||||||
|
`RetrospectiveAnalysis` §4.1's policy panel and `attack-value.rs`'s
|
||||||
|
one-varying-number sweep both exist. Both were built **after** an
|
||||||
|
instance. Nothing asked for either **before**.
|
||||||
|
|
||||||
|
**Verdict: worth having, on the condition it never grows a claim it
|
||||||
|
cannot cash.** The moment its output implies verification, ADR-0006 D3
|
||||||
|
and ADR-0017 D2 both apply and it should be deleted.
|
||||||
|
|
||||||
|
## 4. The check fired on three real rows, and all three were wrong to fire
|
||||||
|
|
||||||
|
First run: F12, F15, F21 — **every one a `note`**.
|
||||||
|
|
||||||
|
A note is by definition a finding **without** a reproduction
|
||||||
|
(GameDesign §3.1). There is no measurement, so there is nothing for a
|
||||||
|
sensitivity statement to be about; the check was demanding one anyway.
|
||||||
|
|
||||||
|
**The fix went into the check, not into three exemptions.** Three
|
||||||
|
hand-written `sensitivity-exempt` comments would have produced green with
|
||||||
|
the rule still wrong, and the next note would have needed a fourth. **That
|
||||||
|
is the difference between a rule and a pile of special cases**, and the
|
||||||
|
exemption mechanism exists for genuine one-offs, not for a category the
|
||||||
|
rule should never have covered.
|
||||||
|
|
||||||
|
## 5. The mutation, and the moment it nearly passed as proof
|
||||||
|
|
||||||
|
The control is that the check **fails on a real row** — a lint that
|
||||||
|
cannot go red is decoration.
|
||||||
|
|
||||||
|
**The first mutation attempt did not fire, and I nearly read that as a
|
||||||
|
failed check.** I had rewritten two phrases in F17's prose ("varies
|
||||||
|
exactly one number" → "uses a policy") and expected red. Green. A debug
|
||||||
|
probe showed F17 *was* being reached, kind `degenerate`, state `raised`,
|
||||||
|
prose found — and still matched, because the text elsewhere said *"one
|
||||||
|
number varied."*
|
||||||
|
|
||||||
|
**The check was right and the mutation was incomplete.** Stripping every
|
||||||
|
trigger word produces:
|
||||||
|
|
||||||
|
```
|
||||||
|
[sensitivity] specs/FindingRegister.md
|
||||||
|
F17 (degenerate) states a quantity and names no variable it depends
|
||||||
|
on (GameDesign §1.4). ... NOTE: this checks presence, not adequacy.
|
||||||
|
```
|
||||||
|
|
||||||
|
and restoring the file returns it to green.
|
||||||
|
|
||||||
|
**This is worth recording because the near-miss is the same family.** *"The
|
||||||
|
lint does not fire"* was a correct observation about the wrong subject — I
|
||||||
|
had measured my mutation, not the check. A debug probe printing what the
|
||||||
|
check actually saw settled it in one run, which is the second-computation
|
||||||
|
mechanism §1 credits with catching all seven.
|
||||||
|
|
||||||
|
**Four self-test controls**, because a check that cannot say NO is
|
||||||
|
decoration and one that cannot say YES fires on everything: it catches a
|
||||||
|
bare arithmetic claim, **clears one that names its variable**, skips
|
||||||
|
non-arithmetic kinds, and skips notes.
|
||||||
|
|
||||||
|
## 6. A vocabulary defect, caught by a gate that already existed
|
||||||
|
|
||||||
|
This workplan shipped as `kind: process`. The vocabulary is
|
||||||
|
**product | meta | mixed**, and `make self-tests` said so immediately.
|
||||||
|
|
||||||
|
**No new control was needed and none was added.** Corrected to `meta` —
|
||||||
|
the workplan changes how the loop constrains its own operation, which is
|
||||||
|
what `meta` means. Worth one line here because the pass that added a gate
|
||||||
|
was itself caught by an older one, working exactly as intended.
|
||||||
|
|
||||||
|
## 7. Cost
|
||||||
|
|
||||||
|
`make cost`, not inlined (§Single source of fact). CB-EV-0019 §4's
|
||||||
|
unbounded chain is still unbounded.
|
||||||
|
|
||||||
|
## Open after this pass
|
||||||
|
|
||||||
|
- **The untagged-facts surface** (§2) — 62 copies, measured, deferred,
|
||||||
|
unclosed.
|
||||||
|
- **The rule catches 3½ of 7 and the family is not closed.** Instances 3
|
||||||
|
and 7 need different controls: a wrong condition is not a number, and a
|
||||||
|
count never made needs counting.
|
||||||
|
- **The lint's honesty is a claim about readers, not code** (§3), and only
|
||||||
|
a later pass reading a green run tests it.
|
||||||
|
- **F18** — `Relations`, `DARVO`, `Player_Mats`, `Glossary` unvendored;
|
||||||
|
still the only open register row with no reproduction.
|
||||||
|
- **Three of four scenarios never dealt**, carried from ADR-0015 D5.
|
||||||
|
|
@ -81,6 +81,38 @@ admissible, remained in the register, and remained queued for transmission
|
||||||
to `ground-game` for four days, because nothing treated a passing artifact
|
to `ground-game` for four days, because nothing treated a passing artifact
|
||||||
as a signal. It was stopped by an adversarial review, not by a control.
|
as a signal. It was stopped by an adversarial review, not by a control.
|
||||||
|
|
||||||
|
### 1.4 It must move when its subject moves
|
||||||
|
|
||||||
|
> **A measured claim names the variable it depends on, and shows the
|
||||||
|
> measurement moving with it.** A number that does not move when its
|
||||||
|
> subject moves is measuring something else.
|
||||||
|
|
||||||
|
ADR-0018. This is the fourth clause and the first three are unchanged —
|
||||||
|
each was bought by an earlier mistake and a rewrite would drop what they
|
||||||
|
paid for.
|
||||||
|
|
||||||
|
**Why it exists.** Seven claims in this project have been wrong the same
|
||||||
|
way: the arithmetic right, the subject wrong. Two of them were caught only
|
||||||
|
because someone varied something —
|
||||||
|
|
||||||
|
- a per-node cost that **fell** as branching **rose** (no per-node cost
|
||||||
|
can), and
|
||||||
|
- a win rate attributed to *the game* that collapsed to 0% under a second
|
||||||
|
policy on the same deals.
|
||||||
|
|
||||||
|
**Both were already printed when they shipped.** Nobody looked at whether
|
||||||
|
the number moved the way its subject demanded.
|
||||||
|
|
||||||
|
**Where it binds:** register findings whose claim is arithmetic, and
|
||||||
|
anything leaving the repo. **Not every number everywhere** — a rule that
|
||||||
|
binds everywhere is obeyed nowhere.
|
||||||
|
|
||||||
|
**What it does not catch, stated so nobody stops looking:** a count that
|
||||||
|
was never made (there is no sensitivity to state), and a claim that is not
|
||||||
|
a number. **No syntactic gate detects a wrong subject in general.** It is
|
||||||
|
a semantic error; `loop-lint` checks that a sensitivity line is *present*,
|
||||||
|
never that it is *adequate*.
|
||||||
|
|
||||||
## 2. Kinds
|
## 2. Kinds
|
||||||
|
|
||||||
Five (ADR-0012 D4). **No kind exists without a finding that instantiates
|
Five (ADR-0012 D4). **No kind exists without a finding that instantiates
|
||||||
|
|
|
||||||
|
|
@ -306,6 +306,74 @@ def check_gate_registry(root=REPO):
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def check_sensitivity_stated(root=REPO):
|
||||||
|
"""GameDesign §1.4 / ADR-0018 — a finding whose claim is arithmetic
|
||||||
|
must name the variable it depends on.
|
||||||
|
|
||||||
|
**This checks PRESENCE, NEVER ADEQUACY.** It cannot tell whether the
|
||||||
|
variable named is the right one; that is the judgement the rule exists
|
||||||
|
to force, and it stays with the author and the reviewer. A green run
|
||||||
|
here means "somebody wrote a sensitivity line", not "the claim was
|
||||||
|
verified" -- and if it is ever read as the second, the control has
|
||||||
|
become a way of not looking.
|
||||||
|
|
||||||
|
Seven claims in this project were arithmetically correct about the
|
||||||
|
wrong subject. Three of them would have been caught by varying
|
||||||
|
something; this is the half of that a machine can see.
|
||||||
|
"""
|
||||||
|
out = []
|
||||||
|
reg = os.path.join(root, "specs", "FindingRegister.md")
|
||||||
|
if not os.path.exists(reg):
|
||||||
|
return out
|
||||||
|
text = open(reg).read()
|
||||||
|
try:
|
||||||
|
block = text.split("<!-- design-register:begin -->")[1] \
|
||||||
|
.split("<!-- design-register:end -->")[0]
|
||||||
|
except IndexError:
|
||||||
|
return out
|
||||||
|
|
||||||
|
# Kinds whose claim is a quantity. `inert` and `unplayed` are about
|
||||||
|
# whether a thing happens at all, which has no denominator to get
|
||||||
|
# wrong.
|
||||||
|
ARITHMETIC = {"inconsistent", "degenerate", "underdetermined"}
|
||||||
|
exempt = set(re.findall(r"^<!-- sensitivity-exempt:\s*(\S+)\s+(.+?)\s*-->$",
|
||||||
|
text, re.M))
|
||||||
|
exempt_ids = {e[0] for e in exempt}
|
||||||
|
|
||||||
|
for line in block.splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if not line.startswith("|") or line.startswith("|---"):
|
||||||
|
continue
|
||||||
|
cells = [c.strip() for c in line.strip("|").split("|")]
|
||||||
|
if len(cells) != 7 or cells[0] == "id":
|
||||||
|
continue
|
||||||
|
fid, kind, state = cells[0], cells[1], cells[2]
|
||||||
|
# Closed rows are history; the rule binds what is still claimed.
|
||||||
|
# And a `note` is by definition a finding WITHOUT a reproduction
|
||||||
|
# (GameDesign §3.1) -- there is no measurement to be sensitive
|
||||||
|
# about, so requiring one would be asking for a sensitivity
|
||||||
|
# statement about nothing.
|
||||||
|
if state in ("withdrawn", "applied", "note") or kind not in ARITHMETIC:
|
||||||
|
continue
|
||||||
|
if fid in exempt_ids:
|
||||||
|
continue
|
||||||
|
# The prose block for this finding must say what moves it.
|
||||||
|
body = ""
|
||||||
|
m = re.search(rf"^- \*\*{re.escape(fid)} [^\n]*(?:\n(?!- \*\*F?U?\d).*)*",
|
||||||
|
text, re.M)
|
||||||
|
if m:
|
||||||
|
body = m.group(0)
|
||||||
|
if not re.search(r"varie[sd]|varying|sensitivit|moves with|held fixed|"
|
||||||
|
r"one number|second policy", body, re.I):
|
||||||
|
out.append(Finding(
|
||||||
|
"sensitivity", "specs/FindingRegister.md",
|
||||||
|
f"{fid} ({kind}) states a quantity and names no variable it "
|
||||||
|
f"depends on (GameDesign §1.4). Say what would move it, or "
|
||||||
|
f"add `<!-- sensitivity-exempt: {fid} <reason> -->`. "
|
||||||
|
f"NOTE: this checks presence, not adequacy."))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
CHECKS = (
|
CHECKS = (
|
||||||
check_loadability,
|
check_loadability,
|
||||||
check_evidence_no_unmeasured,
|
check_evidence_no_unmeasured,
|
||||||
|
|
@ -315,6 +383,7 @@ CHECKS = (
|
||||||
check_review_trail,
|
check_review_trail,
|
||||||
check_reporting_tools_self_test,
|
check_reporting_tools_self_test,
|
||||||
check_gate_registry,
|
check_gate_registry,
|
||||||
|
check_sensitivity_stated,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -374,6 +443,36 @@ def self_test():
|
||||||
body += f"\n```task\nid: CB-WP-0100-T\nstatus: {t}\npriority: high\n```\n"
|
body += f"\n```task\nid: CB-WP-0100-T\nstatus: {t}\npriority: high\n```\n"
|
||||||
with open(os.path.join(tmp, "workplans", "CB-WP-0100-x.md"), "w") as fh:
|
with open(os.path.join(tmp, "workplans", "CB-WP-0100-x.md"), "w") as fh:
|
||||||
fh.write(body)
|
fh.write(body)
|
||||||
|
# GameDesign §1.4 / ADR-0018. Four controls, because a check that
|
||||||
|
# cannot say NO is decoration and one that cannot say YES fires on
|
||||||
|
# everything.
|
||||||
|
os.makedirs(os.path.join(tmp, "specs"), exist_ok=True)
|
||||||
|
|
||||||
|
def reg(kind, state, prose):
|
||||||
|
body = ("<!-- design-register:begin -->\n\n"
|
||||||
|
"| id | kind | state | reproduction | role | raised | owner |\n"
|
||||||
|
"|---|---|---|---|---|---|---|\n"
|
||||||
|
f"| F99 | {kind} | {state} | x.rs | counterexample | 2026-01-01 | us |\n"
|
||||||
|
"\n<!-- design-register:end -->\n\n"
|
||||||
|
f"- **F99 — a claim.** {prose}\n")
|
||||||
|
with open(os.path.join(tmp, "specs", "FindingRegister.md"), "w") as fh:
|
||||||
|
fh.write(body)
|
||||||
|
|
||||||
|
reg("degenerate", "raised", "It is 42.")
|
||||||
|
check("sensitivity: an arithmetic claim with no variable is caught",
|
||||||
|
len(check_sensitivity_stated(tmp)) == 1)
|
||||||
|
reg("degenerate", "raised", "It is 42, and it varies with seat count.")
|
||||||
|
check("sensitivity: naming the variable clears it",
|
||||||
|
not check_sensitivity_stated(tmp),
|
||||||
|
"without this it would fire on everything")
|
||||||
|
reg("inert", "raised", "It is 42.")
|
||||||
|
check("sensitivity: a non-arithmetic kind is not asked",
|
||||||
|
not check_sensitivity_stated(tmp),
|
||||||
|
"`inert` is about whether a thing happens, not how much")
|
||||||
|
reg("degenerate", "note", "It is 42.")
|
||||||
|
check("sensitivity: a note has no measurement to be sensitive about",
|
||||||
|
not check_sensitivity_stated(tmp), "GameDesign §3.1")
|
||||||
|
|
||||||
wp("ready", ["done", "todo"])
|
wp("ready", ["done", "todo"])
|
||||||
f = check_workplan_lifecycle(tmp)
|
f = check_workplan_lifecycle(tmp)
|
||||||
check("lifecycle detects `ready` after work has started",
|
check("lifecycle detects `ready` after work has started",
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,9 @@
|
||||||
---
|
---
|
||||||
id: CB-WP-0030
|
id: CB-WP-0030
|
||||||
kind: process
|
kind: meta
|
||||||
title: "A number that does not move: a control for the wrong-subject family"
|
title: "A number that does not move: a control for the wrong-subject family"
|
||||||
status: ready
|
status: done
|
||||||
|
state_hub_workstream_id: "e6a0dad9-9b45-41fb-92ab-07c8b5690366"
|
||||||
---
|
---
|
||||||
|
|
||||||
# Purpose
|
# Purpose
|
||||||
|
|
@ -84,8 +85,9 @@ applied to itself.**
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: CB-WP-0030-T01
|
id: CB-WP-0030-T01
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
|
state_hub_task_id: "4be3630e-4c67-4ef9-a2f6-f689208b26b1"
|
||||||
```
|
```
|
||||||
|
|
||||||
`decisions/ADR-0018-*.md`.
|
`decisions/ADR-0018-*.md`.
|
||||||
|
|
@ -100,12 +102,31 @@ priority: high
|
||||||
half is mechanical.
|
half is mechanical.
|
||||||
- **Do not bundle the untagged-facts surface.** Name it as separate.
|
- **Do not bundle the untagged-facts surface.** Name it as separate.
|
||||||
|
|
||||||
|
**Done 2026-08-07.**
|
||||||
|
[ADR-0018](../decisions/ADR-0018-a-number-that-does-not-move.md).
|
||||||
|
|
||||||
|
**D1 states what it does not do, first**, because the tempting fix was the
|
||||||
|
wrong one: closing the 62 untagged literals would have caught **1 of 7**,
|
||||||
|
and instance 1's denominator was a registry fact that was *correct*.
|
||||||
|
Deferred as separate work rather than bundled.
|
||||||
|
|
||||||
|
**D3 gives the yield instance by instance: three clear, one partial, three
|
||||||
|
missed.** The two it catches outright are the two that travelled furthest,
|
||||||
|
both bound for `ground-game`. **The misses are named** so nobody stops
|
||||||
|
looking — a count never made has no sensitivity, and instance 3 was not a
|
||||||
|
number.
|
||||||
|
|
||||||
|
**D5 splits it honestly**: the lint sees *presence*, never *adequacy*, and
|
||||||
|
its output says so — otherwise a green run reads as *"the claim was
|
||||||
|
verified"* and the control becomes a way of not looking.
|
||||||
|
|
||||||
## Task: the rule, in the spec that governs findings
|
## Task: the rule, in the spec that governs findings
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: CB-WP-0030-T02
|
id: CB-WP-0030-T02
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
|
state_hub_task_id: "cefcc917-91ef-4dbb-ab92-743266874b81"
|
||||||
```
|
```
|
||||||
|
|
||||||
`specs/GameDesign.md` §1 already requires a reproduction that **exists**,
|
`specs/GameDesign.md` §1 already requires a reproduction that **exists**,
|
||||||
|
|
@ -119,12 +140,18 @@ only sums.
|
||||||
- **the existing three clauses are unchanged** — this is an addition, and
|
- **the existing three clauses are unchanged** — this is an addition, and
|
||||||
a rewrite would silently drop what earlier instances bought.
|
a rewrite would silently drop what earlier instances bought.
|
||||||
|
|
||||||
|
**Done 2026-08-07.** `GameDesign.md` §1.4, with the two instances that
|
||||||
|
motivate it stated in it: a per-node cost that **fell** as branching
|
||||||
|
**rose**, and a win rate attributed to the game that collapsed to 0% under
|
||||||
|
a second policy. **Both were already printed when they shipped.**
|
||||||
|
|
||||||
## Task: make the mechanical half mechanical
|
## Task: make the mechanical half mechanical
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: CB-WP-0030-T03
|
id: CB-WP-0030-T03
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
|
state_hub_task_id: "b773bc85-1f7b-4563-a0b5-3d5007162846"
|
||||||
```
|
```
|
||||||
|
|
||||||
`loop-lint` fails when a register row of an arithmetic kind lacks a
|
`loop-lint` fails when a register row of an arithmetic kind lacks a
|
||||||
|
|
@ -139,12 +166,33 @@ sensitivity line.
|
||||||
a reason — a rule that only binds future rows leaves the six instances
|
a reason — a rule that only binds future rows leaves the six instances
|
||||||
unprotected.
|
unprotected.
|
||||||
|
|
||||||
|
**Done 2026-08-07.** `check_sensitivity_stated` in `loop-lint`.
|
||||||
|
|
||||||
|
**It fired on three real rows immediately — F12, F15, F21 — and all three
|
||||||
|
were `note`s.** A note is by definition a finding *without* a
|
||||||
|
reproduction (§3.1), so there is no measurement to be sensitive about;
|
||||||
|
requiring one would be asking for a sensitivity statement about nothing.
|
||||||
|
**That exclusion belongs in the check, not in three hand-written
|
||||||
|
exemptions**, and putting it there is the difference between a rule and a
|
||||||
|
pile of special cases.
|
||||||
|
|
||||||
|
**Mutation-proven on a real row.** Strip every sensitivity word from F17's
|
||||||
|
prose and the check goes red; restore it and green. The first mutation
|
||||||
|
attempt *failed to fire* because F17's text still said "one number varied"
|
||||||
|
— which is the check working, and worth recording as the moment it was
|
||||||
|
nearly mis-verified.
|
||||||
|
|
||||||
|
Four self-test controls: it catches a bare claim, **clears one that names
|
||||||
|
its variable** (without which it would fire on everything), skips
|
||||||
|
non-arithmetic kinds, and skips notes.
|
||||||
|
|
||||||
## Task: evidence
|
## Task: evidence
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: CB-WP-0030-T04
|
id: CB-WP-0030-T04
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
|
state_hub_task_id: "7206661a-a046-47bb-8f73-4b56328151af"
|
||||||
```
|
```
|
||||||
|
|
||||||
`evidence/CB-EV-0028-*.md`.
|
`evidence/CB-EV-0028-*.md`.
|
||||||
|
|
@ -155,3 +203,22 @@ priority: medium
|
||||||
deferred.
|
deferred.
|
||||||
- **Whether a lint that checks presence is worth having at all**, or
|
- **Whether a lint that checks presence is worth having at all**, or
|
||||||
whether it trains the eye to stop at green.
|
whether it trains the eye to stop at green.
|
||||||
|
|
||||||
|
**Done 2026-08-07.**
|
||||||
|
[CB-EV-0028](../evidence/CB-EV-0028-a-number-that-does-not-move.md).
|
||||||
|
|
||||||
|
**Yield 3 clear + 1 partial of 7** — above the falsifier, and the two it
|
||||||
|
catches outright are the two that travelled furthest. Row 6 is recorded as
|
||||||
|
*partly* rather than as a catch, because the variation would have
|
||||||
|
*prompted* the question, not *tested* it.
|
||||||
|
|
||||||
|
**The untagged surface is worth 1 of 7 against this family** and is still
|
||||||
|
worth closing for drift — recorded as separate, unclosed work.
|
||||||
|
|
||||||
|
**The presence-only lint is worth having on one condition**: it must never
|
||||||
|
grow a claim it cannot cash. The day its output implies verification,
|
||||||
|
ADR-0006 D3 applies and it should be deleted.
|
||||||
|
|
||||||
|
**§6 records a vocabulary defect in this very workplan** — it shipped
|
||||||
|
`kind: process` against `product|meta|mixed` and `make self-tests` caught
|
||||||
|
it. The pass that added a gate was caught by an older one.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue