diff --git a/workplans/CB-WP-0005-assertion-coverage.md b/workplans/CB-WP-0005-assertion-coverage.md index 30a2509..577878e 100644 --- a/workplans/CB-WP-0005-assertion-coverage.md +++ b/workplans/CB-WP-0005-assertion-coverage.md @@ -245,7 +245,7 @@ downgrade inline, so a reader of the spec cannot reach the old claim. ```task id: CB-WP-0005-T04 -status: todo +status: deferred priority: high state_hub_task_id: "35876b28-97ac-4379-bb71-72e74c4116d4" ``` @@ -273,7 +273,7 @@ the length prefix → reject. K11's operative clause is *detection*. ```task id: CB-WP-0005-T05 -status: todo +status: deferred priority: high state_hub_task_id: "8592b6e7-8b83-4b47-b310-90e16a35eb6a" ``` @@ -308,7 +308,7 @@ charged to **AM-4b** (9.4% headroom), not AM-4a. ```task id: CB-WP-0005-T06 -status: todo +status: deferred priority: medium state_hub_task_id: "774b1c8a-71c1-4e0a-91b1-0247a3e397cf" ``` diff --git a/workplans/CB-WP-0006-instrument-the-table.md b/workplans/CB-WP-0006-instrument-the-table.md new file mode 100644 index 0000000..3f62aa9 --- /dev/null +++ b/workplans/CB-WP-0006-instrument-the-table.md @@ -0,0 +1,231 @@ +--- +id: CB-WP-0006 +title: "Instrument the acceptance table, then implement what it exposes" +status: proposed +--- + +# Purpose + +CB-WP-0005 T02 measured M-D1-MUT for the first time: + +```text + M-D1-MUT: 4/14 rows enforced + PARTIAL 2 (AM-7, AM-8 — some clauses live, some inert) + unmutatable 8 (no property to invert) +``` + +**4 of 14, against a predicted 10 of 14.** The prediction was wrong by +2.5×, and wrong about *what* as well as *how much*: CB-RES-0004 diagnosed +three absent kernel rules, and the measurement found that **more than half +the acceptance table has no instrument behind it at all**. + +CB-WP-0005 Phase C was deferred before starting on that basis (T02's stop +condition, maintainer decision 2026-07-31). It was scoped to five rules — +K9, K10, K11, K14, K18 — which does not address the eight. + +This workplan is scoped to what was measured rather than what was +predicted. **Phase A instruments the unmeasured rows; Phase B implements +the kernel work inherited from CB-WP-0005 T04–T06.** That order is the +whole point: the previous pass built a fix before its instrument existed, +and the instrument then contradicted the fix's premise. + +Per InnerLoop §Step 4, no target moves in the commit that measures it. + +## Phase A — instrument the rows that measure nothing + +## Task: AM-6 — assert the throughput target + +```task +id: CB-WP-0006-T01 +status: todo +priority: high +``` + +The sharpest single finding of CB-WP-0005. **Nothing in the workspace +compares any number to 100,000 events/s** — the project's headline +throughput claim, reported as `met, 16.5×`. The Criterion bench emits +throughput; the only asserts in `synthetic.rs` are the stress-gate shape +and the events-per-round pin. + +Deliver a standing assertion — a test, not a bench — that measures applied +events/s on the synthetic workload and fails below the AM-6 target. + +**The trap, stated up front:** a timing assertion in CI is flaky by +nature, and the usual response is to loosen it until it never fires, which +would reproduce exactly the defect being fixed. So the threshold is +**pinned well below the measured 1.65M/s** (AM-6 asks for 100k), the +assertion states its own headroom in the failure message, and a loosening +requires an ADR. A gate tuned until it cannot fire is a gate that counts +names. + +**Verified by:** `make mutation-check --row AM-6` goes from `unmutatable` +to `red`. + +## Task: AM-2, AM-3 — instrument the size metrics + +```task +id: CB-WP-0006-T02 +status: todo +priority: medium +``` + +AM-2 (≤40 spec lines per rule) and AM-3 (≤50 LOC for the synthetic +workload) are both `unmutatable` for the same reason: `make loc` prints +line counts and nothing divides, compares, or gates. + +AM-2 is the **anti-gaming pair** for AM-1 — 100% rule coverage means +nothing if the rules are trivially small — so an uninstrumented AM-2 +leaves AM-1 gameable, which is the specific risk MetricsAndScenarios §1 +names when it introduces M-D1-SPL. + +AM-3 depends on K18 (benches driven from scenario files); if K18 resolves +toward amending the spec rather than implementing it, AM-3 must be +restated or withdrawn with an argument rather than left unmeasured. + +## Task: AM-5, AM-9 — measure or withdraw, but stop leaving them blank + +```task +id: CB-WP-0006-T03 +status: todo +priority: medium +``` + +AM-5 (clean build ≤60 s) is `recorded not gated` — and not recorded +either. AM-9 (peak RSS ≤64 MB) is declared "very unlikely to bind", which +CB-EV-0001 itself calls out as an unmeasured judgment call. + +Both are cheap to measure: one `time` around a clean build, one +`/usr/bin/time -v` around the 100k-event run. **Or withdraw them with an +argument**, which is a legitimate outcome and possibly the right one for +AM-5 on a machine-dependent number. What is not legitimate is a third pass +leaving them blank while the scoreboard reports elsewhere. + +## Task: AM-4c — target it or drop it + +```task +id: CB-WP-0006-T04 +status: todo +priority: low +``` + +`reported, not targeted`, so there is no property to invert and it counts +against M-D1-MUT by ADR-0005 §1. + +Two honest options, and the task must pick one with an argument: give it a +threshold (making it enforceable), or **remove it from the acceptance +table** and keep it as a reported diagnostic. A row in an acceptance table +that cannot fail is decoration, and this pass has learned to say so. + +Note the incentive: ADR-0005 already predicts own-source growth from the +kernel work in Phase B, so AM-4c will move against us. Setting its +threshold *after* seeing that movement would be the retarget InnerLoop +§Step 4 forbids — so it is set here, before Phase B, or not at all. + +## Phase B — the kernel work inherited from CB-WP-0005 + +> These three carry their full analysis in +> [CB-WP-0005](CB-WP-0005-assertion-coverage.md) Phase C, where they were +> written and then deferred unstarted. They are summarized here; read that +> file for the detail, the four replay controls, and the D2 correction on +> what the bundle writer actually requires. + +## Task: K9 and K11 — durable log, LogStore port, real conformance suite + +```task +id: CB-WP-0006-T05 +status: todo +priority: high +``` + +Inherited from CB-WP-0005 T04. K9's acceptance property (snapshot at N + +events N+1..M ≡ genesis fold) is mutation-proven unasserted; K11 has no +durable format at all. The `LogStore` port with a shared +`fn conformance(…)`, retro-fitted to `KernelRng`, is what +finally earns **AM-11** — currently downgraded to unmet because the pair +exists and the suite does not. + +**Verified by:** `make mutation-check --row AM-11` goes from +`unmutatable` to `red`. + +## Task: K10 — the replay bundle and `--replay` + +```task +id: CB-WP-0006-T06 +status: todo +priority: high +``` + +Inherited from CB-WP-0005 T05. INTENT design decision 8 of 10, +unimplemented. All four controls from ADR-0005 §6 are required verbatim, +including the two the adversarial reviewer supplied: truncate-by-one-byte +rejection, and a mutated-seed control so the round-trip is capable of +failing. + +Fixing this is also what lets **AM-7's withdrawn `hash-identical` +clause** be re-earned, since a real replay is what the clause was always +about. + +## Task: K14 and K18 — implement, or amend the spec and say why + +```task +id: CB-WP-0006-T07 +status: todo +priority: medium +``` + +Inherited from CB-WP-0005 T06. `CommitWindow` has zero non-test users; +the bench never touches `ScenarioFile`. Either wire them up or amend the +rules with a recorded argument. Deleting a rule to make a gate green is +forbidden; deleting one *with an argument* is legitimate and probably +right for K18. + +## Phase C — measure, and answer the question this pass inherited + +## Task: control loop — M-D1-MUT before and after + +```task +id: CB-WP-0006-T08 +status: todo +priority: high +``` + +Commit `evidence/CB-EV-0004`. The baseline is **4 of 14**, committed and +reproducible via `make mutation-check`. + +Three tests, all reported: + +1. **Did the enforced count rise, per row?** Against the per-task + predictions above, unmet reported as unmet. +2. **Did any row regress from `red` to `SURVIVED`?** The four currently + enforced rows are the ones with something to lose. +3. **Were the new mutations strong?** CB-WP-0005 T02 wrote two no-op + mutations on its first attempt, either of which would have published a + false accusation against working code. Every mutation added by this + pass must be shown to fail for the *stated* reason — record how that + was checked. + +## Task: retrospective — is a weak mutation the new grep? + +```task +id: CB-WP-0006-T09 +status: todo +priority: medium +``` + +The question CB-WP-0005 T08 raised and this pass is positioned to answer. + +CB-WP-0004 concluded that tooling recovers capacity only where it removes +the manual path. M-D1-MUT does **not** remove a manual path — writing a +weak mutation is exactly as easy as writing a strong one, and the harness +cannot tell the difference. CB-WP-0005 produced two no-op mutations on its +first attempt, so the failure mode is demonstrated, not hypothetical. + +**And it fails in the more dangerous direction:** a weak mutation does not +merely miss a defect, it *indicts working code*. A false `SURVIVED` sends +the next pass to fix something that is not broken. + +So: is M-D1-MUT a real instrument, or a name-counter with extra steps? If +the latter, say so and propose what would actually bind — the candidate +being that the mutation must be written by someone other than the author +of the assertion, which is the adversarial-review principle applied one +level down.