From 1e414d1c999dc639820b893184a057fa180fc55e Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 31 Jul 2026 00:48:04 +0200 Subject: [PATCH] InnerLoop v0.2: loop tiers with chaos roll, adversarial survey review, runnable-baseline option --- specs/InnerLoop.md | 78 ++++++++++++++++++++++++++---- specs/MetricsAndScenarios.md | 4 ++ workplans/CB-WP-0001-inner-loop.md | 11 ++++- 3 files changed, 81 insertions(+), 12 deletions(-) diff --git a/specs/InnerLoop.md b/specs/InnerLoop.md index db6f24a..84b4591 100644 --- a/specs/InnerLoop.md +++ b/specs/InnerLoop.md @@ -1,7 +1,9 @@ # The Inner Loop — Assimilate and Surpass -Status: **v0.1 draft** — becomes v1.0 only after surviving its first full -pass (CB-WP-0001-T09 retrospective). +Status: **v0.2 draft** — becomes v1.0 only after surviving its first full +pass (CB-WP-0001-T09 retrospective). v0.2 adds loop tiers with the chaos +roll, adversarial survey review, and the runnable-baseline option +(maintainer decision, 2026-07-31). Normative process for building every Clay-Borg capability. Referenced by all workplans. The loop's own optimization target is **agentic efficiency**: @@ -22,8 +24,35 @@ agent can judge its own work without a human in the iteration. ``` **Hard gate: no implementation code for a capability exists before its ADR -(step 3) is committed.** Steps 1–3 may be compressed into one session for -small capabilities, but their artifacts are never skipped. +(step 3) is committed.** + +## Loop tiers and the chaos roll + +Every work packet declares a tier before work starts. The tier sets how +heavy steps 1–3 are; steps 4–5 (spec with metrics, code loop with evidence) +are never skipped for code-producing work. + +| Tier | Weight of steps 1–3 | Structural trigger (forces at least this tier) | +|---|---|---| +| **L** | Full: separate survey, adversarial review, ADR | Creates a new capability port, or is named a high-leverage pass by the maintainer | +| **M** | Survey and ADR merged into one document; review optional | Touches a canonical interface, or adds/updates an external dependency | +| **S** | One provenance paragraph in the commit message | Everything else (utilities, fixes, refactors inside a boundary) | + +**The chaos roll.** After deriving the structural tier, roll d10 +(`shuf -i 1-10 -n 1`). On a **10**, the tier is instead picked uniformly at +random (`shuf -e S M L -n 1`), overriding the structural derivation — up or +down. Both rolls are recorded in the tier declaration +(`tier: M (structural L, chaos 10→M)`). Purpose: an occasional random +reweighting keeps the classification honest — arguing everything into S +stops paying off when audits can compare argued tiers against the random +sample — and occasionally forces a deep look at something "obviously +trivial", which is where local optima hide. + +Chaos limits: a rolled-down tier relaxes *process* weight only. Invariants +(zero foreign types in canonical interfaces, determinism, passing +conformance suites) bind at every tier, and a rolled-down pass touching a +canonical interface still requires the interface change to be flagged in +the commit for retrospective review. ### Step 1 — Research @@ -33,13 +62,42 @@ The survey is done when it can name, per dimension, a concrete **benchmark-to-beat**: a number, a property, or a reproducible comparison — not an impression. -### Step 2 — Approve +**Runnable-baseline option.** For passes judged high-leverage (declared by +the maintainer or proposed in the survey and confirmed in the ADR), cited +numbers are not enough: the survey must ship a reproducible **baseline +harness** that runs the leading candidate on our machine against our +workload — the same scenario files where feasible. The harness ships with a +*fidelity note* stating what was and wasn't faithfully reproduced, so a +hastily wired competitor setup cannot silently inflate our advantage. +Where the option is not invoked (or the candidate isn't practically +runnable), comparisons against cited-only numbers are **directional**: the +evidence verdict for those rows caps at `parity`, never `better`. -An explicit recorded judgment inside the ADR: the survey is complete, the -candidates were the right ones, the baselines are trustworthy enough to -measure against. If not approvable, the loop returns to step 1 with the -named gap. Approval is cheap to record and expensive to skip — it is the -point where "we looked at X" becomes contestable. +### Step 2 — Approve (adversarial review) + +For tier-L passes, approval is earned through an **adversarial review**: a +separate session (or agent), given only the survey document, attempts to +break it — an omitted candidate, a stale or unverifiable benchmark, an +unmeasured claim presented as measured. Exactly **one round**: challenge, +then response. The survey is approvable only when every challenge is either +answered with evidence or conceded and folded into the survey. + +**Documentation requirement:** the research process, the challenge, and the +resulting improvements to the research are each documented in timestamped +markdown files under `history/`: + +```text +history/YYMMDD--research.md # how the survey was conducted: sources, + # queries, what was measured vs cited, dead ends +history/YYMMDD--challenge.md # the adversarial attack, verbatim +history/YYMMDD--response.md # answers/concessions and what changed in the survey +``` + +The polished survey artifact remains `research/CB-RES-NNNN-.md`; the +history files preserve the unpolished trail so a later reader can judge how +hard the survey was actually tested. For tier-M passes the review is +optional but, when performed, follows the same format. If not approvable +after the round, the loop returns to step 1 with the named gaps. ### Step 3 — Decide diff --git a/specs/MetricsAndScenarios.md b/specs/MetricsAndScenarios.md index 45e9707..d5f8c2b 100644 --- a/specs/MetricsAndScenarios.md +++ b/specs/MetricsAndScenarios.md @@ -119,6 +119,10 @@ source = "boardgame.io core, cloc, cited from CB-RES-0001" - Comparisons are same-machine where `machine` is set; cross-machine numbers are marked `provenance = cited` and treated as directional. + Per the runnable-baseline option in [InnerLoop.md](InnerLoop.md) §Step 1, + evidence rows compared only against cited numbers cap their verdict at + `parity`; a `better` verdict requires a locally measured baseline from a + fidelity-noted harness. - Regression rule (adopted:criterion): a merge-blocking regression is >3% on any D3 metric against **our own** last evidence file, independent of the SOTA baseline. diff --git a/workplans/CB-WP-0001-inner-loop.md b/workplans/CB-WP-0001-inner-loop.md index 947f0a6..4caea4b 100644 --- a/workplans/CB-WP-0001-inner-loop.md +++ b/workplans/CB-WP-0001-inner-loop.md @@ -87,7 +87,11 @@ template. Survey at minimum: boardgame.io, Tabletop Simulator's scripting model, an event-sourcing kernel, and one ECS-centric approach. For each: data model, mutation mechanism, determinism/replay story, hidden-information handling, simultaneous-action handling, measured or cited performance. -Conclude with the named benchmark-to-beat per dimension. +Conclude with the named benchmark-to-beat per dimension. This pass is +**tier L** and **high-leverage**: the runnable-baseline option is invoked — +ship a fidelity-noted local harness for the leading runnable candidate +(boardgame.io expected). Document the research trail in +history/YYMMDD-game-kernel-research.md per InnerLoop §Step 2. ## Task: Approval and implementation decision (ADR) @@ -98,7 +102,10 @@ priority: high state_hub_task_id: "2024c17d-9a37-48d2-8cb6-0fb3d8a19691" ``` -Record decisions/ADR-0002-game-kernel.md: survey approved, benchmark-to-beat +Adversarial review first (InnerLoop §Step 2): a separate session attacks +the survey; challenge and response are committed as +history/YYMMDD-game-kernel-challenge.md and -response.md, one round. Then +record decisions/ADR-0002-game-kernel.md: survey approved, benchmark-to-beat named, and the assimilate/reimplement/hybrid choice made with the expected advantage stated per dimension. The gate: no kernel code before this ADR is committed.