Commit graph

11 commits

Author SHA1 Message Date
6487d33f27 CB-WP-0023: SOLVE is legal only where it can do something
Some checks failed
ci / check (push) Failing after 4s
Implements ground-game's ruling of 2026-08-03. make all exits 0, 26
scenarios, rule coverage 59/59, and no scenario encoded the bug.

The rule ended up somewhere other than where I put it, and a gate moved
it. It went into legal_commands first; the AM-1 coverage gate then
demanded a scenario for the new GR-P05, and scenarios drive validate, not
the offer layer. A rule enforced only by the offer is enforced only for
clients that ask what is legal -- the browser would be filtered and a
scenario file would walk straight past it. Once GR-P05 moved into
validate, every condition in legal_commands was dead code, and the
layering test said so in those words.

And the reported case was not the one I reported. CB-WP-0018 and the
message to ground-game described SOLVE offered on a FACE-DOWN Problem.
Measured: validate already rejected face-down, so it never was offered.
Problem 1 is the Surface Problem, face-up from the deal -- the
maintainer's three inert SOLVEs were the HAND case, holding no Clarify
for a Clarify Problem. The ruling covers both so nothing is invalidated,
but the record was wrong.

Four conditions asserted separately, because one 'SOLVE is filtered' test
would pass with three of four implemented.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:18:40 +02:00
327bda64ab CB-WP-0006 T07: implement K18, amend K14
Two rules, two different answers, which is the point of a task phrased
"implement, or amend and say why".

K18 is implemented. "Criterion benches driving the same scenario format at
scale" was false — the bench hardcoded its commands and never touched
ScenarioFile, while MetricsAndScenarios §3 pointed at a benchmarks/
directory containing only baselines/. benchmarks/synthetic-3p.yaml now
holds the workload and both the bench and bench_shape read it: the
workload is data, not code.

A second defect surfaced while fixing the first. After the bench switched
to the file, bench_shape still hardcoded the same sequence, so the
workload existed twice — deleting end_round from the YAML broke bench-test
while bench_shape kept passing. Duplicated-fact drift in executable form.
Both now read the same include_str! and deleting a command breaks both.

Explicitly not claimed: this does not unblock AM-3. AM-3's baseline is a
declarative game object — moves, turn order, rules. synthetic-3p.yaml is a
command list; the rules live in games/ground. Marking it as AM-3's subject
would compare a script to a game definition, which is the category error
AM-3 is blocked on. The file says so in its own header, where the next
person will be tempted.

K14 is amended. CommitWindow had zero non-test users and GROUND enforces
the same contract inline. Wiring GROUND through it was rejected: it would
change the serialized shape of `selections`, which four scenario files
assert by dot-path and every state hash depends on, for the sole benefit
of making a sentence literally true.

The deciding argument is INTENT's, not convenience: abstractions are
extracted from working games rather than invented in isolation, and no
concept becomes canonical until it survives a second concrete use.
CommitWindow was invented before any game needed it and has survived none.
Imposing it on GROUND would manufacture the first use rather than discover
it. So K14 states what is actually guaranteed, CommitWindow is marked
provisional in the source, and it carries a delete-by date of 2026-12-31.

Kernel spec->code link 16/18 -> 18/18, stated with the caveat the gate
prints every run: that is about names, not assertions.

Two self-tests broke and both broke correctly. rule-coverage's gate test
hardcoded "unlinked rules exist today" and failed when the last one was
linked; it now computes that and asserts the gate fails iff rules are
unlinked. facts' text check rejected k_unlinked once it became
legitimately empty; empty now renders as "(none)" and the check
distinguishes absent from empty.

M-D1-MUT: 8 of 14, unchanged — K14 and K18 are kernel rules, not
acceptance rows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 12:47:16 +02:00
1edadac9a2 CB-WP-0006 T06: K10 replay bundles, --replay, and AM-7 re-earned
INTENT design decision 8 of 10, unimplemented for six passes. cb-sim had
no flag parsing at all, so --replay had nowhere to go.

The bundle is manifest + commands.log + initial.snapshot + expected.yaml,
dev-only behind the scenarios feature and charged to AM-4b. The command
stream goes through the K11 framing built in T05, so a truncated bundle is
detected rather than replayed short — the two tasks compose rather than
duplicating.

The reviewer's D2 correction was real: this was not "a directory of four
files". Pass carried only the end state, RunOutcome::Failed was a
formatted String, and scenario.rs created an EventLog, appended to it and
never read it. All three had to change.

The first round trip failed to reproduce, and the cause is worth keeping:
state_hash_hex over a serde_json::Value is a different canonical form than
over the typed aggregate — Value's map is key-sorted, a struct serializes
in declaration order. The bundle was written with one basis and verified
with the other. A round trip written to recompute its own comparison value
would have PASSED this bug; it failed because the recorded hash came from
the producing process, which is control 2's entire purpose.

make replay-test implements ADR-0005 §6's four controls, 14/14: a
committed deliberately-failing fixture outside the corpus with covers: []
so it neither fails `make sim` nor inflates AM-1; a tampered recorded hash
must fail; a log short by one byte and a corrupted length prefix must be
rejected; and a mutated manifest seed must fail — which bites only because
replay re-derives the initial state from seed+setup and checks it against
the recorded snapshot, since restoring from the snapshot alone would leave
the seed inert. Plus a control on the controls: the bundle must still
replay after every mutation is reverted.

AM-7's hash-identical clause is re-earned. The probe records a hash per
per-game segment and replays each from its own genesis; folding from the
wrong seed now fails. That is the clause ADR-0005 §4 withdrew as
mutation-proven inert. The scaling >= 0.9x clause is still unenforced, so
AM-7 stays PARTIAL — reported, not rounded up.

Kernel coverage 15/18 -> 16/18. facts-check immediately caught the spec's
copy of that number going stale, on a number that moved the same hour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 11:05:37 +02:00
bb35fcb168 CB-WP-0005 T01: spec->code link over every numbered spec and every crate
AGGREGATE becomes a list of source roots and rule patterns become
per-spec, so the link runs over every numbered spec x every crate rather
than GroundRules.md x games/ground/src/lib.rs.

The prediction held on the first run:

  AM-1b kernel spec->code link: 15/18 (83%) across 10 source files
    unlinked: K10 K14 K18

Kernel rules are link-only by design, and the output says so: they are
kernel invariants with no aggregate, setup preset or command vocabulary,
so scenarios/kernel/*.yaml with covers: [K11] would be a tag in a
directory the runner cannot dispatch. Claiming scenario coverage for them
is the inflation this gate exists to prevent.

Per ADR-0005 §5 the kernel arm reports without feeding the exit code
until 2026-08-31, then binds — the date in the tool, not in prose, with
days remaining printed every run, because open-ended "gate it later" is
how AM-4's targets went unratified for four workplans. The self-test
asserts the gate returns 0 before that date and 2 after.

The zero-rules positive control is replicated on the new denominator: a
kernel regex that stops matching aborts rather than printing 0/0 as
though it were 100%.

The self-test passed while the tool was completely broken. A print(
inside say() became say(), so every real `make coverage` died with
RecursionError while --self-test reported all-ok — it only ever called
kernel_arm(quiet=True) and never executed the reporting path. The control
named the behaviour and did not assert it, which is precisely what this
workplan is about. Fixed by exercising the loud path and asserting it
prints, then verified by re-breaking say() and confirming both new checks
go red. Seventh instance of the harness-does-nothing shape, in the tool
written to find that shape.

Also caught by its own gate: a self-test label that printed "0 K-ids"
beside a passing ">5" assertion, because the detail string rebuilt the
pattern with different escaping. A label that contradicts its own check
is worse than no label.

k_rules, k_linked and k_unlinked are registered facts under facts-check.
A limit of that checker is recorded rather than patched: it is
line-based, so a tagged value that prose-wraps fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 16:54:07 +02:00
53c460c992 CB-WP-0004 T04: fact registry and make facts-check — DFD gets a gate
Duplicated-fact drift is the fourth error class and the only one with no
executable rule. No positive control catches it (both copies are
internally consistent) and re-derivation does not either (the copy
reproduces whatever it was copied from). It is caught only by reading a
copy against its source, which nothing in the loop required.

facts.toml holds 15 facts and is GENERATED by `make facts-gen` from
cb-cost, dep-weight and rule-coverage. The trap this task named — a
hand-maintained registry that becomes another drifting copy — is closed
by facts-check re-running the instruments and failing when the committed
registry disagrees with them. A stale registry cannot certify stale
artifacts.

An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key.
17 occurrences across 5 artifacts are now checked.

Falsified before being believed: changing CostAccounting.md line 158 from
$93.15 to $92.87 — the exact historical drift — produced exit 1 naming
the file, the line and the expected value. Tested against the class it
exists to catch, on a real artifact, not only in its self-test.

It then caught a live tag inside its own documentation example in
InnerLoop.md within the hour. Third time a gate has failed on its own
pass's work.

What it does not close is stated rather than implied: 22 untagged literal
copies remain and are reported, not failed. Tagging is opt-in, a number
can legitimately recur, and a gate that fires on coincidence gets routed
around. Naming the uncovered surface beats claiming the class is closed.

InnerLoop single-source-of-fact moves from prose to executable — v1.3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:24:39 +02:00
4580438f1c Replace the inlined price sheet with a pointer to the live file
The copy in MetricsAndScenarios 1a went stale within hours of T11: it
still showed the sonnet intro rate as a comment after the real sheet
moved dated rates into data. A spec that duplicates a data file will
always drift from it; the copy is now a pointer that says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:27:31 +02:00
9e30bbb2b3 T09: close the spec->code link mechanically (M-D1-LNK)
make coverage now reports a second number: how many claimed rules are
also NAMED in the aggregate source. 58/58 tag coverage was weaker
evidence than it read as, and this says how much weaker.

Measured: 49 of 58. Nine rules are claimed by a scenario and appear
nowhere in games/ground/src/lib.rs --
GR-D07 GR-F02 GR-L03 GR-O03 GR-P01 GR-P02 GR-P03 GR-P04 GR-T01.

The gate REPORTS rather than fails, on purpose. Closing the gap by
adding those IDs to comments would satisfy the check without
establishing that any of the nine is implemented -- the overclaim
InnerLoop implementation rule 2 exists to prevent, and one CB-WP-0001
already committed once. Each needs its implementation confirmed before
it is tagged; promoting M-D1-LNK to a failing gate is correct after
that, not before.

Also added: a phantom check that fails when a rule id appears in code
that the spec does not define (currently zero).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:27:14 +02:00
b96cd94a64 T03: specs/CostAccounting.md — 15 contracts, 8 metrics, each with a command
Normative spec for the cost model (dedup by requestId, per-model and
per-TTL pricing), the attribution contract (ending-at-commit intervals
scoped by sessionId), and the reported shape (composition, not a total).

Every acceptance row names the command that produces its number, per
InnerLoop Step 4. AC-5..AC-8 are the positive control: cb-cost
--self-test must fail on a dedup violation, on zero responses, on a
missing subagent tree, and on 5m cache priced at the 1h rate.

The feasibility check earns its place here: AC-1's $92.87 is reachable
only if CA-06 holds. The earlier $92.21 target was reachable only by a
collector with the exact blind spot the survey documented.

MetricsAndScenarios 1a superseded. Two of its rules struck through
rather than deleted, because both were wrong in instructive ways: "if
the cache split is unknown, count all input at full price" would have
priced 80.5M cache reads at 10x, and "the hub already records tokens"
named as a source what is only ever a lossy sink. M-D2-TOK demoted --
tokens are not comparable across models or cache states.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:46:12 +02:00
38ffd8b7fd Add M-D2-CST token-cost metric with committed model price sheet 2026-07-31 00:53:22 +02:00
1e414d1c99 InnerLoop v0.2: loop tiers with chaos roll, adversarial survey review, runnable-baseline option 2026-07-31 00:48:04 +02:00
5d1c1bc866 Phase A: codify inner loop (specs/InnerLoop.md) and metrics/scenario conventions 2026-07-31 00:31:14 +02:00