Commit graph

8 commits

Author SHA1 Message Date
bb35fcb168 CB-WP-0005 T01: spec->code link over every numbered spec and every crate
AGGREGATE becomes a list of source roots and rule patterns become
per-spec, so the link runs over every numbered spec x every crate rather
than GroundRules.md x games/ground/src/lib.rs.

The prediction held on the first run:

  AM-1b kernel spec->code link: 15/18 (83%) across 10 source files
    unlinked: K10 K14 K18

Kernel rules are link-only by design, and the output says so: they are
kernel invariants with no aggregate, setup preset or command vocabulary,
so scenarios/kernel/*.yaml with covers: [K11] would be a tag in a
directory the runner cannot dispatch. Claiming scenario coverage for them
is the inflation this gate exists to prevent.

Per ADR-0005 §5 the kernel arm reports without feeding the exit code
until 2026-08-31, then binds — the date in the tool, not in prose, with
days remaining printed every run, because open-ended "gate it later" is
how AM-4's targets went unratified for four workplans. The self-test
asserts the gate returns 0 before that date and 2 after.

The zero-rules positive control is replicated on the new denominator: a
kernel regex that stops matching aborts rather than printing 0/0 as
though it were 100%.

The self-test passed while the tool was completely broken. A print(
inside say() became say(), so every real `make coverage` died with
RecursionError while --self-test reported all-ok — it only ever called
kernel_arm(quiet=True) and never executed the reporting path. The control
named the behaviour and did not assert it, which is precisely what this
workplan is about. Fixed by exercising the loud path and asserting it
prints, then verified by re-breaking say() and confirming both new checks
go red. Seventh instance of the harness-does-nothing shape, in the tool
written to find that shape.

Also caught by its own gate: a self-test label that printed "0 K-ids"
beside a passing ">5" assertion, because the detail string rebuilt the
pattern with different escaping. A label that contradicts its own check
is worse than no label.

k_rules, k_linked and k_unlinked are registered facts under facts-check.
A limit of that checker is recorded rather than patched: it is
line-based, so a tagged value that prose-wraps fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 16:54:07 +02:00
53c460c992 CB-WP-0004 T04: fact registry and make facts-check — DFD gets a gate
Duplicated-fact drift is the fourth error class and the only one with no
executable rule. No positive control catches it (both copies are
internally consistent) and re-derivation does not either (the copy
reproduces whatever it was copied from). It is caught only by reading a
copy against its source, which nothing in the loop required.

facts.toml holds 15 facts and is GENERATED by `make facts-gen` from
cb-cost, dep-weight and rule-coverage. The trap this task named — a
hand-maintained registry that becomes another drifting copy — is closed
by facts-check re-running the instruments and failing when the committed
registry disagrees with them. A stale registry cannot certify stale
artifacts.

An artifact quoting a fact tags it: **$93.15** <!-- fact --> with the key.
17 occurrences across 5 artifacts are now checked.

Falsified before being believed: changing CostAccounting.md line 158 from
$93.15 to $92.87 — the exact historical drift — produced exit 1 naming
the file, the line and the expected value. Tested against the class it
exists to catch, on a real artifact, not only in its self-test.

It then caught a live tag inside its own documentation example in
InnerLoop.md within the hour. Third time a gate has failed on its own
pass's work.

What it does not close is stated rather than implied: 22 untagged literal
copies remain and are reported, not failed. Tagging is opt-in, a number
can legitimately recur, and a gate that fires on coincidence gets routed
around. Naming the uncovered surface beats claiming the class is closed.

InnerLoop single-source-of-fact moves from prose to executable — v1.3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 10:24:39 +02:00
4580438f1c Replace the inlined price sheet with a pointer to the live file
The copy in MetricsAndScenarios 1a went stale within hours of T11: it
still showed the sonnet intro rate as a comment after the real sheet
moved dated rates into data. A spec that duplicates a data file will
always drift from it; the copy is now a pointer that says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:27:31 +02:00
9e30bbb2b3 T09: close the spec->code link mechanically (M-D1-LNK)
make coverage now reports a second number: how many claimed rules are
also NAMED in the aggregate source. 58/58 tag coverage was weaker
evidence than it read as, and this says how much weaker.

Measured: 49 of 58. Nine rules are claimed by a scenario and appear
nowhere in games/ground/src/lib.rs --
GR-D07 GR-F02 GR-L03 GR-O03 GR-P01 GR-P02 GR-P03 GR-P04 GR-T01.

The gate REPORTS rather than fails, on purpose. Closing the gap by
adding those IDs to comments would satisfy the check without
establishing that any of the nine is implemented -- the overclaim
InnerLoop implementation rule 2 exists to prevent, and one CB-WP-0001
already committed once. Each needs its implementation confirmed before
it is tagged; promoting M-D1-LNK to a failing gate is correct after
that, not before.

Also added: a phantom check that fails when a rule id appears in code
that the spec does not define (currently zero).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 09:27:14 +02:00
b96cd94a64 T03: specs/CostAccounting.md — 15 contracts, 8 metrics, each with a command
Normative spec for the cost model (dedup by requestId, per-model and
per-TTL pricing), the attribution contract (ending-at-commit intervals
scoped by sessionId), and the reported shape (composition, not a total).

Every acceptance row names the command that produces its number, per
InnerLoop Step 4. AC-5..AC-8 are the positive control: cb-cost
--self-test must fail on a dedup violation, on zero responses, on a
missing subagent tree, and on 5m cache priced at the 1h rate.

The feasibility check earns its place here: AC-1's $92.87 is reachable
only if CA-06 holds. The earlier $92.21 target was reachable only by a
collector with the exact blind spot the survey documented.

MetricsAndScenarios 1a superseded. Two of its rules struck through
rather than deleted, because both were wrong in instructive ways: "if
the cache split is unknown, count all input at full price" would have
priced 80.5M cache reads at 10x, and "the hub already records tokens"
named as a source what is only ever a lossy sink. M-D2-TOK demoted --
tokens are not comparable across models or cache states.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:46:12 +02:00
38ffd8b7fd Add M-D2-CST token-cost metric with committed model price sheet 2026-07-31 00:53:22 +02:00
1e414d1c99 InnerLoop v0.2: loop tiers with chaos roll, adversarial survey review, runnable-baseline option 2026-07-31 00:48:04 +02:00
5d1c1bc866 Phase A: codify inner loop (specs/InnerLoop.md) and metrics/scenario conventions 2026-07-31 00:31:14 +02:00