T11: dated rates and staleness become data, ahead of the 2026-08-31 flip

The price sheet had two defects of one shape -- a schema that could not
hold the fact it needed, the same criticism the cost survey levelled at
the State Hub.

  CA-16  time-boxed rates are DATA. Sonnet's intro price lived in a
         `# intro ...` comment and was invisible to the collector that
         reads the file. Now promo_input/promo_output/promo_until,
         applied per response at its own timestamp.
  CA-17  the 90-day staleness rule was prose in MetricsAndScenarios 1a
         that every M-D2-CST verdict silently inherited. Now `recorded`
         + `max_age_days` in the sheet, and a stale sheet ABORTS.

Both are exercised by make cost-test: the promo rate must apply before
2026-08-31 and lapse after, and a 102-day-old sheet must trip.

Applying CA-16 moved AC-1 from $93.32 to $93.15 -- the $0.17 CB-EV-0002
predicted, now collected rather than noted. That is a legitimate
retarget under T07's distinction: the instrument disproved the target,
and its output is in this commit. The number has now been stated five
times ($248.46, $92.21, $92.87, $93.32, $93.15), each correction from a
different mechanism.

Evidence tables regenerated from the tool rather than hand-patched,
per CA-15 -- which is the rule that exists because hand-typed tables
were the only thing the adversarial review found wrong.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 09:23:29 +02:00
parent 06628e83e1
commit db731aa0bc
6 changed files with 138 additions and 32 deletions

View file

@ -67,6 +67,16 @@ cost = input_tokens × price.input
> reported as an unpriced line with its token counts, never dropped and
> never priced at a default.
> **CA-16.** Promotional and other time-boxed rates are **data**, not
> comments. A response is priced at the rate in force *at its own
> timestamp*: `promo_input`/`promo_output` apply on or before
> `promo_until`, and the standard rate applies after.
> **CA-17.** The price sheet carries `recorded` and `max_age_days`, and a
> sheet older than `max_age_days` **aborts the run**. A stale sheet
> invalidates `better` verdicts, so the staleness rule is a command rather
> than a sentence.
### 1.3 Scope of a measurement
> **CA-06.** A measurement enumerates **every** transcript for the repo:
@ -125,9 +135,9 @@ Each row names the command that produces its number, per InnerLoop §Step 4.
| ID | Metric | Target | Instrument |
|---|---|---|---|
| **AC-1** | reproduces the pinned CB-WP-0001 total | **$93.32** = $92.21 main + $1.11 subagent | `make cost-pin` |
| **AC-1** | reproduces the pinned CB-WP-0001 total | **$93.15** = $92.03 main + $1.11 subagent | `make cost-pin` |
| **AC-2** | reconciliation residual (CA-14) | **$0.00** exactly | same command, `reconciled: ok` line |
| **AC-3** | unattributed share reported (CA-10) | present, and **33%** on the pinned run | `cb-cost --pin fc76445 --by-task` |
| **AC-3** | unattributed share reported (CA-10) | present, and **32.4%** on the pinned run | `cb-cost --pin fc76445 --by-task` |
| **AC-4** | composition reported (CA-13) | all five components present | `cb-cost --pin fc76445 --composition` |
| **AC-5** | dedup invariant asserted (CA-02) | violation exits non-zero | `make cost-test` |
| **AC-6** | positive control: refuses to report on zero responses | exits non-zero | `make cost-test` |
@ -145,7 +155,7 @@ assertion against a fixture whose expected value is known and fails loudly;
Per InnerLoop §Step 4, the acceptance table is checked against the
contracts in this same spec:
- AC-1's $93.32 is reachable only if CA-06 holds (both trees enumerated).
- AC-1's $93.15 is reachable only if CA-06 holds (both trees enumerated).
Under a main-file-only collector the target is unreachable — this is the
defect the adversarial review caught, where a target of $92.21 would have
been hit *only* by a broken collector.
@ -228,6 +238,31 @@ signal — which is exactly the dead-policy failure this replaces, one level
up. The mitigation is that it costs nothing to run and is on the one
command surface.
## 9. The price sheet
`benchmarks/baselines/model-prices.toml` is the single source of rates.
Two defects were fixed here, both of the same shape — **a schema that
could not hold the fact it needed**, which is the criticism this
capability's survey levelled at the State Hub:
1. **A time-boxed rate lived in a comment.** Sonnet's intro price
(2.00/10.00 through 2026-08-31) was written as `# intro …` and was
therefore invisible to the collector, which used the standard rate. Cost
at the pin: **$0.17**, 0.19% of the pass. The real hazard was the
date — on **2026-08-31** the promotion expires and a comment-only
representation would have been wrong in the opposite direction, with
nothing to notice. Now expressed as `promo_*` fields and applied per
CA-16.
2. **The 90-day staleness rule had no check.** It was prose in
`MetricsAndScenarios.md` §1a and every M-D2-CST verdict silently
inherited it. Now `recorded` + `max_age_days` in the sheet, enforced by
CA-17 and exercised by `make cost-test`.
Applying CA-16 moved AC-1 from $93.32 to **$93.15**. This is a *legitimate*
retarget under the distinction CB-WP-0003 T07 draws: the instrument
disproved the target, rather than the implementation missing it. The
instrument's output is in the same commit.
## 8. Revisions to M-D2-CST
`specs/MetricsAndScenarios.md` §1a is superseded by this spec. M-D2-CST is