coulomb/clay-borg
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Replace the inlined price sheet with a pointer to the live file

Browse source 
The copy in MetricsAndScenarios 1a went stale within hours of T11: it
still showed the sonnet intro rate as a comment after the real sheet
moved dated rates into data. A spec that duplicates a data file will
always drift from it; the copy is now a pointer that says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-07-31 09:27:31 +02:00
parent 9e30bbb2b3
commit 4580438f1c
1 changed files with 7 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

29
specs/MetricsAndScenarios.md
View file

@ -94,27 +94,11 @@ benchmarks/baselines/model-prices.toml # the price sheet, updated when prices
``` ```
```toml ```toml
# USD per million tokens; source: Anthropic pricing, recorded 2026-07-31 # The live sheet is benchmarks/baselines/model-prices.toml — read it,
[claude-fable-5] # do not copy it. An inlined copy here went stale within hours: it still
input = 10.00 # showed the sonnet intro rate as a `#` comment after CB-WP-0003 T11 made
output = 50.00 # dated rates into data (promo_input / promo_output / promo_until), and
# added `recorded` + `max_age_days` for the staleness check (CA-16/CA-17).
[claude-opus-5]
input = 5.00
output = 25.00
[claude-sonnet-5]
input = 3.00 # intro 2.00 through 2026-08-31
output = 15.00 # intro 10.00 through 2026-08-31
[claude-haiku-4-5]
input = 1.00
output = 5.00
[cache] # multipliers on the input price
read = 0.1
write_5m = 1.25
write_1h = 2.0
``` ```
Rules: Rules:
@ -135,7 +119,8 @@ Rules:
failed pass scores nothing. failed pass scores nothing.
- Prices are `adopted:anthropic-pricing` with a recorded date; a stale sheet - Prices are `adopted:anthropic-pricing` with a recorded date; a stale sheet
(> 90 days or known price change) invalidates new `better` verdicts on (> 90 days or known price change) invalidates new `better` verdicts on
M-D2-CST until refreshed. M-D2-CST until refreshed. **Now enforced** — CA-17 aborts the run rather
than leaving this as prose every verdict silently inherited.
Determinism is not a metric but an **invariant**: N replays of the same Determinism is not a metric but an **invariant**: N replays of the same
seed and command log must produce bit-identical state hashes. Invariant seed and command log must produce bit-identical state hashes. Invariant