Replace the inlined price sheet with a pointer to the live file
The copy in MetricsAndScenarios 1a went stale within hours of T11: it still showed the sonnet intro rate as a comment after the real sheet moved dated rates into data. A spec that duplicates a data file will always drift from it; the copy is now a pointer that says so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
9e30bbb2b3
commit
4580438f1c
1 changed files with 7 additions and 22 deletions
|
|
@ -94,27 +94,11 @@ benchmarks/baselines/model-prices.toml # the price sheet, updated when prices
|
|||
```
|
||||
|
||||
```toml
|
||||
# USD per million tokens; source: Anthropic pricing, recorded 2026-07-31
|
||||
[claude-fable-5]
|
||||
input = 10.00
|
||||
output = 50.00
|
||||
|
||||
[claude-opus-5]
|
||||
input = 5.00
|
||||
output = 25.00
|
||||
|
||||
[claude-sonnet-5]
|
||||
input = 3.00 # intro 2.00 through 2026-08-31
|
||||
output = 15.00 # intro 10.00 through 2026-08-31
|
||||
|
||||
[claude-haiku-4-5]
|
||||
input = 1.00
|
||||
output = 5.00
|
||||
|
||||
[cache] # multipliers on the input price
|
||||
read = 0.1
|
||||
write_5m = 1.25
|
||||
write_1h = 2.0
|
||||
# The live sheet is benchmarks/baselines/model-prices.toml — read it,
|
||||
# do not copy it. An inlined copy here went stale within hours: it still
|
||||
# showed the sonnet intro rate as a `#` comment after CB-WP-0003 T11 made
|
||||
# dated rates into data (promo_input / promo_output / promo_until), and
|
||||
# added `recorded` + `max_age_days` for the staleness check (CA-16/CA-17).
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
|
@ -135,7 +119,8 @@ Rules:
|
|||
failed pass scores nothing.
|
||||
- Prices are `adopted:anthropic-pricing` with a recorded date; a stale sheet
|
||||
(> 90 days or known price change) invalidates new `better` verdicts on
|
||||
M-D2-CST until refreshed.
|
||||
M-D2-CST until refreshed. **Now enforced** — CA-17 aborts the run rather
|
||||
than leaving this as prose every verdict silently inherited.
|
||||
|
||||
Determinism is not a metric but an **invariant**: N replays of the same
|
||||
seed and command log must produce bit-identical state hashes. Invariant
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue