tegwick
e008b1e787
Collector per ADR-0003: enumerates every transcript including the subagents/ tree, dedups by requestId, prices per model and per cache TTL, attributes on (prev_commit, this_commit] intervals, and reconciles to the cent or aborts. The positive control caught a real defect on its very first run against real data, which is the entire argument for writing it: CA-02 assumed usage is identical across the lines of one requestId. True in the main transcript (206/206 groups, verified twice — by the survey and by the adversarial reviewer). FALSE in the subagents/ tree, where output_tokens is a running count: one response reads 5, 5, 195 across its three lines. First-wins scored it at 5. So CA-02 now splits: input-side counters are charged once and must be identical (assertion retained); output_tokens resolves to the max (CA-02a). AC-9 pins the exact 5,5,195 case as a regression test. The acceptance target moved again as a result, for the third time: $248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same first-wins method the tool just disproved, so the tool failing its target was the tool being correct. Target updated, not the tool. Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual $0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10. make cost / cost-test / cost-pin wired; cost-test added to `make all` and to CI, where it gates the collector's assertions without needing transcripts present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
45 lines
1.7 KiB
YAML
45 lines
1.7 KiB
YAML
name: ci
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
|
|
jobs:
|
|
check:
|
|
runs-on: docker
|
|
container:
|
|
image: rust:1.97
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- run: rustup component add rustfmt clippy
|
|
- run: cargo fmt --all --check
|
|
- run: cargo clippy --workspace --all-targets -- -D warnings
|
|
|
|
# The shipped runtime is a distinct configuration (AM-4a); it must
|
|
# compile and lint on its own, or the feature split rots.
|
|
- run: cargo clippy -p games-ground --no-default-features -- -D warnings
|
|
|
|
- run: cargo test --workspace
|
|
|
|
# No `|| test $? -eq 2`: cb-sim now fails on an unregistered game
|
|
# prefix and on a run that executed nothing. A silent skip is the
|
|
# failure this step exists to catch.
|
|
- run: cargo run -q -p cb-sim -- scenarios/ground/*.yaml
|
|
|
|
# AM-1: every numbered GR-rule is claimed by a scenario, and no
|
|
# scenario claims a rule the spec does not define.
|
|
- run: make coverage
|
|
|
|
# AM-4a/AM-4b: third-party source under audit, per configuration.
|
|
- run: make dep-weight
|
|
|
|
# Positive control for the cost collector (AC-5..AC-9). Does not
|
|
# gate on a dollar figure — transcripts are not present in CI — but
|
|
# proves the collector still detects the failures it claims to.
|
|
- run: make cost-test
|
|
|
|
# InnerLoop v1.0 positive control, enforced rather than asserted in
|
|
# prose: --test runs every benchmark once, so a workload that
|
|
# stalls or produces the wrong event count fails the build instead
|
|
# of silently reporting throughput for work that never happened.
|
|
- run: cargo bench -p games-ground --bench synthetic -- --test
|