tegwick
e008b1e787
Collector per ADR-0003: enumerates every transcript including the subagents/ tree, dedups by requestId, prices per model and per cache TTL, attributes on (prev_commit, this_commit] intervals, and reconciles to the cent or aborts. The positive control caught a real defect on its very first run against real data, which is the entire argument for writing it: CA-02 assumed usage is identical across the lines of one requestId. True in the main transcript (206/206 groups, verified twice — by the survey and by the adversarial reviewer). FALSE in the subagents/ tree, where output_tokens is a running count: one response reads 5, 5, 195 across its three lines. First-wins scored it at 5. So CA-02 now splits: input-side counters are charged once and must be identical (assertion retained); output_tokens resolves to the max (CA-02a). AC-9 pins the exact 5,5,195 case as a regression test. The acceptance target moved again as a result, for the third time: $248.46 -> $92.21 -> $92.87 -> $93.32. AC-1 was computed by the same first-wins method the tool just disproved, so the tool failing its target was the tool being correct. Target updated, not the tool. Measured at the pin: $92.21 main + $1.11 subagent = $93.32, residual $0.000000. 32.5% UNATTRIBUTED, reported as its own line per CA-10. make cost / cost-test / cost-pin wired; cost-test added to `make all` and to CI, where it gates the collector's assertions without needing transcripts present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
54 lines
1.6 KiB
Makefile
54 lines
1.6 KiB
Makefile
# One command surface (InnerLoop §agentic-efficiency #3). Deterministic,
|
|
# greppable output; precursor of the `cb` CLI.
|
|
|
|
CARGO := cargo
|
|
|
|
.PHONY: check test sim bench bench-test coverage dep-weight cost cost-test cost-pin loc all
|
|
|
|
## fmt + clippy (deny warnings) + HashMap deny-lint
|
|
check:
|
|
$(CARGO) fmt --all --check
|
|
$(CARGO) clippy --workspace --all-targets -- -D warnings
|
|
|
|
## unit + scenario-format tests
|
|
test:
|
|
$(CARGO) test --workspace
|
|
|
|
## run all GROUND scenarios through cb-sim
|
|
dep-weight:
|
|
python3 tools/dep-weight.py
|
|
|
|
coverage:
|
|
python3 tools/rule-coverage.py
|
|
|
|
# M-D2-CST (specs/CostAccounting.md). cost-test is the positive control and
|
|
# runs first: a cost number from an unverified collector is void.
|
|
cost: cost-test
|
|
python3 tools/cb-cost.py --composition --by-task
|
|
|
|
cost-test:
|
|
python3 tools/cb-cost.py --self-test
|
|
|
|
cost-pin: cost-test
|
|
python3 tools/cb-cost.py --pin fc76445 --composition --by-task
|
|
|
|
sim:
|
|
$(CARGO) run -q -p cb-sim -- scenarios/ground/*.yaml
|
|
|
|
## Criterion benches (AM-6/AM-7)
|
|
bench:
|
|
$(CARGO) bench -p games-ground
|
|
|
|
## InnerLoop positive control: run every bench once, no measurement.
|
|
## Fails if a workload stalls or produces the wrong event count.
|
|
bench-test:
|
|
$(CARGO) bench -p games-ground --bench synthetic -- --test
|
|
|
|
|
|
## AM-2/AM-3 input: source LOC per crate (excludes tests would need tokei)
|
|
loc:
|
|
@for d in crates/cb-kernel crates/cb-events crates/cb-game-runtime games/ground tools/cb-sim; do \
|
|
printf '%-28s %s\n' $$d "$$(find $$d/src -name '*.rs' | xargs cat | grep -vcE '^\s*(//|$$)')"; \
|
|
done
|
|
|
|
all: check test sim coverage dep-weight cost-test bench-test
|