infospace-bench/src/infospace_bench
tegwick 3ca891de4a fix: review findings from Lefevre live smoke
Two small fixes informed by the 2026-05-18 live OpenRouter chapter-I run.

1. extract-entities templates (trading-literature and general-knowledge):
   the # Entity Title placeholder was interpreted by gpt-4o-mini as a
   literal heading prefix, so every entity came back as "# Entity Title:
   Bucket Shop" etc. The instruction now spells the placeholder out
   with concrete examples and an explicit "not the literal string"
   note, so smaller models hit the intended shape.

2. generate plan grows --model <id>. When supplied, the cost estimate
   pulls per-prompt and per-completion rates from the bundled
   model_rates.yaml instead of multiplying a single blended
   --cost-per-1k value across all tokens. The summary now also returns
   a separate estimated_completion_tokens field plus a cost_source tag
   ("rate_table:<model>" | "cost_per_1k_blended" | None).

This is a stopgap. LLM-WP-0005 (proposed in llm-connect this round)
will move the rate registry and token-shape problem classes upstream
so consumers stop re-implementing them.

The live smoke ran 28k prompt tokens / 7.5k completion / $0.0088
actual. With --model openai/gpt-4o-mini the plan estimate now lands at
$0.0076 (within 14% of actual) versus the prior $8.40 estimate at
--cost-per-1k 0.30.

181 tests pass, 2 skipped (both live OpenRouter smokes).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 04:30:33 +02:00
..
profiles fix: review findings from Lefevre live smoke 2026-05-19 04:30:33 +02:00
__init__.py IB-WP-0014: archive-list, restore, retention annotation, docs (T03-T05) 2026-05-17 11:46:23 +02:00
__main__.py Initial implementation 2026-05-14 11:32:25 +02:00
archive.py IB-WP-0019-T07: archive integration; close IB-WP-0019 2026-05-17 21:53:28 +02:00
budget.py IB-WP-0020-T03: routing CLI flags 2026-05-18 22:08:51 +02:00
checks.py Initial implementation 2026-05-14 11:32:25 +02:00
cli.py fix: review findings from Lefevre live smoke 2026-05-19 04:30:33 +02:00
engine.py engine and lifecycle 2026-05-14 16:26:42 +02:00
errors.py Initial implementation 2026-05-14 11:32:25 +02:00
evaluation.py eval history and metrics 2026-05-14 15:35:04 +02:00
evaluation_io.py fix(lifecycle): _relative_to_root path doubling with relative workspace 2026-05-19 03:26:55 +02:00
generation.py infospace pipeline for wealth of nations example 2026-05-14 18:04:38 +02:00
generator.py fix: review findings from Lefevre live smoke 2026-05-19 04:30:33 +02:00
history.py eval history and metrics 2026-05-14 15:35:04 +02:00
inspection.py Initial implementation 2026-05-14 11:32:25 +02:00
lifecycle.py fix(lifecycle): _relative_to_root path doubling with relative workspace 2026-05-19 03:26:55 +02:00
markdown_adapter.py acceptance matrix and workflow generation 2026-05-14 16:01:32 +02:00
model_rates.yaml IB-WP-0019-T03: rate-table cost computation 2026-05-17 19:54:30 +02:00
models.py Initial implementation 2026-05-14 11:32:25 +02:00
openrouter.py generic source-to-infospace generator 2026-05-14 19:33:22 +02:00
routing.py IB-WP-0020-T03: routing CLI flags 2026-05-18 22:08:51 +02:00
routing_config.py IB-WP-0020-T05: shadow-mode CLI flags; close IB-WP-0020 2026-05-18 23:30:36 +02:00
semantics.py entity relationship model 2026-05-14 15:06:17 +02:00
source_intake.py IB-WP-0016-T02: chapter-aware chunking and stable IDs 2026-05-17 15:52:47 +02:00
viability.py Initial implementation 2026-05-14 11:32:25 +02:00
workflow.py generic source-to-infospace generator 2026-05-14 19:33:22 +02:00