Add image-model selection guide from the intelligence-radar digest
Maps the 2026-10-08 image-generation decision tree onto OpenRouter model IDs with per-image quotes and caps for llm_connect.images, and documents the anchor-win-rate scoring that makes Elo leaderboards usable for Topnotch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 197486@bnt-lap001 Assistant-Session: 8e588643-5640-4ad7-9309-4ed3e12b2423
This commit is contained in:
parent
19fcb19e84
commit
7073d2543c
3 changed files with 371 additions and 281 deletions
88
docs/image-model-selection.md
Normal file
88
docs/image-model-selection.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
# Choosing an image model for `llm_connect.images`
|
||||
|
||||
`llm_connect.images` has no default image model and does not choose one (see
|
||||
[`contracts/functional/images.md`](../contracts/functional/images.md)). This
|
||||
guide is advisory input for the caller. It does not change that contract.
|
||||
|
||||
**Source:** intelligence-radar topic digest
|
||||
`digests/topics/2026-10-08-1527_Image_Generation_with_Generative_AI_Digest.md`
|
||||
(evidence observed 2026-10-08). Rankings and prices in this field change
|
||||
monthly. Re-derive the table from the digest's sources before relying on it
|
||||
after about 2026-11.
|
||||
|
||||
## Picking a model
|
||||
|
||||
All IDs below are OpenRouter model IDs. Prices are for one 1024² image. The
|
||||
quote is the per-image estimate plus some margin, and the cap leaves room for
|
||||
the provider-reported cost to come in above the quote.
|
||||
|
||||
| Need | Model | ≈ $/img | `--quote-usd` | `--max-cost-usd` |
|
||||
|---|---|---|---|---|
|
||||
| Best result, cost secondary (Frontier) | `openai/gpt-image-2.5-sunburst` | 0.21 | 0.25 | 0.30 |
|
||||
| Same quality, faster | `openai/gpt-image-2.5-flare` | 0.21 | 0.25 | 0.30 |
|
||||
| Near-frontier at scale (Efficient) | `google/gemini-nano-banana-2.1` | 0.034 | 0.04 | 0.08 |
|
||||
| Efficient, better-evidenced fallback | `microsoft/mai-image-2.6` | 0.039 | 0.045 | 0.08 |
|
||||
| Routine imagery (Budget) | `meta/muse-image` | 0.010 | 0.012 | 0.03 |
|
||||
| Drafts, thumbnails, bulk variants | `recraft/recraft-v4.1-flash` | 0.007 | 0.008 | 0.02 |
|
||||
| Text-heavy (posters, infographics, CJK) | `openai/gpt-image-2.5-sunburst` | 0.21 | 0.25 | 0.30 |
|
||||
| Brand / consistent style set | `recraft/recraft-v4-styles` | 0.035 | 0.04 | 0.08 |
|
||||
| Vector-style output | `recraft/recraft-v4.1-vector` | 0.08 | 0.09 | 0.15 |
|
||||
|
||||
Decision order:
|
||||
|
||||
1. Need vector output, brand style, or heavy text? Take that row.
|
||||
2. Otherwise decide how much an excellent image is worth compared with a good
|
||||
one:
|
||||
- the image is the product → Frontier;
|
||||
- many images, and quality matters → Efficient;
|
||||
- illustrative or routine → Budget;
|
||||
- exploration → bulk.
|
||||
3. For important assets, **explore cheap, then finish expensive**. Render 10–20
|
||||
Budget variants, then re-render the chosen one or two with the Frontier
|
||||
model. 20 × Muse plus 2 × Sunburst costs about the same as three Frontier
|
||||
attempts. This is a hypothesis to test with quality-ledger rows, not a
|
||||
measured result.
|
||||
|
||||
Not available on OpenRouter, and therefore not reachable from this module:
|
||||
self-hostable open weights (Qwen-Image-2.1, FLUX.2 [dev]), Ideogram,
|
||||
Midjourney and Reve.
|
||||
|
||||
## Scoring rule behind the classes
|
||||
|
||||
Image leaderboards publish pairwise-preference **Elo**. Elo is an interval
|
||||
scale, so "within 10% of the best model" cannot be computed as a ratio of Elo
|
||||
values. The digest makes the scores usable as follows:
|
||||
|
||||
```
|
||||
task_score(m) = P(m beats anchor) = 1 / (1 + 10^(-(Elo_m - Elo_anchor)/400))
|
||||
```
|
||||
|
||||
Here the anchor is a fixed reference model. On the Artificial Analysis board
|
||||
that is FLUX.2 [dev], Elo 1000.
|
||||
|
||||
- **Frontier:** the model with the highest `task_score`.
|
||||
- **Topnotch:** `task_score >= 0.90 × task_score(frontier)`. With the
|
||||
Frontier at Elo 1198, this means Elo ≥ about 1133.
|
||||
- **Efficient:** the cheapest Topnotch model.
|
||||
- **Sufficient / Budget:** a per-task floor. For routine imagery the digest uses
|
||||
a provisional floor of Elo ≳ 1100, the level of the frontier from about ten
|
||||
months earlier. Budget is the cheapest model above that floor.
|
||||
|
||||
Callers that already use `AdaptiveRoutingPolicy` quality floors can use the
|
||||
same mapping. Convert Elo to an anchor win rate first, then apply a ratio
|
||||
floor. Do not threshold raw Elo differences across leaderboards: the scales
|
||||
differ. Arena's top gap is about 95 points where Artificial Analysis shows
|
||||
about 40.
|
||||
|
||||
## Caveats
|
||||
|
||||
- OpenRouter bills most image models through a per-token rate. For Recraft,
|
||||
Krea, Muse, Qwen, Seedream and FLUX 3, about 4,175 image tokens reproduces
|
||||
the vendor's per-image price. For OpenAI and Google, cost depends on the
|
||||
quality tier. Let the receipt's `cost_source=provider` correct the quotes
|
||||
after the first runs.
|
||||
- It is not verified here that every listed model accepts OpenRouter's
|
||||
`/images` endpoint with explicit `size` and `aspect_ratio=1:1`. Do one
|
||||
capped single render per model before running a batch.
|
||||
- Nano Banana 2.1 had only a few thousand arena votes on 2026-10-08. If its
|
||||
score drops, MAI-Image-2.6 becomes the Efficient pick.
|
||||
Loading…
Add table
Add a link
Reference in a new issue