diff --git a/digests/topics/2026-10-08-1527_Image_Generation_with_Generative_AI_Digest.md b/digests/topics/2026-10-08-1527_Image_Generation_with_Generative_AI_Digest.md new file mode 100644 index 0000000..a3a84f7 --- /dev/null +++ b/digests/topics/2026-10-08-1527_Image_Generation_with_Generative_AI_Digest.md @@ -0,0 +1,373 @@ +--- +title: "Image Generation with Generative AI — State of the Field and Model Choice" +date: 2026-10-08T15:27:00+02:00 +project: intelligence-radar +document_type: topic-digest +status: initial-assessment +evidence_observed_at: 2026-10-08 +policy_refs: + - model-ranking-v0.1 +tags: + - intelligence-radar + - image-generation + - text-to-image + - image-editing + - capability-advisor + - openrouter + - model-choice +--- + +# Image Generation with Generative AI — State of the Field and Model Choice + +**Timestamp:** 2026-10-08 15:27 CEST +**Research context:** `intelligence-radar` / Capability Advisor +**Evidence snapshot:** Artificial Analysis Text-to-Image and Image Editing arenas +(read 2026-10-08), Arena text-to-image leaderboard (last updated 2026-10-07), +OpenRouter `/api/v1/models?output_modalities=image` (61 entries, read 2026-10-08). + +> Prices and rankings in this field move monthly. Five of the top six +> text-to-image models were released after June 2026. Treat every number here +> as dated to 2026-10-08. Re-check before you commit to spending money (P7). + +--- + +## 1. Executive summary + +- **Frontier (general quality):** OpenAI **GPT Image 2.5 Sunburst** (Sept 2026). + It is #1 on both independent human-preference arenas and on the editing arena. + **GPT Image 2.5 Flare** is statistically tied with it (rank range 1–2 on + Artificial Analysis). The cost is high: about **$0.21 per image** at max quality. +- **Efficient (Topnotch at the lowest cost):** Google **Nano Banana 2.1** + (≈ $0.034/img). It is close to the Frontier at about one sixth of the price, but + it is only days old and its arena scores are preliminary. The well-evidenced + fallback is **Microsoft MAI-Image-2.6** (≈ $0.039/img). +- **Budget (credibly Sufficient for routine work):** Meta **Muse Image** + (≈ $0.010/img). It is #8 overall on Artificial Analysis and #5–8 for editing, + at about one twentieth of the Frontier price. This is the stand-out value + observation in this digest. +- **Open weights:** Alibaba **Qwen-Image-2.1** leads (Elo 1035, under a + non-commercial Qwen Research licence). The open/closed gap is about 160 Elo, + so the best open model is roughly as good as the closed frontier was about a + year ago. +- **OpenRouter:** every model above **except the open-weights ones** is + available, so the main recommendations carry over unchanged. Missing from + OpenRouter: Ideogram, Midjourney, Reve, Luma, HiDream, Imagen 4, and all + self-hostable weights. +- **Structural shift:** the top of the field is now **natively multimodal + LLM-integrated generators** (GPT Image, Nano Banana/Gemini, MAI, Muse, Grok + Imagine). Pure diffusion specialists (FLUX, Ideogram, Recraft, Midjourney) have + moved into niches: vectors, typography, style systems, self-hosting, and very + low cost. + +--- + +## 2. The landscape (2026-10-08) + +### 2.1 Text-to-image — Artificial Analysis Arena (AA-Image-T2I v2.0, top 20) + +The anchor is FLUX.2 [dev] = 1000. Price is per 1,000 images at 1024², default +settings, from the creator's API. "OR" marks models available on OpenRouter. + +| # (range) | Model | Creator | Elo | $/1k | Released | OR | +|---|---|---|---|---|---|---| +| 1 (1–2) | GPT Image 2.5 Sunburst (max) | OpenAI | 1198 | 210.7 | 2026-09 | ✔ | +| 2 (1–2) | GPT Image 2.5 Flare (max) | OpenAI | 1191 | 210.7 | 2026-09 | ✔ | +| 3 | GPT Image 2 (high) | OpenAI | 1172 | 211.0 | 2026-04 | ✔ | +| 4 (4–6) | Nano Banana 2.1 | Google | 1160 | 33.6 | 2026-10 | ✔ | +| 5 (4–6) | Grok Imagine Image 2.0 | SpaceXAI | 1156 | 60.0 | 2026-08 | ✔ | +| 6 (4–6) | MAI-Image-2.6 | Microsoft AI | 1151 | 38.9 | 2026-08 | ✔ | +| 7 | Nano Banana 2 (Gemini 3.1 Flash Image) | Google | 1126 | 67.0 | 2026-02 | ✔ | +| 8 (8–9) | Muse Image | Meta | 1115 | 10.0 | 2026-07 | ✔ | +| 9 (8–13) | GPT Image 1.5 (high) | OpenAI | 1107 | 133.0 | 2025-12 | — | +| 10 (9–13) | MAI-Image-2.6-Flash | Microsoft AI | 1105 | 19.5 | 2026-09 | ✔ | +| 11 | MAI-Image-2.5 | Microsoft AI | 1104 | 48.1 | 2026-06 | ✔ | +| 12 | Nano Banana Pro (Gemini 3 Pro Image) | Google | 1102 | 134.0 | 2025-11 | ✔ | +| 13 | MAI-Image-2.5-Pro | Microsoft AI | 1100 | 108.5 | 2026-07 | ✔ | +| 14 | Nano Banana 2 Lite | Google | 1095 | 33.6 | 2026-06 | ✔ | +| 15 | Qwen-Image-3.0-Pro | Alibaba | 1088 | 40.0 | 2026-07 | ✔ | +| 16 | Seedream 5.0 Pro | ByteDance Seed | 1081 | 90.0 | 2026-07 | ✔ | +| 17 | Qwen-Image-3.0 | Alibaba | 1075 | 30.0 | 2026-07 | ✔ | +| 18 | grok-imagine-image-quality | SpaceXAI | 1046 | 50.0 | 2026-05 | ✔ | +| 19 (19–24) | **Qwen-Image-2.1** (open weights) | Alibaba | 1035 | no API | 2026-09 | — | +| 20 | MAI-Image-2.5-Flash | Microsoft AI | 1032 | 20.0 | 2026-06 | — | + +Selected further reference points: + +| Model | Elo | $/1k | Note | +|---|---|---|---| +| FLUX.2 [max] / [pro] / [dev] | 1020 / 1004 / 1000 | 70 / 30 / 12 | dev = open weights, arena anchor | +| Ideogram 4.5 (High) / 4.0 (Quality) | 1016 / 1012 | 100 / 100 | 4.0 has open weights; typography specialist | +| Recraft V4.1 Utility / V4.1 / V4.1 Flash | 1022 / 989 / 957 | 35 / 35 / 7 | vector and brand-style specialist | +| FLUX.2 [dev] Flash / Turbo (Fal) | 987 / 998 | 5 / 8 | cheapest strong open-weights serving | +| Z-Image Turbo | 941 | 5 | open weights, very cheap | +| Midjourney v7 Alpha | 879 | no API | no programmatic access; not in contention for API work | +| DALL·E 3 | 777 | 40 | legacy; do not choose | + +### 2.2 Independent cross-check — Arena (arena.ai) text-to-image, 2026-10-07 + +| # | Model | Score | Votes | +|---|---|---|---| +| 1 | gpt-image-2.5-sunburst | 1425 ± 7 (prelim.) | 17,752 | +| 2 | gpt-image-2.5-flare | 1398 ± 7 (prelim.) | 16,889 | +| 3 | gpt-image-2 (**medium**) | 1383 ± 4 | 93,334 | +| 4 | grok-imagine-image-2.0 (canvas) | 1334 ± 12 (prelim.) | 3,113 | +| 5 | mai-image-2.6 | 1332 ± 5 | 25,091 | +| 6 | gemini-nano-banana-2.1 | 1328 ± 9 (prelim.) | 5,310 | +| 7 | reve-2.1 | 1302 ± 8 | 8,249 | +| 9 | muse-image | 1274 ± 5 | 42,262 | +| 12 | seedream-5.0-pro | 1255 ± 4 | 94,181 | + +**Agreement.** Both arenas give the same top-6 *set*, and the same ordering of +vendors at the very top. **Disagreement.** On Arena the OpenAI lead is about +95 points, against about 40 on Artificial Analysis. That makes the "near +Frontier" status of Nano Banana 2.1 and MAI-Image-2.6 less certain than the AA +numbers alone suggest. Arena also shows that GPT Image 2 at **medium** quality +still ranks #3. Quality tier is therefore a large cost lever within the OpenAI +family, but neither leaderboard publishes a per-tier price at that setting. + +### 2.3 Image editing — Artificial Analysis Editing Arena (top tier) + +| Rank | Model | Elo | $/1k | OR | +|---|---|---|---|---| +| 1 | GPT Image 2.5 Sunburst (max) | 1183 | 210.7 | ✔ | +| 2 | GPT Image 2.5 Flare (max) | 1163 | 210.7 | ✔ | +| 3–4 | MAI-Image-2.6 | 1139 | 38.9 | ✔ | +| 3–4 | Nano Banana 2.1 | 1137 | 33.6 | ✔ | +| 5–7 | MAI-Image-2.6-Flash | 1127 | 19.5 | ✔ | +| 5–7 | GPT Image 2 (high) | 1125 | 211.0 | ✔ | +| 5–8 | **Muse Image** | 1120 | **10.0** | ✔ | +| 15–17 | HunyuanImage 3.5 (Preview) | 1087 | 24.0 | ✔ | +| 18–23 | Qwen-Image-2.1 (open) | 1074 | no API | — | +| 19–24 | HunyuanImage 3.0 Instruct (open) | 1068 | no API | — | + +Editing is where the cheap, LLM-integrated models are closest to the +Frontier. MAI-Image-2.6-Flash and Muse Image are statistically tied with +GPT Image 2 (high) for editing, at 1/10 and 1/20 of its price. + +--- + +## 3. Benchmark selection (P4) for "general image generation" + +| # | Instrument | Weight | Construct | Freshness | Known weaknesses | +|---|---|---|---|---|---| +| 1 | Artificial Analysis T2I Arena | 0.35 | Pairwise human preference, fixed prompt set | 2026-10-08 | Aesthetic bias; prompt set not task-specific; new models have few appearances | +| 2 | Arena (arena.ai) text-to-image | 0.25 | Pairwise preference, user-supplied prompts | 2026-10-07 | Self-selected voters and prompts; correlated with #1 (P5), so it is not an independent fifth of the evidence | +| 3 | Artificial Analysis Editing Arena | 0.20 | Preference on instructed edits | 2026-10-08 | Primary instrument for editing tasks, secondary for generation | +| 4 | OneIG-Bench (or a successor) | 0.10 | Automated alignment, text rendering, reasoning, style and diversity | **not refreshed** | Automated judges; lags new releases | +| 5 | LongText-Bench / text-rendering evals | 0.10 | Text rendering accuracy (Latin + CJK) | **not refreshed** | Narrow construct; vendor-reported numbers dominate | + +**Mapping quality: medium.** Instruments #1–#3 are fresh and well populated. +Instruments #4 and #5 were not re-fetched for this digest. The specialist branches +of the decision tree (typography, vector output, photorealism, brand style) therefore +rest mostly on vendor positioning and press coverage, not on measured category +scores. `task-router/BENCHMARK-ATLAS.md` has **no image-generation family yet**. +That is an evidence gap to close (see §7). + +--- + +## 4. Applying the ranking policy (model-ranking-v0.1) + +The policy defines Topnotch as `task_score ≥ 0.90 × task_score(frontier)`. Elo is +an interval scale, so a ratio of Elo values is meaningless. This digest +therefore uses a **digest-local operationalisation**, which is not a versioned +policy: + +> `task_score` = expected win rate against the arena anchor (FLUX.2 [dev], Elo 1000), +> `p = 1 / (1 + 10^(−(Elo − 1000)/400))`. + +Frontier (Sunburst, 1198) → p = 0.758. The Topnotch threshold of 0.90 × 0.758 = 0.683 +corresponds to **Elo ≥ ≈1133** on the AA board. + +| Model | AA Elo | p vs anchor | ratio to Frontier | Head-to-head vs Frontier | $/img | Class | +|---|---|---|---|---|---|---| +| GPT Image 2.5 Sunburst | 1198 | 0.758 | 1.00 | — | 0.211 | **Frontier** | +| GPT Image 2.5 Flare | 1191 | 0.750 | 0.99 | 0.49 | 0.211 | Topnotch | +| GPT Image 2 (high) | 1172 | 0.729 | 0.96 | 0.46 | 0.211 | Topnotch | +| Nano Banana 2.1 | 1160 | 0.715 | 0.94 | 0.45 | **0.034** | Topnotch → **Efficient** | +| Grok Imagine Image 2.0 | 1156 | 0.711 | 0.94 | 0.44 | 0.060 | Topnotch | +| MAI-Image-2.6 | 1151 | 0.705 | 0.93 | 0.43 | 0.039 | Topnotch (Efficient fallback) | +| Nano Banana 2 | 1126 | 0.674 | 0.89 | 0.40 | 0.067 | Sufficient | +| Muse Image | 1115 | 0.660 | 0.87 | 0.38 | **0.010** | Sufficient → **Budget** | +| MAI-Image-2.6-Flash | 1105 | 0.647 | 0.85 | 0.37 | 0.0195 | Sufficient | + +**Sufficient** needs a success threshold per task. No outcome ledger exists for +image tasks yet, so this digest uses a **provisional placeholder**: for routine +illustration and marketing imagery, a model is Sufficient at Elo ≳ 1100. That is +the level of GPT Image 1.5 and Nano Banana Pro, which were the Frontier about ten +months ago. Under that placeholder, **Budget = Muse Image**. For drafts, thumbnails +and exploration (the "exploratory" 0.60 threshold), much cheaper models qualify, +as §5 shows. + +The anchor choice affects class membership near the threshold. If Arena's +gap is used instead of AA's, Nano Banana 2.1 and MAI-Image-2.6 fall just outside +Topnotch. **Confidence in the Efficient pick: medium.** **Confidence in the +Frontier and Budget picks: high.** + +--- + +## 5. Decision tree — general (any access route) + +``` +START: what does the task need? +│ +├─ A. Hard constraint: weights must run on own hardware +│ (data residency, offline, fine-tuning/LoRA, no per-image fees) +│ ├─ Best quality, non-commercial use OK ........ Qwen-Image-2.1 (Elo 1035; Qwen Research licence) +│ ├─ Commercial use, quality first .............. Ideogram 4.0 (open) (1012) or FLUX.2 [dev] (1000) — check each licence +│ ├─ Editing / instruction-following ............ Qwen-Image-2.1 (1074 edit) or HunyuanImage 3.0 Instruct (1068) +│ └─ Throughput / small GPU ..................... FLUX.2 [klein] 9B/4B, Z-Image Turbo (≈940 / 863) +│ +├─ B. Output must be vector (SVG, logos, icons, scalable assets) +│ └─ Recraft V4.1 Vector (budget) / V4.1 Pro Vector (quality) [unmeasured by arenas → evidence weak] +│ +├─ C. Editing an existing image (inpaint, restyle, multi-reference, product shots) +│ ├─ Highest fidelity, cost no object ........... GPT Image 2.5 Sunburst (1183; up to 16 reference images) +│ ├─ Near-frontier, about 6× cheaper ............ MAI-Image-2.6 (1139) ≈ Nano Banana 2.1 (1137) +│ └─ High volume ................................ Muse Image (1120, $0.010) or MAI-Image-2.6-Flash (1127, $0.020) +│ +├─ D. Text-heavy images (posters, infographics, UI mockups, CJK text) +│ ├─ Default .................................... GPT Image 2.5 Sunburst (GPT Image line is strongest on text rendering) +│ ├─ Factual infographic needing current data ... Nano Banana 2.1 / 2 with web-search grounding +│ └─ Typography-centric design, lower cost ...... Ideogram 4.5 / 4.0 [specialist; vendor claim, category score not refreshed] +│ +├─ E. Brand style systems / consistent illustration style across a set +│ └─ Recraft V4 Styles (Pro), or a top model with a fixed reference-image set +│ +└─ F. General generation (photo, illustration, concept art, marketing) + ├─ Is the per-image cost irrelevant compared with the value of the best result? + │ └─ YES → FRONTIER: GPT Image 2.5 Sunburst (max) + │ · Flare if latency matters (tied on quality) + │ · try GPT Image 2 at medium quality first: still #3 on Arena + ├─ Do you need near-frontier quality at scale? + │ └─ YES → EFFICIENT: Nano Banana 2.1 ($0.034) + │ · fallback while 2.1 is preliminary: MAI-Image-2.6 ($0.039) + │ · alternative look: Grok Imagine Image 2.0 ($0.060) + ├─ Is it routine imagery where "good" is good enough? + │ └─ YES → BUDGET: Muse Image ($0.010) + └─ Is it drafts, thumbnails, exploration, or bulk A/B variants? + └─ YES → FLUX.2 [dev] Flash/Turbo ($0.005–0.008), Z-Image Turbo ($0.005), + Recraft V4.1 Flash ($0.007) +``` + +**Escalation pattern (P12).** For important assets, decomposition usually beats +running one model for everything. Explore with a Budget model at 10–20 variants, +pick a composition, then regenerate or edit only the final one or two with the +Frontier model. At these prices, 20 × Muse ($0.20) + 2 × Sunburst ($0.42) costs +about the same as 3 Frontier attempts, and covers much more of the composition +space. This is a hypothesis to test in the outcome ledger, not a measured result. + +--- + +## 6. Decision tree — limited to OpenRouter + +### 6.1 How OpenRouter bills image models + +OpenRouter lists 61 image-output entries (including two `openrouter/auto*` routers). +It expresses most per-image vendor prices as a **per-token rate**. For the +image-token-priced vendors (Recraft, Krea, Muse, Qwen, Seedream, FLUX 3), the +rate × ≈4,175 tokens reproduces Artificial Analysis's per-image price exactly. +Examples: Recraft V4.1 → $0.035, Krea 2 Large → $0.060, Muse → $0.010, +Qwen-Image-3 → $0.030. Prices below are derived from that rate. For OpenAI and +Google the token count depends on the quality and resolution tier. Grok Imagine +lists $0.04 output plus $0.01 per input image, against Artificial Analysis's +$0.060 per generated image. Verify actual spend with OpenRouter's generation +stats after the first runs. + +Calls go through the normal chat-completions endpoint, with image output +requested through `modalities`. One OpenRouter key therefore covers every branch +below. Data handling then depends on the upstream provider, which matters for +branch A. + +### 6.2 Tree + +``` +START (OpenRouter only) +│ +├─ A. Self-hosting / open weights required +│ └─ NOT SATISFIABLE on OpenRouter. Nearest hosted options: +│ black-forest-labs/flux.2-pro or flux.2-klein-4b (open-family models, hosted by BFL) +│ qwen/qwen-image-3 ($0.030) if the requirement is "non-US vendor", not "own hardware" +│ +├─ B. Vector output +│ ├─ recraft/recraft-v4-styles-vector ...................... ≈ $0.050 +│ ├─ recraft/recraft-v4.1-vector ........................... ≈ $0.080 +│ └─ recraft/recraft-v4.1-pro-vector ....................... ≈ $0.300 +│ +├─ C. Editing +│ ├─ Frontier ..... openai/gpt-image-2.5-sunburst +│ ├─ Efficient .... microsoft/mai-image-2.6 or google/gemini-nano-banana-2.1 +│ └─ Budget ....... meta/muse-image (or microsoft/mai-image-2.6-flash) +│ +├─ D. Text-heavy images +│ ├─ Default ....................... openai/gpt-image-2.5-sunburst +│ ├─ Grounded / current facts ...... google/gemini-nano-banana-2.1 (web_search priced separately) +│ ├─ Layered, editable design ...... inclusionai/ming-image-0.1-design-layer (Elo ≈997; no price listed — verify) +│ └─ (Ideogram is NOT on OpenRouter; there is no typography specialist) +│ +├─ E. Brand / style systems +│ └─ recraft/recraft-v4-styles (≈$0.035) / recraft-v4-styles-pro (≈$0.100) +│ +└─ F. General generation + ├─ FRONTIER ..... openai/gpt-image-2.5-sunburst (Flare = faster twin; gpt-image-2 at lower quality to save cost) + ├─ EFFICIENT .... google/gemini-nano-banana-2.1 (fallback: microsoft/mai-image-2.6) + │ x-ai/grok-imagine-image-2.0 (alternative look) + ├─ BUDGET ....... meta/muse-image (≈ $0.010, Elo 1115) + └─ BULK / DRAFT + ├─ recraft/recraft-v4.1-flash .................. ≈ $0.007 (Elo 957) + ├─ black-forest-labs/flux.2-klein-4b ........... ≈ $0.014 (Elo 863) + ├─ bytedance-seed/seedream-5-0-flash ........... ≈ $0.018 (unranked, new 2026-10-01) + └─ black-forest-labs/flux-3-image .............. ≈ $0.021 (unranked, new 2026-10-01 — watch) +``` + +### 6.3 What OpenRouter costs you + +| Need | Best overall | Best on OpenRouter | Loss | +|---|---|---|---| +| Frontier / Efficient / Budget | Sunburst / Nano Banana 2.1 / Muse | same | none | +| Self-hosting, fine-tuning, LoRA | Qwen-Image-2.1, FLUX.2 [dev] | — | **total** | +| Typography specialist | Ideogram 4.5 | GPT Image 2.5 (generalist) | small in quality, but no cheap typography option | +| Ultra-cheap bulk | FLUX.2 [dev] Flash / Z-Image Turbo ($0.005) | Recraft V4.1 Flash ($0.007) | small | +| Distinctive aesthetic | Reve 2.1 (Arena #7), Midjourney | — | depends on taste; not measurable here | + +**Verdict:** for API-driven work that does not need self-hosting, limiting yourself +to OpenRouter costs almost nothing at the top of the market in October 2026. + +--- + +## 7. Evidence gaps and confidence + +| Item | Confidence | Gap | +|---|---|---| +| Frontier = GPT Image 2.5 Sunburst | **high** | Both arenas agree; 14–18k votes each | +| Efficient = Nano Banana 2.1 | **medium-low** | 5–6k votes, preliminary on Arena; the AA vs Arena gap affects Topnotch membership | +| Budget = Muse Image | **high** for routine work | Strong on both arenas (42k Arena votes); Sufficient threshold is a placeholder | +| Specialist branches (B, D, E) | **low–medium** | No refreshed category or text-rendering scores; vendor claims dominate | +| OpenRouter per-image costs | **medium** | Derived from token rates; OpenAI/Google spend depends on quality tier | +| Price of GPT Image 2/2.5 at medium/low quality | **unknown** | The lever with the most potential savings inside the Frontier family is unmeasured here | + +Follow-ups worth doing: + +1. Add an **image-generation family** to `task-router/BENCHMARK-ATLAS.md`, with + AA T2I, Arena T2I, AA Editing, a text-rendering benchmark and a compositional + alignment benchmark. +2. Decide on a versioned rule for applying `topnotch.relative_to_frontier_minimum` + to **pairwise-preference (Elo) scores**. The anchor-win-rate mapping in §4 is a + candidate. +3. Run a small, matched **outcome-ledger trial** (P9). Use about 20 real prompts + across branches C, D and F with Sunburst, Nano Banana 2.1, MAI-2.6 and Muse, + with a human verdict per image. This would test both the Efficient pick and + the §5 escalation hypothesis. +4. Watch list: FLUX 3 Image (unranked), Seedream 5.0 Flash (unranked), Nano Banana 2.1 + (vote count stabilising), Ideogram 4.5 (too few appearances, 2.4k). + +--- + +## Sources + +- Artificial Analysis — Text-to-Image leaderboard: https://artificialanalysis.ai/image/leaderboard/text-to-image (read 2026-10-08) +- Artificial Analysis — Image Editing leaderboard: https://artificialanalysis.ai/image/leaderboard/editing (read 2026-10-08) +- Arena — Text-to-Image leaderboard: https://arena.ai/leaderboard/text-to-image (last updated 2026-10-07) +- OpenRouter models API, image output: https://openrouter.ai/api/v1/models?output_modalities=image (read 2026-10-08) +- GPT Image 2.5 release coverage (2026-09-08/09): https://letsdatascience.com/news/openai-introduces-chatgpt-images-25-with-new-editing-tools-19e7c138 ; https://help.artlist.io/hc/en-us/articles/39018863931165-GPT-Image-2-Family +- FLUX 3 rollout coverage (secondary; dates conflict): https://aipicks.jp/news/20261002-flux-flux-3-image-4k-10 +- Secondary aggregators (discovery only, P11): https://llm-stats.com/leaderboards/best-ai-for-image-generation ; https://budgetpixel.com/arena/leaderboard ; https://fal.ai/learn/tools/best-text-to-image-apis-2026