diff --git a/README.md b/README.md index f518fb8..a0d3b66 100644 --- a/README.md +++ b/README.md @@ -1,280 +1,281 @@ -# llm-connect - -Pluggable LLM adapters for Python and the commandline. Supports OpenRouter, Gemini, -OpenAI, and the Claude Code CLI out of the box, with a clean abstract interface for adding -your own. - -## Quick start - -```python -from llm_connect import create_adapter, RunConfig - -adapter = create_adapter("gemini", model="gemini-2.5-flash") -config = RunConfig(temperature=0.7, max_tokens=1000) -response = adapter.execute_prompt("Summarise the value chain concept.", config) -print(response.content) -``` - -## Installation - -```bash -pip install -e /path/to/llm-connect # local editable install -# or, once published: -pip install llm-connect -``` - -**Requires:** Python 3.10+, `toml` - -## Providers - -| Provider key | Class | Notes | -|---|---|---| -| `"openrouter"` | `OpenRouterAdapter` | OpenAI-compatible endpoint; supports all OpenRouter models | -| `"gemini"` | `GeminiAdapter` | Google Generative Language REST API; supports free tier | - -```python -from llm_connect import create_adapter - -# OpenRouter (default model: moonshotai/kimi-k3) -adapter = create_adapter("openrouter") -adapter = create_adapter("openrouter", model="anthropic/claude-sonnet-4") - -# Gemini (uses GEMINI_API_KEY env var or apikey-geminifree.txt) -adapter = create_adapter("gemini", model="gemini-2.5-flash") - -# OpenAI (uses OPENAI_API_KEY env var) -adapter = create_adapter("openai", model="gpt-4.1-mini") - -# Claude Code CLI (uses locally installed claude binary) -adapter = create_adapter("claude-code") -``` - -## CLI - -```bash -# Run a prompt; content on stdout, tokens + €/USD cost on stderr -llm-connect run "Summarise the value chain concept." -llm-connect run "Hello" --provider mock --model moonshotai/kimi-k3 --json - -# Estimate cost without calling a provider -llm-connect cost estimate --model moonshotai/kimi-k3 \ - --prompt-tokens 1000 --completion-tokens 500 - -# Weekly spend (Mon-based; default timezone Europe/Berlin) -llm-connect spend week # current week: Mon 00:00 → now -llm-connect spend week --last # previous week: Mon → Sun -llm-connect spend week --json - -# Provider prepaid / key remaining (backend-scoped; does not change defaults) -llm-connect balance # current default provider -llm-connect balance --provider openrouter # one-shot backend select -llm-connect balance --json - -# Square text-to-image. The model and the quote are required when a cap is set. -# Receipt JSON on stdout; image bytes in --output. Credentials stay in the env. -llm-connect image render \ - --prompt "precise technical illustration of a gear, no text" \ - --model "" \ - --output visuals/gear.jpg \ - --max-cost-usd 0.15 \ - --quote-usd 0.08 -``` - -Usage events append to a JSONL ledger (default -`~/.local/share/llm-connect/usage.jsonl`; override with `--ledger` or -`LLM_CONNECT_USAGE_LEDGER`). Costs are list-price estimates (USD rate table + -EUR via snapshot or `LLM_CONNECT_EUR_PER_USD`). - -`balance` is **provider-scoped**: it reports the selected backend’s prepaid / -key remaining (OpenRouter today). `--provider` applies only to that -invocation and does not switch the library default provider or model. -Unsupported backends fail clearly instead of falling back to OpenRouter. - -| Env | Purpose | -|---|---| -| `LLM_CONNECT_USAGE_LEDGER` | Usage JSONL path; enables library/server auto-recording when set | -| `LLM_CONNECT_EUR_PER_USD` | Euros per one USD (FX override) | -| `LLM_CONNECT_TZ` | Timezone for weekly spend windows (default `Europe/Berlin`) | - -## API keys - -Keys are resolved in this order (first found wins): - -1. Explicit `api_key` argument to the constructor -2. Environment variable (e.g. `OPENROUTER_API_KEY`, `GEMINI_API_KEY`, `OPENAI_API_KEY`) -3. Key file in the project root (e.g. `apikey-openrouter.txt`, `apikey-geminifree.txt`) - -## Core types - -### `RunConfig` - -Controls a single LLM call. - -```python -from llm_connect import RunConfig - -config = RunConfig( - model_name="gemini-2.5-flash", # overrides adapter default - temperature=0.3, - max_tokens=2000, - timeout_seconds=60, -) -``` - -| Field | Default | Description | -|---|---|---| -| `model_name` | `"gpt-4"` | Model identifier (adapter may override) | -| `temperature` | `0.7` | Sampling temperature | -| `max_tokens` | `2000` | Maximum output tokens | -| `model_params` | `{}` | Portable extras translated by each adapter; see `docs/adapter-model-params.md` | -| `max_depth` | `3` | Max nesting depth for recursive calls | -| `skip_if_exists` | `True` | Skip if identical input hash already processed | -| `timeout_seconds` | `300` | Request timeout | - -### `LLMResponse` - -Returned by every `execute_prompt` call. - -```python -response = adapter.execute_prompt(prompt, config) -print(response.content) # generated text -print(response.model) # model actually used -print(response.usage) # {"prompt_tokens": …, "completion_tokens": …, "total_tokens": …} -print(response.finish_reason) # "stop", "length", etc. -``` - -## Server diagnostics - -Serve mode can include a debug envelope without changing normal responses: - -```bash -LLM_CONNECT_DEBUG=1 python -m llm_connect.server --provider openrouter -curl 'http://127.0.0.1:8080/execute?debug=1' -d '{"prompt":"hi"}' -``` - -Set `LLM_CONNECT_AUDIT_DIR=/path/to/audit` to write per-call replay records, -then parse one without another provider call: - -```bash -python -m llm_connect.replay /path/to/audit/record.json --json -``` - -## Server runtime profiles - -Serve mode enables named runtime profiles by default. A client can send -`config.model_name="custodian-triage-balanced"` and the server resolves it to -the configured provider/model before calling the adapter. - -Useful runtime environment variables: - -```bash -LLM_CONNECT_HOST=0.0.0.0 -LLM_CONNECT_PORT=8080 -LLM_CONNECT_PROVIDER=openrouter -LLM_CONNECT_MODEL=google/gemini-2.5-flash -LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER=openrouter -LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL=google/gemini-2.5-flash -``` - -For local smoke tests without provider credentials: - -```bash -export LLM_CONNECT_MOCK_RESPONSE="$(python -c 'import json; print(json.dumps(json.load(open("fixtures/activity_core/daily-triage-valid-content.json"))))')" -python -m llm_connect.server --provider mock -python scripts/smoke_activity_core_endpoint.py --url http://127.0.0.1:8080 -``` - -Disable profile dispatch with `--disable-profiles`. Set -`LLM_CONNECT_STRICT_PROFILES=1` or pass `--strict-profiles` to reject direct -model names that are not configured profiles. - -## Writing your own adapter - -```python -from llm_connect import LLMAdapter, RunConfig, LLMResponse - -class MyAdapter(LLMAdapter): - def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse: - # call your API here - return LLMResponse(content="...", model="my-model") - - def validate_config(self, config: RunConfig) -> bool: - return True -``` - -## TOML configuration chain - -The `resolve_llm()` function walks a 7-level priority chain to pick a -provider and model. This is used by the `llm-helper` integration but is also -available standalone: - -```python -from llm_connect.toml_config import resolve_llm - -resolved = resolve_llm(app_name="myapp") -print(resolved.provider, resolved.model, resolved.provider_source) -``` - -Priority order (highest first): - -1. CLI flags (`cli_provider`, `cli_model` arguments) -2. Env var `{APP_NAME}_HELPER_MODEL` (model only) -3. User preference — `~/.config/{app_name}/config.toml` `[llm.preference]` -4. Directory preference — `.{app_name}.toml` `[llm.preference]` -5. Directory default — `.{app_name}.toml` `[llm.default]` -6. User default — `~/.config/{app_name}/config.toml` `[llm.default]` -7. Hardcoded fallback — `gemini / gemini-2.5-flash` - -Example config file (`~/.config/myapp/config.toml`): - -```toml -[llm.default] -provider = "gemini" -model = "gemini-2.5-flash" - -[llm.preference] -provider = "openrouter" -model = "anthropic/claude-sonnet-4" -``` - -## Embeddings - -```python -from llm_connect import create_embedding_adapter, EmbeddingCache - -adapter = create_embedding_adapter("openai", model="text-embedding-3-small") -cache = EmbeddingCache(cache_dir=".embeddings") - -# Get embedding (cached after first call) -vec = cache.get_or_compute("my text", lambda t: adapter.embed([t])[0]) -``` - -## Exceptions - -```python -from llm_connect.exceptions import ( - LLMError, # base - LLMConfigurationError,# bad key, unknown provider - LLMAPIError, # HTTP error from provider (has .status_code) - LLMRateLimitError, # 429 - LLMTimeoutError, # request timed out - LLMSubprocessError, # claude CLI failed (has .return_code, .stderr) -) -``` - -## Testing - -```python -from llm_connect import MockLLMAdapter, RunConfig - -mock = MockLLMAdapter(mock_response="Test response") -config = RunConfig() -response = mock.execute_prompt("any prompt", config) -assert response.content == "Test response" -assert mock.call_count == 1 -``` - -## Origin - -Extracted from the [markitect](https://github.com/worsch/markitect) project. -The `markitect.llm` module remains a re-export shim pointing here. +# llm-connect + +Pluggable LLM adapters for Python and the commandline. Supports OpenRouter, Gemini, +OpenAI, and the Claude Code CLI out of the box, with a clean abstract interface for adding +your own. + +## Quick start + +```python +from llm_connect import create_adapter, RunConfig + +adapter = create_adapter("gemini", model="gemini-2.5-flash") +config = RunConfig(temperature=0.7, max_tokens=1000) +response = adapter.execute_prompt("Summarise the value chain concept.", config) +print(response.content) +``` + +## Installation + +```bash +pip install -e /path/to/llm-connect # local editable install +# or, once published: +pip install llm-connect +``` + +**Requires:** Python 3.10+, `toml` + +## Providers + +| Provider key | Class | Notes | +|---|---|---| +| `"openrouter"` | `OpenRouterAdapter` | OpenAI-compatible endpoint; supports all OpenRouter models | +| `"gemini"` | `GeminiAdapter` | Google Generative Language REST API; supports free tier | + +```python +from llm_connect import create_adapter + +# OpenRouter (default model: moonshotai/kimi-k3) +adapter = create_adapter("openrouter") +adapter = create_adapter("openrouter", model="anthropic/claude-sonnet-4") + +# Gemini (uses GEMINI_API_KEY env var or apikey-geminifree.txt) +adapter = create_adapter("gemini", model="gemini-2.5-flash") + +# OpenAI (uses OPENAI_API_KEY env var) +adapter = create_adapter("openai", model="gpt-4.1-mini") + +# Claude Code CLI (uses locally installed claude binary) +adapter = create_adapter("claude-code") +``` + +## CLI + +```bash +# Run a prompt; content on stdout, tokens + €/USD cost on stderr +llm-connect run "Summarise the value chain concept." +llm-connect run "Hello" --provider mock --model moonshotai/kimi-k3 --json + +# Estimate cost without calling a provider +llm-connect cost estimate --model moonshotai/kimi-k3 \ + --prompt-tokens 1000 --completion-tokens 500 + +# Weekly spend (Mon-based; default timezone Europe/Berlin) +llm-connect spend week # current week: Mon 00:00 → now +llm-connect spend week --last # previous week: Mon → Sun +llm-connect spend week --json + +# Provider prepaid / key remaining (backend-scoped; does not change defaults) +llm-connect balance # current default provider +llm-connect balance --provider openrouter # one-shot backend select +llm-connect balance --json + +# Square text-to-image. The model and the quote are required when a cap is set. +# Which model and quote: docs/image-model-selection.md +# Receipt JSON on stdout; image bytes in --output. Credentials stay in the env. +llm-connect image render \ + --prompt "precise technical illustration of a gear, no text" \ + --model "" \ + --output visuals/gear.jpg \ + --max-cost-usd 0.15 \ + --quote-usd 0.08 +``` + +Usage events append to a JSONL ledger (default +`~/.local/share/llm-connect/usage.jsonl`; override with `--ledger` or +`LLM_CONNECT_USAGE_LEDGER`). Costs are list-price estimates (USD rate table + +EUR via snapshot or `LLM_CONNECT_EUR_PER_USD`). + +`balance` is **provider-scoped**: it reports the selected backend’s prepaid / +key remaining (OpenRouter today). `--provider` applies only to that +invocation and does not switch the library default provider or model. +Unsupported backends fail clearly instead of falling back to OpenRouter. + +| Env | Purpose | +|---|---| +| `LLM_CONNECT_USAGE_LEDGER` | Usage JSONL path; enables library/server auto-recording when set | +| `LLM_CONNECT_EUR_PER_USD` | Euros per one USD (FX override) | +| `LLM_CONNECT_TZ` | Timezone for weekly spend windows (default `Europe/Berlin`) | + +## API keys + +Keys are resolved in this order (first found wins): + +1. Explicit `api_key` argument to the constructor +2. Environment variable (e.g. `OPENROUTER_API_KEY`, `GEMINI_API_KEY`, `OPENAI_API_KEY`) +3. Key file in the project root (e.g. `apikey-openrouter.txt`, `apikey-geminifree.txt`) + +## Core types + +### `RunConfig` + +Controls a single LLM call. + +```python +from llm_connect import RunConfig + +config = RunConfig( + model_name="gemini-2.5-flash", # overrides adapter default + temperature=0.3, + max_tokens=2000, + timeout_seconds=60, +) +``` + +| Field | Default | Description | +|---|---|---| +| `model_name` | `"gpt-4"` | Model identifier (adapter may override) | +| `temperature` | `0.7` | Sampling temperature | +| `max_tokens` | `2000` | Maximum output tokens | +| `model_params` | `{}` | Portable extras translated by each adapter; see `docs/adapter-model-params.md` | +| `max_depth` | `3` | Max nesting depth for recursive calls | +| `skip_if_exists` | `True` | Skip if identical input hash already processed | +| `timeout_seconds` | `300` | Request timeout | + +### `LLMResponse` + +Returned by every `execute_prompt` call. + +```python +response = adapter.execute_prompt(prompt, config) +print(response.content) # generated text +print(response.model) # model actually used +print(response.usage) # {"prompt_tokens": …, "completion_tokens": …, "total_tokens": …} +print(response.finish_reason) # "stop", "length", etc. +``` + +## Server diagnostics + +Serve mode can include a debug envelope without changing normal responses: + +```bash +LLM_CONNECT_DEBUG=1 python -m llm_connect.server --provider openrouter +curl 'http://127.0.0.1:8080/execute?debug=1' -d '{"prompt":"hi"}' +``` + +Set `LLM_CONNECT_AUDIT_DIR=/path/to/audit` to write per-call replay records, +then parse one without another provider call: + +```bash +python -m llm_connect.replay /path/to/audit/record.json --json +``` + +## Server runtime profiles + +Serve mode enables named runtime profiles by default. A client can send +`config.model_name="custodian-triage-balanced"` and the server resolves it to +the configured provider/model before calling the adapter. + +Useful runtime environment variables: + +```bash +LLM_CONNECT_HOST=0.0.0.0 +LLM_CONNECT_PORT=8080 +LLM_CONNECT_PROVIDER=openrouter +LLM_CONNECT_MODEL=google/gemini-2.5-flash +LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER=openrouter +LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL=google/gemini-2.5-flash +``` + +For local smoke tests without provider credentials: + +```bash +export LLM_CONNECT_MOCK_RESPONSE="$(python -c 'import json; print(json.dumps(json.load(open("fixtures/activity_core/daily-triage-valid-content.json"))))')" +python -m llm_connect.server --provider mock +python scripts/smoke_activity_core_endpoint.py --url http://127.0.0.1:8080 +``` + +Disable profile dispatch with `--disable-profiles`. Set +`LLM_CONNECT_STRICT_PROFILES=1` or pass `--strict-profiles` to reject direct +model names that are not configured profiles. + +## Writing your own adapter + +```python +from llm_connect import LLMAdapter, RunConfig, LLMResponse + +class MyAdapter(LLMAdapter): + def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse: + # call your API here + return LLMResponse(content="...", model="my-model") + + def validate_config(self, config: RunConfig) -> bool: + return True +``` + +## TOML configuration chain + +The `resolve_llm()` function walks a 7-level priority chain to pick a +provider and model. This is used by the `llm-helper` integration but is also +available standalone: + +```python +from llm_connect.toml_config import resolve_llm + +resolved = resolve_llm(app_name="myapp") +print(resolved.provider, resolved.model, resolved.provider_source) +``` + +Priority order (highest first): + +1. CLI flags (`cli_provider`, `cli_model` arguments) +2. Env var `{APP_NAME}_HELPER_MODEL` (model only) +3. User preference — `~/.config/{app_name}/config.toml` `[llm.preference]` +4. Directory preference — `.{app_name}.toml` `[llm.preference]` +5. Directory default — `.{app_name}.toml` `[llm.default]` +6. User default — `~/.config/{app_name}/config.toml` `[llm.default]` +7. Hardcoded fallback — `gemini / gemini-2.5-flash` + +Example config file (`~/.config/myapp/config.toml`): + +```toml +[llm.default] +provider = "gemini" +model = "gemini-2.5-flash" + +[llm.preference] +provider = "openrouter" +model = "anthropic/claude-sonnet-4" +``` + +## Embeddings + +```python +from llm_connect import create_embedding_adapter, EmbeddingCache + +adapter = create_embedding_adapter("openai", model="text-embedding-3-small") +cache = EmbeddingCache(cache_dir=".embeddings") + +# Get embedding (cached after first call) +vec = cache.get_or_compute("my text", lambda t: adapter.embed([t])[0]) +``` + +## Exceptions + +```python +from llm_connect.exceptions import ( + LLMError, # base + LLMConfigurationError,# bad key, unknown provider + LLMAPIError, # HTTP error from provider (has .status_code) + LLMRateLimitError, # 429 + LLMTimeoutError, # request timed out + LLMSubprocessError, # claude CLI failed (has .return_code, .stderr) +) +``` + +## Testing + +```python +from llm_connect import MockLLMAdapter, RunConfig + +mock = MockLLMAdapter(mock_response="Test response") +config = RunConfig() +response = mock.execute_prompt("any prompt", config) +assert response.content == "Test response" +assert mock.call_count == 1 +``` + +## Origin + +Extracted from the [markitect](https://github.com/worsch/markitect) project. +The `markitect.llm` module remains a re-export shim pointing here. diff --git a/contracts/functional/images.md b/contracts/functional/images.md index 5737de5..b5db4ad 100644 --- a/contracts/functional/images.md +++ b/contracts/functional/images.md @@ -43,7 +43,8 @@ comes from the explicit argument, `OPENROUTER_API_KEY`, or `apikey-openrouter.txt`. Only `b64_json` is accepted. A URL in the response is not fetched. -There is no default image model. +There is no default image model. Advisory selection guidance, outside this +contract: [`docs/image-model-selection.md`](../../docs/image-model-selection.md). ## CLI diff --git a/docs/image-model-selection.md b/docs/image-model-selection.md new file mode 100644 index 0000000..2e1f2e7 --- /dev/null +++ b/docs/image-model-selection.md @@ -0,0 +1,88 @@ +# Choosing an image model for `llm_connect.images` + +`llm_connect.images` has no default image model and does not choose one (see +[`contracts/functional/images.md`](../contracts/functional/images.md)). This +guide is advisory input for the caller. It does not change that contract. + +**Source:** intelligence-radar topic digest +`digests/topics/2026-10-08-1527_Image_Generation_with_Generative_AI_Digest.md` +(evidence observed 2026-10-08). Rankings and prices in this field change +monthly. Re-derive the table from the digest's sources before relying on it +after about 2026-11. + +## Picking a model + +All IDs below are OpenRouter model IDs. Prices are for one 1024² image. The +quote is the per-image estimate plus some margin, and the cap leaves room for +the provider-reported cost to come in above the quote. + +| Need | Model | ≈ $/img | `--quote-usd` | `--max-cost-usd` | +|---|---|---|---|---| +| Best result, cost secondary (Frontier) | `openai/gpt-image-2.5-sunburst` | 0.21 | 0.25 | 0.30 | +| Same quality, faster | `openai/gpt-image-2.5-flare` | 0.21 | 0.25 | 0.30 | +| Near-frontier at scale (Efficient) | `google/gemini-nano-banana-2.1` | 0.034 | 0.04 | 0.08 | +| Efficient, better-evidenced fallback | `microsoft/mai-image-2.6` | 0.039 | 0.045 | 0.08 | +| Routine imagery (Budget) | `meta/muse-image` | 0.010 | 0.012 | 0.03 | +| Drafts, thumbnails, bulk variants | `recraft/recraft-v4.1-flash` | 0.007 | 0.008 | 0.02 | +| Text-heavy (posters, infographics, CJK) | `openai/gpt-image-2.5-sunburst` | 0.21 | 0.25 | 0.30 | +| Brand / consistent style set | `recraft/recraft-v4-styles` | 0.035 | 0.04 | 0.08 | +| Vector-style output | `recraft/recraft-v4.1-vector` | 0.08 | 0.09 | 0.15 | + +Decision order: + +1. Need vector output, brand style, or heavy text? Take that row. +2. Otherwise decide how much an excellent image is worth compared with a good + one: + - the image is the product → Frontier; + - many images, and quality matters → Efficient; + - illustrative or routine → Budget; + - exploration → bulk. +3. For important assets, **explore cheap, then finish expensive**. Render 10–20 + Budget variants, then re-render the chosen one or two with the Frontier + model. 20 × Muse plus 2 × Sunburst costs about the same as three Frontier + attempts. This is a hypothesis to test with quality-ledger rows, not a + measured result. + +Not available on OpenRouter, and therefore not reachable from this module: +self-hostable open weights (Qwen-Image-2.1, FLUX.2 [dev]), Ideogram, +Midjourney and Reve. + +## Scoring rule behind the classes + +Image leaderboards publish pairwise-preference **Elo**. Elo is an interval +scale, so "within 10% of the best model" cannot be computed as a ratio of Elo +values. The digest makes the scores usable as follows: + +``` +task_score(m) = P(m beats anchor) = 1 / (1 + 10^(-(Elo_m - Elo_anchor)/400)) +``` + +Here the anchor is a fixed reference model. On the Artificial Analysis board +that is FLUX.2 [dev], Elo 1000. + +- **Frontier:** the model with the highest `task_score`. +- **Topnotch:** `task_score >= 0.90 × task_score(frontier)`. With the + Frontier at Elo 1198, this means Elo ≥ about 1133. +- **Efficient:** the cheapest Topnotch model. +- **Sufficient / Budget:** a per-task floor. For routine imagery the digest uses + a provisional floor of Elo ≳ 1100, the level of the frontier from about ten + months earlier. Budget is the cheapest model above that floor. + +Callers that already use `AdaptiveRoutingPolicy` quality floors can use the +same mapping. Convert Elo to an anchor win rate first, then apply a ratio +floor. Do not threshold raw Elo differences across leaderboards: the scales +differ. Arena's top gap is about 95 points where Artificial Analysis shows +about 40. + +## Caveats + +- OpenRouter bills most image models through a per-token rate. For Recraft, + Krea, Muse, Qwen, Seedream and FLUX 3, about 4,175 image tokens reproduces + the vendor's per-image price. For OpenAI and Google, cost depends on the + quality tier. Let the receipt's `cost_source=provider` correct the quotes + after the first runs. +- It is not verified here that every listed model accepts OpenRouter's + `/images` endpoint with explicit `size` and `aspect_ratio=1:1`. Do one + capped single render per model before running a batch. +- Nano Banana 2.1 had only a few thousand arena votes on 2026-10-08. If its + score drops, MAI-Image-2.6 becomes the Efficient pick.