Implement LLM-WP-0007: Kimi K3 default and EUR spend reporting
Add moonshotai/kimi-k3 as OpenRouter basemodel default after live smoke, USD→EUR cost conversion, append-only usage ledger, and CLI run/cost/spend week commands with token and euro reporting.
This commit is contained in:
parent
f3121c3f1f
commit
09aa1f3604
21 changed files with 1396 additions and 103 deletions
147
README.md
147
README.md
|
|
@ -35,7 +35,8 @@ pip install llm-connect
|
|||
```python
|
||||
from llm_connect import create_adapter
|
||||
|
||||
# OpenRouter
|
||||
# OpenRouter (default model: moonshotai/kimi-k3)
|
||||
adapter = create_adapter("openrouter")
|
||||
adapter = create_adapter("openrouter", model="anthropic/claude-sonnet-4")
|
||||
|
||||
# Gemini (uses GEMINI_API_KEY env var or apikey-geminifree.txt)
|
||||
|
|
@ -48,6 +49,34 @@ adapter = create_adapter("openai", model="gpt-4.1-mini")
|
|||
adapter = create_adapter("claude-code")
|
||||
```
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
# Run a prompt; content on stdout, tokens + €/USD cost on stderr
|
||||
llm-connect run "Summarise the value chain concept."
|
||||
llm-connect run "Hello" --provider mock --model moonshotai/kimi-k3 --json
|
||||
|
||||
# Estimate cost without calling a provider
|
||||
llm-connect cost estimate --model moonshotai/kimi-k3 \
|
||||
--prompt-tokens 1000 --completion-tokens 500
|
||||
|
||||
# Weekly spend (Mon-based; default timezone Europe/Berlin)
|
||||
llm-connect spend week # current week: Mon 00:00 → now
|
||||
llm-connect spend week --last # previous week: Mon → Sun
|
||||
llm-connect spend week --json
|
||||
```
|
||||
|
||||
Usage events append to a JSONL ledger (default
|
||||
`~/.local/share/llm-connect/usage.jsonl`; override with `--ledger` or
|
||||
`LLM_CONNECT_USAGE_LEDGER`). Costs are list-price estimates (USD rate table +
|
||||
EUR via snapshot or `LLM_CONNECT_EUR_PER_USD`).
|
||||
|
||||
| Env | Purpose |
|
||||
|---|---|
|
||||
| `LLM_CONNECT_USAGE_LEDGER` | Usage JSONL path; enables library/server auto-recording when set |
|
||||
| `LLM_CONNECT_EUR_PER_USD` | Euros per one USD (FX override) |
|
||||
| `LLM_CONNECT_TZ` | Timezone for weekly spend windows (default `Europe/Berlin`) |
|
||||
|
||||
## API keys
|
||||
|
||||
Keys are resolved in this order (first found wins):
|
||||
|
|
@ -73,15 +102,15 @@ config = RunConfig(
|
|||
)
|
||||
```
|
||||
|
||||
| Field | Default | Description |
|
||||
|---|---|---|
|
||||
| `model_name` | `"gpt-4"` | Model identifier (adapter may override) |
|
||||
| `temperature` | `0.7` | Sampling temperature |
|
||||
| `max_tokens` | `2000` | Maximum output tokens |
|
||||
| `model_params` | `{}` | Portable extras translated by each adapter; see `docs/adapter-model-params.md` |
|
||||
| `max_depth` | `3` | Max nesting depth for recursive calls |
|
||||
| `skip_if_exists` | `True` | Skip if identical input hash already processed |
|
||||
| `timeout_seconds` | `300` | Request timeout |
|
||||
| Field | Default | Description |
|
||||
|---|---|---|
|
||||
| `model_name` | `"gpt-4"` | Model identifier (adapter may override) |
|
||||
| `temperature` | `0.7` | Sampling temperature |
|
||||
| `max_tokens` | `2000` | Maximum output tokens |
|
||||
| `model_params` | `{}` | Portable extras translated by each adapter; see `docs/adapter-model-params.md` |
|
||||
| `max_depth` | `3` | Max nesting depth for recursive calls |
|
||||
| `skip_if_exists` | `True` | Skip if identical input hash already processed |
|
||||
| `timeout_seconds` | `300` | Request timeout |
|
||||
|
||||
### `LLMResponse`
|
||||
|
||||
|
|
@ -92,55 +121,55 @@ response = adapter.execute_prompt(prompt, config)
|
|||
print(response.content) # generated text
|
||||
print(response.model) # model actually used
|
||||
print(response.usage) # {"prompt_tokens": …, "completion_tokens": …, "total_tokens": …}
|
||||
print(response.finish_reason) # "stop", "length", etc.
|
||||
```
|
||||
|
||||
## Server diagnostics
|
||||
|
||||
Serve mode can include a debug envelope without changing normal responses:
|
||||
|
||||
```bash
|
||||
LLM_CONNECT_DEBUG=1 python -m llm_connect.server --provider openrouter
|
||||
curl 'http://127.0.0.1:8080/execute?debug=1' -d '{"prompt":"hi"}'
|
||||
```
|
||||
|
||||
Set `LLM_CONNECT_AUDIT_DIR=/path/to/audit` to write per-call replay records,
|
||||
then parse one without another provider call:
|
||||
|
||||
```bash
|
||||
python -m llm_connect.replay /path/to/audit/record.json --json
|
||||
```
|
||||
|
||||
## Server runtime profiles
|
||||
|
||||
Serve mode enables named runtime profiles by default. A client can send
|
||||
`config.model_name="custodian-triage-balanced"` and the server resolves it to
|
||||
the configured provider/model before calling the adapter.
|
||||
|
||||
Useful runtime environment variables:
|
||||
|
||||
```bash
|
||||
LLM_CONNECT_HOST=0.0.0.0
|
||||
LLM_CONNECT_PORT=8080
|
||||
LLM_CONNECT_PROVIDER=openrouter
|
||||
LLM_CONNECT_MODEL=google/gemini-2.5-flash
|
||||
LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER=openrouter
|
||||
LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL=google/gemini-2.5-flash
|
||||
```
|
||||
|
||||
For local smoke tests without provider credentials:
|
||||
|
||||
```bash
|
||||
export LLM_CONNECT_MOCK_RESPONSE="$(python -c 'import json; print(json.dumps(json.load(open("fixtures/activity_core/daily-triage-valid-content.json"))))')"
|
||||
python -m llm_connect.server --provider mock
|
||||
python scripts/smoke_activity_core_endpoint.py --url http://127.0.0.1:8080
|
||||
```
|
||||
|
||||
Disable profile dispatch with `--disable-profiles`. Set
|
||||
`LLM_CONNECT_STRICT_PROFILES=1` or pass `--strict-profiles` to reject direct
|
||||
model names that are not configured profiles.
|
||||
|
||||
## Writing your own adapter
|
||||
print(response.finish_reason) # "stop", "length", etc.
|
||||
```
|
||||
|
||||
## Server diagnostics
|
||||
|
||||
Serve mode can include a debug envelope without changing normal responses:
|
||||
|
||||
```bash
|
||||
LLM_CONNECT_DEBUG=1 python -m llm_connect.server --provider openrouter
|
||||
curl 'http://127.0.0.1:8080/execute?debug=1' -d '{"prompt":"hi"}'
|
||||
```
|
||||
|
||||
Set `LLM_CONNECT_AUDIT_DIR=/path/to/audit` to write per-call replay records,
|
||||
then parse one without another provider call:
|
||||
|
||||
```bash
|
||||
python -m llm_connect.replay /path/to/audit/record.json --json
|
||||
```
|
||||
|
||||
## Server runtime profiles
|
||||
|
||||
Serve mode enables named runtime profiles by default. A client can send
|
||||
`config.model_name="custodian-triage-balanced"` and the server resolves it to
|
||||
the configured provider/model before calling the adapter.
|
||||
|
||||
Useful runtime environment variables:
|
||||
|
||||
```bash
|
||||
LLM_CONNECT_HOST=0.0.0.0
|
||||
LLM_CONNECT_PORT=8080
|
||||
LLM_CONNECT_PROVIDER=openrouter
|
||||
LLM_CONNECT_MODEL=google/gemini-2.5-flash
|
||||
LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER=openrouter
|
||||
LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL=google/gemini-2.5-flash
|
||||
```
|
||||
|
||||
For local smoke tests without provider credentials:
|
||||
|
||||
```bash
|
||||
export LLM_CONNECT_MOCK_RESPONSE="$(python -c 'import json; print(json.dumps(json.load(open("fixtures/activity_core/daily-triage-valid-content.json"))))')"
|
||||
python -m llm_connect.server --provider mock
|
||||
python scripts/smoke_activity_core_endpoint.py --url http://127.0.0.1:8080
|
||||
```
|
||||
|
||||
Disable profile dispatch with `--disable-profiles`. Set
|
||||
`LLM_CONNECT_STRICT_PROFILES=1` or pass `--strict-profiles` to reject direct
|
||||
model names that are not configured profiles.
|
||||
|
||||
## Writing your own adapter
|
||||
|
||||
```python
|
||||
from llm_connect import LLMAdapter, RunConfig, LLMResponse
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue