Add LLM-WP-0008: provider-scoped account balance CLI
Workplan for backend-pluggable prepaid/budget lookup with one-shot --provider selection that does not change library defaults.
This commit is contained in:
parent
ce38c6247f
commit
8159efff8e
1 changed files with 281 additions and 0 deletions
281
workplans/LLM-WP-0008-provider-account-balance-cli.md
Normal file
281
workplans/LLM-WP-0008-provider-account-balance-cli.md
Normal file
|
|
@ -0,0 +1,281 @@
|
|||
---
|
||||
id: LLM-WP-0008
|
||||
type: workplan
|
||||
title: "Provider-scoped account balance CLI"
|
||||
domain: agents
|
||||
repo: llm-connect
|
||||
status: ready
|
||||
owner: codex
|
||||
topic_slug: provider-account-balance-cli
|
||||
planning_priority: high
|
||||
planning_order: 8
|
||||
created: "2026-08-03"
|
||||
updated: "2026-08-03"
|
||||
depends_on_workplans:
|
||||
- LLM-WP-0007
|
||||
related_workplans: []
|
||||
---
|
||||
|
||||
# LLM-WP-0008 — Provider-scoped account balance CLI
|
||||
|
||||
**status:** ready
|
||||
**owner:** codex
|
||||
|
||||
## Purpose
|
||||
|
||||
Add a CLI (and library) surface that reports **remaining prepaid / account
|
||||
budget** for the **selected inference backend**, without treating OpenRouter
|
||||
as the only possible source of truth and **without changing the library or
|
||||
operator default** provider/model.
|
||||
|
||||
Constraints from operator (2026-08-03):
|
||||
|
||||
1. Balance depends on the **backend** (provider). A future inference service
|
||||
must not silently be asked for “the OpenRouter budget”.
|
||||
2. The command must accept an explicit **backend selector for that invocation
|
||||
only** — it must **not** switch or persist the default basemodel/provider.
|
||||
3. If the backend is **not** specified, report balance for the **currently
|
||||
configured default backend** (today: `openrouter` via `LLMConfig.provider`
|
||||
/ factory default).
|
||||
|
||||
## Demand signal
|
||||
|
||||
After LLM-WP-0007, local estimated spend is visible (`spend week`), but the
|
||||
operator still cannot see **remaining OpenRouter prepaid credits / key limit**
|
||||
from `llm-connect`. That gap is backend-specific; the design must stay
|
||||
pluggable.
|
||||
|
||||
## Current repo state (as of 2026-08-03)
|
||||
|
||||
| Area | State |
|
||||
|---|---|
|
||||
| CLI | `rates`, `classes`, `run`, `cost`, `spend` — **no** balance/credits command |
|
||||
| Default provider | `openrouter` (`LLMConfig.provider`, `create_adapter` default) |
|
||||
| Default model | `moonshotai/kimi-k3` (OpenRouter) — **must not** be mutated by balance CLI |
|
||||
| OpenRouter live APIs (checked) | `GET /api/v1/auth/key` → `limit`, `limit_remaining`, `usage`, `limit_reset`, …; `GET /api/v1/credits` → `total_credits`, `total_usage` |
|
||||
| Other providers | Gemini / OpenAI / Claude Code have **no** balance client yet |
|
||||
|
||||
OpenRouter semantics worth distinguishing in the UX:
|
||||
|
||||
| Signal | Source | Meaning |
|
||||
|---|---|---|
|
||||
| **Account credits remaining** | `total_credits - total_usage` from `/credits` | Prepaid balance on the OpenRouter account |
|
||||
| **Key limit remaining** | `limit_remaining` from `/auth/key` | Remaining allowance on **this API key** (may be lower than account credits; may reset monthly) |
|
||||
|
||||
v1 should surface **both** when available, labelled clearly, with EUR display
|
||||
optional via existing FX helpers (amounts from OpenRouter are USD).
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
llm_connect/
|
||||
balance.py # NEW: ProviderBalance protocol + registry + resolve_default_provider()
|
||||
openrouter_balance.py # NEW: OpenRouterBalanceClient (auth/key + credits)
|
||||
# later: openai_balance.py, etc. — only if a real API exists
|
||||
|
||||
cli.py # llm-connect balance [--provider …]
|
||||
config.py # unchanged defaults; only *read* default provider
|
||||
```
|
||||
|
||||
### Provider selection (critical)
|
||||
|
||||
```bash
|
||||
# Uses default backend (today: openrouter). Does NOT change defaults.
|
||||
llm-connect balance
|
||||
|
||||
# One-shot backend override for this command only — no config write, no
|
||||
# mutation of LLMConfig defaults or OpenRouterAdapter._DEFAULT_MODEL.
|
||||
llm-connect balance --provider openrouter
|
||||
llm-connect balance --provider openai # may error: unsupported until implemented
|
||||
```
|
||||
|
||||
Resolution order for **which backend to query**:
|
||||
|
||||
1. Explicit `--provider` / library `provider=` argument
|
||||
2. Else `LLMConfig.provider` default (currently `"openrouter"`)
|
||||
3. Never infer “whatever model string looks like” and never call OpenRouter
|
||||
because “it’s the only client we have” when another provider was selected
|
||||
|
||||
Calling `balance` with `--provider X` **must not**:
|
||||
|
||||
- rewrite config files, env, or key files
|
||||
- change `LLMConfig` class defaults or module-level `_DEFAULT_MODEL`
|
||||
- affect subsequent `llm-connect run` without an explicit provider on that run
|
||||
|
||||
### Pluggable contract
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class AccountBalance:
|
||||
provider: str
|
||||
currency: str # usually "USD" from upstream
|
||||
# Prepaid / wallet style (optional)
|
||||
credits_total: float | None = None
|
||||
credits_used: float | None = None
|
||||
credits_remaining: float | None = None
|
||||
# Key/limit style (optional)
|
||||
limit: float | None = None
|
||||
limit_remaining: float | None = None
|
||||
limit_reset: str | None = None # e.g. "monthly"
|
||||
usage: float | None = None
|
||||
# Display
|
||||
credits_remaining_eur: float | None = None
|
||||
limit_remaining_eur: float | None = None
|
||||
fx_source: str | None = None
|
||||
source: str # e.g. "openrouter:/api/v1/credits+auth/key"
|
||||
raw: dict[str, Any] = field(default_factory=dict) # optional debug
|
||||
|
||||
class ProviderBalanceClient(Protocol):
|
||||
provider_id: str
|
||||
def get_balance(self) -> AccountBalance: ...
|
||||
```
|
||||
|
||||
Registry maps `provider_id → client factory`. Unknown / unimplemented
|
||||
providers raise a clear `LLMConfigurationError` (or dedicated
|
||||
`LLMBalanceUnsupportedError`) listing which providers support balance.
|
||||
|
||||
### OpenRouter client (v1 implementation)
|
||||
|
||||
- Auth: same key resolution as adapters (`OPENROUTER_API_KEY` / key file)
|
||||
- Fetch `/api/v1/credits` and `/api/v1/auth/key` (best-effort both; degrade
|
||||
gracefully if one fails, but fail if both fail)
|
||||
- `credits_remaining = total_credits - total_usage` when credits payload present
|
||||
- Attach key `limit_remaining` / `limit` / `limit_reset` when key payload present
|
||||
- Convert remaining amounts to EUR via existing `resolve_fx_rate` (optional,
|
||||
same as cost estimates)
|
||||
|
||||
### CLI output
|
||||
|
||||
Human (default):
|
||||
|
||||
```
|
||||
provider: openrouter
|
||||
account credits remaining: 12.715255 USD (~11.70 EUR)
|
||||
key limit remaining: 9.983798 USD (~9.19 EUR) [limit=10, reset=monthly]
|
||||
source: openrouter:/api/v1/credits+auth/key
|
||||
```
|
||||
|
||||
`--json` for automation. Never print full API keys; label redaction already
|
||||
matches OpenRouter’s partial label if shown.
|
||||
|
||||
### Unsupported backends
|
||||
|
||||
| Provider | v1 behaviour |
|
||||
|---|---|
|
||||
| `openrouter` | Implemented |
|
||||
| `openai`, `gemini`, `claude-code`, `mock` | Explicit unsupported error with exit ≠ 0 |
|
||||
| unknown | Same as factory: unknown provider error |
|
||||
|
||||
Mock may optionally return a fixed balance for tests only if registered under
|
||||
`mock` — prefer unit-testing OpenRouter client with `urllib` mocks instead of
|
||||
shipping a fake production path.
|
||||
|
||||
## Scope guardrails
|
||||
|
||||
**In scope**
|
||||
|
||||
- Pluggable balance protocol + registry
|
||||
- OpenRouter implementation using real APIs
|
||||
- CLI `balance` with `--provider` one-shot override
|
||||
- Default-provider resolution without mutating defaults
|
||||
- EUR display via existing FX
|
||||
- Tests (mocked HTTP); optional live smoke gated on key
|
||||
- Docs (README + short contract)
|
||||
|
||||
**Out of scope**
|
||||
|
||||
- Changing default provider or default model
|
||||
- Writing balance into the usage ledger (different concern)
|
||||
- Billing reconciliation / invoices
|
||||
- Implementing Gemini/OpenAI balance clients until demand + known API
|
||||
- Interactive provider picker / TUI
|
||||
- Caching balance across process invocations (always live fetch in v1)
|
||||
|
||||
## Tasks
|
||||
|
||||
```task
|
||||
id: LLM-WP-0008-T01
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
**Define `AccountBalance` + `ProviderBalanceClient` protocol and registry** in
|
||||
`balance.py` (or equivalent). Include `resolve_balance_provider(explicit)` that
|
||||
returns the default provider when `explicit` is `None`, without side effects.
|
||||
Tests: default resolution, unknown provider, unsupported provider.
|
||||
|
||||
```task
|
||||
id: LLM-WP-0008-T02
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
**OpenRouter balance client.** Call `/api/v1/credits` and `/api/v1/auth/key`
|
||||
with existing key resolution and HTTP helpers. Map to `AccountBalance` with
|
||||
clear field semantics. Unit tests with mocked responses (including partial
|
||||
failure of one endpoint). Live smoke optional / manual.
|
||||
|
||||
```task
|
||||
id: LLM-WP-0008-T03
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
**CLI `llm-connect balance [--provider …] [--json] [--eur-per-usd …]`.**
|
||||
Explicit `--provider` is per-invocation only. Omit → default backend.
|
||||
Print tokens-free account/key remaining in USD and €. Exit non-zero for
|
||||
unsupported backends. Tests via mocked client registration.
|
||||
|
||||
```task
|
||||
id: LLM-WP-0008-T04
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
**Package exports + contract docs + README.** Export balance types; document
|
||||
that balance is backend-scoped and that `--provider` does not change defaults.
|
||||
Note OpenRouter dual signals (account credits vs key limit).
|
||||
|
||||
```task
|
||||
id: LLM-WP-0008-T05
|
||||
status: todo
|
||||
priority: low
|
||||
```
|
||||
|
||||
**Acceptance polish.** Confirm with a real key (operator or CI secret) that
|
||||
`llm-connect balance` and `llm-connect balance --provider openrouter` match
|
||||
within rounding of OpenRouter dashboard figures; document any known drift
|
||||
(e.g. pending usage).
|
||||
|
||||
## Acceptance
|
||||
|
||||
- [ ] `llm-connect balance` reports OpenRouter remaining budget when default
|
||||
provider is `openrouter`, without changing any defaults
|
||||
- [ ] `llm-connect balance --provider openrouter` same result; still no default
|
||||
mutation
|
||||
- [ ] `llm-connect balance --provider <unsupported>` fails clearly (not with
|
||||
OpenRouter data)
|
||||
- [ ] Account credits remaining and key limit remaining are distinguishable in
|
||||
output when both are available
|
||||
- [ ] EUR amounts shown when FX available
|
||||
- [ ] Adding a future provider is “register a balance client”, not hard-coding
|
||||
OpenRouter into the CLI command body
|
||||
- [ ] Offline unit tests pass without network
|
||||
|
||||
## Risks / notes
|
||||
|
||||
- **Credits vs key limit:** operators often care about prepaid wallet
|
||||
(`/credits`); key-level caps (`/auth/key`) can be lower. Showing only one
|
||||
risks false “I still have money” / “I’m broke” readings — show both.
|
||||
- **Currency:** OpenRouter amounts are USD; EUR is display conversion only
|
||||
(same philosophy as LLM-WP-0007).
|
||||
- **Default provider today is openrouter:** that is correct for “unspecified
|
||||
→ default backend”, but the implementation must not special-case
|
||||
“unspecified always means OpenRouter” in the CLI handler — go through
|
||||
`resolve_balance_provider(None)`.
|
||||
- **No secret leakage:** never print full API keys; raw dumps only behind an
|
||||
explicit future `--debug` if ever added (not in v1 human output).
|
||||
|
||||
## Implementation order
|
||||
|
||||
T01 → T02 → T03 → T04 → T05.
|
||||
Loading…
Add table
Add a link
Reference in a new issue