Implement LLM-WP-0008: provider-scoped account balance CLI
Add pluggable balance registry, OpenRouter credits/key limit client, and llm-connect balance with one-shot --provider that does not change library defaults.
This commit is contained in:
parent
c0d5c4ad08
commit
ed7c632155
11 changed files with 783 additions and 14 deletions
10
README.md
10
README.md
|
|
@ -64,6 +64,11 @@ llm-connect cost estimate --model moonshotai/kimi-k3 \
|
|||
llm-connect spend week # current week: Mon 00:00 → now
|
||||
llm-connect spend week --last # previous week: Mon → Sun
|
||||
llm-connect spend week --json
|
||||
|
||||
# Provider prepaid / key remaining (backend-scoped; does not change defaults)
|
||||
llm-connect balance # current default provider
|
||||
llm-connect balance --provider openrouter # one-shot backend select
|
||||
llm-connect balance --json
|
||||
```
|
||||
|
||||
Usage events append to a JSONL ledger (default
|
||||
|
|
@ -71,6 +76,11 @@ Usage events append to a JSONL ledger (default
|
|||
`LLM_CONNECT_USAGE_LEDGER`). Costs are list-price estimates (USD rate table +
|
||||
EUR via snapshot or `LLM_CONNECT_EUR_PER_USD`).
|
||||
|
||||
`balance` is **provider-scoped**: it reports the selected backend’s prepaid /
|
||||
key remaining (OpenRouter today). `--provider` applies only to that
|
||||
invocation and does not switch the library default provider or model.
|
||||
Unsupported backends fail clearly instead of falling back to OpenRouter.
|
||||
|
||||
| Env | Purpose |
|
||||
|---|---|
|
||||
| `LLM_CONNECT_USAGE_LEDGER` | Usage JSONL path; enables library/server auto-recording when set |
|
||||
|
|
|
|||
44
contracts/functional/account-balance.md
Normal file
44
contracts/functional/account-balance.md
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
# Provider Account Balance
|
||||
|
||||
`llm_connect.balance` reports **prepaid / key remaining** for a selected
|
||||
inference **backend**. Balance is never assumed to be OpenRouter unless that
|
||||
backend is selected (explicitly or as the current default provider).
|
||||
|
||||
## Selection rules
|
||||
|
||||
```python
|
||||
from llm_connect import get_account_balance, resolve_balance_provider
|
||||
|
||||
resolve_balance_provider(None) # → LLMConfig().provider (today openrouter)
|
||||
resolve_balance_provider("openrouter") # one-shot; does not mutate defaults
|
||||
get_account_balance() # default backend
|
||||
get_account_balance("openrouter") # explicit backend for this call only
|
||||
```
|
||||
|
||||
- Explicit `provider` / CLI `--provider` is **invocation-only**.
|
||||
- It must not rewrite config files, env, `LLMConfig` class defaults, or
|
||||
OpenRouter model defaults.
|
||||
- Unspecified → current default provider via `resolve_balance_provider(None)`.
|
||||
|
||||
## Contract
|
||||
|
||||
- `AccountBalance` carries optional wallet fields (`credits_*`) and optional
|
||||
key-limit fields (`limit`, `limit_remaining`, `limit_reset`, `usage`).
|
||||
- EUR display fields use the same FX resolution as cost estimates.
|
||||
- `BalanceClientRegistry` maps provider id → client factory.
|
||||
- Unsupported providers raise `LLMBalanceUnsupportedError` listing supported
|
||||
backends (do not fall back to OpenRouter).
|
||||
|
||||
## OpenRouter (v1)
|
||||
|
||||
- `GET /api/v1/credits` → account prepaid: remaining = total_credits − total_usage
|
||||
- `GET /api/v1/auth/key` → key limit remaining / reset policy
|
||||
- Both are fetched best-effort; one may fail while the other succeeds.
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
llm-connect balance
|
||||
llm-connect balance --provider openrouter
|
||||
llm-connect balance --json
|
||||
```
|
||||
|
|
@ -21,8 +21,18 @@ from llm_connect.embedding_adapter import EmbeddingAdapter
|
|||
from llm_connect.embedding_cache import EmbeddingCache
|
||||
from llm_connect.embedding_factory import create_embedding_adapter
|
||||
from llm_connect.embedding_openai import OpenAICompatibleEmbeddingAdapter
|
||||
from llm_connect.balance import (
|
||||
AccountBalance,
|
||||
BalanceClientRegistry,
|
||||
OpenRouterBalanceClient,
|
||||
ProviderBalanceClient,
|
||||
format_balance_human,
|
||||
get_account_balance,
|
||||
resolve_balance_provider,
|
||||
)
|
||||
from llm_connect.exceptions import (
|
||||
LLMAPIError,
|
||||
LLMBalanceUnsupportedError,
|
||||
LLMBudgetExceededError,
|
||||
LLMConfigurationError,
|
||||
LLMError,
|
||||
|
|
@ -103,6 +113,14 @@ __all__ = [
|
|||
"LLMTimeoutError",
|
||||
"LLMSubprocessError",
|
||||
"LLMBudgetExceededError",
|
||||
"LLMBalanceUnsupportedError",
|
||||
"AccountBalance",
|
||||
"BalanceClientRegistry",
|
||||
"OpenRouterBalanceClient",
|
||||
"ProviderBalanceClient",
|
||||
"format_balance_human",
|
||||
"get_account_balance",
|
||||
"resolve_balance_provider",
|
||||
"EmbeddingAdapter",
|
||||
"OpenAICompatibleEmbeddingAdapter",
|
||||
"EmbeddingCache",
|
||||
|
|
|
|||
|
|
@ -38,7 +38,36 @@ def post_json(
|
|||
headers={"Content-Type": "application/json", **(headers or {})},
|
||||
method="POST",
|
||||
)
|
||||
return _read_json_response(url, req, timeout=timeout)
|
||||
|
||||
|
||||
def get_json(
|
||||
url: str,
|
||||
headers: Optional[Dict[str, str]] = None,
|
||||
timeout: int = 60,
|
||||
) -> Dict[str, Any]:
|
||||
"""GET *url* and return the parsed JSON response body.
|
||||
|
||||
Raises:
|
||||
LLMRateLimitError: on HTTP 429
|
||||
LLMAPIError: on other non-2xx responses
|
||||
LLMTimeoutError: on socket / read timeout
|
||||
"""
|
||||
record_provider_request(url=url, payload=None, headers=headers or {})
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
headers={**(headers or {})},
|
||||
method="GET",
|
||||
)
|
||||
return _read_json_response(url, req, timeout=timeout)
|
||||
|
||||
|
||||
def _read_json_response(
|
||||
url: str,
|
||||
req: urllib.request.Request,
|
||||
*,
|
||||
timeout: int,
|
||||
) -> Dict[str, Any]:
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
body = resp.read().decode()
|
||||
|
|
|
|||
357
llm_connect/balance.py
Normal file
357
llm_connect/balance.py
Normal file
|
|
@ -0,0 +1,357 @@
|
|||
"""Provider-scoped account balance / prepaid remaining.
|
||||
|
||||
Balance lookup is always bound to an explicit or default **backend**
|
||||
(provider). Selecting a provider for a balance query must never mutate
|
||||
library defaults (model or provider).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
|
||||
from llm_connect._http import get_json
|
||||
from llm_connect.config import LLMConfig, find_project_root, resolve_api_key
|
||||
from llm_connect.exceptions import (
|
||||
LLMAPIError,
|
||||
LLMBalanceUnsupportedError,
|
||||
LLMConfigurationError,
|
||||
)
|
||||
from llm_connect.fx import FxRate, resolve_fx_rate
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AccountBalance:
|
||||
"""Normalised prepaid / key-limit snapshot for one backend."""
|
||||
|
||||
provider: str
|
||||
currency: str
|
||||
source: str
|
||||
credits_total: float | None = None
|
||||
credits_used: float | None = None
|
||||
credits_remaining: float | None = None
|
||||
limit: float | None = None
|
||||
limit_remaining: float | None = None
|
||||
limit_reset: str | None = None
|
||||
usage: float | None = None
|
||||
credits_remaining_eur: float | None = None
|
||||
limit_remaining_eur: float | None = None
|
||||
fx_source: str | None = None
|
||||
raw: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
provider = str(self.provider or "").strip()
|
||||
currency = str(self.currency or "").strip().upper()
|
||||
source = str(self.source or "").strip()
|
||||
if not provider:
|
||||
raise ValueError("provider must be a non-empty string")
|
||||
if not currency:
|
||||
raise ValueError("currency must be a non-empty string")
|
||||
if not source:
|
||||
raise ValueError("source must be a non-empty string")
|
||||
object.__setattr__(self, "provider", provider)
|
||||
object.__setattr__(self, "currency", currency)
|
||||
object.__setattr__(self, "source", source)
|
||||
object.__setattr__(self, "raw", dict(self.raw or {}))
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
"""JSON-serialisable view (includes raw for automation)."""
|
||||
return {
|
||||
"provider": self.provider,
|
||||
"currency": self.currency,
|
||||
"source": self.source,
|
||||
"credits_total": self.credits_total,
|
||||
"credits_used": self.credits_used,
|
||||
"credits_remaining": self.credits_remaining,
|
||||
"limit": self.limit,
|
||||
"limit_remaining": self.limit_remaining,
|
||||
"limit_reset": self.limit_reset,
|
||||
"usage": self.usage,
|
||||
"credits_remaining_eur": self.credits_remaining_eur,
|
||||
"limit_remaining_eur": self.limit_remaining_eur,
|
||||
"fx_source": self.fx_source,
|
||||
"raw": dict(self.raw),
|
||||
}
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class ProviderBalanceClient(Protocol):
|
||||
"""Backend that can report account / key prepaid remaining."""
|
||||
|
||||
provider_id: str
|
||||
|
||||
def get_balance(self, *, fx: FxRate | float | None = None) -> AccountBalance:
|
||||
"""Fetch a live balance snapshot for this provider."""
|
||||
...
|
||||
|
||||
|
||||
BalanceClientFactory = Callable[[], ProviderBalanceClient]
|
||||
|
||||
|
||||
def resolve_balance_provider(explicit: str | None = None) -> str:
|
||||
"""Resolve which backend to query for balance.
|
||||
|
||||
1. Non-empty *explicit* provider id (one-shot override)
|
||||
2. Else :attr:`LLMConfig.provider` default (today ``openrouter``)
|
||||
|
||||
Read-only: never mutates config, env, or module defaults.
|
||||
"""
|
||||
if explicit is not None and str(explicit).strip():
|
||||
return str(explicit).strip().lower()
|
||||
return str(LLMConfig().provider).strip().lower()
|
||||
|
||||
|
||||
class BalanceClientRegistry:
|
||||
"""Map provider id → balance client factory."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
factories: Mapping[str, BalanceClientFactory] | None = None,
|
||||
) -> None:
|
||||
self._factories: dict[str, BalanceClientFactory] = {
|
||||
str(key).strip().lower(): factory
|
||||
for key, factory in (factories or {}).items()
|
||||
}
|
||||
|
||||
def register(self, provider_id: str, factory: BalanceClientFactory) -> None:
|
||||
"""Register or replace a factory for *provider_id*."""
|
||||
key = str(provider_id).strip().lower()
|
||||
if not key:
|
||||
raise ValueError("provider_id must be a non-empty string")
|
||||
self._factories[key] = factory
|
||||
|
||||
def supported(self) -> list[str]:
|
||||
"""Return sorted provider ids that implement balance."""
|
||||
return sorted(self._factories)
|
||||
|
||||
def create(self, provider_id: str) -> ProviderBalanceClient:
|
||||
"""Instantiate a client for *provider_id* or raise unsupported."""
|
||||
key = str(provider_id).strip().lower()
|
||||
factory = self._factories.get(key)
|
||||
if factory is None:
|
||||
supported = self.supported()
|
||||
raise LLMBalanceUnsupportedError(
|
||||
f"Account balance is not supported for provider {key!r}. "
|
||||
f"Supported: {', '.join(supported) if supported else '(none)'}",
|
||||
provider=key,
|
||||
supported=supported,
|
||||
)
|
||||
return factory()
|
||||
|
||||
@classmethod
|
||||
def default(cls) -> "BalanceClientRegistry":
|
||||
"""Built-in registry (OpenRouter first; more backends later)."""
|
||||
return cls(
|
||||
{
|
||||
"openrouter": lambda: OpenRouterBalanceClient(),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def get_account_balance(
|
||||
provider: str | None = None,
|
||||
*,
|
||||
registry: BalanceClientRegistry | None = None,
|
||||
fx: FxRate | float | None = None,
|
||||
) -> AccountBalance:
|
||||
"""Fetch balance for *provider* or the current default backend.
|
||||
|
||||
*provider* is a one-shot selector and does not change library defaults.
|
||||
"""
|
||||
resolved = resolve_balance_provider(provider)
|
||||
clients = registry or BalanceClientRegistry.default()
|
||||
return clients.create(resolved).get_balance(fx=fx)
|
||||
|
||||
|
||||
class OpenRouterBalanceClient:
|
||||
"""OpenRouter prepaid credits + API-key limit remaining."""
|
||||
|
||||
provider_id = "openrouter"
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
api_key: str | None = None,
|
||||
api_base: str = "https://openrouter.ai/api/v1",
|
||||
*,
|
||||
timeout: int = 30,
|
||||
get_json_fn: Callable[..., dict[str, Any]] | None = None,
|
||||
) -> None:
|
||||
self._api_base = api_base.rstrip("/")
|
||||
self._timeout = timeout
|
||||
self._get_json = get_json_fn or get_json
|
||||
root = find_project_root()
|
||||
key_file_paths = [root / "apikey-openrouter.txt"] if root else []
|
||||
self._api_key = resolve_api_key(
|
||||
explicit=api_key,
|
||||
env_var="OPENROUTER_API_KEY",
|
||||
key_file_paths=key_file_paths,
|
||||
)
|
||||
|
||||
def get_balance(self, *, fx: FxRate | float | None = None) -> AccountBalance:
|
||||
if not self._api_key:
|
||||
raise LLMConfigurationError(
|
||||
"OpenRouter API key not found for balance lookup "
|
||||
"(set OPENROUTER_API_KEY or apikey-openrouter.txt)",
|
||||
context={"provider": self.provider_id},
|
||||
)
|
||||
|
||||
headers = {"Authorization": f"Bearer {self._api_key}"}
|
||||
credits_payload: dict[str, Any] | None = None
|
||||
key_payload: dict[str, Any] | None = None
|
||||
errors: list[str] = []
|
||||
sources: list[str] = []
|
||||
|
||||
try:
|
||||
credits_payload = self._get_json(
|
||||
f"{self._api_base}/credits",
|
||||
headers=headers,
|
||||
timeout=self._timeout,
|
||||
)
|
||||
sources.append("credits")
|
||||
except LLMAPIError as exc:
|
||||
errors.append(f"credits: {exc}")
|
||||
|
||||
try:
|
||||
key_payload = self._get_json(
|
||||
f"{self._api_base}/auth/key",
|
||||
headers=headers,
|
||||
timeout=self._timeout,
|
||||
)
|
||||
sources.append("auth/key")
|
||||
except LLMAPIError as exc:
|
||||
errors.append(f"auth/key: {exc}")
|
||||
|
||||
if credits_payload is None and key_payload is None:
|
||||
raise LLMAPIError(
|
||||
"OpenRouter balance lookup failed for both /credits and /auth/key: "
|
||||
+ "; ".join(errors),
|
||||
context={"provider": self.provider_id, "errors": errors},
|
||||
)
|
||||
|
||||
credits_total: float | None = None
|
||||
credits_used: float | None = None
|
||||
credits_remaining: float | None = None
|
||||
limit: float | None = None
|
||||
limit_remaining: float | None = None
|
||||
limit_reset: str | None = None
|
||||
usage: float | None = None
|
||||
|
||||
if credits_payload is not None:
|
||||
data = _unwrap_data(credits_payload)
|
||||
credits_total = _optional_float(data.get("total_credits"))
|
||||
credits_used = _optional_float(data.get("total_usage"))
|
||||
if credits_total is not None and credits_used is not None:
|
||||
credits_remaining = credits_total - credits_used
|
||||
|
||||
if key_payload is not None:
|
||||
data = _unwrap_data(key_payload)
|
||||
limit = _optional_float(data.get("limit"))
|
||||
limit_remaining = _optional_float(data.get("limit_remaining"))
|
||||
usage = _optional_float(data.get("usage"))
|
||||
raw_reset = data.get("limit_reset")
|
||||
if raw_reset is not None and str(raw_reset).strip():
|
||||
limit_reset = str(raw_reset).strip()
|
||||
|
||||
fx_rate = resolve_fx_rate(fx)
|
||||
credits_remaining_eur = None
|
||||
limit_remaining_eur = None
|
||||
fx_source = None
|
||||
if fx_rate is not None:
|
||||
fx_source = fx_rate.source
|
||||
if credits_remaining is not None:
|
||||
credits_remaining_eur = fx_rate.usd_to_eur(credits_remaining)
|
||||
if limit_remaining is not None:
|
||||
limit_remaining_eur = fx_rate.usd_to_eur(limit_remaining)
|
||||
|
||||
source_label = "openrouter:" + "+".join(
|
||||
f"/api/v1/{part}" for part in sources
|
||||
)
|
||||
if errors:
|
||||
source_label += f" (partial: {'; '.join(errors)})"
|
||||
|
||||
return AccountBalance(
|
||||
provider=self.provider_id,
|
||||
currency="USD",
|
||||
source=source_label,
|
||||
credits_total=credits_total,
|
||||
credits_used=credits_used,
|
||||
credits_remaining=credits_remaining,
|
||||
limit=limit,
|
||||
limit_remaining=limit_remaining,
|
||||
limit_reset=limit_reset,
|
||||
usage=usage,
|
||||
credits_remaining_eur=credits_remaining_eur,
|
||||
limit_remaining_eur=limit_remaining_eur,
|
||||
fx_source=fx_source,
|
||||
raw={
|
||||
"credits": credits_payload,
|
||||
"auth_key": key_payload,
|
||||
"errors": errors,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def format_balance_human(balance: AccountBalance) -> str:
|
||||
"""Multi-line human-readable balance report."""
|
||||
lines = [f"provider: {balance.provider}"]
|
||||
|
||||
if balance.credits_remaining is not None:
|
||||
eur = _fmt_eur(balance.credits_remaining_eur)
|
||||
lines.append(
|
||||
f"account credits remaining: {_fmt_money(balance.credits_remaining)} "
|
||||
f"{balance.currency}{eur}"
|
||||
)
|
||||
if balance.credits_total is not None and balance.credits_used is not None:
|
||||
lines.append(
|
||||
f" (total={_fmt_money(balance.credits_total)} "
|
||||
f"used={_fmt_money(balance.credits_used)})"
|
||||
)
|
||||
else:
|
||||
lines.append("account credits remaining: unavailable")
|
||||
|
||||
if balance.limit_remaining is not None:
|
||||
eur = _fmt_eur(balance.limit_remaining_eur)
|
||||
extra = []
|
||||
if balance.limit is not None:
|
||||
extra.append(f"limit={_fmt_money(balance.limit)}")
|
||||
if balance.limit_reset:
|
||||
extra.append(f"reset={balance.limit_reset}")
|
||||
suffix = f" [{', '.join(extra)}]" if extra else ""
|
||||
lines.append(
|
||||
f"key limit remaining: {_fmt_money(balance.limit_remaining)} "
|
||||
f"{balance.currency}{eur}{suffix}"
|
||||
)
|
||||
else:
|
||||
lines.append("key limit remaining: unavailable")
|
||||
|
||||
if balance.fx_source:
|
||||
lines.append(f"fx: {balance.fx_source}")
|
||||
lines.append(f"source: {balance.source}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _unwrap_data(payload: Mapping[str, Any]) -> Mapping[str, Any]:
|
||||
data = payload.get("data", payload)
|
||||
if isinstance(data, Mapping):
|
||||
return data
|
||||
return {}
|
||||
|
||||
|
||||
def _optional_float(value: Any) -> float | None:
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _fmt_money(value: float) -> str:
|
||||
return f"{value:.6f}".rstrip("0").rstrip(".") if value != 0 else "0"
|
||||
|
||||
|
||||
def _fmt_eur(value: float | None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
return f" (~{_fmt_money(value)} EUR)"
|
||||
|
|
@ -10,7 +10,12 @@ from datetime import datetime
|
|||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from llm_connect.balance import (
|
||||
format_balance_human,
|
||||
get_account_balance,
|
||||
)
|
||||
from llm_connect.costs import estimate_cost
|
||||
from llm_connect.exceptions import LLMBalanceUnsupportedError, LLMConfigurationError, LLMError
|
||||
from llm_connect.factory import create_adapter
|
||||
from llm_connect.fx import resolve_fx_rate
|
||||
from llm_connect.models import RunConfig
|
||||
|
|
@ -106,6 +111,30 @@ def _build_parser() -> argparse.ArgumentParser:
|
|||
spend_week.add_argument("--json", action="store_true", help="Emit JSON")
|
||||
spend_week.set_defaults(func=_spend_week)
|
||||
|
||||
balance = commands.add_parser(
|
||||
"balance",
|
||||
help=(
|
||||
"Show prepaid/account remaining for a backend "
|
||||
"(default: current default provider; --provider is one-shot only)"
|
||||
),
|
||||
)
|
||||
balance.add_argument(
|
||||
"--provider",
|
||||
default=None,
|
||||
help=(
|
||||
"Backend to query for this command only (does not change library defaults). "
|
||||
"Omit to use the current default provider."
|
||||
),
|
||||
)
|
||||
balance.add_argument("--json", action="store_true", help="Emit JSON")
|
||||
balance.add_argument(
|
||||
"--eur-per-usd",
|
||||
type=float,
|
||||
default=None,
|
||||
help="Override FX rate (euros per one USD)",
|
||||
)
|
||||
balance.set_defaults(func=_balance_show)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
|
|
@ -340,6 +369,27 @@ def _fmt_local(dt: datetime) -> str:
|
|||
return dt.isoformat()
|
||||
|
||||
|
||||
def _balance_show(args: argparse.Namespace) -> int:
|
||||
try:
|
||||
balance = get_account_balance(args.provider, fx=args.eur_per_usd)
|
||||
except LLMBalanceUnsupportedError as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
return 2
|
||||
except LLMConfigurationError as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
return 2
|
||||
except LLMError as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if args.json:
|
||||
print(json.dumps(balance.to_dict(), indent=2, sort_keys=True))
|
||||
return 0
|
||||
|
||||
print(format_balance_human(balance))
|
||||
return 0
|
||||
|
||||
|
||||
def _classes_payload(classes: Iterable[ProblemClass]) -> dict[str, dict[str, Any]]:
|
||||
return {
|
||||
problem_class.name: {
|
||||
|
|
|
|||
|
|
@ -90,6 +90,30 @@ class LLMBudgetExceededError(LLMError):
|
|||
self.requested = requested
|
||||
|
||||
|
||||
class LLMBalanceUnsupportedError(LLMConfigurationError):
|
||||
"""Account balance is not supported for the selected provider.
|
||||
|
||||
Attributes:
|
||||
provider: Provider id that was requested.
|
||||
supported: Provider ids that currently implement balance.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
provider: str = "",
|
||||
supported: Optional[list[str]] = None,
|
||||
cause: Optional[Exception] = None,
|
||||
context: Optional[Dict[str, Any]] = None,
|
||||
):
|
||||
supported_list = list(supported or [])
|
||||
if context is None:
|
||||
context = {"provider": provider, "supported": supported_list}
|
||||
super().__init__(message, cause=cause, context=context)
|
||||
self.provider = provider
|
||||
self.supported = supported_list
|
||||
|
||||
|
||||
class LLMSubprocessError(LLMError):
|
||||
"""Claude Code CLI subprocess failed.
|
||||
|
||||
|
|
|
|||
141
tests/test_balance.py
Normal file
141
tests/test_balance.py
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
import pytest
|
||||
|
||||
from llm_connect.balance import (
|
||||
AccountBalance,
|
||||
BalanceClientRegistry,
|
||||
OpenRouterBalanceClient,
|
||||
format_balance_human,
|
||||
get_account_balance,
|
||||
resolve_balance_provider,
|
||||
)
|
||||
from llm_connect.config import LLMConfig
|
||||
from llm_connect.exceptions import LLMAPIError, LLMBalanceUnsupportedError, LLMConfigurationError
|
||||
from llm_connect.openrouter import _DEFAULT_MODEL as OPENROUTER_DEFAULT_MODEL
|
||||
|
||||
|
||||
def test_resolve_balance_provider_explicit_and_default():
|
||||
assert resolve_balance_provider("OpenRouter") == "openrouter"
|
||||
assert resolve_balance_provider(None) == LLMConfig().provider
|
||||
assert resolve_balance_provider("") == LLMConfig().provider
|
||||
assert resolve_balance_provider(" gemini ") == "gemini"
|
||||
|
||||
|
||||
def test_resolve_balance_provider_does_not_mutate_defaults():
|
||||
before_provider = LLMConfig().provider
|
||||
before_model = OPENROUTER_DEFAULT_MODEL
|
||||
resolve_balance_provider("gemini")
|
||||
resolve_balance_provider("openai")
|
||||
assert LLMConfig().provider == before_provider
|
||||
assert OPENROUTER_DEFAULT_MODEL == before_model
|
||||
|
||||
|
||||
def test_registry_unsupported_provider():
|
||||
registry = BalanceClientRegistry.default()
|
||||
with pytest.raises(LLMBalanceUnsupportedError, match="gemini") as exc_info:
|
||||
registry.create("gemini")
|
||||
assert "openrouter" in exc_info.value.supported
|
||||
|
||||
|
||||
def test_openrouter_balance_client_maps_credits_and_key_limit():
|
||||
def fake_get(url, headers=None, timeout=30):
|
||||
if url.endswith("/credits"):
|
||||
return {"data": {"total_credits": 25.0, "total_usage": 12.5}}
|
||||
if url.endswith("/auth/key"):
|
||||
return {
|
||||
"data": {
|
||||
"limit": 10.0,
|
||||
"limit_remaining": 9.5,
|
||||
"limit_reset": "monthly",
|
||||
"usage": 1.5,
|
||||
}
|
||||
}
|
||||
raise AssertionError(f"unexpected url {url}")
|
||||
|
||||
client = OpenRouterBalanceClient(
|
||||
api_key="sk-test",
|
||||
get_json_fn=fake_get,
|
||||
)
|
||||
balance = client.get_balance(fx=1.0)
|
||||
|
||||
assert balance.provider == "openrouter"
|
||||
assert balance.credits_remaining == pytest.approx(12.5)
|
||||
assert balance.credits_total == pytest.approx(25.0)
|
||||
assert balance.credits_used == pytest.approx(12.5)
|
||||
assert balance.limit_remaining == pytest.approx(9.5)
|
||||
assert balance.limit == pytest.approx(10.0)
|
||||
assert balance.limit_reset == "monthly"
|
||||
assert balance.credits_remaining_eur == pytest.approx(12.5)
|
||||
assert balance.limit_remaining_eur == pytest.approx(9.5)
|
||||
assert "credits" in balance.source
|
||||
assert "auth/key" in balance.source
|
||||
|
||||
|
||||
def test_openrouter_partial_failure_still_returns_one_signal():
|
||||
def fake_get(url, headers=None, timeout=30):
|
||||
if url.endswith("/credits"):
|
||||
raise LLMAPIError("down", status_code=500)
|
||||
if url.endswith("/auth/key"):
|
||||
return {"data": {"limit": 5, "limit_remaining": 4, "limit_reset": "monthly"}}
|
||||
raise AssertionError(url)
|
||||
|
||||
balance = OpenRouterBalanceClient(api_key="sk-test", get_json_fn=fake_get).get_balance(
|
||||
fx=0.92
|
||||
)
|
||||
assert balance.credits_remaining is None
|
||||
assert balance.limit_remaining == pytest.approx(4.0)
|
||||
assert balance.limit_remaining_eur == pytest.approx(3.68)
|
||||
assert "partial" in balance.source
|
||||
|
||||
|
||||
def test_openrouter_both_endpoints_fail():
|
||||
def fake_get(url, headers=None, timeout=30):
|
||||
raise LLMAPIError("nope", status_code=503)
|
||||
|
||||
with pytest.raises(LLMAPIError, match="both"):
|
||||
OpenRouterBalanceClient(api_key="sk-test", get_json_fn=fake_get).get_balance()
|
||||
|
||||
|
||||
def test_openrouter_missing_key():
|
||||
client = OpenRouterBalanceClient(api_key=None, get_json_fn=lambda *a, **k: {})
|
||||
# Force empty key even if env has one
|
||||
client._api_key = None
|
||||
with pytest.raises(LLMConfigurationError, match="API key"):
|
||||
client.get_balance()
|
||||
|
||||
|
||||
def test_get_account_balance_uses_registry():
|
||||
class Stub:
|
||||
provider_id = "stub"
|
||||
|
||||
def get_balance(self, *, fx=None):
|
||||
return AccountBalance(
|
||||
provider="stub",
|
||||
currency="USD",
|
||||
source="stub:test",
|
||||
credits_remaining=1.0,
|
||||
)
|
||||
|
||||
registry = BalanceClientRegistry({"stub": lambda: Stub()})
|
||||
balance = get_account_balance("stub", registry=registry)
|
||||
assert balance.credits_remaining == 1.0
|
||||
|
||||
|
||||
def test_format_balance_human_mentions_both_signals():
|
||||
text = format_balance_human(
|
||||
AccountBalance(
|
||||
provider="openrouter",
|
||||
currency="USD",
|
||||
source="openrouter:/api/v1/credits+auth/key",
|
||||
credits_remaining=12.5,
|
||||
credits_remaining_eur=11.5,
|
||||
limit_remaining=9.5,
|
||||
limit_remaining_eur=8.74,
|
||||
limit=10,
|
||||
limit_reset="monthly",
|
||||
fx_source="explicit",
|
||||
)
|
||||
)
|
||||
assert "account credits remaining" in text
|
||||
assert "key limit remaining" in text
|
||||
assert "EUR" in text
|
||||
assert "openrouter" in text
|
||||
|
|
@ -113,6 +113,74 @@ def test_cost_estimate_cli(capsys):
|
|||
assert payload["cost_eur"] == 0.018
|
||||
|
||||
|
||||
def test_balance_cli_json(monkeypatch, capsys):
|
||||
from llm_connect.balance import AccountBalance
|
||||
|
||||
def fake_get_balance(provider=None, *, registry=None, fx=None):
|
||||
assert provider is None # default path
|
||||
return AccountBalance(
|
||||
provider="openrouter",
|
||||
currency="USD",
|
||||
source="openrouter:/api/v1/credits+auth/key",
|
||||
credits_remaining=12.5,
|
||||
credits_remaining_eur=11.5,
|
||||
limit_remaining=9.5,
|
||||
limit_remaining_eur=8.74,
|
||||
limit=10.0,
|
||||
limit_reset="monthly",
|
||||
fx_source="explicit",
|
||||
)
|
||||
|
||||
monkeypatch.setattr("llm_connect.cli.get_account_balance", fake_get_balance)
|
||||
assert main(["balance", "--json"]) == 0
|
||||
payload = json.loads(capsys.readouterr().out)
|
||||
assert payload["provider"] == "openrouter"
|
||||
assert payload["credits_remaining"] == 12.5
|
||||
assert payload["limit_remaining"] == 9.5
|
||||
|
||||
|
||||
def test_balance_cli_explicit_provider_one_shot(monkeypatch, capsys):
|
||||
from llm_connect.balance import AccountBalance
|
||||
from llm_connect.config import LLMConfig
|
||||
from llm_connect.openrouter import _DEFAULT_MODEL
|
||||
|
||||
seen = {}
|
||||
|
||||
def fake_get_balance(provider=None, *, registry=None, fx=None):
|
||||
seen["provider"] = provider
|
||||
return AccountBalance(
|
||||
provider="openrouter",
|
||||
currency="USD",
|
||||
source="test",
|
||||
credits_remaining=1.0,
|
||||
)
|
||||
|
||||
before_provider = LLMConfig().provider
|
||||
before_model = _DEFAULT_MODEL
|
||||
monkeypatch.setattr("llm_connect.cli.get_account_balance", fake_get_balance)
|
||||
assert main(["balance", "--provider", "openrouter"]) == 0
|
||||
assert seen["provider"] == "openrouter"
|
||||
assert LLMConfig().provider == before_provider
|
||||
assert _DEFAULT_MODEL == before_model
|
||||
assert "account credits remaining" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_balance_cli_unsupported_provider(monkeypatch, capsys):
|
||||
from llm_connect.exceptions import LLMBalanceUnsupportedError
|
||||
|
||||
def boom(provider=None, *, registry=None, fx=None):
|
||||
raise LLMBalanceUnsupportedError(
|
||||
"not supported",
|
||||
provider=provider or "",
|
||||
supported=["openrouter"],
|
||||
)
|
||||
|
||||
monkeypatch.setattr("llm_connect.cli.get_account_balance", boom)
|
||||
assert main(["balance", "--provider", "gemini"]) == 2
|
||||
err = capsys.readouterr().err
|
||||
assert "not supported" in err
|
||||
|
||||
|
||||
def test_spend_week_current_and_last(tmp_path, capsys, monkeypatch):
|
||||
ledger_path = tmp_path / "usage.jsonl"
|
||||
ledger = UsageLedger(ledger_path)
|
||||
|
|
|
|||
|
|
@ -79,3 +79,20 @@ def test_wp_0007_spend_primitives_are_exported_from_package_root():
|
|||
for name in expected_names:
|
||||
assert hasattr(llm_connect, name)
|
||||
assert name in llm_connect.__all__
|
||||
|
||||
|
||||
def test_wp_0008_balance_primitives_are_exported_from_package_root():
|
||||
expected_names = [
|
||||
"AccountBalance",
|
||||
"BalanceClientRegistry",
|
||||
"OpenRouterBalanceClient",
|
||||
"ProviderBalanceClient",
|
||||
"format_balance_human",
|
||||
"get_account_balance",
|
||||
"resolve_balance_provider",
|
||||
"LLMBalanceUnsupportedError",
|
||||
]
|
||||
|
||||
for name in expected_names:
|
||||
assert hasattr(llm_connect, name)
|
||||
assert name in llm_connect.__all__
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Provider-scoped account balance CLI"
|
||||
domain: agents
|
||||
repo: llm-connect
|
||||
status: ready
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: provider-account-balance-cli
|
||||
planning_priority: high
|
||||
|
|
@ -19,7 +19,7 @@ state_hub_workstream_id: "b8302848-5437-44ad-aec9-5c2df8cd5a47"
|
|||
|
||||
# LLM-WP-0008 — Provider-scoped account balance CLI
|
||||
|
||||
**status:** ready
|
||||
**status:** finished
|
||||
**owner:** codex
|
||||
|
||||
## Purpose
|
||||
|
|
@ -196,7 +196,7 @@ shipping a fake production path.
|
|||
|
||||
```task
|
||||
id: LLM-WP-0008-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "7df3a274-4ed9-429b-aabc-15fc52c91bfc"
|
||||
```
|
||||
|
|
@ -208,7 +208,7 @@ Tests: default resolution, unknown provider, unsupported provider.
|
|||
|
||||
```task
|
||||
id: LLM-WP-0008-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "ff564d7c-5131-4e3e-b0cd-51a41e7a1261"
|
||||
```
|
||||
|
|
@ -220,7 +220,7 @@ failure of one endpoint). Live smoke optional / manual.
|
|||
|
||||
```task
|
||||
id: LLM-WP-0008-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "61879d8f-2271-4fa1-9a7b-dbdf7a0d408d"
|
||||
```
|
||||
|
|
@ -232,7 +232,7 @@ unsupported backends. Tests via mocked client registration.
|
|||
|
||||
```task
|
||||
id: LLM-WP-0008-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "9da2c09a-fe32-4602-bcad-5f0f46d4da0f"
|
||||
```
|
||||
|
|
@ -243,7 +243,7 @@ Note OpenRouter dual signals (account credits vs key limit).
|
|||
|
||||
```task
|
||||
id: LLM-WP-0008-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: low
|
||||
state_hub_task_id: "ebf67737-3f3d-458b-9dec-9224f08c8016"
|
||||
```
|
||||
|
|
@ -255,18 +255,18 @@ within rounding of OpenRouter dashboard figures; document any known drift
|
|||
|
||||
## Acceptance
|
||||
|
||||
- [ ] `llm-connect balance` reports OpenRouter remaining budget when default
|
||||
- [x] `llm-connect balance` reports OpenRouter remaining budget when default
|
||||
provider is `openrouter`, without changing any defaults
|
||||
- [ ] `llm-connect balance --provider openrouter` same result; still no default
|
||||
- [x] `llm-connect balance --provider openrouter` same result; still no default
|
||||
mutation
|
||||
- [ ] `llm-connect balance --provider <unsupported>` fails clearly (not with
|
||||
- [x] `llm-connect balance --provider <unsupported>` fails clearly (not with
|
||||
OpenRouter data)
|
||||
- [ ] Account credits remaining and key limit remaining are distinguishable in
|
||||
- [x] Account credits remaining and key limit remaining are distinguishable in
|
||||
output when both are available
|
||||
- [ ] EUR amounts shown when FX available
|
||||
- [ ] Adding a future provider is “register a balance client”, not hard-coding
|
||||
- [x] EUR amounts shown when FX available
|
||||
- [x] Adding a future provider is “register a balance client”, not hard-coding
|
||||
OpenRouter into the CLI command body
|
||||
- [ ] Offline unit tests pass without network
|
||||
- [x] Offline unit tests pass without network
|
||||
|
||||
## Risks / notes
|
||||
|
||||
|
|
@ -285,3 +285,14 @@ within rounding of OpenRouter dashboard figures; document any known drift
|
|||
## Implementation order
|
||||
|
||||
T01 → T02 → T03 → T04 → T05.
|
||||
|
||||
|
||||
## Implementation notes (2026-08-03)
|
||||
|
||||
- Added `llm_connect.balance` with pluggable registry; OpenRouter client uses
|
||||
`/api/v1/credits` + `/api/v1/auth/key` via new `get_json` HTTP helper.
|
||||
- CLI: `llm-connect balance [--provider] [--json] [--eur-per-usd]`.
|
||||
- Live smoke: account credits remaining ~12.72 USD and key limit remaining
|
||||
~9.98 USD; unsupported `gemini` exits 2; defaults unchanged
|
||||
(`openrouter` / `moonshotai/kimi-k3`).
|
||||
- Unit suite: 237 passed.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue