llm-connect/workplans/LLM-WP-0008-provider-account-balance-cli.md

id	type	title	domain	repo	status	owner	topic_slug	planning_priority	planning_order	created	updated	depends_on_workplans	related_workplans	state_hub_workstream_id
LLM-WP-0008	workplan	Provider-scoped account balance CLI	agents	llm-connect	finished	codex	provider-account-balance-cli	high	8	2026-08-03	2026-08-03	LLM-WP-0007		d43fd87d-27de-5e00-b362-9cb96ca5ffa5

LLM-WP-0008 — Provider-scoped account balance CLI

status: finished owner: codex

Purpose

Add a CLI (and library) surface that reports remaining prepaid / account budget for the selected inference backend, without treating OpenRouter as the only possible source of truth and without changing the library or operator default provider/model.

Constraints from operator (2026-08-03):

1. Balance depends on the backend (provider). A future inference service must not silently be asked for “the OpenRouter budget”.
2. The command must accept an explicit backend selector for that invocation only — it must not switch or persist the default basemodel/provider.
3. If the backend is not specified, report balance for the currently configured default backend (today: openrouter via LLMConfig.provider / factory default).

Demand signal

After LLM-WP-0007, local estimated spend is visible (spend week), but the operator still cannot see remaining OpenRouter prepaid credits / key limit from llm-connect. That gap is backend-specific; the design must stay pluggable.

Current repo state (as of 2026-08-03)

Area	State
CLI	rates, classes, run, cost, spend — no balance/credits command
Default provider	openrouter (LLMConfig.provider, create_adapter default)
Default model	moonshotai/kimi-k3 (OpenRouter) — must not be mutated by balance CLI
OpenRouter live APIs (checked)	GET /api/v1/auth/key → limit, limit_remaining, usage, limit_reset, …; GET /api/v1/credits → total_credits, total_usage
Other providers	Gemini / OpenAI / Claude Code have no balance client yet

OpenRouter semantics worth distinguishing in the UX:

Signal	Source	Meaning
Account credits remaining	total_credits - total_usage from /credits	Prepaid balance on the OpenRouter account
Key limit remaining	limit_remaining from /auth/key	Remaining allowance on this API key (may be lower than account credits; may reset monthly)

v1 should surface both when available, labelled clearly, with EUR display optional via existing FX helpers (amounts from OpenRouter are USD).

Architecture sketch

llm_connect/
  balance.py              # NEW: ProviderBalance protocol + registry + resolve_default_provider()
  openrouter_balance.py   # NEW: OpenRouterBalanceClient (auth/key + credits)
  # later: openai_balance.py, etc. — only if a real API exists

  cli.py                  # llm-connect balance [--provider …]
  config.py               # unchanged defaults; only *read* default provider

Provider selection (critical)

# Uses default backend (today: openrouter). Does NOT change defaults.
llm-connect balance

# One-shot backend override for this command only — no config write, no
# mutation of LLMConfig defaults or OpenRouterAdapter._DEFAULT_MODEL.
llm-connect balance --provider openrouter
llm-connect balance --provider openai     # may error: unsupported until implemented

Resolution order for which backend to query:

1. Explicit --provider / library provider= argument
2. Else LLMConfig.provider default (currently "openrouter")
3. Never infer “whatever model string looks like” and never call OpenRouter because “it’s the only client we have” when another provider was selected

Calling balance with --provider X must not:

• rewrite config files, env, or key files
• change LLMConfig class defaults or module-level _DEFAULT_MODEL
• affect subsequent llm-connect run without an explicit provider on that run

Pluggable contract

@dataclass(frozen=True)
class AccountBalance:
    provider: str
    currency: str                    # usually "USD" from upstream
    # Prepaid / wallet style (optional)
    credits_total: float | None = None
    credits_used: float | None = None
    credits_remaining: float | None = None
    # Key/limit style (optional)
    limit: float | None = None
    limit_remaining: float | None = None
    limit_reset: str | None = None   # e.g. "monthly"
    usage: float | None = None
    # Display
    credits_remaining_eur: float | None = None
    limit_remaining_eur: float | None = None
    fx_source: str | None = None
    source: str                      # e.g. "openrouter:/api/v1/credits+auth/key"
    raw: dict[str, Any] = field(default_factory=dict)  # optional debug

class ProviderBalanceClient(Protocol):
    provider_id: str
    def get_balance(self) -> AccountBalance: ...

Registry maps provider_id → client factory. Unknown / unimplemented providers raise a clear LLMConfigurationError (or dedicated LLMBalanceUnsupportedError) listing which providers support balance.

OpenRouter client (v1 implementation)

• Auth: same key resolution as adapters (OPENROUTER_API_KEY / key file)
• Fetch /api/v1/credits and /api/v1/auth/key (best-effort both; degrade gracefully if one fails, but fail if both fail)
• credits_remaining = total_credits - total_usage when credits payload present
• Attach key limit_remaining / limit / limit_reset when key payload present
• Convert remaining amounts to EUR via existing resolve_fx_rate (optional, same as cost estimates)

CLI output

Human (default):

provider: openrouter
account credits remaining: 12.715255 USD (~11.70 EUR)
key limit remaining:        9.983798 USD (~9.19 EUR)  [limit=10, reset=monthly]
source: openrouter:/api/v1/credits+auth/key

--json for automation. Never print full API keys; label redaction already matches OpenRouter’s partial label if shown.

Unsupported backends

Provider	v1 behaviour
openrouter	Implemented
openai, gemini, claude-code, mock	Explicit unsupported error with exit ≠ 0
unknown	Same as factory: unknown provider error

Mock may optionally return a fixed balance for tests only if registered under mock — prefer unit-testing OpenRouter client with urllib mocks instead of shipping a fake production path.

Scope guardrails

In scope

• Pluggable balance protocol + registry
• OpenRouter implementation using real APIs
• CLI balance with --provider one-shot override
• Default-provider resolution without mutating defaults
• EUR display via existing FX
• Tests (mocked HTTP); optional live smoke gated on key
• Docs (README + short contract)

Out of scope

• Changing default provider or default model
• Writing balance into the usage ledger (different concern)
• Billing reconciliation / invoices
• Implementing Gemini/OpenAI balance clients until demand + known API
• Interactive provider picker / TUI
• Caching balance across process invocations (always live fetch in v1)

Tasks

id: LLM-WP-0008-T01
status: done
priority: high
state_hub_task_id: "ba9af2d8-8887-53f3-98d8-6e0a25846125"

Define AccountBalance + ProviderBalanceClient protocol and registry in balance.py (or equivalent). Include resolve_balance_provider(explicit) that returns the default provider when explicit is None, without side effects. Tests: default resolution, unknown provider, unsupported provider.

id: LLM-WP-0008-T02
status: done
priority: high
state_hub_task_id: "4381bf6a-dc61-5b54-b524-158e054bb32f"

OpenRouter balance client. Call /api/v1/credits and /api/v1/auth/key with existing key resolution and HTTP helpers. Map to AccountBalance with clear field semantics. Unit tests with mocked responses (including partial failure of one endpoint). Live smoke optional / manual.

id: LLM-WP-0008-T03
status: done
priority: high
state_hub_task_id: "1cd9a7c1-32c8-5487-8bae-b045bf2c245b"

CLI llm-connect balance [--provider …] [--json] [--eur-per-usd …]. Explicit --provider is per-invocation only. Omit → default backend. Print tokens-free account/key remaining in USD and €. Exit non-zero for unsupported backends. Tests via mocked client registration.

id: LLM-WP-0008-T04
status: done
priority: medium
state_hub_task_id: "0326e771-906d-540d-87d2-86e99b7d1165"

Package exports + contract docs + README. Export balance types; document that balance is backend-scoped and that --provider does not change defaults. Note OpenRouter dual signals (account credits vs key limit).

id: LLM-WP-0008-T05
status: done
priority: low
state_hub_task_id: "a602f04d-15dc-58e6-9b0f-fadf8f7edb62"

Acceptance polish. Confirm with a real key (operator or CI secret) that llm-connect balance and llm-connect balance --provider openrouter match within rounding of OpenRouter dashboard figures; document any known drift (e.g. pending usage).

Acceptance

• llm-connect balance reports OpenRouter remaining budget when default provider is openrouter, without changing any defaults
• llm-connect balance --provider openrouter same result; still no default mutation
• llm-connect balance --provider <unsupported> fails clearly (not with OpenRouter data)
• Account credits remaining and key limit remaining are distinguishable in output when both are available
• EUR amounts shown when FX available
• Adding a future provider is “register a balance client”, not hard-coding OpenRouter into the CLI command body
• Offline unit tests pass without network

Risks / notes

• Credits vs key limit: operators often care about prepaid wallet (/credits); key-level caps (/auth/key) can be lower. Showing only one risks false “I still have money” / “I’m broke” readings — show both.
• Currency: OpenRouter amounts are USD; EUR is display conversion only (same philosophy as LLM-WP-0007).
• Default provider today is openrouter: that is correct for “unspecified → default backend”, but the implementation must not special-case “unspecified always means OpenRouter” in the CLI handler — go through resolve_balance_provider(None).
• No secret leakage: never print full API keys; raw dumps only behind an explicit future --debug if ever added (not in v1 human output).

Implementation order

T01 → T02 → T03 → T04 → T05.

Implementation notes (2026-08-03)

• Added llm_connect.balance with pluggable registry; OpenRouter client uses /api/v1/credits + /api/v1/auth/key via new get_json HTTP helper.
• CLI: llm-connect balance [--provider] [--json] [--eur-per-usd].
• Live smoke: account credits remaining ~12.72 USD and key limit remaining ~9.98 USD; unsupported gemini exits 2; defaults unchanged (openrouter / moonshotai/kimi-k3).
• Unit suite: 237 passed.