llm-connect/tests
tegwick 435da49263 Prefer JSON-bearing envelope fields, skip metadata, in Claude CLI unwrap
The first CUST-WP-0045 canary retry after 9de0f49 still failed schema
validation with `Expecting value: line 1 column 1 (char 0)`. The original
allowlist returned envelope.result verbatim, which on longer prompts
carries the model's conversational preamble ("Triage report generated
and returned via structured output. Key signals: ..."), not the
schema-enforced JSON. The actual structured payload lives in a different
envelope field whose name varies across CLI versions.

Make the unwrap order-aware:
  1. Scan envelope fields and return the first one whose value parses as
     JSON (dict, list, or a string that loads cleanly). Skip well-known
     metadata keys (type, usage, total_cost_usd, etc.) so telemetry can
     never be mistaken for the model payload.
  2. Fall back to the original text-field allowlist only when no field
     carries JSON, so non-schema callers via this same code path still
     see the model's prose.
  3. Surface the raw envelope as last resort.

This is robust against unknown envelope shapes — as long as the schema-
enforced JSON appears somewhere in a non-metadata field, the adapter
will find it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 12:44:25 +02:00
..
conftest.py feat: WP-0001 foundation + WP-0002 core extensions 2026-04-01 22:24:14 +00:00
test_adapter.py feat: WP-0001 foundation + WP-0002 core extensions 2026-04-01 22:24:14 +00:00
test_adaptive_integration.py Adaptive routing initial version 2026-05-18 11:38:12 +02:00
test_adaptive_routing.py Add adaptive cost-quality routing primitives 2026-05-17 21:32:27 +02:00
test_async.py feat: WP-0001 foundation + WP-0002 core extensions 2026-04-01 22:24:14 +00:00
test_budget.py feat: WP-0001 foundation + WP-0002 core extensions 2026-04-01 22:24:14 +00:00
test_claude_code.py Prefer JSON-bearing envelope fields, skip metadata, in Claude CLI unwrap 2026-06-02 12:44:25 +02:00
test_cli.py Implement-LLM-WP-0005-cost-model-estimators 2026-05-19 05:02:20 +02:00
test_costs.py Implement-LLM-WP-0005-cost-model-estimators 2026-05-19 05:02:20 +02:00
test_exceptions.py feat: WP-0001 foundation + WP-0002 core extensions 2026-04-01 22:24:14 +00:00
test_factory.py feat: WP-0003 — RoutingPolicy (FR-2) and HTTP serve mode (FR-1) 2026-04-01 22:34:00 +00:00
test_grading.py Add adaptive cost-quality routing primitives 2026-05-17 21:32:27 +02:00
test_models.py feat: WP-0001 foundation + WP-0002 core extensions 2026-04-01 22:24:14 +00:00
test_package_exports.py Implement-LLM-WP-0005-cost-model-estimators 2026-05-19 05:02:20 +02:00
test_problem_classes.py Implement-LLM-WP-0005-cost-model-estimators 2026-05-19 05:02:20 +02:00
test_quality.py Add adaptive cost-quality routing primitives 2026-05-17 21:32:27 +02:00
test_rates.py Implement-LLM-WP-0005-cost-model-estimators 2026-05-19 05:02:20 +02:00
test_routing.py feat: WP-0003 — RoutingPolicy (FR-2) and HTTP serve mode (FR-1) 2026-04-01 22:34:00 +00:00
test_server.py Preserve llm-connect run config in server mode 2026-05-19 20:55:02 +02:00
test_shadowing.py Add adaptive cost-quality routing primitives 2026-05-17 21:32:27 +02:00