Restore qualified task identities and finish transport protocol types
Assistant: codex Assistant-Model: gpt-5.6-luna Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc
This commit is contained in:
parent
4055986c98
commit
68ffe9ef76
9 changed files with 899 additions and 157 deletions
|
|
@ -71,3 +71,31 @@ review compatibility of the exact CLI/beta combination against the actual
|
|||
provider before accepting a live profile. Local fake-provider evidence cannot
|
||||
close this task or establish a hard live EUR ceiling. Reuse GLAS-WP-0015 identity
|
||||
and native-delivery owner work; completed verifier CCRs are not reopened.
|
||||
|
||||
|
||||
Pre-release quality return: configured repository-wide checks expose 177 Ruff
|
||||
diagnostics and 36 mypy errors, reproduced identically at the original 00560945
|
||||
source baseline. The new transport adds none after its protocol types were
|
||||
completed. The 263 passing tests are not a claim of green full-repository CI.
|
||||
Resolve or explicitly disposition those existing checks before an owner accepts
|
||||
the protected artifact/release. Evidence:
|
||||
`docs/evidence/2026-09-09-request-admission-quality.json`.
|
||||
|
||||
## Repair historical source identities blocking primary synchronization
|
||||
|
||||
```task
|
||||
id: LLM-WP-0009-T04
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
HFACT-WP-0001-T02 side quest: qualified 65 historical parent-local task IDs to
|
||||
match their already-existing Hub record IDs across five finished workplans.
|
||||
Preserved all workplan/task UUIDs, parent links, statuses and task content;
|
||||
qualified local references and retained an exact before/after mapping in
|
||||
`docs/evidence/2026-09-09-legacy-task-qualification.json`. This is source identity
|
||||
repair, not a UUID migration, record recreation or retirement. Six missing
|
||||
historical ad-hoc source bindings also resolve to existing matching Hub UUIDs;
|
||||
only Repo Manager's managed-field write may restore those pointers. The
|
||||
AGENTS.md versus registry prefix disagreement remains a future-instruction
|
||||
issue; published workplan IDs are unchanged.
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@ and state-hub housekeeping.
|
|||
## Tasks
|
||||
|
||||
```task
|
||||
id: T01
|
||||
id: LLM-WP-0001-T01
|
||||
title: 'Create SCOPE.md'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -36,7 +36,7 @@ state_hub_task_id: "c38c5a79-4ce5-4088-9a21-ac65e09b12ba"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T02
|
||||
id: LLM-WP-0001-T02
|
||||
title: 'Fill .claude/rules/ stubs: architecture.md, stack-and-commands.md, repo-boundary.md'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -44,7 +44,7 @@ state_hub_task_id: "6a15c794-d0f7-4d9c-a3ac-850f8c5bd5e9"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T03
|
||||
id: LLM-WP-0001-T03
|
||||
title: 'Create ARCHITECTURE-LAYERS.md with layer map, scorecard stub, next-review date'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -52,7 +52,7 @@ state_hub_task_id: "af1c63ac-e4be-495a-9fdb-68eddebfcb75"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T04
|
||||
id: LLM-WP-0001-T04
|
||||
title: 'Create /contracts/ tree (core/, functional/, config/)'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -60,7 +60,7 @@ state_hub_task_id: "da5a7986-5c47-4c4c-a8f6-a58956127535"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T05
|
||||
id: LLM-WP-0001-T05
|
||||
title: 'Core contract doc: LLMAdapter interface invariants, RunConfig/LLMResponse field contracts'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -68,7 +68,7 @@ state_hub_task_id: "01237203-0582-4bc4-a308-075e991e8e99"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T06
|
||||
id: LLM-WP-0001-T06
|
||||
title: 'Functional contract stubs for all 4 adapters + embedding adapters (maturity: Beta)'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -76,7 +76,7 @@ state_hub_task_id: "2bee5174-d3d7-4267-9cee-6e0e9b5cc731"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T07
|
||||
id: LLM-WP-0001-T07
|
||||
title: 'Create tests/ with conftest.py, wire pytest in pyproject.toml'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -84,7 +84,7 @@ state_hub_task_id: "b6dccf3e-8742-486e-a6a7-82577866a3bc"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T08
|
||||
id: LLM-WP-0001-T08
|
||||
title: 'Unit tests: RunConfig, LLMResponse, MockLLMAdapter, full exception hierarchy'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -92,7 +92,7 @@ state_hub_task_id: "cc05b67d-f956-458a-908f-2ff58b1d33d3"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T09
|
||||
id: LLM-WP-0001-T09
|
||||
title: 'Unit tests: create_adapter (all providers + unknown provider error), create_embedding_adapter'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -100,7 +100,7 @@ state_hub_task_id: "8f9ec054-79ab-411d-8204-9d764bbbed98"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T10
|
||||
id: LLM-WP-0001-T10
|
||||
title: 'Add ruff, mypy to dev deps in pyproject.toml'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -108,7 +108,7 @@ state_hub_task_id: "044ee879-6baa-42fd-a0a4-a43dac0eacbb"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T11
|
||||
id: LLM-WP-0001-T11
|
||||
title: 'CI workflow: pytest + ruff + mypy'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -116,7 +116,7 @@ state_hub_task_id: "699eef00-e9df-4de0-b7e6-61cfaace9617"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T12
|
||||
id: LLM-WP-0001-T12
|
||||
title: 'State hub: register this host path, SBOM refresh'
|
||||
priority: low
|
||||
status: done
|
||||
|
|
@ -125,18 +125,18 @@ state_hub_task_id: "c0853a23-52ae-499e-9a49-e7b65749b508"
|
|||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|-------|----------|--------|
|
||||
| T01 | Create `SCOPE.md` | high | done |
|
||||
| T02 | Fill `.claude/rules/` stubs: `architecture.md`, `stack-and-commands.md`, `repo-boundary.md` | high | done |
|
||||
| T03 | Create `ARCHITECTURE-LAYERS.md` with layer map, scorecard stub, next-review date | high | done |
|
||||
| T04 | Create `/contracts/` tree (`core/`, `functional/`, `config/`) | high | done |
|
||||
| T05 | Core contract doc: `LLMAdapter` interface invariants, `RunConfig`/`LLMResponse` field contracts | high | done |
|
||||
| T06 | Functional contract stubs for all 4 adapters + embedding adapters (maturity: Beta) | medium | done |
|
||||
| T07 | Create `tests/` with `conftest.py`, wire pytest in `pyproject.toml` | high | done |
|
||||
| T08 | Unit tests: `RunConfig`, `LLMResponse`, `MockLLMAdapter`, full exception hierarchy | high | done |
|
||||
| T09 | Unit tests: `create_adapter` (all providers + unknown provider error), `create_embedding_adapter` | high | done |
|
||||
| T10 | Add `ruff`, `mypy` to dev deps in `pyproject.toml` | medium | done |
|
||||
| T11 | CI workflow: pytest + ruff + mypy | medium | done |
|
||||
| T12 | State hub: register this host path, SBOM refresh | low | done |
|
||||
| LLM-WP-0001-T01 | Create `SCOPE.md` | high | done |
|
||||
| LLM-WP-0001-T02 | Fill `.claude/rules/` stubs: `architecture.md`, `stack-and-commands.md`, `repo-boundary.md` | high | done |
|
||||
| LLM-WP-0001-T03 | Create `ARCHITECTURE-LAYERS.md` with layer map, scorecard stub, next-review date | high | done |
|
||||
| LLM-WP-0001-T04 | Create `/contracts/` tree (`core/`, `functional/`, `config/`) | high | done |
|
||||
| LLM-WP-0001-T05 | Core contract doc: `LLMAdapter` interface invariants, `RunConfig`/`LLMResponse` field contracts | high | done |
|
||||
| LLM-WP-0001-T06 | Functional contract stubs for all 4 adapters + embedding adapters (maturity: Beta) | medium | done |
|
||||
| LLM-WP-0001-T07 | Create `tests/` with `conftest.py`, wire pytest in `pyproject.toml` | high | done |
|
||||
| LLM-WP-0001-T08 | Unit tests: `RunConfig`, `LLMResponse`, `MockLLMAdapter`, full exception hierarchy | high | done |
|
||||
| LLM-WP-0001-T09 | Unit tests: `create_adapter` (all providers + unknown provider error), `create_embedding_adapter` | high | done |
|
||||
| LLM-WP-0001-T10 | Add `ruff`, `mypy` to dev deps in `pyproject.toml` | medium | done |
|
||||
| LLM-WP-0001-T11 | CI workflow: pytest + ruff + mypy | medium | done |
|
||||
| LLM-WP-0001-T12 | State hub: register this host path, SBOM refresh | low | done |
|
||||
|
||||
## Exit criteria
|
||||
|
||||
|
|
|
|||
|
|
@ -38,12 +38,12 @@ Both changes are Core-layer modifications under GAAF-2026:
|
|||
`asyncio.get_event_loop().run_in_executor(None, ...)` fallback so existing
|
||||
adapters remain valid; native async overrides are provided per adapter.
|
||||
|
||||
Core contract doc (from WP-0001 T05) must be updated after each change.
|
||||
Core contract doc (from LLM-WP-0001-T05) must be updated after each change.
|
||||
|
||||
## Tasks
|
||||
|
||||
```task
|
||||
id: T01
|
||||
id: LLM-WP-0002-T01
|
||||
title: 'BudgetTracker dataclass: total, spent, remaining(), thread-safe increment'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -51,7 +51,7 @@ state_hub_task_id: "ae27c363-339a-4f78-9737-cf872698f6d8"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T02
|
||||
id: LLM-WP-0002-T02
|
||||
title: 'LLMBudgetExceededError(LLMError) in exceptions.py'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -59,7 +59,7 @@ state_hub_task_id: "ea6f6ef7-2cb2-48e2-b9c9-f2b84a1a242b"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T03
|
||||
id: LLM-WP-0002-T03
|
||||
title: 'Optional budget_tracker field on RunConfig'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -67,7 +67,7 @@ state_hub_task_id: "fe6dbb73-5d04-45e6-aa91-5eff79aae7ee"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T04
|
||||
id: LLM-WP-0002-T04
|
||||
title: 'Enforcement: adapters check/update tracker, raise LLMBudgetExceededError when exceeded'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -75,7 +75,7 @@ state_hub_task_id: "8fd21bc2-598e-4449-8c86-eacde760e23f"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T05
|
||||
id: LLM-WP-0002-T05
|
||||
title: 'Update Core contract doc for BudgetTracker and RunConfig changes'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -83,7 +83,7 @@ state_hub_task_id: "e15745f5-9bb7-45d6-a36b-3a345fb0e9f1"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T06
|
||||
id: LLM-WP-0002-T06
|
||||
title: 'Tests: single call, delegation chain, exceeded error, multi-adapter shared tracker'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -91,7 +91,7 @@ state_hub_task_id: "5af37ade-3dd0-4ce9-8ead-be9887913bab"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T07
|
||||
id: LLM-WP-0002-T07
|
||||
title: 'Add async_execute_prompt to LLMAdapter ABC with default executor fallback'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -99,7 +99,7 @@ state_hub_task_id: "e221e630-658f-4adb-9f00-7b7df7ab8cb4"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T08
|
||||
id: LLM-WP-0002-T08
|
||||
title: 'Native async override in OpenAIAdapter, GeminiAdapter, OpenRouterAdapter'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -107,7 +107,7 @@ state_hub_task_id: "a75c2b2a-e4ef-4cbd-9c5f-7e98c8d3d7e8"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T09
|
||||
id: LLM-WP-0002-T09
|
||||
title: 'Native async for ClaudeCodeAdapter via asyncio.create_subprocess_exec'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -115,7 +115,7 @@ state_hub_task_id: "1c50889f-28ed-4c6e-a788-1fc7dcc5a2c3"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T10
|
||||
id: LLM-WP-0002-T10
|
||||
title: 'Update Core contract doc for async_execute_prompt'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -123,7 +123,7 @@ state_hub_task_id: "fa4f9e80-ddee-4d05-a239-fe09e633b0cb"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T11
|
||||
id: LLM-WP-0002-T11
|
||||
title: 'Tests: asyncio.gather over N adapters, timeout propagation, budget interaction'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -134,22 +134,22 @@ state_hub_task_id: "bca78609-7f7c-4548-8857-a72e4c760dc6"
|
|||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|-------|----------|--------|
|
||||
| T01 | `BudgetTracker` dataclass: `total`, `spent`, `remaining()`, thread-safe increment | high | done |
|
||||
| T02 | `LLMBudgetExceededError(LLMError)` in `exceptions.py` | high | done |
|
||||
| T03 | Optional `budget_tracker: BudgetTracker \| None` field on `RunConfig` | high | done |
|
||||
| T04 | Enforcement: each adapter checks/updates tracker around call; raises on exceeded | high | done |
|
||||
| T05 | Update Core contract doc | medium | done |
|
||||
| T06 | Tests: single call, delegation chain (A→B→C shared tracker), exceeded error, multi-adapter | high | done |
|
||||
| LLM-WP-0002-T01 | `BudgetTracker` dataclass: `total`, `spent`, `remaining()`, thread-safe increment | high | done |
|
||||
| LLM-WP-0002-T02 | `LLMBudgetExceededError(LLMError)` in `exceptions.py` | high | done |
|
||||
| LLM-WP-0002-T03 | Optional `budget_tracker: BudgetTracker \| None` field on `RunConfig` | high | done |
|
||||
| LLM-WP-0002-T04 | Enforcement: each adapter checks/updates tracker around call; raises on exceeded | high | done |
|
||||
| LLM-WP-0002-T05 | Update Core contract doc | medium | done |
|
||||
| LLM-WP-0002-T06 | Tests: single call, delegation chain (A→B→C shared tracker), exceeded error, multi-adapter | high | done |
|
||||
|
||||
### FR-3 — async_execute_prompt
|
||||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|-------|----------|--------|
|
||||
| T07 | Add `async_execute_prompt` to `LLMAdapter` ABC with default executor fallback | high | done |
|
||||
| T08 | Native async override in `OpenAIAdapter`, `GeminiAdapter`, `OpenRouterAdapter` | high | done |
|
||||
| T09 | Native async for `ClaudeCodeAdapter` via `asyncio.create_subprocess_exec` | high | done |
|
||||
| T10 | Update Core contract doc | medium | done |
|
||||
| T11 | Tests: `asyncio.gather` over N adapters, timeout propagation, budget interaction | high | done |
|
||||
| LLM-WP-0002-T07 | Add `async_execute_prompt` to `LLMAdapter` ABC with default executor fallback | high | done |
|
||||
| LLM-WP-0002-T08 | Native async override in `OpenAIAdapter`, `GeminiAdapter`, `OpenRouterAdapter` | high | done |
|
||||
| LLM-WP-0002-T09 | Native async for `ClaudeCodeAdapter` via `asyncio.create_subprocess_exec` | high | done |
|
||||
| LLM-WP-0002-T10 | Update Core contract doc | medium | done |
|
||||
| LLM-WP-0002-T11 | Tests: `asyncio.gather` over N adapters, timeout propagation, budget interaction | high | done |
|
||||
|
||||
## Exit criteria
|
||||
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ Both additions are Functional-layer under GAAF-2026:
|
|||
## Tasks
|
||||
|
||||
```task
|
||||
id: T01
|
||||
id: LLM-WP-0003-T01
|
||||
title: 'RoutingPolicy data model: rules list with task_type, prefer, max_cost_per_1k, fallback'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -45,7 +45,7 @@ state_hub_task_id: "85cf92fd-cddd-4e19-8782-970f6480a37f"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T02
|
||||
id: LLM-WP-0003-T02
|
||||
title: 'policy.resolve(task_type) returns configured LLMAdapter'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -53,7 +53,7 @@ state_hub_task_id: "352701ce-4b21-4f5d-a22e-462136e58fd2"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T03
|
||||
id: LLM-WP-0003-T03
|
||||
title: 'Export RoutingPolicy from llm_connect.__init__ and update __all__'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -61,7 +61,7 @@ state_hub_task_id: "baeb9b39-7fee-4f2b-86cc-ce64ff9e9b95"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T04
|
||||
id: LLM-WP-0003-T04
|
||||
title: 'Functional contract doc for RoutingPolicy'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -69,7 +69,7 @@ state_hub_task_id: "aa4488c6-950e-4cea-99b1-89defa4677ce"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T05
|
||||
id: LLM-WP-0003-T05
|
||||
title: 'Tests: rule match, cost-cap fallback, unknown task_type fallback, no-match default'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -77,7 +77,7 @@ state_hub_task_id: "a4ad9c9e-64a4-44f0-85f3-b9cfe9ef59f7"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T06
|
||||
id: LLM-WP-0003-T06
|
||||
title: 'Design /execute JSON schema (request: provider, model, prompt, config; response: LLMResponse)'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -85,7 +85,7 @@ state_hub_task_id: "cf79bce2-8d1a-4708-90b2-5e6569908b14"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T07
|
||||
id: LLM-WP-0003-T07
|
||||
title: 'Implement llm_connect/server.py: POST /execute, GET /health'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -93,7 +93,7 @@ state_hub_task_id: "c91964ab-7366-4b34-acd4-1ee12f96881e"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T08
|
||||
id: LLM-WP-0003-T08
|
||||
title: 'python -m llm_connect.server --port N --provider X --model Y CLI entry point'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -101,7 +101,7 @@ state_hub_task_id: "e3115bb4-cf3b-4ca0-9992-136e317068ac"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T09
|
||||
id: LLM-WP-0003-T09
|
||||
title: 'Add server optional dep (httpx or aiohttp) to pyproject.toml'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -109,7 +109,7 @@ state_hub_task_id: "2caf5531-8e10-40e9-a595-8652882a10e0"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T10
|
||||
id: LLM-WP-0003-T10
|
||||
title: 'Functional contract doc: HTTP API schema (request/response shapes, error codes)'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -117,7 +117,7 @@ state_hub_task_id: "dc3c81c2-698d-4fee-b1dd-1af156a4276f"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T11
|
||||
id: LLM-WP-0003-T11
|
||||
title: 'Tests: server POST round-trip (MockAdapter), GET /health, error responses'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -128,22 +128,22 @@ state_hub_task_id: "848a1622-abdd-4938-8bb4-3da27f5f9867"
|
|||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|-------|----------|--------|
|
||||
| T01 | `RoutingPolicy` data model: `rules` list with `task_type`, `prefer`, `max_cost_per_1k`, `fallback` | high | done |
|
||||
| T02 | `policy.resolve(task_type)` → returns configured `LLMAdapter` | high | done |
|
||||
| T03 | Export from `llm_connect.__init__` and update `__all__` | medium | done |
|
||||
| T04 | Functional contract doc for `RoutingPolicy` | medium | done |
|
||||
| T05 | Tests: rule match, cost-cap fallback, unknown task_type fallback, no-match default | high | done |
|
||||
| LLM-WP-0003-T01 | `RoutingPolicy` data model: `rules` list with `task_type`, `prefer`, `max_cost_per_1k`, `fallback` | high | done |
|
||||
| LLM-WP-0003-T02 | `policy.resolve(task_type)` → returns configured `LLMAdapter` | high | done |
|
||||
| LLM-WP-0003-T03 | Export from `llm_connect.__init__` and update `__all__` | medium | done |
|
||||
| LLM-WP-0003-T04 | Functional contract doc for `RoutingPolicy` | medium | done |
|
||||
| LLM-WP-0003-T05 | Tests: rule match, cost-cap fallback, unknown task_type fallback, no-match default | high | done |
|
||||
|
||||
### FR-1 — HTTP serve mode
|
||||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|-------|----------|--------|
|
||||
| T06 | Design `/execute` JSON schema (request: provider, model, prompt, config; response: LLMResponse fields) | high | done |
|
||||
| T07 | Implement `llm_connect/server.py` — minimal HTTP server, `POST /execute`, `GET /health` | high | done |
|
||||
| T08 | `python -m llm_connect.server --port N --provider X --model Y` CLI entry point | high | done |
|
||||
| T09 | Add `httpx` or `aiohttp` server dep under `[project.optional-dependencies] server` | medium | done |
|
||||
| T10 | Functional contract doc (API schema — request/response shapes, error codes) | medium | done |
|
||||
| T11 | Tests: spin up server in subprocess or via `TestClient`, POST round-trip (MockAdapter), error responses | high | done |
|
||||
| LLM-WP-0003-T06 | Design `/execute` JSON schema (request: provider, model, prompt, config; response: LLMResponse fields) | high | done |
|
||||
| LLM-WP-0003-T07 | Implement `llm_connect/server.py` — minimal HTTP server, `POST /execute`, `GET /health` | high | done |
|
||||
| LLM-WP-0003-T08 | `python -m llm_connect.server --port N --provider X --model Y` CLI entry point | high | done |
|
||||
| LLM-WP-0003-T09 | Add `httpx` or `aiohttp` server dep under `[project.optional-dependencies] server` | medium | done |
|
||||
| LLM-WP-0003-T10 | Functional contract doc (API schema — request/response shapes, error codes) | medium | done |
|
||||
| LLM-WP-0003-T11 | Tests: spin up server in subprocess or via `TestClient`, POST round-trip (MockAdapter), error responses | high | done |
|
||||
|
||||
## Exit criteria
|
||||
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ The fenced `task` blocks below are the State Hub registration index. Keep them
|
|||
in sync with the detailed task tables that follow.
|
||||
|
||||
```task
|
||||
id: T01
|
||||
id: LLM-WP-0004-T01
|
||||
title: 'QualityObservation dataclass: task_type, adapter_id, model_id, cost_usd, quality_score (0..1), latency_ms, tokens_in, tokens_out, baseline_adapter_id, recorded_at, tags'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -86,7 +86,7 @@ state_hub_task_id: "1c285bec-c30b-45a8-a408-3f91d810a078"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T02
|
||||
id: LLM-WP-0004-T02
|
||||
title: 'QualityLedger append-only JSONL store with file-locked writes, configurable path, simple query helpers (by_task_type, recent, mean_quality)'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -94,7 +94,7 @@ state_hub_task_id: "5249f171-a047-499f-9ec4-cb50e1477765"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T03
|
||||
id: LLM-WP-0004-T03
|
||||
title: 'TTL helpers: prune_before(timestamp) and is_stale(observation, max_age)'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -102,7 +102,7 @@ state_hub_task_id: "adb255cf-7e89-4fea-b822-6be437d99789"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T04
|
||||
id: LLM-WP-0004-T04
|
||||
title: 'Functional contract doc for the ledger schema and quality_score semantics'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -110,7 +110,7 @@ state_hub_task_id: "51a33180-a99d-4aa4-96be-2fcee15bfbc3"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T05
|
||||
id: LLM-WP-0004-T05
|
||||
title: 'Tests: ledger round-trip, concurrent writes, query helpers, TTL, malformed-line resilience'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -118,7 +118,7 @@ state_hub_task_id: "458610c5-c903-4b42-9602-cd511999c9ba"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T06
|
||||
id: LLM-WP-0004-T06
|
||||
title: 'GradingResult dataclass: quality_score, notes, grader_id, baseline_response, candidate_response'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -126,7 +126,7 @@ state_hub_task_id: "c12a595b-90fc-4a80-8394-549edbda2031"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T07
|
||||
id: LLM-WP-0004-T07
|
||||
title: 'BaselineGrader protocol plus PairedGrader that runs baseline and candidate calls and delegates to a Judge'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -134,7 +134,7 @@ state_hub_task_id: "80b98e31-06fc-4462-b030-a12881095f93"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T08
|
||||
id: LLM-WP-0004-T08
|
||||
title: 'Judge protocol and built-ins: ExactMatchJudge, EmbeddingSimilarityJudge, LLMJudge'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -142,7 +142,7 @@ state_hub_task_id: "c2887fe3-bae6-4298-8c26-f9a519264dcf"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T09
|
||||
id: LLM-WP-0004-T09
|
||||
title: 'Functional contract doc covering judge bias caveats'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -150,7 +150,7 @@ state_hub_task_id: "7a4fd87a-b0ba-41b0-8e1a-a60fdaded905"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T10
|
||||
id: LLM-WP-0004-T10
|
||||
title: 'Tests: judges with canned inputs, stable grader result, deterministic LLMJudge rubric seed'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -158,7 +158,7 @@ state_hub_task_id: "8415a11d-d508-4d17-8082-10f93e9d16c5"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T11
|
||||
id: LLM-WP-0004-T11
|
||||
title: 'AdaptiveRoutingPolicy extends RoutingPolicy and selects the cheapest adapter whose observed mean quality clears the floor'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -166,7 +166,7 @@ state_hub_task_id: "0e9f9f8e-5066-4257-913b-a19f5b3fc47d"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T12
|
||||
id: LLM-WP-0004-T12
|
||||
title: 'Tie-breaking: prefer lower observed cost, then explicit preferred adapter from static rules'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -174,7 +174,7 @@ state_hub_task_id: "59d44712-1088-41ac-bad8-5d95db6f3a4f"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T13
|
||||
id: LLM-WP-0004-T13
|
||||
title: 'Cold-start behaviour falls through to static RoutingPolicy.resolve when observations are missing'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -182,7 +182,7 @@ state_hub_task_id: "1927d369-f5f6-48d3-8f53-7e4f1cae370e"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T14
|
||||
id: LLM-WP-0004-T14
|
||||
title: 'Functional contract doc for adaptive policy and sample-size/freshness trade-off'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -190,7 +190,7 @@ state_hub_task_id: "4d4717c1-8849-4fed-8f8d-515901ecafe0"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T15
|
||||
id: LLM-WP-0004-T15
|
||||
title: 'Tests: floor enforcement, tie-break, cold-start, window-size effect, fallback chain'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -198,7 +198,7 @@ state_hub_task_id: "304bd782-db15-4b7a-8d05-49e064a926c3"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T16
|
||||
id: LLM-WP-0004-T16
|
||||
title: 'ShadowingAdapter wraps a candidate adapter, also invokes the baseline adapter, grades, and appends to QualityLedger'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -206,7 +206,7 @@ state_hub_task_id: "62dd507f-536a-4623-8cbd-fa9f78e85ca6"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T17
|
||||
id: LLM-WP-0004-T17
|
||||
title: 'Sampling: caller-configurable shadow_rate so production load is not doubled'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -214,7 +214,7 @@ state_hub_task_id: "ccb73e92-1fca-42f9-8437-9b2b50e6424c"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T18
|
||||
id: LLM-WP-0004-T18
|
||||
title: 'Failure isolation: shadow errors never affect the candidate response returned to the caller'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -222,7 +222,7 @@ state_hub_task_id: "b879d232-d6ce-4ff6-b534-616729ea5ad7"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T19
|
||||
id: LLM-WP-0004-T19
|
||||
title: 'Functional contract doc for ShadowingAdapter'
|
||||
priority: low
|
||||
status: done
|
||||
|
|
@ -230,7 +230,7 @@ state_hub_task_id: "99d2c1bc-f1d8-42b3-9e04-6eea49460943"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T20
|
||||
id: LLM-WP-0004-T20
|
||||
title: 'Tests: candidate response survives baseline failure, ledger sampling rate, sync vs async modes'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -238,7 +238,7 @@ state_hub_task_id: "f533fbf4-484f-4408-8260-7e84e23bdc46"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T21
|
||||
id: LLM-WP-0004-T21
|
||||
title: 'Example script: route fixture batch through three candidate adapters and populate the ledger'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -246,7 +246,7 @@ state_hub_task_id: "7ef0c143-74b0-4740-81fa-819a826cf8f3"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T22
|
||||
id: LLM-WP-0004-T22
|
||||
title: 'Integration test: cold-start, static fallback, first observations, convergence to cheapest qualifying adapter'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -254,61 +254,61 @@ state_hub_task_id: "c4c6743f-157b-4445-8576-9caa6421d463"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T23
|
||||
id: LLM-WP-0004-T23
|
||||
title: 'Consumer integration guide showing how infospace-bench wires task types into adaptive policy'
|
||||
priority: medium
|
||||
status: done
|
||||
state_hub_task_id: "3a073ff7-0170-4a95-9c2a-a5daa84964e6"
|
||||
```
|
||||
|
||||
### T01 — Quality observation data model + ledger
|
||||
### LLM-WP-0004-T01 — Quality observation data model + ledger
|
||||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|------------------------------------
|
||||
---------------------------------------------------------------------------------------------------------------------------------------------------------|----------|--------|
|
||||
| T01 | `QualityObservation` dataclass: `task_type`, `adapter_id`, `model_id`, `cost_usd`, `quality_score` (0..1), `latency_ms`, `tokens_in`, `tokens_out`, `baseline_adapter_id`, `recorded_at`, `tags` | high | done |
|
||||
| T02 | `QualityLedger` append-only JSONL store with file-locked writes, configurable path, simple query helpers (`by_task_type`, `recent`, `mean_quality`) | high | done |
|
||||
| T03 | TTL helpers: `prune_before(timestamp)` and `is_stale(observation, max_age)` so callers can refresh observations without re-reading the whole ledger | medium | done |
|
||||
| T04 | Functional contract doc for the ledger schema and the field semantics of `quality_score` | medium | done |
|
||||
| T05 | Tests: round-trip, concurrent writes, query helpers, TTL, malformed-line resilience | high | done |
|
||||
| LLM-WP-0004-T01 | `QualityObservation` dataclass: `task_type`, `adapter_id`, `model_id`, `cost_usd`, `quality_score` (0..1), `latency_ms`, `tokens_in`, `tokens_out`, `baseline_adapter_id`, `recorded_at`, `tags` | high | done |
|
||||
| LLM-WP-0004-T02 | `QualityLedger` append-only JSONL store with file-locked writes, configurable path, simple query helpers (`by_task_type`, `recent`, `mean_quality`) | high | done |
|
||||
| LLM-WP-0004-T03 | TTL helpers: `prune_before(timestamp)` and `is_stale(observation, max_age)` so callers can refresh observations without re-reading the whole ledger | medium | done |
|
||||
| LLM-WP-0004-T04 | Functional contract doc for the ledger schema and the field semantics of `quality_score` | medium | done |
|
||||
| LLM-WP-0004-T05 | Tests: round-trip, concurrent writes, query helpers, TTL, malformed-line resilience | high | done |
|
||||
|
||||
### T02 — Baseline grader
|
||||
### LLM-WP-0004-T02 — Baseline grader
|
||||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|----------------------------------------------------------------------------------------------------------------------------------------------------|----------|--------|
|
||||
| T06 | `GradingResult` dataclass: `quality_score`, `notes`, `grader_id`, `baseline_response`, `candidate_response` | high | done |
|
||||
| T07 | `BaselineGrader` protocol: `.grade(baseline_adapter, candidate_adapter, prompt, run_config)` → `GradingResult`; built-in concrete `PairedGrader` runs both calls and delegates to a `Judge` | high | done |
|
||||
| T08 | `Judge` protocol + three built-ins: `ExactMatchJudge`, `EmbeddingSimilarityJudge` (uses an embedding adapter), `LLMJudge` (uses a third adapter with a fixed rubric prompt) | high | done |
|
||||
| T09 | Functional contract doc covering judge bias caveats (length bias, format bias, position bias for `LLMJudge`) | medium | done |
|
||||
| T10 | Tests: each judge against canned inputs, grader emits stable result with both responses preserved, deterministic seed for `LLMJudge` rubric | high | done |
|
||||
| LLM-WP-0004-T06 | `GradingResult` dataclass: `quality_score`, `notes`, `grader_id`, `baseline_response`, `candidate_response` | high | done |
|
||||
| LLM-WP-0004-T07 | `BaselineGrader` protocol: `.grade(baseline_adapter, candidate_adapter, prompt, run_config)` → `GradingResult`; built-in concrete `PairedGrader` runs both calls and delegates to a `Judge` | high | done |
|
||||
| LLM-WP-0004-T08 | `Judge` protocol + three built-ins: `ExactMatchJudge`, `EmbeddingSimilarityJudge` (uses an embedding adapter), `LLMJudge` (uses a third adapter with a fixed rubric prompt) | high | done |
|
||||
| LLM-WP-0004-T09 | Functional contract doc covering judge bias caveats (length bias, format bias, position bias for `LLMJudge`) | medium | done |
|
||||
| LLM-WP-0004-T10 | Tests: each judge against canned inputs, grader emits stable result with both responses preserved, deterministic seed for `LLMJudge` rubric | high | done |
|
||||
|
||||
### T03 — Adaptive routing policy
|
||||
### LLM-WP-0004-T03 — Adaptive routing policy
|
||||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|--------------------------------------------------------------------------------------------------------------------------------------------------|----------|--------|
|
||||
| T11 | `AdaptiveRoutingPolicy` extends `RoutingPolicy`: given `task_type` + `quality_floor` + `ledger`, returns the cheapest adapter whose observed mean quality clears the floor over a configurable window | high | done |
|
||||
| T12 | Tie-breaking: when two adapters meet the floor, prefer lower observed cost; if still tied, prefer the explicitly-preferred adapter from the underlying static rules | medium | done |
|
||||
| T13 | Cold-start behaviour: when no observations exist for a `(task_type, adapter)` pair, fall through to the static `RoutingPolicy.resolve` result so the system stays usable on day zero | high | done |
|
||||
| T14 | Functional contract doc; document the trade-off between sample size and freshness | medium | done |
|
||||
| T15 | Tests: floor enforcement, tie-break, cold-start, window-size effect, fallback chain | high | done |
|
||||
| LLM-WP-0004-T11 | `AdaptiveRoutingPolicy` extends `RoutingPolicy`: given `task_type` + `quality_floor` + `ledger`, returns the cheapest adapter whose observed mean quality clears the floor over a configurable window | high | done |
|
||||
| LLM-WP-0004-T12 | Tie-breaking: when two adapters meet the floor, prefer lower observed cost; if still tied, prefer the explicitly-preferred adapter from the underlying static rules | medium | done |
|
||||
| LLM-WP-0004-T13 | Cold-start behaviour: when no observations exist for a `(task_type, adapter)` pair, fall through to the static `RoutingPolicy.resolve` result so the system stays usable on day zero | high | done |
|
||||
| LLM-WP-0004-T14 | Functional contract doc; document the trade-off between sample size and freshness | medium | done |
|
||||
| LLM-WP-0004-T15 | Tests: floor enforcement, tie-break, cold-start, window-size effect, fallback chain | high | done |
|
||||
|
||||
### T04 — Shadow-mode observation wrapper
|
||||
### LLM-WP-0004-T04 — Shadow-mode observation wrapper
|
||||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|--------|
|
||||
| T16 | `ShadowingAdapter` wraps a candidate adapter; on each call, also invokes the baseline adapter (sync or via a thread pool), grades, and appends to a `QualityLedger` | medium | done |
|
||||
| T17 | Sampling: caller-configurable fraction (`shadow_rate=0.1` means grade one call in ten) so production load is not doubled | medium | done |
|
||||
| T18 | Failure isolation: shadow errors never affect the candidate response returned to the caller; failures are logged but not raised | high | done |
|
||||
| T19 | Functional contract doc | low | done |
|
||||
| T20 | Tests: candidate response always returned even when baseline raises, ledger gets exactly `shadow_rate × calls` entries (within tolerance), sync vs async modes | high | done |
|
||||
| LLM-WP-0004-T16 | `ShadowingAdapter` wraps a candidate adapter; on each call, also invokes the baseline adapter (sync or via a thread pool), grades, and appends to a `QualityLedger` | medium | done |
|
||||
| LLM-WP-0004-T17 | Sampling: caller-configurable fraction (`shadow_rate=0.1` means grade one call in ten) so production load is not doubled | medium | done |
|
||||
| LLM-WP-0004-T18 | Failure isolation: shadow errors never affect the candidate response returned to the caller; failures are logged but not raised | high | done |
|
||||
| LLM-WP-0004-T19 | Functional contract doc | low | done |
|
||||
| LLM-WP-0004-T20 | Tests: candidate response always returned even when baseline raises, ledger gets exactly `shadow_rate × calls` entries (within tolerance), sync vs async modes | high | done |
|
||||
|
||||
### T05 — End-to-end example + integration test
|
||||
### LLM-WP-0004-T05 — End-to-end example + integration test
|
||||
|
||||
| ID | Title | Priority | Status |
|
||||
|-----|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|--------|
|
||||
| T21 | Example script: route a small fixture batch through three candidate adapters (one OpenRouter cheap, one OpenRouter mid, `ClaudeCodeAdapter` as baseline), grade each, populate ledger | medium | done |
|
||||
| T22 | Integration test with mocked adapters covering: cold-start → static fallback → first observations → adaptive selection converges to the cheapest qualifying adapter | high | done |
|
||||
| T23 | Brief consumer-integration guide in `docs/` showing how `infospace-bench` (or any caller) wires task-type-per-stage into the adaptive policy | medium | done |
|
||||
| LLM-WP-0004-T21 | Example script: route a small fixture batch through three candidate adapters (one OpenRouter cheap, one OpenRouter mid, `ClaudeCodeAdapter` as baseline), grade each, populate ledger | medium | done |
|
||||
| LLM-WP-0004-T22 | Integration test with mocked adapters covering: cold-start → static fallback → first observations → adaptive selection converges to the cheapest qualifying adapter | high | done |
|
||||
| LLM-WP-0004-T23 | Brief consumer-integration guide in `docs/` showing how `infospace-bench` (or any caller) wires task-type-per-stage into the adaptive policy | medium | done |
|
||||
|
||||
## Risks and open questions
|
||||
|
||||
|
|
@ -364,5 +364,5 @@ state_hub_task_id: "3a073ff7-0170-4a95-9c2a-a5daa84964e6"
|
|||
- Wire the shadow-mode wrapper for the first multi-chapter run so the
|
||||
ledger fills up while real generation proceeds
|
||||
|
||||
That workplan should be drafted after T01–T03 of this workplan land, so
|
||||
That workplan should be drafted after LLM-WP-0004-T01–LLM-WP-0004-T03 of this workplan land, so
|
||||
that the consumer-side wiring is anchored in a stable llm-connect API.
|
||||
|
|
|
|||
|
|
@ -181,7 +181,7 @@ keeps working (LLM-WP-0004 owns the ledger).
|
|||
## Tasks
|
||||
|
||||
```task
|
||||
id: T01
|
||||
id: LLM-WP-0005-T01
|
||||
title: 'ModelRate + ModelRateRegistry data model, YAML loader, default-registry seed of nine OpenRouter models'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -189,7 +189,7 @@ state_hub_task_id: "535d3f12-911e-4b6a-87c3-b539c5986671"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T02
|
||||
id: LLM-WP-0005-T02
|
||||
title: 'CostModel.estimate_cost() pure function; tests for known model, unknown model, registry override, zero-token edge'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -197,7 +197,7 @@ state_hub_task_id: "691dd985-6a97-432d-8bf0-6cb99a9fbdcc"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T03
|
||||
id: LLM-WP-0005-T03
|
||||
title: 'ProblemClass protocol + TokenEstimate + ProblemClassRegistry'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -205,7 +205,7 @@ state_hub_task_id: "ecf263d2-f40a-460e-9195-4e01135ef727"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T04
|
||||
id: LLM-WP-0005-T04
|
||||
title: 'Built-in classes: chunk-summarization, entity-extraction, relation-extraction, judge-eval, report-synthesis'
|
||||
priority: high
|
||||
status: done
|
||||
|
|
@ -213,7 +213,7 @@ state_hub_task_id: "f1860b10-7467-4ce3-9775-ab293cef3ed0"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T05
|
||||
id: LLM-WP-0005-T05
|
||||
title: 'ProblemClass.fit() adapts tunable params from QualityLedger observations'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -221,7 +221,7 @@ state_hub_task_id: "950b74e9-ede8-477a-b6b7-c7af423d4ebb"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T06
|
||||
id: LLM-WP-0005-T06
|
||||
title: 'CLI helpers: llm-connect rates show, llm-connect classes show, llm-connect classes fit <ledger>'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -229,7 +229,7 @@ state_hub_task_id: "c47eca5f-4cb3-4f88-ac1b-38a9ae18e7e6"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T07
|
||||
id: LLM-WP-0005-T07
|
||||
title: 'Functional contract docs under contracts/functional/ for rates, costs, and problem classes'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -237,7 +237,7 @@ state_hub_task_id: "c15fd1dc-48c3-40e9-abca-ba3ffe3684f9"
|
|||
```
|
||||
|
||||
```task
|
||||
id: T08
|
||||
id: LLM-WP-0005-T08
|
||||
title: 'Consumer migration note for infospace-bench: replace plan_generation_summary cost+token math with llm-connect calls'
|
||||
priority: medium
|
||||
status: done
|
||||
|
|
@ -316,7 +316,7 @@ Out of scope:
|
|||
|
||||
## Consumer-side follow-up
|
||||
|
||||
Once T01-T04 land in llm-connect, `infospace-bench` opens a thin
|
||||
Once `LLM-WP-0005-T01` through `LLM-WP-0005-T04` land in llm-connect, `infospace-bench` opens a thin
|
||||
companion workplan to:
|
||||
|
||||
- Replace `_CALLS_PER_CHUNK_BY_WORKFLOW` + `_profile_template_words` +
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue