tegwick
772c1b3e8e
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0cffe-206f-7323-89d3-b20ca4cf4bf9
82 lines
4.7 KiB
Markdown
82 lines
4.7 KiB
Markdown
# Truncation diagnosis
|
|
|
|
Date: 2026-09-25. Reviewer: codex, self-review. Owner: FEP-WP-0002-T02.
|
|
|
|
## Findings
|
|
|
|
Inspected llm-connect checkout `37436bb`. Its `_payload.py` explicitly drops
|
|
`reasoning_effort` before OpenRouter execution. `openrouter.py` retains only three
|
|
aggregate usage counters in its normal LLMResponse, omitting detailed token counts.
|
|
Per-call `/execute?debug=1` already exposes provider-bound settings and raw usage;
|
|
no deployment, credential retrieval or upstream repository edit was required.
|
|
|
|
Fresh local-first `-04`: requested low reasoning effort, 1,000 completion tokens;
|
|
first response ended `length`. Live diagnostics confirmed neither reasoning field
|
|
was sent; the provider reported zero reasoning tokens. Output was 30,595 characters.
|
|
Reported usage: 525 prompt + 1,000 completion tokens. The numeric provider cost field
|
|
was retained; it is not an independent billing reconciliation.
|
|
|
|
Fresh `-05`: omitted the ineffective requested reasoning override and kept the same
|
|
1,000-token limit. The rejected generated text was quarantined under
|
|
`reports/rejected-response.md`, separate from workflow outputs. It contains three
|
|
lines: a claim-table heading and partial header, with 30,544 spaces out of 30,595
|
|
characters. This is direct evidence of pathological table padding in this response,
|
|
not excessive substantive analysis. No raw provider debug body or header is stored.
|
|
|
|
The two diagnostic attempts each stopped after one service request. This diagnosis
|
|
does not retroactively establish the content of prior runs whose text was not saved,
|
|
or prove a general model defect. It identifies an actionable failure mode here.
|
|
|
|
## Intervention
|
|
|
|
Replaced the claims table with labeled bullet records. All four prompts explicitly
|
|
forbid padded tables. Preserved source bytes and historical workspaces. Fresh `-06`
|
|
requested 3,000 completion tokens per response, at most four service requests, no
|
|
client retries, same llm-connect profile; diagnostics enabled. Internal retries are
|
|
outside the client ceiling. No claim that low reasoning effort was applied.
|
|
|
|
The client now retains only allowlisted diagnostics, plus normal generated incomplete
|
|
text as a rejected artifact. Tests cover diagnostic data exclusion and quarantine;
|
|
a compatibility check documents the upstream adapter's current ignored parameter.
|
|
|
|
|
|
## Outcome and editorial review
|
|
|
|
Local-first `-06` completed all four stages with finish reason `stop`; source hash
|
|
is unchanged. [Receipt](../bench/infospaces/local-first-authority-06/reports/research-run.json).
|
|
This successful run supports the compact-format workaround for this packet. It does
|
|
not prove future runs cannot truncate; the client still rejects incomplete output.
|
|
|
|
[Structural screening](LocalFirstAdmissionScreen.json) passes, with admission still
|
|
`not_granted`. The model returned a valid terminal JSON candidate ledger without
|
|
Markdown fences. Screening now accepts one unambiguous terminal JSON document as
|
|
well as a fenced block; it does not repair syntax or infer a ledger from prose.
|
|
Regression tests cover this exact live result and reject ambiguous/malformed JSON.
|
|
|
|
Editorial disposition: retain the bounded negative result; admit no candidates.
|
|
|
|
- C01 faithfully paraphrases the local-primary-copy fragment.
|
|
- C02 substitutes "authoritative version" for "primary copy". Treat that as an
|
|
interpretation requiring additional evidence, not a second source-established fact.
|
|
The assisted critique correctly flagged this leap; the editorial review agrees.
|
|
- The candidate stage explicitly finds insufficient evidence for a complete pattern;
|
|
its candidate ledger and relation array are empty. This is a legitimate research
|
|
outcome, not a pipeline failure or rejection of the existing manual hypothesis.
|
|
- Persistence, concurrency, recovery and independent known uses remain unestablished
|
|
by the packet. VC-02 relevance is a hypothesis, despite the critique's "Pass" label;
|
|
no scenario walkthrough has been performed by this run.
|
|
- Manual review claims C02/C03 from broader source sections are distinct from these
|
|
workspace-local labels. Do not merge them by ID alone.
|
|
|
|
The immediate execution obstacle is resolved for this run. T02's next research step
|
|
is to expand the source packet to located persistence/synchronization/conflict
|
|
passages and independent implementation evidence. Then T03 can compare restart and
|
|
reconnection behavior in VC-01/VC-02. No more live calls are needed for this diagnosis.
|
|
|
|
Completed run reported total tokens: 5660.
|
|
|
|
|
|
Whitespace validation exception: the quarantined `-05/reports/rejected-response.md`
|
|
intentionally retains the provider's trailing spaces as diagnostic evidence. All
|
|
other staged files pass `git diff --check`; trimming this file would destroy the
|
|
captured failure and invalidate its receipt hash.
|