tegwick
2ec23bb22c
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0cffe-206f-7323-89d3-b20ca4cf4bf9
60 lines
3.5 KiB
Markdown
60 lines
3.5 KiB
Markdown
# llm-connect authority pilot
|
|
|
|
Date: 2026-09-24. Reviewer: codex; initial editorial review, not independent validation.
|
|
|
|
## Execution
|
|
|
|
Per user direction, live execution now calls the deployed llm-connect `/execute`
|
|
service. No provider key was retrieved. A temporary Kubernetes port-forward exposed
|
|
`activity-core/llm-connect` locally; no deployment was changed. Requested profile:
|
|
`custodian-triage-balanced`; resolved provider/model: OpenRouter / Gemini 2.5 Flash.
|
|
Credentials and routing remain service-owned. Client retries are disabled; internal
|
|
service retry count and dollar cost are unknown.
|
|
|
|
Receipts under `research/bench/infospaces/` record source/prompt/config digests,
|
|
requested settings, actual model, usage and finish reason:
|
|
|
|
- `carniato-authority-02`: one response, `length`, 4,470 total tokens. Rejected.
|
|
- `carniato-authority-03`: four responses, all `stop`, 12,638 total tokens. Workflow completed.
|
|
- `local-first-authority-02`: one response, `length`, 12,489 total tokens. Rejected.
|
|
|
|
Total reported usage across these six service requests: 29,597 tokens, including
|
|
failed attempts. These totals do not measure dollar cost. The `-01` failed direct
|
|
provider runs remain intact. No successful local-first generated artifact exists.
|
|
Generated output remains unreviewed in the bench; this editorial disposition does
|
|
not rewrite the original run or convert assisted critique into independent review.
|
|
|
|
## Review of the completed Carniato draft
|
|
|
|
Artifacts: [candidate](../bench/infospaces/carniato-authority-03/artifacts/generated/candidates.md),
|
|
[relations](../bench/infospaces/carniato-authority-03/artifacts/generated/relation-proposals.md),
|
|
[assisted critique](../bench/infospaces/carniato-authority-03/output/evaluations/source-review.md).
|
|
|
|
Disposition: **revise candidate; reject proposed graph relations; admit nothing**.
|
|
|
|
- Workspace P01 maps provisionally to the existing Immediate Feedback candidate.
|
|
Its family and scale labels do not satisfy the repository vocabulary contract.
|
|
- The candidate's chat, social, checkbox and form examples have no independently
|
|
located implementations. They are illustrative hypotheses, not known-use evidence.
|
|
Its recurrence assertion is unsupported by this bounded source packet.
|
|
- The proposed `requires` edges target C01/C02 claim IDs as if they were candidate
|
|
IDs. These are evidence references, not valid pattern relation targets.
|
|
- Server authority is contextual in this source. A mandatory server-authority
|
|
prerequisite for immediate feedback is not established, and conflicts with the
|
|
local-first comparison hypothesis in the earlier manual review.
|
|
- The assisted critique approved recurrence and both relation justifications.
|
|
This is a concrete false-positive example for our admission process: automated
|
|
self-review cannot supply missing evidence or validate graph identity.
|
|
|
|
The two narrowly sourced claims are useful starting points, but the larger dynamics,
|
|
failure modes and examples extend beyond the excerpt and need separate evidence.
|
|
No FEP identities, canonical pattern files or graph edges were added.
|
|
|
|
## Follow-up owned by FEP-WP-0002-T02
|
|
|
|
Before another local-first run, select explicit research-appropriate execution
|
|
settings and diagnose repeated truncation within a bounded call/token plan; simply
|
|
raising the cap again is not justified by these results. Refine prompts to prohibit
|
|
invented known uses, distinguish claims from pattern identities, and require
|
|
unsupported recurrence to remain unknown. Verify those rules on this captured
|
|
failure before the next live run. T03 still owns cross-source case walkthroughs.
|