Record first authority investigation and preserve failed live-run evidence
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0cffe-206f-7323-89d3-b20ca4cf4bf9
This commit is contained in:
parent
0839ec0600
commit
1f95f6389a
13 changed files with 268 additions and 79 deletions
6
TREE.md
6
TREE.md
|
|
@ -1,6 +1,6 @@
|
|||
# Repository file inventory
|
||||
|
||||
Project files, including synthetic fixtures and prepared primary-source inputs; updated 2026-09-24.
|
||||
Project files, including pilot receipts and manual reviews; updated 2026-09-24.
|
||||
|
||||
```text
|
||||
.custodian-brief.md
|
||||
|
|
@ -57,6 +57,7 @@ research/bench/infospaces/carniato-authority-01/profile/templates/critique.md
|
|||
research/bench/infospaces/carniato-authority-01/profile/templates/relations.md
|
||||
research/bench/infospaces/carniato-authority-01/profile/workflow.yaml
|
||||
research/bench/infospaces/carniato-authority-01/reports/preflight.json
|
||||
research/bench/infospaces/carniato-authority-01/reports/research-run.json
|
||||
research/bench/infospaces/fixture-smoke/artifacts/generated/candidates.md
|
||||
research/bench/infospaces/fixture-smoke/artifacts/generated/claims.md
|
||||
research/bench/infospaces/fixture-smoke/artifacts/generated/relation-proposals.md
|
||||
|
|
@ -80,12 +81,14 @@ research/bench/infospaces/local-first-authority-01/profile/templates/critique.md
|
|||
research/bench/infospaces/local-first-authority-01/profile/templates/relations.md
|
||||
research/bench/infospaces/local-first-authority-01/profile/workflow.yaml
|
||||
research/bench/infospaces/local-first-authority-01/reports/preflight.json
|
||||
research/bench/infospaces/local-first-authority-01/reports/research-run.json
|
||||
research/bench/profile/templates/candidates.md
|
||||
research/bench/profile/templates/claims.md
|
||||
research/bench/profile/templates/critique.md
|
||||
research/bench/profile/templates/relations.md
|
||||
research/bench/profile/workflow.yaml
|
||||
research/cases/CarniatoGrandUnifyingArchitecture.md
|
||||
research/cases/LocalFirstAuthority.md
|
||||
research/catalog/PatternCandidateCatalog.md
|
||||
research/catalog/SourceCatalog.md
|
||||
research/compendium/FrontendArchitectureCompendium.md
|
||||
|
|
@ -97,6 +100,7 @@ research/plan/OpenQuestions.md
|
|||
research/plan/ResearchProgram.md
|
||||
research/plan/ResearchProtocol.md
|
||||
research/plan/ValidationCases.md
|
||||
research/reviews/AuthorityPilot.md
|
||||
research/reviews/SchemaAndSourcePreparation.md
|
||||
scripts/research_bench.py
|
||||
sequences/CollaborativeApplication.md
|
||||
|
|
|
|||
|
|
@ -99,14 +99,14 @@ provider call was made during setup.
|
|||
|
||||
| Investigation | Input readiness | Work record |
|
||||
| --- | --- | --- |
|
||||
| Carniato first source case | SRC-0001 verified; bounded authority/feedback workspace prepared | FEP-WP-0002-T02 |
|
||||
| Contrasting local-first source | SRC-0002 verified; bounded authority workspace prepared | FEP-WP-0002-T02 |
|
||||
| Carniato first source case | Initial manual review recorded; provider attempt failed | FEP-WP-0002-T02 |
|
||||
| Contrasting local-first source | Initial manual review recorded; provider attempt failed | FEP-WP-0002-T02 |
|
||||
| VC-01 and VC-02 synthesis | Depends on reviewed source claims and candidate patterns | FEP-WP-0002-T03 |
|
||||
|
||||
Do not ingest the existing Carniato research stub as if it were the original article.
|
||||
|
||||
|
||||
Prepared primary-source plans (four calls each; not yet executed):
|
||||
Primary-source plans (four calls each; each run failed on its first call):
|
||||
|
||||
```bash
|
||||
../infospace-bench/.venv/bin/python scripts/research_bench.py plan research/bench/infospaces/carniato-authority-01
|
||||
|
|
@ -115,3 +115,16 @@ Prepared primary-source plans (four calls each; not yet executed):
|
|||
|
||||
These short packets isolate the first comparison question. They do not substitute
|
||||
for full source review, independent known uses or the two architecture walkthroughs.
|
||||
|
||||
|
||||
## Live pilot outcome — 2026-09-24
|
||||
|
||||
Both primary-source runs attempted Gemini 2.5 Flash through OpenRouter and failed
|
||||
before producing research artifacts. A separate credential probe returned HTTP 401.
|
||||
Their failed receipts remain immutable. Read [AuthorityPilot](../reviews/AuthorityPilot.md)
|
||||
for the manual review that continued independently. New runs require a working
|
||||
credential authorized for this client and fresh workspace slugs.
|
||||
|
||||
Failure receipts now capture adapter/model and safe structured diagnostic fields;
|
||||
raw provider error bodies, headers and credential values are never persisted.
|
||||
The initial failed receipts predate this improvement; the review records their model.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
{
|
||||
"root": "/home/worsch/frontend-patterns/research/bench/infospaces/carniato-authority-01",
|
||||
"workflow": "frontend-source-investigation",
|
||||
"source_sha256": "a4ef36bb7d4530106cde8b85281e88c7632e9a28d9f0596cda2f405442d80888",
|
||||
"evidence_kind": "primary",
|
||||
"stages": [
|
||||
"claims",
|
||||
"candidates",
|
||||
"relations",
|
||||
"critique"
|
||||
],
|
||||
"provider_calls": 4,
|
||||
"cost_usd": "unknown; call limit is not a dollar budget",
|
||||
"review_status": "unreviewed",
|
||||
"input_digests": {
|
||||
"infospace.yaml": "0f52be84e24f426e5851f905c462c2ca959f08d528b187b58e84a6170e683952",
|
||||
"profile/templates/candidates.md": "a1b101ec9ad8727be0cfb4b495eaa07b8715b21e5fb089ff3bbf3378d97ab592",
|
||||
"profile/templates/claims.md": "1dbaea092a7d2e5acbf5e0ef0a2a7caa3d1b3ee7a84196d47e34a10b2514fae7",
|
||||
"profile/templates/critique.md": "c8da28d03ae204324136c2f26f0602fdb43fdc2a360f0b305493b96502d934cf",
|
||||
"profile/templates/relations.md": "159310e2b63d84fe67e0866f79c12099232a1b45c7c5e09793a3767a0a03ebe4",
|
||||
"profile/workflow.yaml": "e0f3075dc9d896369dd45dc4da3577eb314cb4ef0e4414f905af68ec77cf5b72"
|
||||
},
|
||||
"status": "failed",
|
||||
"error_type": "InfospaceError",
|
||||
"provider_calls_attempted": 1
|
||||
}
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
{
|
||||
"root": "/home/worsch/frontend-patterns/research/bench/infospaces/local-first-authority-01",
|
||||
"workflow": "frontend-source-investigation",
|
||||
"source_sha256": "c32ea4843ba1be9a2da7ef90e4b6dffab26f106a33dd881aee6048b6c3d027f7",
|
||||
"evidence_kind": "primary",
|
||||
"stages": [
|
||||
"claims",
|
||||
"candidates",
|
||||
"relations",
|
||||
"critique"
|
||||
],
|
||||
"provider_calls": 4,
|
||||
"cost_usd": "unknown; call limit is not a dollar budget",
|
||||
"review_status": "unreviewed",
|
||||
"input_digests": {
|
||||
"infospace.yaml": "4e5a557c5f9e8a3e7c38bb578aa58d6336791bd241ecb84fc58164f3795d8889",
|
||||
"profile/templates/candidates.md": "a1b101ec9ad8727be0cfb4b495eaa07b8715b21e5fb089ff3bbf3378d97ab592",
|
||||
"profile/templates/claims.md": "1dbaea092a7d2e5acbf5e0ef0a2a7caa3d1b3ee7a84196d47e34a10b2514fae7",
|
||||
"profile/templates/critique.md": "c8da28d03ae204324136c2f26f0602fdb43fdc2a360f0b305493b96502d934cf",
|
||||
"profile/templates/relations.md": "159310e2b63d84fe67e0866f79c12099232a1b45c7c5e09793a3767a0a03ebe4",
|
||||
"profile/workflow.yaml": "e0f3075dc9d896369dd45dc4da3577eb314cb4ef0e4414f905af68ec77cf5b72"
|
||||
},
|
||||
"status": "failed",
|
||||
"error_type": "InfospaceError",
|
||||
"provider_calls_attempted": 1
|
||||
}
|
||||
|
|
@ -1,79 +1,25 @@
|
|||
# Case Study — The Grand Unifying Architecture of Frontend
|
||||
# Carniato source review — authority and feedback
|
||||
|
||||
Status: initial research stub
|
||||
Status: extracted; initial manual review, 2026-09-24, codex (self-review).
|
||||
Source: SRC-0001, Ryan Carniato, [original article](https://dev.to/playfulprogramming/the-grand-unifying-architecture-of-frontend-bhk).
|
||||
Edition and date uncertainty: see the source catalog. No API-generated findings.
|
||||
|
||||
Source:
|
||||
Ryan Carniato, "The Grand Unifying Architecture of Frontend".
|
||||
| Claim | Original locator | Source statement, paraphrased | Our interpretation |
|
||||
| --- | --- | --- | --- |
|
||||
| C01 | Core Responsibilities, Content, paragraph 1 | Content authority resides on the server. | Authority placement is an assumption of this model. |
|
||||
| C02 | Core Responsibilities, Affordances, paragraph 1 | Interaction feedback should precede a server round trip. | Local feedback need not imply local authority. |
|
||||
| C03 | Content: Reactive reads over the wire, final paragraph | The live graph must be reconstructible from durable state. | Projection state and durable authority should be distinguished. |
|
||||
|
||||
## Why this source matters
|
||||
C03 was checked manually in the original; it is outside the bench excerpt.
|
||||
|
||||
The article attempts to unify several modern frontend architectures beneath framework-specific terminology.
|
||||
## Limits and counterexample
|
||||
|
||||
Its most useful conceptual move is the decomposition:
|
||||
The article's universal framing is broader than this review supports. Local-first
|
||||
primary data challenges mandatory server authority; it does not refute separating
|
||||
responsibilities. Framework implementation claims and examples were not executed.
|
||||
|
||||
```text
|
||||
Navigation → Content → Affordance
|
||||
```
|
||||
## Disposition
|
||||
|
||||
with a characteristic movement:
|
||||
|
||||
```text
|
||||
client → server → client
|
||||
```
|
||||
|
||||
for server-authoritative applications.
|
||||
|
||||
## Initial interpretation
|
||||
|
||||
### Navigation
|
||||
|
||||
Contains aspects of:
|
||||
|
||||
- addressability,
|
||||
- routing,
|
||||
- user intent,
|
||||
- mutation initiation.
|
||||
|
||||
### Content
|
||||
|
||||
Contains aspects of:
|
||||
|
||||
- authority,
|
||||
- server-side state,
|
||||
- projection.
|
||||
|
||||
### Affordance
|
||||
|
||||
Contains aspects of:
|
||||
|
||||
- local interaction state,
|
||||
- speculative state,
|
||||
- immediate feedback.
|
||||
|
||||
## Main qualification
|
||||
|
||||
The model strongly fits server-authoritative web applications.
|
||||
|
||||
It should not be promoted unchanged into the general canon because:
|
||||
|
||||
- local-first systems may treat local data as primary,
|
||||
- replicated systems may have multiple authorities,
|
||||
- peer-to-peer systems may lack a distinguished server,
|
||||
- microfrontend concerns cut across this axis,
|
||||
- internal modularity is largely orthogonal to it.
|
||||
|
||||
## Research tasks
|
||||
|
||||
- extract concrete architecture examples from the article,
|
||||
- identify all implicit assumptions,
|
||||
- map each example to responsibility dimensions,
|
||||
- identify candidate patterns,
|
||||
- compare against:
|
||||
- local-first,
|
||||
- HTMX/hypermedia,
|
||||
- SPA,
|
||||
- server components,
|
||||
- islands,
|
||||
- LiveView-style persistent server interaction,
|
||||
- identify where the model is descriptive versus normative,
|
||||
- determine which concepts should enter the canon.
|
||||
Retain Server Authority and Immediate Feedback as candidates. Do not infer a
|
||||
requires edge between them. Recurrence and failure behavior need further evidence.
|
||||
Broader architecture comparisons remain open under FEP-WP-0002-T02/T03.
|
||||
|
|
|
|||
25
research/cases/LocalFirstAuthority.md
Normal file
25
research/cases/LocalFirstAuthority.md
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
# Local-first source review — authority placement
|
||||
|
||||
Status: extracted; initial manual review, 2026-09-24, codex (self-review).
|
||||
Source: SRC-0002, Kleppmann, Wiggins, van Hardenberg and McGranaghan,
|
||||
[April 2019 web essay](https://www.inkandswitch.com/essay/local-first/).
|
||||
Conference DOI: 10.1145/3359591.3359737. No API-generated findings.
|
||||
|
||||
| Claim | Original locator | Source statement, paraphrased | Our interpretation |
|
||||
| --- | --- | --- | --- |
|
||||
| C01 | Seven ideals, paragraph 3 | The device's data copy becomes primary rather than subordinate to a server. | Local Authority differs from caching. |
|
||||
| C02 | Ideal 3, paragraph beginning Since local-first applications | Local reading/writing can continue offline; synchronization follows connectivity. | Commit location and synchronization timing are separate decisions. |
|
||||
| C03 | Introduction, paragraph beginning The software we are talking about | The discussion targets documents and personal repositories, excluding several centralized service domains. | Do not generalize the proposal to every transaction. |
|
||||
|
||||
C02/C03 were checked manually beyond the bench excerpt.
|
||||
|
||||
## Limits and counterexample
|
||||
|
||||
A hypothetical shared inventory reservation requires a separate concurrency argument;
|
||||
local durability alone does not prove that a reservation is globally valid.
|
||||
The excerpt does not establish a merge algorithm or access-revocation semantics.
|
||||
|
||||
## Disposition
|
||||
|
||||
Retain Local Authority as a hypothesis candidate. Review persistence, synchronization,
|
||||
conflict handling and independent implementations before admission or maturity changes.
|
||||
|
|
@ -22,3 +22,19 @@ This file tracks concepts before promotion into canonical pattern files.
|
|||
| Failure Boundary | quality | L5/L6 | seed | fault containment |
|
||||
|
||||
`seed` means "worth researching", not "accepted into the canon".
|
||||
|
||||
|
||||
## First investigation dispositions — 2026-09-24
|
||||
|
||||
See [AuthorityPilot](../reviews/AuthorityPilot.md). These are manual source-review
|
||||
outcomes, not successful bench extraction or canonical admission.
|
||||
|
||||
| Candidate | Evidence reference | Disposition / question |
|
||||
| --- | --- | --- |
|
||||
| Server Authority | SRC-0001, Carniato review C01 | Retain; identify precisely which data and decisions it governs |
|
||||
| Immediate Feedback | SRC-0001, Carniato review C02 | Add seed; feedback should distinguish tentative from durable/accepted state |
|
||||
| Local Authority | SRC-0002, local-first review C01 | Retain; test primary ownership versus cache behavior |
|
||||
| Scoped Authority | Cross-source interpretation in AuthorityPilot | Defer new pattern; may belong in context requirements instead |
|
||||
|
||||
No recurrence, established maturity, known uses or required dependency edges are
|
||||
claimed by these dispositions. Other seed candidates remain unassessed.
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@
|
|||
|
||||
| ID | Source and edition | State | Prepared input |
|
||||
| --- | --- | --- | --- |
|
||||
| SRC-0001 | Ryan Carniato, [The Grand Unifying Architecture of Frontend](https://dev.to/playfulprogramming/the-grand-unifying-architecture-of-frontend-bhk); web edition accessed 2026-09-24; displayed Sep 22, year unverified | queued; identity checked, bounded passages prepared | [Authority and feedback packet](../bench/infospaces/carniato-authority-01/artifacts/sources/source.md) |
|
||||
| SRC-0002 | Martin Kleppmann, Adam Wiggins, Peter van Hardenberg, Mark McGranaghan, [Local-first software](https://www.inkandswitch.com/essay/local-first/); April 2019 web essay; DOI 10.1145/3359591.3359737; accessed 2026-09-24 | queued; identity checked, bounded passage prepared | [Local authority packet](../bench/infospaces/local-first-authority-01/artifacts/sources/source.md) |
|
||||
| SRC-0001 | Ryan Carniato, [The Grand Unifying Architecture of Frontend](https://dev.to/playfulprogramming/the-grand-unifying-architecture-of-frontend-bhk); web edition accessed 2026-09-24; displayed Sep 22, year unverified | extracted; [initial manual review](../cases/CarniatoGrandUnifyingArchitecture.md); live run failed | [Authority and feedback packet](../bench/infospaces/carniato-authority-01/artifacts/sources/source.md) |
|
||||
| SRC-0002 | Martin Kleppmann, Adam Wiggins, Peter van Hardenberg, Mark McGranaghan, [Local-first software](https://www.inkandswitch.com/essay/local-first/); April 2019 web essay; DOI 10.1145/3359591.3359737; accessed 2026-09-24 | extracted; [initial manual review](../cases/LocalFirstAuthority.md); live run failed | [Local authority packet](../bench/infospaces/local-first-authority-01/artifacts/sources/source.md) |
|
||||
|
||||
Queued does not mean reviewed. Bibliography and hashes live in each workspace's
|
||||
source artifact and manifest. [Preparation review](../reviews/SchemaAndSourcePreparation.md)
|
||||
|
|
@ -41,3 +41,9 @@ Each source should eventually capture:
|
|||
- contradictions,
|
||||
- known implementations,
|
||||
- evidence quality.
|
||||
|
||||
|
||||
2026-09-24: initial authority-focused reviews are recorded as extracted. They have
|
||||
not passed a full-source or independent review gate. Live calls failed before any
|
||||
artifact generation. See [AuthorityPilot](../reviews/AuthorityPilot.md) for findings,
|
||||
execution evidence and limitations.
|
||||
|
|
|
|||
|
|
@ -42,3 +42,12 @@ retaining a hypothesis requires evidence just as rejecting one does.
|
|||
| Q05 Sequence ordering | The general sequence works with explicit branches | Test offline, collaboration and quality requirements that force earlier decisions | FEP-WP-0003 |
|
||||
| Q06 Scale and boundaries | Composition and deployment are distinct dimensions | Analyze independently owned features that deploy together, and the reverse | FEP-WP-0003 |
|
||||
| Q07 Retrieval | Metadata improves relevant, justified recommendations | Held-out prompts including ambiguity, incompatibility and abstention | FEP-WP-0004 |
|
||||
|
||||
|
||||
## First-cycle update — 2026-09-24
|
||||
|
||||
[AuthorityPilot](../reviews/AuthorityPilot.md) records an initial manual comparison.
|
||||
Q02 remains open: distinguish authority scope and acknowledgement semantics before
|
||||
claiming independent responsibility dimensions. Q03 remains open because source
|
||||
statements do not alone establish recurrence. Q04 awaits VC-01/VC-02 walkthroughs.
|
||||
The failed provider runs are execution failures, not negative evidence for the model.
|
||||
|
|
|
|||
66
research/reviews/AuthorityPilot.md
Normal file
66
research/reviews/AuthorityPilot.md
Normal file
|
|
@ -0,0 +1,66 @@
|
|||
# Authority pilot — first investigation
|
||||
|
||||
Date: 2026-09-24. Reviewer: codex, manual self-review.
|
||||
Scope: prepared passages plus the explicitly located original sections in the two
|
||||
source reviews. This is an initial investigation, not the M1 acceptance review.
|
||||
|
||||
## Execution record
|
||||
|
||||
Both bench workspaces attempted their claims stage with `google/gemini-2.5-flash`.
|
||||
Each stopped after one adapter call; no generated claims, candidates, relations or
|
||||
critiques were produced. The separate OpenRouter credential probe returned HTTP 401.
|
||||
The original failure receipts record the exception type only, so that probe is
|
||||
separate diagnostic evidence rather than a retroactively fabricated provider receipt.
|
||||
|
||||
The existing credential route inspected was for activity-core's llm-connect workload,
|
||||
not generic workstation clients. No production key was retrieved or rotated. No
|
||||
fallback model or hidden retries were used. Completed usage/cost metadata is absent;
|
||||
we do not claim a measured zero charge. Eight planned calls did not take place.
|
||||
|
||||
The adapter now records model, structured error code and HTTP status where available
|
||||
without persisting raw provider error details. Tests exercise the 401 path using a
|
||||
synthetic secret sentinel. Existing failure receipts are preserved unchanged.
|
||||
|
||||
## Reviewed research result
|
||||
|
||||
Read [Carniato review](../cases/CarniatoGrandUnifyingArchitecture.md) and
|
||||
[local-first review](../cases/LocalFirstAuthority.md) for the source claims.
|
||||
The following are our proposed modeling decisions, not conclusions attributed to
|
||||
both authors or proof that any pattern is established:
|
||||
|
||||
1. **Scope authority to particular data and decisions.** One application's draft,
|
||||
published record and shared reservation can have different authority rules.
|
||||
2. **Separate acknowledgement levels.** Interaction feedback, durable local save,
|
||||
and globally accepted mutation should not be interchangeable states.
|
||||
3. **Qualify relations.** Server Authority and Local Authority are alternatives only
|
||||
for the same authority scope. They may coexist in one application. Immediate
|
||||
Feedback does not require Server Authority.
|
||||
4. **Make recovery explicit.** A projection or optimistic view must explain which
|
||||
durable state reconstructs it and what happens after interruption or rejection.
|
||||
|
||||
These proposals need source expansion and scenario testing. Q02 has a useful
|
||||
counterexample to server placement as a universal rule; independent variation of
|
||||
all responsibilities remains unproven. Q03 recurrence and Q04 generativity are open.
|
||||
|
||||
## Candidate dispositions
|
||||
|
||||
| Workspace/candidate origin | Repository candidate | Outcome |
|
||||
| --- | --- | --- |
|
||||
| SRC-0001 C01, manually extracted | Server Authority | Retain candidate; qualify data scope |
|
||||
| SRC-0001 C02, manually extracted | Immediate Feedback | Retain candidate; do not attach mandatory server dependency |
|
||||
| SRC-0002 C01, manually extracted | Local Authority | Retain candidate; distinguish primary data from a disposable cache |
|
||||
| Cross-source review hypothesis | Scoped Authority | Investigate whether this is a context rule rather than another pattern |
|
||||
|
||||
There are no generated P01 identities to map: the provider runs failed. No canonical
|
||||
FEP IDs, pattern files, known uses or graph edges have been admitted in this pilot.
|
||||
|
||||
## Next bounded work
|
||||
|
||||
- FEP-WP-0002-T02: extend review to durability, rejected writes, synchronization and
|
||||
conflicting edits; verify independent implementations. Source catalog remains
|
||||
extracted, not fully reviewed.
|
||||
- FEP-WP-0002-T03: test acknowledgement distinctions on VC-01 and VC-02, using the
|
||||
same changed constraint: lose connectivity, restart, then reconnect.
|
||||
- Live execution: a working credential authorized for this workstation client is
|
||||
needed. Once available, create fresh revision workspaces; do not overwrite failed
|
||||
runs or label manually written content as model-provider output.
|
||||
|
|
@ -4,11 +4,13 @@ from __future__ import annotations
|
|||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
|
||||
import yaml
|
||||
from infospace_bench import create_infospace, load_infospace, register_artifact
|
||||
from infospace_bench.errors import InfospaceError
|
||||
from infospace_bench.workflow import FixtureAssistedGenerationAdapter, run_workflow
|
||||
from infospace_bench.openrouter import OpenRouterAssistedGenerationAdapter
|
||||
|
||||
|
|
@ -107,6 +109,8 @@ def execute(root, adapter, max_calls=4):
|
|||
root / "profile/workflow.yaml"]
|
||||
}
|
||||
manifest["status"] = "started"
|
||||
manifest["adapter"] = type(adapter).__name__
|
||||
manifest["model"] = getattr(adapter, "model", "")
|
||||
receipt.write_text(json.dumps(manifest, indent=2) + "\n")
|
||||
limited = LimitedAdapter(adapter, max_calls)
|
||||
try:
|
||||
|
|
@ -114,6 +118,12 @@ def execute(root, adapter, max_calls=4):
|
|||
manifest.update(status=result.status, run_record=result.run_record_path)
|
||||
except Exception as exc:
|
||||
manifest.update(status="failed", error_type=type(exc).__name__)
|
||||
if isinstance(exc, InfospaceError):
|
||||
manifest["error_code"] = exc.code
|
||||
# Persist only the status code, never provider response bodies or headers.
|
||||
status = re.search(r"HTTP Error (\d{3})", str(exc.detail.get("error", "")))
|
||||
if status:
|
||||
manifest["http_status"] = int(status.group(1))
|
||||
raise
|
||||
finally:
|
||||
manifest["provider_calls_attempted"] = limited.calls
|
||||
|
|
@ -154,7 +164,7 @@ def main():
|
|||
adapter = OpenRouterAssistedGenerationAdapter(model=args.model, retry_limit=0)
|
||||
result = execute(args.root, adapter, args.max_calls)
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
except (ValueError, OSError) as exc:
|
||||
except (ValueError, OSError, InfospaceError) as exc:
|
||||
parser.exit(1, f"{exc}\n")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -95,6 +95,29 @@ def test_failure_is_recorded_and_cannot_be_silently_retried(tmp_path):
|
|||
bench.execute(root, adapter())
|
||||
|
||||
|
||||
def test_provider_diagnostic_keeps_status_but_not_sensitive_detail(tmp_path):
|
||||
from infospace_bench.errors import InfospaceError
|
||||
|
||||
root = initialize(tmp_path)
|
||||
|
||||
class Denied:
|
||||
model = "example/model"
|
||||
|
||||
def generate(self, request):
|
||||
raise InfospaceError("provider_failed", "Request rejected", {
|
||||
"error": "HTTP Error 401: Unauthorized",
|
||||
"headers": {"Authorization": "Bearer TEST-SECRET-MUST-NOT-PERSIST"},
|
||||
})
|
||||
|
||||
with pytest.raises(InfospaceError):
|
||||
bench.execute(root, Denied())
|
||||
text = (root / "reports/research-run.json").read_text()
|
||||
result = json.loads(text)
|
||||
assert result["http_status"] == 401
|
||||
assert result["model"] == "example/model"
|
||||
assert "TEST-SECRET" not in text
|
||||
|
||||
|
||||
def test_unidentifiable_source_is_rejected_before_creating_workspace(tmp_path):
|
||||
source = tmp_path / "unidentified.md"
|
||||
source.write_text("# No bibliography")
|
||||
|
|
|
|||
|
|
@ -122,3 +122,22 @@ T02 in progress: SRC-0001/SRC-0002 verified and indexed. Two original-passage
|
|||
workspaces and four-call preflight plans are prepared under `research/bench/infospaces/`.
|
||||
No live generation or complete source review is claimed. Selection of model/provider,
|
||||
source-grounded investigation, broader case review and counterexamples remain.
|
||||
|
||||
|
||||
## First source investigation — 2026-09-24
|
||||
|
||||
Attempted the two four-stage source investigations with Gemini 2.5 Flash through
|
||||
OpenRouter. Both stopped at the first call; a separate authentication probe returned
|
||||
HTTP 401. No provider-generated research artifacts exist. The existing workload
|
||||
credential route does not apply to a generic workstation client; a correct route or
|
||||
provider was requested from the user. Failed receipts are retained.
|
||||
|
||||
Manual authority-focused reviews are in `research/cases/CarniatoGrandUnifyingArchitecture.md`
|
||||
and `research/cases/LocalFirstAuthority.md`; synthesis/dispositions are in
|
||||
`research/reviews/AuthorityPilot.md`. Source catalog states are extracted, not fully
|
||||
reviewed. Added safe error status/model recording to the adapter and a regression
|
||||
check preventing raw sensitive diagnostic detail from entering receipts.
|
||||
|
||||
T02 remains progress: broader evidence and independent known uses are still needed.
|
||||
Live reruns specifically depend on an authorized working credential; manual research
|
||||
can continue. No failure of the research hypothesis is inferred from provider failure.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue