tegwick
7d6d5a04c8
SCOPE.md now states only what exists (write-only cache, retry on any URLError, optional source, string-compared latest period). INTENT.md sharpens commitments 1 and 2. README and the capability record drop the replay overclaim. WP-0002 reworks T01 (selective retry) and adds T05 (enforce source), T06 (replay from cache), T07 (latest-period rule). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 153021@bnt-lap001 Assistant-Session: 46917be4-6400-4106-b8b7-61953a0caf24
30 lines
1.5 KiB
Markdown
30 lines
1.5 KiB
Markdown
# INTENT
|
|
|
|
`web-fetcher` exists to give research repositories a shared, trustworthy way
|
|
to pull data from the web and from public APIs.
|
|
|
|
**Core commitments**
|
|
|
|
1. **Provenance first.** No value without source, URL, retrieval time and
|
|
payload hash — plus the source's own publication time wherever the source
|
|
states one.
|
|
2. **Never invent.** Missing data is an explicit `not_found`, never a guess,
|
|
and when a source offers several candidates (for example the "latest"
|
|
period) the choice follows a stated rule visible in the record. Language
|
|
models are not a data source here.
|
|
3. **Reusable adapters.** Statistics APIs, CSV/JSON endpoints and
|
|
spreadsheets behind one small interface. New sources are added as adapters
|
|
with offline fixture tests.
|
|
4. **Replayable.** An optional content-addressed cache reproduces a run from
|
|
the exact bytes it used.
|
|
5. **Minimal footprint.** Standard library only, so it runs inside executors
|
|
(for example rein-aharness on railiance01) without dependency management.
|
|
|
|
**Contract.** Consumers depend on the `Observation` fields and the spec
|
|
format documented in `README.md`; adapters may be added freely, but changing
|
|
those fields is a breaking change. The capability is discoverable through
|
|
reuse-surface so research repos reuse it instead of writing their own fetchers.
|
|
|
|
Out of scope: crawling, scraping behind logins, LLM-based extraction,
|
|
scheduling (activity-core owns *when*), and execution orchestration
|
|
(rein-aharness owns *how it runs*).
|