web-fetcher/INTENT.md

# INTENT

`web-fetcher` exists to give research repositories a shared, trustworthy way
to pull data from the web and from public APIs.

**Core commitments**

1. **Provenance first.** No value without source, URL, retrieval time and
   payload hash — plus the source's own publication time wherever the source
   states one.
2. **Never invent.** Missing data is an explicit `not_found`, never a guess,
   and when a source offers several candidates (for example the "latest"
   period) the choice follows a stated rule visible in the record. Language
   models are not a data source here.
3. **Reusable adapters.** Statistics APIs, CSV/JSON endpoints and
   spreadsheets behind one small interface. New sources are added as adapters
   with offline fixture tests.
4. **Replayable.** An optional content-addressed cache reproduces a run from
   the exact bytes it used.
5. **Minimal footprint.** Standard library only, so it runs inside executors
   (for example rein-aharness on railiance01) without dependency management.

**Contract.** Consumers depend on the `Observation` fields and the spec
format documented in `README.md`; adapters may be added freely, but changing
those fields is a breaking change. The capability is discoverable through
reuse-surface so research repos reuse it instead of writing their own fetchers.

Out of scope: crawling, scraping behind logins, LLM-based extraction,
scheduling (activity-core owns *when*), and execution orchestration
(rein-aharness owns *how it runs*).