coulomb/web-fetcher
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
web-fetcher/INTENT.md
tegwick c48d6113dd Seeded repo
2026-10-13 14:04:02 +02:00

982 B
Raw Blame History

INTENT

web-fetcher exists to give research repositories a shared, trustworthy way to pull data from the web and from public APIs.

Core commitments

  1. Provenance first. No value without source, URL, retrieval time and payload hash.
  2. Never invent. Missing data is an explicit not_found, never a guess. Language models are not a data source here.
  3. Reusable adapters. Statistics APIs, CSV/JSON endpoints and spreadsheets behind one small interface. New sources are added as adapters with offline fixture tests.
  4. Replayable. An optional content-addressed cache reproduces a run from the exact bytes it used.
  5. Minimal footprint. Standard library only, so it runs inside executors (for example rein-aharness on railiance01) without dependency management.

Out of scope: crawling, scraping behind logins, LLM-based extraction, scheduling (activity-core owns when), and execution orchestration (rein-aharness owns how it runs).