coulomb/web-fetcher
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
web-fetcher/INTENT.md
tegwick 2ea8c1337a Finish WEBFETCH-WP-0001: refine SCOPE/INTENT, dev workflow, reuse-surface registry, seed WP-0002 
- SCOPE.md rewritten with repo-specific boundaries and current state
- INTENT.md: consumer contract and capability id
- AGENTS.md: developer workflow (test/run/validate)
- registry/: capability.infotech.provenance-fetch (D2/A2/C1/R1)
- workplans: WP-0001 finished, WP-0002 proposed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 153021@bnt-lap001
Assistant-Session: 46917be4-6400-4106-b8b7-61953a0caf24
2026-10-19 13:51:40 +02:00

1.2 KiB
Raw Blame History

INTENT

web-fetcher exists to give research repositories a shared, trustworthy way to pull data from the web and from public APIs.

Core commitments

  1. Provenance first. No value without source, URL, retrieval time and payload hash.
  2. Never invent. Missing data is an explicit not_found, never a guess. Language models are not a data source here.
  3. Reusable adapters. Statistics APIs, CSV/JSON endpoints and spreadsheets behind one small interface. New sources are added as adapters with offline fixture tests.
  4. Replayable. An optional content-addressed cache reproduces a run from the exact bytes it used.
  5. Minimal footprint. Standard library only, so it runs inside executors (for example rein-aharness on railiance01) without dependency management.

Contract. Consumers depend on the Observation fields and the spec format documented in README.md; adapters may be added freely, but changing those fields is a breaking change. Discoverable through reuse-surface as capability.infotech.provenance-fetch.

Out of scope: crawling, scraping behind logins, LLM-based extraction, scheduling (activity-core owns when), and execution orchestration (rein-aharness owns how it runs).