tegwick 96fe5b022b Release 0.2.0: replay, source enforcement, period rule, selective retry (WEBFETCH-WP-0002)

- --replay DIR / replay_dir: offline replay from the cache, hash-verified
- ok values without source become error; Last-Modified recorded as published
- latest period by parsed order (web_fetcher/periods.py); mixed formats refused
- non-numeric values not_found unless value_type: string
- retry only connection errors, timeouts, 429/5xx; honour Retry-After
- opt-in live smoke test; 18 offline tests
- CHANGELOG, README install docs, SCOPE update, capability D2/A3/C2/R2
- WEBFETCH-WP-0003 (repo hygiene) added

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 153021@bnt-lap001
Assistant-Session: 46917be4-6400-4106-b8b7-61953a0caf24

2026-10-19 14:08:35 +02:00

1.8 KiB

Raw Blame History

Changelog

0.2.0 — 2026-10-19

Closes the gaps between SCOPE.md and INTENT.md (WEBFETCH-WP-0002).

Added

Replay: --replay DIR / run_spec(..., replay_dir=) serves every URL from a cache written by --cache, without network access; payload hashes are verified and the original retrieval time is kept, so a replayed run reproduces the original observations. Uncached URLs become error.
HTTP Last-Modified is recorded as published for the http_json, http_csv and xlsx_wide adapters, and stored in the cache index.
period_format (default auto) and value_type spec fields.
Opt-in live smoke test: WEB_FETCHER_LIVE=1 python3 -B -m unittest tests.test_live.

Changed (may affect existing specs)

A value is never returned without a source: an ok observation from a spec without source becomes error (the Eurostat adapter names its own).
"Latest" period is chosen by parsed period order (year, month, date, quarter, semester, week, fixed-width digits). Mixed or unrecognised formats are an error unless period_format is set (string = lexicographic). The rule used is stated in the observation note.
Non-numeric values are not_found unless value_type: string; . is also treated as a missing marker. Ambiguous matches (several CSV rows with different values for the latest period, or several XLSX rows) are error.
Retries only on connection errors, timeouts and HTTP 429/5xx, with exponential backoff (1.5 s, 3 s) or the server's Retry-After (capped at 120 s); other HTTP errors fail immediately.

Verified: all 31 assembly-economics checkpoint specs give identical status/value/period to 0.1.0, and a replay of the cached run is byte-identical.

0.1.0 — 2026-10-12

Initial release: Observation records, Eurostat / JSON / CSV / wide-XLSX adapters, write-only payload cache, CLI.