tegwick
96fe5b022b
- --replay DIR / replay_dir: offline replay from the cache, hash-verified - ok values without source become error; Last-Modified recorded as published - latest period by parsed order (web_fetcher/periods.py); mixed formats refused - non-numeric values not_found unless value_type: string - retry only connection errors, timeouts, 429/5xx; honour Retry-After - opt-in live smoke test; 18 offline tests - CHANGELOG, README install docs, SCOPE update, capability D2/A3/C2/R2 - WEBFETCH-WP-0003 (repo hygiene) added Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 153021@bnt-lap001 Assistant-Session: 46917be4-6400-4106-b8b7-61953a0caf24
37 lines
1.8 KiB
Markdown
37 lines
1.8 KiB
Markdown
# Changelog
|
|
|
|
## 0.2.0 — 2026-10-19
|
|
|
|
Closes the gaps between `SCOPE.md` and `INTENT.md` (WEBFETCH-WP-0002).
|
|
|
|
### Added
|
|
- Replay: `--replay DIR` / `run_spec(..., replay_dir=)` serves every URL from a
|
|
cache written by `--cache`, without network access; payload hashes are
|
|
verified and the original retrieval time is kept, so a replayed run
|
|
reproduces the original observations. Uncached URLs become `error`.
|
|
- HTTP `Last-Modified` is recorded as `published` for the `http_json`,
|
|
`http_csv` and `xlsx_wide` adapters, and stored in the cache index.
|
|
- `period_format` (default `auto`) and `value_type` spec fields.
|
|
- Opt-in live smoke test: `WEB_FETCHER_LIVE=1 python3 -B -m unittest tests.test_live`.
|
|
|
|
### Changed (may affect existing specs)
|
|
- A value is never returned without a source: an `ok` observation from a
|
|
spec without `source` becomes `error` (the Eurostat adapter names its own).
|
|
- "Latest" period is chosen by parsed period order (year, month, date,
|
|
quarter, semester, week, fixed-width digits). Mixed or unrecognised formats
|
|
are an `error` unless `period_format` is set (`string` = lexicographic).
|
|
The rule used is stated in the observation `note`.
|
|
- Non-numeric values are `not_found` unless `value_type: string`; `.` is also
|
|
treated as a missing marker. Ambiguous matches (several CSV rows with
|
|
different values for the latest period, or several XLSX rows) are `error`.
|
|
- Retries only on connection errors, timeouts and HTTP 429/5xx, with
|
|
exponential backoff (1.5 s, 3 s) or the server's `Retry-After` (capped at
|
|
120 s); other HTTP errors fail immediately.
|
|
|
|
Verified: all 31 assembly-economics checkpoint specs give identical
|
|
status/value/period to 0.1.0, and a replay of the cached run is byte-identical.
|
|
|
|
## 0.1.0 — 2026-10-12
|
|
|
|
Initial release: Observation records, Eurostat / JSON / CSV / wide-XLSX
|
|
adapters, write-only payload cache, CLI.
|