fix(legacy-meter): capture evidence over 7 days, not an 8-hour band
capture_legacy_meter_evidence.py fell back to hours=8 whenever --days was omitted, and --hours itself defaulted to 8. Every unattended capture sampled 06:00Z-14:00Z while writing a file named weekly-review with cadence: weekly. 39 of 40 captures ran this way; only 2026-07-08 used a true 7-day window. Calls outside the band were never sampled, so interfaces with live callers reported as retirement candidates -- GET /tasks/?workstream_id was flagged on 2026-08-19 despite traffic on 2026-08-18. Default the script to days=7; keep --hours for spot checks, documented as not retirement evidence. Adds corrected capture for 2026-08-20 and records the residual gap (candidate rule ignores last_seen_at) against STATE-WP-0079-T05. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
5fe38fa9d8
commit
be7f2632c3
4 changed files with 1793 additions and 22 deletions
1716
docs/evidence/legacy-meter-weekly-review-20260820.json
Normal file
1716
docs/evidence/legacy-meter-weekly-review-20260820.json
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -179,27 +179,55 @@ D3 is two MCP tools with no dependants; take it opportunistically.
|
|||
| Slice | Capability | Items | Gate |
|
||||
| --- | --- | --- | --- |
|
||||
| E1 | `suggestion-backlog` → archive | 17 | caller check; no successor runtime |
|
||||
| E2 | `work-records-workplan-legacy` → archive | 13 | legacy-meter — see caveat below |
|
||||
| E2 | `work-records-workplan-legacy` → archive | 13 | legacy-meter — candidate rule, see below |
|
||||
| E3 | `dashboard-meta` | 1 | none |
|
||||
| E4 | `legacy-meter` itself | 9 | **last** — it is the instrument for E2 |
|
||||
|
||||
E4 retires only after every other legacy surface is closed; retiring the meter
|
||||
first destroys the evidence for the retirements it gates.
|
||||
|
||||
### Caveat on E2's evidence
|
||||
### E2 evidence — defect found and partly fixed (2026-08-20)
|
||||
|
||||
The 2026-08-19 review (`docs/evidence/legacy-meter-weekly-review-20260819.json`)
|
||||
reports 19 retirement candidates, all `replacement_verified: true` with zero
|
||||
calls in window. But the window is `06:00Z → 14:00Z` — **8 hours, not the 7 days
|
||||
`cadence: weekly` implies**. Several candidates have very large all-time counts
|
||||
(`GET /workstreams/{id}` 511 406, `GET /workstreams/` 144 086,
|
||||
`POST /workstreams/{id}/dependencies/` 255 865).
|
||||
The window discrepancy flagged on 2026-08-19 was a real defect in
|
||||
`scripts/capture_legacy_meter_evidence.py`, not a labelling nit.
|
||||
|
||||
An 8-hour quiet window is not sufficient evidence to retire an interface at that
|
||||
call volume. Either the window computation is a defect or the cadence label is
|
||||
wrong; either way **E2 should not proceed on the current evidence**. Resolving
|
||||
this is a prerequisite, and it is worth checking before Wave A rather than after,
|
||||
since the same meter gates T05.
|
||||
`_review_query` fell back to `hours=8` whenever `--days` was omitted, and
|
||||
`--hours` defaulted to `8`. Every unattended capture therefore sampled an
|
||||
8-hour band (06:00Z–14:00Z) while writing a file named `weekly-review` with
|
||||
`cadence: weekly` in the payload. **39 of 40 captures ran this way** — only the
|
||||
first, 2026-07-08, used a true 7-day window.
|
||||
|
||||
Calls in the other 16 hours of each day were never sampled, so quiet-in-band
|
||||
interfaces were reported as retirement candidates while still having live
|
||||
callers. On the 2026-08-19 capture three candidates had recent traffic:
|
||||
|
||||
| Interface | All-time calls | Last seen |
|
||||
| --- | --- | --- |
|
||||
| `GET /tasks/?workstream_id` | 594 | 2026-08-18 |
|
||||
| `GET /progress/?workstream_id` | 2 | 2026-08-12 |
|
||||
| `GET /workstreams/` | 144 086 | 2026-08-10 |
|
||||
|
||||
**Fixed:** the script now defaults to `days=7`, matching the endpoint name and
|
||||
the cadence label. `--hours` is retained for spot checks and documented as not
|
||||
retirement evidence. Corrected capture:
|
||||
`docs/evidence/legacy-meter-weekly-review-20260820.json` — 18 candidates of 20
|
||||
interfaces, `GET /tasks/?workstream_id` correctly excluded.
|
||||
|
||||
**Still open — E2 remains gated.** A 7-day window is not sufficient for
|
||||
high-volume interfaces, and two false positives survive it:
|
||||
|
||||
- `GET /progress/?workstream_id` — last seen 2026-08-12T08:55Z, roughly 14 hours
|
||||
before the window opened
|
||||
- `GET /workstreams/` — last seen 2026-08-10, 9 days quiet, 144 086 all-time calls
|
||||
|
||||
`last_seen_at` is unbounded by the window and is the stronger signal; the
|
||||
candidate rule currently ignores it. Before E2 proceeds, the rule should require
|
||||
a minimum quiet period scaled to call volume — an interface with six figures of
|
||||
traffic needs materially more than seven silent days. That is a service-side
|
||||
change in `api/services/legacy_meter.py` and belongs to T05, not here.
|
||||
|
||||
Everything else on the list was last seen 2026-07-31 or earlier, most in early
|
||||
July, so the bulk of E2 is well-evidenced once the rule is tightened.
|
||||
|
||||
## Keep — 43 items
|
||||
|
||||
|
|
|
|||
|
|
@ -71,10 +71,20 @@ def _retire_interfaces(base: str, keys: list[str]) -> list[dict]:
|
|||
return retired
|
||||
|
||||
|
||||
DEFAULT_REVIEW_DAYS = 7
|
||||
|
||||
|
||||
def _review_query(*, days: int | None, hours: int | None) -> str:
|
||||
if days is not None:
|
||||
return f"days={days}"
|
||||
return f"hours={hours or 8}"
|
||||
"""Build the review-window query.
|
||||
|
||||
Defaults to a 7-day window to match the ``weekly-review`` endpoint and the
|
||||
``cadence: weekly`` label in the payload. An hours-scoped window only
|
||||
samples part of each day, so an interface called outside that band reads as
|
||||
unused and is falsely reported as a retirement candidate.
|
||||
"""
|
||||
if hours is not None:
|
||||
return f"hours={hours}"
|
||||
return f"days={days or DEFAULT_REVIEW_DAYS}"
|
||||
|
||||
|
||||
def capture(
|
||||
|
|
@ -101,10 +111,10 @@ def capture(
|
|||
"retired_interfaces": retired,
|
||||
"weekly_review": review,
|
||||
}
|
||||
if days is not None:
|
||||
payload["days"] = days
|
||||
if hours is not None:
|
||||
payload["hours"] = hours
|
||||
else:
|
||||
payload["hours"] = hours or 8
|
||||
payload["days"] = days or DEFAULT_REVIEW_DAYS
|
||||
|
||||
if dry_run:
|
||||
print(json.dumps(payload, indent=2))
|
||||
|
|
@ -124,14 +134,15 @@ def main() -> None:
|
|||
parser.add_argument(
|
||||
"--hours",
|
||||
type=int,
|
||||
default=8,
|
||||
help="Review window in hours (default: 8; used when --days is omitted)",
|
||||
default=None,
|
||||
help="Review window in hours; only samples part of each day — use for "
|
||||
"spot checks, never as retirement evidence",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--days",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Review window in days (weekly cadence; overrides --hours)",
|
||||
help=f"Review window in days (default: {DEFAULT_REVIEW_DAYS}, weekly cadence)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--api-base",
|
||||
|
|
|
|||
|
|
@ -131,6 +131,22 @@ state_hub_task_id: "02e508ed-3cde-4487-907e-d324a8a877d6"
|
|||
Complete retirement of suggestions, workstream aliases, and other `retire`
|
||||
inventory items once meters/callers allow. Keep historical rows archive-readable.
|
||||
|
||||
**Blocker identified (2026-08-20):** the legacy-meter evidence this task gates
|
||||
on was being captured over an 8-hour band, not a week —
|
||||
`capture_legacy_meter_evidence.py` fell back to `hours=8` whenever `--days` was
|
||||
omitted, so 39 of 40 captures sampled only 06:00Z–14:00Z. Interfaces called
|
||||
outside that band read as unused. Three candidates on the 2026-08-19 capture had
|
||||
live callers, including `GET /tasks/?workstream_id` (seen 2026-08-18).
|
||||
|
||||
Script default corrected to `days=7`; corrected capture
|
||||
`docs/evidence/legacy-meter-weekly-review-20260820.json`.
|
||||
|
||||
**Remaining work for this task:** the candidate rule in
|
||||
`api/services/legacy_meter.py` keys only on in-window calls and ignores
|
||||
`last_seen_at`, so `GET /workstreams/` (144 086 all-time calls, quiet 9 days)
|
||||
still reports as a candidate. Require a minimum quiet period scaled to call
|
||||
volume before retiring. See `docs/retirement-cutover-slice-plan.md` § E2.
|
||||
|
||||
## Stabilization window and archive prep
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue