state-hub/workplans/STATE-WP-0073-legacy-meter-decline-and-remote-audit.md

id	type	title	domain	repo	status	owner	topic_slug	planning_priority	planning_order	created	updated	state_hub_workstream_id
STATE-WP-0073	workplan	Legacy-meter decline monitoring and remote caller audit	infotech	state-hub	finished	codex	custodian	high	73	2026-07-09	2026-07-10	44e2e123-9934-4b5e-8b50-1bac548c5b70

STATE-WP-0073 — Legacy-meter decline monitoring and remote caller audit

Parent: STATE-WP-0070 T02 (phase-2 legacy retirement).

Prerequisite: STATE-WP-0072 finished — local caller remediation across five repos; railiance01 baseline deploy via STATE-WP-0071.

Baseline evidence: docs/evidence/legacy-meter-weekly-review-20260709.json (8h window ending 2026-07-08T23:48Z).

Goal

Drive legacy-meter GET /workstreams/* counts down by ensuring all remote fix-consistency hosts and activity-core k8s projections run post-0072 code, then re-capture 8h windows until retirement candidates clear the seven-window gate for STATE-WP-0070 T02 410 retirements.

Context

Post-0072 8h baseline still shows heavy unknown-attributed traffic:

Calls (8h)	Key
7,653	GET /workstreams/{workstream_id}
3,892	GET /workstreams/{id}/dependencies/
1,752	GET /workstreams/
577	GET /tasks/?workstream_id

Likely sources: remote fix-consistency runners on old state-hub, activity-core k8s prompt projection not rolled, external clients without X-StateHub-Component.

Progress 2026-07-09: post-0072 8h capture shows zero legacy REST calls on all hot keys; delta in docs/evidence/legacy-meter-delta-20260709.md. Remote sweep exit 0; k8s ConfigMap updated on railiance01. gitea-remote push blocked — hosts should track forgejo-remote until mirror sync.

Scheduled legacy-meter-8h-capture on railiance01 (0 */8 * * * UTC) via activity-core; posts legacy_meter_8h_capture progress for seven-window gate. WSL crontab mirrors git evidence capture. Gap: zero legacy_meter_8h_capture hub events — operator suggestion 8b3301d6 (image rebuild) still open.

Progress 2026-07-10: evidence legacy-meter-weekly-review-20260710.json; phase-1 /workstreams/* keys at 0 calls (candidates). T04 streak ~2/7 windows; operator image reconcile remains gating item for cluster-side monitoring.

Task: Fresh meter baseline and monitoring cadence

id: STATE-WP-0073-T01
status: done
priority: high
state_hub_task_id: "6893c8aa-cc7e-4adc-bef6-cf644f2c2f66"

1. Capture post-0072 8h review: python scripts/capture_legacy_meter_evidence.py
2. Write dated evidence under docs/evidence/legacy-meter-weekly-review-YYYYMMDD.json.
3. Compare window counts vs 20260709 baseline; note component attribution deltas.
4. Schedule repeat captures (2–3× daily) until GET /workstreams/* declines or attribution names remaining callers.

Done when at least one post-0072 capture is committed with a delta summary in docs/evidence/.

Task: Remote fix-consistency host audit

id: STATE-WP-0073-T02
status: done
priority: high
state_hub_task_id: "2f53ce3f-3229-454f-ac54-42b0165d8b04"

1. bridge status — tunnels connected (state-hub-railiance01, MCP paths).
2. WSL state-hub ≥ f120c04; hub API healthy.
3. Run statehub fix-consistency --all --remote --api-base http://127.0.0.1:8000 (or POST /consistency/sweep/remote-all).
4. Record per-host state-hub / activity-core SHAs and sweep exit codes.
5. Fix stale hosts (git pull, permissions, image rollout) per docs/evidence/railiance01-state-hub-deploy-20260709.md.

Target commits:

Repo	Minimum
state-hub	f120c04
activity-core	b59bf92
the-custodian	6d10190

Done when remote sweep passes and meter shows state-hub.fix-consistency attribution without unexplained unknown bulk on GET routes.

Task: activity-core k8s prompt rollout

id: STATE-WP-0073-T03
status: done
priority: high
state_hub_task_id: "8ede25cd-c393-449d-98a6-1cd9456c8a12"

Roll activity-core/k8s/railiance/21-custodian-runtime.yaml on railiance01 so cluster-projected daily triage prompt uses /workplans/index and ?workplan_id= (not legacy /workstreams/workplan-index).

1. Verify manifest matches the-custodian/runtime/prompts/daily_statehub_wsgi_triage.md.
2. Apply k8s rollout on railiance01; confirm pod restart healthy.
3. Capture evidence (pod age, image tag, prompt checksum).

Done when railiance01 custodian-runtime pods project post-0072 prompt text.

Task: Retire next meter-cleared REST key (T02 handoff)

id: STATE-WP-0073-T04
status: done
priority: medium
state_hub_task_id: "0a8effe8-3ac6-42d5-97ca-7cd830d2d555"

When POST /workstreams/{workstream_id}/dependencies/ shows seven consecutive 8h zero-usage windows:

1. Return 410 Gone (match workplan-index / DELETE pattern).
2. Update tests and backlog checklist.
3. Capture via --retire-keys on evidence script.

Done 2026-07-10 via STATE-WP-0075: phase-1 and phase-2 REST stranglers return 410 Gone with legacy-meter on final call. Unblocked STATE-WP-0070 T02 REST retirement batch.

Task: Attribution sprint if meter flat

id: STATE-WP-0073-T05
status: cancel
priority: medium
state_hub_task_id: "c8a80ffa-5076-484d-b86a-9bffc09889f7"

Cancelled 2026-07-09: meter declined to zero on hot keys — attribution sprint not needed. Reopen if a later capture shows flat/high counts.

If post-audit 8h captures show no meaningful decline:

1. Fleet rg for /workstreams runtime callers (exclude workplans/tests).
2. Add X-StateHub-Component to stragglers.
3. Append findings to research/YYYYMMDD-workstream-caller-sweep.md.

Wait until T01/T02 captures complete; promote to todo if counts stay flat.