--- id: ACTIVITY-WP-ADHOC-2026-10-19 type: workplan title: Activity Core owns recovery of lapsed ops_run leases domain: infotech repo: activity-core status: active owner: claude-activity-core created: "2026-10-19" updated: "2026-10-19" related: [REINAH-WP-0003] state_hub_workstream_id: "b75bd9fb-6308-5195-9a30-d264c0031bf2" --- Origin: rein-aharness REINAH-WP-0003-T05 open point on thread 27cd4c47. When the worker's first heartbeat gets a 409 before dispatch, the run stays `claimed`, because a worker must not close a lease it has lost. Activity Core's position: that is correct, and recovery is Activity Core's job. ## Expire claims that keep losing their lease ```task id: ACTIVITY-WP-ADHOC-2026-10-19-T01 status: done priority: high state_hub_task_id: "63ddf98f-c5e2-5130-8ca3-4a7573fb5c5b" ``` `reopen_stale_claims` returned every lapsed claim to `open` with no attempt ceiling, so a row that kept losing its lease could cycle forever. New `expire_stale_claims` in `ops_run_queue.py`, which runs before each claim and behind `POST /ops-runs/expire-leases`, now reopens lapsed claims below `OPS_RUN_MAX_ATTEMPTS`. At the ceiling it sets terminal `expired`, keeps the owner for audit and records `result.error`. A late close on an expired row is refused as `state_conflict`. The endpoint returns `{"reopened": n, "expired": m}`. Contract documented in `docs/ops-run-queue.md`, with three new tests in `tests/test_ops_run_queue.py`. ## Release the expiry change to production ```task id: ACTIVITY-WP-ADHOC-2026-10-19-T02 status: done priority: medium state_hub_task_id: "d94bfae7-c5ad-585f-96a0-a93a1010c111" ``` Deployed 2026-10-19 (founder-approved attended release): api and worker run `git-9a46350bb3e1192b76aab859218f95312e17fdf4` (`sha256:08966e8dd976d2e6f50711f33b2ab0abaa650aed998b28d743483999729e7d66`), Argo child `activity-core` at 8884b12, Synced Healthy. `POST /ops-runs/expire-leases` returned `{"reopened":0,"expired":0}`. Rollback: f9f2640 / `sha256:fdc631d7`. Evidence: `docs/evidence/2026-10-19-grant-v2-lease-expiry-release.json`. ## Fix the /release/health probe contract ```task id: ACTIVITY-WP-ADHOC-2026-10-19-T03 status: todo priority: medium state_hub_task_id: "0bfc9af8-1b75-59ab-91e6-cef5b09bc08d" ``` Residual of T02. The first live `GET /release/health` reported `healthy:false` although the schedules and report sinks are correct. `release_health.definition_matches()` requires `trigger_config.catchup_window_seconds == 86400`, which the live rows (synced 2026-09-27) do not carry, and it expects the `fep_feedback_intake` sink on all three frontend-patterns definitions, while weekly and monthly use `fep_improvement_review` and `fep_monthly_review`. Fix the probe to check the per-cadence sinks, and decide whether to run an admin definition sync or accept a missing `catchup_window_seconds` when the schedule enforces it.