tegwick
b186746f4a
The probe expected the daily fep_feedback_intake sink on all three frontend-patterns definitions and required an explicit catchup_window_seconds that live rows do not carry. It now checks each cadence's own sink, and accepts an absent window, which resolves to the catchup_latest 24h default (the schedule's catchup_window is still checked). Closes ACTIVITY-WP-ADHOC-2026-10-19-T03. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 244572@bnt-lap001 Assistant-Session: 3b235e1a-ae45-4d0f-ae70-17a9bd78ece8
87 lines
3.6 KiB
Markdown
87 lines
3.6 KiB
Markdown
---
|
|
id: ACTIVITY-WP-ADHOC-2026-10-19
|
|
type: workplan
|
|
title: Activity Core owns recovery of lapsed ops_run leases
|
|
domain: infotech
|
|
repo: activity-core
|
|
status: active
|
|
owner: claude-activity-core
|
|
created: "2026-10-19"
|
|
updated: "2026-10-19"
|
|
related: [REINAH-WP-0003]
|
|
state_hub_workstream_id: "b75bd9fb-6308-5195-9a30-d264c0031bf2"
|
|
---
|
|
|
|
Origin: rein-aharness REINAH-WP-0003-T05 open point on thread 27cd4c47. When
|
|
the worker's first heartbeat gets a 409 before dispatch, the run stays
|
|
`claimed`, because a worker must not close a lease it has lost. Activity
|
|
Core's position: that is correct, and recovery is Activity Core's job.
|
|
|
|
## Expire claims that keep losing their lease
|
|
|
|
```task
|
|
id: ACTIVITY-WP-ADHOC-2026-10-19-T01
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "63ddf98f-c5e2-5130-8ca3-4a7573fb5c5b"
|
|
```
|
|
|
|
`reopen_stale_claims` returned every lapsed claim to `open` with no attempt
|
|
ceiling, so a row that kept losing its lease could cycle forever. New
|
|
`expire_stale_claims` in `ops_run_queue.py`, which runs before each claim and
|
|
behind `POST /ops-runs/expire-leases`, now reopens lapsed claims below
|
|
`OPS_RUN_MAX_ATTEMPTS`. At the ceiling it sets terminal `expired`, keeps the
|
|
owner for audit and records `result.error`. A late close on an expired row is
|
|
refused as `state_conflict`. The endpoint returns
|
|
`{"reopened": n, "expired": m}`. Contract documented in
|
|
`docs/ops-run-queue.md`, with three new tests in `tests/test_ops_run_queue.py`.
|
|
|
|
## Release the expiry change to production
|
|
|
|
```task
|
|
id: ACTIVITY-WP-ADHOC-2026-10-19-T02
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "d94bfae7-c5ad-585f-96a0-a93a1010c111"
|
|
```
|
|
|
|
Deployed 2026-10-19 (founder-approved attended release): api and worker run
|
|
`git-9a46350bb3e1192b76aab859218f95312e17fdf4`
|
|
(`sha256:08966e8dd976d2e6f50711f33b2ab0abaa650aed998b28d743483999729e7d66`),
|
|
Argo child `activity-core` at 8884b12, Synced Healthy. `POST
|
|
/ops-runs/expire-leases` returned `{"reopened":0,"expired":0}`. Rollback:
|
|
f9f2640 / `sha256:fdc631d7`. Evidence:
|
|
`docs/evidence/2026-10-19-grant-v2-lease-expiry-release.json`.
|
|
|
|
## Fix the /release/health probe contract
|
|
|
|
```task
|
|
id: ACTIVITY-WP-ADHOC-2026-10-19-T03
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "0bfc9af8-1b75-59ab-91e6-cef5b09bc08d"
|
|
```
|
|
|
|
Residual of T02. The first live `GET /release/health` reported
|
|
`healthy:false` although the schedules and report sinks are correct.
|
|
`release_health.definition_matches()` requires
|
|
`trigger_config.catchup_window_seconds == 86400`, which the live rows (synced
|
|
2026-09-27) do not carry, and it expects the `fep_feedback_intake` sink on all
|
|
three frontend-patterns definitions, while weekly and monthly use
|
|
`fep_improvement_review` and `fep_monthly_review`. Fix the probe to check the
|
|
per-cadence sinks, and decide whether to run an admin definition sync or
|
|
accept a missing `catchup_window_seconds` when the schedule enforces it.
|
|
|
|
2026-10-20: fixed in `release_health.py`.
|
|
- Each cadence now expects its own sink: daily `fep_feedback_intake`, weekly
|
|
`fep_improvement_review`, monthly `fep_monthly_review`. All sit on topic
|
|
`fe2aaa78` with author activity-core, as in the 2026-10-03
|
|
natural-cadence evidence.
|
|
- A row with no `catchup_window_seconds` is accepted, because the
|
|
`catchup_latest` policy's default in `schedule_manager` is 24h. An explicit
|
|
value other than 86400 is still refused, and the probe still checks the live
|
|
Temporal `catchup_window` separately. No admin definition sync is needed.
|
|
- Two new tests cover these cases. The full suite passes (877 passed,
|
|
1 skipped).
|
|
- Not deployed: the fix ships with the next image release, and live
|
|
`/release/health` stays false until then.
|