activity-core/workplans/ACTIVITY-WP-0036-queue-identity-and-lease-integrity.md
tegwick 3cd525051d
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Record WP0036 production rollout
Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a028de-e2c8-7732-8521-46a7fc5db82f
2026-08-23 13:07:13 +02:00

5.8 KiB

id type title domain repo status owner topic_slug priority created updated related
ACTIVITY-WP-0036 workplan Bind queue mutations to worker identity and active leases infotech activity-core active codex activity-core high 2026-08-23 2026-08-23
ACT-ADR-005
ACTIVITY-WP-0026
ACTIVITY-WP-0032
HARNESS-WP-0003

Bind Queue Mutations to Worker Identity and Active Leases

Origin

Activity Core's review of rein-aharness ADR-002 found two enforcement gaps at the ops_run boundary:

  1. ACTIVITY_CORE_WORKER_TOKEN authenticates a shared caller class, but the authenticated principal is discarded and the caller supplies any worker_id in the request body.
  2. heartbeat, completion, and failure accept an owned claimed row even when its lease has expired, until another claim or explicit expiry request happens to reopen it.

These gaps make queue ownership advisory at precisely the point where a repository executor needs a durable acceptance decision. Rein-side cancellation remains necessary, but cannot replace server-side enforcement.

Boundary and rollout rules

  • Activity Core authenticates queue callers and owns lease acceptance.
  • The queue worker_id is separate from the governed actor (agt) used by Glas and sand-boxer.
  • Operator SSO and break-glass credentials are not worker identities and must not claim, heartbeat, complete, or fail runs through the normal worker API.
  • Existing result normalization and retry ceilings remain unchanged.
  • Production must receive the configured worker identity in the same rollout that activates strict binding; do not strand the live consumer between API and configuration revisions.

Confirm the cross-repo responsibility contract

id: ACTIVITY-WP-0036-T01
status: done
priority: high

Review rein-aharness ADR-002 against the implemented Activity Core boundary. Report any distinction between current state and target guarantees, and require the responsibility matrix to assign both worker authentication and lease acceptance to an enforceable owner.

Activity Core sent required edits in State Hub message 428abb02-75dc-450d-a7f6-56fc715409ab: the ADR must disclose the current self-asserted identity, require credential-to-worker binding, require server-side rejection after lease expiry, and keep operator identity out of the normal worker mutation path.

Reject mutations without an active lease

id: ACTIVITY-WP-0036-T02
status: done
priority: high

Lock the target row while deciding heartbeat, completion, or failure. Require the row to be claimed by the caller and lease_until to be strictly later than the server's current UTC time. A missing lease, or a deadline equal to or before the decision time, is a conflict and must not update result or state. Stale claim reopening must use the same deadline boundary.

Done when unit/API tests cover missing, equal, expired, active, and wrong-owner leases and prove no late completion or heartbeat can revive an expired claim.

Implemented with row-level locking and one strict server-time predicate shared by heartbeat, completion, and failure. Stale reopening now uses the same inclusive expiry boundary (lease_until <= now). Tests prove missing, equal, expired, active, and wrong-owner behavior without mutating late rows.

Bind authenticated worker credentials to the claim owner

id: ACTIVITY-WP-0036-T03
status: done
priority: high

Add an explicit configured queue worker identity and bind a valid worker token to that identity. Compare the body worker_id to the authenticated identity before every worker mutation and persist only the authenticated identity as claim_owner. Refuse production worker mutations when token or identity configuration is incomplete. Preserve an explicitly enabled local-development path without presenting it as authenticated production behavior.

Done when tests prove spoofed worker ids, operator/SSO credentials on worker mutations, missing production identity configuration, and invalid worker tokens fail closed.

Implemented ACTIVITY_CORE_WORKER_ID as the non-secret identity bound to the worker token. Claim and close paths validate the compatibility body field and persist only the authenticated identity. Operator/SSO credentials are removed from normal worker mutations; unauthenticated development requires the existing explicit opt-in, and the former implicit open mode is gone.

Document, deploy, and prove the boundary

id: ACTIVITY-WP-0036-T04
status: progress
priority: high

Update the queue contract, runbook, and Railiance manifest with the non-secret configured worker identity. Run focused and full tests, deploy API and consumer configuration without a compatibility gap, and prove one authenticated claim plus active heartbeat and a rejected mismatched/expired mutation. Return the revision and bounded evidence to rein-aharness for its ADR acknowledgement cycle.

Source verification: 468 tests passed with one live integration test skipped; Python compilation, whitespace checks, and parsing all 16 Railiance Kubernetes documents passed. The manifest binds the existing production worker token to rein-aharness@railiance01. Production rollout and live rejection evidence remain before this task and workplan can finish.

Production rollout completed on 2026-08-23 at revision f0a897e. The API pod is Ready on the new image/configuration, the existing rein-aharness claim loop received HTTP 200 on its first two authenticated polls, and a valid credential paired with a spoofed body identity was rejected HTTP 403. Bounded evidence is in docs/evidence/ACTIVITY-WP-0036-queue-boundary-rollout-2026-08-23.md.

No run was claimed during the observation window, so a natural active heartbeat and rejected late close remain before T04 can be marked done. The sole open row was an intentionally non-automated retention record and was not modified to manufacture evidence.