activity-core/workplans/ACTIVITY-WP-0036-queue-identity-and-lease-integrity.md
tegwick 4d7623ac7a
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
chore(registrar): preserve partial identifier sync
Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a02b22-9638-76d2-bbff-b7ea1770b118
2026-08-23 14:40:44 +02:00

151 lines
6.1 KiB
Markdown

---
id: ACTIVITY-WP-0036
type: workplan
title: "Bind queue mutations to worker identity and active leases"
domain: infotech
repo: activity-core
status: active
owner: codex
topic_slug: activity-core
priority: high
created: "2026-08-23"
updated: "2026-08-23"
related:
- ACT-ADR-005
- ACTIVITY-WP-0026
- ACTIVITY-WP-0032
- HARNESS-WP-0003
state_hub_workstream_id: "01e6d5d4-6e1a-5f0d-81f0-ded97e0f71cd"
---
# Bind Queue Mutations to Worker Identity and Active Leases
## Origin
Activity Core's review of rein-aharness ADR-002 found two enforcement gaps at
the `ops_run` boundary:
1. `ACTIVITY_CORE_WORKER_TOKEN` authenticates a shared caller class, but the
authenticated principal is discarded and the caller supplies any
`worker_id` in the request body.
2. heartbeat, completion, and failure accept an owned `claimed` row even when
its lease has expired, until another claim or explicit expiry request happens
to reopen it.
These gaps make queue ownership advisory at precisely the point where a
repository executor needs a durable acceptance decision. Rein-side
cancellation remains necessary, but cannot replace server-side enforcement.
## Boundary and rollout rules
- Activity Core authenticates queue callers and owns lease acceptance.
- The queue `worker_id` is separate from the governed actor (`agt`) used by
Glas and sand-boxer.
- Operator SSO and break-glass credentials are not worker identities and must
not claim, heartbeat, complete, or fail runs through the normal worker API.
- Existing result normalization and retry ceilings remain unchanged.
- Production must receive the configured worker identity in the same rollout
that activates strict binding; do not strand the live consumer between API
and configuration revisions.
## Confirm the cross-repo responsibility contract
```task
id: ACTIVITY-WP-0036-T01
status: done
priority: high
state_hub_task_id: "fe4013f0-83ba-5697-8a3c-3fea1b0daedc"
```
Review rein-aharness ADR-002 against the implemented Activity Core boundary.
Report any distinction between current state and target guarantees, and require
the responsibility matrix to assign both worker authentication and lease
acceptance to an enforceable owner.
Activity Core sent required edits in State Hub message
`428abb02-75dc-450d-a7f6-56fc715409ab`: the ADR must disclose the current
self-asserted identity, require credential-to-worker binding, require
server-side rejection after lease expiry, and keep operator identity out of the
normal worker mutation path.
## Reject mutations without an active lease
```task
id: ACTIVITY-WP-0036-T02
status: done
priority: high
state_hub_task_id: "57da10c2-db6f-5a41-b306-3629901751da"
```
Lock the target row while deciding heartbeat, completion, or failure. Require
the row to be claimed by the caller and `lease_until` to be strictly later than
the server's current UTC time. A missing lease, or a deadline equal to or before
the decision time, is a conflict and must not update result or state. Stale
claim reopening must use the same deadline boundary.
Done when unit/API tests cover missing, equal, expired, active, and wrong-owner
leases and prove no late completion or heartbeat can revive an expired claim.
Implemented with row-level locking and one strict server-time predicate shared
by heartbeat, completion, and failure. Stale reopening now uses the same
inclusive expiry boundary (`lease_until <= now`). Tests prove missing, equal,
expired, active, and wrong-owner behavior without mutating late rows.
## Bind authenticated worker credentials to the claim owner
```task
id: ACTIVITY-WP-0036-T03
status: done
priority: high
state_hub_task_id: "de090738-01bd-58c7-80fa-6d36db222fc5"
```
Add an explicit configured queue worker identity and bind a valid worker token
to that identity. Compare the body `worker_id` to the authenticated identity
before every worker mutation and persist only the authenticated identity as
`claim_owner`. Refuse production worker mutations when token or identity
configuration is incomplete. Preserve an explicitly enabled local-development
path without presenting it as authenticated production behavior.
Done when tests prove spoofed worker ids, operator/SSO credentials on worker
mutations, missing production identity configuration, and invalid worker
tokens fail closed.
Implemented `ACTIVITY_CORE_WORKER_ID` as the non-secret identity bound to the
worker token. Claim and close paths validate the compatibility body field and
persist only the authenticated identity. Operator/SSO credentials are removed
from normal worker mutations; unauthenticated development requires the
existing explicit opt-in, and the former implicit open mode is gone.
## Document, deploy, and prove the boundary
```task
id: ACTIVITY-WP-0036-T04
status: progress
priority: high
state_hub_task_id: "1228e10a-067b-54db-baa3-a8fc9549492b"
```
Update the queue contract, runbook, and Railiance manifest with the non-secret
configured worker identity. Run focused and full tests, deploy API and consumer
configuration without a compatibility gap, and prove one authenticated claim
plus active heartbeat and a rejected mismatched/expired mutation. Return the
revision and bounded evidence to rein-aharness for its ADR acknowledgement
cycle.
Source verification: 468 tests passed with one live integration test skipped;
Python compilation, whitespace checks, and parsing all 16 Railiance Kubernetes
documents passed. The manifest binds the existing production worker token to
`rein-aharness@railiance01`. Production rollout and live rejection evidence
remain before this task and workplan can finish.
Production rollout completed on 2026-08-23 at revision `f0a897e`. The API pod
is Ready on the new image/configuration, the existing rein-aharness claim loop
received HTTP 200 on its first two authenticated polls, and a valid credential
paired with a spoofed body identity was rejected HTTP 403. Bounded evidence is
in `docs/evidence/ACTIVITY-WP-0036-queue-boundary-rollout-2026-08-23.md`.
No run was claimed during the observation window, so a natural active
heartbeat and rejected late close remain before T04 can be marked done. The
sole open row was an intentionally non-automated retention record and was not
modified to manufacture evidence.