Harden ops run identity and leases
Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a028de-e2c8-7732-8521-46a7fc5db82f
This commit is contained in:
parent
36161d346f
commit
f0a897e088
13 changed files with 522 additions and 50 deletions
135
workplans/ACTIVITY-WP-0036-queue-identity-and-lease-integrity.md
Normal file
135
workplans/ACTIVITY-WP-0036-queue-identity-and-lease-integrity.md
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
---
|
||||
id: ACTIVITY-WP-0036
|
||||
type: workplan
|
||||
title: "Bind queue mutations to worker identity and active leases"
|
||||
domain: infotech
|
||||
repo: activity-core
|
||||
status: active
|
||||
owner: codex
|
||||
topic_slug: activity-core
|
||||
priority: high
|
||||
created: "2026-08-23"
|
||||
updated: "2026-08-23"
|
||||
related:
|
||||
- ACT-ADR-005
|
||||
- ACTIVITY-WP-0026
|
||||
- ACTIVITY-WP-0032
|
||||
- HARNESS-WP-0003
|
||||
---
|
||||
|
||||
# Bind Queue Mutations to Worker Identity and Active Leases
|
||||
|
||||
## Origin
|
||||
|
||||
Activity Core's review of rein-aharness ADR-002 found two enforcement gaps at
|
||||
the `ops_run` boundary:
|
||||
|
||||
1. `ACTIVITY_CORE_WORKER_TOKEN` authenticates a shared caller class, but the
|
||||
authenticated principal is discarded and the caller supplies any
|
||||
`worker_id` in the request body.
|
||||
2. heartbeat, completion, and failure accept an owned `claimed` row even when
|
||||
its lease has expired, until another claim or explicit expiry request happens
|
||||
to reopen it.
|
||||
|
||||
These gaps make queue ownership advisory at precisely the point where a
|
||||
repository executor needs a durable acceptance decision. Rein-side
|
||||
cancellation remains necessary, but cannot replace server-side enforcement.
|
||||
|
||||
## Boundary and rollout rules
|
||||
|
||||
- Activity Core authenticates queue callers and owns lease acceptance.
|
||||
- The queue `worker_id` is separate from the governed actor (`agt`) used by
|
||||
Glas and sand-boxer.
|
||||
- Operator SSO and break-glass credentials are not worker identities and must
|
||||
not claim, heartbeat, complete, or fail runs through the normal worker API.
|
||||
- Existing result normalization and retry ceilings remain unchanged.
|
||||
- Production must receive the configured worker identity in the same rollout
|
||||
that activates strict binding; do not strand the live consumer between API
|
||||
and configuration revisions.
|
||||
|
||||
## Confirm the cross-repo responsibility contract
|
||||
|
||||
```task
|
||||
id: ACTIVITY-WP-0036-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Review rein-aharness ADR-002 against the implemented Activity Core boundary.
|
||||
Report any distinction between current state and target guarantees, and require
|
||||
the responsibility matrix to assign both worker authentication and lease
|
||||
acceptance to an enforceable owner.
|
||||
|
||||
Activity Core sent required edits in State Hub message
|
||||
`428abb02-75dc-450d-a7f6-56fc715409ab`: the ADR must disclose the current
|
||||
self-asserted identity, require credential-to-worker binding, require
|
||||
server-side rejection after lease expiry, and keep operator identity out of the
|
||||
normal worker mutation path.
|
||||
|
||||
## Reject mutations without an active lease
|
||||
|
||||
```task
|
||||
id: ACTIVITY-WP-0036-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Lock the target row while deciding heartbeat, completion, or failure. Require
|
||||
the row to be claimed by the caller and `lease_until` to be strictly later than
|
||||
the server's current UTC time. A missing lease, or a deadline equal to or before
|
||||
the decision time, is a conflict and must not update result or state. Stale
|
||||
claim reopening must use the same deadline boundary.
|
||||
|
||||
Done when unit/API tests cover missing, equal, expired, active, and wrong-owner
|
||||
leases and prove no late completion or heartbeat can revive an expired claim.
|
||||
|
||||
Implemented with row-level locking and one strict server-time predicate shared
|
||||
by heartbeat, completion, and failure. Stale reopening now uses the same
|
||||
inclusive expiry boundary (`lease_until <= now`). Tests prove missing, equal,
|
||||
expired, active, and wrong-owner behavior without mutating late rows.
|
||||
|
||||
## Bind authenticated worker credentials to the claim owner
|
||||
|
||||
```task
|
||||
id: ACTIVITY-WP-0036-T03
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Add an explicit configured queue worker identity and bind a valid worker token
|
||||
to that identity. Compare the body `worker_id` to the authenticated identity
|
||||
before every worker mutation and persist only the authenticated identity as
|
||||
`claim_owner`. Refuse production worker mutations when token or identity
|
||||
configuration is incomplete. Preserve an explicitly enabled local-development
|
||||
path without presenting it as authenticated production behavior.
|
||||
|
||||
Done when tests prove spoofed worker ids, operator/SSO credentials on worker
|
||||
mutations, missing production identity configuration, and invalid worker
|
||||
tokens fail closed.
|
||||
|
||||
Implemented `ACTIVITY_CORE_WORKER_ID` as the non-secret identity bound to the
|
||||
worker token. Claim and close paths validate the compatibility body field and
|
||||
persist only the authenticated identity. Operator/SSO credentials are removed
|
||||
from normal worker mutations; unauthenticated development requires the
|
||||
existing explicit opt-in, and the former implicit open mode is gone.
|
||||
|
||||
## Document, deploy, and prove the boundary
|
||||
|
||||
```task
|
||||
id: ACTIVITY-WP-0036-T04
|
||||
status: progress
|
||||
priority: high
|
||||
```
|
||||
|
||||
Update the queue contract, runbook, and Railiance manifest with the non-secret
|
||||
configured worker identity. Run focused and full tests, deploy API and consumer
|
||||
configuration without a compatibility gap, and prove one authenticated claim
|
||||
plus active heartbeat and a rejected mismatched/expired mutation. Return the
|
||||
revision and bounded evidence to rein-aharness for its ADR acknowledgement
|
||||
cycle.
|
||||
|
||||
Source verification: 468 tests passed with one live integration test skipped;
|
||||
Python compilation, whitespace checks, and parsing all 16 Railiance Kubernetes
|
||||
documents passed. The manifest binds the existing production worker token to
|
||||
`rein-aharness@railiance01`. Production rollout and live rejection evidence
|
||||
remain before this task and workplan can finish.
|
||||
Loading…
Add table
Add a link
Reference in a new issue