55 lines
2.7 KiB
Markdown
55 lines
2.7 KiB
Markdown
|
# ACTIVITY-WP-0039: multi-worker identity cutover (2026-09-23/24)
|
||
|
|
|
||
|
|
Founder go-ahead, build mode. No token value was printed, logged, or committed.
|
||
|
|
|
||
|
|
## Custody
|
||
|
|
|
||
|
|
- railiance-platform T06 retired the static ESO token path. The
|
||
|
|
`openbao-activity-core` store uses Kubernetes auth (RPF-WP-0045).
|
||
|
|
- T03: the founder ran an attended mint of both worker tokens, option (b)
|
||
|
|
(fresh values), with `scripts/wp0039-seed-worker-tokens.sh`:
|
||
|
|
`platform/workloads/activity-core/ops-run-workers/rein-aharness-railiance01`
|
||
|
|
and `.../rein-aharness-metered-railiance01`, field `token`.
|
||
|
|
- T02: railiance-platform granted exact-path read to
|
||
|
|
`workload-kv-read-activity-core-eso`, applied at 2026-09-23T18:06:29Z
|
||
|
|
(CCR-2026-0029/0030, railiance-platform `fe1665d`).
|
||
|
|
- ExternalSecret `actcore-ops-run-worker-tokens` synced both keys at
|
||
|
|
2026-09-23T20:35:35Z. ESO now owns `ACTIVITY_CORE_WORKER_TOKEN` and
|
||
|
|
`ACTIVITY_CORE_WORKER_TOKEN_METERED` in the activity-core runtime Secret.
|
||
|
|
|
||
|
|
## Rollout
|
||
|
|
|
||
|
|
1. With 0 claimed runs, the claim-loop env file was backed up
|
||
|
|
(`claim-loop.env.bak-wp0039-20260923T203559`) and given the new loop token
|
||
|
|
inside a remote process, never printed.
|
||
|
|
2. First API rollout at 20:38Z on `activity-core:wp0039-20260923` (main
|
||
|
|
`1a20c85`): HTTP 500, `column ops_runs.repository_grant does not exist`.
|
||
|
|
Production was at alembic `0009`, and main needs `0010` (WP-0038, never
|
||
|
|
deployed before). The running `fi-publication-20260914` image contained no
|
||
|
|
WP-0038 code. The API was rolled back, and the loop claimed again with HTTP
|
||
|
|
200 at 20:39:23Z. The outage lasted about 1 minute, and no run was affected.
|
||
|
|
3. On 2026-09-24 the founder ran the one-off Job `actcore-migrate-wp0039`
|
||
|
|
(`alembic upgrade head`, new image), giving `0009 -> 0010`, with 0 runs
|
||
|
|
claimed. The old API kept claiming with 200 against the new schema.
|
||
|
|
4. The API was redeployed on `activity-core:wp0039-20260923` at 05:24:18Z.
|
||
|
|
`kubectl diff` showed the image line only. The ConfigMap carries
|
||
|
|
`ACTIVITY_CORE_WORKERS`. The worker and event router remain on
|
||
|
|
`fi-publication-20260914`.
|
||
|
|
|
||
|
|
## Proof (2026-09-24)
|
||
|
|
|
||
|
|
- The claim loop keeps polling `POST /ops-runs/claim` with HTTP 200 as
|
||
|
|
`rein-aharness@railiance01` through the new API (05:24:40Z), with no loop
|
||
|
|
restart needed.
|
||
|
|
- The founder ran this. With the metered token and body
|
||
|
|
`rein-aharness-metered@railiance01` on label `wp0039-proof-no-match`, the
|
||
|
|
response was HTTP 200 `{"items":[],"lease_seconds":900}` and nothing was
|
||
|
|
claimed.
|
||
|
|
- With the same token and body `rein-aharness@railiance01`, the response was
|
||
|
|
HTTP 403 `worker_id does not match authenticated worker`.
|
||
|
|
|
||
|
|
## Side effect: WP-0038 is now live
|
||
|
|
|
||
|
|
This rollout is the first production deployment of the ACTIVITY-WP-0038
|
||
|
|
repository-grant carriage and exact terminal close reconciliation.
|