activity-core/workplans/ACTIVITY-WP-0049-wave2-resource-right-sizing.md
tegwick b7cfe087b2
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Wave-2 right-sizing applied 2026-10-22 (CUST-WP-0071)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 276069@bnt-lap001
Assistant-Session: 2e192b3d-b3e8-4403-94d4-d55a2c8b476c
2026-10-22 01:03:36 +02:00

6.1 KiB

id type title domain repo status flavor owner topic_slug created updated related state_hub_workstream_id
ACTIVITY-WP-0049 workplan Wave-2 resource right-sizing from the 2026-10-21 allocation review infotech activity-core ready implementation claude activity-core 2026-10-21 2026-10-21
CUST-WP-0071
RESOURCE-WP-0008
de34ab8e-3f4c-514b-82b6-671588362a6c

ACTIVITY-WP-0049 — Wave-2 resource right-sizing from the 2026-10-21 allocation review

Receiving workplan in activity-core for the wave-2 request changes below.

Origin: the 2026-10-21 node allocation review (RESOURCE-WP-0008-T03, feeding CUST-WP-0071-T03). Per the founder's 2026-10-21 delivery interview (the-custodian/docs/decisions/2026-10-21-cust-wp-0071-delivery-interview.md) the small adjustments form wave 2: one receiving task per owner repo, applied later in a batch. Applying wave 2 needs a separate founder approval, so every cluster-write task here is a human-gated wait. This file changes no manifest.

Apply the reviewed request changes

id: ACTIVITY-WP-0049-T01
status: wait
blocking_reason: "ArgoCD part (api/worker/event-router) needs an exception to the image-only promotion check; infra, relay and llm-connect applied."
priority: medium
needs_human: true
state_hub_task_id: "588ffa46-5368-5479-a2f2-df8419245a77"

Values are pod-aggregate figures from the budget file (7-day window ending 2026-10-21T06:08Z). Limits stay unchanged unless stated.

Workload (ns/name) Pods CPU request CPU limit CPU p95/peak 7d (m) Memory request Memory limit Mem p95/peak (Mi) Throttle Action
activity-core/activity-core 10 100m → 100m 1000m (8 container(s) without CPU limit) 62.8/132.5 256Mi → 944Mi 1024Mi 811.0/1011.0 0.0 keep + memory up

Source and container detail:

  • activity-core/activity-core: ten pods in k8s/railiance/10-infrastructure.yaml (StatefulSets actcore-app-db, actcore-temporal-db with 50m/256Mi each, actcore-nats; Deployments actcore-temporal, actcore-temporal-ui) and k8s/railiance/20-runtime.yaml. CPU stays at 100m (within tolerance). Memory p95 811Mi vs 256Mi requested: raise the requests to a 944Mi total, distributed by each pod's p95. Eight containers have no CPU request or limit; give each at least the 5m floor.

Evidence: resource-control docs/evidence/2026-10-21-node-allocation-review/2026-10-21-node-budget.md (+ .json, commit 3d9b1bd). Rule: CPU = max(floor, p95x1.5, peakx0.5) rounded up to 5m; memory request = 7d p95 working set x1.15 (only when the change is at least 64Mi).

Steps:

  1. Map the pod-aggregate proposal onto the named container(s) in the source file; where a workload has several containers or pods, split by the per-pod figures in 2026-10-21-workload-usage.json and keep the sum at the proposed value.
  2. Prepare the source change as a reviewed commit (no push, no sync) and record the before values here.
  3. Apply only inside the batched wave-2 rollout, after founder approval, through the owner's normal deploy path (ArgoCD sync or documented helm/kubectl apply).

Verification: kubectl -n <ns> get pod -o jsonpath='{..resources}' shows the new requests; pods Ready with no new restarts or OOMKilled over 24h; for CPU decreases the throttle ratio and request latency stay at or below the 7d baseline; node request total drops by the expected delta.

Rollback: revert the source commit and re-sync/re-apply; the before values in the table above are the rollback target. Keep before/after kubectl get -o yaml captures as the receipt.

Prepared 2026-10-22 (wave-2 worker W1b)

Status stays wait / needs_human. Local commit only: not pushed, not applied. Diff = full owner render vs live (kubectl diff, object names and changed key paths only, Secret objects skipped).

  • Totals kept at the proposal, 100m CPU / 944Mi memory over the 10 live pods (requests only, no new limits): api 5m/96Mi, worker 5m/176Mi, event-router 5m/80Mi, statehub-edge-relay 50m/128Mi -> 10m/96Mi, temporal 35m/240Mi, temporal-ui 5m/16Mi, nats 5m/16Mi, app-db 10m/48Mi, temporal-db 15m/128Mi, llm-connect 50m/128Mi -> 5m/48Mi.
  • Commits: activity-core d3cd1a2 (k8s/gitops/runtime.yaml, k8s/railiance/10-infrastructure.yaml, k8s/railiance/20-runtime.yaml) and llm-connect 19006db (deploy/k8s/activity-core-llm-connect/deployment.yaml).
  • Diff: resource fields only for all objects. The local diff of the gitops render also drops the ArgoCD tracking-id annotation, which is expected. Existing drift: the source already declared app-db/temporal-db 50m/256Mi requests and a 1Gi memory limit that were never applied. Applying 10-infrastructure therefore also adds that owner-declared 1Gi memory limit to both DBs, which the founder should confirm.
  • Restart: yes. app-db and temporal-db (single-replica postgres StatefulSets), nats, temporal, temporal-ui, relay, api, worker, event-router and llm-connect all roll.
  • Gates: (1) api/worker/event-router are ArgoCD-managed with targetRevision pinned in railiance-platform/argocd/railiance01/applications/activity-core.application.yaml, so after push the pin needs a bump. That repo is owned by another worker. (2) scripts/check_gitops_promotion.py is an image-only admission policy and refuses a resources-only gitops change. The founder needs to accept the exception.
  • Apply: kubectl apply -f k8s/railiance/10-infrastructure.yaml; apply only the relay Deployment from 20-runtime.yaml, not the whole file, because it holds immutable Jobs and ArgoCD-owned Deployments; kubectl apply -k ~/llm-connect/deploy/k8s/activity-core-llm-connect; then bump the ArgoCD pin and sync activity-core.

Applied — 2026-10-22 (ACTIVITY-WP-0049-T01)

Partial: the infrastructure is applied (app-db and temporal-db restarted, including the already-declared 1Gi memory limits), and so are the relay and llm-connect. All are Ready. The ArgoCD part (api, worker, event-router) waits: scripts/check_gitops_promotion.py admits image changes only, so it needs an owner exception or a promotion-check change, then a pin bump in railiance-platform. Evidence: the-custodian docs/evidence/2026-10-22-wave2-apply.md.