Schedule Core Hub stabilization checks for T02 sign-off

Add daily and closeout ActivityDefinitions, a manual stabilization script, runbook, and workplan notes for the 2026-07-03 to 2026-07-10 window.
2026-07-08 00:27:13 +02:00 · 2026-07-08 00:27:13 +02:00 · e3f01a46e7
commit e3f01a46e7
parent b042a7077a
6 changed files with 236 additions and 2 deletions
--- a/5
+++ b/5
@ -1,4 +1,4 @@
-.PHONY: install test lint run openapi migrate-validate deployed-smoke operator-cli staging-profile-check container-build playwright-install visual-check
+.PHONY: install test lint run openapi migrate-validate deployed-smoke stabilization-check operator-cli staging-profile-check container-build playwright-install visual-check

 UV ?= /home/worsch/.local/bin/uv
 PYTHONPATH ?= src
@ -28,6 +28,9 @@ migrate-validate:
 deployed-smoke:
 	PYTHONPATH=$(PYTHONPATH) $(UV) run --extra dev python scripts/core_hub_deployed_smoke.py

+stabilization-check:
+	bash scripts/core_hub_stabilization_check.sh
+
 operator-cli:
 	PYTHONPATH=$(PYTHONPATH) $(UV) run --extra dev python scripts/core_hub_cli.py $(CLI_ARGS)

--- a/activity-definitions/core-hub-stabilization-closeout.md
+++ b/activity-definitions/core-hub-stabilization-closeout.md
@ -0,0 +1,47 @@
+---
+id: "c5d9f3a2-7b4e-5f6c-0a1d-3e8f9b2c4d5e"
+name: "Core Hub Stabilization Closeout Check"
+type: activity-definition
+version: "1.0"
+enabled: true
+owner: core-hub
+governance: core-hub
+status: active
+created: "2026-07-07"
+trigger:
+  type: scheduled
+  at: "2026-07-10T17:35:00+00:00"
+  timezone: UTC
+context_sources:
+  - type: core-hub
+    query: stabilization_check
+    required: true
+    params:
+      source: activity-core
+      closeout: true
+      base_url: "https://hub.coulomb.social"
+      window_start: "2026-07-03T00:00:00+00:00"
+      window_end: "2026-07-10T17:35:00+00:00"
+      min_widget_types: 26
+      evidence_sinks:
+        - type: state-hub-progress
+          event_type: core_hub_stabilization_closeout
+          author: activity-core
+          workstream_id: a8d66822-e435-4b1e-ad81-37a298d1795e
+          task_id: 16eb7ce3-b574-4667-a189-c14ff5d0502b
+    bind_to: context.core_hub_stabilization_check
+---
+
+# ActivityDefinition: Core Hub Stabilization Closeout Check
+
+## Purpose
+
+One-shot closeout gate at the end of the Core Hub stabilization window. When
+`overall_pass` is true, the result sets `operator_signoff_needed=true` so the
+operator can approve `CORE-WP-0007-T02` (Inter-Hub rename / rollback retirement).
+
+## Output Contract
+
+- one State Hub progress event with `event_type: core_hub_stabilization_closeout`
+- fails the activity run when closeout checks do not pass
+- operator records the retirement decision separately in State Hub
--- a/activity-definitions/core-hub-stabilization-daily.md
+++ b/activity-definitions/core-hub-stabilization-daily.md
@ -0,0 +1,55 @@
+---
+id: "b4c8e2f1-6a3d-4e5b-9f0c-2d7e8a1b3c4d"
+name: "Core Hub Stabilization Daily Check"
+type: activity-definition
+version: "1.0"
+enabled: true
+owner: core-hub
+governance: core-hub
+status: active
+created: "2026-07-07"
+trigger:
+  type: cron
+  cron_expression: "0 9 * * *"
+  timezone: Europe/Berlin
+  misfire_policy: skip
+context_sources:
+  - type: core-hub
+    query: stabilization_check
+    required: true
+    params:
+      source: activity-core
+      closeout: false
+      base_url: "https://hub.coulomb.social"
+      window_start: "2026-07-03T00:00:00+00:00"
+      window_end: "2026-07-10T17:35:00+00:00"
+      min_widget_types: 26
+      evidence_sinks:
+        - type: state-hub-progress
+          event_type: core_hub_stabilization_check
+          author: activity-core
+          workstream_id: a8d66822-e435-4b1e-ad81-37a298d1795e
+          task_id: 16eb7ce3-b574-4667-a189-c14ff5d0502b
+    bind_to: context.core_hub_stabilization_check
+---
+
+# ActivityDefinition: Core Hub Stabilization Daily Check
+
+## Purpose
+
+Scheduled hub-visible health gate for `CORE-WP-0007-T02` during the post-cutover
+stabilization window (2026-07-03 → 2026-07-10). Complements the manual
+`scripts/core_hub_stabilization_check.sh` script with fleet-side evidence in
+State Hub progress.
+
+## Trigger
+
+Daily at 09:00 Europe/Berlin while the stabilization window is open. Runs
+outside the window are recorded as `skipped` with
+`reason=outside_stabilization_window`.
+
+## Output Contract
+
+- one State Hub progress event with `event_type: core_hub_stabilization_check`
+- compact pass/fail detail for public production surface checks
+- no LLM call and no direct workplan edits
--- a/docs/deployment/stabilization-signoff.md
+++ b/docs/deployment/stabilization-signoff.md
@ -0,0 +1,65 @@
+# Core Hub production stabilization
+
+Post-cutover checks for `CORE-WP-0007-T02` operator sign-off after
+`CORE-WP-0005` moved `hub.coulomb.social` to Core Hub (2026-07-03).
+
+## Window
+
+- **Start:** 2026-07-03 (production cutover)
+- **End:** 2026-07-10T17:35:00Z (closeout job)
+- **Daily job:** 09:00 Europe/Berlin via activity-core
+
+## Automated checks
+
+The `core-hub` context resolver query `stabilization_check` verifies:
+
+| Check | Expected |
+| --- | --- |
+| `GET /healthz` | 200 |
+| `GET /readyz` | 200 |
+| `GET /api/v2/widget-types` | 200, count ≥ 26 |
+| `GET /api/v2/hubs` (unauthenticated) | 401 |
+
+State Hub progress events:
+
+- `core_hub_stabilization_check` — daily while the window is open
+- `core_hub_stabilization_closeout` — one-shot at window end
+
+Activity definitions live in `activity-definitions/` and are projected into
+`activity-core/k8s/railiance/20-runtime.yaml`.
+
+## Manual companion script
+
+```bash
+CORE_HUB_BASE_URL=https://hub.coulomb.social \
+STATE_HUB_URL=http://127.0.0.1:8000 \
+./scripts/core_hub_stabilization_check.sh
+```
+
+Optional full bootstrap smoke (needs approved operator token):
+
+```bash
+CORE_HUB_BASE_URL=https://hub.coulomb.social \
+CORE_HUB_OPERATOR_TOKEN_FILE=/secure/path/to/token \
+make deployed-smoke
+```
+
+## Operator sign-off for T02
+
+Sign-off is a human decision recorded in State Hub after:
+
+1. Daily checks (or closeout) show `overall_pass=true` through the window
+2. No rollback was invoked since cutover
+3. Operator approves retiring the Inter-Hub k8s rollback deployment
+
+Post a progress note or decision referencing workstream
+`a8d66822-e435-4b1e-ad81-37a298d1795e` / task `16eb7ce3-b574-4667-a189-c14ff5d0502b`,
+then proceed with `CORE-WP-0007-T02`.
+
+## Deploy / enable
+
+1. Rebuild and deploy `activity-core` on railiance01 with the new `core-hub`
+   context resolver.
+2. Apply the updated `activity-core/k8s/railiance/20-runtime.yaml` ConfigMap
+   projection for the two new ActivityDefinitions.
+3. Confirm events appear at `GET /progress/?event_type=core_hub_stabilization_check`.
--- a/scripts/core_hub_stabilization_check.sh
+++ b/scripts/core_hub_stabilization_check.sh
@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# Core Hub production stabilization snapshot for CORE-WP-0007-T02.
+# Complements the scheduled activity-core job (core_hub_stabilization_check).
+set -euo pipefail
+
+BASE_URL="${CORE_HUB_BASE_URL:-https://hub.coulomb.social}"
+STATE_HUB_URL="${STATE_HUB_URL:-http://127.0.0.1:8000}"
+
+echo "=== Core Hub public surface (${BASE_URL}) ==="
+for path in /healthz /readyz /api/v2/widget-types; do
+  code=$(curl -sS -o /dev/null -w '%{http_code}' "${BASE_URL}${path}")
+  echo "${path} -> ${code}"
+done
+hubs_code=$(curl -sS -o /dev/null -w '%{http_code}' "${BASE_URL}/api/v2/hubs")
+echo "/api/v2/hubs -> ${hubs_code} (expect 401 hardened contract)"
+widget_count=$(curl -fsS "${BASE_URL}/api/v2/widget-types" | python3 -c "
+import json,sys
+payload=json.load(sys.stdin)
+items=payload if isinstance(payload,list) else payload.get('data',[])
+print(len(items))
+")
+echo "widget_types count=${widget_count} (expect >= 26)"
+
+echo
+echo "=== optional deployed smoke (needs approved token) ==="
+if [[ -n "${CORE_HUB_OPERATOR_TOKEN_FILE:-}" && -f "${CORE_HUB_OPERATOR_TOKEN_FILE}" ]]; then
+  make deployed-smoke
+else
+  echo "skip: set CORE_HUB_OPERATOR_TOKEN_FILE for full bootstrap smoke"
+fi
+
+echo
+echo "=== optional ops-hub gate probe ==="
+if command -v make >/dev/null && [[ -d "${HOME}/ops-hub" ]]; then
+  (cd "${HOME}/ops-hub" && make interhub-gate IHUB_BASE="${BASE_URL}") || true
+else
+  echo "skip: ops-hub checkout unavailable"
+fi
+
+echo
+echo "=== last 6 core_hub_stabilization_check events ==="
+curl -fsS "${STATE_HUB_URL}/progress/?event_type=core_hub_stabilization_check&limit=6" | python3 -c "
+import json,sys
+for e in json.load(sys.stdin):
+    d=e.get('detail') or {}
+    probe=d.get('probe') or {}
+    print(e['created_at'][:19], (e.get('summary') or '')[:72],
+          'pass='+str(probe.get('overall_pass')))
+" 2>/dev/null || echo "no stabilization events yet"
--- a/workplans/CORE-WP-0007-haskell-retirement.md
+++ b/workplans/CORE-WP-0007-haskell-retirement.md
@ -121,4 +121,19 @@ Related finished workplans: `CORE-WP-0004` (compatibility smokes),
  triggers)

 Forgejo repo archived on `gitea.coulomb.social` (2026-07-07); git history and
-diagnostic docs remain available read-only.
+diagnostic docs remain available read-only.
+
+
+## Progress 2026-07-07 — stabilization job scheduled
+
+Post-cutover stabilization checks for T02 operator sign-off:
+
+- `activity-core` `core-hub` context resolver: `stabilization_check`
+- Activity definitions: `activity-definitions/core-hub-stabilization-{daily,closeout}.md`
+- Manual script: `scripts/core_hub_stabilization_check.sh` (`make stabilization-check`)
+- Runbook: `docs/deployment/stabilization-signoff.md`
+- Window: 2026-07-03 → 2026-07-10 (daily 09:00 Europe/Berlin; closeout 2026-07-10T17:35Z)
+- State Hub events: `core_hub_stabilization_check`, `core_hub_stabilization_closeout`
+
+Requires activity-core image rebuild + k8s ConfigMap apply on railiance01 before
+the schedule is live.