diff --git a/README.md b/README.md index 9a8fdb9..870f60c 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ Core Hub is the 3rd-generation Production Interaction Framework for Coulomb / He | Name | Role | | --- | --- | -| **hub-core** | Shared library — `hub_core` utils, models, router factories | +| **hub-core** | Production runtime and shared framework/library | | **state-hub** | Dev coordination — file-first workplans, agent MCP | -| **core-hub** | This repo — production `/api/v2` framework | +| **core-hub** | This repo — previous `/api/v2` runtime, retained as rollback until archive | Ecosystem architecture: `/home/worsch/the-custodian/docs/hub-ecosystem-architecture.md` @@ -42,11 +42,12 @@ It carries forward the lessons from State Hub and Inter-Hub: workplans and progr ## Current Work The contract, FastAPI/Postgres foundation, Inter-Hub migration, Haskell/IHP -retirement, and relocation from CoulombCore to railiance01 are complete. -`CORE-WP-0011` is finished. The only open local workplan is runtime absorption -and eventual archive (`CORE-WP-0010`). Its dual-run design is recorded and S0 -is active against published hub-core revision `7e1ec03`, building the durable -backend and internal candidate before any public traffic changes. +retirement, relocation from CoulombCore, and production absorption into +hub-core are complete. Hub-core revision `055cf49` is public authority. This +repository remains a no-writer rollback through the stabilization window +ending no earlier than 2026-08-28T20:49:50+02:00. New product work belongs in +hub-core; the only open local task is the prepared read-only archive closeout +in [`docs/deployment/archive-closeout.md`](docs/deployment/archive-closeout.md). Operational migration bundles are emitted without raw key material: diff --git a/STATE.md b/STATE.md index 0588247..3e994f1 100644 --- a/STATE.md +++ b/STATE.md @@ -35,7 +35,7 @@ Deployment packaging and rollout truth lives in the `rapp-core-hub` repository. | --- | --- | --- | | `CORE-WP-0001`–`CORE-WP-0009` | finished / archived | Gen3 runtime, contracts, persistence, consumer gates, and hardening delivered | | `CORE-WP-0011` | finished | Production relocated to railiance01 and active CoulombCore runtime retired | -| `CORE-WP-0010` | active | Production cutover done; stabilization/archive gate remains | +| `CORE-WP-0010` | active | Archive packet prepared; timed stabilization/closeout remains | ### Open work @@ -45,7 +45,10 @@ Deployment packaging and rollout truth lives in the `rapp-core-hub` repository. | `CORE-WP-0010-T02` dual-run design | done | Route-group plan recorded in `docs/specs/runtime-absorption-plan.md` | | `CORE-WP-0010-T03` absorption slices | done | Seven-table parity, conformance, restart, and consumer gates passed | | `CORE-WP-0010-T04` production cutover | done | Hub-core authoritative; rollback and forward recovery exercised | -| `CORE-WP-0010-T05` archive | wait | Seven-day stabilization through 2026-08-28T20:49:50+02:00 | +| `CORE-WP-0010-T05` archive | progress | Closure packet ready; stabilization through 2026-08-28T20:49:50+02:00 | + +The executable residual-owner map, retention boundary, and ordered closeout +checklist are in `docs/deployment/archive-closeout.md`. ## Intent position diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index beb44c1..9a60160 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -53,7 +53,7 @@ | task | CORE-WP-0010-T02 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0010-T03 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0010-T04 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | -| task | CORE-WP-0010-T05 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | +| task | CORE-WP-0010-T05 | progress | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0011-T01 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md | | task | CORE-WP-0011-T02 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md | | task | CORE-WP-0011-T03 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md | diff --git a/activity-definitions/core-hub-stabilization-closeout.md b/activity-definitions/core-hub-stabilization-closeout.md index 4872cc7..8c03b2a 100644 --- a/activity-definitions/core-hub-stabilization-closeout.md +++ b/activity-definitions/core-hub-stabilization-closeout.md @@ -3,11 +3,12 @@ id: "c5d9f3a2-7b4e-5f6c-0a1d-3e8f9b2c4d5e" name: "Core Hub Stabilization Closeout Check" type: activity-definition version: "1.0" -enabled: true +enabled: false owner: core-hub governance: core-hub -status: active +status: archived created: "2026-07-07" +updated: "2026-08-21" trigger: type: scheduled at: "2026-07-10T17:35:00+00:00" @@ -34,6 +35,10 @@ context_sources: # ActivityDefinition: Core Hub Stabilization Closeout Check +> Historical definition for the completed July 2026 Core Hub/Inter-Hub +> stabilization window. It is disabled and must not be reused for the August +> hub-core cutover window. + ## Purpose One-shot closeout gate at the end of the Core Hub stabilization window. When @@ -44,4 +49,4 @@ operator can approve `CORE-WP-0007-T02` (Inter-Hub rename / rollback retirement) - one State Hub progress event with `event_type: core_hub_stabilization_closeout` - fails the activity run when closeout checks do not pass -- operator records the retirement decision separately in State Hub \ No newline at end of file +- operator records the retirement decision separately in State Hub diff --git a/activity-definitions/core-hub-stabilization-daily.md b/activity-definitions/core-hub-stabilization-daily.md index 903afee..fc4618c 100644 --- a/activity-definitions/core-hub-stabilization-daily.md +++ b/activity-definitions/core-hub-stabilization-daily.md @@ -3,11 +3,12 @@ id: "b4c8e2f1-6a3d-4e5b-9f0c-2d7e8a1b3c4d" name: "Core Hub Stabilization Daily Check" type: activity-definition version: "1.0" -enabled: true +enabled: false owner: core-hub governance: core-hub -status: active +status: archived created: "2026-07-07" +updated: "2026-08-21" trigger: type: cron cron_expression: "0 9 * * *" @@ -35,6 +36,10 @@ context_sources: # ActivityDefinition: Core Hub Stabilization Daily Check +> Historical definition for the completed July 2026 Core Hub/Inter-Hub +> stabilization window. It is disabled and must not be reused for the August +> hub-core cutover window. + ## Purpose Scheduled hub-visible health gate for `CORE-WP-0007-T02` during the post-cutover @@ -52,4 +57,4 @@ outside the window are recorded as `skipped` with - one State Hub progress event with `event_type: core_hub_stabilization_check` - compact pass/fail detail for public production surface checks -- no LLM call and no direct workplan edits \ No newline at end of file +- no LLM call and no direct workplan edits diff --git a/docs/deployment/archive-closeout.md b/docs/deployment/archive-closeout.md new file mode 100644 index 0000000..cdc9d14 --- /dev/null +++ b/docs/deployment/archive-closeout.md @@ -0,0 +1,85 @@ +# Core Hub archive closeout + +Prepared 2026-08-21 for `CORE-WP-0010-T05`. This document is an executable +closure packet, not evidence that the time-gated archive has happened. + +## Freeze boundary + +Core Hub is no longer a product-development target. Until archive: + +- accept only stabilization, security, rollback, evidence, or archive changes; +- implement new runtime and compatibility behavior in hub-core; +- keep Core Hub Ready with an empty writer set; +- never enable legacy and hub-core writers concurrently; and +- do not delete the legacy schema, immutable image, Helm history, or retained + migration evidence during this task. + +The absence of an approved retention expiry blocks destructive cleanup. It +does not block scaling the rollback workload to zero or archiving this Git +repository read-only after operator approval. + +## Residual ownership + +| Residual | Live owner / record | Archive condition | +| --- | --- | --- | +| Production runtime, compatibility API, migration code, and future product work | `hub-core`; completed baseline `HUB-WP-0005`, new changes require hub-core workplans | No Core Hub implementation remains authoritative | +| Deployment, route map, immutable images, live verification, rollback retirement | `rapp-core-hub`; `RAPPCOREHUB-WP-0002-T05` | Stabilization closes and rollback is scaled down through the package | +| `hub_runtime` ownership, grants, and lease behavior | `rapp-postgres`; `RAPP-POSTGRES-WP-0004-T03` | Rotation evidence is closed or remains explicitly owned there | +| OpenBao/ESO runtime and migration projections | `railiance-platform`; `RAILIANCE-WP-0023-T02` | Rotation evidence is closed or remains explicitly owned there | +| Consumer evidence and hub-port alignment | `activity-core`; `ACTIVITY-WP-0029` | Resolver/evidence path passes on hub-core | +| State Hub strangler, remaining development coordination, and retirement | `state-hub`; `STATE-WP-0079` | No State Hub work is silently absorbed into this archive | +| Registrar/workstation coupling that prevents authoritative UUID allocation | `state-hub`; `STATE-WP-0081` | Existing missing-UUID warnings remain owned outside this repo | +| ops-hub compatibility probe | `ops-hub`; `make interhub-gate` | Public gate passes at closeout | + +The broader intent gaps recorded in `STATE.md`—progress/work projections and +durable decision, deployment, and outcome resources—are future hub-core and +State Hub retirement scope. They are not unfinished Core Hub migration code +and must not be restarted here. + +## Current immutable anchors + +- Public hub-core source/image: `055cf49` / + `sha256:adf580d09a4a9139b1663c41d59d46903a007e03a3ae567e09b8a6ec23708ab8` +- Core Hub rollback image: + `sha256:388a94ec752e13a0031e329a45e970480c41691e969d4e40c31924669e8d0005` +- Deployment package: chart 0.4.0, current Helm revision 20 +- Rehearsed rollback revision: 19 +- Data and rollback evidence: + `docs/evidence/core-hub-hub-core-cutover-2026-08-21.md` + +The final repository revision is intentionally filled in only by the closeout +commit immediately before Forgejo is made read-only. + +## Closeout checklist + +Do not check the time- or approval-dependent items early. + +- [ ] Current time is no earlier than `2026-08-28T20:49:50+02:00` +- [ ] No unresolved stabilization anomaly or rollback occurred after final promotion +- [ ] `make test` passes +- [ ] `make stabilization-check` passes +- [ ] activity-core public resolver/evidence gate passes +- [ ] Counts, identity sets, and canonical hashes remain explained and matching +- [ ] Credential rotation tasks are done or retain explicit external owners +- [ ] State Hub inbox contains no unhandled Core Hub retirement message +- [ ] Final Core Hub Git revision and evidence links are recorded +- [ ] Operator approves rollback retirement and read-only archive + +## Ordered closeout + +1. Run the checklist and append dated results to the cutover evidence record. +2. Record the operator decision and completion progress in State Hub against + `CORE-WP-0010-T05`. +3. Through `rapp-core-hub`, scale the Core Hub rollback workload to zero while + retaining its immutable image, Helm history, schema, and recovery record. +4. Re-run the public consumer and deployment-package gates against hub-core. +5. Set `CORE-WP-0010-T05` to `done`, the workplan to `finished`, and update + `STATE.md`/`README.md` to point all active development to hub-core. +6. Run `statehub fix-consistency`, commit, and push the final revision. +7. Route the Forgejo admin credential through `warden route` and set + `coulomb/core-hub` read-only/archived. Do not expose the credential. +8. Verify anonymous clone/history remains available and pushes are refused. + +If any closeout gate fails, leave the repository writable for evidence fixes, +keep the rollback workload Ready with no writers, and record the failed gate +and owner. Do not improvise concurrent writers or destructive cleanup. diff --git a/docs/deployment/stabilization-signoff.md b/docs/deployment/stabilization-signoff.md index 764694e..56ff580 100644 --- a/docs/deployment/stabilization-signoff.md +++ b/docs/deployment/stabilization-signoff.md @@ -1,65 +1,64 @@ -# Core Hub production stabilization +# hub-core production stabilization -Post-cutover checks for `CORE-WP-0007-T02` operator sign-off after -`CORE-WP-0005` moved `hub.coulomb.social` to Core Hub (2026-07-03). +This is the stabilization and closeout gate for `CORE-WP-0010-T05` after +hub-core replaced Core Hub as the public production runtime. ## Window -- **Start:** 2026-07-03 (production cutover) -- **End:** 2026-07-10T17:35:00Z (closeout job) -- **Daily job:** 09:00 Europe/Berlin via activity-core +- Start: `2026-08-21T20:49:50+02:00` +- Earliest closeout: `2026-08-28T20:49:50+02:00` +- Public endpoint: `https://hub.coulomb.social` +- Production package: `rapp-core-hub` chart 0.4.0, Helm revision 20 at start +- Rollback: Core Hub remains Ready with no writer groups -## Automated checks +The July Core Hub/Inter-Hub activities in `activity-definitions/` are retained +as historical evidence. They must not be treated as the schedule for this +window. Current fleet-side observation is owned by activity-core under +`ACTIVITY-WP-0029`; this repository's manual gate is authoritative for archive +closeout until that projection is confirmed. -The `core-hub` context resolver query `stabilization_check` verifies: +## Daily/manual gate -| Check | Expected | -| --- | --- | -| `GET /healthz` | 200 | -| `GET /readyz` | 200 | -| `GET /api/v2/widget-types` | 200, count ≥ 26 | -| `GET /api/v2/hubs` (unauthenticated) | 401 | - -State Hub progress events: - -- `core_hub_stabilization_check` — daily while the window is open -- `core_hub_stabilization_closeout` — one-shot at window end - -Activity definitions live in `activity-definitions/` and are projected into -`activity-core/k8s/railiance/20-runtime.yaml`. - -## Manual companion script +Run from this repository: ```bash -CORE_HUB_BASE_URL=https://hub.coulomb.social \ -STATE_HUB_URL=http://127.0.0.1:8000 \ -./scripts/core_hub_stabilization_check.sh +make stabilization-check ``` -Optional full bootstrap smoke (needs approved operator token): +The command fails closed unless all of the following pass: -```bash -CORE_HUB_BASE_URL=https://hub.coulomb.social \ -CORE_HUB_OPERATOR_TOKEN_FILE=/secure/path/to/token \ -make deployed-smoke -``` +- public health reports runtime version 0.2.0; +- readiness identifies hub-core, production PostgreSQL, database and + authorization health, all four candidate writer groups, and no legacy + writer group; +- the public widget-type contract has at least 26 entries; +- an unauthenticated protected request returns 401; +- the ops-hub public consumer gate passes; and +- the deployment package verifies both immutable workloads, routing, writer + ownership, credential metadata, policies, and authentication. -## Operator sign-off for T02 +The protected deployed smoke is optional because it needs an approved token +file. If it is used, the value must remain outside Git, State Hub, logs, and +chat. -Sign-off is a human decision recorded in State Hub after: +## Closeout gate -1. Daily checks (or closeout) show `overall_pass=true` through the window -2. No rollback was invoked since cutover -3. Operator approves retiring the Inter-Hub k8s rollback deployment +After the earliest closeout time, require all of these before archive: -Post a progress note or decision referencing workstream -`a8d66822-e435-4b1e-ad81-37a298d1795e` / task `16eb7ce3-b574-4667-a189-c14ff5d0502b`, -then proceed with `CORE-WP-0007-T02`. +1. The manual stabilization gate passes. +2. The activity-core resolver/evidence path passes against the public host. +3. No unexplained data, contract, authorization, or writer-ownership drift is + open. +4. The final source/target comparison or retained reverse-export evidence has + matching counts, identities, and canonical hashes. +5. The runtime/database credential rotation observations in + `RAPP-POSTGRES-WP-0004-T03` and `RAILIANCE-WP-0023-T02` are resolved or + carry an explicit live residual owner. +6. The operator approves rollback retirement and repository archive. -## Deploy / enable +Record the decision and final progress against workplan +`f5a90aa2-e63f-4f94-b052-8be0d0ee55ea`, task +`16525001-f85e-4dae-9dc4-d36cb1eb4fbe`. -1. Rebuild and deploy `activity-core` on railiance01 with the new `core-hub` - context resolver. -2. Apply the updated `activity-core/k8s/railiance/20-runtime.yaml` ConfigMap - projection for the two new ActivityDefinitions. -3. Confirm events appear at `GET /progress/?event_type=core_hub_stabilization_check`. \ No newline at end of file +The complete procedure and residual ownership map are in +`docs/deployment/archive-closeout.md`. diff --git a/scripts/core_hub_stabilization_check.sh b/scripts/core_hub_stabilization_check.sh index 3d3321e..963eb3d 100755 --- a/scripts/core_hub_stabilization_check.sh +++ b/scripts/core_hub_stabilization_check.sh @@ -1,28 +1,59 @@ #!/usr/bin/env bash -# Core Hub production stabilization snapshot for CORE-WP-0007-T02. -# Complements the scheduled activity-core job (core_hub_stabilization_check). +# hub-core production stabilization gate for CORE-WP-0010-T05. set -euo pipefail BASE_URL="${CORE_HUB_BASE_URL:-https://hub.coulomb.social}" STATE_HUB_URL="${STATE_HUB_URL:-http://127.0.0.1:8000}" +OPS_HUB_ROOT="${OPS_HUB_ROOT:-/home/worsch/ops-hub}" +RAPP_CORE_HUB_ROOT="${RAPP_CORE_HUB_ROOT:-/home/worsch/rapp-core-hub}" +EXPECTED_RUNTIME_VERSION="${EXPECTED_RUNTIME_VERSION:-0.2.0}" +export EXPECTED_RUNTIME_VERSION + +echo "=== hub-core public authority (${BASE_URL}) ===" +curl -fsS "${BASE_URL}/healthz" | python3 -c ' +import json, os, sys +payload = json.load(sys.stdin) +assert payload.get("status") == "ok", payload +assert payload.get("version") == os.environ["EXPECTED_RUNTIME_VERSION"], payload +' +echo "/healthz -> ok (version ${EXPECTED_RUNTIME_VERSION})" + +curl -fsS "${BASE_URL}/readyz" | python3 -c ' +import json, sys +payload = json.load(sys.stdin) +checks = payload.get("checks") or {} +assert payload.get("service") == "hub-core", payload +assert payload.get("status") == "ok", payload +assert checks.get("environment") == "production", checks +assert checks.get("configured_backend") == "postgresql", checks +assert checks.get("active_backend") == "postgresql", checks +assert checks.get("database") == "ok", checks +assert checks.get("authorization") == "ok", checks +assert checks.get("legacy_write_groups") == "none", checks +assert set(filter(None, checks.get("v2_write_groups", "").split(","))) == { + "credentials", "deferred", "interaction", "registry" +}, checks +assert {"credentials", "deferred", "interaction", "operator", "registry", "system"} <= set( + filter(None, checks.get("v2_groups", "").split(",")) +), checks +' +echo "/readyz -> ok (PostgreSQL/auth healthy; hub-core is sole writer)" -echo "=== Core Hub public surface (${BASE_URL}) ===" -for path in /healthz /readyz /api/v2/widget-types; do - code=$(curl -sS -o /dev/null -w '%{http_code}' "${BASE_URL}${path}") - echo "${path} -> ${code}" -done hubs_code=$(curl -sS -o /dev/null -w '%{http_code}' "${BASE_URL}/api/v2/hubs") echo "/api/v2/hubs -> ${hubs_code} (expect 401 hardened contract)" -widget_count=$(curl -fsS "${BASE_URL}/api/v2/widget-types" | python3 -c " -import json,sys -payload=json.load(sys.stdin) -items=payload if isinstance(payload,list) else payload.get('data',[]) +test "${hubs_code}" = "401" + +widget_count=$(curl -fsS "${BASE_URL}/api/v2/widget-types" | python3 -c ' +import json, sys +payload = json.load(sys.stdin) +items = payload if isinstance(payload, list) else payload.get("data", []) +assert len(items) >= 26, len(items) print(len(items)) -") +') echo "widget_types count=${widget_count} (expect >= 26)" echo -echo "=== optional deployed smoke (needs approved token) ===" +echo "=== optional protected deployed smoke (needs approved token) ===" if [[ -n "${CORE_HUB_OPERATOR_TOKEN_FILE:-}" && -f "${CORE_HUB_OPERATOR_TOKEN_FILE}" ]]; then make deployed-smoke else @@ -30,12 +61,14 @@ else fi echo -echo "=== optional ops-hub gate probe ===" -if command -v make >/dev/null && [[ -d "${HOME}/ops-hub" ]]; then - (cd "${HOME}/ops-hub" && make interhub-gate IHUB_BASE="${BASE_URL}") || true -else - echo "skip: ops-hub checkout unavailable" -fi +echo "=== required ops-hub consumer gate ===" +test -d "${OPS_HUB_ROOT}" +(cd "${OPS_HUB_ROOT}" && make interhub-gate IHUB_BASE="${BASE_URL}") + +echo +echo "=== required deployment-package gate ===" +test -d "${RAPP_CORE_HUB_ROOT}" +(cd "${RAPP_CORE_HUB_ROOT}" && make verify-live) echo echo "=== last 6 core_hub_stabilization_check events ===" @@ -46,4 +79,7 @@ for e in json.load(sys.stdin): probe=d.get('probe') or {} print(e['created_at'][:19], (e.get('summary') or '')[:72], 'pass='+str(probe.get('overall_pass'))) -" 2>/dev/null || echo "no stabilization events yet" \ No newline at end of file +" 2>/dev/null || echo "no stabilization events yet" + +echo +echo "ok: hub-core stabilization gate passed" diff --git a/workplans/CORE-WP-0010-runtime-absorption-and-archive.md b/workplans/CORE-WP-0010-runtime-absorption-and-archive.md index 91333fa..28104c9 100644 --- a/workplans/CORE-WP-0010-runtime-absorption-and-archive.md +++ b/workplans/CORE-WP-0010-runtime-absorption-and-archive.md @@ -143,7 +143,7 @@ Evidence is recorded in ```task id: CORE-WP-0010-T05 -status: wait +status: progress priority: medium state_hub_task_id: "16525001-f85e-4dae-9dc4-d36cb1eb4fbe" ``` @@ -157,6 +157,14 @@ started at 2026-08-21T20:49:50+02:00 and ends no earlier than repository read-only on Forgejo, and hand any remaining product-intent gaps to live records outside this repository. +Archive preparation started 2026-08-21. `docs/deployment/archive-closeout.md` +now freezes feature development here, records live residual owners, preserves +non-destructive retention boundaries, and defines the ordered final checklist. +`docs/deployment/stabilization-signoff.md` and +`make stabilization-check` define the current fail-closed production gate. +Only elapsed stabilization, final observations, operator approval, rollback +scale-down, and the read-only Forgejo action remain. + ## Acceptance - [x] Absorption inventory complete