diff --git a/INTENT.md b/INTENT.md index 3db2a9f..1b83f8f 100644 --- a/INTENT.md +++ b/INTENT.md @@ -38,36 +38,35 @@ The framework must be usable by humans and agents. Human operators need a stable ## Ecosystem position -`core-hub` is the **production framework layer** in the three-repo hub stack: +`core-hub` is the **previous production service and temporary rollback** in the +three-repo hub stack: | Repo | Role | | --- | --- | | `hub-core` | Surviving framework/runtime — contracts, ports, package, and primary image | | `state-hub` | Dev coordination — file-first workplans, dev MCP | -| `core-hub` | Production service — `/api/v2`, hubs, manifests, operator console | +| `core-hub` | Previous production service retained through cutover stabilization | Canon: `/home/worsch/the-custodian/docs/hub-ecosystem-architecture.md` -**Transition naming:** `core-hub` remains the current production service and -rollback authority while `hub-core` becomes the surviving framework/runtime. +**Transition naming:** `hub-core` is the production service and surviving +framework/runtime. `core-hub` is the tested rollback until archive. ## Current State -Core Hub is the verified production `/api/v2` runtime at -`hub.coulomb.social`, now running on railiance01 through the `rapp-core-hub` -package. The 2026-08-21 relocation preserved the public contract and durable -data, reverified ops-hub and activity-core consumers, and retired the active -CoulombCore runtimes and obsolete tunnel. Haskell Inter-Hub remains retired -(`CORE-WP-0007`). +Hub-core is the verified production `/api/v2` runtime at +`hub.coulomb.social`, running on railiance01 through the `rapp-core-hub` +package. The 2026-08-21 cutover preserved the public contract and exact durable +data, reverified ops-hub and activity-core consumers, enforced one writer per +group, and exercised rollback to Core Hub plus forward recovery. Core Hub is +still deployed with no writer groups through the seven-day stabilization +window; the active CoulombCore runtimes and obsolete tunnel remain retired. +Haskell Inter-Hub remains retired (`CORE-WP-0007`). This delivers the practical Gen3 production reset, but not the whole product -intent. Messaging, progress/work projections, several durable evidence and -decision resources, and a single extension runtime remain outside Core Hub or -only partially represented. `CORE-WP-0010` tracks absorption into hub-core and -repository archive. Hub-core's packaging, contract, port, and conformance -foundations are complete, and the joint route-group migration design is -recorded. Hub-core revision `7e1ec03` publishes that foundation, so S0 is -active: durable PostgreSQL storage, legacy auth/health compatibility, an -immutable image, and an internal candidate deployment come next. The healthy -service remains the production and rollback authority until dual-run consumer -gates prove a replacement. See `STATE.md` for current operational truth. +intent. Fleet development coordination remains in state-hub; progress/work +projections and several durable evidence, decision, deployment, and outcome +resources are still external or only partially represented. `CORE-WP-0010` +now tracks only stabilization, residual-owner handoff, and read-only archive. +Hub-core revision `055cf49` is the production implementation. See `STATE.md` +for current operational truth. diff --git a/STATE.md b/STATE.md index 48f3bcb..0588247 100644 --- a/STATE.md +++ b/STATE.md @@ -5,26 +5,28 @@ ## One-line posture -**Core Hub remains the verified production `/api/v2` authority on railiance01. -Hub-core S0–S5 implementation is published at `8ab1d0c` with immutable image -digest `sha256:2a8b396c5295476d5ce865927b8309a6e054e50eac4e92175ef32cdeaba962fb`; -the next gate is the private production candidate and data import.** +**Hub-core is the verified production `/api/v2` authority on railiance01. +Core Hub remains deployed and healthy with no writer groups as the tested +rollback until the stabilization window closes on 2026-08-28.** ## Production truth | Surface | Current state | | --- | --- | | Public endpoint | `https://hub.coulomb.social` → `92.205.62.239` | -| Runtime | railiance01, Helm revision 4, chart `0.3.2` | +| Runtime | railiance01, Helm revision 20, chart `0.4.0` | | Package | `rapp-core-hub`, verified and bound to `reef-railiance` | -| Image | Forgejo digest `sha256:0db6f7c646c6d4c1fbc844dc266fcf883584637e1c1f47802efe1c6a98558d3c` | -| Database | railiance01 `platform-pg`, database `core_hub`, schema `core_hub` | +| Authoritative image | hub-core `055cf49`, digest `sha256:adf580d09a4a9139b1663c41d59d46903a007e03a3ae567e09b8a6ec23708ab8` | +| Rollback image | Core Hub digest `sha256:388a94ec752e13a0031e329a45e970480c41691e969d4e40c31924669e8d0005` | +| Database | railiance01 `platform-pg`, database `core_hub`, schemas `core_hub` and `hub_runtime` | | TLS | Let's Encrypt certificate Ready through 2026-11-18 | | Consumers | ops-hub gate and activity-core resolver/evidence sink verified | -| Source fallback | CoulombCore production and staging Deployments at zero replicas; source DB read-only | +| Live rollback | Core Hub Deployment Ready, public routes removed, writer set empty | The relocation and retirement evidence is in [`docs/evidence/core-hub-railiance01-stabilization-2026-08-21.md`](docs/evidence/core-hub-railiance01-stabilization-2026-08-21.md). +The hub-core migration and cutover evidence is in +[`docs/evidence/core-hub-hub-core-cutover-2026-08-21.md`](docs/evidence/core-hub-hub-core-cutover-2026-08-21.md). Deployment packaging and rollout truth lives in the `rapp-core-hub` repository. ## Workplans @@ -33,7 +35,7 @@ Deployment packaging and rollout truth lives in the `rapp-core-hub` repository. | --- | --- | --- | | `CORE-WP-0001`–`CORE-WP-0009` | finished / archived | Gen3 runtime, contracts, persistence, consumer gates, and hardening delivered | | `CORE-WP-0011` | finished | Production relocated to railiance01 and active CoulombCore runtime retired | -| `CORE-WP-0010` | active | S0–S5 code done; private candidate/data gates next | +| `CORE-WP-0010` | active | Production cutover done; stabilization/archive gate remains | ### Open work @@ -41,19 +43,21 @@ Deployment packaging and rollout truth lives in the `rapp-core-hub` repository. | --- | --- | --- | | `CORE-WP-0010-T01` inventory | done | Inventory recorded in `docs/specs/runtime-absorption-inventory.md` | | `CORE-WP-0010-T02` dual-run design | done | Route-group plan recorded in `docs/specs/runtime-absorption-plan.md` | -| `CORE-WP-0010-T03` absorption slices | progress | Deploy private candidate, import/compare data, run consumers | -| `CORE-WP-0010-T04` production cutover | wait | Absorption evidence and operator approval | -| `CORE-WP-0010-T05` archive | wait | Cutover complete and residual ownership recorded | +| `CORE-WP-0010-T03` absorption slices | done | Seven-table parity, conformance, restart, and consumer gates passed | +| `CORE-WP-0010-T04` production cutover | done | Hub-core authoritative; rollback and forward recovery exercised | +| `CORE-WP-0010-T05` archive | wait | Seven-day stabilization through 2026-08-28T20:49:50+02:00 | ## Intent position -The operational reset promised by `INTENT.md` is real: the service is small, -containerized, contract-tested, credential-safe, observable, and deployed on -ordinary Kubernetes/PostgreSQL infrastructure. The larger Gen3 product intent -is only partly realized. Core Hub does not yet own the fleet's messaging, -progress/work projections, durable decision/deployment/outcome resources, or a -single extension runtime. Those gaps are not a reason to disturb the healthy -production runtime; they are inputs to the hub-core consolidation decision. +The operational and consolidation goals in `INTENT.md` are now materially +realized: hub-core is the single production runtime, remains contract-tested +and credential-safe, and runs on ordinary Kubernetes/PostgreSQL infrastructure +with a reversible migration. The larger Gen3 product intent is still partial. +Fleet development coordination remains in state-hub, and durable +decision/deployment/outcome resources plus some progress/work projections are +not yet unified behind hub-core. Those residual capabilities need explicit +owners after this repository is archived; they are not reasons to reverse the +healthy production cutover. ## Verify @@ -69,12 +73,11 @@ curl -fsS https://hub.coulomb.social/readyz - `CORE-WP-0011` is finished but has no State Hub UUID; the registrar warning is retained rather than inventing an identifier. - `.custodian-brief.md` is generated and may lag file-backed workplan truth. -- Hub-core runtime/compatibility/migration implementation is published at - `8ab1d0c`; its candidate image digest is recorded above. Public authority - stays here until the private data, conformance, consumer, and rollback gates - pass. +- Hub-core production authority is published at `055cf49`; exact migration, + conformance, consumer, writer-fence, and rollback gates passed. - `make lint` has baseline style findings in unchanged Python files; the full suite passes all 26 tests after adding migration export and write fencing. - The stopped CoulombCore workloads and read-only source database are retained only as controlled fallback evidence. -- Do not archive this repository before `CORE-WP-0010` completes. +- Do not archive this repository before the 2026-08-28 stabilization review + completes `CORE-WP-0010-T05`. diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index dc20df8..beb44c1 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -51,8 +51,8 @@ | task | CORE-WP-0009-T06 | done | — | workplans/CORE-WP-0009-hub-core-adoption-and-integration.md | | task | CORE-WP-0010-T01 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0010-T02 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | -| task | CORE-WP-0010-T03 | progress | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | -| task | CORE-WP-0010-T04 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | +| task | CORE-WP-0010-T03 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | +| task | CORE-WP-0010-T04 | done | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0010-T05 | wait | — | workplans/CORE-WP-0010-runtime-absorption-and-archive.md | | task | CORE-WP-0011-T01 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md | | task | CORE-WP-0011-T02 | done | — | workplans/CORE-WP-0011-railiance01-production-relocation.md | diff --git a/docs/evidence/core-hub-hub-core-cutover-2026-08-21.md b/docs/evidence/core-hub-hub-core-cutover-2026-08-21.md new file mode 100644 index 0000000..edf0fd4 --- /dev/null +++ b/docs/evidence/core-hub-hub-core-cutover-2026-08-21.md @@ -0,0 +1,70 @@ +# Core Hub to hub-core production cutover evidence — 2026-08-21 + +## Outcome + +Hub-core became the public `https://hub.coulomb.social` authority at +2026-08-21T20:49:50+02:00. The current release is Helm revision 20 of chart +`rapp-core-hub` 0.4.0. Core Hub remains Ready with no public route and no +writer groups as the live rollback through the stabilization window. + +## Immutable identities + +- hub-core source revision: `055cf49` +- authoritative hub-core image digest: + `sha256:adf580d09a4a9139b1663c41d59d46903a007e03a3ae567e09b8a6ec23708ab8` +- rollback Core Hub image digest: + `sha256:388a94ec752e13a0031e329a45e970480c41691e969d4e40c31924669e8d0005` +- database: `core_hub` on railiance01 `platform-pg` +- source/target schemas: `core_hub` / `hub_runtime` + +## Data gates + +The full import, idempotent replay, restart-persistence check, delta replay, +and reverse export passed. Source and target had identical record identities +and the following canonical counts and content hashes: + +| Resource | Count | SHA-256 | +| --- | ---: | --- | +| `hubs` | 2 | `1903171468a9ec1e19afc8e6d6c3dccd8ec96abbb2eb7a6d43cb0694a27d1bed` | +| `hubCapabilityManifests` | 2 | `71cc6cf72176558ce656a31f786b81032976355436f4207ef267e079890f0ba5` | +| `apiConsumers` | 3 | `ee909a15b21c514cab56b35219fb86f39ee092c61a2b05e24827b316e5b47710` | +| `apiKeys` | 10 | `a6a5123f410e996f8c40922ab8289ff8a1d91a0324234e25d91197ceabf9f6c0` | +| `widgets` | 16 | `f9b0abfc236e16264b0b02cc22ee680e4a71da58d990db1edae67dfe16a372a1` | +| `interactionEvents` | 4 | `32d617ff66f67d6a9da516b449145c66a92d7f9e0e2ecede3916957f7402213c` | +| `migrationRuns` | 2 | `47318fe8f2fa51c66a8fadf2d548ef1d45d1429c365ea97a50d4620d8e6c9c61` | + +No raw key or credential value was exported or recorded as evidence. + +## Runtime and consumer gates + +- All five route groups (`system`, `registry`, `credentials`, `interaction`, + `deferred`) route to hub-core. +- All four mutable writer groups belong only to hub-core. Core Hub rejects + writes and advertises an empty writer set. +- Hub-core internal conformance passed 8/8. +- `ops-hub` `make interhub-gate` passed against the public host. +- The activity-core stabilization resolver passed all five checks. +- Both Deployments are Ready, dependency-aware readiness reports PostgreSQL + and authentication healthy, and `make verify-live` passed. + +During the registry fence probe, the still-running legacy process briefly +accepted one exact synthetic `must-not-write` hub before its configuration +checksum restart. The identified row was removed after an exact identity/name +check; the restarted legacy runtime then returned 503 and the target/public +dataset remained at the expected two hubs. No user record was affected. + +## Rollback rehearsal + +The final cutover was rolled back with Helm to the pre-cutover configuration, +creating revision 19. Core Hub served healthy public health/readiness checks. +The final source delta replay preserved all counts and hashes. Forward deploy +then created current revision 20, after which image, readiness, writer-fence, +conformance, ops-hub, and activity-core gates all passed again. + +## Stabilization and residual owner + +The seven-day stabilization interval is +2026-08-21T20:49:50+02:00 through no earlier than +2026-08-28T20:49:50+02:00. `CORE-WP-0010-T05` owns the final review, Forgejo +read-only archive, and handoff of any remaining product-intent gaps. Core Hub +must remain available as the no-writer rollback until that task closes. diff --git a/workplans/CORE-WP-0010-runtime-absorption-and-archive.md b/workplans/CORE-WP-0010-runtime-absorption-and-archive.md index d4195ff..91333fa 100644 --- a/workplans/CORE-WP-0010-runtime-absorption-and-archive.md +++ b/workplans/CORE-WP-0010-runtime-absorption-and-archive.md @@ -102,7 +102,7 @@ aligned with hub-core's accepted canonical plan; `HUB-WP-0004-T06` is done. ```task id: CORE-WP-0010-T03 -status: progress +status: done priority: high state_hub_task_id: "c01ddec1-5737-4115-ac73-c402ffb5bdc5" ``` @@ -110,25 +110,20 @@ state_hub_task_id: "c01ddec1-5737-4115-ac73-c402ffb5bdc5" Migrate in slices with consumer smokes (ops-hub, activity-core) after each. Record row counts and fixture replays per slice. -Started 2026-08-21 after hub-core revision `7e1ec03` closed the joint planning -and publication gate. S0 now owns the durable PostgreSQL backend, legacy -auth/health compatibility, immutable image, and internal candidate deployment. -Do not change public traffic. - -Implementation advancement 2026-08-21: hub-core revision `8ab1d0c` adds the -durable PostgreSQL store, audit ledger, all `/api/v2` compatibility groups, -fail-closed writer controls, seven-table idempotent import/export, and reverse -bundle support. Its Forgejo image is pinned at -`sha256:2a8b396c5295476d5ce865927b8309a6e054e50eac4e92175ef32cdeaba962fb`. -This source now exports all seven tables without raw keys and fences writes by -route group. The remaining T03 work is live candidate migration, comparison, -and named consumer evidence. +Completed 2026-08-21. Hub-core revision `055cf49` and immutable image +`sha256:adf580d09a4a9139b1663c41d59d46903a007e03a3ae567e09b8a6ec23708ab8` +provide the durable PostgreSQL store, audit ledger, complete compatibility +surface, fail-closed writer controls, and lossless migration tooling. The live +seven-table import was repeatable and preserved exact counts, identity sets, +and canonical content hashes; restart persistence and reverse export also +matched. Hub-core conformance passed 8/8, and the public ops-hub and +activity-core consumer gates passed before and after rollback recovery. ## Cutover production traffic ```task id: CORE-WP-0010-T04 -status: wait +status: done priority: high state_hub_task_id: "dc414053-a896-4bf0-af05-944b7a7cf750" ``` @@ -136,7 +131,13 @@ state_hub_task_id: "dc414053-a896-4bf0-af05-944b7a7cf750" With operator approval, point production at hub-core runtime; verify consumer smokes and interaction evidence path. -Waiting on completed absorption evidence and explicit operator approval. +Completed 2026-08-21 with operator approval. All five route groups and all +four mutable writer groups now belong to hub-core; Core Hub remains Ready with +an empty writer set as rollback. Helm rollback to the pre-cutover runtime was +exercised at revision 19, followed by successful forward recovery to revision +20 and repeat conformance, consumer, image, readiness, and writer-fence gates. +Evidence is recorded in +`docs/evidence/core-hub-hub-core-cutover-2026-08-21.md`. ## Archive repository @@ -150,12 +151,16 @@ state_hub_task_id: "16525001-f85e-4dae-9dc4-d36cb1eb4fbe" Write completion record (revisions, PRs, residual owners), set read-only archive on forge, and hand residual work as live records outside this repo. -Waiting on production cutover and recorded residual ownership. +Production cutover is complete. Wait through the stabilization interval that +started at 2026-08-21T20:49:50+02:00 and ends no earlier than +2026-08-28T20:49:50+02:00. Then record the final review, archive this +repository read-only on Forgejo, and hand any remaining product-intent gaps to +live records outside this repository. ## Acceptance - [x] Absorption inventory complete - [x] Dual-run design approved/recorded -- [ ] Consumer smokes pass on hub-core host -- [ ] Production traffic cut over with evidence +- [x] Consumer smokes pass on hub-core host +- [x] Production traffic cut over with evidence - [ ] core-hub archived read-only; residuals owned elsewhere