86 repos carry forgejo workflows, but the only runner in the estate is forgejo/forgejo-runner on railiance01, registered against FORGEJO_INSTANCE. No runner on CoulombCore. gitea-runner-status.sh, which prompted the check, is legacy: it defaults to RUNNER_HOST=haskelseed and probes INTER_HUB_IMAGE, both already retired. The gitea-era runner lived on haskelseed and died with it on 2026-08-19 without anything breaking — evidence that gitea-based CI is no longer in use. Six independent methods were needed to enumerate the surface, each finding something the previous could not see. That is the lesson for the G-GEN gate: an inventory is only as complete as the number of independent ways you looked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
390 lines
19 KiB
Markdown
390 lines
19 KiB
Markdown
---
|
|
id: SHR-WP-0002
|
|
type: workplan
|
|
title: "Generation 3 serves from the host being decommissioned"
|
|
domain: infotech
|
|
repo: prj-state-hub-retirement
|
|
status: proposed
|
|
owner: unassigned
|
|
topic_slug: state-hub-retirement
|
|
created: "2026-08-20"
|
|
updated: "2026-08-20"
|
|
state_hub_workstream_id: "c94c3ba1-6c31-4989-a2eb-650726add747"
|
|
---
|
|
|
|
# SHR-WP-0002 — Predecessor generations and deployment reality
|
|
|
|
## The problem in one sentence
|
|
|
|
**Core Hub's production runtime — the generation-3 replacement, cut over
|
|
deliberately and correctly in July — serves from `hub.coulomb.social`, which
|
|
resolves to CoulombCore: the host being decommissioned.** No plan in any
|
|
repository accounts for that.
|
|
|
|
## Correction to this workplan's first draft
|
|
|
|
The first draft of `SHR-WP-0002` (2026-08-20, superseded before any task ran)
|
|
claimed generation 2 had "retired itself by attrition". **That was wrong**, and
|
|
the evidence contradicting it is in `core-hub`'s own archive:
|
|
|
|
- `CORE-WP-0005` finished **2026-07-03**: `hub.coulomb.social` ingress serves
|
|
Core Hub; Inter-Hub compatibility, staging import, dual-run smokes, and
|
|
production cutover gates all closed.
|
|
- `CORE-WP-0007` finished **2026-07-08**: Haskell/IHP infrastructure retired,
|
|
the production Inter-Hub repo renamed/archived, `ihp-railiance-probe`
|
|
archived — after *"a short post-cutover stabilization window"* and explicit
|
|
operator approval to retire the Inter-Hub rollback deployment.
|
|
|
|
The `inter-hub` Deployment scaled `0/0` on railiance01 is that **rollback
|
|
deployment**, held at zero exactly as the workplan describes. It is not a
|
|
lapse; it is the designed end state of a careful retirement.
|
|
|
|
The lesson is the opposite of the one first drafted: **generation 2 was retired
|
|
well.** What nobody checked was whether the successor's runtime was standing on
|
|
durable ground.
|
|
|
|
## What was observed, 2026-08-20
|
|
|
|
| Generation | Position | Observed |
|
|
| --- | --- | --- |
|
|
| 1 — `state-hub` | Being retired by this project, under gates | Running; what the estate uses hourly |
|
|
| 2 — `inter-hub` | Retired `CORE-WP-0005`/`0007`, Jul 2026 | Correctly gone. Rollback deployment at `0/0` as designed |
|
|
| 3 — `core-hub` | The replacement, in production since 2026-07-03 | `hub.coulomb.social` → `92.205.130.254` = **CoulombCore**. No core-hub namespace or Deployment on railiance01 |
|
|
| — `hub-core` | The surviving repository (GOAL §1) | A reusable Python package, not a running service |
|
|
|
|
Three consequences, none recorded anywhere:
|
|
|
|
1. **The gen-1 retirement is being planned onto a runtime that has no home.**
|
|
`G-HUB-RUNTIME` and `G-CORE-ABSORB` presume somewhere to absorb traffic into.
|
|
That somewhere is currently a host with a decommission date, and the gate
|
|
model does not represent that date at all.
|
|
2. **`core-hub/SCOPE.md` is stale in a way that hides this.** It still places
|
|
"retiring production Inter-Hub before migration and smoke evidence exists"
|
|
out of scope and lists `/api/v2` compatibility and "cutover planning from
|
|
Inter-Hub" as in scope — work its own archive shows finished in July. A
|
|
reader cannot tell from the live documents that the cutover already happened.
|
|
3. **`inventory/capabilities.yaml` does not mention `inter-hub`.** The migration
|
|
was real, so this is a smaller gap than first thought — but `G-DISP` still
|
|
has no record of where gen-2 capabilities went, and the estate cannot
|
|
currently answer "does anything still provide the unified operator surface?"
|
|
without reading an archived workplan.
|
|
|
|
## Why this belongs to the project and not to a child repo
|
|
|
|
No single repository can see it. `core-hub` knows its own migration plan;
|
|
`llm-connect` knows its consumers; `ops-warden` knows its access lanes. Only this
|
|
project holds the cross-repository authority, the disposition inventory, and the
|
|
gates — and `G-DISP` ("full disposition inventory"), `G-CORE-ABSORB` and
|
|
`G-HUB-RUNTIME` are all currently gated on assumptions that the observations
|
|
above contradict.
|
|
|
|
Per `SCOPE.md`, this workplan **decides and inventories; it does not implement**.
|
|
Every implementation task it identifies is routed to the owning repository by
|
|
identifier.
|
|
|
|
## Tasks
|
|
|
|
```task
|
|
id: SHR-WP-0002-T01
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "7bf75488-59c9-47ab-a2ff-6ae459dd1444"
|
|
```
|
|
|
|
**Done 2026-08-20 — there was a proper cutover.** `CORE-WP-0005` closed the
|
|
production cutover gates on 2026-07-03; `CORE-WP-0007` retired the Haskell/IHP
|
|
infrastructure by 2026-07-08 with operator approval and a stabilization window.
|
|
The `0/0` rollback deployment is the designed end state.
|
|
|
|
Remaining sub-task: record this in `DECISIONS.md` **with the correction**, so
|
|
the project's own record shows that the first reading was wrong and why. A
|
|
retirement project that misreads a completed retirement as an attrition should
|
|
keep the evidence trail for the next reader.
|
|
|
|
```task
|
|
id: SHR-WP-0002-T02
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "d2dbe425-345f-46cf-af70-0ca4b69a336e"
|
|
```
|
|
|
|
**Extend the disposition inventory backwards to generation 2.** Smaller than
|
|
first thought — the migration was real — but still unrecorded here. `G-DISP` demands
|
|
that every capability reach an explicit destination or retirement decision.
|
|
Applied only to State Hub, it lets gen-2 capabilities vanish unrecorded.
|
|
|
|
Produce `inventory/inter-hub-disposition.yaml` in the shape of the existing
|
|
capability rollup: for each capability gen 2 was to provide — domain hubs, shared
|
|
manifests, widgets, registries, events, unified operator surface — record whether
|
|
it is **served today** (and by what), **planned** (and by whom, under which
|
|
workplan), or **dropped** (and by whose decision).
|
|
|
|
The expected output is uncomfortable and worth having: a list of things the
|
|
estate intended to have, does not have, and has not decided to do without.
|
|
|
|
```task
|
|
id: SHR-WP-0002-T03
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "3254b7d3-e319-44a6-ae63-23b0d080247a"
|
|
```
|
|
|
|
**This is now the workplan's centre of gravity: get the gen-3 runtime off
|
|
CoulombCore.** `G-HUB-RUNTIME` and `G-CORE-ABSORB` presume a runtime to absorb
|
|
traffic into. That runtime is `hub.coulomb.social` on CoulombCore — production
|
|
since 2026-07-03 — plus a `core-hub-staging` tunnel to the same host. There is
|
|
no core-hub presence on railiance01, and `hub-core` is a package, not a service.
|
|
|
|
Decide and record: does core-hub's runtime move to railiance01 first, or does
|
|
consolidation into `hub-core` (GOAL §1) happen directly, skipping a migration to
|
|
a host that is itself scheduled to disappear? Sequence it explicitly **against
|
|
the CoulombCore decommission date**, which is the real constraint and is not
|
|
currently represented in the gate model.
|
|
|
|
Precedent worth reusing: `issue-core` was migrated off CoulombCore on 2026-08-19
|
|
via `ISSUE-WP-0007` and a `rapp-issue-core` package. That is the pattern, and it
|
|
is one week old.
|
|
|
|
**Routed to `core-hub` 2026-08-20; waiting on their answer.** The vehicle is
|
|
`CORE-WP-0010` (runtime absorption into hub-core, `proposed`, all tasks `todo`),
|
|
which depends on `HUB-WP-0004` — itself `proposed`, and still carrying an open
|
|
decision about whether `hub-core` stays an importable library or becomes a
|
|
library plus a permanent thin host.
|
|
|
|
**So the chain that would move production off CoulombCore bottoms out in an
|
|
unanswered architecture question.** That is a sequencing problem rather than an
|
|
implementation one, which is why it sits here and not in a child repo — but the
|
|
answer is core-hub's.
|
|
|
|
Two shapes were put to them:
|
|
|
|
- **(a) Interim move** — package Core Hub as-is onto railiance01, absorb into
|
|
hub-core later on a calm schedule. Costs a migration that would otherwise not
|
|
happen; buys independence from the decommission date.
|
|
- **(b) Absorb directly** — skip the interim host and let `CORE-WP-0010` /
|
|
`HUB-WP-0004` be the migration. Cheaper in total work; couples production
|
|
continuity to finishing an architecture decision under a hard external
|
|
deadline.
|
|
|
|
The project does not choose between them, but records the reasoning either way.
|
|
The one position taken: **leaving it implicit is not acceptable**, because the
|
|
decommission date is a real constraint currently represented in no plan —
|
|
including this project's own gate model, which `T06` addresses.
|
|
|
|
Also asked: whether the decommission date changes `HUB-WP-0004`'s open
|
|
library-vs-thin-host decision. **Pinned 2026-08-20 by the operator: CoulombCore retires by 2026-08-31.**
|
|
Eleven days. See `DECISIONS.md`. On that constraint the project recommends the
|
|
interim move — an architecture decision plus a production migration inside
|
|
eleven days is not a plan. The choice remains core-hub's.
|
|
|
|
```task
|
|
id: SHR-WP-0002-T04
|
|
status: todo
|
|
priority: medium
|
|
state_hub_task_id: "b2196340-fa22-4531-81a2-a34bc217d66c"
|
|
```
|
|
|
|
**Reconcile stated scope against reality across participating repositories.**
|
|
The lineage confusion is visible in the documents: `core-hub/SCOPE.md` guards a
|
|
predecessor that is not running; `llm-connect/SCOPE.md` names a consumer
|
|
relationship that cannot exist; `ops-hub` describes itself as an extension *for
|
|
Core Hub*, a repository this project intends to retire into `hub-core`.
|
|
|
|
Produce a table of every participating repository's stated position versus
|
|
observed reality, and route each correction to its owner. Do **not** edit their
|
|
files from here — the project's authority is the ledger, not the prose.
|
|
|
|
```task
|
|
id: SHR-WP-0002-T05
|
|
status: todo
|
|
priority: medium
|
|
state_hub_task_id: "2b97a7c5-f877-4020-ba83-8eb00c3690e6"
|
|
```
|
|
|
|
**Sweep the downstream lanes that outlived their subject.** A retired generation
|
|
leaves access and credential lanes behind, and they do not expire on their own.
|
|
|
|
Known instance: ops-warden's routing catalog carries `inter-hub-bootstrap-ssh` —
|
|
`status: active`, `risk: high`, last reviewed 2026-06-24 — a bootstrap SSH
|
|
envelope for a system that runs nowhere, whose runbook points at "the ops-hub
|
|
production activation lane tracked by `CUST-WP-0049`". ops-warden cannot retire
|
|
it alone: the lane may still serve ops-hub activation independently of inter-hub's
|
|
runtime, and only this project can see both sides.
|
|
|
|
Ask each participating repository for lanes, tunnels, credentials and scheduled
|
|
jobs whose subject is a retired or lapsed generation. Route retirements to owners;
|
|
record the sweep here so the next generation change has a checklist rather than a
|
|
memory.
|
|
|
|
```task
|
|
id: SHR-WP-0002-T06
|
|
status: todo
|
|
priority: medium
|
|
state_hub_task_id: "9b390f80-1cbb-4485-a958-14b0e24a0b26"
|
|
```
|
|
|
|
**Add a generation-transition gate, so this cannot recur.** The project has gates
|
|
for retiring State Hub deliberately. It has none that would have caught a
|
|
predecessor lapsing, or a successor running only on a host scheduled for
|
|
shutdown.
|
|
|
|
Propose `G-GEN` for `architecture/retirement-gates_v0.1.md`: no generation is
|
|
considered superseded until its capabilities carry explicit dispositions, its
|
|
consumers are reconciled, its downstream lanes are swept, and its successor is
|
|
deployed somewhere that will still exist. Take it through the project's normal
|
|
decision route rather than adding it unilaterally.
|
|
|
|
```task
|
|
id: SHR-WP-0002-T07
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "07636ddf-a26b-4760-8353-2532af9fd938"
|
|
```
|
|
|
|
**The registry dependency, found 2026-08-20 and not previously tracked.**
|
|
`gitea.coulomb.social` is a live container registry on CoulombCore, and two
|
|
railiance01 workloads pull images from it:
|
|
`reuse/Deployment/reuse-surface` and `state-hub/Job/state-hub-alembic-init`
|
|
(checked across Deployments, StatefulSets, DaemonSets, Jobs and CronJobs — that
|
|
is the complete set).
|
|
|
|
Nothing breaks on decommission day, because running pods already hold their
|
|
images. It breaks at the next restart, reschedule or scale-up, as
|
|
`ImagePullBackOff`.
|
|
|
|
**The two are not equally urgent** (corrected 2026-08-20 after checking each
|
|
reference's liveness):
|
|
|
|
- **`reuse-surface` — live and stale.** A running Deployment on the old
|
|
registry, pinned to a 2026-07-07 commit and **22 commits behind** main. Its CI
|
|
switched to forgejo three hours after that commit and has published there ever
|
|
since; the deployment was never repointed. **This is the one that must move
|
|
before 2026-08-31.**
|
|
- **`state-hub` — mostly a false alarm.** The Deployment is already on
|
|
`forgejo.coulomb.social/coulomb/state-hub:main-d8808bf`. The gitea reference is
|
|
a *completed* one-shot Job that will not re-run by itself; the residual risk is
|
|
a chart template recreating it. Worth cleaning, not urgent.
|
|
|
|
There was **no repository rename**. This is a registry migration (gitea →
|
|
forgejo, early July) in which producers moved and some consumers did not.
|
|
|
|
### Full sweep, 2026-08-20 — CoulombCore runs *two* registry services
|
|
|
|
The first pass looked at container images. `gitea.coulomb.social` also serves a
|
|
**PyPI package index** (`/api/packages/coulomb/pypi`), confirmed by
|
|
`railiance-fabric/fabric/interfaces/railiance-forge-python-package-index.yaml`.
|
|
Both die on 2026-08-31.
|
|
|
|
Sweeping every repo for `gitea.coulomb.social` in operational files (yaml, sh,
|
|
py, Makefile, Dockerfile, toml, json) and classifying by whether anything can
|
|
still act on it:
|
|
|
|
**Live, must move before 2026-08-31**
|
|
|
|
| Reference | What breaks |
|
|
| --- | --- |
|
|
| `reuse-surface` Deployment image | Next restart → `ImagePullBackOff`. Also 22 commits stale |
|
|
| `kaizen-agentic` Makefile + README + 5 docs | **Never migrated at all.** Publishes to gitea PyPI and instructs users to `pip install --extra-index-url …gitea…`. No forgejo equivalent exists in the repo |
|
|
|
|
**Live but already dual-pathed — retire the old half**
|
|
|
|
| Reference | Note |
|
|
| --- | --- |
|
|
| `issue-core/Makefile:240` publishes to gitea PyPI | Line 250 already publishes to forgejo. The gitea target is legacy and should go |
|
|
|
|
**Owned by the thing being decommissioned — expected**
|
|
|
|
`railiance-forge` (`manifests/gitea-ingress.yaml`, `helm/gitea-registry-values.yaml`,
|
|
`tools/gitea-runner-status.sh`) owns gitea itself; these retire *with* it.
|
|
|
|
**Not live — leave alone**
|
|
|
|
Tests asserting historical facts (`railiance-fabric`, `markitect-main`,
|
|
`reuse-surface/tests/test_forge_host.py`, `railiance-platform/tests/…`),
|
|
inventory snapshots and asset/data registers (`disaster-control`,
|
|
`domain-tree`, `railiance-fabric` snapshots), agent session blobs, and
|
|
`issue-core/Dockerfile:5` (a comment). These *record* that gitea existed, which
|
|
stays true after it is switched off. Rewriting them would destroy history to
|
|
tidy a grep.
|
|
|
|
**One to check, not ours:** `railiance-platform/argocd/bootstrap/01-railiance-tenants-project.yaml:14`
|
|
permits `sourceRepos: https://gitea.coulomb.social/coulomb/*.git`. railiance01
|
|
has no ArgoCD `applications` resource type, so this appears inert — but it is a
|
|
bootstrap file and should be confirmed rather than assumed.
|
|
|
|
### Credential-lane sweep, 2026-08-20 — a *third* registry service
|
|
|
|
Checking ops-warden's routing catalog found what the file sweep could not: the
|
|
host serves **three** package services, not two.
|
|
|
|
`gitea.coulomb.social/api/packages/coulomb/npm/` is an **npm registry**, and
|
|
ops-warden's catalog lane `whynot-design-npm-publish` — `risk: high`,
|
|
production-exercised (WP-0018 published `@whynot/design@0.4.0` through it) —
|
|
vends the `NPM_AUTH_TOKEN` that publishes to it.
|
|
|
|
So the complete CoulombCore package surface is:
|
|
|
|
| Service | Endpoint | Known consumers |
|
|
| --- | --- | --- |
|
|
| OCI container registry | `gitea.coulomb.social/coulomb/…` | `reuse-surface` (live Deployment) |
|
|
| PyPI index | `…/api/packages/coulomb/pypi` | `kaizen-agentic` (never migrated), `issue-core` (legacy half) |
|
|
| **npm registry** | `…/api/packages/coulomb/npm/` | `@whynot/design` via ops-warden lane `whynot-design-npm-publish` |
|
|
|
|
**Consequence for the credential lane:** after 2026-08-31 that lane routes to a
|
|
registry that does not exist. It does not fail safe — an operator following it
|
|
gets a token for a dead endpoint and debugs the token. `forgejo-admin-api-token`
|
|
already exists as the forgejo-side equivalent and its keywords include
|
|
`forgejo-npm`, so the destination is plausibly in place; ops-warden must confirm
|
|
rather than assume, and cannot repoint the lane before the packages are on
|
|
forgejo — the same publish-before-repoint rule as `KAIZEN-WP-0010`.
|
|
|
|
Routed to ops-warden (lane owner for the catalog entry; `railiance-platform`
|
|
owns the credential itself).
|
|
|
|
### CI runner sweep, 2026-08-20 — clear, and the sweep is now closed
|
|
|
|
86 repositories carry `.forgejo/workflows`, with jobs on `self-hosted` (83),
|
|
`container-build` (11), `ubuntu-latest` (87) and `docker` (1). Those labels
|
|
resolve to registered runners, so the question was where the runners live.
|
|
|
|
**The only runner in the estate is on railiance01**: `forgejo/forgejo-runner`,
|
|
a Deployment up 48 days, registering against `${FORGEJO_INSTANCE}` — forgejo,
|
|
not gitea. No runner exists on CoulombCore.
|
|
|
|
`railiance-forge/tools/gitea-runner-status.sh`, which prompted this check, is a
|
|
**legacy artifact**: it defaults to `RUNNER_HOST=haskelseed` and probes
|
|
`INTER_HUB_IMAGE`. Both are already retired — haskelseed's bridge on 2026-08-19,
|
|
inter-hub in July. So the gitea-era runner lived on haskelseed and died with it a
|
|
day ago, and nothing broke: evidence in itself that gitea-based CI is no longer
|
|
in use.
|
|
|
|
**CoulombCore's dependency surface is therefore fully enumerated** across four
|
|
methods — tunnels, service DNS, workload image references, operational-file
|
|
grep, credential-lane catalog, and CI runners. Each method found something the
|
|
previous one structurally could not see, which is the finding `T06`'s `G-GEN`
|
|
gate should encode: **an inventory is only as complete as the number of
|
|
independent ways you looked.**
|
|
|
|
`forgejo.coulomb.social` is already on railiance01, so the work is retag, push,
|
|
update manifest. Routed to `railiance-platform`; ownership of the `reuse-surface`
|
|
manifest sits with that repo.
|
|
|
|
**Why this is a project task and not a platform ticket:** it was missed because
|
|
the decommission inventory was assembled from tunnels and workplans. A host is
|
|
not free of dependents because nothing tunnels to it. Feed that into `T06`'s
|
|
`G-GEN` gate — a generation or host transition must enumerate what *pulls,
|
|
resolves and authenticates* against the thing being switched off, not only what
|
|
connects to it.
|
|
|
|
## Related
|
|
|
|
- `SHR-WP-0001` — foundation and architecture baseline (finished)
|
|
- `architecture/retirement-gates_v0.1.md` — `G-DISP`, `G-CORE-ABSORB`, `G-HUB-RUNTIME`
|
|
- `inventory/capabilities.yaml` — the gen-1 rollup this extends
|
|
- `core-hub/INTENT.md` — the stated lineage
|
|
- `core-hub` `CORE-WP-0005` / archived `CORE-WP-0007` — the July cutover and
|
|
Haskell retirement, the evidence that corrected this workplan's first draft
|
|
- `core-hub` `CORE-WP-0010` — runtime absorption into `hub-core`, all tasks
|
|
`todo`; the likely vehicle for T03
|
|
- `issue-core` `ISSUE-WP-0007` — the CoulombCore migration precedent
|
|
- ops-warden `inter-hub-bootstrap-ssh` — a lane outliving its subject
|