the-custodian/workplans/CUST-WP-0043-state-hub-repo-extraction.md

404 lines
14 KiB
Markdown
Raw Normal View History

2026-05-17 18:48:31 +02:00
---
id: CUST-WP-0043
type: workplan
title: "State Hub Repo Extraction"
domain: infotech
2026-05-17 18:48:31 +02:00
repo: the-custodian
2026-05-17 21:36:40 +02:00
status: completed
2026-05-17 18:48:31 +02:00
owner: custodian
topic_slug: custodian
planning_priority: high
planning_order: 43
created: "2026-05-17"
updated: "2026-05-17"
state_hub_workstream_id: "01fc91e6-7e4a-4eda-8322-e40a0c6c7cf7"
---
# CUST-WP-0043 - State Hub Repo Extraction
## Goal
Untangle `state-hub/` from `the-custodian` and make `/home/worsch/state-hub`
the standalone source repository for the State Hub service before continuing
State Hub implementation work such as `CUST-WP-0042`.
The intended end state is:
- `the-custodian` owns canon, governance, memory, agent runtime scaffolding, and
cross-domain coordination documents.
- `state-hub` owns the live service: FastAPI app, MCP server, migrations,
dashboard, scripts, policies, task-flow engine, tests, and operational docs.
- State Hub itself knows about the new `state-hub` repo path, so future
workplans can live with the service they modify.
- The recently generated workplan `CUST-WP-0042 - Workplan State Model Cleanup`
can be executed against the standalone `state-hub` repository instead of the
embedded directory.
## Context
`/home/worsch/state-hub` currently exists only as a repo-seed stub with a README,
license, `.gitignore`, and Git metadata. The live implementation still resides
under `the-custodian/state-hub/`.
This creates three forms of coupling:
- Source coupling: implementation code is versioned inside the governance repo.
- Operational coupling: Make targets, MCP registration, dashboard paths, and
consistency sync assume `state-hub/` is a subdirectory of `the-custodian`.
- Planning coupling: State Hub workplans, including `CUST-WP-0042`, are
currently file-backed from `the-custodian/workplans/` even when they primarily
modify State Hub internals.
This workplan is deliberately narrower than the broader FOS hub extraction in
`CUST-WP-0025`: it extracts the existing State Hub into its own repo first. The
generic `hub-core` extraction and any future `dev-hub` rename remain separate
architecture work.
## Scope
In scope:
- Move the existing State Hub implementation into `/home/worsch/state-hub`.
- Preserve useful Git history where practical.
- Exclude generated or environment-local directories such as `.venv/`,
`dashboard/node_modules/`, `dashboard/dist/`, `__pycache__/`, and
`.pytest_cache/`.
- Register or update the `state-hub` repo in State Hub with the local path.
- Update session guidance, runbooks, Make targets, and MCP/API path references
so agents and humans know where State Hub work now lives.
- Re-home State Hub workplans so future implementation work is file-backed from
the standalone repo.
Out of scope:
- Extracting the generic `hub-core` package.
- Renaming State Hub to Dev Hub.
- Changing the State Hub lifecycle/status model itself; that belongs to
`CUST-WP-0042`.
- Changing production deployment topology beyond path and repo ownership.
## Extraction Boundary Inventory
Inventory captured on 2026-05-17 against
`/home/worsch/the-custodian/state-hub` and target stub
`/home/worsch/state-hub`.
### Move To `state-hub`
These paths are tracked service source or service-owned operations assets and
should become authoritative in `/home/worsch/state-hub`:
| Path | Handling |
|------|----------|
| `.dockerignore` | Move; revisit because current Docker context excludes `tests`, `docs`, and `infra`. |
| `.env.example` | Move; remains the safe config template. |
| `Dockerfile` | Move; update build context references after extraction. |
| `Makefile` | Move; becomes the primary State Hub command surface. |
| `README.md` | Move/merge with standalone repo baseline README. |
| `alembic.ini` | Move. |
| `api/` | Move; FastAPI app, models, schemas, routers, event publisher. |
| `custodian_cli.py` | Move; update any repo-root assumptions. |
| `dashboard/observablehq.config.js` | Move. |
| `dashboard/package.json` | Move. |
| `dashboard/package-lock.json` | Move. |
| `dashboard/src/` | Move; dashboard source and data loaders. |
| `docs/` | Move; State Hub operational and architecture docs. |
| `flows/` | Move; task-flow definitions. |
| `infra/` | Move; local Postgres compose and infra docs. |
| `mcp_server/` | Move; MCP server and tool reference. |
| `migrations/` | Move; Alembic migration environment and versions. |
| `policies/` | Move; repo DoI and workplan DoD policies. |
| `prompts/` | Move; State Hub prompt assets. |
| `pyproject.toml` | Move; Python package/dependency definition. |
| `scripts/` | Move; consistency, registration, SBOM, token, image, and repo-sync tooling. |
| `task_flow_engine/` | Move; embedded flow evaluator package. |
| `tests/` | Move; keep in repo even though Docker image currently excludes it. |
| `uv.lock` | Move; required for reproducible `uv sync`. |
### Keep In `the-custodian`
These remain Custodian-owned after extraction:
| Path | Handling |
|------|----------|
| `canon/`, `memory/`, `runtime/`, root `workplans/` | Stay in `the-custodian`; not part of State Hub service source. |
| Root `AGENTS.md`, `CLAUDE.md`, `.custodian-brief.md`, `SCOPE.md` | Stay, but update to point State Hub code work at `/home/worsch/state-hub`. |
| `workplans/CUST-WP-0043-state-hub-repo-extraction.md` | Stay as the bridge plan until extraction is complete. |
| `workplans/CUST-WP-0042-workplan-state-model-cleanup.md` | Re-home or bridge after standalone repo registration; do not duplicate blindly. |
| `state-hub/` | Remove or replace with a small pointer only after standalone verification passes. |
### Regenerate Locally
These paths are useful locally but should not be copied as source:
| Path | Handling |
|------|----------|
| `.venv/` | Regenerate with `make install` / `uv sync`. |
| `dashboard/node_modules/` | Regenerate with the dashboard package manager install. |
| `dashboard/dist/` | Regenerate via Observable build/preview. |
| `dashboard/src/.observablehq/` | Regenerate as Observable cache. |
| `.pytest_cache/` | Regenerate from test runs. |
| `**/__pycache__/`, `*.pyc` | Regenerate from Python execution. |
### Discard / Do Not Copy
These are local, sensitive, or machine-specific:
| Path | Handling |
|------|----------|
| `.env` | Do not copy; secrets/local config stay local. |
| `.claude/settings.local.json` | Do not copy; machine/user-local Claude settings. |
| `kubectl` | Do not copy unless later proven to be a tracked intentional helper; currently ignored and machine-local. |
### Target Stub State
`/home/worsch/state-hub` currently contains only `.git/`, `.gitignore`,
`LICENSE`, and a repo-seed README. It needs a service-specific baseline before
the implementation move:
- README that identifies State Hub as the live FastAPI/MCP/dashboard service.
- `AGENTS.md` with the State Hub session protocol.
- `SCOPE.md` for repo discovery and capability registration.
- A `workplans/` directory or explicit decision about the repo-local workplan
prefix before re-homing `CUST-WP-0042`.
- A tightened `.gitignore` that keeps `uv.lock`, docs, infra, and tests
trackable while excluding runtime/build/cache/secrets.
2026-05-17 18:48:31 +02:00
## Tasks
### T01 - Inventory Extraction Boundary
```task
id: CUST-WP-0043-T01
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "1d1e1cd6-96ac-4840-9c11-2867ada1116d"
```
Create an inventory of everything under `the-custodian/state-hub/` and classify
each path as:
- move to `state-hub`
- keep in `the-custodian`
- regenerate locally
- discard as generated/cache output
Done when the inventory names the handling for API, MCP server, migrations,
dashboard, infra, scripts, tests, docs, policies, prompts, and task-flow engine.
### T02 - Prepare Standalone Repo Baseline
```task
id: CUST-WP-0043-T02
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "24ad0bf0-e11d-4a16-9c00-0f9cca6d208f"
```
Turn `/home/worsch/state-hub` from repo-seed stub into a proper service repo
baseline before moving code:
- replace the seed README with State Hub-specific orientation
- add `AGENTS.md`/session protocol for State Hub work
- add `SCOPE.md`
- decide the repo-local workplan prefix for future State Hub plans
- ensure `.gitignore` excludes local environments, dependency installs, build
output, caches, and secrets
Done when a fresh agent can open `/home/worsch/state-hub` and understand what
the repo owns before any implementation code is copied.
### T03 - Move Implementation With History-Aware Strategy
```task
id: CUST-WP-0043-T03
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "fe1faa7b-fa19-4d08-b12f-b67bbe5d6774"
```
Move the implementation from `the-custodian/state-hub/` into the standalone
repo using a strategy that preserves useful history where possible. Candidate
approaches:
- `git subtree split` from `the-custodian/state-hub/` into the new repo
- `git filter-repo` into a temporary extraction branch, then merge into the stub
- a clean copy only if history preservation is not worth the migration risk
Do not migrate generated or local-only content:
- `.venv/`
- `dashboard/node_modules/`
- `dashboard/dist/`
- `__pycache__/`
- `.pytest_cache/`
- local `.env` files or credentials
Done when `/home/worsch/state-hub` contains the service implementation and has a
clean Git status other than intentional new extraction commits.
### T04 - Decouple The Custodian Repository
```task
id: CUST-WP-0043-T04
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "92c6ee06-c256-4526-9bc4-6f2b1b0f5d8e"
```
Remove `the-custodian/state-hub/` as an owned implementation tree after the
standalone repo is verified. Leave only intentional integration surface in
`the-custodian`, such as:
- documentation pointing to `/home/worsch/state-hub`
- optional proxy Make targets if they remain useful
- session guidance that says State Hub code work belongs in the standalone repo
Done when there is a single authoritative implementation tree and
`the-custodian` no longer looks like it owns State Hub source code.
### T05 - Register State Hub Repo In State Hub
```task
id: CUST-WP-0043-T05
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "5dd3547c-69b3-4760-83a7-f0783053a572"
```
Connect `/home/worsch/state-hub` to State Hub as a managed repo in the
`custodian` domain.
Use the file-first rule for repo-backed workplans. If new workplan files are
created in the standalone repo, write them before running consistency sync or
registration tooling so no ghost workplans are created.
2026-05-17 18:48:31 +02:00
Done when State Hub repo metadata resolves the local host path for
`/home/worsch/state-hub`, and `fix-consistency --here` or the equivalent repo
sync can discover the repo without path overrides.
### T06 - Re-home State Hub Workplans
```task
id: CUST-WP-0043-T06
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "1c537901-1013-43e0-81c6-9c29ed4f7dd2"
```
Move State Hub implementation workplans to the standalone repo once it is
registered. The first priority is the recently generated
`CUST-WP-0042 - Workplan State Model Cleanup`.
The migration should preserve existing State Hub IDs where appropriate and avoid
creating duplicate workplans. If the consistency engine cannot safely move an
existing file-backed workplan across repos, record the limitation and use the
2026-05-17 18:48:31 +02:00
least confusing transition path:
- keep the old file as an archived pointer, or
- create a new State Hub-local continuation plan that references the old
workplan explicitly
2026-05-17 18:48:31 +02:00
Done when future agents can work on the State Hub state-model cleanup from the
standalone repo without editing `the-custodian/state-hub/`.
### T07 - Update Operational Wiring
```task
id: CUST-WP-0043-T07
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "626faa5e-314b-445b-b86b-f11014473ad5"
```
Update path-sensitive wiring after extraction:
- Make targets and scripts that run API, MCP, migrations, tests, dashboard, or
consistency sync
- MCP registration examples and local config patchers
- dashboard/data path references
- docs and runbooks that mention `the-custodian/state-hub`
- deployment or container build contexts
- NATS/activity-core delegation docs if they name repo-local paths
Done when local development and agent session startup use the standalone repo
path and no longer assume the embedded path.
### T08 - Verification Gate
```task
id: CUST-WP-0043-T08
status: done
2026-05-17 18:48:31 +02:00
priority: high
state_hub_task_id: "4571d9fd-0513-406e-ad4e-284e39f16ac5"
```
Run verification from the standalone repo:
- Python test suite
- Alembic migration check
- API health check
- MCP server smoke test
- dashboard smoke test if frontend dependencies are available
- consistency sync for both affected repos
Done when the standalone repo can run the same service checks that previously
ran from `the-custodian/state-hub/`, and the Custodian dashboard shows the new
repo/workplan state accurately.
### T09 - Handoff To State Hub State-Model Work
```task
id: CUST-WP-0043-T09
status: done
2026-05-17 18:48:31 +02:00
priority: medium
state_hub_task_id: "72b737f8-6caa-4a36-9113-f05c4fab2738"
```
After extraction is verified, resume `CUST-WP-0042` from the standalone
`state-hub` repo. Before implementation starts:
- confirm the workplan paths point at the new repo layout
- confirm `state_hub_workstream_id` and task IDs still refer to the intended
workplan/tasks
2026-05-17 18:48:31 +02:00
- update any acceptance criteria that assumed the embedded path
- record a progress event linking this extraction workplan to the resumed
state-model cleanup work
Done when `CUST-WP-0042` is ready to execute in the extracted repository.
## Build Order
```text
T01 -> T02 -> T03 -> T08
T03 -> T04 -> T07 -> T08
T02 -> T05 -> T06 -> T09
T08 -> T09
```
`CUST-WP-0042` should not receive implementation edits until T06 and T08 are
complete, because its changes should land in the standalone `state-hub` repo.
## Acceptance Criteria
- `/home/worsch/state-hub` is the authoritative State Hub implementation repo.
- `the-custodian` no longer owns a live copy of State Hub implementation code.
- State Hub repo metadata knows the standalone repo path on this host.
- New State Hub workplans can be synced from the standalone repo without ghost
workplans.
2026-05-17 18:48:31 +02:00
- Existing active State Hub work, especially `CUST-WP-0042`, has a clear and
non-duplicating handoff path into the standalone repo.
- Local API, MCP, migration, dashboard, test, and consistency workflows are
verified after extraction.
## Risks And Guardrails
- Do not delete the embedded implementation until the standalone repo passes the
verification gate.
- Do not copy secrets, local virtualenvs, dependency installs, or generated
frontend output into the new repo.
- Do not call `create_workstream()` or `create_task()` manually for migrated
file-backed workplans before the files exist in their final repo.
- Prefer a reversible transition: keep an archived pointer or temporary bridge
doc in `the-custodian` until State Hub work has clearly moved.