repo-manager/history/2026-08-09-state-hub-repo-tooling-research.md

Research: How State Hub interacts with repositories

Recorded: 2026-08-09
Audience: Repo Manager + State Hub retirement
Sources: state-hub checkout (API, scripts, MCP, Makefile, docs), activity-core consumers, prior RM extraction inventory

---

1. Executive summary

State Hub is the live index and agent coordination surface for fleet repositories. It does not own Git truth. It:

1. Registers repos (slug, domain, classification, host paths, remotes).
2. Resolves a checkout on the current machine via host_paths / local_path.
3. Parses file-backed work records (workplans, tasks, and related kinds).
4. Reconciles files ↔ Postgres with ADR-001 (“file wins”) via consistency_check / statehub fix-consistency.
5. Mutates files for selected status writebacks (API reconciliation path + fix-consistency writeback), then git commit + push-seal.
6. Serves agents (MCP/REST) and automation (activity-core sweep, SBOM, DoI).

Repo Manager’s job is to take over (1)–(5) as the repository boundary while hub-core keeps messaging, progress, and cross-domain projections.

---

2. Interaction model (as-built)

  Agents (Claude/Codex)          activity-core            Operators
         │                            │                        │
         │ MCP / REST                 │ GET /repos/            │ make / CLI
         │ update_task_status         │ POST sweep             │ register-path
         ▼                            ▼                        ▼
                    ┌─────────────────────────────────┐
                    │           State Hub API         │
                    │  /repos  /workplans  /tasks     │
                    │  /reconciliation  /consistency  │
                    │  /progress  /messages  …        │
                    └───────────────┬─────────────────┘
                                    │ Postgres (index)
                    ┌───────────────┴─────────────────┐
                    │  resolve_repo_path(host_paths)  │
                    │  parse workplans/*.md           │
                    │  writeback + git commit/push    │
                    └───────────────┬─────────────────┘
                                    ▼
                         Git checkouts on hosts
                    (workstation, railiance01, …)

Authority: repository files + Git history.
Hub DB: rebuildable cache/index.
Operator config: host_paths (not in Git).

---

3. Registration and classification tooling

Tool	Entry	What it does
REST POST/GET/PATCH /repos/	API	CRUD managed_repos
POST /repos/{slug}/paths	API / make register-path	Set host_paths[hostname]=path
POST /repos/onboard	API	Onboarding helper
scripts/register_from_classification.py	make register-from-classification	Upsert from .repo-classification.yaml
statehub register / register-project	CLI / make	Scaffold INTENT/AGENTS, register, optional seed WP
list_domain_repos / list_repos_by_classification	MCP	Discovery
register_repo / register_repo_from_classification / update_repo_path	MCP	Agent registration

Classification spine: committed .repo-classification.yaml (category, domain, tags, business fields). Canon allowed vocab in the-custodian. Domain FK on managed_repos derived from primary domain.

Multi-host: each machine registers its checkout path. Consistency uses host_paths[current_hostname] then falls back to local_path.

Pitfall observed: statehub register seeds a bootstrap workplan with a generic REPO-WP-0001 style id; if a real WP already uses that prefix, id collision occurs (seen on repo-manager). Prefer classification register + hand-authored workplans for mature repos.

---

4. Work-record loop (files ↔ index)

4.1 File conventions

• workplans/<PREFIX>-WP-NNNN-*.md with YAML frontmatter + ````task` blocks.
• Hub UUIDs written back as state_hub_workstream_id / state_hub_task_id.
• Generated WORK-RECORDS.md and .custodian-brief.md (fix-consistency).
• Other kinds: intakes/decisions/register entries (C-31/C-32 registration).

4.2 Read path (agents)

Action	Typical path
Orient	get_domain_summary / GET /state/summary
List work	list_workplans / GET /workplans/?…
List tasks	list_tasks / GET /tasks/?workplan_id=
Brief offline	.custodian-brief.md in each repo

4.3 Write path (status)

Action	Path	File effect
MCP update_task_status	REST PATCH task	May go DB-first then consistency writeback
API reconciliation	api/routers/reconciliation.py	Classifies write-through vs deferred; may patch file via workplan_files
statehub fix-consistency --fix	scripts/consistency_check.py	File↔DB drift repair; optional git writeback commit
Push seal	scripts/repo_sync.py	After fix commits: pull-ff gates + push so local≡remote

Git rules (critical):

• C-16 behind remote → skip writes (pull first).
• C-17 ahead + push failed → skip further writes.
• Push-seal: fix runs that create commits must push before return.

---

5. Consistency engine (primary repo tooling)

Script: scripts/consistency_check.py (large monolith).
CLI wrappers: statehub fix-consistency, make check-consistency / make fix-consistency, MCP check_repo_consistency.

Mode	Meaning
--repo SLUG	Single registered repo
--here [PATH]	Infer slug from cwd/remote
--all	All registered repos
--remote --all	Pull then fix (fleet sweep)
--fix	Apply file-wins repairs + writebacks
--no-writeback	Check/report only

Scheduled automation: activity-core (or operator) → POST /consistency/sweep/remote-all → remote-all sweep with wall-clock budget.

C-rule families (repo-relevant):

Group	Examples	Role
Parse/structure	C-01, C-02	workplans/ present and parseable
Binding/drift	C-03–C-06, C-09–C-12, C-15, C-19, C-22	UUID + status/title drift; file wins
Orphans	C-07, C-08, C-14	DB without file / ghost workstreams
Git sync	C-16, C-17	Protect against clobber / open-loop
Classification	C-24	.repo-classification.yaml
Id hygiene	C-26, C-27	prefix + collision
Work-record kinds	C-31–C-33	registry + WORK-RECORDS index
Inbox (not repo)	C-25, C-28, C-29	messages — hub-core territory
Quality soft	C-34	DoR-Ok soft warnings

Parsers also live in api/services/workplan_files.py for in-process API writeback (overlap with consistency_check — dual implementation risk).

---

6. REST surfaces touching repos (selected)

Prefix / route	Purpose
/repos/	Registry list/create
/repos/{slug} PATCH/archive	Metadata / lifecycle
/repos/{slug}/paths	Host path registration
/repos/{slug}/sync	Sync trigger
/repos/{slug}/doi, /doi/summary	Definition of Integrated scoring
/repos/todo-md-staleness	Automation input (activity-core)
/repos/scope-health	SCOPE.md health
/repos/{slug}/dispatch	Agent dispatch orientation
/workplans/, /tasks/	Work index CRUD
/reconciliation	State-change + optional file write-through
/consistency/sweep/remote-all	Fleet reconcile job

Related but not pure repo boundary: progress, messages, suggestions, fabric, token events, service catalog, TPSC, capabilities (mixed hub + repo).

---

7. MCP tooling (repo-facing subset)

Tool	Role
register_repo / register_repo_from_classification	Registration
update_repo_path	Host path
list_domain_repos / list_repos_by_classification	Discovery
check_repo_consistency	Wraps consistency engine
validate_repo_adr	ADR-001 checklist script
ingest_sbom_tool	SBOM ingest for a repo
get_repo_goals / update_repo_goal / get_repo_dispatch	Goals / dispatch
list_workplans / create_workplan / update_workplan*	Work index
list_tasks / update_task_status / bulk status	Task index + mutate
get_domain_summary / get_state_summary	Orientation (includes repo health)

Sanctioned writes vs bootstrap-only are documented in mcp_server/TOOLS.md.

---

8. Adjacent tooling (repo-attached inventories)

Tool	Makefile / script	Notes
SBOM ingest	make ingest-sbom, ingest_sbom.py	Lockfiles + tools yaml
TPSC ingest	ingest_tpsc.py	Service declarations
Capabilities ingest	ingest_capabilities.py	Registry
DoI check	check_doi.py	Integration maturity
ADR validate	validate_repo_adr.py	File layout
Gitea inventory	gitea_inventory.py	Forge listing
Normalize attached WPs	normalize_attached_repo_workplans.py	Fleet hygiene
Edge outbox	api/edge/outbox.py	Offline queue when hub unreachable

These are repo-scoped jobs but not the core ADR-001 consistency loop.

---

9. Downstream consumers

Consumer	Interaction
All agent AGENTS.md templates	Session start brief; end with fix-consistency
activity-core	GET /repos/, todo-md-staleness, SBOM bulk, remote sweep, context resolvers
ops-bridge	Tunnels to API/MCP (:8000 / :18000)
Dashboard	Observable pages over REST
wise-validator / agentic-resources	Progress and legacy field callers (historical)

---

10. Strengths of the established tooling

1. File-first doctrine is operational, not only documented (C-rules + writeback).
2. Multi-host path registry enables workstation + cluster workers on one slug.
3. Push-seal closed the open-loop commit pile-up class of bugs.
4. Classification spine unifies discovery without hard-coding domain folders.
5. MCP + REST + CLI give agents and humans the same coordination surface.
6. Fleet sweep exists for unattended reconcile (activity-core / remote-all).

---

11. Failure modes and technical debt (retirement-relevant)

Issue	Impact
Monolithic consistency_check.py	Hard to own outside state-hub; extract unit is large
Dual parsers (API vs script)	Drift risk between writeback paths
Bootstrap WP id collisions	statehub register unsafe on repos with real WP-0001
Topic spine residual	topic_id still on models; classification is future spine
Inbox C-rules in consistency	Couples messaging to “repo fix”
Index mixed with hub concerns	Progress/messages/catalogs in same service as repo authority
Ghost workstreams from MCP create_*	Bootstrap-only tools still create DB-first risk if misused
Scheduled sweep fragility	Cluster notes: sweeps paused/gaps after cutovers

---

12. Mapping to Repo Manager (current)

State Hub capability	RM status (2026-08-09)
Observe + parse workplans	Partial — rmgr observe/reconcile (JSON index)
Task status writeback + git	Partial — rmgr update-task-status
Full C-rules / push-seal	Not yet — still SH
Registry + host_paths as system of record	Still SH
MCP / FastAPI / Postgres index	Still SH
Dual-run adapter for retirement	Missing — blocks clean strangler

See also: docs/state-hub-extraction-inventory_v0.1.md, docs/observation-command-contracts_v0.1.md.

---

13. Conclusion for retirement sequencing

The retirement-critical path is not a better CLI UX or a prettier model. It is:

1. Make Repo Manager the authoritative executor of reconcile + file writeback for at least one production command path agents already use.
2. Keep State Hub as a strangler facade (MCP/REST) until meters show zero direct need for SH to touch checkouts.
3. Only then move registry/host_paths and drop SH repo mutation.

That sequence is elaborated in specs/ArchitectureBlueprint.md and the next workplan RMGR-WP-0002.