the-custodian/workplans/CUST-WP-0050-repo-classification-registration-redesign.md

389 lines
17 KiB
Markdown
Raw Normal View History

---
id: CUST-WP-0050
type: workplan
title: "Repo Classification & State Hub Registration Redesign"
domain: custodian
repo: the-custodian
status: active
owner: custodian
topic_slug: custodian
planning_priority: high
planning_order: 50
created: "2026-06-22"
updated: "2026-06-22"
started: "2026-06-22"
state_hub_workstream_id: "9f031f48-8de8-48b6-8e69-d2d83ad70a7a"
---
# CUST-WP-0050 - Repo Classification & State Hub Registration Redesign
## Goal
Adopt the **Repo Classification Standard** (`canon/standards/repo-classification-standard_v1.0.md`,
`id: canon-repo-classification`) as the ecosystem-wide model for organising
repositories, and redesign State Hub registration around it so that:
- every known repository carries a committed `.repo-classification.yaml` that is
the **source of truth** for its classification,
- the State Hub can **automatically register all known repos** by reading and
validating those files (local checkout or Gitea API), and
- all **previously registered repos are reclassified** under the new standard,
replacing the current ad-hoc 14-domain model.
End state: one principled, validated taxonomy (category · domain · capability
tags · business stake · business mechanics) spanning the whole portfolio, with
registration that is reproducible from repo-owned metadata rather than
hand-curated DB rows.
## Context
A 2026-06-21 review compared three views of the portfolio and found them out of
sync:
- **Gitea** hosts ~72 repos (70 under `coulomb/`, plus a fork and a personal repo).
- **State Hub** has 57 `managed_repos` across **14 ad-hoc domains** (custodian,
railiance, markitect, coulomb_social, personhood, capabilities, canon,
citation_evidence, helix_forge, inter_hub, netkingdom, stack,
vergabe_teilnahme, whynot).
- **the-custodian** `canon/projects/` froze at the original **6 founding charters**.
Concrete discrepancies to resolve as part of this work:
- ~18 Gitea repos are **unregistered** (e.g. audit-core, binect-chrome, binect-js,
coordination-engine, direkt-vermittlung-de, human-resources, polycode-sim,
ralph-workplan, repo-seed, tegwick-control, tele-mcp, testdrive-jsui,
timeline-svg, vantage-point, whynot-control, whynot-design).
- **Phantom / renamed** registrations: `markitect-project` (registered) vs
`markitect-main` (Gitea) — likely a rename; `railiance-bootstrap` and
`railiance-hosts` registered but absent from Gitea.
- **Duplicate domain**: `vergabe_teilnahme` looks like a second registration of
`vergabe-teilnahme` (already under coulomb_social).
- **Empty domain**: `personhood` has a charter and topic but no repos.
- **Naming drift**: `coulomb.social`/`coulomb_social`, `foerster-capabilities`/`capabilities`.
The new standard fixes the root cause: it separates *category* (work mode),
*domain* (intended market/user), *capability tags* (what it does), and *business
stake* (who cares) — concerns the current 14 "domains" conflate.
### Architecture decision: repo-anchored model, domain derived from classification
The standard's `domain` is a **fixed 14-value market vocabulary** (infotech,
financials, communication, consumer, health, industrials, energy, utilities,
materials, realestate, crypto, agents, space, government) that is *orthogonal* to
the Hub's current 14 coordination domains. Per the steering decision on
2026-06-22, the new market-domain vocabulary **replaces** the Hub's domain model
(rather than augmenting it or running a parallel two-axis model).
The current spine is `Domain → Topic → Workstream`, where `topics.domain_id` and
`workstreams.topic_id` are both **NOT NULL** and the 14 domains are seeded **1:1
with 14 topics** (a data convention — the schema actually allows many topics per
domain, but that has never been used). `workstreams.repo_id` and
`repo_goals.repo_id` already exist, but the *required* anchor is the soft,
hub-only `topic`, while the stable git-managed `repo` link is optional.
Per the 2026-06-22 steering decision, this redesign **flips the polarity**: the
**repo becomes the primary anchor** for workplans, and market-domain is
**derived** from the repo's `.repo-classification.yaml`, not stored as a separate
`topic`/`domain` parent. Concretely:
- `workstreams.repo_id` becomes the **required** anchor; `topic_id` is demoted to
optional (or `topic` is retired) — see T10.
- Market-domain is computed from `repo → classification.domain`; the standalone
`topics.domain_id` / `managed_repos.domain_id` spine is removed.
- `RepoGoal` (already repo-anchored) becomes the goal primitive; `DomainGoal`
becomes a thin strategic rollup keyed by the 14 market domains.
- **Cross-repo workplans** anchor to a dedicated **project repo** that retires to
archive on completion, with results living on in the modified product repos —
see **ADR-005** and Open Questions D1/D1a.
This is consistent with ADR-001: the spine becomes the git-managed repo plus its
committed classification file, so the Hub stays fully rebuildable from repo-owned
files. It is a **breaking migration** of the coordination spine (T04/T05).
## Scope
In scope:
- Promote and steward the standard as custodian canon (done: the standard now
lives at `canon/standards/repo-classification-standard_v1.0.md`).
- A single machine-readable allowed-values source derived from the standard,
consumed by both the per-repo files and the Hub validator.
- A committed `.repo-classification.yaml` for every active repo (agent-assisted
first pass, human-reviewed), authored in each repo.
- State Hub schema/model redesign replacing the domain model with the 14 market
domains and storing the full classification on `managed_repos`.
- A reversible data migration re-homing existing topics/workstreams/goals/
decisions/charters and resolving the discrepancies listed above.
- Auto-registration tooling (bulk, idempotent) that reads classification files
from local checkouts or the Gitea API and registers/reclassifies repos.
- Updates to dashboard, consistency checker, MCP/REST surface, and orientation
docs to the new taxonomy.
In scope (added 2026-06-22):
- Re-anchor workplans to repos (`repo_id` required, `topic` optional/retired) and
derive market-domain from classification (T04/T10).
- Rename `workstream → workplan` across schema, API, and MCP so the Hub
vocabulary matches the repo files and current usage (T10).
Out of scope:
- Re-architecting task-level semantics beyond what the re-anchor and rename force.
- Changing the Gitea hosting model or repo contents beyond adding the
classification file (and, for cross-repo efforts, creating project repos per
ADR-005).
- Classifying throwaway/forked/non-ecosystem repos (explicit exclusion list).
## Repo boundary
This is the **custodian driving/coordination workplan** (it owns the canon
standard and the portfolio decision), consistent with how `CUST-WP-0043` drove
State Hub work. Implementation tasks **T04–T08 execute in `/home/worsch/state-hub`**
and should be re-homed as a state-hub-local workplan once this plan is approved;
per-repo classification files (T02/T03) are authored in each target repo. The
hub remains a read/index model fed by repo-owned files (ADR-001).
## Tasks
### Phase 1 — Standard as a validation source
### T01 - Derive machine-readable allowed-values from the standard
```task
id: CUST-WP-0050-T01
status: done
priority: high
state_hub_task_id: "d978b1f3-4eca-4a17-835b-2c25d13cae22"
```
Extract the standard's controlled vocabularies (5 categories, 14 domains, the
business_stake and business_mechanics enums, and the recommended capability
families) into a single machine-readable artefact (e.g.
`canon/standards/repo-classification.allowed.yaml`) that both the per-repo
`.repo-classification.yaml` linter and the State Hub validator import.
Done when a single allowed-values file exists, is referenced by the standard, and
a small validator can check a `.repo-classification.yaml` against it.
**Delivered (2026-06-22):** `canon/standards/repo-classification.allowed.yaml`
(categories, domains, business_stake, business_mechanics, capability families,
guidance bounds); referenced from the standard §12; validator
`tools/validate_repo_classification.py` (stdlib + PyYAML) with `--self-test`
(PASS) — checks category/domain enums, secondary-domain rules, kebab-case tags,
and stake/mechanics enums.
### Phase 2 — Classify the portfolio (repo-owned source of truth)
### T02 - Classify custodian-owned repos
```task
id: CUST-WP-0050-T02
status: in_progress
priority: high
state_hub_task_id: "b7edfbb5-483f-4600-9356-8f885c78ce58"
```
Author and human-review `.repo-classification.yaml` for the custodian-domain
repos (the-custodian, state-hub, hub-core, inter-hub, activity-core, issue-core,
kaizen-agentic, llm-connect, ops-bridge, ops-warden, email-connect) using the
standard's §16 agent prompt as a first pass.
Done when each custodian repo has a committed file that validates against T01 and
has been reviewed by a human.
**Progress (2026-06-22):** `the-custodian/.repo-classification.yaml` authored
(category: research · domain: infotech · secondary: agents) and validates clean;
flagged `classified_by: agent` pending human review. Remaining 10 custodian repos
(state-hub, hub-core, inter-hub, activity-core, issue-core, kaizen-agentic,
llm-connect, ops-bridge, ops-warden, email-connect) still to classify.
### T03 - Classify the full Gitea inventory
```task
id: CUST-WP-0050-T03
status: todo
priority: high
state_hub_task_id: "81489716-61ef-4207-ab8a-5877843281de"
```
Produce proposed `.repo-classification.yaml` for every active repo in the Gitea
`coulomb` org (~70), prioritising the 57 already-registered and the ~18
unregistered repos. Deliver as per-repo PRs for owner/human review. Maintain an
explicit **exclusion list** (forks, `lando_worsch/python-snake`, archived
`test_domain_v2`) recorded in this workplan.
Done when every non-excluded active repo has a committed, validated classification
file (or is on the recorded exclusion list).
### Phase 3 — State Hub redesign (executed in /home/worsch/state-hub)
### T04 - Redesign schema: replace domains, add classification
```task
id: CUST-WP-0050-T04
status: todo
priority: high
state_hub_task_id: "b61f6267-c2b2-4325-95fa-30ee899ce7d1"
```
Replace the `domains` table contents with the 14 fixed market domains and add
classification storage to `managed_repos`: `category`, primary `domain_id`,
`secondary_domains[]`, `capability_tags[]`, `business_stake[]`,
`business_mechanics[]`, plus provenance (`classified_at`, `classified_by`,
`standard_version`). Enforce the allowed-values from T01 at the API boundary.
Make `repo_id` the **required** workplan anchor, derive market-domain from the
repo's classification, and demote/retire the `topic`/`domain` spine (see D1 and
T10). Provide an Alembic migration and updated SQLAlchemy models + Pydantic
schemas.
Done when the schema/model/API accept and validate the full classification and
reject invalid values, with a forward migration and a tested downgrade path.
### T05 - Migration mapping + data migration
```task
id: CUST-WP-0050-T05
status: todo
priority: high
state_hub_task_id: "171fa385-4d78-41ea-b749-ac3f9082fe47"
```
Define and apply the mapping from the old 14 domains/topics to the new model
(guided by standard §15 Migration Notes), re-pointing existing topics,
workstreams, goals, decisions, progress events, and charter `topic_id`
references with **no orphaned workstreams**. Resolve the 2026-06-21 discrepancies:
reconcile `markitect-project`↔`markitect-main`, retire phantom
`railiance-bootstrap`/`railiance-hosts` (or relink), collapse the
`vergabe_teilnahme` duplicate, and decide `personhood`'s disposition (charter-only
vs retire).
Done when a dry-run migration report is reviewed and the applied migration leaves
zero orphaned coordination records; the discrepancy list is resolved or explicitly
deferred with reasons.
### T06 - Auto-registration tooling
```task
id: CUST-WP-0050-T06
status: todo
priority: high
state_hub_task_id: "6ae14007-d6d2-4395-814e-ace91486a953"
```
Build an idempotent `register-from-classification` capability (Make target +
script + MCP tool) that, given a repo (local path or Gitea API), reads
`.repo-classification.yaml`, validates against T01, and upserts the
`managed_repo` with full classification. Support a **bulk** run over the Gitea
inventory and reclassification of existing rows. Reuse the k3s/Gitea access path
documented during the 2026-06-21 review (Gitea runs in k3s on coulombcore;
reach it via `kubectl port-forward svc/gitea-http`).
Done when one command registers/reclassifies every repo with a valid file and
emits a report of registered / updated / skipped / invalid.
### T07 - Reclassify existing registrations
```task
id: CUST-WP-0050-T07
status: todo
priority: medium
state_hub_task_id: "6411bf3f-9de2-4bcd-9ffe-6209cda6ba93"
```
Run T06 against the classification files for the 57 previously-registered repos,
reconciling each to the new taxonomy and retiring phantom/duplicate records.
Done when all previously-registered repos reflect their new classification and
the managed-repo set matches the (non-excluded) Gitea inventory.
### Phase 4 — Consuming surfaces & cutover
### T08 - Update dashboard, consistency checker, MCP/REST, docs
```task
id: CUST-WP-0050-T08
status: todo
priority: medium
state_hub_task_id: "09951aec-2960-4c50-b73d-4e2e7bd285c9"
```
Update the dashboard to navigate by category/domain/capability/business-stake;
add a consistency rule flagging registered repos lacking a valid
`.repo-classification.yaml`; expose list/filter-by-classification in MCP/REST; and
update orientation docs (`SCOPE.md`, `README.md`, `.claude/rules/*`) that
reference the old "domains".
Done when the dashboard renders the new taxonomy, the consistency checker has a
classification rule, and docs no longer assume the old domain model.
### T09 - Cutover, verification, retire old model
```task
id: CUST-WP-0050-T09
status: todo
priority: medium
state_hub_task_id: "babbb80a-c52d-4ec2-b217-2f6196a2e5f3"
```
Switch orientation/registration tooling to the new model end-to-end, archive the
old domain semantics, and run `make fix-consistency REPO=the-custodian`.
Done when an end-to-end pass (classify → auto-register → dashboard view) is
verified and the old ad-hoc domain model is retired.
### T10 - Re-anchor to repos + rename workstream → workplan
```task
id: CUST-WP-0050-T10
status: todo
priority: high
state_hub_task_id: "bee16416-a67f-4155-93d7-09f278daa04f"
```
Two coupled changes to the coordination spine, executed in `/home/worsch/state-hub`:
1. **Re-anchor:** make `repo_id` the required anchor for a workplan, derive
market-domain from the repo's classification, and remove the
`topic`/`domain` parent spine (or demote `topic` to an optional cross-repo
tag). Promote `RepoGoal` to the goal primitive; reduce `DomainGoal` to a thin
rollup. Cross-repo workplans anchor to a **project repo** per ADR-005.
2. **Rename:** rename `workstream → workplan` across the DB table, SQLAlchemy
models, Pydantic schemas, REST routes, and MCP tools/resources, so the Hub
vocabulary matches the repo `workplans/` files and current usage. Provide
migration + compatibility/redirect notes for existing tool callers.
Sequence with T04/T05 (same migration window where practical). Done when a
workplan is anchored to a repo with no required topic, market-domain resolves
from classification, and the API/MCP surface uses "workplan" with green tests.
## Open Questions / Decisions
- **D1 (RESOLVED 2026-06-22): the repo is the primary anchor.** Workplans bind to
repos (`repo_id` required); market-domain is *derived* from the repo's
classification; `topic`/`domain` stop being the spine (`topic` retires or
becomes an optional cross-repo tag). This supersedes the earlier "keep topic as
an independent coordination unit" proposal. Implemented by T04/T10.
- **D1a (open, follows from D1): anchor for cross-repo workplans.** Per **ADR-005**,
a complex cross-repo effort gets its own **project repo** (`category: project`)
as the anchor, retired to archive on completion with results living in the
modified product repos. Open sub-point: the project-repo **naming convention**
(e.g. `proj-<slug>` vs a dedicated grouping) and the archival trigger details.
- **D2: classification ownership/approval.** Who approves each repo's
`.repo-classification.yaml` — per-repo owner, or central custodian review?
- **D3: exclusion list.** Confirm exclusions (fork `tegwick/the-custodian`,
`lando_worsch/python-snake`, archived `test_domain_v2`, any inactive repos).
- **D4: behavioural vs descriptive.** Do `secondary_domains` / `capability_tags`
/ `business_stake` drive any Hub behaviour initially, or are they descriptive
until a later phase?
## Risks
- **Breaking-migration blast radius** — topics/workstreams/goals/decisions and
charter `topic_id` references all move; mitigate with a reviewed dry-run and a
tested downgrade (T05).
- **Cross-repo coordination** — T03 touches ~70 repos via PRs; sequence behind
T01/T02 so the vocabulary is stable first.
- **Consistency-checker coupling** — existing C-rules assume the current domain
model; update alongside (T08) to avoid mass false positives.
- **Boundary drift** — keep implementation in `state-hub`; this plan coordinates.