coulomb/the-custodian
4
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

feat(workplan): open CUST-WP-0067 hub target resolution
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
Details
CI Smoke / container-smoke (push) Successful in 1s
Details

Browse source 
ADR-010 documented two hub instances sharing port 8000 separated only by IP
family; the condition was never remediated. Repo onboarding from the
workstation has landed in the local cache since 2026-07-08 — 122 cache repos
vs 78 central, a strict subset with 0 central-only records.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
codex 2026-08-24 22:01:55 +02:00
parent 1f56ce55fe
commit c7d67aba84
1 changed files with 211 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

211
workplans/CUST-WP-0067-hub-authority-target-resolution.md Normal file
View file

@ -0,0 +1,211 @@
---
id: CUST-WP-0067
type: workplan
title: "Resolve hub target ambiguity and restore repo onboarding to the authoritative hub"
domain: infotech
repo: the-custodian
status: active
owner: codex
created: "2026-08-24"
updated: "2026-08-24"
quality_dor: DoR-Ok
quality_dor_at: "2026-08-24"
quality_dor_by: codex
quality_dor_note: "Live divergence measured (122 cache / 78 central, strict subset, 0 central-only); root cause located in configuration (port collision plus 127.0.0.1 defaults in four call sites); recovery source confirmed on disk and pushed for 43 of 44 records; owner boundaries follow ADR-010 and hub-record-authority.yaml."
---
# Resolve hub target ambiguity and restore repo onboarding to the authoritative hub
## Goal
Make "the authoritative hub" a resolvable address rather than a policy claim, so
that repository onboarding lands on central instead of a local cache that
believes it is primary. Then recover the seven weeks of workstation-originated
repo registrations that never reached central, and correct the ADR-010 framing
that treats this class of gap as a refresh problem.
## Context
`ADR-010` documented two State Hub instances sharing port 8000, separated only
by IP family, and named the consequence: every tool defaulting to `127.0.0.1`
reaches the local instance while believing it is the primary. That condition was
never remediated. As of 2026-08-24 it is still live:
```text
LISTEN 127.0.0.1:8000 uvicorn (pid 3258) local cache
LISTEN [::1]:8000 ssh -L 8000:10.43.68.154:8000 central
```
The collision is declared in `~/.config/bridge/tunnels.yaml`, where the tunnel
`state-hub-primary` takes `local_port: 8000` — the port the local uvicorn
already holds. `ssh -L` binds both address families, the IPv4 bind loses to
uvicorn, and the IPv6 bind succeeds, so `ExitOnForwardFailure=yes` does not
fire. `_reject_duplicate_local_ports` in `ops-bridge` guards only bridge tunnel
against bridge tunnel; it cannot see a non-bridge listener.
Measured repository divergence, 2026-08-24:
```text
cache (127.0.0.1:8000) 122
central ([::1]:8000) 78
central-only 0
cache-only 44
```
Central is a strict subset. That is not the shape ADR-010 anticipated. A refresh
of the cache would delete the 44 rather than reconcile them, and central holds
no path to re-derive records for repositories it has never been told exist.
Onboarding by registration date shows a clean break:
| Registered | central | cache-only |
|---|---|---|
| 2026-02 – 2026-06 | 74 | 0 |
| 2026-07 | 1 | 24 |
| 2026-08 | 3 | 20 |
Central is otherwise live — `last_state_synced_at` runs to 2026-08-22, and the
three August rows (`repo-manager`, `rail-kubernetes`, `fin-hub`) were registered
from the railiance side. Only workstation-originated onboarding is lost.
Nothing is unrecoverable: 43 of the 44 have a working copy on disk, all 43 have
an `origin` remote and are pushed (`soul-frame` and `rein-openweights` are ahead
of their remote), and 35 carry `.repo-classification.yaml`. `agent-harness` has
no working copy and needs separate disposition.
`hub-record-authority.yaml` assigns `managed_repos` to `repo-manager` as
`file-derived`. Repo Manager owns the record type but exposes no command that
onboards a repository into a hub, and `registrar-reconcile --confirm-primary`
defaults its "primary" to `127.0.0.1:8000` — confirming projections against the
cache and reporting success while central holds nothing.
## Repair the crashing status printer
```task
id: CUST-WP-0067-T01
status: done
priority: low
```
`statehub status` raised `KeyError: 'in_progress'` at `custodian_cli.py:565`.
The task vocabulary is `wait|todo|progress|done|cancel`; `in_progress` and
`blocked` are not task statuses in this schema and the totals block never
carried them. Fix the key names, tolerate the `workplans`/`workstreams` rename,
and print the resolved API base so the operator can see which instance answered.
**Done (2026-08-24):** `custodian_cli.py` `cmd_status` now reads
`progress`/`todo`/`wait`, accepts either totals key for workplans, uses `.get()`
defaults throughout, and prints `API base:` as its second line. Verified against
the live API.
## Eliminate the port collision
```task
id: CUST-WP-0067-T02
status: todo
priority: high
```
Move `state-hub-primary` off `local_port: 8000` to `18000`, matching the port
map already reserved for the primary in the global agent instructions. Leave the
local cache on 8000 so nothing that currently resolves to it changes behaviour —
this step is deliberately non-breaking and must stay that way.
Extend the `ops-bridge` duplicate-port guard so it rejects a `direction: local`
tunnel whose `local_port` is already bound by any listener, not only by another
bridge tunnel. The current guard could not have caught this.
Acceptance: `state-hub-primary` binds `127.0.0.1:18000`; `[::1]:8000` no longer
serves a hub; `curl 127.0.0.1:18000/state/health` reports the central instance
and `curl 127.0.0.1:8000/state/health` reports the cache; the two return
different repo counts; the guard fails a synthetic config that reintroduces the
collision.
## Make the hub target explicit and unspoofable
```task
id: CUST-WP-0067-T03
status: todo
priority: high
```
Remove the `http://127.0.0.1:8000` default from every call site that claims to
reach the primary: `custodian_cli.py:28`, `statehub_register.py:22`,
`repo_manager/cli.py:56,405`, `repo_manager/commands/registrar_reconcile.py:410`.
A default that silently resolves to a cache is worse than a missing one. Give
the hub an identity it can assert — instance role served from the health or
summary endpoint — and make `registrar-reconcile --confirm-primary` refuse to
run against anything that does not assert `primary`. Confirming against a cache
is a false green and is the specific failure that let this run for seven weeks.
Acceptance: `--confirm-primary` against the cache exits non-zero with a message
naming the instance it reached; against central it succeeds; no code path
reaches a hub without an explicitly resolved target.
## Give Repo Manager a real onboarding write path
```task
id: CUST-WP-0067-T04
status: todo
priority: high
```
Repo Manager owns `managed_repos` as `file-derived` and has no command for it.
Add onboarding that follows ADR-010 decision 5: write `.repo-classification.yaml`
in the target repository, commit, push, and have central derive the record.
Central must not accept a push of derived state, so the command's job is to make
the source file correct and reachable, then trigger and verify derivation.
Resolve the bootstrap gap explicitly — central derives from repositories it
already knows about, so a never-registered repository is never scanned. The
onboarding path must be able to introduce a repository central has not seen.
Acceptance: onboarding a fresh repository from the workstation produces a
central record with no manual step; re-running is idempotent; the cache is not
written directly.
## Backfill the 44 cache-only registrations
```task
id: CUST-WP-0067-T05
status: todo
priority: medium
```
Runs only after T02–T04; backfilling before the target is unambiguous refills
the cache. Drive the 43 on-disk repositories through the T04 path. Nine lack
`.repo-classification.yaml` (`binky-control`, `clay-borg`,
`direkt-vermittlung-de`, `polycode-sim`, `railiance-telemetry`, `ralph-workplan`,
`rein-openweights`, `testdrive-jsui`, `timeline-svg`) and need one authored with
the owner rather than guessed — classification is not mechanical, per
`CUST-WP-0065-T01`. Push `soul-frame` and `rein-openweights` first.
`agent-harness` has no working copy: decide restore-from-remote or drop, and
record the decision. Do not preserve it as a hub-only record — that is the
ADR-001 violation ADR-010 calls out.
Acceptance: central and cache repo counts converge; the cache-only set is empty
or every remainder has a written disposition.
## Correct the ADR-010 repo-record framing
```task
id: CUST-WP-0067-T06
status: todo
priority: medium
```
ADR-010 decision 2 says a divergent database is a merge problem and a stale
cache is a refresh problem. For `managed_repos` neither holds: the gap is a
strict subset in the cache's favour, and refreshing destroys rather than
reconciles. Record the third shape — cache-only records whose authoritative
source exists but was never introduced to central — and state that its remedy is
re-derivation from source, not refresh and not merge.
Also correct the implicit assumption that the ADR's own remediation happened.
The measurement stands; the fix did not land, and the ADR reads as though it did.
Acceptance: ADR-010 revised with a superseding note dated and linked to this
workplan; the port-collision remediation recorded as an outcome rather than an
observation.