--- id: CUST-ADR-010 type: architecture-decision-record title: "Hub Authority, Local Cache, and the Two Kinds of Hub Data" status: proposed owner: the-custodian revision: "draft-2" last_reviewed: "2026-08-31" review_interval: 6m decided_by: Bernd Worsch date: "2026-08-17" tags: ["architecture", "state-hub", "repo-manager", "hub-core", "authority", "cache", "offline", "adr-001"] --- # ADR-010: Hub Authority, Local Cache, and the Two Kinds of Hub Data ## Status Proposed, and **partially superseded by `ADR-012`** (accepted 2026-08-25). Decisions 1, 5 and 6 are sharpened or given a mechanism there; see the notes on each below. Everything else in this ADR remains in force. ## Context Investigating a workplan-identifier flip-flop (`ADR-007`) revealed that two State Hub instances were operating as **peer databases**, not as a primary and a replica: | Address | Instance | Workplans | |---|---|---| | `127.0.0.1:8000` (IPv4) | local uvicorn on docker postgres | 955 | | `[::1]:8000` (IPv6) | ssh tunnel `state-hub-primary` → railiance01 | 649 | Both listen on port 8000, separated only by IP family, so every tool defaulting to `127.0.0.1` reached the local instance while believing it was the primary. > **Mechanism identified 2026-08-24 by `CUST-WP-0067`.** This ADR recorded the > shared port as the hazard but not why it was *silent*, which is the part that > let it persist. An `ssh -L` forward with no bind address listens on every > loopback family, and `ExitOnForwardFailure=yes` fires only when **every** > requested bind fails. The IPv4 bind lost to the local uvicorn, the IPv6 bind > succeeded, and the tunnel therefore reported success while serving an address > nothing resolved to by default. The collision was declared in > `~/.config/bridge/tunnels.yaml`, and `ops-bridge` guarded only tunnel against > tunnel, so a non-bridge listener was invisible to it. > > Fixed by pinning local forwards to `127.0.0.1`, so a contested port fails > loudly (`ops-bridge` commit `2213847`). A shared port is recoverable; a shared > port that reports healthy is not. Measured divergence — 320 records exist locally but not on the primary: ```text backed by a file that exists on disk 288 fully rebuildable no backing file recorded 28 not rebuildable as-is same filename, different UUID 4 duplicate registration ``` **No file was missing for any of the 288.** The divergence is therefore almost entirely redundant: it can be discarded and rebuilt from repositories without losing anything. `ADR-001` already says work originates as repository files and the hub is a read model. The two-peer-database situation contradicts that in practice: a second database accumulated 306 net records of its own. ## Decision **1. One authoritative hub, deriving from repository files.** The central State Hub — running on railiance — is authoritative. It is authoritative *as a reading of the repositories*, not as a second place data lives. Repository files remain the source of truth (`ADR-001`). > **Sharpened 2026-08-25 by `ADR-012` decision 1.** "A reading of the > repositories" never said *which* copy of them, and the honest answer was > neither the forge nor any particular working copy: the projection derived from > whichever checkout last ran the sync. The hub holds no repository files at all > and never reads one. `ADR-012` names the forge as the projection source. **2. A local hub is a cache, never a database.** Local instances hold a rebuildable projection. A cache may be discarded and reconstructed from the repositories at any time, and losing it must never lose work. This replaces the peer-database arrangement. It is also why the divergence is tractable: a divergent database is a merge problem, a stale cache is a refresh problem. > **Corrected 2026-08-24 by `CUST-WP-0067`.** Those two shapes are not > exhaustive, and repository records took a third one. Measured that day: > 122 repositories on the cache, 78 on central, **zero central-only**. A strict > subset in the cache's favour is neither a merge problem nor a refresh problem > — refreshing the cache would have *destroyed* the 44 extra records rather than > reconciling them, and central held no path to re-derive repositories it had > never been told existed. > > The third shape: **cache-only records whose authoritative source exists and is > reachable, but was never introduced to central.** Its remedy is re-derivation > from source — not refresh, not merge. Of the 44, 43 had a working copy, all 43 > were pushed, and 34 carried a classification file; nothing was unrecoverable, > but nothing would have recovered itself either. > > Onboarding by date showed a clean break: central's repository registrations > stopped at 2026-07-08 while the workstation kept accepting them. **3. Local work requires no hub at all.** Repository files are self-describing — identifier, status, tasks, all in frontmatter. Working in a repository requires reading files, not querying an index. A cache is an optimization for cross-repo questions, never a prerequisite for doing work. **4. Hub data is classified by origin, and the two kinds have opposite rules.** | | File-derived | Hub-native | |---|---|---| | Examples | workplans, tasks, statuses, dependencies | progress events, decisions, inbox messages, token events | | Source of truth | the repository file | the hub | | Offline behaviour | write the file and commit — the commit *is* the write | buffer locally, replay when reachable | | Conflict model | **none**; conflicts are git conflicts, resolved in git | **none**; append-only merges regardless of order | | Central accepts pushes? | no — it derives | yes | Neither kind needs a hub-side conflict model. That is the point of the split: if central derives file-backed state, it cannot hold a conflicting version of it — it re-derives whatever git settles on. Two people editing one workplan is a git conflict and belongs to git. **5. Central derives file-backed state; it does not accept pushes of it.** "Authoritative" means authoritative *reading*, so nothing may inject derived state directly. Hub-native records are the exception and keep a real write path. > **Sharpened 2026-08-25 by `ADR-012` decision 6.** This was policy, not > practice: nothing derived, and the workstation pushed everything. `ADR-012` > retires push-based sync as the primary path so that "central derives" becomes > true rather than aspirational. **6. Preliminary until confirmed.** Locally registered data and uncommitted repository state are preliminary until the central service has seen them. Mitigation is by changing the repository files and the local cache — never by editing central to match a local view. > **Given a mechanism 2026-08-25 by `ADR-012` decisions 3 and 4.** "Preliminary" > was named here but never built, so in practice locally registered data was > indistinguishable from derived state once it arrived. It is now a labelled > overlay within the same projection — explicitly not a second store — and it > retires when the commit carrying it reaches the forge. The prohibition on > editing central to match a local view is unchanged. Combined with `ADR-007` decision 2 (identifiers derived from `PREFIX-WP-NNNN`), "preliminary" largely stops mattering: a cache computes the same identifier central will, so offline-registered data is already correct on arrival and needs confirmation rather than reconciliation. **7. Every record has exactly one authoritative hub.** The State Hub retirement splits one hub into several. Multiple central hubs are permitted only under this rule: the owning hub is determined by the record's repository and domain. Without it, the same peer-database divergence recurs at larger scale. **8. Cache reads are advisory and must carry their age.** Cross-repo answers from a cache are advisory and should be presented with staleness. For the repository an agent is working in, the file is truth and the cache is never consulted for correctness. ## Orphan disposition The 28 records with no backing file are the only ones a cache rebuild would drop. They fall into three classes, to be separated before any rebuild: 1. **Broken links** — a file exists but `backing_filename` was never recorded. `RMGR-WP-0004` is one: the workplan file exists and is committed. These are metadata repairs, not data loss, and are likely the largest class. 2. **Live hub-first records** — `proposed`, `ready`, or `backlog` with no file, in `activity-core`, `core-hub`, `hub-core`, `issue-core`, `ops-hub`, `prj-forgejo-org-refactor`, `railiance-enablement`, `railiance-infra`, `reef-railiance`. Each needs a repository file written or an explicit drop. These are `ADR-001` violations and must not be preserved as hub-only records. 3. **Closed hub-first records** — `finished` or `archived` with no file. Retain as historical provenance where cheap; do not reconstruct plans that are done. A cache rebuild enforces `ADR-001` retroactively: the only casualties are records that broke it. ## Consequences **Positive.** The divergence becomes discardable rather than mergeable. Offline work is fully supported without a write buffer for file-backed state — the git commit is the write. No hub-side conflict model is needed for either data kind. Authority stops being a policy claim and becomes a structural property. **Negative.** The 28 orphans require case-by-case disposition before a rebuild. Any consumer that treats a local hub as authoritative must be corrected. The one-hub-per-record rule constrains the retirement's hub split. **Rescoping.** `STATE-WP-0068` (offline write buffer and edge relay) is scoped as a single mechanism. Under decision 4, most of what it buffers does not need buffering — only the append-only hub-native stream does. Its scope should be re-examined before more is built on it; this may reduce work rather than add it. **Correction to `ADR-007`.** Decision 2 there calls the workstation instance a "development read replica". It was neither a replica nor smaller — it held 306 more workplans than the primary. Superseded by decisions 1–3 here. ## Implementation Owned by `repo-manager` for file-derived state (decision `747011c6`; it already owns repository representation, file-backed record indexing, and reconciliation) and by `hub-core` for hub-native records. Tracked under `RMGR-WP-0005`. ## References - ADR-001 — workplans originate as repo files; hub is a read model - ADR-007 — identifier uniqueness, derived identifiers, worker topology - Decision `747011c6` — repository standards belong to Repo Manager - `RMGR-WP-0005` — registrar consolidation and deterministic identifiers - `STATE-WP-0068` — offline write buffer and edge relay (rescope candidate) - Divergence measurement, 2026-08-17: 955 local / 649 primary / 320 local-only ## Outcome (2026-08-24) > Added by `CUST-WP-0067`. This ADR read as though its remediation had landed. > It had not: the two-instance condition it measured on 2026-08-17 was still > live seven weeks later, and had continued to accumulate records the whole > time. An ADR that describes a fix in the present tense is easily mistaken for > a fix that happened — worth stating plainly for the ones that follow. The peer-database arrangement is now resolved, by deletion rather than by reconciliation: - **The local hub instance is retired.** Rather than making two instances coexist safely, the second one was removed. Exactly one process binds `127.0.0.1:8000` and it is the tunnel to central. Nothing remains that could impersonate the primary, and no call site needed editing — retiring the impersonator made the existing `127.0.0.1:8000` defaults correct. - **Decision 3 carried the argument.** Because local work requires no hub at all, and Repo Manager already maintains a file-derived index, the local instance was redundant rather than load-bearing. - **Repo Manager gained the write path it was already assigned.** `hub-record-authority.yaml` gives it `managed_repos` as `file-derived`, but it exposed no command for it; the only working path lived in the State Hub repo and defaulted to a local address. `rmgr repo-onboard` closes that, and refuses to onboard a repository whose backing file is uncommitted, unpushed, or has no upstream — a record whose source is only local cannot be re-derived, which is the failure this ADR exists to prevent. - **33 of the 44 were re-derived onto central**, taking it from 78 to 111 repositories. The remaining 11 carry written dispositions rather than guessed values, per the orphan-disposition principle above. One cause sat deeper than the topology: the hub resolved its classification allowed-values file from three hardcoded developer-workstation checkouts, so in a container every classification write failed. Repository classification could *only* be written from a workstation. That is a second instance of this ADR's own theme — authority that depends on where a process happens to run is not authority — and is why "central derives" had never been achievable for this record type. ## Outcome (2026-08-28) > Added by `CUST-WP-0068`. The 2026-08-24 outcome closed the *repository* > divergence. The work-record divergence this ADR originally measured — 955 > local / 649 primary — remained, because the retired instance's database was > still load-bearing. That is now closed. - **Central holds 1167 workplans.** Records that existed only in the cache were re-derived from their files, renamed onto the canonical scheme, or given a written disposition (`docs/recovery/cache-only-disposition-2026-08-28.md`). - **No open work record exists only in the cache.** Remaining cache-only slugs are aliases of recovered records, clay-borg product files (not workplans), or prefix-migration residue. - **The cache database is discarded.** Final dump `~/backups/state-hub-cache-2026-08-28.dump`. Container `infra-postgres-1` and volume `infra_pg_data` removed. Port 5432 is free. - **The local instance is no longer load-bearing for any record type.** Decision 3 is now true in operation, not only in argument.