the-custodian/canon/architecture/adr-010-hub-authority-and-local-cache-model.md

272 lines
14 KiB
Markdown
Raw Normal View History

---
id: CUST-ADR-010
type: architecture-decision-record
title: "Hub Authority, Local Cache, and the Two Kinds of Hub Data"
status: proposed
owner: the-custodian
revision: "draft-2"
last_reviewed: "2026-08-31"
review_interval: 6m
decided_by: Bernd Worsch
date: "2026-08-17"
tags: ["architecture", "state-hub", "repo-manager", "hub-core", "authority", "cache", "offline", "adr-001"]
---
# ADR-010: Hub Authority, Local Cache, and the Two Kinds of Hub Data
## Status
Proposed, and **partially superseded by `ADR-012`** (accepted 2026-08-25).
Decisions 1, 5 and 6 are sharpened or given a mechanism there; see the notes on
each below. Everything else in this ADR remains in force.
## Context
Investigating a workplan-identifier flip-flop (`ADR-007`) revealed that two
State Hub instances were operating as **peer databases**, not as a primary and a
replica:
| Address | Instance | Workplans |
|---|---|---|
| `127.0.0.1:8000` (IPv4) | local uvicorn on docker postgres | 955 |
| `[::1]:8000` (IPv6) | ssh tunnel `state-hub-primary` → railiance01 | 649 |
Both listen on port 8000, separated only by IP family, so every tool defaulting
to `127.0.0.1` reached the local instance while believing it was the primary.
> **Mechanism identified 2026-08-24 by `CUST-WP-0067`.** This ADR recorded the
> shared port as the hazard but not why it was *silent*, which is the part that
> let it persist. An `ssh -L` forward with no bind address listens on every
> loopback family, and `ExitOnForwardFailure=yes` fires only when **every**
> requested bind fails. The IPv4 bind lost to the local uvicorn, the IPv6 bind
> succeeded, and the tunnel therefore reported success while serving an address
> nothing resolved to by default. The collision was declared in
> `~/.config/bridge/tunnels.yaml`, and `ops-bridge` guarded only tunnel against
> tunnel, so a non-bridge listener was invisible to it.
>
> Fixed by pinning local forwards to `127.0.0.1`, so a contested port fails
> loudly (`ops-bridge` commit `2213847`). A shared port is recoverable; a shared
> port that reports healthy is not.
Measured divergence — 320 records exist locally but not on the primary:
```text
backed by a file that exists on disk 288 fully rebuildable
no backing file recorded 28 not rebuildable as-is
same filename, different UUID 4 duplicate registration
```
**No file was missing for any of the 288.** The divergence is therefore almost
entirely redundant: it can be discarded and rebuilt from repositories without
losing anything.
`ADR-001` already says work originates as repository files and the hub is a read
model. The two-peer-database situation contradicts that in practice: a second
database accumulated 306 net records of its own.
## Decision
**1. One authoritative hub, deriving from repository files.** The central State
Hub — running on railiance — is authoritative. It is authoritative *as a reading
of the repositories*, not as a second place data lives. Repository files remain
the source of truth (`ADR-001`).
> **Sharpened 2026-08-25 by `ADR-012` decision 1.** "A reading of the
> repositories" never said *which* copy of them, and the honest answer was
> neither the forge nor any particular working copy: the projection derived from
> whichever checkout last ran the sync. The hub holds no repository files at all
> and never reads one. `ADR-012` names the forge as the projection source.
**2. A local hub is a cache, never a database.** Local instances hold a
rebuildable projection. A cache may be discarded and reconstructed from the
repositories at any time, and losing it must never lose work.
This replaces the peer-database arrangement. It is also why the divergence is
tractable: a divergent database is a merge problem, a stale cache is a refresh
problem.
> **Corrected 2026-08-24 by `CUST-WP-0067`.** Those two shapes are not
> exhaustive, and repository records took a third one. Measured that day:
> 122 repositories on the cache, 78 on central, **zero central-only**. A strict
> subset in the cache's favour is neither a merge problem nor a refresh problem
> — refreshing the cache would have *destroyed* the 44 extra records rather than
> reconciling them, and central held no path to re-derive repositories it had
> never been told existed.
>
> The third shape: **cache-only records whose authoritative source exists and is
> reachable, but was never introduced to central.** Its remedy is re-derivation
> from source — not refresh, not merge. Of the 44, 43 had a working copy, all 43
> were pushed, and 34 carried a classification file; nothing was unrecoverable,
> but nothing would have recovered itself either.
>
> Onboarding by date showed a clean break: central's repository registrations
> stopped at 2026-07-08 while the workstation kept accepting them.
**3. Local work requires no hub at all.** Repository files are self-describing —
identifier, status, tasks, all in frontmatter. Working in a repository requires
reading files, not querying an index. A cache is an optimization for cross-repo
questions, never a prerequisite for doing work.
**4. Hub data is classified by origin, and the two kinds have opposite rules.**
| | File-derived | Hub-native |
|---|---|---|
| Examples | workplans, tasks, statuses, dependencies | progress events, decisions, inbox messages, token events |
| Source of truth | the repository file | the hub |
| Offline behaviour | write the file and commit — the commit *is* the write | buffer locally, replay when reachable |
| Conflict model | **none**; conflicts are git conflicts, resolved in git | **none**; append-only merges regardless of order |
| Central accepts pushes? | no — it derives | yes |
Neither kind needs a hub-side conflict model. That is the point of the split: if
central derives file-backed state, it cannot hold a conflicting version of it —
it re-derives whatever git settles on. Two people editing one workplan is a git
conflict and belongs to git.
**5. Central derives file-backed state; it does not accept pushes of it.**
"Authoritative" means authoritative *reading*, so nothing may inject derived
state directly. Hub-native records are the exception and keep a real write path.
> **Sharpened 2026-08-25 by `ADR-012` decision 6.** This was policy, not
> practice: nothing derived, and the workstation pushed everything. `ADR-012`
> retires push-based sync as the primary path so that "central derives" becomes
> true rather than aspirational.
**6. Preliminary until confirmed.** Locally registered data and uncommitted
repository state are preliminary until the central service has seen them.
Mitigation is by changing the repository files and the local cache — never by
editing central to match a local view.
> **Given a mechanism 2026-08-25 by `ADR-012` decisions 3 and 4.** "Preliminary"
> was named here but never built, so in practice locally registered data was
> indistinguishable from derived state once it arrived. It is now a labelled
> overlay within the same projection — explicitly not a second store — and it
> retires when the commit carrying it reaches the forge. The prohibition on
> editing central to match a local view is unchanged.
Combined with `ADR-007` decision 2 (identifiers derived from `PREFIX-WP-NNNN`),
"preliminary" largely stops mattering: a cache computes the same identifier
central will, so offline-registered data is already correct on arrival and needs
confirmation rather than reconciliation.
**7. Every record has exactly one authoritative hub.** The State Hub retirement
splits one hub into several. Multiple central hubs are permitted only under this
rule: the owning hub is determined by the record's repository and domain. Without
it, the same peer-database divergence recurs at larger scale.
**8. Cache reads are advisory and must carry their age.** Cross-repo answers from
a cache are advisory and should be presented with staleness. For the repository
an agent is working in, the file is truth and the cache is never consulted for
correctness.
## Orphan disposition
The 28 records with no backing file are the only ones a cache rebuild would drop.
They fall into three classes, to be separated before any rebuild:
1. **Broken links** — a file exists but `backing_filename` was never recorded.
`RMGR-WP-0004` is one: the workplan file exists and is committed. These are
metadata repairs, not data loss, and are likely the largest class.
2. **Live hub-first records**`proposed`, `ready`, or `backlog` with no file,
in `activity-core`, `core-hub`, `hub-core`, `issue-core`, `ops-hub`,
`prj-forgejo-org-refactor`, `railiance-enablement`, `railiance-infra`,
`reef-railiance`. Each needs a repository file written or an explicit drop.
These are `ADR-001` violations and must not be preserved as hub-only records.
3. **Closed hub-first records**`finished` or `archived` with no file. Retain
as historical provenance where cheap; do not reconstruct plans that are done.
A cache rebuild enforces `ADR-001` retroactively: the only casualties are records
that broke it.
## Consequences
**Positive.** The divergence becomes discardable rather than mergeable. Offline
work is fully supported without a write buffer for file-backed state — the git
commit is the write. No hub-side conflict model is needed for either data kind.
Authority stops being a policy claim and becomes a structural property.
**Negative.** The 28 orphans require case-by-case disposition before a rebuild.
Any consumer that treats a local hub as authoritative must be corrected. The
one-hub-per-record rule constrains the retirement's hub split.
**Rescoping.** `STATE-WP-0068` (offline write buffer and edge relay) is scoped as
a single mechanism. Under decision 4, most of what it buffers does not need
buffering — only the append-only hub-native stream does. Its scope should be
re-examined before more is built on it; this may reduce work rather than add it.
**Correction to `ADR-007`.** Decision 2 there calls the workstation instance a
"development read replica". It was neither a replica nor smaller — it held 306
more workplans than the primary. Superseded by decisions 13 here.
## Implementation
Owned by `repo-manager` for file-derived state (decision `747011c6`; it already
owns repository representation, file-backed record indexing, and reconciliation)
and by `hub-core` for hub-native records. Tracked under `RMGR-WP-0005`.
## References
- ADR-001 — workplans originate as repo files; hub is a read model
- ADR-007 — identifier uniqueness, derived identifiers, worker topology
- Decision `747011c6` — repository standards belong to Repo Manager
- `RMGR-WP-0005` — registrar consolidation and deterministic identifiers
- `STATE-WP-0068` — offline write buffer and edge relay (rescope candidate)
- Divergence measurement, 2026-08-17: 955 local / 649 primary / 320 local-only
## Outcome (2026-08-24)
> Added by `CUST-WP-0067`. This ADR read as though its remediation had landed.
> It had not: the two-instance condition it measured on 2026-08-17 was still
> live seven weeks later, and had continued to accumulate records the whole
> time. An ADR that describes a fix in the present tense is easily mistaken for
> a fix that happened — worth stating plainly for the ones that follow.
The peer-database arrangement is now resolved, by deletion rather than by
reconciliation:
- **The local hub instance is retired.** Rather than making two instances
coexist safely, the second one was removed. Exactly one process binds
`127.0.0.1:8000` and it is the tunnel to central. Nothing remains that could
impersonate the primary, and no call site needed editing — retiring the
impersonator made the existing `127.0.0.1:8000` defaults correct.
- **Decision 3 carried the argument.** Because local work requires no hub at
all, and Repo Manager already maintains a file-derived index, the local
instance was redundant rather than load-bearing.
- **Repo Manager gained the write path it was already assigned.**
`hub-record-authority.yaml` gives it `managed_repos` as `file-derived`, but it
exposed no command for it; the only working path lived in the State Hub repo
and defaulted to a local address. `rmgr repo-onboard` closes that, and refuses
to onboard a repository whose backing file is uncommitted, unpushed, or has no
upstream — a record whose source is only local cannot be re-derived, which is
the failure this ADR exists to prevent.
- **33 of the 44 were re-derived onto central**, taking it from 78 to 111
repositories. The remaining 11 carry written dispositions rather than guessed
values, per the orphan-disposition principle above.
One cause sat deeper than the topology: the hub resolved its classification
allowed-values file from three hardcoded developer-workstation checkouts, so in
a container every classification write failed. Repository classification could
*only* be written from a workstation. That is a second instance of this ADR's
own theme — authority that depends on where a process happens to run is not
authority — and is why "central derives" had never been achievable for this
record type.
## Outcome (2026-08-28)
> Added by `CUST-WP-0068`. The 2026-08-24 outcome closed the *repository*
> divergence. The work-record divergence this ADR originally measured — 955
> local / 649 primary — remained, because the retired instance's database was
> still load-bearing. That is now closed.
- **Central holds 1167 workplans.** Records that existed only in the cache
were re-derived from their files, renamed onto the canonical scheme, or
given a written disposition (`docs/recovery/cache-only-disposition-2026-08-28.md`).
- **No open work record exists only in the cache.** Remaining cache-only
slugs are aliases of recovered records, clay-borg product files (not
workplans), or prefix-migration residue.
- **The cache database is discarded.** Final dump
`~/backups/state-hub-cache-2026-08-28.dump`. Container `infra-postgres-1`
and volume `infra_pg_data` removed. Port 5432 is free.
- **The local instance is no longer load-bearing for any record type.**
Decision 3 is now true in operation, not only in argument.