the-custodian/canon/architecture/adr-010-hub-authority-and-local-cache-model.md
codex 563e7811b9
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
docs(canon): correct ADR-010 on divergence shapes, mechanism, and outcome
Three dated corrections, following the blockquote convention ADR-007 uses:

- Decision 2's merge/refresh dichotomy is not exhaustive. Repository records
  took a third shape — cache-only records whose source exists and is reachable
  but was never introduced to central. Refreshing would have destroyed them.
- The Context recorded the shared port but not why it was silent: an unbound
  ssh -L binds every loopback family, and ExitOnForwardFailure fires only when
  every bind fails.
- A new Outcome section states that the ADR's own remediation never landed and
  the condition was still live seven weeks later.

Refs CUST-WP-0067-T06

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 23:46:33 +02:00

232 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
id: CUST-ADR-010
type: architecture-decision-record
title: "Hub Authority, Local Cache, and the Two Kinds of Hub Data"
status: proposed
owner: the-custodian
revision: "draft-1"
last_reviewed: "2026-08-17"
review_interval: 6m
decided_by: Bernd Worsch
date: "2026-08-17"
tags: ["architecture", "state-hub", "repo-manager", "hub-core", "authority", "cache", "offline", "adr-001"]
---
# ADR-010: Hub Authority, Local Cache, and the Two Kinds of Hub Data
## Status
Proposed.
## Context
Investigating a workplan-identifier flip-flop (`ADR-007`) revealed that two
State Hub instances were operating as **peer databases**, not as a primary and a
replica:
| Address | Instance | Workplans |
|---|---|---|
| `127.0.0.1:8000` (IPv4) | local uvicorn on docker postgres | 955 |
| `[::1]:8000` (IPv6) | ssh tunnel `state-hub-primary` → railiance01 | 649 |
Both listen on port 8000, separated only by IP family, so every tool defaulting
to `127.0.0.1` reached the local instance while believing it was the primary.
> **Mechanism identified 2026-08-24 by `CUST-WP-0067`.** This ADR recorded the
> shared port as the hazard but not why it was *silent*, which is the part that
> let it persist. An `ssh -L` forward with no bind address listens on every
> loopback family, and `ExitOnForwardFailure=yes` fires only when **every**
> requested bind fails. The IPv4 bind lost to the local uvicorn, the IPv6 bind
> succeeded, and the tunnel therefore reported success while serving an address
> nothing resolved to by default. The collision was declared in
> `~/.config/bridge/tunnels.yaml`, and `ops-bridge` guarded only tunnel against
> tunnel, so a non-bridge listener was invisible to it.
>
> Fixed by pinning local forwards to `127.0.0.1`, so a contested port fails
> loudly (`ops-bridge` commit `2213847`). A shared port is recoverable; a shared
> port that reports healthy is not.
Measured divergence — 320 records exist locally but not on the primary:
```text
backed by a file that exists on disk 288 fully rebuildable
no backing file recorded 28 not rebuildable as-is
same filename, different UUID 4 duplicate registration
```
**No file was missing for any of the 288.** The divergence is therefore almost
entirely redundant: it can be discarded and rebuilt from repositories without
losing anything.
`ADR-001` already says work originates as repository files and the hub is a read
model. The two-peer-database situation contradicts that in practice: a second
database accumulated 306 net records of its own.
## Decision
**1. One authoritative hub, deriving from repository files.** The central State
Hub — running on railiance — is authoritative. It is authoritative *as a reading
of the repositories*, not as a second place data lives. Repository files remain
the source of truth (`ADR-001`).
**2. A local hub is a cache, never a database.** Local instances hold a
rebuildable projection. A cache may be discarded and reconstructed from the
repositories at any time, and losing it must never lose work.
This replaces the peer-database arrangement. It is also why the divergence is
tractable: a divergent database is a merge problem, a stale cache is a refresh
problem.
> **Corrected 2026-08-24 by `CUST-WP-0067`.** Those two shapes are not
> exhaustive, and repository records took a third one. Measured that day:
> 122 repositories on the cache, 78 on central, **zero central-only**. A strict
> subset in the cache's favour is neither a merge problem nor a refresh problem
> — refreshing the cache would have *destroyed* the 44 extra records rather than
> reconciling them, and central held no path to re-derive repositories it had
> never been told existed.
>
> The third shape: **cache-only records whose authoritative source exists and is
> reachable, but was never introduced to central.** Its remedy is re-derivation
> from source — not refresh, not merge. Of the 44, 43 had a working copy, all 43
> were pushed, and 34 carried a classification file; nothing was unrecoverable,
> but nothing would have recovered itself either.
>
> Onboarding by date showed a clean break: central's repository registrations
> stopped at 2026-07-08 while the workstation kept accepting them.
**3. Local work requires no hub at all.** Repository files are self-describing —
identifier, status, tasks, all in frontmatter. Working in a repository requires
reading files, not querying an index. A cache is an optimization for cross-repo
questions, never a prerequisite for doing work.
**4. Hub data is classified by origin, and the two kinds have opposite rules.**
| | File-derived | Hub-native |
|---|---|---|
| Examples | workplans, tasks, statuses, dependencies | progress events, decisions, inbox messages, token events |
| Source of truth | the repository file | the hub |
| Offline behaviour | write the file and commit — the commit *is* the write | buffer locally, replay when reachable |
| Conflict model | **none**; conflicts are git conflicts, resolved in git | **none**; append-only merges regardless of order |
| Central accepts pushes? | no — it derives | yes |
Neither kind needs a hub-side conflict model. That is the point of the split: if
central derives file-backed state, it cannot hold a conflicting version of it —
it re-derives whatever git settles on. Two people editing one workplan is a git
conflict and belongs to git.
**5. Central derives file-backed state; it does not accept pushes of it.**
"Authoritative" means authoritative *reading*, so nothing may inject derived
state directly. Hub-native records are the exception and keep a real write path.
**6. Preliminary until confirmed.** Locally registered data and uncommitted
repository state are preliminary until the central service has seen them.
Mitigation is by changing the repository files and the local cache — never by
editing central to match a local view.
Combined with `ADR-007` decision 2 (identifiers derived from `PREFIX-WP-NNNN`),
"preliminary" largely stops mattering: a cache computes the same identifier
central will, so offline-registered data is already correct on arrival and needs
confirmation rather than reconciliation.
**7. Every record has exactly one authoritative hub.** The State Hub retirement
splits one hub into several. Multiple central hubs are permitted only under this
rule: the owning hub is determined by the record's repository and domain. Without
it, the same peer-database divergence recurs at larger scale.
**8. Cache reads are advisory and must carry their age.** Cross-repo answers from
a cache are advisory and should be presented with staleness. For the repository
an agent is working in, the file is truth and the cache is never consulted for
correctness.
## Orphan disposition
The 28 records with no backing file are the only ones a cache rebuild would drop.
They fall into three classes, to be separated before any rebuild:
1. **Broken links** — a file exists but `backing_filename` was never recorded.
`RMGR-WP-0004` is one: the workplan file exists and is committed. These are
metadata repairs, not data loss, and are likely the largest class.
2. **Live hub-first records**`proposed`, `ready`, or `backlog` with no file,
in `activity-core`, `core-hub`, `hub-core`, `issue-core`, `ops-hub`,
`prj-forgejo-org-refactor`, `railiance-enablement`, `railiance-infra`,
`reef-railiance`. Each needs a repository file written or an explicit drop.
These are `ADR-001` violations and must not be preserved as hub-only records.
3. **Closed hub-first records**`finished` or `archived` with no file. Retain
as historical provenance where cheap; do not reconstruct plans that are done.
A cache rebuild enforces `ADR-001` retroactively: the only casualties are records
that broke it.
## Consequences
**Positive.** The divergence becomes discardable rather than mergeable. Offline
work is fully supported without a write buffer for file-backed state — the git
commit is the write. No hub-side conflict model is needed for either data kind.
Authority stops being a policy claim and becomes a structural property.
**Negative.** The 28 orphans require case-by-case disposition before a rebuild.
Any consumer that treats a local hub as authoritative must be corrected. The
one-hub-per-record rule constrains the retirement's hub split.
**Rescoping.** `STATE-WP-0068` (offline write buffer and edge relay) is scoped as
a single mechanism. Under decision 4, most of what it buffers does not need
buffering — only the append-only hub-native stream does. Its scope should be
re-examined before more is built on it; this may reduce work rather than add it.
**Correction to `ADR-007`.** Decision 2 there calls the workstation instance a
"development read replica". It was neither a replica nor smaller — it held 306
more workplans than the primary. Superseded by decisions 13 here.
## Implementation
Owned by `repo-manager` for file-derived state (decision `747011c6`; it already
owns repository representation, file-backed record indexing, and reconciliation)
and by `hub-core` for hub-native records. Tracked under `RMGR-WP-0005`.
## References
- ADR-001 — workplans originate as repo files; hub is a read model
- ADR-007 — identifier uniqueness, derived identifiers, worker topology
- Decision `747011c6` — repository standards belong to Repo Manager
- `RMGR-WP-0005` — registrar consolidation and deterministic identifiers
- `STATE-WP-0068` — offline write buffer and edge relay (rescope candidate)
- Divergence measurement, 2026-08-17: 955 local / 649 primary / 320 local-only
## Outcome (2026-08-24)
> Added by `CUST-WP-0067`. This ADR read as though its remediation had landed.
> It had not: the two-instance condition it measured on 2026-08-17 was still
> live seven weeks later, and had continued to accumulate records the whole
> time. An ADR that describes a fix in the present tense is easily mistaken for
> a fix that happened — worth stating plainly for the ones that follow.
The peer-database arrangement is now resolved, by deletion rather than by
reconciliation:
- **The local hub instance is retired.** Rather than making two instances
coexist safely, the second one was removed. Exactly one process binds
`127.0.0.1:8000` and it is the tunnel to central. Nothing remains that could
impersonate the primary, and no call site needed editing — retiring the
impersonator made the existing `127.0.0.1:8000` defaults correct.
- **Decision 3 carried the argument.** Because local work requires no hub at
all, and Repo Manager already maintains a file-derived index, the local
instance was redundant rather than load-bearing.
- **Repo Manager gained the write path it was already assigned.**
`hub-record-authority.yaml` gives it `managed_repos` as `file-derived`, but it
exposed no command for it; the only working path lived in the State Hub repo
and defaulted to a local address. `rmgr repo-onboard` closes that, and refuses
to onboard a repository whose backing file is uncommitted, unpushed, or has no
upstream — a record whose source is only local cannot be re-derived, which is
the failure this ADR exists to prevent.
- **33 of the 44 were re-derived onto central**, taking it from 78 to 111
repositories. The remaining 11 carry written dispositions rather than guessed
values, per the orphan-disposition principle above.
One cause sat deeper than the topology: the hub resolved its classification
allowed-values file from three hardcoded developer-workstation checkouts, so in
a container every classification write failed. Repository classification could
*only* be written from a workstation. That is a second instance of this ADR's
own theme — authority that depends on where a process happens to run is not
authority — and is why "central derives" had never been achievable for this
record type.