1eb5fa51ed
Item 13 is the restore RESOURCE-WP-0002 already proved. Item 14 retires the dead Bitnami postgresql-ha deploy path and gates Valkey until a consumer exists. Item 17 publishes the versioned S3 consumer-interface index.
293 lines
13 KiB
Markdown
293 lines
13 KiB
Markdown
---
|
||
id: RAILIANCE-WP-0016
|
||
type: workplan
|
||
title: "Architecture cleanup backlog, WSJF-prioritized"
|
||
domain: financials
|
||
repo: railiance-platform
|
||
status: finished
|
||
owner: codex
|
||
topic_slug: railiance
|
||
created: "2026-08-12"
|
||
updated: "2026-08-15"
|
||
related_repos:
|
||
- railiance-master
|
||
- railiance-infra
|
||
- railiance-cluster
|
||
- railiance-enablement
|
||
- railiance-telemetry
|
||
- railiance-forge
|
||
state_hub_workstream_id: "8611b555-c681-496d-bfba-667198456837"
|
||
---
|
||
|
||
# RAILIANCE-WP-0016 - Architecture cleanup backlog, WSJF-prioritized
|
||
|
||
## Goal
|
||
|
||
Turn the INTENT/SCOPE gap analysis into an ordered backlog, so cleanup happens
|
||
highest-value-least-effort first rather than in the order things were noticed.
|
||
|
||
Source: `history/2026-08-12-intent-scope-gap-analysis.md`.
|
||
|
||
## Boundary — read before working this
|
||
|
||
This workplan **coordinates and orders**; it does not execute. Every item below
|
||
belongs to another repo, and each carries its owning repo in the table. S3 does
|
||
not own fleet architecture cleanup — `railiance-master` does.
|
||
|
||
This backlog lives here because the analysis was done here. **It should be
|
||
adopted by `railiance-master`**, and this workplan closed, once master has a
|
||
place for it. Until then it is better recorded than lost. S3 executes only the
|
||
items marked `railiance-platform`.
|
||
|
||
## Scoring method
|
||
|
||
WSJF = **Cost of Delay ÷ Job Size**, where Cost of Delay = Value + Time
|
||
Criticality + Risk Reduction. Each component 1–10, job size 1–10.
|
||
|
||
Two honesty notes. These are **judgments, not measurements** — their value is
|
||
the relative ordering and the explicit reasoning, not the arithmetic. And job
|
||
sizes for unbuilt things (telemetry, the conformance loop) are the least
|
||
reliable numbers here, because nobody has scoped them yet.
|
||
|
||
## The backlog
|
||
|
||
| # | Item | Owner | V | TC | RR | Size | **WSJF** |
|
||
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||
| 1 | Converge the firewall allowlist; prune two stale grants | `railiance-infra` | 6 | 9 | 10 | 1 | **25.0** |
|
||
| 2 | Ratify the five pending decisions | `railiance-master` | 8 | 7 | 5 | 1 | **20.0** |
|
||
| 3 | Move the S1 goal off `railiance-hosts`, then retire it | `railiance-master` | 5 | 5 | 4 | 1 | **14.0** |
|
||
| 4 | Settle ArgoCD: adopt, retire, or relocate | `railiance-master` | 6 | 6 | 6 | 2 | **9.0** |
|
||
| 5 | Audit base role for further declared-vs-live drift | `railiance-infra` | 5 | 6 | 9 | 3 | **6.7** |
|
||
| 6 | `rapp.yaml` schema + family declaration validator | `railiance-master` | 8 | 5 | 7 | 3 | **6.7** |
|
||
| 7 | Reconcile hub capability attribution (C1) | `railiance-master` | 5 | 4 | 6 | 3 | **5.0** |
|
||
| 8 | Deployable coverage check: every deployable → one rapp | `railiance-master` | 7 | 4 | 7 | 4 | **4.5** |
|
||
| 9a | Telemetry: minimal metrics collection on railiance01 | `railiance-telemetry` | 8 | 5 | 5 | 4 | **4.5** |
|
||
| 9b | Telemetry: one expectation-based alert (needs 10b) | `railiance-telemetry` | 8 | 5 | 2 | 2 | **7.5** |
|
||
| ~~10a~~ | ~~Run `make verify`; tighten the UFW assertion~~ **DONE 2026-08-12** | `railiance-infra` | 8 | 7 | 9 | 1–2 | **12.0** |
|
||
| 10b | Schedule verification; route failures where they are seen | `railiance-infra` / Q7 owner | 7 | 6 | 8 | 3 | **7.0** |
|
||
| 11 | Consolidate the bootstrap path; sort inherited S2 content | `railiance-bootstrap` | 5 | 3 | 4 | 4 | **3.0** |
|
||
| 12 | Forge migration: `rapp-forgejo` + S4 layer transfer | `railiance-forge` / S4 | 6 | 4 | 4 | 5 | **2.8** |
|
||
| ~~13~~ | ~~Prove a restore end to end~~ **DONE 2026-08-15** | `railiance-platform` | 9 | 5 | 9 | 8 | **2.9** |
|
||
| ~~14~~ | ~~Valkey extraction + legacy postgresql-ha retirement~~ **DONE 2026-08-15** | `railiance-platform` | 5 | 4 | 4 | 5 | **2.6** |
|
||
| 15 | Sweep remaining Gitea prose (~31 files each) | `railiance-forge`, `railiance-apps` | 3 | 2 | 2 | 3 | **2.3** |
|
||
| 16 | Declare the Q6 cost boundary with `resource-control`/`fin-hub` | `railiance-master` | 4 | 3 | 3 | 4 | **2.5** |
|
||
| ~~17~~ | ~~Version S3 consumer service interfaces~~ **DONE 2026-08-15** | `railiance-platform` | 6 | 3 | 5 | 6 | **2.3** |
|
||
| 18 | Resolve the C4 ecosystem-modelling overlap | `railiance-master` / `railiance-fabric` | 4 | 3 | 4 | 5 | **2.2** |
|
||
| 19 | S4 build-out: pipeline templates, SDKs, portal | `railiance-enablement` | 7 | 3 | 4 | 9 | **1.6** |
|
||
|
||
## Reading the order
|
||
|
||
**Items 1–3 are nearly free and should happen first.** All three are "decided
|
||
but unmoved" — waiting on ratification or a single command, not on effort.
|
||
Item 1 in particular is a live security exposure: two standing firewall grants
|
||
point at addresses the ISP has since reassigned, and the fix is one convergence
|
||
run.
|
||
|
||
**Item 2 is the highest-leverage thing on the list.** Five pending
|
||
ratifications in `railiance-master` block items 6, 7, 8, 12 and parts of 9 and
|
||
10. It is a day of decisions that unblocks a quarter of the backlog.
|
||
|
||
**Items 9 and 10 are the strategic core** — Q2 and Q7, the two halves of the
|
||
self-evidencing thread five layers independently asked for. They were rescoped
|
||
on 2026-08-12 (see below) and item 10 turned out to be badly mis-sized: the
|
||
conformance checker already exists in `railiance-infra` and has simply not run
|
||
since 2026-03-09. **Slice 10a is now the highest-value item after the
|
||
ratifications** at WSJF 12.0.
|
||
|
||
**Item 13 is done, not blocked.** `RESOURCE-WP-0002` finished 2026-08-15.
|
||
Barman is live on `platform-pg`; T05 recorded full restore and PITR on
|
||
2026-08-14; daily backups have completed since. Pointer:
|
||
`docs/evidence/RAILIANCE-WP-0016-item13-restore-2026-08-15.md`.
|
||
|
||
**Item 19 is last and that is deliberate.** S4 build-out is the widest gap in
|
||
the family, but it is large, and item 12 gives S4 a smaller first
|
||
responsibility to prove delivery on. Building the portal before the forge
|
||
transfer would be starting at the hard end.
|
||
|
||
## Rescoping items 9 and 10 (2026-08-12)
|
||
|
||
### What was assumed
|
||
|
||
That Q2 Observability and Q7 Governance were both greenfield builds of
|
||
comparable size (5), sitting mid-table at WSJF 4.6.
|
||
|
||
### What is actually there
|
||
|
||
`railiance-infra` already has:
|
||
|
||
- `goss/baseline.yaml` — a host baseline assertion suite
|
||
- `ansible/roles/goss` + `ansible/playbooks/verify.yaml` — the runner
|
||
- `make verify` — runs it against all hosts, **commits TAP reports to git**, and
|
||
exits non-zero on failure
|
||
- `reports/*.tap` — an evidence trail, last written **2026-03-09**
|
||
|
||
So a check-and-evidence mechanism exists and is wired. Two things are wrong with
|
||
it, and neither is a build:
|
||
|
||
1. **Nothing runs it.** Five months of silence. A verification suite that is not
|
||
scheduled is a verification suite that does not exist.
|
||
2. **Its firewall assertion is too loose in exactly the place that mattered.**
|
||
It asserts `/6443\/tcp.*ALLOW/` — that the port is allowed. It does not
|
||
assert *from whom*. That regex passes identically whether 6443 is restricted
|
||
to one operator address or open to the entire internet, which is precisely
|
||
the drift that went undetected.
|
||
|
||
It would, however, have caught the *other* finding: it asserts
|
||
`Status: active`, and UFW is entirely inactive on `CoulombCore`.
|
||
|
||
### The rescoped slices
|
||
|
||
**Item 10 — conformance loop**
|
||
|
||
| Slice | Work | Size |
|
||
| --- | --- | --- |
|
||
| 10a | Run `make verify` against both hosts; tighten the UFW assertion to compare the live allowlist against `k3s_api_allowed_sources` rather than matching any ALLOW; fix what it surfaces | **2** |
|
||
| 10b | Schedule it, and route failures somewhere a human or agent actually sees | 3 |
|
||
| 10c | Extend to declaration conformance — rapp/rail/reef schema validation | blocked on item 6 |
|
||
| 10d | Deployable coverage: every live deployable maps to exactly one rapp | blocked on item 8 |
|
||
|
||
**Item 9 — telemetry**
|
||
|
||
| Slice | Work | Size |
|
||
| --- | --- | --- |
|
||
| 9a | Minimal metrics collection on `railiance01` | 4 |
|
||
| 9b | One expectation-based alert on the 10b path | 2 |
|
||
| 9c | Retention sufficient to prove a restore or rollout held | 3 |
|
||
| 9d | Dashboards and agent-queryable surfaces | 4 |
|
||
|
||
### The shared dependency worth naming
|
||
|
||
**10b and 9b are the same plumbing.** Both are "a check failed — now make that
|
||
visible to someone who can act." Building an alert path twice would be waste,
|
||
and building either of 9 or 10 *without* it produces logs nobody reads. That
|
||
path is the real first dependency for both, and it is small.
|
||
|
||
It also has an obvious candidate already in the fleet: the Goss suite commits
|
||
TAP evidence to git, and State Hub takes progress events. Neither is an alert,
|
||
but the gap between them is narrow.
|
||
|
||
### Revised positions
|
||
|
||
| Item | Old WSJF | New WSJF | Effect |
|
||
| --- | --- | --- | --- |
|
||
| 10a run + tighten Goss | 4.6 (as part of 10) | **12.0** (V8 TC7 RR9 / size 2) | rises to 4th overall |
|
||
| 10b schedule + alert path | — | **7.0** (V7 TC6 RR8 / size 3) | rises above ArgoCD |
|
||
| 9b one real alert | — | **7.5** (V8 TC5 RR2 / size 2) | depends on 10b |
|
||
| 9a metrics collection | 4.6 | **4.5** (unchanged) | stays mid-table |
|
||
|
||
**Conclusion: item 10a should be done next, immediately after the ratifications.**
|
||
It is two points of effort, it directly closes the hole that produced this
|
||
session's security finding, and it converts an existing but dormant asset into a
|
||
working control. Item 9 is correctly mid-table and genuinely is a build.
|
||
|
||
### 10a outcome (2026-08-12) — the estimate held, and the check paid for itself immediately
|
||
|
||
Delivered in `railiance-infra`: `goss/baseline.yaml` became a template rendered
|
||
per host from the *same* inventory declaration that converges the host, so the
|
||
assertion cannot drift from what it checks. `verify.yaml` gained the
|
||
`vars_files` the bootstrap play already had — without it the firewall assertions
|
||
rendered empty and **silently asserted nothing**, which is worse than having no
|
||
check at all.
|
||
|
||
Result on `Railiance01`: 32 assertions, 31 pass. And the one failure is a real
|
||
defect the old assertion could never have seen:
|
||
|
||
> **Flannel VXLAN (`8472/udp`) is declared open to Anywhere**, with no source
|
||
> restriction — the same defect pattern as the k3s API rule. It is currently
|
||
> *absent* from the host, so converging the base role would **introduce** the
|
||
> exposure rather than correct drift.
|
||
|
||
Tracked as `RAIL-HO-WP-0009-T06`. Two things worth taking from this:
|
||
|
||
1. **The check earned its keep on first run** — it found a latent defect that
|
||
convergence would have activated, in under two points of effort.
|
||
2. **It is an argument for finishing 10b.** This run happened because a human
|
||
asked. Nothing schedules it, so the next latent defect waits for the next
|
||
person who happens to look.
|
||
|
||
### The lesson worth keeping
|
||
|
||
The original size-5 estimate for item 10 was wrong because nobody looked. The
|
||
gap analysis correctly identified "no conformance loop" as a *capability* gap,
|
||
but the capability was two-thirds built and unrun. **Before sizing a gap as a
|
||
build, check whether it is actually an abandonment.**
|
||
|
||
## Tasks
|
||
|
||
```task
|
||
id: RAILIANCE-WP-0016-T01
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "9fabf938-0479-4282-b1fe-88af0d2fbd36"
|
||
```
|
||
|
||
Produce the INTENT/SCOPE gap analysis across all `railiance-*` repos, comparing
|
||
declared aspiration against declared state against verified reality.
|
||
|
||
Delivered: `history/2026-08-12-intent-scope-gap-analysis.md`. Named the pattern
|
||
that organizes the backlog — every gap is *declared but unbuilt*, *built but
|
||
undeclared*, or *decided but unmoved*, and the third kind is cheapest to clear
|
||
while blocking the most.
|
||
|
||
```task
|
||
id: RAILIANCE-WP-0016-T02
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "5b4d5f69-40fd-4af0-be23-325864cea48b"
|
||
```
|
||
|
||
Score and order the backlog by WSJF, recording owner and reasoning per item so
|
||
the ordering can be argued with rather than merely followed.
|
||
|
||
```task
|
||
id: RAILIANCE-WP-0016-T03
|
||
status: done
|
||
priority: high
|
||
state_hub_task_id: "9ede2fe5-5b6c-4792-b141-9c8f140c1a28"
|
||
```
|
||
|
||
Route the backlog to `railiance-master` for adoption, with the recommendation
|
||
that it takes items 2, 3, 4, 6, 7, 8, 16 and 18 — the ratification and taxonomy
|
||
items — and that this workplan closes once adopted.
|
||
|
||
```task
|
||
id: RAILIANCE-WP-0016-T04
|
||
status: done
|
||
priority: medium
|
||
state_hub_task_id: "aac02de1-6367-4767-b7ee-c980577e778f"
|
||
```
|
||
|
||
Rescope items 9 and 10 (telemetry MVP, conformance loop MVP) into deliverable
|
||
slices before committing to their position. Their size scores are the least
|
||
reliable on the list, and both are strategically central.
|
||
|
||
**Done 2026-08-12, and the answer changes the order.** See "Rescoping items 9
|
||
and 10" below. The headline: item 10 was mis-sized by a wide margin, because the
|
||
conformance checker **already exists** — `railiance-infra` has a Goss baseline
|
||
suite with a `make verify` target that runs it against all hosts and commits TAP
|
||
reports to git. It has not run since **2026-03-09**, five months. The work is not
|
||
to build a loop; it is to run what exists, tighten two assertions, and schedule
|
||
it.
|
||
|
||
```task
|
||
id: RAILIANCE-WP-0016-T05
|
||
status: done
|
||
priority: medium
|
||
state_hub_task_id: "a40faaf1-77db-43f5-a774-6ab3866d8e1d"
|
||
```
|
||
|
||
Execute the S3-owned items in WSJF order — 13 (blocked on `9c21c0e0`), 14, 17 —
|
||
and keep this table current as items complete or their sizes become known.
|
||
|
||
**Done 2026-08-15.** Item 13 unblocked by `RESOURCE-WP-0002` finish: restore
|
||
already proven as T05 there; live `platform-pg-daily` backups completing.
|
||
Item 14: Bitnami postgresql-ha deploy path fail-closed (never live on
|
||
railiance01); Valkey has no S2 instance to extract and no consumer, so
|
||
deploy is gated. Item 17: `docs/s3-consumer-interfaces.md` v1.0.0.
|
||
|
||
## Outcome
|
||
|
||
T01–T05 done. Routed to `railiance-master` (message `ac73b172`); master
|
||
took items 6 and 8 as WP-0021 and declined the rest. S3-owned items 13,
|
||
14 and 17 executed 2026-08-15 after the object-store purchase landed.
|
||
Workplan finished. Remaining backlog rows belong to other repos.
|