Repurpose as the Railiance bootstrap path
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

This repo carried a copied declaration identifying it as the S2 cluster runtime
layer - byte-similar to railiance-cluster, with which it shares no git history
but nearly all of its content. It was one of four repos (bootstrap, cluster,
hosts, infra) created by copying working trees as the S1/S2 layering idea
emerged, and the INTENT was copied along with everything else.

Its own QUICKSTART.md - set up your very first Railiance host - describes what
the repo is actually for. INTENT.md and SCOPE.md now declare that: turning a
single bare virtual machine into a working Railiance, then handing off to the
ordinary layers.

Inherited S2-era content in ansible/, helm/, docs/ and wiki/ is not yet sorted;
SCOPE records that restructuring is in progress rather than claiming it is done.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-11 22:41:29 +02:00
parent 607f3960cf
commit 0814ed3a19
2 changed files with 148 additions and 132 deletions

144
INTENT.md
View file

@ -9,25 +9,31 @@
## One-liner
**The cluster runtime layer — turning hardened servers into a healthy, ready-to-use orchestration platform for workloads.**
**The bootstrap path — turning a single bare virtual machine into a working Railiance, so that standing up a new Railiance is a repeatable procedure rather than an act of memory.**
---
## Why This Exists
A hardened set of servers is not yet somewhere workloads can run.
Something must install and configure the **orchestration runtime** — its
scheduler, networking, ingress, admission controls, and the operators that
extend it — and then **prove the cluster is healthy**.
Railiance's layers each describe how to operate a system that **already
exists**. S1 converges a substrate, S2 runs a cluster on it, S3 provides shared
services, and so on. Every one of them assumes the thing beneath it is already
there.
Without a disciplined runtime layer:
Nothing owns the first step: *you have one virtual machine and an intention —
now what?*
* cluster configuration drifts between environments,
* extension and addon boundaries blur,
* and higher layers deploy onto an unproven runtime.
Without a bootstrap path:
This layer exists to provide that runtime **consistently and verifiably**,
so the layers above can deploy onto a known-good cluster.
* standing up a new Railiance depends on the memory of whoever did it last,
* the ordering between substrate, cluster, secrets, and forge is rediscovered
each time,
* the chicken-and-egg problems (secrets needed to fetch the code that manages
secrets) are solved ad hoc and differently each time,
* and "Railiance is easy to adopt" stays an aspiration nobody can test.
This repo exists so that a new Railiance can be **initiated from nothing**, by
someone who was not present when the first one was built.
---
@ -35,75 +41,78 @@ so the layers above can deploy onto a known-good cluster.
> *Where we are going.*
To become the **canonical home for the cluster runtime** — installation
and baseline configuration of the orchestrator, its networking and
ingress, admission controls, cluster-level operators and addons, and
runtime access — operated to a **verified-healthy** standard.
To become the **canonical entry point into Railiance** — the one repository a
newcomer clones to turn a bare machine into a Railiance that can then manage
itself through the ordinary layers.
This means:
* The runtime is configured to a **consistent baseline** every time
* Cluster health is **proven by tests**, not assumed
* Capabilities are extended through **operators and addons** behind clear
boundaries
* Runtime **access is managed and rotatable**
* The path from **one VM to a working Railiance** is a documented, runnable
sequence
* **Ordering and prerequisites** between layers are explicit, not folklore
* Bootstrap **chicken-and-egg problems** are named and solved deliberately
* The procedure is **testable** — a new Railiance can be raised from scratch to
prove the path still works
* Once bootstrapped, the system **hands off** to the ordinary layers and this
repo steps out of the way
---
## Core Principles
### 1. Runtime, Not Workloads
### 1. From Nothing, Repeatably
Provide the place where things run. Do not own the things that run there.
The path assumes no pre-existing Railiance, no prepared secrets, and no
institutional memory. If a step depends on something that already exists, that
dependency is stated.
### 2. Healthy by Verification
### 2. Hand Off, Do Not Hold
Cluster health is demonstrated by smoke tests and checks, never assumed.
This repo raises a Railiance and then yields. Ongoing operation belongs to S1S5.
Anything this repo keeps operating is a boundary violation.
### 3. Consistent Baseline
### 3. The Path Is Tested by Walking It
The runtime is brought up the same way every time, so environments stay
comparable and predictable.
A bootstrap procedure that has not been run from scratch recently is a
hypothesis. Proving it means raising a new Railiance, not reviewing the
document.
### 4. Built on a Verified Substrate
### 4. Honest About Manual Steps
Assumes a converged, hardened foundation beneath it; it does not reach
down and reconfigure that foundation.
Some bootstrap steps genuinely require a human — provisioning a machine,
entering a root credential. These are named as such rather than hidden behind
automation that pretends otherwise.
### 5. Extensible by Operators
### 5. Adoption Is the Point
Cluster capabilities are added through operators and addons within clear
boundaries, not by ad-hoc mutation.
### 6. Managed Access
Access to the runtime is controlled, auditable, and rotatable.
Ease of initiation is a Railiance goal, not a convenience. A Railiance that only
its author can stand up is not a framework.
---
## What This Is (Conceptually)
This layer is:
The **ignition sequence** for a Railiance:
* a **cluster runtime** layer
* an **orchestrator installation and configuration**
* a **networking, ingress, and admission** baseline
* a host for **cluster-level operators and addons**
* a **health-verification gate**
* **runtime access** management
* the newcomer quickstart and its prerequisites
* the ordering contract between substrate, cluster, secrets, and forge
* bootstrap secret material handling, and how it is replaced by the real custody
path once one exists
* the handoff point where the ordinary layers take over
* a test that the whole path still works
---
## What This Is Not
This layer is not:
* the infrastructure substrate beneath it
* a provider of shared, stateful platform services
* an application or business-capability provider
* an owner of the workloads it runs
It is the **runtime an entire landscape's workloads depend on**.
* **Not the cluster runtime.** k3s, Helm, ingress, CNI, operators and kubeconfig
management belong to `railiance-cluster` (S2). This repo previously declared
itself S2 — that was a copied declaration, not an intent.
* **Not the infrastructure substrate.** OS provisioning, hardening and baseline
belong to `railiance-infra` (S1).
* **Not a second home for layer content.** It sequences the layers; it does not
duplicate what they own.
* **Not an ongoing operations repo.** After handoff, it has no running role.
---
@ -111,14 +120,33 @@ It is the **runtime an entire landscape's workloads depend on**.
This layer is expected to evolve toward:
* Stronger, continuous **health verification**
* Smoother, safer **runtime upgrades**
* Clearer **operator and addon** boundaries
* More robust **access rotation**
* Self-evidencing, **auditable** runtime state
* A **single documented path** from bare VM to working Railiance
* Explicit **prerequisite and ordering** contracts with S1 and S2
* Deliberate handling of **bootstrap secret material** and its retirement
* A **rehearsable** bootstrap — proven by raising a fresh Railiance, not by review
* Reduced manual steps over time, with the remaining ones **named honestly**
* Eventually, bootstrapping onto **substrates other than the current one**
without changing the model
---
## History
This repository previously carried a copied declaration identifying it as the
S2 cluster runtime layer — identical in intent to `railiance-cluster`, with
which it shares no git history but nearly all of its content. It was one of four
repos (`bootstrap`, `cluster`, `hosts`, `infra`) created by copying working
trees as the S1/S2 layering idea emerged.
That declaration was wrong. The repository's own `QUICKSTART.md` — *"set up your
very first Railiance host"* — describes what this repo is actually for, and this
INTENT now states it.
See `railiance-platform/ArchitectureBlueprint.md` §4 for the full lineage and
`history/2026-08-11-railiance-architecture-in-aspiration.md` for the assessment.
---
## Guiding Question
> **How can the runtime an entire landscape's workloads depend on be made consistently healthy, upgradable, and trustworthy?**
> *Could someone who was not there raise a new Railiance from this?*

136
SCOPE.md
View file

@ -8,120 +8,108 @@
## One-liner
S2 Cluster Runtime layer of the Railiance OAS Stack — owns k3s installation, Helm, ingress, CNI, admission controllers, operators, and kubeconfig management.
The Railiance bootstrap path — turning a single bare virtual machine into a working Railiance, then handing off to the ordinary layers.
---
## Core Idea
Railiance is structured as five independent repos per OAS Stack layer. This repo is S2. It installs and configures the Kubernetes cluster runtime: k3s, Helm, ingress controller, CNI plugin, cluster addons and operators. S1 (OS) must be converged before S2 can run. S2 explicitly does not own platform services (PostgreSQL, caches) — those are S3.
Every Railiance layer describes how to operate a system that already exists.
S1 converges a substrate, S2 runs a cluster on it, S3 provides shared services.
None of them owns the first step: *you have one virtual machine and an
intention — now what?*
This repo owns that step. It sequences the layers into a runnable path from
nothing to a working Railiance, and then gets out of the way. One of Railiance's
goals is that initiating a new Railiance should be easy; this repo is where that
claim is made testable.
**Repository status:** this repo previously carried a copied declaration
identifying it as the S2 cluster runtime layer, duplicating `railiance-cluster`.
That was wrong and has been corrected — see `INTENT.md` → History. Restructuring
of the inherited content is in progress; expect S2-era material still present in
`ansible/`, `helm/`, `docs/` and `wiki/` that has not yet been sorted into
"bootstrap path" versus "belongs to S1/S2".
---
## In Scope
- k3s installation and baseline configuration
- Helm chart management
- Ingress controller, CNI plugin
- Admission controllers and cluster operators
- Cluster addons (cert-manager, etc.)
- kubeconfig management and access
- Smoke tests to validate cluster health
- The newcomer path from bare VM to working Railiance (`QUICKSTART.md`)
- Prerequisite and ordering contracts between substrate, cluster, secrets, forge
- Bootstrap secret material: how it is introduced, and how it is retired once
the real custody path (OpenBao, S3) exists
- The handoff point where the ordinary layers take over
- A rehearsable test that the bootstrap path still works
---
## Out of Scope
- OS security hardening, SSH, firewall → railiance-infra (S1)
- Platform services (PostgreSQL HA, Valkey, object storage) → railiance-platform (S3)
- CI/CD and developer tooling → railiance-enablement (S4)
- Application deployments → railiance-apps (S5)
- No re-configuration of S1 concerns from this repo
- OS provisioning, hardening, server baseline → `railiance-infra` (S1)
- k3s, Helm, ingress, CNI, admission controllers, operators, kubeconfig
management → `railiance-cluster` (S2)
- Shared stateful services: databases, cache, secrets, object storage
`railiance-platform` (S3)
- Ongoing operation of anything. After handoff this repo has no running role
- Duplicating layer content — this repo sequences layers, it does not own what
they own
---
## Relevant When
- Setting up or maintaining the Kubernetes cluster runtime
- Installing or updating cluster-level operators and addons
- Diagnosing cluster health (smoke tests)
- k3s upgrades or kubeconfig rotation
---
- Standing up a **new** Railiance from nothing
- Establishing or changing the ordering between substrate, cluster and secrets
- Resolving a bootstrap chicken-and-egg problem (needing a secret to fetch the
code that manages secrets)
- Testing that the adoption path still works by raising a fresh Railiance
## Not Relevant When
- OS-level work (use railiance-infra)
- Platform service configuration (use railiance-platform)
- Application deployments (use railiance-apps)
- Operating, upgrading, or troubleshooting an existing Railiance — that belongs
to the layer that owns the concern
- Deploying a workload (that is `rapp-*` and S5)
---
## Current State
- Status: active / stable
- Implementation: k3s baseline, pgpool HA failover fix, age-encrypted backup, kubeconfig delivery, staged promotion lifecycle, and activity-core/llm-connect reconcile gates all finished (RAIL-BS-WP-0002…0006, RAILIANCE-WP-0012…0014)
- Open work: RAIL-BS-WP-0007 ThreePhoenix HA cluster (active, 0/7); RAIL-BS-WP-0008 activity-core WP-0016 deploy (ready); RAIL-BS-WP-0009 admin-sync smoke (ready)
- Usage: core Kubernetes runtime for all Railiance deployments; runs on COULOMBCORE (92.205.130.254)
- Also deployed at cluster level: cert-manager, ArgoCD, CloudNative PG operator (cnpg), nginx ingress, SSO stack (mfa + sso namespaces via net-kingdom)
- Status: **repurposed, restructuring in progress**
- The bootstrap intent is declared (`INTENT.md`, 2026-08-11); the inherited S2
content has not yet been sorted
- `QUICKSTART.md` is the existing seed of the newcomer path; a near-identical
copy also exists in `railiance-cluster` and the two need consolidating
- `install/` exists in `railiance-cluster` but not here, and is a candidate to
move
- Open: whether the bootstrap path has ever been walked end to end from scratch
---
## How It Fits
- Upstream dependencies: railiance-infra (S1) — OS must be converged and verified
- Downstream consumers: railiance-platform (S3), railiance-enablement (S4), railiance-apps (S5)
- Often used with: railiance-platform (next layer to configure after cluster is up)
- Sequences: `railiance-infra` (S1) → `railiance-cluster` (S2) →
`railiance-platform` (S3), and hands off once they can run themselves
- Downstream: everything, indirectly — this is the entry point
- Often used with: `railiance-master` (which owns the layer model this
sequences), `railiance-platform` (which takes over secret custody after
bootstrap material is retired)
---
## Terminology
- Preferred terms: OAS Stack Level S2, smoke test, pre-condition chain, boundary rule
- Potentially confusing terms: cluster runtime ≠ platform services; Gitea and databases are NOT S2 concerns
---
## Related / Overlapping
- `railiance-infra` (S1) — must be converged before this layer runs
- `railiance-platform` (S3) — consumes the cluster runtime provided by S2
- Preferred terms: bootstrap path, ignition sequence, handoff point,
bootstrap secret material
- Potentially confusing: "bootstrap" here means *initiating a whole Railiance*,
not `cluster bootstrap` in the k3s sense (that is S2), and not ArgoCD
app-of-apps bootstrapping (that is S3's `argocd/bootstrap/`)
---
## Getting Oriented
- Start with: `CLAUDE.md` (session protocol, remote execution via SSH tunnel), `README.md`
- Key files / directories: `workplans/` (4 active), `.sops.yaml` (secret encryption)
- Entry points: `Makefile` targets; remote work requires SSH tunnel to State Hub
---
## Provided Capabilities
```capability
type: infrastructure
title: Kubernetes cluster provisioning (k3s)
description: Install and configure a production k3s cluster including Helm, ingress controller, CNI plugin, and kubeconfig management on Railiance servers.
keywords: [kubernetes, k3s, cluster, helm, ingress, cni, k8s, provisioning]
```
```capability
type: infrastructure
title: Cluster operators and addon management
description: Deploy and manage cluster-wide operators and addons (cert-manager, CloudNative PG operator, ArgoCD, nginx ingress) on the running Railiance Kubernetes cluster.
keywords: [operator, addon, cert-manager, cnpg, argocd, admission, kubernetes, cluster]
```
```capability
type: operations
title: Kubernetes runtime backup (age-encrypted)
description: Daily encrypted backup of k3s cluster state (SQLite hot copy), Helm release values, and kubeconfig to /opt/backup/railiance/cluster/ using age encryption. Run via sudo make backup.
keywords: [backup, restore, age, encryption, k3s, state, helm, kubeconfig, disaster-recovery]
```
---
## Notes
Runs on COULOMBCORE (92.205.130.254). State Hub access via ops-bridge reverse tunnel — `bridge up state-hub-coulombcore` from the workstation (see ADR-004). Gitea Helm values were migrated to S5 (railiance-apps) in RAIL-HO-WP-0004-T06 — boundary violation resolved.
- Start with: `INTENT.md`, then `QUICKSTART.md`
- Context: `railiance-platform/ArchitectureBlueprint.md` §4 records this repo's
lineage and status
- Pre-conditions: one reachable virtual machine, and nothing else