railiance-cluster/SCOPE.md

# SCOPE

> This file helps you quickly understand what this repository is about,
> when it is relevant, and when it is not.
> It is intentionally lightweight and may be incomplete.

---

## One-liner

S2 Cluster Runtime layer of the Railiance OAS Stack - owns k3s installation,
Helm, ingress, CNI, admission controllers, operators, kubeconfig management,
and the substrate facts published to downstream workload rails.

---

## Core Idea

Railiance is structured as five independent repos per OAS Stack layer. This repo
is S2. It installs and configures the Kubernetes cluster runtime: k3s, Helm,
ingress controller, CNI plugin, cluster addons and operators. S1 (OS) must be
converged before S2 can run. S2 explicitly does not own platform services
(PostgreSQL, caches) - those are S3.

Some generic workload lifecycle docs and commands still live here as a temporary
compatibility layer. That material is transition debt pending extraction to a
future `rail-kubernetes` repo and must not be treated as long-term S2 scope.

---

## In Scope

- k3s installation and baseline configuration
- Helm chart management
- Ingress controller, CNI plugin
- Admission controllers and cluster operators
- Cluster addons (cert-manager, etc.)
- kubeconfig management and access
- Smoke tests to validate cluster health
- Published substrate facts and safety expectations for downstream workload rails

---

## Out of Scope

- OS security hardening, SSH, firewall -> railiance-infra (S1)
- Platform services (PostgreSQL HA, Valkey, object storage) -> railiance-platform (S3)
- CI/CD and developer tooling -> railiance-enablement (S4)
- Application deployments -> railiance-apps (S5)
- Generic workload lifecycle semantics, `railiance/app.toml`, and wrapper scaffolding as long-term ownership concerns -> future `rail-kubernetes` / `rapp-*`
- No re-configuration of S1 concerns from this repo

---

## Relevant When

- Setting up or maintaining the Kubernetes cluster runtime
- Installing or updating cluster-level operators and addons
- Diagnosing cluster health (smoke tests)
- k3s upgrades or kubeconfig rotation
- Defining what the current Kubernetes substrate guarantees to downstream workload rails

---

## Not Relevant When

- OS-level work (use railiance-infra)
- Platform service configuration (use railiance-platform)
- Application deployments (use railiance-apps)

---

## Current State

- Status: active / stable
- Implementation: k3s baseline, age-encrypted backup, kubeconfig delivery, and cluster operator/addon management are established; generic workload lifecycle docs/tooling remain here as transition debt pending `rail-kubernetes` extraction
- Capacity evidence collectors are implemented under RCLUSTER-WP-0014.
  Current work status is indexed in WORK-RECORDS.md and workplans/.
- Usage: core Kubernetes runtime for all Railiance deployments. **Two clusters
  are managed**: `railiance01` (92.205.62.239) is the current primary and runs
  the live workloads; `CoulombCore` (92.205.130.254) is the older cluster and is
  under a production freeze
- Verified on `railiance01` 2026-08-12 (k3s v1.35.1+k3s1, node Ready):
  cert-manager, CloudNative PG operator (cnpg), **Traefik** ingress (k3s
  default, in `kube-system`), Kourier + Knative Serving, External Secrets,
  OpenBao, SSO stack (`mfa` + `sso` via net-kingdom)
- **Not** on `railiance01`: ArgoCD (it runs on CoulombCore only) and nginx
  ingress (Traefik is the controller). Earlier revisions of this file listed
  both — the ArgoCD entry was accurate for CoulombCore, the nginx entry was not
- CNPG clusters on `railiance01`: `apps-pg`, `forgejo-db`, `net-kingdom-pg`,
  `platform-pg`, `state-hub-db`, `target-revenue-pg`, `user-engine-pg`
- `rapp-telemetry` records a privately deployed Prometheus/Alertmanager/Grafana
  package on railiance01. `railiance-telemetry` owns Q2 signal contracts and
  reference adapters; its external delivery acceptance remains separate.
  This is a package evidence reference, not a fresh live health observation.
- k3s API access is tunnel-only on Railiance01; see `railiance-infra`
  `RAIL-HO-WP-0009` and ADR-005. It is not a public/source-allowlisted surface.
- Admin credential file modes (RCLUSTER-WP-0019, done 2026-10-20): the
  installer writes the admin kubeconfig with mode 600 and no longer exports it.
  `ansible/k3s-kubeconfig-mode.yml`, applied on railiance01, keeps
  the admin kubeconfig and `registries.yaml` at root 0600.
  `registries.yaml`, which holds a registry login, had been found at 0644.
  Restricted agent accounts on the node therefore cannot read cluster-admin or
  registry credentials.

---

## How It Fits

- Upstream dependencies: railiance-infra (S1) - OS must be converged and verified
- Downstream consumers: railiance-platform (S3), railiance-enablement (S4), railiance-apps (S5)
- Often used with: railiance-platform (next layer to configure after cluster is up)

---

## Terminology

- Preferred terms: OAS Stack Level S2, smoke test, pre-condition chain, boundary rule
- Potentially confusing terms: cluster runtime != platform services; cluster runtime != generic workload rail; Gitea and databases are not S2 concerns

---

## Related / Overlapping

- `railiance-infra` (S1) - must be converged before this layer runs
- `railiance-platform` (S3) - consumes the cluster runtime provided by S2

---

## Getting Oriented

- Start with: `CLAUDE.md`, `README.md`
- Key files / directories: `workplans/`, `.sops.yaml`, `docs/rail-kubernetes-extraction-map.md`, `docs/rail-kubernetes-substrate-profile.md`
- Entry points: `Makefile` targets; remote work requires SSH tunnel to State Hub

---

## Provided Capabilities

```capability
type: infrastructure
title: Kubernetes cluster provisioning (k3s)
description: Install and configure a production k3s cluster including Helm, Traefik ingress, networking, and kubeconfig management on Railiance servers. Admin kubeconfig and registries.yaml are kept root 0600 (ansible/k3s-kubeconfig-mode.yml); the installer does not export the admin kubeconfig.
keywords: [kubernetes, k3s, cluster, helm, ingress, cni, k8s, provisioning]
```

```capability
type: infrastructure
title: Cluster operators and addon management
description: Deploy and manage cluster-wide operators and addons, including cert-manager, CloudNative PG, Traefik, Knative Serving, External Secrets, and OpenBao integration, on the running Kubernetes cluster.
keywords: [operator, addon, cert-manager, cnpg, traefik, knative, admission, kubernetes, cluster]
```

```capability
type: operations
title: Kubernetes runtime backup (age-encrypted)
description: Daily encrypted backup of k3s cluster state (SQLite hot copy), Helm release values, and kubeconfig to /opt/backup/railiance/cluster/ using age encryption. Run via sudo make backup.
keywords: [backup, restore, age, encryption, k3s, state, helm, kubeconfig, disaster-recovery]
```

---

## Notes

The live primary is Railiance01 (`92.205.62.239`). CoulombCore
(`92.205.130.254`) is the older frozen cluster and remains a legacy recovery
surface, not the default deployment target. State Hub and Kubernetes API access
use the corresponding ops-bridge tunnels (see ADR-004/ADR-005).

`bin/railiance` lifecycle subcommands, `railiance/app.toml`, and overlay/wrapper
scaffolding stay in this repo only as a migration bridge. They are not the
steady-state S2 identity.


```capability
type: operations
title: Kubernetes capacity and allocation observation
description: Capture timestamped non-secret cluster capacity, scheduler-effective allocation, workload utilization and measurement gaps with make cluster-observe REMOTE=railiance01. Retain observations for resource-control and release preflight; residual requests are not measured spare processing capacity or admission.
keywords: [capacity, allocation, kubernetes, utilization, sizing, evidence]
```


## Generated work-record index

This repository opts into staged-source WORK-RECORDS.md generation through
`.generated-work-records.json` and the isolated `.generated-work-records.mk`
recipe. `make generated-work-records` reuses the State Hub renderer through
the Custodian adapter; `make generated-work-records-check` verifies byte-exact
agreement with staged Markdown. The pre-commit hook uses the isolated recipe,
preserves unstaged index edits by refusing them, and retains the generated-only
commit guard and existing security checks. It does not contact the hub, alter
workplan statuses or grant deployment authority. This consumes shared tooling;
it does not make this repository the provider of the generator capability.

## Local orientation brief policy — 2026-10-19

`.custodian-brief.md` is a local, ignored State Hub orientation snapshot. Its
existing bytes remain available for offline session startup; refreshing it does
not require a Git commit. Authored workplans and the generated WORK-RECORDS index
retain their existing source and staged-generation contracts. No runtime or
credential change follows from this repository hygiene policy.