railiance-infra/docs/verification.md

Server Verification

railiance-infra ships a declarative baseline spec and a Goss test suite that asserts every managed node matches it. This replaces manual spot-checks with a reproducible, CI-friendly pass/fail verdict.

The spec

spec/server-baseline.yaml is executable through scripts/baseline_contract.py. The dynamic inventory resolves its selected profile into the variables consumed by both Ansible and Goss. It covers:

• Firewall — UFW active, default deny inbound, required ports allowed (SSH 22/tcp; HostEurope Nydus 2224/tcp). The k3s API (6443/tcp) is tunnel-only (ADR-005): k3s_api_allowed_sources is empty, and k3s_api_revoked_sources is pruned on a firewall-tagged converge. Flannel VXLAN (8472/udp) is omitted while the cluster is single-node; peer addresses go in flannel_vxlan_allowed_sources when a second node appears. CoulombCore selects an explicit external-firewall profile and verifies its replacement INPUT default-drop control without rewriting it. A host with an empty 6443 allowlist is recoverable over SSH.
• SSH daemon — root login disabled, password auth disabled, pubkey auth enabled
• Services — ufw, fail2ban, ssh.socket enabled and running
• Packages — ufw, fail2ban, git, curl, vim, htop (age and sops installed as binaries)
• Users — admin user with bash shell and passwordless sudo
• Security — fail2ban sshd jail active, HISTCONTROL=ignorespace in /etc/profile.d/

When you change governed desired state, update this file. The consumers use the same resolved variables; make validate-baseline fails if either consumer is disconnected from the contract.

Running verification

make verify

This runs ansible/playbooks/verify.yaml against all hosts. The playbook:

1. Downloads the Goss binary (pinned version) to /usr/local/bin/goss
2. Copies goss/baseline.yaml to /etc/goss/baseline.yaml on each host
3. Runs goss validate --format tap
4. Fails the play (non-zero exit) if any assertion fails
5. Fetches the TAP report to reports/goss-<host>-<timestamp>.tap
6. Auto-commits the report to git

All assertions passed → exit 0 One or more assertions FAILED → exit non-zero, TAP report in reports/

After convergence

The standard workflow after converging a new or updated node:

make converge   # bring the node to the desired state
make verify     # assert it got there

Run make status for a quick human-readable summary; run make verify when you need a structured, automatable check.

Goss test file

goss/baseline.yaml.j2 is rendered per host from the same inventory lists that converge UFW. The mapping is:

spec section	Goss resource
firewall	command: ufw status stdout patterns
ssh	file: /etc/ssh/sshd_config.d/10-hardening.conf contains
services	service: blocks
packages	package: blocks
users	user: block + command: grep NOPASSWD
security.histcontrol	command: grep -r HISTCONTROL /etc/profile.d/
security.fail2ban_jails	command: fail2ban-client status sshd
age, sops (binary installs)	command: test -x /usr/local/bin/{age,sops}

Adding new assertions

1. Add the desired state to spec/server-baseline.yaml.
2. If it introduces a new control kind, teach both consumers that kind.
3. Run make validate-baseline and the unit tests.
4. Run make converge-firewall and make verify-host HOST=… only in the appropriate reviewed live-change sequence.

An hourly on-host timer (railiance-goss-baseline.timer) reruns the last rendered baseline. See docs/conformance-loop.md.

Reports

TAP reports are committed to reports/ after each make verify run. They are machine-readable and suitable for CI pipelines. A cleanup policy for old reports is tracked as extension point EP 78ef4879.