# Server Verification

`railiance-infra` ships a declarative baseline spec and a Goss test suite that
asserts every managed node matches it. This replaces manual spot-checks with
a reproducible, CI-friendly pass/fail verdict.

## The spec

`spec/server-baseline.yaml` is executable through
`scripts/baseline_contract.py`. The dynamic inventory resolves its selected
profile into the variables consumed by both Ansible and Goss. It covers:

- **Firewall** — UFW active, default deny inbound, required ports allowed
  (SSH 22/tcp; HostEurope Nydus 2224/tcp). The k3s API (6443/tcp) is
  **tunnel-only** (ADR-005): `k3s_api_allowed_sources` is empty, and
  `k3s_api_revoked_sources` is pruned on a firewall-tagged converge. Flannel
  VXLAN (8472/udp) is omitted while the cluster is single-node; peer
  addresses go in `flannel_vxlan_allowed_sources` when a second node appears.
  `CoulombCore` selects an explicit external-firewall profile and verifies its
  replacement INPUT default-drop control without rewriting it. A host with an
  empty 6443 allowlist is recoverable over SSH.
- **SSH daemon** — root login disabled, password auth disabled, pubkey auth enabled
- **Services** — ufw, fail2ban, ssh.socket enabled and running
- **Packages** — ufw, fail2ban, git, curl, vim, htop (age and sops installed as binaries)
- **Users** — admin user with bash shell and passwordless sudo
- **Security** — fail2ban sshd jail active, HISTCONTROL=ignorespace in /etc/profile.d/

When you change governed desired state, update this file. The consumers use the
same resolved variables; `make validate-baseline` fails if either consumer is
disconnected from the contract.

## Running verification

```bash
make verify
```

This runs `ansible/playbooks/verify.yaml` against all hosts. The playbook:

1. Reads the installed Goss binary and `/etc/goss/baseline.yaml` metadata.
2. Renders the selected profile on the controller and requires its SHA-256
   digest to match the installed baseline exactly.
3. Runs the fixed `goss validate --format tap` argument vector.
4. Fails the play if the surface is missing/stale or any assertion fails.
5. Writes TAP evidence only on the controller under `reports/`.

`scripts/handoff_contract.py` statically rejects remote modules other than
`stat`, `assert`, and the exact Goss `command`. Controller writes must be
delegated to `localhost` and remain under `reports/`. It also pins every Goss
command assertion and both profile firewall probes to their reviewed read-only
set. `make verify` therefore does not install packages, rewrite files, reload
systemd, or change services on a managed host.

**All assertions passed** → exit 0
**One or more assertions FAILED** → exit non-zero, TAP report in `reports/`

## After convergence

The standard workflow after converging a new or updated node:

```bash
make converge   # bring the node to the desired state
make verify     # assert it got there
```

Run `make status` for a quick human-readable summary; run `make verify` when
you need a structured, automatable check.

If verification fails because the installed surface is stale, review the
rendered changes and use the explicit mutating interface for only the intended
host:

```bash
make verify-refresh-host \
  HOST=Railiance01 \
  APPROVE_VERIFY_REFRESH=REFRESH-GOSS-Railiance01
```

Refreshing every host requires the distinct approval value
`APPROVE_VERIFY_REFRESH=REFRESH-GOSS-ALL`. Both refresh targets install or
update Goss, its baseline, wrapper, service, and timer before running checks.

## Goss test file

`goss/baseline.yaml.j2` is rendered per host from the same inventory lists
that converge UFW. The mapping is:

| spec section | Goss resource |
|---|---|
| `firewall` | `command: ufw status` stdout patterns |
| `ssh` | `file: /etc/ssh/sshd_config.d/10-hardening.conf` contains |
| `services` | `service:` blocks |
| `packages` | `package:` blocks |
| `users` | `user:` block + `command: grep NOPASSWD` |
| `security.histcontrol` | `command: grep -r HISTCONTROL /etc/profile.d/` |
| `security.fail2ban_jails` | `command: fail2ban-client status sshd` |
| `age`, `sops` (binary installs) | `command: test -x /usr/local/bin/{age,sops}` |

## Adding new assertions

1. Add the desired state to `spec/server-baseline.yaml`.
2. If it introduces a new control kind, teach both consumers that kind.
3. Run `make validate-baseline` and the unit tests.
4. Run the applicable reviewed convergence/refresh command, then use
   `make verify-host HOST=…` as the read-only acceptance check.

An hourly on-host timer (`railiance-goss-baseline.timer`) reruns the last
rendered baseline. See `docs/conformance-loop.md`.

## Reports

TAP reports are transient under ignored `reports/`. They are machine-readable
and suitable for CI pipelines. Promote deliberately retained evidence under
`docs/evidence/` after reviewing it for safe metadata. A cleanup policy for old
reports is tracked as extension point EP `78ef4879`.