Make the k3s API tunnel-only (ADR-005), stop declaring Flannel VXLAN
open to Anywhere, tag the base role so firewall can be scoped, and
schedule the Goss declared-vs-live check. CoulombCore sets ufw_manage
false so a converge cannot enable UFW there. T02 still needs operator
approval for make converge-firewall HOST=Railiance01.
The conformance checker existed but had not run since 2026-03-09, and its
firewall assertion matched /6443\/tcp.*ALLOW/ - asserting the port was allowed
but not from whom. It passed identically whether the API was restricted to one
operator address or open to the entire internet, which is precisely the drift
that went undetected.
goss/baseline.yaml is now a template rendered per host from the same inventory
declaration that converges the host, so the assertion cannot drift from what it
checks. It asserts each declared source is present, that no revoked source
retains access, and that 6443 is never ALLOW Anywhere. verify.yaml gained the
vars_files the bootstrap play already had - without it the firewall assertions
rendered empty and silently asserted nothing, which is worse than no check.
Result on Railiance01: 32 assertions, 31 pass. The one failure is a real find
and is recorded as T06 - Flannel VXLAN is declared open to Anywhere with no
source restriction, the same defect pattern as the k3s rule. It is currently
absent from the host, so converging would INTRODUCE the exposure rather than
fix drift. Must be resolved before RAIL-BS-WP-0007 multi-node HA needs working
VXLAN.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
89.244.90.255 appeared in the live allowlist mid-session. Verified legitimate:
SSH pubkey auth as tegwick from that address on 2026-08-02, and UFW BLOCK
entries on 6443 immediately before it was granted. Now declared.
Both stale grants are deleted from the live host, so the security goal of T02 is
met and the live 6443 allowlist matches the declaration exactly.
Full convergence deliberately held: --check reports 11 changes, mostly unrelated
to the firewall, including MemoryMax=1500M on user-1000.slice which could OOM
running agent workloads, plus an sshd restart. That is a separate decision, and
the base role has no tags so convergence cannot be scoped to UFW alone. Records
both follow-ons.
The allowlist drifting again by hand during the session that was fixing
allowlist drift is now recorded as the strongest argument for T04, removing the
API from the public internet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The live host restricted 6443/tcp to specific operator addresses, added by hand,
while this role still declared the port open to Anywhere with no source
restriction. The declared config was weaker than reality: running the base role
would have REMOVED the restriction and exposed the Kubernetes API to the
internet. Security was tightened on the host and never fed back into the source
of truth.
Found 2026-08-11 while diagnosing lost cluster access, which turned out to be an
ISP lease rotation (89.244.90.246 -> .236) against a hand-maintained allowlist.
Changes:
- defaults: k3s_api_allowed_sources (empty = 6443 closed to all external
sources, the safe failure; SSH unaffected so the host stays recoverable) and
k3s_api_revoked_sources, so rotated addresses are pruned rather than left as
standing grants to whoever the ISP reassigns them to
- tasks: grant approved sources, then remove any blanket rule, then revoke
retired ones. Order matters - grants are added before the blanket rule is
deleted so convergence never opens a window with no API access
- group_vars/all.yaml: the current operator address, plus the two stale grants
(.246 rotated, 85.132.220.102 historic) marked for revocation
- docs/verification.md: state that 6443 is source-restricted rather than
listing it as a plainly allowed port
Not yet converged against the live host - the role change is committed but
running it is a production action needing operator approval.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add ops_bridge_pubkey to group_vars/all.yaml (public key only, safe to
commit) and inject it via ansible.posix.authorized_key in the base role,
immediately after SSH hardening. This ensures ops-bridge tunnel
connectivity is available as soon as SSH infrastructure is up on any
managed host — no manual key provisioning required for new nodes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- base role: allow UFW routing direction (required for k3s flannel
pod networking to function across nodes)
- docs/deploy-stack.md: full S1→S5 ordered deploy runbook with
pre-conditions checklist and layer-by-layer steps
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ansible/playbooks/custodian-agent.yaml: minimal playbook with only
the custodian_agent role — avoids loading base/sops_agent/etc when
all we need is key injection
- Makefile: use custodian-agent.yaml in provision targets; remove
--tags workaround (was fragile; standalone playbook is correct)
Manual invocation (from CoulombCore):
cd ~/railiance-infra/ansible
ansible-playbook playbooks/custodian-agent.yaml -u tegwick --limit Railiance01
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC/V9fe5MGKdhTBz9KwEvC1NE+HjdoCtQocpGxP6Pko9
Generated 2026-03-27 via make custodian-keygen. Private key at workstation
only (~/.ssh/id_custodian_agent), never committed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Establishes a dedicated SSH keypair for the Custodian automation agent:
- ansible/roles/custodian_agent/: authorized_key task (tagged custodian_agent)
- ansible/inventory/group_vars/all.yaml: custodian_agent_user/pubkey vars
- ansible/playbooks/bootstrap.yaml: custodian_agent role added
- Makefile: provision-custodian-agent / provision-custodian-agent-host targets
Keypair generation: cd ~/the-custodian && make custodian-keygen
Then deploy: cd ~/railiance-infra && make provision-custodian-agent
The private key lives at ~/.ssh/id_custodian_agent — never committed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Include time in TAP report filename (ISO 8601: date + HHmmssZ)
- Add changed_when: false to report write task — verify play now shows
changed=0 on a clean run (all green recap)
- make verify auto-commits new reports to repo after a passing run;
exits non-zero before committing if assertions fail
- Register EP-RAIL-001: report pruning extension point for future
implementation when reports/ accumulates beyond a threshold
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes found by running make verify against Railiance01:
- Fix playbook_dir paths (ansible/playbooks/ is 2 levels from repo root)
- age/sops are binary installs, not apt packages — use command checks
- Admin user is tegwick, not admin; sudoers at /etc/sudoers.d/tegwick
- sudo granted via sudoers file, not group membership — remove group assert
- Ubuntu 24.04 socket-activates SSH; assert ssh.socket not ssh.service
- SSH hardening lives in sshd_config.d/10-hardening.conf, not main config
- UFW SSH rule uses app name "OpenSSH", not port 22/tcp
- Replace /regex/i patterns with plain strings (Goss file.contents)
- Update spec/server-baseline.yaml to match all findings
All 27 assertions now pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extend base role with fail2ban, UFW k3s/Flannel rules, HISTCONTROL
- Add handlers dir for fail2ban restart
- Fix inventory script to emit correct dynamic inventory JSON format
- Add roles_path to ansible.cfg so playbook finds roles
- Add Railiance01 (92.205.62.239) to inventory/servers.yaml
- Mark workplan T03/T04/T05 as done
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
