---
agent: host-operator
engagement_id: eng-coulomb-railiance01-ho-001
project: coulomb-railiance01
last_updated: '2026-07-16'
session_count: 3
confidentiality: client_owned
---

# Host Operator Memory — railiance01 pilot

## Engagement Charter

- **Duty:** Keep railiance01 operational and secure; OS currency; load and workload review
- **Cadence:** daily health/load (business days); weekly OS/security pass
- **Change windows:** prefer low-traffic periods; reboot only with human approval
- **Escalation:** coulomb / railiance human operator (Bernd / on-call as designated)
- **Out of scope:** app features; other hosts; secret vending
- **Access:** `ssh railiance01` as tegwick (host_observe verified 2026-07-16)

## Project Context

Coulomb ecosystem production host `railiance01` runs k3s and platform workloads
(including Forgejo/apps per railiance-hosts docs). This engagement is the first
forward-deployed host-operator pilot (KAIZEN-WP-0009).

## Host Profiles

| hostname | role | typical load | services | last review |
|----------|------|--------------|----------|-------------|
| railiance01 | k3s production single-node | overload on 2 cores under current mix | k3s, gitea/forgejo, temporal, activity-core, state-hub edge, traefik, platform pods | 2026-07-16 |

## OS & Patch State

- Ubuntu 24.04.3 LTS (noble); kernel 6.8.0-87-generic (needrestart KSTA=1 — kernel current)
- Many packages upgradable (security-relevant: bind9-*, curl, ca-certificates, dpkg, …)
- `unattended-upgrades.service` flagged by needrestart for restart
- **No package upgrades applied** this session (gated)

## Security Posture

- UFW **active**: default deny in; allow OpenSSH 22, k3s API **6443/tcp Anywhere**, flannel **8472/udp Anywhere**
- Listeners include SSH, k3s components, localhost bridge ports (18000/18001), gitea stack processes
- Journal noise: reverse-forward port 18765 already in use (tunnel contention)
- Public 6443 exposure: policy review recommended (firewall_change gated)
- First snapshot: 2026-07-16 (see health report)

## Load & Workload Envelope

| Field | Value (2026-07-16 sample) |
|-------|---------------------------|
| Cores | 2 |
| Load 1/5/15 | ~12 / 11 / 16 (**saturated**) |
| RAM | 3.8 GiB; no swap; MemAvailable often <0.5 GiB |
| PSI memory full | avg60 ~24%, avg300 ~29% |
| Disk / | 54% of 96G |
| /var/log | ~5.7G (journal ~4.1G) |
| Heavy workloads | k3s server, gitea, temporal-server, activity-core worker/API, state-hub edge uvicorn, traefik, coredns |
| Saturation incident | 2026-07-16 — memory+load critical; k3s API ServiceUnavailable |

**Envelope intent:** treat load > 4 sustained or MemAvailable < 300Mi as **Watch/Critical**; escalate capacity.

## Accumulated Findings

- Undersized RAM for workload density is the dominant risk
- k3s API instability under memory pressure
- Large journald footprint
- Pending OS security updates

## What Worked

- Observe path via `ssh railiance01` (tegwick + id_custodian_agent)
- passwordless `sudo -n` for read-only needrestart/ufw/k3s inspect
- Engagement vault + close-session for durable evidence

## Watch Points

- Single-node production: privileged mistakes have full blast radius
- DinD / Actions runner privilege model (railiance-hosts ADR-004)
- Disk growth from images, logs, and backups
- Do not schedule heavy CI/agents until memory recovers
- Bridge reverse-forward port collisions (18765)

## Recurring Findings

- Memory pressure / no swap · first seen 2026-07-16 · frequency 1
- Load ≫ cores · first seen 2026-07-16 · frequency 1

## Cleared Issues

_None yet (no remediation applied)._

## Open Threads

- Human decision: add RAM and/or temporary swap
- Human approval: journal vacuum; security package upgrades; UFW source restriction for 6443
- Re-check k3s API health after memory improves
- Align inventory hostname with kernel hostname / DNS naming

## Session Log

- 2026-07-16 · railiance01 · standard_review · T09 wire-up smoke: prepare+close-session path verified (no host access) · ok
- 2026-07-16 · railiance01 · first live observe: Critical memory/load; k3s API unavailable; RU checklist complete · ok
- 2026-07-16 · railiance01 · deep_assessment · T10 ramp-up complete: Critical memory/load, k3s API unavailable, RU all done, phase operating · ok