ops: note railiance01 capacity upgrade claim vs live guest size
Some checks failed
CI Smoke / container-smoke (push) Successful in 2s
ci / test (push) Failing after 5s
CI Smoke / host-smoke (push) Successful in 0s

Operator reported 4 vCPU / 16 GiB; verify shows guest still 2 / 3.8 GiB
after recent reboot. Document check report and keep open thread until
nproc/free match the new flavor.
This commit is contained in:
tegwick 2026-07-16 14:33:48 +02:00
parent 06037b0486
commit 776b7f8cc4
7 changed files with 82 additions and 29 deletions

View file

@ -1,3 +1,4 @@
{"agent": "host-operator", "duty": "standard_review", "engagement_id": "eng-coulomb-railiance01-ho-001", "execution_time_s": 5.0, "phase": "staffing", "quality_score": 0.7, "success": true, "timestamp": "2026-07-16T10:08:57Z"}
{"agent": "host-operator", "duty": "deep_assessment", "engagement_id": "eng-coulomb-railiance01-ho-001", "execution_time_s": 900.0, "phase": "operating", "quality_score": 0.9, "success": true, "timestamp": "2026-07-16T10:42:51Z"}
{"agent": "host-operator", "duty": "deep_assessment", "engagement_id": "eng-coulomb-railiance01-ho-001", "execution_time_s": 1200.0, "phase": "operating", "quality_score": 0.88, "success": true, "timestamp": "2026-07-16T11:05:17Z"}
{"agent": "host-operator", "duty": "standard_review", "engagement_id": "eng-coulomb-railiance01-ho-001", "execution_time_s": 60.0, "phase": "operating", "quality_score": 0.7, "success": true, "timestamp": "2026-07-16T12:33:33Z"}

View file

@ -1,9 +1,9 @@
{
"agent": "host-operator",
"avg_execution_time_s": 701.667,
"avg_quality_score": 0.827,
"execution_count": 3,
"last_execution": "2026-07-16T11:05:17Z",
"avg_execution_time_s": 541.25,
"avg_quality_score": 0.795,
"execution_count": 4,
"last_execution": "2026-07-16T12:33:33Z",
"success_rate": 1.0,
"trend": {
"quality_score": "stable",

View file

@ -2,3 +2,4 @@
{"access_surcharge_product": "read_only", "account": "coulomb-ops-kai", "amount_kai": 1600, "apiVersion": "kaizen.agentic/v1", "capability_tier": 4, "created_at": "2026-07-16T10:08:57Z", "currency": "KAI", "engagement_id": "eng-coulomb-railiance01-ho-001", "id": "kai-20260716T100857Z-standard_review", "kind": "KaiLedgerEntry", "metadata": {"access_class": "read_only", "phase": "staffing", "success": true, "target": "railiance01"}, "product": "standard_review", "session_ref": "reports/2026-07-16-standard-review.md", "type": "duty_charge"}
{"access_surcharge_product": "host_observe", "account": "coulomb-ops-kai", "amount_kai": 4900, "apiVersion": "kaizen.agentic/v1", "capability_tier": 4, "created_at": "2026-07-16T10:42:51Z", "currency": "KAI", "engagement_id": "eng-coulomb-railiance01-ho-001", "id": "kai-20260716T104251Z-deep_assessment", "kind": "KaiLedgerEntry", "metadata": {"access_class": "host_observe", "phase": "operating", "success": true, "target": "railiance01"}, "product": "deep_assessment", "session_ref": "reports/2026-07-16-deep-assessment.md", "type": "duty_charge"}
{"access_surcharge_product": "privileged_ops", "account": "coulomb-ops-kai", "amount_kai": 5200, "apiVersion": "kaizen.agentic/v1", "capability_tier": 4, "created_at": "2026-07-16T11:05:17Z", "currency": "KAI", "engagement_id": "eng-coulomb-railiance01-ho-001", "id": "kai-20260716T110517Z-deep_assessment", "kind": "KaiLedgerEntry", "metadata": {"access_class": "privileged_ops", "phase": "operating", "success": true, "target": "railiance01"}, "product": "deep_assessment", "session_ref": "reports/2026-07-16-deep-assessment.md", "type": "duty_charge"}
{"access_surcharge_product": "host_observe", "account": "coulomb-ops-kai", "amount_kai": 1700, "apiVersion": "kaizen.agentic/v1", "capability_tier": 4, "created_at": "2026-07-16T12:33:33Z", "currency": "KAI", "engagement_id": "eng-coulomb-railiance01-ho-001", "id": "kai-20260716T123333Z-standard_review", "kind": "KaiLedgerEntry", "metadata": {"access_class": "host_observe", "phase": "operating", "success": true, "target": "railiance01"}, "product": "standard_review", "session_ref": "reports/2026-07-16-standard-review.md", "type": "duty_charge"}

View file

@ -0,0 +1,50 @@
# Capacity upgrade check — railiance01 — 2026-07-16
**Engagement:** eng-coulomb-railiance01-ho-001
**Operator report:** VM upgraded to **4 vCPU / 16GiB RAM**
**Agent verify:** observe via `ssh railiance01`
## Result: not yet visible in the guest OS
| Resource | Operator target | Live guest (`/proc`, `lscpu`) |
|----------|-----------------|------------------------------|
| vCPU | 4 | **2** (`nproc` / `lscpu`) |
| RAM | 16GiB | **~3.8GiB** (`MemTotal` ≈ 4009884 kB) |
| Swap | (our /swapfile) | 4GiB file, ~300Mi used |
| Uptime at check | — | ~12 minutes (recent reboot) |
| Hypervisor | — | KVM / OpenStack Nova (ConfigDrive) |
| k3s node | — | **Ready** |
## Guest health snapshot (at check)
- Load ~2 on 2 cores (much better than Critical era load ~12)
- MemAvailable ~0.9GiB; PSI memory full avg60 ~0.2% (comfortable **for current size**)
- Swap lightly used (~300Mi)
So the **workload is calmer after reboot**, but the **instance size has not expanded** to 4/16 from the kernels point of view.
## Likely causes
1. Resize still pending or applied to a different VM/instance
2. Provider resize requires a further console action / hard reboot after flavor change
3. Resize failed partially (boot on old flavor)
4. Wrong host checked (we used SSH host alias `railiance01` → 92.205.62.239)
## Recommended operator steps (provider / OpenStack)
1. Confirm in HostEurope / OpenStack console that **this** instance (IP 92.205.62.239) shows flavor **4 vCPU / 16GiB**
2. If resize is `VERIFY_RESIZE` / confirm-pending, **confirm** it
3. Hard reboot if the panel says so after resize
4. Re-check from shell:
```bash
ssh railiance01 'nproc; free -h; lscpu | grep -E "CPU\\(s\\)|Model"'
# expect: CPU(s): 4 and Mem ~1516Gi
```
5. Optional later: reduce reliance on 4Gi swapfile once 16Gi is confirmed stable (keep small swap as cushion, or leave as-is)
## Agent action
- Vault/baseline **not** updated to 4/16 until live numbers match
- Open thread retained: “confirm capacity upgrade applied to guest”

View file

@ -4,10 +4,10 @@
- **Duty:** standard_review
- **Targets:** railiance01
- **Outcome:** success
- **Phase:** staffing
- **Phase:** operating
## Summary
T09 wire-up smoke: prepare+close-session path verified (no host access)
Operator reported 4CPU/16G upgrade; live guest still 2CPU/3.8G after reboot; k3s Ready; load calmer
_Billing metadata only in commercial/ledger.jsonl; no secrets in this report._

View file

@ -2,9 +2,7 @@
| Risk | Severity | Mitigation / owner | Status |
|------|----------|--------------------|--------|
| RAM still 3.8G; swap heavily used (~1.9G/4G) | High | Provider RAM upgrade | open |
| Load still elevated (~6 on 2 cores) | Medium | Reduce concurrent workloads; capacity | open |
| Stale user sessions after apt upgrade | Low | Re-login SSH | open |
| UFW 6443 allowlist may miss new admin IPs | Medium | Add `ufw allow from <ip> to any port 6443 proto tcp` | open |
| 8472 closed externally | Low | Re-open if multi-node flannel peers appear | accepted (single-node) |
| fwupd.service failed restart during upgrade | Low | Investigate if firmware tooling needed | open |
| Capacity upgrade not visible in guest (still 2 vCPU / 3.8 GiB) | High | Confirm OpenStack/HostEurope flavor + resize complete on 92.205.62.239 | open |
| If upgrade never lands, original under-sizing returns under load | High | Finish resize or re-open capacity work | open |
| Swap still 4G (OK cushion once 16G lands) | Low | Optional shrink after 16G stable | watch |
| UFW 6443 allowlist may miss new admin IPs | Medium | access-plan procedure | open |

View file

@ -3,7 +3,7 @@ agent: host-operator
engagement_id: eng-coulomb-railiance01-ho-001
project: coulomb-railiance01
last_updated: '2026-07-16'
session_count: 4
session_count: 5
confidentiality: client_owned
---
@ -28,7 +28,8 @@ forward-deployed host-operator pilot (KAIZEN-WP-0009).
| hostname | role | typical load | services | last review |
|----------|------|--------------|----------|-------------|
| railiance01 | k3s production single-node | overload on 2 cores under current mix | k3s, gitea/forgejo, temporal, activity-core, state-hub edge, traefik, platform pods | 2026-07-16 |
| railiance01 | k3s production single-node | calmer after reboot (~load 2 on 2 cores) | k3s, gitea, temporal, activity-core, state-hub | 2026-07-16 capacity check |
## OS & Patch State
@ -47,18 +48,18 @@ forward-deployed host-operator pilot (KAIZEN-WP-0009).
## Load & Workload Envelope
| Field | Post-remediation 2026-07-16 |
|-------|----------------------------|
| Cores | 2 |
| Load 1m | ~6 (was ~12) — still elevated |
| RAM | 3.8 GiB; MemAvailable ~625 Mi |
| Swap | **4 GiB /swapfile**, ~1.9 GiB used |
| PSI memory full avg60 | ~8% (was ~24%) |
| Disk / | ~55% |
| /var/log/journal | ~461 Mi (was ~4.1 Gi) |
| k3s node | Ready |
| Field | 2026-07-16 capacity check (post operator upgrade claim) |
|-------|--------------------------------------------------------|
| Cores (guest) | **still 2** — operator reported 4; not visible yet |
| RAM (guest) | **still ~3.8 GiB** — operator reported 16 GiB; not visible yet |
| Load 1m | ~2 (improved vs Critical era) |
| MemAvailable | ~0.9 GiB |
| Swap | 4 GiB file, ~300 Mi used |
| PSI memory full avg60 | ~0.2% |
| k3s | Ready |
Envelope: swap is emergency cushion; plan RAM upgrade if swap stays high.
Do **not** treat 4/16 as baseline until `nproc` and `free -h` match.
See reports/2026-07-16-capacity-upgrade-check.md
## Accumulated Findings
@ -99,10 +100,10 @@ Envelope: swap is emergency cushion; plan RAM upgrade if swap stays high.
## Open Threads
- Plan hardware/provider RAM increase (swap is temporary relief)
- Re-login stale SSH sessions after package upgrade
- Add UFW allow for new admin IPs when they change
- Monitor if 8472 needed for any multi-node peer (currently none)
- **Confirm VM resize:** operator reported 4 vCPU / 16 GiB; live guest still 2 / 3.8 GiB (2026-07-16 check)
- After 4/16 confirmed: update baseline; consider whether to keep full 4G swap or shrink
- Re-login stale SSH sessions after earlier apt upgrade if not done
- UFW: add new admin public IPs when they change (see access-plan allowlist procedure)
## Session Log
@ -112,3 +113,5 @@ Envelope: swap is emergency cushion; plan RAM upgrade if swap stays high.
- 2026-07-16 · railiance01 · deep_assessment · T10 ramp-up complete: Critical memory/load, k3s API unavailable, RU all done, phase operating · ok
- 2026-07-16 · railiance01 · privileged remediation P1P4 applied (swap, journal, apt, ufw) · ok
- 2026-07-16 · railiance01 · deep_assessment · Approved P1-P4 done: 4G swap, journal vacuum, apt upgrade, UFW k3s API allowlist; k3s Ready; RAM still tight · ok
- 2026-07-16 · railiance01 · capacity check after operator 4CPU/16G report: guest still 2/3.8G · watch
- 2026-07-16 · railiance01 · standard_review · Operator reported 4CPU/16G upgrade; live guest still 2CPU/3.8G after reboot; k3s Ready; load calmer · ok