Pin existing k3s hardening candidate and runtime review gates

docs/changes/CUST-WP-0073/existing-k3s-host-r1/PROPOSAL.md

@@ -0,0 +1,94 @@
+# Existing Railiance01 k3s service hardening — review addendum
+
+Part of CUST73-CREDENTIAL-BOUNDARY-R1, next review revision; original revision-2
+packet remains unchanged. No execution approval, mutation or restart occurred.
+This is the existing-host mode change only, not Custodian assignment admission.
+
+## Observed target and exact proposed change
+
+Existing SSH configuration resolves railiance01 to 92.205.62.239 as tegwick,
+without proxy/local commands. The read used that existing access, no certificate
+issuance and no sudo. Warden ssh-cert-host-access route was checked first.
+
+Sanitized observation refreshed at remote UTC 2026-10-19T12:00:53.721961+00:00,
+with local transport start/finish timestamps separately retained. Strict SSH
+host-key checking was required. The expected IPv4 is present on the host;
+observation.json binds hostname and hashed machine-id. These are captured facts,
+not timeless target assertions; rerun identity checks before any approval/execution.
+
+k3s is loaded/active/running, main PID935, fragment
+/etc/systemd/system/k3s.service, no drop-ins, no pending daemon reload. Its
+loaded ExecStart and fragment each have exactly one mode644 flag. The fragment
+is regular root:root0644, device2049/inode7875, SHA256:
+`b5398c385430ce225faadc1f3724597fe4b05dd999a04b8abbca64d4b0d68c0e`.
+The default config.yaml and config.yaml.d are absent.
+Admin kubeconfig metadata is root:root0644, device2049/inode786464; its contents
+were not read. The service.env file is root:root0600 and inaccessible without
+privilege; only its metadata was read, not contents or a hash.
+
+The exact proposed unit transformation changes the single mode argument from
+644 to 600, preserving every other byte and total length. Two bytes change.
+The privately computed candidate SHA256 is:
+`866ea58bfab00215e5bd3352287ebb0230d4ddb7dd3ed4b1cd853e74c6698d7d`.
+No candidate file was written on the host and raw unit content was never emitted.
+`unit_transform.py` provides the pure guarded transform, with pinned original and
+candidate hashes, root owner/mode/device/inode checks, O_NOFOLLOW, link-count and
+concurrent-read checks. It has no write/reload/restart code. Five tests cover
+exact change and refusal on drift, absent/duplicate flag or wrong candidate.
+
+## Execution sequence to include in owner-reviewed privileged revision
+
+Owner: railiance-cluster for unit/install config, railiance-infra for account
+recovery. The narrow approved executor must:
+
+1. Verify fresh host identity, service state, original fragment identity/hash,
+   no new drop-ins/config paths, and no pending reload. Refuse drift; do not
+   auto-rebase this proposal onto another service revision.
+2. Retain private owner-only predecessor bytes and metadata outside agent reach.
+   Confirm separate attended console/admin recovery and single-node headroom.
+3. After the protected-environment gate below, generate the candidate in private
+   root staging and require exact candidate digest, root ownership and mode0644.
+   Recheck the original object before atomic replacement. Converge only the
+   admin kubeconfig owner/group/mode to root:root0600 without reading its content.
+4. Run systemctl daemon-reload; verify loaded mode600 and no unrelated fragment
+   or drop-in change. Restart only k3s in the approved interruption window.
+   No reinstall, package upgrade, generic bootstrap or kubeconfig export.
+5. Verify active/running service, fresh mode600 ExecStart, kubeconfig root:root0600
+   after readiness, cluster API readiness through attended admin access and no
+   workload rollout regression. Receipts expose only booleans/state and pinned
+   hashes. Do not capture kubectl error dumps, unit/env/config text or credentials.
+6. Keep future-installer source correction from the original packet coupled to
+   this change, so a future installer does not reintroduce644 or controller export.
+
+## Explicit remaining gate
+
+The protected /etc/systemd/system/k3s.service.env cannot be read by the current
+non-sudo observation path. No privilege escalation or content retrieval is
+requested here. The owner must independently return only nonsecret predicates:
+whether alternate config selection or K3S_KUBECONFIG_MODE is defined, whether it
+can conflict with the selected launch change, and whether additional config/drop-in
+sources exist. Do not return values or raw environment. Unknown means refusal,
+not an assumption that the default config absence covers every launch input.
+Fresh exact action, execution identity, expected interruption bound, validation
+and recovery must be bound into the supervisor-approved revision before use.
+
+## Abort and recovery
+
+Before any write: a failed predicate aborts with no effect. After fragment
+replacement but before service restart: the owner can CAS-restore the private
+preimage only if the candidate digest is still exact, then daemon-reload; retain
+kubeconfig0600 and do not restart the old mode644 service. This rollback returns
+the reviewed predecessor unit, not a claim that credential separation succeeded.
+
+After restart: prefer attended diagnosis with mode600 retained. Do not
+unconditionally restore644 or expose admin credentials to recover readiness.
+If restoring the old launch is necessary, stop all agent admission and obtain
+explicit supervisor disposition acknowledging mode644 regeneration; use retained
+console access, exact candidate/preimage CAS and immediate post-start0600
+convergence. This exceptional fallback is not authorized by this preparation.
+Preserve failure evidence and re-establish persistent600 before agent admission.
+
+No change to old SSH principals, workstation kubeconfigs, native provider
+credentials or actual interactive assignment isolation is established here.
+Estimated extra infrastructure allocation is zero; restart can interrupt the
+single control plane. Monetary loss is unquantified; autopilot remains disabled.

docs/changes/CUST-WP-0073/existing-k3s-host-r1/SHA256SUMS

@@ -0,0 +1,8 @@
+41aff887b1164256409c0f1506e4bf751c8b30edf89c69dc790fd9374fca13ce  PROPOSAL.md
+6360990f880ef55cc1f41893c15f880fdbb436dd0658657a52feebf5abafcc14  candidate.json
+7ad5169d89fc9c546c3a2a82fe095736eca450fbbad4d7bdf6de101e9549e81f  derive.py
+2858375cf9bae3b867dc00799ad6877f5320e08dc6b0a2e047526e0718c8c4eb  observation.json
+b32df5eaac5433fb76fd67f6d359c2b8e9de4431e3ced2f0d2bae01cb796e446  observe.py
+8c85ebe236aaee4b48b170ca67e6b0386183c1f2c4966c54ea61819af16f0557  test_unit_transform.py
+2d8bc17ee5d9ca0f649222a766226f4f8bca4ed63f788f7ce69d744f8922fe7b  unit_transform.py
+d8704792f62da027320ab8f6d31194ab6fa2639b732d5e5ead9cdb618188c923  validation.json

docs/changes/CUST-WP-0073/existing-k3s-host-r1/candidate.json

@@ -0,0 +1,13 @@
+{
+  "unit_preimage_sha256": "b5398c385430ce225faadc1f3724597fe4b05dd999a04b8abbca64d4b0d68c0e",
+  "unit_candidate_sha256": "866ea58bfab00215e5bd3352287ebb0230d4ddb7dd3ed4b1cd853e74c6698d7d",
+  "changed_argument_count": 1,
+  "same_length": true,
+  "bytes_changed": 2,
+  "env_file_readable_without_sudo": false,
+  "candidate_written": false,
+  "raw_unit_emitted": false,
+  "credential_content_read": false,
+  "revalidated_observation_sha256": "2858375cf9bae3b867dc00799ad6877f5320e08dc6b0a2e047526e0718c8c4eb",
+  "binding_note": "Candidate digest was computed read-only from the same pinned unit preimage; fresh host-bound observation confirms that preimage unchanged."
+}

docs/changes/CUST-WP-0073/existing-k3s-host-r1/derive.py

@@ -0,0 +1,14 @@
+import hashlib,json,os,re,stat
+from pathlib import Path
+p=Path('/etc/systemd/system/k3s.service')
+expected='b5398c385430ce225faadc1f3724597fe4b05dd999a04b8abbca64d4b0d68c0e'
+s=p.lstat();raw=p.read_bytes()
+assert stat.S_ISREG(s.st_mode) and not p.is_symlink() and (s.st_uid,s.st_gid,s.st_dev,s.st_ino)==(0,0,2049,7875)
+assert hashlib.sha256(raw).hexdigest()==expected
+# Exact one-argument change, with all other unit bytes retained privately.
+new,count=re.subn(rb'(--write-kubeconfig-mode(?:=|[ \t]+)[\x22\x27]?)644\b',rb'\g<1>600',raw)
+assert count==1
+print(json.dumps({'unit_preimage_sha256':expected,'unit_candidate_sha256':hashlib.sha256(new).hexdigest(),
+ 'changed_argument_count':count,'same_length':len(new)==len(raw),'bytes_changed':sum(a!=b for a,b in zip(raw,new)),
+ 'env_file_readable_without_sudo':os.access('/etc/systemd/system/k3s.service.env',os.R_OK),
+ 'candidate_written':False,'raw_unit_emitted':False,'credential_content_read':False}))

docs/changes/CUST-WP-0073/existing-k3s-host-r1/observation.json

@@ -0,0 +1,91 @@
+{
+  "credential_file_content_read": false,
+  "directories": {
+    "/etc/rancher/k3s/config.yaml.d": {
+      "entry_count": 0,
+      "exists": false,
+      "metadata_accessible": true
+    },
+    "/etc/systemd/system/k3s.service.d": {
+      "entry_count": 0,
+      "exists": false,
+      "metadata_accessible": true
+    }
+  },
+  "files": {
+    "/etc/rancher/k3s/config.yaml": {
+      "exists": false
+    },
+    "/etc/rancher/k3s/k3s.yaml": {
+      "device": 2049,
+      "exists": true,
+      "gid": 0,
+      "inode": 786464,
+      "mode": "0o644",
+      "regular": true,
+      "symlink": false,
+      "uid": 0
+    },
+    "/etc/systemd/system/k3s.service": {
+      "device": 2049,
+      "execstart_count": 1,
+      "exists": true,
+      "gid": 0,
+      "has_mode_env_assignment": false,
+      "inode": 7875,
+      "mode": "0o644",
+      "mode_config_values": [],
+      "mode_flags": [
+        "644"
+      ],
+      "readable": true,
+      "regular": true,
+      "sha256": "b5398c385430ce225faadc1f3724597fe4b05dd999a04b8abbca64d4b0d68c0e",
+      "symlink": false,
+      "uid": 0
+    },
+    "/etc/systemd/system/k3s.service.env": {
+      "device": 2049,
+      "exists": true,
+      "gid": 0,
+      "inode": 7873,
+      "mode": "0o600",
+      "regular": true,
+      "symlink": false,
+      "uid": 0
+    }
+  },
+  "host_identity": {
+    "expected_ipv4": "92.205.62.239",
+    "expected_ipv4_present": true,
+    "hostname": "239.62.205.92.host.secureserver.net",
+    "machine_id_sha256": "fe7367a60951922122af7ba18ed2cde65c7e0a232bbedfb045cb575de2094c2f",
+    "remote_captured_at": "2026-10-19T12:00:53.721961+00:00",
+    "ssh_alias": "railiance01"
+  },
+  "read_only": true,
+  "service": {
+    "ActiveState": "active",
+    "DropInPaths": [],
+    "FragmentPath": [
+      "/etc/systemd/system/k3s.service"
+    ],
+    "LoadState": "loaded",
+    "MainPID": "935",
+    "NeedDaemonReload": "no",
+    "SubState": "running",
+    "effective_mode_flags": [
+      "644"
+    ]
+  },
+  "sudo_used": false,
+  "uid": 1000,
+  "transport_capture": {
+    "started_at_utc": "2026-10-19T12:00:52.301688+00:00",
+    "finished_at_utc": "2026-10-19T12:00:52.834076+00:00",
+    "ssh_alias": "railiance01",
+    "strict_host_key_checking": true,
+    "existing_credentials_only": true
+  },
+  "observer_sha256": "b32df5eaac5433fb76fd67f6d359c2b8e9de4431e3ced2f0d2bae01cb796e446"
+}

docs/changes/CUST-WP-0073/existing-k3s-host-r1/observe.py

@@ -0,0 +1,55 @@
+import datetime,hashlib,json,os,re,stat,subprocess,socket
+from pathlib import Path
+
+def attrs(path, content=False):
+    p=Path(path)
+    try:
+        s=p.lstat()
+        out={'exists':True,'uid':s.st_uid,'gid':s.st_gid,'mode':oct(stat.S_IMODE(s.st_mode)),
+             'regular':stat.S_ISREG(s.st_mode),'symlink':stat.S_ISLNK(s.st_mode),'device':s.st_dev,'inode':s.st_ino}
+        if content and stat.S_ISREG(s.st_mode):
+            try:
+                raw=p.read_bytes();out['sha256']=hashlib.sha256(raw).hexdigest()
+                text=raw.decode('utf-8',errors='replace')
+                out['mode_flags']=re.findall(r'--write-kubeconfig-mode(?:=|\s+)[\x22\x27]?(0?[0-7]{3})\b',text)
+                out['mode_config_values']=re.findall(r'^\s*write-kubeconfig-mode\s*:\s*[\x22\x27]?(0?[0-7]{3})\b',text,re.M)
+                out['execstart_count']=len(re.findall(r'^ExecStart=',text,re.M))
+                out['has_mode_env_assignment']=bool(re.search(r'K3S_KUBECONFIG_MODE\s*=',text))
+                out['readable']=True
+            except (OSError,ValueError):out['readable']=False
+        return out
+    except FileNotFoundError:return {'exists':False}
+    except OSError:return {'metadata_accessible':False}
+
+network=subprocess.run(['ip','-j','address','show','scope','global'],capture_output=True,text=True,timeout=10)
+try:
+    addresses=[a.get('local') for interface in json.loads(network.stdout) for a in interface.get('addr_info',[])]
+    expected_ip_present=network.returncode==0 and '92.205.62.239' in addresses
+except (ValueError,TypeError):expected_ip_present=False
+host_identity={'ssh_alias':'railiance01','expected_ipv4':'92.205.62.239',
+ 'expected_ipv4_present':expected_ip_present,'hostname':socket.gethostname(),
+ 'machine_id_sha256':hashlib.sha256(Path('/etc/machine-id').read_bytes()).hexdigest(),
+ 'remote_captured_at':datetime.datetime.now(datetime.timezone.utc).isoformat()}
+result={'host_identity':host_identity,'read_only':True,'sudo_used':False,'credential_file_content_read':False,'uid':os.getuid(),'service':{}}
+for prop in ('LoadState','ActiveState','SubState','MainPID','FragmentPath','DropInPaths','NeedDaemonReload'):
+    p=subprocess.run(['systemctl','show','k3s','--property='+prop,'--value'],capture_output=True,text=True,timeout=10)
+    value=p.stdout.strip()
+    if prop in ('FragmentPath','DropInPaths'):
+        paths=value.split()
+        safe=all(re.fullmatch(r'/[A-Za-z0-9_./@+-]+',x) for x in paths)
+        result['service'][prop]=paths if p.returncode==0 and safe else 'unavailable'
+    else:
+        result['service'][prop]=value if p.returncode==0 and re.fullmatch(r'[A-Za-z0-9_-]*',value) else 'unavailable'
+p=subprocess.run(['systemctl','show','k3s','--property=ExecStart','--value'],capture_output=True,text=True,timeout=10)
+result['service']['effective_mode_flags']=re.findall(r'--write-kubeconfig-mode(?:=|\s+)[\x22\x27]?(0?[0-7]{3})\b',p.stdout) if p.returncode==0 else None
+result['files']={name:attrs(name,content) for name,content in [('/etc/systemd/system/k3s.service',True),('/etc/rancher/k3s/config.yaml',True),('/etc/rancher/k3s/k3s.yaml',False),('/etc/systemd/system/k3s.service.env',False)]}
+for directory in ('/etc/systemd/system/k3s.service.d','/etc/rancher/k3s/config.yaml.d'):
+    d=Path(directory)
+    try:
+        entries=list(d.iterdir()) if d.exists() else []
+        result['directories' if 'directories' in result else 'directories']=result.get('directories',{})
+        result['directories'][directory]={'exists':d.exists(),'entry_count':len(entries),'metadata_accessible':True}
+        for child in entries:
+            if re.fullmatch(r'[A-Za-z0-9_.+-]+',child.name):result['files'][str(child)]=attrs(child,True)
+    except OSError:result.setdefault('directories',{})[directory]={'metadata_accessible':False}
+print(json.dumps(result,sort_keys=True))

docs/changes/CUST-WP-0073/existing-k3s-host-r1/test_unit_transform.py

@@ -0,0 +1,31 @@
+import hashlib
+import unittest
+from unit_transform import transform
+
+
+def sha(value):return hashlib.sha256(value).hexdigest()
+
+
+class TransformTests(unittest.TestCase):
+    def test_exact_flag_preserves_every_other_byte(self):
+        raw=b'[Service]\nExecStart=/usr/local/bin/k3s server --write-kubeconfig-mode=644 --other=kept\n'
+        expected=raw.replace(b'mode=644',b'mode=600')
+        self.assertEqual(transform(raw,before=sha(raw),after=sha(expected)),expected)
+
+    def test_changed_preimage_refuses(self):
+        with self.assertRaisesRegex(ValueError,'preimage'):transform(b'unknown')
+
+    def test_multiple_flags_refuse(self):
+        raw=b'--write-kubeconfig-mode=644 --write-kubeconfig-mode=644'
+        with self.assertRaisesRegex(ValueError,'not_exact'):transform(raw,before=sha(raw),after=sha(raw))
+
+    def test_missing_flag_refuses(self):
+        raw=b'--write-kubeconfig-mode=600'
+        with self.assertRaisesRegex(ValueError,'not_exact'):transform(raw,before=sha(raw),after=sha(raw))
+
+    def test_wrong_candidate_refuses(self):
+        raw=b'--write-kubeconfig-mode=644'
+        with self.assertRaisesRegex(ValueError,'candidate_mismatch'):transform(raw,before=sha(raw),after=sha(raw))
+
+
+if __name__=='__main__':unittest.main()

docs/changes/CUST-WP-0073/existing-k3s-host-r1/unit_transform.py

@@ -0,0 +1,40 @@
+"""Pure reviewed k3s unit transformation. No write, reload or restart capability."""
+import hashlib
+import os
+from pathlib import Path
+import re
+import stat
+
+BEFORE = 'b5398c385430ce225faadc1f3724597fe4b05dd999a04b8abbca64d4b0d68c0e'
+AFTER = '866ea58bfab00215e5bd3352287ebb0230d4ddb7dd3ed4b1cd853e74c6698d7d'
+PATH = Path('/etc/systemd/system/k3s.service')
+
+
+def transform(raw, *, before=BEFORE, after=AFTER):
+    if hashlib.sha256(raw).hexdigest() != before:
+        raise ValueError('unit_preimage_changed')
+    candidate, count = re.subn(rb'(--write-kubeconfig-mode(?:=|[ \t]+)[\x22\x27]?)644\b', rb'\g<1>600', raw)
+    if count != 1 or len(candidate) != len(raw) or sum(a != b for a,b in zip(raw,candidate)) != 2:
+        raise ValueError('unit_change_not_exact')
+    if hashlib.sha256(candidate).hexdigest() != after:
+        raise ValueError('unit_candidate_mismatch')
+    return candidate
+
+
+def prepare():
+    # Open without symlink following and bind metadata/read to the same object.
+    fd = os.open(PATH, os.O_RDONLY | os.O_NOFOLLOW)
+    try:
+        st = os.fstat(fd)
+        if (not stat.S_ISREG(st.st_mode) or stat.S_IMODE(st.st_mode) != 0o644
+                or (st.st_uid, st.st_gid, st.st_dev, st.st_ino) != (0,0,2049,7875)
+                or st.st_nlink != 1):
+            raise ValueError('unit_identity_changed')
+        with os.fdopen(os.dup(fd), 'rb') as stream:
+            candidate = transform(stream.read())
+        end = os.fstat(fd)
+        if (end.st_mtime_ns,end.st_ctime_ns,end.st_size) != (st.st_mtime_ns,st.st_ctime_ns,st.st_size):
+            raise ValueError('unit_changed_during_read')
+        return candidate
+    finally:
+        os.close(fd)

docs/changes/CUST-WP-0073/existing-k3s-host-r1/validation.json

@@ -0,0 +1,9 @@
+{
+  "tested_at": "2026-10-19T12:00:25.851998+00:00",
+  "unit_transform_tests_passed": 5,
+  "production_reads": "sanitized unit/service and file metadata only",
+  "credential_contents_read": false,
+  "sudo": false,
+  "mutations": false,
+  "full_assignment_admission": false
+}