the-custodian/docs/changes/CUST-WP-0073/ATTENDED-CUTOVER.md

CUST-WP-0073 — attended cutover runbook (2026-10-20)

Founder interview 2026-10-20 (Bernd Worsch) fixed: one isolated workstation account hosts Codex, Claude Code and Grok Build; attended execution with per-step confirmation; owner workers commit and push to main; privileged actions go through the new gated-executor repo; curated read RBAC; short-lived, re-issued credentials; open egress (builder mode — re-arm an egress allowlist at production close-down); Forgejo bot user without write to GitOps-sourced repos, scoped per tenant (current work = tenant zero, the platform building itself); transport mixed (Codex desktop via loopback SSH, Claude Code/Grok Build via wsl -d custodian-agents).

Shared names: workstation distro custodian-agents, account custodian-agt (workstation and railiance01), SSH principal agt-custodian, ServiceAccount custodian-agents/custodian-agent, executor account gexec.

Owner deliveries (source, pushed; nothing live)

Owner	Workplan	Commit	Attended step
railiance-platform	RPF-WP-0062	567d2c6, ded5922	T03 Argo sync
railiance-platform	(OpenBao agt-role)	e415da9	apply ssh roles
railiance-cluster	RCLUSTER-WP-0019	1d8d11d	T03 mode-600 playbook
railiance-infra	RAIL-HO-WP-0014	c62b187	T02 account/principal
ops-warden	WARDEN-WP-0045	e62763c	T03 inventory + first issuance
gated-executor	GEXEC-WP-0001	80b1973	T05 register, T06 install
the-custodian	this dir	workstation-agents-r4/	distro creation

Order (each step: check/diff first, then apply, then verify)

0. Recovery access: keep an open ssh tegwick@railiance01 session through steps 3–4. Migrate railiance-platform/scripts/application_backup_credentials.py (reads admin kubeconfig as non-root) to sudo or gexec before step 3.
1. OpenBao agt-role: railiance-platform/scripts/openbao-apply-ssh-engine.sh (dry-run first). Warden: copy the agt-custodian actor block from ops-warden/examples/inventory.seed.yaml into ~/.config/warden/inventory.yaml.
2. RBAC: in railiance-platform make argocd-pin-drift; then argocd app sync railiance-apps-root --revision ded5922 --resource argoproj.io:Application:custodian-agents; argocd app sync custodian-agents.
3. k3s mode 600 (railiance-cluster): ansible-playbook ansible/k3s-kubeconfig-mode.yml -e target=<railiance01> --check --diff, then without --check. API pause ~30–90 s. Rollback in playbook header.
4. Host account (railiance-infra/ansible): ansible-playbook playbooks/bootstrap-ssh-ca.yaml --limit Railiance01 --tags restricted_agent_account --check --diff, then apply. Confirm a fresh ssh tegwick@railiance01 still works before closing step 0's session.
5. Workstation distro (operator distro, this repo):

   powershell.exe -NoProfile -ExecutionPolicy Bypass -File "$(wslpath -w workstation-agents-r4/import.ps1)" -IAmAttended
   tar -C workstation-agents-r4 -cf - . | wsl.exe -d custodian-agents -u root -- bash -c 'mkdir -p /root/r4 && tar -C /root/r4 -xf - && bash /root/r4/guest-setup.sh --i-am-attended'
   wsl.exe --terminate custodian-agents
   

   Never wsl --shutdown, never touch another distro.
6. Harness install + login inside the distro (as custodian-agt, interactive by Bernd): Codex, Claude Code, Grok Build installers; each logs in with its own account. Codex desktop: add the Windows-side public key to /etc/ssh/authorized_keys/custodian-agt (root), connect to custodian-agt@127.0.0.1:22273.
7. Credentials into the distro (from the operator account, daily):
   • kubectl -n custodian-agents create token custodian-agent --duration=24h → kubeconfig at /home/custodian-agt/.kube/config (server URL, CA, token only).
   • agent keypair generated inside the distro; operator runs warden sign agt-custodian --pubkey <pub> > <cert>; place cert beside key.
   • Forgejo bot token (tenant-zero bot, no GitOps-repo write).
8. gated-executor: register project (custodian register-project), then GEXEC-WP-0001-T06 attended install on railiance01.
9. Proof (T02 acceptance), run inside custodian-agents as custodian-agt:
   • kubectl auth whoami → system:serviceaccount:custodian-agents:custodian-agent.
   • credential-boundary-r1/verify_agent_denials.py with supervisor namespace inventory.
   • ssh custodian-agt@railiance01: sudo -n true fails; cat /etc/rancher/k3s/k3s.yaml fails.
   • ssh tegwick@railiance01 refused (no cert for that principal); no ~/.kube admin config, no /mnt/c, wsl.exe/powershell.exe absent, no SSH_AUTH_SOCK, no OpenBao token.
   • gexec: one approved no-op executes; tampered/replayed/expired refused; sanitized output only. Record sanitized output under docs/evidence/.

Re-arm list (builder-mode conveniences to tighten at production close-down)

• egress allowlist for custodian-agents; loopback forwards from the operator distro are reachable (shared WSL2 network namespace)
• retire agt-task-bridge / agt-interhub-bootstrap → tegwick (RAIL-HO-WP-0014-T03)
• gexec credential fetch via custody owner (GEXEC-WP-0001-T07) instead of a pre-placed file
• T05 rotation of the 2026-09-23 exposed Secrets stays founder-triggered