net-kingdom/docs/evidence/2026-10-30-nk54-bind-split/README.md

LLDAP bind split, live run 2026-10-30 (NK-WP-0054-T05)

Founder-run in the founder's terminal following sso-mfa/k8s/lldap/BIND-SPLIT-RUNBOOK.md, 17:39-18:53 UTC, agent (Claude Code) guiding. All receipts are value-free (values_recorded: false).

Step	Result	Receipt
1 prepare custody	five keys changed; custody copy re-sealed to keys/age.pub	01-prepare.json
2 plan	five create, admin in lldap_admin, no collision	02-plan.json
3 accounts	preview, then apply; all five sign in	03a-accounts-preview.json, 03b-accounts.json
4 verify	pass	04-verify.json
5 identity-provisioner	Secret identity-provisioner-ldap created; image sha256:6d1f3632…4199 ready (readiness includes directory login)	operator output
6 Authelia	ldap_password rotated to svc-authelia; configmap applied; founder sign-in with password and OTP	operator output
7 KeyCape	directory-bind-rollout.py apply, uid unchanged, other_secret_data_unchanged: true; sign-in ok	operator output
8 privacyIDEA resolver	realm_bootstrap.py --switch-bind --apply; OTP sign-in ok	operator output
9 LLDAP	LLDAP_LDAP_USER_PASS rotated; configured super-admin ld-admin	operator output
10 retire shared password	admin password changed by the founder; old shared password refused; pass	10-verify.json

Deviations and findings

1. Custody copy not at the runbook path. Step 1 failed with FileNotFoundError (nothing written): only the sealed sso-mfa/bootstrap/secrets.enc/ exists. The founder decrypted it with make sops-custody-shell + decrypt-secrets.sh, ran step 1, re-sealed with --no-shred, and sealed and shredded after step 10. Runbook setup now says so.

2. Wrong kubeconfig for the Secret scripts. apply-service-secrets.py calls plain kubectl, which used the limited default identity (ops-mason-builder, cannot read sso Secrets); step 5 was refused before any write (presence check: NotFound). Fixed by export KUBECONFIG=$HOME/.kube/config-railiance01; the runbook setup now exports it.

3. Stale custody value: AUTHELIA_KEYCAPE_CLIENT_SECRET. Step 6 was refused before any write: a value-free comparison showed the four other Authelia values equal to live, ldap_password different (intended), and the custody client secret not matching the live keycape_client_secret_hash. The KeyCape-Authelia client secret was changed after the March custody seal without updating custody. Step 6 used a one-off helper that calls the same publish() path for ldap_password only (uid, resourceVersion and full-data test, readback) and skips only the client-secret check. Reconciled the same evening with sso-mfa/bootstrap/reconcile-keycape-client-secret.py (founder-run, value-free: live pair agreed, custody updated and re-checked, re-sealed).

4. Rollouts of identity-provisioner, KeyCape and LLDAP use Recreate: each had a short outage of about 30 s.

5. privacyIDEA resolver was still bound as admin (found 2026-10-30 evening). After step 10 every OTP check failed: KeyCape logged mfa_failed (openbao-admin) and mfa_check_error (user-engine-portal); privacyIDEA logged LDAP Bind unsuccessful: invalidCredentials (49) from 19:00:10 UTC, last successful lookup 18:49:30 UTC, step 10 at 18:53 UTC. Founder found Bind DN uid=admin,… on lldap-coulomb in the privacyIDEA UI and changed it to uid=svc-privacyidea,…; the Bind DN persisted but the hand-entered bind password did not match. A one-off attended script (reusing realm_bootstrap.py's client and custody reader) then wrote only BINDPW from the custody LLDAP_SVC_PRIVACYIDEA_PASS, checked the other resolver fields unchanged and resolved platform-root through the resolver; the founder's OpenBao OIDC login with OTP then succeeded. Prefer the custody-fed write over UI entry: the German UI offered only the two test buttons, and pasting the value by hand is error-prone. No repository tool rewrites the resolver, so step 8's apply most likely did not take effect; its OTP check passed only because the old shared password was still valid. Runbook step 8 now requires the resolver Bind DN and test, and step 10 a fresh OTP sign-in.

Closing condition still open: the restore drill on the first backup generation after this run (NK-WP-0053 stages 2-5, T17/T19 under the new accounts).