# LLDAP bind split, live run 2026-10-30 (NK-WP-0054-T05)

Founder-run in the founder's terminal following `sso-mfa/k8s/lldap/BIND-SPLIT-RUNBOOK.md`, 17:39-18:53
UTC, agent (Claude Code) guiding. All receipts are value-free (`values_recorded: false`).

| Step | Result | Receipt |
| --- | --- | --- |
| 1 prepare custody | five keys changed; custody copy re-sealed to `keys/age.pub` | `01-prepare.json` |
| 2 plan | five `create`, `admin` in `lldap_admin`, no collision | `02-plan.json` |
| 3 accounts | preview, then apply; all five sign in | `03a-accounts-preview.json`, `03b-accounts.json` |
| 4 verify | `pass` | `04-verify.json` |
| 5 identity-provisioner | Secret `identity-provisioner-ldap` created; image `sha256:6d1f3632…4199` ready (readiness includes directory login) | operator output |
| 6 Authelia | `ldap_password` rotated to `svc-authelia`; configmap applied; founder sign-in with password and OTP | operator output |
| 7 KeyCape | `directory-bind-rollout.py apply`, uid unchanged, `other_secret_data_unchanged: true`; sign-in ok | operator output |
| 8 privacyIDEA resolver | `realm_bootstrap.py --switch-bind --apply`; OTP sign-in ok | operator output |
| 9 LLDAP | `LLDAP_LDAP_USER_PASS` rotated; configured super-admin `ld-admin` | operator output |
| 10 retire shared password | `admin` password changed by the founder; old shared password refused; `pass` | `10-verify.json` |

## Deviations and findings

1. **Custody copy not at the runbook path.** Step 1 failed with `FileNotFoundError` (nothing written):
   only the sealed `sso-mfa/bootstrap/secrets.enc/` exists. The founder decrypted it with
   `make sops-custody-shell` + `decrypt-secrets.sh`, ran step 1, re-sealed with `--no-shred`, and sealed
   and shredded after step 10. Runbook setup now says so.
2. **Wrong kubeconfig for the Secret scripts.** `apply-service-secrets.py` calls plain `kubectl`, which
   used the limited default identity (`ops-mason-builder`, cannot read `sso` Secrets); step 5 was
   refused before any write (presence check: `NotFound`). Fixed by
   `export KUBECONFIG=$HOME/.kube/config-railiance01`; the runbook setup now exports it.
3. **Stale custody value: `AUTHELIA_KEYCAPE_CLIENT_SECRET`.** Step 6 was refused before any write: a
   value-free comparison showed the four other Authelia values equal to live, `ldap_password` different
   (intended), and the custody client secret **not** matching the live `keycape_client_secret_hash`.
   The KeyCape-Authelia client secret was changed after the March custody seal without updating custody.
   Step 6 used a one-off helper that calls the same `publish()` path for `ldap_password` only (uid,
   resourceVersion and full-data test, readback) and skips only the client-secret check. Reconciled the
   same evening with `sso-mfa/bootstrap/reconcile-keycape-client-secret.py` (founder-run, value-free:
   live pair agreed, custody updated and re-checked, re-sealed).
4. Rollouts of identity-provisioner, KeyCape and LLDAP use `Recreate`: each had a short outage of
   about 30 s.

5. **privacyIDEA resolver was still bound as `admin` (found 2026-10-30 evening).** After step 10
   every OTP check failed: KeyCape logged `mfa_failed` (`openbao-admin`) and `mfa_check_error`
   (`user-engine-portal`); privacyIDEA logged `LDAP Bind unsuccessful: invalidCredentials (49)` from
   19:00:10 UTC, last successful lookup 18:49:30 UTC, step 10 at 18:53 UTC. Founder found Bind DN
   `uid=admin,…` on `lldap-coulomb` in the privacyIDEA UI and changed it to `uid=svc-privacyidea,…`;
   the Bind DN persisted but the hand-entered bind password did not match. A one-off attended script
   (reusing `realm_bootstrap.py`'s client and custody reader) then wrote only `BINDPW` from the custody
   `LLDAP_SVC_PRIVACYIDEA_PASS`, checked the other resolver fields unchanged and resolved
   `platform-root` through the resolver; the founder's OpenBao OIDC login with OTP then succeeded.
   Prefer the custody-fed write over UI entry: the German UI offered only the two test buttons, and
   pasting the value by hand is error-prone. No repository tool rewrites the
   resolver, so step 8's apply most likely did not take effect; its OTP check passed only because the
   old shared password was still valid. Runbook step 8 now requires the resolver Bind DN and test, and
   step 10 a fresh OTP sign-in.

Closing condition still open: the restore drill on the first backup generation after this run
(NK-WP-0053 stages 2-5, T17/T19 under the new accounts).