net-kingdom/sso-mfa/k8s/lldap/BIND-SPLIT-RUNBOOK.md

LLDAP Bind Split: Founder Runbook

NK-WP-0054-T05, founder decision 9f0f7a87 (2026-10-28). Moves Authelia, KeyCape, the privacyIDEA resolver and identity-provisioner off the shared human admin bind onto their own accounts, makes ld-admin LLDAP's configured super-admin, and makes the admin password human-only. You run every live step in your own terminal; agents prepared and rehearsed it on synthetic directories (docs/evidence/2026-10-28-nk54-bind-split-rehearsal/).

No step prints a password. Passwords come from the private custody copy or a hidden prompt. Keep the receipts under ~/nk54/; they are value-free and go to docs/evidence/ afterwards.

Fixed rules

• Order matters. bind-split.py accounts runs before the LLDAP deployment switch: LLDAP never creates a missing configured admin once lldap_admin has a member (rehearsal --stage startup).
• The human admin password stays valid until step 10. Until then every consumer can be put back on the old bind, and a backup from any point restores (the restore launchers take each identity from the backup itself).
• One consumer at a time. Check before the next.

Setup (once per session)

cd ~/net-kingdom && git pull --ff-only
bridge up k3s-api-railiance01
K="kubectl --kubeconfig $HOME/.kube/config-hosteurope --server https://127.0.0.1:16444"
CUSTODY=sso-mfa/bootstrap/secrets      # your private custody copy, as for make creds-apply
mkdir -p -m 700 ~/nk54

0. Preconditions

1. Restore drill on the current backup (NK-WP-0053-T02/T04) has passed. It proves today's backup before the change.
2. Pin LLDAP to what runs. Done 2026-10-29: the running digest is v0.6.2 and deployment.yaml is pinned. Read the running digest (no Secret is read):

   $K -n sso get pod -l app.kubernetes.io/name=lldap -o jsonpath='{.items[0].status.containerStatuses[0].imageID}'; echo
   

   If it ends in 9e605a66c02514bfcffd1b67cafb1e98d50992216bb2871d7ae44622047dd09d (v0.6.2, the rehearsed version), set image: in sso-mfa/k8s/lldap/deployment.yaml to docker.io/lldap/lldap@sha256:9e605a66…09d and commit. If it is anything else, stop: the rehearsals must be rerun on that digest first; never pin to an older version.
3. Provisioner image. Done 2026-10-29: sha256:6d1f3632…4199, built by the Forgejo workflow from main after 2ebc4f0, is pinned. Identity provider journey acceptance is green on 4261a5a (same provisioner code; that commit only made one test runnable without pytest). Originally: built from commit 2ebc4f0 or later through its owner pipeline (it reads LLDAP_ADMIN_USERNAME). Put its digest in sso-mfa/k8s/identity-provisioner/deployment.yaml and commit. Do not deploy it yet.

1. Prepare custody

python3 sso-mfa/k8s/lldap/bind-split.py prepare-custody "$CUSTODY" --receipt ~/nk54/01-prepare.json

Adds the four LLDAP_SVC_*_PASS keys and replaces LLDAP_LDAP_USER_PASS with a new ld-admin password. It refuses a second run. Re-seal the custody copy the way you normally do after a custody change, and put the ld-admin password into your password safe as the sealed LLDAP service-admin (crisis list; it is LLDAP_LDAP_USER_PASS in the custody copy).

2. Plan (read-only)

python3 sso-mfa/k8s/lldap/bind-split.py plan --endpoint https://lldap.coulomb.social --receipt ~/nk54/02-plan.json

Prompts for the admin password. Expect every target "action": "create" and admin.in_lldap_admin: true. Any refuse means an existing account collides: stop.

3. Create the accounts

python3 sso-mfa/k8s/lldap/bind-split.py accounts --endpoint https://lldap.coulomb.social "$CUSTODY" --receipt ~/nk54/03a-accounts-preview.json
python3 sso-mfa/k8s/lldap/bind-split.py accounts --endpoint https://lldap.coulomb.social "$CUSTODY" --apply --receipt ~/nk54/03b-accounts.json

Additive only: nothing uses the new accounts yet. Needs docker and the pinned image locally (it sets passwords with LLDAP's own lldap_set_password). Expect signs_in all true.

4. Verify the accounts

python3 sso-mfa/k8s/lldap/bind-split.py verify --endpoint https://lldap.coulomb.social "$CUSTODY" --receipt ~/nk54/04-verify.json

"verdict": "pass" before any consumer moves.

5. identity-provisioner

bash sso-mfa/k8s/identity-provisioner/create-secrets.sh "$CUSTODY"
$K apply -f sso-mfa/k8s/identity-provisioner/deployment.yaml
$K -n sso rollout status deploy/identity-provisioner --timeout=180s

Its readiness includes a directory login, so rollout status succeeding means svc-provisioner works. Rollback: $K -n sso rollout undo deploy/identity-provisioner.

6. Authelia

bash sso-mfa/k8s/authelia/create-secrets.sh "$CUSTODY" --rotate-key LLDAP_SVC_AUTHELIA_PASS
$K apply -f sso-mfa/k8s/authelia/configmap.yaml
$K -n sso rollout restart deploy/authelia && $K -n sso rollout status deploy/authelia --timeout=180s

Check: sign in at https://auth.coulomb.social as yourself (password and OTP). Rollback (the admin password still works until step 10):

git show 145ca8e~1:sso-mfa/k8s/authelia/configmap.yaml | $K apply -f -
read -rsp 'admin password: ' P; echo
printf '[{"op":"replace","path":"/data/ldap_password","value":"%s"}]' "$(printf %s "$P" | base64 -w0)" \
  | $K -n sso patch secret authelia-secrets --type=json --patch-file=/dev/stdin; unset P
$K -n sso rollout restart deploy/authelia

7. KeyCape

python3 sso-mfa/k8s/keycape/directory-bind-rollout.py inspect --custody-root "$CUSTODY"
# note before.uid and before.resource_version, then:
python3 sso-mfa/k8s/keycape/directory-bind-rollout.py dry-run --custody-root "$CUSTODY" --expected-uid <uid> --expected-resource-version <rv>
python3 sso-mfa/k8s/keycape/directory-bind-rollout.py apply --custody-root "$CUSTODY" --expected-uid <uid> --expected-resource-version <rv> --attended
$K -n sso rollout restart deploy/keycape && $K -n sso rollout status deploy/keycape --timeout=180s

Expect prior_bind: human, change_needed: true. Check: a KeyCape sign-in (for example the users portal or the OpenBao UI through openbao-ui-railiance01). Rollback: the same three commands with --operation revert (prompts for the admin password), then restart.

8. privacyIDEA resolver

python3 sso-mfa/k8s/privacyidea/realm_bootstrap.py "$CUSTODY" --switch-bind               # preview
python3 sso-mfa/k8s/privacyidea/realm_bootstrap.py "$CUSTODY" --switch-bind --apply

Expect actions: ["switch-bind"], one write. Check: a sign-in with OTP (step 6 again). There is no automated rollback; the old bind is the admin account, which still works, so the attended repair (--attended, prompts) can set it back if needed.

9. LLDAP

bash sso-mfa/k8s/lldap/create-secrets.sh "$CUSTODY" --rotate-key LLDAP_LDAP_USER_PASS
$K apply -f sso-mfa/k8s/lldap/deployment.yaml
$K -n sso rollout status deploy/lldap --timeout=180s

The configured super-admin is now ld-admin. Check: sign in to the LLDAP UI as ld-admin once, then as admin (still admin through lldap_admin).

10. Make the human password human-only

Sign in to the LLDAP UI as admin, change your password, store it in your password safe. Then:

python3 sso-mfa/k8s/lldap/bind-split.py verify --endpoint https://lldap.coulomb.social "$CUSTODY" --old-admin-retired --receipt ~/nk54/10-verify.json

It prompts for the new admin password, then the old shared one, and requires that the old one is refused. "verdict": "pass" ends the live run.

11. Afterwards

• End-to-end: a full KeyCape sign-in with MFA, and one provisioning call if one is due.
• Copy ~/nk54/*.json to docs/evidence/<date>-nk54-bind-split/ and commit.
• After the next daily backup, run the restore drill on that generation (NK-WP-0053 stages 2 to 5). It proves T17/T19 under the new accounts and closes NK-WP-0054-T05; then exception SI-X1 is retired.