net-kingdom/docs/keycape-exposure-resolver-reconciliation.md

KeyCape exposure: privacyIDEA resolver reconciliation

Incident: KEYCAPE-EXPOSURE-20260823-01 Historical workplan: NK-WP-0033; source maintenance: NK-WP-0049-T02 Exercise status: core helper exercised 2026-10-14 by custodian-codex against local disposable privacyIDEA 3.12.2 and LLDAP, including actual LDAP bind, password update, preserved configuration, exact user lookup and HOTP. Terminal handling and failure paths have automated tests. The complete new attended procedure has not run against the production cluster or KeyCape. Evidence: Custodian docs/evidence/2026-10-14-netkingdom-security/resolver-cutover.json.

Both sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh and update-lldap-resolver-live.sh delegate to resolver_cutover.py. The update helper changes only an existing lldap-coulomb resolver's bind password; the reconcile helper additionally checks cluster readiness, discovery, exact user resolution, token-backed MFA and the supplied predecessor's HTTP denial. Neither helper bootstraps a missing resolver or realm, repairs missing tuning, or changes policies, tokens, directory credentials or Kubernetes resources.

Authority and target pins

The historical platform recovery contract is railiance-platform 453fed3; owner cutover receipt is State Hub message 45b236c8-052f-43d3-a472-44f8e9694da2. These identify historical evidence, not a fresh live grant. Founder-deferred exposure rotation remains deferred.

Before any live run, the existing owner/supervisor path must bind a private approval record to the current NetKingdom source revision and dependencies, incident, provider-approved replacement, owner receipt, exact start/end window, attended driver, independent abort operator and recovery plan. Confirm predecessor provenance separately. Do not restore an exposed predecessor or bundle. A preview digest and TTY are technical preconditions, not authorization.

Reconcile requires an explicit absolute kubeconfig, context, expected API server and kube-system namespace UID from the reviewed owner packet. It checks the server and UID before workload reads. It never reads Kubernetes Secrets. The five existing deployments must each have the current generation observed and exactly one desired, actual, updated, ready and available replica, without a deletion timestamp. No ambient default context is accepted.

Default service origins are https://pink.coulomb.social, https://lldap.coulomb.social and https://kc.coulomb.social. Explicit literal loopback URLs with ports are supported for approved forwards or fixtures. Environment URL/tuning overrides and the old --note option are removed. KeyCape discovery must identify the canonical https://kc.coulomb.social issuer and its own authorization, token and JWKS endpoints, even through a forward. The shared transport disables proxies and redirects and bounds strict JSON responses; each extra service profile permits only its one required route and never receives the privacyIDEA authorization token.

Preview, apply and check

Use an attended terminal from the reviewed checkout. These are command templates; replace public placeholders with the approved target values. No secret belongs in argv, environment, Git, State Hub, chat or ordinary logs.

bash sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh --preview \
  --kubeconfig /absolute/approved/kubeconfig --context APPROVED_CONTEXT \
  --server https://APPROVED_SERVER:6443 --cluster-uid APPROVED_NAMESPACE_UID

Preview prompts only for pi-admin, authenticates with an admin-role check, validates the existing resolver/core/tuning/realm, and returns a public-state snapshot_sha256. It performs no configuration write. It creates an auth session and can produce provider audit events. The digest excludes the bind password and binds the target settings and observed non-secret configuration. It is not a credential fingerprint.

Run the same command with --apply --expected-digest REVIEWED_DIGEST in place of --preview, retaining all target and user options. The helper prompts silently for pi-admin, the approved replacement and the actual predecessor. Inputs stay in Python memory; no temporary secret files are created. Inherited shell tracing is disabled before input handling. A fresh OTP/PIN is requested only after configuration and exact user lookup succeed. --user defaults to platform-root.

Before mutation the helper authenticates the replacement to LLDAP and uses privacyIDEA /resolver/test with the candidate password to prove an actual LDAP bind. It tests supplied predecessor denial at LLDAP HTTP login and rechecks the observed configuration. Apply sends exactly one resolver update containing only type and BINDPW. Native privacyIDEA 3.12.2 preserves omitted fields. Readback must preserve all other resolver data and every realm; a native resolver test then proves the stored bind works. The API has no compare-and-swap: an exclusive owner window remains required, and ambiguous writes are never retried.

--check replaces --preview to perform the proofs without a configuration write; it needs no digest. Check proves the stored bind works, not that its censored value equals the supplied replacement. Authentication, audit and OTP counter effects still occur. It is not an effect-free operation.

The smaller update-lldap-resolver-live.sh supports the same preview/apply/check sequence without cluster flags. It proves candidate and stored binds plus exact configuration preservation, but does not claim MFA, readiness or predecessor acceptance. It must not substitute for full incident acceptance.

Receipts and recovery

Receipts contain fixed phase names, booleans, write counts and a public-state digest; no raw responses, exceptions or free-form notes. configured means the update helper completed its apply; checked means its no-write proof completed. Reconcile returns verified only when all helper proofs pass. Passing a wrong arbitrary password also produces denial: supplied_predecessor_denied is not proof of predecessor provenance. The owner must match the input to custody.

Use --predecessor-unavailable honestly when it cannot be produced. Reconcile then returns incomplete, exit 2, even if the other proofs pass; never type a placeholder to obtain a passing receipt. Failures and handled interruptions return unverified, exit 1, with write_attempted showing whether a partial change may exist. Exit 0 still leaves incident_acceptance: unverified: the separate owner receipt and actual KeyCape MFA path are outside this helper.

Stop on drift, wrong target, failed health/readiness or any unverified result. After an attempted write, reconcile observed state through the approved owner path before another attempt. Recover forward with an approved replacement; no automatic rollback to an exposed predecessor. Record only sanitized evidence, source revision, target references and operator outcome. NK-WP-0049 T03 remains the owner-native recovery/acceptance gate; this source repair does not close it.

Historical receipts and corrections

The old --check --predecessor-unavailable printed PASS on 2026-09-23 when run by Bernd Worsch from railiance01 checkout 6096c395 (receipt in NK-WP-0033). That remains historical check evidence, with predecessor denial unproven; it cannot establish the new revision's acceptance. Historical apply last ran on 2026-08-27 before fixes in 4a38511.

The August account attributed missing tuning to whole-object replacement. Native privacyIDEA 3.12.2 and source inspection corrected that interpretation on 2026-10-13: omitted fields are preserved, while sending explicit defaults can overwrite custom tuning. The October helper therefore sends only the bind password and type. Historic claims, default-overwriting commands, temporary-file handling and raw --note receipts are superseded by this procedure.