coulomb/net-kingdom
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix: preserve resolver configuration during attended password cutover
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
Details
CI Smoke / container-smoke (push) Successful in 1s
Details

Browse source 
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a1346b-72a4-7d40-b8bd-8b8ca346cfa2
This commit is contained in:
tegwick 2026-10-14 01:52:37 +02:00
parent ab25dde80d
commit 4dffe2fe7a
11 changed files with 802 additions and 548 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/attended-procedure-inventory.md
View file

@ -30,9 +30,9 @@ runbook or a dedicated procedure. Missing runbooks are listed separately.
| `privacyidea/check-user-mfa-state.sh` | [smooth-bootstrap-guide.md](../docs/smooth-bootstrap-guide.md) | source repaired: explicit admitted private JWT, scoped read-only counts; live diagnostic delivery not admitted by this repair | | `privacyidea/check-user-mfa-state.sh` | [smooth-bootstrap-guide.md](../docs/smooth-bootstrap-guide.md) | source repaired: explicit admitted private JWT, scoped read-only counts; live diagnostic delivery not admitted by this repair |
| `privacyidea/create-secrets.sh` | [README.md](../sso-mfa/k8s/privacyidea/README.md), [smooth-bootstrap-guide.md](../docs/smooth-bootstrap-guide.md) | unknown | | `privacyidea/create-secrets.sh` | [README.md](../sso-mfa/k8s/privacyidea/README.md), [smooth-bootstrap-guide.md](../docs/smooth-bootstrap-guide.md) | unknown |
| `privacyidea/enckey-bootstrap.sh` | [README.md](../sso-mfa/k8s/privacyidea/README.md) | unknown | | `privacyidea/enckey-bootstrap.sh` | [README.md](../sso-mfa/k8s/privacyidea/README.md) | unknown |
| `privacyidea/reconcile-lldap-resolver-live.sh` | [keycape-exposure-resolver-reconciliation.md](../docs/keycape-exposure-resolver-reconciliation.md) | attempted by Bernd Worsch 2026-08-27; no completed PASS receipt | | `privacyidea/reconcile-lldap-resolver-live.sh` | [keycape-exposure-resolver-reconciliation.md](../docs/keycape-exposure-resolver-reconciliation.md) | historical check 2026-09-23 with predecessor unproven; current core exercised 2026-10-14 on disposable PI/LLDAP, full production procedure unexercised |
| `privacyidea/repair-realm-live.sh` | [privacyIDEA repair contract](../sso-mfa/k8s/privacyidea/README.md#configuration-preserving-realm-repair), [verify-t06.md](../docs/verify-t06.md) | source repaired: attended in-memory input and propagated functional proof; live acceptance pending | | `privacyidea/repair-realm-live.sh` | [privacyIDEA repair contract](../sso-mfa/k8s/privacyidea/README.md#configuration-preserving-realm-repair), [verify-t06.md](../docs/verify-t06.md) | source repaired: attended in-memory input and propagated functional proof; live acceptance pending |
| `privacyidea/update-lldap-resolver-live.sh` | None found | unknown | | `privacyidea/update-lldap-resolver-live.sh` | [keycape-exposure-resolver-reconciliation.md](keycape-exposure-resolver-reconciliation.md) | core exercised 2026-10-14 on disposable PI/LLDAP; production unexercised |
| `user-engine/verify-operability.sh` | None found | unknown | | `user-engine/verify-operability.sh` | None found | unknown |
| `verify-identity-cutover-dependencies.sh` | None found | unknown | | `verify-identity-cutover-dependencies.sh` | None found | unknown |
| `verify-t02.sh` | [README.md](../sso-mfa/k8s/README.md) | unknown | | `verify-t02.sh` | [README.md](../sso-mfa/k8s/README.md) | unknown |
@ -46,7 +46,6 @@ runbook or a dedicated procedure. Missing runbooks are listed separately.
## Scripts without a matching runbook ## Scripts without a matching runbook
- `sso-mfa/k8s/keycape/register-coulomb-social.sh` — header usage is available, but a runbook with prerequisites, verification and exercise history was not found in the reviewed runbook set. - `sso-mfa/k8s/keycape/register-coulomb-social.sh` — header usage is available, but a runbook with prerequisites, verification and exercise history was not found in the reviewed runbook set.
- `sso-mfa/k8s/privacyidea/update-lldap-resolver-live.sh` — header usage is available, but a runbook with prerequisites, verification and exercise history was not found in the reviewed runbook set.
- `sso-mfa/k8s/user-engine/verify-operability.sh` — header usage is available, but a runbook with prerequisites, verification and exercise history was not found in the reviewed runbook set. - `sso-mfa/k8s/user-engine/verify-operability.sh` — header usage is available, but a runbook with prerequisites, verification and exercise history was not found in the reviewed runbook set.
- `sso-mfa/k8s/verify-identity-cutover-dependencies.sh` — header usage is available, but a runbook with prerequisites, verification and exercise history was not found in the reviewed runbook set. - `sso-mfa/k8s/verify-identity-cutover-dependencies.sh` — header usage is available, but a runbook with prerequisites, verification and exercise history was not found in the reviewed runbook set.

8
docs/attended-procedure-standard.md
View file

@ -17,9 +17,11 @@ The first execution, on 2026-08-27, found four defects:
2. `GET /user/` parsed as a dict when privacyIDEA returns a list — an 2. `GET /user/` parsed as a dict when privacyIDEA returns a list — an
`AttributeError` outside the `except` clause, so the run died as a traceback `AttributeError` outside the `except` clause, so the run died as a traceback
rather than a receipt. rather than a receipt.
3. Resolver writes omitting `TIMEOUT`, `CACHE_TIMEOUT`, `SIZELIMIT` — a resolver 3. Missing resolver tuning (`TIMEOUT`, `CACHE_TIMEOUT`, `SIZELIMIT`) was
write replaces the whole object, so each run silently un-repaired a resolver historically attributed to whole-object replacement. Correction, 2026-10-14:
an operator had just fixed by hand. native privacyIDEA 3.12.2 preserves omitted fields; sending explicit defaults
can overwrite custom tuning. The maintained cutover helper updates only the
bind password and verifies preservation. See the current resolver runbook.
4. No way to declare an unavailable predecessor — the prompt could not be left 4. No way to declare an unavailable predecessor — the prompt could not be left
empty, so an operator had to type a placeholder, which also fails the bind empty, so an operator had to type a placeholder, which also fails the bind
and was recorded as a **passing** denial proof. and was recorded as a **passing** denial proof.

224
docs/keycape-exposure-resolver-reconciliation.md
View file

@ -1,138 +1,132 @@
# KeyCape exposure: privacyIDEA resolver reconciliation # KeyCape exposure: privacyIDEA resolver reconciliation
Incident: `KEYCAPE-EXPOSURE-20260823-01` Incident: `KEYCAPE-EXPOSURE-20260823-01`
Workplan: `NK-WP-0033` Historical workplan: `NK-WP-0033`; source maintenance: `NK-WP-0049-T02`
NetKingdom procedure: `sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh` (repaired 2026-08-27 in `4a38511`; the previously pinned revision could not complete a run — see NK-WP-0033 T05) Exercise status: core helper exercised 2026-10-14 by custodian-codex against
Platform recovery contract: railiance-platform `453fed3` local disposable privacyIDEA 3.12.2 and LLDAP, including actual LDAP bind,
Owner cutover receipt: State Hub message `45b236c8-052f-43d3-a472-44f8e9694da2` password update, preserved configuration, exact user lookup and HOTP. Terminal
Exercise status: `--check --predecessor-unavailable` exercised to PASS on 2026-09-23 by Bernd Worsch (railiance01 checkout `6096c395`, receipt in NK-WP-0033). `--apply` last ran on 2026-08-27 before the fixes in `4a38511`. Run it from your own terminal with `ssh -t`: Claude Code's `!` prefix has no TTY, and the script refuses to run without one. Take the pi-admin and LLDAP values from OpenBao `operators` custody (`privacyidea/pi-admin`, `lldap/admin`). See `attended-procedure-standard.md`. handling and failure paths have automated tests. The complete new attended
procedure has not run against the production cluster or KeyCape.
Evidence: Custodian `docs/evidence/2026-10-14-netkingdom-security/resolver-cutover.json`.
This is the remaining attended provider-admin operation after the owner-reported Both `sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh` and
four-class cutover. It updates only privacyIDEA resolver `lldap-coulomb` so the `update-lldap-resolver-live.sh` delegate to `resolver_cutover.py`. The update
resolver uses the replacement LLDAP bind credential. It does not modify realms, helper changes only an existing `lldap-coulomb` resolver's bind password; the
policies, tokens, KeyCape Secrets, or any other resolver. reconcile helper additionally checks cluster readiness, discovery, exact user
resolution, token-backed MFA and the supplied predecessor's HTTP denial.
Neither helper bootstraps a missing resolver or realm, repairs missing tuning,
or changes policies, tokens, directory credentials or Kubernetes resources.
No password, hash, token, Secret payload, or manifest belongs in this document, ## Authority and target pins
State Hub, Git, chat, command arguments, or ordinary logs.
## Resolver update semantics — corrected 2026-10-13 The historical platform recovery contract is railiance-platform `453fed3`;
owner cutover receipt is State Hub message
`45b236c8-052f-43d3-a472-44f8e9694da2`. These identify historical evidence,
not a fresh live grant. Founder-deferred exposure rotation remains deferred.
Disposable native privacyIDEA 3.12.2 tests and its owner API implementation prove Before any live run, the existing owner/supervisor path must bind a private
that omitted resolver fields are preserved. The historical general statement approval record to the current NetKingdom source revision and dependencies,
below that every POST replaces the whole object is superseded for this verified incident, provider-approved replacement, owner receipt, exact start/end window,
version. Sending explicit defaults can still overwrite custom tuning. Historical attended driver, independent abort operator and recovery plan. Confirm predecessor
incident receipts and their target-version limitations remain unchanged. provenance separately. Do not restore an exposed predecessor or bundle.
The current general realm repair sends only missing tuning fields on an existing A preview digest and TTY are technical preconditions, not authorization.
resolver and never rotates its bind credential or replays MFA policies. The
incident-specific attended cutover remains a separate approved procedure; do not
substitute or combine the two.
## Authority and pinning gate Reconcile requires an explicit absolute kubeconfig, context, expected API server
and `kube-system` namespace UID from the reviewed owner packet. It checks the
server and UID before workload reads. It never reads Kubernetes Secrets. The
five existing deployments must each have the current generation observed and
exactly one desired, actual, updated, ready and available replica, without a
deletion timestamp. No ambient default context is accepted.
The operator must record a private approval receipt containing, at minimum: Default service origins are `https://pink.coulomb.social`,
`https://lldap.coulomb.social` and `https://kc.coulomb.social`. Explicit literal
loopback URLs with ports are supported for approved forwards or fixtures.
Environment URL/tuning overrides and the old `--note` option are removed.
KeyCape discovery must identify the canonical `https://kc.coulomb.social`
issuer and its own authorization, token and JWKS endpoints, even through a forward.
The shared transport disables proxies and redirects and bounds strict JSON
responses; each extra service profile permits only its one required route and
never receives the privacyIDEA authorization token.
- incident `KEYCAPE-EXPOSURE-20260823-01`; ## Preview, apply and check
- the NetKingdom revision containing `reconcile-lldap-resolver-live.sh` and its adjacent `pi_api.py` and platform contract `453fed3`;
- the owner receipt `45b236c8-052f-43d3-a472-44f8e9694da2`;
- the exact start/end window, attended driver, and independent abort operator;
- confirmation that the replacement LLDAP credential is the provider-approved
value and that no exposed predecessor will be restored.
No live action is permitted if any revision, owner, cluster, or approval Use an attended terminal from the reviewed checkout. These are command templates;
identifier differs from the receipt. replace public placeholders with the approved target values. No secret belongs
in argv, environment, Git, State Hub, chat or ordinary logs.
## Preflight (metadata and health only) ```bash
bash sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh --preview \
--kubeconfig /absolute/approved/kubeconfig --context APPROVED_CONTEXT \
--server https://APPROVED_SERVER:6443 --cluster-uid APPROVED_NAMESPACE_UID
```
Run from the approved operator workstation, with the cluster context and Preview prompts only for pi-admin, authenticates with an admin-role check,
provider endpoint already authorized. Do not render any Secret data. validates the existing resolver/core/tuning/realm, and returns a public-state
`snapshot_sha256`. It performs no configuration write. It creates an auth session
and can produce provider audit events. The digest excludes the bind password and
binds the target settings and observed non-secret configuration. It is not a
credential fingerprint.
1. Verify the approved checkout contains `reconcile-lldap-resolver-live.sh` Run the same command with `--apply --expected-digest REVIEWED_DIGEST` in place
with mode `0755`; inspect its source, not live credential material. of `--preview`, retaining all target and user options. The helper prompts silently
2. Confirm LLDAP, privacyIDEA, KeyCape, Authelia, and for pi-admin, the approved replacement and the actual predecessor. Inputs stay
identity-provisioner are Ready using deployment/pod status fields only. in Python memory; no temporary secret files are created. Inherited shell tracing
3. Confirm privacyIDEA availability via the documented unauthenticated is disabled before input handling. A fresh OTP/PIN is requested only after
`/token/` probe (401/403 is expected) and KeyCape discovery health, configuration and exact user lookup succeed. `--user` defaults to `platform-root`.
discarding response bodies. Do not use a command that prints a bearer token
or configuration response.
4. Confirm the approved window, driver, abort operator, provider custody, and
cleanup workspace are ready. Stop on any drift or missing owner.
## Apply (one attended operation) Before mutation the helper authenticates the replacement to LLDAP and uses
privacyIDEA `/resolver/test` with the candidate password to prove an actual LDAP
bind. It tests supplied predecessor denial at LLDAP HTTP login and rechecks the
observed configuration. Apply sends exactly one resolver update containing only
`type` and `BINDPW`. Native privacyIDEA 3.12.2 preserves omitted fields. Readback
must preserve all other resolver data and every realm; a native resolver test
then proves the stored bind works. The API has no compare-and-swap: an exclusive
owner window remains required, and ambiguous writes are never retried.
1. From the approved checkout, run exactly: `--check` replaces `--preview` to perform the proofs without a configuration
write; it needs no digest. Check proves the stored bind works, not that its
censored value equals the supplied replacement. Authentication, audit and OTP
counter effects still occur. It is not an effect-free operation.
```text The smaller `update-lldap-resolver-live.sh` supports the same preview/apply/check
bash sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh --apply sequence without cluster flags. It proves candidate and stored binds plus exact
``` configuration preservation, but does not claim MFA, readiness or predecessor
acceptance. It must not substitute for full incident acceptance.
The helper securely prompts for pi-admin, replacement and predecessor ## Receipts and recovery
LLDAP passwords, and one approved MFA code; authenticates to privacyIDEA;
performs exactly one `POST /resolver/lldap-coulomb`; proves replacement
resolver lookup, privacyIDEA MFA, replacement LLDAP authentication, and
predecessor denial; rechecks readiness/health; securely cleans up; and
emits one sanitized PASS/FAIL receipt. It must not be combined with
`repair-realm-live.sh`, `bootstrap-realm.sh`, `creds-rotate.sh`, or any
full-bundle generator.
3. Stop immediately on any non-success response, timeout, unexpected endpoint,
or failed cleanup. Do not restore the exposed bundle or predecessor
credential.
## Postflight and predecessor denial Receipts contain fixed phase names, booleans, write counts and a public-state
digest; no raw responses, exceptions or free-form notes. `configured` means the
update helper completed its apply; `checked` means its no-write proof completed.
Reconcile returns `verified` only when all helper proofs pass. Passing a wrong
arbitrary password also produces denial: `supplied_predecessor_denied` is not
proof of predecessor provenance. The owner must match the input to custody.
Record only status codes, readiness, timestamps, revision identifiers, and Use `--predecessor-unavailable` honestly when it cannot be produced. Reconcile
boolean results. then returns `incomplete`, exit 2, even if the other proofs pass; never type a
placeholder to obtain a passing receipt. Failures and handled interruptions
return `unverified`, exit 1, with `write_attempted` showing whether a partial
change may exist. Exit 0 still leaves `incident_acceptance: unverified`: the
separate owner receipt and actual KeyCape MFA path are outside this helper.
1. Confirm privacyIDEA, LLDAP, KeyCape, Authelia, and identity-provisioner are Stop on drift, wrong target, failed health/readiness or any unverified result.
Ready again. Confirm the privacyIDEA health endpoint succeeds and the After an attempted write, reconcile observed state through the approved owner
resolver endpoint returns success without retaining its response body. path before another attempt. Recover forward with an approved replacement; no
2. Exercise one approved KeyCape MFA path that requires the `coulomb` realm. automatic rollback to an exposed predecessor. Record only sanitized evidence,
Record pass/fail only; never record the token or response body. source revision, target references and operator outcome. NK-WP-0049 T03 remains
3. Using protected file inputs, prove an LDAP bind with the replacement value the owner-native recovery/acceptance gate; this source repair does not close it.
succeeds and a bind with the exposed predecessor fails. The predecessor
test must be a boolean result and must not put the password in argv or
stdout. A failed predecessor bind is required evidence; do not retry it
against another provider.
4. Confirm the KeyCape owner’s existing four-class positive/negative receipt
remains associated with this resolver update. If any class lacks a receipt,
keep T05 open.
5. Securely remove the temporary workspace and record only cleanup success.
## Abort and rollback ## Historical receipts and corrections
Abort before mutation on revision drift, missing authority, unavailable health, The old `--check --predecessor-unavailable` printed PASS on 2026-09-23 when run by
uncertain workspace cleanup, or any unsafe helper output. Abort forward after Bernd Worsch from railiance01 checkout `6096c395` (receipt in NK-WP-0033).
mutation on a failed resolver response, failed readiness, failed replacement That remains historical check evidence, with predecessor denial unproven; it
MFA, or missing predecessor denial. The exposed bundle and every exposed cannot establish the new revision's acceptance. Historical apply last ran on
predecessor are never rollback material. Recovery after a partial write must 2026-08-27 before fixes in `4a38511`.
use a newly approved replacement value and revision, not a stale local bundle.
## Completion evidence The August account attributed missing tuning to whole-object replacement.
Native privacyIDEA 3.12.2 and source inspection corrected that interpretation on
T03 may move to done only after the helper run and cleanup receipt are recorded 2026-10-13: omitted fields are preserved, while sending explicit defaults can
by the attended operator. T05 may move to done only after the resolver’s overwrite custom tuning. The October helper therefore sends only the bind
replacement success, predecessor denial, owner cutover receipt, and all password and type. Historic claims, default-overwriting commands, temporary-file
residual limitations are recorded as sanitized evidence. handling and raw `--note` receipts are superseded by this procedure.
## Revision note — 2026-08-27
The pinned revision of `reconcile-lldap-resolver-live.sh` had never completed a
run. Four defects were found by running it and are fixed in `4a38511`; see
`NK-WP-0033` T05 for the full findings. Two change how this procedure is
invoked:
- `--predecessor-unavailable` — use when the exposed predecessor cannot be
produced. The denial bind is not attempted and the receipt records
`predecessor denial=NOT-PROVEN`. Do **not** type a placeholder at the
predecessor prompt instead: a wrong value also fails the bind, and the run
records it as a *passing* denial proof — a receipt asserting a test that never
ran.
- `--note TEXT` — operator context carried verbatim in the receipt line, so a
claim and its caveat travel together. One line, 200 characters, no
credentials.
`TIMEOUT`, `CACHE_TIMEOUT` and `SIZELIMIT` are now sent with the resolver body
(default 5 / 120 / 500, overridable via the matching `LDAP_*` environment
variables). Before this fix, `--apply` dropped them, because a resolver write
replaces the whole object. A resolver with them unset still resolves users, but
the WebUI refuses to save or test it — so a hand repair was silently reverted by
the next run.

8
sso-mfa/k8s/privacyidea/README.md
View file

@ -311,3 +311,11 @@ replacement claim; it does not reclassify historical deployment evidence.
Reference: [provider resolver API](https://privacyidea.readthedocs.io/en/v3.12.2/modules/api/resolver.html). Reference: [provider resolver API](https://privacyidea.readthedocs.io/en/v3.12.2/modules/api/resolver.html).
Provider/package checks do not prove live LDAP reachability, current custody, Provider/package checks do not prove live LDAP reachability, current custody,
installed hooks, user enrollment or a complete KeyCape browser sign-in. installed hooks, user enrollment or a complete KeyCape browser sign-in.
## Attended resolver password reconciliation
The update and reconcile helpers share `resolver_cutover.py`. Follow the
[current procedure](../../../docs/keycape-exposure-resolver-reconciliation.md)
for preview, digest-bound minimal apply, explicit cluster pins and truthful
partial receipts. Credentials stay in attended process memory. Native fixture
proof is not production acceptance or authority to perform exposure rotation.

28
sso-mfa/k8s/privacyidea/pi_api.py
View file

@ -11,12 +11,36 @@ from mfa_state import NoRedirect, endpoint, unique_object
MAX_BODY = 1024 * 1024 MAX_BODY = 1024 * 1024
def request(url, payload=None, token=None, *, method=None): def service_endpoint(value, profile='privacyidea'):
"""Named destinations; profiles never inherit each other's credential scope."""
if profile == 'privacyidea':
return endpoint(value)
hosts = {'lldap-auth': 'lldap.coulomb.social', 'keycape-discovery': 'kc.coulomb.social'}
if profile not in hosts:
raise ValueError()
if value in ('https://' + hosts[profile], 'https://' + hosts[profile] + ':443'):
return value
base = endpoint(value)
if urllib.parse.urlsplit(base).hostname not in ('127.0.0.1', '::1'):
raise ValueError()
return base
def request(url, payload=None, token=None, *, method=None, profile='privacyidea'):
try: try:
parsed = urllib.parse.urlsplit(url) parsed = urllib.parse.urlsplit(url)
base = endpoint(urllib.parse.urlunsplit((parsed.scheme, parsed.netloc, '', '', ''))) base = service_endpoint(urllib.parse.urlunsplit((parsed.scheme, parsed.netloc, '', '', '')), profile)
if parsed.fragment or not parsed.path.startswith('/') or any(ord(c) < 32 for c in url): if parsed.fragment or not parsed.path.startswith('/') or any(ord(c) < 32 for c in url):
return 0, None return 0, None
verb = method or ('POST' if payload is not None else 'GET')
if profile != 'privacyidea':
if token is not None or parsed.query: return 0, None
if profile == 'lldap-auth':
if (verb != 'POST' or parsed.path != '/auth/simple/login' or not isinstance(payload, dict)
or set(payload) != {'username', 'password'} or payload['username'] != 'admin'
or not isinstance(payload['password'], str) or not payload['password']): return 0, None
elif verb != 'GET' or parsed.path != '/.well-known/openid-configuration' or payload is not None:
return 0, None
# Origin validated before creating a request containing protected values. # Origin validated before creating a request containing protected values.
headers = {'Accept': 'application/json'} headers = {'Accept': 'application/json'}
data = None data = None

294
sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh
View file

@ -1,293 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# reconcile-lldap-resolver-live.sh — one-command attended resolver cutover. # Attended inputs stay in Python memory; disable inherited tracing first.
# set +x
# Performs the narrow privacyIDEA lldap-coulomb update and its safe proof set:
# replacement LLDAP authentication, resolver lookup, one privacyIDEA MFA check,
# predecessor LLDAP denial, health/readiness, and cleanup. It never reads a
# Kubernetes Secret and never prints a credential, token, response body, or
# manifest.
#
# Usage:
# ./reconcile-lldap-resolver-live.sh --check # read-only proof
# ./reconcile-lldap-resolver-live.sh --apply # resolver update + proof
#
# Options:
# --note TEXT operator context recorded verbatim in the receipt
# --predecessor-unavailable the exposed predecessor cannot be produced; the
# denial bind is not attempted and the receipt
# records it as NOT-PROVEN
set -euo pipefail set -euo pipefail
exec python3 "$(dirname -- "${BASH_SOURCE[0]}")/resolver_cutover.py" reconcile "$@"
MODE=""
NOTE=""
PREDECESSOR="required"
usage() {
echo "Usage: $0 --check|--apply [--note TEXT] [--predecessor-unavailable]" >&2
exit 2
}
while [[ $# -gt 0 ]]; do
case "$1" in
--check|--apply)
if [[ -n "$MODE" ]]; then usage; fi
MODE="$1"; shift ;;
--note)
# Never put a credential here: the receipt goes to a terminal and
# is copied into the incident record.
if [[ -z "${2:-}" ]]; then usage; fi
NOTE="$2"; shift 2 ;;
--predecessor-unavailable)
# Typing a placeholder at the predecessor prompt also fails the
# bind, and would be recorded as a passing denial proof. An absent
# proof is honest; a fabricated one asserts a test that never ran.
PREDECESSOR="unavailable"; shift ;;
*)
usage ;;
esac
done
if [[ "$MODE" != "--apply" && "$MODE" != "--check" ]]; then
usage
fi
# One bounded line: a receipt that breaks its own format is not evidence
# anyone can read back.
NOTE="${NOTE//$'\n'/ }"
NOTE="${NOTE//$'\r'/ }"
if (( ${#NOTE} > 200 )); then
echo "ERROR: --note must be 200 characters or fewer." >&2
exit 2
fi
if [[ ! -t 0 ]]; then
echo "ERROR: --check/--apply requires an interactive terminal." >&2
exit 2
fi
PI_URL="${PI_URL:-https://pink.coulomb.social}"
LLDAP_AUTH_URL="${LLDAP_AUTH_URL:-https://lldap.coulomb.social/auth/simple/login}"
LLDAP_URL="${LLDAP_URL:-ldap://lldap.sso.svc.cluster.local:3890}"
LLDAP_BASE_DN="${LLDAP_BASE_DN:-dc=netkingdom,dc=local}"
LLDAP_BIND_DN="${LLDAP_BIND_DN:-uid=admin,ou=people,dc=netkingdom,dc=local}"
RESOLVER_NAME="${RESOLVER_NAME:-lldap-coulomb}"
MFA_USER="${MFA_USER:-platform-root}"
MFA_REALM="${MFA_REALM:-coulomb}"
KEYCAPE_DISCOVERY_URL="${KEYCAPE_DISCOVERY_URL:-https://kc.coulomb.social/.well-known/openid-configuration}"
# A resolver write replaces the whole object, so every field the body omits is
# dropped. A resolver with these unset still resolves users, but the WebUI
# refuses to save or test it until they are filled in — so omitting them here
# silently un-repairs a resolver an operator fixed by hand.
LDAP_TIMEOUT="${LDAP_TIMEOUT:-5}"
LDAP_CACHE_TIMEOUT="${LDAP_CACHE_TIMEOUT:-120}"
LDAP_SIZELIMIT="${LDAP_SIZELIMIT:-500}"
for _v in LDAP_TIMEOUT LDAP_CACHE_TIMEOUT LDAP_SIZELIMIT; do
if [[ ! "${!_v}" =~ ^[0-9]+$ ]]; then
echo "ERROR: $_v must be a non-negative integer, got '${!_v}'." >&2
exit 2
fi
done
unset _v
tmp="$(mktemp -d)"
chmod 700 "$tmp"
cleanup() {
for file in pi-admin lldap-new lldap-old otp phase; do
if [[ -f "$tmp/$file" ]]; then
shred -u "$tmp/$file" 2>/dev/null || rm -f "$tmp/$file"
fi
done
rmdir "$tmp" 2>/dev/null || true
}
cleanup_ok=0
trap cleanup EXIT INT TERM
prompt_secret() {
local label="$1" target="$2"
printf '%s: ' "$label" >&2
IFS= read -r -s value
printf '\n' >&2
if [[ -z "$value" ]]; then
echo "ERROR: empty protected input." >&2
exit 2
fi
printf '%s' "$value" > "$target"
unset value
chmod 600 "$target"
}
prompt_secret "privacyIDEA pi-admin password" "$tmp/pi-admin"
prompt_secret "replacement LLDAP bind/admin password" "$tmp/lldap-new"
if [[ "$PREDECESSOR" == "required" ]]; then
prompt_secret "exposed predecessor LLDAP password (denial proof only)" "$tmp/lldap-old"
else
echo " [INFO] predecessor unavailable: denial proof recorded as NOT-PROVEN." >&2
fi
prompt_secret "one-time MFA code for $MFA_USER@$MFA_REALM" "$tmp/otp"
PI_HELPER_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
if PYTHONPATH="$PI_HELPER_DIR${PYTHONPATH:+:$PYTHONPATH}" python3 - "$tmp/pi-admin" "$tmp/lldap-new" "$tmp/lldap-old" "$tmp/otp" \
"$PI_URL" "$LLDAP_AUTH_URL" "$LLDAP_URL" "$LLDAP_BASE_DN" "$LLDAP_BIND_DN" \
"$RESOLVER_NAME" "$MFA_USER" "$MFA_REALM" "$KEYCAPE_DISCOVERY_URL" "$MODE" "$tmp/phase" "$PREDECESSOR" \
"$LDAP_TIMEOUT" "$LDAP_CACHE_TIMEOUT" "$LDAP_SIZELIMIT" <<'PY'
import json
import subprocess
import sys
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path
(
pi_path, new_path, old_path, otp_path, pi_url, lldap_auth_url, lldap_url,
base_dn, bind_dn, resolver_name, mfa_user, mfa_realm, discovery_url, mode, phase_path,
predecessor, ldap_timeout, ldap_cache_timeout, ldap_sizelimit,
) = sys.argv[1:]
def phase(name: str) -> None:
Path(phase_path).write_text(name, encoding="ascii")
def secret(path: str) -> str:
value = Path(path).read_text(encoding="utf-8")
if not value:
raise RuntimeError("empty protected input")
return value
from pi_api import request
def check_k8s_ready() -> None:
workloads = (("sso", "lldap"), ("mfa", "privacyidea"), ("sso", "keycape"),
("sso", "authelia"), ("sso", "identity-provisioner"))
for namespace, name in workloads:
proc = subprocess.run(
["kubectl", "get", "deployment", name, "-n", namespace,
"-o", "jsonpath={.status.readyReplicas}/{.status.replicas}"],
capture_output=True, text=True, timeout=20,
)
if proc.returncode != 0 or proc.stdout.strip() != "1/1":
raise RuntimeError(f"readiness failed: {namespace}/{name}")
def check_health() -> None:
# privacyIDEA intentionally exposes no unauthenticated HTTP health route;
# /token/ is documented by the deployment as a safe availability probe.
for label, url, accepted in (
("privacyidea", pi_url + "/token/", {200, 401, 403}),
("keycape", discovery_url, set(range(200, 400))),
):
status, _ = request(url)
if status not in accepted:
raise RuntimeError(f"health failed: {label}")
def lldap_login(password: str) -> tuple[int, bool]:
status, body = request(lldap_auth_url, {"username": "admin", "password": password})
return status, status == 200 and isinstance(body, dict) and bool(body.get("token"))
try:
phase("preflight")
check_k8s_ready()
check_health()
phase("privacyidea-auth")
status, auth = request(pi_url + "/auth", {"username": "pi-admin", "password": secret(pi_path)})
pi_token = str((auth or {}).get("result", {}).get("value", {}).get("token", ""))
if status != 200 or not pi_token:
raise RuntimeError("privacyIDEA authentication failed")
# Prove the provider-approved replacement and predecessor disposition
# before any resolver mutation is attempted.
phase("replacement-lldap-auth")
new_status, new_authenticated = lldap_login(secret(new_path))
if new_status != 200 or not new_authenticated:
raise RuntimeError("replacement LLDAP authentication failed")
if predecessor == "unavailable":
# Not a pass. The bind is not attempted, so this run claims nothing
# about the predecessor's disposition.
phase("predecessor-denial-not-proven")
else:
phase("predecessor-denial")
old_status, old_authenticated = lldap_login(secret(old_path))
if old_status not in (401, 403) or old_authenticated:
raise RuntimeError("predecessor LLDAP authentication was not denied")
resolver_body = {
"type": "ldapresolver", "LDAPURI": lldap_url, "BINDDN": bind_dn,
"BINDPW": secret(new_path), "LDAPBASE": base_dn,
"LOGINNAMEATTRIBUTE": "uid", "LDAPSEARCHFILTER": "(objectClass=inetOrgPerson)",
"LDAPFILTER": "(&(objectClass=inetOrgPerson)(uid=%s))",
"USERINFO": json.dumps({"username": "uid", "phone": "telephoneNumber", "mobile": "mobile", "email": "mail", "surname": "sn", "givenname": "givenName"}),
"UIDTYPE": "uid", "NOREFERRALS": True, "NOSCHEMAS": True,
"TIMEOUT": int(ldap_timeout), "CACHE_TIMEOUT": int(ldap_cache_timeout),
"SIZELIMIT": int(ldap_sizelimit),
}
if mode == "--apply":
phase("resolver-update")
status, result = request(pi_url + "/resolver/" + resolver_name, resolver_body, pi_token)
if status != 200 or (result or {}).get("result", {}).get("status") not in (True, "true", "True"):
raise RuntimeError("resolver update failed")
else:
phase("resolver-read-only-check")
phase("resolver-lookup")
status, result = request(
pi_url + f"/user/?realm={mfa_realm}&username={urllib.parse.quote(mfa_user)}",
token=pi_token,
)
# privacyIDEA returns result.value as a list of user objects for /user/.
# Accept the dict-with-"users" shape too: other endpoints use it, and a
# lookup that guesses wrong raises AttributeError past the except clause.
_value = (result or {}).get("result", {}).get("value", [])
if isinstance(_value, dict):
_value = _value.get("users", [])
users = _value if isinstance(_value, list) else []
if status != 200:
phase(f"resolver-lookup-http-{status}")
raise RuntimeError("replacement resolver lookup failed")
if not users:
phase("resolver-lookup-empty")
raise RuntimeError("replacement resolver lookup failed")
phase("privacyidea-mfa")
status, result = request(
pi_url + "/validate/check",
{"user": mfa_user, "realm": mfa_realm, "pass": secret(otp_path)},
pi_token,
)
if status != 200 or (result or {}).get("result", {}).get("value") is not True:
raise RuntimeError("replacement MFA validation failed")
phase("postflight")
check_k8s_ready()
check_health()
except (OSError, RuntimeError, subprocess.SubprocessError,
AttributeError, TypeError, KeyError, ValueError) as exc:
# A malformed reply must still produce a receipt naming the phase it died
# in. A traceback tells the operator nothing about what was proven.
raise SystemExit(1)
PY
then
rc=0
else
rc=$?
fi
phase_result="$(cat "$tmp/phase" 2>/dev/null || echo unknown)"
if [[ "$PREDECESSOR" == "required" ]]; then
pred_text="predecessor denial"
else
pred_text="predecessor denial=NOT-PROVEN"
fi
note_text=""
if [[ -n "$NOTE" ]]; then
note_text="; note=$NOTE"
fi
cleanup
trap - EXIT INT TERM
if [[ "$rc" -ne 0 ]]; then
echo "NK-WP-0033 receipt FAIL: phase=$phase_result; $pred_text; cleanup=PASS$note_text" >&2
exit "$rc"
fi
if [[ -d "$tmp" ]]; then
echo "NK-WP-0033 receipt FAIL: cleanup=FAIL$note_text" >&2
exit 1
fi
if [[ "$MODE" == "--check" ]]; then
echo "NK-WP-0033 receipt PASS: read-only resolver lookup, privacyIDEA MFA, $pred_text, readiness, health, cleanup=PASS$note_text"
else
echo "NK-WP-0033 receipt PASS: resolver lookup, privacyIDEA MFA, $pred_text, readiness, health, cleanup=PASS$note_text"
fi

218
sso-mfa/k8s/privacyidea/resolver_cutover.py Normal file
View file

@ -0,0 +1,218 @@
#!/usr/bin/env python3
"""Attended resolver password cutover; no secret files or ambient target overrides."""
import copy
import getpass
import hashlib
import json
from pathlib import Path
import re
import signal
import subprocess
import sys
import urllib.parse
from mfa_state import QuietParser, endpoint
from pi_api import request, service_endpoint
import realm_bootstrap as realm
from realm_bootstrap import Client, Refused, password
from verify_mfa import verify
WORKLOADS = (('sso', 'lldap'), ('mfa', 'privacyidea'), ('sso', 'keycape'),
('sso', 'authelia'), ('sso', 'identity-provisioner'))
def public_snapshot(snapshot):
result = copy.deepcopy(snapshot)
result['resolver']['data'].pop('BINDPW', None)
return result
def existing(snapshot):
changes = realm.plan(snapshot)
if any(changes.values()):
raise Refused()
# Never fingerprint a password, even if a provider stops censoring it.
data = snapshot['resolver']['data']
if data.get('BINDPW') != '__CENSORED__':
raise Refused()
return snapshot
def digest(snapshot, target):
return hashlib.sha256(json.dumps({'state': public_snapshot(snapshot), 'target': target},
sort_keys=True, separators=(',', ':')).encode()).hexdigest()
def native_bind(client, snapshot, candidate='__CENSORED__'):
payload = dict(snapshot['resolver']['data'], resolver=realm.RESOLVER,
type='ldapresolver', BINDPW=password(candidate))
if client.call('/resolver/test', payload) is not True:
raise Refused()
def login(base, candidate, *, denied=False):
status, body = request(base + '/auth/simple/login',
{'username': 'admin', 'password': password(candidate)}, profile='lldap-auth')
if denied:
if status not in (401, 403): raise Refused()
elif status != 200 or not isinstance(body, dict) or not isinstance(body.get('token'), str) or not body['token']:
raise Refused()
def cluster_ready(target):
prefix = ['kubectl', '--kubeconfig', target['kubeconfig'], '--context', target['context'], '--request-timeout=15s']
def read(args):
result = subprocess.run(prefix + args, capture_output=True, text=True, timeout=20)
if result.returncode or len(result.stdout) > 4096: raise Refused()
return result.stdout.strip()
if read(['config', 'view', '--minify', '-o', 'jsonpath={.clusters[0].cluster.server}']) != target['server']:
raise Refused()
if read(['get', 'namespace', 'kube-system', '-o', 'jsonpath={.metadata.uid}']) != target['cluster_uid']:
raise Refused()
fields = ('metadata.generation', 'status.observedGeneration', 'spec.replicas', 'status.replicas',
'status.updatedReplicas', 'status.readyReplicas', 'status.availableReplicas', 'metadata.deletionTimestamp')
expression = '{"|"}'.join('{.' + field + '}' for field in fields)
for namespace, name in WORKLOADS:
values = read(['get', 'deployment', name, '-n', namespace, '-o', 'jsonpath=' + expression]).split('|')
if (len(values) != 8 or not values[0].isdigit() or int(values[0]) < 1 or values[0] != values[1]
or values[2:7] != ['1'] * 5 or values[7]): raise Refused()
def health(target):
status, _ = request(target['pi_url'] + '/token/')
if status not in (200, 401, 403): raise Refused()
base = target['keycape_url']
issuer = 'https://kc.coulomb.social'
status, body = request(base + '/.well-known/openid-configuration', profile='keycape-discovery')
if status != 200 or not isinstance(body, dict) or body.get('issuer') != issuer: raise Refused()
for key in ('authorization_endpoint', 'token_endpoint', 'jwks_uri'):
value = body.get(key)
if not isinstance(value, str) or not value.startswith(issuer + '/'):
raise Refused()
parsed = urllib.parse.urlsplit(value)
if parsed.fragment or parsed.username or parsed.password or any(ord(c) <= 32 for c in value): raise Refused()
def run(client, target, mode, *, replacement=None, predecessor=None, expected_digest=None,
mfa_proof=None, gate=None, report=None):
report = report if report is not None else {}
report.update(result='unverified', mode=mode, helper=target['helper'], phase='preflight', write_attempted=False, configuration_writes=0,
mfa_verified=False, supplied_predecessor_denied=False, incident_acceptance='unverified')
full = target['helper'] == 'reconcile'
if full: gate()
report['phase'] = 'snapshot'
initial = existing(client.snapshot())
fingerprint = digest(initial, target)
report['snapshot_sha256'] = fingerprint
if mode == 'preview':
report.update(result='planned', phase='complete')
return report
if mode == 'apply' and expected_digest != fingerprint: raise Refused()
report['phase'] = 'replacement-authentication'
if replacement == '__CENSORED__': raise Refused()
login(target['lldap_url'], replacement)
native_bind(client, initial, replacement)
if full and predecessor is not None:
report['phase'] = 'predecessor-denial'
if predecessor == replacement: raise Refused()
login(target['lldap_url'], predecessor, denied=True)
report['supplied_predecessor_denied'] = True
# No CAS in privacyIDEA: owner must still reserve an exclusive change window.
report['phase'] = 'prewrite-recheck'
if client.snapshot() != initial: raise Refused()
if mode == 'apply':
report.update(phase='resolver-update', write_attempted=True)
value = client.call('/resolver/' + realm.RESOLVER, {'type': 'ldapresolver', 'BINDPW': replacement})
if type(value) is not int or value < 1: raise Refused()
report['configuration_writes'] = 1
report['phase'] = 'stored-bind-verification'
after = existing(client.snapshot())
if public_snapshot(after) != public_snapshot(initial): raise Refused()
native_bind(client, after)
if full:
report['phase'] = 'exact-user-lookup'
query = urllib.parse.urlencode({'realm': realm.REALM, 'username': target['user']})
users = client.call('/user/?' + query)
if (not isinstance(users, list) or len(users) != 1 or not isinstance(users[0], dict)
or users[0].get('username') != target['user'] or users[0].get('resolver') != realm.RESOLVER):
raise Refused()
report['phase'] = 'token-backed-mfa'
if mfa_proof().get('result') != 'PASS': raise Refused()
report['mfa_verified'] = True
report['phase'] = 'postflight'
gate()
if client.snapshot() != after: raise Refused()
# An arbitrary wrong password can also be denied. Owner provenance is a
# separate acceptance requirement; this receipt only describes supplied input.
report.update(phase='complete', result=('verified' if predecessor is not None else 'incomplete') if full else ('configured' if mode == 'apply' else 'checked'))
return report
def arguments():
parser = QuietParser(description=__doc__)
parser.add_argument('helper', choices=('update', 'reconcile'))
modes = parser.add_mutually_exclusive_group(required=True)
for mode in ('preview', 'check', 'apply'): modes.add_argument('--' + mode, dest='mode', action='store_const', const=mode)
parser.add_argument('--expected-digest')
parser.add_argument('--pi-url', default='https://pink.coulomb.social')
parser.add_argument('--lldap-url', default='https://lldap.coulomb.social')
parser.add_argument('--keycape-url', default='https://kc.coulomb.social')
parser.add_argument('--user', default='platform-root')
parser.add_argument('--predecessor-unavailable', action='store_true')
for name in ('kubeconfig', 'context', 'server', 'cluster-uid'): parser.add_argument('--' + name)
args = parser.parse_args()
args.pi_url = endpoint(args.pi_url)
args.lldap_url = service_endpoint(args.lldap_url, 'lldap-auth')
args.keycape_url = service_endpoint(args.keycape_url, 'keycape-discovery')
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._-]{0,127}', args.user): raise Refused()
if args.mode == 'apply':
if not re.fullmatch(r'[0-9a-f]{64}', args.expected_digest or ''): raise Refused()
elif args.expected_digest is not None: raise Refused()
if args.helper == 'reconcile':
if not args.kubeconfig or not Path(args.kubeconfig).is_absolute(): raise Refused()
for value in (args.context, args.cluster_uid):
if not isinstance(value, str) or not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._:-]{0,199}', value): raise Refused()
parsed = urllib.parse.urlsplit(args.server or '')
if (parsed.scheme != 'https' or not parsed.hostname or parsed.username or parsed.password
or parsed.query or parsed.fragment or parsed.path or any(ord(c) <= 32 for c in args.server)):
raise Refused()
if not sys.stdin.isatty() or not sys.stderr.isatty(): raise Refused()
return args
def main():
report = {'result': 'unverified', 'phase': 'input', 'write_attempted': False}
try:
args = arguments()
# Do not silently fall back to echoed stdin when terminal access fails.
import warnings
warnings.simplefilter('error', getpass.GetPassWarning)
signal.signal(signal.SIGTERM, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt()))
target = {key: getattr(args, key) for key in ('helper', 'pi_url', 'lldap_url', 'keycape_url', 'user',
'kubeconfig', 'context', 'server', 'cluster_uid')}
gate = lambda: (cluster_ready(target), health(target))
if args.helper == 'reconcile': gate()
report['phase'] = 'privacyidea-authentication'
admin = password(getpass.getpass('privacyIDEA pi-admin password: '))
client = Client(args.pi_url, admin)
replacement = predecessor = None
if args.mode != 'preview':
replacement = password(getpass.getpass('Approved replacement LLDAP password: '))
if args.helper == 'reconcile' and not args.predecessor_unavailable:
predecessor = password(getpass.getpass('Actual predecessor LLDAP password (denial proof only): '))
proof = lambda: verify(args.pi_url, args.user, realm.REALM, realm.RESOLVER, admin,
lambda: password(getpass.getpass('Fresh MFA code (include token PIN if required): ')))
run(client, target, args.mode, replacement=replacement, predecessor=predecessor,
expected_digest=args.expected_digest, mfa_proof=proof, gate=gate, report=report)
print(json.dumps(report, sort_keys=True))
return 2 if report['result'] == 'incomplete' else 0
except (Exception, KeyboardInterrupt):
report['result'] = 'unverified'
print(json.dumps(report, sort_keys=True), file=sys.stderr)
return 1
if __name__ == '__main__':
raise SystemExit(main())

126
sso-mfa/k8s/privacyidea/update-lldap-resolver-live.sh
View file

@ -1,125 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# update-lldap-resolver-live.sh — attended, resolver-only privacyIDEA update. # Attended inputs stay in Python memory; disable inherited tracing first.
# set +x
# This does not repair realms or policies. It updates only the persisted
# lldap-coulomb resolver after the LLDAP bind credential has changed.
#
# Usage:
# ./update-lldap-resolver-live.sh --apply
#
# The operator supplies both passwords interactively. Values are kept in a
# mode-0700 temporary directory and mode-0600 files, passed to a child process
# by pathname, and never printed or placed in command arguments.
set -euo pipefail set -euo pipefail
exec python3 "$(dirname -- "${BASH_SOURCE[0]}")/resolver_cutover.py" update "$@"
if [[ "${1:-}" != "--apply" || "${2:-}" != "" ]]; then
echo "Usage: $0 --apply" >&2
exit 2
fi
if [[ ! -t 0 ]]; then
echo "ERROR: --apply requires an interactive terminal." >&2
exit 2
fi
PI_URL="${PI_URL:-https://pink.coulomb.social}"
RESOLVER_NAME="${RESOLVER_NAME:-lldap-coulomb}"
LLDAP_URL="${LLDAP_URL:-ldap://lldap.sso.svc.cluster.local:3890}"
LLDAP_BASE_DN="${LLDAP_BASE_DN:-dc=netkingdom,dc=local}"
LLDAP_BIND_DN="${LLDAP_BIND_DN:-uid=admin,ou=people,dc=netkingdom,dc=local}"
tmp="$(mktemp -d)"
chmod 700 "$tmp"
cleanup() {
if [[ -f "$tmp/pi-admin" ]]; then shred -u "$tmp/pi-admin" 2>/dev/null || rm -f "$tmp/pi-admin"; fi
if [[ -f "$tmp/lldap-bind" ]]; then shred -u "$tmp/lldap-bind" 2>/dev/null || rm -f "$tmp/lldap-bind"; fi
rmdir "$tmp" 2>/dev/null || true
}
trap cleanup EXIT INT TERM
printf 'privacyIDEA pi-admin password: ' >&2
IFS= read -r -s pi_admin_password
printf '\n' >&2
printf 'LLDAP bind/admin password: ' >&2
IFS= read -r -s lldap_bind_password
printf '\n' >&2
if [[ -z "$pi_admin_password" || -z "$lldap_bind_password" ]]; then
echo "ERROR: passwords must not be empty." >&2
exit 2
fi
printf '%s' "$pi_admin_password" > "$tmp/pi-admin"
printf '%s' "$lldap_bind_password" > "$tmp/lldap-bind"
unset pi_admin_password lldap_bind_password
chmod 600 "$tmp/pi-admin" "$tmp/lldap-bind"
python3 - "$tmp/pi-admin" "$tmp/lldap-bind" "$PI_URL" "$RESOLVER_NAME" \
"$LLDAP_URL" "$LLDAP_BASE_DN" "$LLDAP_BIND_DN" <<'PY'
import json
import sys
import urllib.error
import urllib.request
from pathlib import Path
pi_path, ldap_path, base_url, resolver, ldap_url, base_dn, bind_dn = sys.argv[1:]
def read_secret(path: str) -> str:
value = Path(path).read_text(encoding="utf-8")
if not value:
raise RuntimeError("empty protected input")
return value
def post(path: str, payload: dict, token: str | None = None) -> dict:
headers = {"Content-Type": "application/json"}
if token is not None:
headers["Authorization"] = token
request = urllib.request.Request(
base_url.rstrip("/") + path,
data=json.dumps(payload).encode("utf-8"),
headers=headers,
method="POST",
)
try:
with urllib.request.urlopen(request, timeout=20) as response:
return json.load(response)
except urllib.error.HTTPError as exc:
raise RuntimeError(f"HTTP status {exc.code}") from None
except (urllib.error.URLError, TimeoutError):
raise RuntimeError("request failed") from None
try:
auth = post("/auth", {"username": "pi-admin", "password": read_secret(pi_path)})
token = str(auth.get("result", {}).get("value", {}).get("token", ""))
if not token:
raise RuntimeError("privacyIDEA authentication failed")
resolver_body = {
"type": "ldapresolver",
"LDAPURI": ldap_url,
"BINDDN": bind_dn,
"BINDPW": read_secret(ldap_path),
"LDAPBASE": base_dn,
"LOGINNAMEATTRIBUTE": "uid",
"LDAPSEARCHFILTER": "(objectClass=inetOrgPerson)",
"LDAPFILTER": "(&(objectClass=inetOrgPerson)(uid=%s))",
"USERINFO": json.dumps({
"username": "uid",
"phone": "telephoneNumber",
"mobile": "mobile",
"email": "mail",
"surname": "sn",
"givenname": "givenName",
}),
"UIDTYPE": "uid",
"NOREFERRALS": True,
"NOSCHEMAS": True,
}
result = post(f"/resolver/{resolver}", resolver_body, token)
status = result.get("result", {}).get("status")
if status not in (True, "true", "True"):
raise RuntimeError("privacyIDEA resolver update was rejected")
except RuntimeError as exc:
print(f"ERROR: {exc}", file=sys.stderr)
raise SystemExit(1)
print("privacyIDEA resolver update: PASS")
PY

141
tests/resolver_cutover_native_fixture.py Normal file
View file

@ -0,0 +1,141 @@
"""Local LLDAP + privacyIDEA 3.12.2 bind/cutover/HOTP proof; synthetic data only."""
import contextlib
import io
import json
import logging
import os
from pathlib import Path
import sys
import tempfile
import threading
def provider_run(lldap_url, ldap_url, replacement):
for name in list(os.environ):
if name.startswith('PRIVACYIDEA_'):del os.environ[name]
logging.disable(logging.CRITICAL)
sys.path.insert(0,str(Path(__file__).resolve().parents[1]/'sso-mfa/k8s/privacyidea'))
import realm_bootstrap as r
import resolver_cutover as cut
from privacyidea.lib.token import init_token
from privacyidea.lib.user import User
r.CORE = dict(LDAPURI=ldap_url, LDAPBASE="dc=example,dc=test", BINDDN="uid=admin,ou=people,dc=example,dc=test")
from privacyidea.app import create_app
from privacyidea.models import db
from privacyidea.lib.auth import create_db_admin
from privacyidea.lib.policy import PolicyClass,set_policy
from privacyidea.lib.resolver import get_resolver_config,save_resolver
from privacyidea.lib.realm import set_realm,set_default_realm
from werkzeug.serving import make_server
with tempfile.TemporaryDirectory(prefix='realm-native-') as directory:
root=Path(directory);(root/'enckey').write_bytes(os.urandom(96))
cfg=root/'fixture.cfg'
config=dict(SQLALCHEMY_DATABASE_URI='sqlite:///'+str(root/'db.sqlite'),SECRET_KEY='fixture-secret-key',
PI_PEPPER='fixture-pepper',PI_ENCFILE=str(root/'enckey'),PI_NO_RESPONSE_SIGN=True,
PI_AUDIT_NO_SIGN=True,PI_LOGFILE=str(root/'log'),PI_TRUSTED_JWT=[],
PI_AUDIT_SQL_URI='sqlite:///'+str(root/'audit.sqlite'))
cfg.write_text('\n'.join(k+'='+repr(v) for k,v in config.items()))
app=create_app(config_name='testing',config_file=str(cfg),silent=True)
with app.app_context():
assert str(db.engine.url)==config['SQLALCHEMY_DATABASE_URI'];db.create_all()
from privacyidea.lib.audit import getAudit
from privacyidea.lib.auditmodules.sqlaudit import LogEntry
LogEntry.__table__.create(getAudit(app.config).engine,checkfirst=True)
create_db_admin('pi-admin',password='fixture-admin-password')
policies=PolicyClass().policies
server=make_server('127.0.0.1',0,app);thread=threading.Thread(target=server.serve_forever,daemon=True);thread.start()
try:
client=r.Client(f'http://127.0.0.1:{server.server_port}','fixture-admin-password')
r.repair(client,lambda:'synthetic-invalid-predecessor',apply=True)
with app.app_context():
save_resolver({'resolver':r.RESOLVER,'type':'ldapresolver','TIMEOUT':'7','SIZELIMIT':'777','CACHE_TIMEOUT':'0'})
set_realm('other',[]);set_default_realm('other')
set_policy(name='mfa-passthru-phase1',scope='authentication',action='passthru',realm='coulomb',active=False)
policies=PolicyClass().policies
target=dict(helper='update',pi_url=client.url,lldap_url=lldap_url,user='admin')
before=client.snapshot()
try:cut.native_bind(client,before)
except r.Refused:pass
else:raise AssertionError('old bind unexpectedly works')
preview=cut.run(client,target,'preview')
assert client.snapshot()==before
report=cut.run(client,target,'apply',replacement=replacement,expected_digest=preview['snapshot_sha256'])
assert report['result']=='configured' and report['configuration_writes']==1
assert cut.public_snapshot(client.snapshot())==cut.public_snapshot(before)
with app.app_context():
assert get_resolver_config(r.RESOLVER)['BINDPW']==replacement
assert PolicyClass().policies==policies
token=init_token({'type':'hotp','genkey':1,'pin':''},user=User('admin',r.REALM,r.RESOLVER))
otp=token.get_otp()[2]
target['helper']='reconcile'
# Fixture has actual PI/LDAP/MFA; cluster and KeyCape gates are explicitly substituted.
report=cut.run(client,target,'check',replacement=replacement,predecessor='synthetic-invalid-predecessor',
gate=lambda:None,mfa_proof=lambda:cut.verify(client.url,'admin',r.REALM,r.RESOLVER,
'fixture-admin-password',lambda:otp))
assert report['result']=='verified' and report['mfa_verified'] and report['configuration_writes']==0
return {'native_candidate_and_stored_ldap_bind':True,'native_old_bind_refused':True,
'native_minimal_password_update':True,'native_custom_config_and_realms_preserved':True,
'native_policies_preserved':True,'native_exact_user_and_hotp':True,
'synthetic_wrong_predecessor_http_denied':True,'cluster_and_keycape_gates_substituted':True,
'production_acceptance':False}
finally:server.shutdown();thread.join();server.server_close()
IMAGE = 'sha256:2a8454b668c1aba7157e832eab0e242e1e7eb5fb7591d7e7774ba05286511ca8'
def run():
import re
import secrets
import subprocess
import time
sys.path.insert(0,str(Path(__file__).resolve().parents[1]/'sso-mfa/k8s/privacyidea'))
from resolver_cutover import login
def command(args):
result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=60)
if result.returncode:raise RuntimeError('fixture command failed')
return result.stdout.strip()
contexts=json.loads(command(['context','inspect']))
# DOCKER_CONTEXT takes precedence over DOCKER_HOST. Reject any remote
# configured endpoint, including conflicting environment selections.
hosts=[contexts[0]['Endpoints']['docker']['Host']]
if os.environ.get('DOCKER_HOST'):hosts.append(os.environ['DOCKER_HOST'])
assert all(host.startswith('unix://') for host in hosts)
assert command(['image','inspect',IMAGE,'--format','{{.Id}}'])==IMAGE
container=None
try:
with tempfile.TemporaryDirectory(prefix='resolver-lldap-native-') as directory:
root=Path(directory);password=secrets.token_hex(24)
source=root/'container.env'
fd=os.open(source,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
with os.fdopen(fd,'w') as stream:
stream.write('LLDAP_LDAP_USER_PASS='+password+'\nLLDAP_JWT_SECRET='+secrets.token_hex(32)+'\nLLDAP_LDAP_BASE_DN=dc=example,dc=test\n')
container=command(['run','--detach','--rm','--pull=never','--memory=256m','--cpus=0.5','--pids-limit=128',
'--publish','127.0.0.1::17170','--publish','127.0.0.1::3890',
'--tmpfs','/data:rw,nosuid,nodev,size=32m,uid=1000,gid=1000','--env-file',str(source),IMAGE])
assert re.fullmatch(r'[0-9a-f]{64}',container)
ports=json.loads(command(['inspect',container,'--format','{{json .NetworkSettings.Ports}}']))
assert all(ports[p][0]['HostIp']=='127.0.0.1' for p in ('17170/tcp','3890/tcp'))
base='http://127.0.0.1:'+ports['17170/tcp'][0]['HostPort']
ldap='ldap://127.0.0.1:'+ports['3890/tcp'][0]['HostPort']
for _ in range(80):
try:login(base,password);break
except Exception:time.sleep(.25)
else:raise RuntimeError('fixture startup failed')
result=provider_run(base,ldap,password)
finally:
if container and re.fullmatch(r'[0-9a-f]{64}',container):command(['rm','--force',container])
result['fixture_removed']=True
return result
if __name__=='__main__':
try:
with contextlib.redirect_stdout(io.StringIO()),contextlib.redirect_stderr(io.StringIO()):result=run()
print(json.dumps(result,sort_keys=True))
except Exception as exc:
import traceback
frame=traceback.extract_tb(exc.__traceback__)[-1]
print(json.dumps({'result':'failed','type':type(exc).__name__,'file':Path(frame.filename).name,'line':frame.lineno}))
raise SystemExit(1)

252
tests/test_resolver_cutover.py Normal file
View file

@ -0,0 +1,252 @@
"""Cutover state machine and credential-scoped HTTP regression checks."""
import copy
import json
import os
from pathlib import Path
import subprocess
import sys
from types import SimpleNamespace
import pytest
from test_realm_bootstrap import snapshot, server, HELPERS
sys.path.insert(0, str(HELPERS))
import resolver_cutover as c
import pi_api as p
TARGET = dict(helper='update', pi_url='http://127.0.0.1:1234', lldap_url='http://127.0.0.1:1235',
keycape_url='http://127.0.0.1:1236', user='fixture-user', kubeconfig='/tmp/fixture',
context='fixture', server='https://127.0.0.1:6443', cluster_uid='fixture-uid')
class Client:
def __init__(self):
self.state = snapshot(); self.calls = []; self.reads = 0; self.drift = False
self.bad_bind = False; self.post_failure = False; self.users = [{'username':'fixture-user','resolver':c.realm.RESOLVER}]
def snapshot(self):
self.reads += 1
if self.drift and self.reads == 2: self.state['resolver']['data']['TIMEOUT'] = '99'
return copy.deepcopy(self.state)
def call(self, path, payload=None):
self.calls.append((path, copy.deepcopy(payload)))
if path == '/resolver/test': return not self.bad_bind
if path.startswith('/user/'): return self.users
if path == '/resolver/' + c.realm.RESOLVER:
if self.post_failure: raise c.Refused()
return 1
raise AssertionError()
@pytest.fixture
def client(monkeypatch):
monkeypatch.setattr(c, 'login', lambda *a, **kw: None)
return Client()
def apply(client, target=TARGET, **kw):
options = dict(replacement='synthetic-new', expected_digest=c.digest(client.state, target))
options.update(kw)
return c.run(client, target, 'apply', **options)
def writes(client): return [x for x in client.calls if x[0] == '/resolver/' + c.realm.RESOLVER]
def test_minimal_update_and_no_fabricated_acceptance(client):
before=copy.deepcopy(client.state)
report=apply(client)
assert writes(client)==[('/resolver/'+c.realm.RESOLVER, {'type':'ldapresolver','BINDPW':'synthetic-new'})]
assert client.state==before and report['result']=='configured'
assert report['incident_acceptance']=='unverified' and not report['mfa_verified']
tests=[payload for path,payload in client.calls if path=='/resolver/test']
assert [x['BINDPW'] for x in tests]==['synthetic-new','__CENSORED__']
assert tests[0]['SIZELIMIT']=='700'
assert 'synthetic-new' not in json.dumps(report)
def test_preview_without_candidate_or_write(client):
report=c.run(client,TARGET,'preview')
assert report['result']=='planned' and not client.calls
@pytest.mark.parametrize('change', ['missing','core','tuning','realm','uncensored'])
def test_incomplete_or_conflicting_setup_refused(client,change):
if change=='missing':client.state['resolver']=None
elif change=='core':client.state['resolver']['data']['LDAPURI']='ldap://other'
elif change=='tuning':del client.state['resolver']['data']['TIMEOUT']
elif change=='realm':client.state['realms']['coulomb']['resolver']=[]
else:client.state['resolver']['data']['BINDPW']='synthetic-uncensored'
with pytest.raises(c.Refused):c.run(client,TARGET,'apply',replacement='synthetic-new',expected_digest='0'*64)
assert not writes(client)
@pytest.mark.parametrize('failure',['stale','drift','candidate'])
def test_prewrite_failures_never_mutate(client,failure):
client.drift=failure=='drift';client.bad_bind=failure=='candidate'
opts={'expected_digest':'0'*64} if failure=='stale' else {}
with pytest.raises(c.Refused):apply(client,**opts)
assert not writes(client)
def test_ambiguous_write_is_not_retried_and_receipt_remains_unverified(client):
client.post_failure=True;report={}
with pytest.raises(c.Refused):apply(client,report=report)
assert report['result']=='unverified' and report['write_attempted']
assert report['configuration_writes']==0 and len(writes(client))==1
@pytest.mark.parametrize('predecessor,result',[(None,'incomplete'),('synthetic-old','verified')])
def test_full_proofs_and_honest_predecessor_result(client,predecessor,result):
target=dict(TARGET,helper='reconcile');gates=[]
report=apply(client,target,predecessor=predecessor,gate=lambda:gates.append(1),mfa_proof=lambda:{'result':'PASS'})
assert report['result']==result and report['mfa_verified'] and len(gates)==2
assert report['supplied_predecessor_denied']==bool(predecessor)
assert report['incident_acceptance']=='unverified'
@pytest.mark.parametrize('users',[[],[{'username':'other','resolver':c.realm.RESOLVER}],
[{'username':'fixture-user','resolver':'wrong'}],[{'username':'fixture-user','resolver':c.realm.RESOLVER}]*2])
def test_exact_user_before_mfa(client,users):
client.users=users
with pytest.raises(c.Refused):apply(client,dict(TARGET,helper='reconcile'),gate=lambda:None,mfa_proof=lambda:pytest.fail('OTP too early'))
@pytest.mark.parametrize('status',[0,200,302,400,500])
def test_predecessor_network_or_unexpected_result_not_denial(monkeypatch,status):
monkeypatch.setattr(c,'request',lambda *a,**kw:(status,{'token':'synthetic'}))
with pytest.raises(c.Refused):c.login(TARGET['lldap_url'],'old',denied=True)
@pytest.mark.parametrize('profile,path,payload',[
('lldap-auth','/auth/simple/login',{'username':'admin','password':'synthetic'}),
('keycape-discovery','/.well-known/openid-configuration',None)])
def test_named_profile_actual_http(server,profile,path,payload):
base,state=server
assert p.request(base+path,payload,profile=profile)[0]==200
assert len(state['requests'])==1
assert state['requests'][0][2]==(json.dumps(payload).encode() if payload else b'')
assert 'Authorization' not in state['requests'][0][1]
@pytest.mark.parametrize('profile,path,payload,token,method',[
('lldap-auth','/other',{'username':'admin','password':'synthetic'},None,None),
('lldap-auth','/auth/simple/login',{'username':'pi-admin','password':'synthetic'},None,None),
('lldap-auth','/auth/simple/login',{'username':'admin','password':'synthetic'},'pi-token',None),
('lldap-auth','/auth/simple/login?x=1',{'username':'admin','password':'synthetic'},None,None),
('lldap-auth','/auth/simple/login',{'username':'admin','password':'synthetic'},None,'PUT'),
('keycape-discovery','/.well-known/openid-configuration',{'password':'synthetic'},None,None),
('keycape-discovery','/.well-known/openid-configuration',None,'pi-token',None),
('unknown','/auth',{},None,None)])
def test_profile_refusal_before_socket(server,profile,path,payload,token,method):
base,state=server
assert p.request(base+path,payload,token,profile=profile,method=method)==(0,None)
assert not state['requests']
@pytest.mark.parametrize('profile',['lldap-auth','keycape-discovery'])
@pytest.mark.parametrize('origin',['http://lldap.coulomb.social','https://evil.invalid','https://pink.coulomb.social','http://localhost:1234'])
def test_origin_scope(profile,origin):
with pytest.raises(Exception):p.service_endpoint(origin,profile)
@pytest.mark.parametrize('status',[301,302,307,308,500])
def test_profile_no_redirect(server,status):
base,state=server;state['status']=status
assert p.request(base+'/auth/simple/login',{'username':'admin','password':'synthetic'},profile='lldap-auth')==(status,None)
assert len(state['requests'])==1
@pytest.mark.parametrize('bad',['server','uid','generation','replicas','deletion'])
def test_cluster_pin_and_current_generation(monkeypatch,bad):
calls=[]
def command(argv,**kw):
calls.append(argv)
if 'view' in argv:out='https://wrong:6443' if bad=='server' else TARGET['server']
elif 'namespace' in argv:out='wrong' if bad=='uid' else TARGET['cluster_uid']
else:out={'generation':'4|3|1|1|1|1|1|','replicas':'4|4|1|2|1|1|1|','deletion':'4|4|1|1|1|1|1|now'}.get(bad,'4|4|1|1|1|1|1|')
return SimpleNamespace(returncode=0,stdout=out)
monkeypatch.setattr(c.subprocess,'run',command)
with pytest.raises(c.Refused):c.cluster_ready(TARGET)
assert all(argv[1:5]==['--kubeconfig','/tmp/fixture','--context','fixture'] for argv in calls)
assert not any('secret' in argv or '--raw' in argv for argv in calls)
@pytest.mark.parametrize('script',['update-lldap-resolver-live.sh','reconcile-lldap-resolver-live.sh'])
def test_unattended_and_unsafe_legacy_argument_refuse_without_echo(script):
result=subprocess.run(['bash','-x',str(HELPERS/script),'--note','synthetic-private-note'],capture_output=True,text=True)
assert result.returncode!=0 and 'synthetic-private-note' not in result.stdout+result.stderr
assert 'Traceback' not in result.stderr and 'password:' not in result.stderr
@pytest.mark.parametrize('broken',['issuer','endpoint','redirect','malformed'])
def test_health_semantics(monkeypatch,broken):
issuer='https://kc.coulomb.social'
body={'issuer':issuer,**{k:issuer+'/'+k for k in ('authorization_endpoint','token_endpoint','jwks_uri')}}
if broken=='issuer':body['issuer']='https://other.invalid'
if broken=='endpoint':body['token_endpoint']='https://kc.coulomb.social.evil.invalid/token'
if broken=='malformed':body=[]
monkeypatch.setattr(c,'request',lambda url,**kw:(401,None) if url.endswith('/token/') else (302 if broken=='redirect' else 200,body))
with pytest.raises(c.Refused):c.health(TARGET)
@pytest.mark.parametrize('script',['update-lldap-resolver-live.sh','reconcile-lldap-resolver-live.sh'])
def test_pty_tracing_never_discloses_password(server,tmp_path,script):
import pty
import select
import signal
import time
if script.startswith('reconcile'):
# Pin validation must happen before any credential prompt.
argv=['--preview','--server','https://bad.invalid']
else:
argv=['--preview','--pi-url',server[0]]
server[1]['response']={'result':{'status':True,'value':{'role':'admin','token':'fixture-token'}}}
pid,fd=pty.fork()
if pid==0:
os.environ['TMPDIR']=str(tmp_path)
os.execv('/bin/bash',['bash','-x',str(HELPERS/script),*argv])
output=b'';sent=False;status=None
try:
deadline=time.monotonic()+10
while time.monotonic()<deadline:
if select.select([fd],[],[],.1)[0]:
try:chunk=os.read(fd,65536)
except OSError:break
if not chunk:break
output+=chunk
if b'pi-admin password:' in output and not sent:
os.write(fd,b'fixture-private-terminal-value\n');sent=True
done,value=os.waitpid(pid,os.WNOHANG)
if done:status=value;break
if status is None:
done,value=os.waitpid(pid,os.WNOHANG)
if not done:
os.kill(pid,signal.SIGKILL);os.waitpid(pid,0);pytest.fail('attended helper timed out')
status=value
assert os.waitstatus_to_exitcode(status)!=0
assert sent == script.startswith('update')
assert b'fixture-private-terminal-value' not in output and b'Traceback' not in output
assert not list(tmp_path.iterdir())
finally:os.close(fd)
def test_postwrite_drift_stops_before_success(client,monkeypatch):
original=client.call
def change(path,payload=None):
value=original(path,payload)
if path=='/resolver/'+c.realm.RESOLVER:client.state['resolver']['data']['SIZELIMIT']='500'
return value
monkeypatch.setattr(client,'call',change);report={}
with pytest.raises(c.Refused):apply(client,report=report)
assert report['write_attempted'] and report['configuration_writes']==1
assert report['result']=='unverified' and report['phase']=='stored-bind-verification'
def test_mfa_failure_never_reports_success(client):
report={}
with pytest.raises(c.Refused):apply(client,dict(TARGET,helper='reconcile'),gate=lambda:None,mfa_proof=lambda:{'result':'FAIL'},report=report)
assert report['phase']=='token-backed-mfa' and not report['mfa_verified']
def test_censored_sentinel_is_not_a_replacement(client):
with pytest.raises(c.Refused):apply(client,replacement='__CENSORED__')
assert not client.calls

46
workplans/NK-WP-0049-bootstrap-secret-free-recovery-handoff.md
View file

@ -204,17 +204,11 @@ Remaining closure checklist (source review, not proof of current live use):
below: explicit private input, scoped counts and reader-visibility proof replace below: explicit private input, scoped counts and reader-visibility proof replace
retired Secret retrieval. Live diagnostic delivery/execution still requires retired Secret retrieval. Live diagnostic delivery/execution still requires
separate owner admission; KeyCape's workload lane is not silently shared. separate owner admission; KeyCape's workload lane is not silently shared.
- `lldap/bootstrap-users.sh` still sources credentials, sends password/token in - LDAP user-lifecycle shortcuts are retired and group callers are repaired below
argv and can print auth responses. Inspect its group/user provisioning callers through the existing User Engine/service-group boundary. Inventory completeness
against the current User Engine owner path before replacing or retiring them. and live provider admission still need owner evidence.
Targeted source findings also include `manage-group-members.sh`, `create-user.sh`, - General realm repair and both incident resolver helpers are repaired below.
`break-glass.sh`, `dry-run-nonroot-user.sh` and `verify-t07.sh`; their caller and Full caller/root-output inventory and live owner-scoped acceptance remain open.
lifecycle review remains incomplete.
- `privacyidea/repair-realm-live.sh` and `bootstrap-realm.sh` are repaired below:
attended in-memory or literal private-file input, minimal configuration repair,
and preservation of owner-managed policies. Current live acceptance remains
owner-scoped; incident-specific resolver cutover/rotation callers still need
the complete inventory and configuration-preservation review.
- Reconcile the complete `docs/attended-procedure-inventory.md` against current - Reconcile the complete `docs/attended-procedure-inventory.md` against current
scripts and Make/bootstrap/root-output callers. The selected findings above scripts and Make/bootstrap/root-output callers. The selected findings above
are not an exhaustive closure audit. Retain historical revocation and are not an exhaustive closure audit. Retain historical revocation and
@ -361,6 +355,36 @@ No live estate credentials or directory state were read or changed. Incident
cutover helpers, full caller/root-output inventory and owner-approved native cutover helpers, full caller/root-output inventory and owner-approved native
recovery/estate acceptance keep T02 progress and T03 wait. recovery/estate acceptance keep T02 progress and T03 wait.
2026-10-14 incident resolver helpers reconciled: both legacy shell bodies now
invoke `resolver_cutover.py` with tracing disabled and attended memory-only
inputs. The shared JSON transport has separate destination/route profiles for
LLDAP authentication and KeyCape discovery; no privacyIDEA token crosses those
profiles. Existing resolver/realm configuration is mandatory. Preview binds the
observed public state and exact targets; apply rechecks it, performs one minimal
BINDPW/type write, and verifies all other fields and realms are preserved.
Native candidate and stored LDAP bind tests prevent cached-lookup false proof.
Full reconcile pins kubeconfig/context/server/cluster UID, checks current single
replica generations, validates discovery issuer/endpoints, requires one exact
user and token-backed MFA, and tests supplied predecessor HTTP denial. Missing
predecessor returns incomplete/exit 2; an ambiguous write remains unverified and
is never retried. Owner input provenance and actual KeyCape flow acceptance stay
external. No secret files, ambient endpoint overrides or raw --note receipts.
The runbook, historical merge-semantics correction and inventory are updated.
Validation: 171 affected tests pass (including terminal tracing tests), plus
actual disposable privacyIDEA 3.12.2 and LLDAP prove old-bind refusal, candidate
and stored-bind success, exact password-only update with custom settings/realms/
policies preserved, and real user/HOTP validation. Cluster/KeyCape gates are
substituted only in that native fixture; production acceptance is unverified.
The initial run had one test-fixture error (missing-resolver digest before the
refusal call), corrected and retained in the evidence receipt. No live change
or credential retrieval. Custodian evidence:
`docs/evidence/2026-10-14-netkingdom-security/resolver-cutover.json`.
This completes the bounded two-helper source repair; the full caller/root-output
inventory and owner-native recovery/consumer acceptance still keep T02 progress
and T03 wait. Founder-deferred exposure rotation remains deferred.
## Verify owner-approved recovery and current native callers ## Verify owner-approved recovery and current native callers
```task ```task