coulomb/key-cape
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
key-cape/history/KEY-WP-0011-live-secret-exposure-recovery.md
tegwick 93704fd242 ops: restart identity provisioner during keycape rotation 
Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a02e3f-7301-7622-9be1-12e5f352881c
2026-08-23 14:42:44 +02:00

3.3 KiB
Raw Blame History

KEY-WP-0011 — Live Secret exposure recovery evidence

Date: 2026-08-23

Outcome

The accidental rendering of the sso/keycape-config Secret was treated as a live credential exposure. With explicit operator approval for session invalidation and brief SSO disruption, recovery replaced:

  • the KeyCape RS256 signing key;
  • the shared LLDAP admin/bind credential used by LLDAP, Authelia, and KeyCape;
  • the Authelia-KeyCape OIDC client credential and verifier; and
  • the privacyIDEA application signing secret, invalidating the exposed admin JWT.

KeyCape now carries the non-credential marker ROTATED_DISABLED_ADMIN_TOKEN instead of a privacyIDEA admin JWT. The current client set has no enrollment handoff URL, and requireForAll: true keeps the normal MFA path fail-closed. A durable, least-privilege privacyIDEA service credential remains a follow-up architecture gap.

Custody

The authoritative replacement material is encrypted to the NetKingdom age recipient in KEY-WP-0011-rotated-credentials-2026-08-23-final.tar.age (mode 0600). It contains no pre-exposure value. The temporary first-cutover archive was securely removed after its contents were superseded by the final rotation. The final archive was not decrypted during verification.

Non-secret verification evidence

  • Pre-recovery public JWKS fingerprint: 4172efd1109e2a07b728c59b28558bf6e0171b22afb95d1692f87fccf284039b
  • First-cutover public JWKS fingerprint: 8ec2651168468caa595f185bb7021cfdec228eb4992f1f6c1521c6e62762c73e
  • Final public JWKS fingerprint: c6faac5dfeef2453daf9cfc14671f62b535dd321befdf6014e3ee5c2cf1f1156
  • Final Secret resource versions:
    • sso/keycape-config: 51346058
    • sso/keycape-pi-token: 51344120
    • sso/authelia-secrets: 51346049
    • sso/lldap-secrets: 51345775
    • mfa/privacyidea-config: 51345601
  • KeyCape, Authelia, LLDAP, privacyIDEA, and identity-provisioner each reported one Ready, updated, available replica.
  • LLDAP accepted the replacement credential before and after removal of LLDAP_FORCE_LDAP_USER_PASS_RESET; the final deployment has zero copies of that emergency setting.
  • Authelia accepted the replacement KeyCape client credential and rejected a deliberately wrong credential. Its active ConfigMap verifier matches the copy held in authelia-secrets.
  • privacyIDEA rejected the synthetic invalid MFA proof with HTTP 400, result.status=false, and error code 907.
  • OIDC discovery returned issuer https://kc.coulomb.social.
  • The registered OpenBao local callback returned HTTP 302 to auth.coulomb.social.

Follow-up disposition

NetKingdom resolved the manifest mismatch in revision c956ceb: Authelia's OIDC client list now uses the supported template filter to read the verifier from the Secret-mounted file. The revision was applied live and Authelia returned Ready with startup complete.

The credential-routing catalog also has no concrete native rotation entries for these legacy KeyCape backend credentials; generic and interactive-login routes were insufficient for incident execution.

The remaining cross-repository follow-up is the privacyIDEA lldap-coulomb resolver. It needs an attended provider-admin operation to replace its persisted LLDAP bind password and prove a user lookup; no unattended provider-admin credential was retained after the exposed JWT was disabled.