net-kingdom/sso-mfa/k8s
tegwick 4a38511d11 fix(privacyidea): repair the resolver reconciliation script (NK-WP-0033)
reconcile-lldap-resolver-live.sh had never completed a run. Four defects,
found by running it on 2026-08-27:

1. request() set Content-Type: application/json on every call, including
   bodyless GETs. Werkzeug 3.x rejects those in front of privacyIDEA, so
   every GET returned an HTML 400 while POSTs succeeded — the resolver
   write landed and the lookup immediately after it did not.
   bootstrap-realm.sh already fixed this in pi_api and said why; this
   script was written later and did not inherit it.

2. GET /user/ returns result.value as a list of user objects, not a dict
   carrying "users". With the 400 fixed, the lookup finally reached the
   parse and raised AttributeError past the except clause, so the run
   died as a traceback instead of a receipt. Both shapes now accepted,
   and the except clause catches parse errors so a failed run still
   names the phase it died in.

3. A resolver write replaces the whole object, so TIMEOUT,
   CACHE_TIMEOUT and SIZELIMIT were dropped by every --apply. A resolver
   with them unset still resolves users, but the WebUI refuses to save
   or test it — so the script silently un-repaired a resolver an
   operator had fixed by hand. Now sent, defaulting to the verified
   5/120/500 and overridable per run. Same omission fixed in
   bootstrap-realm.sh, which created the resolver that way originally.

4. The predecessor prompt could not be left empty, so an operator who
   had lost the exposed credential had to type a placeholder — which
   also fails the bind and was recorded as a PASSING denial proof.
   --predecessor-unavailable skips the bind and records NOT-PROVEN.
   --note carries operator context into the receipt line itself, so the
   claim and its caveat travel together.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 3377672@bnt-lap001
Assistant-Session: 15463ccf-238f-4e13-b163-93aa25c6d166
2026-08-27 22:20:03 +02:00
..
authelia fix(authelia): load rotated client verifier from secret file 2026-08-23 14:39:32 +02:00
backup feat(t09): backup, break-glass, DR drill — NK-WP-0003-T09 done 2026-03-25 23:56:40 +00:00
cert-manager custodian integration and some cleanuo 2026-03-04 23:31:28 +01:00
identity-provisioner Finish public registration and JIT acceptance 2026-08-14 19:15:06 +02:00
keycape feat(NK-WP-0032): admit OpenBao operator loopback callback 2026-08-23 14:01:57 +02:00
lldap Implement NK-WP-0021 activity-core ops SSO least-privilege. 2026-07-22 15:47:26 +02:00
namespaces feat(sso-mfa): T02 K8s foundations manifests (NK-WP-0001-T02) 2026-03-02 09:49:39 +01:00
network-policies Allow user-engine OIDC exchange with KeyCape 2026-07-29 21:40:52 +02:00
postgresql feat(sso-mfa): T02/T03 live apply — age-encrypted secrets, CNPG cluster (NK-WP-0001-T02/T03) 2026-03-20 02:57:41 +00:00
privacyidea fix(privacyidea): repair the resolver reconciliation script (NK-WP-0033) 2026-08-27 22:20:03 +02:00
storage feat(sso-mfa): T02 K8s foundations manifests (NK-WP-0001-T02) 2026-03-02 09:49:39 +01:00
tenant-engine Complete live registration acceptance 2026-08-14 17:39:08 +02:00
user-engine NK-WP-0026 finished: user-engine caller identity verified live against railiance01 2026-08-19 22:01:12 +02:00
README.md feat(sso-mfa): T05 SSO stack pivot — Keycloak → Authelia + LLDAP + KeyCape (NK-WP-0001-T05) 2026-03-19 08:31:51 +00:00
verify-identity-cutover-dependencies.sh Verify railiance01 identity dependencies 2026-07-28 12:59:12 +02:00
verify-t02.sh Allow user-engine OIDC exchange with KeyCape 2026-07-29 21:40:52 +02:00
verify-t03.sh feat(sso-mfa): T02/T03 live apply — age-encrypted secrets, CNPG cluster (NK-WP-0001-T02/T03) 2026-03-20 02:57:41 +00:00
verify-t04.sh docs(sso-mfa): fix stale Keycloak refs and add T04 apply section to WORKPLAN 2026-03-20 07:33:47 +00:00
verify-t05.sh feat(sso-mfa): T05 SSO stack pivot — Keycloak → Authelia + LLDAP + KeyCape (NK-WP-0001-T05) 2026-03-19 08:31:51 +00:00
verify-t06.sh Add KeyCape privacyIDEA token repair flow 2026-05-29 03:07:17 +02:00
verify-t07.sh feat(NK-WP-0032): admit OpenBao operator loopback callback 2026-08-23 14:01:57 +02:00
verify-t08.sh feat(t09): backup, break-glass, DR drill — NK-WP-0003-T09 done 2026-03-25 23:56:40 +00:00

T02 — K8s Foundations

Phase 1 of NK-WP-0001: namespaces, NetworkPolicies, cert-manager, StorageClass.

SSO stack overview

The sso namespace hosts three components:

  • KeyCape (kc.coulomb.social) — OIDC orchestration layer, stateless
  • Authelia (auth.coulomb.social) — password authentication frontend
  • LLDAP (lldap.coulomb.social) — lightweight LDAP directory (admin UI restricted)

The mfa namespace hosts:

  • privacyIDEA (pink.coulomb.social) — MFA engine, called by KeyCape

Prerequisites

  • K3s cluster running (ThreePhoenix HA or single-node dev)
  • T01 Phase 0a complete (KeePassXC vault populated, ops bundle exported)
  • kubectl configured with cluster access

Apply order

# 1. Install cert-manager (if not already on cluster)
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true

# Wait for cert-manager to be ready
kubectl rollout status deployment/cert-manager -n cert-manager

# 2. Create namespaces
kubectl apply -f namespaces/namespaces.yaml

# 3. Apply NetworkPolicies
kubectl apply -f network-policies/netpol-sso.yaml
kubectl apply -f network-policies/netpol-mfa.yaml
kubectl apply -f network-policies/netpol-databases.yaml

# 4. Create ClusterIssuers
#    Edit issuers.yaml first: replace ACME_EMAIL with your address
kubectl apply -f cert-manager/issuers.yaml

# 5. Verify cert-manager with test certificate
kubectl apply -f cert-manager/test-certificate.yaml
kubectl wait --for=condition=Ready certificate/selfsigned-test \
  -n cert-manager-test --timeout=60s
kubectl delete namespace cert-manager-test

# 6. Verify StorageClass
kubectl apply -f storage/verify-pvc.yaml
kubectl wait --for=condition=Ready pod/storage-test \
  -n storage-test --timeout=60s
kubectl logs -n storage-test storage-test
kubectl delete namespace storage-test

# 7. Run the full verification script
chmod +x verify-t02.sh
./verify-t02.sh

NetworkPolicy design

All three namespaces follow a default-deny-all posture. Only the minimal required paths are opened:

Source Destination Port Purpose
Traefik (kube-system) KeyCape (sso) 8080 OIDC endpoints — public
Traefik (kube-system) Authelia (sso) 9091 Login portal — public
Traefik (kube-system) LLDAP (sso) 17170 Admin web UI — IP-restricted
Traefik (kube-system) privacyIDEA (mfa) 8080 MFA portal — public
KeyCape (sso) Authelia (sso) 9091 OIDC token exchange
KeyCape (sso) LLDAP (sso) 3890 User attribute lookup
KeyCape (sso) privacyIDEA (mfa) 8080 MFA challenge + validation
Authelia (sso) LLDAP (sso) 3890 Credential validation
privacyIDEA (mfa) PostgreSQL (databases) 5432 DB
CNPG operator (cnpg-system) PostgreSQL (databases) 5432/9187 Operator + metrics
All pods kube-dns (kube-system) 53 DNS resolution
CNPG pods K8s API 6443 Status updates

Verifying denied paths (manual)

After applying NetworkPolicies, confirm that illegal paths are blocked:

# Test: KeyCape → privacyIDEA (should be ALLOWED)
kubectl run test-allowed -n sso --rm -it --image=busybox --restart=Never \
  -- nc -zv privacyidea.mfa.svc.cluster.local 8080

# Test: Authelia → privacyIDEA (should be DENIED — only KeyCape calls privacyIDEA)
kubectl run test-denied -n sso --rm -it --image=busybox --restart=Never \
  -l app.kubernetes.io/name=authelia \
  -- nc -zv -w3 privacyidea.mfa.svc.cluster.local 8080

# Test: databases → sso (should be DENIED — DB pods must not initiate connections)
kubectl run test-denied2 -n databases --rm -it --image=busybox --restart=Never \
  -- nc -zw3 keycape.sso.svc.cluster.local 8080

Notes

  • net-kingdom/component labels on namespaces are used by NetworkPolicy namespaceSelector rules. Do not remove them.
  • cnpg.io/cluster: net-kingdom-pg in netpol-databases.yaml must match the name of the CloudNativePG Cluster CR you create in T03.
  • The letsencrypt-prod ClusterIssuer requires public DNS and port 80 open to Let's Encrypt servers. Update ACME_EMAIL before applying.