net-kingdom/sso-mfa/k8s
Bernd Worsch c054241a5c feat(t09): backup, break-glass, DR drill — NK-WP-0003-T09 done
- Apply SQLite backup CronJobs (LLDAP, Authelia, privacyIDEA) — all verified running
- Fix authelia-backup: remove scale-down/up dance; concurrent local-path PVC mount
  works on single-node k3s, sqlite3 .backup is safe for concurrent access
- Fix privacyidea-backup: add supplementalGroups: [999] so uid=1000 can read enckey
- Add allow-backup-to-kube-api NetworkPolicy (backup pod → 10.43.0.1:443)
- Create break-glass LLDAP account (net-kingdom-admins); fix ((PASS++)) set-e trap
- SQLite restore drill: LLDAP backup valid (2 users, all tables)
- verify-t08.sh: PASS=15, FAIL=0; fix counter bug + enckey PVC path (/etc/privacyidea)
- Update DR-RUNBOOK.md Authelia restore procedure
- T09 deferred: CNPG backup (needs MinIO/S3), Prometheus (needs kube-prometheus-stack)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:56:40 +00:00
..
authelia fix(sso-mfa): commit T02–T06 fixes and workplan status updates 2026-03-21 20:25:03 +00:00
backup feat(t09): backup, break-glass, DR drill — NK-WP-0003-T09 done 2026-03-25 23:56:40 +00:00
cert-manager custodian integration and some cleanuo 2026-03-04 23:31:28 +01:00
keycape fix(keycape): NK-WP-0003-T07 — fix deployment image + add demo-app client 2026-03-22 00:30:58 +00:00
lldap feat(t09): backup, break-glass, DR drill — NK-WP-0003-T09 done 2026-03-25 23:56:40 +00:00
namespaces feat(sso-mfa): T02 K8s foundations manifests (NK-WP-0001-T02) 2026-03-02 09:49:39 +01:00
network-policies feat(t09): backup, break-glass, DR drill — NK-WP-0003-T09 done 2026-03-25 23:56:40 +00:00
postgresql feat(sso-mfa): T02/T03 live apply — age-encrypted secrets, CNPG cluster (NK-WP-0001-T02/T03) 2026-03-20 02:57:41 +00:00
privacyidea fix(privacyidea): create pi-admin-all-rights policy in bootstrap-admin.sh 2026-03-25 11:49:26 +00:00
storage feat(sso-mfa): T02 K8s foundations manifests (NK-WP-0001-T02) 2026-03-02 09:49:39 +01:00
README.md feat(sso-mfa): T05 SSO stack pivot — Keycloak → Authelia + LLDAP + KeyCape (NK-WP-0001-T05) 2026-03-19 08:31:51 +00:00
verify-t02.sh fix(sso-mfa): commit T02–T06 fixes and workplan status updates 2026-03-21 20:25:03 +00:00
verify-t03.sh feat(sso-mfa): T02/T03 live apply — age-encrypted secrets, CNPG cluster (NK-WP-0001-T02/T03) 2026-03-20 02:57:41 +00:00
verify-t04.sh docs(sso-mfa): fix stale Keycloak refs and add T04 apply section to WORKPLAN 2026-03-20 07:33:47 +00:00
verify-t05.sh feat(sso-mfa): T05 SSO stack pivot — Keycloak → Authelia + LLDAP + KeyCape (NK-WP-0001-T05) 2026-03-19 08:31:51 +00:00
verify-t06.sh feat(sso-mfa): T06 realm config & MFA flow manifests (NK-WP-0001-T06) 2026-03-19 09:04:07 +00:00
verify-t07.sh feat(sso-mfa): T07/T08 user mgmt, backups, DR & break-glass (NK-WP-0001-T07/T08) 2026-03-19 09:17:03 +00:00
verify-t08.sh feat(t09): backup, break-glass, DR drill — NK-WP-0003-T09 done 2026-03-25 23:56:40 +00:00

T02 — K8s Foundations

Phase 1 of NK-WP-0001: namespaces, NetworkPolicies, cert-manager, StorageClass.

SSO stack overview

The sso namespace hosts three components:

  • KeyCape (kc.coulomb.social) — OIDC orchestration layer, stateless
  • Authelia (auth.coulomb.social) — password authentication frontend
  • LLDAP (lldap.coulomb.social) — lightweight LDAP directory (admin UI restricted)

The mfa namespace hosts:

  • privacyIDEA (pink.coulomb.social) — MFA engine, called by KeyCape

Prerequisites

  • K3s cluster running (ThreePhoenix HA or single-node dev)
  • T01 Phase 0a complete (KeePassXC vault populated, ops bundle exported)
  • kubectl configured with cluster access

Apply order

# 1. Install cert-manager (if not already on cluster)
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true

# Wait for cert-manager to be ready
kubectl rollout status deployment/cert-manager -n cert-manager

# 2. Create namespaces
kubectl apply -f namespaces/namespaces.yaml

# 3. Apply NetworkPolicies
kubectl apply -f network-policies/netpol-sso.yaml
kubectl apply -f network-policies/netpol-mfa.yaml
kubectl apply -f network-policies/netpol-databases.yaml

# 4. Create ClusterIssuers
#    Edit issuers.yaml first: replace ACME_EMAIL with your address
kubectl apply -f cert-manager/issuers.yaml

# 5. Verify cert-manager with test certificate
kubectl apply -f cert-manager/test-certificate.yaml
kubectl wait --for=condition=Ready certificate/selfsigned-test \
  -n cert-manager-test --timeout=60s
kubectl delete namespace cert-manager-test

# 6. Verify StorageClass
kubectl apply -f storage/verify-pvc.yaml
kubectl wait --for=condition=Ready pod/storage-test \
  -n storage-test --timeout=60s
kubectl logs -n storage-test storage-test
kubectl delete namespace storage-test

# 7. Run the full verification script
chmod +x verify-t02.sh
./verify-t02.sh

NetworkPolicy design

All three namespaces follow a default-deny-all posture. Only the minimal required paths are opened:

Source Destination Port Purpose
Traefik (kube-system) KeyCape (sso) 8080 OIDC endpoints — public
Traefik (kube-system) Authelia (sso) 9091 Login portal — public
Traefik (kube-system) LLDAP (sso) 17170 Admin web UI — IP-restricted
Traefik (kube-system) privacyIDEA (mfa) 8080 MFA portal — public
KeyCape (sso) Authelia (sso) 9091 OIDC token exchange
KeyCape (sso) LLDAP (sso) 3890 User attribute lookup
KeyCape (sso) privacyIDEA (mfa) 8080 MFA challenge + validation
Authelia (sso) LLDAP (sso) 3890 Credential validation
privacyIDEA (mfa) PostgreSQL (databases) 5432 DB
CNPG operator (cnpg-system) PostgreSQL (databases) 5432/9187 Operator + metrics
All pods kube-dns (kube-system) 53 DNS resolution
CNPG pods K8s API 6443 Status updates

Verifying denied paths (manual)

After applying NetworkPolicies, confirm that illegal paths are blocked:

# Test: KeyCape → privacyIDEA (should be ALLOWED)
kubectl run test-allowed -n sso --rm -it --image=busybox --restart=Never \
  -- nc -zv privacyidea.mfa.svc.cluster.local 8080

# Test: Authelia → privacyIDEA (should be DENIED — only KeyCape calls privacyIDEA)
kubectl run test-denied -n sso --rm -it --image=busybox --restart=Never \
  -l app.kubernetes.io/name=authelia \
  -- nc -zv -w3 privacyidea.mfa.svc.cluster.local 8080

# Test: databases → sso (should be DENIED — DB pods must not initiate connections)
kubectl run test-denied2 -n databases --rm -it --image=busybox --restart=Never \
  -- nc -zw3 keycape.sso.svc.cluster.local 8080

Notes

  • net-kingdom/component labels on namespaces are used by NetworkPolicy namespaceSelector rules. Do not remove them.
  • cnpg.io/cluster: net-kingdom-pg in netpol-databases.yaml must match the name of the CloudNativePG Cluster CR you create in T03.
  • The letsencrypt-prod ClusterIssuer requires public DNS and port 80 open to Let's Encrypt servers. Update ACME_EMAIL before applying.