Fix plus-address sign-in and finish NK-WP-0041

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a0e31f-2bcc-7051-a050-70d8cb2dfa49

.custodian-brief.md

@@ -2,7 +2,7 @@
 # Custodian Brief — net-kingdom
 
 **Domain:** infotech  
-**Last synced:** 2026-09-27 11:08 UTC  
+**Last synced:** 2026-09-27 14:02 UTC  
 **State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
 
 ## Active Workstreams
@@ -14,12 +14,6 @@ Progress: 0/2 done  |  workplan_id: `e2533f3a-aa43-59b3-bff3-8e64b6149487`
 - · Draft the receipt fields as a Playbook Capability Contract amendment  `9d02685a`
 - · Agree the evidence holder and schema with audit-core and Railiance  `963120c1`
 
-### Fix onboarding-journey defects found in the 2026-09-23 human run
-Progress: 2/3 done  |  workplan_id: `98168f50-7a4d-5bb5-a462-1e031563b89f`
-
-**Open tasks:**
-- ! Plus-addressed email sign-in fails with an LDAP filter error  `83633649`
-
 ### Take in the flex-auth to access-engine repository-coordinate rename
 Progress: 2/4 done  |  workplan_id: `284a8ac2-61dc-5bee-b74a-0a62d9808edb`
 
@@ -60,11 +54,6 @@ Progress: 7/8 done  |  workplan_id: `d76ddccc-00c8-548a-b141-2cd660fa38da`
 **Open tasks:**
 - ! T08 - Final deletion and closure  `42a3b4c0`
 
-## Inbox Hygiene
-
-**Stale unread:** 3 message(s) older than 3 day(s) — triage at session start.
-**Missing thread_id:** 2 unread message(s) lack supersession chains.
-
 ---
 ## MCP Orientation (when available)
 

WORK-RECORDS.md

@@ -34,7 +34,7 @@
 | workplan | NK-WP-0038 | finished | — | workplans/NK-WP-0038-tenant-scoped-identity-lifecycle.md |
 | workplan | NK-WP-0039 | active | — | workplans/NK-WP-0039-flex-auth-access-engine-coordinate-intake.md |
 | workplan | NK-WP-0040 | ready | — | workplans/NK-WP-0040-execution-attribution-receipt.md |
-| workplan | NK-WP-0041 | active | — | workplans/NK-WP-0041-onboarding-journey-usability.md |
+| workplan | NK-WP-0041 | finished | — | workplans/NK-WP-0041-onboarding-journey-usability.md |
 | workplan | NK-WP-0042 | backlog | — | workplans/NK-WP-0042-workload-mfa-step-up.md |
 | task | NK-WP-ADHOC-2026-07-02-T01 | done | — | workplans/ADHOC-2026-07-02.md |
 | task | NK-WP-ADHOC-2026-07-02-T02 | done | — | workplans/ADHOC-2026-07-02.md |
@@ -152,7 +152,7 @@
 | task | NK-WP-0040-T01 | todo | — | workplans/NK-WP-0040-execution-attribution-receipt.md |
 | task | NK-WP-0040-T02 | todo | — | workplans/NK-WP-0040-execution-attribution-receipt.md |
 | task | NK-WP-0041-T01 | done | — | workplans/NK-WP-0041-onboarding-journey-usability.md |
-| task | NK-WP-0041-T02 | wait | — | workplans/NK-WP-0041-onboarding-journey-usability.md |
+| task | NK-WP-0041-T02 | done | — | workplans/NK-WP-0041-onboarding-journey-usability.md |
 | task | NK-WP-0041-T03 | done | — | workplans/NK-WP-0041-onboarding-journey-usability.md |
 | task | NK-WP-0042-T01 | todo | — | workplans/NK-WP-0042-workload-mfa-step-up.md |
 | task | NK-WP-0042-T02 | todo | — | workplans/NK-WP-0042-workload-mfa-step-up.md |

sso-mfa/k8s/authelia/README.md

@@ -117,3 +117,32 @@ kubectl run -n sso --rm -it auth-test --image=busybox --restart=Never \
 # OIDC discovery (should return issuer + endpoints)
 curl -s https://auth.coulomb.social/.well-known/openid-configuration | jq .
 ```
+
+## 4.39 compatibility and regression check
+
+KeyCape needs `preferred_username` in the signed upstream ID token for its
+directory and MFA lookup. Authelia 4.39 requires the explicit `keycape`
+claims policy in `configmap.yaml`; deploy it together with the pinned image.
+KeyCape's in-cluster token requests must also carry the public HTTPS forwarded
+scheme and host (KeyCape `3b0446e` or later). Neither a health check nor an
+authorization redirect exercises these requirements.
+
+Run the isolated token regression with an Authelia 4.39.28 binary and Python
+`requests`, `PyYAML`, and `cryptography` installed:
+
+```bash
+python3 sso-mfa/k8s/authelia/tests/probe_claims.py --authelia-bin /path/to/authelia
+```
+
+It uses disposable local users, keys and SQLite databases, checks real signed
+authorization-code tokens with and without the policy, and terminates the
+scratch processes. It uses a file backend, so it does not prove LDAP email
+lookup. Live acceptance also requires a fresh plus-addressed email sign-in
+through KeyCape to both Vergabe and the account portal, with successful
+callback and token issuance evidence.
+
+Before a 4.38 → 4.39 rollout, stop Authelia and preserve the SQLite database
+and any sidecar files. Keep the previous ConfigMap and image reference too.
+Rollback requires stopping 4.39 and restoring all three: database, ConfigMap,
+and image. The migrated database and 4.39 claims policy cannot be used by
+4.38. See `NK-WP-0041` for the exercised rollout and backup receipt.

sso-mfa/k8s/authelia/configmap.yaml

@@ -121,10 +121,17 @@ data:
     # KeyCape is the only registered client.
     identity_providers:
       oidc:
+        # Authelia 4.39 no longer includes profile claims in ID tokens by
+        # default. KeyCape uses this verified claim for directory/MFA lookup;
+        # its opaque OIDC subject is not a directory username (NK-WP-0041).
+        claims_policies:
+          keycape:
+            id_token: [preferred_username]
         # hmac_secret: injected via AUTHELIA_IDENTITY_PROVIDERS_OIDC_HMAC_SECRET_FILE
         # issuer_private_key: injected via AUTHELIA_IDENTITY_PROVIDERS_OIDC_ISSUER_PRIVATE_KEY_FILE
         clients:
           - id: keycape
+            claims_policy: keycape
             description: "KeyCape IAM Orchestration Layer"
             # OIDC clients are a list, so Authelia's *_FILE environment
             # mechanism cannot override this field. The template filter reads

sso-mfa/k8s/authelia/deployment.yaml

@@ -45,7 +45,8 @@ spec:
       containers:
         - name: authelia
           # Pin to a specific 4.x release. Check https://hub.docker.com/r/authelia/authelia
-          image: docker.io/authelia/authelia@sha256:46021dc20efdcc5cdc38a29e3050b8835429a155ae6215388ed3b793a02eb0ab  # 4.38 (rolled back from 4.39.28, NK-WP-0041-T02)
+          # Requires the KeyCape claims policy in configmap.yaml (NK-WP-0041).
+          image: docker.io/authelia/authelia@sha256:bd97cff4fcbf715b5ff1f9ae286afbe6033afce385302520b0368122d43a6f54  # 4.39.28
           imagePullPolicy: IfNotPresent
 
           ports:

sso-mfa/k8s/authelia/tests/probe_claims.py

@@ -0,0 +1,214 @@
+"""Integration regression for NK-WP-0041; needs requests, PyYAML, cryptography."""
+
+import argparse
+import base64
+import json
+import pathlib
+import socket
+import subprocess
+import tempfile
+import time
+import urllib.parse
+
+import requests
+import yaml
+from cryptography.hazmat.primitives import hashes, serialization
+from cryptography.hazmat.primitives.asymmetric import padding, rsa
+
+
+def decode(value):
+    return base64.urlsafe_b64decode(value + "=" * (-len(value) % 4))
+
+parser = argparse.ArgumentParser(
+    description="Exercise real Authelia ID tokens with and without the KeyCape claims policy; scratch data only."
+)
+parser.add_argument("--authelia-bin", required=True)
+binary = str(pathlib.Path(parser.parse_args().authelia_bin).resolve())
+key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
+pem = key.private_bytes(
+    serialization.Encoding.PEM,
+    serialization.PrivateFormat.PKCS8,
+    serialization.NoEncryption(),
+).decode()
+source = yaml.safe_load(
+    yaml.safe_load(
+        (pathlib.Path(__file__).resolve().parents[1] / "configmap.yaml").read_text()
+    )["data"]["configuration.yml"]
+)["identity_providers"]["oidc"]
+for enabled in (False, True):
+    with tempfile.TemporaryDirectory(prefix="nk-claims-") as tmp:
+        p = pathlib.Path(tmp)
+        with socket.socket() as s:
+            s.bind(("127.0.0.1", 0))
+            port = s.getsockname()[1]
+        password = "scratch-password-only"
+        digest = (
+            subprocess.check_output(
+                [
+                    binary,
+                    "crypto",
+                    "hash",
+                    "generate",
+                    "argon2",
+                    "--password",
+                    password,
+                ],
+                text=True,
+            )
+            .strip()
+            .split("Digest: ")[1]
+        )
+        (p / "users.yml").write_text(
+            yaml.safe_dump(
+                {
+                    "users": {
+                        "nk-probe": {
+                            "displayname": "Scratch User",
+                            "password": digest,
+                            "email": "nk-probe+x@example.com",
+                            "groups": [],
+                        }
+                    }
+                }
+            )
+        )
+        client = dict(source["clients"][0])
+        client.update(
+            secret="scratch-client-secret-only",
+            redirect_uris=["https://client.example.com/callback"],
+        )
+        if not enabled:
+            client.pop("claims_policy")
+        oidc = {
+            "hmac_secret": "h" * 64,
+            "jwks": [{"key": pem, "algorithm": "RS256", "use": "sig"}],
+            "clients": [client],
+        }
+        if enabled:
+            oidc["claims_policies"] = source["claims_policies"]
+        cfg = {
+            "ntp": {"disable_startup_check": True},
+            "server": {"address": f"tcp://127.0.0.1:{port}/"},
+            "log": {"level": "info"},
+            "authentication_backend": {"file": {"path": str(p / "users.yml")}},
+            "session": {
+                "secret": "s" * 64,
+                "cookies": [
+                    {
+                        "domain": "example.com",
+                        "authelia_url": "https://auth.example.com",
+                    }
+                ],
+            },
+            "storage": {
+                "encryption_key": "e" * 64,
+                "local": {"path": str(p / "db.sqlite3")},
+            },
+            "notifier": {"filesystem": {"filename": str(p / "notifications")}},
+            "access_control": {"default_policy": "one_factor"},
+            "identity_validation": {"reset_password": {"jwt_secret": "j" * 64}},
+            "identity_providers": {"oidc": oidc},
+        }
+        (p / "config.yml").write_text(yaml.safe_dump(cfg))
+        with (p / "log").open("w") as log:
+            proc = subprocess.Popen(
+                [binary, "--config", str(p / "config.yml")], stdout=log, stderr=log
+            )
+            try:
+                session = requests.Session()
+                session.trust_env = False
+                base = f"http://127.0.0.1:{port}"
+                session.headers.update(
+                    {
+                        "Host": "auth.example.com",
+                        "X-Forwarded-Proto": "https",
+                        "X-Forwarded-Host": "auth.example.com",
+                    }
+                )
+                for _ in range(100):
+                    if proc.poll() is not None:
+                        raise RuntimeError((p / "log").read_text())
+                    try:
+                        if (
+                            session.get(base + "/api/health", timeout=1).status_code
+                            == 200
+                        ):
+                            break
+                    except requests.ConnectionError:
+                        pass
+                    time.sleep(0.1)
+                else:
+                    raise RuntimeError("Scratch Authelia did not become healthy")
+                r = session.post(
+                    base + "/api/firstfactor",
+                    json={
+                        "username": "nk-probe",
+                        "password": password,
+                        "keepMeLoggedIn": False,
+                    },
+                    timeout=5,
+                )
+                assert r.status_code == 200, (r.status_code, r.text)
+                session.headers["Cookie"] = "; ".join(
+                    c.name + "=" + c.value for c in session.cookies
+                )
+                r = session.get(
+                    base + "/api/oidc/authorization",
+                    params={
+                        "client_id": "keycape",
+                        "redirect_uri": "https://client.example.com/callback",
+                        "response_type": "code",
+                        "scope": "openid profile email groups",
+                        "state": "scratch-state-long-enough",
+                        "nonce": "scratch-nonce-long-enough",
+                    },
+                    allow_redirects=False,
+                    timeout=5,
+                )
+                loc = r.headers.get("Location", "")
+                query = urllib.parse.parse_qs(urllib.parse.urlparse(loc).query)
+                assert "code" in query, (
+                    r.status_code,
+                    loc,
+                    r.text,
+                    (p / "log").read_text(),
+                )
+                r = session.post(
+                    base + "/api/oidc/token",
+                    auth=("keycape", "scratch-client-secret-only"),
+                    data={
+                        "grant_type": "authorization_code",
+                        "code": query["code"][0],
+                        "redirect_uri": "https://client.example.com/callback",
+                    },
+                    timeout=5,
+                )
+                assert r.status_code == 200, (r.status_code, r.text)
+                parts = r.json()["id_token"].split(".")
+                key.public_key().verify(
+                    decode(parts[2]),
+                    (".".join(parts[:2])).encode(),
+                    padding.PKCS1v15(),
+                    hashes.SHA256(),
+                )
+                claims = json.loads(decode(parts[1]))
+                assert claims["iss"] == "https://auth.example.com"
+                assert "keycape" in claims["aud"]
+                assert claims["nonce"] == "scratch-nonce-long-enough"
+                assert claims["exp"] > time.time()
+                assert (claims.get("preferred_username") == "nk-probe") == enabled
+                assert claims["sub"] != "nk-probe"
+                print(
+                    json.dumps(
+                        {
+                            "claims_policy": enabled,
+                            "signed_id_token_verified": True,
+                            "preferred_username_present": "preferred_username"
+                            in claims,
+                            "subject_is_directory_username": False,
+                        }
+                    )
+                )
+            finally:
+                proc.terminate()
+                proc.wait(timeout=10)

workplans/NK-WP-0041-onboarding-journey-usability.md

@@ -4,7 +4,7 @@ type: workplan
 title: "Fix onboarding-journey defects found in the 2026-09-23 human run"
 domain: infotech
 repo: net-kingdom
-status: active
+status: finished
 flavor: implementation
 owner: claude-code
 topic_slug: netkingdom
@@ -59,7 +59,7 @@ to a user; note it here if the password manager still misbehaves.
 
 ```task
 id: NK-WP-0041-T02
-status: wait
+status: done
 priority: medium
 state_hub_task_id: "83633649-3e5c-57e7-8207-609380a29c25"
 ```
@@ -260,3 +260,52 @@ explained. The plus-address filter defect remains on 4.38.
 `bernd.worsch-99` and could log in. The email address still cannot.
 That is the restored 4.38 behavior. The session closed there, with T02
 still `wait`.
+
+
+2026-09-27, third attempt: isolated diagnosis and claims-policy correction.
+Authelia 4.39 intentionally removed profile claims from default ID tokens
+([release notes](https://www.authelia.com/blog/4.39-release-notes/)). The deployed
+KeyCape commit `3b0446e` reads `preferred_username` from the verified ID token
+and falls back to `sub`; that opaque subject is not the LDAP username needed
+by privacyIDEA. The live config supplied no claims policy. This explains the
+MFA lookup failure; the old telemetry did not capture the exact lookup error.
+
+Added a KeyCape-only claims policy emitting `preferred_username`, retaining
+its existing scopes, audience behavior, one-factor policy and secret template.
+No MFA bypass was added. The regression probe at
+`sso-mfa/k8s/authelia/tests/probe_claims.py` runs an isolated real 4.39.28
+provider with disposable users/keys and completes authorization-code exchanges:
+without the policy the signed ID token omits the username; with the policy it
+contains the expected directory username. The subject is distinct in both
+cases. Both signatures and the public HTTPS issuer are checked. Scratch NTP
+startup checking is disabled so this local test does not depend on external
+clock services; production NTP configuration is unchanged. The complete repo
+config also passes 4.39.28 validation with placeholder secrets (legacy
+configuration deprecation warnings only).
+
+The operator confirmed availability for browser acceptance. Codex stopped
+Authelia before copying SQLite and verified `PRAGMA quick_check = ok`.
+Fresh rollback copy on the PVC:
+`backups/pre-4.39.28-claims-20260927T135544Z/db.sqlite3` (2,244,608 bytes).
+The old ConfigMap and deployment snapshots plus rollback helper are in
+`/tmp/nk-wp0041/` for this session. Rollback must restore the old config as
+well as the database and 4.38 image, since claims policies require 4.39.
+The claims policy and pinned 4.39.28 image were applied together.
+
+Closure evidence, 2026-09-27 (UTC):
+
+- Authelia is healthy on the declared 4.39.28 digest. The known LDAP startup
+  race caused two restarts; startup completed at 13:57:00.
+- At 13:57:44–45 both invalid-address probes (with and without `+`) returned
+  generic HTTP 401 and logged `user not found`, with no filter compile error.
+- Vergabe: KeyCape `auth_success` at 13:58:38 and authorization-code
+  `token_issued` at 13:58:39 for `vergabe-demo-company`.
+- Account portal: `auth_success` at 13:59:47 and authorization-code
+  `token_issued` at 13:59:48 for `user-engine-portal`.
+- The operator explicitly confirmed "Both sign-ins work" when asked to use
+  `bernd.worsch+99@gmail.com` in a fresh private window for both sites.
+
+T02 is done and NK-WP-0041 is finished. The scoped claims policy fixes the
+username propagation without changing MFA requirements. Transactional email
+link delivery remains the separately owned USER-WP-0035-T02 item described
+under T03; it is not a remaining task in this workplan.