net-kingdom/workplans/NK-WP-0041-onboarding-journey-usability.md

---
id: NK-WP-0041
type: workplan
title: "Fix onboarding-journey defects found in the 2026-09-23 human run"
domain: infotech
repo: net-kingdom
status: finished
flavor: implementation
owner: claude-code
topic_slug: netkingdom
created: "2026-09-23"
updated: "2026-09-27"
related: [NK-WP-0036, NK-WP-0037, KEY-WP-0033]
state_hub_workstream_id: "98168f50-7a4d-5bb5-a462-1e031563b89f"
---

Found by the operator's native onboarding run (NK-WP-0036-T05) and the
blocked Vergabe sign-in (NK-WP-0037-T02). Each item names its owner, and only
items in this repository are implemented here.

## Password managers and sign-in after password setup

```task
id: NK-WP-0041-T01
status: done
priority: medium
state_hub_task_id: "82ce581a-b635-5b5b-9b08-91b61e7eb9bd"
```

The setup form had no username field, so browsers stored the new password
without a login. After setup there was no way on to the sign-in page. The
source is fixed in `identity-provisioner`:

- The setup form shows the grant's login name as a read-only
  `autocomplete="username"` field. The submitted value is ignored, and the
  form no longer sets `autocomplete="off"`.
- The completion page links to `PASSWORD_SETUP_SIGNIN_URL` (HTTPS only)
  when there is no company return. The deployment declares
  `https://users.coulomb.social/`.
- The completion page still does not show the login name, as an existing
  test requires.

Four new HTTP tests cover this, and the 32 provisioner tests pass.

Released on 2026-09-24. CI published `main-6c4fcaf` as
`identity-provisioner@sha256:ffacd5d7…`, and the operator applied the
declaration on railiance01. Only the image and the new env changed; the
other resources were unchanged. The new pod checks out:

- `/healthz` returns 200 and `/readyz` returns 200 (`dependency: directory`).
- An invalid setup link returns 400.
- The served code carries the `autocomplete="username"` field.
- `PASSWORD_SETUP_SIGNIN_URL` is set.

Rollback digest: `3317a261…`. The next real setup link will show the field
to a user; note it here if the password manager still misbehaves.

## Plus-addressed email sign-in fails with an LDAP filter error

```task
id: NK-WP-0041-T02
status: done
priority: medium
state_hub_task_id: "83633649-3e5c-57e7-8207-609380a29c25"
```

Authelia's `users_filter` accepts `uid` or `mail`. Sign-in as
`bernd.worsch+99@gmail.com` failed with "LDAP Result Code 201 Filter Compile
Error: invalid characters for escape", not a clean result. A probe with fake
addresses (`nk-probe@example.invalid` compared with `nk-probe+x@…`) showed
that only the `+` form triggers the error. Establish whether Authelia 4.38
escapes `+` DN-style inside the filter, and fix it through the reference
configuration or the version. Users who use plus-addressing cannot sign in
by email until then.

Diagnosis, 2026-09-24. This is an upstream Authelia defect.
`internal/authentication/ldap_util.go` `ldapEscape()` in v4.38.0 through
v4.38.19 (the last 4.38 release) and in v4.39.0 applies `ldap.EscapeFilter`
and then DN-escapes `, # + < > ; " =` as `\c`. Inside a filter, an escape
must be `\XX` hex, so the go-ldap filter compile fails for any username or
email containing one of those eight characters. v4.39.28 (2026-09-17) builds
the filter with `ldap.EscapeFilter(input)` only (`ldap_user_provider.go:720`).
No configuration workaround exists.

Upgrade prepared. The target is `authelia/authelia:4.39.28`, pinned as
`sha256:bd97cff4…`. The live image is the floating tag `4.38`
(`sha256:46021dc2…`). The live `authelia-config` was validated locally with
placeholder secrets under both versions: each returned exit 0 with no errors
and the same set of auto-mapped deprecation warnings. Storage is SQLite on
the PVC, and 4.39 migrates the schema on start. The rollback therefore needs
the pre-upgrade copy (`backups/db.sqlite3.pre-4.39.28`) as well as the old
digest. The daily backups continue.

## Route the portal findings to user-engine

```task
id: NK-WP-0041-T03
status: done
priority: low
state_hub_task_id: "7350f35a-6893-5ac8-bfc5-fb2e40b3cf5d"
```

These belong to user-engine:

- The setup link is shown to the provider instead of being delivered to the
  recipient (journey U04/T03: mail delivery unresolved).
- The derived login name (`bernd.worsch-99`) is not obvious to the recipient.
- There is no sign-in link from the user entry once a password has been set.

Sent, and the reply is recorded (user-engine, 2026-09-23, hub message
`a500cc67`, thread `21b0b7e4`). user-engine opened USER-WP-0035
(`35aa6adf-255a-5705-9294-a50d98e45f00`).

Findings 2 and 3 are implemented in source as USER-WP-0035-T01
(`a26b22ae-05fd-52f4-accc-068ccba7bbda`): the tenant-admin user entry shows
the sign-in address beside the login name and states that the email address
is not the login name, and the password-setup handoff repeats the sign-in
address and says the portal does not deliver the link. Their regression is
`test_journey_roles.UserJourneys.test_password_handoff_names_actual_login_and_failure_can_retry`.
That is source, not a live release. They will not suggest that email sign-in
works while NK-WP-0041-T02 is open.

Finding 1 stays USER-WP-0035-T02
(`fae614ba-ffc0-511d-9d22-2d91fbab057f`, status wait). Delivery needs the
governed transactional mail lane, which is operator-owned. A portal-rendered
link is not delivery evidence.

### Incident 2026-09-24: 4.39.28 rollout broke KeyCape sign-in; rolled back

Timeline (UTC, 2026-09-23 on the server clock):

- **23:34.** A pre-upgrade database copy was taken
  (`backups/db.sqlite3.pre-4.39.28`, 2,023,424 bytes).
- **23:35.** The operator rolled out v4.39.28. The schema migrated 15 → 29.
  The first start failed the LDAP startup check (network not ready) and
  restarted clean. Health, discovery and the probes passed; the `+` address
  became a clean not-found. KeyCape's redirect checks passed.
- **23:45 and 23:49.** Real sign-ins passed Authelia's first factor. Then
  Authelia rejected KeyCape's back-channel token request: "Error occurred
  determining the effective issuer … invalid X-Forwarded-Proto header value
  'http'" (`POST /api/oidc/token`). Every KeyCape sign-in was broken: the
  portal, Vergabe and the OpenBao browser login. The pre-rollout checks were
  redirect-only and could not see this. The 23:43 attempts had failed
  separately, because of a leading space in the pasted username (4.39 does
  not trim it).
- **23:53.** Rolled back with `/tmp/authelia-rollback.sh`: scale to 0,
  restore the pre-upgrade database via a helper pod (the migrated copy is
  kept as `backups/db.sqlite3.4.39.28-migrated`), then the 4.38 digest
  `sha256:46021dc2…`. The schema is "already up to date" (15), and health
  returned 200. Two restarts come from the same LDAP startup race.

Exposure: about 18 minutes in which KeyCape sign-ins failed.

Before retrying: KeyCape calls Authelia's token endpoint in-cluster over
plain HTTP, and 4.39 will not derive its issuer from an `http` forwarded
scheme. Resolve that first, either with KeyCape sending
`X-Forwarded-Proto: https` and the public host, or through an Authelia
4.39 setting for the in-cluster endpoint. Also add a real back-channel token
exchange to the upgrade acceptance, because redirect-only checks miss it.
The repository now pins the exact 4.38 digest instead of the floating tag.

Separate finding: Authelia's LDAP startup check fails on the first start
after a pod is scheduled, then passes on restart.

Operator confirmation after the rollback, 2026-09-24: sign-in as
`bernd.worsch-99` works again. `bernd.worsch+99@gmail.com` still fails on
4.38, as expected.

Root cause of the 4.39 break. KeyCape's `tokenBaseURL` is the in-cluster
`http://authelia.sso.svc.cluster.local:9091` (declared in
`sso-mfa/k8s/keycape/create-secrets.sh`). The token request sets no
forwarded headers (key-cape `adapter.go`), so Authelia sees an `http`
scheme. 4.38 accepts that; 4.39 will not derive an issuer from it.

Path forward. Stage the fix on 4.38 first, then upgrade with only one
variable changing:

- **A (preferred, config only).** Point `tokenBaseURL` at
  `https://auth.coulomb.social`, so the call goes through the ingress with
  `X-Forwarded-Proto: https`. First check that the KeyCape pod can reach the
  public hostname in-cluster (hairpin). Apply through the guarded KeyCape
  config lane with unrelated bytes preserved, then prove it with a real
  login on 4.38.
- **B (code).** key-cape sends `X-Forwarded-Proto: https` and
  `X-Forwarded-Host` on back-channel calls. Whether 4.39 trusts forwarded
  headers from a pod is unverified.

Then retry 4.39.28. The acceptance must include a real human sign-in
through KeyCape, and the `+` email sign-in. The task waits on the choice
of A or B, with key-cape consulted.

2026-09-27, checked through the Vergabe demo-company pilot
(`https://vergabe-teilnahme.coulomb.social/demo-company/`, customer
`tenant:trial:demo-company`). The welcome page is up. Starting sign-in
returns 302 to KeyCape `client_id=vergabe-demo-company` with PKCE, `prompt=login`,
and the company callback. KeyCape then sends the browser to
`https://auth.coulomb.social/api/oidc/authorization` with `max_age=10` and
client `keycape`. No credential was submitted.

The plus-address defect is still live on that same Authelia 4.38. A first-factor
probe for `nk-probe@example.invalid` is logged as user not found. The same probe
for `nk-probe+x@example.invalid` is logged as LDAP Result Code 201 Filter Compile
Error on `+`. Both responses to the browser are the generic authentication
failure, so the filter error is not visible on the sign-in page. Vendor
`tenant:friendly:binky` is a different tenant and is not admitted by this
client.

Path A is not reachable with the current network policy. Namespace `sso` is
default-deny. KeyCape egress allows only TCP 9091 to Authelia pods, TCP 3890
to LLDAP, TCP 8080 to the mfa namespace, and DNS. The pod has no egress to
the ingress or to port 443, so `tokenBaseURL: https://auth.coulomb.social`
would fail before any hairpin. The in-cluster token URL remains the only
allowed back-channel. key-cape is asked whether it will send
`X-Forwarded-Proto: https` and `X-Forwarded-Host: auth.coulomb.social` on
that existing call (path B). Authelia 4.39 trusting those headers from a
pod, rather than from Traefik, is still unverified. No live config or image
was changed.

2026-09-27, path B shipped by key-cape and staged on Authelia 4.38.
`ADMINISTER @ realm:kubernetes/railiance01`, `activation=APPROVED` for this
image change only. Commit `3b0446e`, tag `main-3b0446e`, digest
`sha256:7c99cd6c6fdf63afaf22e47dad31e20dcb3a8b2079206f198cb425047be495b3`.
The token call stays in-cluster and now sets those two headers from
`browserBaseURL`. KeyCape rollback digest `sha256:82f1e5ac…`.

2026-09-27, the founder reached the Vergabe confirmation page for
`bernd.worsch-99` ("Sie fahren mit bernd.worsch-99 fort") on this KeyCape
image with Authelia still 4.38. That is a completed demo-company sign-in,
so the header-bearing token call works against 4.38.

Authelia was then moved to 4.39.28, index digest `sha256:bd97cff4…`, under
the same `ADMINISTER @ realm:kubernetes/railiance01` approval, with rollback
required if the plus-address sign-in failed. The pre-upgrade database copy
is `backups/db.sqlite3.pre-4.39.28-20260927` (2,228,224 bytes). Startup
migrated storage schema 15 → 29, restarted once on the known LDAP race, and
then listened on 9091. Probes for `nk-probe@example.invalid` and
`nk-probe+x@example.invalid` both logged "user not found". The filter
compile error on `+` was gone.

The founder's following sign-in, using `bernd.worsch+99@gmail.com`, reached
KeyCape `/authorize/callback` and failed there. Telemetry at
2026-09-27T13:24:11Z is `error_type=mfa_check_error` for
`vergabe-demo-company`. The account site then failed the same way
(`user-engine-portal`, 13:25:02Z through 13:25:36Z) and showed
"Sign-in could not be completed". That error is raised only after Authelia's
callback has already returned a username, so the issuer rejection from the
2026-09-24 rollout did not recur. `decideAssurance` returned an error
(privacyIDEA token lookup or the policy store). The telemetry line does not
carry the underlying error.

Authelia was rolled back immediately. The 20260927 database copy was restored
over `db.sqlite3`, any sqlite wal/shm/journal beside it was removed, and the
image returned to `sha256:46021dc2…`. v4.38.19 started with the schema
already up to date and is listening. KeyCape stayed on `sha256:7c99cd6c…`.
T02 stays waiting: 4.39.28 identified the user, and the MFA check then
failed closed. Do not roll 4.39.28 forward again until that check is
explained. The plus-address filter defect remains on 4.38.

2026-09-27, after that rollback the founder signed in again as
`bernd.worsch-99` and could log in. The email address still cannot.
That is the restored 4.38 behavior. The session closed there, with T02
still `wait`.


2026-09-27, third attempt: isolated diagnosis and claims-policy correction.
Authelia 4.39 intentionally removed profile claims from default ID tokens
([release notes](https://www.authelia.com/blog/4.39-release-notes/)). The deployed
KeyCape commit `3b0446e` reads `preferred_username` from the verified ID token
and falls back to `sub`; that opaque subject is not the LDAP username needed
by privacyIDEA. The live config supplied no claims policy. This explains the
MFA lookup failure; the old telemetry did not capture the exact lookup error.

Added a KeyCape-only claims policy emitting `preferred_username`, retaining
its existing scopes, audience behavior, one-factor policy and secret template.
No MFA bypass was added. The regression probe at
`sso-mfa/k8s/authelia/tests/probe_claims.py` runs an isolated real 4.39.28
provider with disposable users/keys and completes authorization-code exchanges:
without the policy the signed ID token omits the username; with the policy it
contains the expected directory username. The subject is distinct in both
cases. Both signatures and the public HTTPS issuer are checked. Scratch NTP
startup checking is disabled so this local test does not depend on external
clock services; production NTP configuration is unchanged. The complete repo
config also passes 4.39.28 validation with placeholder secrets (legacy
configuration deprecation warnings only).

The operator confirmed availability for browser acceptance. Codex stopped
Authelia before copying SQLite and verified `PRAGMA quick_check = ok`.
Fresh rollback copy on the PVC:
`backups/pre-4.39.28-claims-20260927T135544Z/db.sqlite3` (2,244,608 bytes).
The old ConfigMap and deployment snapshots plus rollback helper are in
`/tmp/nk-wp0041/` for this session. Rollback must restore the old config as
well as the database and 4.38 image, since claims policies require 4.39.
The claims policy and pinned 4.39.28 image were applied together.

Closure evidence, 2026-09-27 (UTC):

- Authelia is healthy on the declared 4.39.28 digest. The known LDAP startup
  race caused two restarts; startup completed at 13:57:00.
- At 13:57:44–45 both invalid-address probes (with and without `+`) returned
  generic HTTP 401 and logged `user not found`, with no filter compile error.
- Vergabe: KeyCape `auth_success` at 13:58:38 and authorization-code
  `token_issued` at 13:58:39 for `vergabe-demo-company`.
- Account portal: `auth_success` at 13:59:47 and authorization-code
  `token_issued` at 13:59:48 for `user-engine-portal`.
- The operator explicitly confirmed "Both sign-ins work" when asked to use
  `bernd.worsch+99@gmail.com` in a fresh private window for both sites.

T02 is done and NK-WP-0041 is finished. The scoped claims policy fixes the
username propagation without changing MFA requirements. Transactional email
link delivery remains the separately owned USER-WP-0035-T02 item described
under T03; it is not a remaining task in this workplan.