ops: record native tenant success and prepare attended identity repair

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a07ff8-19d0-7820-b4d0-1353833cb7fc

WORK-RECORDS.md

@@ -131,7 +131,9 @@
 | task | NK-WP-0035-T05 | done | — | workplans/NK-WP-0035-emission-cadence-security-profile.md |
 | task | NK-WP-0036-T01 | done | — | workplans/NK-WP-0036-restore-user-portal-client-registration.md |
 | task | NK-WP-0036-T02 | done | — | workplans/NK-WP-0036-restore-user-portal-client-registration.md |
-| task | NK-WP-0036-T03 | progress | — | workplans/NK-WP-0036-restore-user-portal-client-registration.md |
+| task | NK-WP-0036-T03 | done | — | workplans/NK-WP-0036-restore-user-portal-client-registration.md |
+| task | NK-WP-0036-T04 | wait | — | workplans/NK-WP-0036-restore-user-portal-client-registration.md |
+| task | NK-WP-0036-T05 | todo | — | workplans/NK-WP-0036-restore-user-portal-client-registration.md |
 | intake | NK-IN-0001 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |
 | intake | NK-IN-0002 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |
 | intake | NET-IN-0001 | open | — | intakes/intakes.md |

docs/evidence/2026-09-11-native-portal-repair.md

@@ -36,4 +36,27 @@ Local validation: Flex Auth make test (including race tests), 27 policy tests an
 User Engine make test: 169 tests, 3 optional integration skips, layer check passed;
 regression proves authority 403 reaches the browser as redacted 403 without
 creating an administrator. Source fixes: flex-auth dd8dd51, user-engine 3c85e56.
-CI publication/live promotion are pending; these are not live success claims.
+CI succeeded: Flex Auth smoke 159/image 160, User Engine smoke 116/image 117.
+Tenant PDP Helm revision 3 now serves
+sha256:05a03a8790c2210c48ea92391441c77ddf640d0cd32f5ec09838f5393171fcbd.
+User Engine now serves
+sha256:9369d6709269395751a8db301820dd8d04160d7126c3c8f509a70341d409fde8.
+Both are ready at their prior 25m and 50m requests. Nine live checks from the
+Tenant Engine consumer confirm intended allow/deny outcomes and missing-token
+401. Native User Engine adapter readback returns demo-company not found, so the
+failed form submission left no tenant. A fresh native operator login was opened
+for the continued tenant onboarding in RAPPS-WP-0014-T02.
+
+
+At 19:03:18 UTC the operator's second native tenant-form submission returned 200.
+Native operator readback and the deployed User Engine adapter confirm
+`tenant:trial:demo-company`, display `demo-company`, lifecycle active, version 1.
+The tenant administration page confirms the chosen first administrator exists
+with invited account status. No private contact address is recorded here.
+Ordinary demo users, application deployment and onboarding remain RAPPS-WP-0014.
+
+The observation helper mistakenly collected full browser storage every five
+seconds; Playwright opened temporary pages for previously visited origins.
+After the operator reported the repeated page activity, the helper was stopped
+and changed to cookie-only persistence. The confirmed tenant submission was not
+repeated. This was helper-induced UI disruption, not evidence of a tenant failure.

docs/identity-provisioner-bind-repair.md

@@ -0,0 +1,68 @@
+# Repair the identity provisioner's stored LLDAP credential
+
+NK-WP-0036-T04, 2026-09-11. Prepared and tested; live apply has not run.
+
+The native User Engine Create login request reaches identity-provisioner, whose
+LLDAP admin login returns HTTP 401 before identity creation. Reloading the
+existing lldap-secrets reference did not fix it. The tenant and user-domain
+records are independent and remain intact. Root portal login is working.
+
+This is a proposed attended consumer-reference reconciliation. It does not
+rotate the LLDAP account, change signing keys, restore an exposed predecessor,
+read a Secret payload, or rebuild KeyCape's configuration. The operator must
+supply the currently working LLDAP `admin` password from existing custody through
+a hidden terminal prompt. The platform-root portal password is a different input.
+No value belongs in chat, shell arguments or work records. Do not fetch values
+through the unresolved warden pointer lane or a Kubernetes Secret export.
+
+The owner contract is warden route net-kingdom-lldap-bind-credential, with
+railiance-platform custody and the NetKingdom attended procedure. Operator
+acceptance of this narrow repair is required before its explicit apply command;
+the older full-bundle incident approval is not reused. If the current working
+credential is unavailable, stop at the custody-owner recovery boundary.
+
+The helper is sso-mfa/k8s/lldap/identity-provisioner-reconcile.py. Run inspect
+first; it emits only the exact Secret UID/resourceVersion and rejects controller
+ownership or the wrong cluster:
+
+```sh
+python3 sso-mfa/k8s/lldap/identity-provisioner-reconcile.py inspect
+python3 sso-mfa/k8s/lldap/identity-provisioner-reconcile.py check
+```
+
+`check` requires an interactive terminal and a hidden current-password prompt.
+It authenticates the existing admin against the pinned in-cluster LLDAP URL and
+performs a directory read. It changes no provider or consumer state.
+
+After explicit operator acceptance, use the UID/resourceVersion from inspect:
+
+```sh
+python3 sso-mfa/k8s/lldap/identity-provisioner-reconcile.py apply   --expected-uid c6a9e6be-5bb5-47e6-9faa-06b8d72afec3   --expected-resource-version 51345775
+```
+
+The helper validates the candidate against the provider before any write, server
+ dry-runs a JSON patch, then updates only lldap-secrets/LLDAP_LDAP_USER_PASS with
+UID/resourceVersion tests. Values travel only in child stdin/process memory;
+there is no temporary credential file, Secret export or full-manifest output.
+It restarts only identity-provisioner, waits for readiness and verifies the
+reloaded credential through the consumer. Failure after a successful field
+update is reported as incomplete verification; the rejected old value is never
+restored. Other login services and the LLDAP provider are not restarted.
+
+Seven synthetic tests cover exact patch scope, stale metadata and controller
+refusal, candidate rejection before writes, check-only behavior, stdin-only
+value handling, redaction of child errors and the apply/reload/proof sequence.
+The live metadata-only inspect succeeds. Live check/apply requires operator
+input and is not claimed by those tests.
+
+After success, retry Create login only for the existing intended user, inspect
+the returned password-setup page and record the native identity linkage.
+Password setup links are private, short-lived and must not enter evidence.
+User Engine's display name is not necessarily its directory username: the
+current provider derives a name from email unless preferred_username is sent.
+Keep the requested demo login-name mapping explicit before provisioning.
+
+Follow-up remains necessary for credential custody/publication and a functional
+provisioner preflight: the current /healthz confirms process health while the
+directory connection is broken. The HTTP handler also fails to catch the
+upstream HTTPError. These are tracked in NK-WP-0036-T05.

sso-mfa/k8s/lldap/identity-provisioner-reconcile.py

@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+"""Attended repair of one consumer field using an operator-held current password.
+
+Never reads a Secret payload, changes a provider password, or copies a bundle.
+"""
+from __future__ import annotations
+
+import argparse
+import base64
+import getpass
+import json
+import re
+import subprocess
+import sys
+
+CLUSTER_UID = 'a553c742-0115-43d4-99a4-a5ca56fe0786'
+KUBE = ['kubectl', '--kubeconfig', '/home/worsch/.kube/config-hosteurope',
+        '--server', 'https://127.0.0.1:16444', '--request-timeout=20s']
+NAMESPACE = 'sso'
+SECRET = 'lldap-secrets'
+KEY = 'LLDAP_LDAP_USER_PASS'
+# The password and returned directory token remain in this child process only.
+# Child stdin carries the operator input; no credential is a process argument.
+VERIFY = '''import json,os,sys
+from urllib.error import HTTPError
+from provisioner import LLDAPProvisioner
+try:
+ assert os.environ['LLDAP_URL']=='http://lldap.sso.svc.cluster.local:17170'
+ p=LLDAPProvisioner(base_url=os.environ['LLDAP_URL'],admin_password=json.load(sys.stdin)['password'])
+ token=p._login()
+ p._gql(token,'query { groups { id } }',{})
+except Exception:
+ print('directory-check-failed');sys.exit(1)
+print('directory-check-passed')
+'''
+VERIFY_RELOADED = '''import os,sys
+from provisioner import LLDAPProvisioner
+try:
+ p=LLDAPProvisioner(base_url=os.environ['LLDAP_URL'],admin_password=os.environ['LLDAP_ADMIN_PASSWORD'])
+ token=p._login()
+ p._gql(token,'query { groups { id } }',{})
+except Exception:
+ print('reloaded-check-failed');sys.exit(1)
+print('reloaded-check-passed')
+'''
+
+
+class Refused(Exception):
+    """Fixed safe code only; never carry child output or an exception repr."""
+
+
+def run(arguments, *, data=None, timeout=30):
+    try:
+        result = subprocess.run(KUBE + arguments, input=data, text=True,
+                                capture_output=True, timeout=timeout)
+    except (OSError, subprocess.TimeoutExpired):
+        raise Refused('command_unavailable') from None
+    if result.returncode:
+        raise Refused('command_refused') from None
+    return result.stdout.strip()
+
+
+def inspect():
+    uid = run(['get', 'namespace', 'kube-system', '-o', 'jsonpath={.metadata.uid}'])
+    if uid != CLUSTER_UID:
+        raise Refused('wrong_cluster')
+    # Do not include annotations: last-applied configuration can contain data.
+    fields = run(['-n', NAMESPACE, 'get', 'secret', SECRET, '-o',
+                  'go-template={{.metadata.uid}} {{.metadata.resourceVersion}} {{if .metadata.ownerReferences}}{{len .metadata.ownerReferences}}{{else}}0{{end}}']).split()
+    if len(fields) != 3 or not re.fullmatch(r'[a-f0-9-]{36}', fields[0]) or not fields[1].isdigit():
+        raise Refused('invalid_metadata')
+    if fields[2] != '0':
+        raise Refused('controller_owned_secret')
+    return {'uid': fields[0], 'resource_version': fields[1]}
+
+
+def patch(password, metadata):
+    return [
+        {'op': 'test', 'path': '/metadata/uid', 'value': metadata['uid']},
+        {'op': 'test', 'path': '/metadata/resourceVersion', 'value': metadata['resource_version']},
+        {'op': 'replace', 'path': '/data/' + KEY,
+         'value': base64.b64encode(password.encode()).decode()},
+    ]
+
+
+def reconcile(password, *, mode, expected_uid=None, expected_resource_version=None):
+    if not password or len(password) > 4096 or '\x00' in password:
+        raise Refused('invalid_protected_input')
+    metadata = inspect()
+    if mode == 'apply' and metadata != {'uid': expected_uid, 'resource_version': expected_resource_version}:
+        raise Refused('stale_metadata')
+    result = run(['-n', NAMESPACE, 'exec', '-i', 'deployment/identity-provisioner',
+                  '--', 'python', '-c', VERIFY], data=json.dumps({'password': password}))
+    if result != 'directory-check-passed':
+        raise Refused('current_provider_credential_unverified')
+    if mode == 'check':
+        return {'result': 'checked', 'provider_login': True, 'secret_written': False, **metadata}
+    payload = json.dumps(patch(password, metadata))
+    command = ['-n', NAMESPACE, 'patch', 'secret', SECRET, '--type=json',
+               '--patch-file=/dev/stdin', '-o', 'name']
+    run(command + ['--dry-run=server'], data=payload)
+    run(command, data=payload)
+    # Do not restore the known nonworking field on failure. Report partial repair.
+    try:
+        run(['-n', NAMESPACE, 'rollout', 'restart', 'deployment/identity-provisioner'])
+        run(['-n', NAMESPACE, 'rollout', 'status', 'deployment/identity-provisioner',
+             '--timeout=120s'], timeout=150)
+        result = run(['-n', NAMESPACE, 'exec', 'deployment/identity-provisioner',
+                      '--', 'python', '-c', VERIFY_RELOADED])
+        if result != 'reloaded-check-passed':
+            raise Refused('consumer_verification_failed')
+    except Refused:
+        raise Refused('field_updated_consumer_verification_incomplete') from None
+    return {'result': 'reconciled', 'provider_login': True, 'consumer_login': True,
+            'provider_password_changed': False, 'secret_field': KEY, **inspect()}
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('mode', choices=['inspect', 'check', 'apply'])
+    parser.add_argument('--expected-uid')
+    parser.add_argument('--expected-resource-version')
+    args = parser.parse_args()
+    try:
+        if args.mode == 'inspect':
+            result = {'result': 'inspected', **inspect()}
+        else:
+            if args.mode == 'apply' and not (args.expected_uid and args.expected_resource_version):
+                raise Refused('exact_metadata_required')
+            if not sys.stdin.isatty():
+                raise Refused('attended_terminal_required')
+            password = getpass.getpass('Current working LLDAP admin password (hidden): ')
+            result = reconcile(password, mode=args.mode, expected_uid=args.expected_uid,
+                               expected_resource_version=args.expected_resource_version)
+            del password
+        print(json.dumps(result, sort_keys=True))
+        return 0
+    except Refused as error:
+        print(json.dumps({'result': 'refused', 'code': str(error)}))
+        return 1
+
+
+if __name__ == '__main__':
+    raise SystemExit(main())

sso-mfa/k8s/lldap/test_identity_provisioner_reconcile.py

@@ -0,0 +1,74 @@
+import base64
+import importlib.util
+import io
+import json
+from pathlib import Path
+import subprocess
+import unittest
+from unittest.mock import patch
+
+spec=importlib.util.spec_from_file_location('repair',Path(__file__).with_name('identity-provisioner-reconcile.py'))
+repair=importlib.util.module_from_spec(spec)
+spec.loader.exec_module(repair)
+META={'uid':'c6a9e6be-5bb5-47e6-9faa-06b8d72afec3','resource_version':'1234'}
+SYNTHETIC='synthetic-only-current-password'
+
+
+class ReconcileTests(unittest.TestCase):
+    def test_patch_has_exact_guards_and_changes_only_the_named_field(self):
+        changes=repair.patch(SYNTHETIC,META)
+        self.assertEqual(['test','test','replace'],[x['op'] for x in changes])
+        self.assertEqual('/metadata/uid',changes[0]['path'])
+        self.assertEqual('/metadata/resourceVersion',changes[1]['path'])
+        self.assertEqual('/data/LLDAP_LDAP_USER_PASS',changes[2]['path'])
+        self.assertEqual(SYNTHETIC,base64.b64decode(changes[2]['value']).decode())
+
+    def test_stale_metadata_stops_before_credential_use(self):
+        with patch.object(repair,'inspect',return_value=META), patch.object(repair,'run') as run:
+            with self.assertRaisesRegex(repair.Refused,'stale_metadata'):
+                repair.reconcile(SYNTHETIC,mode='apply',expected_uid=META['uid'],expected_resource_version='old')
+            run.assert_not_called()
+
+    def test_provider_refusal_stops_before_any_write(self):
+        with patch.object(repair,'inspect',return_value=META), patch.object(repair,'run',side_effect=repair.Refused('command_refused')) as run:
+            with self.assertRaises(repair.Refused):
+                repair.reconcile(SYNTHETIC,mode='apply',expected_uid=META['uid'],expected_resource_version=META['resource_version'])
+            self.assertEqual(1,run.call_count)
+            self.assertNotIn('patch',run.call_args.args[0])
+
+    def test_check_does_not_write_and_keeps_password_out_of_arguments(self):
+        with patch.object(repair,'inspect',return_value=META), patch.object(repair,'run',return_value='directory-check-passed') as run:
+            result=repair.reconcile(SYNTHETIC,mode='check')
+            self.assertFalse(result['secret_written'])
+            self.assertNotIn(SYNTHETIC,str(run.call_args.args))
+            self.assertNotIn(SYNTHETIC,json.dumps(result))
+            self.assertEqual(SYNTHETIC,json.loads(run.call_args.kwargs['data'])['password'])
+
+    def test_apply_dry_runs_then_writes_reloads_and_verifies(self):
+        results=['directory-check-passed','secret/lldap-secrets','secret/lldap-secrets','restarted','ready','reloaded-check-passed']
+        with patch.object(repair,'inspect',return_value=META), patch.object(repair,'run',side_effect=results) as run:
+            result=repair.reconcile(SYNTHETIC,mode='apply',expected_uid=META['uid'],expected_resource_version=META['resource_version'])
+            self.assertEqual('reconciled',result['result'])
+            self.assertFalse(result['provider_password_changed'])
+            calls=run.call_args_list
+            self.assertIn('--dry-run=server',calls[1].args[0])
+            self.assertNotIn('--dry-run=server',calls[2].args[0])
+            for call in calls:
+                self.assertNotIn(SYNTHETIC,str(call.args))
+                self.assertNotIn(base64.b64encode(SYNTHETIC.encode()).decode(),str(call.args))
+
+    def test_child_errors_never_reveal_output(self):
+        failed=subprocess.CompletedProcess([],1,stdout=SYNTHETIC,stderr=SYNTHETIC)
+        with patch.object(subprocess,'run',return_value=failed):
+            with self.assertRaises(repair.Refused) as result:
+                repair.run(['anything'])
+            self.assertEqual('command_refused',str(result.exception))
+
+    def test_wrong_cluster_and_controller_owner_refused(self):
+        with patch.object(repair,'run',return_value='foreign-cluster'):
+            with self.assertRaisesRegex(repair.Refused,'wrong_cluster'): repair.inspect()
+        with patch.object(repair,'run',side_effect=[repair.CLUSTER_UID,META['uid']+' 1234 1']):
+            with self.assertRaisesRegex(repair.Refused,'controller_owned_secret'): repair.inspect()
+
+
+if __name__=='__main__': unittest.main()

workplans/NK-WP-0036-restore-user-portal-client-registration.md

@@ -70,7 +70,7 @@ reached the platform form. See docs/evidence/2026-09-11-native-portal-repair.md.
 
 ```task
 id: NK-WP-0036-T03
-status: progress
+status: done
 priority: high
 state_hub_task_id: "378b03b3-aedc-50c3-8de7-94064794582b"
 ```
@@ -94,3 +94,58 @@ actual allowed/denied contract with authenticated consumer calls. Then reconcile
 native demo tenant existence before retrying the operator form. RAPPS-WP-0014
 retains actual tenant onboarding and product placement; this workplan does not
 claim tenant existence or application readiness from component tests alone.
+
+Completed: both owner suites and CI images passed, the Tenant Engine PDP and
+User Engine runtime were independently promoted by immutable digest, nine live
+policy/caller checks passed, and native demo-company readback returns not-found.
+Remaining actual operator submission, administrator onboarding and application
+placement remain live in RAPPS-WP-0014-T02 and VERGABE-WP-0019-T03/T04. This
+integration repair is complete; the invited product pilot is not yet admitted.
+
+
+Follow-through: the operator submitted the native form successfully at 19:03:18
+UTC. Independent authority readback confirms demo-company active/version 1;
+User Engine shows the first administrator as invited. Remaining ordinary members,
+identity activation and product accounts are retained in RAPPS-WP-0014-T02.
+
+
+## Reconcile the identity provisioner's rejected directory credential
+
+```task
+id: NK-WP-0036-T04
+status: wait
+priority: high
+needs_human: true
+intervention_note: "LLDAP rejects the provisioner's current admin login with 401 after reload. Prepared consumer-only reconciliation requires explicit operator acceptance and the current working LLDAP admin credential via hidden terminal input; never send it in chat."
+```
+
+A subsequent native Create login fails before directory identity creation, at
+LLDAPProvisioner._login, HTTP 401. The existing identity-provisioner image
+5b460f5c was restarted to reload its lldap-secrets reference; readiness passed
+but contained provider authentication still returns 401. No provider credential
+was changed and no rejected create was replayed. Native user records remain.
+
+The non-printing, field-only attended repair and seven synthetic tests are
+prepared at sso-mfa/k8s/lldap/identity-provisioner-reconcile.py. See
+ docs/identity-provisioner-bind-repair.md for exact scope, operator input,
+metadata guard, candidate authentication, apply, reload and verification.
+Live metadata inspect succeeds; live check/apply is pending current operator
+custody and acceptance. The unresolved warden route is a pointer, not authority
+to export another live Secret. Retain NK-WP-0033's separate incident residuals.
+
+## Make dependency failure visible before another human onboarding attempt
+
+```task
+id: NK-WP-0036-T05
+status: todo
+priority: high
+```
+
+Cover the real provisioner-to-directory contract in a contained preflight,
+return a redacted structured dependency failure instead of an uncaught HTTPError,
+and make CI execute those contract tests. Keep process liveness distinct from
+functional readiness. Publish and promote through a reviewed immutable image
+lane, then verify provider login, user creation/linkage and password setup as a
+single native onboarding journey. Preserve credential secrecy and avoid
+unbounded password-check retries. Actual demo users and application admission
+remain RAPPS-WP-0014 and VERGABE-WP-0019.