net-kingdom/workplans/NK-WP-0034-verification-that-verifies.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	related	state_hub_workstream_id
NK-WP-0034	workplan	Make the SSO/MFA verification actually verify	infotech	net-kingdom	blocked	codex	infotech	2026-08-28	2026-09-05	NK-WP-0033 CUST-ADR-012	c87e142c-4eda-5c1b-84a9-ee5a848f3f63

Make the SSO/MFA verification actually verify

Goal

Close the blind spot NK-WP-0033 exposed: verification that passes while the thing it verifies cannot work.

Why this exists

reconcile-lldap-resolver-live.sh was approved, documented, revision-pinned and referenced by an incident workplan as the supported attended operation. Its first execution on 2026-08-27 found four defects, two of which meant it could never have completed a run under any circumstances.

verify-t06.sh reported success throughout.

It checked that a resolver and a realm existed. It did not check that GET /user/?realm=coulomb&username=<user> returns a user — which is the entire purpose of a resolver, and would have caught two of the four on the day they landed. The resolver was also created without TIMEOUT, CACHE_TIMEOUT and SIZELIMIT, so the WebUI had refused to save or test it for as long as it had existed, and nothing noticed.

A verification that asserts its objects exist rather than that its property holds is not a weaker check. It is a check that reports green while the system is broken, which is worse than no check, because it is believed.

Assert the property, not the objects

id: NK-WP-0034-T01
status: wait
priority: high
state_hub_task_id: "9b5d8034-0ef1-53f9-ad4d-37de536bdc62"

Rewrite verify-t06.sh so it proves what the realm and resolver are for: that a known user resolves through the realm, and that a privacyIDEA MFA validation against that user succeeds. Existence checks may stay as diagnostics; they may not stand in for the outcome.

Include the resolver's numeric parameters in what is asserted — a resolver the WebUI cannot save or test is misconfigured whether or not it currently answers.

Acceptance: with the tuning fields cleared by hand, verify-t06.sh fails. With them restored, it passes. Demonstrated in both directions, not argued.

Audit the other verify scripts for the same shape

id: NK-WP-0034-T02
status: done
priority: medium
state_hub_task_id: "0bd09a40-8b4d-5f9d-8d4c-81c55aa0c565"

verify-t02.sh through verify-t08.sh were written the same way by the same hand. Read each and record, per script, the property it is supposed to establish and whether it establishes it.

The output is a list, not a rewrite. Fixing them all is a larger decision than this task should make on its own.

Acceptance: one table — script, intended property, actual assertion, verdict.

Label every attended procedure with its exercise status

id: NK-WP-0034-T03
status: done
priority: medium
state_hub_task_id: "becb4dce-4971-5755-888f-5276c4a56fe7"

docs/attended-procedure-standard.md requires an Exercise status: line in each runbook. There are 32 shell scripts under sso-mfa/k8s/; the reconciliation runbook is the only one carrying the line.

Work through them and record, honestly, which have ever been run. "Unknown" is a permitted answer and a more useful one than a guess — the point is that an operator learns before running whether they are the first.

Acceptance: every attended procedure's runbook carries the line; scripts with no runbook are listed as a separate finding.

One helper per API

id: NK-WP-0034-T04
status: done
priority: low
state_hub_task_id: "d43ae8e8-eda6-5c1d-9e1b-01966b0e92da"

The Content-Type-on-bodyless-GET defect was found and fixed in bootstrap-realm.sh's pi_api, with a comment explaining it, and then reintroduced verbatim in a script written later in the same directory.

Extract one privacyIDEA request helper and have both scripts use it. A fix applied to a copy is a fix that expires the next time someone writes a script.

Acceptance: one implementation of the privacyIDEA request path in sso-mfa/k8s/privacyidea/; both callers use it; the GET behaviour has a test that fails if the header returns.

Review and implementation — 2026-09-05

Reviewed the proposal against current code and implemented all repository-local changes. T01 waits only for the attended scratch/provider exercise required by its acceptance. No live credentials were requested and no production resolver was changed.

• T01: replaced warning-only T06 success with an attended known-user and TOTP/HOTP verifier, requiring realm binding, numeric tuning, exact user and resolver, and token-backed success. Credentials are prompted in process; missing attendance fails. Updated realm-repair and DR callers. Eighteen tests exercise the actual shared HTTP transport, including clearing and restoring every tuning field and rejecting passthrough/static passwords. See docs/verify-t06.md for the remaining attended acceptance and limits.
• T02: completed the per-script property/assertion/verdict table in docs/verification-audit-2026-09-05.md. Other verify scripts retain their implementations; their gaps are explicitly recorded for follow-up.
• T03: inventoried every shell script under sso-mfa/k8s/, added exercise headers to component runbooks, recorded unknown history and scripts without runbooks in docs/attended-procedure-inventory.md. Corrected the reconciliation runbook's attempted-run claim: no successful completion receipt exists.
• T04: privacyidea/pi_api.py now supplies the request transport to both realm bootstrap and resolver reconciliation (and T06). Authentication also uses the shared transport in bootstrap. Its shell adapter keeps the token/body off argv. Fixed bootstrap's first-success set -e counter exit found during the review. The HTTP fixture rejects JSON Content-Type on GET and verifies the real Python and shell adapter requests.

Remaining gate: an attended operator runs the scratch target failure/restoration procedure and records receipts. An automated fixture is not reported as an exercised provider run. NK-WP-0033 still needs its own incident receipt and predecessor-disposition ruling.

Validation: python3 -m pytest tests tools -q passed 106 tests, including 18 privacyIDEA and 15 cadence cases. Ruff lint/format, shell syntax, embedded Python compilation and git diff --check passed.

State Hub reconciliation was attempted with both the installed CLI and current checkout. Full reconciliation remains pending because API queries/writes timed out or returned connection-refused errors. Generated index/intake metadata was reviewed; the source files remain authoritative.