net-kingdom/workplans/NK-WP-0034-verification-that-verifies.md
tegwick 4e07d60ff1
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Validate cadence contract and require functional MFA verification
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a06ea3-7939-7b63-8125-699f8b50bedd
2026-09-05 01:28:05 +02:00

164 lines
6.6 KiB
Markdown

---
id: NK-WP-0034
type: workplan
title: "Make the SSO/MFA verification actually verify"
domain: infotech
repo: net-kingdom
status: blocked
owner: codex
topic_slug: infotech
created: "2026-08-28"
updated: "2026-09-05"
related:
- NK-WP-0033
- CUST-ADR-012
state_hub_workstream_id: "c87e142c-4eda-5c1b-84a9-ee5a848f3f63"
---
# Make the SSO/MFA verification actually verify
## Goal
Close the blind spot `NK-WP-0033` exposed: verification that passes while the
thing it verifies cannot work.
## Why this exists
`reconcile-lldap-resolver-live.sh` was approved, documented, revision-pinned and
referenced by an incident workplan as the supported attended operation. Its first
execution on 2026-08-27 found four defects, two of which meant it could never
have completed a run under any circumstances.
`verify-t06.sh` reported success throughout.
It checked that a resolver and a realm existed. It did not check that
`GET /user/?realm=coulomb&username=<user>` returns a user — which is the entire
purpose of a resolver, and would have caught two of the four on the day they
landed. The resolver was also created without `TIMEOUT`, `CACHE_TIMEOUT` and
`SIZELIMIT`, so the WebUI had refused to save or test it for as long as it had
existed, and nothing noticed.
A verification that asserts its objects exist rather than that its property
holds is not a weaker check. It is a check that reports green while the system
is broken, which is worse than no check, because it is believed.
## Assert the property, not the objects
```task
id: NK-WP-0034-T01
status: wait
priority: high
state_hub_task_id: "9b5d8034-0ef1-53f9-ad4d-37de536bdc62"
```
Rewrite `verify-t06.sh` so it proves what the realm and resolver are *for*: that
a known user resolves through the realm, and that a privacyIDEA MFA validation
against that user succeeds. Existence checks may stay as diagnostics; they may
not stand in for the outcome.
Include the resolver's numeric parameters in what is asserted — a resolver the
WebUI cannot save or test is misconfigured whether or not it currently answers.
Acceptance: with the tuning fields cleared by hand, `verify-t06.sh` fails. With
them restored, it passes. Demonstrated in both directions, not argued.
## Audit the other verify scripts for the same shape
```task
id: NK-WP-0034-T02
status: done
priority: medium
state_hub_task_id: "0bd09a40-8b4d-5f9d-8d4c-81c55aa0c565"
```
`verify-t02.sh` through `verify-t08.sh` were written the same way by the same
hand. Read each and record, per script, the property it is supposed to establish
and whether it establishes it.
The output is a list, not a rewrite. Fixing them all is a larger decision than
this task should make on its own.
Acceptance: one table — script, intended property, actual assertion, verdict.
## Label every attended procedure with its exercise status
```task
id: NK-WP-0034-T03
status: done
priority: medium
state_hub_task_id: "becb4dce-4971-5755-888f-5276c4a56fe7"
```
`docs/attended-procedure-standard.md` requires an `Exercise status:` line in
each runbook. There are 32 shell scripts under `sso-mfa/k8s/`; the reconciliation
runbook is the only one carrying the line.
Work through them and record, honestly, which have ever been run. "Unknown"
is a permitted answer and a more useful one than a guess — the point is that an
operator learns before running whether they are the first.
Acceptance: every attended procedure's runbook carries the line; scripts with no
runbook are listed as a separate finding.
## One helper per API
```task
id: NK-WP-0034-T04
status: done
priority: low
state_hub_task_id: "d43ae8e8-eda6-5c1d-9e1b-01966b0e92da"
```
The `Content-Type`-on-bodyless-GET defect was found and fixed in
`bootstrap-realm.sh`'s `pi_api`, with a comment explaining it, and then
reintroduced verbatim in a script written later in the same directory.
Extract one privacyIDEA request helper and have both scripts use it. A fix
applied to a copy is a fix that expires the next time someone writes a script.
Acceptance: one implementation of the privacyIDEA request path in
`sso-mfa/k8s/privacyidea/`; both callers use it; the GET behaviour has a test
that fails if the header returns.
## Review and implementation — 2026-09-05
Reviewed the proposal against current code and implemented all repository-local
changes. T01 waits only for the attended scratch/provider exercise required by
its acceptance. No live credentials were requested and no production resolver
was changed.
- T01: replaced warning-only T06 success with an attended known-user and
TOTP/HOTP verifier, requiring realm binding, numeric tuning, exact user and
resolver, and token-backed success. Credentials are prompted in process;
missing attendance fails. Updated realm-repair and DR callers. Eighteen
tests exercise the actual shared HTTP transport, including clearing and
restoring every tuning field and rejecting passthrough/static passwords.
See `docs/verify-t06.md` for the remaining attended acceptance and limits.
- T02: completed the per-script property/assertion/verdict table in
`docs/verification-audit-2026-09-05.md`. Other verify scripts retain their
implementations; their gaps are explicitly recorded for follow-up.
- T03: inventoried every shell script under `sso-mfa/k8s/`, added exercise
headers to component runbooks, recorded unknown history and scripts without
runbooks in `docs/attended-procedure-inventory.md`. Corrected the reconciliation
runbook's attempted-run claim: no successful completion receipt exists.
- T04: `privacyidea/pi_api.py` now supplies the request transport to both realm
bootstrap and resolver reconciliation (and T06). Authentication also uses the
shared transport in bootstrap. Its shell adapter keeps the token/body off
argv. Fixed bootstrap's first-success `set -e` counter exit found during the
review. The HTTP fixture rejects JSON Content-Type on GET and verifies the
real Python and shell adapter requests.
Remaining gate: an attended operator runs the scratch target failure/restoration
procedure and records receipts. An automated fixture is not reported as an
exercised provider run. NK-WP-0033 still needs its own incident receipt and
predecessor-disposition ruling.
Validation: `python3 -m pytest tests tools -q` passed 106 tests, including 18
privacyIDEA and 15 cadence cases. Ruff lint/format, shell syntax, embedded Python
compilation and `git diff --check` passed.
State Hub reconciliation was attempted with both the installed CLI and current
checkout. Full reconciliation remains pending because API queries/writes timed
out or returned connection-refused errors. Generated index/intake metadata was
reviewed; the source files remain authoritative.