From 13d41b953e7e879c2fdeb72ccdcc1557918fc61c Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 28 Aug 2026 11:50:51 +0200 Subject: [PATCH] feat(workplan): open NK-WP-0034 for verification that verifies MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit verify-t06.sh reported success against a resolver that had been unsaveable since creation and a reconciliation script that could never complete a run. It asserted that objects existed rather than that the property held. A check that reports green while the system is broken is worse than no check, because it is believed. T01 rewrites it to prove a user resolves and MFA validates, and must be demonstrated failing with the tuning fields cleared — not argued. T02 audits the sibling scripts for the same shape, output a table rather than a rewrite. T03 labels 32 attended scripts with exercise status, where "unknown" is permitted and more useful than a guess. T04 extracts the one privacyIDEA request helper whose absence let a fixed defect be reintroduced verbatim in the same directory. Co-Authored-By: Claude Opus 5 Assistant: claude-code Assistant-Model: opus Assistant-Process: 3377672@bnt-lap001 Assistant-Session: 15463ccf-238f-4e13-b163-93aa25c6d166 --- .../NK-WP-0034-verification-that-verifies.md | 115 ++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 workplans/NK-WP-0034-verification-that-verifies.md diff --git a/workplans/NK-WP-0034-verification-that-verifies.md b/workplans/NK-WP-0034-verification-that-verifies.md new file mode 100644 index 0000000..e118db0 --- /dev/null +++ b/workplans/NK-WP-0034-verification-that-verifies.md @@ -0,0 +1,115 @@ +--- +id: NK-WP-0034 +type: workplan +title: "Make the SSO/MFA verification actually verify" +domain: infotech +repo: net-kingdom +status: proposed +owner: codex +topic_slug: infotech +created: "2026-08-28" +related: + - NK-WP-0033 + - CUST-ADR-012 +--- + +# Make the SSO/MFA verification actually verify + +## Goal + +Close the blind spot `NK-WP-0033` exposed: verification that passes while the +thing it verifies cannot work. + +## Why this exists + +`reconcile-lldap-resolver-live.sh` was approved, documented, revision-pinned and +referenced by an incident workplan as the supported attended operation. Its first +execution on 2026-08-27 found four defects, two of which meant it could never +have completed a run under any circumstances. + +`verify-t06.sh` reported success throughout. + +It checked that a resolver and a realm existed. It did not check that +`GET /user/?realm=coulomb&username=` returns a user — which is the entire +purpose of a resolver, and would have caught two of the four on the day they +landed. The resolver was also created without `TIMEOUT`, `CACHE_TIMEOUT` and +`SIZELIMIT`, so the WebUI had refused to save or test it for as long as it had +existed, and nothing noticed. + +A verification that asserts its objects exist rather than that its property +holds is not a weaker check. It is a check that reports green while the system +is broken, which is worse than no check, because it is believed. + +## Assert the property, not the objects + +```task +id: NK-WP-0034-T01 +status: todo +priority: high +``` + +Rewrite `verify-t06.sh` so it proves what the realm and resolver are *for*: that +a known user resolves through the realm, and that a privacyIDEA MFA validation +against that user succeeds. Existence checks may stay as diagnostics; they may +not stand in for the outcome. + +Include the resolver's numeric parameters in what is asserted — a resolver the +WebUI cannot save or test is misconfigured whether or not it currently answers. + +Acceptance: with the tuning fields cleared by hand, `verify-t06.sh` fails. With +them restored, it passes. Demonstrated in both directions, not argued. + +## Audit the other verify scripts for the same shape + +```task +id: NK-WP-0034-T02 +status: todo +priority: medium +``` + +`verify-t02.sh` through `verify-t08.sh` were written the same way by the same +hand. Read each and record, per script, the property it is supposed to establish +and whether it establishes it. + +The output is a list, not a rewrite. Fixing them all is a larger decision than +this task should make on its own. + +Acceptance: one table — script, intended property, actual assertion, verdict. + +## Label every attended procedure with its exercise status + +```task +id: NK-WP-0034-T03 +status: todo +priority: medium +``` + +`docs/attended-procedure-standard.md` requires an `Exercise status:` line in +each runbook. There are 32 shell scripts under `sso-mfa/k8s/`; the reconciliation +runbook is the only one carrying the line. + +Work through them and record, honestly, which have ever been run. "Unknown" +is a permitted answer and a more useful one than a guess — the point is that an +operator learns before running whether they are the first. + +Acceptance: every attended procedure's runbook carries the line; scripts with no +runbook are listed as a separate finding. + +## One helper per API + +```task +id: NK-WP-0034-T04 +status: todo +priority: low +``` + +The `Content-Type`-on-bodyless-GET defect was found and fixed in +`bootstrap-realm.sh`'s `pi_api`, with a comment explaining it, and then +reintroduced verbatim in a script written later in the same directory. + +Extract one privacyIDEA request helper and have both scripts use it. A fix +applied to a copy is a fix that expires the next time someone writes a script. + +Acceptance: one implementation of the privacyIDEA request path in +`sso-mfa/k8s/privacyidea/`; both callers use it; the GET behaviour has a test +that fails if the header returns.