tegwick
c99b8da41f
Three gaps from NK-WP-0033, none of them the fault we set out to fix. Operator custody (platform-root-custody.md). This document named a password-safe entry that had never been created, so the 2026-08-23 rotation had nowhere to put the new LLDAP admin value and the outgoing one survived only in a browser entry that was overwritten on 2026-08-27. Decided: operators/<subsystem>/<account> in OpenBao for everything reachable once logged in, plus a short closed offline list for what you need when OpenBao is not — platform-root's own password, unseal shares, break-glass. Anything on that list in OpenBao would be circular; anything not on it does not belong in the packet. The rule that closes it: every rotation names, as a numbered step, where the outgoing and incoming values go. NK-WP-0033-T04 rotated four classes with no such step, and that is the root cause of the session that followed — not the four software defects it also surfaced. Secret disclosure by trust stage, tied to the existing S0-S6 table. A fingerprint may be spoken to confirm two parties hold the same credential; the whole value never may. Used ad hoc throughout 2026-08-27 and worth writing down. Attended procedures (new standard). reconcile-lldap-resolver-live.sh was approved, documented, revision-pinned and had never run; its first execution found four defects, two of which meant it could never have completed. verify-t06.sh passed throughout. An unexecuted procedure is unfinished work and must say so in its runbook header. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3377672@bnt-lap001 Assistant-Session: 15463ccf-238f-4e13-b163-93aa25c6d166
3.3 KiB
Attended procedures: a script that has never run is not a procedure
Adopted 2026-08-28, after NK-WP-0033.
What happened
sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh was written, reviewed,
documented in an approved runbook, pinned by revision, and referenced by an
incident workplan as the supported one-command attended operation. Its own
workplan entry said plainly: "It has not been run."
The first execution, on 2026-08-27, found four defects:
Content-Type: application/jsonon bodyless GETs — Werkzeug rejects those in front of the application, so every GET the script made returned an HTML 400. POSTs worked, which made the failure look like a server-state problem.GET /user/parsed as a dict when privacyIDEA returns a list — anAttributeErroroutside theexceptclause, so the run died as a traceback rather than a receipt.- Resolver writes omitting
TIMEOUT,CACHE_TIMEOUT,SIZELIMIT— a resolver write replaces the whole object, so each run silently un-repaired a resolver an operator had just fixed by hand. - No way to declare an unavailable predecessor — the prompt could not be left empty, so an operator had to type a placeholder, which also fails the bind and was recorded as a passing denial proof.
The first two meant the script could never have completed a run under any circumstances. It had been treated as ready for an incident for four days.
verify-t06.sh reported success throughout.
The rule
An unexecuted procedure is unfinished work, and must be labelled as such.
A procedure is exercised when it has completed successfully at least once against a real or scratch target, and the operator who ran it is recorded. Until then it is unexercised, whatever its review status.
Every attended procedure carries this in its runbook, in the header:
Exercise status: exercised 2026-08-27 by <operator> | unexercised
An unexercised procedure may still be the approved path — sometimes the incident is the first run. What it may not do is present itself as routine. Naming the status sets the operator's expectations correctly: run it expecting to debug it, budget time accordingly, and do not schedule it as the last step of a window.
Consequences for verification
A verification script that passes against a procedure that cannot execute is not
verifying anything. Where a verify-*.sh exists, it must assert the property the
procedure is supposed to establish — not that its objects exist, and not that its
API calls returned 200.
verify-t06.sh checked that a resolver and realm existed. It did not check that
GET /user/?realm=coulomb&username=<user> returns a user, which is the entire
point of a resolver and would have caught defects 1 and 3 on the day they landed.
That gap is open and tracked in NK-WP-0033-T05.
Consequences for shared helpers
Defect 1 had already been found and fixed in bootstrap-realm.sh's pi_api
helper, in the same directory, with a comment explaining the cause. The reconcile
script was written later and reimplemented the request path without the fix.
Two implementations of the same API call in one directory is one too many. New scripts against an existing API extend the existing helper or import it; they do not start a second one.