tegwick
c99b8da41f
Three gaps from NK-WP-0033, none of them the fault we set out to fix. Operator custody (platform-root-custody.md). This document named a password-safe entry that had never been created, so the 2026-08-23 rotation had nowhere to put the new LLDAP admin value and the outgoing one survived only in a browser entry that was overwritten on 2026-08-27. Decided: operators/<subsystem>/<account> in OpenBao for everything reachable once logged in, plus a short closed offline list for what you need when OpenBao is not — platform-root's own password, unseal shares, break-glass. Anything on that list in OpenBao would be circular; anything not on it does not belong in the packet. The rule that closes it: every rotation names, as a numbered step, where the outgoing and incoming values go. NK-WP-0033-T04 rotated four classes with no such step, and that is the root cause of the session that followed — not the four software defects it also surfaced. Secret disclosure by trust stage, tied to the existing S0-S6 table. A fingerprint may be spoken to confirm two parties hold the same credential; the whole value never may. Used ad hoc throughout 2026-08-27 and worth writing down. Attended procedures (new standard). reconcile-lldap-resolver-live.sh was approved, documented, revision-pinned and had never run; its first execution found four defects, two of which meant it could never have completed. verify-t06.sh passed throughout. An unexecuted procedure is unfinished work and must say so in its runbook header. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3377672@bnt-lap001 Assistant-Session: 15463ccf-238f-4e13-b163-93aa25c6d166
71 lines
3.3 KiB
Markdown
71 lines
3.3 KiB
Markdown
# Attended procedures: a script that has never run is not a procedure
|
|
|
|
*Adopted 2026-08-28, after `NK-WP-0033`.*
|
|
|
|
## What happened
|
|
|
|
`sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh` was written, reviewed,
|
|
documented in an approved runbook, pinned by revision, and referenced by an
|
|
incident workplan as *the* supported one-command attended operation. Its own
|
|
workplan entry said plainly: "It has not been run."
|
|
|
|
The first execution, on 2026-08-27, found four defects:
|
|
|
|
1. `Content-Type: application/json` on bodyless GETs — Werkzeug rejects those in
|
|
front of the application, so **every** GET the script made returned an HTML
|
|
400. POSTs worked, which made the failure look like a server-state problem.
|
|
2. `GET /user/` parsed as a dict when privacyIDEA returns a list — an
|
|
`AttributeError` outside the `except` clause, so the run died as a traceback
|
|
rather than a receipt.
|
|
3. Resolver writes omitting `TIMEOUT`, `CACHE_TIMEOUT`, `SIZELIMIT` — a resolver
|
|
write replaces the whole object, so each run silently un-repaired a resolver
|
|
an operator had just fixed by hand.
|
|
4. No way to declare an unavailable predecessor — the prompt could not be left
|
|
empty, so an operator had to type a placeholder, which also fails the bind
|
|
and was recorded as a **passing** denial proof.
|
|
|
|
The first two meant the script could never have completed a run under any
|
|
circumstances. It had been treated as ready for an incident for four days.
|
|
|
|
`verify-t06.sh` reported success throughout.
|
|
|
|
## The rule
|
|
|
|
**An unexecuted procedure is unfinished work, and must be labelled as such.**
|
|
|
|
A procedure is *exercised* when it has completed successfully at least once
|
|
against a real or scratch target, and the operator who ran it is recorded. Until
|
|
then it is *unexercised*, whatever its review status.
|
|
|
|
Every attended procedure carries this in its runbook, in the header:
|
|
|
|
```markdown
|
|
Exercise status: exercised 2026-08-27 by <operator> | unexercised
|
|
```
|
|
|
|
An unexercised procedure may still be the approved path — sometimes the incident
|
|
*is* the first run. What it may not do is present itself as routine. Naming the
|
|
status sets the operator's expectations correctly: run it expecting to debug it,
|
|
budget time accordingly, and do not schedule it as the last step of a window.
|
|
|
|
## Consequences for verification
|
|
|
|
A verification script that passes against a procedure that cannot execute is not
|
|
verifying anything. Where a `verify-*.sh` exists, it must assert the property the
|
|
procedure is supposed to establish — not that its objects exist, and not that its
|
|
API calls returned 200.
|
|
|
|
`verify-t06.sh` checked that a resolver and realm existed. It did not check that
|
|
`GET /user/?realm=coulomb&username=<user>` returns a user, which is the entire
|
|
point of a resolver and would have caught defects 1 and 3 on the day they landed.
|
|
That gap is open and tracked in `NK-WP-0033-T05`.
|
|
|
|
## Consequences for shared helpers
|
|
|
|
Defect 1 had already been found and fixed in `bootstrap-realm.sh`'s `pi_api`
|
|
helper, in the same directory, with a comment explaining the cause. The reconcile
|
|
script was written later and reimplemented the request path without the fix.
|
|
|
|
Two implementations of the same API call in one directory is one too many. New
|
|
scripts against an existing API extend the existing helper or import it; they do
|
|
not start a second one.
|