From c99b8da41f2bd770ccdebef4eb2fc4a833708580 Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 28 Aug 2026 10:33:48 +0200 Subject: [PATCH] docs: name where operator credentials live, and label unexercised procedures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three gaps from NK-WP-0033, none of them the fault we set out to fix. Operator custody (platform-root-custody.md). This document named a password-safe entry that had never been created, so the 2026-08-23 rotation had nowhere to put the new LLDAP admin value and the outgoing one survived only in a browser entry that was overwritten on 2026-08-27. Decided: operators// in OpenBao for everything reachable once logged in, plus a short closed offline list for what you need when OpenBao is not — platform-root's own password, unseal shares, break-glass. Anything on that list in OpenBao would be circular; anything not on it does not belong in the packet. The rule that closes it: every rotation names, as a numbered step, where the outgoing and incoming values go. NK-WP-0033-T04 rotated four classes with no such step, and that is the root cause of the session that followed — not the four software defects it also surfaced. Secret disclosure by trust stage, tied to the existing S0-S6 table. A fingerprint may be spoken to confirm two parties hold the same credential; the whole value never may. Used ad hoc throughout 2026-08-27 and worth writing down. Attended procedures (new standard). reconcile-lldap-resolver-live.sh was approved, documented, revision-pinned and had never run; its first execution found four defects, two of which meant it could never have completed. verify-t06.sh passed throughout. An unexecuted procedure is unfinished work and must say so in its runbook header. Co-Authored-By: Claude Opus 5 Assistant: claude-code Assistant-Model: opus Assistant-Process: 3377672@bnt-lap001 Assistant-Session: 15463ccf-238f-4e13-b163-93aa25c6d166 --- docs/attended-procedure-standard.md | 71 ++++++++++++++ ...eycape-exposure-resolver-reconciliation.md | 1 + docs/platform-root-custody.md | 92 ++++++++++++++++++- 3 files changed, 160 insertions(+), 4 deletions(-) create mode 100644 docs/attended-procedure-standard.md diff --git a/docs/attended-procedure-standard.md b/docs/attended-procedure-standard.md new file mode 100644 index 0000000..f7e90f4 --- /dev/null +++ b/docs/attended-procedure-standard.md @@ -0,0 +1,71 @@ +# Attended procedures: a script that has never run is not a procedure + +*Adopted 2026-08-28, after `NK-WP-0033`.* + +## What happened + +`sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh` was written, reviewed, +documented in an approved runbook, pinned by revision, and referenced by an +incident workplan as *the* supported one-command attended operation. Its own +workplan entry said plainly: "It has not been run." + +The first execution, on 2026-08-27, found four defects: + +1. `Content-Type: application/json` on bodyless GETs — Werkzeug rejects those in + front of the application, so **every** GET the script made returned an HTML + 400. POSTs worked, which made the failure look like a server-state problem. +2. `GET /user/` parsed as a dict when privacyIDEA returns a list — an + `AttributeError` outside the `except` clause, so the run died as a traceback + rather than a receipt. +3. Resolver writes omitting `TIMEOUT`, `CACHE_TIMEOUT`, `SIZELIMIT` — a resolver + write replaces the whole object, so each run silently un-repaired a resolver + an operator had just fixed by hand. +4. No way to declare an unavailable predecessor — the prompt could not be left + empty, so an operator had to type a placeholder, which also fails the bind + and was recorded as a **passing** denial proof. + +The first two meant the script could never have completed a run under any +circumstances. It had been treated as ready for an incident for four days. + +`verify-t06.sh` reported success throughout. + +## The rule + +**An unexecuted procedure is unfinished work, and must be labelled as such.** + +A procedure is *exercised* when it has completed successfully at least once +against a real or scratch target, and the operator who ran it is recorded. Until +then it is *unexercised*, whatever its review status. + +Every attended procedure carries this in its runbook, in the header: + +```markdown +Exercise status: exercised 2026-08-27 by | unexercised +``` + +An unexercised procedure may still be the approved path — sometimes the incident +*is* the first run. What it may not do is present itself as routine. Naming the +status sets the operator's expectations correctly: run it expecting to debug it, +budget time accordingly, and do not schedule it as the last step of a window. + +## Consequences for verification + +A verification script that passes against a procedure that cannot execute is not +verifying anything. Where a `verify-*.sh` exists, it must assert the property the +procedure is supposed to establish — not that its objects exist, and not that its +API calls returned 200. + +`verify-t06.sh` checked that a resolver and realm existed. It did not check that +`GET /user/?realm=coulomb&username=` returns a user, which is the entire +point of a resolver and would have caught defects 1 and 3 on the day they landed. +That gap is open and tracked in `NK-WP-0033-T05`. + +## Consequences for shared helpers + +Defect 1 had already been found and fixed in `bootstrap-realm.sh`'s `pi_api` +helper, in the same directory, with a comment explaining the cause. The reconcile +script was written later and reimplemented the request path without the fix. + +Two implementations of the same API call in one directory is one too many. New +scripts against an existing API extend the existing helper or import it; they do +not start a second one. diff --git a/docs/keycape-exposure-resolver-reconciliation.md b/docs/keycape-exposure-resolver-reconciliation.md index 136f7ec..e816e31 100644 --- a/docs/keycape-exposure-resolver-reconciliation.md +++ b/docs/keycape-exposure-resolver-reconciliation.md @@ -5,6 +5,7 @@ Workplan: `NK-WP-0033` NetKingdom procedure: `sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh` (repaired 2026-08-27 in `4a38511`; the previously pinned revision could not complete a run — see NK-WP-0033 T05) Platform recovery contract: railiance-platform `453fed3` Owner cutover receipt: State Hub message `45b236c8-052f-43d3-a472-44f8e9694da2` +Exercise status: exercised 2026-08-27 by Bernd Worsch — first run ever; four defects found and fixed (`4a38511`). See `attended-procedure-standard.md`. This is the remaining attended provider-admin operation after the owner-reported four-class cutover. It updates only privacyIDEA resolver `lldap-coulomb` so the diff --git a/docs/platform-root-custody.md b/docs/platform-root-custody.md index 541b451..6f3fa15 100644 --- a/docs/platform-root-custody.md +++ b/docs/platform-root-custody.md @@ -99,12 +99,15 @@ LLDAP deliberately has no public registration flow. The first-user process is administrator-provisioned: 1. Log in to `https://lldap.coulomb.social` as `admin`. -2. Retrieve `LLDAP_LDAP_USER_PASS` from the operator password safe entry - `net-kingdom/LLDAP/admin`. +2. Retrieve `LLDAP_LDAP_USER_PASS` from operator custody — + `operators/lldap/admin` in OpenBao (see *Operator Credential Custody*). + Until 2026-08-28 this step named a password-safe entry that did not exist; + see that section for what that cost. 3. Create the dedicated `platform-root` or `king` user. 4. Add the user to `net-kingdom-admins` for the current lightweight path. -5. Store the new account password only in the password safe or offline custody - packet. +5. Store the new account password in operator custody (`operators/`) or, for a + bootstrap identity, in the offline packet — never only in a browser + password manager. 6. Use `pi-admin` in privacyIDEA to confirm that the LLDAP resolver can see the new user and that self-enrollment is allowed. 7. Log in to privacyIDEA self-service as `platform-root` and enroll the TOTP @@ -124,6 +127,65 @@ custody unless the recorded king credential was intentionally changed to `platform-admin`. `platform-admin` is the later non-root operator path; it is useful, but it is not the same as proving the platform-root custody identity. +## Operator Credential Custody + +*Decided 2026-08-28, after `NK-WP-0033`.* + +Every credential a human operator holds has exactly one recorded home. Before +this was written, this document named a password-safe entry +(`net-kingdom/LLDAP/admin`) that had never been created. The consequence was not +theoretical: the LLDAP admin credential was rotated on 2026-08-23 with nowhere +to put the new value, the outgoing value survived only in a browser password +manager, and it was destroyed on 2026-08-27 when that entry was overwritten with +the replacement. A full working session went into diagnosing a system whose +predecessor credential no longer existed anywhere. + +### Two stores, and the line between them + +**`operators/` in OpenBao — everything reachable once you are logged in.** +Service account passwords, provider PATs, the LLDAP admin bind, privacyIDEA +`pi-admin`, machine identities like `state-hub-svc`. Auditable, backed up with +the rest of the platform, reachable from any machine, and writable through +`warden desk --act paste_once_provision` so the value never reaches a terminal +or shell history. + +Path convention: `operators//` — for example +`operators/lldap/admin`, `operators/privacyidea/pi-admin`, +`operators/forgejo/state-hub-svc`. Distinct from `platform/workloads/…`, which +holds credentials that *workloads* consume; `operators/` holds credentials that +*people* use. + +**The offline packet — what you need when OpenBao is unreachable.** This list is +deliberately short, closed, and enumerated here rather than left to judgement: + +- `platform-root`'s own password and TOTP recovery codes +- OpenBao unseal shares and root token +- the `break-glass` account password (`sso-mfa/k8s/lldap/break-glass.sh`) + +Anything on that list in `operators/` would be a circular dependency: you would +need the store to reach the credential that opens the store. Anything *not* on +that list belongs in `operators/`, not in the packet — the packet is not a +general-purpose safe and does not scale. + +### The rule that closes the gap + +**Every rotation procedure must name, as a numbered step, where the outgoing and +incoming values go.** A rotation that changes a credential without saying what +happens to custody will lose one of the two, and which one it loses is a matter +of luck. `NK-WP-0033-T04` rotated four credential classes and had no such step; +that is the root cause of the session that followed, not any of the four +software defects it also surfaced. + +The outgoing value is retained until its successor is verified *and* the +predecessor is proven rejected. Only then is it destroyed, and its destruction +is recorded. + +### Not yet built + +The `operators/` mount and its policy do not exist yet. Until they do, this +section describes intent rather than practice, and the same failure remains +available. Building it is a `railiance-platform` act, not an ops-mason one. + ## Trust Progression The platform moves through explicit trust stages: @@ -141,6 +203,28 @@ The platform moves through explicit trust stages: This prevents early bootstrap convenience from silently becoming permanent platform sovereignty. +### Secret disclosure by stage + +*Decided 2026-08-27, during `NK-WP-0033`.* Diagnosis stalls when two people +cannot establish whether they are discussing the same credential. What follows +is what may be said aloud — in a terminal, a transcript, a chat message, or a +ticket — about a secret's *value*, by stage. + +| Stage | Permitted about a value | +| --- | --- | +| S0–S4 | A fingerprint: length, last three characters, or `sha256 \| head -c 8`. Enough to confirm two parties hold the same credential and no more. | +| S5+ | Fingerprint by hash only (`sha256 \| head -c 8`). No plaintext characters, on the grounds that a live platform's credentials outlive the conversation about them. | +| Any | Never the whole value, never a working prefix, never a screenshot of one. | + +Prefer the hash form at every stage: it disambiguates just as well, it is what +this document's own evidence rules already ask for elsewhere ("public-key +fingerprints"), and it does not become a liability when a transcript is read +back later by someone who was not in the room. + +This is a disclosure rule, not a handling rule. It says what may be *spoken* +about a value; it does not license reading a value out of a store that a role is +not otherwise entitled to read. + ## Required Bootstrap Use Cases The guided bootstrap experience must cover at least: