docs: name where operator credentials live, and label unexercised procedures
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

Three gaps from NK-WP-0033, none of them the fault we set out to fix.

Operator custody (platform-root-custody.md). This document named a
password-safe entry that had never been created, so the 2026-08-23
rotation had nowhere to put the new LLDAP admin value and the outgoing
one survived only in a browser entry that was overwritten on 2026-08-27.
Decided: operators/<subsystem>/<account> in OpenBao for everything
reachable once logged in, plus a short closed offline list for what you
need when OpenBao is not — platform-root's own password, unseal shares,
break-glass. Anything on that list in OpenBao would be circular;
anything not on it does not belong in the packet.

The rule that closes it: every rotation names, as a numbered step, where
the outgoing and incoming values go. NK-WP-0033-T04 rotated four classes
with no such step, and that is the root cause of the session that
followed — not the four software defects it also surfaced.

Secret disclosure by trust stage, tied to the existing S0-S6 table. A
fingerprint may be spoken to confirm two parties hold the same
credential; the whole value never may. Used ad hoc throughout 2026-08-27
and worth writing down.

Attended procedures (new standard). reconcile-lldap-resolver-live.sh was
approved, documented, revision-pinned and had never run; its first
execution found four defects, two of which meant it could never have
completed. verify-t06.sh passed throughout. An unexecuted procedure is
unfinished work and must say so in its runbook header.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 3377672@bnt-lap001
Assistant-Session: 15463ccf-238f-4e13-b163-93aa25c6d166
This commit is contained in:
tegwick 2026-08-28 10:33:48 +02:00
parent fd86397f52
commit c99b8da41f
3 changed files with 160 additions and 4 deletions

View file

@ -0,0 +1,71 @@
# Attended procedures: a script that has never run is not a procedure
*Adopted 2026-08-28, after `NK-WP-0033`.*
## What happened
`sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh` was written, reviewed,
documented in an approved runbook, pinned by revision, and referenced by an
incident workplan as *the* supported one-command attended operation. Its own
workplan entry said plainly: "It has not been run."
The first execution, on 2026-08-27, found four defects:
1. `Content-Type: application/json` on bodyless GETs — Werkzeug rejects those in
front of the application, so **every** GET the script made returned an HTML
400. POSTs worked, which made the failure look like a server-state problem.
2. `GET /user/` parsed as a dict when privacyIDEA returns a list — an
`AttributeError` outside the `except` clause, so the run died as a traceback
rather than a receipt.
3. Resolver writes omitting `TIMEOUT`, `CACHE_TIMEOUT`, `SIZELIMIT` — a resolver
write replaces the whole object, so each run silently un-repaired a resolver
an operator had just fixed by hand.
4. No way to declare an unavailable predecessor — the prompt could not be left
empty, so an operator had to type a placeholder, which also fails the bind
and was recorded as a **passing** denial proof.
The first two meant the script could never have completed a run under any
circumstances. It had been treated as ready for an incident for four days.
`verify-t06.sh` reported success throughout.
## The rule
**An unexecuted procedure is unfinished work, and must be labelled as such.**
A procedure is *exercised* when it has completed successfully at least once
against a real or scratch target, and the operator who ran it is recorded. Until
then it is *unexercised*, whatever its review status.
Every attended procedure carries this in its runbook, in the header:
```markdown
Exercise status: exercised 2026-08-27 by <operator> | unexercised
```
An unexercised procedure may still be the approved path — sometimes the incident
*is* the first run. What it may not do is present itself as routine. Naming the
status sets the operator's expectations correctly: run it expecting to debug it,
budget time accordingly, and do not schedule it as the last step of a window.
## Consequences for verification
A verification script that passes against a procedure that cannot execute is not
verifying anything. Where a `verify-*.sh` exists, it must assert the property the
procedure is supposed to establish — not that its objects exist, and not that its
API calls returned 200.
`verify-t06.sh` checked that a resolver and realm existed. It did not check that
`GET /user/?realm=coulomb&username=<user>` returns a user, which is the entire
point of a resolver and would have caught defects 1 and 3 on the day they landed.
That gap is open and tracked in `NK-WP-0033-T05`.
## Consequences for shared helpers
Defect 1 had already been found and fixed in `bootstrap-realm.sh`'s `pi_api`
helper, in the same directory, with a comment explaining the cause. The reconcile
script was written later and reimplemented the request path without the fix.
Two implementations of the same API call in one directory is one too many. New
scripts against an existing API extend the existing helper or import it; they do
not start a second one.

View file

@ -5,6 +5,7 @@ Workplan: `NK-WP-0033`
NetKingdom procedure: `sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh` (repaired 2026-08-27 in `4a38511`; the previously pinned revision could not complete a run — see NK-WP-0033 T05)
Platform recovery contract: railiance-platform `453fed3`
Owner cutover receipt: State Hub message `45b236c8-052f-43d3-a472-44f8e9694da2`
Exercise status: exercised 2026-08-27 by Bernd Worsch — first run ever; four defects found and fixed (`4a38511`). See `attended-procedure-standard.md`.
This is the remaining attended provider-admin operation after the owner-reported
four-class cutover. It updates only privacyIDEA resolver `lldap-coulomb` so the

View file

@ -99,12 +99,15 @@ LLDAP deliberately has no public registration flow. The first-user process is
administrator-provisioned:
1. Log in to `https://lldap.coulomb.social` as `admin`.
2. Retrieve `LLDAP_LDAP_USER_PASS` from the operator password safe entry
`net-kingdom/LLDAP/admin`.
2. Retrieve `LLDAP_LDAP_USER_PASS` from operator custody —
`operators/lldap/admin` in OpenBao (see *Operator Credential Custody*).
Until 2026-08-28 this step named a password-safe entry that did not exist;
see that section for what that cost.
3. Create the dedicated `platform-root` or `king` user.
4. Add the user to `net-kingdom-admins` for the current lightweight path.
5. Store the new account password only in the password safe or offline custody
packet.
5. Store the new account password in operator custody (`operators/`) or, for a
bootstrap identity, in the offline packet — never only in a browser
password manager.
6. Use `pi-admin` in privacyIDEA to confirm that the LLDAP resolver can see
the new user and that self-enrollment is allowed.
7. Log in to privacyIDEA self-service as `platform-root` and enroll the TOTP
@ -124,6 +127,65 @@ custody unless the recorded king credential was intentionally changed to
`platform-admin`. `platform-admin` is the later non-root operator path; it is
useful, but it is not the same as proving the platform-root custody identity.
## Operator Credential Custody
*Decided 2026-08-28, after `NK-WP-0033`.*
Every credential a human operator holds has exactly one recorded home. Before
this was written, this document named a password-safe entry
(`net-kingdom/LLDAP/admin`) that had never been created. The consequence was not
theoretical: the LLDAP admin credential was rotated on 2026-08-23 with nowhere
to put the new value, the outgoing value survived only in a browser password
manager, and it was destroyed on 2026-08-27 when that entry was overwritten with
the replacement. A full working session went into diagnosing a system whose
predecessor credential no longer existed anywhere.
### Two stores, and the line between them
**`operators/` in OpenBao — everything reachable once you are logged in.**
Service account passwords, provider PATs, the LLDAP admin bind, privacyIDEA
`pi-admin`, machine identities like `state-hub-svc`. Auditable, backed up with
the rest of the platform, reachable from any machine, and writable through
`warden desk --act paste_once_provision` so the value never reaches a terminal
or shell history.
Path convention: `operators/<subsystem>/<account>` — for example
`operators/lldap/admin`, `operators/privacyidea/pi-admin`,
`operators/forgejo/state-hub-svc`. Distinct from `platform/workloads/…`, which
holds credentials that *workloads* consume; `operators/` holds credentials that
*people* use.
**The offline packet — what you need when OpenBao is unreachable.** This list is
deliberately short, closed, and enumerated here rather than left to judgement:
- `platform-root`'s own password and TOTP recovery codes
- OpenBao unseal shares and root token
- the `break-glass` account password (`sso-mfa/k8s/lldap/break-glass.sh`)
Anything on that list in `operators/` would be a circular dependency: you would
need the store to reach the credential that opens the store. Anything *not* on
that list belongs in `operators/`, not in the packet — the packet is not a
general-purpose safe and does not scale.
### The rule that closes the gap
**Every rotation procedure must name, as a numbered step, where the outgoing and
incoming values go.** A rotation that changes a credential without saying what
happens to custody will lose one of the two, and which one it loses is a matter
of luck. `NK-WP-0033-T04` rotated four credential classes and had no such step;
that is the root cause of the session that followed, not any of the four
software defects it also surfaced.
The outgoing value is retained until its successor is verified *and* the
predecessor is proven rejected. Only then is it destroyed, and its destruction
is recorded.
### Not yet built
The `operators/` mount and its policy do not exist yet. Until they do, this
section describes intent rather than practice, and the same failure remains
available. Building it is a `railiance-platform` act, not an ops-mason one.
## Trust Progression
The platform moves through explicit trust stages:
@ -141,6 +203,28 @@ The platform moves through explicit trust stages:
This prevents early bootstrap convenience from silently becoming permanent
platform sovereignty.
### Secret disclosure by stage
*Decided 2026-08-27, during `NK-WP-0033`.* Diagnosis stalls when two people
cannot establish whether they are discussing the same credential. What follows
is what may be said aloud — in a terminal, a transcript, a chat message, or a
ticket — about a secret's *value*, by stage.
| Stage | Permitted about a value |
| --- | --- |
| S0S4 | A fingerprint: length, last three characters, or `sha256 \| head -c 8`. Enough to confirm two parties hold the same credential and no more. |
| S5+ | Fingerprint by hash only (`sha256 \| head -c 8`). No plaintext characters, on the grounds that a live platform's credentials outlive the conversation about them. |
| Any | Never the whole value, never a working prefix, never a screenshot of one. |
Prefer the hash form at every stage: it disambiguates just as well, it is what
this document's own evidence rules already ask for elsewhere ("public-key
fingerprints"), and it does not become a liability when a transcript is read
back later by someone who was not in the room.
This is a disclosure rule, not a handling rule. It says what may be *spoken*
about a value; it does not license reading a value out of a store that a role is
not otherwise entitled to read.
## Required Bootstrap Use Cases
The guided bootstrap experience must cover at least: