Rename open to stalled and autopilot to auto-confirm; add pushed state; declare idempotency; propose maturity and projects model (RBT-WP-0008)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: sonnet
Assistant-Process: 295952@bnt-lap001
Assistant-Session: e93f64ad-516c-46eb-9666-aad8d300c477

docs/maturity-and-projects.md

@@ -0,0 +1,106 @@
+# Runbook maturity, projects and idempotency
+
+Status: **proposal for decision** (RBT-WP-0008-T01). Raised by Bernd on 2026-09-29.
+
+## The problem
+
+`exercise_status: unexercised | exercised <date> by <operator>` mixes three different things:
+
+1. **What the runbook is like** (does it change state, can it be repeated, can it be undone).
+2. **How much the runbook can be trusted** (has it been rehearsed, and where).
+3. **What state an environment is in** (has this runbook been applied to *this* system).
+
+A run that succeeded on one system says little about another system, so "exercised" is not a
+property of the runbook alone. Environments differ, and applying a runbook to an environment is a
+state of that environment, not of the runbook.
+
+## Terminology survey
+
+From general DevOps, SRE, configuration-management and ITIL practice, written from memory, not from
+a source review. Please treat the names as candidates.
+
+| Area | Terms in common use | Relevance here |
+| --- | --- | --- |
+| Automation maturity | manual, semi-automated (guided), automated; the "runbook automation ladder"; toil reduction (SRE) | our modes: manual, auto-confirm, later auto-pilot with execution |
+| Rehearsal | game day, fire drill, disaster-recovery drill, dry run, rehearsal in staging, `--check` mode (Ansible), `plan` before `apply` (Terraform) | how a runbook earns trust without touching production |
+| Readiness | production readiness review (PRR), operational readiness, definition of done | a maturity gate before a runbook is used on production |
+| Repeatability | idempotent, convergent (declare desired state), re-runnable, resumable, reversible; drift; "converged" | idempotency is a *declared characteristic*, not a state |
+| Change control (ITIL) | standard change (pre-approved because proven repeatable), normal change, change model | maps to "proven" runbooks that need no fresh review |
+| Environments | environment / stage (dev, test, staging, production), target, inventory and group variables (Ansible), workspace (Terraform), context (kubectl), installation, instance | the thing runbooks are applied to |
+| Decay | runbook rot, stale, drift between docs and system | maturity must expire when the runbook or the system changes |
+
+## Proposal: three separate axes
+
+### 1. Characteristics: declared by the author, static
+
+- **Idempotency**: `idempotent | non-idempotent | conditional` (with a note). Declared per step; the
+  runbook's value is derived (all idempotent, or any non-idempotent, or unknown if a step is undeclared).
+  **Implemented 2026-09-29** in the v0.2 schema and validator. Values are words, not yes/no, because
+  YAML reads unquoted `yes`/`no` as booleans.
+- **Reversibility**: already declared per step (`reversible`), with rollback text.
+- **Effect class** (candidate): read-only, changes-state, destructive.
+- Consequences to build: retry of a failed non-idempotent step should require the operator to confirm
+  that the earlier attempt did not partly apply; re-running a whole idempotent runbook against a
+  converged environment is safe and is the way to verify it.
+
+### 2. Maturity: earned, independent of any environment, bound to a pack revision
+
+Derived from run history, not typed in. A run records a digest of the pack revision it used.
+
+| Level | Meaning (candidate names) |
+| --- | --- |
+| `draft` | valid against the spec, never run |
+| `reviewed` | a second person has read it (recorded) |
+| `rehearsed` | completed cleanly (no deviation) in a **rehearsal** environment |
+| `proven` | completed cleanly in at least two distinct environments by at least two operators |
+| `stale` | the pack changed since it was proven: maturity is bound to a revision and expires when it changes |
+
+This replaces `exercise_status`. Deviations (overrides, waivers, push-through) never count towards
+maturity, which matches the current rule for exercising.
+
+### 3. Projects: environments the runbooks are applied to
+
+A **project** groups runbooks that share one environment. It holds what they have in common and provides
+it as datapoint defaults (like Ansible group variables), so a value such as the target host or tenant is
+entered once.
+
+```yaml
+project: railiance01-identity
+environment: production          # rehearsal | staging | production (class, not free text)
+datapoints: {tenant: coulomb, target_host: railiance01}     # non-secret; never secrets
+runbooks:
+  - {pack: nk.ssh-certificates}
+  - {pack: nk.openbao-operating-path, after: [nk.ssh-certificates]}
+```
+
+Project state is derived from the latest run of each member at the current pack revision:
+
+| State (candidate) | Meaning |
+| --- | --- |
+| `planned` | no member applied yet |
+| `applying` | a run is in progress, or some members are applied and others not (the "running" state) |
+| `applied` | every member's latest run succeeded cleanly at the current revision |
+| `stale` | a member's pack changed, or a run failed or was abandoned, since it was applied |
+| `drifted` (later) | verification against the live system no longer matches |
+
+A **rehearsal** project is where maturity evidence comes from. "Exercised" then has a precise meaning:
+*applied successfully in a rehearsal project*. A production project is `applied`, not "exercised".
+
+## Consequences for what exists
+
+- Runs record `project`, the environment class and the pack digest. Scores become per project as well as per
+  runbook. Run records stay non-secret and local (`.local/`), so shared history remains an open question.
+- `exercise_status` in `pack.yaml` is superseded. Until this is decided it stays as the interim marker.
+- Project files are declarative and non-secret. Candidate location: the repository that owns the system
+  (for example `net-kingdom/projects/<name>/project.yaml`), next to `runbooks/`.
+
+## Open questions
+
+1. **Names.** *Project* is Bernd's word. Alternatives: *target*, *engagement*, *installation*, *rollout*. Is
+   *environment class* (`rehearsal|staging|production`) the right way to separate rehearsal from real?
+2. **Maturity thresholds.** Is "two environments, two operators" the right bar for `proven`? Should `reviewed`
+   be part of the ladder?
+3. **Where projects live**, and whether one runbook may belong to several projects (yes, one project per
+   environment, so a runbook applies to many).
+4. **Retry policy** for non-idempotent steps, as described above.
+5. **Shared run history**: committing a redacted run summary versus keeping records local.