Maturity from run history, pack revision digests, attribute sets, projects and templates (RBT-WP-0008)

Also fixes run lists and scores mixing runs of different packs in one repository.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: sonnet
Assistant-Process: 295952@bnt-lap001
Assistant-Session: e93f64ad-516c-46eb-9666-aad8d300c477

docs/maturity-and-projects.md

@@ -1,6 +1,7 @@
 # Runbook maturity, projects and idempotency
 
-Status: **proposal for decision** (RBT-WP-0008-T01). Raised by Bernd on 2026-09-29.
+Status: **decided 2026-09-29** (Bernd: "go with your suggestions and keep project"), with the attribute-set and template
+extension below. Tracked as RBT-WP-0008.
 
 ## The problem
 
@@ -94,13 +95,84 @@ A **rehearsal** project is where maturity evidence comes from. "Exercised" then
 - Project files are declarative and non-secret. Candidate location: the repository that owns the system
   (for example `net-kingdom/projects/<name>/project.yaml`), next to `runbooks/`.
 
-## Open questions
+## Decisions (2026-09-29, Bernd)
 
-1. **Names.** *Project* is Bernd's word. Alternatives: *target*, *engagement*, *installation*, *rollout*. Is
-   *environment class* (`rehearsal|staging|production`) the right way to separate rehearsal from real?
-2. **Maturity thresholds.** Is "two environments, two operators" the right bar for `proven`? Should `reviewed`
-   be part of the ladder?
-3. **Where projects live**, and whether one runbook may belong to several projects (yes, one project per
-   environment, so a runbook applies to many).
-4. **Retry policy** for non-idempotent steps, as described above.
-5. **Shared run history**: committing a redacted run summary versus keeping records local.
+1. **Name: project.** Environment class `rehearsal | staging | production`.
+2. **Maturity:** `draft`, `reviewed`, `rehearsed`, `proven`, `stale`; `proven` = reviewed, rehearsed, clean in at least two
+   environments by at least two operators. Evidence flags are independent; the level is the highest one achieved.
+3. **Project files live in the repository that owns the system**, next to `runbooks/`.
+4. **Retry of a failed non-idempotent step requires the operator to confirm the earlier attempt did not partly apply.**
+
+## Extension: attribute sets, targets and project templates
+
+A project may have a **goal**, several **targets** (installations, tenants, hosts) and shared facts. All of these are
+**attribute sets**: named collections of non-secret attributes that runbooks and other attribute sets can reuse. This is what
+makes runbooks and projects composable, and it is the basis for reusable **project templates**.
+
+### Attribute sets
+
+```yaml
+spec: attribute-set/v0.1
+id: railiance01-host
+title: The railiance01 host
+extends: [base-cluster]                # composition: later sets override earlier ones
+attributes:
+  target_host: railiance01             # a value
+  tenant: {requirement: required}      # needed, no value yet
+  motto: {requirement: optional}
+```
+
+Every attribute has:
+
+- a **requirement**, declared: `required` (needed) or `optional`;
+- a **state**, derived, using the same words as run datapoints: `nil` (no value), `set`, `err` (fails a consuming runbook's
+  datapoint definition), `ok` (confirmed by a run's check), plus a **modified** flag when a later layer overrides a value that an
+  earlier layer provided;
+- a **source**: which layer gave the value (set, project, target or member). Provenance is what makes a wrong value traceable.
+
+Values are never secret (`secret` is not allowed; secret markers are refused). Only references to secrets are allowed.
+
+### Layers, in order (later wins)
+
+attribute sets listed by the project (each after its own `extends`) then the project's own `attributes`, then the target's
+sets and attributes, then the member's own `attributes`. The resolved attributes become the defaults of the member runbook's
+datapoints, so a value shared by several runbooks is entered once.
+
+### Projects and targets
+
+```yaml
+spec: project/v0.1
+id: railiance01-identity
+title: Identity on railiance01
+goal: Lightweight identity is reachable and its custody is documented.
+environment: production                # rehearsal | staging | production
+template: identity-stack               # set when created from a template
+attribute_sets: [railiance01-host]
+attributes: {tenant: coulomb}
+targets:
+  coulomb: {attribute_sets: [], attributes: {tenant: coulomb}}
+runbooks:
+  - {pack: nk.ssh-certificates, target: coulomb}
+  - {pack: nk.openbao-operating-path, after: [nk.ssh-certificates]}
+```
+
+Project state is derived from each member's latest run at the current pack revision (see the table above, extended by
+member status `not-applied | applying | applied | stale | failed`).
+
+### Templates
+
+A **project template** (`spec: project-template/v0.1`) is a project skeleton: goal, members, attribute sets and attributes that
+still need values. Instantiating a template creates a project file, records which template it came from, and shows which required
+attributes are still needed. Templates make a set of runbooks reusable as a product.
+
+### Modes
+
+Any project run can use any mode: **manual** (operator confirms everything), **auto-confirm** (the engine confirms when
+preconditions hold), and later **auto-pilot** (the engine also executes steps; RBT-WP-0006, opt-in and audited). Because the
+project supplies the attributes, an auto-pilot run needs no operator input except what the template marks as `required` and unset.
+
+## Still open
+
+1. Sharing run history between machines (committing a redacted run summary versus keeping records local).
+2. Whether attribute sets may declare validation of their own (types), independent of the runbooks that consume them.
+3. The exact shape of member ordering (`after`) once runbooks can also depend on runbooks (`requires_packs`, RBT-WP-0006).

docs/run-spec.md

@@ -105,3 +105,13 @@ Rules the engine enforces:
 - **Stalled**: an unfinished run is counted as `stalled` in the score, and as `abandoned` after a week.
 - **Modes** are `manual` and `auto-confirm`. Auto-confirm applies confirmation only; running the step's own
   command is a separate, later level ("auto-pilot with execution", RBT-WP-0006).
+
+## Context and revision
+
+Every run (v0.1 and v0.2) records `pack_digest`, the revision of the pack directory it used (a hash of its files,
+excluding review records, local state and `exercise_status:` lines), and `context`: the optional non-secret
+`project`, `target` and `environment` class (`rehearsal | staging | production`) it was applied to. Maturity and project
+state are derived from these (`docs/maturity-and-projects.md`). Run records of all packs in a repository share one
+folder; a pack's runs are those whose `pack_id` matches.
+
+Run timestamps have microsecond precision, so runs and feedback made within the same second still sort correctly.