runbook-tutorials/docs/run-spec.md

# Run record `runbook-run/v0.1`

One run of one native pack, stored as JSON in
`<owning repo>/.local/runbook-runs/<run_id>.json` (git-ignored). Non-secret only.

| Field | Meaning |
| --- | --- |
| `run_id` | Timestamp plus random suffix |
| `pack_id`, `pack_path` | The pack run |
| `operator` | One word: the person who ran it |
| `params` | Resolved parameter values (validated, never secret) |
| `steps.<id>.state` | `todo`, `running`, `done`, `failed`, `skipped` |
| `steps.<id>.verification` | `kind`, `passed`, and for commands `exit_code`, `output_bytes`, `seconds`. Output text is never stored |
| `steps.<id>.evidence` | The facts the step declared, supplied by the operator |

## Rules

- Steps run in order; a step starts only after its predecessors are done or skipped.
- A step completes only with a passing verification and its declared evidence.
- A verify command runs only when the operator confirms (`--yes`), with a 60 s timeout.
- Only read-only steps may be skipped, and a run with skips can never succeed.
- Values matching the secret markers are refused everywhere they could be recorded.
- The engine never edits `pack.yaml`. How far a pack can be trusted is not a field in it: it is derived from run history
  (`rtut maturity`, `docs/maturity-and-projects.md`).

CLI: `rtut run new|list|status|cmd|start|verify|done|fail|retry|skip|feedback|finalize|capture <pack.yaml> ...`

## Outcome and score

A run is unfinished (**stalled**) until finalized. `rtut run finalize` (or the UI's *Close this run*) records
an `outcome`:

| Outcome | Rule |
| --- | --- |
| `succeeded` | every step is `done` (skipped steps do not count) |
| `failed` | at least one step has failed; otherwise abandon |
| `abandoned` | any time. An unfinished run untouched for 7 days also counts as abandoned in the score |

A finished run is read-only, except for feedback. The **score** of a runbook is derived from
its run records: tried, succeeded, failed, abandoned, stalled (unfinished), plus a tally of feedback ratings
(`rtut score <root>`; shown in the UI). Run records live in `.local/` of the machine that ran
them, so the score is per machine until run histories are shared.

Finalizing never edits the pack. A clean `succeeded` run becomes evidence for the pack's **maturity** at the
revision it ran (`docs/maturity-and-projects.md`); this replaces the earlier `exercise_status` field and the
"Record pack as exercised" step, both retired. Old packs that still carry `exercise_status` load unchanged; the
field is ignored.

## Feedback

`feedback` entries (`rating` good|neutral|poor, `comment`, `at`) are added after a run; history
is kept and the latest counts. `rtut feedback <pack.yaml>` lists them.

## Captures

A step may carry optional `captures`: the command actually run, and the output that came
back, as documentation or for inspection when the engine cannot reach the system itself. Limits:
20 000 characters of output, 2 000 of command. Obvious secret markers are refused; the engine cannot
detect every secret, so the operator stays responsible. For `output-contains` steps the output can
be **pasted for verification** (`source: pasted-for-verification`), which checks the `contains` text
without running anything. Exit-code checks cannot be verified from pasted output.

## Run record `runbook-run/v0.2` (packs of `runbook-pack/v0.2`)

Implemented in `rtut/run2.py` (graph in `rtut/flow.py`). It adds to the v0.1 record:

| Field | Meaning |
| --- | --- |
| `mode` | `manual` or `auto-confirm` |
| `datapoints.<id>` | `state` nil, set, err, ok or pushed; `value`, `how` (initial, manual, produced, check, manual confirmation), `error` |
| `steps.<id>` | as v0.1 (`pushed` is an additional state), plus `alternative` (chosen `one_of` branch), `authorized_by` |
| `gates.<id>` | `state` open, passed, failed, waived or pushed; per-check `results`; `waiver` or `pushed` |
| `deviations[]` | every override, waiver and push-through (see `docs/pack-spec-v0.2.md`) |
| `auto-confirm` | `preauthorized` irreversible steps, `authorized_by`, and `stopped` if a timeout stopped the run |

Rules the engine enforces:

- **Preconditions** of an item are its `requires` (and those of enclosing stages), the `set`/`ok`
  state of the datapoints it `uses`, and, for a gate, completion of the steps of its stage.
- **Recommended next action**: the running item first, else the first ready item in the pack's canonical
  order. `next_actions(run)` returns the recommended item, all ready items (parallel work) and the blocked
  ones with what they wait for.
- **Alternatives**: a `one_of` step is started with an alternative (default: the first whose
  `recommended_when` matches the datapoints).
- **Datapoints**: invalid input leaves the datapoint `err` and is never stored; a step's `produces` become `set`
  when it completes; `confirmed_by` promotes them to `ok` when that step or gate passes.
- **Override** (manual mode only): starting an item with unmet preconditions needs a reason and an acknowledged
  downstream risk; it is recorded as a deviation.
- **Waiver**: only for gates that declare `waivable`, with a reason; recorded as a deviation.
- **Confirmation**: irreversible steps need the exact confirmation phrase at start. In auto-confirm they run only
  if the run pre-authorized them, and never if the step is `auto_confirm: manual-only`.
- **Auto-confirm** (`auto-confirm_tick`): the engine applies confirmation when preconditions are met: it starts the
  recommended item, runs its checks (manual checks are auto-confirmed and marked so) and completes it. It
  never runs a step's own command; the work is done by the operator or, later, by opt-in execution. A step
  with `evidence` waits for the operator. If checks keep failing until the item's `timeout` (or the pack
  default), the run stops, or with `on_timeout: push-through` the item is marked done with
  state `pushed` and a deviation is recorded.
- **Deviations and maturity**: a run with any deviation or pushed-through step can be finalized but never counts as maturity evidence.
  `finalize(succeeded)` also needs every gate passed or waived.
- **Pushed is its own state.** A step, gate or datapoint passed over by push-through is `pushed`: not `done`,
  not `set`, not `err`. Dependents may proceed (that is the point of pushing through), but a pushed datapoint
  has no value, so a command that needs it fails its check instead of running with nothing. A run with pushed
  items can be finished as `failed` (never `succeeded`) and never counts as maturity evidence.
- **Stalled**: an unfinished run is counted as `stalled` in the score, and as `abandoned` after a week.
- **Modes** are `manual` and `auto-confirm`. Auto-confirm applies confirmation only; running the step's own
  command is a separate, later level ("auto-pilot with execution", RBT-WP-0006).

## Closed runs are locked and sealed

Closing a run (`finalize`) makes its record final:

- **Locked.** A closed record cannot be saved with changes: the engine refuses, naming what would have changed. This holds for runs closed
  before sealing existed too.
- **Feedback.** It locks with the run. Give it before closing, or with the closing form: `finalize(..., rating, comment)` records an optional
  rating and comment together with the outcome (validated first, so a refused close changes nothing). This keeps abandoned and failed runs from
  losing what the operator wanted to say.
- **Sealed.** The record carries a SHA-256 `seal` of its content (excluding `seal` and `updated`; a legacy `exercise_recorded` key on old records is ignored), and the file is made
  read-only. `seal_state` reports `open`, `sealed`, `broken` (changed outside the app after closing) or `unsealed` (closed before sealing existed).
  A broken seal is called out in the UI, and such a run no longer counts as maturity evidence. This is tamper-evidence, not prevention: someone
  with file access can still edit a file, but not without being noticed.

## Changing a value after steps used it

A v0.1 run fixes its parameters when it starts. In a v0.2 run datapoints can be edited (guided flow questions, data view), but a value
that steps have already started with is *in use*. Changing it needs the operator's override (a reason and an acknowledged risk), only in
manual mode, and is recorded as a `changed-after-use` deviation naming the steps and the previous value. A save containing any such
change is refused as a whole unless the override is given, so a refused save applies nothing. Values nobody used yet change freely,
which is what the questionnaire is for. A run with any deviation never counts towards maturity.

## Timestamps and durations

`created` is the start and `finished` (set when the run is closed) the end. `elapsed` is start to finish, or to now while the run is
unfinished; `working` is the time steps were in progress; each step has its own `started` and `finished`. The runbook's score carries
the average, median and fastest **elapsed time of succeeded runs only**: unfinished, failed and abandoned runs say nothing about how
long a working run takes. The UI shows this on the run, on each finished step, in the runs list and next to the score.

## Artefacts: open and download

An artefact is a reference to something a run produced. The UI links it when it is a URL, or when it is exactly the artefact's
resolved local file (`GET /run/<i>/<run>/artefact/<id>`, `?download=1` for an attachment). It refuses, with the reason: a file that
does not exist yet, anything that is not a regular file, files over 5 MB, files in sensitive places (`.ssh`, `.gnupg`, `.aws`, `.kube`,
`.git`, key and certificate files, run records), and files whose beginning looks like a secret. Text is served as plain text, never as a
page, with `nosniff` and a sandboxing CSP; other content is only offered as a download. Symlinks are judged by their real target.

## Context and revision

Every run (v0.1 and v0.2) records `pack_digest`, the revision of the pack directory it used (a hash of its files,
excluding review records, local state and `exercise_status:` lines), and `context`: the optional non-secret
`project`, `target` and `environment` class (`rehearsal | staging | production`) it was applied to. Maturity and project
state are derived from these (`docs/maturity-and-projects.md`). Run records of all packs in a repository share one
folder; a pack's runs are those whose `pack_id` matches.

Run timestamps have microsecond precision, so runs and feedback made within the same second still sort correctly.
- **Retry**: a failed step that is not declared `idempotent` (or is undeclared) is retried only with `acknowledge_partial`, the operator's
  confirmation that the earlier attempt did not partly apply; it is recorded in `retries`.
- **Mode switch**: `switch_mode` hands a run over to auto-confirm or takes it over again; each change is recorded in `mode_history`.
- **Views for renderers**: `overview`, `datapoint_rows` (grouped or filtered by dimension), `dimension_keys` and `artefact_rows` give a UI
  everything the flow, data and overview views show, without knowing the record layout.