runbook-tutorials/docs/run-spec.md

Run record runbook-run/v0.1

One run of one native pack, stored as JSON in <owning repo>/.local/runbook-runs/<run_id>.json (git-ignored). Non-secret only.

Field	Meaning
run_id	Timestamp plus random suffix
pack_id, pack_path	The pack run
operator	One word: the person who ran it
params	Resolved parameter values (validated, never secret)
steps.<id>.state	todo, running, done, failed, skipped
steps.<id>.verification	kind, passed, and for commands exit_code, output_bytes, seconds. Output text is never stored
steps.<id>.evidence	The facts the step declared, supplied by the operator

Rules

• Steps run in order; a step starts only after its predecessors are done or skipped.
• A step completes only with a passing verification and its declared evidence.
• A verify command runs only when the operator confirms (--yes), with a 60 s timeout.
• Only read-only steps may be skipped, and a run with skips can never succeed.
• Values matching the secret markers are refused everywhere they could be recorded.
• The engine never edits pack.yaml. How far a pack can be trusted is not a field in it: it is derived from run history (rtut maturity, docs/maturity-and-projects.md).

CLI: rtut run new|list|status|cmd|start|verify|done|fail|retry|skip|feedback|finalize|capture <pack.yaml> ...

Outcome and score

A run is unfinished (stalled) until finalized. rtut run finalize (or the UI's Close this run) records an outcome:

Outcome	Rule
succeeded	every step is done (skipped steps do not count)
failed	at least one step has failed; otherwise abandon
abandoned	any time. An unfinished run untouched for 7 days also counts as abandoned in the score

A finished run is read-only, except for feedback. The score of a runbook is derived from its run records: tried, succeeded, failed, abandoned, stalled (unfinished), plus a tally of feedback ratings (rtut score <root>; shown in the UI). Run records live in .local/ of the machine that ran them, so the score is per machine until run histories are shared.

Finalizing never edits the pack. A clean succeeded run becomes evidence for the pack's maturity at the revision it ran (docs/maturity-and-projects.md); this replaces the earlier exercise_status field and the "Record pack as exercised" step, both retired. Old packs that still carry exercise_status load unchanged; the field is ignored.

Feedback

feedback entries (rating good|neutral|poor, comment, at) are added after a run; history is kept and the latest counts. rtut feedback <pack.yaml> lists them.

Captures

A step may carry optional captures: the command actually run, and the output that came back, as documentation or for inspection when the engine cannot reach the system itself. Limits: 20 000 characters of output, 2 000 of command. Obvious secret markers are refused; the engine cannot detect every secret, so the operator stays responsible. For output-contains steps the output can be pasted for verification (source: pasted-for-verification), which checks the contains text without running anything. Exit-code checks cannot be verified from pasted output.

Run record runbook-run/v0.2 (packs of runbook-pack/v0.2)

Implemented in rtut/run2.py (graph in rtut/flow.py). It adds to the v0.1 record:

Field	Meaning
mode	manual or auto-confirm
datapoints.<id>	state nil, set, err, ok or pushed; value, how (initial, manual, produced, check, manual confirmation), error
steps.<id>	as v0.1 (pushed is an additional state), plus alternative (chosen one_of branch), authorized_by
gates.<id>	state open, passed, failed, waived or pushed; per-check results; waiver or pushed
deviations[]	every override, waiver and push-through (see docs/pack-spec-v0.2.md)
auto-confirm	preauthorized irreversible steps, authorized_by, and stopped if a timeout stopped the run

Rules the engine enforces:

• Preconditions of an item are its requires (and those of enclosing stages), the set/ok state of the datapoints it uses, and, for a gate, completion of the steps of its stage.
• Recommended next action: the running item first, else the first ready item in the pack's canonical order. next_actions(run) returns the recommended item, all ready items (parallel work) and the blocked ones with what they wait for.
• Alternatives: a one_of step is started with an alternative (default: the first whose recommended_when matches the datapoints).
• Datapoints: invalid input leaves the datapoint err and is never stored; a step's produces become set when it completes; confirmed_by promotes them to ok when that step or gate passes.
• Override (manual mode only): starting an item with unmet preconditions needs a reason and an acknowledged downstream risk; it is recorded as a deviation.
• Waiver: only for gates that declare waivable, with a reason; recorded as a deviation.
• Confirmation: irreversible steps need the exact confirmation phrase at start. In auto-confirm they run only if the run pre-authorized them, and never if the step is auto_confirm: manual-only.
• Auto-confirm (auto-confirm_tick): the engine applies confirmation when preconditions are met: it starts the recommended item, runs its checks (manual checks are auto-confirmed and marked so) and completes it. It never runs a step's own command; the work is done by the operator or, later, by opt-in execution. A step with evidence waits for the operator. If checks keep failing until the item's timeout (or the pack default), the run stops, or with on_timeout: push-through the item is marked done with state pushed and a deviation is recorded.
• Deviations and maturity: a run with any deviation or pushed-through step can be finalized but never counts as maturity evidence. finalize(succeeded) also needs every gate passed or waived.
• Pushed is its own state. A step, gate or datapoint passed over by push-through is pushed: not done, not set, not err. Dependents may proceed (that is the point of pushing through), but a pushed datapoint has no value, so a command that needs it fails its check instead of running with nothing. A run with pushed items can be finished as failed (never succeeded) and never counts as maturity evidence.
• Stalled: an unfinished run is counted as stalled in the score, and as abandoned after a week.
• Modes are manual and auto-confirm. Auto-confirm applies confirmation only; running the step's own command is a separate, later level ("auto-pilot with execution", RBT-WP-0006).

Closed runs are locked and sealed

Closing a run (finalize) makes its record final:

• Locked. A closed record cannot be saved with changes: the engine refuses, naming what would have changed. This holds for runs closed before sealing existed too.
• Feedback. It locks with the run. Give it before closing, or with the closing form: finalize(..., rating, comment) records an optional rating and comment together with the outcome (validated first, so a refused close changes nothing). This keeps abandoned and failed runs from losing what the operator wanted to say.
• Sealed. The record carries a SHA-256 seal of its content (excluding seal and updated; a legacy exercise_recorded key on old records is ignored), and the file is made read-only. seal_state reports open, sealed, broken (changed outside the app after closing) or unsealed (closed before sealing existed). A broken seal is called out in the UI, and such a run no longer counts as maturity evidence. This is tamper-evidence, not prevention: someone with file access can still edit a file, but not without being noticed.

Exporting a closed run as evidence

A closed run's record can be downloaded as JSON from its page (GET /run/<i>/<run>/record.json; project runs: /project-run/<id>/record.json), exactly as stored. An open run has no final record (409). The response carries X-Rtut-Seal (sealed, broken or unsealed), so a record that was changed on disk is still downloadable but flagged. rtut seal <file> checks an exported record anywhere: it recomputes the SHA-256 over the record (without seal and updated) and prints sealed (exit 0), broken (1), or unsealed/open (2). The record is non-secret by construction (secret-looking text is refused everywhere it could be recorded); it does include any output the operator chose to capture, so read it before sharing.

Changing a value after steps used it

A v0.1 run fixes its parameters when it starts. In a v0.2 run datapoints can be edited (guided flow questions, data view), but a value that steps have already started with is in use. Changing it needs the operator's override (a reason and an acknowledged risk), only in manual mode, and is recorded as a changed-after-use deviation naming the steps and the previous value. A save containing any such change is refused as a whole unless the override is given, so a refused save applies nothing. Values nobody used yet change freely, which is what the questionnaire is for. A run with any deviation never counts towards maturity.

Timestamps and durations

created is the start and finished (set when the run is closed) the end. elapsed is start to finish, or to now while the run is unfinished; working is the time steps were in progress; each step has its own started and finished. The runbook's score carries the average, median and fastest elapsed time of succeeded runs only: unfinished, failed and abandoned runs say nothing about how long a working run takes. The UI shows this on the run, on each finished step, in the runs list and next to the score.

Artefacts: open and download

An artefact is a reference to something a run produced. The UI links it when it is a URL, or when it is exactly the artefact's resolved local file (GET /run/<i>/<run>/artefact/<id>, ?download=1 for an attachment). It refuses, with the reason: a file that does not exist yet, anything that is not a regular file, files over 5 MB, files in sensitive places (.ssh, .gnupg, .aws, .kube, .git, key and certificate files, run records), and files whose beginning looks like a secret. Text is served as plain text, never as a page, with nosniff and a sandboxing CSP; other content is only offered as a download. Symlinks are judged by their real target.

Context and revision

Every run (v0.1 and v0.2) records pack_digest, the revision of the pack directory it used (a hash of its files, excluding review records, local state and exercise_status: lines), and context: the optional non-secret project, target and environment class (rehearsal | staging | production) it was applied to. Maturity and project state are derived from these (docs/maturity-and-projects.md). Run records of all packs in a repository share one folder; a pack's runs are those whose pack_id matches.

Run timestamps have microsecond precision, so runs and feedback made within the same second still sort correctly.

• Retry: a failed step that is not declared idempotent (or is undeclared) is retried only with acknowledge_partial, the operator's confirmation that the earlier attempt did not partly apply; it is recorded in retries.
• Mode switch: switch_mode hands a run over to auto-confirm or takes it over again; each change is recorded in mode_history.
• Views for renderers: overview, datapoint_rows (grouped or filtered by dimension), dimension_keys and artefact_rows give a UI everything the flow, data and overview views show, without knowing the record layout.