coulomb/runbook-tutorials
5
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Retire exercise_status: maturity is the only notion; add CLI tests (RBT-WP-0009-T01)

Browse source 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: sonnet
Assistant-Process: 295952@bnt-lap001
Assistant-Session: e93f64ad-516c-46eb-9666-aad8d300c477
This commit is contained in:
tegwick 2026-09-29 21:28:56 +02:00
parent b89e8eaa74
commit 6d3414d8b7
29 changed files with 444 additions and 218 deletions
Download patch file Download diff file Expand all files Collapse all files

29
docs/run-spec.md
View file

@ -18,14 +18,14 @@ One run of one native pack, stored as JSON in
- Steps run in order; a step starts only after its predecessors are done or skipped.
- A step completes only with a passing verification and its declared evidence.
- A verify command runs only when the operator confirms (`--yes`), with a 60 s timeout.
- Only read-only steps may be skipped, and a run with skips never exercises a pack.
- Only read-only steps may be skipped, and a run with skips can never succeed.
- Values matching the secret markers are refused everywhere they could be recorded.
- The engine never edits `pack.yaml`. `rtut run finish` shows the exercise-status
diff; `--apply` writes it, and only for a run in which every step is done.
- The engine never edits `pack.yaml`. How far a pack can be trusted is not a field in it: it is derived from run history
(`rtut maturity`, `docs/maturity-and-projects.md`).
CLI: `rtut run new|list|status|cmd|start|verify|done|fail|retry|skip|finish <pack.yaml> ...`
CLI: `rtut run new|list|status|cmd|start|verify|done|fail|retry|skip|feedback|finalize|capture <pack.yaml> ...`
## Outcome, score and exercise status
## Outcome and score
A run is unfinished (**stalled**) until finalized. `rtut run finalize` (or the UI's *Close this run*) records
an `outcome`:
@ -41,11 +41,10 @@ its run records: tried, succeeded, failed, abandoned, stalled (unfinished), plus
(`rtut score <root>`; shown in the UI). Run records live in `.local/` of the machine that ran
them, so the score is per machine until run histories are shared.
Finalizing never edits the pack by itself. For a `succeeded` run the UI offers **Record pack
as exercised**, which shows what will be written, and on click sets
`exercise_status: exercised <date> by <operator>` in `pack.yaml` (`rtut run finalize ... --outcome
succeeded --apply` does the same). The change is an ordinary edit in the owning repository, to
be reviewed and committed there.
Finalizing never edits the pack. A clean `succeeded` run becomes evidence for the pack's **maturity** at the
revision it ran (`docs/maturity-and-projects.md`); this replaces the earlier `exercise_status` field and the
"Record pack as exercised" step, both retired. Old packs that still carry `exercise_status` load unchanged; the
field is ignored.
## Feedback
@ -96,12 +95,12 @@ Rules the engine enforces:
with `evidence` waits for the operator. If checks keep failing until the item's `timeout` (or the pack
default), the run stops, or with `on_timeout: push-through` the item is marked done with
state `pushed` and a deviation is recorded.
- **Deviations and exercise**: a run with any deviation or pushed-through step can be finalized but never exercises a pack.
- **Deviations and maturity**: a run with any deviation or pushed-through step can be finalized but never counts as maturity evidence.
`finalize(succeeded)` also needs every gate passed or waived.
- **Pushed is its own state.** A step, gate or datapoint passed over by push-through is `pushed`: not `done`,
not `set`, not `err`. Dependents may proceed (that is the point of pushing through), but a pushed datapoint
has no value, so a command that needs it fails its check instead of running with nothing. A run with pushed
items can be finished as `failed` (never `succeeded`) and never exercises a pack.
items can be finished as `failed` (never `succeeded`) and never counts as maturity evidence.
- **Stalled**: an unfinished run is counted as `stalled` in the score, and as `abandoned` after a week.
- **Modes** are `manual` and `auto-confirm`. Auto-confirm applies confirmation only; running the step's own
command is a separate, later level ("auto-pilot with execution", RBT-WP-0006).
@ -110,12 +109,12 @@ Rules the engine enforces:
Closing a run (`finalize`) makes its record final:
- **Locked.** A closed record cannot be saved with changes: the engine refuses, naming what would have changed. The only later update it
accepts is `exercise_recorded`, written by the legacy "Record pack as exercised" step. This holds for runs closed before sealing existed too.
- **Locked.** A closed record cannot be saved with changes: the engine refuses, naming what would have changed. This holds for runs closed
before sealing existed too.
- **Feedback.** It locks with the run. Give it before closing, or with the closing form: `finalize(..., rating, comment)` records an optional
rating and comment together with the outcome (validated first, so a refused close changes nothing). This keeps abandoned and failed runs from
losing what the operator wanted to say.
- **Sealed.** The record carries a SHA-256 `seal` of its content (excluding `seal`, `updated` and `exercise_recorded`), and the file is made
- **Sealed.** The record carries a SHA-256 `seal` of its content (excluding `seal` and `updated`; a legacy `exercise_recorded` key on old records is ignored), and the file is made
read-only. `seal_state` reports `open`, `sealed`, `broken` (changed outside the app after closing) or `unsealed` (closed before sealing existed).
A broken seal is called out in the UI, and such a run no longer counts as maturity evidence. This is tamper-evidence, not prevention: someone
with file access can still edit a file, but not without being noticed.