Engine v0.2: flow graph, datapoint states, gates, modes, overrides, deviations (RBT-WP-0007)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: sonnet Assistant-Process: 295952@bnt-lap001 Assistant-Session: e93f64ad-516c-46eb-9666-aad8d300c477
This commit is contained in:
parent
3fc279a6ed
commit
60d983769f
13 changed files with 1017 additions and 22 deletions
|
|
@ -34,6 +34,10 @@ pack
|
|||
| **Step** | `todo`, `running`, `done`, `failed` (`skipped` for read-only) | Has it been applied? |
|
||||
| **Gate** | `open`, `passed`, `failed`, `waived` | Is the quality condition met? |
|
||||
|
||||
A datapoint becomes `ok` when the step or gate named in its `confirmed_by` passes. A gate must not
|
||||
require a datapoint (`datapoints_ok`) that it is itself `confirmed_by`: that is circular and the validator
|
||||
rejects it.
|
||||
|
||||
A v0.1 `parameter` is a datapoint that has a value type and is asked as a
|
||||
question. Steps `uses:` datapoints; steps may `produces:` datapoints
|
||||
(for example an artefact location).
|
||||
|
|
|
|||
|
|
@ -60,3 +60,41 @@ back, as documentation or for inspection when the engine cannot reach the system
|
|||
detect every secret, so the operator stays responsible. For `output-contains` steps the output can
|
||||
be **pasted for verification** (`source: pasted-for-verification`), which checks the `contains` text
|
||||
without running anything. Exit-code checks cannot be verified from pasted output.
|
||||
|
||||
## Run record `runbook-run/v0.2` (packs of `runbook-pack/v0.2`)
|
||||
|
||||
Implemented in `rtut/run2.py` (graph in `rtut/flow.py`). It adds to the v0.1 record:
|
||||
|
||||
| Field | Meaning |
|
||||
| --- | --- |
|
||||
| `mode` | `manual` or `autopilot` |
|
||||
| `datapoints.<id>` | `state` nil, set, err or ok; `value`, `how` (initial, manual, produced, check, manual confirmation), `error` |
|
||||
| `steps.<id>` | as v0.1, plus `alternative` (chosen `one_of` branch), `authorized_by`, `pushed_through` |
|
||||
| `gates.<id>` | `state` open, passed, failed or waived; per-check `results`; `waiver` |
|
||||
| `deviations[]` | every override, waiver and push-through (see `docs/pack-spec-v0.2.md`) |
|
||||
| `autopilot` | `preauthorized` irreversible steps, `authorized_by`, and `stopped` if a timeout stopped the run |
|
||||
|
||||
Rules the engine enforces:
|
||||
|
||||
- **Preconditions** of an item are its `requires` (and those of enclosing stages), the `set`/`ok`
|
||||
state of the datapoints it `uses`, and, for a gate, completion of the steps of its stage.
|
||||
- **Recommended next action**: the running item first, else the first ready item in the pack's canonical
|
||||
order. `next_actions(run)` returns the recommended item, all ready items (parallel work) and the blocked
|
||||
ones with what they wait for.
|
||||
- **Alternatives**: a `one_of` step is started with an alternative (default: the first whose
|
||||
`recommended_when` matches the datapoints).
|
||||
- **Datapoints**: invalid input leaves the datapoint `err` and is never stored; a step's `produces` become `set`
|
||||
when it completes; `confirmed_by` promotes them to `ok` when that step or gate passes.
|
||||
- **Override** (manual mode only): starting an item with unmet preconditions needs a reason and an acknowledged
|
||||
downstream risk; it is recorded as a deviation.
|
||||
- **Waiver**: only for gates that declare `waivable`, with a reason; recorded as a deviation.
|
||||
- **Confirmation**: irreversible steps need the exact confirmation phrase at start. In autopilot they run only
|
||||
if the run pre-authorized them, and never if the step is `autopilot: manual-only`.
|
||||
- **Autopilot** (`autopilot_tick`): the engine applies confirmation when preconditions are met: it starts the
|
||||
recommended item, runs its checks (manual checks are auto-confirmed and marked so) and completes it. It
|
||||
never runs a step's own command; the work is done by the operator or, later, by opt-in execution. A step
|
||||
with `evidence` waits for the operator. If checks keep failing until the item's `timeout` (or the pack
|
||||
default), the run stops, or with `on_timeout: push-through` the item is marked done with
|
||||
`pushed_through` and a deviation is recorded.
|
||||
- **Deviations and exercise**: a run with any deviation or pushed-through step can be finalized but never exercises a pack.
|
||||
`finalize(succeeded)` also needs every gate passed or waived.
|
||||
|
|
|
|||
69
rtut/flow.py
Normal file
69
rtut/flow.py
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
"""The flow graph of a runbook-pack/v0.2: stages, steps, alternatives, gates, canonical order."""
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
class Flow:
|
||||
def __init__(self, data: dict):
|
||||
self.data = data
|
||||
self.stages: dict[str, dict] = {}
|
||||
self.steps: dict[str, dict] = {}
|
||||
self.gates: dict[str, dict] = {}
|
||||
self.alts: dict[str, str] = {} # alternative id -> parent step id
|
||||
self.parent: dict[str, str | None] = {} # flow id -> enclosing stage id
|
||||
self.order: list[str] = [] # canonical order of steps and gates
|
||||
self.datapoints = {d["id"]: d for d in data.get("datapoints", [])}
|
||||
self.actors = {a["id"]: a for a in data.get("actors", [])}
|
||||
self._walk(data["stages"], None)
|
||||
|
||||
def _walk(self, stages: list[dict], parent: str | None) -> None:
|
||||
for st in stages:
|
||||
self.stages[st["id"]] = st
|
||||
self.parent[st["id"]] = parent
|
||||
for s in st.get("steps", []):
|
||||
self.steps[s["id"]] = s
|
||||
self.parent[s["id"]] = st["id"]
|
||||
self.order.append(s["id"])
|
||||
for a in s.get("one_of", []):
|
||||
self.alts[a["id"]] = s["id"]
|
||||
self._walk(st.get("stages", []), st["id"])
|
||||
for g in st.get("gates", []):
|
||||
self.gates[g["id"]] = g
|
||||
self.parent[g["id"]] = st["id"]
|
||||
self.order.append(g["id"])
|
||||
|
||||
# -- structure ------------------------------------------------------------------------
|
||||
def kind(self, id_: str) -> str:
|
||||
return "stage" if id_ in self.stages else "step" if id_ in self.steps else "gate" if id_ in self.gates else "?"
|
||||
|
||||
def node(self, id_: str) -> dict:
|
||||
return self.stages.get(id_) or self.steps.get(id_) or self.gates[id_]
|
||||
|
||||
def resolve(self, ref: str) -> str | None:
|
||||
if ref.startswith("gate:"):
|
||||
return ref[5:] if ref[5:] in self.gates else None
|
||||
return ref if ref in self.stages or ref in self.steps else None
|
||||
|
||||
def ancestors(self, id_: str) -> list[str]:
|
||||
out, p = [], self.parent.get(id_)
|
||||
while p:
|
||||
out.append(p)
|
||||
p = self.parent.get(p)
|
||||
return out
|
||||
|
||||
def members(self, stage_id: str, include_gates: bool = True) -> list[str]:
|
||||
"""Steps (and gates) inside a stage, recursively."""
|
||||
out = []
|
||||
st = self.stages[stage_id]
|
||||
out += [s["id"] for s in st.get("steps", [])]
|
||||
for child in st.get("stages", []):
|
||||
out += self.members(child["id"], include_gates)
|
||||
if include_gates:
|
||||
out += [g["id"] for g in st.get("gates", [])]
|
||||
return out
|
||||
|
||||
def requires(self, id_: str) -> list[str]:
|
||||
"""Own requires plus those of every enclosing stage, as resolved flow ids."""
|
||||
refs = list(self.node(id_).get("requires", []))
|
||||
for a in self.ancestors(id_):
|
||||
refs += self.stages[a].get("requires", [])
|
||||
return [r for r in (self.resolve(x) for x in refs) if r]
|
||||
|
|
@ -112,7 +112,7 @@ def load(path: Path) -> Pack:
|
|||
|
||||
def render(command: str, values: dict[str, str], pack: Pack) -> str:
|
||||
"""Substitute parameters into a command template. Unknown or unset -> error."""
|
||||
params = {p["id"]: p for p in pack.data.get("parameters", [])}
|
||||
params = {p["id"]: p for p in (pack.data.get("parameters") or pack.data.get("datapoints") or [])}
|
||||
|
||||
def sub(m: re.Match) -> str:
|
||||
name = m.group(1)
|
||||
|
|
|
|||
11
rtut/run.py
11
rtut/run.py
|
|
@ -72,8 +72,11 @@ def _writable(run: Run) -> None:
|
|||
def create(pack: Pack, operator: str, params: dict[str, str] | None = None) -> Run:
|
||||
if not pack.ok or pack.data.get("engine") != "native":
|
||||
raise RunError("only valid native packs can be run")
|
||||
if pack.spec == "runbook-pack/v0.2":
|
||||
from . import run2
|
||||
return run2.create(pack, operator, params)
|
||||
if pack.spec != "runbook-pack/v0.1":
|
||||
raise RunError(f"{pack.spec} packs cannot be run yet (RBT-WP-0007)")
|
||||
raise RunError(f"{pack.spec} packs cannot be run")
|
||||
operator = _clean(operator.strip(), "operator")
|
||||
if not re.fullmatch(r"\S+", operator):
|
||||
raise RunError("operator must be one word (name or handle)")
|
||||
|
|
@ -239,6 +242,10 @@ def exercise_update(run: Run, today: date | None = None) -> tuple[str, str]:
|
|||
"""
|
||||
if any(v["state"] != "done" for v in run.data["steps"].values()):
|
||||
raise RunError("every step must be done (not skipped) to exercise the pack")
|
||||
if run.data.get("deviations"):
|
||||
raise RunError("a run with deviations (overrides, waivers, push-through) cannot exercise the pack")
|
||||
if any(v.get("pushed_through") for v in run.data["steps"].values()):
|
||||
raise RunError("a run with pushed-through steps cannot exercise the pack")
|
||||
today = today or date.today()
|
||||
old = run.pack.path.read_text()
|
||||
new = STATUS_LINE.sub(f"exercise_status: exercised {today.isoformat()} by {run.data['operator']}",
|
||||
|
|
@ -337,6 +344,8 @@ def finalize(run: Run, outcome: str, note: str = "") -> None:
|
|||
states = [v["state"] for v in run.data["steps"].values()]
|
||||
if outcome == "succeeded" and any(x != "done" for x in states):
|
||||
raise RunError("a run succeeds only when every step is done (skipped steps do not count)")
|
||||
if outcome == "succeeded" and any(g["state"] not in ("passed", "waived") for g in run.data.get("gates", {}).values()):
|
||||
raise RunError("a run succeeds only when every gate has passed or been waived")
|
||||
if outcome == "failed" and "failed" not in states:
|
||||
raise RunError("a run can be finished as failed only when a step has failed; otherwise abandon it")
|
||||
run.data["outcome"] = outcome
|
||||
|
|
|
|||
539
rtut/run2.py
Normal file
539
rtut/run2.py
Normal file
|
|
@ -0,0 +1,539 @@
|
|||
"""Runs of runbook-pack/v0.2 packs (record runbook-run/v0.2).
|
||||
|
||||
Adds to the v0.1 run: datapoint states (nil/set/err/ok), a dependency graph with a recommended
|
||||
next action, alternatives, QA gates with waivers, manual and autopilot modes with timeouts, and
|
||||
a deviation record. Shared helpers (feedback, finalize, captures, score) live in run.py.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from .flow import Flow
|
||||
from .pack import Pack, render
|
||||
from .run import Run, RunError, _clean, _now, runs_dir # noqa: F401 (re-exported for callers)
|
||||
|
||||
DEFAULT_PHRASE = "I understand this cannot be undone"
|
||||
VERIFY_TIMEOUT = 60
|
||||
DONE_STEP = ("done", "skipped")
|
||||
DONE_GATE = ("passed", "waived")
|
||||
|
||||
|
||||
def flow(run: Run) -> Flow:
|
||||
if not hasattr(run, "_flow"):
|
||||
run._flow = Flow(run.pack.data) # type: ignore[attr-defined]
|
||||
return run._flow # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def _values(run: Run) -> dict[str, str]:
|
||||
return {k: v["value"] for k, v in run.data["datapoints"].items() if v["state"] in ("set", "ok")}
|
||||
|
||||
|
||||
def _sync(run: Run) -> None:
|
||||
run.data["params"] = _values(run)
|
||||
|
||||
|
||||
# ---- datapoints ---------------------------------------------------------------------------------
|
||||
|
||||
def _coerce(run: Run, dp_id: str, value: str) -> tuple[str | None, str | None]:
|
||||
"""(clean value, None) or (None, error)."""
|
||||
dp = flow(run).datapoints[dp_id]
|
||||
value = str(value)
|
||||
if dp["type"] == "int" and not re.fullmatch(r"-?\d+", value):
|
||||
return None, "not an integer"
|
||||
if dp["type"] == "bool" and value not in ("true", "false"):
|
||||
return None, "must be true or false"
|
||||
try:
|
||||
render("{{%s}}" % dp_id, {dp_id: value}, run.pack) # values, pattern, unsafe characters
|
||||
_clean(value, f"datapoint {dp_id}")
|
||||
except (ValueError, KeyError, RunError) as e:
|
||||
return None, str(e)
|
||||
return value, None
|
||||
|
||||
|
||||
def set_datapoint(run: Run, dp_id: str, value: str, how: str = "manual") -> str:
|
||||
"""Give a datapoint a value. Invalid input leaves it in state 'err' (never stores it)."""
|
||||
_writable(run)
|
||||
if dp_id not in flow(run).datapoints:
|
||||
raise RunError(f"no such datapoint: {dp_id}")
|
||||
clean, err = _coerce(run, dp_id, value)
|
||||
rec = run.data["datapoints"][dp_id]
|
||||
if err:
|
||||
rec.update(state="err", error=err, at=_now())
|
||||
rec.pop("value", None)
|
||||
else:
|
||||
rec.update(state="set", value=clean, how=how, at=_now())
|
||||
rec.pop("error", None)
|
||||
_sync(run)
|
||||
run.save()
|
||||
return rec["state"]
|
||||
|
||||
|
||||
def _confirm_datapoints(run: Run, by: str, how: str) -> None:
|
||||
for d in flow(run).datapoints.values():
|
||||
rec = run.data["datapoints"][d["id"]]
|
||||
if d.get("confirmed_by") == by and rec["state"] == "set":
|
||||
rec.update(state="ok", how=how, at=_now())
|
||||
|
||||
|
||||
# ---- creation -------------------------------------------------------------------------------------
|
||||
|
||||
def create(pack: Pack, operator: str, values: dict[str, str] | None = None, mode: str = "manual",
|
||||
preauthorize: list[str] | None = None) -> Run:
|
||||
if not pack.ok or pack.data.get("engine") != "native" or pack.spec != "runbook-pack/v0.2":
|
||||
raise RunError("only valid native runbook-pack/v0.2 packs can be run here")
|
||||
if mode not in ("manual", "autopilot"):
|
||||
raise RunError("mode must be manual or autopilot")
|
||||
operator = _clean(operator.strip(), "operator")
|
||||
if not re.fullmatch(r"\S+", operator):
|
||||
raise RunError("operator must be one word (name or handle)")
|
||||
fl = Flow(pack.data)
|
||||
for u in values or {}:
|
||||
if u not in fl.datapoints:
|
||||
raise RunError(f"unknown datapoint {u}")
|
||||
pre = list(preauthorize or [])
|
||||
for sid in pre:
|
||||
if sid not in fl.steps or effective_step(pack, fl, sid, None).get("reversible", True):
|
||||
raise RunError(f"{sid}: only irreversible steps can be pre-authorized")
|
||||
data = {
|
||||
"spec": "runbook-run/v0.2", "run_id": time.strftime("%Y%m%d-%H%M%S-") + uuid.uuid4().hex[:6],
|
||||
"pack_id": pack.id, "pack_path": str(pack.path), "operator": operator, "created": _now(),
|
||||
"mode": mode, "params": {}, "deviations": [],
|
||||
"autopilot": {"preauthorized": pre, "authorized_by": operator if pre else None},
|
||||
"datapoints": {i: {"state": "nil"} for i in fl.datapoints},
|
||||
"steps": {i: {"state": "todo"} for i in fl.steps},
|
||||
"gates": {i: {"state": "open", "results": {}} for i in fl.gates},
|
||||
}
|
||||
run = Run(pack, data)
|
||||
run._flow = fl # type: ignore[attr-defined]
|
||||
for i, d in fl.datapoints.items():
|
||||
v = (values or {}).get(i, d.get("default"))
|
||||
if v is not None:
|
||||
set_datapoint(run, i, str(v).lower() if isinstance(v, bool) else str(v), how="initial")
|
||||
if any(x["state"] == "err" for x in data["datapoints"].values()):
|
||||
bad = {k: x["error"] for k, x in data["datapoints"].items() if x["state"] == "err"}
|
||||
raise RunError(f"invalid datapoint values: {bad}")
|
||||
run.save()
|
||||
return run
|
||||
|
||||
|
||||
def _writable(run: Run) -> None:
|
||||
if run.data.get("outcome"):
|
||||
raise RunError(f"run is finished (outcome: {run.data['outcome']}); start a new run")
|
||||
|
||||
|
||||
# ---- step definitions and alternatives ------------------------------------------------------------------
|
||||
|
||||
def effective_step(pack: Pack, fl: Flow, step_id: str, alt: str | None) -> dict:
|
||||
"""The step with the chosen alternative's command, verify, reversibility and rollback applied."""
|
||||
s = dict(fl.steps[step_id])
|
||||
if s.get("one_of") and alt:
|
||||
a = next(x for x in s["one_of"] if x["id"] == alt)
|
||||
for k in ("command", "verify", "reversible", "rollback", "title"):
|
||||
if k in a:
|
||||
s[k] = a[k]
|
||||
return s
|
||||
|
||||
|
||||
def recommended_alt(run: Run, step_id: str) -> str | None:
|
||||
vals = _values(run)
|
||||
for a in flow(run).steps[step_id].get("one_of", []):
|
||||
rw = a.get("recommended_when") or {}
|
||||
if rw and all(vals.get(k) in v for k, v in rw.items()):
|
||||
return a["id"]
|
||||
return None
|
||||
|
||||
|
||||
def step_def(run: Run, step_id: str) -> dict:
|
||||
if step_id not in flow(run).steps:
|
||||
raise RunError(f"no such step: {step_id}")
|
||||
return effective_step(run.pack, flow(run), step_id, run.data["steps"][step_id].get("alternative"))
|
||||
|
||||
|
||||
def command_for(run: Run, step_id: str) -> str | None:
|
||||
cmd = step_def(run, step_id).get("command")
|
||||
return render(cmd, run.data["params"], run.pack) if cmd else None
|
||||
|
||||
|
||||
def phrase_for(run: Run, step_id: str) -> str:
|
||||
return flow(run).steps[step_id].get("confirm_phrase") or run.pack.data.get("confirm_phrase") or DEFAULT_PHRASE
|
||||
|
||||
|
||||
def confirmation_style(run: Run, step_id: str) -> str:
|
||||
"""'confirm' (button) for reversible steps, 'phrase' (typed) for irreversible ones."""
|
||||
return "confirm" if step_def(run, step_id).get("reversible", True) else "phrase"
|
||||
|
||||
|
||||
# ---- readiness, unmet preconditions, the recommended next action -----------------------------------------------
|
||||
|
||||
def is_complete(run: Run, id_: str) -> bool:
|
||||
fl = flow(run)
|
||||
k = fl.kind(id_)
|
||||
if k == "step":
|
||||
return run.data["steps"][id_]["state"] in DONE_STEP
|
||||
if k == "gate":
|
||||
return run.data["gates"][id_]["state"] in DONE_GATE
|
||||
return all(is_complete(run, m) for m in fl.members(id_))
|
||||
|
||||
|
||||
def unmet(run: Run, id_: str) -> list[str]:
|
||||
fl = flow(run)
|
||||
out = []
|
||||
for r in fl.requires(id_):
|
||||
if not is_complete(run, r):
|
||||
out.append(f"{fl.kind(r)} {r} is not complete")
|
||||
if fl.kind(id_) == "step":
|
||||
for dp in fl.steps[id_].get("uses", []):
|
||||
st = run.data["datapoints"][dp]["state"]
|
||||
if st not in ("set", "ok"):
|
||||
out.append(f"datapoint {dp} is {st}")
|
||||
elif fl.kind(id_) == "gate":
|
||||
stage = fl.parent[id_]
|
||||
for m in fl.members(stage, include_gates=False):
|
||||
if not is_complete(run, m):
|
||||
out.append(f"step {m} of the stage is not done")
|
||||
return out
|
||||
|
||||
|
||||
def _item_open(run: Run, id_: str) -> bool:
|
||||
fl = flow(run)
|
||||
return not is_complete(run, id_) if fl.kind(id_) in ("step", "gate") else False
|
||||
|
||||
|
||||
def next_actions(run: Run) -> dict:
|
||||
"""{'recommended': id|None, 'ready': [...], 'blocked': [{'id', 'unmet'}...]} in canonical order."""
|
||||
fl = flow(run)
|
||||
running = [i for i in fl.order if fl.kind(i) == "step" and run.data["steps"][i]["state"] == "running"]
|
||||
ready, blocked = [], []
|
||||
for i in fl.order:
|
||||
if not _item_open(run, i) or i in running:
|
||||
continue
|
||||
u = unmet(run, i)
|
||||
(blocked if u else ready).append({"id": i, "unmet": u} if u else i)
|
||||
rec = (running + ready + [None])[0]
|
||||
return {"recommended": rec, "running": running, "ready": ready, "blocked": blocked}
|
||||
|
||||
|
||||
# ---- deviations --------------------------------------------------------------------------------------------------
|
||||
|
||||
def _deviate(run: Run, kind: str, item: str, unmet_list: list[str], reason: str, risk: str, ack: bool) -> None:
|
||||
run.data["deviations"].append({
|
||||
"kind": kind, "item": item, "mode": run.data["mode"], "unmet": unmet_list,
|
||||
"downstream_risk": risk, "risk_acknowledged": ack, "reason": _clean(reason.strip(), "reason"),
|
||||
"who": run.data["operator"], "at": _now()})
|
||||
|
||||
|
||||
def _risk_text(run: Run, id_: str) -> str:
|
||||
return flow(run).node(id_).get("downstream_risk") or "Steps that rely on this may break or give wrong results."
|
||||
|
||||
|
||||
def _gate_precondition(run: Run, id_: str, override_reason: str, acknowledge_risk: bool) -> None:
|
||||
u = unmet(run, id_)
|
||||
if not u:
|
||||
return
|
||||
if not override_reason.strip():
|
||||
raise RunError(f"{id_} has unmet preconditions: {'; '.join(u)}. Risk: {_risk_text(run, id_)} "
|
||||
"To continue anyway give an override reason and acknowledge the risk.")
|
||||
if not acknowledge_risk:
|
||||
raise RunError("an override needs the operator to acknowledge the downstream risk")
|
||||
if run.data["mode"] != "manual":
|
||||
raise RunError("overrides are made by the operator in manual mode")
|
||||
_deviate(run, "override", id_, u, override_reason, _risk_text(run, id_), True)
|
||||
|
||||
|
||||
# ---- steps -----------------------------------------------------------------------------------------------------------
|
||||
|
||||
def start(run: Run, step_id: str, alt: str | None = None, phrase: str | None = None,
|
||||
override_reason: str = "", acknowledge_risk: bool = False) -> None:
|
||||
_writable(run)
|
||||
fl = flow(run)
|
||||
if step_id not in fl.steps:
|
||||
raise RunError(f"no such step: {step_id}")
|
||||
st = run.data["steps"][step_id]
|
||||
if st["state"] in DONE_STEP or st["state"] == "running":
|
||||
raise RunError(f"step {step_id} already {st['state']}")
|
||||
_gate_precondition(run, step_id, override_reason, acknowledge_risk)
|
||||
s = fl.steps[step_id]
|
||||
if s.get("one_of"):
|
||||
alt = alt or recommended_alt(run, step_id)
|
||||
if alt not in {a["id"] for a in s["one_of"]}:
|
||||
raise RunError(f"step {step_id}: choose one of {', '.join(a['id'] for a in s['one_of'])}")
|
||||
st["alternative"] = alt
|
||||
elif alt:
|
||||
raise RunError(f"step {step_id} has no alternatives")
|
||||
eff = effective_step(run.pack, fl, step_id, st.get("alternative"))
|
||||
if not eff.get("reversible", True): # irreversible: the typed phrase is enforced here
|
||||
pre = run.data["autopilot"]["preauthorized"]
|
||||
if step_id in pre and run.data["mode"] == "autopilot":
|
||||
st["authorized_by"] = run.data["autopilot"]["authorized_by"]
|
||||
elif (phrase or "") != phrase_for(run, step_id):
|
||||
raise RunError(f"step {step_id} is irreversible: type the confirmation phrase exactly: "
|
||||
f"{phrase_for(run, step_id)!r}")
|
||||
else:
|
||||
st["authorized_by"] = run.data["operator"]
|
||||
st.pop("verification", None)
|
||||
st.update(state="running", started=_now())
|
||||
run.save()
|
||||
|
||||
|
||||
def _check_result(run: Run, ck: dict, confirm: bool, manual_ok: bool | None, auto_manual: bool = False) -> dict:
|
||||
parts, passed = {}, True
|
||||
dps = ck.get("datapoints_ok", [])
|
||||
if dps:
|
||||
states = {d: run.data["datapoints"][d]["state"] for d in dps}
|
||||
parts["datapoints_ok"] = all(v == "ok" for v in states.values())
|
||||
passed &= parts["datapoints_ok"]
|
||||
exp = ck.get("expect") or (None if dps else "manual")
|
||||
if exp == "manual":
|
||||
if auto_manual:
|
||||
parts["manual"] = True
|
||||
elif manual_ok is None:
|
||||
raise RunError("a manual check needs the operator's yes/no")
|
||||
else:
|
||||
parts["manual"] = bool(manual_ok)
|
||||
passed &= parts["manual"]
|
||||
elif exp in ("exit-0", "output-contains"):
|
||||
if not confirm:
|
||||
raise RunError("running a check command needs operator confirmation")
|
||||
cmd = render(ck["command"], run.data["params"], run.pack)
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
cp = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=VERIFY_TIMEOUT,
|
||||
cwd=run.pack.path.resolve().parents[2])
|
||||
code, out = cp.returncode, cp.stdout + cp.stderr
|
||||
except subprocess.TimeoutExpired:
|
||||
code, out = -1, ""
|
||||
parts.update(exit_code=code, output_bytes=len(out.encode()), seconds=round(time.monotonic() - t0, 2))
|
||||
ok = code == 0
|
||||
if exp == "output-contains":
|
||||
ok = ok and render(ck["contains"], run.data["params"], run.pack) in out
|
||||
parts["command"] = ok
|
||||
passed &= ok
|
||||
return {"passed": bool(passed), "kind": exp or "datapoints", "parts": parts, "at": _now(),
|
||||
**({"auto_confirmed": True} if auto_manual and exp == "manual" else {})}
|
||||
|
||||
|
||||
def verify(run: Run, step_id: str, confirm: bool = False, manual_ok: bool | None = None) -> dict:
|
||||
_writable(run)
|
||||
st = run.data["steps"][step_id]
|
||||
if st["state"] != "running":
|
||||
raise RunError(f"step {step_id} is not running")
|
||||
res = _check_result(run, step_def(run, step_id)["verify"], confirm, manual_ok)
|
||||
st["verification"] = res
|
||||
run.save()
|
||||
return res
|
||||
|
||||
|
||||
def complete(run: Run, step_id: str, evidence: dict[str, str] | None = None, note: str = "") -> None:
|
||||
_writable(run)
|
||||
st = run.data["steps"][step_id]
|
||||
if st["state"] != "running":
|
||||
raise RunError(f"step {step_id} is not running")
|
||||
if not st.get("verification", {}).get("passed"):
|
||||
raise RunError(f"step {step_id} has no passing verification")
|
||||
s = flow(run).steps[step_id]
|
||||
evidence = evidence or {}
|
||||
missing = [n for n in s.get("evidence", []) if n not in evidence]
|
||||
if missing:
|
||||
raise RunError(f"evidence required: {', '.join(missing)}")
|
||||
st["evidence"] = {k: _clean(str(v), f"evidence {k}") for k, v in evidence.items()}
|
||||
if note.strip():
|
||||
st["note"] = _clean(note.strip(), "note")
|
||||
st.update(state="done", finished=_now())
|
||||
for dp in s.get("produces", []):
|
||||
if dp in run.data["datapoints"] and run.data["datapoints"][dp]["state"] not in ("set", "ok"):
|
||||
val = st["evidence"].get(dp) or ("true" if flow(run).datapoints[dp]["type"] == "bool" else "(produced)")
|
||||
run.data["datapoints"][dp].update(state="set", value=val, how="produced", at=_now())
|
||||
_confirm_datapoints(run, step_id, st["verification"]["kind"] if st["verification"]["kind"] != "datapoints" else "check")
|
||||
_sync(run)
|
||||
run.save()
|
||||
|
||||
|
||||
def fail(run: Run, step_id: str, note: str = "") -> None:
|
||||
_writable(run)
|
||||
st = run.data["steps"][step_id]
|
||||
if st["state"] != "running":
|
||||
raise RunError(f"step {step_id} is not running")
|
||||
st.update(state="failed", note=_clean(note.strip(), "note"), finished=_now())
|
||||
for dp in flow(run).steps[step_id].get("produces", []):
|
||||
if dp in run.data["datapoints"] and run.data["datapoints"][dp]["state"] == "nil":
|
||||
run.data["datapoints"][dp].update(state="err", error=f"producing step {step_id} failed", at=_now())
|
||||
run.save()
|
||||
|
||||
|
||||
def retry(run: Run, step_id: str) -> None:
|
||||
_writable(run)
|
||||
st = run.data["steps"][step_id]
|
||||
if st["state"] != "failed":
|
||||
raise RunError(f"step {step_id} has not failed")
|
||||
for dp in flow(run).steps[step_id].get("produces", []):
|
||||
rec = run.data["datapoints"].get(dp)
|
||||
if rec and rec["state"] == "err" and rec.get("error", "").startswith("producing step"):
|
||||
run.data["datapoints"][dp] = {"state": "nil"}
|
||||
st.pop("verification", None)
|
||||
st.update(state="running", started=_now())
|
||||
run.save()
|
||||
|
||||
|
||||
# ---- gates ------------------------------------------------------------------------------------------------------------
|
||||
|
||||
def gate_check(run: Run, gate_id: str, index: int, confirm: bool = False, manual_ok: bool | None = None,
|
||||
override_reason: str = "", acknowledge_risk: bool = False, _auto: bool = False) -> dict:
|
||||
"""Run one check of a gate and update the gate state (passed when all checks pass)."""
|
||||
_writable(run)
|
||||
g = flow(run).gates.get(gate_id)
|
||||
if g is None:
|
||||
raise RunError(f"no such gate: {gate_id}")
|
||||
gs = run.data["gates"][gate_id]
|
||||
if gs["state"] in DONE_GATE:
|
||||
raise RunError(f"gate {gate_id} is already {gs['state']}")
|
||||
if index < 0 or index >= len(g["checks"]):
|
||||
raise RunError("no such check")
|
||||
if not gs.get("overridden") and unmet(run, gate_id):
|
||||
_gate_precondition(run, gate_id, override_reason, acknowledge_risk)
|
||||
gs["overridden"] = True
|
||||
gs.setdefault("started", _now())
|
||||
res = _check_result(run, g["checks"][index], confirm, manual_ok, auto_manual=_auto)
|
||||
gs["results"][str(index)] = res
|
||||
done = all(str(i) in gs["results"] for i in range(len(g["checks"])))
|
||||
if done:
|
||||
gs["state"] = "passed" if all(r["passed"] for r in gs["results"].values()) else "failed"
|
||||
if gs["state"] == "passed":
|
||||
_confirm_datapoints(run, gate_id, "check")
|
||||
_sync(run)
|
||||
run.save()
|
||||
return res
|
||||
|
||||
|
||||
def waive(run: Run, gate_id: str, reason: str) -> None:
|
||||
"""Waive a gate that declares 'waivable'. The waiver is a recorded deviation."""
|
||||
_writable(run)
|
||||
g = flow(run).gates.get(gate_id)
|
||||
if g is None:
|
||||
raise RunError(f"no such gate: {gate_id}")
|
||||
w = g.get("waivable")
|
||||
if not w:
|
||||
raise RunError(f"gate {gate_id} is not waivable; use an override at the dependent step instead")
|
||||
if not reason.strip():
|
||||
raise RunError("a waiver needs a reason")
|
||||
gs = run.data["gates"][gate_id]
|
||||
if gs["state"] == "passed":
|
||||
raise RunError("the gate has already passed")
|
||||
gs.update(state="waived", waiver={"by_actor": w["by"], "who": run.data["operator"], "reason": _clean(reason.strip(), "reason"), "at": _now()})
|
||||
_deviate(run, "waiver", gate_id, [f"check {i} not passed" for i, r in gs["results"].items() if not r["passed"]] or ["not evaluated"],
|
||||
reason, _risk_text(run, gate_id), True)
|
||||
run.save()
|
||||
|
||||
|
||||
# ---- autopilot ----------------------------------------------------------------------------------------------------------
|
||||
|
||||
def _timeout_for(run: Run, id_: str) -> dict | None:
|
||||
return flow(run).node(id_).get("timeout") or run.pack.data.get("timeout")
|
||||
|
||||
|
||||
def _elapsed(started: str, now: datetime) -> float:
|
||||
return (now - datetime.fromisoformat(started)).total_seconds()
|
||||
|
||||
|
||||
def _stop(run: Run, item: str, why: str) -> dict:
|
||||
run.data["autopilot"]["stopped"] = {"item": item, "reason": why, "at": _now()}
|
||||
run.save()
|
||||
return {"action": "stopped", "item": item, "detail": why}
|
||||
|
||||
|
||||
def autopilot_tick(run: Run, now: datetime | None = None) -> dict:
|
||||
"""One autopilot decision. Confirmation is applied when preconditions are met; a timeout stops
|
||||
the run or pushes through, recording a deviation. The engine never runs a step's own command."""
|
||||
_writable(run)
|
||||
if run.data["mode"] != "autopilot":
|
||||
raise RunError("autopilot_tick needs an autopilot run")
|
||||
ap = run.data["autopilot"]
|
||||
if ap.get("stopped"):
|
||||
return {"action": "stopped", "item": ap["stopped"]["item"], "detail": ap["stopped"]["reason"]}
|
||||
now = now or datetime.now(timezone.utc)
|
||||
fl = flow(run)
|
||||
na = next_actions(run)
|
||||
item = na["recommended"]
|
||||
if item is None:
|
||||
if all(is_complete(run, i) for i in fl.order):
|
||||
return {"action": "finished", "item": None, "detail": "every step is done and every gate passed or waived"}
|
||||
first = na["blocked"][0]
|
||||
return _stop(run, first["id"], "nothing can advance: " + "; ".join(first["unmet"]))
|
||||
if fl.kind(item) == "step":
|
||||
return _autopilot_step(run, item, now)
|
||||
return _autopilot_gate(run, item, now)
|
||||
|
||||
|
||||
def _on_timeout(run: Run, item: str, cfg: dict, detail: str) -> dict:
|
||||
if cfg.get("on_timeout", "stop") == "stop":
|
||||
return _stop(run, item, f"timed out after {cfg['seconds']}s: {detail}")
|
||||
return {"action": "push-through", "item": item, "detail": detail}
|
||||
|
||||
|
||||
def _autopilot_step(run: Run, sid: str, now: datetime) -> dict:
|
||||
fl = flow(run)
|
||||
s = fl.steps[sid]
|
||||
st = run.data["steps"][sid]
|
||||
if s.get("autopilot") == "manual-only":
|
||||
return {"action": "awaiting-operator", "item": sid, "detail": "step is manual-only"}
|
||||
if st["state"] == "failed":
|
||||
return {"action": "awaiting-operator", "item": sid, "detail": "step failed; retry or stop"}
|
||||
alt = st.get("alternative") or recommended_alt(run, sid)
|
||||
eff = effective_step(run.pack, fl, sid, alt)
|
||||
if not eff.get("reversible", True) and sid not in run.data["autopilot"]["preauthorized"]:
|
||||
return {"action": "awaiting-operator", "item": sid, "detail": "irreversible and not pre-authorized"}
|
||||
if s.get("one_of") and not alt:
|
||||
return {"action": "awaiting-operator", "item": sid, "detail": "no alternative is recommended; the operator must choose"}
|
||||
if st["state"] == "todo":
|
||||
start(run, sid, alt=alt)
|
||||
return {"action": "started", "item": sid, "detail": alt or ""}
|
||||
res = _check_result(run, eff["verify"], True, None, auto_manual=True)
|
||||
st["verification"] = res
|
||||
if res["passed"]:
|
||||
if s.get("evidence"):
|
||||
run.save()
|
||||
return {"action": "awaiting-operator", "item": sid, "detail": "evidence must be provided by the operator: " + ", ".join(s["evidence"])}
|
||||
complete(run, sid)
|
||||
return {"action": "completed", "item": sid, "detail": ""}
|
||||
cfg = _timeout_for(run, sid)
|
||||
run.save()
|
||||
if cfg and _elapsed(st["started"], now) >= cfg["seconds"]:
|
||||
out = _on_timeout(run, sid, cfg, "verification did not pass")
|
||||
if out["action"] == "push-through":
|
||||
st.update(state="done", finished=_now(), pushed_through=True)
|
||||
_deviate(run, "push-through", sid, ["verification did not pass"],
|
||||
f"autopilot timeout after {cfg['seconds']}s", _risk_text(run, sid), False)
|
||||
for dp in s.get("produces", []):
|
||||
rec = run.data["datapoints"].get(dp)
|
||||
if rec and rec["state"] == "nil":
|
||||
rec.update(state="err", error="pushed through without verification", at=_now())
|
||||
run.save()
|
||||
return out
|
||||
return {"action": "waiting", "item": sid, "detail": "verification did not pass yet"}
|
||||
|
||||
|
||||
def _autopilot_gate(run: Run, gid: str, now: datetime) -> dict:
|
||||
g = flow(run).gates[gid]
|
||||
gs = run.data["gates"][gid]
|
||||
gs.setdefault("started", _now())
|
||||
gs["results"] = {}
|
||||
gs["state"] = "open"
|
||||
for i in range(len(g["checks"])):
|
||||
gate_check(run, gid, i, confirm=True, _auto=True)
|
||||
if gs["state"] == "passed":
|
||||
return {"action": "completed", "item": gid, "detail": "gate passed"}
|
||||
cfg = _timeout_for(run, gid)
|
||||
run.save()
|
||||
if cfg and _elapsed(gs["started"], now) >= cfg["seconds"]:
|
||||
out = _on_timeout(run, gid, cfg, "gate did not pass")
|
||||
if out["action"] == "push-through":
|
||||
failed = [f"check {i} not passed" for i, r in gs["results"].items() if not r["passed"]]
|
||||
gs.update(state="waived", waiver={"by_actor": "autopilot", "who": run.data["operator"],
|
||||
"reason": f"push-through after {cfg['seconds']}s", "at": _now()})
|
||||
_deviate(run, "push-through", gid, failed, f"autopilot timeout after {cfg['seconds']}s", _risk_text(run, gid), False)
|
||||
run.save()
|
||||
return out
|
||||
return {"action": "waiting", "item": gid, "detail": "gate did not pass yet"}
|
||||
|
|
@ -151,6 +151,11 @@ def _check_refs(m: _Model, data: dict) -> None:
|
|||
m.errs.append(f"datapoint {d['id']}: produced_by names unknown step {d['produced_by']}")
|
||||
if d.get("confirmed_by") and d["confirmed_by"] not in steps | gates:
|
||||
m.errs.append(f"datapoint {d['id']}: confirmed_by names unknown step or gate")
|
||||
for g_id in gates: # a gate that requires a datapoint to be ok cannot be what confirms it
|
||||
for ck in m.flow[g_id]["checks"]:
|
||||
for d in ck.get("datapoints_ok", []):
|
||||
if dps.get(d, {}).get("confirmed_by") == g_id:
|
||||
m.errs.append(f"gate {g_id}: checks datapoint {d} that it is itself confirmed_by (circular)")
|
||||
for a in data.get("artefacts", []):
|
||||
if a["location"]["datapoint"] not in dps:
|
||||
m.errs.append(f"artefact {a['id']}: location names unknown datapoint")
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ datapoints:
|
|||
- {id: weather, label: Tonight's weather, type: enum, values: [clear, fog, storm], default: fog, dimensions: {kind: condition}, ask_in: dusk}
|
||||
- {id: motto, label: Motto, type: text, default: Steady light for those at sea, dimensions: {actor: keeper, kind: identity}, ask_in: dusk}
|
||||
- {id: logdir, label: Scratch folder, type: path, pattern: "/tmp/rtut-[a-z0-9-]+", default: /tmp/rtut-lighthouse, dimensions: {kind: location, subsystem: logbook}, ask_in: dusk}
|
||||
- {id: lamp_lit, label: Lamp is lit, type: bool, dimensions: {actor: keeper, kind: state}, produced_by: light-lamp, confirmed_by: lamp-check}
|
||||
- {id: lamp_lit, label: Lamp is lit, type: bool, dimensions: {actor: keeper, kind: state}, produced_by: light-lamp, confirmed_by: light-lamp}
|
||||
|
||||
stages:
|
||||
- id: dusk
|
||||
|
|
|
|||
316
tests/test_run2.py
Normal file
316
tests/test_run2.py
Normal file
|
|
@ -0,0 +1,316 @@
|
|||
import copy
|
||||
import subprocess
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from rtut import pack as P, run as R1, run2 as R
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
SRC = yaml.safe_load((ROOT / "runbooks/showcase-lighthouse/pack.v0.2.yaml").read_text())
|
||||
PHRASE = "I understand this cannot be undone"
|
||||
|
||||
|
||||
def mkpack(tmp_path, mutate=None):
|
||||
d = copy.deepcopy(SRC)
|
||||
if mutate:
|
||||
mutate(d)
|
||||
f = tmp_path / "runbooks" / "lh" / "pack.yaml"
|
||||
f.parent.mkdir(parents=True, exist_ok=True)
|
||||
f.write_text(yaml.safe_dump(d, sort_keys=False))
|
||||
pk = P.load(f)
|
||||
assert pk.ok, pk.errors
|
||||
return pk
|
||||
|
||||
|
||||
_SCRATCH: list[str] = []
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _cleanup_scratch():
|
||||
yield
|
||||
import shutil
|
||||
while _SCRATCH:
|
||||
shutil.rmtree(_SCRATCH.pop(), ignore_errors=True)
|
||||
|
||||
|
||||
def new(tmp_path, mutate=None, **kw):
|
||||
pk = mkpack(tmp_path, mutate)
|
||||
logdir = f"/tmp/rtut-{uuid.uuid4().hex[:10]}"
|
||||
_SCRATCH.append(logdir)
|
||||
vals = {"logdir": logdir, **kw.pop("values", {})}
|
||||
return R.create(pk, "tester", vals, **kw)
|
||||
|
||||
|
||||
def do_step(r, sid, alt=None, phrase=None, manual=True, run_cmd=True, evidence=None):
|
||||
"""What an operator does: start, run the command, check, mark done."""
|
||||
R.start(r, sid, alt=alt, phrase=phrase)
|
||||
cmd = R.command_for(r, sid)
|
||||
if cmd and run_cmd:
|
||||
subprocess.run(cmd, shell=True, check=True, capture_output=True)
|
||||
R.verify(r, sid, confirm=True, manual_ok=manual)
|
||||
R.complete(r, sid, evidence)
|
||||
|
||||
|
||||
def pass_gate(r, gid):
|
||||
for i, ck in enumerate(R.flow(r).gates[gid]["checks"]):
|
||||
R.gate_check(r, gid, i, confirm=True, manual_ok=True)
|
||||
|
||||
|
||||
# ---- datapoints -----------------------------------------------------------------------------------
|
||||
|
||||
def test_create_sets_datapoint_states(tmp_path):
|
||||
r = new(tmp_path)
|
||||
dp = r.data["datapoints"]
|
||||
assert dp["keeper_name"]["state"] == "set" and dp["keeper_name"]["value"] == "Ada"
|
||||
assert dp["lamp_lit"]["state"] == "nil"
|
||||
assert r.data["params"]["weather"] == "fog"
|
||||
|
||||
|
||||
def test_invalid_datapoint_is_err_and_not_stored(tmp_path):
|
||||
r = new(tmp_path)
|
||||
assert R.set_datapoint(r, "weather", "hail") == "err"
|
||||
assert "value" not in r.data["datapoints"]["weather"] and "weather" not in r.data["params"]
|
||||
assert R.set_datapoint(r, "weather", "storm") == "set"
|
||||
assert R.set_datapoint(r, "logdir", "/etc") == "err"
|
||||
assert R.set_datapoint(r, "motto", 'x"; rm -rf /') == "err"
|
||||
with pytest.raises(R.RunError):
|
||||
R.create(mkpack(tmp_path), "t", {"weather": "hail"})
|
||||
|
||||
|
||||
# ---- readiness, recommendation, parallelism, alternatives ---------------------------------------------
|
||||
|
||||
def test_recommended_next_is_first_ready_in_canonical_order_and_others_are_parallel(tmp_path):
|
||||
r = new(tmp_path)
|
||||
na = R.next_actions(r)
|
||||
assert na["recommended"] == "read-the-sign"
|
||||
assert na["ready"] == ["read-the-sign", "light-lamp", "read-the-sky"] # parallel, canonical order
|
||||
blocked = {b["id"]: b["unmet"] for b in na["blocked"]}
|
||||
assert "lamp-check" in blocked and "warn" in blocked and "close-log" in blocked
|
||||
R.start(r, "light-lamp") # any ready item may be taken
|
||||
assert R.next_actions(r)["recommended"] == "light-lamp" # a running item comes first
|
||||
|
||||
|
||||
def test_alternatives_recommended_by_datapoint_and_explicit_choice(tmp_path):
|
||||
for weather, alt in (("fog", "sound-horn"), ("clear", "wave-flags"), ("storm", "shelter")):
|
||||
r = new(tmp_path, values={"weather": weather})
|
||||
assert R.recommended_alt(r, "warn") == alt
|
||||
r = new(tmp_path)
|
||||
for s in ("read-the-sign", "light-lamp", "read-the-sky"):
|
||||
do_step(r, s)
|
||||
pass_gate(r, "lamp-check")
|
||||
with pytest.raises(R.RunError, match="choose one of"):
|
||||
R.start(r, "warn", alt="dance")
|
||||
R.start(r, "warn", alt="wave-flags") # operator may override the recommendation
|
||||
assert R.command_for(r, "warn").startswith("echo") and "signals with flags" in R.command_for(r, "warn")
|
||||
assert r.data["steps"]["warn"]["alternative"] == "wave-flags"
|
||||
|
||||
|
||||
def test_datapoint_ok_via_confirmed_by_and_gate_flow(tmp_path):
|
||||
r = new(tmp_path)
|
||||
assert R.unmet(r, "lamp-check") # stage steps not done yet
|
||||
do_step(r, "light-lamp")
|
||||
dp = r.data["datapoints"]["lamp_lit"]
|
||||
assert dp["state"] == "ok" and dp["how"] # produced, then confirmed by the step's own check
|
||||
for s in ("read-the-sign", "read-the-sky"):
|
||||
do_step(r, s)
|
||||
assert R.unmet(r, "lamp-check") == []
|
||||
R.gate_check(r, "lamp-check", 0, manual_ok=True)
|
||||
assert r.data["gates"]["lamp-check"]["state"] == "open" # not all checks evaluated yet
|
||||
R.gate_check(r, "lamp-check", 1) # datapoints_ok is automatic
|
||||
assert r.data["gates"]["lamp-check"]["state"] == "passed"
|
||||
assert "warn" in R.next_actions(r)["ready"]
|
||||
|
||||
|
||||
def test_failed_step_marks_produced_datapoint_err_and_retry_clears_it(tmp_path):
|
||||
r = new(tmp_path)
|
||||
R.start(r, "light-lamp"); R.fail(r, "light-lamp", "no oil")
|
||||
assert r.data["datapoints"]["lamp_lit"]["state"] == "err"
|
||||
R.retry(r, "light-lamp")
|
||||
assert r.data["datapoints"]["lamp_lit"]["state"] == "nil"
|
||||
|
||||
|
||||
# ---- overrides, waivers, deviations ------------------------------------------------------------------
|
||||
|
||||
def test_override_needs_reason_and_acknowledged_risk_and_is_recorded(tmp_path):
|
||||
r = new(tmp_path)
|
||||
with pytest.raises(R.RunError, match="Risk:"):
|
||||
R.start(r, "warn", alt="sound-horn") # gate not passed: names unmet + risk
|
||||
with pytest.raises(R.RunError, match="acknowledge"):
|
||||
R.start(r, "warn", alt="sound-horn", override_reason="need to see the horn text")
|
||||
R.start(r, "warn", alt="sound-horn", override_reason="need to see the horn text", acknowledge_risk=True)
|
||||
dev = r.data["deviations"][0]
|
||||
assert dev["kind"] == "override" and dev["item"] == "warn" and dev["mode"] == "manual"
|
||||
assert any("lamp-check" in u for u in dev["unmet"]) and dev["risk_acknowledged"] and dev["reason"]
|
||||
assert "not complete" in dev["unmet"][0]
|
||||
|
||||
|
||||
def test_waiver_only_where_declared_with_reason_and_recorded(tmp_path):
|
||||
r = new(tmp_path)
|
||||
for s in ("read-the-sign", "light-lamp", "read-the-sky"):
|
||||
do_step(r, s)
|
||||
R.gate_check(r, "lamp-check", 0, manual_ok=False)
|
||||
R.gate_check(r, "lamp-check", 1)
|
||||
assert r.data["gates"]["lamp-check"]["state"] == "failed"
|
||||
with pytest.raises(R.RunError, match="reason"):
|
||||
R.waive(r, "lamp-check", " ")
|
||||
R.waive(r, "lamp-check", "harbour master accepts the risk")
|
||||
assert r.data["gates"]["lamp-check"]["state"] == "waived"
|
||||
assert r.data["deviations"][-1]["kind"] == "waiver" and "check 0 not passed" in r.data["deviations"][-1]["unmet"]
|
||||
assert "warn" in R.next_actions(r)["ready"] # a waived gate unblocks dependents
|
||||
with pytest.raises(R.RunError, match="not waivable"):
|
||||
R.waive(r, "night-on-record", "try")
|
||||
|
||||
|
||||
def test_deviations_block_exercise_but_not_finalize(tmp_path):
|
||||
r = new(tmp_path)
|
||||
R.start(r, "warn", alt="sound-horn", override_reason="probe", acknowledge_risk=True)
|
||||
r.data["steps"]["warn"]["state"] = "failed"
|
||||
assert r.data["deviations"]
|
||||
with pytest.raises(R1.RunError, match="deviations"):
|
||||
for k in r.data["steps"]:
|
||||
r.data["steps"][k]["state"] = "done"
|
||||
R1.exercise_update(r)
|
||||
|
||||
|
||||
# ---- confirmation styles ------------------------------------------------------------------------------
|
||||
|
||||
def test_irreversible_step_needs_the_typed_phrase(tmp_path):
|
||||
r = new(tmp_path)
|
||||
assert R.confirmation_style(r, "read-the-sign") == "confirm" and R.confirmation_style(r, "close-log") == "phrase"
|
||||
for s in ("read-the-sign", "light-lamp", "read-the-sky"):
|
||||
do_step(r, s)
|
||||
pass_gate(r, "lamp-check")
|
||||
do_step(r, "warn")
|
||||
do_step(r, "write-logbook")
|
||||
with pytest.raises(R.RunError, match="confirmation phrase"):
|
||||
R.start(r, "close-log")
|
||||
with pytest.raises(R.RunError, match="confirmation phrase"):
|
||||
R.start(r, "close-log", phrase="yes")
|
||||
R.start(r, "close-log", phrase=PHRASE)
|
||||
assert r.data["steps"]["close-log"]["authorized_by"] == "tester"
|
||||
|
||||
|
||||
def test_preauthorization_only_for_irreversible_steps(tmp_path):
|
||||
with pytest.raises(R.RunError, match="only irreversible"):
|
||||
new(tmp_path, mode="autopilot", preauthorize=["light-lamp"])
|
||||
|
||||
|
||||
# ---- autopilot ------------------------------------------------------------------------------------------------
|
||||
|
||||
def drive(r, ticks=60, now=None, operator=None):
|
||||
log = []
|
||||
for _ in range(ticks):
|
||||
out = R.autopilot_tick(r, now)
|
||||
log.append(out)
|
||||
if out["action"] in ("finished", "stopped", "awaiting-operator", "waiting"):
|
||||
break
|
||||
if out["action"] == "started" and operator:
|
||||
operator(out["item"]) # the operator runs the step's command while autopilot confirms
|
||||
return log
|
||||
|
||||
|
||||
def test_autopilot_confirms_when_preconditions_are_met_and_waits_for_the_operator_on_manual_only(tmp_path):
|
||||
r = new(tmp_path, mode="autopilot")
|
||||
|
||||
def operator(sid):
|
||||
cmd = R.command_for(r, sid)
|
||||
if cmd:
|
||||
subprocess.run(cmd, shell=True, check=True, capture_output=True)
|
||||
|
||||
log = drive(r, operator=operator)
|
||||
assert log[-1] == {"action": "awaiting-operator", "item": "close-log", "detail": "step is manual-only"}
|
||||
done = [x["item"] for x in log if x["action"] == "completed"]
|
||||
assert done[:3] == ["read-the-sign", "light-lamp", "read-the-sky"] and "lamp-check" in done and "write-logbook" in done
|
||||
assert r.data["steps"]["warn"]["alternative"] == "sound-horn" # recommended by the fog datapoint
|
||||
assert r.data["gates"]["lamp-check"]["state"] == "passed"
|
||||
assert r.data["steps"]["read-the-sign"]["verification"].get("auto_confirmed")
|
||||
assert r.data["deviations"] == []
|
||||
R.start(r, "close-log", phrase=PHRASE) # the operator takes over the last step
|
||||
subprocess.run(R.command_for(r, "close-log"), shell=True, check=True, capture_output=True)
|
||||
R.verify(r, "close-log", confirm=True); R.complete(r, "close-log")
|
||||
assert R.autopilot_tick(r)["action"] == "completed" # gate night-on-record
|
||||
assert R.autopilot_tick(r)["action"] == "finished"
|
||||
R1.finalize(r, "succeeded")
|
||||
assert r.data["outcome"] == "succeeded" and not r.data["deviations"]
|
||||
|
||||
|
||||
def test_autopilot_timeout_stop(tmp_path):
|
||||
def m(d):
|
||||
d["timeout"] = {"seconds": 30, "on_timeout": "stop"}
|
||||
r = new(tmp_path, m, mode="autopilot")
|
||||
log = drive(r) # write-logbook's file check cannot pass: nobody ran its command
|
||||
assert log[-1]["action"] == "waiting" and log[-1]["item"] == "write-logbook"
|
||||
late = datetime.now(timezone.utc) + timedelta(seconds=31)
|
||||
out = R.autopilot_tick(r, late)
|
||||
assert out["action"] == "stopped" and "timed out after 30s" in out["detail"]
|
||||
assert R.autopilot_tick(r)["action"] == "stopped" # stays stopped
|
||||
assert r.data["deviations"] == [] # stopping is not a deviation
|
||||
|
||||
|
||||
def test_autopilot_push_through_records_deviation_and_moves_on(tmp_path):
|
||||
def m(d):
|
||||
d["timeout"] = {"seconds": 30, "on_timeout": "push-through"}
|
||||
r = new(tmp_path, m, mode="autopilot")
|
||||
drive(r)
|
||||
late = datetime.now(timezone.utc) + timedelta(seconds=31)
|
||||
out = R.autopilot_tick(r, late)
|
||||
assert out["action"] == "push-through" and out["item"] == "write-logbook"
|
||||
assert r.data["steps"]["write-logbook"]["state"] == "done" and r.data["steps"]["write-logbook"]["pushed_through"]
|
||||
dev = r.data["deviations"][0]
|
||||
assert dev["kind"] == "push-through" and dev["mode"] == "autopilot" and dev["risk_acknowledged"] is False
|
||||
nxt = R.autopilot_tick(r, late) # continues to the manual-only step
|
||||
assert nxt["item"] == "close-log" and nxt["action"] == "awaiting-operator"
|
||||
with pytest.raises(R1.RunError, match="deviations"):
|
||||
for k in r.data["steps"]:
|
||||
r.data["steps"][k]["state"] = "done"
|
||||
R1.exercise_update(r)
|
||||
|
||||
|
||||
def test_autopilot_gate_push_through_and_stop(tmp_path):
|
||||
def m(d):
|
||||
d["stages"][0]["gates"][0]["checks"].append({"done_when": "always fails", "command": "false", "expect": "exit-0"})
|
||||
d["stages"][0]["gates"][0]["timeout"] = {"seconds": 10, "on_timeout": "push-through"}
|
||||
r = new(tmp_path, m, mode="autopilot")
|
||||
drive(r)
|
||||
assert r.data["gates"]["lamp-check"]["state"] == "failed" # all checks evaluated, one failing
|
||||
out = R.autopilot_tick(r, datetime.now(timezone.utc) + timedelta(seconds=11))
|
||||
assert out["action"] == "push-through" and r.data["gates"]["lamp-check"]["state"] == "waived"
|
||||
assert r.data["deviations"][0]["item"] == "lamp-check" and r.data["deviations"][0]["kind"] == "push-through"
|
||||
|
||||
|
||||
def test_autopilot_never_confirms_irreversible_steps_unless_preauthorized(tmp_path):
|
||||
def m(d):
|
||||
for st in d["stages"]:
|
||||
for s in st.get("steps", []):
|
||||
s.pop("autopilot", None) # allow automation of close-log
|
||||
r = new(tmp_path, m, mode="autopilot")
|
||||
drive(r, operator=lambda sid: (subprocess.run(R.command_for(r, sid), shell=True, check=True, capture_output=True) if R.command_for(r, sid) else None))
|
||||
last = R.autopilot_tick(r)
|
||||
assert last["action"] == "awaiting-operator" and "not pre-authorized" in last["detail"]
|
||||
r2 = new(tmp_path, m, mode="autopilot", preauthorize=["close-log"])
|
||||
drive(r2, operator=lambda sid: (subprocess.run(R.command_for(r2, sid), shell=True, check=True, capture_output=True) if R.command_for(r2, sid) else None))
|
||||
assert r2.data["steps"]["close-log"]["state"] in ("running", "done")
|
||||
assert r2.data["steps"]["close-log"]["authorized_by"] == "tester"
|
||||
|
||||
|
||||
def test_manual_mode_rejects_autopilot_and_autopilot_rejects_overrides(tmp_path):
|
||||
with pytest.raises(R.RunError, match="autopilot run"):
|
||||
R.autopilot_tick(new(tmp_path))
|
||||
r = new(tmp_path, mode="autopilot")
|
||||
with pytest.raises(R.RunError, match="manual mode"):
|
||||
R.start(r, "warn", alt="sound-horn", override_reason="x", acknowledge_risk=True)
|
||||
|
||||
|
||||
def test_finished_v02_run_is_read_only_and_scores_count_it(tmp_path):
|
||||
r = new(tmp_path)
|
||||
R1.finalize(r, "abandoned")
|
||||
with pytest.raises(R.RunError, match="finished"):
|
||||
R.start(r, "read-the-sign")
|
||||
with pytest.raises(R.RunError, match="finished"):
|
||||
R.set_datapoint(r, "weather", "clear")
|
||||
assert R1.scorecard(r.pack)["abandoned"] == 1
|
||||
assert R1.load(r.pack, r.data["run_id"]).data["spec"] == "runbook-run/v0.2" # reloads through the shared loader
|
||||
|
|
@ -35,9 +35,10 @@ def test_showcase_v02_is_valid_and_loads_through_pack_load():
|
|||
assert pk.spec == "runbook-pack/v0.2"
|
||||
|
||||
|
||||
def test_v02_packs_cannot_be_run_yet():
|
||||
with pytest.raises(R.RunError, match="cannot be run yet"):
|
||||
R.create(P.load(FILE), "tester")
|
||||
def test_v02_packs_run_through_run_create():
|
||||
import uuid
|
||||
r = R.create(P.load(FILE), "tester", {"logdir": f"/tmp/rtut-{uuid.uuid4().hex[:8]}"})
|
||||
assert r.data["spec"] == "runbook-run/v0.2" and r.data["mode"] == "manual"
|
||||
|
||||
|
||||
CASES = [
|
||||
|
|
@ -104,3 +105,8 @@ def test_legacy_v02_needs_only_schema():
|
|||
d = {"spec": "runbook-pack/v0.2", "id": "x.y", "title": "Legacy x", "owner": "o",
|
||||
"exercise_status": "unexercised", "engine": "legacy-command", "legacy": {"command": ["true"]}}
|
||||
assert v2.validate(d) == []
|
||||
|
||||
|
||||
def test_gate_cannot_confirm_a_datapoint_it_requires():
|
||||
out = errs(lambda d: next(x for x in d["datapoints"] if x["id"] == "lamp_lit").update(confirmed_by="lamp-check"))
|
||||
assert any("circular" in e for e in out), out
|
||||
|
|
|
|||
|
|
@ -105,7 +105,7 @@ state_hub_task_id: "6740457f-81a7-5761-a768-b9b0faee7552"
|
|||
scratch logbook under /tmp/rtut-*). `pack.yaml` (v0.1) runs today and is the test bed for
|
||||
UI refinement. `pack.v0.2.yaml` expresses the same story in the proposed v0.2 model
|
||||
(stages, parallel steps, alternatives, gates, actors, datapoints, artefacts, irreversible
|
||||
confirmation). It now validates against the v0.2 schema and semantic checks. It cannot be run until RBT-WP-0007 lands.
|
||||
confirmation). It validates against the v0.2 schema and semantic checks and can be run by the v0.2 engine (`rtut/run2.py`); a UI for it is RBT-WP-0004.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
|
|
|
|||
|
|
@ -91,12 +91,12 @@ step instead of the top of the page.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0004-T06
|
||||
status: wait
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "261bf74f-f5cd-55bb-9a96-12767418c980"
|
||||
```
|
||||
|
||||
Depends on RBT-WP-0007. The serial guided flow is the main interface: stages with
|
||||
Unblocked: RBT-WP-0007 (engine) is finished. The serial guided flow is the main interface: stages with
|
||||
progress, the recommended next action, other ready actions, framing shown at the
|
||||
start of each stage, and gates shown at stage exit.
|
||||
|
||||
|
|
@ -104,12 +104,12 @@ start of each stage, and gates shown at stage exit.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0004-T07
|
||||
status: wait
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "16db09c7-7707-5d17-aab3-5d8eab2c1fce"
|
||||
```
|
||||
|
||||
Depends on RBT-WP-0007. A random-access table of datapoints (with `nil/set/err/ok`
|
||||
Unblocked: RBT-WP-0007 (engine) is finished. A random-access table of datapoints (with `nil/set/err/ok`
|
||||
state) grouped or filtered by dimension (actor, role, privilege, subsystem, kind),
|
||||
editable independently of the flow, plus the artefacts view.
|
||||
|
||||
|
|
@ -117,12 +117,12 @@ editable independently of the flow, plus the artefacts view.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0004-T08
|
||||
status: wait
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "23d3f5cc-e844-5077-9a4a-0435c189d98b"
|
||||
```
|
||||
|
||||
Depends on RBT-WP-0007. State transparency at a glance: stage progress, gate states
|
||||
Unblocked: RBT-WP-0007 (engine) is finished. State transparency at a glance: stage progress, gate states
|
||||
and waivers, datapoint completeness by dimension.
|
||||
|
||||
## Visual design
|
||||
|
|
@ -154,12 +154,12 @@ Confirm button for reversible steps, typed phrase for irreversible ones, per the
|
|||
|
||||
```task
|
||||
id: RBT-WP-0004-T11
|
||||
status: wait
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "99320417-8261-556b-8932-4947cfd8ce11"
|
||||
```
|
||||
|
||||
Depends on RBT-WP-0007 T06-T07. Choose manual or autopilot per run (with timeout and
|
||||
Unblocked: RBT-WP-0007 T06-T07 are finished. Choose manual or autopilot per run (with timeout and
|
||||
push-through controls). In manual mode, an override dialog lists unmet preconditions and
|
||||
the downstream risk before continuing; deviations show in the overview.
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ a dimension view and an overview from the same run data.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "55dd86f9-5f89-59a2-8a0a-85fbedd95483"
|
||||
```
|
||||
|
|
@ -35,7 +35,7 @@ canonical order; explain what a blocked item waits for.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "571e8685-6eeb-54ce-bb1a-4616ea951f81"
|
||||
```
|
||||
|
|
@ -47,7 +47,7 @@ how `ok` is reached (check or recorded manual confirmation).
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "707cca55-24a1-5355-8722-10740c5a2862"
|
||||
```
|
||||
|
|
@ -59,7 +59,7 @@ exit, record waivers with who, when and why.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "73307bb9-2739-563b-92c7-a034a360b444"
|
||||
```
|
||||
|
|
@ -70,7 +70,7 @@ state_hub_task_id: "73307bb9-2739-563b-92c7-a034a360b444"
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "0933ff4a-8478-50d9-aa99-991948bc2379"
|
||||
```
|
||||
|
|
@ -82,7 +82,7 @@ engine so no renderer can skip it.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T06
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "1656597c-3748-5480-b136-e2872436bab4"
|
||||
```
|
||||
|
|
@ -96,7 +96,7 @@ are never auto-confirmed without a recorded pre-authorization.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T07
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "0ef958ef-88b6-565b-8797-df730cf11ad3"
|
||||
```
|
||||
|
|
@ -107,6 +107,15 @@ deviation. Waivers only where a gate declares `waivable`. Autopilot timeouts
|
|||
(`stop` or `push-through`) record a deviation and, for push-through, move on. A run with
|
||||
any deviation is flagged and can never exercise a pack.
|
||||
|
||||
## Delivered 2026-09-29
|
||||
|
||||
`rtut/flow.py` and `rtut/run2.py`, documented in `docs/run-spec.md` (run record v0.2) and covered by
|
||||
`tests/test_run2.py` (25 tests: readiness and recommendation, parallel items, alternatives, datapoint
|
||||
states, gates, overrides, waivers, confirmation phrase, autopilot confirm / stop / push-through,
|
||||
manual-only and pre-authorization, read-only finished runs). Two spec corrections came out of it:
|
||||
a gate cannot require a datapoint it is itself `confirmed_by` (validator rule), and `run.finalize`
|
||||
also requires gates passed or waived. Not done here: any UI or CLI for v0.2 runs (RBT-WP-0004).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The old console's stages, gates and datapoints can be expressed and computed
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue