Adopt spec v0.2: schema, semantic validator, operating modes; workplans updated
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: sonnet Assistant-Process: 295952@bnt-lap001 Assistant-Session: e93f64ad-516c-46eb-9666-aad8d300c477
This commit is contained in:
parent
b6868a3a58
commit
79996ab249
15 changed files with 1090 additions and 64 deletions
|
|
@ -4,7 +4,7 @@ A user interface specialized to present runbook tutorials that with parameters
|
|||
can guide through actual IT administration and operation tasks.
|
||||
|
||||
- **Intent:** `INTENT.md` · **Scope:** `SCOPE.md`
|
||||
- **Pack specification:** `docs/pack-spec.md`, schema in `schema/`
|
||||
- **Pack specification:** `docs/pack-spec.md` (v0.1, runnable) and `docs/pack-spec-v0.2.md` (v0.2: stages, gates, datapoints, actors, modes; validates, not yet runnable). Schemas in `schema/`. UX findings: `docs/ux-review-2026-09-29.md`
|
||||
- **Packs live where the procedure lives:** `runbooks/<pack>/pack.yaml` in the owning repo.
|
||||
Examples: `net-kingdom/runbooks/`, and here `runbooks/example-hello/` and
|
||||
`runbooks/showcase-lighthouse/` (a harmless story to try the UI on).
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
# Runbook pack specification v0.2 - proposal
|
||||
# Runbook pack specification v0.2
|
||||
|
||||
Status: **proposed** (review with Bernd; RBT-WP-0002-T04). Derived from
|
||||
`docs/ux-review-2026-09-29.md`. v0.1 stays valid; v0.2 is a superset.
|
||||
Status: **adopted 2026-09-29** (Bernd's review answers are in the Decisions section).
|
||||
Derived from `docs/ux-review-2026-09-29.md`. v0.1 stays valid; v0.2 is a superset.
|
||||
Machine-checkable form: `schema/runbook-pack-v0.2.schema.json` plus the semantic
|
||||
checks in `rtut/v2.py`. The engine and UI do not run v0.2 packs yet (RBT-WP-0007).
|
||||
|
||||
## Principle
|
||||
|
||||
|
|
@ -141,23 +143,65 @@ things to keep in mind, shown before the work starts. They are not gates. v0.1
|
|||
- No embedded scripting, and no execution mode (still RBT-WP-0006).
|
||||
- No secret values anywhere.
|
||||
|
||||
## Recommendation
|
||||
## Operating modes, overrides and deviations
|
||||
|
||||
Keep `runbook-pack/v0.1` as shipped (three NetKingdom packs use it). Adopt this
|
||||
proposal as v0.2 and port the four in-repo packs, then use the console's seven
|
||||
runbooks and 14 gates as the first real v0.2 test: if the model can express them
|
||||
without special cases, it is expressive enough.
|
||||
A **run** has a mode. The mode belongs to the run, not to the pack.
|
||||
|
||||
## Open questions
|
||||
| Mode | Who confirms | Checks, gates, preconditions |
|
||||
| --- | --- | --- |
|
||||
| **manual** | the operator | shown with their state; the operator confirms each |
|
||||
| **autopilot** | the engine, when every precondition is met | evaluated by the engine; confirmation is applied automatically |
|
||||
|
||||
1. **Waivers:** allowed at all, and by whom? Proposal: only where the pack declares
|
||||
`waivable`, always with a recorded reason.
|
||||
2. **Gate checks against live systems:** may a gate run a command (as steps' verify
|
||||
do, operator-confirmed), or only read datapoint states and step results?
|
||||
Proposal: both, with the same confirmation rule.
|
||||
3. **Dimension vocabulary:** free tags first, or start from a canon list? Proposal:
|
||||
free tags, with a `dimensions` registry file that packs may reference, promoted
|
||||
to canon later.
|
||||
4. **Nesting depth:** three levels enough?
|
||||
5. **Datapoint `ok`:** who confirms it, the operator or a check? Proposal: either;
|
||||
a check makes it `ok`, a manual confirmation is recorded as such.
|
||||
### Manual mode: override
|
||||
|
||||
If preconditions are not met the operator may **override** and continue anyway.
|
||||
Doing so must (1) show a dialog that names the unmet preconditions and the
|
||||
downstream risk (`downstream_risk` text declared on the step or gate, else a
|
||||
generic warning), (2) require an explicit confirmation, and (3) record a
|
||||
**deviation** in the run.
|
||||
|
||||
### Autopilot mode: timeout and push-through
|
||||
|
||||
- A step or gate may declare `timeout: {seconds: N, on_timeout: stop|push-through}`;
|
||||
a run may set a default. Without a timeout, autopilot waits.
|
||||
- `stop` (default): on timeout the run stops and reports.
|
||||
- `push-through`: on timeout the engine confirms the item, records a deviation,
|
||||
and moves on, ignoring the problem. This gathers the most information about
|
||||
problems in one run and reduces refinement runs.
|
||||
- Irreversible steps are never confirmed automatically unless the run was started
|
||||
with an explicit, recorded pre-authorization naming those steps (and the pack's
|
||||
`confirm_phrase`). Push-through does not bypass that rule. This is a safety
|
||||
default, open to review.
|
||||
- A step may declare `autopilot: manual-only` to forbid automation entirely.
|
||||
|
||||
### Deviations
|
||||
|
||||
Every override, waiver and push-through is a **deviation** in the run record:
|
||||
`kind` (`override|waiver|push-through`), `item` (step or gate id), `mode`,
|
||||
`unmet` (what was not satisfied), `risk_acknowledged`, `reason` (required for
|
||||
overrides and waivers; automatic text for push-through), `who`, `at`. A run with
|
||||
deviations is marked so in the overview and can never exercise a pack. A **waiver** is
|
||||
declared in advance (`waivable` on a gate); an **override** is not.
|
||||
|
||||
## Decisions (2026-09-29, Bernd)
|
||||
|
||||
1. **Waivers:** allowed only where a gate declares `waivable`, always with a reason.
|
||||
Undeclared departures are overrides (above), also recorded.
|
||||
2. **Gate checks:** a gate may run a command (operator-confirmed in manual mode) or
|
||||
read datapoint states and step results.
|
||||
3. **Dimension vocabulary:** free tags first; a shared registry may be promoted to canon.
|
||||
4. **Nesting:** at most three levels. If procedures grow beyond that, dependencies
|
||||
between runbooks are needed (see below), not deeper nesting.
|
||||
5. **Datapoint `ok`:** reached by a check, or by a recorded manual confirmation.
|
||||
Handling of missing preconditions is defined by the operating modes above.
|
||||
|
||||
## Later: dependencies between runbooks
|
||||
|
||||
Not in v0.2. When a procedure needs another runbook (for example "OpenBao operating
|
||||
path requires the SSH access pack"), packs will declare `requires_packs` with a
|
||||
required result (exercised, or a named gate passed). Tracked in RBT-WP-0006.
|
||||
|
||||
## Recommendation (carried out)
|
||||
|
||||
Keep `runbook-pack/v0.1` as shipped (three NetKingdom packs use it). Use the old
|
||||
console's seven runbooks and 14 gates as the first real v0.2 test.
|
||||
|
|
@ -9,6 +9,8 @@ from . import pack as P
|
|||
|
||||
|
||||
def _packs(root: str) -> list[P.Pack]:
|
||||
if Path(root).is_file():
|
||||
return [P.load(Path(root))]
|
||||
return [P.load(p) for p in P.discover(Path(root))]
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -26,6 +26,10 @@ class Pack:
|
|||
def id(self) -> str:
|
||||
return str(self.data.get("id", self.path.parent.name))
|
||||
|
||||
@property
|
||||
def spec(self) -> str:
|
||||
return str(self.data.get("spec", ""))
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return not self.errors
|
||||
|
|
@ -97,6 +101,9 @@ def load(path: Path) -> Pack:
|
|||
return Pack(path, {}, [f"invalid YAML: {e}"])
|
||||
if not isinstance(data, dict):
|
||||
return Pack(path, {}, ["manifest must be a mapping"])
|
||||
if data.get("spec") == "runbook-pack/v0.2":
|
||||
from . import v2
|
||||
return Pack(path, data, v2.validate(data))
|
||||
errs = _schema_errors(data)
|
||||
if not errs:
|
||||
errs = _semantic_errors(data)
|
||||
|
|
|
|||
|
|
@ -66,6 +66,8 @@ class Run:
|
|||
def create(pack: Pack, operator: str, params: dict[str, str] | None = None) -> Run:
|
||||
if not pack.ok or pack.data.get("engine") != "native":
|
||||
raise RunError("only valid native packs can be run")
|
||||
if pack.spec != "runbook-pack/v0.1":
|
||||
raise RunError(f"{pack.spec} packs cannot be run yet (RBT-WP-0007)")
|
||||
operator = _clean(operator.strip(), "operator")
|
||||
if not re.fullmatch(r"\S+", operator):
|
||||
raise RunError("operator must be one word (name or handle)")
|
||||
|
|
|
|||
|
|
@ -48,6 +48,9 @@ class App:
|
|||
d = p.data
|
||||
if not p.ok:
|
||||
rows.append(f"<li class=bad>{e(p.id)}: invalid ({e(p.errors[0])})</li>")
|
||||
elif d.get("spec") != "runbook-pack/v0.1" and d["engine"] == "native":
|
||||
rows.append(f"<li>{e(d['title'])} <span class=pill>{e(d.get('spec'))}</span> "
|
||||
"<span class=note>cannot be run yet (RBT-WP-0007)</span></li>")
|
||||
elif d["engine"] != "native":
|
||||
rows.append(f"<li>{e(d['title'])} <span class=pill>legacy console</span> "
|
||||
f"<span class=note>start with: python -m rtut launch {e(p.path)}</span></li>")
|
||||
|
|
|
|||
213
rtut/v2.py
Normal file
213
rtut/v2.py
Normal file
|
|
@ -0,0 +1,213 @@
|
|||
"""Validation for runbook-pack/v0.2: JSON Schema plus semantic checks."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from jsonschema import Draft202012Validator
|
||||
|
||||
from .pack import PLACEHOLDER, SECRET_RE
|
||||
|
||||
SCHEMA_PATH = Path(__file__).resolve().parent.parent / "schema" / "runbook-pack-v0.2.schema.json"
|
||||
MAX_DEPTH = 3
|
||||
|
||||
|
||||
def schema_errors(data: dict) -> list[str]:
|
||||
v = Draft202012Validator(json.loads(SCHEMA_PATH.read_text()))
|
||||
return [f"{'/'.join(map(str, e.absolute_path)) or '<root>'}: {e.message}"
|
||||
for e in sorted(v.iter_errors(data), key=lambda e: list(map(str, e.absolute_path)))]
|
||||
|
||||
|
||||
class _Model:
|
||||
"""Flattened view of a v0.2 pack used by the semantic checks."""
|
||||
|
||||
def __init__(self, data: dict):
|
||||
self.data = data
|
||||
self.errs: list[str] = []
|
||||
self.flow: dict[str, dict] = {} # stages, steps, gates share one id space
|
||||
self.kind: dict[str, str] = {}
|
||||
self.parent: dict[str, str | None] = {}
|
||||
self.children: dict[str, list[str]] = {}
|
||||
self.alts: dict[str, str] = {} # alternative id -> parent step id
|
||||
self.depth_ok = True
|
||||
|
||||
def add(self, id_: str, kind: str, node: dict, parent: str | None) -> None:
|
||||
if id_ in self.flow or id_ in self.alts:
|
||||
self.errs.append(f"duplicate id: {id_}")
|
||||
return # keep the first definition; never overwrite the tree
|
||||
self.flow[id_] = node
|
||||
self.kind[id_] = kind
|
||||
self.parent[id_] = parent
|
||||
if parent:
|
||||
self.children.setdefault(parent, []).append(id_)
|
||||
|
||||
def walk(self, stages: list[dict], parent: str | None, depth: int) -> None:
|
||||
for st in stages:
|
||||
if depth > MAX_DEPTH:
|
||||
self.errs.append(f"stage {st['id']}: nesting deeper than {MAX_DEPTH} levels")
|
||||
self.add(st["id"], "stage", st, parent)
|
||||
for s in st.get("steps", []):
|
||||
self.add(s["id"], "step", s, st["id"])
|
||||
for a in s.get("one_of", []):
|
||||
if a["id"] in self.flow or a["id"] in self.alts:
|
||||
self.errs.append(f"duplicate id: {a['id']}")
|
||||
self.alts[a["id"]] = s["id"]
|
||||
for g in st.get("gates", []):
|
||||
self.add(g["id"], "gate", g, st["id"])
|
||||
self.walk(st.get("stages", []), st["id"], depth + 1)
|
||||
|
||||
def resolve(self, ref: str) -> str | None:
|
||||
"""requires reference -> flow id, or None if unresolved."""
|
||||
if ref.startswith("gate:"):
|
||||
g = ref[5:]
|
||||
return g if self.kind.get(g) == "gate" else None
|
||||
return ref if self.kind.get(ref) in ("stage", "step") else None
|
||||
|
||||
|
||||
def _check_refs(m: _Model, data: dict) -> None:
|
||||
actors = {a["id"] for a in data.get("actors", [])}
|
||||
dps = {d["id"]: d for d in data.get("datapoints", [])}
|
||||
arts = {a["id"] for a in data.get("artefacts", [])}
|
||||
steps = {i for i, k in m.kind.items() if k == "step"} | set(m.alts)
|
||||
gates = {i for i, k in m.kind.items() if k == "gate"}
|
||||
stages = {i for i, k in m.kind.items() if k == "stage"}
|
||||
for label, ids in (("actor", [a["id"] for a in data.get("actors", [])]),
|
||||
("datapoint", list(dps)), ("artefact", [a["id"] for a in data.get("artefacts", [])])):
|
||||
if len(ids) != len(set(ids)):
|
||||
m.errs.append(f"duplicate {label} id")
|
||||
|
||||
def check_ck(ck: dict, where: str, uses: set[str]) -> None:
|
||||
exp = ck.get("expect", "manual")
|
||||
if exp in ("exit-0", "output-contains") and not ck.get("command"):
|
||||
m.errs.append(f"{where}: verify expect {exp} needs a command")
|
||||
if exp == "output-contains" and not ck.get("contains"):
|
||||
m.errs.append(f"{where}: verify output-contains needs 'contains'")
|
||||
for d in ck.get("datapoints_ok", []):
|
||||
if d not in dps:
|
||||
m.errs.append(f"{where}: datapoints_ok names unknown datapoint {d}")
|
||||
for t in (ck.get("command", ""), ck.get("contains", "")):
|
||||
for name in PLACEHOLDER.findall(t or ""):
|
||||
if name not in dps:
|
||||
m.errs.append(f"{where}: undefined placeholder {{{{{name}}}}}")
|
||||
elif uses and name not in uses:
|
||||
m.errs.append(f"{where}: placeholder {name} not listed in uses")
|
||||
|
||||
def check_text(text: str, where: str, uses: set[str]) -> None:
|
||||
for name in PLACEHOLDER.findall(text or ""):
|
||||
if name not in dps:
|
||||
m.errs.append(f"{where}: undefined placeholder {{{{{name}}}}}")
|
||||
elif name not in uses:
|
||||
m.errs.append(f"{where}: placeholder {name} not listed in uses")
|
||||
|
||||
for id_, node in m.flow.items():
|
||||
kind = m.kind[id_]
|
||||
if kind == "step":
|
||||
if node["actor"] not in actors:
|
||||
m.errs.append(f"step {id_}: unknown actor {node['actor']}")
|
||||
uses = set(node.get("uses", []))
|
||||
for u in uses:
|
||||
if u not in dps:
|
||||
m.errs.append(f"step {id_}: uses unknown datapoint {u}")
|
||||
for p in node.get("produces", []):
|
||||
if p not in dps and p not in arts:
|
||||
m.errs.append(f"step {id_}: produces unknown datapoint or artefact {p}")
|
||||
parts = [(node, id_)] + [(a, a["id"]) for a in node.get("one_of", [])]
|
||||
for part, pid in parts:
|
||||
check_text(part.get("command", ""), f"step {pid}", uses)
|
||||
check_text(part.get("rollback", ""), f"step {pid}", uses)
|
||||
if "verify" in part:
|
||||
check_ck(part["verify"], f"step {pid}", uses)
|
||||
if part.get("reversible", node.get("reversible", True)) is False:
|
||||
if not (node.get("confirm_phrase") or data.get("confirm_phrase")):
|
||||
m.errs.append(f"step {pid}: irreversible step needs a confirm_phrase (step or pack)")
|
||||
for k, vals in (part.get("recommended_when") or {}).items():
|
||||
if k not in dps:
|
||||
m.errs.append(f"step {pid}: recommended_when names unknown datapoint {k}")
|
||||
elif dps[k].get("values"):
|
||||
for v in vals:
|
||||
if v not in dps[k]["values"]:
|
||||
m.errs.append(f"step {pid}: recommended_when {k}={v} not in values")
|
||||
if node.get("one_of") and node.get("command"):
|
||||
m.errs.append(f"step {id_}: a step with one_of must not also have its own command")
|
||||
elif kind == "gate":
|
||||
by = (node.get("waivable") or {}).get("by")
|
||||
if by and by not in actors:
|
||||
m.errs.append(f"gate {id_}: waivable.by names unknown actor {by}")
|
||||
for ck in node["checks"]:
|
||||
check_ck(ck, f"gate {id_}", set())
|
||||
for d in dps.values():
|
||||
if d["type"] == "enum" and not d.get("values"):
|
||||
m.errs.append(f"datapoint {d['id']}: enum without values")
|
||||
dv = d.get("default")
|
||||
if dv is not None:
|
||||
if d.get("values") and dv not in d["values"]:
|
||||
m.errs.append(f"datapoint {d['id']}: default not in values")
|
||||
if d.get("pattern") and isinstance(dv, str) and not re.fullmatch(d["pattern"], dv):
|
||||
m.errs.append(f"datapoint {d['id']}: default violates pattern")
|
||||
if d.get("ask_in") and d["ask_in"] not in stages:
|
||||
m.errs.append(f"datapoint {d['id']}: ask_in names unknown stage {d['ask_in']}")
|
||||
if d.get("produced_by") and d["produced_by"] not in steps:
|
||||
m.errs.append(f"datapoint {d['id']}: produced_by names unknown step {d['produced_by']}")
|
||||
if d.get("confirmed_by") and d["confirmed_by"] not in steps | gates:
|
||||
m.errs.append(f"datapoint {d['id']}: confirmed_by names unknown step or gate")
|
||||
for a in data.get("artefacts", []):
|
||||
if a["location"]["datapoint"] not in dps:
|
||||
m.errs.append(f"artefact {a['id']}: location names unknown datapoint")
|
||||
if a["produced_by"] not in steps:
|
||||
m.errs.append(f"artefact {a['id']}: produced_by names unknown step")
|
||||
if a.get("lives_until") and a["lives_until"] not in steps:
|
||||
m.errs.append(f"artefact {a['id']}: lives_until names unknown step")
|
||||
|
||||
|
||||
def _check_graph(m: _Model) -> None:
|
||||
deps: dict[str, set[str]] = {i: set() for i in m.flow}
|
||||
for id_, node in m.flow.items():
|
||||
for ref in node.get("requires", []):
|
||||
r = m.resolve(ref)
|
||||
if r is None:
|
||||
m.errs.append(f"{m.kind[id_]} {id_}: requires unknown item {ref}")
|
||||
else:
|
||||
deps[id_].add(r)
|
||||
# a node also waits for whatever its enclosing stages wait for
|
||||
p, seen = m.parent[id_], set()
|
||||
while p and p not in seen:
|
||||
seen.add(p)
|
||||
for ref in m.flow[p].get("requires", []):
|
||||
r = m.resolve(ref)
|
||||
if r:
|
||||
deps[id_].add(r)
|
||||
p = m.parent[p]
|
||||
# a stage is finished when its contents are
|
||||
for c in m.children.get(id_, []):
|
||||
deps[id_].add(c)
|
||||
state: dict[str, int] = {}
|
||||
|
||||
def visit(n: str, path: list[str]) -> None:
|
||||
if state.get(n) == 2:
|
||||
return
|
||||
if state.get(n) == 1:
|
||||
m.errs.append("requires cycle: " + " -> ".join(path[path.index(n):] + [n]))
|
||||
return
|
||||
state[n] = 1
|
||||
for d in sorted(deps[n]):
|
||||
visit(d, path + [n])
|
||||
state[n] = 2
|
||||
|
||||
for n in sorted(deps):
|
||||
visit(n, [])
|
||||
|
||||
|
||||
def validate(data: dict) -> list[str]:
|
||||
errs = schema_errors(data)
|
||||
if errs:
|
||||
return errs
|
||||
if data["engine"] == "legacy-command":
|
||||
return []
|
||||
m = _Model(data)
|
||||
m.walk(data["stages"], None, 1)
|
||||
_check_refs(m, data)
|
||||
_check_graph(m)
|
||||
if SECRET_RE.search(json.dumps(data)):
|
||||
m.errs.append("contains secret-looking marker")
|
||||
return sorted(set(m.errs))
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# DRAFT - not loaded by the engine. Written against docs/pack-spec-v0.2-proposal.md.
|
||||
# DRAFT - not loaded by the engine. Written against docs/pack-spec-v0.2.md.
|
||||
# Purpose: an acceptance test for the proposed model. If the model cannot express
|
||||
# this story without special cases, the proposal must change. When v0.2 is adopted
|
||||
# this file becomes pack.yaml (and the v0.1 file is retired or kept as a
|
||||
|
|
@ -55,6 +55,7 @@ stages:
|
|||
produces: [lamp_lit]
|
||||
command: echo "{{keeper_name}} lights the {{lamp_fuel}} lamp."
|
||||
reversible: true
|
||||
downstream_risk: Without a lit lamp the night watch has no purpose and ships are not warned.
|
||||
verify: {done_when: The lamp is lit, expect: manual}
|
||||
- id: read-the-sky
|
||||
title: Read the sky
|
||||
|
|
@ -69,6 +70,8 @@ stages:
|
|||
title: The lamp burns steadily
|
||||
goal: safe-navigation # quality objective; a canon standard id where one exists
|
||||
blocking: true
|
||||
timeout: {seconds: 120, on_timeout: stop}
|
||||
downstream_risk: Ships may steer by a light that is not burning.
|
||||
checks:
|
||||
- {done_when: The light is visible from the water, expect: manual}
|
||||
- {datapoints_ok: [lamp_lit]}
|
||||
|
|
@ -109,7 +112,7 @@ stages:
|
|||
owner: the-lighthouse
|
||||
requires: [warn]
|
||||
uses: [keeper_name, weather, lamp_fuel, motto, logdir]
|
||||
produces: [logbook_file]
|
||||
produces: [logbook-file]
|
||||
command: mkdir -p {{logdir}} && echo "Night watch by {{keeper_name}} - {{weather}}, {{lamp_fuel}} lamp. {{motto}}." >> {{logdir}}/logbook.txt
|
||||
reversible: true
|
||||
rollback: rm -rf {{logdir}}
|
||||
|
|
@ -122,6 +125,7 @@ stages:
|
|||
steps:
|
||||
- id: close-log
|
||||
title: Close the log for good
|
||||
autopilot: manual-only # never confirmed by the engine on its own
|
||||
actor: harbourmaster
|
||||
owner: the-lighthouse
|
||||
uses: [logdir]
|
||||
|
|
@ -137,7 +141,7 @@ stages:
|
|||
- {done_when: No files remain under the scratch folder, command: 'test ! -e {{logdir}}', expect: exit-0}
|
||||
|
||||
artefacts:
|
||||
- id: logbook_file
|
||||
- id: logbook-file
|
||||
title: The night's logbook entry
|
||||
location: {datapoint: logdir, path: logbook.txt} # a reference, never a value
|
||||
produced_by: write-logbook
|
||||
621
schema/runbook-pack-v0.2.schema.json
Normal file
621
schema/runbook-pack-v0.2.schema.json
Normal file
|
|
@ -0,0 +1,621 @@
|
|||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://runbook-tutorials.invalid/schema/runbook-pack/v0.2",
|
||||
"title": "Runbook pack manifest v0.2",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"spec",
|
||||
"id",
|
||||
"title",
|
||||
"owner",
|
||||
"exercise_status",
|
||||
"engine"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"spec": {
|
||||
"const": "runbook-pack/v0.2"
|
||||
},
|
||||
"id": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9]+([.-][a-z0-9]+)*$"
|
||||
},
|
||||
"title": {
|
||||
"type": "string",
|
||||
"minLength": 3
|
||||
},
|
||||
"owner": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"outcome": {
|
||||
"type": "string"
|
||||
},
|
||||
"exercise_status": {
|
||||
"type": "string",
|
||||
"pattern": "^(unexercised|exercised [0-9]{4}-[0-9]{2}-[0-9]{2} by \\S+)$"
|
||||
},
|
||||
"engine": {
|
||||
"enum": [
|
||||
"native",
|
||||
"legacy-command"
|
||||
]
|
||||
},
|
||||
"confirm_phrase": {
|
||||
"type": "string",
|
||||
"minLength": 8,
|
||||
"description": "Typed by the operator to confirm an irreversible step."
|
||||
},
|
||||
"timeout": {
|
||||
"$ref": "#/$defs/timeout",
|
||||
"description": "Default for autopilot runs."
|
||||
},
|
||||
"legacy": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"command"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"command": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"minItems": 1
|
||||
},
|
||||
"cwd": {
|
||||
"type": "string"
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
},
|
||||
"note": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"framing": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "Things to keep in mind; shown first; not gates."
|
||||
},
|
||||
"actors": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/actor"
|
||||
}
|
||||
},
|
||||
"datapoints": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/datapoint"
|
||||
}
|
||||
},
|
||||
"stages": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"$ref": "#/$defs/stage"
|
||||
}
|
||||
},
|
||||
"artefacts": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/artefact"
|
||||
}
|
||||
},
|
||||
"ownership": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/ownership"
|
||||
}
|
||||
}
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"if": {
|
||||
"properties": {
|
||||
"engine": {
|
||||
"const": "native"
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"required": [
|
||||
"stages",
|
||||
"framing"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": {
|
||||
"properties": {
|
||||
"engine": {
|
||||
"const": "legacy-command"
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"required": [
|
||||
"legacy"
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"$defs": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9-]*$"
|
||||
},
|
||||
"dpid": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9_]*$"
|
||||
},
|
||||
"timeout": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"seconds"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"seconds": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"on_timeout": {
|
||||
"enum": [
|
||||
"stop",
|
||||
"push-through"
|
||||
],
|
||||
"default": "stop"
|
||||
}
|
||||
}
|
||||
},
|
||||
"actor": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"role",
|
||||
"description"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"id": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"role": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": {
|
||||
"type": "string"
|
||||
},
|
||||
"privileges": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"datapoint": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"label",
|
||||
"type"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"id": {
|
||||
"$ref": "#/$defs/dpid"
|
||||
},
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"type": {
|
||||
"enum": [
|
||||
"string",
|
||||
"text",
|
||||
"enum",
|
||||
"path",
|
||||
"bool",
|
||||
"int"
|
||||
]
|
||||
},
|
||||
"values": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"default": {
|
||||
"type": [
|
||||
"string",
|
||||
"integer",
|
||||
"boolean"
|
||||
]
|
||||
},
|
||||
"pattern": {
|
||||
"type": "string"
|
||||
},
|
||||
"help": {
|
||||
"type": "string"
|
||||
},
|
||||
"dimensions": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"ask_in": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"produced_by": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"confirmed_by": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"secret": {
|
||||
"const": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"check": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"done_when": {
|
||||
"type": "string",
|
||||
"minLength": 3
|
||||
},
|
||||
"command": {
|
||||
"type": "string"
|
||||
},
|
||||
"expect": {
|
||||
"enum": [
|
||||
"manual",
|
||||
"exit-0",
|
||||
"output-contains"
|
||||
]
|
||||
},
|
||||
"contains": {
|
||||
"type": "string"
|
||||
},
|
||||
"datapoints_ok": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/dpid"
|
||||
},
|
||||
"minItems": 1
|
||||
}
|
||||
},
|
||||
"anyOf": [
|
||||
{
|
||||
"required": [
|
||||
"done_when"
|
||||
]
|
||||
},
|
||||
{
|
||||
"required": [
|
||||
"datapoints_ok"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"alternative": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"title",
|
||||
"verify"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"id": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"command": {
|
||||
"type": "string"
|
||||
},
|
||||
"reversible": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"rollback": {
|
||||
"type": "string"
|
||||
},
|
||||
"verify": {
|
||||
"$ref": "#/$defs/check"
|
||||
},
|
||||
"recommended_when": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"step": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"title",
|
||||
"actor",
|
||||
"owner"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"id": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"actor": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"owner": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": {
|
||||
"type": "string"
|
||||
},
|
||||
"command": {
|
||||
"type": "string"
|
||||
},
|
||||
"reversible": {
|
||||
"type": "boolean",
|
||||
"default": true
|
||||
},
|
||||
"rollback": {
|
||||
"type": "string"
|
||||
},
|
||||
"confirm_phrase": {
|
||||
"type": "string",
|
||||
"minLength": 8
|
||||
},
|
||||
"verify": {
|
||||
"$ref": "#/$defs/check"
|
||||
},
|
||||
"requires": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"uses": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/dpid"
|
||||
}
|
||||
},
|
||||
"produces": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"evidence": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"one_of": {
|
||||
"type": "array",
|
||||
"minItems": 2,
|
||||
"items": {
|
||||
"$ref": "#/$defs/alternative"
|
||||
}
|
||||
},
|
||||
"autopilot": {
|
||||
"enum": [
|
||||
"allowed",
|
||||
"manual-only"
|
||||
],
|
||||
"default": "allowed"
|
||||
},
|
||||
"timeout": {
|
||||
"$ref": "#/$defs/timeout"
|
||||
},
|
||||
"downstream_risk": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"anyOf": [
|
||||
{
|
||||
"required": [
|
||||
"verify"
|
||||
]
|
||||
},
|
||||
{
|
||||
"required": [
|
||||
"one_of"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"gate": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"title",
|
||||
"goal",
|
||||
"checks"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"id": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"goal": {
|
||||
"type": "string",
|
||||
"minLength": 3
|
||||
},
|
||||
"blocking": {
|
||||
"type": "boolean",
|
||||
"default": true
|
||||
},
|
||||
"checks": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"$ref": "#/$defs/check"
|
||||
}
|
||||
},
|
||||
"waivable": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"by",
|
||||
"needs_reason"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"by": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"needs_reason": {
|
||||
"const": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"timeout": {
|
||||
"$ref": "#/$defs/timeout"
|
||||
},
|
||||
"downstream_risk": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"stage": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"title",
|
||||
"goal"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"id": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"goal": {
|
||||
"type": "string",
|
||||
"minLength": 3
|
||||
},
|
||||
"framing": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"requires": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/step"
|
||||
}
|
||||
},
|
||||
"gates": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/gate"
|
||||
}
|
||||
},
|
||||
"stages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/stage"
|
||||
}
|
||||
}
|
||||
},
|
||||
"anyOf": [
|
||||
{
|
||||
"required": [
|
||||
"steps"
|
||||
]
|
||||
},
|
||||
{
|
||||
"required": [
|
||||
"stages"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"artefact": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"title",
|
||||
"location",
|
||||
"produced_by"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"id": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"location": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"datapoint"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"datapoint": {
|
||||
"$ref": "#/$defs/dpid"
|
||||
},
|
||||
"path": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"produced_by": {
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"lives_until": {
|
||||
"$ref": "#/$defs/id"
|
||||
}
|
||||
}
|
||||
},
|
||||
"ownership": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"concern",
|
||||
"owner"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"concern": {
|
||||
"type": "string"
|
||||
},
|
||||
"owner": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -3,7 +3,6 @@ import shutil
|
|||
import subprocess
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from rtut import pack as P, run as R
|
||||
|
||||
|
|
@ -22,28 +21,6 @@ def test_showcase_is_valid():
|
|||
assert P.load(SRC / "pack.yaml").ok
|
||||
|
||||
|
||||
def test_v02_draft_is_wellformed_and_self_consistent():
|
||||
d = yaml.safe_load((SRC / "pack.v0.2.draft.yaml").read_text())
|
||||
assert d["spec"] == "runbook-pack/v0.2"
|
||||
actors = {a["id"] for a in d["actors"]}
|
||||
dps = {x["id"] for x in d["datapoints"]}
|
||||
steps = {}
|
||||
for st in d["stages"]:
|
||||
for s in st["steps"]:
|
||||
steps[s["id"]] = s
|
||||
assert s["actor"] in actors
|
||||
for u in s.get("uses", []):
|
||||
assert u in dps, u
|
||||
gates = {g["id"] for st in d["stages"] for g in st.get("gates", [])}
|
||||
stages = {st["id"] for st in d["stages"]}
|
||||
for st in d["stages"]:
|
||||
for r in st.get("requires", []):
|
||||
assert r.removeprefix("gate:") in gates | stages
|
||||
assert any(s.get("reversible") is False for s in steps.values())
|
||||
assert any("one_of" in s for s in steps.values())
|
||||
assert not any(k.startswith("secret") for x in d["datapoints"] for k in x)
|
||||
|
||||
|
||||
def test_scripted_playthrough(pk):
|
||||
r = R.create(pk, "tester", {"keeper": "Ada", "weather": "storm", "motto": "Light on"})
|
||||
for s in pk.data["steps"]:
|
||||
|
|
|
|||
106
tests/test_v2.py
Normal file
106
tests/test_v2.py
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
import copy
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from rtut import pack as P, run as R, v2
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
FILE = ROOT / "runbooks/showcase-lighthouse/pack.v0.2.yaml"
|
||||
GOOD = yaml.safe_load(FILE.read_text())
|
||||
|
||||
|
||||
def errs(mutate):
|
||||
d = copy.deepcopy(GOOD)
|
||||
mutate(d)
|
||||
return v2.validate(d)
|
||||
|
||||
|
||||
def step(d, sid):
|
||||
for st in d["stages"]:
|
||||
for s in st.get("steps", []):
|
||||
if s["id"] == sid:
|
||||
return s
|
||||
raise KeyError(sid)
|
||||
|
||||
|
||||
def gate(d, gid):
|
||||
return next(g for st in d["stages"] for g in st.get("gates", []) if g["id"] == gid)
|
||||
|
||||
|
||||
def test_showcase_v02_is_valid_and_loads_through_pack_load():
|
||||
pk = P.load(FILE)
|
||||
assert pk.ok, pk.errors
|
||||
assert pk.spec == "runbook-pack/v0.2"
|
||||
|
||||
|
||||
def test_v02_packs_cannot_be_run_yet():
|
||||
with pytest.raises(R.RunError, match="cannot be run yet"):
|
||||
R.create(P.load(FILE), "tester")
|
||||
|
||||
|
||||
CASES = [
|
||||
("unknown actor", lambda d: step(d, "light-lamp").update(actor="nobody"), "unknown actor"),
|
||||
("unknown datapoint in uses", lambda d: step(d, "light-lamp").update(uses=["nope"]), "uses unknown datapoint"),
|
||||
("placeholder not in uses", lambda d: step(d, "light-lamp").update(uses=["keeper_name"]), "not listed in uses"),
|
||||
("undefined placeholder", lambda d: step(d, "light-lamp").update(command="echo {{ghost}}"), "undefined placeholder"),
|
||||
("requires unknown", lambda d: step(d, "write-logbook").update(requires=["gate:ghost"]), "requires unknown item"),
|
||||
("duplicate id", lambda d: d["stages"][0]["gates"][0].update(id="dusk"), "duplicate id"),
|
||||
("irreversible without phrase", lambda d: (d.pop("confirm_phrase"), None), "confirm_phrase"),
|
||||
("waivable by unknown actor", lambda d: gate(d, "lamp-check").update(waivable={"by": "ghost", "needs_reason": True}), "unknown actor"),
|
||||
("datapoints_ok unknown", lambda d: gate(d, "lamp-check")["checks"].append({"datapoints_ok": ["ghost"]}), "unknown datapoint"),
|
||||
("ask_in unknown stage", lambda d: d["datapoints"][0].update(ask_in="ghost"), "unknown stage"),
|
||||
("artefact produced_by unknown", lambda d: d["artefacts"][0].update(produced_by="ghost"), "unknown step"),
|
||||
("recommended_when bad value", lambda d: step(d, "warn")["one_of"][0].update(recommended_when={"weather": ["hail"]}), "not in values"),
|
||||
("step with one_of and command", lambda d: step(d, "warn").update(command="echo x"), "must not also have its own command"),
|
||||
("secret marker", lambda d: d["framing"].append("token hvs.ABCDEFGHIJKLMN"), "secret-looking"),
|
||||
("exit-0 without command", lambda d: step(d, "write-logbook")["verify"].pop("command"), "needs a command"),
|
||||
("secret datapoint", lambda d: d["datapoints"][0].update(secret=True), "secret"),
|
||||
("gate without goal", lambda d: gate(d, "lamp-check").pop("goal"), "goal"),
|
||||
("waiver must need a reason", lambda d: gate(d, "lamp-check").update(waivable={"by": "harbourmaster", "needs_reason": False}), "needs_reason"),
|
||||
("bad timeout action", lambda d: gate(d, "lamp-check").update(timeout={"seconds": 5, "on_timeout": "explode"}), "on_timeout"),
|
||||
("bad autopilot value", lambda d: step(d, "close-log").update(autopilot="always"), "autopilot"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name,fn,frag", CASES, ids=[c[0] for c in CASES])
|
||||
def test_invalid(name, fn, frag):
|
||||
out = errs(fn)
|
||||
assert any(frag in e for e in out), out
|
||||
|
||||
|
||||
def test_cycle_detected():
|
||||
out = errs(lambda d: d["stages"][0].update(requires=["night"]))
|
||||
assert any("cycle" in e for e in out), out
|
||||
|
||||
|
||||
def test_child_requiring_its_parent_is_a_cycle():
|
||||
def m(d):
|
||||
step(d, "read-the-sky").update(requires=["dusk"])
|
||||
assert any("cycle" in e for e in errs(m))
|
||||
|
||||
|
||||
def test_depth_limit():
|
||||
def m(d):
|
||||
inner = {"id": "l4", "title": "Level four", "goal": "too deep", "steps": [copy.deepcopy(step(d, "read-the-sign"))]}
|
||||
inner["steps"][0]["id"] = "deep-step"
|
||||
l3 = {"id": "l3", "title": "Level three", "goal": "three", "stages": [inner]}
|
||||
l2 = {"id": "l2", "title": "Level two", "goal": "two", "stages": [l3]}
|
||||
d["stages"].append({"id": "l1", "title": "Level one", "goal": "one", "stages": [l2]})
|
||||
assert any("nesting deeper" in e for e in errs(m))
|
||||
|
||||
|
||||
def test_three_levels_allowed():
|
||||
def m(d):
|
||||
l3 = {"id": "l3", "title": "Level three", "goal": "three", "steps": [copy.deepcopy(step(d, "read-the-sign"))]}
|
||||
l3["steps"][0]["id"] = "deep-step"
|
||||
l2 = {"id": "l2", "title": "Level two", "goal": "two", "stages": [l3]}
|
||||
d["stages"].append({"id": "l1", "title": "Level one", "goal": "one", "stages": [l2]})
|
||||
assert errs(m) == []
|
||||
|
||||
|
||||
def test_legacy_v02_needs_only_schema():
|
||||
d = {"spec": "runbook-pack/v0.2", "id": "x.y", "title": "Legacy x", "owner": "o",
|
||||
"exercise_status": "unexercised", "engine": "legacy-command", "legacy": {"command": ["true"]}}
|
||||
assert v2.validate(d) == []
|
||||
|
|
@ -61,27 +61,24 @@ unsafe characters.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0002-T04
|
||||
status: progress
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
`docs/pack-spec-v0.2-proposal.md` (stages and hierarchy, dependency graph with a
|
||||
`docs/pack-spec-v0.2.md` (stages and hierarchy, dependency graph with a
|
||||
recommended serialization, datapoints with `nil/set/err/ok`, QA gates tied to stages
|
||||
and goals, actors and roles, artefacts, reversibility-based confirmation, framing).
|
||||
Review with Bernd, resolve the five open questions, and decide: freeze v0.1 as
|
||||
shipped and adopt v0.2. Supersedes the earlier plan to freeze v0.1 alone.
|
||||
Done 2026-09-29: Bernd answered the five questions; v0.2 adopted as `docs/pack-spec-v0.2.md` (with operating modes, overrides, deviations); v0.1 stays as shipped. Supersedes the earlier plan to freeze v0.1 alone.
|
||||
|
||||
## Write the v0.2 schema and validator
|
||||
|
||||
```task
|
||||
id: RBT-WP-0002-T05
|
||||
status: wait
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
After T04: `schema/runbook-pack-v0.2.schema.json`, semantic checks (acyclic
|
||||
requires, gate and datapoint references resolve, one stage per gate, no secret
|
||||
datapoints), v0.1 packs still load, tests.
|
||||
Done 2026-09-29: `schema/runbook-pack-v0.2.schema.json` and `rtut/v2.py` (acyclic requires, references resolve, nesting depth 3, irreversible steps need a confirm phrase, no secret datapoints). v0.1 packs still load. 61 tests including one negative test per rule.
|
||||
|
||||
## Port the in-repo and NetKingdom packs
|
||||
|
||||
|
|
@ -106,10 +103,9 @@ state_hub_task_id: "6740457f-81a7-5761-a768-b9b0faee7552"
|
|||
|
||||
`runbooks/showcase-lighthouse/` is a harmless interactive-fiction runbook (poem, story,
|
||||
scratch logbook under /tmp/rtut-*). `pack.yaml` (v0.1) runs today and is the test bed for
|
||||
UI refinement. `pack.v0.2.draft.yaml` expresses the same story in the proposed v0.2 model
|
||||
UI refinement. `pack.v0.2.yaml` expresses the same story in the proposed v0.2 model
|
||||
(stages, parallel steps, alternatives, gates, actors, datapoints, artefacts, irreversible
|
||||
confirmation). The draft is checked for internal consistency by tests. When v0.2 is adopted
|
||||
the validator must accept it unchanged, or the proposal changes.
|
||||
confirmation). It now validates against the v0.2 schema and semantic checks. It cannot be run until RBT-WP-0007 lands.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ priority: high
|
|||
state_hub_task_id: "8309d778-3688-5cb3-bb34-18f87b66dbf3"
|
||||
```
|
||||
|
||||
Done 2026-09-29 with Bernd. Findings and consequences are in `docs/ux-review-2026-09-29.md`; the model they imply is proposed in `docs/pack-spec-v0.2-proposal.md`.
|
||||
Done 2026-09-29 with Bernd. Findings and consequences are in `docs/ux-review-2026-09-29.md`; the model they imply is proposed in `docs/pack-spec-v0.2.md`.
|
||||
|
||||
## Build the step view
|
||||
|
||||
|
|
@ -146,6 +146,18 @@ state_hub_task_id: "ddbca5e1-dcf6-5c2a-b3e2-497d2a82ba23"
|
|||
Confirm button for reversible steps, typed phrase for irreversible ones, per the v0.2
|
||||
`reversible` field.
|
||||
|
||||
## Mode switch and override dialog
|
||||
|
||||
```task
|
||||
id: RBT-WP-0004-T11
|
||||
status: wait
|
||||
priority: high
|
||||
```
|
||||
|
||||
Depends on RBT-WP-0007 T06-T07. Choose manual or autopilot per run (with timeout and
|
||||
push-through controls). In manual mode, an override dialog lists unmet preconditions and
|
||||
the downstream risk before continuing; deviations show in the overview.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Bernd can complete the SSH tutorial pack in the UI without pasting outputs to an agent.
|
||||
|
|
|
|||
|
|
@ -56,3 +56,15 @@ state_hub_task_id: "ce82d1ae-76a3-59c8-9d26-4813e8dc9745"
|
|||
Operator feedback on a step feeds an assistant that knows the runbook and may
|
||||
diagnose the deployment (read-only access first), giving a feedback, retry,
|
||||
feedback, confirm loop. Needs decisions on access, identity and what is sent where.
|
||||
|
||||
## Dependencies between runbooks
|
||||
|
||||
```task
|
||||
id: RBT-WP-0006-T04
|
||||
status: wait
|
||||
priority: low
|
||||
```
|
||||
|
||||
Packs declare `requires_packs` with a required result (exercised, or a named gate
|
||||
passed), instead of deeper nesting (v0.2 caps nesting at three levels). Needs a decision
|
||||
on how a pack in another repo is identified and trusted.
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ a dimension view and an overview from the same run data.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T01
|
||||
status: wait
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "55dd86f9-5f89-59a2-8a0a-85fbedd95483"
|
||||
```
|
||||
|
|
@ -35,7 +35,7 @@ canonical order; explain what a blocked item waits for.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T02
|
||||
status: wait
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "571e8685-6eeb-54ce-bb1a-4616ea951f81"
|
||||
```
|
||||
|
|
@ -47,7 +47,7 @@ how `ok` is reached (check or recorded manual confirmation).
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T03
|
||||
status: wait
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "707cca55-24a1-5355-8722-10740c5a2862"
|
||||
```
|
||||
|
|
@ -59,7 +59,7 @@ exit, record waivers with who, when and why.
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T04
|
||||
status: wait
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "73307bb9-2739-563b-92c7-a034a360b444"
|
||||
```
|
||||
|
|
@ -70,7 +70,7 @@ state_hub_task_id: "73307bb9-2739-563b-92c7-a034a360b444"
|
|||
|
||||
```task
|
||||
id: RBT-WP-0007-T05
|
||||
status: wait
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "0933ff4a-8478-50d9-aa99-991948bc2379"
|
||||
```
|
||||
|
|
@ -78,6 +78,33 @@ state_hub_task_id: "0933ff4a-8478-50d9-aa99-991948bc2379"
|
|||
Derive confirm button versus typed phrase from `reversible`; enforce the phrase in the
|
||||
engine so no renderer can skip it.
|
||||
|
||||
## Operating modes: manual and autopilot
|
||||
|
||||
```task
|
||||
id: RBT-WP-0007-T06
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
A run has a mode (`docs/pack-spec-v0.2.md`). Manual: the operator confirms; every
|
||||
check, gate and precondition is shown with its state. Autopilot: the engine confirms
|
||||
when all preconditions are met. `autopilot: manual-only` steps and irreversible steps
|
||||
are never auto-confirmed without a recorded pre-authorization.
|
||||
|
||||
## Overrides, waivers, push-through and the deviation record
|
||||
|
||||
```task
|
||||
id: RBT-WP-0007-T07
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Override in manual mode when preconditions are unmet: name the unmet items and the
|
||||
declared `downstream_risk`, require explicit confirmation and a reason, record a
|
||||
deviation. Waivers only where a gate declares `waivable`. Autopilot timeouts
|
||||
(`stop` or `push-through`) record a deviation and, for push-through, move on. A run with
|
||||
any deviation is flagged and can never exercise a pack.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The old console's stages, gates and datapoints can be expressed and computed
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue