Lock and seal closed runs; feedback with the closing form; Home headlines link to their sections; drop the back link
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: sonnet Assistant-Process: 295952@bnt-lap001 Assistant-Session: e93f64ad-516c-46eb-9666-aad8d300c477
This commit is contained in:
parent
b7edb8f023
commit
711c4e1149
13 changed files with 381 additions and 42 deletions
|
|
@ -106,6 +106,20 @@ Rules the engine enforces:
|
|||
- **Modes** are `manual` and `auto-confirm`. Auto-confirm applies confirmation only; running the step's own
|
||||
command is a separate, later level ("auto-pilot with execution", RBT-WP-0006).
|
||||
|
||||
## Closed runs are locked and sealed
|
||||
|
||||
Closing a run (`finalize`) makes its record final:
|
||||
|
||||
- **Locked.** A closed record cannot be saved with changes: the engine refuses, naming what would have changed. The only later update it
|
||||
accepts is `exercise_recorded`, written by the legacy "Record pack as exercised" step. This holds for runs closed before sealing existed too.
|
||||
- **Feedback.** It locks with the run. Give it before closing, or with the closing form: `finalize(..., rating, comment)` records an optional
|
||||
rating and comment together with the outcome (validated first, so a refused close changes nothing). This keeps abandoned and failed runs from
|
||||
losing what the operator wanted to say.
|
||||
- **Sealed.** The record carries a SHA-256 `seal` of its content (excluding `seal`, `updated` and `exercise_recorded`), and the file is made
|
||||
read-only. `seal_state` reports `open`, `sealed`, `broken` (changed outside the app after closing) or `unsealed` (closed before sealing existed).
|
||||
A broken seal is called out in the UI, and such a run no longer counts as maturity evidence. This is tamper-evidence, not prevention: someone
|
||||
with file access can still edit a file, but not without being noticed.
|
||||
|
||||
## Changing a value after steps used it
|
||||
|
||||
A v0.1 run fixes its parameters when it starts. In a v0.2 run datapoints can be edited (guided flow questions, data view), but a value
|
||||
|
|
|
|||
|
|
@ -53,7 +53,8 @@ def _env_key(ctx: dict) -> str:
|
|||
|
||||
def _evidence(pack: Pack, dig: str, runs: list[R.Run]) -> dict:
|
||||
clean = [r for r in runs if r.data.get("pack_digest") == dig and r.data.get("outcome") == "succeeded"
|
||||
and not r.data.get("deviations") and r.data.get("context", {}).get("environment")]
|
||||
and not r.data.get("deviations") and r.data.get("context", {}).get("environment")
|
||||
and R.seal_state(r) != "broken"] # a record changed after closing is not evidence
|
||||
reviewed = any(rv.get("digest") == dig for rv in _reviews(pack))
|
||||
rehearsal = [r for r in clean if r.data["context"]["environment"] == "rehearsal"]
|
||||
envs = {_env_key(r.data["context"]) for r in clean}
|
||||
|
|
|
|||
68
rtut/run.py
68
rtut/run.py
|
|
@ -44,10 +44,22 @@ class Run:
|
|||
def path(self) -> Path:
|
||||
return runs_dir(self.pack) / f"{self.data['run_id']}.json"
|
||||
|
||||
def save(self) -> None:
|
||||
def save(self, allow: tuple = ()) -> None:
|
||||
"""Write the record. A closed run is locked: nothing may change except the keys named in `allow`."""
|
||||
disk = _read_json(self.path)
|
||||
if disk and disk.get("outcome"):
|
||||
keys = set(disk) | set(self.data)
|
||||
changed = {k for k in keys if disk.get(k) != self.data.get(k)} - {"updated"} - set(allow)
|
||||
if changed:
|
||||
raise RunError(f"this run is closed and locked ({disk['outcome']}); it cannot be changed "
|
||||
f"({', '.join(sorted(changed))})")
|
||||
self.data["updated"] = _now()
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if self.path.exists():
|
||||
self.path.chmod(0o644) # a sealed file is read-only; the allowed update reopens it briefly
|
||||
self.path.write_text(json.dumps(self.data, indent=2, sort_keys=True) + "\n")
|
||||
if self.data.get("outcome"):
|
||||
self.path.chmod(0o444)
|
||||
|
||||
def step_def(self, step_id: str) -> dict:
|
||||
if self.pack.spec == "runbook-pack/v0.2": # steps live in stages; alternatives are applied
|
||||
|
|
@ -280,12 +292,19 @@ def exercise_update(run: Run, today: date | None = None) -> tuple[str, str]:
|
|||
RATINGS = {"good": ":)", "neutral": ":|", "poor": ":("}
|
||||
|
||||
|
||||
def add_feedback(run: Run, rating: str, comment: str = "") -> dict:
|
||||
"""Record how the runbook felt. Allowed at any time; the latest entry is the current one."""
|
||||
def _feedback_entry(rating: str, comment: str) -> dict:
|
||||
if rating not in RATINGS:
|
||||
raise RunError(f"rating must be one of {', '.join(RATINGS)}")
|
||||
entry = {"rating": rating, "comment": _clean(comment.strip(), "comment"),
|
||||
"at": datetime.now(timezone.utc).isoformat(timespec="microseconds")} # sortable even within one second
|
||||
return {"rating": rating, "comment": _clean(comment.strip(), "comment"),
|
||||
"at": datetime.now(timezone.utc).isoformat(timespec="microseconds")} # sortable even within one second
|
||||
|
||||
|
||||
def add_feedback(run: Run, rating: str, comment: str = "") -> dict:
|
||||
"""Record how the runbook felt. Possible while the run is open (the latest entry counts); a closed run is locked,
|
||||
so give the feedback before closing, or with the closing form."""
|
||||
if run.data.get("outcome"):
|
||||
raise RunError("this run is closed and locked: its feedback can no longer be changed")
|
||||
entry = _feedback_entry(rating, comment)
|
||||
run.data.setdefault("feedback", []).append(entry)
|
||||
run.save()
|
||||
return entry
|
||||
|
|
@ -359,8 +378,37 @@ OUTCOMES = ("succeeded", "failed", "abandoned")
|
|||
STALE_DAYS = 7
|
||||
|
||||
|
||||
def finalize(run: Run, outcome: str, note: str = "") -> None:
|
||||
"""Close a run. 'succeeded' needs every step done; 'failed' needs a failed step."""
|
||||
def _seal_of(data: dict) -> str:
|
||||
"""Fingerprint of a closed record. Timestamps of allowed later updates and the exercise marker are not part of it."""
|
||||
import hashlib
|
||||
body = {k: v for k, v in data.items() if k not in ("seal", "updated", "exercise_recorded")}
|
||||
return hashlib.sha256(json.dumps(body, sort_keys=True).encode()).hexdigest()
|
||||
|
||||
|
||||
def _read_json(path) -> dict | None:
|
||||
try:
|
||||
return json.loads(Path(path).read_text())
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def seal_state(run: Run) -> str:
|
||||
"""'open' | 'sealed' | 'broken' (changed after closing, outside the app) | 'unsealed' (closed before sealing existed).
|
||||
|
||||
Judged on what is on disk, not on the in-memory copy."""
|
||||
disk = _read_json(run.path) or run.data
|
||||
if not disk.get("outcome"):
|
||||
return "open"
|
||||
if not disk.get("seal"):
|
||||
return "unsealed"
|
||||
return "sealed" if disk["seal"] == _seal_of(disk) else "broken"
|
||||
|
||||
|
||||
def finalize(run: Run, outcome: str, note: str = "", rating: str | None = None, comment: str = "") -> None:
|
||||
"""Close a run: its outcome is recorded, its record sealed and locked. An optional rating and comment are
|
||||
recorded with it, because a closed run can no longer take feedback.
|
||||
|
||||
'succeeded' needs every step done; 'failed' needs a failed or pushed-through step."""
|
||||
if run.data.get("outcome"):
|
||||
raise RunError(f"run already finished ({run.data['outcome']})")
|
||||
if outcome not in OUTCOMES:
|
||||
|
|
@ -373,10 +421,14 @@ def finalize(run: Run, outcome: str, note: str = "") -> None:
|
|||
gstates = [g["state"] for g in run.data.get("gates", {}).values()]
|
||||
if outcome == "failed" and not ({"failed", "pushed"} & set(states) or {"failed", "pushed"} & set(gstates)):
|
||||
raise RunError("a run can be finished as failed only when a step or gate has failed or was pushed through; otherwise abandon it")
|
||||
entry = _feedback_entry(rating, comment) if rating else None # validated before anything changes
|
||||
run.data["outcome"] = outcome
|
||||
run.data["finished"] = _now()
|
||||
if note.strip():
|
||||
run.data["outcome_note"] = _clean(note.strip(), "note")
|
||||
if entry:
|
||||
run.data.setdefault("feedback", []).append(entry)
|
||||
run.data["seal"] = _seal_of(run.data)
|
||||
run.save()
|
||||
|
||||
|
||||
|
|
@ -462,5 +514,5 @@ def apply_exercise(run: Run, today: date | None = None) -> str:
|
|||
status = STATUS_LINE.search(new).group(0).split(": ", 1)[1]
|
||||
run.pack.data["exercise_status"] = status
|
||||
run.data["exercise_recorded"] = status
|
||||
run.save()
|
||||
run.save(allow=("exercise_recorded",)) # the one thing a closed run may still receive
|
||||
return status
|
||||
|
|
|
|||
60
rtut/ui.py
60
rtut/ui.py
|
|
@ -75,6 +75,7 @@ a.chip:hover{background:var(--hi);color:#141414;border-color:var(--hi)}.chip.on{
|
|||
.pill[data-s=failed],.pill[data-s=err],.pill[data-s=invalid]{color:var(--bad);border-color:var(--bad)}
|
||||
.pill[data-s=changed-after-use],.pill[data-s=pushed],.pill[data-s=waived],.pill[data-s=stale],.pill[data-s=override],.pill[data-s=push-through],.pill[data-s=waiver],.pill[data-s=modified],.pill[data-s=abandoned]{color:var(--warn);border-color:var(--warn)}
|
||||
.pill[data-s=nil],.pill[data-s=todo],.pill[data-s=open],.pill[data-s=planned],.pill[data-s="not checked"],.pill[data-s=not-applied]{color:var(--mut);border-style:dashed}
|
||||
.hl a{text-decoration:none}.hl a::after{content:" →";opacity:.35;font-weight:400}.hl a:hover::after{opacity:1}
|
||||
.feed{list-style:none;padding:0}.feed li{padding:.3rem 0;border-bottom:1px dashed var(--line)}
|
||||
.checks{list-style:none;padding:0;margin:.4rem 0}.checks li{padding:.35rem 0;border-bottom:1px dashed var(--line)}.checks li:last-child{border:0}
|
||||
#current{scroll-margin-top:1rem}
|
||||
|
|
@ -143,6 +144,13 @@ def e(s) -> str:
|
|||
return html.escape(str(s))
|
||||
|
||||
|
||||
def heading(level: str, text: str, href: str | None = None, extra: str = "") -> str:
|
||||
"""A section headline. On the Home page it links to the matching sidebar section, as a shortcut."""
|
||||
if href:
|
||||
return f"<{level} class=hl><a href='{href}'>{html.escape(text)}</a>{extra}</{level}>"
|
||||
return f"<{level}>{html.escape(text)}{extra}</{level}>"
|
||||
|
||||
|
||||
SMILEYS = (("good", "🙂", "Went well"), ("neutral", "😐", "So-so"), ("poor", "🙁", "Poor"))
|
||||
|
||||
|
||||
|
|
@ -177,15 +185,15 @@ class App:
|
|||
return (f"<input type=hidden name={name} value='{e(u['operator'])}'>"
|
||||
f"<p class=note>Running as <b>{label}</b> · <a href='/user'>change</a></p>")
|
||||
|
||||
def activities_html(self, limit: int = 12, level: str = "h2") -> str:
|
||||
def activities_html(self, limit: int = 12, level: str = "h2", link: bool = False) -> str:
|
||||
events = ACT.latest(self.packs, limit)
|
||||
if not events:
|
||||
return f"<{level}>Latest activities</{level}><p class=note>Nothing has happened yet. Start a run or a project.</p>"
|
||||
return f"{heading(level, 'Latest activities', '/activity' if link else None)}<p class=note>Nothing has happened yet. Start a run or a project.</p>"
|
||||
rows = "".join(
|
||||
f"<li><span class=note>{e(x['at'][:16].replace('T', ' '))}</span> · <b>{e(x['who'])}</b> {e(x['text'])} "
|
||||
f"<span class=note>· {e(x['pack'])}{' · project ' + e(x['project']) if x.get('project') else ''}</span>"
|
||||
f"{' · <a href=' + chr(39) + x['link'] + chr(39) + '>open</a>' if x['link'] else ''}</li>" for x in events)
|
||||
return (f"<{level}>Latest activities <span class=note style='font-weight:400;font-size:.85rem'>(times in UTC)</span></{level}>"
|
||||
return (f"{heading(level, 'Latest activities', '/activity' if link else None, ' <span class=note style=' + chr(39) + 'font-weight:400;font-size:.85rem' + chr(39) + '>(times in UTC)</span>')}"
|
||||
f"<ul class=feed>{rows}</ul>")
|
||||
|
||||
def user_page(self, msg: str = "") -> bytes:
|
||||
|
|
@ -209,7 +217,7 @@ class App:
|
|||
return f"<input type=hidden name=_t value='{self.token}'>"
|
||||
|
||||
def index(self) -> bytes:
|
||||
return page("Home", UP.index_sections(self) + self.activities_html() + self.runbooks_html())
|
||||
return page("Home", UP.index_sections(self, home=True) + self.activities_html(link=True) + self.runbooks_html(link=True))
|
||||
|
||||
def runbooks_page(self) -> bytes:
|
||||
return page("Runbooks", self.runbooks_html("h1"))
|
||||
|
|
@ -220,7 +228,7 @@ class App:
|
|||
def activity_page(self) -> bytes:
|
||||
return page("Activity", self.activities_html(100, "h1"))
|
||||
|
||||
def runbooks_html(self, level: str = "h2") -> str:
|
||||
def runbooks_html(self, level: str = "h2", link: bool = False) -> str:
|
||||
rows = []
|
||||
for i, p in enumerate(self.packs):
|
||||
d = p.data
|
||||
|
|
@ -232,7 +240,7 @@ class App:
|
|||
else:
|
||||
rows.append(f"<li><a href='/pack/{i}'>{e(d['title'])}</a> {UP.maturity_pill(p)} "
|
||||
f"<span class=note>{e(score_line(R.scorecard(p)))}</span></li>")
|
||||
return f"<{level}>Runbooks</{level}><ul>{''.join(rows) or '<li>No packs found.</li>'}</ul>"
|
||||
return f"{heading(level, 'Runbooks', '/runbooks' if link else None)}<ul>{''.join(rows) or '<li>No packs found.</li>'}</ul>"
|
||||
|
||||
def pack_page(self, i: int) -> bytes:
|
||||
p = self.packs[i]; d = p.data
|
||||
|
|
@ -291,7 +299,7 @@ class App:
|
|||
outcome = r.data.get("outcome")
|
||||
if outcome:
|
||||
note = f" - {e(r.data['outcome_note'])}" if r.data.get("outcome_note") else ""
|
||||
box = f"<p><b>Run finished: {e(outcome)}</b>{note}</p>"
|
||||
box = f"<p><b>Run finished: {e(outcome)}</b>{note}</p>{self.locked_note(r)}"
|
||||
if outcome == "succeeded":
|
||||
if r.data.get("exercise_recorded"):
|
||||
box += f"<p>Pack recorded as <b>{e(r.data['exercise_recorded'])}</b>. Commit the change in the owning repository.</p>"
|
||||
|
|
@ -315,7 +323,7 @@ class App:
|
|||
"<p class=note>Finishing records the outcome in the pack's score. A finished run is read-only. "
|
||||
"Unfinished runs count as stalled, and as abandoned after a week.</p>"
|
||||
f"<form method=post action='/run/{i}/{e(rid)}/finalize'>{self.hidden()}"
|
||||
"<p><input name=note class=grow style='width:100%' placeholder='comment (optional)'></p>"
|
||||
f"<p><input name=note class=grow style='width:100%' placeholder='comment (optional)'></p>{self.close_feedback_fields(r)}"
|
||||
f"<p class=row>{' '.join(btns)}</p></form></div>")
|
||||
|
||||
def feedback_form(self, i: int, rid: str, r) -> str:
|
||||
|
|
@ -328,14 +336,41 @@ class App:
|
|||
"Never include secrets.</p>"
|
||||
f"<p><textarea name=comment rows=3 style='width:100%' placeholder='optional comment'>"
|
||||
f"{e(fb['comment']) if fb else ''}</textarea></p><p class=row>{btns}</p></form>")
|
||||
if fb: # collected: close the section, suggest the next step
|
||||
if r.data.get("outcome"): # a closed run is locked: show what was said, offer no change
|
||||
if not fb:
|
||||
return ""
|
||||
face = next(f for k, f, _ in SMILEYS if k == fb["rating"])
|
||||
return (f"<div id=feedback class=step><h2>Feedback</h2><p>{face} {e(fb['comment'] or '(no comment)')}</p>"
|
||||
"<p class=note>The run is closed and locked, so its feedback can no longer be changed.</p></div>")
|
||||
if fb: # collected while the run is open: close the section
|
||||
face = next(f for k, f, _ in SMILEYS if k == fb["rating"])
|
||||
return ("<div id=feedback class=step><p><b>Thank you.</b> Recorded: "
|
||||
f"{face} {e(fb['comment'] or '(no comment)')}</p>"
|
||||
"<p><a href='/'>← Back to the runbook list</a></p>"
|
||||
f"<details><summary class=note>Change my feedback</summary>{form}</details></div>")
|
||||
return f"<div id=feedback class=step><h2>How was this runbook?</h2>{form}</div>"
|
||||
|
||||
def close_feedback_fields(self, r) -> str:
|
||||
"""Optional rating and comment inside the closing form: a closed run cannot take feedback afterwards."""
|
||||
if R.latest_feedback(r):
|
||||
return ""
|
||||
faces = " ".join(f"<label title='{e(t)}'><input type=radio name=rating value={k}> <span style='font-size:1.4rem'>{face}</span></label>"
|
||||
for k, face, t in SMILEYS)
|
||||
return ("<fieldset><legend>How was this runbook? <span class=note>(optional; it cannot be added after closing)</span></legend>"
|
||||
f"<p class=row>{faces}</p><textarea name=fb_comment rows=2 style='width:100%' "
|
||||
"placeholder='anything strange or unclear, or an idea to make it better (no secrets)'></textarea></fieldset>")
|
||||
|
||||
def locked_note(self, r) -> str:
|
||||
"""What 'closed' means: the record is sealed and locked. A record changed outside the app is called out."""
|
||||
state = R.seal_state(r)
|
||||
if state == "broken":
|
||||
return ("<p class=bad>⚠ This closed run's record was changed outside the app after it was sealed. "
|
||||
"It no longer counts towards maturity.</p>")
|
||||
base = ("This run is <b>closed and locked</b>: its record is sealed and cannot be changed. "
|
||||
"Start a new run to try again.")
|
||||
if state == "unsealed":
|
||||
base += " (It was closed before sealing existed, so its record carries no seal.)"
|
||||
return f"<p class=note>{base}</p>"
|
||||
|
||||
def run_page(self, i: int, rid: str, msg: str = "", query: dict | None = None) -> bytes:
|
||||
p = self.packs[i]; r = R.load(p, rid); d = p.data
|
||||
if p.spec == "runbook-pack/v0.2":
|
||||
|
|
@ -424,7 +459,7 @@ class App:
|
|||
if r.complete:
|
||||
out.append("<p><b>All steps done.</b> To record this run as exercising the pack, review and apply: "
|
||||
f"<code>python -m rtut run finish {e(p.path)} {e(rid)} --apply</code></p>")
|
||||
if r.complete:
|
||||
if r.complete or r.data.get("outcome") or R.latest_feedback(r):
|
||||
out.append(self.feedback_form(i, rid, r))
|
||||
out.append(self.close_run_form(i, rid, r, p))
|
||||
out.append("<h2>Threat checks</h2><ul>" + "".join(f"<li>{e(t)}</li>" for t in d["threat_checks"]) + "</ul>")
|
||||
|
|
@ -567,7 +602,8 @@ def make_handler(app: App, port_holder: list):
|
|||
i, rid = int(parts[1]), parts[2]
|
||||
r = R.load(app.packs[i], rid)
|
||||
if parts[3] == "finalize":
|
||||
R.finalize(r, form.get("outcome", ""), form.get("note", ""))
|
||||
R.finalize(r, form.get("outcome", ""), form.get("note", ""), rating=form.get("rating") or None,
|
||||
comment=form.get("fb_comment", ""))
|
||||
else:
|
||||
R.apply_exercise(r)
|
||||
app.packs[i] = P.load(app.packs[i].path) # pick up the new exercise status
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@ def maturity_block(app, i: int, pack) -> str:
|
|||
|
||||
# ---- index sections ---------------------------------------------------------------------------------------------
|
||||
|
||||
def index_sections(app) -> str:
|
||||
def index_sections(app, home: bool = False) -> str:
|
||||
reg = app.registry()
|
||||
prj = []
|
||||
for pid, d in reg.projects.items():
|
||||
|
|
@ -75,8 +75,10 @@ def index_sections(app) -> str:
|
|||
tpl = [f"<li><a href='/template/{e(tid)}'>{e(d.data['title'])}</a> "
|
||||
f"<span class=note>{e(d.data.get('goal', ''))}</span></li>" if d.ok else f"<li class=bad>{e(tid)}: invalid</li>"
|
||||
for tid, d in reg.templates.items()]
|
||||
out = f"<h1>Projects</h1><ul>{''.join(prj) or '<li class=note>No projects yet. Start one from a template.</li>'}</ul>"
|
||||
out += f"<h2>Project templates</h2><ul>{''.join(tpl) or '<li class=note>No templates found.</li>'}</ul>"
|
||||
from .ui import heading
|
||||
link = "/projects" if home else None
|
||||
out = f"{heading('h1', 'Projects', link)}<ul>{''.join(prj) or '<li class=note>No projects yet. Start one from a template.</li>'}</ul>"
|
||||
out += f"{heading('h2', 'Project templates', link)}<ul>{''.join(tpl) or '<li class=note>No templates found.</li>'}</ul>"
|
||||
return out
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ def _flow_view(c: Ctx, ov: dict) -> str:
|
|||
out.append(_next_banner(c))
|
||||
for st in fl.data["stages"]:
|
||||
out.append(_stage(c, st))
|
||||
if c.finished or _all_complete(run):
|
||||
if c.finished or _all_complete(run) or R1.latest_feedback(run):
|
||||
out.append(c.app.feedback_form(c.i, c.rid, run))
|
||||
out.append(_close(c))
|
||||
return "".join(out)
|
||||
|
|
@ -603,9 +603,8 @@ def _close(c: Ctx) -> str:
|
|||
run = c.run
|
||||
if c.finished:
|
||||
note = f" - {e(run.data['outcome_note'])}" if run.data.get("outcome_note") else ""
|
||||
return (f"<div id=close class=step><h2>Run finished: {e(run.data['outcome'])}</h2><p>{note}</p>"
|
||||
"<p class=note>The outcome is part of this runbook's score and maturity evidence at this revision. "
|
||||
"<a href='/'>← Back to the runbook list</a></p></div>")
|
||||
return (f"<div id=close class=step><h2>Run finished: {e(run.data['outcome'])}</h2><p>{note}</p>{c.app.locked_note(run)}"
|
||||
"<p class=note>The outcome is part of this runbook's score and maturity evidence at this revision.</p></div>")
|
||||
fl = c.fl
|
||||
steps = [run.data["steps"][s]["state"] for s in fl.steps]
|
||||
gates = [run.data["gates"][g]["state"] for g in fl.gates]
|
||||
|
|
@ -619,7 +618,7 @@ def _close(c: Ctx) -> str:
|
|||
"<p class=note>Finishing records the outcome. A run with deviations can be finished but does not count towards maturity. "
|
||||
"A finished run is read-only.</p>"
|
||||
f"<form method=post action='{c.base}/finalize'>{c.h()}<p><input name=note class=grow style='width:100%' placeholder='comment (optional)'></p>"
|
||||
f"<p class=row>{' '.join(btns)}</p></form></div>")
|
||||
f"{c.app.close_feedback_fields(run)}<p class=row>{' '.join(btns)}</p></form></div>")
|
||||
|
||||
|
||||
# ---- actions -------------------------------------------------------------------------------------------------------------------
|
||||
|
|
@ -638,7 +637,7 @@ def handle_post(app, i: int, rid: str, parts: list[str], form: dict) -> str:
|
|||
R1.add_feedback(run, form.get("rating", ""), form.get("comment", ""))
|
||||
return f"{base}#feedback"
|
||||
if parts == ["finalize"]:
|
||||
R1.finalize(run, form.get("outcome", ""), form.get("note", ""))
|
||||
R1.finalize(run, form.get("outcome", ""), form.get("note", ""), rating=form.get("rating") or None, comment=form.get("fb_comment", ""))
|
||||
return f"{base}#close"
|
||||
if parts == ["mode"]:
|
||||
R.switch_mode(run, form.get("to", ""))
|
||||
|
|
|
|||
|
|
@ -42,16 +42,17 @@ def test_finalize_rules(pk):
|
|||
R.finalize(r, "abandoned") # already finished
|
||||
|
||||
|
||||
def test_finished_run_is_read_only_but_takes_feedback(pk):
|
||||
def test_finished_run_is_read_only_and_feedback_is_locked_too(pk):
|
||||
r = R.create(pk, "bernd"); R.finalize(r, "abandoned")
|
||||
for call in (lambda: R.start(r, "read-the-sign"), lambda: R.add_capture(r, "read-the-sign", "x")):
|
||||
with pytest.raises(R.RunError, match="finished"):
|
||||
call()
|
||||
R.add_feedback(r, "poor", "gave up: unclear") # feedback stays possible
|
||||
with pytest.raises(R.RunError, match="closed and locked"):
|
||||
R.add_feedback(r, "poor", "gave up: unclear") # feedback goes in before closing, or with the closing form
|
||||
|
||||
|
||||
def test_scorecard_counts_every_outcome_and_stale_open_runs(pk):
|
||||
ok = R.create(pk, "a"); play_all(ok); R.finalize(ok, "succeeded"); R.add_feedback(ok, "good")
|
||||
ok = R.create(pk, "a"); play_all(ok); R.finalize(ok, "succeeded", rating="good")
|
||||
bad = R.create(pk, "b"); R.start(bad, "read-the-sign"); R.fail(bad, "read-the-sign", "x"); R.finalize(bad, "failed")
|
||||
gone = R.create(pk, "c"); R.finalize(gone, "abandoned")
|
||||
stalled = R.create(pk, "d")
|
||||
|
|
@ -151,3 +152,72 @@ def test_unfinished_run_elapsed_runs_to_now_and_no_average_without_succeeded_run
|
|||
t = R.run_times(r, now=datetime.now(timezone.utc) + timedelta(seconds=90))
|
||||
assert not t["finished_run"] and 89 <= t["elapsed"] <= 91
|
||||
assert R.scorecard(pk)["avg_seconds"] is None and R.scorecard(pk)["timed_runs"] == 0
|
||||
|
||||
|
||||
# ---- a closed run is locked and sealed --------------------------------------------------------------------------
|
||||
|
||||
def test_closing_records_feedback_with_the_outcome_and_validates_before_changing_anything(pk):
|
||||
r = R.create(pk, "bernd")
|
||||
with pytest.raises(R.RunError, match="rating must be"):
|
||||
R.finalize(r, "abandoned", rating="great")
|
||||
assert not r.data.get("outcome") and "seal" not in r.data # a refused close changed nothing
|
||||
R.finalize(r, "abandoned", "gave up", rating="poor", comment="step 2 was unclear")
|
||||
assert R.latest_feedback(r)["comment"] == "step 2 was unclear" and r.data["outcome"] == "abandoned"
|
||||
assert R.load(pk, r.data["run_id"]).data["feedback"][0]["rating"] == "poor"
|
||||
|
||||
|
||||
def test_a_closed_record_cannot_be_saved_with_changes_but_the_exercise_marker_is_allowed(pk):
|
||||
r = R.create(pk, "bernd"); R.finalize(r, "abandoned")
|
||||
r2 = R.load(pk, r.data["run_id"])
|
||||
r2.data["steps"]["read-the-sign"]["state"] = "done"
|
||||
with pytest.raises(R.RunError, match="closed and locked"):
|
||||
r2.save()
|
||||
r3 = R.load(pk, r.data["run_id"])
|
||||
r3.data["operator"] = "someone-else"
|
||||
with pytest.raises(R.RunError, match="operator"):
|
||||
r3.save()
|
||||
assert R.load(pk, r.data["run_id"]).data["operator"] == "bernd" # nothing reached the disk
|
||||
r4 = R.load(pk, r.data["run_id"])
|
||||
r4.data["exercise_recorded"] = "x"
|
||||
r4.save(allow=("exercise_recorded",)) # the one permitted later update
|
||||
assert R.load(pk, r.data["run_id"]).data["exercise_recorded"] == "x" and R.seal_state(R.load(pk, r.data["run_id"])) == "sealed"
|
||||
|
||||
|
||||
def test_closing_seals_the_file_read_only_and_tampering_is_detected(pk):
|
||||
import json, stat
|
||||
r = R.create(pk, "bernd")
|
||||
assert R.seal_state(r) == "open" and r.path.stat().st_mode & stat.S_IWUSR
|
||||
R.finalize(r, "abandoned")
|
||||
assert R.seal_state(r) == "sealed" and not (r.path.stat().st_mode & (stat.S_IWUSR | stat.S_IWGRP | stat.S_IWOTH))
|
||||
assert len(r.data["seal"]) == 64
|
||||
r.path.chmod(0o644) # someone edits the file by hand
|
||||
data = json.loads(r.path.read_text()); data["operator"] = "forged"
|
||||
r.path.write_text(json.dumps(data))
|
||||
assert R.seal_state(R.load(pk, r.data["run_id"])) == "broken"
|
||||
|
||||
|
||||
def test_a_run_closed_before_sealing_existed_is_reported_as_unsealed_not_broken(pk):
|
||||
import json
|
||||
r = R.create(pk, "bernd"); R.finalize(r, "abandoned")
|
||||
r.path.chmod(0o644)
|
||||
data = json.loads(r.path.read_text()); data.pop("seal"); r.path.write_text(json.dumps(data))
|
||||
assert R.seal_state(R.load(pk, r.data["run_id"])) == "unsealed"
|
||||
|
||||
|
||||
def test_a_tampered_run_no_longer_counts_as_maturity_evidence(pk):
|
||||
import json
|
||||
from rtut import maturity as M
|
||||
M.add_review(pk, "alice")
|
||||
r = R.create(pk, "bob", {"logdir": "/tmp/rtut-lock-mat"}, {"project": "lab", "environment": "rehearsal"})
|
||||
for s in pk.data["steps"]:
|
||||
sid = s["id"]; R.start(r, sid)
|
||||
cmd = R.command_for(r, sid)
|
||||
if cmd:
|
||||
subprocess.run(cmd, shell=True, check=True, capture_output=True)
|
||||
R.verify(r, sid, confirm=True, manual_ok=True if s["verify"]["expect"] == "manual" else None)
|
||||
R.complete(r, sid, {n: "1" for n in s.get("evidence", [])})
|
||||
R.finalize(r, "succeeded")
|
||||
assert M.maturity(pk)["evidence"]["rehearsal_runs"] == 1
|
||||
r.path.chmod(0o644)
|
||||
data = json.loads(r.path.read_text()); data["operator"] = "forged"; r.path.write_text(json.dumps(data))
|
||||
assert M.maturity(pk)["evidence"]["clean_runs"] == 0 # changed after closing: not evidence
|
||||
|
|
|
|||
|
|
@ -205,18 +205,18 @@ def test_feedback_form_after_all_steps_done_and_recording(srv):
|
|||
t, loc, _ = _showcase_run(srv)
|
||||
base = loc.split("#")[0]
|
||||
_, _, body = req(srv, "GET", base)
|
||||
assert "How was this runbook?" not in body # not before the run is complete
|
||||
assert "<h2>How was this runbook?</h2>" not in body # not before the run is complete
|
||||
_finish_showcase(srv, t, base)
|
||||
_, _, body = req(srv, "GET", base)
|
||||
assert "All steps done." in body and "How was this runbook?" in body
|
||||
assert body.index("All steps done.") < body.index("How was this runbook?")
|
||||
assert "All steps done." in body and "<h2>How was this runbook?</h2>" in body
|
||||
assert body.index("All steps done.") < body.index("<h2>How was this runbook?</h2>")
|
||||
assert "🙂" in body and "😐" in body and "🙁" in body and "name=comment" in body
|
||||
st, loc2, _ = req(srv, "POST", base + "/feedback", {"_t": t, "rating": "neutral", "comment": "step 3 was unclear"})
|
||||
assert st == 303 and loc2.endswith("#feedback")
|
||||
_, _, body = req(srv, "GET", base)
|
||||
assert "Thank you." in body and "step 3 was unclear" in body
|
||||
assert "How was this runbook?" not in body # the section closed once feedback was collected
|
||||
assert "<a href='/'>← Back to the runbook list</a>" in body # the probable next step
|
||||
assert "<h2>How was this runbook?</h2>" not in body # the section closed once feedback was collected
|
||||
assert "Back to the runbook list" not in body # the sidebar's Home replaces that link
|
||||
assert "Change my feedback" in body # the form is still reachable, folded away
|
||||
assert req(srv, "GET", "/")[0] == 200
|
||||
|
||||
|
|
|
|||
152
tests/test_ui_locked.py
Normal file
152
tests/test_ui_locked.py
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
import http.client
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import threading
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlencode
|
||||
|
||||
import pytest
|
||||
|
||||
from rtut import run as R1, run2 as R, ui
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
_scratch = []
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def srv(tmp_path):
|
||||
for name, src in (("v1", "showcase-lighthouse"), ("v2", "showcase-lighthouse-v2")):
|
||||
d = tmp_path / "runbooks" / name; d.mkdir(parents=True)
|
||||
shutil.copy(ROOT / f"runbooks/{src}/pack.yaml", d / "pack.yaml")
|
||||
s = ui.serve([tmp_path], 0)
|
||||
threading.Thread(target=s.serve_forever, daemon=True).start()
|
||||
s.repo = tmp_path
|
||||
yield s
|
||||
s.shutdown()
|
||||
while _scratch:
|
||||
shutil.rmtree(_scratch.pop(), ignore_errors=True)
|
||||
|
||||
|
||||
def req(s, method, path, form=None):
|
||||
c = http.client.HTTPConnection("127.0.0.1", s.server_address[1])
|
||||
body = urlencode(form) if form is not None else None
|
||||
c.request(method, path, body, {"Host": "127.0.0.1", "Content-Type": "application/x-www-form-urlencoded"})
|
||||
r = c.getresponse()
|
||||
return r.status, r.getheader("Location"), r.read().decode()
|
||||
|
||||
|
||||
def post(s, path, **form):
|
||||
return req(s, "POST", path, {"_t": s.app.token, **form})
|
||||
|
||||
|
||||
def start(s, pack_id):
|
||||
i = s.app.pack_index(pack_id)
|
||||
logdir = f"/tmp/rtut-lock-{uuid.uuid4().hex[:8]}"
|
||||
_scratch.append(logdir)
|
||||
st, loc, _ = post(s, f"/run/new/{i}", operator="bernd", p_logdir=logdir)
|
||||
assert st == 303
|
||||
return loc.split("#")[0], i
|
||||
|
||||
|
||||
def run_of(s, base):
|
||||
return R1.load(s.app.packs[int(base.split("/")[2])], base.rsplit("/", 1)[1])
|
||||
|
||||
|
||||
# ---- Home headlines are shortcuts ---------------------------------------------------------------------------
|
||||
|
||||
def test_home_headlines_link_to_the_sidebar_sections_and_the_section_pages_do_not(srv):
|
||||
_, _, home = req(srv, "GET", "/")
|
||||
assert "<h1 class=hl><a href='/projects'>Projects</a></h1>" in home
|
||||
assert "<h2 class=hl><a href='/projects'>Project templates</a></h2>" in home
|
||||
assert "<a href='/activity'>Latest activities</a>" in home and "<h2 class=hl><a href='/runbooks'>Runbooks</a></h2>" in home
|
||||
_, _, projects = req(srv, "GET", "/projects")
|
||||
assert "<h1>Projects</h1>" in projects and "class=hl" not in projects # on its own page the headline is just a headline
|
||||
assert "<h1>Runbooks</h1>" in req(srv, "GET", "/runbooks")[2]
|
||||
assert "<h1>Latest activities" in req(srv, "GET", "/activity")[2]
|
||||
|
||||
|
||||
# ---- no "back to the runbook list" -------------------------------------------------------------------------------
|
||||
|
||||
def test_there_is_no_back_to_the_runbook_list_link_in_either_run_view(srv):
|
||||
for pack in ("rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"):
|
||||
base, i = start(srv, pack)
|
||||
assert post(srv, f"{base}/feedback", rating="good", comment="fine")[0] == 303
|
||||
assert "Back to the runbook list" not in req(srv, "GET", base)[2]
|
||||
assert post(srv, f"{base}/finalize", outcome="abandoned")[0] == 303
|
||||
assert "Back to the runbook list" not in req(srv, "GET", base)[2]
|
||||
|
||||
|
||||
# ---- closed runs are locked ------------------------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
|
||||
def test_a_closed_run_shows_its_feedback_read_only_without_a_change_option(srv, pack):
|
||||
base, _ = start(srv, pack)
|
||||
assert post(srv, f"{base}/feedback", rating="neutral", comment="a bit long")[0] == 303
|
||||
assert "Change my feedback" in req(srv, "GET", base)[2] # while open it can still be changed
|
||||
assert post(srv, f"{base}/finalize", outcome="abandoned")[0] == 303
|
||||
body = req(srv, "GET", base)[2]
|
||||
assert "Change my feedback" not in body and f"action='{base}/feedback'" not in body
|
||||
assert "<h2>Feedback</h2>" in body and "a bit long" in body and "closed and locked, so its feedback can no longer be changed" in body
|
||||
st, _, err = post(srv, f"{base}/feedback", rating="good", comment="changed my mind")
|
||||
assert st == 400 and "closed and locked" in err
|
||||
assert run_of(srv, base).data["feedback"][0]["comment"] == "a bit long" # nothing reached the record
|
||||
|
||||
|
||||
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
|
||||
def test_a_closed_run_without_feedback_shows_no_feedback_section(srv, pack):
|
||||
base, _ = start(srv, pack)
|
||||
assert post(srv, f"{base}/finalize", outcome="abandoned")[0] == 303
|
||||
body = req(srv, "GET", base)[2]
|
||||
assert "<h2>Feedback</h2>" not in body and "<h2>How was this runbook?</h2>" not in body and "Change my feedback" not in body
|
||||
|
||||
|
||||
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
|
||||
def test_the_closing_form_takes_optional_feedback_so_abandoned_runs_are_not_silent(srv, pack):
|
||||
base, _ = start(srv, pack)
|
||||
body = req(srv, "GET", base)[2]
|
||||
assert "How was this runbook? <span class=note>(optional; it cannot be added after closing)</span>" in body and "name=fb_comment" in body
|
||||
assert post(srv, f"{base}/finalize", outcome="abandoned", rating="poor", fb_comment="step 1 was unclear")[0] == 303
|
||||
body = req(srv, "GET", base)[2]
|
||||
assert "<h2>Feedback</h2>" in body and "step 1 was unclear" in body
|
||||
assert run_of(srv, base).data["feedback"][0]["rating"] == "poor"
|
||||
|
||||
|
||||
def test_the_closing_form_does_not_ask_again_when_feedback_was_already_given(srv):
|
||||
base, _ = start(srv, "rtut.showcase-lighthouse-v2")
|
||||
post(srv, f"{base}/feedback", rating="good", comment="ok")
|
||||
assert "name=fb_comment" not in req(srv, "GET", base)[2]
|
||||
|
||||
|
||||
def test_a_bad_rating_on_closing_refuses_the_whole_close(srv):
|
||||
base, _ = start(srv, "rtut.showcase-lighthouse-v2")
|
||||
st, _, err = post(srv, f"{base}/finalize", outcome="abandoned", rating="great")
|
||||
assert st == 400 and "rating must be" in err
|
||||
assert not run_of(srv, base).data.get("outcome") # still open
|
||||
|
||||
|
||||
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
|
||||
def test_a_closed_run_says_it_is_locked_and_refuses_every_change(srv, pack):
|
||||
base, _ = start(srv, pack)
|
||||
post(srv, f"{base}/finalize", outcome="abandoned")
|
||||
body = req(srv, "GET", base)[2]
|
||||
assert "closed and locked" in body and "its record is sealed and cannot be changed" in body
|
||||
first = "read-the-sign"
|
||||
for path, form in ((f"{base}/{first}/start", {}), (f"{base}/mode", {"to": "auto-confirm"}), (f"{base}/data", {"d_weather": "clear"}),
|
||||
(f"{base}/{first}/capture", {"output": "x"}), (f"{base}/finalize", {"outcome": "succeeded"})):
|
||||
st, _, err = post(srv, path, **form)
|
||||
assert st in (400,) and ("closed" in err or "finished" in err or "already" in err), path
|
||||
assert run_of(srv, base).data["outcome"] == "abandoned"
|
||||
|
||||
|
||||
def test_the_record_file_is_read_only_after_closing_and_tampering_is_called_out(srv):
|
||||
base, _ = start(srv, "rtut.showcase-lighthouse-v2")
|
||||
post(srv, f"{base}/finalize", outcome="abandoned")
|
||||
r = run_of(srv, base)
|
||||
assert r.path.stat().st_mode & 0o222 == 0 and R1.seal_state(r) == "sealed"
|
||||
assert "changed outside the app" not in req(srv, "GET", base)[2]
|
||||
r.path.chmod(0o644)
|
||||
data = json.loads(r.path.read_text()); data["operator"] = "forged"; r.path.write_text(json.dumps(data))
|
||||
body = req(srv, "GET", base)[2]
|
||||
assert "changed outside the app after it was sealed" in body and "no longer counts towards maturity" in body
|
||||
|
|
@ -59,7 +59,7 @@ def make_project(s, pid="harbour", host=None):
|
|||
|
||||
def test_index_has_projects_templates_runbooks_and_maturity(psrv):
|
||||
_, _, body = req(psrv, "GET", "/")
|
||||
assert "<h1>Projects</h1>" in body and "Project templates" in body and "<h2>Runbooks</h2>" in body
|
||||
assert "<h1 class=hl><a href='/projects'>Projects</a></h1>" in body and "Project templates" in body and "<a href='/runbooks'>Runbooks</a>" in body
|
||||
assert "/template/lighthouse-watch" in body
|
||||
assert ">draft<" in body # maturity pill on every native runbook
|
||||
|
||||
|
|
|
|||
|
|
@ -91,7 +91,7 @@ def test_sections_are_separate_pages(srv):
|
|||
_, _, body = req(srv, "GET", "/activity")
|
||||
assert "<h1>Latest activities" in body and "(times in UTC)" in body and "started a run" in body
|
||||
_, _, body = req(srv, "GET", "/")
|
||||
assert "<h1>Projects</h1>" in body and "Latest activities" in body and "<h2>Runbooks</h2>" in body # Home shows all three
|
||||
assert "<h1 class=hl><a href='/projects'>Projects</a></h1>" in body and "Latest activities" in body and "<h2 class=hl><a href='/runbooks'>Runbooks</a></h2>" in body # Home shows all three, as shortcuts
|
||||
|
||||
|
||||
# ---- attribute sets ----------------------------------------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -314,9 +314,9 @@ def test_finishing_a_completed_run_feedback_and_the_read_only_result(s2):
|
|||
do_step(s2, base, "close-log", phrase=PHRASE)
|
||||
pass_gate(s2, base, "night-on-record")
|
||||
body = page(s2, base)
|
||||
assert "Everything is done." in body and "Finish: succeeded" in body and "How was this runbook?" in body
|
||||
assert "Everything is done." in body and "Finish: succeeded" in body and "<h2>How was this runbook?</h2>" in body
|
||||
assert post(s2, f"{base}/feedback", rating="good", comment="clear")[0] == 303
|
||||
assert "Back to the runbook list" in page(s2, base)
|
||||
assert "Back to the runbook list" not in page(s2, base) and "Change my feedback" in page(s2, base) # still open: changeable
|
||||
assert post(s2, f"{base}/finalize", outcome="succeeded", note="smooth")[0] == 303
|
||||
body = page(s2, base)
|
||||
assert "Run finished: succeeded" in body and "Start this step" not in body
|
||||
|
|
|
|||
|
|
@ -246,6 +246,19 @@ pages; attribute sets listed, inspected with their resolved values and sources,
|
|||
secrets, bad names, unknown and circular `extends` refused, a refused save changes nothing); the project name, description and goal
|
||||
editable from the project page. Bernd confirmed the theme and navigation. Not done: deleting an attribute set, editing templates.
|
||||
|
||||
## Closed runs locked; Home headlines as shortcuts
|
||||
|
||||
```task
|
||||
id: RBT-WP-0004-T16
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Done 2026-09-29 from Bernd's feedback: Home headlines link to their sidebar sections; the "Back to the runbook list" link is gone; closed runs are
|
||||
locked (engine refuses changes, feedback included), sealed with a hash and made read-only, tampering is shown and disqualifies the run as maturity
|
||||
evidence; the closing form takes optional feedback so abandoned runs are not silent; a closed run shows its feedback read-only (no "Change my
|
||||
feedback"), or no feedback section at all. See `docs/run-spec.md`.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Bernd can complete the SSH tutorial pack in the UI without pasting outputs to an agent.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue