Lock and seal closed runs; feedback with the closing form; Home headlines link to their sections; drop the back link
Some checks failed
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 3s
Work Records / validate (push) Has been cancelled

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: sonnet
Assistant-Process: 295952@bnt-lap001
Assistant-Session: e93f64ad-516c-46eb-9666-aad8d300c477
This commit is contained in:
tegwick 2026-09-29 18:03:12 +02:00
parent b7edb8f023
commit 711c4e1149
13 changed files with 381 additions and 42 deletions

View file

@ -106,6 +106,20 @@ Rules the engine enforces:
- **Modes** are `manual` and `auto-confirm`. Auto-confirm applies confirmation only; running the step's own
command is a separate, later level ("auto-pilot with execution", RBT-WP-0006).
## Closed runs are locked and sealed
Closing a run (`finalize`) makes its record final:
- **Locked.** A closed record cannot be saved with changes: the engine refuses, naming what would have changed. The only later update it
accepts is `exercise_recorded`, written by the legacy "Record pack as exercised" step. This holds for runs closed before sealing existed too.
- **Feedback.** It locks with the run. Give it before closing, or with the closing form: `finalize(..., rating, comment)` records an optional
rating and comment together with the outcome (validated first, so a refused close changes nothing). This keeps abandoned and failed runs from
losing what the operator wanted to say.
- **Sealed.** The record carries a SHA-256 `seal` of its content (excluding `seal`, `updated` and `exercise_recorded`), and the file is made
read-only. `seal_state` reports `open`, `sealed`, `broken` (changed outside the app after closing) or `unsealed` (closed before sealing existed).
A broken seal is called out in the UI, and such a run no longer counts as maturity evidence. This is tamper-evidence, not prevention: someone
with file access can still edit a file, but not without being noticed.
## Changing a value after steps used it
A v0.1 run fixes its parameters when it starts. In a v0.2 run datapoints can be edited (guided flow questions, data view), but a value

View file

@ -53,7 +53,8 @@ def _env_key(ctx: dict) -> str:
def _evidence(pack: Pack, dig: str, runs: list[R.Run]) -> dict:
clean = [r for r in runs if r.data.get("pack_digest") == dig and r.data.get("outcome") == "succeeded"
and not r.data.get("deviations") and r.data.get("context", {}).get("environment")]
and not r.data.get("deviations") and r.data.get("context", {}).get("environment")
and R.seal_state(r) != "broken"] # a record changed after closing is not evidence
reviewed = any(rv.get("digest") == dig for rv in _reviews(pack))
rehearsal = [r for r in clean if r.data["context"]["environment"] == "rehearsal"]
envs = {_env_key(r.data["context"]) for r in clean}

View file

@ -44,10 +44,22 @@ class Run:
def path(self) -> Path:
return runs_dir(self.pack) / f"{self.data['run_id']}.json"
def save(self) -> None:
def save(self, allow: tuple = ()) -> None:
"""Write the record. A closed run is locked: nothing may change except the keys named in `allow`."""
disk = _read_json(self.path)
if disk and disk.get("outcome"):
keys = set(disk) | set(self.data)
changed = {k for k in keys if disk.get(k) != self.data.get(k)} - {"updated"} - set(allow)
if changed:
raise RunError(f"this run is closed and locked ({disk['outcome']}); it cannot be changed "
f"({', '.join(sorted(changed))})")
self.data["updated"] = _now()
self.path.parent.mkdir(parents=True, exist_ok=True)
if self.path.exists():
self.path.chmod(0o644) # a sealed file is read-only; the allowed update reopens it briefly
self.path.write_text(json.dumps(self.data, indent=2, sort_keys=True) + "\n")
if self.data.get("outcome"):
self.path.chmod(0o444)
def step_def(self, step_id: str) -> dict:
if self.pack.spec == "runbook-pack/v0.2": # steps live in stages; alternatives are applied
@ -280,12 +292,19 @@ def exercise_update(run: Run, today: date | None = None) -> tuple[str, str]:
RATINGS = {"good": ":)", "neutral": ":|", "poor": ":("}
def add_feedback(run: Run, rating: str, comment: str = "") -> dict:
"""Record how the runbook felt. Allowed at any time; the latest entry is the current one."""
def _feedback_entry(rating: str, comment: str) -> dict:
if rating not in RATINGS:
raise RunError(f"rating must be one of {', '.join(RATINGS)}")
entry = {"rating": rating, "comment": _clean(comment.strip(), "comment"),
"at": datetime.now(timezone.utc).isoformat(timespec="microseconds")} # sortable even within one second
return {"rating": rating, "comment": _clean(comment.strip(), "comment"),
"at": datetime.now(timezone.utc).isoformat(timespec="microseconds")} # sortable even within one second
def add_feedback(run: Run, rating: str, comment: str = "") -> dict:
"""Record how the runbook felt. Possible while the run is open (the latest entry counts); a closed run is locked,
so give the feedback before closing, or with the closing form."""
if run.data.get("outcome"):
raise RunError("this run is closed and locked: its feedback can no longer be changed")
entry = _feedback_entry(rating, comment)
run.data.setdefault("feedback", []).append(entry)
run.save()
return entry
@ -359,8 +378,37 @@ OUTCOMES = ("succeeded", "failed", "abandoned")
STALE_DAYS = 7
def finalize(run: Run, outcome: str, note: str = "") -> None:
"""Close a run. 'succeeded' needs every step done; 'failed' needs a failed step."""
def _seal_of(data: dict) -> str:
"""Fingerprint of a closed record. Timestamps of allowed later updates and the exercise marker are not part of it."""
import hashlib
body = {k: v for k, v in data.items() if k not in ("seal", "updated", "exercise_recorded")}
return hashlib.sha256(json.dumps(body, sort_keys=True).encode()).hexdigest()
def _read_json(path) -> dict | None:
try:
return json.loads(Path(path).read_text())
except (OSError, ValueError):
return None
def seal_state(run: Run) -> str:
"""'open' | 'sealed' | 'broken' (changed after closing, outside the app) | 'unsealed' (closed before sealing existed).
Judged on what is on disk, not on the in-memory copy."""
disk = _read_json(run.path) or run.data
if not disk.get("outcome"):
return "open"
if not disk.get("seal"):
return "unsealed"
return "sealed" if disk["seal"] == _seal_of(disk) else "broken"
def finalize(run: Run, outcome: str, note: str = "", rating: str | None = None, comment: str = "") -> None:
"""Close a run: its outcome is recorded, its record sealed and locked. An optional rating and comment are
recorded with it, because a closed run can no longer take feedback.
'succeeded' needs every step done; 'failed' needs a failed or pushed-through step."""
if run.data.get("outcome"):
raise RunError(f"run already finished ({run.data['outcome']})")
if outcome not in OUTCOMES:
@ -373,10 +421,14 @@ def finalize(run: Run, outcome: str, note: str = "") -> None:
gstates = [g["state"] for g in run.data.get("gates", {}).values()]
if outcome == "failed" and not ({"failed", "pushed"} & set(states) or {"failed", "pushed"} & set(gstates)):
raise RunError("a run can be finished as failed only when a step or gate has failed or was pushed through; otherwise abandon it")
entry = _feedback_entry(rating, comment) if rating else None # validated before anything changes
run.data["outcome"] = outcome
run.data["finished"] = _now()
if note.strip():
run.data["outcome_note"] = _clean(note.strip(), "note")
if entry:
run.data.setdefault("feedback", []).append(entry)
run.data["seal"] = _seal_of(run.data)
run.save()
@ -462,5 +514,5 @@ def apply_exercise(run: Run, today: date | None = None) -> str:
status = STATUS_LINE.search(new).group(0).split(": ", 1)[1]
run.pack.data["exercise_status"] = status
run.data["exercise_recorded"] = status
run.save()
run.save(allow=("exercise_recorded",)) # the one thing a closed run may still receive
return status

View file

@ -75,6 +75,7 @@ a.chip:hover{background:var(--hi);color:#141414;border-color:var(--hi)}.chip.on{
.pill[data-s=failed],.pill[data-s=err],.pill[data-s=invalid]{color:var(--bad);border-color:var(--bad)}
.pill[data-s=changed-after-use],.pill[data-s=pushed],.pill[data-s=waived],.pill[data-s=stale],.pill[data-s=override],.pill[data-s=push-through],.pill[data-s=waiver],.pill[data-s=modified],.pill[data-s=abandoned]{color:var(--warn);border-color:var(--warn)}
.pill[data-s=nil],.pill[data-s=todo],.pill[data-s=open],.pill[data-s=planned],.pill[data-s="not checked"],.pill[data-s=not-applied]{color:var(--mut);border-style:dashed}
.hl a{text-decoration:none}.hl a::after{content:" →";opacity:.35;font-weight:400}.hl a:hover::after{opacity:1}
.feed{list-style:none;padding:0}.feed li{padding:.3rem 0;border-bottom:1px dashed var(--line)}
.checks{list-style:none;padding:0;margin:.4rem 0}.checks li{padding:.35rem 0;border-bottom:1px dashed var(--line)}.checks li:last-child{border:0}
#current{scroll-margin-top:1rem}
@ -143,6 +144,13 @@ def e(s) -> str:
return html.escape(str(s))
def heading(level: str, text: str, href: str | None = None, extra: str = "") -> str:
"""A section headline. On the Home page it links to the matching sidebar section, as a shortcut."""
if href:
return f"<{level} class=hl><a href='{href}'>{html.escape(text)}</a>{extra}</{level}>"
return f"<{level}>{html.escape(text)}{extra}</{level}>"
SMILEYS = (("good", "🙂", "Went well"), ("neutral", "😐", "So-so"), ("poor", "🙁", "Poor"))
@ -177,15 +185,15 @@ class App:
return (f"<input type=hidden name={name} value='{e(u['operator'])}'>"
f"<p class=note>Running as <b>{label}</b> · <a href='/user'>change</a></p>")
def activities_html(self, limit: int = 12, level: str = "h2") -> str:
def activities_html(self, limit: int = 12, level: str = "h2", link: bool = False) -> str:
events = ACT.latest(self.packs, limit)
if not events:
return f"<{level}>Latest activities</{level}><p class=note>Nothing has happened yet. Start a run or a project.</p>"
return f"{heading(level, 'Latest activities', '/activity' if link else None)}<p class=note>Nothing has happened yet. Start a run or a project.</p>"
rows = "".join(
f"<li><span class=note>{e(x['at'][:16].replace('T', ' '))}</span> · <b>{e(x['who'])}</b> {e(x['text'])} "
f"<span class=note>· {e(x['pack'])}{' · project ' + e(x['project']) if x.get('project') else ''}</span>"
f"{' · <a href=' + chr(39) + x['link'] + chr(39) + '>open</a>' if x['link'] else ''}</li>" for x in events)
return (f"<{level}>Latest activities <span class=note style='font-weight:400;font-size:.85rem'>(times in UTC)</span></{level}>"
return (f"{heading(level, 'Latest activities', '/activity' if link else None, ' <span class=note style=' + chr(39) + 'font-weight:400;font-size:.85rem' + chr(39) + '>(times in UTC)</span>')}"
f"<ul class=feed>{rows}</ul>")
def user_page(self, msg: str = "") -> bytes:
@ -209,7 +217,7 @@ class App:
return f"<input type=hidden name=_t value='{self.token}'>"
def index(self) -> bytes:
return page("Home", UP.index_sections(self) + self.activities_html() + self.runbooks_html())
return page("Home", UP.index_sections(self, home=True) + self.activities_html(link=True) + self.runbooks_html(link=True))
def runbooks_page(self) -> bytes:
return page("Runbooks", self.runbooks_html("h1"))
@ -220,7 +228,7 @@ class App:
def activity_page(self) -> bytes:
return page("Activity", self.activities_html(100, "h1"))
def runbooks_html(self, level: str = "h2") -> str:
def runbooks_html(self, level: str = "h2", link: bool = False) -> str:
rows = []
for i, p in enumerate(self.packs):
d = p.data
@ -232,7 +240,7 @@ class App:
else:
rows.append(f"<li><a href='/pack/{i}'>{e(d['title'])}</a> {UP.maturity_pill(p)} "
f"<span class=note>{e(score_line(R.scorecard(p)))}</span></li>")
return f"<{level}>Runbooks</{level}><ul>{''.join(rows) or '<li>No packs found.</li>'}</ul>"
return f"{heading(level, 'Runbooks', '/runbooks' if link else None)}<ul>{''.join(rows) or '<li>No packs found.</li>'}</ul>"
def pack_page(self, i: int) -> bytes:
p = self.packs[i]; d = p.data
@ -291,7 +299,7 @@ class App:
outcome = r.data.get("outcome")
if outcome:
note = f" - {e(r.data['outcome_note'])}" if r.data.get("outcome_note") else ""
box = f"<p><b>Run finished: {e(outcome)}</b>{note}</p>"
box = f"<p><b>Run finished: {e(outcome)}</b>{note}</p>{self.locked_note(r)}"
if outcome == "succeeded":
if r.data.get("exercise_recorded"):
box += f"<p>Pack recorded as <b>{e(r.data['exercise_recorded'])}</b>. Commit the change in the owning repository.</p>"
@ -315,7 +323,7 @@ class App:
"<p class=note>Finishing records the outcome in the pack's score. A finished run is read-only. "
"Unfinished runs count as stalled, and as abandoned after a week.</p>"
f"<form method=post action='/run/{i}/{e(rid)}/finalize'>{self.hidden()}"
"<p><input name=note class=grow style='width:100%' placeholder='comment (optional)'></p>"
f"<p><input name=note class=grow style='width:100%' placeholder='comment (optional)'></p>{self.close_feedback_fields(r)}"
f"<p class=row>{' '.join(btns)}</p></form></div>")
def feedback_form(self, i: int, rid: str, r) -> str:
@ -328,14 +336,41 @@ class App:
"Never include secrets.</p>"
f"<p><textarea name=comment rows=3 style='width:100%' placeholder='optional comment'>"
f"{e(fb['comment']) if fb else ''}</textarea></p><p class=row>{btns}</p></form>")
if fb: # collected: close the section, suggest the next step
if r.data.get("outcome"): # a closed run is locked: show what was said, offer no change
if not fb:
return ""
face = next(f for k, f, _ in SMILEYS if k == fb["rating"])
return (f"<div id=feedback class=step><h2>Feedback</h2><p>{face} {e(fb['comment'] or '(no comment)')}</p>"
"<p class=note>The run is closed and locked, so its feedback can no longer be changed.</p></div>")
if fb: # collected while the run is open: close the section
face = next(f for k, f, _ in SMILEYS if k == fb["rating"])
return ("<div id=feedback class=step><p><b>Thank you.</b> Recorded: "
f"{face} {e(fb['comment'] or '(no comment)')}</p>"
"<p><a href='/'>← Back to the runbook list</a></p>"
f"<details><summary class=note>Change my feedback</summary>{form}</details></div>")
return f"<div id=feedback class=step><h2>How was this runbook?</h2>{form}</div>"
def close_feedback_fields(self, r) -> str:
"""Optional rating and comment inside the closing form: a closed run cannot take feedback afterwards."""
if R.latest_feedback(r):
return ""
faces = " ".join(f"<label title='{e(t)}'><input type=radio name=rating value={k}> <span style='font-size:1.4rem'>{face}</span></label>"
for k, face, t in SMILEYS)
return ("<fieldset><legend>How was this runbook? <span class=note>(optional; it cannot be added after closing)</span></legend>"
f"<p class=row>{faces}</p><textarea name=fb_comment rows=2 style='width:100%' "
"placeholder='anything strange or unclear, or an idea to make it better (no secrets)'></textarea></fieldset>")
def locked_note(self, r) -> str:
"""What 'closed' means: the record is sealed and locked. A record changed outside the app is called out."""
state = R.seal_state(r)
if state == "broken":
return ("<p class=bad>⚠ This closed run's record was changed outside the app after it was sealed. "
"It no longer counts towards maturity.</p>")
base = ("This run is <b>closed and locked</b>: its record is sealed and cannot be changed. "
"Start a new run to try again.")
if state == "unsealed":
base += " (It was closed before sealing existed, so its record carries no seal.)"
return f"<p class=note>{base}</p>"
def run_page(self, i: int, rid: str, msg: str = "", query: dict | None = None) -> bytes:
p = self.packs[i]; r = R.load(p, rid); d = p.data
if p.spec == "runbook-pack/v0.2":
@ -424,7 +459,7 @@ class App:
if r.complete:
out.append("<p><b>All steps done.</b> To record this run as exercising the pack, review and apply: "
f"<code>python -m rtut run finish {e(p.path)} {e(rid)} --apply</code></p>")
if r.complete:
if r.complete or r.data.get("outcome") or R.latest_feedback(r):
out.append(self.feedback_form(i, rid, r))
out.append(self.close_run_form(i, rid, r, p))
out.append("<h2>Threat checks</h2><ul>" + "".join(f"<li>{e(t)}</li>" for t in d["threat_checks"]) + "</ul>")
@ -567,7 +602,8 @@ def make_handler(app: App, port_holder: list):
i, rid = int(parts[1]), parts[2]
r = R.load(app.packs[i], rid)
if parts[3] == "finalize":
R.finalize(r, form.get("outcome", ""), form.get("note", ""))
R.finalize(r, form.get("outcome", ""), form.get("note", ""), rating=form.get("rating") or None,
comment=form.get("fb_comment", ""))
else:
R.apply_exercise(r)
app.packs[i] = P.load(app.packs[i].path) # pick up the new exercise status

View file

@ -59,7 +59,7 @@ def maturity_block(app, i: int, pack) -> str:
# ---- index sections ---------------------------------------------------------------------------------------------
def index_sections(app) -> str:
def index_sections(app, home: bool = False) -> str:
reg = app.registry()
prj = []
for pid, d in reg.projects.items():
@ -75,8 +75,10 @@ def index_sections(app) -> str:
tpl = [f"<li><a href='/template/{e(tid)}'>{e(d.data['title'])}</a> "
f"<span class=note>{e(d.data.get('goal', ''))}</span></li>" if d.ok else f"<li class=bad>{e(tid)}: invalid</li>"
for tid, d in reg.templates.items()]
out = f"<h1>Projects</h1><ul>{''.join(prj) or '<li class=note>No projects yet. Start one from a template.</li>'}</ul>"
out += f"<h2>Project templates</h2><ul>{''.join(tpl) or '<li class=note>No templates found.</li>'}</ul>"
from .ui import heading
link = "/projects" if home else None
out = f"{heading('h1', 'Projects', link)}<ul>{''.join(prj) or '<li class=note>No projects yet. Start one from a template.</li>'}</ul>"
out += f"{heading('h2', 'Project templates', link)}<ul>{''.join(tpl) or '<li class=note>No templates found.</li>'}</ul>"
return out

View file

@ -149,7 +149,7 @@ def _flow_view(c: Ctx, ov: dict) -> str:
out.append(_next_banner(c))
for st in fl.data["stages"]:
out.append(_stage(c, st))
if c.finished or _all_complete(run):
if c.finished or _all_complete(run) or R1.latest_feedback(run):
out.append(c.app.feedback_form(c.i, c.rid, run))
out.append(_close(c))
return "".join(out)
@ -603,9 +603,8 @@ def _close(c: Ctx) -> str:
run = c.run
if c.finished:
note = f" - {e(run.data['outcome_note'])}" if run.data.get("outcome_note") else ""
return (f"<div id=close class=step><h2>Run finished: {e(run.data['outcome'])}</h2><p>{note}</p>"
"<p class=note>The outcome is part of this runbook's score and maturity evidence at this revision. "
"<a href='/'>← Back to the runbook list</a></p></div>")
return (f"<div id=close class=step><h2>Run finished: {e(run.data['outcome'])}</h2><p>{note}</p>{c.app.locked_note(run)}"
"<p class=note>The outcome is part of this runbook's score and maturity evidence at this revision.</p></div>")
fl = c.fl
steps = [run.data["steps"][s]["state"] for s in fl.steps]
gates = [run.data["gates"][g]["state"] for g in fl.gates]
@ -619,7 +618,7 @@ def _close(c: Ctx) -> str:
"<p class=note>Finishing records the outcome. A run with deviations can be finished but does not count towards maturity. "
"A finished run is read-only.</p>"
f"<form method=post action='{c.base}/finalize'>{c.h()}<p><input name=note class=grow style='width:100%' placeholder='comment (optional)'></p>"
f"<p class=row>{' '.join(btns)}</p></form></div>")
f"{c.app.close_feedback_fields(run)}<p class=row>{' '.join(btns)}</p></form></div>")
# ---- actions -------------------------------------------------------------------------------------------------------------------
@ -638,7 +637,7 @@ def handle_post(app, i: int, rid: str, parts: list[str], form: dict) -> str:
R1.add_feedback(run, form.get("rating", ""), form.get("comment", ""))
return f"{base}#feedback"
if parts == ["finalize"]:
R1.finalize(run, form.get("outcome", ""), form.get("note", ""))
R1.finalize(run, form.get("outcome", ""), form.get("note", ""), rating=form.get("rating") or None, comment=form.get("fb_comment", ""))
return f"{base}#close"
if parts == ["mode"]:
R.switch_mode(run, form.get("to", ""))

View file

@ -42,16 +42,17 @@ def test_finalize_rules(pk):
R.finalize(r, "abandoned") # already finished
def test_finished_run_is_read_only_but_takes_feedback(pk):
def test_finished_run_is_read_only_and_feedback_is_locked_too(pk):
r = R.create(pk, "bernd"); R.finalize(r, "abandoned")
for call in (lambda: R.start(r, "read-the-sign"), lambda: R.add_capture(r, "read-the-sign", "x")):
with pytest.raises(R.RunError, match="finished"):
call()
R.add_feedback(r, "poor", "gave up: unclear") # feedback stays possible
with pytest.raises(R.RunError, match="closed and locked"):
R.add_feedback(r, "poor", "gave up: unclear") # feedback goes in before closing, or with the closing form
def test_scorecard_counts_every_outcome_and_stale_open_runs(pk):
ok = R.create(pk, "a"); play_all(ok); R.finalize(ok, "succeeded"); R.add_feedback(ok, "good")
ok = R.create(pk, "a"); play_all(ok); R.finalize(ok, "succeeded", rating="good")
bad = R.create(pk, "b"); R.start(bad, "read-the-sign"); R.fail(bad, "read-the-sign", "x"); R.finalize(bad, "failed")
gone = R.create(pk, "c"); R.finalize(gone, "abandoned")
stalled = R.create(pk, "d")
@ -151,3 +152,72 @@ def test_unfinished_run_elapsed_runs_to_now_and_no_average_without_succeeded_run
t = R.run_times(r, now=datetime.now(timezone.utc) + timedelta(seconds=90))
assert not t["finished_run"] and 89 <= t["elapsed"] <= 91
assert R.scorecard(pk)["avg_seconds"] is None and R.scorecard(pk)["timed_runs"] == 0
# ---- a closed run is locked and sealed --------------------------------------------------------------------------
def test_closing_records_feedback_with_the_outcome_and_validates_before_changing_anything(pk):
r = R.create(pk, "bernd")
with pytest.raises(R.RunError, match="rating must be"):
R.finalize(r, "abandoned", rating="great")
assert not r.data.get("outcome") and "seal" not in r.data # a refused close changed nothing
R.finalize(r, "abandoned", "gave up", rating="poor", comment="step 2 was unclear")
assert R.latest_feedback(r)["comment"] == "step 2 was unclear" and r.data["outcome"] == "abandoned"
assert R.load(pk, r.data["run_id"]).data["feedback"][0]["rating"] == "poor"
def test_a_closed_record_cannot_be_saved_with_changes_but_the_exercise_marker_is_allowed(pk):
r = R.create(pk, "bernd"); R.finalize(r, "abandoned")
r2 = R.load(pk, r.data["run_id"])
r2.data["steps"]["read-the-sign"]["state"] = "done"
with pytest.raises(R.RunError, match="closed and locked"):
r2.save()
r3 = R.load(pk, r.data["run_id"])
r3.data["operator"] = "someone-else"
with pytest.raises(R.RunError, match="operator"):
r3.save()
assert R.load(pk, r.data["run_id"]).data["operator"] == "bernd" # nothing reached the disk
r4 = R.load(pk, r.data["run_id"])
r4.data["exercise_recorded"] = "x"
r4.save(allow=("exercise_recorded",)) # the one permitted later update
assert R.load(pk, r.data["run_id"]).data["exercise_recorded"] == "x" and R.seal_state(R.load(pk, r.data["run_id"])) == "sealed"
def test_closing_seals_the_file_read_only_and_tampering_is_detected(pk):
import json, stat
r = R.create(pk, "bernd")
assert R.seal_state(r) == "open" and r.path.stat().st_mode & stat.S_IWUSR
R.finalize(r, "abandoned")
assert R.seal_state(r) == "sealed" and not (r.path.stat().st_mode & (stat.S_IWUSR | stat.S_IWGRP | stat.S_IWOTH))
assert len(r.data["seal"]) == 64
r.path.chmod(0o644) # someone edits the file by hand
data = json.loads(r.path.read_text()); data["operator"] = "forged"
r.path.write_text(json.dumps(data))
assert R.seal_state(R.load(pk, r.data["run_id"])) == "broken"
def test_a_run_closed_before_sealing_existed_is_reported_as_unsealed_not_broken(pk):
import json
r = R.create(pk, "bernd"); R.finalize(r, "abandoned")
r.path.chmod(0o644)
data = json.loads(r.path.read_text()); data.pop("seal"); r.path.write_text(json.dumps(data))
assert R.seal_state(R.load(pk, r.data["run_id"])) == "unsealed"
def test_a_tampered_run_no_longer_counts_as_maturity_evidence(pk):
import json
from rtut import maturity as M
M.add_review(pk, "alice")
r = R.create(pk, "bob", {"logdir": "/tmp/rtut-lock-mat"}, {"project": "lab", "environment": "rehearsal"})
for s in pk.data["steps"]:
sid = s["id"]; R.start(r, sid)
cmd = R.command_for(r, sid)
if cmd:
subprocess.run(cmd, shell=True, check=True, capture_output=True)
R.verify(r, sid, confirm=True, manual_ok=True if s["verify"]["expect"] == "manual" else None)
R.complete(r, sid, {n: "1" for n in s.get("evidence", [])})
R.finalize(r, "succeeded")
assert M.maturity(pk)["evidence"]["rehearsal_runs"] == 1
r.path.chmod(0o644)
data = json.loads(r.path.read_text()); data["operator"] = "forged"; r.path.write_text(json.dumps(data))
assert M.maturity(pk)["evidence"]["clean_runs"] == 0 # changed after closing: not evidence

View file

@ -205,18 +205,18 @@ def test_feedback_form_after_all_steps_done_and_recording(srv):
t, loc, _ = _showcase_run(srv)
base = loc.split("#")[0]
_, _, body = req(srv, "GET", base)
assert "How was this runbook?" not in body # not before the run is complete
assert "<h2>How was this runbook?</h2>" not in body # not before the run is complete
_finish_showcase(srv, t, base)
_, _, body = req(srv, "GET", base)
assert "All steps done." in body and "How was this runbook?" in body
assert body.index("All steps done.") < body.index("How was this runbook?")
assert "All steps done." in body and "<h2>How was this runbook?</h2>" in body
assert body.index("All steps done.") < body.index("<h2>How was this runbook?</h2>")
assert "🙂" in body and "😐" in body and "🙁" in body and "name=comment" in body
st, loc2, _ = req(srv, "POST", base + "/feedback", {"_t": t, "rating": "neutral", "comment": "step 3 was unclear"})
assert st == 303 and loc2.endswith("#feedback")
_, _, body = req(srv, "GET", base)
assert "Thank you." in body and "step 3 was unclear" in body
assert "How was this runbook?" not in body # the section closed once feedback was collected
assert "<a href='/'>← Back to the runbook list</a>" in body # the probable next step
assert "<h2>How was this runbook?</h2>" not in body # the section closed once feedback was collected
assert "Back to the runbook list" not in body # the sidebar's Home replaces that link
assert "Change my feedback" in body # the form is still reachable, folded away
assert req(srv, "GET", "/")[0] == 200

152
tests/test_ui_locked.py Normal file
View file

@ -0,0 +1,152 @@
import http.client
import json
import shutil
import subprocess
import threading
import uuid
from pathlib import Path
from urllib.parse import urlencode
import pytest
from rtut import run as R1, run2 as R, ui
ROOT = Path(__file__).resolve().parent.parent
_scratch = []
@pytest.fixture
def srv(tmp_path):
for name, src in (("v1", "showcase-lighthouse"), ("v2", "showcase-lighthouse-v2")):
d = tmp_path / "runbooks" / name; d.mkdir(parents=True)
shutil.copy(ROOT / f"runbooks/{src}/pack.yaml", d / "pack.yaml")
s = ui.serve([tmp_path], 0)
threading.Thread(target=s.serve_forever, daemon=True).start()
s.repo = tmp_path
yield s
s.shutdown()
while _scratch:
shutil.rmtree(_scratch.pop(), ignore_errors=True)
def req(s, method, path, form=None):
c = http.client.HTTPConnection("127.0.0.1", s.server_address[1])
body = urlencode(form) if form is not None else None
c.request(method, path, body, {"Host": "127.0.0.1", "Content-Type": "application/x-www-form-urlencoded"})
r = c.getresponse()
return r.status, r.getheader("Location"), r.read().decode()
def post(s, path, **form):
return req(s, "POST", path, {"_t": s.app.token, **form})
def start(s, pack_id):
i = s.app.pack_index(pack_id)
logdir = f"/tmp/rtut-lock-{uuid.uuid4().hex[:8]}"
_scratch.append(logdir)
st, loc, _ = post(s, f"/run/new/{i}", operator="bernd", p_logdir=logdir)
assert st == 303
return loc.split("#")[0], i
def run_of(s, base):
return R1.load(s.app.packs[int(base.split("/")[2])], base.rsplit("/", 1)[1])
# ---- Home headlines are shortcuts ---------------------------------------------------------------------------
def test_home_headlines_link_to_the_sidebar_sections_and_the_section_pages_do_not(srv):
_, _, home = req(srv, "GET", "/")
assert "<h1 class=hl><a href='/projects'>Projects</a></h1>" in home
assert "<h2 class=hl><a href='/projects'>Project templates</a></h2>" in home
assert "<a href='/activity'>Latest activities</a>" in home and "<h2 class=hl><a href='/runbooks'>Runbooks</a></h2>" in home
_, _, projects = req(srv, "GET", "/projects")
assert "<h1>Projects</h1>" in projects and "class=hl" not in projects # on its own page the headline is just a headline
assert "<h1>Runbooks</h1>" in req(srv, "GET", "/runbooks")[2]
assert "<h1>Latest activities" in req(srv, "GET", "/activity")[2]
# ---- no "back to the runbook list" -------------------------------------------------------------------------------
def test_there_is_no_back_to_the_runbook_list_link_in_either_run_view(srv):
for pack in ("rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"):
base, i = start(srv, pack)
assert post(srv, f"{base}/feedback", rating="good", comment="fine")[0] == 303
assert "Back to the runbook list" not in req(srv, "GET", base)[2]
assert post(srv, f"{base}/finalize", outcome="abandoned")[0] == 303
assert "Back to the runbook list" not in req(srv, "GET", base)[2]
# ---- closed runs are locked ------------------------------------------------------------------------------------------
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
def test_a_closed_run_shows_its_feedback_read_only_without_a_change_option(srv, pack):
base, _ = start(srv, pack)
assert post(srv, f"{base}/feedback", rating="neutral", comment="a bit long")[0] == 303
assert "Change my feedback" in req(srv, "GET", base)[2] # while open it can still be changed
assert post(srv, f"{base}/finalize", outcome="abandoned")[0] == 303
body = req(srv, "GET", base)[2]
assert "Change my feedback" not in body and f"action='{base}/feedback'" not in body
assert "<h2>Feedback</h2>" in body and "a bit long" in body and "closed and locked, so its feedback can no longer be changed" in body
st, _, err = post(srv, f"{base}/feedback", rating="good", comment="changed my mind")
assert st == 400 and "closed and locked" in err
assert run_of(srv, base).data["feedback"][0]["comment"] == "a bit long" # nothing reached the record
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
def test_a_closed_run_without_feedback_shows_no_feedback_section(srv, pack):
base, _ = start(srv, pack)
assert post(srv, f"{base}/finalize", outcome="abandoned")[0] == 303
body = req(srv, "GET", base)[2]
assert "<h2>Feedback</h2>" not in body and "<h2>How was this runbook?</h2>" not in body and "Change my feedback" not in body
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
def test_the_closing_form_takes_optional_feedback_so_abandoned_runs_are_not_silent(srv, pack):
base, _ = start(srv, pack)
body = req(srv, "GET", base)[2]
assert "How was this runbook? <span class=note>(optional; it cannot be added after closing)</span>" in body and "name=fb_comment" in body
assert post(srv, f"{base}/finalize", outcome="abandoned", rating="poor", fb_comment="step 1 was unclear")[0] == 303
body = req(srv, "GET", base)[2]
assert "<h2>Feedback</h2>" in body and "step 1 was unclear" in body
assert run_of(srv, base).data["feedback"][0]["rating"] == "poor"
def test_the_closing_form_does_not_ask_again_when_feedback_was_already_given(srv):
base, _ = start(srv, "rtut.showcase-lighthouse-v2")
post(srv, f"{base}/feedback", rating="good", comment="ok")
assert "name=fb_comment" not in req(srv, "GET", base)[2]
def test_a_bad_rating_on_closing_refuses_the_whole_close(srv):
base, _ = start(srv, "rtut.showcase-lighthouse-v2")
st, _, err = post(srv, f"{base}/finalize", outcome="abandoned", rating="great")
assert st == 400 and "rating must be" in err
assert not run_of(srv, base).data.get("outcome") # still open
@pytest.mark.parametrize("pack", ["rtut.showcase-lighthouse", "rtut.showcase-lighthouse-v2"])
def test_a_closed_run_says_it_is_locked_and_refuses_every_change(srv, pack):
base, _ = start(srv, pack)
post(srv, f"{base}/finalize", outcome="abandoned")
body = req(srv, "GET", base)[2]
assert "closed and locked" in body and "its record is sealed and cannot be changed" in body
first = "read-the-sign"
for path, form in ((f"{base}/{first}/start", {}), (f"{base}/mode", {"to": "auto-confirm"}), (f"{base}/data", {"d_weather": "clear"}),
(f"{base}/{first}/capture", {"output": "x"}), (f"{base}/finalize", {"outcome": "succeeded"})):
st, _, err = post(srv, path, **form)
assert st in (400,) and ("closed" in err or "finished" in err or "already" in err), path
assert run_of(srv, base).data["outcome"] == "abandoned"
def test_the_record_file_is_read_only_after_closing_and_tampering_is_called_out(srv):
base, _ = start(srv, "rtut.showcase-lighthouse-v2")
post(srv, f"{base}/finalize", outcome="abandoned")
r = run_of(srv, base)
assert r.path.stat().st_mode & 0o222 == 0 and R1.seal_state(r) == "sealed"
assert "changed outside the app" not in req(srv, "GET", base)[2]
r.path.chmod(0o644)
data = json.loads(r.path.read_text()); data["operator"] = "forged"; r.path.write_text(json.dumps(data))
body = req(srv, "GET", base)[2]
assert "changed outside the app after it was sealed" in body and "no longer counts towards maturity" in body

View file

@ -59,7 +59,7 @@ def make_project(s, pid="harbour", host=None):
def test_index_has_projects_templates_runbooks_and_maturity(psrv):
_, _, body = req(psrv, "GET", "/")
assert "<h1>Projects</h1>" in body and "Project templates" in body and "<h2>Runbooks</h2>" in body
assert "<h1 class=hl><a href='/projects'>Projects</a></h1>" in body and "Project templates" in body and "<a href='/runbooks'>Runbooks</a>" in body
assert "/template/lighthouse-watch" in body
assert ">draft<" in body # maturity pill on every native runbook

View file

@ -91,7 +91,7 @@ def test_sections_are_separate_pages(srv):
_, _, body = req(srv, "GET", "/activity")
assert "<h1>Latest activities" in body and "(times in UTC)" in body and "started a run" in body
_, _, body = req(srv, "GET", "/")
assert "<h1>Projects</h1>" in body and "Latest activities" in body and "<h2>Runbooks</h2>" in body # Home shows all three
assert "<h1 class=hl><a href='/projects'>Projects</a></h1>" in body and "Latest activities" in body and "<h2 class=hl><a href='/runbooks'>Runbooks</a></h2>" in body # Home shows all three, as shortcuts
# ---- attribute sets ----------------------------------------------------------------------------------------------------

View file

@ -314,9 +314,9 @@ def test_finishing_a_completed_run_feedback_and_the_read_only_result(s2):
do_step(s2, base, "close-log", phrase=PHRASE)
pass_gate(s2, base, "night-on-record")
body = page(s2, base)
assert "Everything is done." in body and "Finish: succeeded" in body and "How was this runbook?" in body
assert "Everything is done." in body and "Finish: succeeded" in body and "<h2>How was this runbook?</h2>" in body
assert post(s2, f"{base}/feedback", rating="good", comment="clear")[0] == 303
assert "Back to the runbook list" in page(s2, base)
assert "Back to the runbook list" not in page(s2, base) and "Change my feedback" in page(s2, base) # still open: changeable
assert post(s2, f"{base}/finalize", outcome="succeeded", note="smooth")[0] == 303
body = page(s2, base)
assert "Run finished: succeeded" in body and "Start this step" not in body

View file

@ -246,6 +246,19 @@ pages; attribute sets listed, inspected with their resolved values and sources,
secrets, bad names, unknown and circular `extends` refused, a refused save changes nothing); the project name, description and goal
editable from the project page. Bernd confirmed the theme and navigation. Not done: deleting an attribute set, editing templates.
## Closed runs locked; Home headlines as shortcuts
```task
id: RBT-WP-0004-T16
status: done
priority: high
```
Done 2026-09-29 from Bernd's feedback: Home headlines link to their sidebar sections; the "Back to the runbook list" link is gone; closed runs are
locked (engine refuses changes, feedback included), sealed with a hash and made read-only, tampering is shown and disqualifies the run as maturity
evidence; the closing form takes optional feedback so abandoned runs are not silent; a closed run shows its feedback read-only (no "Change my
feedback"), or no feedback section at all. See `docs/run-spec.md`.
## Acceptance Criteria
- Bernd can complete the SSH tutorial pack in the UI without pasting outputs to an agent.