From 639224b2eb69a3f8a85a73fcc8c789d6d224f5d1 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sat, 3 Oct 2026 00:01:14 +0200 Subject: [PATCH] Retire the markdown tutorial verifier; close NK-WP-0044 and NK-WP-0009 on rehearsal receipts Co-Authored-By: Claude Sonnet 5.5 Assistant: claude-code Assistant-Model: sonnet Assistant-Process: 6969@bnt-lap001 Assistant-Session: e8ae73f2-8385-4c9c-b714-94c3c52cdbc0 --- Makefile | 8 +- SCOPE.md | 6 +- docs/tutorials/README.md | 52 ++++--------- docs/tutorials/TEMPLATE.md | 41 ---------- docs/tutorials/openbao-operating-path.md | 2 +- docs/tutorials/protected-system-flex-auth.md | 2 +- .../tutorials/ssh-certificates-and-tunnels.md | 2 +- .../tests/test_tutorial_verify.py | 48 ------------ tools/tutorial-verify/tutorial_verify.py | 78 ------------------- ...9-netkingdom-security-pattern-tutorials.md | 9 ++- ...044-runbook-packs-for-runbook-tutorials.md | 22 ++++-- 11 files changed, 43 insertions(+), 227 deletions(-) delete mode 100644 docs/tutorials/TEMPLATE.md delete mode 100644 tools/tutorial-verify/tests/test_tutorial_verify.py delete mode 100644 tools/tutorial-verify/tutorial_verify.py diff --git a/Makefile b/Makefile index 425c938..1880c2f 100644 --- a/Makefile +++ b/Makefile @@ -186,17 +186,11 @@ openbao-init-unseal: ## SOPS-held OpenBao init/unseal (NET-WP-0020 T2; requires openbao-init-unseal-dry-run: ## Dry-run the SOPS-held OpenBao init/unseal path @bash sso-mfa/bootstrap/openbao-init-unseal.sh --dry-run -tutorials-verify: ## Verify docs/tutorials structure, ownership tags, and references - python3 tools/tutorial-verify/tutorial_verify.py - RTUT_HOME ?= $(HOME)/runbook-tutorials runbooks-validate: ## Validate runbook packs under runbooks/ with the runbook-tutorials engine PYTHONPATH=$(RTUT_HOME) python3 -m rtut validate . -tutorials-verify-test: ## Run tutorial verifier tests - python3 -m pytest tools/tutorial-verify/tests - iam-profile-conformance-test: ## Run current IAM Profile conformance fixture tests python3 -m pytest tools/iam-profile-conformance/tests @@ -384,7 +378,7 @@ security-bootstrap-ui: security-bootstrap-metadata-init ## Serve local custody a --host "$(SECURITY_BOOTSTRAP_HOST)" \ --port "$(SECURITY_BOOTSTRAP_PORT)" -.PHONY: runbooks-validate tutorials-verify tutorials-verify-test help hooks hooks-test sops-setup sops-edit sops-encrypt sops-decrypt sops-rotate \ +.PHONY: runbooks-validate help hooks hooks-test sops-setup sops-edit sops-encrypt sops-decrypt sops-rotate \ check-secrets creds-init creds-generate creds-bundle creds-apply creds-verify \ creds-status creds-rotate \ creds-agent-init creds-agent-status creds-emergency-reprint \ diff --git a/SCOPE.md b/SCOPE.md index 9c698a4..ab073b4 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -139,14 +139,14 @@ The current owner/evidence gates are: machine-readable authoritative ownership and E2 freshness metadata. - NK-WP-0035: corrected candidate contract pins do not resolve source migration, local-identity's missing heartbeat, or the absent source/observer proof. -- NK-WP-0039: obsolete flex-auth reference objects have been removed; remaining - tenant-engine references and repository-rename pointers await their owners. +- NK-WP-0039: finished; flex-auth and tenant-engine reference objects are retired and + `sso-mfa/k8s/tenant-engine/runtime.yaml` holds owner pointers only. - NK-WP-0040: execution-attribution receipt emission, custody and schema require owner agreement before an end-to-end implementation claim. - NK-WP-0042: reuse delivered enrollment/policy components for the agreed pilot; generic workload step-up is not established by those two scoped clients. -Tutorials (NK-WP-0009) and enterprise federation (NK-WP-0011) remain backlog. +Tutorials (NK-WP-0009) are delivered as rehearsed runbook packs under `runbooks/`; enterprise federation (NK-WP-0011) remains backlog. NK-WP-0030's deterministic composition and NK-WP-0031's local feedback tooling are implemented; neither autonomously changes policy. Use the workplan files and generated `WORK-RECORDS.md` for changing task state, rather than treating diff --git a/docs/tutorials/README.md b/docs/tutorials/README.md index 543bebb..58ba5e8 100644 --- a/docs/tutorials/README.md +++ b/docs/tutorials/README.md @@ -1,41 +1,21 @@ # NetKingdom Security Pattern Tutorials -Hands-on paths for operating the canonical NetKingdom security patterns -(NK-WP-0009). Each tutorial is a file in this directory, written from -[`TEMPLATE.md`](TEMPLATE.md) and checked by `make tutorials-verify`. +The tutorials are now **runbook packs** in [`runbooks/`](../../runbooks/) for the +`runbook-tutorials` engine (NK-WP-0009, NK-WP-0044). Run them through the engine's UI +or `rtut`; check them with `make runbooks-validate`. The markdown verifier is retired. -> **Moving.** Tutorials are becoming runbook packs in `runbooks/` for the -> `runbook-tutorials` engine (NK-WP-0044). Packs: `runbooks/ssh-certificates/`, `runbooks/openbao-operating-path/`, -> `runbooks/protected-system-flex-auth/`. -> These markdown files stay until their packs are exercised. +| Pack | Replaces | Status | +| --- | --- | --- | +| `runbooks/object-storage-sts/` | (new) vend temporary S3 credentials | rehearsed 2026-10-02 | +| `runbooks/ssh-certificates/` | [ssh-certificates-and-tunnels.md](ssh-certificates-and-tunnels.md) | rehearsed 2026-10-02 | +| `runbooks/openbao-operating-path/` | [openbao-operating-path.md](openbao-operating-path.md) | rehearsed 2026-10-02 | +| `runbooks/protected-system-flex-auth/` | [protected-system-flex-auth.md](protected-system-flex-auth.md) | rehearsed 2026-10-02 | -## Rules +Maturity is derived by the engine from run history, not declared here. The three markdown +files are kept as superseded reference. The flex-auth one still holds check N4 (an expired +token is refused), which needs a ten-minute wait and is not in the pack. -1. **Exercise status is mandatory.** Per - [`docs/attended-procedure-standard.md`](../attended-procedure-standard.md), a - tutorial header says `exercised by ` or `unexercised`. - Nothing is labelled exercised until someone has run it. -2. **Every concrete step names its owning repo.** This repo owns canon and - reference tooling only (see `SCOPE.md`); deployment belongs to owners. -3. **Verification and rollback are required**, not optional happy-path extras. -4. **No secrets, ever.** Tutorials show paths and commands, never values. -5. **Consume, don't copy.** Link owner runbooks; do not paste runtime - manifests. Use the named `openbao-ui-railiance01` tunnel, never a public - Bao URL (`bao.coulomb.social` is retired). - -## Index - -| Tutorial | Workplan task | Owners | Status | -| --- | --- | --- | --- | -| [OpenBao: consume, attend, recover](openbao-operating-path.md) | T03 | railiance-platform, net-kingdom | unexercised | -| [Short-lived SSH credentials](ssh-certificates-and-tunnels.md) | T04 | ops-warden, ops-bridge | unexercised | -| [Add a protected system to flex-auth](protected-system-flex-auth.md) | T05 | flex-auth, package owner | unexercised (offline part run) | - -Deferred (see NK-WP-0009): T02 object-storage STS (needs an owner-backed -issuer and refusal/lease proof — ADR-0008 is architecture, not evidence). - -## Pattern mapping - -NK-WP-0008 (the pattern library) has no file in this repo, so tutorials map to -the canonical documents directly: `docs/platform-identity-security-architecture.md`, -`docs/responsibility-map.md`, `docs/platform-root-custody.md`. +Rules that still apply to every pack: name the owning repo for each step, include +verification and rollback, never put secrets in a pack, link owner runbooks instead of +copying manifests, and use the named `openbao-ui-railiance01` tunnel, never a public Bao +URL (`bao.coulomb.social` is retired). diff --git a/docs/tutorials/TEMPLATE.md b/docs/tutorials/TEMPLATE.md deleted file mode 100644 index 30a862f..0000000 --- a/docs/tutorials/TEMPLATE.md +++ /dev/null @@ -1,41 +0,0 @@ -# - -Exercise status: unexercised -Workplan task: NK-WP-NNNN-TNN -Pattern(s): - -## Outcome - - - -## Prerequisites - -- - -## Architecture context - - - -## Steps - -1. **[owner: ]** - -## Verification - -Done when: - -- - -## Rollback - -- - -## Threat checks - -- - -## Ownership notes - -| Concern | Owner | -| --- | --- | -| | | diff --git a/docs/tutorials/openbao-operating-path.md b/docs/tutorials/openbao-operating-path.md index 4eab152..300ba73 100644 --- a/docs/tutorials/openbao-operating-path.md +++ b/docs/tutorials/openbao-operating-path.md @@ -1,6 +1,6 @@ # OpenBao: consume, attend, recover -Exercise status: unexercised +Status: superseded by the runbook pack `runbooks/openbao-operating-path/` (rehearsed 2026-10-02). Kept as reference; the pack is authoritative. Workplan task: NK-WP-0009-T03 Pattern(s): platform-root custody (`docs/platform-root-custody.md`); credential routing diff --git a/docs/tutorials/protected-system-flex-auth.md b/docs/tutorials/protected-system-flex-auth.md index 900ca22..f958321 100644 --- a/docs/tutorials/protected-system-flex-auth.md +++ b/docs/tutorials/protected-system-flex-auth.md @@ -1,6 +1,6 @@ # Add a protected system to flex-auth -Exercise status: unexercised +Status: superseded by the runbook pack `runbooks/protected-system-flex-auth/` (rehearsed 2026-10-02). Kept as reference; the pack is authoritative. Workplan task: NK-WP-0009-T05 Pattern(s): authorization boundary (`docs/platform-identity-security-architecture.md`, ADR-0006); IAM Profile v0.3 consumption diff --git a/docs/tutorials/ssh-certificates-and-tunnels.md b/docs/tutorials/ssh-certificates-and-tunnels.md index 12f8f75..f07f28e 100644 --- a/docs/tutorials/ssh-certificates-and-tunnels.md +++ b/docs/tutorials/ssh-certificates-and-tunnels.md @@ -1,6 +1,6 @@ # Short-lived SSH credentials for admins, agents and automations -Exercise status: unexercised +Status: superseded by the runbook pack `runbooks/ssh-certificates/` (rehearsed 2026-10-02). Kept as reference; the pack is authoritative. Workplan task: NK-WP-0009-T04 Pattern(s): credential routing; ops-warden `AccessManagementDirective` diff --git a/tools/tutorial-verify/tests/test_tutorial_verify.py b/tools/tutorial-verify/tests/test_tutorial_verify.py deleted file mode 100644 index b942d11..0000000 --- a/tools/tutorial-verify/tests/test_tutorial_verify.py +++ /dev/null @@ -1,48 +0,0 @@ -import sys -from pathlib import Path - -sys.path.insert(0, str(Path(__file__).resolve().parents[1])) -import tutorial_verify as tv # noqa: E402 - -ROOT = Path(__file__).resolve().parents[3] -GOOD = (ROOT / "docs/tutorials/TEMPLATE.md").read_text() - - -def write(tmp_path, text): - p = tmp_path / "t.md" - p.write_text(text) - return p - - -def test_real_tutorials_pass(): - for p in (ROOT / "docs/tutorials").glob("*.md"): - if p.name in ("README.md", "TEMPLATE.md"): - continue - assert tv.check(p, ROOT) == [], p.name - - -def test_missing_status(tmp_path): - errs = tv.check(write(tmp_path, GOOD.replace("Exercise status: unexercised\n", "")), ROOT) - assert any("Exercise status" in e for e in errs) - - -def test_missing_rollback(tmp_path): - errs = tv.check(write(tmp_path, GOOD.replace("## Rollback", "## Other")), ROOT) - assert "missing section: Rollback" in errs - - -def test_step_without_owner(tmp_path): - errs = tv.check(write(tmp_path, GOOD.replace("**[owner: ]** ", "")), ROOT) - assert any("owner tag" in e for e in errs) - - -def test_secret_marker(tmp_path): - errs = tv.check(write(tmp_path, GOOD + "\nhvs.ABCDEFGHIJKLMNOP\n"), ROOT) - assert "contains secret-looking marker" in errs - - -def test_retired_endpoint_and_missing_path(tmp_path): - txt = GOOD + "\nOpen https://bao.coulomb.social now. See `docs/nope.md`.\n" - errs = tv.check(write(tmp_path, txt), ROOT) - assert any("bao.coulomb.social" in e for e in errs) - assert "references missing path: docs/nope.md" in errs diff --git a/tools/tutorial-verify/tutorial_verify.py b/tools/tutorial-verify/tutorial_verify.py deleted file mode 100644 index f58846a..0000000 --- a/tools/tutorial-verify/tutorial_verify.py +++ /dev/null @@ -1,78 +0,0 @@ -#!/usr/bin/env python3 -"""Structural verifier for docs/tutorials (NK-WP-0009-T06). - -Fails a tutorial that is prose-only: missing required sections, missing or -invalid exercise status, no per-step owner tags, retired endpoints, secret -markers, or references to repo paths that do not exist. -""" -from __future__ import annotations - -import re -import sys -from pathlib import Path - -REQUIRED_SECTIONS = [ - "Outcome", "Prerequisites", "Architecture context", "Steps", - "Verification", "Rollback", "Threat checks", "Ownership notes", -] -STATUS_RE = re.compile( - r"^Exercise status: (unexercised|exercised \d{4}-\d{2}-\d{2} by \S+)\s*$", re.M) -OWNER_TAG_RE = re.compile(r"\*\*\[owner: [^\]]+\]\*\*") -SECRET_RE = re.compile( - r"(hvs\.[A-Za-z0-9]{8,}|s\.[A-Za-z0-9]{24}|-----BEGIN [A-Z ]*PRIVATE KEY|otpauth://)") -PATH_RE = re.compile(r"`((?:docs|tools|canon|workplans)/[\w./-]+)`") -RETIRED_MENTION_OK = "retired" - - -def check(path: Path, root: Path) -> list[str]: - text = path.read_text() - errs: list[str] = [] - if not STATUS_RE.search(text): - errs.append("missing or invalid 'Exercise status:' header") - headings = set(re.findall(r"^## (.+?)\s*$", text, re.M)) - for s in REQUIRED_SECTIONS: - if s not in headings: - errs.append(f"missing section: {s}") - steps = re.search(r"^## Steps\s*$(.*?)(?=^## |\Z)", text, re.M | re.S) - if steps: - items = re.findall(r"^\d+\. .*$", steps.group(1), re.M) - if not items: - errs.append("Steps has no numbered steps") - for i in items: - if not OWNER_TAG_RE.search(i): - errs.append(f"step lacks owner tag: {i[:50]}") - ver = re.search(r"^## Verification\s*$(.*?)(?=^## |\Z)", text, re.M | re.S) - if ver and "Done when" not in ver.group(1): - errs.append("Verification lacks a 'Done when' outcome") - for line in text.splitlines(): - if "bao.coulomb.social" in line and RETIRED_MENTION_OK not in line: - errs.append("references bao.coulomb.social without marking it retired") - if SECRET_RE.search(text): - errs.append("contains secret-looking marker") - for ref in PATH_RE.findall(text): - if "<" in ref or "*" in ref: - continue - if not (root / ref).exists(): - errs.append(f"references missing path: {ref}") - return errs - - -def main(argv: list[str]) -> int: - root = Path(__file__).resolve().parents[2] - tdir = Path(argv[1]) if len(argv) > 1 else root / "docs" / "tutorials" - files = sorted(p for p in tdir.glob("*.md") if p.name not in ("README.md", "TEMPLATE.md")) - if not files: - print("no tutorials found", file=sys.stderr) - return 1 - failed = 0 - for f in files: - errs = check(f, root) - print(f"{'FAIL' if errs else 'ok '} {f.name}") - for e in errs: - print(f" - {e}") - failed += bool(errs) - return 1 if failed else 0 - - -if __name__ == "__main__": - sys.exit(main(sys.argv)) diff --git a/workplans/NK-WP-0009-netkingdom-security-pattern-tutorials.md b/workplans/NK-WP-0009-netkingdom-security-pattern-tutorials.md index 46a4c4d..6e55c84 100644 --- a/workplans/NK-WP-0009-netkingdom-security-pattern-tutorials.md +++ b/workplans/NK-WP-0009-netkingdom-security-pattern-tutorials.md @@ -4,14 +4,14 @@ type: workplan title: NetKingdom Security Pattern Tutorials domain: infotech repo: net-kingdom -status: active +status: finished flavor: implementation owner: codex topic_slug: netkingdom planning_priority: medium planning_order: 9 created: 2026-05-17 -updated: "2026-09-28" +updated: "2026-10-03" depends_on: - NK-WP-0008 state_hub_workstream_id: "d4d02dbf-3974-502d-8b87-b776fc63e17e" @@ -128,7 +128,7 @@ implementation. ```task id: NK-WP-0009-T05 -status: progress +status: done priority: medium state_hub_task_id: "c380ef19-4bc2-5ade-b127-baf18c64bf35" ``` @@ -185,3 +185,6 @@ Bernd ran three packs cleanly in project `nk-rehearsal` (rehearsal environment, and was retried with the missing `WARDEN_CONFIG`), `nk.openbao-operating-path` (20261002-234245-d73656, T03). T02, T03 and T04 are done. Limits: the S3 pack proves the contract offline, not a live service; the SSH tunnel stage is recorded not proven and the rehearsal config bypasses the flex-auth policy gate; the OpenBao pack is read-only. T05 (`nk.protected-system-flex-auth`) has no run yet. + +`nk.protected-system-flex-auth` (run 20261002-235239-c46227) completed cleanly in `nk-rehearsal`, so T05 is done and all four tutorials are +rehearsed packs. The markdown verifier is retired (NK-WP-0044-T06). diff --git a/workplans/NK-WP-0044-runbook-packs-for-runbook-tutorials.md b/workplans/NK-WP-0044-runbook-packs-for-runbook-tutorials.md index c428a67..6c91172 100644 --- a/workplans/NK-WP-0044-runbook-packs-for-runbook-tutorials.md +++ b/workplans/NK-WP-0044-runbook-packs-for-runbook-tutorials.md @@ -4,12 +4,12 @@ type: workplan title: "Provide NetKingdom runbook packs for the runbook-tutorials engine" domain: infotech repo: net-kingdom -status: active +status: finished flavor: implementation owner: claude topic_slug: netkingdom created: "2026-09-29" -updated: "2026-09-29" +updated: "2026-10-03" related: [NK-WP-0009] state_hub_workstream_id: "82414324-e9cf-5581-af9e-b0122411d7bf" --- @@ -80,9 +80,7 @@ checkout (`RTUT_HOME`, default `~/runbook-tutorials`). ```task id: NK-WP-0044-T05 -status: wait -needs_human: true -blocking_reason: "Bernd runs the three v0.2 packs in the UI against the cluster; the engine records the outcomes." +status: done priority: high state_hub_task_id: "2d99180e-25ff-51db-8208-b0320c8ec7e3" ``` @@ -130,9 +128,7 @@ specifics). ```task id: NK-WP-0044-T06 -status: wait -depends_on: [NK-WP-0044-T05] -blocking_reason: "Once the tutorials are packs (T05)." +status: done priority: low state_hub_task_id: "1b747058-d4df-5f5c-bd63-e85d46dd094e" ``` @@ -159,3 +155,13 @@ none of the gaps G1 to G7 in `runbook-tutorials/docs/console-mapping.md`. The co - The console still works and is launched by the engine without changes to it. - Every NetKingdom pack validates; maturity is derived from run history, not declared. + +## Rehearsal complete and verifier retired — 2026-10-03 + +Bernd ran `nk.protected-system-flex-auth` (run 20261002-235239-c46227): 15 steps and both gates passed, no deviations; the engine derives +`rehearsed`. All four packs are applied in `nk-rehearsal` (T05). T06: `tools/tutorial-verify`, its tests, `make tutorials-verify*` and +`docs/tutorials/TEMPLATE.md` are removed; `docs/tutorials/README.md` is a pointer; the three markdown tutorials stay as superseded +reference (the flex-auth one still holds check N4, which the pack omits). Recorded feedback: ssh "more understandable", openbao neutral +("we need to make it easier to use, but it helps"), flex-auth neutral ("ok, but this is hard tasks"). The four HTTP codes of the live checks were +not captured as evidence in the run. Pack improvements (explicit rehearsal-config step and check in the ssh pack, `route` BrokenPipe noise, +easier-to-use feedback) are deliberately deferred to the next pack revision so the rehearsed evidence is not reset now.