NK-WP-0056-T12 done: pilot operations; synthetic rotation and revert pass
operations.py: OpenBao KV v2 CAS, LLDAP self-rotation (forward and revert), ExternalSecret force-sync and status, rollout restart (RolloutFailed is a definite failure), pilot checks. synthetic_pilot.py on the T08 stack plus OpenBao 2.5.4 with CCR-2026-0058's policy: rotation completed; a one-off wrong delivery reverted in declared order with the earlier value restored. Runner: revert steps run in declared order (BEA-10), definite failures revert, unknown outcomes stop, receipts carry error types. Found and corrected (in railiance-platform) CCR-2026-0058's required_parameters refusing reads. Packaging moved to new T13. 29 executor tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 994874@bnt-lap001 Assistant-Session: 6017c416-d619-47eb-bd1a-f670a02d7d87
This commit is contained in:
parent
9b7701a32a
commit
db56d8468e
11 changed files with 1088 additions and 33 deletions
|
|
@ -354,7 +354,8 @@
|
||||||
| task | NK-WP-0056-T09 | done | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
| task | NK-WP-0056-T09 | done | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
||||||
| task | NK-WP-0056-T10 | todo | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
| task | NK-WP-0056-T10 | todo | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
||||||
| task | NK-WP-0056-T11 | done | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
| task | NK-WP-0056-T11 | done | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
||||||
| task | NK-WP-0056-T12 | todo | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
| task | NK-WP-0056-T12 | done | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
||||||
|
| task | NK-WP-0056-T13 | todo | — | workplans/NK-WP-0056-bounded-executor-path.md |
|
||||||
| intake | NK-IN-0001 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |
|
| intake | NK-IN-0001 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |
|
||||||
| intake | NK-IN-0002 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |
|
| intake | NK-IN-0002 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |
|
||||||
| intake | NET-IN-0001 | open | — | intakes/intakes.md |
|
| intake | NET-IN-0001 | open | — | intakes/intakes.md |
|
||||||
|
|
|
||||||
|
|
@ -28,9 +28,23 @@ Job per admission in `sso` (T12).
|
||||||
- **receipt:** one `execution.attribution.receipt` per run, with PDP `not-evaluated` in the pilot. It never holds a
|
- **receipt:** one `execution.attribution.receipt` per run, with PDP `not-evaluated` in the pilot. It never holds a
|
||||||
value that an operation handled.
|
value that an operation handled.
|
||||||
|
|
||||||
Operations, checks, the approval carrier, the trust oracle, the audit sink, alerts and clocks are injected. T12 adds
|
Operations, checks, the approval carrier, the trust oracle, the audit sink, alerts and clocks are injected.
|
||||||
the pilot's real operations (OpenBao CAS write, LLDAP self-rotation, ExternalSecret sync, rollout, synthetic
|
|
||||||
journey), the CLI, the Containerfile and the image workflow.
|
`operations.py` (T12) holds the pilot's real operations:
|
||||||
|
|
||||||
|
- OpenBao KV v2 with CAS;
|
||||||
|
- LLDAP self-rotation;
|
||||||
|
- ExternalSecret force-sync and its status;
|
||||||
|
- rollout restart, where a rollout that never becomes ready is a definite failure and is reverted;
|
||||||
|
- the synthetic journey check.
|
||||||
|
|
||||||
|
An operation signals a known-bad outcome with `definite_failure`, and the runner reverts. Any other error stops
|
||||||
|
without guessing.
|
||||||
|
|
||||||
|
`synthetic_pilot.py` runs a rotation and a deliberate revert on the T08 stack plus OpenBao:
|
||||||
|
`uv run bounded-executor/synthetic_pilot.py`.
|
||||||
|
|
||||||
|
T13 adds the CLI, the Containerfile, the image workflow and the rapp package.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 -m pytest bounded-executor/tests # or: make bounded-executor-test
|
python3 -m pytest bounded-executor/tests # or: make bounded-executor-test
|
||||||
|
|
|
||||||
|
|
@ -12,9 +12,10 @@ Runs one Bounded Execution Admission (canon/standards/bounded-execution-admissio
|
||||||
started `execution.run.started` to the audit sink; no custody, no run
|
started `execution.run.started` to the audit sink; no custody, no run
|
||||||
steps in plan order, one consumer at a time; checks after each mutating step, after
|
steps in plan order, one consumer at a time; checks after each mutating step, after
|
||||||
their settle time; the kill switch and the budget are checked between steps
|
their settle time; the kill switch and the budget are checked between steps
|
||||||
failure a failed check or an exhausted budget reverts the mutated steps in reverse
|
failure a failed check, an exhausted budget or the kill switch runs the revert steps of
|
||||||
order (or stops, as the admission says) and re-runs their checks; an error of
|
the mutated steps in the admission's declared order (or stops, as it says), each
|
||||||
unknown outcome stops without guessing; a failed revert stops and alerts
|
proving the restored state with its own checks; an error of unknown outcome stops
|
||||||
|
without guessing; a failed revert stops and alerts
|
||||||
receipt one `execution.attribution.receipt` per run (PDP `not-evaluated` in the pilot)
|
receipt one `execution.attribution.receipt` per run (PDP `not-evaluated` in the pilot)
|
||||||
|
|
||||||
Every effect goes through an injected interface: Carrier, operation and check callables, TrustOracle,
|
Every effect goes through an injected interface: Carrier, operation and check callables, TrustOracle,
|
||||||
|
|
@ -52,6 +53,12 @@ class Unknown(Exception):
|
||||||
"""An operation's outcome is unknown: stop, do not guess, do not revert blindly."""
|
"""An operation's outcome is unknown: stop, do not guess, do not revert blindly."""
|
||||||
|
|
||||||
|
|
||||||
|
class StepFailed(Exception):
|
||||||
|
"""A definite, known-bad outcome (e.g. a rollout that never became ready): handled like a failed check.
|
||||||
|
Operations outside this module may instead set `definite_failure = True` on their exception class."""
|
||||||
|
definite_failure = True
|
||||||
|
|
||||||
|
|
||||||
class Carrier(Protocol):
|
class Carrier(Protocol):
|
||||||
def get(self, approval_id: str) -> dict: ...
|
def get(self, approval_id: str) -> dict: ...
|
||||||
def consume(self, approval_id: str, request_digest: str, decision_id: str) -> dict: ...
|
def consume(self, approval_id: str, request_digest: str, decision_id: str) -> dict: ...
|
||||||
|
|
@ -203,9 +210,13 @@ class Run:
|
||||||
def _budget_left(self) -> float:
|
def _budget_left(self) -> float:
|
||||||
return self.spec["budget"]["wall_time_seconds"] - (self.d.monotonic() - self.start_mono)
|
return self.spec["budget"]["wall_time_seconds"] - (self.d.monotonic() - self.start_mono)
|
||||||
|
|
||||||
def _record(self, step: dict, kind: str, outcome: str, checks: list[dict] | None = None) -> None:
|
def _record(self, step: dict, kind: str, outcome: str, checks: list[dict] | None = None,
|
||||||
self.steps.append({"id": step["id"], "kind": kind, "operation": step["operation"], "target": step["target"],
|
error: BaseException | None = None) -> None:
|
||||||
"consumer": step["consumer"], "outcome": outcome, "checks": checks or []})
|
entry = {"id": step["id"], "kind": kind, "operation": step["operation"], "target": step["target"],
|
||||||
|
"consumer": step["consumer"], "outcome": outcome, "checks": checks or []}
|
||||||
|
if error is not None:
|
||||||
|
entry["error_type"] = type(error).__name__ # the type only: messages can carry values
|
||||||
|
self.steps.append(entry)
|
||||||
|
|
||||||
def _run_checks(self, ids: list[str], ctx: Context) -> tuple[bool, list[dict]]:
|
def _run_checks(self, ids: list[str], ctx: Context) -> tuple[bool, list[dict]]:
|
||||||
results = []
|
results = []
|
||||||
|
|
@ -230,18 +241,20 @@ class Run:
|
||||||
return True, results
|
return True, results
|
||||||
|
|
||||||
def _revert(self, done: list[dict], ctx: Context) -> bool:
|
def _revert(self, done: list[dict], ctx: Context) -> bool:
|
||||||
"""Undo the mutated steps in reverse order; each revert step must pass its own checks."""
|
"""Undo the mutated steps. Revert steps run in the order the admission declares them (the author knows
|
||||||
reverts = {s["id"]: s for s in self.spec["plan"].get("revert_steps", [])}
|
the recovery order: data before delivery before restart), limited to those whose forward step ran.
|
||||||
for step in reversed(done):
|
Each must pass its own checks; an irreversible step that ran makes the revert fail."""
|
||||||
if not step["mutates"]:
|
mutated = [s for s in done if s["mutates"]]
|
||||||
|
if any(s.get("irreversible") for s in mutated):
|
||||||
|
return False
|
||||||
|
wanted = {s["revert"] for s in mutated}
|
||||||
|
for revert in self.spec["plan"].get("revert_steps", []):
|
||||||
|
if revert["id"] not in wanted:
|
||||||
continue
|
continue
|
||||||
if step.get("irreversible"):
|
|
||||||
return False
|
|
||||||
revert = reverts[step["revert"]]
|
|
||||||
try:
|
try:
|
||||||
self.d.operations[revert["operation"]](revert, ctx)
|
self.d.operations[revert["operation"]](revert, ctx)
|
||||||
except Exception:
|
except Exception as error:
|
||||||
self._record(revert, "revert", "error")
|
self._record(revert, "revert", "error", error=error)
|
||||||
return False
|
return False
|
||||||
ok, results = self._run_checks(revert.get("checks_after", []), ctx)
|
ok, results = self._run_checks(revert.get("checks_after", []), ctx)
|
||||||
self._record(revert, "revert", "done" if ok else "check_failed", results)
|
self._record(revert, "revert", "done" if ok else "check_failed", results)
|
||||||
|
|
@ -262,11 +275,15 @@ class Run:
|
||||||
return reason
|
return reason
|
||||||
try:
|
try:
|
||||||
self.d.operations[step["operation"]](step, ctx)
|
self.d.operations[step["operation"]](step, ctx)
|
||||||
except Unknown:
|
except Exception as error:
|
||||||
self._record(step, "step", "unknown")
|
if getattr(error, "definite_failure", False) and step["mutates"]:
|
||||||
return "unknown"
|
# known-bad state: treat like a failed check, the step counts as done for the revert
|
||||||
except Exception:
|
done.append(step)
|
||||||
self._record(step, "step", "unknown")
|
self._record(step, "step", "failed", error=error)
|
||||||
|
if policy["on_check_failure"] == "revert":
|
||||||
|
return "reverted" if self._revert(done, ctx) else "revert_failed"
|
||||||
|
return "stopped"
|
||||||
|
self._record(step, "step", "unknown", error=error) # outcome not established: stop
|
||||||
return "unknown"
|
return "unknown"
|
||||||
done.append(step)
|
done.append(step)
|
||||||
ok, results = self._run_checks(step.get("checks_after", []), ctx) if step["mutates"] else (True, [])
|
ok, results = self._run_checks(step.get("checks_after", []), ctx) if step["mutates"] else (True, [])
|
||||||
|
|
|
||||||
332
bounded-executor/operations.py
Normal file
332
bounded-executor/operations.py
Normal file
|
|
@ -0,0 +1,332 @@
|
||||||
|
"""Pilot operations and checks for the bounded executor (NK-WP-0056-T12).
|
||||||
|
|
||||||
|
The T06 pilot rotates one directory bind (svc-authelia) under one admission:
|
||||||
|
|
||||||
|
openbao.kv.write-next-version read version N, write a fresh value as N+1 with cas=N
|
||||||
|
openbao.kv.write-previous-value-as-next-version revert: write the previous value as the next version (CAS)
|
||||||
|
lldap.user.change-own-password-from-kv set the bind's LLDAP password to the current KV value,
|
||||||
|
signing in as the bind with a value it still accepts
|
||||||
|
(self-rotation, founder decision 3d0c8f86); forward and revert
|
||||||
|
eso.force-sync ask the delivery to re-project the value (ExternalSecret)
|
||||||
|
k8s.rollout-restart restart the consumer and wait for the rollout
|
||||||
|
|
||||||
|
Checks: kv-advanced, bind-signs-in, eso-synced, synthetic-journey.
|
||||||
|
|
||||||
|
Values live only in Context.data (memory) and in the requests that carry them; nothing here prints, logs or
|
||||||
|
returns a value. Every adapter is stdlib HTTP so the image needs no client libraries.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
import secrets
|
||||||
|
import ssl
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
HERE = pathlib.Path(__file__).resolve().parent
|
||||||
|
REPO = HERE.parent
|
||||||
|
_spec = importlib.util.spec_from_file_location("journey", REPO / "sso-mfa/k8s/synthetic-journey/journey.py")
|
||||||
|
journey = importlib.util.module_from_spec(_spec)
|
||||||
|
_spec.loader.exec_module(journey)
|
||||||
|
|
||||||
|
FIELD = "LDAP_BIND_PASSWORD"
|
||||||
|
|
||||||
|
|
||||||
|
class OperationError(Exception):
|
||||||
|
"""A definite failure: the operation did not take effect."""
|
||||||
|
|
||||||
|
|
||||||
|
class RolloutFailed(Exception):
|
||||||
|
"""The restart was issued but the consumer never became ready: a definite, known-bad outcome that the
|
||||||
|
executor reverts (executor.StepFailed semantics)."""
|
||||||
|
definite_failure = True
|
||||||
|
|
||||||
|
|
||||||
|
def _opener(ca_file: str | None):
|
||||||
|
context = ssl.create_default_context(cafile=ca_file) if ca_file else ssl.create_default_context()
|
||||||
|
return urllib.request.build_opener(urllib.request.ProxyHandler({}), urllib.request.HTTPSHandler(context=context))
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
# OpenBao KV v2
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class OpenBao:
|
||||||
|
def __init__(self, addr: str, *, token: str | None = None, k8s_role: str | None = None,
|
||||||
|
jwt_file: str | None = None, k8s_mount: str = "kubernetes", ca_file: str | None = None,
|
||||||
|
timeout: float = 15):
|
||||||
|
self.addr = addr.rstrip("/")
|
||||||
|
self.open = _opener(ca_file).open
|
||||||
|
self.timeout = timeout
|
||||||
|
self.token = token
|
||||||
|
if self.token is None:
|
||||||
|
jwt = pathlib.Path(jwt_file).read_text().strip()
|
||||||
|
auth = self._call("POST", f"/v1/auth/{k8s_mount}/login", {"role": k8s_role, "jwt": jwt}, auth=False)
|
||||||
|
self.token = auth["auth"]["client_token"]
|
||||||
|
|
||||||
|
def _call(self, method: str, path: str, body: dict | None = None, auth: bool = True) -> dict:
|
||||||
|
headers = {"Content-Type": "application/json"}
|
||||||
|
if auth:
|
||||||
|
headers["X-Vault-Token"] = self.token
|
||||||
|
request = urllib.request.Request(self.addr + path, method=method, headers=headers,
|
||||||
|
data=json.dumps(body).encode() if body is not None else None)
|
||||||
|
try:
|
||||||
|
with self.open(request, timeout=self.timeout) as response:
|
||||||
|
raw = response.read()
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
error.close()
|
||||||
|
raise OperationError(f"openbao {method} {error.code}") from None
|
||||||
|
return json.loads(raw) if raw else {}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _api(kv_path: str) -> str:
|
||||||
|
mount, _, rest = kv_path.partition("/")
|
||||||
|
return f"/v1/{mount}/data/{rest}"
|
||||||
|
|
||||||
|
def read(self, kv_path: str) -> tuple[dict, int]:
|
||||||
|
body = self._call("GET", self._api(kv_path))["data"]
|
||||||
|
return body["data"], int(body["metadata"]["version"])
|
||||||
|
|
||||||
|
def write(self, kv_path: str, data: dict, cas: int) -> int:
|
||||||
|
return int(self._call("POST", self._api(kv_path), {"options": {"cas": cas}, "data": data})["data"]["version"])
|
||||||
|
|
||||||
|
def revoke_self(self) -> None:
|
||||||
|
try:
|
||||||
|
self._call("POST", "/v1/auth/token/revoke-self", {})
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
# LLDAP
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Directory:
|
||||||
|
"""Sign-in over LLDAP's HTTP API; password changes through the pinned image's lldap_set_password (OPAQUE)."""
|
||||||
|
|
||||||
|
def __init__(self, base_url: str, set_password_command: list[str], *, ca_file: str | None = None,
|
||||||
|
tool_base_url: str | None = None, timeout: float = 15):
|
||||||
|
self.base = base_url.rstrip("/")
|
||||||
|
self.tool_base = (tool_base_url or base_url).rstrip("/")
|
||||||
|
self.command = set_password_command
|
||||||
|
self.open = _opener(ca_file).open
|
||||||
|
self.timeout = timeout
|
||||||
|
|
||||||
|
def login(self, user: str, password: str) -> str | None:
|
||||||
|
request = urllib.request.Request(self.base + "/auth/simple/login", method="POST",
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
data=json.dumps({"username": user, "password": password}).encode())
|
||||||
|
try:
|
||||||
|
with self.open(request, timeout=self.timeout) as response:
|
||||||
|
return json.load(response).get("token")
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
error.close()
|
||||||
|
return None
|
||||||
|
|
||||||
|
def set_own_password(self, token: str, user: str, password: str) -> None:
|
||||||
|
done = subprocess.run([*self.command, "--base-url", self.tool_base, "--token", token, "--username", user],
|
||||||
|
env={"PATH": os.environ.get("PATH", "/usr/bin:/bin"), "LLDAP_USER_PASSWORD": password},
|
||||||
|
capture_output=True, timeout=60, check=False)
|
||||||
|
if done.returncode:
|
||||||
|
raise OperationError("lldap_set_password refused")
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
# Kubernetes (stdlib client, in-cluster ServiceAccount)
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Kube:
|
||||||
|
def __init__(self, api: str = "https://kubernetes.default.svc",
|
||||||
|
token_file: str = "/var/run/secrets/kubernetes.io/serviceaccount/token",
|
||||||
|
ca_file: str | None = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt", timeout: float = 15):
|
||||||
|
self.api = api.rstrip("/")
|
||||||
|
self.token_file = token_file
|
||||||
|
self.open = _opener(ca_file).open
|
||||||
|
self.timeout = timeout
|
||||||
|
|
||||||
|
def _call(self, method: str, path: str, body: dict | None = None) -> dict:
|
||||||
|
headers = {"Authorization": "Bearer " + pathlib.Path(self.token_file).read_text().strip(),
|
||||||
|
"Accept": "application/json"}
|
||||||
|
data = None
|
||||||
|
if body is not None:
|
||||||
|
headers["Content-Type"] = "application/merge-patch+json"
|
||||||
|
data = json.dumps(body).encode()
|
||||||
|
request = urllib.request.Request(self.api + path, method=method, headers=headers, data=data)
|
||||||
|
try:
|
||||||
|
with self.open(request, timeout=self.timeout) as response:
|
||||||
|
return json.load(response)
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
error.close()
|
||||||
|
raise OperationError(f"kube {method} {error.code}") from None
|
||||||
|
|
||||||
|
def get(self, path: str) -> dict:
|
||||||
|
return self._call("GET", path)
|
||||||
|
|
||||||
|
def patch(self, path: str, body: dict) -> dict:
|
||||||
|
return self._call("PATCH", path, body)
|
||||||
|
|
||||||
|
|
||||||
|
def now_utc() -> dt.datetime:
|
||||||
|
return dt.datetime.now(dt.UTC).replace(microsecond=0)
|
||||||
|
|
||||||
|
|
||||||
|
class KubeDelivery:
|
||||||
|
"""eso.force-sync and its eso-synced check, through the ExternalSecret API (no Secret read)."""
|
||||||
|
|
||||||
|
def __init__(self, kube: Kube, namespace: str, external_secret: str, clock: Callable[[], dt.datetime] = now_utc):
|
||||||
|
self.kube, self.clock = kube, clock
|
||||||
|
self.path = f"/apis/external-secrets.io/v1/namespaces/{namespace}/externalsecrets/{external_secret}"
|
||||||
|
|
||||||
|
def force_sync(self) -> dt.datetime:
|
||||||
|
at = self.clock()
|
||||||
|
self.kube.patch(self.path, {"metadata": {"annotations": {"force-sync": str(int(at.timestamp()))}}})
|
||||||
|
return at
|
||||||
|
|
||||||
|
def synced(self, since: dt.datetime) -> bool:
|
||||||
|
status = self.kube.get(self.path).get("status", {})
|
||||||
|
ready = any(c.get("type") == "Ready" and c.get("status") == "True" for c in status.get("conditions", []))
|
||||||
|
refreshed = status.get("refreshTime")
|
||||||
|
return ready and bool(refreshed) and dt.datetime.fromisoformat(refreshed.replace("Z", "+00:00")) >= since
|
||||||
|
|
||||||
|
|
||||||
|
class KubeRestart:
|
||||||
|
"""k8s.rollout-restart: patch the restartedAt annotation, then wait until the rollout is complete."""
|
||||||
|
|
||||||
|
def __init__(self, kube: Kube, namespace: str, deployment: str, timeout: float = 300, poll: float = 3,
|
||||||
|
clock: Callable[[], dt.datetime] = now_utc, sleep: Callable[[float], None] = time.sleep,
|
||||||
|
monotonic: Callable[[], float] = time.monotonic):
|
||||||
|
self.kube, self.timeout, self.poll, self.clock, self.sleep, self.monotonic = kube, timeout, poll, clock, sleep, monotonic
|
||||||
|
self.path = f"/apis/apps/v1/namespaces/{namespace}/deployments/{deployment}"
|
||||||
|
|
||||||
|
def restart(self) -> None:
|
||||||
|
stamp = self.clock().isoformat().replace("+00:00", "Z")
|
||||||
|
self.kube.patch(self.path, {"spec": {"template": {"metadata": {"annotations": {
|
||||||
|
"kubectl.kubernetes.io/restartedAt": stamp}}}}})
|
||||||
|
deadline = self.monotonic() + self.timeout
|
||||||
|
while self.monotonic() < deadline:
|
||||||
|
d = self.kube.get(self.path)
|
||||||
|
spec, status = d.get("spec", {}), d.get("status", {})
|
||||||
|
want = spec.get("replicas", 1)
|
||||||
|
if (status.get("observedGeneration", 0) >= d.get("metadata", {}).get("generation", 0)
|
||||||
|
and status.get("updatedReplicas") == want and status.get("availableReplicas") == want
|
||||||
|
and status.get("replicas") == want and not status.get("unavailableReplicas")):
|
||||||
|
return
|
||||||
|
self.sleep(self.poll)
|
||||||
|
raise RolloutFailed("rollout did not complete") # known-bad: the runner reverts
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
# The pilot registry
|
||||||
|
# --------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def pilot_registry(*, bao: OpenBao, directory: Directory, delivery, restart, bind_kv: str, bind_user: str,
|
||||||
|
journey_kv: str, journey_config, journey_state: dict | None = None):
|
||||||
|
"""Return (operations, checks) for the bind-rotation admission. `delivery` has force_sync()/synced(since);
|
||||||
|
`restart` has restart(). Values stay in ctx.data. `journey_state` carries the last one-time-code time step
|
||||||
|
across runs (privacyIDEA accepts each 30 s step once)."""
|
||||||
|
shared = journey_state if journey_state is not None else {}
|
||||||
|
|
||||||
|
def kv_target(step: dict) -> str:
|
||||||
|
target = step["target"]
|
||||||
|
if not target.startswith("openbao:") or target[len("openbao:"):] != bind_kv:
|
||||||
|
raise OperationError("target outside the configured bind")
|
||||||
|
return bind_kv
|
||||||
|
|
||||||
|
def generate(step, ctx):
|
||||||
|
path = kv_target(step)
|
||||||
|
data, version = bao.read(path)
|
||||||
|
if set(data) != {FIELD}:
|
||||||
|
raise OperationError("kv shape")
|
||||||
|
fresh = secrets.token_urlsafe(32)
|
||||||
|
ctx.data.update(previous=data[FIELD], fresh=fresh)
|
||||||
|
ctx.data["expect"] = (bao.write(path, {FIELD: fresh}, cas=version), "fresh")
|
||||||
|
|
||||||
|
def restore_kv(step, ctx):
|
||||||
|
path = kv_target(step)
|
||||||
|
_, version = bao.read(path)
|
||||||
|
ctx.data["expect"] = (bao.write(path, {FIELD: ctx.data["previous"]}, cas=version), "previous")
|
||||||
|
|
||||||
|
def change_own_password(step, ctx):
|
||||||
|
if step["target"] != "lldap:" + bind_user:
|
||||||
|
raise OperationError("target outside the configured bind")
|
||||||
|
want = bao.read(bind_kv)[0][FIELD]
|
||||||
|
if directory.login(bind_user, want):
|
||||||
|
return # already in effect
|
||||||
|
for known in (ctx.data.get("previous"), ctx.data.get("fresh")):
|
||||||
|
if known and known != want:
|
||||||
|
token = directory.login(bind_user, known)
|
||||||
|
if token:
|
||||||
|
directory.set_own_password(token, bind_user, want)
|
||||||
|
if not directory.login(bind_user, want):
|
||||||
|
raise OperationError("password change not effective")
|
||||||
|
return
|
||||||
|
raise OperationError("no known value signs in as the bind")
|
||||||
|
|
||||||
|
def force_sync(step, ctx):
|
||||||
|
ctx.data["delivered_at"] = delivery.force_sync()
|
||||||
|
|
||||||
|
def rollout_restart(step, ctx):
|
||||||
|
restart.restart()
|
||||||
|
|
||||||
|
def check_kv_advanced(check, ctx):
|
||||||
|
version, which = ctx.data["expect"]
|
||||||
|
data, current = bao.read(bind_kv)
|
||||||
|
return current == version and data.get(FIELD) == ctx.data[which]
|
||||||
|
|
||||||
|
def check_bind_signs_in(check, ctx):
|
||||||
|
return directory.login(bind_user, bao.read(bind_kv)[0][FIELD]) is not None
|
||||||
|
|
||||||
|
def check_eso_synced(check, ctx):
|
||||||
|
deadline = time.monotonic() + check["timeout_seconds"] * 0.8
|
||||||
|
while True:
|
||||||
|
if delivery.synced(ctx.data["delivered_at"]):
|
||||||
|
return True
|
||||||
|
if time.monotonic() > deadline:
|
||||||
|
return False
|
||||||
|
time.sleep(2)
|
||||||
|
|
||||||
|
def check_journey(check, ctx):
|
||||||
|
creds, _ = bao.read(journey_kv)
|
||||||
|
cfg = _load_config(journey_config() if callable(journey_config) else journey_config)
|
||||||
|
last = shared.get("journey_step")
|
||||||
|
receipt = journey.run_journey(cfg, creds["PASSWORD"], journey.decode_seed(creds["TOTP_SEED"]),
|
||||||
|
min_time_step=None if last is None else last + 1)
|
||||||
|
if receipt.get("otp_time_step") is not None:
|
||||||
|
shared["journey_step"] = receipt["otp_time_step"]
|
||||||
|
ctx.data.setdefault("journeys", []).append({k: receipt.get(k) for k in ("result", "failed_step", "reason")})
|
||||||
|
return receipt["result"] == "pass"
|
||||||
|
|
||||||
|
operations = {
|
||||||
|
"openbao.kv.write-next-version": generate,
|
||||||
|
"openbao.kv.write-previous-value-as-next-version": restore_kv,
|
||||||
|
"lldap.user.change-own-password-from-kv": change_own_password,
|
||||||
|
"eso.force-sync": force_sync,
|
||||||
|
"k8s.rollout-restart": rollout_restart,
|
||||||
|
}
|
||||||
|
checks = {"kv-advanced": check_kv_advanced, "bind-signs-in": check_bind_signs_in,
|
||||||
|
"eso-synced": check_eso_synced, "synthetic-journey": check_journey}
|
||||||
|
return operations, checks
|
||||||
|
|
||||||
|
|
||||||
|
def _load_config(config: dict) -> dict:
|
||||||
|
import tempfile
|
||||||
|
fd, path = tempfile.mkstemp(suffix=".json", prefix="journey-")
|
||||||
|
try:
|
||||||
|
with os.fdopen(fd, "w") as out:
|
||||||
|
json.dump(config, out)
|
||||||
|
return journey.load_config(path)
|
||||||
|
finally:
|
||||||
|
os.unlink(path)
|
||||||
280
bounded-executor/synthetic_pilot.py
Normal file
280
bounded-executor/synthetic_pilot.py
Normal file
|
|
@ -0,0 +1,280 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# /// script
|
||||||
|
# dependencies = ["PyYAML>=6,<7", "jsonschema>=4,<5"]
|
||||||
|
# ///
|
||||||
|
"""Synthetic pilot: the executor rotates the svc-authelia bind on the T08 stack (NK-WP-0056-T12).
|
||||||
|
|
||||||
|
Agent-runnable, local Docker only, generated values, everything removed afterwards. Starts the T08 identity
|
||||||
|
stack (LLDAP, privacyIDEA, Authelia, KeyCape on the pinned images) and an OpenBao dev server on the platform's
|
||||||
|
pinned 2.5.4 image (KV v2 at `platform`, `cas_required` on the bind). The executor gets its own token with
|
||||||
|
exactly CCR-2026-0058's policy file plus a journey-read policy, never the root token.
|
||||||
|
|
||||||
|
Not on this stack: approval-engine (an in-process carrier with its consume semantics stands in), audit-core
|
||||||
|
(an in-memory sink), and Kubernetes. Stand-ins replace the ExternalSecret delivery (re-project the KV value
|
||||||
|
into Authelia's configuration) and the rollout (restart the Authelia container). The real Kubernetes adapters
|
||||||
|
(operations.KubeDelivery, KubeRestart) are covered by unit tests and exercised live in T06.
|
||||||
|
|
||||||
|
Runs:
|
||||||
|
rotation one admission, the bind rotates; afterwards the old value is refused and the journey passes
|
||||||
|
revert a second admission whose first delivery projects a wrong value while reporting success:
|
||||||
|
Authelia does not become ready after the restart (definite failure), the executor reverts in
|
||||||
|
declared order, the previous value is back and the journey passes again
|
||||||
|
|
||||||
|
Writes a value-free report. Exit 0 when both runs end as expected.
|
||||||
|
Run: uv run bounded-executor/synthetic_pilot.py [--report FILE]
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import base64
|
||||||
|
import copy
|
||||||
|
import datetime as dt
|
||||||
|
import hashlib
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import pathlib
|
||||||
|
import secrets
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
HERE = pathlib.Path(__file__).resolve().parent
|
||||||
|
REPO = HERE.parent
|
||||||
|
|
||||||
|
|
||||||
|
def _load(name, path):
|
||||||
|
spec = importlib.util.spec_from_file_location(name, path)
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
sys.modules[name] = module
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
X = _load("executor", HERE / "executor.py")
|
||||||
|
OPS = _load("operations", HERE / "operations.py")
|
||||||
|
stack_mod = _load("stack", REPO / "sso-mfa/k8s/synthetic-journey/stack.py")
|
||||||
|
bea = X.bea
|
||||||
|
|
||||||
|
OPENBAO_IMAGE = "quay.io/openbao/openbao@sha256:436eaf9778cad75507ff70ea26ace30dcbe15606e619ac3823495663d7f7c115" # 2.5.4
|
||||||
|
BIND_KV = "platform/workloads/net-kingdom/directory-binds/svc-authelia"
|
||||||
|
JOURNEY_KV = "platform/workloads/net-kingdom/synthetic-journey/syn-journey"
|
||||||
|
POLICY_FILE = REPO.parent / "railiance-platform/openbao/policies/net-kingdom-bind-executor-svc-authelia-write.hcl"
|
||||||
|
JOURNEY_POLICY = 'path "platform/data/workloads/net-kingdom/synthetic-journey/syn-journey" { capabilities = ["read"] }\n'
|
||||||
|
EXECUTOR = "service:nk-bind-executor"
|
||||||
|
APPROVER_SUB = "synthetic-admin-sub"
|
||||||
|
|
||||||
|
|
||||||
|
class OpenBaoDev:
|
||||||
|
def __init__(self, network: str, tag: str):
|
||||||
|
self.name = f"nk56-executor-{tag}-openbao"
|
||||||
|
self.root = secrets.token_hex(16)
|
||||||
|
stack_mod.docker("run", "--detach", "--pull=never", "--name", self.name, "--network", network,
|
||||||
|
"--publish", "127.0.0.1::8200", "--cap-add", "IPC_LOCK", "--env", "BAO_DEV_ROOT_TOKEN_ID",
|
||||||
|
OPENBAO_IMAGE, "server", "-dev", "-dev-listen-address=0.0.0.0:8200",
|
||||||
|
env={"BAO_DEV_ROOT_TOKEN_ID": self.root})
|
||||||
|
self.addr = f"http://127.0.0.1:{stack_mod.port(self.name, 8200)}"
|
||||||
|
stack_mod.wait_http(self.addr + "/v1/sys/health", "openbao")
|
||||||
|
self.admin = OPS.OpenBao(self.addr, token=self.root)
|
||||||
|
|
||||||
|
def call(self, method, path, body=None):
|
||||||
|
return self.admin._call(method, path, body)
|
||||||
|
|
||||||
|
def setup(self, bind_value: str, journey_password: str, seed: bytes) -> str:
|
||||||
|
self.call("POST", "/v1/sys/mounts/platform", {"type": "kv", "options": {"version": "2"}})
|
||||||
|
self.admin.write(BIND_KV, {OPS.FIELD: bind_value}, cas=0)
|
||||||
|
self.call("POST", "/v1/platform/metadata/workloads/net-kingdom/directory-binds/svc-authelia", {"cas_required": True})
|
||||||
|
self.admin.write(JOURNEY_KV, {"PASSWORD": journey_password,
|
||||||
|
"TOTP_SEED": base64.b32encode(seed).decode()}, cas=0)
|
||||||
|
self.call("PUT", "/v1/sys/policies/acl/bind-executor-write", {"policy": POLICY_FILE.read_text()})
|
||||||
|
self.call("PUT", "/v1/sys/policies/acl/journey-read", {"policy": JOURNEY_POLICY})
|
||||||
|
token = self.call("POST", "/v1/auth/token/create", {"policies": ["bind-executor-write", "journey-read"],
|
||||||
|
"no_default_policy": True, "ttl": "15m"})
|
||||||
|
return token["auth"]["client_token"]
|
||||||
|
|
||||||
|
def remove(self):
|
||||||
|
stack_mod.docker("rm", "--force", self.name, check=False)
|
||||||
|
|
||||||
|
|
||||||
|
class Delivery:
|
||||||
|
"""Stand-in for the ExternalSecret: projects the KV value into Authelia's configuration file."""
|
||||||
|
|
||||||
|
def __init__(self, stack, bao_admin):
|
||||||
|
self.stack, self.bao, self.corrupt = stack, bao_admin, 0 # number of next deliveries to corrupt
|
||||||
|
|
||||||
|
def force_sync(self) -> dt.datetime:
|
||||||
|
at = OPS.now_utc()
|
||||||
|
value = self.bao.read(BIND_KV)[0][OPS.FIELD]
|
||||||
|
if self.corrupt:
|
||||||
|
self.corrupt -= 1
|
||||||
|
value = secrets.token_urlsafe(32) # a one-off wrong projection that still reports success
|
||||||
|
path = self.stack.workdir / "authelia.yml"
|
||||||
|
config = yaml.safe_load(path.read_text())
|
||||||
|
config["authentication_backend"]["ldap"]["password"] = value
|
||||||
|
path.write_text(yaml.safe_dump(config)) # same inode: the bind mount sees it
|
||||||
|
self.synced_at = OPS.now_utc()
|
||||||
|
return at
|
||||||
|
|
||||||
|
def synced(self, since: dt.datetime) -> bool:
|
||||||
|
return getattr(self, "synced_at", None) is not None and self.synced_at >= since
|
||||||
|
|
||||||
|
|
||||||
|
class Restart:
|
||||||
|
"""Stand-in for the rollout: restart the Authelia container and wait for it to answer."""
|
||||||
|
|
||||||
|
def __init__(self, stack):
|
||||||
|
self.stack = stack
|
||||||
|
|
||||||
|
def restart(self):
|
||||||
|
stack_mod.docker("restart", self.stack.names["authelia"])
|
||||||
|
# docker assigns a new ephemeral host port on restart (an artifact of this stand-in)
|
||||||
|
self.stack.ports["authelia"] = stack_mod.port(self.stack.names["authelia"], 9091)
|
||||||
|
try:
|
||||||
|
stack_mod.wait_http(f"http://127.0.0.1:{self.stack.ports['authelia']}/api/health", "authelia",
|
||||||
|
{"Host": "auth.journey.test", "X-Forwarded-Proto": "https"}, attempts=60)
|
||||||
|
except stack_mod.SetupError:
|
||||||
|
raise OPS.RolloutFailed("authelia did not become ready") from None
|
||||||
|
|
||||||
|
|
||||||
|
class MemoryCarrier:
|
||||||
|
"""approval-engine's approval object and consume semantics, in memory."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.approvals = {}
|
||||||
|
|
||||||
|
def approve(self, approval_id, admission):
|
||||||
|
spec = admission["spec"]
|
||||||
|
self.approvals[approval_id] = {
|
||||||
|
"id": approval_id, "status": "approved",
|
||||||
|
"binding": {"action": "bounded-execution", "actor": EXECUTOR, "principal": "admin", "purpose": spec["purpose"],
|
||||||
|
"target": {"admission_id": spec["id"], "admission_digest": bea.admission_digest(admission)},
|
||||||
|
"human_control": True, "pdp_path": False},
|
||||||
|
"validity": {"not_before": spec["window"]["not_before"], "expires_at": spec["window"]["not_after"]},
|
||||||
|
"entries": [{"subject_id": APPROVER_SUB, "assurance": "aal2", "principal_type": "human",
|
||||||
|
"approved_at": OPS.now_utc().isoformat()}]}
|
||||||
|
|
||||||
|
def get(self, approval_id):
|
||||||
|
return copy.deepcopy(self.approvals[approval_id])
|
||||||
|
|
||||||
|
def consume(self, approval_id, request_digest, decision_id):
|
||||||
|
a = self.approvals[approval_id]
|
||||||
|
if a["status"] != "approved":
|
||||||
|
raise RuntimeError("409")
|
||||||
|
a["status"] = "consumed"
|
||||||
|
a["consumption"] = {"request_digest": request_digest, "decision_id": decision_id,
|
||||||
|
"consumed_at": OPS.now_utc().isoformat()}
|
||||||
|
return {"approval_id": approval_id, "status": "consumed", "request_digest": request_digest,
|
||||||
|
"decision_id": decision_id, "consumed_at": a["consumption"]["consumed_at"], "idempotent": False}
|
||||||
|
|
||||||
|
|
||||||
|
def admission(admission_id: str, attempts: int) -> dict:
|
||||||
|
"""The pilot example, pinned to the current files, with a window around now."""
|
||||||
|
doc = bea.load(REPO / "examples/bounded-execution-admission/bind-rotation-svc-authelia.yaml")
|
||||||
|
spec = doc["spec"]
|
||||||
|
spec["id"] = admission_id
|
||||||
|
now = dt.datetime.now(dt.UTC).replace(microsecond=0)
|
||||||
|
spec["window"] = {"not_before": (now - dt.timedelta(minutes=5)).isoformat().replace("+00:00", "Z"),
|
||||||
|
"not_after": (now + dt.timedelta(hours=2)).isoformat().replace("+00:00", "Z")}
|
||||||
|
spec["budget"]["attempts"] = attempts
|
||||||
|
files = []
|
||||||
|
for f in spec["plan"]["source"]["files"]:
|
||||||
|
files.append({"path": f["path"], "sha256": hashlib.sha256((REPO / f["path"]).read_bytes()).hexdigest()})
|
||||||
|
spec["plan"]["source"]["files"] = files
|
||||||
|
pins = {f["path"]: f["sha256"] for f in files}
|
||||||
|
for c in spec["checks"]:
|
||||||
|
c["sha256"] = pins[c["ref"]]
|
||||||
|
spec["plan_digest"] = bea.plan_digest(doc)
|
||||||
|
report = bea.evaluate(doc, json.loads(bea.SCHEMA.read_text()))
|
||||||
|
if not report["valid"]:
|
||||||
|
raise SystemExit(f"synthetic admission invalid: {report['findings']}")
|
||||||
|
return doc
|
||||||
|
|
||||||
|
|
||||||
|
def lldap_command():
|
||||||
|
return ["docker", "run", "--rm", "--pull=never", "--network", "host", "--env", "LLDAP_USER_PASSWORD",
|
||||||
|
"--entrypoint", "/app/lldap_set_password", stack_mod.LLDAP_IMAGE]
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv=None):
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
|
||||||
|
parser.add_argument("--report")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
report = {"schema": "netkingdom.bounded-executor.synthetic-pilot/v1",
|
||||||
|
"started_at": OPS.now_utc().isoformat(), "images": {"openbao": OPENBAO_IMAGE},
|
||||||
|
"policy_file_sha256": hashlib.sha256(POLICY_FILE.read_bytes()).hexdigest(), "runs": []}
|
||||||
|
ok = False
|
||||||
|
with tempfile.TemporaryDirectory(prefix="nk56-executor-") as tmp:
|
||||||
|
stack = stack_mod.Stack(pathlib.Path(tmp), pi_cache_timeout=120)
|
||||||
|
bao = None
|
||||||
|
try:
|
||||||
|
stack.up()
|
||||||
|
bao = OpenBaoDev(stack.network, stack.tag)
|
||||||
|
executor_token = bao.setup(stack.pw["svc-authelia"], stack.pw[stack_mod.USER], stack.seed)
|
||||||
|
executor_bao = OPS.OpenBao(bao.addr, token=executor_token)
|
||||||
|
directory = OPS.Directory(stack.lldap, lldap_command())
|
||||||
|
delivery = Delivery(stack, bao.admin)
|
||||||
|
journey_state = {}
|
||||||
|
operations, checks = OPS.pilot_registry(
|
||||||
|
bao=executor_bao, directory=directory, delivery=delivery, restart=Restart(stack),
|
||||||
|
bind_kv=BIND_KV, bind_user="svc-authelia", journey_kv=JOURNEY_KV,
|
||||||
|
journey_config=stack.journey_config, journey_state=journey_state)
|
||||||
|
carrier, events, alerts = MemoryCarrier(), [], []
|
||||||
|
audit = type("Sink", (), {"emit": lambda _s, e: events.append(e)})()
|
||||||
|
cfg = X.Config(executor_identity=EXECUTOR, runtime_principal="synthetic:executor-token",
|
||||||
|
tenant="tenant:platform", environment="TEST", approver_subjects={"admin": APPROVER_SUB})
|
||||||
|
deps = X.Deps(carrier=carrier, operations=operations, checks=checks, trust=lambda s: True,
|
||||||
|
audit=audit, alert=lambda kind, info: alerts.append(kind))
|
||||||
|
|
||||||
|
def run(admission_id, attempts, corrupt):
|
||||||
|
doc = admission(admission_id, attempts)
|
||||||
|
approval = "apr-" + admission_id
|
||||||
|
carrier.approve(approval, doc)
|
||||||
|
before_value, before_version = bao.admin.read(BIND_KV)
|
||||||
|
delivery.corrupt = 1 if corrupt else 0
|
||||||
|
result = X.Run(doc, approval, cfg, deps).execute()
|
||||||
|
delivery.corrupt = 0
|
||||||
|
after_value, after_version = bao.admin.read(BIND_KV)
|
||||||
|
changed = after_value != before_value
|
||||||
|
old_refused = directory.login("svc-authelia", before_value[OPS.FIELD]) is None
|
||||||
|
new_works = directory.login("svc-authelia", after_value[OPS.FIELD]) is not None
|
||||||
|
receipt = result.get("receipt", {}).get("data", {})
|
||||||
|
return {"admission": admission_id, "outcome": result["outcome"], "reason": result.get("reason"),
|
||||||
|
"kv_version": [before_version, after_version], "kv_value_changed": changed,
|
||||||
|
"previous_value_refused_by_lldap": old_refused, "current_kv_value_signs_in": new_works,
|
||||||
|
"steps": [(s["id"], s["kind"], s["outcome"], [c["result"] for c in s.get("checks", [])],
|
||||||
|
s.get("error_type"))
|
||||||
|
for s in receipt.get("steps", [])],
|
||||||
|
"reverted": receipt.get("reverted"), "pdp": receipt.get("pdp"),
|
||||||
|
"events": [e["type"] for e in events if e["correlation_id"] == result.get("run_id")]}
|
||||||
|
|
||||||
|
rotation = run("BEA-2026-9001", 1, corrupt=False)
|
||||||
|
rotation["met"] = (rotation["outcome"] == "completed" and rotation["kv_value_changed"]
|
||||||
|
and rotation["previous_value_refused_by_lldap"] and rotation["current_kv_value_signs_in"])
|
||||||
|
report["runs"].append(rotation)
|
||||||
|
revert = run("BEA-2026-9002", 1, corrupt=True)
|
||||||
|
revert["met"] = (revert["outcome"] == "reverted" and not revert["kv_value_changed"]
|
||||||
|
and revert["current_kv_value_signs_in"]
|
||||||
|
and any(s[0] == "restart" and s[2] in ("check_failed", "failed") for s in revert["steps"])
|
||||||
|
and all(s[2] == "done" for s in revert["steps"] if s[1] == "revert"))
|
||||||
|
report["runs"].append(revert)
|
||||||
|
report["alerts"] = alerts
|
||||||
|
secrets_in_report = [stack.pw["svc-authelia"], executor_token, bao.root]
|
||||||
|
text = json.dumps(report)
|
||||||
|
report["no_values_in_report"] = not any(v in text for v in secrets_in_report)
|
||||||
|
ok = rotation["met"] and revert["met"] and report["no_values_in_report"]
|
||||||
|
finally:
|
||||||
|
if bao:
|
||||||
|
bao.remove()
|
||||||
|
stack.down()
|
||||||
|
report["all_met"] = ok
|
||||||
|
report["finished_at"] = OPS.now_utc().isoformat()
|
||||||
|
text = json.dumps(report, indent=2, default=str)
|
||||||
|
if args.report:
|
||||||
|
pathlib.Path(args.report).write_text(text + "\n")
|
||||||
|
print(text)
|
||||||
|
return 0 if ok else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
|
|
@ -299,11 +299,12 @@ class FailureTests(Base):
|
||||||
self.w.fail_check["eso-synced"] = 1
|
self.w.fail_check["eso-synced"] = 1
|
||||||
result = self.run_it()
|
result = self.run_it()
|
||||||
self.assertEqual(result["outcome"], "reverted", result)
|
self.assertEqual(result["outcome"], "reverted", result)
|
||||||
self.assertEqual(self.ops(), ["generate", "set-directory", "deliver", "redeliver", "restore-directory", "restore-kv"])
|
# declared recovery order, only for steps that ran (restart did not run, so no restart-again)
|
||||||
|
self.assertEqual(self.ops(), ["generate", "set-directory", "deliver", "restore-kv", "restore-directory", "redeliver"])
|
||||||
data = result["receipt"]["data"]
|
data = result["receipt"]["data"]
|
||||||
self.assertTrue(data["reverted"])
|
self.assertTrue(data["reverted"])
|
||||||
reverts = [x for x in data["steps"] if x["kind"] == "revert"]
|
reverts = [x for x in data["steps"] if x["kind"] == "revert"]
|
||||||
self.assertEqual([x["checks"][0]["id"] for x in reverts], ["eso-synced", "bind-signs-in", "kv-advanced"])
|
self.assertEqual([x["checks"][0]["id"] for x in reverts], ["kv-advanced", "bind-signs-in", "eso-synced"])
|
||||||
self.assertTrue(all(x["outcome"] == "done" for x in reverts))
|
self.assertTrue(all(x["outcome"] == "done" for x in reverts))
|
||||||
self.assertEqual(data["attempts"], 1)
|
self.assertEqual(data["attempts"], 1)
|
||||||
|
|
||||||
|
|
@ -319,10 +320,21 @@ class FailureTests(Base):
|
||||||
def test_revert_whose_check_fails_counts_as_failed(self):
|
def test_revert_whose_check_fails_counts_as_failed(self):
|
||||||
self.doc = synthetic_admission(self.root, lambda s: s["budget"].update(attempts=1))
|
self.doc = synthetic_admission(self.root, lambda s: s["budget"].update(attempts=1))
|
||||||
self.w = World(self.doc, self.clock)
|
self.w = World(self.doc, self.clock)
|
||||||
self.w.fail_check["eso-synced"] = 2 # the forward check and the revert's own check
|
self.w.fail_check["eso-synced"] = 1
|
||||||
|
self.w.fail_check["kv-advanced"] = 0
|
||||||
|
original = self.w.check
|
||||||
|
calls = {"kv": 0}
|
||||||
|
def check(c, ctx): # kv-advanced passes forward, fails on the revert
|
||||||
|
if c["id"] == "kv-advanced":
|
||||||
|
calls["kv"] += 1
|
||||||
|
if calls["kv"] == 2:
|
||||||
|
self.w.calls.append("check:kv-advanced")
|
||||||
|
return False
|
||||||
|
return original(c, ctx)
|
||||||
|
self.w.check = check
|
||||||
result = self.run_it()
|
result = self.run_it()
|
||||||
self.assertEqual(result["outcome"], "revert_failed")
|
self.assertEqual(result["outcome"], "revert_failed")
|
||||||
self.assertNotIn("restore-directory", self.ops())
|
self.assertNotIn("restore-directory", self.ops()) # stop at the failed revert
|
||||||
|
|
||||||
def test_failed_revert_stops_and_alerts(self):
|
def test_failed_revert_stops_and_alerts(self):
|
||||||
self.w.fail_check["eso-synced"] = 1
|
self.w.fail_check["eso-synced"] = 1
|
||||||
|
|
@ -330,7 +342,7 @@ class FailureTests(Base):
|
||||||
result = self.run_it()
|
result = self.run_it()
|
||||||
self.assertEqual(result["outcome"], "revert_failed")
|
self.assertEqual(result["outcome"], "revert_failed")
|
||||||
self.assertIn("revert_failed", self.w.alerts)
|
self.assertIn("revert_failed", self.w.alerts)
|
||||||
self.assertNotIn("restore-kv", self.ops()) # stop at the failed revert, do not continue blindly
|
self.assertNotIn("redeliver", self.ops()) # stop at the failed revert, do not continue blindly
|
||||||
|
|
||||||
def test_unknown_outcome_stops_without_revert(self):
|
def test_unknown_outcome_stops_without_revert(self):
|
||||||
self.w.unknown_op = {"eso.force-sync"}
|
self.w.unknown_op = {"eso.force-sync"}
|
||||||
|
|
@ -340,6 +352,23 @@ class FailureTests(Base):
|
||||||
self.assertFalse(result["receipt"]["data"]["reverted"])
|
self.assertFalse(result["receipt"]["data"]["reverted"])
|
||||||
self.assertIn("unknown", self.w.alerts)
|
self.assertIn("unknown", self.w.alerts)
|
||||||
|
|
||||||
|
def test_definite_failure_reverts_like_a_failed_check(self):
|
||||||
|
self.doc = synthetic_admission(self.root, lambda s: s["budget"].update(attempts=1))
|
||||||
|
self.w = World(self.doc, self.clock)
|
||||||
|
original = self.w.op
|
||||||
|
def op(step, ctx):
|
||||||
|
if step["id"] == "restart":
|
||||||
|
self.w.calls.append("restart")
|
||||||
|
raise X.StepFailed("never became ready")
|
||||||
|
return original(step, ctx)
|
||||||
|
self.w.op = op
|
||||||
|
result = self.run_it()
|
||||||
|
self.assertEqual(result["outcome"], "reverted")
|
||||||
|
self.assertEqual(self.ops(), ["generate", "set-directory", "deliver", "restart",
|
||||||
|
"restore-kv", "restore-directory", "redeliver", "restart-again"])
|
||||||
|
restart = next(s for s in result["receipt"]["data"]["steps"] if s["id"] == "restart")
|
||||||
|
self.assertEqual((restart["outcome"], restart["error_type"]), ("failed", "StepFailed"))
|
||||||
|
|
||||||
def test_budget_exhausted_reverts(self):
|
def test_budget_exhausted_reverts(self):
|
||||||
self.w.slow_op = 700.0 # wall_time_seconds is 1200
|
self.w.slow_op = 700.0 # wall_time_seconds is 1200
|
||||||
result = self.run_it()
|
result = self.run_it()
|
||||||
|
|
|
||||||
133
bounded-executor/tests/test_operations.py
Normal file
133
bounded-executor/tests/test_operations.py
Normal file
|
|
@ -0,0 +1,133 @@
|
||||||
|
"""Kubernetes adapters of the pilot operations against a fake API server (NK-WP-0056-T12). No cluster."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import pathlib
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
|
import unittest
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
|
||||||
|
HERE = pathlib.Path(__file__).resolve().parent
|
||||||
|
spec = importlib.util.spec_from_file_location("operations", HERE.parent / "operations.py")
|
||||||
|
OPS = importlib.util.module_from_spec(spec)
|
||||||
|
sys.modules["operations"] = OPS
|
||||||
|
spec.loader.exec_module(OPS)
|
||||||
|
|
||||||
|
ES = "/apis/external-secrets.io/v1/namespaces/sso/externalsecrets/authelia-secrets"
|
||||||
|
DEP = "/apis/apps/v1/namespaces/sso/deployments/authelia"
|
||||||
|
T0 = dt.datetime(2026, 11, 3, 9, 30, tzinfo=dt.UTC)
|
||||||
|
|
||||||
|
|
||||||
|
class Api:
|
||||||
|
def __init__(self):
|
||||||
|
self.patches = []
|
||||||
|
self.auth = []
|
||||||
|
self.es_status = {"conditions": [{"type": "Ready", "status": "True"}], "refreshTime": "2026-11-03T09:29:00Z"}
|
||||||
|
self.dep = {"metadata": {"generation": 5}, "spec": {"replicas": 1},
|
||||||
|
"status": {"observedGeneration": 5, "replicas": 1, "updatedReplicas": 1, "availableReplicas": 1}}
|
||||||
|
self.gets_until_ready = 0
|
||||||
|
self.forbidden = False
|
||||||
|
|
||||||
|
|
||||||
|
def serve(api: Api):
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
def _send(self, code, body):
|
||||||
|
raw = json.dumps(body).encode()
|
||||||
|
self.send_response(code)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(raw)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(raw)
|
||||||
|
|
||||||
|
def do_GET(self):
|
||||||
|
api.auth.append(self.headers.get("Authorization"))
|
||||||
|
if api.forbidden:
|
||||||
|
return self._send(403, {"kind": "Status"})
|
||||||
|
if self.path == ES:
|
||||||
|
return self._send(200, {"status": api.es_status})
|
||||||
|
if self.path == DEP:
|
||||||
|
if api.gets_until_ready > 0:
|
||||||
|
api.gets_until_ready -= 1
|
||||||
|
return self._send(200, {**api.dep, "status": {**api.dep["status"], "updatedReplicas": 0,
|
||||||
|
"unavailableReplicas": 1}})
|
||||||
|
return self._send(200, api.dep)
|
||||||
|
self._send(404, {})
|
||||||
|
|
||||||
|
def do_PATCH(self):
|
||||||
|
api.patches.append((self.path, self.headers.get("Content-Type"),
|
||||||
|
json.loads(self.rfile.read(int(self.headers["Content-Length"])))))
|
||||||
|
if api.forbidden:
|
||||||
|
return self._send(403, {"kind": "Status"})
|
||||||
|
if self.path == DEP:
|
||||||
|
api.dep["metadata"]["generation"] += 1
|
||||||
|
api.dep["status"]["observedGeneration"] = api.dep["metadata"]["generation"]
|
||||||
|
self._send(200, {})
|
||||||
|
|
||||||
|
def log_message(self, *a):
|
||||||
|
return
|
||||||
|
|
||||||
|
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||||
|
threading.Thread(target=server.serve_forever, kwargs={"poll_interval": 0.02}, daemon=True).start()
|
||||||
|
return server
|
||||||
|
|
||||||
|
|
||||||
|
class KubeTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.api = Api()
|
||||||
|
self.server = serve(self.api)
|
||||||
|
self.addCleanup(self.server.shutdown)
|
||||||
|
tmp = tempfile.TemporaryDirectory()
|
||||||
|
self.addCleanup(tmp.cleanup)
|
||||||
|
token = pathlib.Path(tmp.name) / "token"
|
||||||
|
token.write_text("synthetic-sa-token\n")
|
||||||
|
self.kube = OPS.Kube(api=f"http://127.0.0.1:{self.server.server_port}", token_file=str(token), ca_file=None)
|
||||||
|
|
||||||
|
def test_force_sync_patches_only_the_annotation(self):
|
||||||
|
delivery = OPS.KubeDelivery(self.kube, "sso", "authelia-secrets", clock=lambda: T0)
|
||||||
|
self.assertEqual(delivery.force_sync(), T0)
|
||||||
|
((path, ctype, body),) = self.api.patches
|
||||||
|
self.assertEqual((path, ctype), (ES, "application/merge-patch+json"))
|
||||||
|
self.assertEqual(body, {"metadata": {"annotations": {"force-sync": str(int(T0.timestamp()))}}})
|
||||||
|
self.assertEqual(self.api.auth, []) # no read before the patch
|
||||||
|
|
||||||
|
def test_synced_needs_ready_and_a_refresh_after_the_sync(self):
|
||||||
|
delivery = OPS.KubeDelivery(self.kube, "sso", "authelia-secrets")
|
||||||
|
self.assertFalse(delivery.synced(T0)) # refreshed before T0
|
||||||
|
self.api.es_status["refreshTime"] = "2026-11-03T09:30:05Z"
|
||||||
|
self.assertTrue(delivery.synced(T0))
|
||||||
|
self.api.es_status["conditions"][0]["status"] = "False"
|
||||||
|
self.assertFalse(delivery.synced(T0))
|
||||||
|
|
||||||
|
def test_restart_patches_restarted_at_and_waits_for_the_rollout(self):
|
||||||
|
self.api.gets_until_ready = 2
|
||||||
|
restart = OPS.KubeRestart(self.kube, "sso", "authelia", timeout=10, poll=0.01, clock=lambda: T0)
|
||||||
|
restart.restart()
|
||||||
|
((path, _, body),) = self.api.patches
|
||||||
|
self.assertEqual(path, DEP)
|
||||||
|
self.assertEqual(body["spec"]["template"]["metadata"]["annotations"]["kubectl.kubernetes.io/restartedAt"],
|
||||||
|
"2026-11-03T09:30:00Z")
|
||||||
|
self.assertEqual(self.api.gets_until_ready, 0)
|
||||||
|
|
||||||
|
def test_rollout_that_never_becomes_ready_is_a_definite_failure(self):
|
||||||
|
self.api.gets_until_ready = 10 ** 6
|
||||||
|
ticks = iter(range(10 ** 6))
|
||||||
|
restart = OPS.KubeRestart(self.kube, "sso", "authelia", timeout=5, poll=0, monotonic=lambda: next(ticks),
|
||||||
|
sleep=lambda s: None)
|
||||||
|
with self.assertRaises(OPS.RolloutFailed) as caught:
|
||||||
|
restart.restart()
|
||||||
|
self.assertTrue(caught.exception.definite_failure)
|
||||||
|
|
||||||
|
def test_forbidden_is_an_operation_error_and_sends_the_bearer(self):
|
||||||
|
self.api.forbidden = True
|
||||||
|
with self.assertRaises(OPS.OperationError):
|
||||||
|
OPS.KubeDelivery(self.kube, "sso", "authelia-secrets").synced(T0)
|
||||||
|
self.assertEqual(self.api.auth, ["Bearer synthetic-sa-token"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -105,7 +105,7 @@ for up to 120 s, so a check run sooner proves nothing.
|
||||||
| BEA-7 | No `human-intrinsic` step. A `root-class` step needs `high` risk |
|
| BEA-7 | No `human-intrinsic` step. A `root-class` step needs `high` risk |
|
||||||
| BEA-8 | Every step's consumer is in the blast radius. `medium` and `high` risk change one consumer at a time |
|
| BEA-8 | Every step's consumer is in the blast radius. `medium` and `high` risk change one consumer at a time |
|
||||||
| BEA-9 | Every mutating step names checks, revert steps included: a revert must prove the restored state with its own checks. Every check is used and pinned: its `ref` and `sha256` appear in `plan.source.files` |
|
| BEA-9 | Every mutating step names checks, revert steps included: a revert must prove the restored state with its own checks. Every check is used and pinned: its `ref` and `sha256` appear in `plan.source.files` |
|
||||||
| BEA-10 | Every mutating step has a revert step, or is `irreversible` at `high` risk. Revert steps mutate and do not nest. Step ids are unique |
|
| BEA-10 | Every mutating step has a revert step, or is `irreversible` at `high` risk. Revert steps mutate and do not nest. Step ids are unique. The executor runs revert steps in their declared order (the recovery order: data, then delivery, then restart), limited to those whose forward step ran |
|
||||||
| BEA-11 | A revertable plan reverts on check failure. A plan with nothing to revert stops |
|
| BEA-11 | A revertable plan reverts on check failure. A plan with nothing to revert stops |
|
||||||
| BEA-12 | The wall time fits the window, the checks (timeouts plus settle times) fit the wall time, and attempts are at most 3 (1 at `high` risk) |
|
| BEA-12 | The wall time fits the window, the checks (timeouts plus settle times) fit the wall time, and attempts are at most 3 (1 at `high` risk) |
|
||||||
| BEA-13 | The executed source is pinned by file and commit |
|
| BEA-13 | The executed source is pinned by file and commit |
|
||||||
|
|
@ -208,6 +208,8 @@ consume, and `execute-step` after it. The T06 pilot runs without the gate: its r
|
||||||
gate as `not-evaluated` and claims no allow. Any further admission needs the gate, or an exception
|
gate as `not-evaluated` and claims no allow. Any further admission needs the gate, or an exception
|
||||||
the founder records (founder decision 2026-10-31).
|
the founder records (founder decision 2026-10-31).
|
||||||
|
|
||||||
|
**Failure classes.** A failed check, or an operation that reports a definite failure (for example a rollout that never becomes ready), reverts per `on_check_failure`. An error whose outcome is not established stops (`on_unknown: stop`) without reverting. Receipts name the error type, never its message.
|
||||||
|
|
||||||
**Credentials during a run.** Write roles exist only during the admitted window: the platform creates
|
**Credentials during a run.** Write roles exist only during the admitted window: the platform creates
|
||||||
them when an admission is approved and deletes them after the receipt (CCR-2026-0058). A directory
|
them when an admission is approved and deletes them after the receipt (CCR-2026-0058). A directory
|
||||||
bind rotates its own password, so no password-manager account is needed (founder decisions
|
bind rotates its own password, so no password-manager account is needed (founder decisions
|
||||||
|
|
|
||||||
41
docs/evidence/2026-10-31-nk56-t12-synthetic-pilot/README.md
Normal file
41
docs/evidence/2026-10-31-nk56-t12-synthetic-pilot/README.md
Normal file
|
|
@ -0,0 +1,41 @@
|
||||||
|
# NK-WP-0056-T12: the executor rotates a bind on the synthetic stack (2026-10-31)
|
||||||
|
|
||||||
|
Agent-run, local Docker, generated values, everything removed afterwards: `uv run bounded-executor/synthetic_pilot.py`.
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
|
||||||
|
- The T08 stack: LLDAP, privacyIDEA, Authelia and KeyCape on the pinned images.
|
||||||
|
- An OpenBao dev server on the platform's pinned 2.5.4 image, with KV v2 at `platform` and `cas_required` on the
|
||||||
|
`svc-authelia` bind.
|
||||||
|
- The executor gets its own token with CCR-2026-0058's policy file plus a journey-read policy.
|
||||||
|
- Stand-ins:
|
||||||
|
- approval-engine: an in-process carrier with its consume semantics;
|
||||||
|
- audit-core: an in-memory sink;
|
||||||
|
- the ExternalSecret: re-projects the KV value into Authelia's configuration;
|
||||||
|
- the rollout: restarts the Authelia container.
|
||||||
|
- The Kubernetes adapters are unit-tested against a fake API server (`bounded-executor/tests/test_operations.py`).
|
||||||
|
|
||||||
|
Report: [`pilot-report.json`](pilot-report.json). It is value-free: scanned, and the harness asserts that no generated
|
||||||
|
value is present.
|
||||||
|
|
||||||
|
| Run | Outcome | Evidence |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| BEA-2026-9001 rotation | `completed` | KV v1→v2; LLDAP refuses the old value and accepts the new; delivery, restart and synthetic journey pass; started event and receipt emitted |
|
||||||
|
| BEA-2026-9002 one-off wrong delivery | `reverted` | Authelia never becomes ready after the restart (`RolloutFailed`, a definite failure). Revert in declared order, each step passing its own check: `restore-kv` → `restore-directory` → `redeliver` → `restart-again`. The value from before the run is current again (KV v4); the journey passes |
|
||||||
|
|
||||||
|
## Findings that changed the design
|
||||||
|
|
||||||
|
1. **CCR-2026-0058's policy refused reads.** OpenBao 2.5.4 applies `required_parameters = ["options"]` to GET as well,
|
||||||
|
so the executor could not read the version it must name for CAS. The executor token got 403 on every read.
|
||||||
|
- Without that line, read and CAS write work.
|
||||||
|
- A stale CAS and a write without `options` are refused, by `cas_required` on the key.
|
||||||
|
- Metadata, delete, sibling reads and new paths are still denied.
|
||||||
|
- The fix is in railiance-platform's renderer and policy file. CCR-2026-0058 needs re-approval.
|
||||||
|
2. **Revert order.** Undoing in strict reverse order would restart Authelia before its password was restored. Revert
|
||||||
|
steps now run in the order the admission declares (data, then delivery, then restart), limited to steps that ran.
|
||||||
|
The standard's BEA-10 says so.
|
||||||
|
3. **Definite failures revert.** A rollout that never becomes ready was first handled like an unknown outcome
|
||||||
|
(stop, no revert). Operations can now signal a definite failure (`definite_failure`, `executor.StepFailed`),
|
||||||
|
which the runner reverts like a failed check. Truly ambiguous errors still stop.
|
||||||
|
4. **Error types in receipts.** Receipts now carry the error *type* of a failed or unknown step, never its message.
|
||||||
|
5. **Stand-in artifact:** a restarted container gets a new host port. Fixed in the harness; not relevant to Kubernetes.
|
||||||
|
|
@ -0,0 +1,162 @@
|
||||||
|
{
|
||||||
|
"schema": "netkingdom.bounded-executor.synthetic-pilot/v1",
|
||||||
|
"started_at": "2026-10-31T15:21:55+00:00",
|
||||||
|
"images": {
|
||||||
|
"openbao": "quay.io/openbao/openbao@sha256:436eaf9778cad75507ff70ea26ace30dcbe15606e619ac3823495663d7f7c115"
|
||||||
|
},
|
||||||
|
"policy_file_sha256": "f14f0e0b885302b5931c1e84e258b5ee32a596fafab5b961d88becf7825eeeb0",
|
||||||
|
"runs": [
|
||||||
|
{
|
||||||
|
"admission": "BEA-2026-9001",
|
||||||
|
"outcome": "completed",
|
||||||
|
"reason": null,
|
||||||
|
"kv_version": [
|
||||||
|
1,
|
||||||
|
2
|
||||||
|
],
|
||||||
|
"kv_value_changed": true,
|
||||||
|
"previous_value_refused_by_lldap": true,
|
||||||
|
"current_kv_value_signs_in": true,
|
||||||
|
"steps": [
|
||||||
|
[
|
||||||
|
"generate",
|
||||||
|
"step",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"set-directory",
|
||||||
|
"step",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"deliver",
|
||||||
|
"step",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"restart",
|
||||||
|
"step",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
]
|
||||||
|
],
|
||||||
|
"reverted": false,
|
||||||
|
"pdp": "not-evaluated",
|
||||||
|
"events": [
|
||||||
|
"execution.run.started",
|
||||||
|
"execution.attribution.receipt"
|
||||||
|
],
|
||||||
|
"met": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"admission": "BEA-2026-9002",
|
||||||
|
"outcome": "reverted",
|
||||||
|
"reason": null,
|
||||||
|
"kv_version": [
|
||||||
|
2,
|
||||||
|
4
|
||||||
|
],
|
||||||
|
"kv_value_changed": false,
|
||||||
|
"previous_value_refused_by_lldap": false,
|
||||||
|
"current_kv_value_signs_in": true,
|
||||||
|
"steps": [
|
||||||
|
[
|
||||||
|
"generate",
|
||||||
|
"step",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"set-directory",
|
||||||
|
"step",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"deliver",
|
||||||
|
"step",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"restart",
|
||||||
|
"step",
|
||||||
|
"failed",
|
||||||
|
[],
|
||||||
|
"RolloutFailed"
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"restore-kv",
|
||||||
|
"revert",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"restore-directory",
|
||||||
|
"revert",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"redeliver",
|
||||||
|
"revert",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"restart-again",
|
||||||
|
"revert",
|
||||||
|
"done",
|
||||||
|
[
|
||||||
|
"pass"
|
||||||
|
],
|
||||||
|
null
|
||||||
|
]
|
||||||
|
],
|
||||||
|
"reverted": true,
|
||||||
|
"pdp": "not-evaluated",
|
||||||
|
"events": [
|
||||||
|
"execution.run.started",
|
||||||
|
"execution.attribution.receipt"
|
||||||
|
],
|
||||||
|
"met": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"alerts": [],
|
||||||
|
"no_values_in_report": true,
|
||||||
|
"all_met": true,
|
||||||
|
"finished_at": "2026-10-31T15:22:57+00:00"
|
||||||
|
}
|
||||||
|
|
@ -470,7 +470,7 @@ Standard and validator change: BEA-9 now applies to revert steps, which name the
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: NK-WP-0056-T12
|
id: NK-WP-0056-T12
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
assignee: claude-code
|
assignee: claude-code
|
||||||
depends_on: [NK-WP-0056-T11, NK-WP-0056-T08]
|
depends_on: [NK-WP-0056-T11, NK-WP-0056-T08]
|
||||||
|
|
@ -489,7 +489,51 @@ Run them on the T08 synthetic stack, including a revert. Add the Containerfile a
|
||||||
net-kingdom. The `rapp-bounded-executor` package is created with `rmgr rapp wrap`, and the founder creates the
|
net-kingdom. The `rapp-bounded-executor` package is created with `rmgr rapp wrap`, and the founder creates the
|
||||||
Forgejo remote. It runs one Job per admission in `sso` with an egress-only NetworkPolicy.
|
Forgejo remote. It runs one Job per admission in `sso` with an egress-only NetworkPolicy.
|
||||||
|
|
||||||
**Done when:** a synthetic rotation completes and a deliberately failing one reverts, with receipts; the image builds.
|
**Done when:** a synthetic rotation completes and a deliberately failing one reverts, with receipts.
|
||||||
|
|
||||||
|
**Done 2026-10-31** ([evidence](../docs/evidence/2026-10-31-nk56-t12-synthetic-pilot/README.md)).
|
||||||
|
`bounded-executor/operations.py`:
|
||||||
|
|
||||||
|
- the OpenBao KV v2 CAS client;
|
||||||
|
- LLDAP self-rotation, the same operation forward and in revert;
|
||||||
|
- the Kubernetes ExternalSecret force-sync and status check, and the rollout restart that raises `RolloutFailed`;
|
||||||
|
- the pilot checks.
|
||||||
|
|
||||||
|
`bounded-executor/synthetic_pilot.py`: rotation `completed`, and a one-off wrong delivery `reverted` in declared order
|
||||||
|
with the earlier value restored. 29 tests.
|
||||||
|
|
||||||
|
Findings:
|
||||||
|
|
||||||
|
- CCR-2026-0058's `required_parameters` refused reads. Corrected in railiance-platform; needs re-approval.
|
||||||
|
- Revert steps now run in declared order (BEA-10).
|
||||||
|
- Definite failures revert; unknown outcomes stop.
|
||||||
|
- Receipts carry error types.
|
||||||
|
|
||||||
|
The image part moved to T13 (split agreed 2026-10-31).
|
||||||
|
|
||||||
|
## Executor runtime: packaging
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: NK-WP-0056-T13
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
assignee: claude-code
|
||||||
|
depends_on: [NK-WP-0056-T12]
|
||||||
|
```
|
||||||
|
|
||||||
|
The CLI entry point for one admission:
|
||||||
|
|
||||||
|
- reads the admission, the approval id and the configuration;
|
||||||
|
- uses Kubernetes auth to OpenBao;
|
||||||
|
- uses the in-cluster Kubernetes and audit sender;
|
||||||
|
- ships the pinned `lldap_set_password` in the image.
|
||||||
|
|
||||||
|
Also: the Containerfile and the Forgejo image workflow in net-kingdom, and the `rapp-bounded-executor` package
|
||||||
|
(`rmgr rapp wrap`, founder creates the Forgejo remote). The package holds one Job per admission in `sso` with the
|
||||||
|
T03 ServiceAccount, Role and an egress-only NetworkPolicy.
|
||||||
|
|
||||||
|
**Done when:** the image builds in the workflow, the rapp renders and passes `server-dry-run` against a synthetic
|
||||||
|
manifest check, and nothing is applied.
|
||||||
|
|
||||||
## Pilot: unattended bind rotation under one admission
|
## Pilot: unattended bind rotation under one admission
|
||||||
|
|
||||||
|
|
@ -500,7 +544,7 @@ priority: high
|
||||||
assignee: claude-code
|
assignee: claude-code
|
||||||
needs_human: true
|
needs_human: true
|
||||||
intervention_note: "Founder signs the admission for the single live pilot rotation."
|
intervention_note: "Founder signs the admission for the single live pilot rotation."
|
||||||
depends_on: [NK-WP-0056-T01, NK-WP-0056-T03, NK-WP-0056-T04, NK-WP-0056-T05, NK-WP-0056-T10, NK-WP-0056-T12]
|
depends_on: [NK-WP-0056-T01, NK-WP-0056-T03, NK-WP-0056-T04, NK-WP-0056-T05, NK-WP-0056-T10, NK-WP-0056-T13]
|
||||||
state_hub_task_id: "6c3939f8-23a9-5f70-a3b1-94f4b291706b"
|
state_hub_task_id: "6c3939f8-23a9-5f70-a3b1-94f4b291706b"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue