NK-WP-0056-T06: executor waits for every endpoint; errors named by phase, never mislabelled as no-effect
All checks were successful
All checks were successful
Two live runs of BEA-2026-0002 ended setup:URLError while four read-only diagnostic pods (same image, labels, mounts and config) reached every endpoint. Likely cause: a fresh Job pod reaches OpenBao before the sso/approval-engine ingress rules admit it (NetworkPolicy is programmed per destination). Now: - build waits until OpenBao, KeyCape, approval-engine, audit-core and LLDAP all answer HTTP (one shared 30 s budget); an unreachable one is named: setup:<name> not reachable. - a preflight error is a refusal named preflight_error:<Type> (preflight has no effect). - an error in the consume read-back is outcome unknown, consume_unconfirmed:<Type>, alerted; before, any exception after setup was reported as 'refused, no effect' even after consume. - any other escaping error after setup is outcome unknown (exit 5), never 'no effect'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 523928@bnt-lap001 Assistant-Session: 6c737afb-9c14-4152-9f02-477214b9c350
This commit is contained in:
parent
a962bc8ecc
commit
c2a0f2f4e0
5 changed files with 116 additions and 9 deletions
|
|
@ -62,7 +62,8 @@ def _opener(ca_file: str | None):
|
|||
|
||||
|
||||
def wait_reachable(addr: str, *, timeout: float = 30, interval: float = 1, ca_file: str | None = None,
|
||||
opener=None, clock=time.monotonic, sleep=time.sleep) -> float:
|
||||
opener=None, clock=time.monotonic, sleep=time.sleep, path: str = "/v1/sys/health",
|
||||
name: str = "openbao") -> float:
|
||||
"""Wait until OpenBao answers on the network, before the first login. k3s applies NetworkPolicy to a new pod a
|
||||
moment after it starts, and connections in that window are refused (seen live 2026-10-31: refused at 0 s,
|
||||
open at 3 s). Polls the unauthenticated sys/health. Any HTTP answer, including 429/473/503, counts as reachable;
|
||||
|
|
@ -70,7 +71,7 @@ def wait_reachable(addr: str, *, timeout: float = 30, interval: float = 1, ca_fi
|
|||
open_ = opener or _opener(ca_file).open
|
||||
start = clock()
|
||||
while True:
|
||||
request = urllib.request.Request(addr.rstrip("/") + "/v1/sys/health", method="GET")
|
||||
request = urllib.request.Request(addr.rstrip("/") + path, method="GET")
|
||||
try:
|
||||
with open_(request, timeout=5):
|
||||
return clock() - start
|
||||
|
|
@ -79,10 +80,23 @@ def wait_reachable(addr: str, *, timeout: float = 30, interval: float = 1, ca_fi
|
|||
return clock() - start
|
||||
except (urllib.error.URLError, OSError):
|
||||
if clock() - start >= timeout:
|
||||
raise Unreachable("openbao not reachable") from None
|
||||
raise Unreachable(name + " not reachable") from None
|
||||
sleep(interval)
|
||||
|
||||
|
||||
def wait_all_reachable(endpoints: dict, *, timeout: float = 30, **kwargs) -> dict:
|
||||
"""Wait until every named endpoint answers HTTP at all (any status counts), within one shared timeout. A new
|
||||
pod's NetworkPolicy is programmed per destination: OpenBao can already be reachable while the sso and
|
||||
approval-engine ingress rules do not yet admit the pod (NK-WP-0056-T06, two live runs refused setup:URLError).
|
||||
`endpoints` maps name -> (base url, path). Returns seconds waited per endpoint; raises Unreachable(name)."""
|
||||
clock = kwargs.get("clock", time.monotonic)
|
||||
start, waited = clock(), {}
|
||||
for name, (base, path) in endpoints.items():
|
||||
left = max(timeout - (clock() - start), 0.5)
|
||||
waited[name] = round(wait_reachable(base, timeout=left, path=path, name=name, **kwargs), 2)
|
||||
return waited
|
||||
|
||||
|
||||
class Unreachable(Exception):
|
||||
pass
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue