Bound /readyz so kubelet probes cannot hang the Service
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 2s

/readyz walked the hash chain and opened pooled connections with no
libpq connect_timeout, so a 2s kubelet probe never saw a response and
the pod stayed unready. Informed Decision accept is blocked on that.

Probe health() only, under a 1.5s budget, publish last-known
tamper_evidence, and fail TCP handshake in 1s. Integrity stays on
/v1/integrity.

Assistant: grok
Assistant-Session: 01a0a182-bab7-7f11-b32b-d06f3af52082
This commit is contained in:
tegwick 2026-09-14 22:13:55 +02:00
parent e7e054d8d6
commit b0e6792cf0
9 changed files with 276 additions and 18 deletions

View file

@ -29,7 +29,7 @@ warden route show database-dynamic-credentials --json
| Check | Meaning |
| --- | --- |
| `GET /healthz` | Process is up. Liveness uses this. A database outage must **not** restart the pod. |
| `GET /readyz` | Custody is reachable and `custody_class=operational`. Also reports `recoverable_days` and `tamper_evidence`. Readiness uses this; the pod leaves the Service rather than accept events it cannot store. |
| `GET /readyz` | Custody is reachable (`SELECT 1`) within a 1.5s budget. Reports `custody_class`, `recoverable_days`, and last-known `tamper_evidence`. Does **not** walk the hash chain — kubelet `timeoutSeconds` is 2, and a hung walk takes the pod out of the Service. Readiness uses this; the pod leaves the Service rather than accept events it cannot store. Integrity evaluation is `/v1/integrity`. |
| `GET /v1/stats` | In-process counters since start (`accepted`, `duplicate`, `conflict`, `rejected`, `unauthorized`, `forbidden`, `unavailable`, `error`). Resets on restart. Requires `may_read` and full tenant scope. |
| `GET /v1/integrity` | Hash-chain walk: `{intact, events, head, first_break}`. No payloads. Requires `may_read` and full tenant scope. A break is a custody defect, not a sender retry. |