key-cape/workplans/KEY-WP-0025-runtime-lifecycle-and-readiness.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated
KEY-WP-0025	workplan	Give the runtime real readiness, graceful shutdown and stated operational limits	infotech	key-cape	finished	claude	runtime-lifecycle-and-readiness	2026-09-08	2026-09-08

Closes gap G08. /healthz returned a constant without probing anything, the server called ListenAndServe with no signal handling, and the operational limits of in-memory state, startup-loaded keys and local-only logout were spread across code comments rather than stated anywhere an operator would look.

G08 explicitly does not require shared storage or refresh tokens. The work is therefore to make readiness real, make restarts survivable for in-flight requests, and write the limits down honestly.

Separate readiness from liveness

id: KEY-WP-0025-T01
status: done
priority: medium

Added internal/server/readiness and a /readyz endpoint probing LLDAP, Authelia and privacyIDEA. /healthz stays liveness and deliberately probes nothing: wiring liveness to dependency health means an orchestrator restarts KeyCape when a dependency blinks, and since a restart also discards every in-flight login, the reaction is worse than the condition.

Design decisions worth keeping:

• LLDAP is probed with a bind, not a dial. A rotated or revoked service password leaves the port open and every lookup failing — precisely the state readiness exists to catch, and precisely what a TCP dial would miss.
• The body names the failing check but never the reason. The endpoint is unauthenticated, and upstream error text carries hostnames and sometimes credentials-in-URLs. Detail goes to the log. A test asserts a probe error containing a credentialed URL does not reach the response.
• Results are cached for 2s. Not an optimisation: without it, anyone able to reach an unauthenticated endpoint could drive one upstream request per dependency per call.
• Probes run concurrently, each bounded at 3s, so readiness costs the slowest probe rather than their sum, and a hung dependency makes the endpoint answer rather than hang alongside it.

Shut down without dropping requests

id: KEY-WP-0025-T02
status: done
priority: medium

SIGTERM/SIGINT now drain in-flight requests for up to 15 seconds. The grace period sits under the 30s read/write timeouts so a stuck request cannot outlive the window an orchestrator allows before SIGKILL.

In-memory login state is deliberately not preserved: a browser mid-login must start again after a restart. That is a property of the single-replica topology, documented rather than papered over.

State the operational limits

id: KEY-WP-0025-T03
status: done
priority: medium

docs/operations.md states the supported topology — exactly one replica, because authorization codes, login sessions and handoffs are process-local and no sticky configuration makes a second replica safe — and the consequences: restarts drop in-flight logins while issued tokens keep working, and rolling deploys have a window where new logins fail.

It also records three limits that are easy to get wrong in operation:

• the key ID is the constant key-1, so rotating the key without changing it can leave consumers caching JWKS by kid rejecting freshly issued tokens;
• removing a client registration does not revoke its issued tokens, since there is no revocation or introspection endpoint — they stay valid until expiry;
• /logout clears the local session only, not the Authelia session or any token.

No throughput or resource figures are given: nothing here benchmarks KeyCape, so any number would be invented.

Verify in the running server

id: KEY-WP-0025-T04
status: done
priority: medium

Checked against the built executable rather than the handlers, following the lesson from KEY-WP-0023:

• with every dependency down, /readyz returns 503 naming all three as failed while /healthz still returns 200 — the split doing its job;
• starting LLDAP flips its check to ok while the others stay failed, so the probes are measuring what they claim;
• 40 requests issued across a SIGTERM all returned 200, and the log shows the signal handled and shutdown completed.

Seven unit tests cover the reporter, including the disclosure, caching, timeout and concurrency properties above.