key-cape/docs/operations.md
tegwick d568b79223
All checks were successful
Build and Publish Container Image / build-and-push (push) Successful in 41s
Give the runtime real readiness, graceful shutdown and stated limits
Closes gap G08. /healthz returned a constant without probing anything, the server
called ListenAndServe with no signal handling, and the operational limits of
in-memory state, startup-loaded keys and local-only logout lived in code comments
rather than anywhere an operator would look.

/readyz probes LLDAP, Authelia and privacyIDEA; /healthz stays liveness and
probes nothing. Keeping them distinct matters: wiring liveness to dependency
health means an orchestrator restarts KeyCape when a dependency blinks, and a
restart also discards every in-flight login, so the reaction is worse than the
condition it reacts to.

LLDAP is probed with a bind rather than a dial, since a rotated or revoked
service password leaves the port open and every lookup failing -- exactly what
readiness should catch and exactly what a dial would miss. The response names the
failing check but never the reason: the endpoint is unauthenticated and upstream
error text carries hostnames and sometimes credentials-in-URLs. Results are
cached for 2s so an unauthenticated endpoint cannot be used to drive unbounded
upstream traffic, and probes run concurrently under a 3s bound so a hung
dependency makes the endpoint answer rather than hang with it.

SIGTERM and SIGINT now drain in-flight requests for 15s, under the 30s read/write
timeouts so a stuck request cannot outlive the window before SIGKILL.

docs/operations.md states the single-replica topology and why, and three limits
easy to get wrong: the constant key-1 key ID makes same-kid rotation a trap for
consumers caching JWKS, removing a client does not revoke its issued tokens, and
/logout is local only. No throughput figures are given, since nothing here
benchmarks KeyCape. Shared storage and refresh tokens stay excluded, as G08
allows.

Verified in the running executable: 503 naming all three checks failed while
/healthz returned 200, the LLDAP check flipping to ok once started, and 40/40
requests succeeding across a SIGTERM.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 713576@bnt-lap001
Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 09:43:49 +02:00

4.9 KiB

Operating KeyCape

The supported deployment topology and the limits that come with it. These are deliberate boundaries of the current implementation, not defects awaiting a fix: where a limit is a consequence of a design choice, the choice is named.

Topology: exactly one replica

Run one instance per issuer. Authorization codes, login sessions and registration/enrollment handoffs are held in process memory. There is no shared store, and no sticky-session configuration makes this safe: a browser that starts a login on one replica and returns from Authelia to another finds no session and must start again. Two replicas do not halve the failure rate, they roughly double the login failure rate.

This is a documented exclusion rather than a missing feature. Adding a shared session store is a real option if the profile ever requires horizontal scale; it is not required today.

Consequences to plan for:

  • A restart drops in-flight logins. Anyone mid-login gets an error and must start again. Issued tokens are unaffected — they are self-contained JWTs and stay valid until they expire.
  • Rolling deployments are single-instance rollovers, so expect a brief window where new logins fail. Draining (below) protects requests already in flight, not logins waiting on a human at Authelia's password prompt.

Liveness and readiness

Endpoint Answers Use it for
/healthz Is the process up? Probes nothing. Liveness.
/readyz Are LLDAP, Authelia and privacyIDEA reachable? Readiness / traffic gating.

Keep these distinct. Wiring liveness to /readyz means an orchestrator restarts KeyCape whenever a dependency blinks, turning someone else's blip into an outage of your own — and a restart also discards every in-flight login, making it worse than the condition it reacted to.

/readyz returns 200 with status: ready, or 503 with status: not_ready and the failing check named. It reports which check failed, never why: the endpoint is unauthenticated and upstream error text carries hostnames and occasionally credentials-in-URLs. The reason is in the server log.

Probes are reachability and credential checks, not functional tests. LLDAP is probed with a bind, so a rotated or revoked service password is caught — that leaves the port open and every lookup failing, which is exactly what readiness should catch. Authelia and privacyIDEA are probed with a plain HTTP GET: any response means something is listening and speaking HTTP.

Results are cached for 2 seconds and each probe is bounded at 3 seconds. The cache is not an optimisation: the endpoint is necessarily unauthenticated, and without it anyone able to reach it could drive one upstream request per dependency per call.

Shutdown

On SIGTERM or SIGINT the server stops accepting connections and gives in-flight requests up to 15 seconds to finish. The grace period is deliberately under the 30-second read/write timeouts, so a stuck request cannot outlive the window an orchestrator typically allows before SIGKILL.

In-memory login and authorization state is not preserved across shutdown, by design — see the topology section.

Key and registration lifecycle

The signing key and all client registrations are read once at startup. Both are changed by editing configuration and restarting; there is no reload signal and no rotation service.

The key ID is the constant key-1. Rotating the signing key while keeping that identifier is a trap: a consumer caching JWKS by kid may keep the old key and reject freshly issued tokens until its cache expires. Plan rotation as "publish new keys, let consumers refetch, then issue with the new key", and treat a same-kid swap as a breaking change for anyone caching aggressively.

Removing a client registration does not revoke tokens already issued to it. There is no introspection or revocation endpoint, so an issued token stays valid until it expires — 15 minutes by default, or the client's tokenLifetime. To cut off a compromised client, remove the registration and wait out the lifetime, or rotate the signing key if you cannot.

/logout clears the local KeyCape session only. It does not end the Authelia session and does not revoke any issued token. A user who logs out and back in may not be prompted for credentials, because the upstream session is still valid.

Transport

The server speaks plain HTTP. TLS termination belongs to the deployment. KeyCape deliberately does not check or gate on the transport used to reach its upstream providers either; upstream ID tokens are verified cryptographically instead, so that assurance does not depend on the network being what we believe it is (KEY-WP-0019).

What is not claimed

No resource-efficiency or throughput bounds are asserted here. Nothing in this repository benchmarks KeyCape, so any figure would be invention. Measure it in your own deployment before sizing against it.