tegwick
dd8d5781e3
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
118 lines
4.6 KiB
Markdown
118 lines
4.6 KiB
Markdown
---
|
|
id: KEY-WP-0025
|
|
type: workplan
|
|
title: "Give the runtime real readiness, graceful shutdown and stated operational limits"
|
|
domain: infotech
|
|
repo: key-cape
|
|
status: finished
|
|
owner: claude
|
|
topic_slug: runtime-lifecycle-and-readiness
|
|
created: "2026-09-08"
|
|
updated: "2026-09-08"
|
|
state_hub_workstream_id: "70f74bbd-6748-5bbe-b261-a310b6f24e0c"
|
|
---
|
|
|
|
Closes gap G08. `/healthz` returned a constant without probing anything, the
|
|
server called `ListenAndServe` with no signal handling, and the operational
|
|
limits of in-memory state, startup-loaded keys and local-only logout were spread
|
|
across code comments rather than stated anywhere an operator would look.
|
|
|
|
G08 explicitly does not require shared storage or refresh tokens. The work is
|
|
therefore to make readiness real, make restarts survivable for in-flight
|
|
requests, and write the limits down honestly.
|
|
|
|
## Separate readiness from liveness
|
|
|
|
```task
|
|
id: KEY-WP-0025-T01
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "0e0d3349-efaf-546c-a58d-2a9b736278a9"
|
|
```
|
|
|
|
Added `internal/server/readiness` and a `/readyz` endpoint probing LLDAP,
|
|
Authelia and privacyIDEA. `/healthz` stays liveness and deliberately probes
|
|
nothing: wiring liveness to dependency health means an orchestrator restarts
|
|
KeyCape when a dependency blinks, and since a restart also discards every
|
|
in-flight login, the reaction is worse than the condition.
|
|
|
|
Design decisions worth keeping:
|
|
|
|
- **LLDAP is probed with a bind, not a dial.** A rotated or revoked service
|
|
password leaves the port open and every lookup failing — precisely the state
|
|
readiness exists to catch, and precisely what a TCP dial would miss.
|
|
- **The body names the failing check but never the reason.** The endpoint is
|
|
unauthenticated, and upstream error text carries hostnames and sometimes
|
|
credentials-in-URLs. Detail goes to the log. A test asserts a probe error
|
|
containing a credentialed URL does not reach the response.
|
|
- **Results are cached for 2s.** Not an optimisation: without it, anyone able to
|
|
reach an unauthenticated endpoint could drive one upstream request per
|
|
dependency per call.
|
|
- **Probes run concurrently, each bounded at 3s**, so readiness costs the slowest
|
|
probe rather than their sum, and a hung dependency makes the endpoint answer
|
|
rather than hang alongside it.
|
|
|
|
## Shut down without dropping requests
|
|
|
|
```task
|
|
id: KEY-WP-0025-T02
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "0bea0780-523a-587e-a157-d429df15307b"
|
|
```
|
|
|
|
`SIGTERM`/`SIGINT` now drain in-flight requests for up to 15 seconds. The grace
|
|
period sits under the 30s read/write timeouts so a stuck request cannot outlive
|
|
the window an orchestrator allows before `SIGKILL`.
|
|
|
|
In-memory login state is deliberately not preserved: a browser mid-login must
|
|
start again after a restart. That is a property of the single-replica topology,
|
|
documented rather than papered over.
|
|
|
|
## State the operational limits
|
|
|
|
```task
|
|
id: KEY-WP-0025-T03
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "97eba4bb-f8a0-5941-bdc8-6d59b0fa2942"
|
|
```
|
|
|
|
`docs/operations.md` states the supported topology — exactly one replica, because
|
|
authorization codes, login sessions and handoffs are process-local and no sticky
|
|
configuration makes a second replica safe — and the consequences: restarts drop
|
|
in-flight logins while issued tokens keep working, and rolling deploys have a
|
|
window where new logins fail.
|
|
|
|
It also records three limits that are easy to get wrong in operation:
|
|
|
|
- the key ID is the constant `key-1`, so rotating the key without changing it can
|
|
leave consumers caching JWKS by `kid` rejecting freshly issued tokens;
|
|
- removing a client registration does not revoke its issued tokens, since there
|
|
is no revocation or introspection endpoint — they stay valid until expiry;
|
|
- `/logout` clears the local session only, not the Authelia session or any token.
|
|
|
|
No throughput or resource figures are given: nothing here benchmarks KeyCape, so
|
|
any number would be invented.
|
|
|
|
## Verify in the running server
|
|
|
|
```task
|
|
id: KEY-WP-0025-T04
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "bc64fdc2-0a31-5dab-afd1-5e7b6d11d2b3"
|
|
```
|
|
|
|
Checked against the built executable rather than the handlers, following the
|
|
lesson from KEY-WP-0023:
|
|
|
|
- with every dependency down, `/readyz` returns 503 naming all three as failed
|
|
while `/healthz` still returns 200 — the split doing its job;
|
|
- starting LLDAP flips its check to `ok` while the others stay failed, so the
|
|
probes are measuring what they claim;
|
|
- 40 requests issued across a `SIGTERM` all returned 200, and the log shows the
|
|
signal handled and shutdown completed.
|
|
|
|
Seven unit tests cover the reporter, including the disclosure, caching, timeout
|
|
and concurrency properties above.
|