coulomb/rapp-canned-prompts
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Lease-watch verdict must check what it claims

Browse source 
Version 3 reported 'survived past the lease TTL' after a clean 12-minute window
on a 13-minute-old pod. Every measurement in it was accurate; the conclusion
was not, because nothing checked that the observation window had actually
exceeded the 30-minute lease it claimed to have outlasted.

Fifth defect in this instrument, and the first to err toward reassurance.
Versions 1 and 2 cried wolf, which provokes investigation. This one would have
been believed, and readiness_state: verified recorded on it — the same way
live-image-digest-match would have been believed. A check reporting success it
has not established is indistinguishable from one that works, until it matters.

The verdict now requires uptime > lease TTL and reports INCONCLUSIVE when a
window is clean but too short. 'Clean' and 'proven' are different claims and
only one of them was being measured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bjefh8NUiEiahN4JLwoSKM

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 388925@bnt-lap001
Assistant-Session: 3507023f-e0fd-4a1e-9d90-a0d4217d1502
This commit is contained in:
tegwick 2026-09-08 10:48:35 +02:00
parent bd5ee3dfbb
commit bdbef58259
8 changed files with 55 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

17
docs/evidence/RCP-WP-0002-T04-first-deployment-2026-09-08.md
View file

@ -102,6 +102,7 @@ unable to distinguish their own failure from the failure they were watching for:
| `check_readiness` (earlier) | one `except` reported "database unreachable" for a reachable but unmigrated database |
| lease watcher v1 | empty `kubectl` output counted as an outage |
| lease watcher v2 | `grep -c` exits 1 on zero matches, so `\|\| echo "?"` appended a marker to a legitimate `0` and the verdict could never read clean |
| lease watcher v3 | printed "survived past the lease TTL" after a 12-minute window on a 13-minute-old pod — it never checked the one thing it claimed |
A verification step that cannot fail correctly is worse than none, because it is
trusted. That is the durable lesson from this deployment, more than any single
@ -118,3 +119,19 @@ direction; the same class of bug pointing the other way is what let
So the verdict logic is now itself tested — the fix was accompanied by feeding
the counter a matching line and a non-matching one and confirming it returns 1
and 0 — rather than assumed correct because it looked right.
**Version 3 then failed in the dangerous direction.** It reported
`RESULT: survived — continuously ready past the lease TTL` after a clean
12-minute window on a pod that had been up for 13 minutes. Every measurement in
it was accurate; the *conclusion* was not, because nothing checked that the
observation window exceeded the 30-minute lease it claimed to have outlasted.
That is the failure mode worth fearing. Versions 1 and 2 cried wolf, which
provokes investigation. Version 3 would have been believed — and `verified`
recorded on it — for the same reason `live-image-digest-match` would have been
believed: a check reporting success it has not established is indistinguishable
from a check that works, right up until it matters.
The verdict now requires `uptime > lease TTL` and reports `INCONCLUSIVE` when a
window is clean but too short, because "clean" and "proven" are different
claims and only one of them was ever being measured.

2
docs/evidence/RCP-WP-0002-T05-lease-rotation-2026-09-08.log Normal file
View file

@ -0,0 +1,2 @@
started 2026-09-08T10:48:16+02:00 — 22m watch (runtime lease TTL 30m)
2026-09-08T10:48:17+02:00 ready=1/1