Add a read-only headroom preflight before State Hub promotion.
STATE-WP-0091: refuse Helm when remaining CPU cannot cover the API surge, migrate hook, or unrelated pending demand. 65m fails, 105m is narrowly sufficient, not factory admission. Assistant: grok Assistant-Session: 01a09dc1-b21e-77e1-919e-fcad2f82b267
This commit is contained in:
parent
7a4f58b90c
commit
261a48772c
9 changed files with 446 additions and 5 deletions
47
docs/evidence/2026-09-14-release-headroom.md
Normal file
47
docs/evidence/2026-09-14-release-headroom.md
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
# STATE-WP-0091 — release headroom and demand handoff
|
||||
|
||||
Date: 2026-09-14
|
||||
|
||||
## T01 — preflight
|
||||
|
||||
`scripts/release_headroom_preflight.py` is on the promotion path
|
||||
(`make railiance-state-hub-headroom`, `PROMOTE.md` step 0). It is read-only.
|
||||
|
||||
Proved against fixtures of the 2026-09-10 incident:
|
||||
|
||||
| Case | Remaining | Result |
|
||||
| --- | --- | --- |
|
||||
| Revision 61 | 65m | refuse API 100m surge |
|
||||
| Revision 63 | 105m | narrowly sufficient; not factory admission |
|
||||
| 105m plus 50m unrelated pending | 55m effective | refuse |
|
||||
|
||||
Atomic rollback stays `--atomic`. Requests are not lowered by this tool.
|
||||
|
||||
Live kubectl on 2026-09-14 refused: remaining 15m, pending unrelated 25m,
|
||||
migrate hook 50m unmet. The preflight is doing its job on the current node.
|
||||
|
||||
## T02 — MCP demand and margin
|
||||
|
||||
STATE-WP-0090 recorded 1,078 five-minute samples in a seven-day window
|
||||
(partial coverage): MCP p99 2.35m, peak 30.45m. The MCP request was cut
|
||||
50m → 10m; limit 500m, memory and replicas unchanged.
|
||||
|
||||
That sample set is still not a week of complete coverage. It is enough to
|
||||
say:
|
||||
|
||||
- 10m request is above p99 and below peak; bursts still use the 500m limit.
|
||||
- 105m node remainder is only 5m above the API surge. Concurrent MCP surge
|
||||
(10m) or any unrelated pending pod ≥ 6m makes it insufficient.
|
||||
- **Durable release margin for State Hub:** remaining CPU after current
|
||||
allocations must cover API surge 100m + migrate hook 50m sequenced
|
||||
separately, and must refuse when unrelated pending demand would consume
|
||||
the 5m sliver. Do not treat 105m as shared-capacity admission.
|
||||
|
||||
Shared allocation and provisioning stay with other owners. Receiving records:
|
||||
|
||||
- `RCLUSTER-WP-0015` (railiance-cluster) — node remaining vs surge, pending pods
|
||||
- `RESOURCE-WP-0007` (resource-control) — reef-railiance-k3s portfolio, not admission
|
||||
- `RPF-WP-0041` (railiance-platform) — no platform provisioning change from 105m
|
||||
- `CUST-WP-0071` already owns weekly fleet allocation review
|
||||
|
||||
HFACT T01/T04 factory demand is not admitted by this 105m figure.
|
||||
Loading…
Add table
Add a link
Reference in a new issue