Apply rail-knative's declared Knative CPU requests in the installer (RAIL-BS-WP-0015).
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 8s

install.sh now renders the checksum-verified upstream assets through kustomize
overlays: CRDs first and verbatim, then serving-core and kourier with the six
CPU requests lowered live on 2026-09-21, the Kourier Service as ClusterIP and
the Envoy image pinned. verify.sh checks the requests read-only, and
tests/test_knative_render.py proves the render offline against upstream and
rail-knative's declaration. Not run against the cluster.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 63291@bnt-lap001
Assistant-Session: 8bd77868-ca68-4f49-bb1e-d539ecc0d703
This commit is contained in:
codex 2026-09-21 18:45:47 +02:00
parent 4abd781ce3
commit 3a5432270e
11 changed files with 465 additions and 16 deletions

View file

@ -2,7 +2,20 @@
`install.sh` verifies repository-pinned SHA-256 checksums for the upstream
Serving and Kourier v1.22.0 assets before applying them over SSH. It is
idempotent. Kourier is kept `ClusterIP`; public entry through Traefik, DNS, and
idempotent.
It does not apply the upstream manifests as published. `render.sh` (also
usable on its own, with no cluster contact) renders them through the kustomize
overlays in `overlays/`: CRDs verbatim and first, because `serving-core.yaml`
repeats one of them; then serving-core and kourier with the CPU requests that
rail-knative declares in `substrate/v1.22.0/cpu-requests.patch.yaml`
(activator 50m, autoscaler/controller/webhook/net-kourier-controller 30m,
3scale-kourier-gateway 50m), the Kourier Service as `ClusterIP` and the Envoy
gateway image pinned to `ENVOY_IMAGE`. Memory requests and all limits stay
upstream. Applying the unpatched upstream files restores 300m/200m/100m and
exhausts railiance01's CPU requests again. `tests/test_knative_render.py`
proves the render offline and fails if these overlays drift from rail-knative's
declaration. `install.sh` needs a local `kubectl` for `kubectl kustomize`. Kourier is kept `ClusterIP`; public entry through Traefik, DNS, and
TLS requires separate reef admission evidence.
The installer enables only Knative's
@ -10,7 +23,8 @@ The installer enables only Knative's
containers for fail-closed admission checks such as verifying that egress
policy has reconciled before application code starts.
Run `install.sh railiance01`, then `verify.sh railiance01`.
Run `install.sh railiance01`, then `verify.sh railiance01`; `verify.sh` is
read-only and also checks the six CPU requests and the Envoy pin.
Before workload admission, rollback deletes Kourier, Serving core, then CRDs
using the same verified assets. After Knative Services exist, removal requires

View file

@ -1,25 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
root="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$root/release-lock.env"
target="${1:-railiance01}"
stage="$(mktemp -d)"
trap 'rm -rf "$stage"' EXIT
download() {
curl -fsSL --retry 3 "$1" -o "$2"
printf '%s %s\n' "$3" "$2" | sha256sum --check --status
}
serving="https://github.com/knative/serving/releases/download/knative-v${KNATIVE_VERSION}"
kourier="https://github.com/knative-extensions/net-kourier/releases/download/knative-v${KNATIVE_VERSION}"
download "$serving/serving-crds.yaml" "$stage/crds.yaml" "$SERVING_CRDS_SHA256"
download "$serving/serving-core.yaml" "$stage/core.yaml" "$SERVING_CORE_SHA256"
download "$kourier/kourier.yaml" "$stage/kourier.yaml" "$KOURIER_SHA256"
# Checksum-verified upstream assets, rendered with the declared CPU requests
# (rail-knative substrate/v1.22.0), Kourier ClusterIP and the Envoy pin.
"$root/render.sh" "$stage"
# CRDs first and separately: serving-core.yaml repeats one of them.
ssh "$target" kubectl apply -f - < "$stage/crds.yaml"
ssh "$target" kubectl apply -f - < "$stage/core.yaml"
ssh "$target" kubectl apply -f - < "$stage/serving-core.rendered.yaml"
ssh "$target" kubectl wait --for=condition=Available deployment --all -n knative-serving --timeout=300s
ssh "$target" kubectl apply -f - < "$stage/kourier.yaml"
ssh "$target" kubectl set image deployment/3scale-kourier-gateway -n kourier-system "kourier-gateway=$ENVOY_IMAGE"
ssh "$target" kubectl apply -f - < "$stage/kourier.rendered.yaml"
ssh "$target" kubectl patch configmap/config-network -n knative-serving --type merge -p '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}'
ssh "$target" kubectl patch configmap/config-features -n knative-serving --type merge -p '{"data":{"kubernetes.podspec-init-containers":"enabled"}}'
ssh "$target" kubectl patch service/kourier -n kourier-system --type merge -p '{"spec":{"type":"ClusterIP"}}'
ssh "$target" kubectl wait --for=condition=Available deployment --all -n kourier-system --timeout=300s

View file

@ -0,0 +1,36 @@
# CPU requests declared by rail-knative (substrate/v1.22.0/cpu-requests.patch.yaml),
# the documents of that file that target kourier.yaml. Only CPU requests are set;
# memory requests and all limits stay upstream. Live on railiance01 since
# 2026-09-21 (ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED by
# the founder). Keep in step with rail-knative: tests/test_knative_render.py
# fails if the two declarations disagree.
---
# upstream v1.22.0 request: 200m
apiVersion: apps/v1
kind: Deployment
metadata:
name: net-kourier-controller
namespace: knative-serving
spec:
template:
spec:
containers:
- name: controller
resources:
requests:
cpu: 30m
---
# upstream v1.22.0 request: 200m
apiVersion: apps/v1
kind: Deployment
metadata:
name: 3scale-kourier-gateway
namespace: kourier-system
spec:
template:
spec:
containers:
- name: kourier-gateway
resources:
requests:
cpu: 50m

View file

@ -0,0 +1,10 @@
# Overlay over the checksum-verified upstream kourier.yaml, which render.sh
# stages next to this file as upstream.yaml. render.sh appends the `images:`
# pin for the Envoy gateway from ENVOY_IMAGE in release-lock.env.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- upstream.yaml
patches:
- path: cpu-requests.patch.yaml
- path: service-clusterip.patch.yaml

View file

@ -0,0 +1,10 @@
# Kourier stays ClusterIP; public entry needs separate reef admission evidence.
# Declared here rather than patched after apply, so a re-run never flips the
# live Service to the upstream LoadBalancer type, even briefly.
apiVersion: v1
kind: Service
metadata:
name: kourier
namespace: kourier-system
spec:
type: ClusterIP

View file

@ -0,0 +1,66 @@
# CPU requests declared by rail-knative (substrate/v1.22.0/cpu-requests.patch.yaml),
# the documents of that file that target serving-core.yaml. Only CPU requests are set;
# memory requests and all limits stay upstream. Live on railiance01 since
# 2026-09-21 (ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED by
# the founder). Keep in step with rail-knative: tests/test_knative_render.py
# fails if the two declarations disagree.
---
# upstream v1.22.0 request: 300m
apiVersion: apps/v1
kind: Deployment
metadata:
name: activator
namespace: knative-serving
spec:
template:
spec:
containers:
- name: activator
resources:
requests:
cpu: 50m
---
# upstream v1.22.0 request: 100m
apiVersion: apps/v1
kind: Deployment
metadata:
name: autoscaler
namespace: knative-serving
spec:
template:
spec:
containers:
- name: autoscaler
resources:
requests:
cpu: 30m
---
# upstream v1.22.0 request: 100m
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller
namespace: knative-serving
spec:
template:
spec:
containers:
- name: controller
resources:
requests:
cpu: 30m
---
# upstream v1.22.0 request: 100m
apiVersion: apps/v1
kind: Deployment
metadata:
name: webhook
namespace: knative-serving
spec:
template:
spec:
containers:
- name: webhook
resources:
requests:
cpu: 30m

View file

@ -0,0 +1,9 @@
# Overlay over the checksum-verified upstream serving-core.yaml, which render.sh
# stages next to this file as upstream.yaml. Do not add the CRDs here: core.yaml
# repeats one of them, so they are applied separately and first.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- upstream.yaml
patches:
- path: cpu-requests.patch.yaml

42
install/knative/render.sh Executable file
View file

@ -0,0 +1,42 @@
#!/usr/bin/env bash
# Render exactly what install.sh applies, without touching any cluster.
# render.sh <stage-dir>
# Downloads the upstream v1.22.0 assets (reusing files already in <stage-dir>
# only when their pinned SHA-256 matches), verifies every checksum, and writes:
# <stage-dir>/crds.yaml upstream CRDs, verbatim (applied first)
# <stage-dir>/serving-core.rendered.yaml serving-core + declared CPU requests
# <stage-dir>/kourier.rendered.yaml kourier + CPU requests, ClusterIP, Envoy pin
# Needs curl, sha256sum and a local kubectl with built-in kustomize.
set -euo pipefail
root="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$root/release-lock.env"
stage="${1:?usage: render.sh <stage-dir>}"
mkdir -p "$stage"
fetch() {
local url="$1" out="$2" sum="$3"
if ! printf '%s %s\n' "$sum" "$out" | sha256sum --check --status 2>/dev/null; then
curl -fsSL --retry 3 "$url" -o "$out"
printf '%s %s\n' "$sum" "$out" | sha256sum --check --status \
|| { echo "checksum mismatch: $url" >&2; rm -f "$out"; exit 1; }
fi
}
serving="https://github.com/knative/serving/releases/download/knative-v${KNATIVE_VERSION}"
kourier="https://github.com/knative-extensions/net-kourier/releases/download/knative-v${KNATIVE_VERSION}"
fetch "$serving/serving-crds.yaml" "$stage/crds.yaml" "$SERVING_CRDS_SHA256"
fetch "$serving/serving-core.yaml" "$stage/core.yaml" "$SERVING_CORE_SHA256"
fetch "$kourier/kourier.yaml" "$stage/kourier.yaml" "$KOURIER_SHA256"
build() {
local overlay="$1" asset="$2" out="$3" dir="$stage/overlay-$1"
rm -rf "$dir"
cp -r "$root/overlays/$overlay" "$dir"
cp "$stage/$asset" "$dir/upstream.yaml"
if [ "$overlay" = kourier ]; then
printf 'images:\n - name: docker.io/envoyproxy/envoy\n newName: %s\n digest: %s\n' \
"${ENVOY_IMAGE%@*}" "${ENVOY_IMAGE#*@}" >> "$dir/kustomization.yaml"
fi
kubectl kustomize "$dir" > "$stage/$out"
}
build serving-core core.yaml serving-core.rendered.yaml
build kourier kourier.yaml kourier.rendered.yaml

View file

@ -1,10 +1,28 @@
#!/usr/bin/env bash
# Read-only checks of the installed Knative substrate.
set -euo pipefail
root="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$root/release-lock.env"
target="${1:-railiance01}"
ssh "$target" 'set -e
test "$(kubectl get namespace knative-serving -o go-template="{{index .metadata.labels \"app.kubernetes.io/version\"}}")" = "1.22.0"
test "$(kubectl get service kourier -n kourier-system -o jsonpath="{.spec.type}")" = "ClusterIP"
test "$(kubectl get configmap config-network -n knative-serving -o jsonpath="{.data.ingress-class}")" = "kourier.ingress.networking.knative.dev"
test "$(kubectl get configmap config-features -n knative-serving -o jsonpath="{.data.kubernetes\\.podspec-init-containers}")" = "enabled"
test "$(kubectl get deployment 3scale-kourier-gateway -n kourier-system -o jsonpath="{.spec.template.spec.containers[?(@.name==\"kourier-gateway\")].image}")" = "'"$ENVOY_IMAGE"'"
kubectl wait --for=condition=Available deployment --all -n knative-serving --timeout=120s
kubectl wait --for=condition=Available deployment --all -n kourier-system --timeout=120s'
# Declared CPU requests (rail-knative substrate/v1.22.0/cpu-requests.patch.yaml).
while read -r ns deploy container cpu; do
got="$(ssh "$target" kubectl get deployment "$deploy" -n "$ns" \
-o "jsonpath={.spec.template.spec.containers[?(@.name==\"$container\")].resources.requests.cpu}")"
[ "$got" = "$cpu" ] || { echo "cpu request $ns/$deploy/$container: want $cpu, got ${got:-<none>}" >&2; exit 1; }
done <<'LIST'
knative-serving activator activator 50m
knative-serving autoscaler autoscaler 30m
knative-serving controller controller 30m
knative-serving webhook webhook 30m
knative-serving net-kourier-controller controller 30m
kourier-system 3scale-kourier-gateway kourier-gateway 50m
LIST
echo "knative substrate verified"

View file

@ -0,0 +1,131 @@
"""Offline proof that install/knative renders the declared CPU requests.
Runs install/knative/render.sh (download + SHA-256 verification + kustomize;
no cluster contact) and compares the result with the upstream assets: the six
declared CPU requests change, the Kourier Service is ClusterIP, the gateway
image is the pinned Envoy digest, and nothing else differs. Skips when the
upstream assets cannot be fetched or kubectl is missing.
"""
from __future__ import annotations
import copy
import hashlib
import os
import re
import shutil
import subprocess
import tempfile
import unittest
from pathlib import Path
try:
import yaml
except ImportError: # pragma: no cover
yaml = None
ROOT = Path(__file__).resolve().parents[1]
KNATIVE = ROOT / "install" / "knative"
RAIL_KNATIVE = Path(os.environ.get("RAIL_KNATIVE_DIR", ROOT.parent / "rail-knative"))
DECLARED = { # (namespace, deployment, container) -> cpu request
("knative-serving", "activator", "activator"): "50m",
("knative-serving", "autoscaler", "autoscaler"): "30m",
("knative-serving", "controller", "controller"): "30m",
("knative-serving", "webhook", "webhook"): "30m",
("knative-serving", "net-kourier-controller", "controller"): "30m",
("kourier-system", "3scale-kourier-gateway", "kourier-gateway"): "50m",
}
def lock() -> dict[str, str]:
lines = (KNATIVE / "release-lock.env").read_text().splitlines()
return dict(line.split("=", 1) for line in lines if "=" in line)
def load(path: Path) -> list[dict]:
return [d for d in yaml.safe_load_all(path.read_text()) if d]
def key(doc: dict) -> tuple:
meta = doc["metadata"]
return (doc["apiVersion"], doc["kind"], meta.get("namespace"), meta["name"])
def patch_requests(path: Path) -> dict[tuple, str]:
out = {}
for doc in load(path):
for c in doc["spec"]["template"]["spec"]["containers"]:
out[(doc["metadata"]["namespace"], doc["metadata"]["name"], c["name"])] = c["resources"]["requests"]["cpu"]
return out
@unittest.skipIf(yaml is None, "PyYAML not installed")
class DeclarationTests(unittest.TestCase):
def test_overlays_carry_the_declared_requests(self) -> None:
got = {}
for overlay in ("serving-core", "kourier"):
got.update(patch_requests(KNATIVE / "overlays" / overlay / "cpu-requests.patch.yaml"))
self.assertEqual(DECLARED, got)
def test_verify_checks_the_declared_requests(self) -> None:
rows = re.search(r"<<'LIST'\n(.*?)\nLIST", (KNATIVE / "verify.sh").read_text(), re.S).group(1)
got = {tuple(r.split()[:3]): r.split()[3] for r in rows.splitlines()}
self.assertEqual(DECLARED, got)
def test_matches_rail_knative_declaration(self) -> None:
source = RAIL_KNATIVE / "substrate" / "v1.22.0" / "cpu-requests.patch.yaml"
if not source.exists():
self.skipTest(f"rail-knative not checked out at {RAIL_KNATIVE}")
self.assertEqual(DECLARED, patch_requests(source))
@unittest.skipIf(yaml is None or shutil.which("kubectl") is None, "needs PyYAML and kubectl")
class RenderTests(unittest.TestCase):
stage: Path
@classmethod
def setUpClass(cls) -> None:
cls.stage = Path(os.environ.get("KNATIVE_STAGE_DIR") or tempfile.mkdtemp(prefix="knative-render-"))
result = subprocess.run([str(KNATIVE / "render.sh"), str(cls.stage)], capture_output=True, text=True)
if result.returncode != 0:
if "checksum mismatch" in result.stderr:
raise AssertionError(result.stderr)
raise unittest.SkipTest(f"render.sh failed (offline?): {result.stderr.strip()[:200]}")
def compare(self, upstream_file: str, rendered_file: str) -> dict[tuple, dict]:
upstream = {key(d): d for d in load(self.stage / upstream_file)}
rendered = {key(d): d for d in load(self.stage / rendered_file)}
self.assertEqual(set(upstream), set(rendered))
return {k: (upstream[k], rendered[k]) for k in upstream}
def check(self, upstream_file: str, rendered_file: str) -> set:
seen = set()
envoy = lock()["ENVOY_IMAGE"]
for k, (up, out) in self.compare(upstream_file, rendered_file).items():
expected = copy.deepcopy(up)
if k[1] == "Deployment":
for c in expected["spec"]["template"]["spec"]["containers"]:
ident = (k[2], k[3], c["name"])
if ident in DECLARED:
c["resources"]["requests"]["cpu"] = DECLARED[ident]
seen.add(ident)
if c["image"].startswith("docker.io/envoyproxy/envoy:"):
c["image"] = envoy
if k[1:] == ("Service", "kourier-system", "kourier"):
expected["spec"]["type"] = "ClusterIP"
self.assertEqual(expected, out, f"{k} differs beyond the declared changes")
return seen
def test_crds_are_the_pinned_upstream_file(self) -> None:
digest = hashlib.sha256((self.stage / "crds.yaml").read_bytes()).hexdigest()
self.assertEqual(lock()["SERVING_CRDS_SHA256"], digest)
for doc in load(self.stage / "crds.yaml"):
self.assertEqual("CustomResourceDefinition", doc["kind"])
def test_rendered_carries_declared_requests_and_nothing_else_changes(self) -> None:
seen = self.check("core.yaml", "serving-core.rendered.yaml")
seen |= self.check("kourier.yaml", "kourier.rendered.yaml")
self.assertEqual(set(DECLARED), seen)
if __name__ == "__main__":
unittest.main()

View file

@ -0,0 +1,121 @@
---
id: RAIL-BS-WP-0015
type: workplan
title: "Make the Knative installer apply rail-knative's declared CPU requests"
domain: financials
repo: railiance-cluster
status: finished
flavor: implementation
owner: codex
topic_slug: railiance
created: "2026-09-21"
updated: "2026-09-21"
related:
- RAIL-BS-WP-0013
- RAIL-KNATIVE-WP-0002
---
# RAIL-BS-WP-0015 - Knative installer applies the declared CPU requests
On 2026-09-21 the CPU requests of six Knative Deployments were lowered live on
railiance01 (`ADMINISTER @ realm:kubernetes/railiance01`, `activation=APPROVED`
by the founder), to free CPU requests on a node where the identity and
user-engine backups could not be scheduled. The record is
`the-custodian/docs/kubernetes-change-gate-decision.md`. rail-knative declares
the values in `substrate/v1.22.0/cpu-requests.patch.yaml`
(RAIL-KNATIVE-WP-0002). `install/knative/install.sh` still applied the
unpatched upstream manifests, so re-running it would have restored
300m/200m/100m and stalled the backups again.
| Deployment | upstream | declared |
| --- | --- | --- |
| knative-serving/activator | 300m | 50m |
| knative-serving/autoscaler | 100m | 30m |
| knative-serving/controller | 100m | 30m |
| knative-serving/webhook | 100m | 30m |
| knative-serving/net-kourier-controller | 200m | 30m |
| kourier-system/3scale-kourier-gateway | 200m | 50m |
rail-knative's declaration, the change-gate record and the live cluster agree
on all six values.
## T01 - Render the upstream assets through overlays
```task
id: RAIL-BS-WP-0015-T01
status: done
priority: high
```
`install/knative/render.sh <dir>` downloads the three upstream assets, keeps
the SHA-256 check against `release-lock.env`, and renders with the local
`kubectl kustomize`: `crds.yaml` verbatim (applied first and separately, since
`serving-core.yaml` repeats a CRD), `serving-core.rendered.yaml` and
`kourier.rendered.yaml`. The overlays in `install/knative/overlays/` split
rail-knative's patch file by asset (kustomize refuses a patch document with no
matching resource) and keep each document as rail-knative wrote it. Only CPU
requests change; memory requests and all limits stay upstream.
The kourier overlay also declares the Service as `ClusterIP` and pins the
Envoy image from `ENVOY_IMAGE`. The old installer applied the upstream
`LoadBalancer` Service and `envoy:v1.37-latest`, then patched both back; on a
re-run that flipped the live Service and rolled the gateway twice.
## T02 - install.sh and verify.sh
```task
id: RAIL-BS-WP-0015-T02
status: done
priority: high
```
`install.sh` applies the three rendered files in the old order and keeps the
two ConfigMap patches. It drops `kubectl set image` and the Service patch,
because the render now carries both. `verify.sh` stays read-only and
additionally checks the six CPU requests and the Envoy digest.
## T03 - Offline proof and a test
```task
id: RAIL-BS-WP-0015-T03
status: done
priority: high
```
`tests/test_knative_render.py` (in `make test-unit`) runs `render.sh` and
compares every rendered object with upstream. The only differences are the six
CPU requests, the Kourier Service type and the gateway image. The test also
checks the CRD file's checksum, checks that `verify.sh` lists the same six
values, and checks that the overlays match
`rail-knative/substrate/v1.22.0/cpu-requests.patch.yaml` when rail-knative is
checked out next to this repository. It skips when offline.
Read-only evidence, 2026-09-21: `ssh railiance01 'kubectl diff -f -'` against
each rendered file.
- `crds.yaml` and `kourier.rendered.yaml` show no diff.
- `serving-core.rendered.yaml` shows no diff on any Deployment. It does show a
diff on the `webhook.serving.knative.dev` and
`validation.webhook.serving.knative.dev` webhook configurations. Their
`rules` are filled at runtime by the Knative webhook, and the unpatched
upstream file shows the same diff, so the change here did not cause it.
- For contrast, the unpatched upstream `serving-core.yaml` diffs the four
serving Deployments back to 300m/100m.
## T04 - Re-running the installer (not run)
```task
id: RAIL-BS-WP-0015-T04
status: done
priority: medium
```
This session did not run the installer. The values are already live, so
running the patched installer changes none of the six Deployments, the CRDs,
Kourier, the Service type or the gateway image. The one exception is the two
webhook configurations: apply resets them to the upstream skeleton and the
Knative webhook fills them in again, exactly as any earlier re-run did. There
is nothing to apply now. Any future run is
`ADMINISTER @ realm:kubernetes/railiance01` and needs the founder's go-ahead:
`install/knative/install.sh railiance01`, then `install/knative/verify.sh railiance01`.
Rollback of this change is a revert of its commit; the live values need no
rollback.