Render rail-knative's HPA caps and 10m CPU requests in the Knative installer (RCLUSTER-WP-0020).

Follow rail-knative 76cb9c7 (RAIL-KNATIVE-WP-0003): overlays set 10m CPU
requests for the six substrate containers and add hpa-caps.patch.yaml
(maxReplicas activator 3, webhook 2, 3scale-kourier-gateway 3); verify.sh
checks both; tests/test_knative_render.py checks overlays, verify.sh, the
rail-knative declaration and the render. Not applied. Adds the receiving
task for wave-2 railiance-cluster allocation (CUST-WP-0071, RESOURCE-WP-0008).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 8340@bnt-lap001
Assistant-Session: 2e192b3d-b3e8-4403-94d4-d55a2c8b476c

WORK-RECORDS.md

@@ -23,6 +23,7 @@
 | workplan | RCLUSTER-WP-0017 | active | — | workplans/RCLUSTER-WP-0017-netkingdom-signer-runtime-storage-review.md |
 | workplan | RCLUSTER-WP-0018 | active | — | workplans/RCLUSTER-WP-0018-keycape-original-tls-recovery.md |
 | workplan | RCLUSTER-WP-0019 | active | — | workplans/RCLUSTER-WP-0019-k3s-admin-kubeconfig-mode.md |
+| workplan | RCLUSTER-WP-0020 | active | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md |
 | task | RCLUSTER-WP-ADHOC-2026-07-27-T01 | done | — | workplans/ADHOC-2026-07-27.md |
 | task | RAIL-BS-WP-0008-T01 | done | — | workplans/RAIL-BS-WP-0008-activity-core-wp0016-triage-output-deploy.md |
 | task | RAIL-BS-WP-0008-T02 | done | — | workplans/RAIL-BS-WP-0008-activity-core-wp0016-triage-output-deploy.md |
@@ -74,3 +75,6 @@
 | task | RCLUSTER-WP-0019-T01 | done | — | workplans/RCLUSTER-WP-0019-k3s-admin-kubeconfig-mode.md |
 | task | RCLUSTER-WP-0019-T02 | done | — | workplans/RCLUSTER-WP-0019-k3s-admin-kubeconfig-mode.md |
 | task | RCLUSTER-WP-0019-T03 | done | — | workplans/RCLUSTER-WP-0019-k3s-admin-kubeconfig-mode.md |
+| task | RCLUSTER-WP-0020-T01 | done | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md |
+| task | RCLUSTER-WP-0020-T02 | wait | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md |
+| task | RCLUSTER-WP-0020-T03 | todo | — | workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md |

install/knative/README.md

@@ -7,13 +7,15 @@ idempotent.
 It does not apply the upstream manifests as published. `render.sh` (also
 usable on its own, with no cluster contact) renders them through the kustomize
 overlays in `overlays/`: CRDs verbatim and first, because `serving-core.yaml`
-repeats one of them; then serving-core and kourier with the CPU requests that
-rail-knative declares in `substrate/v1.22.0/cpu-requests.patch.yaml`
-(activator 50m, autoscaler/controller/webhook/net-kourier-controller 30m,
-3scale-kourier-gateway 50m), the Kourier Service as `ClusterIP` and the Envoy
-gateway image pinned to `ENVOY_IMAGE`. Memory requests and all limits stay
+repeats one of them; then serving-core and kourier with the CPU requests and
+HPA ceilings that rail-knative declares in `substrate/v1.22.0/`
+(`cpu-requests.patch.yaml`: 10m for activator, autoscaler, controller, webhook,
+net-kourier-controller and 3scale-kourier-gateway; `hpa-caps.patch.yaml`:
+maxReplicas activator 3, webhook 2, 3scale-kourier-gateway 3; RAIL-KNATIVE-WP-0003),
+the Kourier Service as `ClusterIP` and the Envoy
+gateway image pinned to `ENVOY_IMAGE`. Memory requests, limits, minReplicas and HPA targets stay
 upstream. Applying the unpatched upstream files restores 300m/200m/100m and
-exhausts railiance01's CPU requests again. `tests/test_knative_render.py`
+HPA ceilings 20/5/10, and exhausts railiance01's CPU requests again. `tests/test_knative_render.py`
 proves the render offline and fails if these overlays drift from rail-knative's
 declaration. `install.sh` needs a local `kubectl` for `kubectl kustomize`. Kourier is kept `ClusterIP`; public entry through Traefik, DNS, and
 TLS requires separate reef admission evidence.
@@ -24,7 +26,7 @@ containers for fail-closed admission checks such as verifying that egress
 policy has reconciled before application code starts.
 
 Run `install.sh railiance01`, then `verify.sh railiance01`; `verify.sh` is
-read-only and also checks the six CPU requests and the Envoy pin.
+read-only and also checks the six CPU requests, the three HPA ceilings and the Envoy pin.
 
 Before workload admission, rollback deletes Kourier, Serving core, then CRDs
 using the same verified assets. After Knative Services exist, removal requires

install/knative/overlays/kourier/cpu-requests.patch.yaml

@@ -1,8 +1,9 @@
 # CPU requests declared by rail-knative (substrate/v1.22.0/cpu-requests.patch.yaml),
 # the documents of that file that target kourier.yaml. Only CPU requests are set;
-# memory requests and all limits stay upstream. Live on railiance01 since
+# memory requests and all limits stay upstream. First set live on railiance01
 # 2026-09-21 (ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED by
-# the founder). Keep in step with rail-knative: tests/test_knative_render.py
+# the founder); trimmed to 10m each by RAIL-KNATIVE-WP-0003 (2026-10-21).
+# Keep in step with rail-knative: tests/test_knative_render.py
 # fails if the two declarations disagree.
 ---
 # upstream v1.22.0 request: 200m
@@ -18,7 +19,7 @@ spec:
         - name: controller
           resources:
             requests:
-              cpu: 30m
+              cpu: 10m
 ---
 # upstream v1.22.0 request: 200m
 apiVersion: apps/v1
@@ -33,4 +34,4 @@ spec:
         - name: kourier-gateway
           resources:
             requests:
-              cpu: 50m
+              cpu: 10m

install/knative/overlays/kourier/hpa-caps.patch.yaml

@@ -0,0 +1,13 @@
+# HPA ceilings declared by rail-knative (substrate/v1.22.0/hpa-caps.patch.yaml),
+# the documents of that file that target kourier.yaml. Only maxReplicas changes;
+# minReplicas and the CPU target stay upstream (RAIL-KNATIVE-WP-0003). Keep in
+# step with rail-knative: tests/test_knative_render.py fails if they disagree.
+---
+# upstream v1.22.0: minReplicas 1, maxReplicas 10
+apiVersion: autoscaling/v2
+kind: HorizontalPodAutoscaler
+metadata:
+  name: 3scale-kourier-gateway
+  namespace: kourier-system
+spec:
+  maxReplicas: 3

install/knative/overlays/kourier/kustomization.yaml

@@ -7,4 +7,5 @@ resources:
   - upstream.yaml
 patches:
   - path: cpu-requests.patch.yaml
+  - path: hpa-caps.patch.yaml
   - path: service-clusterip.patch.yaml

install/knative/overlays/serving-core/cpu-requests.patch.yaml

@@ -1,8 +1,9 @@
 # CPU requests declared by rail-knative (substrate/v1.22.0/cpu-requests.patch.yaml),
 # the documents of that file that target serving-core.yaml. Only CPU requests are set;
-# memory requests and all limits stay upstream. Live on railiance01 since
+# memory requests and all limits stay upstream. First set live on railiance01
 # 2026-09-21 (ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED by
-# the founder). Keep in step with rail-knative: tests/test_knative_render.py
+# the founder); trimmed to 10m each by RAIL-KNATIVE-WP-0003 (2026-10-21).
+# Keep in step with rail-knative: tests/test_knative_render.py
 # fails if the two declarations disagree.
 ---
 # upstream v1.22.0 request: 300m
@@ -18,7 +19,7 @@ spec:
         - name: activator
           resources:
             requests:
-              cpu: 50m
+              cpu: 10m
 ---
 # upstream v1.22.0 request: 100m
 apiVersion: apps/v1
@@ -33,7 +34,7 @@ spec:
         - name: autoscaler
           resources:
             requests:
-              cpu: 30m
+              cpu: 10m
 ---
 # upstream v1.22.0 request: 100m
 apiVersion: apps/v1
@@ -48,7 +49,7 @@ spec:
         - name: controller
           resources:
             requests:
-              cpu: 30m
+              cpu: 10m
 ---
 # upstream v1.22.0 request: 100m
 apiVersion: apps/v1
@@ -63,4 +64,4 @@ spec:
         - name: webhook
           resources:
             requests:
-              cpu: 30m
+              cpu: 10m

install/knative/overlays/serving-core/hpa-caps.patch.yaml

@@ -0,0 +1,22 @@
+# HPA ceilings declared by rail-knative (substrate/v1.22.0/hpa-caps.patch.yaml),
+# the documents of that file that target serving-core.yaml. Only maxReplicas changes;
+# minReplicas and the CPU target stay upstream (RAIL-KNATIVE-WP-0003). Keep in
+# step with rail-knative: tests/test_knative_render.py fails if they disagree.
+---
+# upstream v1.22.0: minReplicas 1, maxReplicas 20
+apiVersion: autoscaling/v2
+kind: HorizontalPodAutoscaler
+metadata:
+  name: activator
+  namespace: knative-serving
+spec:
+  maxReplicas: 3
+---
+# upstream v1.22.0: minReplicas 1, maxReplicas 5
+apiVersion: autoscaling/v2
+kind: HorizontalPodAutoscaler
+metadata:
+  name: webhook
+  namespace: knative-serving
+spec:
+  maxReplicas: 2

install/knative/overlays/serving-core/kustomization.yaml

@@ -7,3 +7,4 @@ resources:
   - upstream.yaml
 patches:
   - path: cpu-requests.patch.yaml
+  - path: hpa-caps.patch.yaml

install/knative/verify.sh

@@ -18,11 +18,20 @@ while read -r ns deploy container cpu; do
     -o "jsonpath={.spec.template.spec.containers[?(@.name==\"$container\")].resources.requests.cpu}")"
   [ "$got" = "$cpu" ] || { echo "cpu request $ns/$deploy/$container: want $cpu, got ${got:-<none>}" >&2; exit 1; }
 done <<'LIST'
-knative-serving activator activator 50m
-knative-serving autoscaler autoscaler 30m
-knative-serving controller controller 30m
-knative-serving webhook webhook 30m
-knative-serving net-kourier-controller controller 30m
-kourier-system 3scale-kourier-gateway kourier-gateway 50m
+knative-serving activator activator 10m
+knative-serving autoscaler autoscaler 10m
+knative-serving controller controller 10m
+knative-serving webhook webhook 10m
+knative-serving net-kourier-controller controller 10m
+kourier-system 3scale-kourier-gateway kourier-gateway 10m
 LIST
+# Declared HPA ceilings (rail-knative substrate/v1.22.0/hpa-caps.patch.yaml).
+while read -r ns hpa max; do
+  got="$(ssh "$target" kubectl get hpa "$hpa" -n "$ns" -o 'jsonpath={.spec.maxReplicas}')"
+  [ "$got" = "$max" ] || { echo "hpa maxReplicas $ns/$hpa: want $max, got ${got:-<none>}" >&2; exit 1; }
+done <<'HPAS'
+knative-serving activator 3
+knative-serving webhook 2
+kourier-system 3scale-kourier-gateway 3
+HPAS
 echo "knative substrate verified"

tests/test_knative_render.py

@@ -1,8 +1,8 @@
-"""Offline proof that install/knative renders the declared CPU requests.
+"""Offline proof that install/knative renders the declared CPU requests and HPA caps.
 
 Runs install/knative/render.sh (download + SHA-256 verification + kustomize;
 no cluster contact) and compares the result with the upstream assets: the six
-declared CPU requests change, the Kourier Service is ClusterIP, the gateway
+declared CPU requests and HPA maxReplicas change, the Kourier Service is ClusterIP, the gateway
 image is the pinned Envoy digest, and nothing else differs. Skips when the
 upstream assets cannot be fetched or kubectl is missing.
 """
@@ -27,12 +27,17 @@ ROOT = Path(__file__).resolve().parents[1]
 KNATIVE = ROOT / "install" / "knative"
 RAIL_KNATIVE = Path(os.environ.get("RAIL_KNATIVE_DIR", ROOT.parent / "rail-knative"))
 DECLARED = {  # (namespace, deployment, container) -> cpu request
-    ("knative-serving", "activator", "activator"): "50m",
-    ("knative-serving", "autoscaler", "autoscaler"): "30m",
-    ("knative-serving", "controller", "controller"): "30m",
-    ("knative-serving", "webhook", "webhook"): "30m",
-    ("knative-serving", "net-kourier-controller", "controller"): "30m",
-    ("kourier-system", "3scale-kourier-gateway", "kourier-gateway"): "50m",
+    ("knative-serving", "activator", "activator"): "10m",
+    ("knative-serving", "autoscaler", "autoscaler"): "10m",
+    ("knative-serving", "controller", "controller"): "10m",
+    ("knative-serving", "webhook", "webhook"): "10m",
+    ("knative-serving", "net-kourier-controller", "controller"): "10m",
+    ("kourier-system", "3scale-kourier-gateway", "kourier-gateway"): "10m",
+}
+DECLARED_HPA = {  # (namespace, hpa) -> maxReplicas
+    ("knative-serving", "activator"): 3,
+    ("knative-serving", "webhook"): 2,
+    ("kourier-system", "3scale-kourier-gateway"): 3,
 }
 
 
@@ -58,6 +63,10 @@ def patch_requests(path: Path) -> dict[tuple, str]:
     return out
 
 
+def patch_hpa_caps(path: Path) -> dict[tuple, int]:
+    return {(d["metadata"]["namespace"], d["metadata"]["name"]): d["spec"]["maxReplicas"] for d in load(path)}
+
+
 @unittest.skipIf(yaml is None, "PyYAML not installed")
 class DeclarationTests(unittest.TestCase):
     def test_overlays_carry_the_declared_requests(self) -> None:
@@ -66,16 +75,28 @@ class DeclarationTests(unittest.TestCase):
             got.update(patch_requests(KNATIVE / "overlays" / overlay / "cpu-requests.patch.yaml"))
         self.assertEqual(DECLARED, got)
 
+    def test_overlays_carry_the_declared_hpa_caps(self) -> None:
+        got = {}
+        for overlay in ("serving-core", "kourier"):
+            got.update(patch_hpa_caps(KNATIVE / "overlays" / overlay / "hpa-caps.patch.yaml"))
+        self.assertEqual(DECLARED_HPA, got)
+
     def test_verify_checks_the_declared_requests(self) -> None:
         rows = re.search(r"<<'LIST'\n(.*?)\nLIST", (KNATIVE / "verify.sh").read_text(), re.S).group(1)
         got = {tuple(r.split()[:3]): r.split()[3] for r in rows.splitlines()}
         self.assertEqual(DECLARED, got)
 
+    def test_verify_checks_the_declared_hpa_caps(self) -> None:
+        rows = re.search(r"<<'HPAS'\n(.*?)\nHPAS", (KNATIVE / "verify.sh").read_text(), re.S).group(1)
+        got = {tuple(r.split()[:2]): int(r.split()[2]) for r in rows.splitlines()}
+        self.assertEqual(DECLARED_HPA, got)
+
     def test_matches_rail_knative_declaration(self) -> None:
         source = RAIL_KNATIVE / "substrate" / "v1.22.0" / "cpu-requests.patch.yaml"
         if not source.exists():
             self.skipTest(f"rail-knative not checked out at {RAIL_KNATIVE}")
         self.assertEqual(DECLARED, patch_requests(source))
+        self.assertEqual(DECLARED_HPA, patch_hpa_caps(source.with_name("hpa-caps.patch.yaml")))
 
 
 @unittest.skipIf(yaml is None or shutil.which("kubectl") is None, "needs PyYAML and kubectl")
@@ -110,6 +131,9 @@ class RenderTests(unittest.TestCase):
                         seen.add(ident)
                     if c["image"].startswith("docker.io/envoyproxy/envoy:"):
                         c["image"] = envoy
+            if k[1] == "HorizontalPodAutoscaler" and k[2:] in DECLARED_HPA:
+                expected["spec"]["maxReplicas"] = DECLARED_HPA[k[2:]]
+                seen.add(k[2:])
             if k[1:] == ("Service", "kourier-system", "kourier"):
                 expected["spec"]["type"] = "ClusterIP"
             self.assertEqual(expected, out, f"{k} differs beyond the declared changes")
@@ -124,7 +148,7 @@ class RenderTests(unittest.TestCase):
     def test_rendered_carries_declared_requests_and_nothing_else_changes(self) -> None:
         seen = self.check("core.yaml", "serving-core.rendered.yaml")
         seen |= self.check("kourier.yaml", "kourier.rendered.yaml")
-        self.assertEqual(set(DECLARED), seen)
+        self.assertEqual(set(DECLARED) | set(DECLARED_HPA), seen)
 
 
 if __name__ == "__main__":

workplans/RCLUSTER-WP-0020-node-budget-knative-caps-and-wave2-allocation.md

@@ -0,0 +1,92 @@
+---
+id: RCLUSTER-WP-0020
+type: workplan
+title: "Receive the 2026-10-21 node budget: Knative HPA caps/requests and wave-2 allocation"
+domain: financials
+repo: railiance-cluster
+status: active
+owner: claude
+topic_slug: railiance
+created: "2026-10-21"
+updated: "2026-10-21"
+flavor: implementation
+depends_on: []
+related: [CUST-WP-0071, RESOURCE-WP-0008, RAIL-KNATIVE-WP-0003]
+---
+
+# RCLUSTER-WP-0020 - Receive the 2026-10-21 node budget
+
+resource-control's node allocation review of 2026-10-21
+(`resource-control/docs/evidence/2026-10-21-node-allocation-review.md`, per-workload
+table in `.../2026-10-21-node-allocation-review/2026-10-21-node-budget.md`)
+reconciles railiance01 only conditionally: lean decreases accepted, releases,
+builds and Knative scale-out serialized, and Knative HPA ceilings capped.
+This plan receives the railiance-cluster parts.
+
+## T01 - Installer renders rail-knative's RAIL-KNATIVE-WP-0003 declaration
+
+```task
+id: RCLUSTER-WP-0020-T01
+status: done
+priority: high
+```
+
+rail-knative `76cb9c7` declares 10m CPU requests for the six substrate
+containers (was 50/30/30/30/30/50m) and HPA `maxReplicas` activator 20→3,
+webhook 5→2, 3scale-kourier-gateway 10→3. `install/knative/overlays/` now
+carries both (`cpu-requests.patch.yaml`, new `hpa-caps.patch.yaml`, split by
+upstream asset), `verify.sh` checks the six requests and the three ceilings,
+and `tests/test_knative_render.py` asserts the overlays, verify.sh and the
+rail-knative checkout agree and that the render differs from upstream only in
+the declared fields. A read-only render-vs-live comparison showed exactly the
+9 intended fields differing. Not applied.
+
+## T02 - Attended apply and verify on railiance01
+
+```task
+id: RCLUSTER-WP-0020-T02
+status: wait
+priority: high
+needs_human: true
+depends_on: [RCLUSTER-WP-0020-T01]
+blocking_reason: "Coordinator applies wave-1 changes one owner at a time after review (RAIL-KNATIVE-WP-0003-T02)"
+```
+
+`install/knative/install.sh railiance01`, then `install/knative/verify.sh railiance01`.
+
+## T03 - Wave-2 allocation for railiance-cluster workloads (prepare only)
+
+```task
+id: RCLUSTER-WP-0020-T03
+status: todo
+priority: medium
+```
+
+Receiving task for the seven railiance-cluster rows of the node budget.
+Requests only; limits unchanged. Locate the source of each (k3s packaged
+manifests / HelmChartConfig under `install/`, cert-manager install) before
+changing anything; k3s-packaged components need a HelmChartConfig or
+`--disable` + owned manifest so the change survives k3s restarts.
+
+| Workload | CPU req→prop (m) | Mem req→prop (Mi) | p95/peak CPU 7d (m) | Action |
+| --- | ---: | ---: | ---: | --- |
+| cert-manager/cainjector | 0→10 | 0→96 | 1.1/2.8 | increase + memory up |
+| cert-manager/cert-manager | 0→10 | 0→80 | 1.4/3.5 | increase + memory up |
+| cert-manager/webhook | 0→10 | 0→0 | 0.4/0.9 | increase |
+| kube-system/coredns | 100→50 | 70→70 | 7.8/10.3 | decrease (DNS floor 50m) |
+| kube-system/local-path-provisioner | 0→10 | 0→0 | 0.1/0.3 | increase |
+| kube-system/metrics-server | 100→10 | 70→70 | 4.9/5.1 | decrease |
+| kube-system/traefik | 0→10 | 0→80 | 3.8/16.0 | increase + memory up |
+
+Measurement-only items in the same task, no manifest changes yet:
+
+- Produce a 7-day host/control-plane series (non-pod cgroup CPU/memory); the
+  900m host figure is a 70-second kubelet sample. Coordinate with rapp-telemetry
+  (node-exporter is disabled, Prometheus has no non-pod series).
+- `kube-reserved`/`system-reserved` (~900m/2.5Gi) are set **only after** the
+  wave-2 request decreases across owners have landed; applying them first makes
+  the current requests unschedulable.
+
+Done when the seven request changes are declared in this repo with a render or
+diff proof, applied attended, and the host series plus the reserved-resources
+decision are recorded.