NK-WP-0026 finished: user-engine caller identity verified live against railiance01

T02 done. Deployed user-engine digest c501aeb2 reads the projected flex-auth
token per decision (verified in the running container); flex-auth-user-engine
138aa347 serves with caller-auth enforce. Probes: valid 200
decision:d9aef25f08e17b84, missing token 401, wrong-system 403.

Closes manifest drift: runtime.yaml pinned e3b5f65b, the digest T01 warned
against, while the cluster ran c501aeb2. Re-applying it would have rolled the
portal back to an image that cannot authenticate to a PDP now in enforce.
kubectl diff is now empty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-08-19 22:01:12 +02:00
parent dae0dc681a
commit 4a915ce6c7
5 changed files with 133 additions and 8 deletions

View file

@ -18,6 +18,7 @@
| workplan | NK-WP-0023 | finished | — | workplans/NK-WP-0023-user-engine-portal-platform-integration.md |
| workplan | NK-WP-0024 | finished | — | workplans/NK-WP-0024-user-engine-portal-integration-expansion.md |
| workplan | NK-WP-0025 | finished | — | workplans/NK-WP-0025-public-self-registration-and-application-jit.md |
| workplan | NK-WP-0026 | finished | — | workplans/NK-WP-0026-flex-auth-caller-identity-rollout.md |
| task | ADHOC-2026-07-02-T01 | done | — | workplans/ADHOC-2026-07-02.md |
| task | ADHOC-2026-07-02-T02 | done | — | workplans/ADHOC-2026-07-02.md |
| task | ADHOC-2026-08-14-T01 | done | — | workplans/ADHOC-2026-08-14.md |
@ -72,5 +73,7 @@
| task | NK-WP-0025-T03 | done | — | workplans/NK-WP-0025-public-self-registration-and-application-jit.md |
| task | NK-WP-0025-T04 | done | — | workplans/NK-WP-0025-public-self-registration-and-application-jit.md |
| task | NK-WP-0025-T05 | done | — | workplans/NK-WP-0025-public-self-registration-and-application-jit.md |
| task | NK-WP-0026-T01 | done | — | workplans/NK-WP-0026-flex-auth-caller-identity-rollout.md |
| task | NK-WP-0026-T02 | done | — | workplans/NK-WP-0026-flex-auth-caller-identity-rollout.md |
| intake | NK-IN-0001 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |
| intake | NK-IN-0002 | closed | blue | docs/intakes/activity-core-ops-sso-operators.md |

View file

@ -114,9 +114,11 @@ Rules:
Performance model:
- the live-lookup API must be fast and highly available, since `flex-auth`
calls it synchronously on the request path for privileged actions; slow
or unavailable `tenant-engine` lookups fail closed (deny), never fail open;
- the live-lookup API must be fast and highly available, since `flex-auth` is
required to call it synchronously on the request path for privileged actions;
slow or unavailable `tenant-engine` lookups fail closed (deny), never fail
open. The adapter existing without a production caller does not satisfy this
contract and does not permit either service to claim I3;
- the cached-claim read API used by `key-cape` may be slower, since it only
runs at token issuance, not per-request;
- request-scoped memoization within a single `flex-auth` decision is

View file

@ -40,12 +40,13 @@ spec:
labels: *labels
spec:
automountServiceAccountToken: false
serviceAccountName: user-engine
securityContext:
runAsNonRoot: true
seccompProfile: {type: RuntimeDefault}
containers:
- name: portal
image: forgejo.coulomb.social/coulomb/user-engine@sha256:e3b5f65bafc1c0260dfdf2567a52766e67506ceb878a51759a2e9a307c4b5eb8
image: forgejo.coulomb.social/coulomb/user-engine@sha256:c501aeb204154d85017565a87fd34cff12e5c48f600d124d3f8c82f11bb9d59c
imagePullPolicy: IfNotPresent
ports: [{name: http, containerPort: 8080}]
env:
@ -62,6 +63,7 @@ spec:
- {name: USER_ENGINE_OIDC_REDIRECT_URI, value: "https://users.92-205-62-239.nip.io/oidc/callback"}
- {name: USER_ENGINE_OIDC_BACKEND_URL, value: "http://keycape.sso.svc.cluster.local:8080"}
- {name: USER_ENGINE_FLEX_AUTH_URL, value: "http://flex-auth-user-engine.flex-auth.svc.cluster.local:8080"}
- {name: USER_ENGINE_FLEX_AUTH_TOKEN_FILE, value: "/var/run/secrets/flex-auth-caller/token"}
- {name: USER_ENGINE_EVENT_URL, value: "http://audit-core.audit-core.svc.cluster.local:8080/v1/events"}
- {name: USER_ENGINE_MAIL_URL, value: "http://email-connect.email-connect.svc.cluster.local:8080/v1/send"}
- name: USER_ENGINE_EVENT_TOKEN
@ -90,6 +92,10 @@ spec:
allowPrivilegeEscalation: false
capabilities: {drop: ["ALL"]}
readOnlyRootFilesystem: true
volumeMounts:
- name: flex-auth-caller
mountPath: /var/run/secrets/flex-auth-caller
readOnly: true
resources:
requests: {cpu: 50m, memory: 64Mi}
limits: {cpu: 500m, memory: 256Mi}
@ -103,6 +109,15 @@ spec:
livenessProbe:
httpGet: {path: /healthz, port: http}
periodSeconds: 20
volumes:
- name: flex-auth-caller
projected:
defaultMode: 0440
sources:
- serviceAccountToken:
audience: flex-auth
expirationSeconds: 3600
path: token
---
apiVersion: v1
kind: Service
@ -111,6 +126,11 @@ spec:
selector: {app.kubernetes.io/name: user-engine}
ports: [{name: http, port: 8080, targetPort: http}]
---
apiVersion: v1
kind: ServiceAccount
metadata: {name: user-engine, namespace: user-engine}
automountServiceAccountToken: false
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: user-engine-default-deny, namespace: user-engine}

View file

@ -19,11 +19,14 @@ class UserEnginePlatformExpansionContractTests(unittest.TestCase):
self.assertIn("dead-letter", text)
self.assertIn("OpenBao", text)
def test_live_manifest_does_not_enable_partial_integration(self):
def test_desired_manifest_authenticates_flex_auth_calls(self):
manifest = (ROOT / "sso-mfa/k8s/user-engine/runtime.yaml").read_text()
self.assertNotIn("USER_ENGINE_FLEX_AUTH_URL", manifest)
self.assertNotIn("USER_ENGINE_EVENT_URL", manifest)
self.assertNotIn("USER_ENGINE_MAIL_URL", manifest)
self.assertIn("USER_ENGINE_FLEX_AUTH_URL", manifest)
self.assertIn("USER_ENGINE_FLEX_AUTH_TOKEN_FILE", manifest)
self.assertIn("serviceAccountName: user-engine", manifest)
self.assertIn("audience: flex-auth", manifest)
self.assertIn("mountPath: /var/run/secrets/flex-auth-caller", manifest)
self.assertIn("automountServiceAccountToken: false", manifest)
if __name__ == "__main__":

View file

@ -0,0 +1,97 @@
---
id: NK-WP-0026
type: workplan
title: "Promote user-engine workload identity to flex-auth"
domain: infotech
repo: net-kingdom
status: finished
owner: codex
topic_slug: netkingdom
created: "2026-08-18"
updated: "2026-08-19"
state_hub_workstream_id: "3a9b7b04-d283-4101-8b6a-31624c6d5de6"
---
# NK-WP-0026 — user-engine caller identity rollout
```task
id: NK-WP-0026-T01
status: done
priority: high
state_hub_task_id: "8da592be-cbc7-49a4-bb2e-1c04e5c80336"
```
Declare a non-automounted `user-engine` ServiceAccount and a one-hour projected
token with audience `flex-auth`; mount only that token into the portal. Source
completed 2026-08-18. Existing NetworkPolicy already permits only the named
flex-auth user-engine instance on port 8080.
```task
id: NK-WP-0026-T02
status: done
priority: high
state_hub_task_id: "9dbaf763-31fc-4538-93a2-5df4bb9fa6ae"
```
Both sides promoted and probed. Verified live on railiance01, 2026-08-19, by
net-kingdom against the cluster rather than against another repo's assertion.
**Deployed digests**
| Side | Object | Digest |
| --- | --- | --- |
| user-engine | `deploy/user-engine` (ns `user-engine`), pod `user-engine-8569d7cb87-fp7px` | `forgejo.coulomb.social/coulomb/user-engine@sha256:c501aeb204154d85017565a87fd34cff12e5c48f600d124d3f8c82f11bb9d59c` |
| flex-auth | `deploy/flex-auth-user-engine` (ns `flex-auth`), pod `flex-auth-user-engine-b745b6b5c-zhsfv` | `forgejo.coulomb.social/coulomb/flex-auth@sha256:138aa3471c46bca6e814691fa1e6520aedda3dffd743e6b09141ab433afdb64b` |
flex-auth serves with `--caller-auth-mode enforce` and
`--caller-binding user-engine=system:serviceaccount:user-engine:user-engine`.
The warn stage was run on flex-auth's side under FLEX-WP-0011 / FLEX-WP-0015-T02;
net-kingdom observed the end state, not the intermediate warn window.
**The T01 caveat is cleared.** The old digest did not read the projected file.
The deployed digest does: `user_engine/runtime.py:45` makes
`USER_ENGINE_FLEX_AUTH_TOKEN_FILE` a *required* setting (`_required(...)`, so the
portal cannot start without it), and
`user_engine/adapters/flex_auth.py:62-68` re-reads the file **per decision**
deliberately, so projected-token rotation needs no restart — and sends it as
`Authorization: Bearer`. Both were read out of the running container, not out of
the source tree.
**Probes** — `POST /v1/check` against
`flex-auth-user-engine.flex-auth.svc.cluster.local:8080`, run from inside the
workload pods so NetworkPolicy and identity are exercised as in production. All
are read-only decision queries; nothing in the cluster was mutated.
| Probe | Run from | Credential | Result |
| --- | --- | --- | --- |
| valid | `user-engine-8569d7cb87-fp7px` | the pod's own projected token, audience `flex-auth` | **HTTP 200**, `decision:d9aef25f08e17b84`, `effect: allow`, `reason: self_service`, `matched_rule: self_service`, `policy_version: v1` |
| missing token | `user-engine-8569d7cb87-fp7px` | no `Authorization` header | **HTTP 401** `unauthenticated` |
| wrong system | `user-engine-8569d7cb87-fp7px` | valid user-engine token, body asserting `resource.system: tenant-engine` | **HTTP 403** `forbidden` |
| wrong system (network path) | `tenant-engine-6c6dcbddbb-4v26w` | tenant-engine's own projected token, against the *user-engine* PDP | connection refused — NetworkPolicy denies the cross-service path before flex-auth sees it |
Note on the wrong-system probe, because it differs from what flex-auth reported.
flex-auth's `tenancy.yaml` states "a token bound to the other protected system
returns 403". net-kingdom could not reproduce that literal case: the
tenant-engine pod cannot reach the user-engine PDP at all (Errno 111), so the
403 never arises on that path. What *is* reproducible, and is the same
substitution attack, is a caller authenticated as one system asserting another
system in the request body — ADR-0004's stated threat, "a caller could represent
another protected system" — and that returns 403. The estate position is
therefore stronger than claimed (two independent layers refuse it), but the
claimed evidence is not the evidence that exists. Recorded rather than smoothed.
**Manifest drift, found and closed.** `sso-mfa/k8s/user-engine/runtime.yaml`
still pinned `sha256:e3b5f65b…` — the digest T01 warned must not be applied —
while the cluster ran `c501aeb2…`. Anyone re-applying the repo's own manifest
would have rolled the portal back to an image that does not read the token file,
against a PDP now in `enforce`: every authorization call would 401 and the portal
would fail closed. The manifest is now pinned to the deployed digest and
`kubectl diff -f sso-mfa/k8s/user-engine/runtime.yaml` is **empty, exit 0**
declared state and live state agree.
**Observation routed to flex-auth, not a blocker.** A syntactically invalid
bearer token returns **HTTP 503 `caller authentication unavailable`**, not 401.
ADR-0004 reserves 503 for TokenReview *unavailability* and puts invalid tokens
with the 401 class. It fails closed either way, so there is no bypass, but a
malformed credential is being reported as a flex-auth outage — which will
mislead whoever reads the first page of an incident.