--- id: FLEX-WP-0015 type: workplan title: "Tenancy posture declaration and inbound caller authentication" domain: infotech repo: flex-auth status: finished owner: claude topic_slug: netkingdom planning_priority: P1 planning_order: 150 depends_on_workplans: [] related_workplans: - FLEX-WP-0004 - FLEX-WP-0011 created: "2026-08-17" updated: "2026-08-19" state_hub_workstream_id: "31846b19-c2a3-428e-950b-5985bc9146eb" --- # FLEX-WP-0015 - Tenancy posture declaration and inbound caller authentication Answer `rapp-postgres`'s review request on **NetKingdom Tenancy Posture v0.1** (message `82ec43ce`), publish flex-auth's posture vector, and close the one real defect the review surfaced. **Review output:** `docs/tenancy-posture-review.md` **Declaration:** `tenancy.yaml` (repo root, per framework §5.1) ## Why this workplan exists The original five axes described flex-auth unevenly; draft-8's sixth availability axis and canonical provider declaration now cover the remaining shape. Assessing against them turned up two things we did not know we were carrying: 1. **`/v1/check` authenticates no caller.** Any workload with network reach to the ClusterIP Service can assert any subject and any tenant and receive an authoritative allow. flex-auth is the estate's authorization oracle and trusts its callers completely. Framework level `A0`. 2. **The tenant-engine live-roles adapter is built and unwired.** No non-test caller exists. The IAM Profile live re-query reads as a capability we have and is not one. The framework itself needs six amendments, two of them corrections of statements it makes about flex-auth. Those are in the review document and go back to `rapp-postgres` as review output, not as work here. **Scope discipline:** this workplan closes flex-auth's own gaps and answers the review. It does not build fleet-level conformance tooling, does not adopt AuthZEN, and does not touch production enforcement (`policy.enabled` stays off per FLEX-WP-0007). ## Tasks ```task id: FLEX-WP-0015-T01 status: done priority: high state_hub_task_id: "0a32500d-511a-4bd7-972e-de4cd3e62dd9" ``` **Publish posture and answer the review.** Write `tenancy.yaml` and `docs/tenancy-posture-review.md`; reply to `rapp-postgres` with the posture vector, the A4 ruling, the internal-hop ruling, and the two factual corrections. Done 2026-08-17. ```task id: FLEX-WP-0015-T02 status: done priority: high state_hub_task_id: "b2e87a81-b63d-4be4-ad44-01426b7e6f74" ``` **Close the A0 — authenticate callers of `/v1/check`.** Decide first, build second: this is a production-facing security boundary on a service two consumers already depend on, and the wrong instrument is worse than the delay. **Waiting on `ops-warden`** (asked 2026-08-17): it owns how workloads prove identity in this estate, and it is a flex-auth PEP, so it implements the calling side of whatever we choose. `warden route find` was run first per the credential-routing rules — the catalog holds credential lanes and has no entry for service-to-service caller authentication as a pattern, which is part of what was asked. Our stated instinct is ServiceAccount TokenReview; the question is open pending their answer. Nothing is built until it lands. Record an ADR covering at minimum: - What identifies a caller — Kubernetes ServiceAccount token review, mTLS via the cluster's own CA, or a shared secret header. Note that the Deployment currently sets `automountServiceAccountToken: false`, so option one changes the pod spec. - Whether caller identity only authenticates, or also constrains: may `user-engine` ask questions about resources owned by `tenant-engine`? A caller allowlist per `system` is the natural shape and flex-auth's registry already keys on `system`. - Fail-closed vs fail-open during rollout. Given `policy.enabled` is off in production and both consumers are ours, a warn-only phase that logs unauthenticated callers before rejecting them is available and probably right. - Whether NetworkPolicy is a sufficient interim control to declare `A1` honestly before the real fix lands. Target level `A2` — a single inbound choke point binding caller identity — with the §13 evidence artifact: a test that an unbound request is refused. Ship through the FLEX-WP-0011 staged-promotion path, not by direct apply. 2026-08-18 source outcome: ADR 0004 selects audience-scoped Kubernetes ServiceAccount TokenReview with a separately projected reviewer identity and exact `resource.system` → ServiceAccount bindings. Both check endpoints share the choke point; missing/mismatched identity fails 401/403 and reviewer outage fails 503. Unit and handler tests include the required unbound request. Source committed as `1e1e077`. 2026-08-19 overlay outcome: the sanctioned Helm chart did not render caller auth at all, so a FLEX-WP-0011 pin of the new digest would have stayed `disabled`. That is now wired. Desired emergency manifests and production values select **`warn`**, not `enforce` — applying the previous enforce manifests was the hazard. CI image `sha256:138aa3471c46bca6e814691fa1e6520aedda3dffd743e6b09141ab433afdb64b` (`main-3de72fe`, built 2026-08-18T13:25Z) contains the TokenReview flags. The running cluster digest is still unchanged, so `tenancy.current.A` remains 0 while `tenancy.implemented.A` is 2. **Better path than a global enforce flip.** The two production Deployments are independently rollable (FLEX-WP-0011). USER-WP-0023-T03 only probes `flex-auth-user-engine`. Serializing that probe on tenant-engine's caller status would 401 tenant-engine writes if we enforced them together, and would delay user-engine A2 evidence for no safety gain. Sequence: 1. Pin the caller-auth digest in overlay + emergency manifests — **done 2026-08-19**, first mode `warn` on both consumers. 2. Operator: promote each pin in warn via `helm upgrade --install flex-auth- charts/flex-auth --namespace flex-auth -f values/.yaml`. Isolated canary may boot the same digest with caller-auth disabled. Do not `kubectl apply` an enforce manifest. **user-engine done 2026-08-19** (Helm release `flex-auth-user-engine` rev 4). Live objects were kubectl-managed; they were labelled for Helm adoption first. Two production bugs had to be fixed during warn: the reviewer volume needed `fsGroup: 65532`, and the reviewer token must use the API server's default audiences (a custom `https://kubernetes.default.svc` audience 401s TokenReview on this k3s). tenant-engine is still the old digest. 3. Confirm warn logs per consumer. `user-engine` is migrated and deployed as of 2026-08-18 (image `sha256:c501aeb2…`, token at `/var/run/secrets/flex-auth-caller/token`). On 2026-08-19 they confirmed USER-WP-0023-T03 stays wait and they will not run the three assertions against warn (message `f963906d`). `tenant-engine` source and desired manifests already project that token and send it; live digest is still unconfirmed. Warn is safe either way. `ops-warden` adopts the calling side on its own schedule. 4. Flip **user-engine only** to `enforce` (`callerAuth.mode: enforce` in `values/user-engine.yaml`, same digest). Then they capture the live negative probe from a user-engine pod. The three assertions are already written in user-engine `docs/flex-auth-caller-identity.md`; they must not run against warn, because "no token returns 401" would still be a decision. **Done 2026-08-19** from pod `user-engine-8569d7cb87-fp7px`: (1) Bearer projected token → 200 `decision:350b46fb42989606`; (2) no Authorization → **401** `unauthenticated`; (3) `resource.system=tenant-engine` → **403** `forbidden`. Digests: flex-auth `sha256:138aa347…`, user-engine `sha256:c501aeb2…`. Live user-engine also needed `fsGroup: 10001` or the projected token was `0440 root:root` and uid 10001 could not send it — persist that in rapp-user-engine `manifests/runtime.yaml`. 5. Flip tenant-engine to enforce only after its warn logs are clean. **Done 2026-08-19.** Live tenant-engine image `sha256:44ca65f3…` did not send Authorization; source had the client uncommitted. Committed as tenant-engine `2063470`, CI image `sha256:7dc9d7bf…`, then patched the live Deployment with ServiceAccount `tenant-engine`, projected token, `TENANT_ENGINE_FLEX_AUTH_TOKEN_FILE`, and `runAsUser/runAsGroup/fsGroup 10001`. flex-auth-tenant-engine Helm rev 2 is enforce on `sha256:138aa347…`. Probe from `tenant-engine-6c6dcbddbb-4v26w`: (1) Bearer → 200 `decision:ef18e8bbd2dead6a`; (2) no Authorization → **401** `unauthenticated`; (3) `resource.system=user-engine` → **403** `forbidden`. 6. Only then may `policy.enabled` flip anywhere — `ops-warden` names that the real deadline, and it is the same gate as FLEX-WP-0007. Not done here; FLEX-WP-0007 remains that gate. **Operator note.** railiance01 kubeconfig is `~/.kube/config-railiance01`. Both production pins are Helm-managed (`flex-auth-user-engine` rev 4, `flex-auth-tenant-engine` rev 2). `RISK-F-0001` (risk-nexus) tracks this A0 externally. Their NetworkPolicy question was answered 2026-08-18: both Deployments carry an ingress policy restricted to one namespace *and* one pod label, predating this work — so the reachable set was never "any pod in the cluster", and a sentence of mine that said so was corrected to them. ```task id: FLEX-WP-0015-T03 status: done priority: medium state_hub_task_id: "64eb7652-3b67-4bfb-879b-8588deeec8b5" ``` **Wire or delete `internal/adapters/tenantengine`.** The adapter is complete and has no non-test caller; `decision.Engine` has no field for it. Both outcomes are conformant, silence is not. Deciding factor is whether any live policy needs `tenant_roles` for a privileged, destructive, credential-vending or `aal2`-class decision — the IAM Profile's own trigger list. If yes, wire it into the decision path with a timeout and an explicit unavailable-path decision (`AttachToContext` already carries `tenant_roles_available: false` for exactly this), and note the new synchronous dependency on tenant-engine's availability. If no, delete it and say so in the review record rather than leaving it as latent capability. Update `tenancy.yaml` `I` either way. Completed 2026-08-18 by deletion. Repository-wide search found no non-test caller and no policy consuming `tenant_roles`; wiring it would add a synchronous dependency without a decision that uses the result. The adapter and its isolated tests were removed, and the review records that any future live-role policy must introduce the dependency explicitly. ```task id: FLEX-WP-0015-T04 status: cancel priority: low state_hub_task_id: "06432560-c28c-4de2-a672-87e74be54a6a" ``` **AuthZEN evaluation endpoint (framework `A4`).** Deliberately deferred, not backlogged. `POST /access/v1/evaluation` translating onto the existing engine beside `/v1/check`, non-breaking. **Trigger to start:** either a PEP arrives that flex-auth's operator does not control, or a delegated backend under FLEX-WP-0004 needs uniform addressing. Until one of those, A4 buys swappability nobody can use. Cancelled 2026-08-19 so this workplan can finish. The workplan's own scope says it does not adopt AuthZEN. The trigger above still holds: file a new workplan when a PEP we do not control, or a FLEX-WP-0004 backend, needs `POST /access/v1/evaluation`. Note when starting: AuthZEN standardises the envelope, not the action vocabulary — it does not end action-string copying between repos. The cheaper fix for that is serving `registry.Actions` read-only, which is independent of this task and may be worth doing first. ```task id: FLEX-WP-0015-T05 status: done priority: medium state_hub_task_id: "9195ba20-ab0f-4d75-b293-86978073beb1" ``` **Guard the declaration.** Framework §12 requires verifying the declared posture still holds — against our own declaration, not a universal maximum. Small mechanical check, `make verify-posture`, in the shape tenant-engine used for `verify-pin` (bash, no dependency tree). Assert what code can actually prove: that the `/v1/check` handler has caller authentication iff `tenancy.yaml` claims `A >= 2`; that `tenantengine` has a non-test caller iff `I >= 3`; that the production Deployment carries no `--log` and no volume iff `R` is `n/a`. Exit 1 on drift. Designed for invisible failure per §12: posture regression returns correct-looking responses, exactly like the two silent pin rollbacks the estate has already been bitten by. Completed 2026-08-18. `make verify-posture` checks the current/implemented distinction, both authenticated handlers, warn-or-enforce desired manifests and overlay pins, TokenReview RBAC, absence of a latent tenant-engine caller, and the stateless deployment assumptions behind `R: n/a`. First production pin is warn; enforce is the end state, flipped per consumer.