--- id: TEN-WP-0008 type: workplan title: "Bring tenant-engine under the staged-promotion contract" domain: infotech repo: tenant-engine status: ready owner: claude topic_slug: tenant-guardrails created: "2026-08-16" updated: "2026-08-16" depends_on: - TEN-WP-0007 unblocks: [] state_hub_workstream_id: "e9da1c15-7beb-48d9-97ab-b657fdf9366e" --- # TEN-WP-0008 - Staged-promotion onboarding `tenant-engine` runs in production but was never brought under the RAIL-BS-WP-0006 staged-promotion contract: no `railiance/app.toml`, no stage commands, no canary or approval evidence. Rollouts are a hand-edited digest in `deploy/tenant-engine.yaml` plus a `kubectl apply`. ## Why now Two silent pin rollbacks inside one week, discovered by accident rather than by any signal: - **flex-auth** rolled back to a four-action policy — lifecycle mutations denied with `403 write_denied`. - **tenant-engine** rolled back to the pre-lifecycle TEN-WP-0004 image — lifecycle routes absent, `404`. Neither produced an alert. Both looked like ordinary responses rather than failures, so nothing watching for crashes or non-2xx would have noticed. `user-engine` was the live consumer of both and its contract-level tests passed throughout, because they were right about the contract while production was not serving it. TEN-WP-0007 added `make verify-pin`, which compares the repo's digest, the Deployment spec, the running pod, and the live routes. That is worth having but it is not the answer: it runs when someone remembers to run it, from a workstation, against one deployment. **The failure being guarded against is precisely the one nobody was watching for.** A local check makes us individually lucky; staged promotion makes "the digest that should be live" a recorded fact that drift can be measured against. ## Boundary note This workplan onboards *this repo* to a contract owned by `railiance-bootstrap`. It does not modify the contract, the CLI, or the schema, and it does not build a drift-detection system — if staged promotion already records the intended digest, drift detection is a query against something that exists. If the contract needs changing to fit a plain-manifest workload, that is a `railiance-bootstrap` decision to request, not one to make here. ## T01 - Write and validate `railiance/app.toml` ```task id: TEN-WP-0008-T01 status: todo priority: high state_hub_task_id: "0231e4d1-2e1a-4e77-92ba-2cfaf23692ab" ``` Author `railiance/app.toml` against `railiance.app.v1` (`railiance-bootstrap/schemas/railiance-app.schema.json`). Required sections: `app`, `source`, `platform`, `secrets`, `observability`, `rollback`, `stages` (1–3), `checks`. Model it on `qonto-assistant/railiance/app.toml` — the same fleet convention this repo follows for small headless services — not on a larger workload whose shape does not apply. Decisions this task must make and record: - **`criticality`.** `tenant-engine` is a data source `flex-auth` consults for authorization decisions and `key-cape` consults at token issuance. It holds no secrets and is not internet-facing, but an outage makes a PDP fail closed fleet-wide. Argue the level rather than defaulting it. - **`secrets.references`.** This repo has none — runtime secret custody is OpenBao's per `SCOPE.md`. An empty list is the correct answer, and stating it explicitly is the point. - **Health endpoints** for stage1 (local) and stage2 (cluster), using `/health` on port **8090**. - **Rollback strategy**, which must record the corrected ladder from TEN-WP-0007: `44ca65f3…` (lifecycle + guardrails) → `08be0b1d…` (lifecycle only) → `2249e8c6…` (neither). Rolling back past `08be0b1d…` removes lifecycle authority `user-engine` depends on. Validate with `bin/railiance run --pretty` in plan mode. Save the JSON result as non-secret evidence. Done when the file validates against the schema and stage 1 passes in plan mode. ## T02 - Reconcile the deployment shape with the contract ```task id: TEN-WP-0008-T02 status: todo priority: high state_hub_task_id: "5f2ad011-1de8-45df-8e8f-dd6bcf51ece4" ``` Expected friction, and the real work of this workplan. The stage 3 promote path expects a Helm chart or a kustomization; `tenant-engine` ships a single five-document manifest (`Namespace`, `PVC`, `Deployment`, `Service`, `NetworkPolicy`) applied with plain `kubectl apply -f`. `qonto-assistant` uses kustomize, so it is not a precedent for this shape. Establish which of these is true and act on it: 1. the contract already supports a plain-manifest workload and the app.toml just needs the right stage commands; 2. converting to kustomize is cheap and correct; 3. the contract genuinely does not cover this shape, in which case **ask `railiance-bootstrap`** rather than working around it. Do not paper over a gap in the contract with a bespoke local step — that reproduces the problem this workplan exists to fix. A further constraint: the Deployment uses `Recreate` because the SQLite PVC is `ReadWriteOnce`. A canary that runs alongside the stable pod cannot mount the same volume, so `canary_mode` must reflect that a true side-by-side canary is impossible here. Say so explicitly rather than declaring a canary that cannot run. Done when the stage commands in `app.toml` are ones that actually work against this repo's deployment shape, verified in plan mode. ## T03 - Record current production as the stable baseline ```task id: TEN-WP-0008-T03 status: todo priority: medium state_hub_task_id: "c4b79dc1-c280-428e-9dc5-b17c05b084c6" ``` Register the live digest `@sha256:44ca65f3…` as the recorded stable release, so there is an authoritative answer to "what should be running" that drift can be measured against. Until this exists, `make verify-pin` compares the cluster to a repo file that a human edits by hand — better than nothing, but still two copies of the truth rather than one. Done when the stable digest is recorded through the contract's own mechanism rather than only in `deploy/tenant-engine.yaml`. ## T04 - Hand the pattern back to the fleet ```task id: TEN-WP-0008-T04 status: todo priority: low state_hub_task_id: "b816a862-51e0-4abd-a2f3-ee40359526cc" ``` Tell `flex-auth` what onboarding actually cost, including anything the contract did not cover. They filed FLEX-WP-0011 for the same gap and asked whether there is a fleet-level answer to drift; a second data point on the same contract is more useful to them than our local script was. If T02 surfaced a genuine contract gap, make sure `railiance-bootstrap` has it as a request rather than leaving two overlay repos to work around it independently. Done when the note is sent. ## Out of scope - **Changing the staged-promotion contract, CLI, or schema.** Request, do not implement. - **Building a drift-detection service.** The point of this workplan is that staged promotion should make one unnecessary. - **Re-deploying.** Production is healthy on `44ca65f3…`; onboarding describes what is already running, it does not ship anything new.