Plan staged-promotion onboarding (TEN-WP-0008)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

workplans/TEN-WP-0008-staged-promotion-onboarding.md

@@ -0,0 +1,173 @@
+---
+id: TEN-WP-0008
+type: workplan
+title: "Bring tenant-engine under the staged-promotion contract"
+domain: infotech
+repo: tenant-engine
+status: ready
+owner: claude
+topic_slug: tenant-guardrails
+created: "2026-08-16"
+updated: "2026-08-16"
+depends_on:
+  - TEN-WP-0007
+unblocks: []
+---
+
+# TEN-WP-0008 - Staged-promotion onboarding
+
+`tenant-engine` runs in production but was never brought under the
+RAIL-BS-WP-0006 staged-promotion contract: no `railiance/app.toml`, no stage
+commands, no canary or approval evidence. Rollouts are a hand-edited digest in
+`deploy/tenant-engine.yaml` plus a `kubectl apply`.
+
+## Why now
+
+Two silent pin rollbacks inside one week, discovered by accident rather than by
+any signal:
+
+- **flex-auth** rolled back to a four-action policy — lifecycle mutations
+  denied with `403 write_denied`.
+- **tenant-engine** rolled back to the pre-lifecycle TEN-WP-0004 image —
+  lifecycle routes absent, `404`.
+
+Neither produced an alert. Both looked like ordinary responses rather than
+failures, so nothing watching for crashes or non-2xx would have noticed.
+`user-engine` was the live consumer of both and its contract-level tests passed
+throughout, because they were right about the contract while production was not
+serving it.
+
+TEN-WP-0007 added `make verify-pin`, which compares the repo's digest, the
+Deployment spec, the running pod, and the live routes. That is worth having but
+it is not the answer: it runs when someone remembers to run it, from a
+workstation, against one deployment. **The failure being guarded against is
+precisely the one nobody was watching for.** A local check makes us
+individually lucky; staged promotion makes "the digest that should be live" a
+recorded fact that drift can be measured against.
+
+## Boundary note
+
+This workplan onboards *this repo* to a contract owned by
+`railiance-bootstrap`. It does not modify the contract, the CLI, or the schema,
+and it does not build a drift-detection system — if staged promotion already
+records the intended digest, drift detection is a query against something that
+exists. If the contract needs changing to fit a plain-manifest workload, that
+is a `railiance-bootstrap` decision to request, not one to make here.
+
+## T01 - Write and validate `railiance/app.toml`
+
+```task
+id: TEN-WP-0008-T01
+status: todo
+priority: high
+```
+
+Author `railiance/app.toml` against `railiance.app.v1`
+(`railiance-bootstrap/schemas/railiance-app.schema.json`). Required sections:
+`app`, `source`, `platform`, `secrets`, `observability`, `rollback`, `stages`
+(1–3), `checks`.
+
+Model it on `qonto-assistant/railiance/app.toml` — the same fleet convention
+this repo follows for small headless services — not on a larger workload whose
+shape does not apply.
+
+Decisions this task must make and record:
+
+- **`criticality`.** `tenant-engine` is a data source `flex-auth` consults for
+  authorization decisions and `key-cape` consults at token issuance. It holds
+  no secrets and is not internet-facing, but an outage makes a PDP fail closed
+  fleet-wide. Argue the level rather than defaulting it.
+- **`secrets.references`.** This repo has none — runtime secret custody is
+  OpenBao's per `SCOPE.md`. An empty list is the correct answer, and stating it
+  explicitly is the point.
+- **Health endpoints** for stage1 (local) and stage2 (cluster), using `/health`
+  on port **8090**.
+- **Rollback strategy**, which must record the corrected ladder from
+  TEN-WP-0007: `44ca65f3…` (lifecycle + guardrails) → `08be0b1d…` (lifecycle
+  only) → `2249e8c6…` (neither). Rolling back past `08be0b1d…` removes
+  lifecycle authority `user-engine` depends on.
+
+Validate with `bin/railiance run <repo> --pretty` in plan mode. Save the JSON
+result as non-secret evidence.
+
+Done when the file validates against the schema and stage 1 passes in plan
+mode.
+
+## T02 - Reconcile the deployment shape with the contract
+
+```task
+id: TEN-WP-0008-T02
+status: todo
+priority: high
+```
+
+Expected friction, and the real work of this workplan. The stage 3 promote path
+expects a Helm chart or a kustomization; `tenant-engine` ships a single
+five-document manifest (`Namespace`, `PVC`, `Deployment`, `Service`,
+`NetworkPolicy`) applied with plain `kubectl apply -f`. `qonto-assistant` uses
+kustomize, so it is not a precedent for this shape.
+
+Establish which of these is true and act on it:
+
+1. the contract already supports a plain-manifest workload and the app.toml
+   just needs the right stage commands;
+2. converting to kustomize is cheap and correct;
+3. the contract genuinely does not cover this shape, in which case **ask
+   `railiance-bootstrap`** rather than working around it.
+
+Do not paper over a gap in the contract with a bespoke local step — that
+reproduces the problem this workplan exists to fix.
+
+A further constraint: the Deployment uses `Recreate` because the SQLite PVC is
+`ReadWriteOnce`. A canary that runs alongside the stable pod cannot mount the
+same volume, so `canary_mode` must reflect that a true side-by-side canary is
+impossible here. Say so explicitly rather than declaring a canary that cannot
+run.
+
+Done when the stage commands in `app.toml` are ones that actually work against
+this repo's deployment shape, verified in plan mode.
+
+## T03 - Record current production as the stable baseline
+
+```task
+id: TEN-WP-0008-T03
+status: todo
+priority: medium
+```
+
+Register the live digest `@sha256:44ca65f3…` as the recorded stable release, so
+there is an authoritative answer to "what should be running" that drift can be
+measured against. Until this exists, `make verify-pin` compares the cluster to
+a repo file that a human edits by hand — better than nothing, but still two
+copies of the truth rather than one.
+
+Done when the stable digest is recorded through the contract's own mechanism
+rather than only in `deploy/tenant-engine.yaml`.
+
+## T04 - Hand the pattern back to the fleet
+
+```task
+id: TEN-WP-0008-T04
+status: todo
+priority: low
+```
+
+Tell `flex-auth` what onboarding actually cost, including anything the contract
+did not cover. They filed FLEX-WP-0011 for the same gap and asked whether there
+is a fleet-level answer to drift; a second data point on the same contract is
+more useful to them than our local script was.
+
+If T02 surfaced a genuine contract gap, make sure `railiance-bootstrap` has it
+as a request rather than leaving two overlay repos to work around it
+independently.
+
+Done when the note is sent.
+
+## Out of scope
+
+- **Changing the staged-promotion contract, CLI, or schema.** Request, do not
+  implement.
+- **Building a drift-detection service.** The point of this workplan is that
+  staged promotion should make one unnecessary.
+- **Re-deploying.** Production is healthy on `44ca65f3…`; onboarding describes
+  what is already running, it does not ship anything new.