--- id: WARDEN-WP-0027 type: workplan title: "Tamper-resistant credential governance + mass rotation/lockdown (Strand B)" domain: infotech repo: ops-warden status: active owner: codex topic_slug: custodian planning_priority: medium planning_order: 27 created: "2026-07-16" updated: "2026-08-22" state_hub_workstream_id: "7d697c52-766a-4562-b2ad-a722880bcdcb" --- # Tamper-resistant credential governance + mass rotation/lockdown (Strand B) ## Origin Explicitly deferred from `WARDEN-WP-0026` (credential disclosure hygiene, Strand A). WP-0026 "Out of scope" carves out the heavyweight governance machinery and directs that it be "capture[d] separately if/when justified." This workplan is that capture. Strand A makes accidental disclosure structurally hard and gives every lane **advisory** rotation guidance. Strand B is the **executable, tamper-resistant** layer: turning that advisory guidance into one-command action and hardening policy governance against silent drift or malicious change. ## Status: active, narrowly on T02 Activated by the operator on 2026-08-22 after the second activation gate became concrete. `RAILIANCE-WP-0024-T03` now requires an approved recovery window, fresh encrypted Raft snapshot evidence, provider-console access, named abort authority, and the live **2-of-3 Shamir quorum** before a coordinated reboot. That is fleet policy requiring a designed break-glass path, not speculative heavyweight machinery. Activation is deliberately narrow. No mass-disclosure incident triggers T01, and no fleet policy currently mandates T03's signed policy-manifest reconcile. Those tasks remain `cancel`; cancellation here continues to mean deferred, not abandoned. T02 alone is `progress`. ## Activation gate (promote to `ready` only when ≥1 holds) - A real disclosure incident requires rotating **more than a couple** of secrets at once, making manual per-lane rotation (Strand A) too slow. - Fleet policy for OpenBao mandates tamper-evident policy governance or a designed break-glass path. - Audit/compliance requires provable "reconcile policy to a known-good commit." ## Goal Make credential response **executable and trustworthy at fleet scale**: one command rotates or re-establishes a set of tainted secrets; policy changes are tamper-evident and reconcilable to a known-good state; a graded lockdown / break-glass path exists with an explicit trust-root. ## Scope boundary (unchanged from Strand A) ops-warden still **custodies no secret values**. Strand B orchestrates the owner's tools (OpenBao, provider re-mint, railiance-platform credential broker) as the caller — it does not hold or vend secrets. Rotation execution runs the owner-native path; ops-warden sequences and verifies it. ## Task: Executable mass rotation driver ```task id: WARDEN-WP-0027-T01 status: cancel priority: high state_hub_task_id: "604aad14-d398-4c02-85e5-7ff37a905a1b" ``` Turn Strand A's per-lane `rotation:` guidance (WP-0026 T06) into an executable driver: `warden rotate ` (single lane) and `warden rotate --tainted` (all lanes marked EXPOSED per WP-0026 T05). Each step runs the owner-native command (provider re-mint, `bao kv put`, re-encrypt+reupload for re-establish lanes) as the caller, verifies via capabilities (never reads the value), and clears the `exposed_at` taint on success. Dry-run first; idempotent; per-lane failure isolates. **Done when:** a tainted set of lanes can be rotated/re-established with one command, each verified capabilities-safe, taint cleared only on success. **Depends on:** WP-0026 T05 (taint convention), T06 (rotation registry). ## Task: Graded lockdown / break-glass with explicit trust-root ```task id: WARDEN-WP-0027-T02 status: progress priority: medium state_hub_task_id: "9d004d8f-6215-4178-bd13-5785d24fd152" ``` Design and document a graded lockdown: (a) soft — deny agent roles read on all high-risk lanes; (b) hard — seal + re-key with a pre-designed trust-root and quorum. Define the trust-root (who holds unseal shares, recovery keys), the break-glass invocation, and the re-entry path. **Design + runbook first**; any executable seal step is opt-in and attended. **Done when:** a documented, rehearsed break-glass path exists with a named trust-root and quorum, and soft-lockdown is executable via capabilities-based policy toggles. **Parked input — the warden-sign AppRole (2026-08-11).** secrets-engine holds a validated dry-run for a `warden-sign` AppRole (policy + role, `update` on `ssh/sign/{agt,adm,atm}-role` only). It was **parked** rather than withdrawn specifically because it is an input to this task: today the credential broker is the only *autonomous* path to a signing token, so if its issuer lapses, recovery is a founder OIDC act. Whether a second autonomous path should exist — and whether it is the AppRole or something this trust-root design supersedes — is a break-glass question, not a routing one. Resolve it here. Context and the counter-argument (AppRole is narrower in capability than placing the broker's issuer token on a remote host) are in `workplans/ADHOC-2026-08-11.md` T03 (ad-hoc **finished** 2026-08-15; question lives here now); secrets-engine was told to stop holding apply readiness until this task or the ops-bridge cutover fires (msg `863c7b57`). **Rebaselined 2026-08-22.** The July description predates the authoritative OpenBao migration. The current barrier is a rotated **2-of-3**, not the aspirational 3-of-5 value still present in ops-warden's legacy posture descriptor. `RMASTER-WP-0020` records two attended restart/unseal cycles on 2026-08-03 with shares supplied through hidden prompts, matching inventory, audit continuity, exact-path read, and sibling denial. That is valid rehearsal evidence for re-entry, but it is not misreported as a current production emergency-seal drill. The soft half is also already live: `RAILIANCE-WP-0022` reports total concrete high-risk path coverage and a dedicated coding-agent AppRole proving deny-wins without reading a secret value. ops-warden consumes that owner control through `scripts/check_agent_read_boundary.py`; it does not take ownership of the live OpenBao policy. The consumer contract and remaining acceptance gate are now documented in `docs/credential-governance-break-glass.md`. One attended production emergency seal/unseal drill remains. It must use railiance-platform's evidence template and validator, have snapshot/quorum/console/driver/abort receipts before the seal hold point, and be executed by the platform owner—not by a coding agent. **Parked AppRole disposition:** do not unpark the standalone `warden-sign` AppRole for T02 break-glass. The normal broker remains autonomous; an issuer failure deliberately crosses into attended OIDC and, below that, the 2-of-3 platform trust-root. A standing AppRole placed outside the broker would weaken that boundary and still would not recover a sealed OpenBao. A separately approved ops-bridge unattended-signing design may re-evaluate the narrow AppRole under its own workplan; that is service access, not recovery authority. **Read-only readiness 2026-08-22.** The owner `node-reboot` preflight passed every automated check without observing a secret value: node/k3s, platform-pg and continuous archiving, a 16.37-hour successful backup, initialized/unsealed OpenBao with `shares=3` and `threshold=2`, ESO stores/projections, and the reviewed audit-core digest were healthy. `ready_for_live_execution` correctly remained false because no approved window, current encrypted off-host snapshot receipt, provider-console attestation, quorum attestation, owner-ack flag, or named abort operator was supplied. That preflight also exposed a Warden safety defect: the ceremony request matched the generic `openbao-api-key` template and proposed paste-once provisioning. `openbao-shamir-recovery-ceremony` now distinguishes the owner-operated approval ceremony from secret retrieval. It is a non-value-bearing pointer and can never offer `--fetch`, `--exec`, `--out`, `--wrap`, or paste-once share transport. **Scenario established 2026-08-22.** The operator delegated preparation to the agent and reserved their involvement for the final GO/NO-GO. The bounded, non-secret scenario is `docs/evidence/WARDEN-WP-0027-T02-drill-scenario-2026-08-22.md`: ops-warden coordinates and enforces the hold point; railiance-platform is the proposed OpenBao driver; railiance-infra is the proposed independent console/abort owner; and railiance-master is asked only for a metadata-only 2-of-3 availability attestation. All three owner receipts and a green fully parameterized preflight are required before the final operator question. The railiance-infra role is scoped by the authoritative review contract `WARDEN-WP-0027-T02-DRILL-20260822-01-INFRA` in `interfaces/reviews/WARDEN-WP-0027-T02-DRILL-20260822-01-railiance-infra.json`. It binds the exact scenario artifact and expiry, requires explicit independent provider-console and distinct-abort attestations, permits metadata-only evidence, and states `authorizes_execution: false`. **Preparation approved 2026-08-22.** Decision `9da57559-712a-4521-b46e-a4c69729f9d2` authorizes preparation only and narrows the possible live scope to one intentional OpenBao seal followed by the existing 2-of-3 unseal ceremony. It excludes a host reboot, re-key, restore, policy change, PVC mutation, credential disclosure, and general workload restart. Railiance-infra satisfied its direct contract with receipt `01a02b4b-7295-7836-b288-f29407008524`. The revisioned, non-authorizing gate matrix is in `docs/evidence/WARDEN-WP-0027-T02-drill-preparation-checklist-2026-08-22.md`; platform snapshot/driver and master quorum contracts remain required before the fully parameterized read-only preflight. ## Task: Tamper-evident policy governance + reconcile ```task id: WARDEN-WP-0027-T03 status: cancel priority: medium state_hub_task_id: "94d5dcaf-abf0-417c-a6ef-26e8c50905a8" ``` Policy-as-code for OpenBao policies/roles with tamper-evidence: a signed/hashed manifest of the intended policy set, drift **detection** against live OpenBao, and a reconcile-to-known-good-commit path. (Drift detection may land independently even if full reconcile stays gated.) Coordinate ownership with railiance-platform (OpenBao deployment) — ops-warden contributes the ops-access policy surface, not the cluster. **Done when:** live OpenBao policy state is diffable against a signed known-good manifest, and an attended reconcile can restore it. **Cross-repo:** railiance-platform (OpenBao cluster + policy custody). ## References - `WARDEN-WP-0026` — Strand A (disclosure hygiene, advisory rotation) — prerequisite - `.claude/rules/credential-routing.md` — ops-warden custodies no secret values - `railiance-platform` — OpenBao cluster, policy custody, credential broker - `wiki/AccessRouting.md` — issue vs route vs assist boundary - `workplans/ADHOC-2026-08-11.md` T03 — parked warden-sign AppRole (ad-hoc finished 2026-08-15); T02 resolves it as unsuitable for break-glass, while a separately approved ops-bridge unattended-signing design may still re-evaluate it cert_command cutover - `secrets-engine` `SECRETS-WP-0004` — the parked AppRole apply/handoff