railiance-platform/docs/credential-change-approval.md

# Credential Change Approval Workflow

This document sketches the operator workflow we want for it-sec and credential
changes. The goal is to remove raw OpenBao command authoring from routine human
operation while preserving explicit human approval, auditability, and safe
handling of secret values.

## Problem

The current workflow still asks operators to translate a reviewed intent into
OpenBao commands by hand:

- create or update policies;
- create auth roles with the right bound claims;
- create or rotate secret paths and fields;
- verify positive and negative access;
- tell ops-warden or another access front door when a lane may become active.

That is inefficient and easy to get wrong. It is also hard to review because
the actual unit of work is spread across chat, workplans, OpenBao UI screens,
State Hub notes, and shell commands.

## Direction

Treat OpenBao as the enforcement and audit engine, not the primary review UI.
Add a small approval control plane in front of it:

1. an agent or CLI creates a structured, non-secret credential change request;
2. humans review the rendered proposal, risk notes, generated OpenBao plan, and
   verification plan;
3. a human approves or denies with a comment;
4. only approved requests can be applied by an operator-controlled helper;
5. the helper records non-secret evidence and marks the request active,
   rejected, deactivated, rotated, or compromised.

This can be implemented with repo files, State Hub, and CLI/chat integration
first. An OpenBao UI extension can come later if the workflow proves itself.

## Core Object

The canonical unit is a credential change request, abbreviated `CCR`.

The CCR must be non-secret. It may contain:

- stable request id and title;
- requester, reviewer, approver, and applier identities;
- target domain, tenant, workload, environment, and purpose;
- OpenBao mount, path, field names, policy names, and auth role names;
- exact non-secret policy HCL or generated policy references;
- proposed auth bindings and bound claims;
- delivery surface such as ops-warden, External Secrets, CSI, or direct caller
  fetch;
- machine-readable front-door readiness, including `readiness` and
  `resolvable`;
- risk classification and approval requirements;
- generated apply plan;
- verification plan;
- rollback, deactivate, rotate, and compromise response plan;
- comments, approvals, denials, and timestamps;
- non-secret OpenBao audit request ids or timestamps after execution.

It must not contain:

- secret values;
- wrapped token values;
- root, platform-admin, or issuer tokens;
- passwords, API keys, private keys, OTP seeds, unseal shares, or recovery
  codes;
- command output that includes secret values.

## State Machine

Suggested states:

```text
draft
in_flight
proposed
needs_changes
approved
denied
apply_pending
applied
verified
active
deactivated
rotated
compromised
superseded
cancelled
```

`in_flight` is the only nonterminal state that may explicitly omit completion-only fields.
It must declare every omission in `in_flight.missing_fields`, name the owner and
blocking reason, and remain non-resolvable. The validator still checks every
other field. This is not an applyable state and must never be used to hide a
malformed active lane.

An incomplete request withdrawn by its owner may instead be `cancelled`, retaining
its `in_flight` declaration as history. This requires `cancellation.owner`,
`cancellation.reason` and a non-secret `cancellation.source_ref`, with front-door
readiness `disabled` and `resolvable: false`. Only the same declared omissions are
allowed; all other validation remains. Cancellation cannot authorize apply or
resolve a reader. Moving back to an approval/apply state requires completing the
binding; keeping the historical declaration does not bypass that requirement.

Only `approved` requests may be applied. Only `verified` requests may become
`active`.

Emergency break-glass work may create a request after the fact, but it must be
marked as break-glass, reviewed retrospectively, and linked to audit evidence.

## Review Surface

A reviewer should see a concise rendered proposal:

```text
Request: whynot-design npm publish token lane
Type: workload-kv-read
Mount/path/field:
  platform/workloads/coulomb/whynot-design/npm-publish
  NPM_AUTH_TOKEN
Policy:
  workload-kv-read-whynot-design-npm-publish
Auth binding:
  netkingdom OIDC role whynot-design-workload-kv-read
  bound claim: groups includes whynot-design
Access front door:
  ops-warden whynot-design-npm-publish
  readiness: template
  resolvable: false
Risk:
  grants read access to npm publish credential
Checks:
  positive whynot fetch, negative non-whynot denial, OpenBao audit evidence
Decision:
  approve | deny | needs changes
Comment:
  free text
```

The reviewer should not need to know the exact `bao write` syntax. They should
be able to discuss the proposal in chat, request changes, and then make a
formal decision.

## Minimal Implementation

Version 1 should be boring:

- store CCR files under `credential-change-requests/`;
- validate CCR schema offline;
- render a human-readable review summary;
- generate OpenBao apply plans from approved CCRs;
- require an approval record before apply;
- apply only non-secret policy/auth/path metadata;
- prompt or delegate separately for secret value entry;
- record non-secret evidence in State Hub.

When the schema adds or strengthens a required field, the same change must
include a migration pass over every existing CCR. Active declarations must
describe the live authentication path; incomplete requests must move to the
explicit `in_flight` state rather than relying on a filename exception or a
loosened repository-wide assertion.

The first implemented CLI slice is:

```bash
make credential-change-validate
make credential-change-render CREDENTIAL_CHANGE=CCR-2026-0001
make credential-change-plan CREDENTIAL_CHANGE=CCR-2026-0001
make credential-change-status CREDENTIAL_CHANGE=CCR-2026-0001
make credential-change-status-json CREDENTIAL_CHANGE=CCR-2026-0001
scripts/credential-change.py confirm-binding CCR-2026-0001 --reviewer <name> --comment "..."
scripts/credential-change.py approve CCR-2026-0001 --reviewer <name> --comment "..."
scripts/credential-change.py deny CCR-2026-0001 --reviewer <name> --comment "..."
scripts/credential-change.py needs-changes CCR-2026-0001 --reviewer <name> --comment "..."
make credential-change-sync-decision CREDENTIAL_CHANGE=CCR-2026-0001
make credential-change-apply-plan CREDENTIAL_CHANGE=CCR-2026-0001
make credential-change-operator-commands CREDENTIAL_CHANGE=CCR-2026-0001
make credential-change-runbook CREDENTIAL_CHANGE=CCR-2026-0001
scripts/credential-change.py runbook CCR-2026-0001 --execute-metadata --actor <operator> --confirm "APPLY CCR-2026-0001"
scripts/credential-change.py record-evidence CCR-2026-0001 --actor <operator> --kind positive_verification --result passed --detail "<non-secret audit reference>" --record-state-hub
make credential-change-lifecycle-plan CREDENTIAL_CHANGE=CCR-2026-0001 CREDENTIAL_CHANGE_LIFECYCLE_ACTION=deactivate
scripts/credential-change.py lifecycle-event CCR-2026-0001 --action compromise --actor <operator> --reason "<non-secret reason>" --detail "<non-secret evidence>" --blast-radius "<non-secret scope>" --follow-up "<task/ref>" --record-state-hub
scripts/credential-change.py import-inventory CCR-YYYY-NNNN --title "existing lane" --tenant <tenant> --workload <workload> --environment production --purpose "<purpose>" --kv-path platform/workloads/<tenant>/<workload>/<purpose> --field <FIELD_NAME> --auth-method oidc --auth-mount netkingdom --auth-role <role> --bound-claim groups=<group> --bound-claims-confirmed --frontdoor-type ops-warden --catalog-id <catalog-id> --reason "Imported existing lane without secret values"
```

`apply-plan` and `operator-commands` are intentionally guarded: they refuse
anything not approved and refuse unconfirmed auth bindings. `operator-commands`
renders the reviewed non-secret `bao policy write` and `bao write auth/.../role`
commands for a platform operator; the actual secret value is still provisioned
through approved OpenBao/operator custody only.

The same operations can be exposed through chat by having the agent create the
proposal, show the rendered summary, then call the CLI only after the human
gives an explicit approval phrase.

## State Hub Role

State Hub should hold:

- request lifecycle events;
- review comments;
- approval/denial decisions;
- non-secret apply and verification evidence;
- links to workplans and CCR files.

State Hub should not hold secret values. It can be the first review UI because
it already supports messages, progress, task status, and cross-repo
coordination.

For CCR review, create a pending State Hub decision that links to the CCR and
contains only non-secret coordinates. Operators can inspect it in the dashboard
at `http://127.0.0.1:3000/decisions` and resolve it with a rationale beginning
with `APPROVE:`, `DENY:`, or `NEEDS_CHANGES:`. Then run
`make credential-change-sync-decision CREDENTIAL_CHANGE=<CCR>` to copy the
resolved decision back into the CCR file-backed state.

## OpenBao Role

OpenBao remains authoritative for:

- policy enforcement;
- auth method configuration;
- token issuance and revocation;
- secret storage;
- audit logs.

Where OpenBao supports non-secret metadata on secret paths or auth roles, we can
mirror CCR ids and status labels. The workflow must not depend on OpenBao being
the only index, because operators need to see proposed, rejected, deactivated,
rotated, and compromised items across repos and access front doors.

## ops-warden Role

ops-warden should consume only approved and active access lanes.

For draft requests, ops-warden may create a draft catalog entry that points to
the CCR, but it should not activate the entry until the CCR is verified.

For `warden access --fetch` / `--exec`, the catalog should include the CCR id
and refuse active use when the CCR state is not `active`, `readiness` is not
`ready`, or `resolvable` is not `true`.

## Interactive Runbook Role

The interactive runbook is the operator bridge:

1. load a CCR;
2. show the rendered summary and exact generated plan;
3. confirm the request is approved;
4. acquire operator authority through an approved path;
5. apply the plan;
6. ask for attended secret entry when needed;
7. run positive and negative verification;
8. record non-secret evidence;
9. notify downstream front doors such as ops-warden.

`credential-change.py runbook <CCR>` renders the checklist and exact final
confirmation phrase. `--execute-metadata` is intentionally opt-in and requires
that phrase; it uses the local `bao` CLI with ambient approved operator
authority, writes only policy/auth metadata, and records a non-secret
`metadata_apply` evidence entry. Secret value provisioning stays outside the
script through approved OpenBao/operator custody. Verification, activation, and
manual custody events are recorded with `record-evidence`, whose comments are
scanned for known secret markers before the CCR file or State Hub is updated.

This lets operators safely drive privileged work without needing to remember
every OpenBao command.

## Compromise And Deactivation

Every active CCR needs a deactivate and rotate path:

- `deactivated`: access intentionally disabled but not necessarily compromised;
- `rotated`: secret value replaced and old value no longer valid;
- `compromised`: emergency state requiring immediate disablement, rotation,
  blast-radius notes, and incident follow-up.

`lifecycle-plan` renders the attended checklist for each case, including the
front-door state change and OpenBao metadata disable commands for deactivation
or compromise. `lifecycle-event` records the non-secret lifecycle event in the
CCR, sets the CCR status, and marks the access front door disabled, pending
verification, or compromised as appropriate. For compromise events it accepts
non-secret blast-radius notes and follow-up task references. Existing lanes that
predate CCRs can be imported with `import-inventory`, which writes a CCR and
matching read-policy artifact from metadata only; it never asks for or stores
the secret value.

The workflow must support marking an existing credential or lane as compromised
even when the original request predates this system.

## Near-Term Target

Use the whynot-design npm token lane as the pilot:

1. encode the existing non-secret lane as a CCR;
2. render it for review;
3. approve or request changes from chat;
4. generate/apply the OpenBao policy and auth role only after approval;
5. provision the secret value by attended operator custody;
6. verify and activate the ops-warden catalog entry.

Once that path feels good, reuse it for the sibling workload-KV lanes and the
credential broker's OpenBao token-role gates.

## Self-Service Intake (RPF-WP-0061)

Clients file standard CCRs without a railiance-platform worker session. The
platform's judgement is in [`registry/ccr-standard-profile.yaml`](../registry/ccr-standard-profile.yaml);
`review-standard` applies it, and `.forgejo/workflows/ccr-intake.yaml` re-checks
it in CI. Allowed submitters: **ops-mason** only (profile `submitters`).

**In profile** means: a `workload-kv-read` lane on mount `platform`; one exact path
`platform/workloads/<a>/<b>[/<c>]` or `platform/suppliers/<supplier>/<account>`;
`metadata_read: false`; a policy file equal to the generated HCL; Kubernetes auth
with role `external-secrets-*`, audience `openbao`, all ttls 15m or less,
`token_no_default_policy: true`; one service account in one non-platform
namespace; risk `low` or `medium`; and no other live CCR reading the path.
Everything else (OIDC roles, AppRole, token roles, second readers, high risk)
still goes to the `railiance-platform` inbox.

### Client steps

Run these in a railiance-platform checkout that is up to date with `main`:

```bash
# 0. Generate the policy instead of writing it, so it is byte-equal by construction.
python3 scripts/credential-change.py render-policy --kv-path <platform/...> > <policy.hcl>
#    Optional: plan_ref: {repo, path, commit} in the draft links the client's
#    construction plan; submit keeps it and review-standard only checks its shape.

# 1. File the draft. Allocates the CCR id, copies the policy, runs the review.
python3 scripts/credential-change.py submit \
  --ccr <draft.yaml> --policy <policy.hcl> --slug <slug> \
  --requester ops-mason --push --notify
#    [NEEDS-PLATFORM] lines mean out of profile: stop. --notify messaged
#    railiance-platform. Do not edit the review comment to get past it.

# 2. Wait for the "CCR intake gate" run on ccr/<id>-<slug> to pass, then
#    fast-forward main to it. Never merge a red or unchecked ccr/* branch.
git switch main && git merge --ff-only ccr/<id>-<slug> && git push origin main

# 3. Open the founder decision and commit the decision id.
python3 scripts/credential-change.py request-decision <id>
git commit -m "<id>: request founder decision" -- credential-change-requests/<id>-<slug>.yaml
git push origin main
```

After step 3, the client waits and does not edit `status` or `review.comments`
itself. Approvals reach the file only through `sync-decision`, even when the
founder approved the client's plan elsewhere. The hub decision is the single
approval record for both the client plan and the CCR; clients copy their plan
approval *from* it after the sync. The **founder** resolves the State Hub decision.
Then a **platform session or the founder** runs `sync-decision <id>`. Clients do
not run `sync-decision` on their own CCRs. State Hub does not authenticate who
resolved a decision (`decided_by` is free text). Until founder approval can be
verified (RPF-WP-0061 follow-up B), the sync must come from someone other than
the requester. After the CCR is `approved`, the usual steps apply:
`confirm-binding` against native state, apply (ops-mason under
ops-mason-construct for in-profile lanes, otherwise the applier), then
`record-evidence`.

### Known limits

- All workstation agents push to Forgejo as the same account, so `ccr/*`
  cannot yet be restricted to ops-mason. The submitter list and the
  merge-only-green rule are enforced by tooling and convention, not by Forgejo.
  The main-branch `check-ids` job and `check-branch` detect violations after the
  fact (follow-up C).
- The automated review is a source review only. It never sets `approved` and
  never touches OpenBao.