ops-mason/docs/kubernetes-plane.md
tegwick 36445ae679 Enforce readiness tiers and reconcile blocked workplans
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a0e75a-fc5c-7913-9dba-9846210c766d
2026-09-28 11:40:57 +02:00

5.2 KiB

Guarded Kubernetes plane automation

ops-mason plane constructs small security foundations from an immutable, reviewed bundle. It is not a general Kubernetes deployment wrapper.

Safety model

Every bundle declares its exact object identities, source revisions, expected cluster context, namespace, dependency assertions, forbidden kinds, manifest hashes, approved construction plan, and evidence destination.

Read-only commands (render, preflight, verify, rollback-plan) do not require approval. apply refuses unless all of these hold:

  1. the construction plan is status: approved and names its approver/date;
  2. --confirm exactly matches the plan ID;
  3. --expect-digest exactly matches the freshly rendered bundle digest;
  4. the repository is committed and clean;
  5. the Kubernetes context, create RBAC, live-object drift, dependency assertions, client validation, and available server validation pass;
  6. the bundle contains exactly the allowlisted objects, no Pod or Secret, no data/stringData, no cross-namespace object, and no token-bearing ServiceAccount.

The executor applies one pinned manifest at a time with server-side apply and field manager ops-mason. Namespaced manifests receive server dry-run after the Namespace exists and before they are persisted. A failed partial apply is reported; the executor does not automatically delete a Namespace.

Successful apply immediately performs metadata-only verification, checks that no Pod or Secret exists in the plane namespace, and writes JSON evidence with object UIDs/resource versions and generated rollback commands. It never reads or records a Secret value.

Commands

ops-mason plane render --bundle bundles/<id>.yaml
ops-mason plane preflight --bundle bundles/<id>.yaml
ops-mason plane apply --bundle bundles/<id>.yaml \
  --confirm <approved-plan-id> \
  --expect-digest <digest-from-preflight>
ops-mason plane verify --bundle bundles/<id>.yaml
ops-mason plane rollback-plan --bundle bundles/<id>.yaml

Rollback output is a plan, never an action. Review live inventory immediately before using it.

Readiness gate

apply checks ops_mason.readiness.inspect_readiness after approval/digest checks and before any Kubernetes command. preflight reports the result even when direct apply would be refused.

Verified target state Direct apply under APPROVED
declared, installed, verified Allowed with the existing approved-plan checks
production-approved, including a later evidence lapse Refused; use the manifest repository and ArgoCD
deprecated Retains the previous tier; missing history means production
missing, unknown, invalid or stale readiness Production; refused
whitehat namespace, explicit founder placement of 2026-09-21 Non-production, until a binding supersedes the placement
rapp-policy-nexus, production tier Transition only before 2026-12-21; evidence labels it production

ArgoCD now exists on railiance01, so the historical blanket transition for all production targets is not enabled. The policy-nexus transition remains bounded by its review date. The gate does not decide authorization or contact an authorization engine.

Bundles include a reviewed namespace-to-binding mapping and a source pin:

readiness:
  target: {kind: rapp, rapp_id: rapp-user-engine, namespace: user-engine}
  source:
    repo: reef-railiance
    path: bindings/rapps.yaml
    revision: <full-40-character-commit-id>
    sha256: <sha256-of-file-at-that-commit>

The approved bundle is the mapping record: the namespace must match its object scope, and any namespace mapping in the source must agree. The only explicit namespace placement is kind: namespace, namespace: whitehat for bundle whitehat-foundational-plane, using the same pinned source so a newly declared whitehat binding invalidates the placement. Other namespace-only targets and platform objects remain production-tier.

Use --readiness-repo /path/to/reef-railiance on preflight or apply to locate the source; the default is the sibling checkout. The gate verifies its file digest, commit ancestry, unchanged content through local HEAD, clean source file, and complete Git history. Refresh that checkout before review: the gate checks local HEAD, not an unfetched remote. Missing source/history fails closed. Any production approval in the reachable file history retains production tier; a deliberate re-scope needs a separately reviewed gate change, not just lowering the readiness field. Source pins and mapping changes alter the bundle digest and need fresh review/confirmation. Existing live evidence remains historical.

For emergency direct apply, keep all normal plan/digest requirements and add:

ops-mason plane apply --bundle bundles/<id>.yaml \
  --confirm <approved-plan-id> --expect-digest <digest> \
  --activation BREAK_GLASS --break-glass-reason '<incident and justification>'

The JSON evidence and CLI output record activation, local executing account, time, reason, resolved tier/source, and the obligation to commit the same change to the manifest repository ArgoCD reconciles. The command does not open that change or grant emergency authority itself. An empty reason is refused.