railiance-platform/argocd/platform-addons/secret-annotation-guard/README.md

Secret annotation guard

Implemented under the existing CUST-WP-0073-T03; no new platform workplan. Native admission policy denies the last-applied annotation on Secret CREATE and UPDATE, including an empty value. No new workload or controller.

Before the first manual sync, run scripts/secret_annotation_maintenance.py on railiance01 through the supervised admin path, first without arguments, then with --clean. It removes only the duplicate annotation; it never prints kubectl output or Secret values. Concurrent Secret churn can stop cleanup; inspect the receipt before retrying. Preserve only names, counts and booleans.

Declare the policy through the pinned secret-annotation-guard Argo Application; automated sync and prune are off. Platform-addons AppProject must allow both admission kinds. Sync policy first and inspect typeChecking; bind only after cleanup. Test clean CREATE/UPDATE and denied annotated CREATE/UPDATE using synthetic data in whitehat; verify client-side apply is denied, then delete only the test fixture. Secret writers must use server-side apply or replace.

Rollback is a manual Argo sync of a reviewed revision without the binding (with explicit resource-scoped pruning), or attended break-glass deletion of the binding followed by Git reconciliation. Removing the binding reopens this leak path. The policy does not rotate credentials or isolate agent accounts.

September 28 rollout and rollback

Policy source 800cbfa and Application declaration 54885ac were deployed through manual, resource-scoped Argo sync. Native type checking passed. Nine synthetic checks passed: clean creation/update/server apply, rejection of annotated create/update including empty values, client-side apply rejection, and fixture cleanup.

Enforcement was rolled back when ESO v0.16.1 targets stopped refreshing. That version copies ExternalSecret metadata when no target template exists; 31 current ExternalSecrets have no template. Removing annotations from targets alone cannot stop the controller adding them back. All 39 ExternalSecrets recovered after binding deletion; failed controllers were explicitly refreshed. The policy remains installed but UNBOUND. binding.pending.yaml is deliberately absent from kustomization. No ordinary sync of this revision enables enforcement.

Before enabling: add explicit metadata templates in the 31 owning declarations, preserving intended labels/annotations except last-applied; apply through owner paths; verify actual ESO refresh with annotation-free target Secrets. Retain existing data templates and remote references. Do not waive ESO from the policy or upgrade the controller as a shortcut. Rerun cleanup, native admission tests and an ESO refresh integration test, then include the binding and update the pin.

The absent platform-pg-drill namespace still has an orphan Secret drill-minio, a Deployment referencing it, a PVC and Service. Metadata patch fails because the namespace is absent. No orphan object was deleted or namespace recreated. Its disposition stays in CUST-WP-0073-T03; do not report cluster-wide cleanup complete.

Source for the diagnosed behavior: https://github.com/external-secrets/external-secrets/blob/v0.16.1/pkg/controllers/externalsecret/externalsecret_controller_template.go

Detailed receipts: the-custodian/docs/evidence/2026-09-28-secret-annotation-*.json.