railiance-platform/argocd/platform-addons/secret-annotation-guard/README.md

# Secret annotation guard

Implemented under the existing CUST-WP-0073-T03; no new platform workplan.
Native admission policy denies the last-applied annotation on Secret CREATE
and UPDATE, including an empty value. No new workload or controller.

Before the first manual sync, run `scripts/secret_annotation_maintenance.py`
on railiance01 through the supervised admin path, first without arguments,
then with `--clean`. It removes only the duplicate annotation; it never prints
kubectl output or Secret values. Concurrent Secret churn can stop cleanup;
inspect the receipt before retrying. Preserve only names, counts and booleans.

Declare the policy through the pinned `secret-annotation-guard` Argo Application;
automated sync and prune are off. Platform-addons AppProject must allow both
admission kinds. Sync policy first and inspect typeChecking; bind only after
cleanup. Test clean CREATE/UPDATE and denied annotated CREATE/UPDATE using
synthetic data in whitehat; verify client-side apply is denied, then delete
only the test fixture. Secret writers must use server-side apply or replace.

Rollback is a manual Argo sync of a reviewed revision without the binding
(with explicit resource-scoped pruning), or attended break-glass deletion of
the binding followed by Git reconciliation. Removing the binding reopens this
leak path. The policy does not rotate credentials or isolate agent accounts.

## September 28 rollout and rollback

Policy source `800cbfa` and Application declaration `54885ac` were deployed
through manual, resource-scoped Argo sync. Native type checking passed. Nine
synthetic checks passed: clean creation/update/server apply, rejection of
annotated create/update including empty values, client-side apply rejection,
and fixture cleanup.

Enforcement was rolled back when ESO v0.16.1 targets stopped refreshing.
That version copies ExternalSecret metadata when no target template exists;
31 current ExternalSecrets have no template. Removing annotations from targets
alone cannot stop the controller adding them back. All 39 ExternalSecrets
recovered after binding deletion; failed controllers were explicitly refreshed.
The policy remains installed but UNBOUND. `binding.pending.yaml` is deliberately
absent from kustomization. No ordinary sync of this revision enables enforcement.

Before enabling: add explicit metadata templates in the 31 owning declarations,
preserving intended labels/annotations except last-applied; apply through owner
paths; verify actual ESO refresh with annotation-free target Secrets. Retain
existing data templates and remote references. Do not waive ESO from the policy
or upgrade the controller as a shortcut. Rerun cleanup, native admission tests
and an ESO refresh integration test, then include the binding and update the pin.

The absent `platform-pg-drill` namespace still has an orphan Secret `drill-minio`,
a Deployment referencing it, a PVC and Service. Metadata patch fails because the
namespace is absent. No orphan object was deleted or namespace recreated. Its
disposition stays in CUST-WP-0073-T03; do not report cluster-wide cleanup complete.

Source for the diagnosed behavior:
https://github.com/external-secrets/external-secrets/blob/v0.16.1/pkg/controllers/externalsecret/externalsecret_controller_template.go

Detailed receipts: the-custodian/docs/evidence/2026-09-28-secret-annotation-*.json.