railiance-platform/argocd/platform-addons/secret-annotation-guard/README.md

# Secret annotation guard

Implemented under the existing CUST-WP-0073-T03; no new platform workplan.
Native admission policy denies the last-applied annotation on Secret CREATE
and UPDATE, including an empty value. No new workload or controller.

Before the first manual sync, run `scripts/secret_annotation_maintenance.py`
on railiance01 through the supervised admin path, first without arguments,
then with `--clean`. It removes only the duplicate annotation; it never prints
kubectl output or Secret values. Concurrent Secret churn can stop cleanup;
inspect the receipt before retrying. Preserve only names, counts and booleans.

Declare the policy through the pinned `secret-annotation-guard` Argo Application;
automated sync and prune are off. Platform-addons AppProject must allow both
admission kinds. Sync policy first and inspect typeChecking; bind only after
cleanup. Test clean CREATE/UPDATE and denied annotated CREATE/UPDATE using
synthetic data in whitehat; verify client-side apply is denied, then delete
only the test fixture. Secret writers must use server-side apply or replace.

Rollback is a manual Argo sync of a reviewed revision without the binding
(with explicit resource-scoped pruning), or attended break-glass deletion of
the binding followed by Git reconciliation. Removing the binding reopens this
leak path. The policy does not rotate credentials or isolate agent accounts.

## September 28 rollout and rollback

Policy source `800cbfa` and Application declaration `54885ac` were deployed
through manual, resource-scoped Argo sync. Native type checking passed. Nine
synthetic checks passed: clean creation/update/server apply, rejection of
annotated create/update including empty values, client-side apply rejection,
and fixture cleanup.

Enforcement was rolled back when ESO v0.16.1 targets stopped refreshing.
That version copies ExternalSecret metadata when no target template exists;
31 current ExternalSecrets have no template. Removing annotations from targets
alone cannot stop the controller adding them back. All 39 ExternalSecrets
recovered after binding deletion; failed controllers were explicitly refreshed.
The rollback revision `6016f72` left the policy UNBOUND and excluded the binding
from kustomization. That revision remains the immediate rollback target.

Re-enablement prerequisite: add explicit metadata templates in the 31 owning declarations,
preserving intended labels/annotations except last-applied; apply through owner
paths; verify actual ESO refresh with annotation-free target Secrets. Retain
existing data templates and remote references. Do not waive ESO from the policy
or upgrade the controller as a shortcut. Rerun cleanup, native admission tests
and an ESO refresh integration test, then include the binding and update the pin.

An orphan Secret `platform-pg-drill/drill-minio` initially blocked complete
cleanup because its namespace was absent. The founder subsequently approved
deleting only that Secret. UID-bound deletion and absence were verified; the
bound 3 GiB PVC was unchanged. No other resources or namespace were changed.
Receipt: the-custodian/docs/evidence/2026-09-28-orphan-secret-deletion.json.

Source for the diagnosed behavior:
https://github.com/external-secrets/external-secrets/blob/v0.16.1/pkg/controllers/externalsecret/externalsecret_controller_template.go

Detailed receipts: the-custodian/docs/evidence/2026-09-28-secret-annotation-*.json.

## Corrected writer rollout

On September 28 the founder authorized continuing the 31-declaration correction.
Metadata-only templates are committed and published across the 12 owning repos.
Server dry-run proved all 31 changes preserve the remaining ExternalSecret spec.
Direct owner resources used metadata-only SSA; Target Revenue used a selective
Argo sync of its ExternalSecret, with no migration/bootstrap hooks. All 39
ExternalSecrets completed fresh successful refreshes before binding activation.

`binding.yaml` is included for the reviewed re-enablement. After syncing, repeat
the native admission proof and require all 39 ExternalSecrets to complete fresh
refreshes with the binding present. Keep the safe annotation scan receipt and
report the absent-namespace orphan separately. No Secret values are read or
rotated by the metadata maintenance. ESO uses its normal credential refresh path.

Detailed preflight, publication and before/after integration receipts live in
`the-custodian/docs/evidence/2026-09-28-eso-*.json`. The earlier rollout failure
and rescue remain recorded; remediation does not count as unchanged success.

## Verified completion

The binding is active at source `7daf7e9`, application declaration `db51ec8`.
Argo reports Synced/Healthy. All nine native admission checks pass and all 39
ExternalSecrets completed fresh successful refreshes under Deny. All 257 Secrets
in existing namespaces passed the forbidden-annotation scan. The one orphan
exception was subsequently resolved as above. CUST-WP-0073-T03 is complete;
agent-runtime isolation and deferred credential rotation are separate tasks.