build: backup-object-storage AppRole lane delivered

Empty-structure plus xxx example fields. Founder replaces values
in the OpenBao UI before any bucket create.

plans/audit-core-openbao-runtime-custody.md

@@ -0,0 +1,146 @@
+---
+id: audit-core-openbao-runtime-custody
+demand_source: audit-core/workplans/AUDIT-WP-0005-T02
+consumer_repo: audit-core
+credential_type: openbao-approle-kv
+status: built
+approved_by: "Bernd Worsch"
+approved_at: "2026-08-13"
+created: "2026-08-13"
+updated: "2026-08-13"
+---
+
+# Construction plan: audit-core OpenBao runtime custody
+
+## 1. Demand
+
+audit-core on railiance01 needs durable PostgreSQL custody and a sender
+registry without the founder typing OpenBao tokens or running `bao kv put`.
+The founder’s remaining acts are: approve this plan, and unseal OpenBao if
+it is sealed. Mechanics belong to ops-mason (structure) and ops-warden
+(routing / founder desk), not to an ad-hoc terminal session.
+
+## 2. Existing-structure survey
+
+Checked 2026-08-13 against live railiance01 and the routing catalog:
+
+- **Database leases already exist.** `rapp-postgres` provisioned `audit_core`
+  with `audit_core_app` / `audit_core_migrate`, and OpenBao already vends
+  `database/creds/audit-core-runtime` and `database/creds/audit-core-migration`
+  through railiance-platform grants `rapp-postgres/audit-core-runtime` and
+  `rapp-postgres/audit-core-migration`. `warden route show
+  database-dynamic-credentials` points at that playbook. Do not create a
+  second database engine or a KV copy of a rotating lease.
+- **ESO token already exists.** `external-secrets/openbao-audit-core-eso-token`
+  was minted 2026-08-13 with policy `external-secrets-audit-core` (read on
+  the two database cred paths and on `platform/data/workloads/audit-core/senders`).
+  ClusterSecretStore YAML is drafted; it is not yet applied.
+- **KV senders path does not exist.** A founder `bao kv put` against
+  `platform/workloads/audit-core/senders` 403’d because the workstation
+  `~/.vault-token` is not an operator identity. That is the failure this
+  plan exists to retire.
+- **Catalog.** `database-dynamic-credentials` already covers the lease.
+  There is no `audit-core-senders` entry. `openbao-api-key` is the generic
+  template and must not become the concrete lane.
+- **Auth topology.** Same as user-engine / rapp-qonto: railiance01 ESO
+  cannot use CoulombCore Kubernetes auth. AppRole is the transitional
+  bridge; same-cluster Kubernetes auth remains the target.
+- **Does an existing lane already satisfy this?** The database half, yes —
+  bind to the existing OpenBao database roles. The sender-registry half,
+  no — there is no path, no AppRole, and no catalog entry. Reuse the
+  email-connect / user-engine *shape* (namespace-limited store, 15m
+  tokens, eight uses), do not widen those identities.
+
+## 3. Proposed changes
+
+| # | Action | Object | Reuse-vs-new rationale |
+|---|---|---|---|
+| 1 | reuse | OpenBao roles `audit-core-runtime` / `audit-core-migration` | Already vended; a KV copy of a lease recreates the delivery gap |
+| 2 | reuse | policy `external-secrets-audit-core` | Already written; covers database creds + senders read |
+| 3 | create | AppRole `external-secrets-audit-core` | Replace the long-lived ESO token with the same 15m/30m/8-use bridge as user-engine and rapp-qonto |
+| 4 | deliver | Secret `external-secrets/openbao-audit-core-approle` | Mason write-only delivery of role_id/secret_id; never logged |
+| 5 | create | ClusterSecretStore `openbao-audit-core` | Namespace-limited to `audit-core`; AppRole auth (token store is the interim until this builds) |
+| 6 | reuse | ClusterSecretStore `openbao-audit-core-database` + ExternalSecret | Namespaced VaultDynamicSecret cannot read the ESO token in `external-secrets`; CSS can. One `dataFrom` extract per lease. Applied 2026-08-13, SecretSynced |
+| 7 | create | empty KV path `platform/workloads/audit-core/senders` | Structure only. Values are **not** founder-pasted: first deploy mints sender tokens in-cluster into Secret `audit-core-senders`. A later wrap-migrate (user-engine pattern) can move them into this path without printing |
+| 8 | propose | ops-warden catalog `audit-core-senders` | Pointer, `warden_executes: false`, starts `draft` |
+| 9 | retire | founder `bao kv put` / token-prompt as the T02 delivery path | That is the error-prone interaction this demand forbids |
+
+No sibling workload path is readable. Mason does not print, log, or return
+sender tokens or database passwords.
+
+## 4. Review notes
+
+- **Naming:** `external-secrets-audit-core` / `openbao-audit-core` matches
+  email-connect and user-engine. Database grant names stay with rapp-postgres.
+- **TTL/scoping:** 15-minute tokens, 30-minute max, eight uses. Store
+  condition is `audit-core` only.
+- **Redundancy:** does not create a second database role or a KV cache of
+  the lease. Does not widen `openbao-email-connect` or
+  `openbao-user-engine-runtime`.
+- **Compaction:** retires the attended `bao kv put` path as the way T02
+  completes. The already-minted static ESO token is interim and is revoked
+  after the AppRole store is Ready.
+- **Secret values:** sender tokens are generated by us, not imported from a
+  vendor. Founder paste-once is the wrong desk for them. Generate in-cluster;
+  optional later wrap-migrate into the empty KV path.
+- **Founder remaining acts:** approve this plan; unseal OpenBao if sealed.
+  No token paste, no `bao` command review.
+
+## 5. Executive summary
+
+**One-line ask:** approve letting ops-mason finish audit-core’s OpenBao
+lane (AppRole + namespace-limited store + empty senders path + catalog
+pointer) so the receiver can run without you typing OpenBao tokens.
+
+**Who gets access:** only External Secrets, through AppRole
+`external-secrets-audit-core`. Application pods keep seeing mounted
+files and `AUDIT_CORE_SENDERS`. Nothing else gains a new identity.
+
+**To what:** read `database/creds/audit-core-runtime`,
+`database/creds/audit-core-migration`, and (later)
+`platform/workloads/audit-core/senders`. No sibling workload or tenant
+path.
+
+**For how long:** each AppRole login is 15 minutes, 30 minutes maximum,
+eight uses. The AppRole secret ID lasts until rotated or replaced by
+same-cluster Kubernetes auth.
+
+**Blast radius if the credential leaks:** append to the audit trail and
+read the sender registry; cannot rewrite or delete stored events (append-only
+trigger). Revoke the AppRole secret ID, rotate sender tokens overlap-first,
+and roll the receiver. Database leases expire on their own.
+
+**Cost to reverse:** delete the AppRole, its bootstrap Secret, the
+ClusterSecretStore, and the empty KV path. Database roles and the
+`audit_core` database stay (owned by rapp-postgres). No other consumer
+is on this identity.
+
+**Decision:** approve, reject, or send back to phase 1.
+Approval authorizes Mason build of structure and in-cluster sender
+minting. It does not authorize printing any secret value.
+
+## 6. Build result
+
+Built 2026-08-13 after founder approve, via
+`ops_mason.executor.build_approle_kv_lane` with `reuse_policy=True` so
+the existing `external-secrets-audit-core` policy (database creds +
+senders read) was not rewritten.
+
+- reused policy `external-secrets-audit-core`;
+- created AppRole `external-secrets-audit-core` (15m / 30m / 8 uses);
+- delivered role-id/secret-id into Secret
+  `external-secrets/openbao-audit-core-approle`; local delivery dir shredded;
+- switched ClusterSecretStores `openbao-audit-core` and
+  `openbao-audit-core-database` to AppRole auth; both `Valid`;
+- ExternalSecrets remain `SecretSynced`; receiver `/readyz` still
+  `custody_class=archive`;
+- deleted interim Secret `external-secrets/openbao-audit-core-eso-token`;
+- catalog entry `audit-core-senders` already drafted in ops-warden
+  (`warden route find "audit-core senders" --all`).
+
+Empty KV path `platform/workloads/audit-core/senders` was not written:
+mason does not put values, and senders already live in-cluster. A later
+wrap-migrate can create that path without a founder paste.
+
+No secret value was printed, committed, or logged. The 15-minute
+platform-admin child used for the build was revoked.

plans/backup-object-storage.md

@@ -0,0 +1,120 @@
+---
+id: backup-object-storage
+demand_source: resource-control/workplans/RESOURCE-WP-0002-T04
+consumer_repo: rapp-postgres
+credential_type: openbao-kv-founder-provision
+status: built
+approved_by: "Bernd Worsch"
+approved_at: "2026-08-14"
+created: "2026-08-14"
+updated: "2026-08-14"
+---
+
+# Construction plan: resource-control scoped backup key
+
+## 1. Demand
+
+`RESOURCE-WP-0002` T04 needs a Scaleway API key for **backup object
+storage in general**, not a Barman-only identity. Founder uses IAM
+application `resource-control` and binds policy / API-key
+`Scoped backup access` on the project that owns the backup bucket.
+The key lives in OpenBao. External Secrets later projects it into
+consumer Secrets (first: `platform-pg-backup-s3` in `databases`).
+The bootstrap key cannot create IAM. ops-mason builds the empty
+lane. Values never appear in git or chat.
+
+## 2. Existing-structure survey
+
+Checked 2026-08-14, retargeted the same day:
+
+- **KV path exists (version 1, 2026-08-14).** Founder put
+  `ACCESS_KEY`, `SECRET_KEY`, `DEFAULT_ORGANIZATION_ID`,
+  `DEFAULT_PROJECT_ID`. Not placeholders. No `APPLICATION_ID`.
+  The earlier draft path `…/backup/platform-pg-backup-s3` was not
+  built and must not be used.
+- **CCR-2026-0012** now reserves
+  `platform/workloads/railiance/backup/object-storage`. Reuse it.
+- **CCR-2026-0011** is the bootstrap key
+  (`platform/workloads/railiance/scaleway/bootstrap`). Different
+  purpose. Do not reuse those values. Do not hang that key on
+  application `resource-control`.
+- **CCR-2026-0004** is Nextcloud/age offsite. Different provider.
+- **Catalog.** No `backup-object-storage` lane.
+- **Auth.** First delivery is External Secrets into `databases`.
+  Kubernetes auth role `external-secrets-backup-object-storage` is
+  the CCR-declared steady state. railiance01 may need the interim
+  AppRole/token pattern; that apply is after placeholders are
+  replaced.
+- **Does an existing lane already satisfy this?** No live secret.
+
+## 3. Proposed changes
+
+| # | Action | Object | Reuse-vs-new rationale |
+|---|---|---|---|
+| 1 | reuse | CCR-2026-0012 path | Already reserved after retarget |
+| 2 | create | policy `workload-kv-read-backup-object-storage` | File already written; apply after approval |
+| 3 | create | empty KV v2 path (placeholders `xxx`) | Paste-once / UI replace has a home |
+| 4 | defer | Kubernetes auth role / AppRole / ESO store | After real values exist |
+| 5 | defer | catalog `backup-object-storage` | Pointer only; `warden_executes: false` until verified |
+| 6 | defer | Scaleway bucket policy principal | Needs `APPLICATION_ID` of `resource-control` |
+
+No new rail. Scaleway operates S3. No second IAM application.
+
+## 4. Review notes
+
+- **Naming:** path is `…/backup/object-storage`. IAM application is
+  `resource-control`. Policy and key label are `Scoped backup access`.
+- **IAM fact:** policies attach to the application, not one key. Only
+  backup-scoped keys belong on `resource-control`.
+- **TTL/scoping:** the Scaleway key is long-lived until rotated. The
+  OpenBao reader is ESO, 15m Kubernetes auth TTL in the CCR.
+- **Redundancy:** does not reuse the bootstrap path or the Nextcloud
+  offsite lane. Supersedes the Barman-named draft of the same CCR.
+- **Secret values:** mason does not read, write, or log them. Founder
+  uses the OpenBao UI or
+  `railiance-platform/docs/put-backup-object-storage.md`.
+- **Founder remaining acts:** approve this plan and CCR-2026-0012;
+  create or reuse application `resource-control`; bind policy and key
+  `Scoped backup access`; replace `xxx`; say “the backup key is in
+  bao.”
+
+## 5. Executive summary
+
+**One-line ask:** approve an OpenBao home for the resource-control
+backup key so first consumer `rapp-postgres` (and later backup
+consumers) can receive a projected Secret without putting keys in git
+or chat.
+
+**Who gets access:** you (founder) to write the three fields; later,
+External Secrets, first bound to namespace `databases`.
+
+**To what:** only
+`platform/workloads/railiance/backup/object-storage`
+(`ACCESS_KEY_ID`, `ACCESS_SECRET_KEY`, `APPLICATION_ID`). Not the
+bootstrap path.
+
+**For how long:** until you rotate or revoke the Scaleway key.
+
+**Blast radius if the credential leaks:** read/write/delete objects
+on the backup bucket. Revoke the key in the Scaleway console, delete
+the KV version, rotate.
+
+**Cost to reverse:** delete the KV path, policy, and unused drafts.
+WAL is not enabled yet, so there is no production archive to migrate.
+
+**Decision:** approved in chat 2026-08-14 (“Lets go”).
+
+## 6. Build result
+
+Built 2026-08-14:
+
+- policy `workload-kv-read-backup-object-storage` (read on the KV data
+  and metadata paths only)
+- AppRole `external-secrets-backup-object-storage` (15m/30m, 8 uses)
+- Kubernetes Secret `external-secrets/openbao-backup-object-storage-approle`
+  delivered; local files shredded
+- ClusterSecretStore `openbao-backup-object-storage` Valid
+- ExternalSecret synced `databases/platform-pg-backup-s3` (two keys,
+  values not read)
+
+Did not write Scaleway secret values. Did not enable WAL.

plans/reef-storage-scaleway-bootstrap.md

@@ -0,0 +1,144 @@
+---
+id: reef-storage-scaleway-bootstrap
+demand_source: resource-control/workplans/RESOURCE-WP-0002-T03
+consumer_repo: reef-storage
+credential_type: openbao-kv-founder-provision
+status: built
+approved_by: "Bernd Worsch"
+approved_at: "2026-08-14"
+created: "2026-08-14"
+updated: "2026-08-14"
+---
+
+# Construction plan: Scaleway bootstrap key for reef-storage
+
+## 1. Demand
+
+We need to create a private Scaleway Multi-AZ bucket in `nl-ams` for
+postgres backup (`RESOURCE-WP-0002`). The founder has a new Scaleway
+account and an API key in the usual Terraform snippet shape
+(`access_key`, `secret_key`, `organization_id`, `project_id` in
+`scaleway.auto.tfvars`). Those four values must live in OpenBao, not in
+git or chat. ops-mason builds the empty lane. The founder pastes the
+values once (desk or local tfvars ingest). reef-storage then creates the
+bucket by reading OpenBao and never printing the keys.
+
+## 2. Existing-structure survey
+
+Checked 2026-08-14:
+
+- **KV path does not exist.**
+  `platform/metadata/workloads/railiance/scaleway/bootstrap` is 404.
+  Same for the later backup path
+  `platform/workloads/railiance/backup/object-storage`.
+- **CCR already reserved the path.**
+  `railiance-platform` `CCR-2026-0011` names
+  `platform/workloads/railiance/scaleway/bootstrap` and the four fields.
+  Reuse that path. Do not invent a second one.
+- **CCR-2026-0004** is Nextcloud/age offsite backup
+  (`platform/workloads/railiance/backup/offsite-lane`). Different
+  provider, different fields. Do not merge.
+- **Catalog.** No `scaleway` / `s3-backup` lane. `openbao-api-key` is the
+  generic template and must not become this concrete key.
+- **Auth.** This is an operator-workstation bootstrap key for a one-shot
+  (or rare) `scw` create, not an in-cluster ESO consumer. No AppRole.
+  Founder paste-once / local tfvars ingest is the right desk. The later
+  scoped Barman key (T04) is a different path and may get a workload
+  CCR then.
+- **Does an existing lane already satisfy this?** No live secret. The
+  reserved CCR path is the one to build.
+
+Terraform snippet → OpenBao fields (same four facts, scw-native names):
+
+| Terraform (`scaleway.auto.tfvars`) | OpenBao field |
+| --- | --- |
+| `access_key` | `ACCESS_KEY` |
+| `secret_key` | `SECRET_KEY` |
+| `organization_id` | `DEFAULT_ORGANIZATION_ID` |
+| `project_id` | `DEFAULT_PROJECT_ID` |
+
+## 3. Proposed changes
+
+| # | Action | Object | Reuse-vs-new rationale |
+|---|---|---|---|
+| 1 | reuse | CCR-2026-0011 path `platform/workloads/railiance/scaleway/bootstrap` | Already reserved; do not fork |
+| 2 | create | empty KV v2 path (metadata only, no secret data) | Structure so paste-once / ingest has a home |
+| 3 | create | policy `operator-kv-scaleway-bootstrap` (read/write that path only) | Founder desk and the bucket-create script; no sibling paths |
+| 4 | reuse | founder paste-once desk + optional local tfvars ingest | ops-mason never sees values; four fields, four desk pastes or one founder-run ingest |
+| 5 | propose | catalog `scaleway-bootstrap` (draft) + playbook | Pointer only; `warden_executes: false` |
+| 6 | defer | AppRole / ESO / scoped backup key | T04 / CCR for `backup/object-storage`; not this plan |
+
+No new rail. Scaleway operates S3.
+
+## 4. Review notes
+
+- **Naming:** path follows `platform/workloads/railiance/…`. Policy
+  `operator-kv-scaleway-bootstrap` matches “operator workstation, not
+  workload ESO.”
+- **TTL/scoping:** no AppRole token. The *downstream* Scaleway key is
+  long-lived until the founder deletes it after the scoped key exists.
+- **Redundancy:** does not reuse Nextcloud offsite-lane. Does not put
+  org-admin keys in the Barman runtime path.
+- **Compaction:** none. This is the first Scaleway lane.
+- **Secret values:** mason does not read, write, or log them. Founder
+  uses `warden desk --act paste_once_provision` four times, or
+  `reef-storage/tools/ingest-scaleway-tfvars.sh` on a local 0600
+  `scaleway.auto.tfvars` they already have.
+- **Founder remaining acts:** approve this plan; paste four fields (or
+  run ingest once); say “bootstrap is in OpenBao.”
+
+## 5. Executive summary
+
+**One-line ask:** approve an OpenBao home for the four Scaleway Terraform
+variables so we can create the backup bucket without putting keys in git
+or chat.
+
+**Who gets access:** you (founder / operator token) to write the four
+fields; the bucket-create script, using that same operator token, to
+read them once. No new AppRole. No cluster workload. Nothing else gains
+a path.
+
+**To what:** only
+`platform/workloads/railiance/scaleway/bootstrap`
+(`ACCESS_KEY`, `SECRET_KEY`, `DEFAULT_ORGANIZATION_ID`,
+`DEFAULT_PROJECT_ID`). Not the later Barman key path. Not other
+`platform/workloads/railiance/backup/*` secrets.
+
+**For how long:** the KV version stays until you rotate or delete it.
+This bootstrap key should be revoked at Scaleway after the scoped bucket
+key works (T04).
+
+**Blast radius if the credential leaks:** create/list/delete Object
+Storage (and whatever else that API key can do in the project). Revoke
+the key in the Scaleway console, delete the KV version, rotate to a
+scoped bucket-only key.
+
+**Cost to reverse:** delete the KV path, draft catalog entry, and
+policy. No other consumer is bound. The bucket, once created, stays
+until you empty and delete it.
+
+**Decision:** approve, reject, or send back to phase 1.
+Approval authorizes structure (empty path, policy, catalog draft) and
+your paste-once / tfvars ingest. It does not authorize ops-mason to
+handle the Terraform secret values.
+
+## 6. Build result
+
+Executed 2026-08-14 after founder asked to bootstrap with example
+data and replace values in the OpenBao UI.
+
+Created (structure + placeholders only; values are the string `xxx`):
+
+- policy `operator-kv-scaleway-bootstrap` (read/write that one path)
+- KV v2 `platform/workloads/railiance/scaleway/bootstrap` version 1
+  fields `ACCESS_KEY`, `SECRET_KEY`, `DEFAULT_ORGANIZATION_ID`,
+  `DEFAULT_PROJECT_ID`
+- custom_metadata `placeholder=true`, `ccr=CCR-2026-0011`,
+  `replace_in_ui=true`
+
+Catalog draft `scaleway-bootstrap` already proposed.
+
+**Founder next:** OpenBao UI → secret
+`platform/workloads/railiance/scaleway/bootstrap` → create version 2
+with the real Terraform values. Then say “bootstrap is in OpenBao.”
+The bucket-create script refuses `xxx` placeholders.

src/ops_mason/executor.py

@@ -53,6 +53,9 @@ class AppRoleKVSpec:
     delivery_dir: Path | None = None
     bao_bin: str = "bao"
     audit_log_path: Path | None = None
+    # When True the named policy already exists (and may grant more than one
+    # KV path). Do not rewrite it — AppRole bind only.
+    reuse_policy: bool = False
 
 
 @dataclass
@@ -111,8 +114,9 @@ def build_approle_kv_lane(plan: ConstructionPlan, spec: AppRoleKVSpec) -> dict[s
             f"approved_at={plan.approved_at!r}) — refusing to build"
         )
 
-    policy_hcl = _policy_hcl(spec.kv_path, spec.kv_capabilities)
-    _run(spec.bao_bin, ["policy", "write", spec.policy_name, "-"], input_text=policy_hcl)
+    if not spec.reuse_policy:
+        policy_hcl = _policy_hcl(spec.kv_path, spec.kv_capabilities)
+        _run(spec.bao_bin, ["policy", "write", spec.policy_name, "-"], input_text=policy_hcl)
 
     _run(
         spec.bao_bin,

tests/test_executor.py

@@ -117,6 +117,29 @@ def test_build_writes_policy_approle_and_delivers_credentials(tmp_path) -> None:
     assert "secret-id-value" not in audit_text
 
 
+def test_reuse_policy_does_not_rewrite_existing_policy(tmp_path) -> None:
+    plan = _plan(tmp_path)
+    spec = _spec(tmp_path)
+    spec.reuse_policy = True
+
+    def fake_run(cmd, input=None, capture_output=True, text=True, timeout=30):
+        result = MagicMock(returncode=0, stderr="")
+        if cmd[1:3] == ["read", "-field=role_id"]:
+            result.stdout = "role-id-value\n"
+        elif "-field=secret_id" in cmd:
+            result.stdout = "secret-id-value\n"
+        else:
+            result.stdout = ""
+        return result
+
+    with patch("ops_mason.executor.subprocess.run", side_effect=fake_run) as run:
+        build_approle_kv_lane(plan, spec)
+
+    bao_cmds = [c.args[0] for c in run.call_args_list]
+    assert not any(cmd[:2] == ["bao", "policy"] for cmd in bao_cmds)
+    assert any(cmd[1:3] == ["write", "auth/approle/role/test-lane"] for cmd in bao_cmds)
+
+
 def test_build_appends_audit_record(tmp_path) -> None:
     plan = _plan(tmp_path)
     spec = _spec(tmp_path)