diff --git a/plans/audit-core-openbao-runtime-custody.md b/plans/audit-core-openbao-runtime-custody.md new file mode 100644 index 0000000..edb9bd1 --- /dev/null +++ b/plans/audit-core-openbao-runtime-custody.md @@ -0,0 +1,146 @@ +--- +id: audit-core-openbao-runtime-custody +demand_source: audit-core/workplans/AUDIT-WP-0005-T02 +consumer_repo: audit-core +credential_type: openbao-approle-kv +status: built +approved_by: "Bernd Worsch" +approved_at: "2026-08-13" +created: "2026-08-13" +updated: "2026-08-13" +--- + +# Construction plan: audit-core OpenBao runtime custody + +## 1. Demand + +audit-core on railiance01 needs durable PostgreSQL custody and a sender +registry without the founder typing OpenBao tokens or running `bao kv put`. +The founder’s remaining acts are: approve this plan, and unseal OpenBao if +it is sealed. Mechanics belong to ops-mason (structure) and ops-warden +(routing / founder desk), not to an ad-hoc terminal session. + +## 2. Existing-structure survey + +Checked 2026-08-13 against live railiance01 and the routing catalog: + +- **Database leases already exist.** `rapp-postgres` provisioned `audit_core` + with `audit_core_app` / `audit_core_migrate`, and OpenBao already vends + `database/creds/audit-core-runtime` and `database/creds/audit-core-migration` + through railiance-platform grants `rapp-postgres/audit-core-runtime` and + `rapp-postgres/audit-core-migration`. `warden route show + database-dynamic-credentials` points at that playbook. Do not create a + second database engine or a KV copy of a rotating lease. +- **ESO token already exists.** `external-secrets/openbao-audit-core-eso-token` + was minted 2026-08-13 with policy `external-secrets-audit-core` (read on + the two database cred paths and on `platform/data/workloads/audit-core/senders`). + ClusterSecretStore YAML is drafted; it is not yet applied. +- **KV senders path does not exist.** A founder `bao kv put` against + `platform/workloads/audit-core/senders` 403’d because the workstation + `~/.vault-token` is not an operator identity. That is the failure this + plan exists to retire. +- **Catalog.** `database-dynamic-credentials` already covers the lease. + There is no `audit-core-senders` entry. `openbao-api-key` is the generic + template and must not become the concrete lane. +- **Auth topology.** Same as user-engine / rapp-qonto: railiance01 ESO + cannot use CoulombCore Kubernetes auth. AppRole is the transitional + bridge; same-cluster Kubernetes auth remains the target. +- **Does an existing lane already satisfy this?** The database half, yes — + bind to the existing OpenBao database roles. The sender-registry half, + no — there is no path, no AppRole, and no catalog entry. Reuse the + email-connect / user-engine *shape* (namespace-limited store, 15m + tokens, eight uses), do not widen those identities. + +## 3. Proposed changes + +| # | Action | Object | Reuse-vs-new rationale | +|---|---|---|---| +| 1 | reuse | OpenBao roles `audit-core-runtime` / `audit-core-migration` | Already vended; a KV copy of a lease recreates the delivery gap | +| 2 | reuse | policy `external-secrets-audit-core` | Already written; covers database creds + senders read | +| 3 | create | AppRole `external-secrets-audit-core` | Replace the long-lived ESO token with the same 15m/30m/8-use bridge as user-engine and rapp-qonto | +| 4 | deliver | Secret `external-secrets/openbao-audit-core-approle` | Mason write-only delivery of role_id/secret_id; never logged | +| 5 | create | ClusterSecretStore `openbao-audit-core` | Namespace-limited to `audit-core`; AppRole auth (token store is the interim until this builds) | +| 6 | reuse | ClusterSecretStore `openbao-audit-core-database` + ExternalSecret | Namespaced VaultDynamicSecret cannot read the ESO token in `external-secrets`; CSS can. One `dataFrom` extract per lease. Applied 2026-08-13, SecretSynced | +| 7 | create | empty KV path `platform/workloads/audit-core/senders` | Structure only. Values are **not** founder-pasted: first deploy mints sender tokens in-cluster into Secret `audit-core-senders`. A later wrap-migrate (user-engine pattern) can move them into this path without printing | +| 8 | propose | ops-warden catalog `audit-core-senders` | Pointer, `warden_executes: false`, starts `draft` | +| 9 | retire | founder `bao kv put` / token-prompt as the T02 delivery path | That is the error-prone interaction this demand forbids | + +No sibling workload path is readable. Mason does not print, log, or return +sender tokens or database passwords. + +## 4. Review notes + +- **Naming:** `external-secrets-audit-core` / `openbao-audit-core` matches + email-connect and user-engine. Database grant names stay with rapp-postgres. +- **TTL/scoping:** 15-minute tokens, 30-minute max, eight uses. Store + condition is `audit-core` only. +- **Redundancy:** does not create a second database role or a KV cache of + the lease. Does not widen `openbao-email-connect` or + `openbao-user-engine-runtime`. +- **Compaction:** retires the attended `bao kv put` path as the way T02 + completes. The already-minted static ESO token is interim and is revoked + after the AppRole store is Ready. +- **Secret values:** sender tokens are generated by us, not imported from a + vendor. Founder paste-once is the wrong desk for them. Generate in-cluster; + optional later wrap-migrate into the empty KV path. +- **Founder remaining acts:** approve this plan; unseal OpenBao if sealed. + No token paste, no `bao` command review. + +## 5. Executive summary + +**One-line ask:** approve letting ops-mason finish audit-core’s OpenBao +lane (AppRole + namespace-limited store + empty senders path + catalog +pointer) so the receiver can run without you typing OpenBao tokens. + +**Who gets access:** only External Secrets, through AppRole +`external-secrets-audit-core`. Application pods keep seeing mounted +files and `AUDIT_CORE_SENDERS`. Nothing else gains a new identity. + +**To what:** read `database/creds/audit-core-runtime`, +`database/creds/audit-core-migration`, and (later) +`platform/workloads/audit-core/senders`. No sibling workload or tenant +path. + +**For how long:** each AppRole login is 15 minutes, 30 minutes maximum, +eight uses. The AppRole secret ID lasts until rotated or replaced by +same-cluster Kubernetes auth. + +**Blast radius if the credential leaks:** append to the audit trail and +read the sender registry; cannot rewrite or delete stored events (append-only +trigger). Revoke the AppRole secret ID, rotate sender tokens overlap-first, +and roll the receiver. Database leases expire on their own. + +**Cost to reverse:** delete the AppRole, its bootstrap Secret, the +ClusterSecretStore, and the empty KV path. Database roles and the +`audit_core` database stay (owned by rapp-postgres). No other consumer +is on this identity. + +**Decision:** approve, reject, or send back to phase 1. +Approval authorizes Mason build of structure and in-cluster sender +minting. It does not authorize printing any secret value. + +## 6. Build result + +Built 2026-08-13 after founder approve, via +`ops_mason.executor.build_approle_kv_lane` with `reuse_policy=True` so +the existing `external-secrets-audit-core` policy (database creds + +senders read) was not rewritten. + +- reused policy `external-secrets-audit-core`; +- created AppRole `external-secrets-audit-core` (15m / 30m / 8 uses); +- delivered role-id/secret-id into Secret + `external-secrets/openbao-audit-core-approle`; local delivery dir shredded; +- switched ClusterSecretStores `openbao-audit-core` and + `openbao-audit-core-database` to AppRole auth; both `Valid`; +- ExternalSecrets remain `SecretSynced`; receiver `/readyz` still + `custody_class=archive`; +- deleted interim Secret `external-secrets/openbao-audit-core-eso-token`; +- catalog entry `audit-core-senders` already drafted in ops-warden + (`warden route find "audit-core senders" --all`). + +Empty KV path `platform/workloads/audit-core/senders` was not written: +mason does not put values, and senders already live in-cluster. A later +wrap-migrate can create that path without a founder paste. + +No secret value was printed, committed, or logged. The 15-minute +platform-admin child used for the build was revoked. diff --git a/plans/backup-object-storage.md b/plans/backup-object-storage.md new file mode 100644 index 0000000..1513c47 --- /dev/null +++ b/plans/backup-object-storage.md @@ -0,0 +1,120 @@ +--- +id: backup-object-storage +demand_source: resource-control/workplans/RESOURCE-WP-0002-T04 +consumer_repo: rapp-postgres +credential_type: openbao-kv-founder-provision +status: built +approved_by: "Bernd Worsch" +approved_at: "2026-08-14" +created: "2026-08-14" +updated: "2026-08-14" +--- + +# Construction plan: resource-control scoped backup key + +## 1. Demand + +`RESOURCE-WP-0002` T04 needs a Scaleway API key for **backup object +storage in general**, not a Barman-only identity. Founder uses IAM +application `resource-control` and binds policy / API-key +`Scoped backup access` on the project that owns the backup bucket. +The key lives in OpenBao. External Secrets later projects it into +consumer Secrets (first: `platform-pg-backup-s3` in `databases`). +The bootstrap key cannot create IAM. ops-mason builds the empty +lane. Values never appear in git or chat. + +## 2. Existing-structure survey + +Checked 2026-08-14, retargeted the same day: + +- **KV path exists (version 1, 2026-08-14).** Founder put + `ACCESS_KEY`, `SECRET_KEY`, `DEFAULT_ORGANIZATION_ID`, + `DEFAULT_PROJECT_ID`. Not placeholders. No `APPLICATION_ID`. + The earlier draft path `…/backup/platform-pg-backup-s3` was not + built and must not be used. +- **CCR-2026-0012** now reserves + `platform/workloads/railiance/backup/object-storage`. Reuse it. +- **CCR-2026-0011** is the bootstrap key + (`platform/workloads/railiance/scaleway/bootstrap`). Different + purpose. Do not reuse those values. Do not hang that key on + application `resource-control`. +- **CCR-2026-0004** is Nextcloud/age offsite. Different provider. +- **Catalog.** No `backup-object-storage` lane. +- **Auth.** First delivery is External Secrets into `databases`. + Kubernetes auth role `external-secrets-backup-object-storage` is + the CCR-declared steady state. railiance01 may need the interim + AppRole/token pattern; that apply is after placeholders are + replaced. +- **Does an existing lane already satisfy this?** No live secret. + +## 3. Proposed changes + +| # | Action | Object | Reuse-vs-new rationale | +|---|---|---|---| +| 1 | reuse | CCR-2026-0012 path | Already reserved after retarget | +| 2 | create | policy `workload-kv-read-backup-object-storage` | File already written; apply after approval | +| 3 | create | empty KV v2 path (placeholders `xxx`) | Paste-once / UI replace has a home | +| 4 | defer | Kubernetes auth role / AppRole / ESO store | After real values exist | +| 5 | defer | catalog `backup-object-storage` | Pointer only; `warden_executes: false` until verified | +| 6 | defer | Scaleway bucket policy principal | Needs `APPLICATION_ID` of `resource-control` | + +No new rail. Scaleway operates S3. No second IAM application. + +## 4. Review notes + +- **Naming:** path is `…/backup/object-storage`. IAM application is + `resource-control`. Policy and key label are `Scoped backup access`. +- **IAM fact:** policies attach to the application, not one key. Only + backup-scoped keys belong on `resource-control`. +- **TTL/scoping:** the Scaleway key is long-lived until rotated. The + OpenBao reader is ESO, 15m Kubernetes auth TTL in the CCR. +- **Redundancy:** does not reuse the bootstrap path or the Nextcloud + offsite lane. Supersedes the Barman-named draft of the same CCR. +- **Secret values:** mason does not read, write, or log them. Founder + uses the OpenBao UI or + `railiance-platform/docs/put-backup-object-storage.md`. +- **Founder remaining acts:** approve this plan and CCR-2026-0012; + create or reuse application `resource-control`; bind policy and key + `Scoped backup access`; replace `xxx`; say “the backup key is in + bao.” + +## 5. Executive summary + +**One-line ask:** approve an OpenBao home for the resource-control +backup key so first consumer `rapp-postgres` (and later backup +consumers) can receive a projected Secret without putting keys in git +or chat. + +**Who gets access:** you (founder) to write the three fields; later, +External Secrets, first bound to namespace `databases`. + +**To what:** only +`platform/workloads/railiance/backup/object-storage` +(`ACCESS_KEY_ID`, `ACCESS_SECRET_KEY`, `APPLICATION_ID`). Not the +bootstrap path. + +**For how long:** until you rotate or revoke the Scaleway key. + +**Blast radius if the credential leaks:** read/write/delete objects +on the backup bucket. Revoke the key in the Scaleway console, delete +the KV version, rotate. + +**Cost to reverse:** delete the KV path, policy, and unused drafts. +WAL is not enabled yet, so there is no production archive to migrate. + +**Decision:** approved in chat 2026-08-14 (“Lets go”). + +## 6. Build result + +Built 2026-08-14: + +- policy `workload-kv-read-backup-object-storage` (read on the KV data + and metadata paths only) +- AppRole `external-secrets-backup-object-storage` (15m/30m, 8 uses) +- Kubernetes Secret `external-secrets/openbao-backup-object-storage-approle` + delivered; local files shredded +- ClusterSecretStore `openbao-backup-object-storage` Valid +- ExternalSecret synced `databases/platform-pg-backup-s3` (two keys, + values not read) + +Did not write Scaleway secret values. Did not enable WAL. diff --git a/plans/reef-storage-scaleway-bootstrap.md b/plans/reef-storage-scaleway-bootstrap.md new file mode 100644 index 0000000..f04d989 --- /dev/null +++ b/plans/reef-storage-scaleway-bootstrap.md @@ -0,0 +1,144 @@ +--- +id: reef-storage-scaleway-bootstrap +demand_source: resource-control/workplans/RESOURCE-WP-0002-T03 +consumer_repo: reef-storage +credential_type: openbao-kv-founder-provision +status: built +approved_by: "Bernd Worsch" +approved_at: "2026-08-14" +created: "2026-08-14" +updated: "2026-08-14" +--- + +# Construction plan: Scaleway bootstrap key for reef-storage + +## 1. Demand + +We need to create a private Scaleway Multi-AZ bucket in `nl-ams` for +postgres backup (`RESOURCE-WP-0002`). The founder has a new Scaleway +account and an API key in the usual Terraform snippet shape +(`access_key`, `secret_key`, `organization_id`, `project_id` in +`scaleway.auto.tfvars`). Those four values must live in OpenBao, not in +git or chat. ops-mason builds the empty lane. The founder pastes the +values once (desk or local tfvars ingest). reef-storage then creates the +bucket by reading OpenBao and never printing the keys. + +## 2. Existing-structure survey + +Checked 2026-08-14: + +- **KV path does not exist.** + `platform/metadata/workloads/railiance/scaleway/bootstrap` is 404. + Same for the later backup path + `platform/workloads/railiance/backup/object-storage`. +- **CCR already reserved the path.** + `railiance-platform` `CCR-2026-0011` names + `platform/workloads/railiance/scaleway/bootstrap` and the four fields. + Reuse that path. Do not invent a second one. +- **CCR-2026-0004** is Nextcloud/age offsite backup + (`platform/workloads/railiance/backup/offsite-lane`). Different + provider, different fields. Do not merge. +- **Catalog.** No `scaleway` / `s3-backup` lane. `openbao-api-key` is the + generic template and must not become this concrete key. +- **Auth.** This is an operator-workstation bootstrap key for a one-shot + (or rare) `scw` create, not an in-cluster ESO consumer. No AppRole. + Founder paste-once / local tfvars ingest is the right desk. The later + scoped Barman key (T04) is a different path and may get a workload + CCR then. +- **Does an existing lane already satisfy this?** No live secret. The + reserved CCR path is the one to build. + +Terraform snippet → OpenBao fields (same four facts, scw-native names): + +| Terraform (`scaleway.auto.tfvars`) | OpenBao field | +| --- | --- | +| `access_key` | `ACCESS_KEY` | +| `secret_key` | `SECRET_KEY` | +| `organization_id` | `DEFAULT_ORGANIZATION_ID` | +| `project_id` | `DEFAULT_PROJECT_ID` | + +## 3. Proposed changes + +| # | Action | Object | Reuse-vs-new rationale | +|---|---|---|---| +| 1 | reuse | CCR-2026-0011 path `platform/workloads/railiance/scaleway/bootstrap` | Already reserved; do not fork | +| 2 | create | empty KV v2 path (metadata only, no secret data) | Structure so paste-once / ingest has a home | +| 3 | create | policy `operator-kv-scaleway-bootstrap` (read/write that path only) | Founder desk and the bucket-create script; no sibling paths | +| 4 | reuse | founder paste-once desk + optional local tfvars ingest | ops-mason never sees values; four fields, four desk pastes or one founder-run ingest | +| 5 | propose | catalog `scaleway-bootstrap` (draft) + playbook | Pointer only; `warden_executes: false` | +| 6 | defer | AppRole / ESO / scoped backup key | T04 / CCR for `backup/object-storage`; not this plan | + +No new rail. Scaleway operates S3. + +## 4. Review notes + +- **Naming:** path follows `platform/workloads/railiance/…`. Policy + `operator-kv-scaleway-bootstrap` matches “operator workstation, not + workload ESO.” +- **TTL/scoping:** no AppRole token. The *downstream* Scaleway key is + long-lived until the founder deletes it after the scoped key exists. +- **Redundancy:** does not reuse Nextcloud offsite-lane. Does not put + org-admin keys in the Barman runtime path. +- **Compaction:** none. This is the first Scaleway lane. +- **Secret values:** mason does not read, write, or log them. Founder + uses `warden desk --act paste_once_provision` four times, or + `reef-storage/tools/ingest-scaleway-tfvars.sh` on a local 0600 + `scaleway.auto.tfvars` they already have. +- **Founder remaining acts:** approve this plan; paste four fields (or + run ingest once); say “bootstrap is in OpenBao.” + +## 5. Executive summary + +**One-line ask:** approve an OpenBao home for the four Scaleway Terraform +variables so we can create the backup bucket without putting keys in git +or chat. + +**Who gets access:** you (founder / operator token) to write the four +fields; the bucket-create script, using that same operator token, to +read them once. No new AppRole. No cluster workload. Nothing else gains +a path. + +**To what:** only +`platform/workloads/railiance/scaleway/bootstrap` +(`ACCESS_KEY`, `SECRET_KEY`, `DEFAULT_ORGANIZATION_ID`, +`DEFAULT_PROJECT_ID`). Not the later Barman key path. Not other +`platform/workloads/railiance/backup/*` secrets. + +**For how long:** the KV version stays until you rotate or delete it. +This bootstrap key should be revoked at Scaleway after the scoped bucket +key works (T04). + +**Blast radius if the credential leaks:** create/list/delete Object +Storage (and whatever else that API key can do in the project). Revoke +the key in the Scaleway console, delete the KV version, rotate to a +scoped bucket-only key. + +**Cost to reverse:** delete the KV path, draft catalog entry, and +policy. No other consumer is bound. The bucket, once created, stays +until you empty and delete it. + +**Decision:** approve, reject, or send back to phase 1. +Approval authorizes structure (empty path, policy, catalog draft) and +your paste-once / tfvars ingest. It does not authorize ops-mason to +handle the Terraform secret values. + +## 6. Build result + +Executed 2026-08-14 after founder asked to bootstrap with example +data and replace values in the OpenBao UI. + +Created (structure + placeholders only; values are the string `xxx`): + +- policy `operator-kv-scaleway-bootstrap` (read/write that one path) +- KV v2 `platform/workloads/railiance/scaleway/bootstrap` version 1 + fields `ACCESS_KEY`, `SECRET_KEY`, `DEFAULT_ORGANIZATION_ID`, + `DEFAULT_PROJECT_ID` +- custom_metadata `placeholder=true`, `ccr=CCR-2026-0011`, + `replace_in_ui=true` + +Catalog draft `scaleway-bootstrap` already proposed. + +**Founder next:** OpenBao UI → secret +`platform/workloads/railiance/scaleway/bootstrap` → create version 2 +with the real Terraform values. Then say “bootstrap is in OpenBao.” +The bucket-create script refuses `xxx` placeholders. diff --git a/src/ops_mason/executor.py b/src/ops_mason/executor.py index 622ba1e..6f1208f 100644 --- a/src/ops_mason/executor.py +++ b/src/ops_mason/executor.py @@ -53,6 +53,9 @@ class AppRoleKVSpec: delivery_dir: Path | None = None bao_bin: str = "bao" audit_log_path: Path | None = None + # When True the named policy already exists (and may grant more than one + # KV path). Do not rewrite it — AppRole bind only. + reuse_policy: bool = False @dataclass @@ -111,8 +114,9 @@ def build_approle_kv_lane(plan: ConstructionPlan, spec: AppRoleKVSpec) -> dict[s f"approved_at={plan.approved_at!r}) — refusing to build" ) - policy_hcl = _policy_hcl(spec.kv_path, spec.kv_capabilities) - _run(spec.bao_bin, ["policy", "write", spec.policy_name, "-"], input_text=policy_hcl) + if not spec.reuse_policy: + policy_hcl = _policy_hcl(spec.kv_path, spec.kv_capabilities) + _run(spec.bao_bin, ["policy", "write", spec.policy_name, "-"], input_text=policy_hcl) _run( spec.bao_bin, diff --git a/tests/test_executor.py b/tests/test_executor.py index e4c5fea..8cd09d5 100644 --- a/tests/test_executor.py +++ b/tests/test_executor.py @@ -117,6 +117,29 @@ def test_build_writes_policy_approle_and_delivers_credentials(tmp_path) -> None: assert "secret-id-value" not in audit_text +def test_reuse_policy_does_not_rewrite_existing_policy(tmp_path) -> None: + plan = _plan(tmp_path) + spec = _spec(tmp_path) + spec.reuse_policy = True + + def fake_run(cmd, input=None, capture_output=True, text=True, timeout=30): + result = MagicMock(returncode=0, stderr="") + if cmd[1:3] == ["read", "-field=role_id"]: + result.stdout = "role-id-value\n" + elif "-field=secret_id" in cmd: + result.stdout = "secret-id-value\n" + else: + result.stdout = "" + return result + + with patch("ops_mason.executor.subprocess.run", side_effect=fake_run) as run: + build_approle_kv_lane(plan, spec) + + bao_cmds = [c.args[0] for c in run.call_args_list] + assert not any(cmd[:2] == ["bao", "policy"] for cmd in bao_cmds) + assert any(cmd[1:3] == ["write", "auth/approle/role/test-lane"] for cmd in bao_cmds) + + def test_build_appends_audit_record(tmp_path) -> None: plan = _plan(tmp_path) spec = _spec(tmp_path)