ops-mason/plans/rein-openweights-openrouter-approle.md
tegwick 9bcb5819de Fix KV v2 policy path shape; live end-to-end verification succeeded
Two more real bugs found completing this lane for real:

1. platform-admin's own policy had no entry for the new reins/ mount --
   the founder's paste-once-provision write 403'd because the admin
   identity that created the mount was never granted access to operate
   on it. Fixed live (added path "reins/*" matching every other mount
   already in that policy).

2. _policy_hcl wrote the bare KV-v1-shaped path
   (reins/rein-openweights/openrouter) instead of KV v2's data/+metadata/
   sub-paths -- bao token capabilities on the bare path even reported
   full access, but the actual kv get still 403'd, because OpenBao
   evaluates the real request against the data/-prefixed path. Caught
   when the AppRole's own read failed during live verification. Fixed
   in code (now emits both data/ and metadata/ paths), locked in with a
   dedicated unit test, and re-applied to the live policy.

Live end-to-end verification succeeded after both fixes: real AppRole
login, real KV v2 read via the corrected policy, real OpenRouter call,
real commit -- with OPENROUTER_API_KEY unset the whole time. Plan status:
catalogued. glas-harness/GLAS-WP-0002-T02 is closed by this.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 01:50:18 +02:00

245 lines
14 KiB
Markdown

---
id: rein-openweights-openrouter-approle
demand_source: glas-harness/workplans/GLAS-WP-0002-T02
consumer_repo: rein-openweights
credential_type: openbao-approle-kv
status: catalogued
approved_by: "Bernd Worsch"
approved_at: "2026-07-27"
created: "2026-07-27"
updated: "2026-07-27"
---
# Construction plan: rein-openweights OpenBao AppRole
## 1. Demand
`rein-openweights` (the OpenRouter-driven rein in the glas-harness family)
needs to authenticate to OpenBao **non-interactively** — no operator OIDC
session, no human in the loop at run time — to fetch its OpenRouter API
key when it runs unattended (e.g. via a future scheduled trigger, or a
glas-harness gateway invocation with no operator present). This was
scoped as `glas-harness/GLAS-WP-0002-T02` ("Option B" — see that
workplan's discussion) and is currently blocked: the only credential path
exercised so far is the `OPENROUTER_API_KEY` env-var short-circuit in
`rein_openweights/credentials.py`; the AppRole/vault branch
(`_acquire_token`, `bao kv get`) is real code, live-verifiable, but has
never actually talked to OpenBao.
## 2. Existing-structure survey
**Checked:** `ops-warden/registry/routing/catalog.yaml` (readable directly
— no OpenBao session needed for this). **Not checked:** live `bao policy
list` / `bao auth list` / `bao secrets list` — this session has no valid
OpenBao token (`bao token lookup` returns `403`). Whoever executes phase 4
must re-verify against live state before applying anything below; this
survey is current as of what's on disk, not as of live OpenBao ACL state.
**Relevant existing catalog entry: `openrouter-llm-connect`**
(`ops-warden/registry/routing/catalog.yaml:283`) — an OpenRouter key
already exists at
`platform/workloads/activity-core/llm-connect/llm-connect-provider-secrets`
(field `OPENROUTER_API_KEY`), gated by policy
`workload-kv-read-llm-connect-provider-secrets`, `auth_method: caller's
own OpenBao token (operator OIDC via key-cape, or a token carrying
[that policy])`, marked `risk: high` ("provider API key with spend impact").
`rein-aharness`'s own mail-triage/brief-daily already reuse this exact
lane (see `binky-control/integrations/executor-worker-secrets.md`, Lane 1
— "REUSE (verified)").
**Does this existing lane satisfy the demand? Partially, by design, not
fully — reuse considered and declined:**
- The existing lane's `auth_method` is interactive-caller-shaped (OIDC /
"a token carrying the policy") — it was never scoped for a bare AppRole
login the way `agent-harness-binky-mail`'s Lane 3 was. Binding a new
AppRole to the *existing* policy is mechanically possible (Option A
from the original discussion) but would give `rein-openweights`
standing, non-interactive access to a policy explicitly flagged
`risk: high` for spend impact, shared with `activity-core`'s broader
usage — a wider blast radius than a single rein needs.
- **Recommendation: do not reuse.** Build a dedicated, narrowly-scoped
AppRole + KV path for `rein-openweights` specifically (matches
`credentials.py`'s existing default `reins/rein-openweights/openrouter`
path already in the code) — same reasoning `agent-harness-binky-mail`
used for its own dedicated lane rather than widening an existing one.
This keeps a leak scoped to exactly one consumer, one credential.
## 3. Proposed changes
| # | Action | Object | Reuse-vs-new rationale |
|---|---|---|---|
| 1 | create | KV v2 secret path `reins/rein-openweights/openrouter` (field `api_key`) | New, narrowly scoped — matches `credentials.py`'s existing default path; declined reuse of the shared `activity-core` path (see §2) |
| 2 | create | Policy `workload-kv-read-rein-openweights-openrouter` — read-only, scoped to exactly path #1 | New — mirrors the shape of `workload-kv-read-llm-connect-provider-secrets` but scoped to one path, not shared |
| 3 | create | AppRole `rein-openweights`, bound to policy #2 | New — mirrors `agent-harness-binky-mail`'s AppRole shape: `token_ttl=15m`, `token_max_ttl=30m`, bounded `token_num_uses`, `secret_id_ttl` per current posture (build-phase; see `ops-warden/wiki/WorkloadSecurityPosture.md`) |
| 4 | deliver | `role_id`/`secret_id` files, `0600`, to wherever the live verification runs (workstation or a future Railiance deployment) | New — matches `agent-harness-binky-mail`'s delivery pattern (`~/.local/agent-harness/approle-binky-mail`), analogous path for this consumer |
| 5 | propose | ops-warden catalog entry `rein-openweights-openrouter-approle` — pointer-only, `warden_executes: false`, `status: draft` | New — no existing entry covers this AppRole; `openrouter-llm-connect` entry stays as-is, untouched |
No tear-down in this plan — no existing lane is being retired or
compacted; the existing `openrouter-llm-connect` lane is explicitly kept
as-is for its current interactive-caller consumers.
## 4. Review notes (phase 2)
- **Naming convention check:** follows `<consumer>-<credential-purpose>`
shape used by `agent-harness-binky-mail` — proposed
`rein-openweights-openrouter-approle` for the AppRole/catalog id,
`workload-kv-read-rein-openweights-openrouter` for the policy (matches
the `workload-kv-read-<path-slug>` convention already visible in the
existing catalog).
- **TTL/scoping check:** `token_ttl=15m`/`token_max_ttl=30m`/bounded
`token_num_uses` matches `agent-harness-binky-mail` exactly — no reason
to diverge, this consumer's access pattern (one-shot credential fetch
per run) is identical in shape.
- **Redundancy check:** confirmed no second existing AppRole or policy
already scoped to `reins/rein-openweights/*` — this is genuinely new
structure, not a duplicate.
- **Compaction opportunity:** none identified — nothing existing becomes
redundant once this lands; the shared `activity-core` lane keeps its
own consumers.
- **Ease of use for the consumer:** matches directly — `credentials.py`
already reads `REIN_OPENWEIGHTS_APPROLE_DIR` (defaults to a
`role_id`/`secret_id` file pair) and a KV path override via
`REIN_OPENWEIGHTS_OPENROUTER_KV_PATH` (defaulting to exactly
`reins/rein-openweights/openrouter`, field `api_key`) — this plan's
path #1 and delivery #4 need **zero code changes** in
`rein-openweights` to consume; the code was already written expecting
this exact shape.
- **Posture check:** build phase (one founder-operator, pre-revenue) per
`ops-warden/wiki/WorkloadSecurityPosture.md``agent-harness-binky-mail`
used `secret_id_ttl=0` (no expiry) at this same posture; carrying that
forward here rather than adding a rotation schedule neither
`agent-harness-binky-mail` nor current posture requires. **Explicit
assumption, not a silent default** — flagged again in the executive
summary below for the approval decision to confirm or override.
## 5. Executive summary (phase 3)
**One-line ask:** create a narrowly-scoped, non-interactive credential
lane so `rein-openweights` can fetch its own OpenRouter API key at run
time without an operator present.
**Who/what gets access:** a new OpenBao AppRole named `rein-openweights`.
Nothing else — no existing role, human, or service gains anything new.
**To what:** read-only access to exactly one KV path,
`reins/rein-openweights/openrouter` (field `api_key`) — a path that does
not exist today and holds nothing else. The AppRole cannot read the
existing shared `activity-core`/`llm-connect` secret or any other path.
**For how long:** each login is a 15-minute token, renewable up to a
30-minute cap, bounded number of uses per token — matches the existing
`agent-harness-binky-mail` lane exactly. The AppRole credential itself
(`role_id`/`secret_id`) does not expire on a schedule at current
(build-phase) posture, same choice already made for
`agent-harness-binky-mail`.
**Blast radius if the credential leaks:** whoever holds a valid
`role_id`/`secret_id` pair can mint a short-lived token that reads one
OpenRouter API key — nothing else in OpenBao. Actual damage from the
OpenRouter key itself is real but bounded (API spend on that key, no
data-store or infra access) and independent of this plan (same exposure
`openrouter-llm-connect`'s existing consumers already carry).
**Cost to reverse:** delete one AppRole, one policy, one KV path — three
`bao` commands, no other lane or consumer affected. Fully isolated
blast/reversal radius by construction (that was the point of declining
reuse in §2).
**Decision needed:** approve as proposed, reject, or send back to phase 1
with feedback (e.g. different TTL, different path name, or "actually
reuse the shared lane instead").
**Decision: approved as proposed** (Bernd Worsch, 2026-07-27). Ready for
phase 4 — still blocked on a real OpenBao session existing in whatever
environment executes the build (`bao token lookup` from this workstation
returns `403`; unchanged since `GLAS-WP-0002-T02` first flagged it).
## 6. Build result (phase 4)
**Built 2026-07-27**, executed via `ops_mason.executor.build_approle_kv_lane`
against real OpenBao (`bao login -method=oidc -path=netkingdom
role=platform-admin`, `platform-admin` policy, ~1h token). Object names only
below — no secret material.
- **Mount created:** `reins/` (KV v2) — did not exist; `bao secrets list`
showed only `platform/`, `secret/`, `tenants/` at execution time. Per §2's
own caveat ("survey current as of what's on disk, not live OpenBao
ACL state"), this was a real, live discrepancy from what the approved
plan assumed. **Confirmed with the founder before mounting** (a new
top-level secrets engine is a bigger action than "one KV path" implied
in the executive summary's blast-radius framing) — approved to proceed
exactly as originally planned rather than substitute an existing mount.
- **Policy created:** `workload-kv-read-rein-openweights-openrouter`
read-only, scoped to exactly `reins/rein-openweights/openrouter`.
- **AppRole created:** `rein-openweights``token_ttl=15m`,
`token_max_ttl=30m`, `secret_id_ttl=0`.
- **Correction made post-build:** `token_num_uses` was built as `0`
(OpenBao's own default) instead of the plan's own stated `8`
(mirroring `agent-harness-binky-mail`) — `0` means *unlimited* in
OpenBao, the opposite of "bounded." Caught by checking the live
AppRole config against the reference lane immediately after building,
fixed with `bao write auth/approle/role/rein-openweights
token_num_uses=8` (verified). **Root cause fixed in code, not just
patched live:** `AppRoleKVSpec.token_num_uses` no longer has a default
in `ops_mason/executor.py` — it must be passed explicitly every time,
so this specific mistake can't recur silently on a future plan.
- **role_id/secret_id delivered** to `~/.local/rein-openweights/approle/`
(mode `0600`), matching `credentials.py`'s existing
`REIN_OPENWEIGHTS_APPROLE_DIR` default.
- **Catalog entry proposed and merged:** `ops-warden` commit `c0a50bc`,
`rein-openweights-openrouter-approle`, `status: draft` (promotes to
`active` once the founder completes paste-once-provision and
`GLAS-WP-0002-T02`'s live verification succeeds). Playbook:
`ops-warden/wiki/playbooks/rein-openweights-openrouter-approle.md`.
`ops-warden`'s full test suite (326 tests) verified green after the
addition.
**Remaining, not part of this build:** the founder still needs to paste
the real OpenRouter API key into `reins/rein-openweights/openrouter`
(field `api_key`) through ops-warden's `paste_once_provision` desk —
ops-mason built structure only, per `INTENT.md`.
## 7. Post-build corrections (found completing phases 4-onward for real)
Two more real bugs surfaced only by actually exercising the built lane,
both fixed at the root cause, not just patched live:
- **`platform-admin`'s own policy had no entry for the new `reins/`
mount.** The founder's `paste_once_provision` desk write failed
(`403`, "preflight capability check") because the admin identity that
created the mount was never granted access to operate on it — every
other KV mount (`platform/`, `secret/`, `tenants/`, ...) is explicitly
listed in `platform-admin`'s policy; `reins/*` wasn't, since the policy
predates the mount. Fixed by adding a `path "reins/*" {...}` block
matching the existing entries exactly. **Process gap, not just a code
bug:** `build_approle_kv_lane` creates the *consumer's* policy/AppRole
but never checks whether the *founder's own* admin policy can reach a
brand-new mount — worth a follow-up task to check this automatically
for any future plan that creates a new mount, not just this one.
- **The generated policy used the wrong path shape for a KV v2 mount.**
`_policy_hcl` wrote `path "reins/rein-openweights/openrouter"` — the
bare, KV-v1-shaped path. KV v2 routes actual reads/writes through
`<mount>/data/<path>` (with a separate `<mount>/metadata/<path>` for
version operations); a policy against the bare path silently denies
everything on a v2 mount. `bao token capabilities` on the bare path
had even reported full access earlier in this build — misleading,
since OpenBao evaluates the real request against the `data/`-prefixed
path, not whatever a caller checks capabilities against. Caught when
the AppRole's own `kv get` 403'd during the live `GLAS-WP-0002-T02`
verification. Fixed in `ops_mason/executor.py::_policy_hcl` (now
emits both `data/` and `metadata/` sub-paths) and re-applied to the
live policy; a dedicated unit test
(`test_policy_hcl_uses_kv_v2_data_and_metadata_paths`) locks the
correct shape in for every future plan.
**Live end-to-end verification succeeded (2026-07-27)** after both
fixes: real AppRole login → real KV v2 read via the corrected policy →
real OpenRouter API call → real 2-turn tool-calling loop → real commit
(`e30d520c79a3493aeea5e8ea4d5ec331512b781d`), with `OPENROUTER_API_KEY`
unset the whole time — the vault round trip, not the env-var
short-circuit, is what ran. `glas-harness/GLAS-WP-0002-T02` is closed by
this.
**Plan status: catalogued** — ops-warden entry promoted `draft` ->
`active` (see that repo's commit history).