coulomb/secrets-engine
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(mvp): working secrets-engine CLI for the whynot-design npm publish lane

Browse source 
Implements SECRETS-WP-0002 end to end as a uv-managed Python package:

- catalog: non-secret lane registry + strict validator (build/test/prod)
- stage roles + OpenBao ACL policies; guards refuse wildcards, sys/, identity/,
  admin names, and cross-stage paths before any backend call
- plan/apply: dry-run-first, idempotent policy + approle apply, decision-gated
- decisions: State Hub lookup with local-fixture fallback; non-secret evidence
  to JSONL + hub progress, scrubbed of any value
- provision/verify: mode-0600 file import + generated test values; positive/
  negative checks that never print the value
- exec delivery: `exec --catalog ... -- npm publish` injects the token via a
  temp .npmrc for the child only, cleaned up on exit/failure/interrupt
- ops-warden routing contract + hardening backlog docs
- 34 tests incl. live OpenBao integration; scripts/demo-e2e.sh runs the full
  chain against a throwaway bao dev server

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
tegwick 2026-06-28 12:28:45 +02:00
parent 58c24cff53
commit a852d3f1ff
47 changed files with 3743 additions and 122 deletions
Download patch file Download diff file Expand all files Collapse all files

78
docs/cli.md Normal file
View file

@ -0,0 +1,78 @@
# secrets-engine CLI
## Install
```bash
uv venv && uv pip install -e ".[dev]"
source .venv/bin/activate
secrets-engine --version
```
## Environment
| Var | Default | Purpose |
| --- | --- | --- |
| `BAO_ADDR` | `http://127.0.0.1:8200` | OpenBao address |
| `BAO_TOKEN` | _(unset)_ | OpenBao token (or use `--bootstrap-token-file`) |
| `SECRETS_ENGINE_HUB_URL` | `http://127.0.0.1:8000` | State Hub for decisions + evidence (empty to disable) |
| `SECRETS_ENGINE_CATALOG` | `./catalog` | catalog directory |
| `SECRETS_ENGINE_EVIDENCE` | `./.evidence` | local non-secret evidence log |
## Commands
```text
secrets-engine catalog list
secrets-engine catalog show <catalog-id>
secrets-engine decision inspect <decision-or-ccr-id>
secrets-engine plan <ref> --stage <build|test|prod>
secrets-engine apply <ref> --stage <stage> [--dry-run] [--bootstrap-token-file F]
secrets-engine provision <catalog-id> --stage <stage> --field NAME (--from-file F | --generate)
secrets-engine verify <catalog-id> --field NAME [--positive] [--negative]
secrets-engine exec --catalog <catalog-id> [--field NAME] [--mode auto|npm-config|exec-env] -- CMD...
secrets-engine route <catalog-id> [--json]
secrets-engine revoke <catalog-id> [--dry-run]
```
`<ref>` is a catalog id or a decision/CCR ref (matched against
`approval.decision_ref`). `plan` and `apply --dry-run` never mutate OpenBao.
## Exit codes
| Code | Meaning |
| --- | --- |
| 0 | success |
| 2 | catalog error (missing/invalid lane) |
| 3 | decision error (unapproved / superseded / missing) |
| 4 | policy guard error (out-of-stage / wildcard / broad admin) |
| 5 | backend error (OpenBao unreachable / failed) |
| 6 | provisioning error (bad file mode / inside repo / missing field) |
| 7 | verification failed |
| 8 | delivery error |
## End-to-end demo
```bash
SECRETS_ENGINE_HUB_URL="" bash scripts/demo-e2e.sh
```
Boots a throwaway in-memory OpenBao dev server and runs the whole pilot chain:
plan → apply → provision → verify(+/-) → exec (npm-config injection) → route →
revoke. Nothing is persisted; the token is a throwaway local string.
## Pilot: whynot-design npm publish
```bash
# 1. inspect the approved decision
secrets-engine decision inspect whynot-design-npm-publish
# 2. preview the OpenBao changes (no mutation)
secrets-engine plan whynot-design-npm-publish --stage prod
# 3. apply policy + approle
secrets-engine apply whynot-design-npm-publish --stage prod
# 4. provision the token from a mode-0600 file OUTSIDE the repo
secrets-engine provision whynot-design-npm-publish --stage prod \
--field npm_token --from-file ~/.secrets-engine/whynot.token
# 5. prove access without printing the value
secrets-engine verify whynot-design-npm-publish --field npm_token --positive --negative
# 6. publish with the token injected into the child only
secrets-engine exec --catalog whynot-design-npm-publish -- npm publish
```

58
docs/hardening-backlog.md Normal file
View file

@ -0,0 +1,58 @@
# Hardening Backlog — Exit From Bootstrap Mode
The MVP runs in **bootstrap mode**: stage roles may be driven by temporary
root-created OpenBao tokens read from mode-0600 files. That is acceptable setup
material, **not** the steady state. This backlog tracks the work to retire it.
The MVP is honest about this: bootstrap mode is a documented, bounded phase with
explicit revocation tasks — not a hidden permanent security posture.
## H0 — Revoke outstanding bootstrap tokens (always-on hygiene)
Every minted bootstrap token has a revocation task. Track each here:
| Token file | Stage | Minted | TTL | Revoked? |
| --- | --- | --- | --- | --- |
| `~/.secrets-engine/bootstrap/prod.token` | prod | (n/a — demo uses dev server) | 1h | n/a |
Revoke: `bao token revoke -accessor <accessor>` then `shred -u <file>`.
## H1 — Replace bootstrap token files with OIDC / service auth
- Stand up an OpenBao auth method (OIDC or AppRole bound to a workload identity)
for each stage role.
- secrets-engine logs in via that method instead of reading a token file.
- Remove `--bootstrap-token-file` from the steady-state path (keep only for true
break-glass, heavily audited).
## H2 — Response-wrapped handoff
- Add a `wrapped` delivery mode using OpenBao response wrapping for operator
handoff flows where exec-time injection does not fit.
## H3 — Production dual-control
- Require two-person approval (`approval.model: dual-control`) for prod value
provisioning before automating raw-value writes beyond the pilot.
## H4 — Rotation & lifecycle states
- Implement `rotate`, and explicit `compromised` / `deactivated` lane states with
evidence, beyond the current `revoke` (metadata delete).
## H5 — Audit report command
- `secrets-engine audit <catalog-id>` summarizing non-secret evidence (decision,
applies, provisions, verifications, execs, revokes) for a lane.
## H6 — API service mode
- Expose the stabilized CLI semantics as a local service API for ops-warden,
CI, agents, and a future UI — **after** the CLI contract is proven.
## Exit criteria for "bootstrap mode is over"
- No steady-state flow reads a bootstrap token file.
- Every stage role authenticates through OIDC/service auth.
- Production provisioning beyond the pilot requires dual-control.
- Rotation and deactivation are first-class, evidenced operations.

22
docs/netkingdom-security-infrastructure.md Normal file
View file

@ -0,0 +1,22 @@
# NetKingdom Security Infrastructure Boundary
The canonical document lives in the NetKingdom repository:
```text
net-kingdom/docs/secrets-engine-security-infrastructure-boundary.md
```
Local checkout path:
```text
/home/worsch/net-kingdom/docs/secrets-engine-security-infrastructure-boundary.md
```
This secrets-engine file is intentionally only a pointer. The canonical document
belongs to NetKingdom because it defines cross-system security infrastructure
responsibilities and boundaries across OpenBao, flex-auth, user-engine/key-cape,
ops-warden, ops-bridge, info-tech-canon, State Hub, and agents.
secrets-engine consumes that boundary and implements the secrets workflow,
catalog, stage policies, OpenBao apply/delivery mechanics, and evidence model
that the canonical document assigns to it.

79
docs/openbao-stage-roles.md Normal file
View file

@ -0,0 +1,79 @@
# OpenBao Stage Roles & Bootstrap
secrets-engine talks to OpenBao through three **stage roles**, never as root or a
platform admin. Each role is confined to one stage's KV prefix and a small,
explicit capability set.
| Role | Policy file | KV prefix | May do | May NOT do |
| --- | --- | --- | --- | --- |
| `secrets-engine-build` | `policies/secrets-engine-build.hcl` | `secret/.../build/` | manage build metadata + generated test values; own `se-build-*` policies/roles | touch test/prod, `sys/*`, `auth/token/*`, `identity/*`, act as root |
| `secrets-engine-test` | `policies/secrets-engine-test.hcl` | `secret/.../test/` | manage test metadata + values; run positive/negative checks; own `se-test-*` | touch build/prod, `sys/*`, `auth/token/*`, `identity/*`, act as root |
| `secrets-engine-prod` | `policies/secrets-engine-prod.hcl` | owner-scoped prod lanes | apply approved prod ACL policies + approle roles; write approved values; own `se-prod-*` | reach build/test, edit the stage roles themselves, admin `sys/auth`, `sys/mounts`, `identity/*`, `auth/token/*`, act as root |
The **product requirement** is the stage *distinction* and the *denials*, not the
exact policy names — those may evolve as info-tech-canon hardens.
## Negative guarantees (enforced two ways)
1. **In OpenBao** — each policy carries explicit `deny` stanzas for other stages
and for `sys/*`, `auth/token/*`, `identity/*`.
2. **In secrets-engine** — `roles.assert_path_in_stage()` and
`roles.assert_policy_safe()` reject any *plan* that would touch another
stage's prefix, use a wildcard, name itself like an admin policy, or carry a
capability outside `create/read/update/delete/list`. A bad plan fails closed
**before** any OpenBao call.
Run the negative checks:
```bash
pytest tests/test_guards.py -q
```
## Bootstrap token files (temporary)
Until OIDC/service auth exists (see the hardening backlog), a **platform-root
operator** may mint a short-lived OpenBao token for a stage role and hand the
agent the *file path* — never the token value.
Requirements for a bootstrap token file:
- mode **0600**, owned by the invoking user;
- located **outside** any Git worktree (e.g. `~/.secrets-engine/bootstrap/`);
- named by role + environment only, never by value
(e.g. `prod.token`, not `npm_abc123.token`);
- **revocable** and **temporary** — tracked with a revocation task;
- referenced by path, e.g.:
```bash
secrets-engine apply whynot-design-npm-publish --stage prod \
--bootstrap-token-file ~/.secrets-engine/bootstrap/prod.token
```
secrets-engine refuses a bootstrap token file that is group/other-readable or
that lives inside the repo worktree (`provision`/`apply` check `st_mode & 0o077`).
### Minting (operator, one-time, root context)
```bash
# Write each stage policy into OpenBao.
bao policy write secrets-engine-build policies/secrets-engine-build.hcl
bao policy write secrets-engine-test policies/secrets-engine-test.hcl
bao policy write secrets-engine-prod policies/secrets-engine-prod.hcl
# Mint a short-lived token for one stage role, store mode-0600 outside the repo.
install -m 700 -d ~/.secrets-engine/bootstrap
bao token create -policy=secrets-engine-prod -ttl=1h -field=token \
> ~/.secrets-engine/bootstrap/prod.token
chmod 600 ~/.secrets-engine/bootstrap/prod.token
```
### Revocation (always have a path)
```bash
# Revoke by accessor (preferred) or delete the file when the TTL is short.
bao token revoke -accessor <accessor>
shred -u ~/.secrets-engine/bootstrap/prod.token
```
Every minted bootstrap token MUST have a corresponding revocation task in the
hardening backlog (`docs/hardening-backlog.md`).

62
docs/ops-warden-routing-contract.md Normal file
View file

@ -0,0 +1,62 @@
# ops-warden → secrets-engine Routing Contract
ops-warden issues **SSH certificates only**. Every other credential need (API
keys, provider tokens, DB passwords, npm publish tokens) routes to
**secrets-engine**, which is OpenBao-backed. ops-warden must never request, hold,
cache, or vend a raw secret value. A route result is a **pointer**, not a key.
## What ops-warden calls
```bash
secrets-engine route <catalog-id> --json
```
## What it returns (the contract)
```json
{
"catalog_id": "whynot-design-npm-publish",
"owner": "whynot-design",
"stage": "prod",
"decision_status": "resolved",
"decision_ref": "whynot-design-npm-publish",
"review_url": "http://127.0.0.1:8000/decisions/<id>",
"metadata_applied": true,
"value_present": true,
"ready": true,
"next_command": "secrets-engine exec --catalog whynot-design-npm-publish -- <command...>",
"missing": ""
}
```
| Field | Meaning |
| --- | --- |
| `decision_status` | `resolved`/`approved` => approved; `missing`/`pending` => not yet |
| `metadata_applied` | OpenBao ACL policy + approle exist for the lane |
| `value_present` | the secret value has been provisioned (boolean only — value never read) |
| `ready` | approved **and** applied **and** provisioned |
| `next_command` | the single safe command the caller should run next |
| `missing` | the one human/provisioning step still outstanding |
## Guarantees
- **No value crosses this boundary.** `route` reports a boolean `value_present`,
derived from a metadata/presence check — it never reads the secret.
- **Actionable when not ready.** If a lane is unapproved, unapplied, or
unprovisioned, `next_command` + `missing` tell the caller exactly what to do.
- **Idempotent / read-only.** `route` performs no mutation.
## whynot-design retry flow
1. whynot-design CI needs a publish token → asks ops-warden.
2. ops-warden runs `secrets-engine route whynot-design-npm-publish --json`.
3. If `ready=false`, it surfaces `missing` + `next_command` to the human (e.g.
"needs approved decision" or "needs provisioning").
4. Once `ready=true`, the workload runs
`secrets-engine exec --catalog whynot-design-npm-publish -- npm publish`.
## Anti-patterns (forbidden)
- ops-warden `POST /messages/` asking for `NPM_TOKEN` / `OPENROUTER_API_KEY`.
- Caching `value_present` as if it were the value.
- Inventing `warden secret` / `warden bao` — they do not exist.