docs: align scope with implemented capabilities
Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a0217e-8c4c-7383-be6b-f50a6e485306
This commit is contained in:
parent
3ca0e63bed
commit
7799716c73
4 changed files with 454 additions and 159 deletions
217
history/2026-08-23-scope-intent-capability-gap-assessment.md
Normal file
217
history/2026-08-23-scope-intent-capability-gap-assessment.md
Normal file
|
|
@ -0,0 +1,217 @@
|
|||
# Scope and Intent Capability Gap Assessment
|
||||
|
||||
Date: 2026-08-23
|
||||
Author: codex
|
||||
Compared: `SCOPE.md` against `INTENT.md` and the current implementation
|
||||
|
||||
## Executive assessment
|
||||
|
||||
The repository has a credible, tested CLI MVP, but it is not yet the complete
|
||||
secure interaction and automation layer described by `INTENT.md`.
|
||||
|
||||
The strongest implemented slice is:
|
||||
|
||||
```text
|
||||
catalog -> approval-status check -> guarded plan -> policy/AppRole apply
|
||||
-> bounded presence/capability verification -> CLI-local exec/handoff
|
||||
-> non-secret route/evidence pointer
|
||||
```
|
||||
|
||||
The weakest slice is lifecycle safety beyond that pilot chain. Rotation is
|
||||
metadata only, KV provisioning is not merge-safe for multi-field paths, KV
|
||||
revoke is destructive and incomplete, approval integration is still a pilot
|
||||
resolver rather than a production authorization contract, and service mode does
|
||||
not exist.
|
||||
|
||||
The intent remains directionally sound. The gap is maturity, not a need to move
|
||||
the system boundary: OpenBao should remain the backend; flex-auth/State Hub
|
||||
should authorize; key-cape/user-engine should own identity; ops-warden should
|
||||
route; and secrets-engine should remain the guarded workflow layer.
|
||||
|
||||
## Open-work assessment
|
||||
|
||||
The only active repository workplan is `SECRETS-WP-0006`:
|
||||
|
||||
- T05 waits on explicit per-lane approval and scoped production OpenBao apply
|
||||
authority for five reviewed existing production lanes.
|
||||
- T06 waits on successful native verification before ops-warden proxy retirement.
|
||||
|
||||
No approval or production authority was available during this assessment. The
|
||||
latest ops-warden message also parks the standalone `warden-sign` AppRole for
|
||||
break-glass use and explicitly requests no credential action. There was no safe,
|
||||
actionable live task, so documentation reconciliation was the appropriate work.
|
||||
|
||||
## Intent scorecard
|
||||
|
||||
| Intent area | Assessment | Current evidence | Material gap |
|
||||
| --- | --- | --- | --- |
|
||||
| Decision first | Partial | Privileged CLI handlers require a resolved status and reject missing, unapproved, or superseded decisions. | Resolution accepts tracked local fixtures, checks only coarse status, has no freshness/purpose/actor semantics, and does not enforce actual dual control. The five new CCR strings do not resolve through the State Hub UUID endpoint. |
|
||||
| OpenBao enforces; engine orchestrates | Substantially aligned | Backend operations are isolated in `openbao.py`; catalog, plan, guards, delivery, routing, and evidence are separate modules. | Only the local `bao`/`vault` subprocess adapter exists. Some safety depends on local validation rather than backend-bound per-request authorization. |
|
||||
| Stage-aware privilege | Partial | Stage names, stage mismatch rejection, build/test prefixes, static stage policies, and consumer policy/AppRole naming exist. | Production has no local owner-prefix invariant (`STAGE_PREFIX["prod"]` is empty). Static bootstrap stage policies and supplied credentials remain wider and longer-lived than per-action identity. |
|
||||
| No values in coordination surfaces | Partial, with a concrete implementation risk | Catalog validation rejects secret-looking inline values; evidence is scrubbed; CLI output avoids values; exec output is redacted. | `kv_put` passes the raw value in the local `bao` subprocess argument vector. Exec/verification parse the complete KV path response in process memory. These do not put values in Git/State Hub, but they fall short of the strongest custody boundary in the intent. |
|
||||
| Least friction without broad power | Partial | Human-readable dry-runs, one-command exec, route pointers, and exact consumer policies reduce hand-written OpenBao work. | Routine production still needs manual decision and credential bridging. There is no OIDC/service login, service API, or durable request orchestration. |
|
||||
| Bootstrap honestly, then harden | Partial | Bootstrap files are mode checked, rejected inside Git worktrees, documented, and tracked in the hardening backlog. | Bootstrap mode is still part of the live CLI; one historical cleanup/revocation record remains incomplete; tracked local approval mirrors remain usable for production commands. |
|
||||
| Prefer exec-time delivery | Partial | `exec-env` and `npm-config` are implemented, including temporary npm config cleanup and child-output redaction. | `exec-file` and `wrapped` are schema-only. Scoped login tokens are not explicitly revoked after exec/verify, and child delivery is local CLI execution only. |
|
||||
| Auditable and reversible | Weak/partial | Scrubbed JSONL evidence and best-effort State Hub progress events exist; OpenBao remains the audit source. | Many failure paths produce no evidence, State Hub posting silently drops failures, actors are not cryptographically bound, no audit-report command exists, and KV revoke is destructive rather than reversible. |
|
||||
| Typed multi-application catalog | Substantially aligned as metadata | Multiple KV lanes, an auth-capability lane, explicit org/repo/stage/consumer/field data, existing-mount semantics, and high-risk ownership checks are implemented. | This is not runtime multi-tenancy. Consumer claims are descriptive and are not authenticated or authorized by this process. |
|
||||
| CLI and service API | CLI achieved; API absent | The documented CLI surface exists and the pilot flow has integration coverage. | No daemon, API, queue, scheduler, UI, remote protocol, or concurrency model exists. |
|
||||
| Lifecycle: establish/access/rotate/deactivate | Establish/access partial; lifecycle incomplete | Apply, provision, verify, exec, handoff, and two forms of revoke exist. | No rotate command, compromise/suspend state, provider rotation, lease/accessor lifecycle, rollback orchestration, or safe soft deactivation exists. |
|
||||
|
||||
## Highest-priority gaps
|
||||
|
||||
### P0 — Provisioning is unsafe for shared multi-field paths
|
||||
|
||||
`provision_from_file` reads one declared field and calls `OpenBaoClient.kv_put`.
|
||||
That adapter invokes:
|
||||
|
||||
```text
|
||||
bao kv put <mount>/<path> <field>=<value>
|
||||
```
|
||||
|
||||
Two consequences matter:
|
||||
|
||||
1. `kv put` replaces the data at a KV v2 path rather than merging sibling
|
||||
fields. Updating one field can remove the others.
|
||||
2. The raw value is present in the local subprocess argument vector while the
|
||||
command runs.
|
||||
|
||||
This is acceptable only as a constrained pilot/bootstrap primitive on a
|
||||
single-field or disposable path. It must not be used on the newly admitted
|
||||
issue-core, reuse-surface, Forgejo, or email paths. A production design needs a
|
||||
merge-safe update mechanism that does not expose values through argv, plus
|
||||
tests proving sibling preservation and failure cleanup.
|
||||
|
||||
### P0 — KV revoke is destructive, incomplete, and inconsistent with dry-run
|
||||
|
||||
For a KV lane, live `revoke` deletes all KV metadata/versions but leaves the
|
||||
consumer AppRole and ACL policy. The dry-run text says it would delete both KV
|
||||
metadata and the AppRole, which the live path does not do.
|
||||
|
||||
This does not meet the intent's reversible lifecycle principle. The command
|
||||
needs explicit operations with different approvals, for example:
|
||||
|
||||
- suspend delivery by deleting/disabling AppRole credentials;
|
||||
- deactivate consumer policy/auth;
|
||||
- revoke leases/tokens by accessor;
|
||||
- delete current value versions;
|
||||
- irreversibly destroy KV metadata only under an explicit destruction action.
|
||||
|
||||
Until then, KV revoke should be treated as destructive path deletion, not a
|
||||
general revocation or deactivation workflow.
|
||||
|
||||
### P0 — Production approval is not yet a complete authorization contract
|
||||
|
||||
The resolver accepts `resolved`, `approved`, or `accepted` status and absence of
|
||||
`superseded_by`. It does not validate action type, lane/stage binding, actor,
|
||||
expiry, allowed fields, operation, approval count, or flex-auth result.
|
||||
`dual-control` is a catalog label but is not enforced as two approvals.
|
||||
|
||||
Tracked local decision mirrors are useful for demos and historical pilots but
|
||||
are too permissive as a steady-state production authority source. Production
|
||||
should require a canonical decision object bound to the exact action and should
|
||||
fail closed when State Hub/flex-auth cannot prove that binding.
|
||||
|
||||
## Important hardening gaps
|
||||
|
||||
### P1 — Steady-state authentication is absent
|
||||
|
||||
The CLI accepts a token from an environment variable or a mode-0600 bootstrap
|
||||
file. It does not perform OpenBao OIDC, workload/service auth, short-lived
|
||||
issuer exchange, or per-action identity binding. This keeps routine production
|
||||
work dependent on attended external credential bridging.
|
||||
|
||||
### P1 — Verification is narrower than the intent
|
||||
|
||||
KV positive verification checks one field, defaulting to the first. Route
|
||||
readiness also checks only the first field, so a multi-field lane can appear
|
||||
ready while sibling fields are missing. KV negative verification uses a fixed
|
||||
garbage token, not a real unrelated workload identity. Provider behavior and
|
||||
application health are outside the check.
|
||||
|
||||
High-risk lane acceptance should verify every declared field, use a real
|
||||
unrelated identity or capabilities proof, capture OpenBao audit request ids,
|
||||
and, where appropriate, call a value-safe provider/application smoke.
|
||||
|
||||
### P1 — Delivery token lifecycle is implicit
|
||||
|
||||
Exec and verification create AppRole Secret IDs and login tokens. Secret ID
|
||||
use counts and token TTL/use bounds limit them, but the code does not revoke the
|
||||
issued token explicitly on success, child failure, interruption, or verification
|
||||
failure. Evidence does not include a safe lease/accessor lifecycle.
|
||||
|
||||
### P1 — Evidence is informative, not a durable audit workflow
|
||||
|
||||
Local JSONL is append-only by convention, not tamper-evident. State Hub posting
|
||||
is best-effort and silently ignored on error. Apply/provision/handoff failures
|
||||
are generally recorded only if a caller adds separate evidence; there is no
|
||||
transaction id joining State Hub evidence to OpenBao audit entries, and no
|
||||
`audit` command summarizes a lane.
|
||||
|
||||
### P1 — Lifecycle metadata is not lifecycle automation
|
||||
|
||||
Catalog entries require rotation/deactivation descriptions and owners, which is
|
||||
valuable governance metadata. The engine does not execute those plans, manage
|
||||
provider credential overlap, coordinate consumers, maintain compromised or
|
||||
deactivated state, or verify rollback.
|
||||
|
||||
## Longer-term product gaps
|
||||
|
||||
### P2 — Service mode
|
||||
|
||||
The intent describes a CLI and service API. Only the CLI exists. A service
|
||||
should wait until approval binding, auth, delivery token lifecycle, evidence,
|
||||
and destructive-operation semantics are hardened; exposing today's CLI directly
|
||||
would amplify its pilot shortcuts.
|
||||
|
||||
### P2 — Runtime tenancy and consumer identity
|
||||
|
||||
The catalog can describe many applications and logical owners, but the engine
|
||||
does not authenticate a tenant, resolve claims, isolate concurrent callers, or
|
||||
enforce per-tenant quotas/policies. The current product is multi-application
|
||||
metadata and path orchestration, not a multi-tenant service.
|
||||
|
||||
### P2 — Backend abstraction
|
||||
|
||||
OpenBao-specific behavior is isolated reasonably well, but there is only one
|
||||
subprocess backend. “OpenBao as the initial enforcement backend” remains an
|
||||
architectural option, not a demonstrated portable backend contract.
|
||||
|
||||
## Current evolution phase
|
||||
|
||||
Against the phases in `INTENT.md`:
|
||||
|
||||
1. **Bootstrap:** implemented and still active.
|
||||
2. **MVP:** achieved for the whynot-design npm pilot.
|
||||
3. **Stage separation:** implemented in catalog/plan/static policy form, but
|
||||
steady-state stage authentication and stronger production owner binding are
|
||||
incomplete.
|
||||
4. **Hardening:** started; existing-mount admission, risk classification,
|
||||
exact-field guards, bounded AppRoles, and fail-closed decisions exist. The P0
|
||||
and P1 gaps above remain.
|
||||
5. **Service mode:** not started.
|
||||
|
||||
## Recommended next sequence
|
||||
|
||||
1. Keep `SECRETS-WP-0006-T05` limited to approved metadata apply and explicit
|
||||
per-field, positive/negative verification. Do not provision or revoke the
|
||||
existing shared paths through the current KV lifecycle commands.
|
||||
2. Fix provisioning transport and merge semantics, with sibling-preservation
|
||||
and process-argument tests.
|
||||
3. Split KV suspend/deactivate/destroy semantics and make dry-run match live
|
||||
behavior.
|
||||
4. Replace coarse/local production decision resolution with an action-bound,
|
||||
expiring, dual-control-capable authorization contract.
|
||||
5. Add steady-state OIDC/service auth and explicit child-token revocation.
|
||||
6. Strengthen multi-field/readiness and unrelated-identity verification, then
|
||||
correlate evidence with OpenBao audit request ids.
|
||||
7. Implement rotation state and wrapped/file delivery before designing service
|
||||
mode.
|
||||
|
||||
## Conclusion
|
||||
|
||||
`INTENT.md` still describes the right destination, but the accurate present
|
||||
tense is a guarded OpenBao CLI MVP with one proven native workload pilot and
|
||||
several reviewed-but-not-live production lane plans. The repository should not
|
||||
yet advertise itself as a service API, complete lifecycle engine, runtime
|
||||
multi-tenant system, wrapped-delivery broker, or production-safe general KV
|
||||
updater.
|
||||
Loading…
Add table
Add a link
Reference in a new issue