From 70371649afc4baf1c058d6a140877fb08f3388ca Mon Sep 17 00:00:00 2001 From: tegwick Date: Sun, 23 Aug 2026 14:15:42 +0200 Subject: [PATCH] Harden production authorization and service auth Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a0217e-8c4c-7383-be6b-f50a6e485306 --- README.md | 6 +- SCOPE.md | 53 ++- WORK-RECORDS.md | 2 +- docs/cli.md | 11 +- docs/hardening-backlog.md | 18 +- docs/service-auth.md | 28 ++ scripts/demo-e2e.sh | 2 + scripts/npm-publish-demo.sh | 1 + src/secrets_engine/audit.py | 2 +- src/secrets_engine/authorization.py | 314 ++++++++++++++++++ src/secrets_engine/cli.py | 40 ++- src/secrets_engine/evidence.py | 67 +++- src/secrets_engine/service_auth.py | 217 ++++++++++++ tests/test_action_authorization.py | 225 +++++++++++++ tests/test_audit.py | 18 + tests/test_decisions.py | 34 ++ tests/test_privileged_cli_evidence.py | 25 ++ tests/test_redact_evidence.py | 64 +++- tests/test_service_auth.py | 159 +++++++++ ...-WP-0007-production-lifecycle-hardening.md | 36 +- 20 files changed, 1268 insertions(+), 54 deletions(-) create mode 100644 docs/service-auth.md create mode 100644 src/secrets_engine/authorization.py create mode 100644 src/secrets_engine/service_auth.py create mode 100644 tests/test_action_authorization.py create mode 100644 tests/test_service_auth.py diff --git a/README.md b/README.md index 9a82794..bbf0571 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,7 @@ SECRETS_ENGINE_HUB_URL="" bash scripts/demo-e2e.sh - ops-warden routing contract: [docs/ops-warden-routing-contract.md](docs/ops-warden-routing-contract.md) - Hardening backlog (exit bootstrap mode): [docs/hardening-backlog.md](docs/hardening-backlog.md) - Existing-lane catalog admission: [docs/catalog-admission.md](docs/catalog-admission.md) +- KeyCape service-auth consumer boundary: [docs/service-auth.md](docs/service-auth.md) The implementation is a Python package (`src/secrets_engine/`). OpenBao is reached only through the `bao` CLI adapter (`openbao.py`); the rest of the code @@ -68,7 +69,8 @@ speaks in lanes and guarded plans. workplans, or normal logs. - OpenBao is the backend custody and audit authority. - Build, test, and production have separate policy boundaries. -- Production actions require approved decisions except explicit break-glass - flows. +- Production live actions fail closed until the durable State Hub + action-authorization endpoint is available; local approval mirrors are + throwaway-demo material only. - Temporary bootstrap OpenBao credentials must live outside repos, use mode 0600, be revocable, and be removed after narrower auth is working. diff --git a/SCOPE.md b/SCOPE.md index e569da0..104400a 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -32,10 +32,16 @@ provider, credential broker, or general secrets API. ### Decision-gated planning and OpenBao metadata apply - Resolves a catalog lane by catalog id or `approval.decision_ref`. -- Resolves an approval from State Hub by id, with tracked local YAML mirrors as - a pilot/offline fallback. -- Fails privileged live commands closed when a required decision is missing, - unapproved, or superseded. Plans and `apply --dry-run` remain non-mutating. +- Resolves legacy lane decisions from State Hub by id, with tracked local YAML + mirrors retained only for non-production and explicit throwaway demos. +- Fails every production live command closed while State Hub lacks the durable + action-authorization endpoint. A local mirror can unlock a prod-labeled lane + only when an explicit unsafe-demo switch, disabled Hub URL, and loopback + OpenBao target are all present. Plans and `apply --dry-run` remain usable. +- Builds and validates the flex-auth `ActionAuthorization` profile, including + exact lane/stage/action/target/actor/purpose matching, bounded validity, + State Hub authority, request digest and decision binding, accepted policy + package/version, and an independently required distinct-approver threshold. - Renders guarded OpenBao plans for exact consumer ACL policies and AppRoles. - Applies policy and AppRole metadata idempotently. Existing mounts render a non-mutating check and are never created by apply. @@ -107,9 +113,10 @@ cannot be recovered through that AppRole. status, metadata/value-presence booleans, missing declared field names, readiness, and a safe next command. Every declared field must be present. - Records scrubbed local JSONL evidence and posts a minimal State Hub progress - event on a best-effort basis. Each requested State Hub delivery receives an - append-only local `delivered`, `failed`, or `skipped-no-topic` companion - record. + event on a best-effort basis. Posts carry stable idempotency/source headers. + Each requested State Hub delivery receives an append-only local `delivered`, + `queued`, `failed`, or `skipped-no-topic` companion record; edge-relay queued + receipts retain only the non-secret outbox id. - Every live privileged CLI handler records an attempt before lane-approval resolution and a terminal success, verification failure, rejection, interruption, or typed backend/input failure. Failure evidence contains the @@ -119,9 +126,25 @@ cannot be recovered through that AppRole. delivery outcomes. It never re-emits arbitrary evidence detail. - Keeps OpenBao audit logs as the backend source of truth. -State Hub evidence delivery is not queued or transactional; the local receipt -makes failure visible but does not replay it. Route and audit do not replace -exact-action authorization or OpenBao audit logs. +The engine recognizes edge-relay queued receipts but does not own or initiate +outbox replay. Direct State Hub failure remains locally visible. Route and audit +do not replace exact-action authorization or OpenBao audit logs. + +### Steady-state service-auth scaffold + +- Implements an explicit KeyCape `client_credentials` exchange for the + accepted `secrets-engine-openbao` service identity. +- Reads the confidential-client secret only from a mode-0600 file outside Git, + sends it through HTTP Basic authentication, and rejects ID/refresh tokens. +- Preflights the exact issuer, subject, audience, principal type, tenant, role, + scope, assurance, RS256 algorithm, and 15-minute lifetime. The in-memory JWT + is excluded from object representations and is renewed at the three-minute + boundary. +- Never retries into or falls back to bootstrap, operator, or AppRole auth. + +This provider is deliberately not connected to OpenBao. Signature verification +and token issuance remain with the platform-owned exact-bound OpenBao JWT role, +whose mount/role contract is still outstanding. Treat their output as operational guidance, not complete attestation for high-risk lanes. @@ -181,14 +204,18 @@ verification, `approle-login` is auth-capability handoff metadata, and ## Not Implemented - A service API, daemon, UI, queue, scheduler, or remote multi-user service. -- OpenBao OIDC/service-auth login for steady-state secrets-engine operation. +- OpenBao JWT login and platform materialization for the implemented KeyCape + service-auth provider. - Native `exec-file` or response-wrapped delivery. - Provider-side rotation or coordinated multi-consumer rollout. - First-class rotate, compromise, reactivate, lease-status, or audit report commands; lifecycle operations currently execute plans without persistent lane state. -- Dual-control enforcement beyond accepting the catalog label. -- Direct flex-auth evaluation, claim validation, or identity authentication. +- Resolution of a durable State Hub action authorization and wiring its + validated approval threshold to each production handler. +- Direct flex-auth evaluation, JWT signature verification, or identity + authentication. KeyCape claims receive only a consumer preflight; OpenBao is + responsible for cryptographic JWT validation. - Runtime tenancy isolation; `org`, `repo`, consumers, and stages are catalog metadata plus local path/policy guards, not a tenant control plane. - Management or health verification of ESO, Kubernetes Secrets, deployments, diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index c5b1e05..5bf81c8 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -52,7 +52,7 @@ | task | SECRETS-WP-0007-T01 | done | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | | task | SECRETS-WP-0007-T02 | done | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | | task | SECRETS-WP-0007-T03 | progress | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | -| task | SECRETS-WP-0007-T04 | wait | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | +| task | SECRETS-WP-0007-T04 | progress | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | | task | SECRETS-WP-0007-T05 | progress | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | | task | SECRETS-WP-0007-T06 | progress | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | | task | SECRETS-WP-0007-T07 | wait | — | workplans/SECRETS-WP-0007-production-lifecycle-hardening.md | diff --git a/docs/cli.md b/docs/cli.md index d19eb05..138755c 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -38,6 +38,7 @@ secrets-engine --version | `SECRETS_ENGINE_HUB_URL` | `http://127.0.0.1:8000` | State Hub for decisions + evidence (empty to disable) | | `SECRETS_ENGINE_CATALOG` | `./catalog` | catalog directory | | `SECRETS_ENGINE_EVIDENCE` | `./.evidence` | local non-secret evidence log | +| `SECRETS_ENGINE_UNSAFE_DEMO` | _(unset)_ | allow a prod-labeled lane only when Hub is disabled and OpenBao is loopback; throwaway demos only | ## Commands @@ -67,7 +68,11 @@ var the token is injected under, per the netkingdom publication-scope policy `` is a catalog id or a decision/CCR ref (matched against `approval.decision_ref`). `plan` and `apply --dry-run` never mutate OpenBao. For decision-gated lanes they may render with `decision: ` when the -approval object is not reachable; non-dry-run `apply` remains decision-gated. +approval object is not reachable. Production live commands remain disabled +until State Hub exposes the durable exact-action authorization object. A legacy +local decision is accepted for a prod-labeled lane only with +`SECRETS_ENGINE_UNSAFE_DEMO=1`, an empty Hub URL, and loopback OpenBao; the demo +scripts set those three conditions themselves. `handoff` is for `kind: auth-capability` lanes such as `warden-sign`. It mints a fresh AppRole `secret_id` and writes `role_id` plus `secret_id` to caller-chosen @@ -109,8 +114,8 @@ and evidence stores only a short accessor fingerprint plus cleanup outcome. counts, canonical decision references, session cleanup outcomes, and State Hub delivery outcomes for one cataloged lane. Its parser allowlists those fields and does not echo arbitrary JSONL detail. State Hub failures are recorded locally as -append-only companion receipts; they are visible but are not automatically -replayed. +append-only companion receipts. Edge-relay queued receipts and their non-secret +outbox ids are recorded too; replay remains an operator/State Hub responsibility. Live `apply`, `provision`, `verify`, `handoff`, `exec`, `revoke`, `suspend`, and `deactivate` share one evidence guard. It records an attempt before lane approval diff --git a/docs/hardening-backlog.md b/docs/hardening-backlog.md index e40b76e..b16c526 100644 --- a/docs/hardening-backlog.md +++ b/docs/hardening-backlog.md @@ -31,8 +31,11 @@ contents in this repo. ## H1 — Replace bootstrap token files with OIDC / service auth -- Stand up an OpenBao auth method (OIDC or AppRole bound to a workload identity) - for each stage role. +- Implemented consumer scaffold: exact KeyCape client-credentials exchange, + claim/lifetime preflight, strict client-secret file input, renewal boundary, + and no implicit fallback. +- Remaining: railiance-platform stands up the exact-bound OpenBao JWT auth + mount/role and publishes its non-secret connection contract. - secrets-engine logs in via that method instead of reading a token file. - Remove `--bootstrap-token-file` from the steady-state path (keep only for true break-glass, heavily audited). @@ -44,8 +47,11 @@ contents in this repo. ## H3 — Production dual-control -- Require two-person approval (`approval.model: dual-control`) for prod value - provisioning before automating raw-value writes beyond the pilot. +- Implemented consumer validator: exact action request, decision digest/binding, + validity, State Hub provenance, and caller-selected minimum distinct approval + count. +- Remaining: State Hub stores/serves the durable object and production handlers + resolve it; all production live actions currently fail closed. ## H4 — Rotation & lifecycle states @@ -59,7 +65,9 @@ contents in this repo. - Implemented: `secrets-engine audit ` summarizes allowlisted local action/result, decision, session-cleanup, and State Hub delivery evidence. -- Remaining: durable/queued State Hub delivery and OpenBao audit request-id +- Implemented: idempotency/source headers and edge-relay queued-receipt/outbox-id + visibility. +- Remaining: operator replay verification and OpenBao audit request-id correlation. ## H6 — API service mode diff --git a/docs/service-auth.md b/docs/service-auth.md new file mode 100644 index 0000000..1ca6a39 --- /dev/null +++ b/docs/service-auth.md @@ -0,0 +1,28 @@ +# KeyCape service-auth consumer boundary + +secrets-engine now implements the consumer half of KeyCape's accepted +`secrets-engine-openbao` service-auth contract. It performs one explicit HTTPS +`client_credentials` exchange, keeps the resulting JWT in memory, and validates +the non-cryptographic contract before any future OpenBao login: + +- subject `service:secrets-engine`, audience/client `secrets-engine-openbao`; +- service principal in `tenant:coulomb`, role `secrets-engine`; +- exact `openbao:login` scope and KeyCape AAL1 client-secret assurance; +- RS256 declaration, bounded issue/expiry timestamps, maximum 15-minute life, + and renewal when no more than three minutes remain; +- no ID token, refresh token, implicit retry provider, or fallback identity. + +The confidential-client secret must be an explicitly selected mode-0600 file +outside every Git worktree. It is sent with HTTP Basic authentication and never +placed in a request body, command argument, evidence record, or object +representation. + +This is a scaffold, not a live OpenBao authentication path. JWT payload parsing +does not prove a signature. railiance-platform still owns the exact OpenBao JWT +auth mount/role, issuer keys, claim bindings, token policy, TTL/use limits, and +cryptographic verification. Until that contract is materialized, the provider +is not selected by the CLI and bootstrap/AppRole behavior is not used as an +implicit fallback. + +Canonical provider contract: +`key-cape/docs/openbao-service-auth-contract.md` (reviewed 2026-08-23). diff --git a/scripts/demo-e2e.sh b/scripts/demo-e2e.sh index dae5f5f..e42c10f 100755 --- a/scripts/demo-e2e.sh +++ b/scripts/demo-e2e.sh @@ -19,6 +19,8 @@ BAO_BIN="$(command -v bao)" WORK="$(mktemp -d)" export BAO_ADDR="http://127.0.0.1:8270" export BAO_TOKEN="se-demo-root" +export SECRETS_ENGINE_HUB_URL="" # offline; uses local decision fixture +export SECRETS_ENGINE_UNSAFE_DEMO="1" # required for live prod-lane demos # Keep secret material OUTSIDE the repo worktree (engine enforces this). TOKENFILE="$WORK/whynot.token" NEGATIVE_TOKENFILE="$WORK/unrelated.token" diff --git a/scripts/npm-publish-demo.sh b/scripts/npm-publish-demo.sh index f02d32c..5d09053 100755 --- a/scripts/npm-publish-demo.sh +++ b/scripts/npm-publish-demo.sh @@ -20,6 +20,7 @@ WORK="$(mktemp -d)" export BAO_ADDR="http://127.0.0.1:8271" export BAO_TOKEN="se-npm-demo-root" export SECRETS_ENGINE_HUB_URL="" # offline; uses local decision fixture +export SECRETS_ENGINE_UNSAFE_DEMO="1" # required for live prod-lane demos TOKENFILE="$WORK/whynot.token" PKG="$WORK/pkg" diff --git a/src/secrets_engine/audit.py b/src/secrets_engine/audit.py index b18ab82..cbd6a86 100644 --- a/src/secrets_engine/audit.py +++ b/src/secrets_engine/audit.py @@ -12,7 +12,7 @@ from typing import Any _LABEL = re.compile(r"^[a-z][a-z0-9-]{0,79}$") _DECISION_REF = re.compile(r"^[A-Z][A-Z0-9-]{2,80}$") -_DELIVERY_RESULTS = {"delivered", "failed", "skipped-no-topic"} +_DELIVERY_RESULTS = {"delivered", "failed", "queued", "skipped-no-topic"} _VERIFY_RESULT = re.compile(r"^(positive|negative):(pass|fail)$") _ERROR_RESULT = re.compile( r"^failed-(Catalog|Decision|PolicyGuard|Backend|Provisioning|Verification|Delivery)Error$" diff --git a/src/secrets_engine/authorization.py b/src/secrets_engine/authorization.py new file mode 100644 index 0000000..e3bba9d --- /dev/null +++ b/src/secrets_engine/authorization.py @@ -0,0 +1,314 @@ +"""Fail-closed consumer validation for flex-auth action authorizations. + +The canonical contract is flex-auth revision c473f19. State Hub does not yet +provide the durable authoritative endpoint, so this module validates supplied +objects but does not resolve or enable production actions by itself. +""" +from __future__ import annotations + +import hashlib +import json +import uuid +from dataclasses import dataclass +from datetime import datetime, timezone +from typing import Any + +from secrets_engine.catalog import CatalogEntry +from secrets_engine.errors import DecisionError + +SCHEMA_VERSION = "0.1" +AUTHORITY = "state-hub" + + +@dataclass(frozen=True) +class ValidatedActionAuthorization: + authorization_id: str + decision_id: str + action: str + subject_id: str + expires_at: str + + +def build_action_request( + entry: CatalogEntry, + action: str, + *, + subject_id: str, + subject_type: str, + purpose: str, + fields: list[str] | tuple[str, ...] = (), + policy_targets: list[str] | tuple[str, ...] = (), + auth_targets: list[str] | tuple[str, ...] = (), + request_id: str = "", +) -> dict[str, Any]: + """Build the exact normalized secrets-engine profile for flex-auth.""" + if not action or not subject_id or not subject_type or not purpose: + raise DecisionError( + "action request requires action, subject id/type, and purpose" + ) + request: dict[str, Any] = {} + if request_id: + request["id"] = request_id + request.update( + { + "subject": {"id": subject_id, "type": subject_type}, + "action": action, + "resource": { + "id": f"catalog:{entry.id}", + "type": "secret-catalog-lane", + "system": "secrets-engine", + "attributes": { + "stage": entry.stage, + "fields": sorted(set(fields)), + "policy_targets": sorted(set(policy_targets)), + "auth_targets": sorted(set(auth_targets)), + }, + }, + "context": {"purpose": purpose}, + } + ) + return request + + +def _required_dict(container: dict[str, Any], name: str) -> dict[str, Any]: + value = container.get(name) + if not isinstance(value, dict): + raise DecisionError(f"action authorization requires object '{name}'") + return value + + +def _required_text(container: dict[str, Any], name: str) -> str: + value = container.get(name) + if not isinstance(value, str) or not value: + raise DecisionError(f"action authorization requires non-empty '{name}'") + return value + + +def _parse_time(value: object, name: str) -> datetime: + if not isinstance(value, str): + raise DecisionError(f"action authorization requires timestamp '{name}'") + try: + parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError as e: + raise DecisionError(f"action authorization has invalid timestamp '{name}'") from e + if parsed.tzinfo is None: + raise DecisionError(f"action authorization timestamp '{name}' needs timezone") + return parsed.astimezone(timezone.utc) + + +def _sorted_map(value: object) -> dict[str, Any]: + if not isinstance(value, dict): + return {} + return {key: _canonical_map_value(value[key]) for key in sorted(value)} + + +def _canonical_map_value(value: Any) -> Any: + if isinstance(value, dict): + return _sorted_map(value) + if isinstance(value, list): + return [_canonical_map_value(item) for item in value] + return value + + +def _subject_ref(value: object) -> dict[str, Any]: + if not isinstance(value, dict): + raise DecisionError("action authorization subject must be an object") + subject: dict[str, Any] = {"id": _required_text(value, "id")} + for name in ("type", "tenant"): + if value.get(name): + subject[name] = _required_text(value, name) + if value.get("attributes") is not None: + subject["attributes"] = _sorted_map(value.get("attributes")) + return subject + + +def _resource_ref(value: object) -> dict[str, Any]: + if not isinstance(value, dict): + raise DecisionError("action authorization resource must be an object") + resource: dict[str, Any] = {"id": _required_text(value, "id")} + for name in ("type", "system", "tenant"): + if value.get(name): + resource[name] = _required_text(value, name) + if value.get("attributes") is not None: + resource["attributes"] = _sorted_map(value.get("attributes")) + return resource + + +def canonical_check_request(request: object) -> dict[str, Any]: + """Match Go encoding/json field order used by flex-auth request digests.""" + if not isinstance(request, dict): + raise DecisionError("action authorization request must be an object") + canonical: dict[str, Any] = {} + if request.get("id"): + canonical["id"] = _required_text(request, "id") + if request.get("tenant"): + canonical["tenant"] = _required_text(request, "tenant") + canonical["subject"] = _subject_ref(request.get("subject")) + canonical["action"] = _required_text(request, "action") + canonical["resource"] = _resource_ref(request.get("resource")) + if request.get("context") is not None: + canonical["context"] = _sorted_map(request.get("context")) + if request.get("caring_context") is not None: + canonical["caring_context"] = _canonical_map_value( + request.get("caring_context") + ) + if request.get("policy_version"): + canonical["policy_version"] = _required_text(request, "policy_version") + return canonical + + +def request_digest(request: object) -> str: + canonical = canonical_check_request(request) + encoded = json.dumps( + canonical, ensure_ascii=False, separators=(",", ":") + ).encode("utf-8") + return "sha256:" + hashlib.sha256(encoded).hexdigest() + + +def _require_exact_target_sets(request: dict[str, Any]) -> None: + resource = _required_dict(request, "resource") + attributes = resource.get("attributes", {}) + if not isinstance(attributes, dict): + raise DecisionError("action authorization resource attributes must be an object") + for name in ("fields", "policy_targets", "auth_targets"): + values = attributes.get(name, []) + if not isinstance(values, list) or not all( + isinstance(item, str) and item for item in values + ): + raise DecisionError(f"action authorization target set '{name}' is invalid") + if values != sorted(set(values)): + raise DecisionError( + f"action authorization target set '{name}' must be sorted and unique" + ) + + +def validate_action_authorization( + envelope: object, + expected_request: object, + *, + accepted_policy_packages: set[str], + accepted_policy_versions: set[str], + minimum_approval_count: int = 1, + now: datetime | None = None, +) -> ValidatedActionAuthorization: + """Validate exact request binding and dual control; never parse prose.""" + if minimum_approval_count < 1: + raise DecisionError("minimum approval count must be positive") + if not accepted_policy_packages or not accepted_policy_versions: + raise DecisionError("accepted flex-auth policy package/version is required") + if not isinstance(envelope, dict): + raise DecisionError("action authorization must be an object") + if envelope.get("schema_version") != SCHEMA_VERSION: + raise DecisionError("unsupported action authorization schema version") + authorization_id = _required_text(envelope, "id") + try: + parsed_authorization_id = uuid.UUID(authorization_id) + except ValueError as e: + raise DecisionError("action authorization id must be a canonical UUID") from e + if str(parsed_authorization_id) != authorization_id: + raise DecisionError("action authorization id must be a canonical UUID") + if envelope.get("status") != "approved": + raise DecisionError("action authorization status is not approved") + if envelope.get("superseded_by"): + raise DecisionError("action authorization is superseded") + provenance = _required_dict(envelope, "provenance") + if provenance.get("authority") != AUTHORITY: + raise DecisionError("action authorization authority is not State Hub") + + request = canonical_check_request(envelope.get("request")) + expected = canonical_check_request(expected_request) + _require_exact_target_sets(request) + _require_exact_target_sets(expected) + if request != expected: + raise DecisionError("action authorization request does not exactly match action") + + validity = _required_dict(envelope, "validity") + expires = _parse_time(validity.get("expires_at"), "expires_at") + not_before = ( + _parse_time(validity.get("not_before"), "not_before") + if validity.get("not_before") is not None + else None + ) + current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc) + if not_before is not None and current < not_before: + raise DecisionError("action authorization window has not started") + if current >= expires: + raise DecisionError("action authorization has expired") + + approvals = _required_dict(envelope, "approvals") + required_count = approvals.get("required_count") + entries = approvals.get("entries") + if not isinstance(required_count, int) or required_count < 1: + raise DecisionError("action authorization approval count is invalid") + if required_count < minimum_approval_count: + raise DecisionError("action authorization approval threshold is insufficient") + if not isinstance(entries, list): + raise DecisionError("action authorization approval entries are invalid") + approvers: set[str] = set() + for entry in entries: + if not isinstance(entry, dict): + raise DecisionError("action authorization approval entry is invalid") + subject_id = _required_text(entry, "subject_id") + approved_at = _parse_time(entry.get("approved_at"), "approved_at") + if approved_at > current or approved_at >= expires: + raise DecisionError("action authorization approval time is outside window") + if not_before is not None and approved_at < not_before: + raise DecisionError("action authorization approval time is outside window") + if subject_id in approvers: + raise DecisionError("action authorization contains duplicate approver") + approvers.add(subject_id) + if len(approvers) < required_count: + raise DecisionError("action authorization has insufficient distinct approvals") + + decision = _required_dict(envelope, "decision") + if decision.get("effect") != "allow": + raise DecisionError("flex-auth decision effect is not allow") + decision_id = _required_text(decision, "id") + if request.get("id") and decision.get("request_id") != request["id"]: + raise DecisionError("flex-auth decision request id does not match request") + binding = _required_dict(decision, "binding") + bound_request: dict[str, Any] = {} + if binding.get("tenant"): + bound_request["tenant"] = binding["tenant"] + bound_request.update( + { + "subject": binding.get("subject"), + "action": binding.get("action"), + "resource": binding.get("resource"), + "context": binding.get("context", {}), + } + ) + expected_bound: dict[str, Any] = {} + if request.get("tenant"): + expected_bound["tenant"] = request["tenant"] + expected_bound.update( + { + "subject": request["subject"], + "action": request["action"], + "resource": request["resource"], + "context": request.get("context", {}), + } + ) + if canonical_check_request(bound_request) != canonical_check_request( + expected_bound + ): + raise DecisionError("flex-auth decision binding does not match request") + if binding.get("request_digest") != request_digest(request): + raise DecisionError("flex-auth request digest does not match request") + if _subject_ref(decision.get("subject")) != request["subject"]: + raise DecisionError("flex-auth decision subject does not match request") + if _resource_ref(decision.get("resource")) != request["resource"]: + raise DecisionError("flex-auth decision resource does not match request") + decision_provenance = _required_dict(decision, "provenance") + if decision_provenance.get("policy_package") not in accepted_policy_packages: + raise DecisionError("flex-auth policy package is not accepted") + if decision_provenance.get("policy_version") not in accepted_policy_versions: + raise DecisionError("flex-auth policy version is not accepted") + + return ValidatedActionAuthorization( + authorization_id=authorization_id, + decision_id=decision_id, + action=request["action"], + subject_id=request["subject"]["id"], + expires_at=expires.isoformat(), + ) diff --git a/src/secrets_engine/cli.py b/src/secrets_engine/cli.py index 50720cb..4f25520 100644 --- a/src/secrets_engine/cli.py +++ b/src/secrets_engine/cli.py @@ -21,8 +21,10 @@ Every privileged action is decision-gated and writes non-secret evidence. from __future__ import annotations import argparse +import os import sys from pathlib import Path +from urllib.parse import urlparse from secrets_engine import __version__ from secrets_engine.apply import apply_plan @@ -83,8 +85,28 @@ def _privileged_evidence( ) -def _require_lane_approval(cfg: Config, entry): - """Resolve and enforce the lane approval for a privileged live action.""" +def _unsafe_local_demo_enabled(cfg: Config) -> bool: + """Return true only for an explicit, offline, loopback-only demo.""" + host = (urlparse(cfg.bao_addr).hostname or "").lower() + return ( + os.environ.get("SECRETS_ENGINE_UNSAFE_DEMO") == "1" + and not cfg.hub_url + and host in {"127.0.0.1", "localhost", "::1"} + ) + + +def _require_lane_approval(cfg: Config, entry, action: str = ""): + """Resolve approval for a live action, failing production closed. + + The durable State Hub action-authorization endpoint is not available yet. + Production therefore cannot rely on a coarse lane decision. The one narrow + exception is an explicit offline demo against a loopback OpenBao instance. + """ + if entry.stage == "prod" and not _unsafe_local_demo_enabled(cfg): + raise DecisionError( + f"production action '{action or 'unknown'}' requires a durable " + "State Hub action authorization; live production remains disabled" + ) if not entry.approval_required(): return None decision = resolve_decision( @@ -207,7 +229,7 @@ def cmd_apply(cfg: Config, args) -> int: return 0 with _privileged_evidence(cfg, entry, "apply") as evidence: - decision = _require_lane_approval(cfg, entry) + decision = _require_lane_approval(cfg, entry, "apply") evidence.mark_approved(decision) plan = build_plan( entry, args.stage, decision_id=decision.id if decision else "" @@ -236,7 +258,7 @@ def cmd_provision(cfg: Config, args) -> int: raise ProvisioningError( f"lane '{entry.id}' is stage '{entry.stage}', not '{args.stage}'" ) - decision = _require_lane_approval(cfg, entry) + decision = _require_lane_approval(cfg, entry, "provision") evidence.mark_approved(decision) client = OpenBaoClient.resolve( cfg.bao_addr, bootstrap_token_file=args.bootstrap_token_file @@ -269,7 +291,7 @@ def cmd_verify(cfg: Config, args) -> int: "negative_requested": negative, }, ) as evidence: - decision = _require_lane_approval(cfg, entry) + decision = _require_lane_approval(cfg, entry, "verify") evidence.mark_approved(decision) client = OpenBaoClient.resolve( cfg.bao_addr, bootstrap_token_file=args.bootstrap_token_file @@ -345,7 +367,7 @@ def cmd_handoff(cfg: Config, args) -> int: raise ProvisioningError( f"lane '{entry.id}' is {entry.kind}; handoff needs auth-capability" ) - decision = _require_lane_approval(cfg, entry) + decision = _require_lane_approval(cfg, entry, "handoff") evidence.mark_approved(decision) client = OpenBaoClient.resolve( cfg.bao_addr, bootstrap_token_file=args.bootstrap_token_file @@ -397,7 +419,7 @@ def cmd_exec(cfg: Config, args) -> int: }, ) as evidence: # require approval + readiness before running. - decision = _require_lane_approval(cfg, entry) + decision = _require_lane_approval(cfg, entry, "exec") evidence.mark_approved(decision) if not args.command: from secrets_engine.errors import DeliveryError @@ -476,7 +498,7 @@ def cmd_revoke(cfg: Config, args) -> int: with _privileged_evidence( cfg, entry, "revoke", detail={"operation": plan.operation} ) as evidence: - decision = _require_lane_approval(cfg, entry) + decision = _require_lane_approval(cfg, entry, "deactivate") evidence.mark_approved(decision) client = OpenBaoClient.resolve( cfg.bao_addr, bootstrap_token_file=args.bootstrap_token_file @@ -524,7 +546,7 @@ def cmd_lifecycle(cfg: Config, args) -> int: "live destroy is disabled until an exact-action destruction " "approval contract is available; use --dry-run to inspect targets" ) - decision = _require_lane_approval(cfg, entry) + decision = _require_lane_approval(cfg, entry, args.operation) evidence.mark_approved(decision) client = OpenBaoClient.resolve( cfg.bao_addr, bootstrap_token_file=args.bootstrap_token_file diff --git a/src/secrets_engine/evidence.py b/src/secrets_engine/evidence.py index 5915e31..14660b6 100644 --- a/src/secrets_engine/evidence.py +++ b/src/secrets_engine/evidence.py @@ -47,6 +47,7 @@ class EvidenceWriter: topic_id: str = "" workstream_id: str = "" author: str = "secrets-engine" + repo_slug: str = "secrets-engine" actor: str = field(default_factory=lambda: os.environ.get("USER", "unknown")) def __post_init__(self) -> None: @@ -91,8 +92,13 @@ class EvidenceWriter: } self._append_local(record) if hub_requested: - delivery_result = self._post_hub( - action, result, catalog_id, stage, decision_id + delivery = self._post_hub( + action, + result, + catalog_id, + stage, + decision_id, + record_id=record_id, ) # Append-only companion evidence makes an unavailable State Hub # visible without rewriting or delaying the primary local record. @@ -102,23 +108,32 @@ class EvidenceWriter: "related_record_id": record_id, "ts": datetime.now(timezone.utc).isoformat(), "action": "evidence-delivery", - "result": delivery_result, + "result": delivery.status, "actor": self.actor, "catalog_id": catalog_id, "stage": stage, "decision_id": decision_id, - "detail": {}, + "detail": {"outbox_id": delivery.outbox_id} + if delivery.outbox_id + else {}, "hub_delivery_requested": False, } ) return record def _post_hub( - self, action: str, result: str, catalog_id: str, stage: str, decision_id: str - ) -> str: + self, + action: str, + result: str, + catalog_id: str, + stage: str, + decision_id: str, + *, + record_id: str, + ) -> "HubDelivery": """Best-effort progress note; return a non-secret delivery outcome.""" if not self.topic_id: - return "skipped-no-topic" + return HubDelivery("skipped-no-topic") summary = f"secrets-engine {action}: {result}" if catalog_id: summary += f" [{catalog_id}{'/' + stage if stage else ''}]" @@ -133,17 +148,47 @@ class EvidenceWriter: if decision_id: payload["detail"] = {"decision_id": decision_id, "catalog_id": catalog_id} try: + idempotency_key = f"secrets-engine:{record_id}" req = urllib.request.Request( self.hub_url.rstrip("/") + "/progress/", data=json.dumps(payload).encode(), - headers={"Content-Type": "application/json"}, + headers={ + "Content-Type": "application/json", + "Idempotency-Key": idempotency_key, + "X-StateHub-Source-Agent": self.author, + "X-StateHub-Repo-Slug": self.repo_slug, + }, method="POST", ) - urllib.request.urlopen(req, timeout=3).read() - return "delivered" + response = urllib.request.urlopen(req, timeout=3) + body = response.read() + status = getattr(response, "status", 200) + if status == 202: + try: + receipt = json.loads(body or b"{}") + except (json.JSONDecodeError, TypeError): + receipt = {} + if isinstance(receipt, dict) and receipt.get("queued") is True: + outbox_id = receipt.get("outbox_id", "") + if isinstance(outbox_id, str): + try: + if str(uuid.UUID(outbox_id)) == outbox_id: + return HubDelivery("queued", outbox_id=outbox_id) + except ValueError: + pass + return HubDelivery("failed") + return HubDelivery("delivered") except (urllib.error.URLError, OSError, ValueError): # Hub being offline must never block secret work or leak anything. - return "failed" + return HubDelivery("failed") + + +@dataclass(frozen=True) +class HubDelivery: + """Non-secret outcome returned by State Hub or its edge relay.""" + + status: str + outbox_id: str = "" @dataclass diff --git a/src/secrets_engine/service_auth.py b/src/secrets_engine/service_auth.py new file mode 100644 index 0000000..b465b62 --- /dev/null +++ b/src/secrets_engine/service_auth.py @@ -0,0 +1,217 @@ +"""Explicit KeyCape service-JWT provider for future OpenBao JWT login. + +This module implements the accepted KeyCape consumer contract without wiring it +into OpenBao yet. The platform owner still needs to publish the exact OpenBao +JWT auth mount and role. Keeping provider selection separate prevents an auth +failure from falling back to bootstrap, operator, or AppRole credentials. + +JWT parsing here is a claim preflight, not signature verification. OpenBao must +verify the RS256 signature against the configured issuer before issuing a token. +""" +from __future__ import annotations + +import base64 +import binascii +import json +from dataclasses import dataclass, field +from datetime import datetime, timezone +from pathlib import Path +from typing import Any, Callable +from urllib.error import HTTPError, URLError +from urllib.parse import urlencode +from urllib.request import Request, urlopen + +from secrets_engine.errors import BackendError +from secrets_engine.openbao import read_strict_token_file + +CLIENT_ID = "secrets-engine-openbao" +SUBJECT = "service:secrets-engine" +PRINCIPAL_TYPE = "service" +TENANT = "tenant:coulomb" +ROLE = "secrets-engine" +SCOPE = "openbao:login" +MAX_TOKEN_SECONDS = 15 * 60 +RENEW_WINDOW_SECONDS = 3 * 60 + + +@dataclass(frozen=True) +class KeyCapeServiceAuthConfig: + token_url: str + issuer: str + client_secret_file: Path + client_id: str = CLIENT_ID + subject: str = SUBJECT + audience: str = CLIENT_ID + tenant: str = TENANT + required_role: str = ROLE + scope: str = SCOPE + timeout_seconds: float = 10.0 + + def __post_init__(self) -> None: + if not self.token_url.startswith("https://"): + raise BackendError("KeyCape token URL must use HTTPS") + if not self.issuer.startswith("https://"): + raise BackendError("KeyCape issuer must use HTTPS") + if self.client_id != CLIENT_ID or self.audience != CLIENT_ID: + raise BackendError("KeyCape service client/audience must match accepted contract") + if ( + self.subject != SUBJECT + or self.tenant != TENANT + or self.required_role != ROLE + or self.scope != SCOPE + ): + raise BackendError("KeyCape service identity claims must match accepted contract") + if self.timeout_seconds <= 0: + raise BackendError("KeyCape timeout must be positive") + + +@dataclass(frozen=True) +class ServiceJWT: + token: str = field(repr=False) + issued_at: int + expires_at: int + claims: dict[str, Any] = field(repr=False) + + def needs_renewal(self, now: datetime | None = None) -> bool: + current = int((now or datetime.now(timezone.utc)).timestamp()) + return self.expires_at - current <= RENEW_WINDOW_SECONDS + + +def _decode_segment(value: str, label: str) -> dict[str, Any]: + try: + padded = value + "=" * (-len(value) % 4) + decoded = base64.urlsafe_b64decode(padded.encode("ascii")) + parsed = json.loads(decoded) + except (UnicodeEncodeError, binascii.Error, json.JSONDecodeError) as e: + raise BackendError(f"KeyCape JWT has invalid {label}") from e + if not isinstance(parsed, dict): + raise BackendError(f"KeyCape JWT {label} must be an object") + return parsed + + +def preflight_service_jwt( + token: str, + config: KeyCapeServiceAuthConfig, + *, + now: datetime | None = None, +) -> ServiceJWT: + """Validate non-cryptographic JWT shape/claims before OpenBao login.""" + parts = token.split(".") + if len(parts) != 3 or not all(parts): + raise BackendError("KeyCape access token is not a compact JWT") + header = _decode_segment(parts[0], "header") + claims = _decode_segment(parts[1], "payload") + if header.get("alg") != "RS256": + raise BackendError("KeyCape JWT algorithm is not RS256") + + exact_claims = { + "iss": config.issuer, + "sub": config.subject, + "aud": config.audience, + "principal_type": PRINCIPAL_TYPE, + "tenant": config.tenant, + "groups": [], + } + for name, expected in exact_claims.items(): + if claims.get(name) != expected: + raise BackendError(f"KeyCape JWT claim '{name}' does not match contract") + if claims.get("roles") != [config.required_role]: + raise BackendError("KeyCape JWT roles do not match contract") + if claims.get("scope") != config.scope: + raise BackendError("KeyCape JWT scope does not match contract") + + assurance = claims.get("assurance") + if not isinstance(assurance, dict) or ( + assurance.get("aal") != "AAL1" + or assurance.get("method") != "client_secret" + or assurance.get("mfa") is not False + or assurance.get("source") != "key-cape" + ): + raise BackendError("KeyCape JWT assurance does not match contract") + + issued_at = claims.get("iat") + expires_at = claims.get("exp") + if ( + not isinstance(issued_at, int) + or isinstance(issued_at, bool) + or not isinstance(expires_at, int) + or isinstance(expires_at, bool) + ): + raise BackendError("KeyCape JWT iat/exp must be integer timestamps") + current = int((now or datetime.now(timezone.utc)).timestamp()) + if issued_at > current + 60: + raise BackendError("KeyCape JWT issue time is in the future") + if expires_at <= current: + raise BackendError("KeyCape JWT has expired") + if expires_at <= issued_at or expires_at - issued_at > MAX_TOKEN_SECONDS: + raise BackendError("KeyCape JWT lifetime exceeds accepted 15-minute bound") + return ServiceJWT( + token=token, + issued_at=issued_at, + expires_at=expires_at, + claims=dict(claims), + ) + + +Transport = Callable[..., Any] + + +@dataclass(frozen=True) +class KeyCapeServiceAuthProvider: + config: KeyCapeServiceAuthConfig + transport: Transport = field(default=urlopen, repr=False, compare=False) + + def exchange(self, *, now: datetime | None = None) -> ServiceJWT: + """Perform one client-credentials exchange; never retry or fall back.""" + secret = read_strict_token_file( + self.config.client_secret_file, + purpose="KeyCape client secret", + ) + basic = base64.b64encode( + f"{self.config.client_id}:{secret}".encode("utf-8") + ).decode("ascii") + body = urlencode( + {"grant_type": "client_credentials", "scope": self.config.scope} + ).encode("ascii") + request = Request( + self.config.token_url, + data=body, + method="POST", + headers={ + "Authorization": f"Basic {basic}", + "Content-Type": "application/x-www-form-urlencoded", + "Accept": "application/json", + }, + ) + try: + response = self.transport(request, timeout=self.config.timeout_seconds) + with response: + status = getattr(response, "status", 200) + raw = response.read() + except HTTPError as e: + raise BackendError(f"KeyCape token exchange failed with HTTP {e.code}") from e + except (URLError, TimeoutError, OSError) as e: + raise BackendError("KeyCape token exchange failed") from e + if status != 200: + raise BackendError(f"KeyCape token exchange failed with HTTP {status}") + try: + payload = json.loads(raw) + except (UnicodeDecodeError, json.JSONDecodeError) as e: + raise BackendError("KeyCape token response is not valid JSON") from e + if not isinstance(payload, dict): + raise BackendError("KeyCape token response must be an object") + if payload.get("id_token") or payload.get("refresh_token"): + raise BackendError("KeyCape service exchange returned a forbidden extra token") + token = payload.get("access_token") + if not isinstance(token, str) or not token: + raise BackendError("KeyCape token response has no access token") + if str(payload.get("token_type", "")).lower() != "bearer": + raise BackendError("KeyCape token response type is not Bearer") + expires_in = payload.get("expires_in") + if ( + not isinstance(expires_in, int) + or isinstance(expires_in, bool) + or not 0 < expires_in <= MAX_TOKEN_SECONDS + ): + raise BackendError("KeyCape token response lifetime is outside contract") + return preflight_service_jwt(token, self.config, now=now) diff --git a/tests/test_action_authorization.py b/tests/test_action_authorization.py new file mode 100644 index 0000000..e69b821 --- /dev/null +++ b/tests/test_action_authorization.py @@ -0,0 +1,225 @@ +import copy +from datetime import datetime, timezone + +import pytest + +from secrets_engine.authorization import ( + build_action_request, + request_digest, + validate_action_authorization, +) +from secrets_engine.catalog import validate_entry +from secrets_engine.errors import DecisionError +from tests.test_catalog import VALID + + +NOW = datetime(2026, 8, 23, 10, 5, tzinfo=timezone.utc) + + +def _request(): + entry = validate_entry(copy.deepcopy(VALID)) + return build_action_request( + entry, + "deactivate", + subject_id="user:alice", + subject_type="Human", + purpose="contract-test", + fields=["api_token"], + policy_targets=[entry.policy_name], + auth_targets=[entry.role_name], + request_id="check:test-lane-deactivate", + ) + + +def _envelope(): + request = _request() + return { + "schema_version": "0.1", + "id": "8bfc20be-47a4-4fb0-97a2-bf0a920afad8", + "status": "approved", + "request": request, + "validity": { + "not_before": "2026-08-23T10:00:00Z", + "expires_at": "2026-08-23T10:15:00Z", + }, + "approvals": { + "required_count": 2, + "entries": [ + { + "subject_id": "user:alice", + "approved_at": "2026-08-23T10:01:00Z", + }, + { + "subject_id": "user:bob", + "approved_at": "2026-08-23T10:02:00Z", + }, + ], + }, + "decision": { + "id": "decision:test-lane-deactivate", + "request_id": request["id"], + "effect": "allow", + "resource": copy.deepcopy(request["resource"]), + "subject": copy.deepcopy(request["subject"]), + "binding": { + "subject": copy.deepcopy(request["subject"]), + "action": request["action"], + "resource": copy.deepcopy(request["resource"]), + "context": copy.deepcopy(request["context"]), + "request_digest": request_digest(request), + }, + "provenance": { + "evaluator": "flex-auth/local", + "mode": "standalone", + "policy_package": "secrets-engine.lifecycle", + "policy_version": "v1", + }, + }, + "provenance": {"authority": "state-hub"}, + } + + +def _validate(envelope, expected=None): + return validate_action_authorization( + envelope, + expected or _request(), + accepted_policy_packages={"secrets-engine.lifecycle"}, + accepted_policy_versions={"v1"}, + minimum_approval_count=2, + now=NOW, + ) + + +def test_digest_matches_flex_auth_contract_example(): + request = { + "id": "check:secrets-engine-destroy-example", + "subject": {"id": "user:alice", "type": "Human"}, + "action": "destroy", + "resource": { + "id": "catalog:example-build-test-token", + "type": "secret-catalog-lane", + "system": "secrets-engine", + "attributes": { + "stage": "build", + "fields": ["token"], + "policy_targets": [], + "auth_targets": [], + }, + }, + "context": {"purpose": "contract-test"}, + } + # Generated independently with flex-auth's Go api.CheckRequest and + # encoding/json. The action_authorization.json example carried a stale + # digest when this consumer contract was implemented. + assert request_digest(request) == ( + "sha256:73d5d7d5b3363f1a1db8f4c0e79c8f33dae5d77ffb97f21e449438bc0defa4c3" + ) + + +def test_valid_exact_action_authorization_passes(): + result = _validate(_envelope()) + assert result.authorization_id == "8bfc20be-47a4-4fb0-97a2-bf0a920afad8" + assert result.action == "deactivate" + assert result.subject_id == "user:alice" + + +@pytest.mark.parametrize( + ("mutation", "match"), + [ + (lambda doc: doc.update(status="superseded"), "status is not approved"), + ( + lambda doc: doc["request"]["resource"].update(id="catalog:wrong"), + "does not exactly match", + ), + ( + lambda doc: doc["request"].update(action="destroy"), + "does not exactly match", + ), + ( + lambda doc: doc["request"]["resource"]["attributes"].update( + fields=["other"] + ), + "does not exactly match", + ), + ( + lambda doc: doc["request"]["context"].update(purpose="wrong"), + "does not exactly match", + ), + ( + lambda doc: doc["decision"].update(effect="deny"), + "effect is not allow", + ), + ( + lambda doc: doc["decision"]["binding"].update( + request_digest="sha256:" + "0" * 64 + ), + "digest does not match", + ), + ( + lambda doc: doc["approvals"]["entries"][1].update( + subject_id="user:alice" + ), + "duplicate approver", + ), + ( + lambda doc: doc["approvals"].update(required_count=1), + "threshold is insufficient", + ), + ( + lambda doc: doc["decision"].update(request_id="check:wrong"), + "request id does not match", + ), + ( + lambda doc: doc["provenance"].update(authority="local-fixture"), + "authority is not State Hub", + ), + ], +) +def test_invalid_authorizations_fail_closed(mutation, match): + envelope = _envelope() + mutation(envelope) + with pytest.raises(DecisionError, match=match): + _validate(envelope) + + +def test_expired_authorization_fails_closed(): + with pytest.raises(DecisionError, match="expired"): + validate_action_authorization( + _envelope(), + _request(), + accepted_policy_packages={"secrets-engine.lifecycle"}, + accepted_policy_versions={"v1"}, + now=datetime(2026, 8, 23, 10, 15, tzinfo=timezone.utc), + ) + + +def test_noncanonical_authorization_uuid_is_rejected(): + envelope = _envelope() + envelope["id"] = envelope["id"].replace("-", "") + with pytest.raises(DecisionError, match="canonical UUID"): + _validate(envelope) + + +def test_approval_timestamp_must_be_inside_current_authorization_window(): + envelope = _envelope() + envelope["approvals"]["entries"][1]["approved_at"] = "2026-08-23T10:06:00Z" + with pytest.raises(DecisionError, match="approval time is outside window"): + _validate(envelope) + + +def test_unsorted_or_duplicate_target_sets_are_rejected(): + envelope = _envelope() + envelope["request"]["resource"]["attributes"]["fields"] = [ + "second", + "first", + "first", + ] + with pytest.raises(DecisionError, match="sorted and unique"): + _validate(envelope, expected=envelope["request"]) + + +def test_unaccepted_policy_revision_is_rejected(): + envelope = _envelope() + envelope["decision"]["provenance"]["policy_version"] = "v2" + with pytest.raises(DecisionError, match="policy version is not accepted"): + _validate(envelope) diff --git a/tests/test_audit.py b/tests/test_audit.py index 1dcd47e..dd76244 100644 --- a/tests/test_audit.py +++ b/tests/test_audit.py @@ -96,6 +96,24 @@ def test_lane_audit_empty_directory_is_a_valid_empty_summary(tmp_path): assert summary.render().startswith("Lane audit summary for 'test-lane'") +def test_lane_audit_counts_edge_queued_delivery(tmp_path): + path = tmp_path / "evidence-2026-08-23.jsonl" + path.write_text( + json.dumps( + { + "catalog_id": "test-lane", + "action": "evidence-delivery", + "result": "queued", + "detail": {"outbox_id": "3f12014e-47c1-48a5-9c8f-774c1dac1853"}, + } + ) + + "\n", + encoding="utf-8", + ) + summary = summarize_lane_evidence(tmp_path, "test-lane") + assert summary.hub_delivery == {"queued": 1} + + def test_cli_parser_exposes_audit_json_command(): args = build_parser().parse_args(["audit", "test-lane", "--json"]) assert args.catalog_id == "test-lane" diff --git a/tests/test_decisions.py b/tests/test_decisions.py index ece6f60..e827606 100644 --- a/tests/test_decisions.py +++ b/tests/test_decisions.py @@ -72,3 +72,37 @@ def test_privileged_lane_helper_accepts_local_approval(tmp_path, monkeypatch): monkeypatch.setattr(cli, "repo_root", lambda: tmp_path) decision = _require_lane_approval(SimpleNamespace(hub_url=""), _approved()) assert decision.id == "x" + + +def test_production_action_fails_closed_before_legacy_decision(monkeypatch): + import secrets_engine.cli as cli + + entry = validate_entry(dict(VALID, stage="prod", approval={"model": "bootstrap-only"})) + cfg = SimpleNamespace(hub_url="http://127.0.0.1:8000", bao_addr="http://127.0.0.1:8200") + monkeypatch.delenv("SECRETS_ENGINE_UNSAFE_DEMO", raising=False) + with pytest.raises(DecisionError, match="production action 'apply'"): + _require_lane_approval(cfg, entry, "apply") + + +def test_production_demo_requires_all_three_safety_conditions(tmp_path, monkeypatch): + import secrets_engine.cli as cli + + (tmp_path / ".decisions").mkdir() + (tmp_path / ".decisions" / "x.yaml").write_text( + "id: x\ntitle: approved\nstatus: resolved\nsuperseded_by: null\n" + ) + entry = validate_entry( + dict(VALID, stage="prod", approval={"model": "decision", "decision_ref": "x"}) + ) + monkeypatch.setattr(cli, "repo_root", lambda: tmp_path) + monkeypatch.setenv("SECRETS_ENGINE_UNSAFE_DEMO", "1") + + allowed = SimpleNamespace(hub_url="", bao_addr="http://127.0.0.1:8200") + assert _require_lane_approval(allowed, entry, "apply").id == "x" + + for cfg in ( + SimpleNamespace(hub_url="http://127.0.0.1:8000", bao_addr="http://127.0.0.1:8200"), + SimpleNamespace(hub_url="", bao_addr="https://bao.example.test"), + ): + with pytest.raises(DecisionError, match="live production remains disabled"): + _require_lane_approval(cfg, entry, "apply") diff --git a/tests/test_privileged_cli_evidence.py b/tests/test_privileged_cli_evidence.py index 12bd626..9a91e47 100644 --- a/tests/test_privileged_cli_evidence.py +++ b/tests/test_privileged_cli_evidence.py @@ -96,3 +96,28 @@ def test_provision_decision_rejection_is_recorded_before_backend( ] assert records[-1]["detail"]["approval_status"] == "rejected" assert records[-1]["detail"]["decision_ref"] == "CCR-2026-0001" + + +def test_production_handler_fails_closed_before_backend(tmp_path, monkeypatch): + data = copy.deepcopy(VALID) + data.update( + stage="prod", + approval={"model": "decision", "decision_ref": "CCR-2026-0001"}, + ) + entry = validate_entry(data) + monkeypatch.setattr(cli, "get_entry", lambda *_args: entry) + monkeypatch.delenv("SECRETS_ENGINE_UNSAFE_DEMO", raising=False) + monkeypatch.setattr( + cli.OpenBaoClient, + "resolve", + lambda *_args, **_kwargs: pytest.fail("backend must not be reached"), + ) + + with pytest.raises(DecisionError, match="production action 'provision'"): + cli.cmd_provision(_config(tmp_path), _provision_args(entry)) + + records = _records(tmp_path) + assert [record["result"] for record in records] == [ + "attempt", + "failed-DecisionError", + ] diff --git a/tests/test_redact_evidence.py b/tests/test_redact_evidence.py index 19132d9..7379ea2 100644 --- a/tests/test_redact_evidence.py +++ b/tests/test_redact_evidence.py @@ -47,9 +47,15 @@ def test_evidence_record_has_no_value(tmp_path): def test_evidence_records_append_only_hub_delivery_success(tmp_path, monkeypatch): + captured = {} + + def delivered(request, **_kwargs): + captured["headers"] = dict(request.header_items()) + return SimpleNamespace(status=200, read=lambda: b"{}") + monkeypatch.setattr( "urllib.request.urlopen", - lambda *_args, **_kwargs: SimpleNamespace(read=lambda: b"{}"), + delivered, ) writer = EvidenceWriter( evidence_dir=tmp_path, @@ -68,6 +74,62 @@ def test_evidence_records_append_only_hub_delivery_success(tmp_path, monkeypatch assert lines[1]["action"] == "evidence-delivery" assert lines[1]["result"] == "delivered" assert lines[1]["related_record_id"] == primary["record_id"] + assert captured["headers"]["Idempotency-key"] == ( + f"secrets-engine:{primary['record_id']}" + ) + assert captured["headers"]["X-statehub-source-agent"] == "secrets-engine" + assert captured["headers"]["X-statehub-repo-slug"] == "secrets-engine" + + +def test_evidence_records_edge_queued_receipt_as_queued(tmp_path, monkeypatch): + outbox_id = "3f12014e-47c1-48a5-9c8f-774c1dac1853" + monkeypatch.setattr( + "urllib.request.urlopen", + lambda *_args, **_kwargs: SimpleNamespace( + status=202, + read=lambda: json.dumps( + {"queued": True, "outbox_id": outbox_id} + ).encode(), + ), + ) + writer = EvidenceWriter( + evidence_dir=tmp_path, + hub_url="http://edge.invalid", + topic_id="topic-id", + ) + + writer.record("verify", result="pass", catalog_id="lane") + + lines = [ + json.loads(line) + for line in next(tmp_path.glob("evidence-*.jsonl")).read_text().splitlines() + ] + assert lines[-1]["action"] == "evidence-delivery" + assert lines[-1]["result"] == "queued" + assert lines[-1]["detail"] == {"outbox_id": outbox_id} + assert "upstream" not in json.dumps(lines[-1]) + + +def test_evidence_rejects_malformed_queued_receipt(tmp_path, monkeypatch): + monkeypatch.setattr( + "urllib.request.urlopen", + lambda *_args, **_kwargs: SimpleNamespace( + status=202, + read=lambda: b'{"queued":true,"outbox_id":"not-a-uuid"}', + ), + ) + writer = EvidenceWriter( + evidence_dir=tmp_path, + hub_url="http://edge.invalid", + topic_id="topic-id", + ) + writer.record("verify", result="pass", catalog_id="lane") + lines = [ + json.loads(line) + for line in next(tmp_path.glob("evidence-*.jsonl")).read_text().splitlines() + ] + assert lines[-1]["result"] == "failed" + assert lines[-1]["detail"] == {} def test_evidence_records_hub_failure_without_raising(tmp_path, monkeypatch): diff --git a/tests/test_service_auth.py b/tests/test_service_auth.py new file mode 100644 index 0000000..ca2cafc --- /dev/null +++ b/tests/test_service_auth.py @@ -0,0 +1,159 @@ +import base64 +import json +from datetime import datetime, timezone +from pathlib import Path + +import pytest + +from secrets_engine.errors import BackendError +from secrets_engine.service_auth import ( + KeyCapeServiceAuthConfig, + KeyCapeServiceAuthProvider, + preflight_service_jwt, +) + +NOW = datetime(2026, 8, 23, 12, 0, tzinfo=timezone.utc) + + +def _segment(value): + return base64.urlsafe_b64encode( + json.dumps(value, separators=(",", ":")).encode() + ).decode().rstrip("=") + + +def _jwt(**overrides): + now = int(NOW.timestamp()) + claims = { + "iss": "https://keycape.example.test", + "sub": "service:secrets-engine", + "aud": "secrets-engine-openbao", + "iat": now, + "exp": now + 900, + "principal_type": "service", + "tenant": "tenant:coulomb", + "roles": ["secrets-engine"], + "groups": [], + "scope": "openbao:login", + "assurance": { + "aal": "AAL1", + "method": "client_secret", + "mfa": False, + "source": "key-cape", + }, + } + claims.update(overrides) + return f"{_segment({'alg': 'RS256', 'typ': 'JWT'})}.{_segment(claims)}.signature" + + +def _config(secret_file=Path("/tmp/keycape-client-secret")): + return KeyCapeServiceAuthConfig( + token_url="https://keycape.example.test/token", + issuer="https://keycape.example.test", + client_secret_file=secret_file, + ) + + +def test_preflight_accepts_exact_service_contract_and_renewal_window(): + token = _jwt() + result = preflight_service_jwt(token, _config(), now=NOW) + assert result.expires_at - result.issued_at == 900 + assert not result.needs_renewal(NOW) + assert result.needs_renewal(datetime.fromtimestamp(result.expires_at - 180, timezone.utc)) + assert token not in repr(result) + + +@pytest.mark.parametrize( + ("override", "match"), + [ + ({"iss": "https://attacker.test"}, "'iss'"), + ({"sub": "service:operator"}, "'sub'"), + ({"aud": "different"}, "'aud'"), + ({"tenant": "tenant:other"}, "'tenant'"), + ({"roles": ["admin"]}, "roles"), + ({"scope": "openbao:login admin"}, "scope"), + ({"exp": int(NOW.timestamp()) + 901}, "15-minute"), + ], +) +def test_preflight_rejects_claim_or_lifetime_drift(override, match): + with pytest.raises(BackendError, match=match): + preflight_service_jwt(_jwt(**override), _config(), now=NOW) + + +class _Response: + status = 200 + + def __init__(self, payload): + self.payload = payload + + def __enter__(self): + return self + + def __exit__(self, *_args): + return False + + def read(self): + return json.dumps(self.payload).encode() + + +def test_exchange_uses_basic_auth_and_never_places_secret_in_body(tmp_path): + secret_file = tmp_path / "client.secret" + secret_file.write_text("client-secret-value") + secret_file.chmod(0o600) + captured = {} + + def transport(request, *, timeout): + captured["authorization"] = request.get_header("Authorization") + captured["body"] = request.data.decode() + captured["timeout"] = timeout + return _Response( + { + "access_token": _jwt(), + "token_type": "Bearer", + "expires_in": 900, + } + ) + + provider = KeyCapeServiceAuthProvider(_config(secret_file), transport=transport) + result = provider.exchange(now=NOW) + assert result.expires_at == int(NOW.timestamp()) + 900 + assert captured["authorization"].startswith("Basic ") + assert "client-secret-value" not in captured["body"] + assert captured["body"] == "grant_type=client_credentials&scope=openbao%3Alogin" + + +def test_exchange_rejects_refresh_or_id_tokens_without_exposing_secret(tmp_path): + secret = "do-not-leak-this-client-secret" + secret_file = tmp_path / "client.secret" + secret_file.write_text(secret) + secret_file.chmod(0o600) + + def transport(_request, *, timeout): + return _Response( + { + "access_token": _jwt(), + "token_type": "Bearer", + "expires_in": 900, + "refresh_token": "also-secret", + } + ) + + provider = KeyCapeServiceAuthProvider(_config(secret_file), transport=transport) + with pytest.raises(BackendError, match="forbidden extra token") as error: + provider.exchange(now=NOW) + assert secret not in str(error.value) + + +def test_provider_config_is_exact_and_has_no_bootstrap_fallback(): + with pytest.raises(BackendError, match="HTTPS"): + KeyCapeServiceAuthConfig( + token_url="http://keycape.example.test/token", + issuer="https://keycape.example.test", + client_secret_file=Path("/tmp/unused"), + ) + with pytest.raises(BackendError, match="accepted contract"): + KeyCapeServiceAuthConfig( + token_url="https://keycape.example.test/token", + issuer="https://keycape.example.test", + client_secret_file=Path("/tmp/unused"), + client_id="operator", + ) diff --git a/workplans/SECRETS-WP-0007-production-lifecycle-hardening.md b/workplans/SECRETS-WP-0007-production-lifecycle-hardening.md index 8d9a50b..13bb815 100644 --- a/workplans/SECRETS-WP-0007-production-lifecycle-hardening.md +++ b/workplans/SECRETS-WP-0007-production-lifecycle-hardening.md @@ -198,15 +198,23 @@ Acceptance: ```task id: SECRETS-WP-0007-T04 -status: wait +status: progress priority: high ``` -Waiting 2026-08-23 on the canonical external contract rather than parsing -decision prose or inventing authorization ownership locally. Contract requests -were sent to State Hub (`24663321-0263-43fe-8d48-e9c7e06d7bb9`) and flex-auth -(`ef8ff95d-6b4e-46f8-b1a9-497d06cf7c9a`). Until resolved, local fixtures cannot -unlock the new destroy path and that live operation remains disabled. +Progress 2026-08-23. flex-auth revision `c473f19` now supplies execute-time +structured binding and the proposed durable `ActionAuthorization` schema. The +engine builds and fail-closed validates the exact secrets-engine profile, +validity, State Hub authority, decision digest/binding, accepted policy +revision, and a caller-selected minimum distinct-approver threshold. Every live +production handler now carries its action into the shared gate and fails before +OpenBao while State Hub lacks the durable endpoint. Legacy local decisions are +reachable for prod-labeled lanes only with an explicit unsafe-demo switch, +disabled Hub URL, and loopback OpenBao. The throwaway scripts declare that +exception. Live destroy remains disabled independently. State Hub endpoint and +authenticated approval storage are still outstanding; the flex-auth example's +committed digest was also found stale against its Go `CheckRequest` encoder and +must be corrected upstream. Define and enforce the decision contract needed by production commands. A resolved approval must bind at least: @@ -256,6 +264,13 @@ to railiance-platform (`8f910aff-3a94-43c6-8805-eb9276e46fc0`) and key-cape (`0627ca55-115e-43dc-b2f6-3195be3bc90d`); steady-state engine authentication remains external-contract work. +KeyCape has accepted the `secrets-engine-openbao` service contract. An explicit +consumer scaffold now performs HTTPS client-credentials exchange, strict +out-of-repo client-secret input, exact claim/assurance/lifetime preflight, +three-minute renewal classification, and rejection of ID/refresh tokens. It has +no bootstrap/operator/AppRole fallback and is deliberately not wired into the +CLI until railiance-platform materializes the exact OpenBao JWT mount and role. + Refactor AppRole login into a scoped session object that retains only the minimum non-secret lifecycle handle needed to revoke the issued token in a `finally` path. Apply it consistently to exec, KV verification, auth-capability @@ -304,8 +319,13 @@ rejection, backend/input exceptions, interruption, verification failure, and success are recorded without exception prose. Tests prove decision and backend failures stop before inappropriate backend work and exclude fake secret text. -The complete repository suite passes with 115 tests after these changes, -including throwaway OpenBao integration coverage. +State Hub evidence posts now carry stable idempotency, source-agent, and repo +headers. HTTP 202 edge-relay receipts are classified as `queued` and retain only +their non-secret outbox id in the companion evidence record and lane audit. +Outbox replay remains owned by State Hub/operator tooling. + +The complete repository suite is re-run at each implementation slice, including +throwaway OpenBao integration coverage. Make verification and routing truthful for multi-field and high-risk lanes: