Stabilize credential-change test suite (RAILIANCE-WP-0014)
Five failures in full credential test discovery, none of them broker regressions: - CCR-2026-0009 referenced a policy file that was never added, and used a schema-invalid access_frontdoor.readiness value. Add the least-privilege workload-kv-read-qonto-assistant.hcl (read-only on tenants/binky/qonto-api) and set readiness to pending-review. The lane stays proposed and non-resolvable. - Three refusal tests used the live CCR-2026-0002 file as their "unapproved CCR" fixture. That lane is now approved, applied and active, so the gates correctly permitted it and the tests failed; applier-apply then walked into its interactive confirmation prompt and raised EOFError under a non-interactive runner. Add an unapproved_ccr() helper that materializes a normalized temp copy so approval state is no longer read off a mutable production artifact. - The approve/unconfirmed-claim test demoted an active CCR to approved while leaving resolvable=true, tripping a correct validation rule. Build it from the same helper. No gate, blocker, validation rule, or grant semantic was changed. Verified: credential discovery 52/52 and full discovery 61/61 pass non-interactively, make credential-change-validate passes all nine CCRs, the grant catalog validates, and both audit-core openbao-database-credential grants retain exec-env-only delivery and revoke-on-exec-exit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
9f6bdffec4
commit
b7aef386d5
4 changed files with 195 additions and 32 deletions
|
|
@ -59,7 +59,7 @@ access_frontdoor:
|
|||
selector: qonto-assistant workload Qonto API key
|
||||
command: warden access qonto-assistant-workload-kv --fetch API_KEY,API_USER
|
||||
resolvable: false
|
||||
readiness: proposed
|
||||
readiness: pending-review
|
||||
delivery:
|
||||
surface: external-secrets
|
||||
target: >-
|
||||
|
|
|
|||
15
openbao/policies/workload-kv-read-qonto-assistant.hcl
Normal file
15
openbao/policies/workload-kv-read-qonto-assistant.hcl
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
# Least-privilege read policy for the qonto-assistant workload access lane.
|
||||
# Tenant mount (WARDEN-WP-0028) — client commercial secrets, not platform workloads.
|
||||
# Live path: tenants/binky/qonto-api (fields API_KEY, API_USER).
|
||||
# Second, workload-scoped lane onto the same value CCR-2026-0008 vends to the
|
||||
# human/OIDC admin lane; see CCR-2026-0009. Kubernetes auth subject is
|
||||
# external-secrets/external-secrets, ClusterSecretStore scoped to the
|
||||
# qonto-assistant namespace only. Read-only; no secret writes from this lane.
|
||||
|
||||
path "tenants/data/binky/qonto-api" {
|
||||
capabilities = ["read"]
|
||||
}
|
||||
|
||||
path "tenants/metadata/binky/qonto-api" {
|
||||
capabilities = ["read"]
|
||||
}
|
||||
|
|
@ -29,6 +29,26 @@ class CredentialChangeTests(unittest.TestCase):
|
|||
/ "credential-change-requests/CCR-2026-0002-issue-core-ingestion-api-key.yaml"
|
||||
)
|
||||
|
||||
def unapproved_ccr(self, source: Path | None = None) -> Path:
|
||||
"""Return a temp CCR copy that is genuinely pre-approval.
|
||||
|
||||
Refusal tests must not read approval state off a live repo CCR: once a
|
||||
lane is applied and activated, the file stops being an unapproved
|
||||
fixture and the expected refusal silently turns into a success.
|
||||
"""
|
||||
source = source or self.issue_core
|
||||
tmp = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, tmp, True)
|
||||
copied = tmp / source.name
|
||||
shutil.copy2(source, copied)
|
||||
data = credential_change.load_yaml(copied)
|
||||
data["status"] = "proposed"
|
||||
data["review"]["comments"] = []
|
||||
data["access_frontdoor"]["readiness"] = "pending-review"
|
||||
data["access_frontdoor"]["resolvable"] = False
|
||||
credential_change.dump_yaml(copied, data)
|
||||
return copied
|
||||
|
||||
def test_sample_ccr_validates_without_bound_claim_warning(self) -> None:
|
||||
ccr, errors, warnings = credential_change.validate_ccr(self.sample)
|
||||
self.assertEqual(errors, [])
|
||||
|
|
@ -249,38 +269,34 @@ class CredentialChangeTests(unittest.TestCase):
|
|||
def test_operator_commands_refuse_unapproved_ccr(self) -> None:
|
||||
with self.assertRaises(SystemExit):
|
||||
credential_change.command_operator_commands(
|
||||
type("Args", (), {"ref": str(self.issue_core)})()
|
||||
type("Args", (), {"ref": str(self.unapproved_ccr())})()
|
||||
)
|
||||
|
||||
def test_approve_records_comment_but_unconfirmed_claim_still_blocks_apply(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp_path = Path(tmp)
|
||||
ccr_dir = tmp_path / "ccrs"
|
||||
ccr_dir.mkdir()
|
||||
copied = ccr_dir / self.issue_core.name
|
||||
shutil.copy2(self.issue_core, copied)
|
||||
old_ccr_dir = os.environ.get("CCR_DIR")
|
||||
os.environ["CCR_DIR"] = str(ccr_dir)
|
||||
try:
|
||||
credential_change.append_decision(
|
||||
copied, "approved", "unit-test", "looks right"
|
||||
copied = self.unapproved_ccr()
|
||||
ccr_dir = copied.parent
|
||||
old_ccr_dir = os.environ.get("CCR_DIR")
|
||||
os.environ["CCR_DIR"] = str(ccr_dir)
|
||||
try:
|
||||
credential_change.append_decision(
|
||||
copied, "approved", "unit-test", "looks right"
|
||||
)
|
||||
copied_data = credential_change.load_yaml(copied)
|
||||
copied_data["openbao"]["auth"]["bound_claims_confirmed"] = False
|
||||
credential_change.dump_yaml(copied, copied_data)
|
||||
ccr, errors, _warnings = credential_change.validate_ccr(copied)
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(ccr["status"], "approved")
|
||||
self.assertEqual(ccr["review"]["comments"][-1]["comment"], "looks right")
|
||||
with self.assertRaises(SystemExit):
|
||||
credential_change.command_apply_plan(
|
||||
type("Args", (), {"ref": "CCR-2026-0002"})()
|
||||
)
|
||||
copied_data = credential_change.load_yaml(copied)
|
||||
copied_data["openbao"]["auth"]["bound_claims_confirmed"] = False
|
||||
credential_change.dump_yaml(copied, copied_data)
|
||||
ccr, errors, _warnings = credential_change.validate_ccr(copied)
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(ccr["status"], "approved")
|
||||
self.assertEqual(ccr["review"]["comments"][-1]["comment"], "looks right")
|
||||
with self.assertRaises(SystemExit):
|
||||
credential_change.command_apply_plan(
|
||||
type("Args", (), {"ref": "CCR-2026-0002"})()
|
||||
)
|
||||
finally:
|
||||
if old_ccr_dir is None:
|
||||
os.environ.pop("CCR_DIR", None)
|
||||
else:
|
||||
os.environ["CCR_DIR"] = old_ccr_dir
|
||||
finally:
|
||||
if old_ccr_dir is None:
|
||||
os.environ.pop("CCR_DIR", None)
|
||||
else:
|
||||
os.environ["CCR_DIR"] = old_ccr_dir
|
||||
|
||||
def test_confirm_binding_records_comment_and_clears_warning(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
|
|
@ -324,7 +340,7 @@ class CredentialChangeTests(unittest.TestCase):
|
|||
|
||||
def test_applier_dry_run_refuses_unapproved_ccr(self) -> None:
|
||||
exit_code = credential_change.command_applier_dry_run(
|
||||
type("Args", (), {"ref": str(self.issue_core), "json": False})()
|
||||
type("Args", (), {"ref": str(self.unapproved_ccr()), "json": False})()
|
||||
)
|
||||
self.assertEqual(exit_code, 1)
|
||||
|
||||
|
|
@ -406,7 +422,7 @@ class CredentialChangeTests(unittest.TestCase):
|
|||
"Args",
|
||||
(),
|
||||
{
|
||||
"ref": str(self.issue_core),
|
||||
"ref": str(self.unapproved_ccr()),
|
||||
"actor": "unit-test",
|
||||
"confirm": None,
|
||||
"bao_bin": "bao",
|
||||
|
|
@ -509,7 +525,7 @@ class CredentialChangeTests(unittest.TestCase):
|
|||
"Args",
|
||||
(),
|
||||
{
|
||||
"ref": str(self.issue_core),
|
||||
"ref": str(self.unapproved_ccr()),
|
||||
"json": False,
|
||||
"execute_metadata": False,
|
||||
"actor": "unit-test",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,132 @@
|
|||
---
|
||||
id: RAILIANCE-WP-0014
|
||||
type: workplan
|
||||
title: "Credential-change test suite stabilization"
|
||||
domain: financials
|
||||
repo: railiance-platform
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: railiance
|
||||
created: "2026-08-11"
|
||||
updated: "2026-08-11"
|
||||
related_repos:
|
||||
- rapp-postgres
|
||||
- ops-warden
|
||||
---
|
||||
|
||||
# RAILIANCE-WP-0014 - Credential-change test suite stabilization
|
||||
|
||||
## Goal
|
||||
|
||||
Make `python3 -m unittest discover -s tests -p 'test_credential*.py'` pass
|
||||
non-interactively again, without weakening any approval or front-door gate.
|
||||
|
||||
Origin: the `rapp-postgres` coordination message of 2026-08-11, a residual from
|
||||
`RAPP-POSTGRES-WP-0002-T04`. The focused credential broker suite was green
|
||||
(9/9) at commit `9f6bdff`, but full discovery reported five failures unrelated
|
||||
to the broker work.
|
||||
|
||||
## Diagnosis
|
||||
|
||||
None of the five failures were broker regressions. Two were missing or invalid
|
||||
artifacts in `CCR-2026-0009`; three were test fixtures that had silently
|
||||
decayed as real CCR lanes advanced through their lifecycle.
|
||||
|
||||
The decay is the important finding. Three refusal tests used the live
|
||||
`CCR-2026-0002` file as their "unapproved CCR" fixture. That lane has since
|
||||
been approved, applied and activated, so the refusal those tests assert stopped
|
||||
being the correct behaviour for that input — the gate did exactly the right
|
||||
thing and the tests failed anyway. `applier-apply` then walked past its
|
||||
(correctly ordered) blocker check into the interactive confirmation prompt and
|
||||
raised `EOFError` under a non-interactive runner. A refusal test that reads its
|
||||
approval state off a mutable production artifact will keep breaking every time
|
||||
a lane advances.
|
||||
|
||||
## Boundaries
|
||||
|
||||
This workplan may:
|
||||
|
||||
- repair credential-change test fixtures and add missing CCR source artifacts
|
||||
- correct schema-invalid field values in credential change requests
|
||||
|
||||
It must not:
|
||||
|
||||
- relax `applier_readiness_blockers`, `runbook_readiness_blockers`, or the
|
||||
front-door `resolvable`/`status` coupling in `scripts/credential-change.py`
|
||||
- change grant delivery or revocation semantics in `credential-grants/catalog.yaml`
|
||||
- write or move any secret value
|
||||
|
||||
## Tasks
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Repair `CCR-2026-0009` (qonto-assistant workload KV read lane) so it validates:
|
||||
|
||||
- add the missing `openbao/policies/workload-kv-read-qonto-assistant.hcl`,
|
||||
scoped read-only to `tenants/{data,metadata}/binky/qonto-api` and mirroring
|
||||
the `workload-kv-read-binky-qonto-api.hcl` shape
|
||||
- replace the schema-invalid `access_frontdoor.readiness: proposed` with
|
||||
`pending-review`, which is the correct state for a CCR still in review
|
||||
|
||||
The CCR stays `status: proposed` with `resolvable: false` — this task adds the
|
||||
missing source artifact, it does not advance the lane.
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T02
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Stop the refusal tests depending on live CCR lifecycle state. Add a
|
||||
`unapproved_ccr()` helper to `tests/test_credential_change.py` that materializes
|
||||
a temp copy normalized to `status: proposed`, no review comments,
|
||||
`readiness: pending-review`, `resolvable: false`, and point
|
||||
`test_applier_dry_run_refuses_unapproved_ccr`,
|
||||
`test_applier_apply_refuses_unapproved_ccr`,
|
||||
`test_runbook_refuses_unapproved_ccr` and
|
||||
`test_operator_commands_refuse_unapproved_ccr` at it.
|
||||
|
||||
This also removes the `EOFError`: with a genuinely unapproved CCR,
|
||||
`command_applier_apply` returns 1 at its blocker check and never reaches the
|
||||
confirmation prompt. The gate ordering in `scripts/credential-change.py` was
|
||||
already correct and was left untouched.
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T03
|
||||
status: done
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Fix `test_approve_records_comment_but_unconfirmed_claim_still_blocks_apply`,
|
||||
which demoted an active CCR to `approved` while leaving
|
||||
`access_frontdoor.resolvable: true`, tripping the
|
||||
`resolvable=true requires status active` rule. Build the fixture from
|
||||
`unapproved_ccr()` so the front-door state is consistent with the status the
|
||||
test actually wants. The validation rule is correct and unchanged.
|
||||
|
||||
```task
|
||||
id: RAILIANCE-WP-0014-T04
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Confirm acceptance against the requested criteria:
|
||||
|
||||
- `python3 -m unittest discover -s tests -p 'test_credential*.py' < /dev/null`
|
||||
— 52 tests, OK
|
||||
- full repo discovery `-p 'test_*.py'` — 61 tests, OK
|
||||
- `make credential-change-validate` — all nine CCRs OK
|
||||
- `python3 scripts/credential-grants-validate.py` — catalog valid, 3 grants
|
||||
- both `rapp-postgres/audit-core-*` `openbao-database-credential` grants still
|
||||
carry `delivery.allowed: [exec-env]` with `child_only`/`redact_logs`, and
|
||||
`revocation.required: true` with `on_exec_exit: true`
|
||||
|
||||
## Outcome
|
||||
|
||||
Suite green non-interactively; no gate, blocker, validation rule or grant
|
||||
semantic was modified. The one behavioural change outside tests is the addition
|
||||
of a previously missing least-privilege policy artifact.
|
||||
Loading…
Add table
Add a link
Reference in a new issue