key-cape/workplans/KEY-WP-0022-replacement-harness-and-external-conformance.md

---
id: KEY-WP-0022
type: workplan
title: "Make the replacement harness runnable and target conformance externally"
domain: infotech
repo: key-cape
status: finished
owner: claude
topic_slug: replacement-harness-and-external-conformance
created: "2026-09-07"
updated: "2026-09-07"
state_hub_workstream_id: "94d74b17-1b2f-57dd-b6c3-50f10a7dccd9"
---

Closes the runnable half of gap G04 of
`history/2026-09-05-011726-scope-intent-assessment.md`. The Scenario B and C
scripts cannot execute: they reference compose files that do not exist, call
binaries at `src/bin/` where the Makefile builds to root `bin/`, pass `--base-dn`
to `lldap-to-ldap` whose flag is `--basedn`, and invoke a hardcoded workstation Go
path from outside the Go module.

The deeper problem is that repairing the shell would not prove anything. Both
scripts set `KEYCAPE_TEST_ISSUER`, but the profile suite builds its own `httptest`
server and never reads it, so the tests pass identically whether or not the
external provider is running. A harness that cannot fail for the reason it exists
is worse than a missing one.

## Repair the harness prerequisites

```task
id: KEY-WP-0022-T01
status: done
priority: high
state_hub_task_id: "1d552328-e75b-57fd-83bc-e469ae46b397"
```

Fix the binary paths, the `--basedn` flag, and the Go invocation so the scripts
run from a clean checkout with `make build`: use the module directory rather than
a workstation-specific Go path, and fail with a clear message when a prerequisite
binary or a required environment value is absent instead of part-way through.

Scenario B rewritten: root `bin/`, `go -C src` so the module root is right,
prerequisite checks before anything starts, and a cleanup trap so a failure does
not leave a stack running. Scenario C repaired in place — `bin/` paths, `--basedn`
for `lldap-to-ldap` (`lldap-export` correctly takes `--base-dn`; the two flags
genuinely differ), and the same module-relative test invocation.

## Add the missing stacks

```task
id: KEY-WP-0022-T02
status: done
priority: high
state_hub_task_id: "06f34e8b-f3d9-53ed-95c5-02e5063774a0"
```

Provide `docker-compose.scenario-b.yml` (Keycloak over the migrated realm) and
`docker-compose.scenario-c.yml` (OpenLDAP seeded from generated LDIF, plus
Keycloak). Validate them with `docker compose config` rather than assuming they
parse. Whether a full scenario passes end to end is a separate claim from whether
the stack definition is valid, and only the second is established here.

Both files added and validated with `docker compose config`. Scenario B was also
started for real; Scenario C's stack has not been run.

## Target conformance at an external issuer

```task
id: KEY-WP-0022-T03
status: done
priority: high
state_hub_task_id: "d837730d-e9d0-5adb-9dda-ffc47661fca3"
```

Add a suite that reads `KEYCAPE_TEST_ISSUER` and exercises that issuer over HTTP:
discovery metadata, the JWKS parsing under the same strict rules the runtime
applies, and — when client credentials are supplied — a real token exchange whose
claims are verified against the published keys. It must skip cleanly when the
variable is unset so ordinary `make test` is unaffected, and it must fail when
pointed at an issuer that does not meet the profile, which is the property the
current harness lacks.

`src/tests/conformance` checks that the issuer advertises itself, the profile
authorization surface (code, authorization_code, S256, RS256), that published
keys parse under `internal/jose` — the same rules the runtime applies, so a
provider publishing keys KeyCape would refuse fails here rather than at first
login — that excluded grants are absent, and, with credentials, that an issued
token verifies against the published keys and carries a matching issuer.

## Prove the suite against a real provider

```task
id: KEY-WP-0022-T04
status: done
priority: high
state_hub_task_id: "cb4d8af7-76a4-50b8-bee5-eb1c52c2f894"
```

Run the new suite against an actual Keycloak instance, not only against KeyCape,
since the point of the scenario is a replacement provider. Record what genuinely
passed and what remains unproven. Do not describe a partially exercised scenario
as a demonstrated live swap.

Run against Keycloak 26.0. Discovery, authorization surface and published-key
checks passed. A `client_credentials` exchange against a service client created
through the admin API returned a token that verified against Keycloak's JWKS via
`internal/jose`, with a matching issuer claim — the end-to-end property a relying
party depends on, proved against a replacement provider rather than against
KeyCape's own handlers.

The suite also failed, correctly. Stock Keycloak advertises `implicit` and
`password` in its discovery document. In Keycloak these are server capabilities,
not per-client toggles, so no realm the transformer emits can remove them: a
migrated Keycloak presents a wider grant surface than KeyCape. Scenario B
therefore reports a real failure today, which is the harness working. KeyCape
itself was not targeted in this run — the dev stack needs key material absent
from the checkout — so the suite is proved against the replacement provider, not
yet against both sides of the swap.

## Reconcile the records

```task
id: KEY-WP-0022-T05
status: done
priority: medium
state_hub_task_id: "2cb63801-8182-5c71-ab8d-9a1c377f418c"
```

Update `SCOPE.md` and G04's status with what the harness now does and what
replacement claims remain unproven — in particular that directory migration, MFA
and unchanged relying-party behaviour against a live Keycloak are not established
by this work. G04 does not fully close here.

SCOPE.md and G04's status record both the working harness and the grant-surface
finding, which substantiates with evidence what SCOPE previously asserted without
it. Scenario C has still never been run end to end.