key-cape/workplans/KEY-WP-0022-replacement-harness-and-external-conformance.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
KEY-WP-0022	workplan	Make the replacement harness runnable and target conformance externally	infotech	key-cape	finished	claude	replacement-harness-and-external-conformance	2026-09-07	2026-09-07	94d74b17-1b2f-57dd-b6c3-50f10a7dccd9

Closes the runnable half of gap G04 of history/2026-09-05-011726-scope-intent-assessment.md. The Scenario B and C scripts cannot execute: they reference compose files that do not exist, call binaries at src/bin/ where the Makefile builds to root bin/, pass --base-dn to lldap-to-ldap whose flag is --basedn, and invoke a hardcoded workstation Go path from outside the Go module.

The deeper problem is that repairing the shell would not prove anything. Both scripts set KEYCAPE_TEST_ISSUER, but the profile suite builds its own httptest server and never reads it, so the tests pass identically whether or not the external provider is running. A harness that cannot fail for the reason it exists is worse than a missing one.

Repair the harness prerequisites

id: KEY-WP-0022-T01
status: done
priority: high
state_hub_task_id: "1d552328-e75b-57fd-83bc-e469ae46b397"

Fix the binary paths, the --basedn flag, and the Go invocation so the scripts run from a clean checkout with make build: use the module directory rather than a workstation-specific Go path, and fail with a clear message when a prerequisite binary or a required environment value is absent instead of part-way through.

Scenario B rewritten: root bin/, go -C src so the module root is right, prerequisite checks before anything starts, and a cleanup trap so a failure does not leave a stack running. Scenario C repaired in place — bin/ paths, --basedn for lldap-to-ldap (lldap-export correctly takes --base-dn; the two flags genuinely differ), and the same module-relative test invocation.

Add the missing stacks

id: KEY-WP-0022-T02
status: done
priority: high
state_hub_task_id: "06f34e8b-f3d9-53ed-95c5-02e5063774a0"

Provide docker-compose.scenario-b.yml (Keycloak over the migrated realm) and docker-compose.scenario-c.yml (OpenLDAP seeded from generated LDIF, plus Keycloak). Validate them with docker compose config rather than assuming they parse. Whether a full scenario passes end to end is a separate claim from whether the stack definition is valid, and only the second is established here.

Both files added and validated with docker compose config. Scenario B was also started for real; Scenario C's stack has not been run.

Target conformance at an external issuer

id: KEY-WP-0022-T03
status: done
priority: high
state_hub_task_id: "d837730d-e9d0-5adb-9dda-ffc47661fca3"

Add a suite that reads KEYCAPE_TEST_ISSUER and exercises that issuer over HTTP: discovery metadata, the JWKS parsing under the same strict rules the runtime applies, and — when client credentials are supplied — a real token exchange whose claims are verified against the published keys. It must skip cleanly when the variable is unset so ordinary make test is unaffected, and it must fail when pointed at an issuer that does not meet the profile, which is the property the current harness lacks.

src/tests/conformance checks that the issuer advertises itself, the profile authorization surface (code, authorization_code, S256, RS256), that published keys parse under internal/jose — the same rules the runtime applies, so a provider publishing keys KeyCape would refuse fails here rather than at first login — that excluded grants are absent, and, with credentials, that an issued token verifies against the published keys and carries a matching issuer.

Prove the suite against a real provider

id: KEY-WP-0022-T04
status: done
priority: high
state_hub_task_id: "cb4d8af7-76a4-50b8-bee5-eb1c52c2f894"

Run the new suite against an actual Keycloak instance, not only against KeyCape, since the point of the scenario is a replacement provider. Record what genuinely passed and what remains unproven. Do not describe a partially exercised scenario as a demonstrated live swap.

Run against Keycloak 26.0. Discovery, authorization surface and published-key checks passed. A client_credentials exchange against a service client created through the admin API returned a token that verified against Keycloak's JWKS via internal/jose, with a matching issuer claim — the end-to-end property a relying party depends on, proved against a replacement provider rather than against KeyCape's own handlers.

The suite also failed, correctly. Stock Keycloak advertises implicit and password in its discovery document. In Keycloak these are server capabilities, not per-client toggles, so no realm the transformer emits can remove them: a migrated Keycloak presents a wider grant surface than KeyCape. Scenario B therefore reports a real failure today, which is the harness working. KeyCape itself was not targeted in this run — the dev stack needs key material absent from the checkout — so the suite is proved against the replacement provider, not yet against both sides of the swap.

Reconcile the records

id: KEY-WP-0022-T05
status: done
priority: medium
state_hub_task_id: "2cb63801-8182-5c71-ab8d-9a1c377f418c"

Update SCOPE.md and G04's status with what the harness now does and what replacement claims remain unproven — in particular that directory migration, MFA and unchanged relying-party behaviour against a live Keycloak are not established by this work. G04 does not fully close here.

SCOPE.md and G04's status record both the working harness and the grant-surface finding, which substantiates with evidence what SCOPE previously asserted without it. Scenario C has still never been run end to end.