key-cape/workplans/KEY-WP-0027-rollout-readiness-and-live-state.md

122 lines
5 KiB
Markdown
Raw Normal View History

Establish the live state and find a rollout precondition for G10 G10 waits on custody and platform owners and cannot close from here. What was doable: verify the handoffs actually went out, replace a remembered live state with an observed one, and find out whether main is safe to deploy. The last question found a defect in this repository's own recent work. Handoffs verified independently rather than trusted: all seven messages are in the hub with receipt ids. This gap was reopened once for claimed-but-unsent delivery, so the claim deserved the same scrutiny. Live state read from the cluster read-only: image main-153258b, only the Qonto secret materialized so the approval clients remain unprovisioned, four registered clients, no tenantEngine block. That also corrects an earlier claim of mine -- the deployed config sets userOU explicitly, so the KEY-WP-0023 default fix was never a production issue. The precondition: KEY-WP-0019 discovers the expected issuer from authelia.tokenBaseURL, and the deployed Authelia derives its advertised issuer from the request Host, advertising the in-cluster address to KeyCape and the browser-facing one to browsers. Verification fails closed, so a mismatch breaks every human login and looks like a broken login rather than a misconfiguration. Which value the token carries needs a real login against production to settle and was not determined here. Two mitigations: docs/operations.md documents pinning authelia.issuer and jwksUrl, with the curl that reveals what the provider advertises for a given Host; and the authentication failure event now carries a specific reason, so id_token_issuer_mismatch is distinguishable from a signature failure or an unreachable key set. The browser still learns nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 11:41:42 +02:00
---
id: KEY-WP-0027
type: workplan
title: "Establish the live state and the rollout gap for G10"
domain: infotech
repo: key-cape
status: finished
owner: claude
topic_slug: rollout-readiness-and-live-state
created: "2026-09-08"
updated: "2026-09-08"
state_hub_workstream_id: "2ab60558-ee14-542c-9f19-bb487de8e668"
Establish the live state and find a rollout precondition for G10 G10 waits on custody and platform owners and cannot close from here. What was doable: verify the handoffs actually went out, replace a remembered live state with an observed one, and find out whether main is safe to deploy. The last question found a defect in this repository's own recent work. Handoffs verified independently rather than trusted: all seven messages are in the hub with receipt ids. This gap was reopened once for claimed-but-unsent delivery, so the claim deserved the same scrutiny. Live state read from the cluster read-only: image main-153258b, only the Qonto secret materialized so the approval clients remain unprovisioned, four registered clients, no tenantEngine block. That also corrects an earlier claim of mine -- the deployed config sets userOU explicitly, so the KEY-WP-0023 default fix was never a production issue. The precondition: KEY-WP-0019 discovers the expected issuer from authelia.tokenBaseURL, and the deployed Authelia derives its advertised issuer from the request Host, advertising the in-cluster address to KeyCape and the browser-facing one to browsers. Verification fails closed, so a mismatch breaks every human login and looks like a broken login rather than a misconfiguration. Which value the token carries needs a real login against production to settle and was not determined here. Two mitigations: docs/operations.md documents pinning authelia.issuer and jwksUrl, with the curl that reveals what the provider advertises for a given Host; and the authentication failure event now carries a specific reason, so id_token_issuer_mismatch is distinguishable from a signature failure or an unreachable key set. The browser still learns nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 11:41:42 +02:00
---
G10 cannot be closed from this repository: it waits on named custody and platform
owners. What was doable was to stop guessing about the live state, verify the
handoffs actually went out, and find out whether main is safe to roll out. The
last question turned up a defect in this repository's own recent work.
## Verify the handoff receipts independently
```task
id: KEY-WP-0027-T01
status: done
priority: medium
state_hub_task_id: "00de2bad-04da-5b58-8cde-aaa144272541"
Establish the live state and find a rollout precondition for G10 G10 waits on custody and platform owners and cannot close from here. What was doable: verify the handoffs actually went out, replace a remembered live state with an observed one, and find out whether main is safe to deploy. The last question found a defect in this repository's own recent work. Handoffs verified independently rather than trusted: all seven messages are in the hub with receipt ids. This gap was reopened once for claimed-but-unsent delivery, so the claim deserved the same scrutiny. Live state read from the cluster read-only: image main-153258b, only the Qonto secret materialized so the approval clients remain unprovisioned, four registered clients, no tenantEngine block. That also corrects an earlier claim of mine -- the deployed config sets userOU explicitly, so the KEY-WP-0023 default fix was never a production issue. The precondition: KEY-WP-0019 discovers the expected issuer from authelia.tokenBaseURL, and the deployed Authelia derives its advertised issuer from the request Host, advertising the in-cluster address to KeyCape and the browser-facing one to browsers. Verification fails closed, so a mismatch breaks every human login and looks like a broken login rather than a misconfiguration. Which value the token carries needs a real login against production to settle and was not determined here. Two mitigations: docs/operations.md documents pinning authelia.issuer and jwksUrl, with the curl that reveals what the provider advertises for a given Host; and the authentication failure event now carries a specific reason, so id_token_issuer_mismatch is distinguishable from a signature failure or an unreachable key set. The browser still learns nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 11:41:42 +02:00
```
KEY-WP-0009 was reopened once because delivery was claimed without receipts, so
the claim that it has now been sent deserved the same scrutiny rather than being
taken on trust. `GET /messages/?from_agent=key-cape` shows all seven present and
dated 2026-09-08T08:21Z: the four KEY-WP-0009 handoffs (netkingdom,
secrets-engine, ops-warden, railiance-platform) and the three admission requests
(railiance-platform custody, approval-engine human callback, ops-warden token
split). No replies had arrived at the time of writing, which is expected hours
after sending and is not itself a blocker to record.
## Record the live state as observed, not as remembered
```task
id: KEY-WP-0027-T02
status: done
priority: medium
state_hub_task_id: "2d513229-1dbd-500e-9100-c0173be7b4de"
Establish the live state and find a rollout precondition for G10 G10 waits on custody and platform owners and cannot close from here. What was doable: verify the handoffs actually went out, replace a remembered live state with an observed one, and find out whether main is safe to deploy. The last question found a defect in this repository's own recent work. Handoffs verified independently rather than trusted: all seven messages are in the hub with receipt ids. This gap was reopened once for claimed-but-unsent delivery, so the claim deserved the same scrutiny. Live state read from the cluster read-only: image main-153258b, only the Qonto secret materialized so the approval clients remain unprovisioned, four registered clients, no tenantEngine block. That also corrects an earlier claim of mine -- the deployed config sets userOU explicitly, so the KEY-WP-0023 default fix was never a production issue. The precondition: KEY-WP-0019 discovers the expected issuer from authelia.tokenBaseURL, and the deployed Authelia derives its advertised issuer from the request Host, advertising the in-cluster address to KeyCape and the browser-facing one to browsers. Verification fails closed, so a mismatch breaks every human login and looks like a broken login rather than a misconfiguration. Which value the token carries needs a real login against production to settle and was not determined here. Two mitigations: docs/operations.md documents pinning authelia.issuer and jwksUrl, with the curl that reveals what the provider advertises for a given Host; and the authentication failure event now carries a specific reason, so id_token_issuer_mismatch is distinguishable from a signature failure or an unreachable key set. The browser still learns nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 11:41:42 +02:00
```
Read-only cluster inspection, `sso` namespace, 2026-09-08:
- image `forgejo.coulomb.social/coulomb/key-cape:main-153258b`, predating every
change from KEY-WP-0016 onward;
- environment carries `KEYCAPE_RAPP_QONTO_CLIENT_SECRET` only, so the two
approval clients are still not materialized — KEY-WP-0013-T02 remains correctly
blocked;
- four registered clients: `demo-app`, `netkingdom-bootstrap-console` and
`openbao-admin` (public, authorization_code) and `rapp-qonto-client`
(confidential, client_credentials);
- no `tenantEngine` block, so KEY-WP-0024 is inert on rollout.
**Correction to an earlier claim of mine.** I reported the `ou=people` fix
(KEY-WP-0023) as a production-relevant correction to human login. The deployed
config sets `userOU: "ou=people"` explicitly, so production was never affected.
The fix matters for anyone relying on the default — the dev stack, the scenario
scripts, a fresh deployment — not for this one.
## Find out whether main is safe to roll out
```task
id: KEY-WP-0027-T03
status: done
priority: high
state_hub_task_id: "23c8d04b-c2b7-5347-80a0-6388c1c41c53"
Establish the live state and find a rollout precondition for G10 G10 waits on custody and platform owners and cannot close from here. What was doable: verify the handoffs actually went out, replace a remembered live state with an observed one, and find out whether main is safe to deploy. The last question found a defect in this repository's own recent work. Handoffs verified independently rather than trusted: all seven messages are in the hub with receipt ids. This gap was reopened once for claimed-but-unsent delivery, so the claim deserved the same scrutiny. Live state read from the cluster read-only: image main-153258b, only the Qonto secret materialized so the approval clients remain unprovisioned, four registered clients, no tenantEngine block. That also corrects an earlier claim of mine -- the deployed config sets userOU explicitly, so the KEY-WP-0023 default fix was never a production issue. The precondition: KEY-WP-0019 discovers the expected issuer from authelia.tokenBaseURL, and the deployed Authelia derives its advertised issuer from the request Host, advertising the in-cluster address to KeyCape and the browser-facing one to browsers. Verification fails closed, so a mismatch breaks every human login and looks like a broken login rather than a misconfiguration. Which value the token carries needs a real login against production to settle and was not determined here. Two mitigations: docs/operations.md documents pinning authelia.issuer and jwksUrl, with the curl that reveals what the provider advertises for a given Host; and the authentication failure event now carries a specific reason, so id_token_issuer_mismatch is distinguishable from a signature failure or an unreachable key set. The browser still learns nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 11:41:42 +02:00
```
**Defect found in KEY-WP-0019, before it shipped.** Upstream verification
discovers the expected issuer server-side from `authelia.tokenBaseURL`. The
deployed Authelia derives its advertised issuer from the request Host, confirmed
directly:
| Host presented | issuer advertised |
| --- | --- |
| `authelia.sso.svc.cluster.local:9091` | `http://authelia.sso.svc.cluster.local:9091` |
| `auth.coulomb.social` | `http://auth.coulomb.social` |
KeyCape fetches over the in-cluster address, so it would expect the first value
while tokens minted for the browser-facing host carry the second. Verification
fails closed, so if they differ **every human login fails after rollout**, and it
presents as a broken login rather than a configuration error.
Which value the ID token actually carries could not be settled from here without
performing a real login against production Authelia, which was not done. The risk
is therefore recorded as a rollout precondition, not as a confirmed breakage.
Two mitigations, both in this repository:
- `docs/operations.md` documents pinning `authelia.issuer` and `authelia.jwksUrl`
explicitly, with the one-line `curl` that reveals the value the provider will
advertise for a given Host. The overrides already existed; nothing said when
they were needed.
- The failure is now diagnosable. The authentication failure event carries a
specific reason — `id_token_issuer_mismatch`, distinct from
`id_token_signature`, `provider_keys_unavailable` and the rest — so an operator
can tell a misconfiguration from an attack. Five cases cover the mapping. The
browser still learns nothing.
## Record what each owner owes
```task
id: KEY-WP-0027-T04
status: done
priority: medium
state_hub_task_id: "7ba84b61-0458-5820-8ded-c6f59d820303"
Establish the live state and find a rollout precondition for G10 G10 waits on custody and platform owners and cannot close from here. What was doable: verify the handoffs actually went out, replace a remembered live state with an observed one, and find out whether main is safe to deploy. The last question found a defect in this repository's own recent work. Handoffs verified independently rather than trusted: all seven messages are in the hub with receipt ids. This gap was reopened once for claimed-but-unsent delivery, so the claim deserved the same scrutiny. Live state read from the cluster read-only: image main-153258b, only the Qonto secret materialized so the approval clients remain unprovisioned, four registered clients, no tenantEngine block. That also corrects an earlier claim of mine -- the deployed config sets userOU explicitly, so the KEY-WP-0023 default fix was never a production issue. The precondition: KEY-WP-0019 discovers the expected issuer from authelia.tokenBaseURL, and the deployed Authelia derives its advertised issuer from the request Host, advertising the in-cluster address to KeyCape and the browser-facing one to browsers. Verification fails closed, so a mismatch breaks every human login and looks like a broken login rather than a misconfiguration. Which value the token carries needs a real login against production to settle and was not determined here. Two mitigations: docs/operations.md documents pinning authelia.issuer and jwksUrl, with the curl that reveals what the provider advertises for a given Host; and the authentication failure event now carries a specific reason, so id_token_issuer_mismatch is distinguishable from a signature failure or an unreachable key set. The browser still learns nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 11:41:42 +02:00
```
G10's status now names the live state, the outstanding requests with their
receipt ids, and the issuer precondition that must be checked before main is
deployed. The precondition was also sent to railiance-platform as the deployment
owner (message c8b1ad10), with the exact `curl` to settle it, the config keys to
pin, and the `error_type` to look for if it bites — plus a note that the two
approval clients remain unprovisioned, confirmed by inspection rather than
asserted. G10 stays open: nothing here admits custody, registers the human
Establish the live state and find a rollout precondition for G10 G10 waits on custody and platform owners and cannot close from here. What was doable: verify the handoffs actually went out, replace a remembered live state with an observed one, and find out whether main is safe to deploy. The last question found a defect in this repository's own recent work. Handoffs verified independently rather than trusted: all seven messages are in the hub with receipt ids. This gap was reopened once for claimed-but-unsent delivery, so the claim deserved the same scrutiny. Live state read from the cluster read-only: image main-153258b, only the Qonto secret materialized so the approval clients remain unprovisioned, four registered clients, no tenantEngine block. That also corrects an earlier claim of mine -- the deployed config sets userOU explicitly, so the KEY-WP-0023 default fix was never a production issue. The precondition: KEY-WP-0019 discovers the expected issuer from authelia.tokenBaseURL, and the deployed Authelia derives its advertised issuer from the request Host, advertising the in-cluster address to KeyCape and the browser-facing one to browsers. Verification fails closed, so a mismatch breaks every human login and looks like a broken login rather than a misconfiguration. Which value the token carries needs a real login against production to settle and was not determined here. Two mitigations: docs/operations.md documents pinning authelia.issuer and jwksUrl, with the curl that reveals what the provider advertises for a given Host; and the authentication failure event now carries a specific reason, so id_token_issuer_mismatch is distinguishable from a signature failure or an unreachable key set. The browser still learns nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV9oijZukGyGbRQGGKnK4P Assistant: claude-code Assistant-Model: opus Assistant-Process: 713576@bnt-lap001 Assistant-Session: 384c511d-9bce-4cb8-a676-2aef6c0c8df6
2026-09-08 11:41:42 +02:00
callback, or reconciles the OpenBao-token boundary, and no repo-local change can.