tegwick
0f5f56275c
platform-root login is restored and the resolver holds the current bind
credential, so T04's outstanding reconciliation is done. No green receipt
yet — the run of record is a FAIL at resolver-lookup, and T05 stays open.
Records four things the session established:
- the predecessor is dead, observed twice by hand, but not by receipt;
- the value is unrecoverable, because no KeePassXC database has ever
existed despite platform-root-custody.md naming a safe entry, and the
only copy lived in a Firefox entry overwritten during the session.
T04 replaced the credential with no step to update operator custody —
the root cause of the whole session;
- the reconciliation script had never completed a run (4a38511);
- verify-t06.sh reported success at bootstrap against a resolver that
was misconfigured and a reconciliation path that could not execute.
That blind spot is open and is the finding worth acting on.
Also corrects the record: the stale bind credential was real but did not
cause the lookup failure. The HTTP 400 was our own request builder.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 3377672@bnt-lap001
Assistant-Session: 15463ccf-238f-4e13-b163-93aa25c6d166
220 lines
10 KiB
Markdown
220 lines
10 KiB
Markdown
---
|
|
id: NK-WP-0033
|
|
type: workplan
|
|
title: "Contain and rotate the exposed KeyCape credential bundle"
|
|
domain: infotech
|
|
repo: net-kingdom
|
|
status: active
|
|
owner: codex
|
|
topic_slug: net-kingdom
|
|
planning_priority: P0
|
|
created: "2026-08-23"
|
|
updated: "2026-08-23"
|
|
origin: routed
|
|
origin_ref: "State Hub message 8cc44a39-683c-4fab-80dd-b2275d0728e0"
|
|
related:
|
|
- NK-WP-0032
|
|
state_hub_workstream_id: "63665674-6880-593e-96e6-bab3211b1352"
|
|
---
|
|
|
|
# NK-WP-0033 — KeyCape credential exposure containment and rotation
|
|
|
|
KeyCape reported that an agent tool rendered the base64 data map of the live
|
|
`sso/keycape-config` Secret. Base64 is encoding, not protection. Treat the
|
|
carried RS256 signing private key, LLDAP bind credential, Authelia client
|
|
credential, and privacyIDEA admin token as exposed even though no value was
|
|
decoded, reused, or intentionally repeated.
|
|
|
|
No task in this workplan authorizes a credential value in Git, State Hub,
|
|
agent output, command arguments, or non-secret evidence. Live mutation requires
|
|
the exact provider/custody owners, a reviewed sequence, abort and rollback
|
|
conditions, and the authority stated by the owning execution path.
|
|
|
|
## T01 — Contain, classify, and route the exposure
|
|
|
|
```task
|
|
id: NK-WP-0033-T01
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "28a655da-7cd6-58e2-8ca8-64d192de63a6"
|
|
```
|
|
|
|
Mark every credential class in the rendered bundle exposed, stop further
|
|
rendering, consult credential routing, notify custody and issuer owners, and
|
|
publish a sanitized risk notice.
|
|
|
|
Completed 2026-08-23. State Hub decision
|
|
`c43eef14-a209-41b1-ba67-781ca14c5ea2` requires governed rotation. Coordination
|
|
was routed to railiance-platform in `acf98be3-ff6b-4270-bd21-0193bebd806b`
|
|
and KeyCape in `c774f748-6240-4733-89c7-518079a30f61`; risk-nexus received
|
|
sanitized notice `c5d67801-1bc3-4e66-98c9-e08a58313970`. No value was inspected
|
|
or copied.
|
|
|
|
## T02 — Agree the value-safe rotation and issuer-continuity contract
|
|
|
|
```task
|
|
id: NK-WP-0033-T02
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "c5ca817c-d601-5172-a106-1546c0743635"
|
|
```
|
|
|
|
KeyCape and railiance-platform must name the authoritative provider, custody,
|
|
and consumer update for all four classes; select one revision-pinned bundle or
|
|
field-safe sequence that cannot replay a stale exposed value; specify positive
|
|
and predecessor-negative verification; and state abort, rollback, and evidence
|
|
rules.
|
|
|
|
KeyCape currently loads one private key and signs under static `kid=key-1` even
|
|
though its JWKS type can serve multiple keys. The contract must either add and
|
|
verify an active-key plus retained-public-key overlap or deliberately accept
|
|
immediate token invalidation and verifier refresh. Evidence was routed in
|
|
`2b24ccc2-c031-453e-ac70-52f0aaa127d5` and
|
|
`9277d962-b511-4da7-9fc6-2c79bac448c1`.
|
|
|
|
Railiance-platform published its forward-only owner contract at revision
|
|
`453fed3`. NetKingdom review is recorded in
|
|
`history/2026-08-23-keycape-exposure-dependency-map.md`. The review found that
|
|
the LLDAP credential also feeds identity-provisioner and privacyIDEA's
|
|
persisted LDAP resolver. It also found that privacyIDEA `/auth` issues an
|
|
expiring administrative session JWT with no individual revocation operation;
|
|
`/token/revoke` applies to MFA authenticators, not that JWT. The owner recovery
|
|
selected the wider privacyIDEA signing-secret invalidation path, so predecessor
|
|
JWT denial is covered by the live cutover outcome.
|
|
|
|
Completed 2026-08-23. KeyCape reported that its owner-controlled recovery
|
|
replaced all four exposed classes, including the privacyIDEA signing secret,
|
|
and that positive/negative checks passed. The live outcome resolves the
|
|
predecessor disposition; sanitized receipts remain in the owning workplan.
|
|
|
|
Warden routing has no concrete entries for these four KeyCape lanes. Catalog
|
|
repair was requested in `e5794329-cda5-4a6f-8234-a6eaa1fc318c`; generic or
|
|
unrelated routes do not authorize execution.
|
|
|
|
## T03 — Implement a non-printing, non-stale rotation path
|
|
|
|
```task
|
|
id: NK-WP-0033-T03
|
|
status: progress
|
|
priority: high
|
|
state_hub_task_id: "f0f6d6c3-9c45-56b7-9fd6-52fb0ea9054a"
|
|
```
|
|
|
|
After T02 review, implement or select an owner-controlled path that receives
|
|
replacement material through hidden input or mode-0600 files, updates every
|
|
provider and consumer in the agreed order, never reconstructs the bundle from
|
|
stale local inputs, and retains no plaintext artifact after verification.
|
|
|
|
Provider-specific helpers must cover field-safe updates for `lldap-secrets`
|
|
and `authelia-secrets`, the persistent LLDAP admin password, the
|
|
identity-provisioner restart, privacyIDEA resolver `lldap-coulomb`, the
|
|
Authelia client hash, and the privacyIDEA JWT expiry disposition. Static or
|
|
dry-run tests must prove replacements do not enter argv or stdout.
|
|
|
|
The source-of-truth reconciliation completed in revision `c956ceb`: Authelia
|
|
4.38 OIDC client entries cannot consume `*_FILE` environment overrides because
|
|
clients are a list, so the verifier now uses Authelia's supported template
|
|
filter over the Secret-mounted file. The revision was applied live and
|
|
Authelia returned Ready with startup complete.
|
|
|
|
NetKingdom now provides the supported one-command attended operation in
|
|
`sso-mfa/k8s/privacyidea/reconcile-lldap-resolver-live.sh`. It is explicitly
|
|
gated by `--apply`, requires an interactive terminal, uses protected 0600
|
|
files, updates only `lldap-coulomb`, proves replacement resolver lookup and
|
|
privacyIDEA MFA, proves predecessor LLDAP denial with explicit 401/403 status,
|
|
rechecks readiness/health, and emits one sanitized PASS/FAIL receipt after
|
|
cleanup. It has not been run; the exact attended runbook is pinned in
|
|
`docs/keycape-exposure-resolver-reconciliation.md`.
|
|
|
|
Do not use `sso-mfa/bootstrap/creds-rotate.sh` through an agent as currently
|
|
written: it prints generated replacement values and its signing-key path
|
|
causes immediate invalidation. Do not use a helper that reads the live Secret
|
|
unless its output and temporary-state behavior have been independently
|
|
reviewed for this incident.
|
|
|
|
## T04 — Execute the governed replacement cutover
|
|
|
|
```task
|
|
id: NK-WP-0033-T04
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "751cca48-2bc9-5d44-80a0-60478717e99e"
|
|
```
|
|
|
|
Under the separately approved authority/window, replace the signing material,
|
|
LLDAP bind credential, Authelia client credential, and privacyIDEA admin token;
|
|
roll or reload every affected provider/consumer; preserve or explicitly end
|
|
JWT continuity according to T02; and abort on partial-bundle or stale-source
|
|
evidence. The task was gated on the owner-controlled T02 contract and is now
|
|
complete.
|
|
|
|
Completed by the KeyCape owner-controlled recovery path on 2026-08-23. The
|
|
owner reported all four affected deployments Ready and positive/negative
|
|
checks passing. NetKingdom additionally restarted identity-provisioner after
|
|
the LLDAP bind rotation and applied the Authelia template-filter fix. The
|
|
privacyIDEA resolver still awaits attended provider-admin reconciliation.
|
|
|
|
## T05 — Prove replacement, predecessor rejection, and cleanup
|
|
|
|
```task
|
|
id: NK-WP-0033-T05
|
|
status: progress
|
|
priority: high
|
|
state_hub_task_id: "41e55d5c-ae28-5ee2-be25-b9a3758428f7"
|
|
```
|
|
|
|
Record only sanitized identifiers, public-key fingerprints, revisions,
|
|
rollout status, positive authentication/MFA outcomes, predecessor rejection or
|
|
expiry, cleanup receipts, and residual limitations. Close the incident only
|
|
after all four credential classes have an authoritative disposition and no
|
|
temporary plaintext or stale bundle remains.
|
|
|
|
### Attended session 2026-08-27 — findings
|
|
|
|
The resolver reconciliation was attempted and the operator's `platform-root`
|
|
login is restored. `bao login -method=oidc` succeeds, and
|
|
`GET /user/?realm=coulomb&username=platform-root` returns the record. **No
|
|
green receipt has been produced yet**; the run below is the state of record.
|
|
|
|
**LLDAP bind credential — reconciled.** The `--apply` run wrote the current
|
|
`LLDAP_LDAP_USER_PASS` into resolver `lldap-coulomb`, and the replacement
|
|
authenticated against LLDAP in the same run. The resolver had been holding the
|
|
pre-cutover value since 2026-08-23, which is what T04 left outstanding.
|
|
|
|
**Predecessor disposition — NOT PROVEN by receipt; observed twice by hand.**
|
|
The exposed value is dead: the operator was locked out of the LLDAP WebUI on
|
|
2026-08-27 after logging out of a persisted session, and the `--check` run the
|
|
same day passed `predecessor-denial` with the real value before it was lost.
|
|
Neither observation is carried by an emitted receipt. The value is
|
|
unrecoverable — see the custody gap below.
|
|
|
|
**Custody gap — the actual root cause of this session.** T04 replaced the
|
|
credential without a step that updates operator custody, and there was no
|
|
custody store to update it into. `platform-root-custody.md` names a password
|
|
safe entry `net-kingdom/LLDAP/admin`; **no KeePassXC database exists** and never
|
|
did. The value lived only in a Firefox password-manager entry dated 2026-06-28,
|
|
which was overwritten with the replacement on 2026-08-27, destroying the
|
|
predecessor. Any rotation runbook that does not name where the outgoing value is
|
|
retained will lose it the same way.
|
|
|
|
**The reconciliation script had never completed a run.** Four defects, fixed in
|
|
`4a38511`: `Content-Type` on bodyless GETs (HTTP 400 on every GET), `/user/`
|
|
parsed as a dict when it returns a list (traceback past the except clause),
|
|
resolver writes dropping `TIMEOUT`/`CACHE_TIMEOUT`/`SIZELIMIT` (silently
|
|
un-repairing a hand-fixed resolver), and no way to declare an unavailable
|
|
predecessor (forcing a placeholder that records a *passing* denial proof).
|
|
|
|
**Verification blind spot — open.** `verify-t06.sh` reported success at
|
|
bootstrap against a resolver created without the three tuning parameters, by a
|
|
reconciliation path that could not execute. A verification that passes while its
|
|
subject cannot run is not verification. This is the finding worth acting on
|
|
beyond the four fixes, and it is not yet addressed.
|
|
|
|
**Diagnosis correction.** The "stale bind credential" reading in T04 was real but
|
|
was not what blocked the resolver lookup; the HTTP 400 was our own request
|
|
builder throughout. The two faults were independent and looked like one, which
|
|
is why each partial fix appeared to change nothing.
|
|
|
|
Remaining before T05 can close: a green receipt from the repaired script, and a
|
|
ruling on whether a predecessor disposition observed but not receipted is an
|
|
acceptable close for this incident.
|