docs: record provider incident and core hub rotation
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

This commit is contained in:
codex 2026-08-20 22:47:38 +02:00
parent 131502e6e7
commit ceed3c051d
2 changed files with 57 additions and 1 deletions

View file

@ -4,7 +4,7 @@ type: workplan
title: "llm-connect OpenRouter Provider Key Lane"
domain: financials
repo: railiance-platform
status: finished
status: active
owner: codex
topic_slug: railiance
planning_priority: high
@ -290,6 +290,51 @@ Acceptance:
- Compromise response names the immediate front-door disable, provider-key
rotation, and follow-up incident workplan path.
## T08 - Reconcile the railiance01 consumer after provider 401
```task
id: RAILIANCE-WP-0010-T08
status: wait
priority: high
```
Restore the approved OpenBao-to-ESO delivery path for the railiance01
llm-connect instance after its bootstrap key began returning OpenRouter 401.
Read-only diagnosis on 2026-08-20 found a healthy, 61-day-old Deployment using
an unmanaged Secret created on 2026-06-19. No `ExternalSecret` exists on
railiance01. `ClusterSecretStore/openbao-activity-core` is Valid, but its ESO
token reports `deny` for both the canonical llm-connect data and metadata
paths because the activity-core token bootstrap default omits
`workload-kv-read-llm-connect-provider-secrets`.
Repair the durable bootstrap policy set, mint a replacement limited ESO token
through the existing attended platform-admin handoff, apply the already
reviewed llm-connect `ExternalSecret`, and wait for `SecretSynced=True`. Restart
the Deployment only after sync, then run the in-cluster fixture smoke. If the
OpenBao-delivered key still returns 401, stop and require provider-side
rotation; do not copy or inspect either key value.
Done when railiance01 consumes the canonical lane, the fixture smoke passes,
and rollback/non-secret evidence is recorded.
Progress 2026-08-20: repaired the durable delivery path. The activity-core ESO
bootstrap now includes the approved exact-path llm-connect policy; a new
periodic token was minted through the existing mode-0600 platform-admin
handoff, and capabilities report `read` on only the requested data and metadata
paths. The reviewed llm-connect `ExternalSecret` was applied on railiance01,
reported `Ready=True` / `SecretSynced`, and took ownership of the one-key
Secret. `deployment/llm-connect` rolled out successfully, and the real
`actcore-worker` reaches its Service `/health` with `status=ok`.
The full fixture still returns sanitized OpenRouter HTTP 401 when run inside
the new llm-connect pod. Therefore the canonical provider key itself is no
longer accepted. T08 waits on an attended OpenRouter account owner to mint a
replacement key. Keep the old key active until the new value is written from a
mode-0600 file to the existing KV path, ESO reports a new successful refresh,
the Deployment is restarted, and the fixture passes; then revoke the old key.
No provider-key value was read or printed during diagnosis or repair.
## Exit Criteria
- `CCR-2026-0003` is reviewed, approved, applied, verified, and active.

View file

@ -166,6 +166,17 @@ Prefer the mounted-file approach for steady state. A time-bounded automatic
restart contract may be used for the relocation only if rollback and lease
overlap are proven and the temporary limitation is recorded.
Progress 2026-08-20: package chart `0.3.0` implements the bounded relocation
contract. Static API, runtime database, and migration database credentials are
mounted from three disjoint Secrets; the database URL is constructed only in
the supervised child environment. A PID 1 watcher fingerprints mounted files
without logging values and exits 75 on change so Kubernetes restarts the API
on the refreshed lease. The migration Job mounts only its role. Seven package
tests prove separation, URL encoding, no credential output, and rotation exit;
Helm lint, family validation, normal server dry-run, and migration-enabled
server dry-run pass. T03 remains `progress` until T04 proves a live ESO refresh,
container replacement, and readiness before the old runtime lease expires.
## T04 — Declare and apply the OpenBao/ESO lanes
```task