Prepare railiance01 delivery: dynamic leases, migrate Job, operator runbook
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s

VaultDynamicSecret pulls database/creds/* so a rotating lease is not frozen
into KV. Runtime sets AUDIT_CORE_AUTO_MIGRATE=0; schema is a Job with the
migration lease. Image base is digest-pinned. Namespace and NetworkPolicies
are on the cluster; Deployment waits for the attended OpenBao ESO token.
This commit is contained in:
tegwick 2026-08-13 00:58:49 +02:00
parent bbf86b8373
commit 3a7d63e18f
18 changed files with 826 additions and 33 deletions

View file

@ -8,7 +8,7 @@ status: active
owner: codex
topic_slug: netkingdom
created: "2026-08-10"
updated: "2026-08-10"
updated: "2026-08-13"
depends_on:
- AUDIT-WP-0004
- RAPP-POSTGRES-WP-0002
@ -179,10 +179,37 @@ after confirming no active sessions; the group roles are untouched and remain
NOLOGIN. The harness now uses a per-run random password, `VALID UNTIL` one
hour, and an exit trap that drops the roles in every mode including on failure.
Remaining before done: the `openbao-audit-core` ClusterSecretStore and the
`rapp-postgres/audit-core-runtime` OpenBao role are prerequisites owned by
rapp-postgres/railiance-platform, not this repo. Once they exist, apply and
verify a live lease and a live rotation.
Progress 2026-08-13: the in-cluster delivery path is now honest about what
kind of secret it is pulling.
The previous ExternalSecret pointed at KV
`platform/workloads/audit-core/database/audit-core-runtime`. That path does
not exist, and copying a dynamic lease into KV would freeze it — the same
delivery-gap problem the mounted directory exists to avoid. Database
credentials now come from ESO `VaultDynamicSecret` against
`database/creds/audit-core-runtime` and `database/creds/audit-core-migration`,
refreshed every 15m into one-file-per-field Secrets. The sender registry
stays on ClusterSecretStore `openbao-audit-core` (KV), which is the right
tool for a document.
A second finding: `PostgresAuditBackend` applied schema on every process
start. The runtime role cannot `CREATE TABLE` (isolation-test, ADR-0001).
Production now sets `AUDIT_CORE_AUTO_MIGRATE=0`; `python -m audit_core
migrate` runs as Job `audit-core-migrate` with the migration lease.
Package side in this repo: `deploy/clustersecretstore.yaml`,
`deploy/vaultdynamicsecrets.yaml`, `deploy/migrate-job.yaml`,
`scripts/openbao-eso-token-apply.sh`. Platform side drafted in
`railiance-platform` (`openbao/policies/external-secrets-audit-core.hcl` and
the ClusterSecretStore add-on). The OpenBao database roles themselves were
already delivered by RAPP-POSTGRES-WP-0002-T04.
Remaining before done: attended `platform-admin` OpenBao login to write the
ESO policy, mint `external-secrets/openbao-audit-core-eso-token`, and
`bao kv put` the sender registry. Then apply CSS + generators + ExternalSecrets
and verify a live lease plus a live rotation against the mounted directory.
That step is flagged `needs_human` — this session has no token that can write
policies.
## T03 - Deploy the receiver
@ -261,8 +288,25 @@ other scrape target, so an exposition endpoint would be built for a scrape path
that does not exist. This is usable with curl today and a small step from
`/metrics` when a metrics stack lands.
Remaining before done: build and publish the image to get a real digest, apply,
and verify restart/rescheduling and the rollback path on the cluster.
Progress 2026-08-13: Containerfile base is digest-pinned
(`python:3.12-slim@sha256:d764629ce0…`, same digest as the deployed
user-engine image) and labelled with `org.opencontainers.image.revision`.
`make image-build` / `make image-publish` exist. Namespace `audit-core`
(with `railiance.io/postgres-client: platform-pg`) and the four
NetworkPolicies are applied on railiance01. Remaining manifests validate
`--dry-run=server --validate=strict`. Deployment is not applied: it still
carries `sha256:REPLACE_AT_RELEASE` until the ESO secrets exist, and
applying it now would ImagePullBackOff and then CrashLoop on a missing
sender registry.
An image was published to `forgejo.coulomb.social/coulomb/audit-core` during
this session; rebuild and pin after the commit that contains these files so
the label matches the content.
Remaining before done: pin the post-commit digest in
`deploy/audit-core.yaml` and `deploy/migrate-job.yaml`, apply migrate Job
then Deployment once T02 secrets exist, and verify restart, reschedule, and
`kubectl rollout undo`.
## T04 - Migrate existing SQLite records
@ -389,7 +433,7 @@ OpenBao-leased credentials.
```task
id: AUDIT-WP-0005-T06
status: todo
status: progress
priority: medium
state_hub_task_id: "0856c80d-abe1-4bff-ba8d-87295cf76819"
```
@ -404,3 +448,17 @@ provides — the retention window audit-core declares must not exceed the
retention the platform guarantees.
Done when the runbook exists and the restore path has been walked once.
Progress 2026-08-13: `docs/operator-runbook.md` covers lookup by correlation
id, dead-letter inspection, accepted-event replay (`python -m audit_core
replay`), overlap-first sender rotation, restart-free database lease
rotation, `/readyz` vs `/healthz` vs `/v1/stats`, and rollback.
`python -m audit_core replay` is the operator surface that WP-0004
deliberately left off HTTP.
Restore is written down and is **not** walked. rapp-postgres T06 is `wait`
on an S3 Barman target; the Nextcloud logical-dump lane cannot provide WAL
or PITR; physical restore is instance-wide. audit-core currently declares
no finite `retention_days`, so it does not outrun a platform window that
does not yet exist — but it also cannot promise an RPO until the platform
drills land. The walk listed in the runbook is what closes this task.