docs: finish PostgreSQL production cutover

This commit is contained in:
tegwick 2026-08-21 01:20:54 +02:00
parent a738a6bce4
commit 7a1725160f
3 changed files with 122 additions and 9 deletions

70
docs/intakes/residuals.md Normal file
View file

@ -0,0 +1,70 @@
# tenant-engine residual and finding intakes
Durable source records for actionable work that is not part of the workplan
being closed. State Hub is the live routing index.
---
## TEN-IN-0001 — Externalize tenant audit evidence to audit-core
```yaml
id: TEN-IN-0001
kind: intake
title: "Externalize tenant-engine audit evidence to audit-core"
lane: yellow
status: open
priority: high
owner: tenant-engine
repo: tenant-engine
origin: residual
origin_ref: TEN-WP-0009
description: |
TEN-WP-0009 made PostgreSQL the production authority but deliberately did
not add a second cross-service write path. The local events table shares a
database and runtime credential boundary with the facts it attests to, so a
database-credential compromise could forge both the fact and its local
evidence.
Design and implement a governed tenant-engine to audit-core emission path:
1. Agree the event envelope, sender registration and credential custody with
audit-core without putting a secret in Git or a work record.
2. Preserve mutation atomicity, idempotency and fail-closed behavior while
defining retry, duplicate and audit-core-unavailable semantics.
3. Prove the external copy cannot be rewritten through tenant-engine's
runtime database credential.
4. Decide explicitly whether the seven pre-cutover event classes require a
bounded backfill, and record the decision.
Promote this intake to a normal workplan before implementation because the
delivery semantics and cross-service dependency require design and review.
state_hub_intake_id: "01a02176-8ca7-725d-b2c2-4353eb4fd3f5"
```
---
## TEN-IN-0002 — Remove or authorize the unfiltered event-read interface
```yaml
id: TEN-IN-0002
kind: intake
title: "Remove or authorize the tenant-engine unfiltered event-read interface"
lane: red
status: open
priority: high
owner: tenant-engine
repo: tenant-engine
origin: risk-finding
origin_ref: RISK-F-0004
description: |
RISK-F-0004 correctly observes that TenantStore.events() returns every
tenant's event and payload. The accessibility part of the initial finding is
overstated: this is currently an in-process store interface used by tests,
and tenant-engine exposes no HTTP event-read route.
Make the boundary explicit by removing events() from the production protocol
or replacing it with an authorized, deliberately scoped export/query
interface. Add cross-tenant negative tests and give risk-nexus the final
accessibility and payload reading.
state_hub_intake_id: "01a02176-c121-7952-8d52-2cbe282fd883"
```

View file

@ -8,7 +8,7 @@ status: ready
owner: claude
topic_slug: tenant-guardrails
created: "2026-08-16"
updated: "2026-08-16"
updated: "2026-08-21"
depends_on:
- TEN-WP-0007
unblocks: []
@ -130,6 +130,14 @@ run.
Done when the stage commands in `app.toml` are ones that actually work against
this repo's deployment shape, verified in plan mode.
Constraint update 2026-08-21: TEN-WP-0009 removed the `ReadWriteOnce` volume
from the production Deployment, moved the workload to PostgreSQL and changed
the strategy to `RollingUpdate`. A side-by-side canary is now storage-feasible;
T02 must no longer encode the old "canary impossible" exception. The staged
contract still owns the exact canary mode and promotion commands. Current
production digest:
`sha256:a8e8086ffc5b772c1391b166f5e1884b90f7d327b152c205eceae129df555c24`.
## T03 - Record current production as the stable baseline
```task

View file

@ -4,11 +4,11 @@ type: workplan
title: "PostgreSQL as the production store, SQLite for dev and test"
domain: infotech
repo: tenant-engine
status: active
status: finished
owner: claude
topic_slug: tenant-lifecycle
created: "2026-08-16"
updated: "2026-08-16"
updated: "2026-08-21"
depends_on:
- TEN-WP-0007
unblocks:
@ -123,7 +123,7 @@ building a fleet drift-detector because we got bitten. Raised with
```task
id: TEN-WP-0009-T01
status: progress
status: done
priority: high
state_hub_task_id: "b4701256-f235-4474-9dcf-7cb09f62b873"
```
@ -337,6 +337,15 @@ covers per-database dumps, our single-consumer restore comes from there rather
than from a scratch-restored instance, and the slow path stops being the only
path. Asked.
Completed 2026-08-21. `Database/tenant-engine`, the namespace admission label,
exact-scope OpenBao runtime/migration roles, exact-namespace SecretStore and
split ExternalSecrets are live. The runtime lease was forcibly refreshed
without a pod restart, both local and live isolation suites passed 19/19, and
the shared placement remains within the reviewed four-consumer ceiling. The
move-to-dedicated triggers and honest backup/restore boundary remain recorded
above. Live evidence is in
`rapp-postgres/docs/evidence/tenant-engine-postgres-cutover-2026-08-21.md`.
## T02 - Implement `PostgresTenantStore`
```task
@ -421,7 +430,7 @@ are externally visible without disclosing a DSN.
```task
id: TEN-WP-0009-T04
status: progress
status: done
priority: high
state_hub_task_id: "51a83eae-9e9f-4676-bc1d-9c2d884bac0d"
```
@ -461,11 +470,20 @@ tenant lifecycle/version checks. Its PostgreSQL 16 integration test exercises
all tables and the next event sequence. The live scale-to-zero/export/import is
not performed and T04 remains progress.
Completed 2026-08-21. Writes were stopped before the source snapshot. The
transfer verified all seven physical tables by exact row comparison and
deterministic digest: 5 tenants, 13 events, 8 receipts, 3 guardrail changes, 1
override, and empty grants/plans. The known active v1 and retired v5 tenant
records were unchanged. Two failed attempts rolled back to an empty target;
the successful retry also proved the next event sequence. The stopped-write
SQLite snapshot hash and full per-table digests are recorded in the linked
rapp-postgres evidence.
## T05 - Cut over the deployment
```task
id: TEN-WP-0009-T05
status: progress
status: done
priority: high
state_hub_task_id: "03aeae3b-5e0f-4c77-a763-787ae08078f5"
```
@ -501,11 +519,21 @@ own lease and NetworkPolicy. Replicas deliberately remain one. The manifests
are not applied and the pinned old digest does not yet contain this code, so
T05 remains progress and no live PostgreSQL claim is made.
Completed 2026-08-21. Production is healthy with backend `postgresql` at
immutable digest
`sha256:a8e8086ffc5b772c1391b166f5e1884b90f7d327b152c205eceae129df555c24`.
The repo pin, Deployment spec and running image ID agree; `make verify-pin` and
the package `verify-live` target passed. The Deployment is `RollingUpdate`,
does not mount the SQLite PVC, keeps one deliberate replica, and retained all
five required routes plus the TEN-WP-0007 guardrail/lifecycle checks. The old
PVC is a time-bounded rollback artifact, not a current authority; its post-soak
retirement is `RAPP-IN-0005`.
## T06 - Tell the fleet, and close the loop on TEN-WP-0008
```task
id: TEN-WP-0009-T06
status: todo
status: done
priority: medium
state_hub_task_id: "e7adbc64-ad1b-4f28-bfd0-0b07034629db"
```
@ -520,6 +548,13 @@ state_hub_task_id: "e7adbc64-ad1b-4f28-bfd0-0b07034629db"
Done when the notes are sent and TEN-WP-0008-T02 is updated.
Completed 2026-08-21. Non-secret completion notices were sent to flex-auth
(State Hub message `6bb6f923-8909-45d0-9c57-2b6d7c3e41ad`) and user-engine
(`9512cf0c-f2f0-486b-a710-f737b9c79510`), stating the availability/storage
change and unchanged service contracts. TEN-WP-0008-T02 now records that the
RWO constraint is gone and that side-by-side canary execution is
storage-feasible.
## Out of scope
- **Removing SQLite.** It stays as the dev and test backend, and as the thing
@ -536,8 +571,8 @@ Done when the notes are sent and TEN-WP-0008-T02 is updated.
- **Emitting the audit trail to `audit-core`.** Identified under T01 as a real
gap — our `events` table shares a database with the rows it attests to, so
ADR-0001 §5 forges both together. It is a new cross-service dependency rather
than part of a store migration, and it needs its own workplan. Recorded here
so it is not rediscovered by accident.
than part of a store migration. Live residual `TEN-IN-0001` owns promotion
into a reviewed workplan; it is no longer parked only in this prose.
- **Per-tenant isolation.** A future plan tier may sell dedicated
infrastructure. `tenant-engine` already records plan assignment by id, and
`adaptive-pricing` owns what a plan *means*, so neither the tier definition