From 7a1725160f5ed3e4b7d14b360562935613e8b3dd Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 21 Aug 2026 01:20:54 +0200 Subject: [PATCH] docs: finish PostgreSQL production cutover --- docs/intakes/residuals.md | 70 +++++++++++++++++++ ...TEN-WP-0008-staged-promotion-onboarding.md | 10 ++- .../TEN-WP-0009-postgres-production-store.md | 51 +++++++++++--- 3 files changed, 122 insertions(+), 9 deletions(-) create mode 100644 docs/intakes/residuals.md diff --git a/docs/intakes/residuals.md b/docs/intakes/residuals.md new file mode 100644 index 0000000..95d616a --- /dev/null +++ b/docs/intakes/residuals.md @@ -0,0 +1,70 @@ +# tenant-engine residual and finding intakes + +Durable source records for actionable work that is not part of the workplan +being closed. State Hub is the live routing index. + +--- + +## TEN-IN-0001 โ€” Externalize tenant audit evidence to audit-core + +```yaml +id: TEN-IN-0001 +kind: intake +title: "Externalize tenant-engine audit evidence to audit-core" +lane: yellow +status: open +priority: high +owner: tenant-engine +repo: tenant-engine +origin: residual +origin_ref: TEN-WP-0009 +description: | + TEN-WP-0009 made PostgreSQL the production authority but deliberately did + not add a second cross-service write path. The local events table shares a + database and runtime credential boundary with the facts it attests to, so a + database-credential compromise could forge both the fact and its local + evidence. + + Design and implement a governed tenant-engine to audit-core emission path: + + 1. Agree the event envelope, sender registration and credential custody with + audit-core without putting a secret in Git or a work record. + 2. Preserve mutation atomicity, idempotency and fail-closed behavior while + defining retry, duplicate and audit-core-unavailable semantics. + 3. Prove the external copy cannot be rewritten through tenant-engine's + runtime database credential. + 4. Decide explicitly whether the seven pre-cutover event classes require a + bounded backfill, and record the decision. + + Promote this intake to a normal workplan before implementation because the + delivery semantics and cross-service dependency require design and review. +state_hub_intake_id: "01a02176-8ca7-725d-b2c2-4353eb4fd3f5" +``` + +--- + +## TEN-IN-0002 โ€” Remove or authorize the unfiltered event-read interface + +```yaml +id: TEN-IN-0002 +kind: intake +title: "Remove or authorize the tenant-engine unfiltered event-read interface" +lane: red +status: open +priority: high +owner: tenant-engine +repo: tenant-engine +origin: risk-finding +origin_ref: RISK-F-0004 +description: | + RISK-F-0004 correctly observes that TenantStore.events() returns every + tenant's event and payload. The accessibility part of the initial finding is + overstated: this is currently an in-process store interface used by tests, + and tenant-engine exposes no HTTP event-read route. + + Make the boundary explicit by removing events() from the production protocol + or replacing it with an authorized, deliberately scoped export/query + interface. Add cross-tenant negative tests and give risk-nexus the final + accessibility and payload reading. +state_hub_intake_id: "01a02176-c121-7952-8d52-2cbe282fd883" +``` diff --git a/workplans/TEN-WP-0008-staged-promotion-onboarding.md b/workplans/TEN-WP-0008-staged-promotion-onboarding.md index e436609..e61cac7 100644 --- a/workplans/TEN-WP-0008-staged-promotion-onboarding.md +++ b/workplans/TEN-WP-0008-staged-promotion-onboarding.md @@ -8,7 +8,7 @@ status: ready owner: claude topic_slug: tenant-guardrails created: "2026-08-16" -updated: "2026-08-16" +updated: "2026-08-21" depends_on: - TEN-WP-0007 unblocks: [] @@ -130,6 +130,14 @@ run. Done when the stage commands in `app.toml` are ones that actually work against this repo's deployment shape, verified in plan mode. +Constraint update 2026-08-21: TEN-WP-0009 removed the `ReadWriteOnce` volume +from the production Deployment, moved the workload to PostgreSQL and changed +the strategy to `RollingUpdate`. A side-by-side canary is now storage-feasible; +T02 must no longer encode the old "canary impossible" exception. The staged +contract still owns the exact canary mode and promotion commands. Current +production digest: +`sha256:a8e8086ffc5b772c1391b166f5e1884b90f7d327b152c205eceae129df555c24`. + ## T03 - Record current production as the stable baseline ```task diff --git a/workplans/TEN-WP-0009-postgres-production-store.md b/workplans/TEN-WP-0009-postgres-production-store.md index e1f397a..4fe7fa0 100644 --- a/workplans/TEN-WP-0009-postgres-production-store.md +++ b/workplans/TEN-WP-0009-postgres-production-store.md @@ -4,11 +4,11 @@ type: workplan title: "PostgreSQL as the production store, SQLite for dev and test" domain: infotech repo: tenant-engine -status: active +status: finished owner: claude topic_slug: tenant-lifecycle created: "2026-08-16" -updated: "2026-08-16" +updated: "2026-08-21" depends_on: - TEN-WP-0007 unblocks: @@ -123,7 +123,7 @@ building a fleet drift-detector because we got bitten. Raised with ```task id: TEN-WP-0009-T01 -status: progress +status: done priority: high state_hub_task_id: "b4701256-f235-4474-9dcf-7cb09f62b873" ``` @@ -337,6 +337,15 @@ covers per-database dumps, our single-consumer restore comes from there rather than from a scratch-restored instance, and the slow path stops being the only path. Asked. +Completed 2026-08-21. `Database/tenant-engine`, the namespace admission label, +exact-scope OpenBao runtime/migration roles, exact-namespace SecretStore and +split ExternalSecrets are live. The runtime lease was forcibly refreshed +without a pod restart, both local and live isolation suites passed 19/19, and +the shared placement remains within the reviewed four-consumer ceiling. The +move-to-dedicated triggers and honest backup/restore boundary remain recorded +above. Live evidence is in +`rapp-postgres/docs/evidence/tenant-engine-postgres-cutover-2026-08-21.md`. + ## T02 - Implement `PostgresTenantStore` ```task @@ -421,7 +430,7 @@ are externally visible without disclosing a DSN. ```task id: TEN-WP-0009-T04 -status: progress +status: done priority: high state_hub_task_id: "51a83eae-9e9f-4676-bc1d-9c2d884bac0d" ``` @@ -461,11 +470,20 @@ tenant lifecycle/version checks. Its PostgreSQL 16 integration test exercises all tables and the next event sequence. The live scale-to-zero/export/import is not performed and T04 remains progress. +Completed 2026-08-21. Writes were stopped before the source snapshot. The +transfer verified all seven physical tables by exact row comparison and +deterministic digest: 5 tenants, 13 events, 8 receipts, 3 guardrail changes, 1 +override, and empty grants/plans. The known active v1 and retired v5 tenant +records were unchanged. Two failed attempts rolled back to an empty target; +the successful retry also proved the next event sequence. The stopped-write +SQLite snapshot hash and full per-table digests are recorded in the linked +rapp-postgres evidence. + ## T05 - Cut over the deployment ```task id: TEN-WP-0009-T05 -status: progress +status: done priority: high state_hub_task_id: "03aeae3b-5e0f-4c77-a763-787ae08078f5" ``` @@ -501,11 +519,21 @@ own lease and NetworkPolicy. Replicas deliberately remain one. The manifests are not applied and the pinned old digest does not yet contain this code, so T05 remains progress and no live PostgreSQL claim is made. +Completed 2026-08-21. Production is healthy with backend `postgresql` at +immutable digest +`sha256:a8e8086ffc5b772c1391b166f5e1884b90f7d327b152c205eceae129df555c24`. +The repo pin, Deployment spec and running image ID agree; `make verify-pin` and +the package `verify-live` target passed. The Deployment is `RollingUpdate`, +does not mount the SQLite PVC, keeps one deliberate replica, and retained all +five required routes plus the TEN-WP-0007 guardrail/lifecycle checks. The old +PVC is a time-bounded rollback artifact, not a current authority; its post-soak +retirement is `RAPP-IN-0005`. + ## T06 - Tell the fleet, and close the loop on TEN-WP-0008 ```task id: TEN-WP-0009-T06 -status: todo +status: done priority: medium state_hub_task_id: "e7adbc64-ad1b-4f28-bfd0-0b07034629db" ``` @@ -520,6 +548,13 @@ state_hub_task_id: "e7adbc64-ad1b-4f28-bfd0-0b07034629db" Done when the notes are sent and TEN-WP-0008-T02 is updated. +Completed 2026-08-21. Non-secret completion notices were sent to flex-auth +(State Hub message `6bb6f923-8909-45d0-9c57-2b6d7c3e41ad`) and user-engine +(`9512cf0c-f2f0-486b-a710-f737b9c79510`), stating the availability/storage +change and unchanged service contracts. TEN-WP-0008-T02 now records that the +RWO constraint is gone and that side-by-side canary execution is +storage-feasible. + ## Out of scope - **Removing SQLite.** It stays as the dev and test backend, and as the thing @@ -536,8 +571,8 @@ Done when the notes are sent and TEN-WP-0008-T02 is updated. - **Emitting the audit trail to `audit-core`.** Identified under T01 as a real gap โ€” our `events` table shares a database with the rows it attests to, so ADR-0001 ยง5 forges both together. It is a new cross-service dependency rather - than part of a store migration, and it needs its own workplan. Recorded here - so it is not rediscovered by accident. + than part of a store migration. Live residual `TEN-IN-0001` owns promotion + into a reviewed workplan; it is no longer parked only in this prose. - **Per-tenant isolation.** A future plan tier may sell dedicated infrastructure. `tenant-engine` already records plan assignment by id, and `adaptive-pricing` owns what a plan *means*, so neither the tier definition