diff --git a/Makefile b/Makefile index e1b1461..5a9ce85 100644 --- a/Makefile +++ b/Makefile @@ -193,3 +193,10 @@ agents-update: agents-validate: @echo "Validating agents..." @kaizen-agentic validate agents/ + +.PHONY: artifact-adr008 +artifact-adr008: + python3 tools/render-artifact.py canon/architecture/adr-008-multi-tenancy-model.md \ + --output build/adr-008.html \ + --title "Tenancy Posture" \ + --subtitle "A framework for describing, holding and improving multi-tenancy — including where we are not there yet." diff --git a/canon/architecture/adr-008-multi-tenancy-model.md b/canon/architecture/adr-008-multi-tenancy-model.md index 91c6e55..a49bef1 100644 --- a/canon/architecture/adr-008-multi-tenancy-model.md +++ b/canon/architecture/adr-008-multi-tenancy-model.md @@ -1,554 +1,34 @@ --- id: ADR-008 type: architecture-decision-record -title: "Multi-Tenancy Framework: Four Planes, Graduated Levels, Declared Posture" -status: proposed +title: "Multi-Tenancy Framework — relocated to NetKingdom" +status: superseded decided_by: Bernd Worsch date: "2026-08-17" -revision: "draft-2" -tags: ["architecture", "multi-tenancy", "isolation", "placement", "maturity", "tenant-engine", "flex-auth", "rapp-postgres", "scaling"] --- -# ADR-008: Multi-Tenancy Framework — Four Planes, Graduated Levels, Declared Posture +# ADR-008 — relocated -## Status +Drafts 1–4 of the multi-tenancy framework were written here. On 2026-08-17 the +operator determined that multi-tenancy is part of the IT-security framework +NetKingdom provides, so the framework belongs in NetKingdom canon beside the +IAM Profile and the tenant-engine boundary contract. -**Proposed, draft-2.** Draft-1 (2026-08-17) proposed a single model with fixed -characteristics. It was rejected on the grounds that the estate does not need -one model of multi-tenancy — it needs a framework able to describe, hold, and -improve several situations at once, including repos that do not separate -tenants rigorously today and must be developed toward doing so. Draft-2 is -that reframe. +**It now lives at `net-kingdom/canon/standards/tenancy-posture_v0.1.md`.** -This document decides things currently owned by `tenant-engine`, `flex-auth`, -`rapp-postgres`, `adaptive-pricing`, and `railiance-platform`. It must be -reviewed by those owners before ratification; §18 lists what each is being -asked to accept. +Two things changed on relocation: -## 1. Context +1. The five dimensions were renamed from **planes** to **axes**. NetKingdom's + accepted `platform-identity-security-architecture` already uses *plane* for + a trust and deployment layer (bootstrap / platform control / tenant), and + two senses of one word in one canon is a concept-ownership collision. +2. The opening claim that the estate "has never written down what it is + building" was corrected. That architecture document has described the trust + and tenant model since 2026-07-23. What was actually missing is a way to say + how far a given service has got. -The estate has been building multi-tenancy for months and has never written -down what it is building. Five documents each cover a slice: +This stub remains so that the ADR-008 identifier resolves rather than +dangling. It is not a second copy and must not be edited as one. -| Document | Covers | Status | -|---|---|---| -| `iam-profile_v0.3` (NetKingdom) | Tenant identifier shape, `tenant_roles` claim, staleness rules | Ratified | -| `tenant-engine-boundary-contract_v0.1` (NetKingdom) | Who owns tenant records, roles, plan assignment | Ratified | -| `business-app-service-contract_v0.1` §1 (Custodian) | Business apps: instance-per-client, tenant-keyed data | Ratified | -| `rapp-postgres/docs/adr/ADR-0001` | Consumer + tenant isolation in PostgreSQL | Proposed, governs one repo | -| `rapp-postgres/docs/canon-drafts/shared-platform-relational-storage_v0.1` | The stacked-boundary gap | Routed 2026-08-10, **still unratified** | - -Four failures follow. - -**The gap has been diagnosed once and the fix stalled.** The -shared-platform-relational-storage draft was written to fill this hole. It has -sat unratified for a week and appears in neither canon directory. §19 attaches -a ratification path so this one does not join it. - -**Placement is owned by nobody.** `user-engine-pg` and `target-revenue-pg` are -dedicated; `apps-pg`, `net-kingdom-pg`, `platform-pg`, `state-hub-db` and -`forgejo-db` are shared. Both patterns are live, neither is written down, and -each new service copies whichever neighbour it consulted. `tenant-engine` -raised this with `railiance-platform` on 2026-08-16; unanswered. - -**Two contradictory tenancy defaults are already ratified.** -`business-app-service-contract` §1.2 makes instance-per-client the default for -business apps. Platform services pool all tenants into one database. Nothing -states which shape a new service takes, and no definition distinguishes the -two categories. - -**There is no honest way to describe a repo that is not there yet.** This is -the failure draft-1 missed. The estate absorbs and will keep absorbing repos -with weak or absent tenant separation — app-local accounts, no tenant column, -ad-hoc filtering. Today such a repo is simply non-conformant, which gives it -two bad options: misrepresent its posture, or stay outside the framework -entirely. Both are worse than a framework that can say "this repo is at level -1, its target is level 3, here is the gap and the evidence." - -## 2. What this document is - -**A framework, not a model.** It does not specify one correct multi-tenancy -implementation with a fixed set of characteristics. It supplies: - -- **Terminology** — four planes, and graduated levels within each (§3, §4). -- **A declaration** — the posture vector, so any repo can state where it - actually is (§5). -- **A conformance rule** — accuracy, not altitude (§6). -- **Methodology** — analyze, establish, improve, guard (§12). -- **Evidence definitions** — what proves a claimed level (§13). - -A service is conformant when its declared posture is accurate and its -trajectory is recorded. A service is non-conformant when it claims a level it -cannot evidence — regardless of how high or low that level is. - -## 3. Four orthogonal planes - -"Is this multi-tenant?" is treated as one question. It is four, and they are -independent: - -| Plane | Question | Vocabulary owner | -|---|---|---| -| **Identity (I)** | How is a tenant named and validated? | `tenant-engine` / IAM Profile | -| **Authorization (A)** | How is a request bound to the tenants it may act for? | `flex-auth` | -| **Enforcement (E)** | Where, mechanically, is the tenant boundary enforced? | This framework | -| **Placement (P)** | Which physical substrate holds a tenant's data? | `railiance-platform` | - -Treating them as one produces concrete errors today. `rapp-postgres`'s -`PostgresConsumer` carries a field `tenantIsolation: consumer-service-boundary` -— an **E**-plane fact recorded in a **P**-plane artifact, reading as though the -storage platform enforces something it does not. The "dedicated versus shared" -argument routinely mixes P (capacity, blast radius) with E (correctness). - -The planes are separated **precisely so that each may sit at a different -level**. A repo can be strong on identity and weak on enforcement, or the -reverse, and the framework must state that without flattening it to a single -verdict. That is the whole point, and draft-1 lost it by collapsing E into a -single fixed rule. - -**Decision 3.1:** every document, declaration, and plan tier that says -"isolation" MUST name which plane it means. - -**Decision 3.2:** the planes couple at their top levels — `E4` is only -reachable at `P3` or above — and that coupling MUST be stated where it applies -rather than used to argue the planes are one. - -## 4. Graduated levels - -Each plane carries an ordered ladder. Higher is stronger, not better: the -right level is the one a service can evidence and its risk warrants. - -### 4.1 Identity (I) — how the tenant is named - -| Level | State | -|---|---| -| **I0** | No tenant concept. Data is not attributable to a tenant. | -| **I1** | A local tenant notion exists but does not conform to `tenant::`. | -| **I2** | Canonical identifiers used, validated against the ratified vocabulary; `tenant-engine` is the source of existence. | -| **I3** | I2 plus capability roles (`PLTF`/`IAM`/`VEN`/`CUS`) honoured, with live `tenant-engine` re-query for privileged, destructive, credential-vending or `aal2`-class decisions rather than trusting the cached `tenant_roles` claim. | - -Note `business-app-service-contract` §2.1 sets app-local accounts as the v1 -baseline for business apps — an explicitly sanctioned low level with recorded -triggers for moving up. That is the pattern this framework generalises. - -### 4.2 Authorization (A) — how a request is bound to a tenant - -| Level | State | -|---|---| -| **A0** | No authorization, or tenant context not carried on the request. | -| **A1** | Ad-hoc checks scattered through handlers. | -| **A2** | A single local authorization boundary; tenant context bound once, centrally. | -| **A3** | Decisions delegated to `flex-auth` as PDP, with live re-query where the IAM Profile requires it. | - -### 4.3 Enforcement (E) — where the tenant boundary actually holds - -This is the reframed core. Draft-1 asserted one rule for everyone; this is a -ladder whose top is platform-enforced. - -| Level | Mechanism | Bypassed by | -|---|---|---| -| **E0** | None. Data not tenant-keyed; separation is incidental or absent. | Anything. | -| **E1** | Data tenant-keyed, filtering applied per query at call sites. | One missing predicate. | -| **E2** | Filtering centralised at a single service-side choke point that binds authenticated identity to permitted tenants. | Code that bypasses the choke point. | -| **E3** | Platform-assisted: row-level security keyed on a session GUC set at pool checkout, or an equivalent data-access layer the application cannot trivially route around. | A `SECURITY DEFINER` function, a missing policy, or a wrong GUC. | -| **E4** | Structural: the connection a workload holds cannot address another tenant's data at all. Requires per-tenant credentials and per-tenant substrate. | Only a platform-level compromise. | - -Three properties of this ladder matter. - -**E3 is real and is not forbidden.** Draft-1 rejected RLS on the grounds that -a control depending on the consumer setting a session variable "looks like a -database guarantee without being one". The observation is correct; the -conclusion was wrong. E3 is materially stronger than E2 — it converts "every -call site must remember" into "one checkout path must be right" — and it is -the standard mitigation in pool-model systems. What was actually wrong was -*calling E3 by E4's name*. The honesty requirement survives; the prohibition -does not. - -**Higher enforcement is the direction of travel, not an optional extra.** -Services holding data whose cross-tenant exposure would be a reportable breach -SHOULD target E3 or above. The framework's default expectation for a new -platform service is **E2 at first serve, E3 recorded as target**. - -**Platform enforcement is a platform obligation, not only a consumer one.** -Draft-1 placed the whole burden on consumers permanently. It is now stated -that reaching E3 requires the storage platform to *offer* the mechanism — -provisioned RLS policies, a documented GUC contract, and a probe. -`rapp-postgres` ADR-0001 §4 already commits to provisioning it on request; -that offer becomes an obligation with a defined contract (§18.6). Where a -consumer wants E3 and the platform has not supplied the mechanism, the gap is -the platform's, not the consumer's. - -### 4.4 Placement (P) — which substrate holds the data - -Draft-1's `R0` ("shared tables, tenant column") is removed: that was an -E-plane state mislabelled as placement. Corrected ladder: - -| Level | Shape | Live occupants | -|---|---|---| -| **P0** | Shares a database with another consumer (schema or tables co-resident). | None sanctioned; the state absorbed repos often arrive in. | -| **P1** | Database per consumer, shared cluster. | `audit-core`, `tenant-engine` on `platform-pg` | -| **P2** | Dedicated cluster per consumer. | `user-engine-pg`, `target-revenue-pg` | -| **P3** | Dedicated cluster per tenant. | Business apps per `business-app-service-contract` §1.2 | -| **P4** | P3 plus separate region or jurisdiction. | None | - -**P0 → P1 → P2 does not raise the E level.** Those steps buy consumer -isolation, capacity predictability, independent backup retention, and a -smaller operational blast radius. They leave the tenant boundary exactly where -the E level puts it. Only P3 makes E4 reachable. This is the single most -misusable fact in the framework and §11 governs how it may be described -commercially. - -**Decision 4.5:** P1 is the default for platform services; P3 is the default -for client-facing business apps, as already ratified. A service unsure which -it is must resolve that before choosing (§18.4). - -## 5. The posture vector - -A service states its tenancy posture as one level per plane, plus a target and -a date: - -```yaml -tenancy: - current: { I: 2, A: 3, E: 2, P: 1 } - target: { I: 2, A: 3, E: 3, P: 1 } - reviewed: "2026-08-17" - gap: - E: "Choke point exists and is tested; RLS not yet provisioned. Blocked on - rapp-postgres offering the GUC contract. Target Q4." -``` - -Worked examples, as best assessed today and subject to owner correction: - -| Service | Current | Notes | -|---|---|---| -| `tenant-engine` | `I2 A3 E2 P1` | Moving to P1 under TEN-WP-0009; E2 via its own service boundary. | -| `audit-core` | `I2 A3 E2 P1` | Same shape; holds audit data, so E3 is the more urgent target. | -| A newly absorbed repo | `I1 A1 E1 P0` | Conformant **if declared**, with a recorded path. | - -The vector is what makes the framework hold both situations honestly. A repo -at `I1 A1 E1 P0` is not a failure — it is a described state with a direction. -A repo claiming `E3` with no RLS policy is a failure regardless of how good the -rest of it is. - -**Decision 5.1:** the posture vector is declared in the repo, not in the hub, -consistent with local-files-are-source-of-truth. - -## 6. Honest absorption — conformance is accuracy, not altitude - -The governing rule of this framework: - -> **A service is conformant when its declared posture is accurate, its target -> is recorded, and it does not claim a level it cannot evidence. It is -> non-conformant when it overclaims — at any altitude.** - -Consequences, stated deliberately: - -- Declaring `E0` is conformant. Concealing `E0` is not. -- A repo may be absorbed at any posture. It may not be absorbed silently. -- No service is blocked from the estate for being low on a ladder. Services - MAY be blocked from *specific work* — serving a tenant grouping, holding a - data class, carrying a plan tier — by requirements expressed as minimum - levels (§11). -- Downgrading is permitted and must be declared. A regression discovered by - guarding (§12) is a defect; a regression declared in advance is a decision. - -This is what the plane separation is *for*. Without it, "not rigorous about -tenant separation" is a single verdict that a repo either passes or fails. -With it, the same repo is `I1 A1 E1 P0` with a path — which is a plan, not an -indictment. - -## 7. Portability across placement levels - -Movement between P levels must be an operational change, not a rebuild. -Adopted from `tenant-engine`'s TEN-WP-0009 design: - -- Connect by injected credential only — no cluster, host, namespace, or - database name in source. -- Own a whole database, never tables inside someone else's. Sharing a cluster - is reversible; sharing a database makes relocation a schema merge. -- Idempotent schema creation. -- No cross-database joins or co-location assumptions. - -A service meeting these moves P1 → P2 by: create target, dump, restore, swap -the credential reference, restart. - -**Decision 7.1:** mandatory at P1 and above. At P3, SHOULD rather than MUST — -the consolidation `business-app-service-contract` §1.3 preserves is a real -benefit, but a per-client instance that never moves is not misconformant for -naming its own database. - -## 8. Placement triggers - -A level change decided under pressure is decided badly. Each service records -at provisioning time what would move it: - -- Noisy neighbour on a latency-critical path -- A compliance or residency requirement -- A plan tier requiring a higher minimum (§11) -- Backup retention or RTO no longer fitting -- Connection or memory ceiling reached (§16) - -**Decision 8.1:** triggers MUST be *monitored*, not merely recorded. A trigger -in a YAML comment that nobody re-reads is documentation, not control. The -`platform-pg` ceiling in §16 is the first that requires an actual alert. - -**Decision 8.2:** placement policy ownership is proposed to -`railiance-platform`. This framework supplies the ladder, the triggers, and -the declaration; it does not appoint per-workload placement. See §18.2 — this -remains their call and their open request. - -## 9. Credentials as a tenancy control - -Short-lived leased credentials from OpenBao, re-read at connection checkout, -with overlap-first rotation, bound the residual risk at every E level below -E4: a leaked credential expires rather than persisting. This is stronger than -the industry norm of a long-lived per-service secret and belongs in the -tenancy model, not only in credential-management canon. - -**Decision 9.1:** static long-lived database credentials are not a sanctioned -path for any service above E0. - -## 10. Blast radius must be published - -**Decision 10.1:** every platform holding consumer data MUST publish, in -concrete terms, what a leaked runtime credential can and cannot reach at the -levels it operates. `rapp-postgres` ADR-0001 §5 is the reference -implementation. Where the model cannot provide a guarantee, the platform says -so and names the escalation, rather than describing a weaker control in -stronger language. - -## 11. Commercial expression — room to be successful - -Isolation will be sold. The framework must let commercial and presentation -work rather than obstruct them, while preventing a claim that cannot be -evidenced. - -**Decision 11.1:** plan tiers are expressed internally as **minimum levels**, -not as implementation labels. A tier may require `E3 P2`; it need not print -that string anywhere customer-facing. - -**Decision 11.2:** marketing and product language is free. There is no -requirement to expose level labels, ladder names, or this document to -customers. "Dedicated infrastructure", "isolated tenancy", "private instance" -all remain available. - -**Decision 11.3:** the constraint is on **evidence, not vocabulary**. Any -customer-facing isolation or availability claim must map to a minimum level -the delivering service actually holds, recorded once when the tier is defined. -The review is internal and happens at tier definition — not per campaign, per -page, or per conversation. - -**Decision 11.4:** the one hard line — a claim that another tenant *cannot* -reach the customer's data requires **E4**. Everything softer than that -absolute has room. This is the claim that reaches compliance questionnaires -and contracts, and it is the one that must be true. - -`adaptive-pricing` owns plan terms; `tenant-engine` owns which plan a tenant -holds. Neither currently maps a tier to minimum levels; §18.5 asks them to, -for tiers making isolation or availability claims only. - -## 12. Methodology — analyze, establish, improve, guard - -The framework is operated as a cycle, not consulted as a reference. - -**Analyze.** Assess a repo against the four ladders and produce its current -vector. Output: the `tenancy.current` block with the reasoning recorded. -Applies to new services and absorbed ones alike. - -**Establish.** Declare the target vector and the gap. The target is set by the -data class held, the tenant groupings served, and any plan tier carried — not -by ambition. Output: `tenancy.target` plus a recorded gap per plane. - -**Improve.** Move one plane at a time. Cross-plane moves are where mistakes -happen — raising P while leaving E untouched is exactly the misstep §4.4 -warns about. Each step is ordinary repo work with ordinary evidence. - -**Guard.** Verify continuously that the declared posture still holds. Guarding -checks a service **against its own declaration**, not against a universal -maximum. This is what makes the verification problem tractable: nobody has to -prove every service is at E4; the check is that nobody is below what they -declared, and nobody claims what they cannot evidence. - -Regression found by guarding is a defect; regression declared in advance is a -decision. The estate has been bitten twice by silent pin rollbacks that -produced ordinary-looking 403s and 404s rather than errors. Posture regression -will look the same, so guarding must be designed for invisible failure rather -than for crashes. - -## 13. Evidence per level - -Guarding requires each level to have a defined evidence artifact. Proposed: - -| Level | Evidence | -|---|---| -| **I2** | Identifiers validated against the ratified vocabulary; rejection test for a malformed tenant id. | -| **I3** | Live re-query demonstrated on an `aal2`-class path; cached-claim path shown not to be used there. | -| **A2** | Single choke point identified; test that an unbound request is refused. | -| **A3** | Live decision against `flex-auth` with a denial observed at the endpoint, not only at the decision surface. | -| **E1** | Every tenant-owned table carries the tenant key. Schema-level check, mechanical. | -| **E2** | Choke point identified; adversarial test that an identity bound to tenant A cannot read tenant B. | -| **E3** | RLS policies present on every tenant-owned table; probe that a session without the GUC set reads nothing; probe that a wrong GUC reads nothing. | -| **E4** | Per-tenant credential demonstrated unable to connect to another tenant's substrate. | -| **P1–P4** | Provisioning declaration plus the platform's own isolation probes. | - -**Decision 13.1:** a level is claimed only with its evidence artifact present. -This turns §6's accuracy rule from an honour system into a check. - -**Decision 13.2:** the E2 and E3 evidence artifacts do not exist anywhere in -the estate today. `rapp-postgres` runs 15 adversarial probes, all against the -*consumer* boundary, none against the tenant boundary inside a consumer. This -is the framework's largest live gap and §18.3 seeks an owner. - -## 14. Alternatives considered - -**One fixed model with a single set of characteristics** (draft-1). *Rejected -by the operator:* it cannot describe a repo that is not there yet, forcing -absorbed repos either to misrepresent their posture or to stay outside the -framework. A framework that can only describe its own end state is not a -framework. - -**A maturity model with a single overall level** (bronze/silver/gold). -*Rejected:* collapses the plane separation that makes the framework useful. A -service strong on identity and weak on enforcement has a specific, actionable -gap; one composite score hides it and invites averaging. - -**Schema-per-consumer in one database** (sanctioning P0). *Rejected:* -`pg_catalog` is readable per-database, so every co-resident enumerates every -other's table and column names regardless of `GRANT`s. Retained in the ladder -as a describable state, never as a sanctioned target. - -**Row-level security as the *consumer* boundary.** *Rejected:* puts a -correctness-critical boundary between two services inside query predicates. -Retained as E3 for the *tenant* boundary, where the realistic alternative is -application code rather than a database. - -**Mandating E4 for everyone.** *Rejected:* the tenant grouping taxonomy -includes `consumer` (private individuals) and `family`. A cluster per private -individual is economically impossible; the taxonomy is itself evidence that -pooling is required. - -**Leaving each repo to its own model.** *Rejected:* this is the status quo, -and it produced two contradictory ratified defaults plus an unowned placement -question. - -## 15. Challenge against outside practice - -**Where the framework is stronger than typical.** Most SaaS multi-tenancy -literature (AWS's silo/pool/bridge framing among them) models exactly one -boundary — tenant-to-tenant. This estate has **two stacked boundaries**: -platform-service-to-platform-service, and tenant-to-tenant inside a consumer. -Naming them separately and refusing to enforce both with one mechanism is -uncommon and correct. - -Graduated per-plane levels are also stronger than the usual silo/pool/bridge -trichotomy, which describes deployment shape only. Silo/pool/bridge is -approximately this framework's P plane with the other three missing — which is -why it cannot express "pooled infrastructure, structurally enforced -boundary", or the reverse. - -**Where it is weaker, and this is the finding that matters.** The pool model's -standard mitigation is a *verified* enforcement layer that every service is -demonstrably routed through. The estate has the concept (E2/E3) and none of -the verification (§13.2). A single missing `WHERE tenant_id = ?` in any -consumer is a cross-tenant breach that no platform control catches, no probe -fails, and no log shows as an error. - -**Where the estate has quietly adopted a good pattern without naming it.** -Short-lived leased credentials re-read at checkout, with overlap-first -rotation, beat the industry norm of a long-lived per-service secret. §9 -promotes it to a tenancy control. - -**Cell-based architecture remains unexplored.** Neither P nor E describes a -cell — a fixed-size slice of infrastructure serving a bounded set of tenants, -sized so one cell's failure is survivable and cell count scales linearly. -Given the estate spans railiance, NetKingdom, HelixForge and Coulomb, cells -may fit better than per-tenant clusters at the point P1 runs out (§18.8). - -## 16. Scaling demands - -Measured against the live `platform-pg` specification, not estimated. - -``` -instances: 1 (no HA; single-node rail) -max_connections: 100 -memory limit: 1Gi -per consumer: 14 connections (12 runtime + 2 migration) -``` - -**Connection ceiling: roughly six consumers.** Seven consumers request 98 of -100 before CNPG's instance manager, metrics exporter, and reserved slots. Two -are provisioned today. **We are at roughly a third of capacity and the third -request will not feel like a scaling event.** - -**Memory likely binds first.** 100 backends against 1Gi is ~10MB per backend -for shared buffers, work_mem and overhead. Connection exhaustion errors -clearly; memory pressure OOM-kills and degrades every co-resident at once. - -**Raising E is not free on a shared substrate.** RLS with a per-checkout GUC -does not itself increase connection count, but it forecloses aggressive -transaction-level pooling for consumers that adopt it. E3 on P1 interacts with -the ceiling above and should be planned against it. - -**Restore time couples all consumers.** Physical backup is instance-wide, so a -consumer's RTO is a function of *total* instance size, not its own. One large -consumer silently degrades every co-resident's recovery promise. - -**`instances: 1` means no P1 tenant has HA.** A tier promising uptime cannot -be satisfied at P1 as currently built — an availability floor belongs in §11's -minimum-level vocabulary alongside isolation. - -## 17. Consequences - -- The estate gains one vocabulary and, more importantly, a way to be honest - about partial adoption. -- Absorbed repos get a described state and a path instead of a failing grade. -- `tenantIsolation` in `PostgresConsumer` is revealed as a mislabelled field - (§18.1). -- The two ratified defaults are reconciled by scope, at the cost of needing a - classification rule. -- The verification problem becomes tractable: guard against declaration, not - against a universal maximum. -- Draft-1's prohibition on RLS is reversed; `rapp-postgres` acquires an - obligation to define and offer the E3 mechanism. -- Nothing here changes a running system. - -## 18. Open questions - -1. **`tenantIsolation` field** — `rapp-postgres`: rename to name its plane and - carry a level (`tenancy.E: 2`), or move it out of the storage declaration. -2. **Placement ownership** — `railiance-platform`: accept the ladder, the - triggers, and the monitoring obligation in §8.1, and appoint a recorded - placement owner per workload. -3. **E2/E3 evidence artifacts** — *owner needed.* Who builds the adversarial - cross-tenant probes and the RLS conformance check? Both `audit-core` and - `tenant-engine` have declined fleet-scope work on correct boundary - reasoning, so this needs appointing. Highest-severity gap (§13.2). -4. **Business app vs platform service** — Custodian canon: a classification - rule, so a new repo knows whether P3 or P1 is its default. Candidate: reuse - `repo-classification-standard_v1.0` rather than invent a second taxonomy. -5. **Tier → minimum level mapping** — `adaptive-pricing` and `tenant-engine`: - required only for tiers making isolation or availability claims (§11). -6. **E3 mechanism** — `rapp-postgres`: what exactly is the GUC contract, and - is it provisioned per consumer on request or offered as a standard option? -7. **Engine neutrality** — the P ladder rests on a PostgreSQL property - (per-database catalogs). State it engine-specifically and say so, or - abstract it and risk a non-Postgres implementation that silently differs? - Inherited unresolved from the v0.1 draft. -8. **Cell-based partitioning** — evaluate at consumer #4 rather than now? - Neither P nor E describes a cell. - -## 19. Ratification path - -The predecessor draft stalled because it was routed without one. - -1. Reviewed by `tenant-engine`, `flex-auth`, `rapp-postgres`, - `railiance-platform`, and `adaptive-pricing` against §18. -2. Each of those repos publishes its own posture vector (§5) as part of - review. **The framework is validated by whether it can describe them - accurately** — if a repo cannot express itself in these four ladders, the - ladders are wrong and this document changes, not the repo. -3. On acceptance, **supersedes** the routing of - `rapp-postgres/docs/canon-drafts/shared-platform-relational-storage_v0.1-draft.md`, - whose §§3–8 are absorbed here. That draft is then withdrawn rather than - left pending. -4. On acceptance, `rapp-postgres/docs/adr/ADR-0001` moves to `accepted` and is - annotated as the PostgreSQL implementation of the E and P ladders. +The publication renderer that lived in `tools/` moved to `policy-nexus`, which +owns publication. diff --git a/research/2026-08-17-adr008-gap-research-erasure-and-noisy-neighbour.md b/research/2026-08-17-adr008-gap-research-erasure-and-noisy-neighbour.md new file mode 100644 index 0000000..8a95cf0 --- /dev/null +++ b/research/2026-08-17-adr008-gap-research-erasure-and-noisy-neighbour.md @@ -0,0 +1,265 @@ +# Gap research — erasure mechanism (R4) and noisy-neighbour evidence + +> Research artefact — 2026-08-17. Fifth digest supporting +> `canon/architecture/adr-008-multi-tenancy-model.md` (draft-3). Closes the two +> gaps the four plane digests left open: how `R4` (verified erasure) can be +> reached at all, and what a noisy-neighbour evidence artifact should actually +> assert. +> **Both findings change a level definition in draft-3.** +> Index: `2026-08-17-adr008-multi-tenancy-research-index.md`. + +--- + +# Part 1 — Erasure: R4 needs a mechanism it does not currently have + +## 1.1 The problem draft-3 left standing + +ADR-008 §4.5 defines `R4` as "deletion proven complete across live data, +backups and derived copies within the published horizon", and `rapp-postgres` +ADR-0002 establishes that on shared substrate the horizon is the instance +maximum — every co-resident consumer's retention window, whichever is longest. + +Read together, those say: **erasure completes by waiting.** A tenant exercising +a deletion right is told the data is gone in up to N days, where N is set by +somebody else's retention requirement. That is honest, and it is a weak answer. +It also makes `R4` unreachable in any useful sense, because "we waited" is not +a proof. + +## 1.2 Crypto-shredding is the established mechanism + +The industry answer to erasure under immutable or long-retention storage is to +make the data unreadable rather than absent. + +> "Crypto-shredding for backups encrypts each subject (or tenant) with a unique +> key derived from the subject identifier, and on erasure, destroys the key. +> The backups remain but are cryptographically inaccessible." +> — [oneuptime — setting up crypto-shredding for GDPR right-to-erasure](https://oneuptime.com/blog/post/2026-02-17-how-to-set-up-crypto-shredding-for-gdpr-right-to-erasure-compliance-in-google-cloud/view) + +It is aimed precisely at our situation: the named use cases are "append-only or +immutable storage (event sourcing, WORM, immutable backups) where physical +deletion is impossible" +([Granit — crypto-shredding: GDPR erasure without deleting rows](https://granit-fx.dev/blog/crypto-shredding-gdpr-erasure-without-deleting-rows/)). + +**Regulatory standing, stated carefully because overclaiming here would be +worse than in any other section of this framework.** The EDPB has *not* formally +endorsed crypto-shredding as Article 17 erasure. Several data protection +authorities have accepted it where physical deletion would be manifestly +disproportionate effort under Article 17(1), and EDPB Guidelines 5/2019, the UK +ICO and the French CNIL are cited as recognising it **provided three conditions +hold**: strong encryption (AES-256), irreversible key destruction, and auditable +destruction +([pg_hardstorage — GDPR Art. 17 crypto-shred](https://github.com/cybertec-postgresql/pg_hardstorage/blob/main/docs/compliance/gdpr-art-17-crypto-shred.md), +[Opsio — right to erasure in cloud systems](https://opsiocloud.com/blogs/right-to-erasure-article-17-gdpr-deletion-cloud-systems/)). + +The third condition is the one this estate is well placed for and the one most +implementations skip: **destruction must be auditable**. ADR-0002 already +requires every sweep to emit an evidence record. Key destruction is the same +shape. + +## 1.3 The finding: shredding granularity is bounded by the key boundary + +Crypto-shredding "requires per-entity key isolation: each entity instance gets +its own encryption key" (oneuptime, ibid.). That single requirement determines +what is available to us, and it is not uniform across the estate. + +| What we want to shred | Key boundary needed | Available today? | +|---|---|---| +| A consumer's logical exports | Per-consumer key | **Close.** The offsite lane already encrypts with `age`, and OpenBao holds key material. This is a key-management change, not an architecture change. | +| A consumer's physical backup | Per-consumer key on the Barman object store | **No.** One WAL stream, one credential, one encryption context for the whole instance. Same constraint that forced the derived maximum in ADR-0002 §2. | +| A single tenant's rows inside a consumer | Per-tenant key, applied by the application before the data reaches storage | **No, and not the platform's to build.** Requires application-level per-tenant encryption. | + +The third row is the architecturally interesting one. Per-tenant crypto-shred +means the consumer encrypts tenant data under a tenant-specific key before +writing it — which is an **enforcement-plane** property, not a retention one. It +also happens to be a strong one: data encrypted per tenant is unreadable across +the boundary even by a credential that can address the rows, which is E4-class +protection achieved without per-tenant substrate. + +**This is a new coupling and it belongs in §3.2.** Draft-3 records two +couplings (E4 needs P3; R's horizon is bounded by P). There is a third: +**R4 at tenant granularity requires per-tenant keys, which is an E-plane +capability.** Reaching the top of the retention ladder is not a retention +project. + +A reference implementation exists for the consumer-granularity case: +`pg_hardstorage` "implements crypto-shred by destroying the per-tenant KEK that +wraps every encrypted backup's DEK" (ibid.). Per the §14 adoption stance we +take the **structure** — envelope encryption with a per-consumer key-encryption +key wrapping each backup's data-encryption key, and destruction of the KEK as +the erasure act — and build it on our own OpenBao and `age` lane rather than +adopting the extension. + +## 1.4 The tension nobody should paper over: erasure versus audit + +`audit-core` exists to hold a tamper-evident trail. Crypto-shredding a tenant's +audit records destroys evidence, and ADR-0001 §2 deliberately built the role +model so that "a compromised runtime credential could not rewrite history at +the schema level". Erasure is a sanctioned way to do exactly that. + +The literature treats this as a genuine design tension rather than a solved +problem ([Remote Reason — balancing auditability and privacy with crypto-shredding](https://remotereason.com/blog/balancing-auditability-and-privacy-with-crypto-shredding)). +The usual resolution is to separate the *fact* of an event (retained, not +personal) from its *personal payload* (encrypted per subject, shreddable), so +the trail survives erasure with its personal content removed. + +**This is a question for `audit-core`, not an answer this framework should +supply.** Flagging it because a naive "R4 everywhere" target would silently +instruct the audit service to destroy its own evidence. + +## 1.5 Proposed amendments to draft-3 + +1. **`R4` admits two routes, and must name which one a service uses:** + - *Horizon-elapsed* — deletion is complete once the published erasure + horizon has passed. Available to everyone, proves little, slow. + - *Key-destroyed* — data is rendered unreadable immediately by destroying the + key that decrypts it. Requires per-entity keys, AES-256-class encryption, + and an auditable destruction record. +2. **Add the third coupling to §3.2:** R4 at tenant granularity requires an + E-plane capability (per-tenant encryption). R4 at consumer granularity + requires per-consumer keys on the retained artifacts. +3. **State the regulatory position honestly** wherever R4 is described: DPAs + have accepted key destruction as erasure under conditions; the EDPB has not + formally endorsed it. A plan tier claiming "your data is deleted" on the + strength of crypto-shredding is making a defensible but not settled claim, + and §11.4's hard line should say so rather than implying R4 is a clean yes. +4. **Raise the audit tension** as an open question addressed to `audit-core`. + +--- + +# Part 2 — Noisy neighbour: the evidence artifact asserts the wrong thing + +## 2.1 The problem draft-3 left standing + +ADR-008 §13 proposes a P1/P2 evidence artifact: "one consumer saturating its +connection or CPU allowance demonstrably **does not breach** another's." + +That is not achievable, and the reference guidance says so in its first line of +solution text: + +> "Sharing a single resource inherently carries the risk of noisy neighbor +> problems that you can't completely avoid." +> — [Azure Architecture Center — Noisy Neighbor antipattern](https://learn.microsoft.com/en-us/azure/architecture/antipatterns/noisy-neighbor/noisy-neighbor) + +An artifact that can only ever fail, or that passes by being run gently enough, +is worse than no artifact — it is the overclaim §6 prohibits, dressed as +evidence. + +## 2.2 There are two failure shapes, and we only guard against one + +Azure distinguishes them explicitly: + +- **One greedy tenant.** "At peak times, tenant A uses all of the system's + resources, which means that any requests that tenant B makes fail." (ibid.) +- **Aggregate of many modest tenants.** "The noisy neighbor problem also occurs + when each individual tenant consumes only a small portion of the system's + capacity. However, the combined resource usage of many tenants can result in + a peak in overall usage." (ibid.) + +Our per-consumer governance — `connection_limit`, `statement_timeout`, +`idle_in_transaction_session_timeout`, applied at database scope per ADR-0001 +§3 — addresses the first shape well. It does nothing about the second. + +**And the second shape is exactly what §17 computes.** Seven consumers, each +politely within its 14-connection allowance, request 98 of 100 connections. +Every one of them is well-behaved; the instance still fails. The connection +ceiling in §17 is not merely a capacity number — **it is the aggregate +noisy-neighbour bound**, and framing it as a scaling statistic understates what +it is. + +## 2.3 The detection signature we can actually build + +Azure's service-side diagnostic is specific and unintuitive: + +> "**Failures when performing an operation for a tenant:** Look for failures +> that occur when a tenant isn't consuming a large share of the system's +> resources. This pattern might indicate that the tenant is experiencing a +> noisy neighbor problem. Track resource consumption by tenant." (ibid.) + +The signal is a failure for a *quiet* consumer. That requires per-consumer +resource attribution, and we already have the raw material: ADR-0001 stamps +`application_name` per role (`/runtime`), PostgreSQL logs carry +database, user, application name and client address, and +`make postgres-ops-status` already reports sessions by application name. + +What is missing is a **baseline**. "It's important to clearly understand your +normal baseline resource usage and to configure monitoring and alerts to detect +spikes" (ibid.). No baseline for `platform-pg` has been recorded. Without one, +a spike is indistinguishable from a Tuesday. + +## 2.4 A normative requirement we are not meeting + +> "Be transparent with clients about any throttling mechanisms or usage quotas +> that you enforce. It's important that they handle failed requests gracefully +> and aren't caught off guard by limitations." (ibid.) + +`tenant-engine` was told its connection and timeout limits during provisioning, +which is good practice by accident rather than by rule. Nothing requires it. A +consumer discovering its `statement_timeout` by hitting it in production is the +failure this warns about. + +**Proposed:** fold quota transparency into §10 (blast radius must be published) +— the same disclosure obligation, extended from what a leaked credential can +reach to what the platform will refuse to do for you. + +## 2.5 Mitigations we have, and the ones we do not + +| Azure-recommended control | Our status | +|---|---| +| Query time limits / max returnable records | **Have.** `statement_timeout`, per-role since the tenant-engine change. | +| Connection quotas per tenant | **Have.** Database-scoped, per ADR-0001 §3. | +| Monitor overall *and* per-tenant resource usage | **Partial.** Attribution exists; no baseline, no alerting. | +| Throttling / rate limiting | **None.** No request-rate control at the database boundary. | +| Quality-of-service prioritisation | **None.** All consumers are equal, including on the synchronous authorization path where `tenant-engine` now sits. | +| Run heavy non-time-sensitive work off-peak | **Implicit.** Backups at 02:15; nothing else scheduled. | +| Rebalance tenants across stamps | **Not possible.** One instance, no second cell (§19.8). | + +The QoS absence is worth noting against a specific fact: `tenant-engine` chose +a 5s statement timeout because `flex-auth` calls it synchronously on the +authorization path. That is a latency-critical consumer sharing an instance +with `audit-core`, which is not. Nothing prioritises one over the other, and +the framework has no vocabulary for saying one consumer's latency matters more. + +## 2.6 Proposed amendment to draft-3 + +Replace the §13 P1/P2 artifact with one that asserts something true: + +> **Noisy-neighbour characterisation (P1/P2, adversarial, load-generated).** +> A recorded baseline of normal per-consumer resource usage; a load run in +> which one consumer saturates its declared allowance; evidence that (a) the +> governance controls bind — the greedy consumer is throttled at its declared +> limits rather than exceeding them, and (b) the measured degradation +> experienced by co-resident consumers is recorded and judged acceptable. +> Carries a review date and the aggregate headroom at time of measurement. + +That is achievable, it is falsifiable, and it produces a number — degradation +under contention — that a plan tier promising performance would need anyway. + +--- + +## 3. Summary of changes these two gaps force + +| # | Change | Where | +|---|---|---| +| A | `R4` splits into *horizon-elapsed* and *key-destroyed* routes; a service must name which | §4.5 | +| B | Third coupling: tenant-granularity R4 requires an E-plane capability (per-tenant encryption) | §3.2 | +| C | R4's regulatory standing stated honestly — accepted by DPAs under conditions, not formally endorsed by the EDPB | §4.5, §11.4 | +| D | Erasure-versus-audit tension raised as an open question to `audit-core` | §19 | +| E | Noisy-neighbour artifact reworded from "does not breach" to characterisation with a recorded baseline and measured degradation | §13 | +| F | §17's connection ceiling reframed as the aggregate noisy-neighbour bound, not only a capacity figure | §17 | +| G | Quota transparency added as a disclosure obligation | §10 | +| H | No QoS vocabulary despite a latency-critical consumer sharing with a batch one | §19, new | + +## Sources + +- [oneuptime — How to set up crypto-shredding for GDPR right-to-erasure compliance](https://oneuptime.com/blog/post/2026-02-17-how-to-set-up-crypto-shredding-for-gdpr-right-to-erasure-compliance-in-google-cloud/view) +- [cybertec-postgresql/pg_hardstorage — GDPR Art. 17 crypto-shred](https://github.com/cybertec-postgresql/pg_hardstorage/blob/main/docs/compliance/gdpr-art-17-crypto-shred.md) +- [Granit — Crypto-shredding: GDPR erasure without deleting a single row](https://granit-fx.dev/blog/crypto-shredding-gdpr-erasure-without-deleting-rows/) +- [Granit — Crypto-shredding (dotnet/compliance)](https://granit-fx.dev/dotnet/compliance/crypto-shredding/) +- [Opsio — Right to erasure, Article 17 GDPR, deletion in cloud systems](https://opsiocloud.com/blogs/right-to-erasure-article-17-gdpr-deletion-cloud-systems/) +- [Remote Reason — Balancing auditability and privacy with crypto-shredding](https://remotereason.com/blog/balancing-auditability-and-privacy-with-crypto-shredding) +- [Azure Architecture Center — Noisy Neighbor antipattern](https://learn.microsoft.com/en-us/azure/architecture/antipatterns/noisy-neighbor/noisy-neighbor) +- [Neon — The noisy neighbor problem in multitenant architectures](https://neon.com/blog/noisy-neighbor-multitenant) +- [PingCAP — Playbook: stop noisy neighbors, multi-tenant MySQL at scale](https://www.pingcap.com/playbook-noisy-neighbor-multi-tenant-mysql/) +- [Causal inference for quantifying noisy neighbor effects in multi-tenant cloud environments (arXiv 2604.03145)](https://arxiv.org/pdf/2604.03145) +- [Azure Architecture Center — Throttling pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/throttling) +- [Azure Architecture Center — Rate Limiting pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/rate-limiting-pattern) diff --git a/research/2026-08-17-adr008-multi-tenancy-research-index.md b/research/2026-08-17-adr008-multi-tenancy-research-index.md index 1712b3d..9f50e8a 100644 --- a/research/2026-08-17-adr008-multi-tenancy-research-index.md +++ b/research/2026-08-17-adr008-multi-tenancy-research-index.md @@ -55,7 +55,7 @@ Ranked by how badly they mislead. | # | Finding | Effect | |---|---|---| -| O1 | **pgTAP + pgrls** (67 RLS lint rules, semantic policy-diff for CI gating, pytest isolation plugin) exist off the shelf. `rapp-postgres` already runs a 15-probe harness of the same shape. | The E3 half of open question §18.3 is far cheaper than assumed. Splits the question into a tractable mechanical half and a human half. | +| O1 | **pgTAP + pgrls** exist off the shelf; `rapp-postgres` already runs a 15-probe harness of the same shape. pgrls **verified directly** 2026-08-17: MIT, 67 rules (54 security, incl. BYPASSRLS / LEAKPROOF / `SECURITY DEFINER` / cross-tenant leaks), 19 auto-fixable, Z3 isolation proofs, SAFE/BREAKING policy-diff for CI gating, pytest plugin, PG 15–17. **But it is a 26-star beta project** — adopt the approach with confidence, treat the dependency as a supply-chain decision. pgTAP is long-established and carries no such caveat. | The E3 half of open question §18.3 is far cheaper than assumed. Splits the question into a tractable mechanical half and a human half. | | O2 | **OWASP Multi-Tenant Security Cheat Sheet** is the closest external baseline to our framework and is referenced nowhere in the estate. | Adopt as the named external baseline in ADR §15. | | O3 | **OpenID AuthZEN Authorization API 1.0** reached Final Specification January 2026; Keycloak shipped experimental support May 2026. Demonstrated PDP interoperability across 5+ engines. | `flex-auth`'s bespoke `CheckRequest` is pre-standard. Propose an `A4` rung for standard-interface PDP delegation. | | O4 | **Token versioning** as a middle rung between cached claim and live re-query. | Cheap global invalidation without a hot-path lookup. | @@ -101,7 +101,75 @@ digest: 7. Route G7 to `tenant-engine` separately. It is a critique of ratified canon, not of this ADR, and folding it in would overreach. -## 7. Method and limits +## 7. Adoption stance — structure, not tooling (operator decision, 2026-08-17) + +Research output is used as **design input**, not as a shopping list. The +operator's rule: + +> Use the structure that supports, extends or challenges our design. Do not +> adopt the tooling unless it is an established industry standard with broad +> application. Otherwise build ground up, so we can optimize and refactor as we +> see fit. + +Applied to this survey: + +| Finding | Adopt? | How | +|---|---|---| +| OWASP Multi-Tenant Security Cheat Sheet | **Yes — baseline** | A published security baseline, not tooling. Cite as the external reference our ladders answer to. | +| OWASP API Security Top 10 (BOLA / API1) | **Yes — vocabulary** | Names our gap in terms a security reviewer already holds. | +| OpenID AuthZEN Authorization API 1.0 | **Yes — standard** | Final Specification from a standards body with demonstrated multi-vendor interop. Qualifies as broad and established. Propose as the `A4` rung. | +| Azure tenancy models / AWS SaaS Lens / cell architecture | **Yes — structure** | Taxonomies and patterns. Corroborate the spectrum reframe and the E↔P coupling. | +| PostgreSQL RLS mechanics (`FORCE`, `BYPASSRLS`, `SET LOCAL`, leakproof) | **Yes — facts** | Engine behaviour, not a vendor's product. | +| **pgrls** | **No — structure only** | Take its rule *taxonomy* (54 security rules over RLS enablement, BYPASSRLS, LEAKPROOF, `SECURITY DEFINER`, cross-tenant leak; the SAFE/BREAKING/REQUIRES_REVIEW/DANGEROUS policy-diff classification) as a checklist for probes we write ourselves. 26 stars and beta was already a supply-chain caveat; the stance settles it. | +| **pgTAP** | **No** | Established in the Postgres world but still tooling, and we already have the ground-up equivalent: `scripts/isolation-test.sh`, 15 probes, bash + psql, no dependency tree. Extend that harness to the tenant boundary. | + +The practical consequence is small and good: `rapp-postgres` already owns a +probe harness that found four real defects in its own provisioning SQL. The E2 +and E3 evidence artifacts become new probes in an existing tool, informed by +pgrls's taxonomy of what to look for, with no new dependency. The +policy-diff idea — classifying a policy change as safe or breaking before it +lands — is the genuinely novel structural contribution worth reimplementing. + +## 8. Retention and erasure — designed, not just flagged + +Gap G1 (tenant offboarding and deletion) has been taken up rather than left as +an open question. See `rapp-postgres/docs/adr/ADR-0002-data-retention-and-erasure.md`. + +The shape, per operator direction: retention is declared per consumer as +`N` days defaulting to 30, `rapp-postgres` provides the abstract mechanism for +any dataset, and a consumer or its governance layer builds policy on top of +that interface. + +The finding that emerged while designing it: **per-consumer physical retention +is not implementable.** CNPG has one WAL stream and one window, so the instance +retention is derived as the maximum across consumers and every consumer's +erasure horizon is that maximum. A consumer declaring 7 days beside one +declaring 90 gets 90. This is the retention analogue of ADR-0001 §5 — state the +coupling rather than imply an isolation that is not there — and it produces two +things the estate could not previously say: + +- The **erasure horizon**: the interval between deleting data and it ceasing to + be recoverable from anything the platform holds. Deleted rows survive in + backups for the whole window; that is the difference between "deleted" and + "erased", and it had never been written down. +- **Retention as a placement trigger.** A consumer needing a shorter horizon + than the instance floor cannot have one at P1. It moves to a dedicated + cluster for a reason unrelated to performance — which is exactly why it needs + recording, since nobody looks for a retention argument when reviewing + placement. + +Deletion splits mechanism from policy: the platform sweeps whole datasets and +records an opaque `deletionPolicyRef` it never interprets, so every deletion +traces to the policy that authorised it. Rows are not a dataset. +`consumerDatabase` is deliberately not sweepable. + +**Open for draft-3:** whether this is a fifth plane (`R` — retention and +erasure, with its own ladder from "no policy" through "verified erasure within +a published horizon") or a lifecycle stage cutting across the existing four. +The interface is ladder-shaped, which argues for a plane — at the cost of the +framework's name. + +## 9. Method and limits Four search-and-read passes, one per plane, against vendor architecture guidance (AWS Well-Architected SaaS Lens, Azure Architecture Center), standards diff --git a/research/2026-08-17-adr008-plane-e-enforcement.md b/research/2026-08-17-adr008-plane-e-enforcement.md index 1968399..fa65b2b 100644 --- a/research/2026-08-17-adr008-plane-e-enforcement.md +++ b/research/2026-08-17-adr008-plane-e-enforcement.md @@ -143,10 +143,25 @@ expected. cross-tenant isolation tests are the only reliable guard" ([Blair Jordan — testing RLS policies with pgTAP](https://blair-devmode.medium.com/testing-row-level-security-rls-policies-in-postgresql-with-pgtap-a-supabase-example-b435c1852602), [MakerKit — database testing with pgTAP](https://makerkit.dev/docs/next-supabase-turbo/development/database-tests)). -- **pgrls** is a purpose-built static analyzer: 67 lint rules for tenant and - per-user row-scoping bugs, performance traps and hygiene, 20 auto-fixable, a - semantic policy-diff command for CI gating, and a pytest plugin for isolation - tests ([pgrls on GitHub](https://github.com/pgrls/pgrls)). +- **pgrls** is a purpose-built static analyzer, **verified directly 2026-08-17** + rather than taken from a search snippet + ([pgrls on GitHub](https://github.com/pgrls/pgrls)). Confirmed: MIT licence; + 67 rules across SEC (54), PERF (5), HYG (4) and VIEW (4); 19 auto-fixable; + Z3 SMT-solver verification of isolation properties; a policy-diff command + classifying changes SAFE / BREAKING / REQUIRES_REVIEW / DANGEROUS; a + `pgrls.testing` pytest plugin with role-switching and per-test transactions; + SARIF, JUnit XML and GitHub-annotation output; tested against PostgreSQL + 15–17. Its SEC rules cover exactly the traps in §3 — BYPASSRLS, LEAKPROOF, + `SECURITY DEFINER`, cross-tenant leaks. + + **Maturity caveat, stated because this digest recommends adoption.** It is + 26 stars, 2 forks, 527 commits, self-described "Beta — actively maintained". + The engineering is serious (Z3 proofs, a precision corpus) but this is a + small project, not an established dependency. Adopt the *approach* with + confidence; treat the *tool* as a supply-chain decision requiring the estate's + normal SBOM and pinning discipline, and be prepared to reimplement its + handful of load-bearing rules if it goes unmaintained. pgTAP, by contrast, is + long-established and carries no such caveat. The named common failures match ours exactly: "a forgotten role, a pool reusing connections with the wrong tenant context, or an admin path bypassing RLS