coulomb/the-custodian
4
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

research: external survey of multi-tenancy practice, one digest per ADR-008 plane

Browse source 
Four search-and-read passes against AWS Well-Architected SaaS Lens, Azure
Architecture Center, OWASP, the OpenID Foundation, and PostgreSQL practitioner
literature. Each digest holds one of our ladders against outside practice and
lists its sources.

Headline: the draft-1 -> draft-2 reframe is corroborated. Microsoft states
"isolation is a spectrum, not a discrete property" and derives our E/P coupling
independently. The shape is sound; the details are not.

Five corrections draft-2 needs:

- E3 overclaims. Any session can re-SET the GUC, so RLS is strong against a
  forgotten predicate and useless against SQL injection or a compromised
  process. The ADR says "the application cannot trivially route around" - the
  exact overclaim it prohibits elsewhere.
- The pooling claim in section 16 is backwards. Transaction-scoped SET LOCAL is
  what makes E3 safe under pooling; statement-level pooling is what serves
  other tenants' rows, only under production concurrency.
- E3 layers on E2 rather than superseding it.
- Placement cannot be expressed per tenant, but the commercial section requires
  exactly that - a tier bought by some tenants puts a service at two P levels
  at once, which the posture vector cannot represent.
- FORCE ROW LEVEL SECURITY is mandatory: ADR-0001 already established that our
  migration role owns the tables it creates, so it would silently bypass every
  policy.

Also: pgTAP and pgrls make the E3 evidence artifact far cheaper than assumed;
OWASP publishes a Multi-Tenant Security Cheat Sheet we reference nowhere; our
tenant-boundary gap is BOLA, OWASP API1, top of that list since it launched;
AuthZEN 1.0 went Final in January and flex-auth is pre-standard; and no plane
covers tenant offboarding, which OWASP makes a MUST.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
codex 2026-08-17 13:00:02 +02:00
parent 9a6e14e733
commit 04f888399f
5 changed files with 1052 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

122
research/2026-08-17-adr008-multi-tenancy-research-index.md Normal file
View file

@ -0,0 +1,122 @@
# ADR-008 multi-tenancy — external research index
> Research artefact — 2026-08-17. Four deep-research digests, one per plane of
> `canon/architecture/adr-008-multi-tenancy-model.md` (draft-2), surveying prior
> knowledge and best practice and holding our ladders against it.
>
> - `2026-08-17-adr008-plane-i-identity.md`
> - `2026-08-17-adr008-plane-a-authorization.md`
> - `2026-08-17-adr008-plane-e-enforcement.md`
> - `2026-08-17-adr008-plane-p-placement.md`
>
> Purpose: feed draft-3. Nothing here amends the ADR by itself.
## 1. Headline: the reframe is externally corroborated
The operator's draft-1 → draft-2 instruction — stop specifying one model, build
a framework of graduated levels — matches the most rigorous public guidance
almost verbatim:
> "Instead of viewing isolation as a discrete property, consider it a spectrum.
> You can deploy components of your architecture that are more isolated or less
> isolated than other components in the same architecture."
> — [Azure Architecture Center, Tenancy models](https://learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenancy-models)
Microsoft also derives our E↔P coupling independently: shared deployment ⇒
enforcement lives in application code; dedicated deployment ⇒ enforcement is
structural. Draft-2's shape is sound. What follows is about its details.
## 2. Corrections to draft-2 — things it currently states wrongly
Ranked by how badly they mislead.
| # | Correction | Plane | Where |
|---|---|---|---|
| C1 | **E3 does not prevent an application from routing around it.** Any session can re-`SET` the GUC; SQL injection or a compromised process defeats it entirely. E3 is strong against *accident*, weak against *compromise*. The ADR's phrase "the application cannot trivially route around" overclaims — the exact fault §6 prohibits. | E | §4.3 |
| C2 | **The pooling claim in §16 is backwards.** Transaction-scoped context (`SET LOCAL`) is what makes E3 *safe* under pooling; **statement**-level pooling is what breaks it — returning other tenants' rows, only under production concurrency. The ADR says E3 "forecloses aggressive transaction-level pooling". | E | §16 |
| C3 | **"Set at pool checkout" is the wrong scope.** Should be set transaction-locally inside an explicit transaction. | E | §4.3 |
| C4 | **E3 does not supersede E2 — it layers on it.** Practice treats them as coexisting; a service dropping E2 on reaching E3 is worse off, because E3 fails open under injection. Claiming E3 should require the E2 artifact too. | E | §4.3, §13 |
| C5 | **P cannot express per-tenant placement, which §11 requires.** A tier requiring `E3 P2` bought by only some tenants puts the service at two P levels at once. The posture vector cannot represent it, forcing an over- or under-claim. | P | §5, §11 |
## 3. Gaps — real concerns no plane covers
| # | Gap | Suggested home |
|---|---|---|
| G1 | **Tenant offboarding and deletion.** OWASP lists complete data deletion on offboarding as a MUST. `tenant-engine` has reversible retirement and explicitly no hard-delete; `platform-pg` retains backups 30 days. Falls between two repos' declared scopes. | New open question; a lifecycle stage across all four planes, not a fifth plane. |
| G2 | **Non-database stores.** Caches, search indices and background jobs are named leak surfaces. Our ladders are datastore-shaped throughout. | Scope note on the E ladder, or widened evidence artifacts. |
| G3 | **Identity-provider placement.** Realm-per-tenant vs Keycloak Organizations is the same silo/pool decision, live in our estate, undecided. Realm-per-tenant has a stated ceiling of 5–20 tenants. | Note in §3 that P scopes to data substrate; identity placement is parallel and unowned. |
| G4 | **Noisy-neighbour evidence.** Azure requires testing both that data does not leak *and* that noisy-neighbour outcomes are acceptable. §13 covers only leakage. | Add a P1/P2 evidence artifact. |
| G5 | **Tenant→deployment mapping record.** A first-class component in the reference guidance. Our placement facts in §1 were assembled by inspection. | Follows from G3/C5. |
| G6 | **Cell sizing.** `platform-pg` is an uncapped cell with a computed ~6-consumer ceiling and no overflow target. | Reframes §18.8: ask what its declared maximum size is. |
| G7 | **Tenant identifier encodes mutable headcount bands.** `tenant:<grouping>:<name>` embeds `small`/`medium`/`large`, which change. Contradicts identifier-design consensus. | Not ADR-008's; raise to `tenant-engine` / NetKingdom canon re ADR-0013. |
| G8 | **Logging.** OWASP MUST: include tenant context in all log entries. Unmodelled. | Minor; fold into evidence artifacts. |
## 4. Opportunities — cheaper or better than we assumed
| # | Finding | Effect |
|---|---|---|
| O1 | **pgTAP + pgrls** (67 RLS lint rules, semantic policy-diff for CI gating, pytest isolation plugin) exist off the shelf. `rapp-postgres` already runs a 15-probe harness of the same shape. | The E3 half of open question §18.3 is far cheaper than assumed. Splits the question into a tractable mechanical half and a human half. |
| O2 | **OWASP Multi-Tenant Security Cheat Sheet** is the closest external baseline to our framework and is referenced nowhere in the estate. | Adopt as the named external baseline in ADR §15. |
| O3 | **OpenID AuthZEN Authorization API 1.0** reached Final Specification January 2026; Keycloak shipped experimental support May 2026. Demonstrated PDP interoperability across 5+ engines. | `flex-auth`'s bespoke `CheckRequest` is pre-standard. Propose an `A4` rung for standard-interface PDP delegation. |
| O4 | **Token versioning** as a middle rung between cached claim and live re-query. | Cheap global invalidation without a hot-path lookup. |
| O5 | **BOLA / OWASP API1.** Our tenant-boundary gap is the industry's #1 API vulnerability, top of the list since it launched. | Cite by name. Converts an internal judgement into an external consensus for reviewers with a security background. |
## 5. Specific technical risks for the E3 implementation contract (§18.6)
Concrete requirements the contract must carry, each sourced in the enforcement
digest:
- `FORCE ROW LEVEL SECURITY` on every tenant-owned table. Without it the table
owner bypasses policies **silently** — and ADR-0001 already established that
objects created by `<consumer>_migrate` are owned by `_migrate`, so our
migration role would bypass every policy on tables it created.
- No `BYPASSRLS` attribute on any leased role.
- `SECURITY INVOKER` for ordinary logic; `SECURITY DEFINER` reserved for
deliberate, reviewed cross-tenant access.
- Context set with `SET LOCAL` inside an explicit transaction, never
session-scoped.
- An `EXPLAIN` comparison before and after enabling RLS: functional indexes
using non-leakproof functions (`lower`, `enum_eq`) stop being used under RLS.
Relevant to `tenant-engine`, which sits on the synchronous authorization path
and just requested a 5s statement timeout.
## 6. Recommended shape of draft-3
1. Apply corrections C1–C4 to §4.3, §13 and §16. These are the ones a reviewer
will otherwise catch, and C1 is the ADR overclaiming in exactly the way it
forbids.
2. Resolve C5 by making placement expressible per tenant class — a default plus
exceptions in the posture vector — or by declaring per-tenant placement out
of scope and accepting that isolation tiers become unmodellable.
3. Split open question §18.3 into a mechanical half (E1/E3, tooling exists) and
a semantic half (E2, adversarial review on a recorded cadence). Cross-tenant
findings "require a human tester who can set up separate tenant contexts and
compare the responses" — a green CI run is not E2 evidence.
4. Add G1 (offboarding/deletion) and G3 (identity placement) as open questions;
add G2 as a scope note.
5. Cite the external baselines — OWASP Multi-Tenant Security Cheat Sheet, Azure
tenancy models, AWS SaaS Lens — in §15, and BOLA/API1 in §13.2.
6. Reframe §18.8 from "evaluate cells" to "declare `platform-pg`'s maximum
size and its overflow target".
7. Route G7 to `tenant-engine` separately. It is a critique of ratified canon,
not of this ADR, and folding it in would overreach.
## 7. Method and limits
Four search-and-read passes, one per plane, against vendor architecture
guidance (AWS Well-Architected SaaS Lens, Azure Architecture Center), standards
bodies (OpenID Foundation), security baselines (OWASP), PostgreSQL practitioner
literature, and IdP vendor documentation. Full source lists are in each digest.
Limits worth stating:
- No primary academic sources beyond the Zanzibar lineage; this is an
engineering-practice survey, not a literature review.
- Vendor guidance is not neutral — AWS and Azure both describe patterns their
platforms sell. The Azure tenancy-models page is the most rigorous taxonomy
found and is used heavily in the P digest; readers should discount its
Azure-service-specific recommendations accordingly.
- Nothing here was tested against our own systems. Every claim about our estate
is drawn from repo documents, not from probing running infrastructure.
- The E digest's corrections are the highest-confidence findings, being
PostgreSQL mechanics rather than architectural opinion.

224
research/2026-08-17-adr008-plane-a-authorization.md Normal file
View file

@ -0,0 +1,224 @@
# Plane A — Authorization: prior knowledge and best practices
> Research artefact — 2026-08-17. Supports `canon/architecture/adr-008-multi-tenancy-model.md`
> (draft-2), plane A of four. Surveys external practice for binding a request
> to the tenants it may act for, and holds our `A0–A3` ladder against it.
> Companions: plane I (identity), plane E (enforcement), plane P (placement).
## 1. What the plane covers
How a request is bound to the tenants its principal may act for, where that
decision is made, and how the decision point is kept honest. Our ladder:
| Level | State |
|---|---|
| A0 | None, or tenant context not carried |
| A1 | Ad-hoc checks scattered through handlers |
| A2 | Single local authorization boundary; context bound once |
| A3 | Delegated to `flex-auth` as PDP, live re-query where the IAM Profile requires |
## 2. This is the industry's #1 vulnerability class, by name
The single most important external finding: what our framework calls a tenant
boundary failure, the security industry calls **Broken Object Level
Authorization**, and it is not a marginal concern.
- BOLA is **API1** in the OWASP API Security Top 10 and "has held the top spot
since the list launched"
([Decryption Digest — BOLA and IDOR testing guide](https://www.decryptiondigest.com/blog/bola-idor-api-authorization-test-remediation)).
- It is "consistently the most common, the most exploitable, and the most
damaging API vulnerability across bug bounty submissions, public disclosures,
and production penetration tests" (ibid.).
- The multi-tenant framing is explicit: "a single missing authorization check
can let one tenant access another's storage buckets, database records, or
configuration data"
([Wiz — OWASP API security top 10](https://www.wiz.io/academy/api-security/owasp-api-security)).
**Against our model:** ADR-008 §13.2 identifies the missing cross-tenant
evidence artifact as the framework's largest gap. This research upgrades that
from an internal judgement to an industry consensus: we have no test coverage
for the vulnerability class that ranks first globally. The ADR should cite
BOLA/API1 by name so the gap is legible to anyone who arrives with a security
background rather than an architecture one.
## 3. OWASP has a Multi-Tenant Security Cheat Sheet, and we should adopt it wholesale
This is the most directly applicable artefact found in the whole research
effort, and neither ADR-008 nor any repo document references it
([OWASP Multi-Tenant Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Multi_Tenant_Security_Cheat_Sheet.html)).
Its normative requirements, mapped to our ladders:
| OWASP requirement | Our plane | Status |
|---|---|---|
| "Never trust client-supplied tenant IDs without validation" — derive from verified JWT claims | I2 | Implied, not stated. Tighten. |
| "Establish tenant context early in the request lifecycle (middleware/interceptor)"; bind to the authenticated session | A2 | **Matches A2 exactly.** |
| "Always validate that requested resources belong to the current tenant" | E1–E2 | Matches. |
| "Include tenant context in all log entries" | — | **Not covered by any plane.** |
| Tenant-isolated audit trails | — | `audit-core`'s concern; not in the framework. |
| "Ensure complete data deletion on tenant offboarding" | — | **Not covered. See §6.** |
| Prefix all cache keys with tenant identifiers | — | **Not covered** (see plane I §3). |
| Per-tenant rate limiting with tier-based quotas | — | Adjacent to `tenant-engine` guardrails. |
| Separate API keys per tenant | E4-adjacent | Our E4 is credential-based; consistent. |
Explicitly rejected anti-patterns, two of which we should check ourselves
against:
- **"Skipping tenant validation for internal services."** Our estate is mostly
internal service-to-service calls — `flex-auth` calls `tenant-engine`
synchronously on the authorization path, `user-engine` calls it for tenant
creation. Our A ladder describes a request arriving with a subject; it does
not say whether a *service* identity calling on behalf of a tenant carries
and revalidates tenant context. This is a real question for A3.
- **"Exposing sequential or guessable resource IDs."** Not assessed anywhere.
**Proposed action:** reference the cheat sheet from ADR-008 as the external
baseline, and fold its uncovered requirements (logging, cache keys, offboarding
deletion) into either the ladders or an explicit scope exclusion.
## 4. The PDP/PEP split is right, and there is now a standard for the wire
Our A3 delegates decisions to `flex-auth` as PDP, which is the mainstream
architecture — a Policy Enforcement Point intercepts an action and queries a
Policy Decision Point
([DEV — RBAC vs ABAC vs ReBAC](https://dev.to/kanywst/rbac-vs-abac-vs-rebac-how-to-choose-and-implement-access-control-models-3i2d)).
What is new since our contracts were written:
- **OpenID AuthZEN Authorization API 1.0** was approved as an OpenID Final
Specification in January 2026 and published Standards Track in March 2026
([OpenID Foundation — Authorization API 1.0 final specification approved](https://openid.net/authorization-api-1-0-final-specification-approved/),
[spec](https://openid.github.io/authzen/)).
- Its purpose is exactly our seam: letting PDPs and PEPs "communicate
authorization requests and decisions to each other without requiring
knowledge of each other's inner workings"
([Curity — introduction to AuthZEN](https://curity.io/resources/learn/authzen/)).
- Interoperability is demonstrated, not theoretical: one PEP implementation
driving 5+ different PDPs (Topaz, Axiomatics, OpenFGA) by switching endpoint
URLs alone
([DEV — AuthZEN deep dive](https://dev.to/kanywst/authzen-authorization-api-10-deep-dive-the-standard-api-that-separates-authorization-decisions-1m2a)).
- Keycloak shipped experimental AuthZEN support in May 2026
([Keycloak — AuthZEN as experimental feature](https://www.keycloak.org/2026/05/authzen-as-experimental-feature)).
**Against our model:** `flex-auth` uses a bespoke `CheckRequest` shape and a
bespoke action vocabulary (`tenant.update`, `tenant.guardrail.set`, …), with
action strings copied verbatim between repos to avoid re-derivation. That
coupling is exactly what AuthZEN standardises away. We are not wrong — we are
pre-standard, and the standard landed seven months ago.
**Proposed amendment:** A3 currently says "delegated to `flex-auth` as PDP".
Consider distinguishing:
- **A3** — externalised to a PDP (any protocol).
- **A4** — externalised over a standard interface (AuthZEN), so the PDP is
swappable and the PEP is not coupled to one engine's request shape.
That gives the ladder somewhere to go and makes the current coupling visible
rather than invisible. It also reframes `key-cape`/Keycloak's AuthZEN support
as a fleet opportunity rather than an unrelated upstream feature.
## 5. Model choice: our vocabulary is RBAC-shaped, our problem may be ReBAC-shaped
- ReBAC originates in Google's **Zanzibar**, modelling authorization as a graph
of typed relationships, with authorization data stored as explicit tuples in
a dedicated datastore
([AuthZed — introduction to Google Zanzibar](https://authzed.com/learn/google-zanzibar)).
- It is described as "well-suited to multi-tenant enterprise environments where
access patterns are relational rather than role-based", and specifically for
modelling "complex customer organizational structures while maintaining
strict data isolation" (ibid.,
[AuthZed — ABAC vs ReBAC](https://authzed.com/learn/abac-vs-rebac-when-to-use-which)).
- Open implementations: SpiceDB, OpenFGA, Authzed
([IAM Day by Day — Zanzibar & ReBAC](https://iamdaybyday.com/patterns/authorization/zanzibar/)).
**Against our model:** our capability roles (`PLTF`/`IAM`/`VEN`/`CUS`) are
tenant-level RBAC, and `user-engine` owns memberships separately. The moment a
question like "may this user act for this tenant, given they are a member of an
organisation that administers it" arises, that is a relationship query, and
role-flattening will start to strain. Not a present problem — flagged because
the estate's own tenant taxonomy (`family`, `community`, `association`) is
overtly relational and will produce these questions.
## 6. Finding: nothing in the framework covers tenant exit
OWASP lists "ensure complete data deletion on tenant offboarding" as a **must**
([OWASP Multi-Tenant Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Multi_Tenant_Security_Cheat_Sheet.html)).
Our position today: `tenant-engine`'s lifecycle contract provides reversible
retirement and states plainly that **there is no hard-delete**. That is a
defensible design for an audit-bearing estate, but it means:
- No plane in ADR-008 has anything to say about tenant exit, data deletion, or
the interaction with retention.
- Physical backups on `platform-pg` retain 30 days, so even a hypothetical
hard-delete leaves tenant data recoverable for a month — which is a GDPR
erasure question, not just an architecture one.
- `rapp-postgres` SCOPE explicitly excludes tenant identity, and
`tenant-engine` explicitly excludes storage. Deletion falls between them.
**Proposed action:** this is a fifth concern the framework does not model, and
it is not obviously a fifth *plane* — it is a lifecycle stage cutting across
all four. Recommend adding it as an explicit open question rather than
inventing a plane for it. The estate's `gdpr_report` tooling in the hub
suggests someone has thought about this elsewhere; worth locating before
drafting.
## 7. Finding: automated conformance testing may not be sufficient for this plane
This challenges the recommendation I gave in the ADR review (§18.3, "build a
fleet conformance suite").
- "Finding cross-tenant vulnerabilities requires semantic understanding of what
data belongs to which context, and that understanding requires a human tester
who can set up separate tenant contexts and compare the responses."
- "Across 119 SaaS assessments, the highest-impact findings were concentrated
in access control, business logic, and tenant isolation, precisely the
categories that require context-aware human testing rather than automated
tool execution."
([Bugstrix — multi-tenant SaaS security testing](https://bugstrix.com/blogs/multi-tenant-saas-security-testing-how-to-prevent-cross-tenant-data-leaks/))
**Against our model:** this does not invalidate the conformance suite — a
mechanical check that every tenant-owned table carries the key (E1), or that a
session without the GUC reads nothing (E3), is precisely the kind of structural
assertion automation does well. What it invalidates is the hope that a suite
*replaces* adversarial review at E2, where the boundary lives in business
logic and the failure is semantic.
**Proposed amendment to §13:** split the evidence artifacts by what can be
automated. E1 and E3 are mechanical and belong in CI. E2 is semantic and needs
a periodic adversarial review with a recorded date, closer to a pen-test
cadence than a test suite. Claiming E2 on the strength of a green CI run would
be exactly the overclaim §6 of the ADR prohibits.
## 8. Summary against our ladder
| Finding | Verdict |
|---|---|
| Tenant boundary failure = BOLA = OWASP API1, top of the list since launch | **Reframe.** Cite by name; our largest gap is the industry's largest risk. |
| OWASP Multi-Tenant Security Cheat Sheet exists and is unreferenced | **Adopt.** Closest external baseline to our framework. |
| A2 matches OWASP's "establish context early, bind to session" | **Aligned.** |
| "Skipping tenant validation for internal services" is a named anti-pattern | **Check ourselves.** Our estate is mostly internal service-to-service. |
| PDP/PEP split is mainstream | **Aligned.** |
| AuthZEN 1.0 Final (Jan 2026) standardises the PDP wire; Keycloak experimental May 2026 | **Gap.** `flex-auth` is pre-standard and bespoke. Propose A4. |
| ReBAC/Zanzibar suits relational tenant structures | **Watch.** Our `family`/`community`/`association` groupings are relational. |
| Tenant offboarding deletion is a MUST we do not model | **Finding.** No plane covers exit; interacts with 30-day backup retention. |
| Cross-tenant testing needs human semantic review, not only automation | **Amend §13.** E1/E3 mechanical, E2 adversarial-review with a date. |
## Sources
- [OWASP — Multi-Tenant Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Multi_Tenant_Security_Cheat_Sheet.html)
- [Wiz — OWASP API Security Top 10 risks and how to mitigate them](https://www.wiz.io/academy/api-security/owasp-api-security)
- [Decryption Digest — BOLA and IDOR testing guide](https://www.decryptiondigest.com/blog/bola-idor-api-authorization-test-remediation)
- [SecPortal — Broken Object Level Authorization (BOLA) guide](https://secportal.io/vulnerabilities/broken-object-level-authorization)
- [Bugstrix — Multi-tenant SaaS security testing: preventing cross-tenant data leaks](https://bugstrix.com/blogs/multi-tenant-saas-security-testing-how-to-prevent-cross-tenant-data-leaks/)
- [OpenID Foundation — Authorization API 1.0 Final Specification approved](https://openid.net/authorization-api-1-0-final-specification-approved/)
- [OpenID AuthZEN — Authorization API 1.0 specification](https://openid.github.io/authzen/)
- [OpenID Foundation — AuthZEN working group specifications](https://openid.net/wg/authzen/specifications/)
- [Curity — An introduction to Authorization Exchange (AuthZEN)](https://curity.io/resources/learn/authzen/)
- [DEV — AuthZEN Authorization API 1.0 deep dive](https://dev.to/kanywst/authzen-authorization-api-10-deep-dive-the-standard-api-that-separates-authorization-decisions-1m2a)
- [Keycloak — AuthZEN as an experimental feature (May 2026)](https://www.keycloak.org/2026/05/authzen-as-experimental-feature)
- [Auth0 — A guide to OpenID AuthZEN's Authorization API 1.0](https://auth0.com/blog/implementing-authzen-guide-openid-authorization-api/)
- [AuthZed — An introduction to Google Zanzibar and ReBAC](https://authzed.com/learn/google-zanzibar)
- [AuthZed — ABAC vs ReBAC: when to use which](https://authzed.com/learn/abac-vs-rebac-when-to-use-which)
- [IAM Day by Day — Zanzibar & relationship-based access control](https://iamdaybyday.com/patterns/authorization/zanzibar/)
- [DEV — RBAC vs ABAC vs ReBAC: how to choose](https://dev.to/kanywst/rbac-vs-abac-vs-rebac-how-to-choose-and-implement-access-control-models-3i2d)

228
research/2026-08-17-adr008-plane-e-enforcement.md Normal file
View file

@ -0,0 +1,228 @@
# Plane E — Enforcement: prior knowledge and best practices
> Research artefact — 2026-08-17. Supports `canon/architecture/adr-008-multi-tenancy-model.md`
> (draft-2), plane E of four. Surveys external practice for where the tenant
> boundary is mechanically enforced, and holds our `E0–E4` ladder against it.
> **This digest contains a correction to the ADR's own definition of E3.**
> Companions: plane I (identity), plane A (authorization), plane P (placement).
## 1. What the plane covers
Where the tenant boundary actually holds, and what defeats it. Our ladder:
| Level | Mechanism | ADR claims it is defeated by |
|---|---|---|
| E0 | None | Anything |
| E1 | Tenant-keyed, filtered per query | One missing predicate |
| E2 | Single service-side choke point | Code bypassing the choke point |
| E3 | RLS keyed on a session GUC set at pool checkout | `SECURITY DEFINER`, missing policy, wrong GUC |
| E4 | Per-tenant credential and substrate | Platform-level compromise |
## 2. Correction: E3's description is wrong, and the error matters
ADR-008 draft-2 describes E3 as "platform-assisted: row-level security keyed on
a session GUC set at pool checkout, or an equivalent data-access layer **the
application cannot trivially route around**".
That last clause is false for the GUC mechanism, and the literature is blunt
about it:
> "Any session can `SET` any custom parameter. RLS policies that rely on
> `current_setting('app.tenant_id')` are only as secure as the application's
> control over SQL execution. Users with direct access can trivially
> `SET app.tenant_id = '999'` and bypass the policy."
> — [kmoppel — Yes, Postgres can do session vars, but should you use them?](https://kmoppel.github.io/2025-06-03-yes-postgres-can-do-session-variables-but-should-you-use-them/)
And the corollary: "SQL injection could enable an attacker to issue their own
`SET` command, therefore accessing other customer's data. The session variable
based approach is only safe when you protect yourself against SQL injections"
(ibid.).
**What this means for the framework.** Draft-1's instinct was not baseless
after all — but neither was draft-2's reversal. The honest resolution is that
E3's strength is **threat-model dependent**, and the ladder must say so:
| Threat | Does E3 help? |
|---|---|
| Developer forgets a `WHERE tenant_id = ?` | **Yes, decisively.** This is the common case and the reason E3 exists. |
| A new code path bypasses the repository/choke point | **Yes.** The database still filters. |
| SQL injection reaching the connection | **No.** The attacker can re-`SET` the GUC. |
| Compromised application process | **No.** It holds a credential that can address all tenants. |
So E3 is a strong control against **accident** and a weak one against
**compromise**. E4 is the only rung that holds against both, because the
credential itself cannot address another tenant's data.
**Proposed amendment to §4.3 of the ADR:** replace "the application cannot
trivially route around" with an explicit threat-model row. The current wording
overclaims in precisely the way §6 of the ADR prohibits — an irony worth fixing
before reviewers find it.
## 3. Three PostgreSQL mechanics that will bite our specific role model
These are not generic warnings; each interacts with `rapp-postgres` ADR-0001 as
built.
### 3.1 The table owner bypasses RLS silently — and our migration role owns tables
> "By default, the table owner bypasses RLS policies unless the table is
> altered with `FORCE ROW LEVEL SECURITY`."
> — [pganalyze — RLS, security invoker views and LEAKPROOF functions](https://pganalyze.com/blog/5mins-postgres-row-level-security-bypassrls-security-invoker-views-leakproof-functions)
> "The `FORCE` keyword is critical — without it, the table owner bypasses
> policies silently."
> — [Kawshik — securing pgvector with PostgreSQL RLS](https://kawshik.dev/blog/multi-tenant-rag-pgvector-postgres-rls.html)
ADR-0001 already discovered — via `isolation-test.sh`, not review — that
**objects created by `<consumer>_migrate` are owned by `_migrate`**, not by
`_owner`, because default privileges key on the creating role. That finding was
about grants. It has a second consequence nobody has drawn: under RLS, the
`_migrate` role would silently bypass every policy on every table it created.
Add `BYPASSRLS` to the picture — roles with that attribute are exempt
([oneuptime — securing multi-tenant data with RLS](https://oneuptime.com/blog/post/2026-01-25-row-level-security-postgresql/view))
— and the E3 provisioning contract needs to state explicitly: `FORCE ROW LEVEL
SECURITY` on every tenant-owned table, no `BYPASSRLS` on any leased role, and
the runtime role must not own the objects (which ADR-0001 already guarantees,
for a different reason).
### 3.2 Connection pooling can silently serve the wrong tenant's rows
This is the failure mode most likely to reach production undetected.
> "RLS with `SET` or `SET LOCAL` will not work properly with pgbouncer in
> statement pooling mode; you will likely return rows for the wrong users and
> it may only happen in production when multiple people are hitting the app at
> once." — [Daniel Imfeld — PostgreSQL row level security](https://imfeld.dev/notes/postgresql_row_level_security)
> "In highly concurrent environments with connection pooling (like pgbouncer),
> managing this context correctly is a major source of subtle, hard-to-debug
> bugs. If the context leaks between connections, so might the data."
> — [ShiftAsia — an intro to PostgreSQL's RLS](https://shiftasia.com/community/why-your-database-needs-boundaries-an-intro-to-postgresqls-row-level-security-rls/)
The mitigation is consistent across sources: set the context **transaction-locally**
(`SET LOCAL` inside an explicit transaction), never session-locally, so pooling
mode cannot leak it. Our ADR says "set at pool checkout", which is the
session-scoped phrasing and the wrong one.
**Against our model:** §16 of the ADR notes that E3 "forecloses aggressive
transaction-level pooling". That is backwards — transaction-scoped context is
what makes E3 *safe* under pooling; it is **statement**-level pooling that is
incompatible. Correct the scaling section as well as §4.3.
This also sharpens the ADR's own §12 point about designing guarding for
invisible failure: an RLS context leak produces correct-looking rows for the
wrong tenant. No error, no log line, no crash.
### 3.3 RLS silently disables some indexes
> "Due to security concerns, functional indices using leakproof functions
> cannot be used for RLS-secured queries. Functions such as `lower` or
> `enum_eq` are not leakproof. This means indices such as
> `some_table(lower(email))` will not be used when RLS is active."
> — [sigmoid.at — notes on PostgreSQL row level security](https://sigmoid.at/post/2021/09/23/postgresql_row_level_security/)
A service adopting E3 may see specific queries fall off an index with no schema
change. Relevant to `tenant-engine` in particular, which sits on `flex-auth`'s
synchronous authorization path and just requested a 5s statement timeout on
exactly that basis.
**Proposed addition:** the E3 provisioning contract (ADR §18.6) should require
an EXPLAIN comparison before and after enabling RLS, not just a correctness
probe.
## 4. The verification gap has off-the-shelf answers
ADR §13.2 calls the missing E2/E3 evidence artifacts the framework's largest
live gap and asks who should build them. The research says: less building than
expected.
- **pgTAP** is the established way to assert RLS behaviour in CI. "RLS policies
should be tested in CI with pgTAP, not just in dev, since policy regressions
are silent (no query error, just wrong data returned), and automated
cross-tenant isolation tests are the only reliable guard"
([Blair Jordan — testing RLS policies with pgTAP](https://blair-devmode.medium.com/testing-row-level-security-rls-policies-in-postgresql-with-pgtap-a-supabase-example-b435c1852602),
[MakerKit — database testing with pgTAP](https://makerkit.dev/docs/next-supabase-turbo/development/database-tests)).
- **pgrls** is a purpose-built static analyzer: 67 lint rules for tenant and
per-user row-scoping bugs, performance traps and hygiene, 20 auto-fixable, a
semantic policy-diff command for CI gating, and a pytest plugin for isolation
tests ([pgrls on GitHub](https://github.com/pgrls/pgrls)).
The named common failures match ours exactly: "a forgotten role, a pool reusing
connections with the wrong tenant context, or an admin path bypassing RLS
entirely" ([Kawshik](https://kawshik.dev/blog/multi-tenant-rag-pgvector-postgres-rls.html)).
**Against our model:** the E3 evidence artifact in §13 ("RLS policies present on
every tenant-owned table; probe that a session without the GUC reads nothing;
probe that a wrong GUC reads nothing") is well specified and is roughly what
pgrls lints for plus what pgTAP asserts. `rapp-postgres` already runs
`scripts/isolation-test.sh` with 15 probes against a throwaway PostgreSQL 16 in
Docker — the same harness shape, pointed at the consumer boundary. Extending it
to the tenant boundary is incremental, not novel.
**This materially changes the answer to open question §18.3.** The E3 half is
tooling-assisted and cheap. The E2 half remains semantic and human (see plane A
§7). Splitting the question that way makes it assignable.
## 5. The layered consensus: RLS as defence in depth, not as the only layer
Vendor and community guidance converges on the same layering our ADR describes,
which is reassuring for the ladder's shape even where the details need fixing.
- AWS documents RLS as the mechanism for pooled multi-tenant isolation and
pairs it with a non-owner application role
([AWS — multi-tenant data isolation with PostgreSQL RLS](https://aws.amazon.com/blogs/database/multi-tenant-data-isolation-with-postgresql-row-level-security/)).
- OWASP lists RLS under database-level enforcement and application-layer ORM
filters as *complementary*, recommending "database-level isolation as
defense-in-depth" rather than as a replacement
([OWASP Multi-Tenant Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Multi_Tenant_Security_Cheat_Sheet.html)).
- The application-layer half is a custom ORM/repository that automatically
appends tenant filters and injects `tenant_id` on write (ibid.) — which is
precisely our E2.
- "Use `SECURITY INVOKER` for typical multi-tenant logic so that the caller's
RLS context applies"; reserve `SECURITY DEFINER` for deliberate, controlled
cross-tenant access
([oneuptime](https://oneuptime.com/blog/post/2026-01-25-row-level-security-postgresql/view)).
**Against our model:** the E ladder is ordinal — each rung replacing the last —
but practice treats E2 and E3 as **layers that coexist**. A service at E3 that
dropped its E2 choke point would be worse off, not better, because E3 alone
fails open under SQL injection (§2).
**Proposed amendment:** state that E3 *presupposes* E2 rather than superseding
it, and that a claim of E3 requires the E2 evidence artifact as well. That is a
small change with real consequences for how the ladder is read — it converts
"move up a rung" into "add a layer", which is what the sources actually
describe.
## 6. Summary against our ladder
| Finding | Verdict |
|---|---|
| Any session can re-`SET` the GUC; E3 fails against SQL injection and app compromise | **Correction.** E3's "cannot trivially route around" overclaims. Add a threat-model table. |
| E3 is decisive against forgotten predicates — the common case | **Confirms the reversal.** Draft-2's direction is right, its wording is not. |
| Table owner bypasses RLS silently without `FORCE ROW LEVEL SECURITY` | **Specific risk.** Our `_migrate` role owns the tables it creates (ADR-0001). |
| `BYPASSRLS` roles are exempt | Provisioning contract must forbid it on leased roles. |
| Statement-pooling + `SET` returns other tenants' rows, only under production concurrency | **Correction.** Use `SET LOCAL` in-transaction; ADR §16's pooling claim is backwards. |
| RLS disables functional indexes using non-leakproof functions | **New.** Require an EXPLAIN comparison in the E3 contract. |
| pgTAP + pgrls (67 lint rules, policy-diff CI gate, pytest plugin) exist | **Adopt.** §18.3's E3 half is far cheaper than assumed. |
| Policy regressions are silent — wrong data, no error | **Confirms §12.** Guarding must target invisible failure. |
| Practice treats E2 and E3 as coexisting layers, not successive rungs | **Amend.** E3 should presuppose E2, not replace it. |
## Sources
- [kmoppel — Yes, Postgres can do session vars, but should you use them?](https://kmoppel.github.io/2025-06-03-yes-postgres-can-do-session-variables-but-should-you-use-them/)
- [Daniel Imfeld — PostgreSQL row level security (notes)](https://imfeld.dev/notes/postgresql_row_level_security)
- [pganalyze — RLS in Postgres, security invoker views and why LEAKPROOF functions matter](https://pganalyze.com/blog/5mins-postgres-row-level-security-bypassrls-security-invoker-views-leakproof-functions)
- [AWS Database Blog — Multi-tenant data isolation with PostgreSQL Row Level Security](https://aws.amazon.com/blogs/database/multi-tenant-data-isolation-with-postgresql-row-level-security/)
- [oneuptime — How to secure multi-tenant data with RLS in PostgreSQL](https://oneuptime.com/blog/post/2026-01-25-row-level-security-postgresql/view)
- [sigmoid.at — Notes on PostgreSQL row level security](https://sigmoid.at/post/2021/09/23/postgresql_row_level_security/)
- [ShiftAsia — Why your database needs boundaries: an intro to PostgreSQL's RLS](https://shiftasia.com/community/why-your-database-needs-boundaries-an-intro-to-postgresqls-row-level-security-rls/)
- [pgEdge — It depends: using session variables in Postgres](https://www.pgedge.com/blog/it-depends-using-session-variables-in-postgres)
- [Kawshik — The multi-tenant RAG nightmare: securing pgvector with PostgreSQL RLS](https://kawshik.dev/blog/multi-tenant-rag-pgvector-postgres-rls.html)
- [pgrls — static analyzer for Postgres Row-Level Security](https://github.com/pgrls/pgrls)
- [Blair Jordan — Testing RLS policies in PostgreSQL with pgTAP](https://blair-devmode.medium.com/testing-row-level-security-rls-policies-in-postgresql-with-pgtap-a-supabase-example-b435c1852602)
- [MakerKit — Database testing with pgTAP](https://makerkit.dev/docs/next-supabase-turbo/development/database-tests)
- [Rico Fritzsche — Mastering PostgreSQL RLS for rock-solid multi-tenancy](https://ricofritzsche.me/mastering-postgresql-row-level-security-rls-for-rock-solid-multi-tenancy/)
- [pganalyze — Using Postgres row-level security in Ruby on Rails](https://pganalyze.com/blog/postgres-row-level-security-ruby-rails)
- [OWASP — Multi-Tenant Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Multi_Tenant_Security_Cheat_Sheet.html)

234
research/2026-08-17-adr008-plane-i-identity.md Normal file
View file

@ -0,0 +1,234 @@
# Plane I — Identity: prior knowledge and best practices
> Research artefact — 2026-08-17. Supports `canon/architecture/adr-008-multi-tenancy-model.md`
> (draft-2), plane I of four. Surveys external practice for how multi-tenant
> systems name, carry, and validate tenant identity, and holds our `I0–I3`
> ladder against it.
> Companions: plane A (authorization), plane E (enforcement), plane P (placement).
> Index: `2026-08-17-adr008-multi-tenancy-research-index.md`.
## 1. What the plane covers
How a tenant is named, where that name is bound to a principal, how it travels
with a request, and how fresh the binding must be. Our ladder:
| Level | State |
|---|---|
| I0 | No tenant concept |
| I1 | Local tenant notion, non-canonical |
| I2 | Canonical `tenant:<grouping>:<name>`, validated; `tenant-engine` owns existence |
| I3 | I2 + capability roles honoured, live re-query for `aal2`-class decisions |
## 2. Consensus practice: bind at registration, carry in the token
The dominant pattern across vendor guidance is that tenant identity is bound
to the principal **at the identity provider**, not derived per-application.
- AWS calls the unpacked token claims the **tenant context**, and frames it as
environment-like: it "influences how the SaaS application processes the
request" rather than being one parameter among many
([SaaS Lens — preventing cross-tenant access](https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/preventing-cross-tenant-access.html)).
- Binding is explicitly a **registration-time** responsibility: storing the
tenant identifier as a claim or attribute on the user in the IdP "shifts the
responsibility of mapping identities to tenants from each application to the
user registration process"
([AWS Prescriptive Guidance — tenant onboarding and registration](https://docs.aws.amazon.com/prescriptive-guidance/latest/saas-multitenant-api-access-authorization/avp-design-onboarding-registration.html)).
- The recommendation is to define tenant context **at the IAM layer** rather
than relying on the application layer
([AWS — 5 multi-tenant SaaS architecture best practices](https://aws.amazon.com/isv/resources/5-multi-tenant-saas-architecture/)).
**Against our model:** this is what `I2` describes and what the IAM Profile
already mandates (`tenant` claim required on every token accepted by profile
consumers). We are aligned with consensus here, and `key-cape`/Keycloak
stamping the claim at issuance is the sanctioned shape.
**Gap:** the ladder's I2 says "canonical identifiers used, validated". It does
not say *where the binding happens*. A service could satisfy I2 by parsing a
tenant id out of a request body. Consider tightening I2 to require the binding
be IdP-sourced, with request-supplied tenant ids explicitly at I1.
## 3. Propagation is a separate discipline from naming
Practice treats "the tenant id must reach every layer" as its own requirement,
not a consequence of having one:
- A stable identifier should follow every request and stay consistent across
databases, services, logs, billing records, and background processes; without
a reliable tenant registry, other isolation controls are hard to enforce
consistently ([WorkOS — developer's guide to SaaS multi-tenant architecture](https://workos.com/blog/developers-guide-saas-multi-tenant-architecture)).
- Requests should execute in a tenant context established by middleware or
context variables, tagging each request with its owner
([SSOJet — multi-tenant identity management](https://ssojet.com/blog/multi-tenant-identity-management)).
- Two leak surfaces get called out repeatedly and are **not** database
concerns: search indices (Elasticsearch queries that omit the tenant filter)
and cache keys that do not include the tenant id
([WorkOS](https://workos.com/blog/developers-guide-saas-multi-tenant-architecture)).
**Against our model:** ADR-008 has nothing to say about background jobs,
caches, or search indices. Our E ladder is described almost entirely in
database terms (tenant-keyed tables, RLS, per-tenant credentials). A service
could sit honestly at `E3` and still leak through a Redis key or an unfiltered
index.
**Proposed amendment:** either widen the E-ladder evidence artifacts to cover
non-database stores, or add an explicit scope note that the E ladder describes
the *primary datastore* only and other stores are assessed separately. The
second is cheaper and more honest.
## 4. Identity-plane placement is its own decision — and it has a known ceiling
Identity providers face the same silo/pool question the P plane describes, and
the industry has converged on an answer with a number attached.
- Keycloak offers **realm-per-tenant** (strong isolation, heavyweight) versus
**Organizations** (multiple tenants inside one realm, stable since Keycloak
26, designed for the SaaS case)
([Phase Two — multi-tenancy options in Keycloak](https://phasetwo.io/blog/multi-tenancy-options-keycloak/),
[Skycloak — multitenancy using Organizations](https://skycloak.io/blog/multitenancy-in-keycloak-using-the-organizations-feature/)).
- The operational verdict is blunt: "realm-per-tenant works for 5–20 tenants;
beyond that, you are managing an IAM platform, not building your product"
([Izylife — Keycloak multi-tenancy: realm per tenant or Organizations?](https://izylifesolutions.com/articles/keycloak-multi-tenancy-b2b-iam/)).
- Auth0's equivalent is Organizations within a single Auth0 tenant, with
per-organization connections
([KeycloakPro — Keycloak multi-tenancy with Organizations](https://keycloakpro.com/blog/keycloak-multi-tenancy-organizations-guide)).
**Against our model:** ADR-008 treats placement as a property of *data
substrate* only. Identity has a placement axis too, it is live in our estate
(`key-cape`/Keycloak), and nobody has decided realm-per-tenant versus
Organizations. Given the tenant taxonomy includes `consumer` (private
individuals) and `family`, realm-per-tenant is as economically impossible here
as cluster-per-tenant — the 5–20 ceiling is well below our target tenant count.
**Proposed amendment:** note in §3 of the ADR that the P plane, as written,
scopes to data substrate; identity-provider placement is a parallel decision
with the same silo/pool shape and should be recorded, probably by whoever owns
`key-cape`.
## 5. Freshness: the tradeoff we already made, confirmed
Our I3 requires live `tenant-engine` re-query for privileged, destructive,
credential-vending or `aal2`-class decisions, with the cached `tenant_roles`
claim acceptable for ordinary ones. External practice supports the split and
names the cost precisely.
- The tradeoff is explicit: "tighter revocation controls often increase latency
and operational overhead, requiring organisations to balance immediate cutoff
against service reliability"
([NHI Mgmt Group — JWT vs OAuth revocation](https://nhimg.org/faq/why-do-jwt-revocation-and-oauth-revocation-need-different-handling/)).
- For high-stakes decisions the sanctioned answer is opaque tokens plus
introspection (RFC 7662) — "a real kill switch at the cost of a lookup on the
hot path" ([oneuptime — how to handle JWT revocation](https://oneuptime.com/blog/post/2026-02-02-jwt-revocation/view)).
- Short lifetimes are the mainstream mitigation: "a 5-minute access token caps
how stale any resource server's view can be, which is the quiet reason the
default is 5 minutes and not an hour" (ibid.). Our IAM Profile's 5–30 minute
service/agent token lifetime sits inside this norm.
- Known failure mode directly relevant to us: controls "break down when
services cache JWT validation results" (ibid.). Our hybrid explicitly permits
caching for ordinary decisions, so the boundary between ordinary and
high-stakes is load-bearing and must be enforced, not documented.
- **Token versioning** is an alternative we do not use: a `token_version` per
subject, bumped on change, rejecting stale tokens service-side
([TechNet Experts — JWT token versioning](https://www.technetexperts.com/jwt-token-versioning-stale-data/)).
It gives cheap global invalidation without a per-request lookup, at the cost
of a version read. Worth considering as a middle rung between our cached
claim and full live re-query.
**Against our model:** we match best practice and, unusually, we wrote the
boundary down. The gap is evidential rather than architectural — our I3
evidence artifact ("live re-query demonstrated on an `aal2`-class path; cached
path shown not to be used there") is exactly right and does not exist yet for
any service.
## 6. Finding: our tenant identifier encodes a mutable attribute
This is the significant new finding, and it is a critique of ratified canon
rather than of ADR-008.
Our identifier is `tenant:<grouping>:<name>`, where `<grouping>` is an
onboarding-risk / entity-shape classification from ADR-0013: `trial`,
`friendly`, `single`, `small` (≤10 employees), `medium` (≤100), `large`
(≤1000), `enterprise` (1001+), `consumer`, `family`, `community`,
`association`, `agentic`.
Several of those are **explicitly headcount bands**, and headcount changes. A
tenant onboarded as `tenant:small:acme` that grows past ten employees is
either misclassified forever or must be renamed — and renaming a tenant
identifier is a estate-wide data migration, since the identifier is the key
shared with `user-engine`, appears in OpenBao paths (`tenants/<name>/...`),
and is carried in every token.
External practice is unambiguous:
- "Avoid encoding mutable data or business logic in the identifier" — the
**ID Entanglement Effect**, where composite ids encode business relationships
or mutable state that is later parsed
([Cogna — the unreasonable power of identifiers](https://cogna.co/blog/the-unreasonable-power-of-identifiers/)).
- Opaque identifiers "decouple identification from descriptive attributes,
thereby enhancing stability against entity changes"
([Grokipedia — unique identifier](https://grokipedia.com/page/Unique_identifier)).
- The FAIR/persistent-identifier literature reaches the same conclusion: make
identifiers opaque from the start, so that when a name changes "just the
`rdfs:label` and some other properties may change — but not the URI"
([Cool URIs for FAIR Knowledge Graphs](https://arxiv.org/pdf/2407.09237)).
- A structured id is acceptable *only if treated as opaque* — "the key is that
we treat it as an unchanging reference and don't try to parse meaning from
its structure" ([Cogna](https://cogna.co/blog/the-unreasonable-power-of-identifiers/)).
The IAM Profile half-anticipates this. It states that capability roles were
deliberately separated from the identifier so "a tenant's roles may change
without renaming its identifier", and that grouping "does not gate which
capability roles a tenant may hold". The same reasoning applies to grouping
itself and was not carried through.
**Two honest readings**, and this needs an owner decision rather than an
assertion from this document:
1. **Grouping is onboarding-time and frozen by design.** It records the shape
of the entity *when it was onboarded*, not its current headcount, and is
never expected to change. If so, this should be stated explicitly in
ADR-0013 — because every reader will otherwise assume `small` means
currently-small, and some system will eventually make a decision on it.
2. **Grouping is current classification.** Then the identifier encodes a
mutable attribute, the estate has a rename problem it has not hit yet
because tenant counts are tiny, and the fix (opaque id + grouping as a
mutable property on the tenant record) gets more expensive every month.
Either way, `rapp-postgres` is already correct to treat these as opaque
strings — ADR-0001 §4 prohibits the storage platform from parsing them. The
risk sits with any consumer that *does* parse the grouping segment.
**Proposed action:** raise as a question to `tenant-engine` and NetKingdom
canon rather than folding into ADR-008. It is out of ADR-008's scope but
materially affects plane I's top rung.
## 7. Summary against our ladder
| Finding | Verdict |
|---|---|
| Bind tenant at IdP, carry in token, treat as context not parameter | **Aligned.** Matches I2 + IAM Profile. |
| I2 does not require the binding be IdP-sourced | **Gap.** Tighten wording; request-supplied ids belong at I1. |
| Propagation must cover caches, search indices, background jobs | **Gap.** ADR-008's ladders are datastore-shaped. Add a scope note. |
| Identity-provider placement is its own silo/pool decision, ceiling ~5–20 tenants for realm-per-tenant | **Missing.** P plane scopes to data only; identity placement is undecided and unowned. |
| Hybrid cached-claim / live-re-query for high-stakes | **Aligned and unusually well specified.** Evidence artifact does not exist yet. |
| Token versioning as a middle option | **Unconsidered.** Cheap global invalidation without hot-path lookup. |
| Identifier encodes mutable headcount bands | **Finding.** Contradicts identifier-design consensus; needs an ADR-0013 clarification. |
## Sources
- [AWS SaaS Lens — Preventing cross-tenant access](https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/preventing-cross-tenant-access.html)
- [AWS Well-Architected SaaS Lens (PDF)](https://docs.aws.amazon.com/pdfs/wellarchitected/latest/saas-lens/wellarchitected-saas-lens.pdf)
- [AWS Prescriptive Guidance — Tenant onboarding and user tenant registration](https://docs.aws.amazon.com/prescriptive-guidance/latest/saas-multitenant-api-access-authorization/avp-design-onboarding-registration.html)
- [AWS — 5 multi-tenant SaaS architecture best practices](https://aws.amazon.com/isv/resources/5-multi-tenant-saas-architecture/)
- [AWS Security Blog — SaaS tenant isolation with ABAC using STS support for tags in JWT](https://aws.amazon.com/blogs/security/saas-tenant-isolation-with-abac-using-aws-sts-support-for-tags-in-jwt/)
- [WorkOS — The developer's guide to SaaS multi-tenant architecture](https://workos.com/blog/developers-guide-saas-multi-tenant-architecture)
- [SSOJet — Multi-tenant identity management for SaaS](https://ssojet.com/blog/multi-tenant-identity-management)
- [Phase Two — Understanding multi-tenancy options in Keycloak](https://phasetwo.io/blog/multi-tenancy-options-keycloak/)
- [Skycloak — Multitenancy in Keycloak using the Organizations feature](https://skycloak.io/blog/multitenancy-in-keycloak-using-the-organizations-feature/)
- [Izylife — Keycloak multi-tenancy: realm per tenant or Organizations?](https://izylifesolutions.com/articles/keycloak-multi-tenancy-b2b-iam/)
- [KeycloakPro — Keycloak multi-tenancy with Organizations](https://keycloakpro.com/blog/keycloak-multi-tenancy-organizations-guide)
- [oneuptime — How to handle JWT revocation](https://oneuptime.com/blog/post/2026-02-02-jwt-revocation/view)
- [NHI Management Group — Why JWT revocation and OAuth revocation need different handling](https://nhimg.org/faq/why-do-jwt-revocation-and-oauth-revocation-need-different-handling/)
- [TechNet Experts — How to fix stale JWTs via token versioning](https://www.technetexperts.com/jwt-token-versioning-stale-data/)
- [Cogna — The unreasonable power of identifiers](https://cogna.co/blog/the-unreasonable-power-of-identifiers/)
- [Grokipedia — Unique identifier](https://grokipedia.com/page/Unique_identifier)
- [Cool URIs for FAIR Knowledge Graphs (arXiv 2407.09237)](https://arxiv.org/pdf/2407.09237)

244
research/2026-08-17-adr008-plane-p-placement.md Normal file
View file

@ -0,0 +1,244 @@
# Plane P — Placement: prior knowledge and best practices
> Research artefact — 2026-08-17. Supports `canon/architecture/adr-008-multi-tenancy-model.md`
> (draft-2), plane P of four. Surveys external practice for which substrate
> holds a tenant's data, and holds our `P0–P4` ladder against it.
> **Contains strong external validation of draft-2's reframe, and one
> expressiveness failure in the P ladder itself.**
> Companions: plane I (identity), plane A (authorization), plane E (enforcement).
## 1. What the plane covers
Which physical substrate holds a tenant's data, who decides, and what moving
costs. Our ladder:
| Level | Shape |
|---|---|
| P0 | Shares a database with another consumer |
| P1 | Database per consumer, shared cluster |
| P2 | Dedicated cluster per consumer |
| P3 | Dedicated cluster per tenant |
| P4 | P3 plus separate region or jurisdiction |
## 2. Validation: "isolation is a spectrum, not a discrete property"
The operator's instruction to reframe draft-1 from a fixed model into a
graduated framework is not merely defensible — it is Microsoft's stated
position, in almost the same words:
> "Instead of viewing isolation as a discrete property, consider it a spectrum.
> You can deploy components of your architecture that are more isolated or less
> isolated than other components in the same architecture, depending on your
> requirements."
> — [Azure Architecture Center — Tenancy models for a multitenant solution](https://learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenancy-models)
The same page decomposes isolation across tiers exactly as our planes do —
"you can use different levels of isolation for each tier", with a shared UI
tier, a shared middle tier, and isolated databases given as the worked example.
**Against our model:** draft-2's core move is externally corroborated by the
most rigorous public taxonomy available. Worth citing in ADR §15, because it
converts "we decided to do it this way" into "this matches the reference
guidance", which matters for reviewers who will ask whether we invented a
private vocabulary.
## 3. Validation: our E↔P coupling claim is independently confirmed
ADR §3.2 asserts that the planes couple at the top — `E4` is only reachable at
`P3` or above — and §4.4 asserts that `P0 → P1 → P2` does not raise the E
level. Microsoft states the mechanism plainly:
> "A key difference between tenants and deployments is how isolation is
> enforced. When multiple tenants share a single deployment (a set of
> infrastructure), you typically rely on your application code and a tenant
> identifier that's in a database to keep each tenant's data separate. When
> tenants have their own dedicated deployments, they have their own
> infrastructure, so it might be less important for your code to account for a
> multitenant environment." (ibid.)
That is our E/P relationship derived independently: shared deployment ⇒
enforcement is application code (our E1–E3); dedicated deployment ⇒ enforcement
is structural (our E4).
## 4. Finding: our P ladder cannot express per-tenant placement, and our own commercial section requires it
This is the significant gap.
Azure names two partitioning axes that our single ladder collapses:
- **Vertically partitioned deployments** — "a combination of single-tenant and
multitenant deployments. For example, you might have most of your customers'
data and application tiers on multitenant infrastructures, but you deploy
single-tenant infrastructures for customers who require higher performance or
data isolation." Explicitly monetisable: "You can even charge customers a
higher rate to use a single-tenant deployment." (ibid.)
- **Horizontally partitioned deployments** — shared application tier, dedicated
databases per tenant, targeted at the component that carries the load: "if
you identify that specific components cause most of the load on your system,
then you can deploy separate components for each tenant." (ibid.)
Our P ladder assigns **one level per service**. `tenant-engine` is `P1`.
`audit-core` is `P1`. There is no way to say *"P1 for most tenants, P3 for the
three enterprise tenants who bought isolation"* — which is precisely the
vertically partitioned model, and precisely what ADR §11's minimum-level plan
tiers imply.
The contradiction is internal to draft-2: §11.1 says "a tier may require
`E3 P2`", but if only some tenants hold that tier, the delivering service
occupies two P levels at once. The posture vector as specified cannot represent
that state, so a service in it would have to either overclaim (declare P2 for
everyone) or underclaim (declare P1 and misrepresent what the tier delivers).
Both are §6 violations.
**Proposed amendment.** Either:
1. **Make P a per-tenant-class level rather than a per-service one** — the
vector carries a default plus exceptions:
```yaml
tenancy:
current: { I: 2, A: 3, E: 2, P: 1 }
placement_exceptions:
- tenants: ["tenant:enterprise:*"]
P: 3
reason: "isolation tier"
```
2. Or **declare per-tenant placement out of scope** for the framework and say
so, accepting that plan tiers selling isolation are then unmodellable.
Option 1 is more work and is the honest one, because the estate has already
anticipated tiered isolation. Azure also flags the cost: "your codebase needs
to be designed to support both multitenant and single-tenant deployments", and
migration between them must be planned for (ibid.) — which is exactly what our
§7 portability rules already provide, so we are better positioned for this than
most.
## 5. Cell-based architecture: the pattern our ladder skips
ADR §18.8 asks whether to evaluate cells. The literature gives enough to
sharpen the question.
- A cell is "a complete, independent instance of the service" with **a fixed
maximum size** — compute, storage, supporting services and configuration for
a defined blast radius
([Rackspace — cell-based architecture on AWS](https://www.rackspace.com/blog/cell-based-architecture-aws),
[AWS guidance for cell-based architecture](https://github.com/aws-solutions-library-samples/guidance-for-cell-based-architecture-on-aws)).
- Cells are logical partitions on a **partition key** — customer ID ranges or
tenant ID — and "the data required for or generated by each cell is also
partitioned, so that there is no replication of data between cells" (ibid.).
- The sizing argument is the point: "by capping the size of each cell… you can
potentially predict and address failures more easily. This leads to a higher
MTBF" (ibid.). And "it is important to identify the proper partition key to
minimize cross-cell interactions" (ibid.).
- Cells are the bulkhead pattern applied to tenancy — AWS files it under
fault isolation, REL10-BP04
([AWS Well-Architected — use bulkhead architectures to limit scope of impact](https://docs.aws.amazon.com/en_us/wellarchitected/2022-03-31/framework/rel_fault_isolation_use_bulkhead.html)).
- Azure's equivalent is the **Deployment Stamps pattern**: "each stamp serves a
predefined number of tenants, allowing you to scale almost linearly and serve
increasing numbers of tenants"
([Azure — Deployment Stamps pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/deployment-stamp)).
Azure also calls deployments "supertenants or stamps" and describes the
tenant→deployment mapping table as a first-class component.
**Against our model:** a cell is not a rung on our P ladder — it is a different
axis. P measures *how much is shared*; a cell measures *how large a shared unit
is allowed to get before you make another one*. Our `platform-pg` is, in cell
terms, an uncapped cell: §16 of the ADR computes a ceiling of roughly six
consumers but nothing enforces it, and there is no second cell to overflow
into.
That reframes §18.8 usefully. The question is not "should we adopt cells" but
"**what is `platform-pg`'s declared maximum size, and what happens at the
boundary?**" — a question we can answer now, cheaply, and which the connection
ceiling already forces.
Azure also names **bin packing** as the assignment strategy and stresses
maintaining "a record of your customers and the infrastructure where their data
and applications reside so that you can route their traffic to the correct
location" — a tenant→deployment mapping table. We have no such record; the
placement facts in ADR §1 were assembled by inspection.
## 6. Placement is a commercial decision, and the guidance says so first
Our §11 treats commercial expression as a constraint to be managed. Azure puts
it the other way round:
> "Selecting a tenancy model isn't only a technical decision. It's also a
> commercial decision." — considering business objectives, compliance, scale,
> automation capacity, and SLAs (ibid.).
And on the economics, matching our §14 rejection of R3/P3-everywhere: "if a
single tenant requires a specific infrastructure cost, 100 tenants probably
require 100 times that cost", with ongoing maintenance time-consuming at scale
(ibid.). Conversely single-tenant is appropriate "if you expect that your
business will have only a few customers… even though it's more costly".
**Against our model:** ADR §8.2 proposes placement ownership to
`railiance-platform`, an operations-shaped repo. This guidance suggests
placement policy is *jointly* commercial and technical — which strengthens the
case that `adaptive-pricing` must be a co-signer on §18.2, not only on §18.5.
## 7. Our tenant taxonomy is a recognised hard case
Azure's B2C guidance describes our exact situation:
> "If your customers are consumers, it's often more complicated to relate
> customers, tenants, and users… consider whether your solution might be used
> by families, groups of friends, clubs, associations, or other groups that
> might need to access and manage their data together. For example, a music
> streaming service might support both individual users and families, and it
> might treat each of these account types differently when it separates them
> into tenants." (ibid.)
Our ADR-0013 taxonomy contains `consumer`, `family`, `community`, and
`association` verbatim. It also notes that if tenants are individuals, "you
might need to consider how you handle personal data and about the data
sovereignty laws in each jurisdiction" — which is our `P4` and, combined with
plane A §6, the tenant-deletion gap.
## 8. Testing placement is a named obligation
> "Whichever isolation model you choose, be sure to test your solution to verify
> that one tenant's data isn't accidentally leaked to another **and that any
> noisy neighbor outcomes are acceptable**." (ibid.)
Two evidence artifacts, not one. ADR §13 covers the leak half (P1–P4:
"provisioning declaration plus the platform's own isolation probes") and says
nothing about noisy-neighbour behaviour. Azure suggests deliberate fault
injection (Chaos Studio) to verify resilience.
**Proposed amendment to §13:** add a noisy-neighbour evidence artifact for P1
and P2 — a demonstration that one consumer saturating its connection or CPU
allowance does not breach another's. `rapp-postgres` already sets
database-scoped `connection_limit`, `statement_timeout` and
`idle_in_transaction_session_timeout` precisely for this, and ADR-0001 §3 argues
resource exhaustion is an isolation failure — but nothing proves the limits
hold under load.
## 9. Summary against our ladder
| Finding | Verdict |
|---|---|
| "Isolation is a spectrum, not a discrete property"; different tiers at different levels | **Validates draft-2.** Cite in §15. |
| Shared deployment ⇒ app-code enforcement; dedicated ⇒ structural | **Confirms E↔P coupling** independently. |
| Vertical partitioning: some tenants pooled, some dedicated, explicitly monetised | **Gap.** P is per-service; cannot express per-tenant placement that §11 requires. |
| Horizontal partitioning: shared app tier, per-tenant DB for the hot component | **Unmodelled.** Our P assumes whole-service granularity. |
| A cell has a **fixed maximum size**; stamps scale near-linearly | **Reframes §18.8.** Ask "what is `platform-pg`'s declared max size?", not "should we adopt cells". |
| Tenant→deployment mapping table is a first-class component | **Missing.** Our placement facts were assembled by inspection. |
| Tenancy model is a commercial decision as much as technical | **Amend §8.2.** `adaptive-pricing` should co-sign placement ownership. |
| B2C/family/association tenancy is a recognised hard case | Our taxonomy hits it verbatim; jurisdiction concerns feed P4. |
| Test for leakage **and** for acceptable noisy-neighbour outcomes | **Amend §13.** No noisy-neighbour evidence artifact exists. |
## Sources
- [Azure Architecture Center — Tenancy models for a multitenant solution](https://learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenancy-models)
- [Azure Architecture Center — Architectural approaches for a multitenant solution](https://learn.microsoft.com/en-us/azure/architecture/guide/multitenant/approaches/overview)
- [Azure Architecture Center — Deployment Stamps pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/deployment-stamp)
- [Azure Architecture Center — Considerations for updating a multitenant solution](https://learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/updates)
- [AWS Well-Architected — REL10-BP04 Use bulkhead architectures to limit scope of impact](https://docs.aws.amazon.com/en_us/wellarchitected/2022-03-31/framework/rel_fault_isolation_use_bulkhead.html)
- [AWS Well-Architected — How do you use fault isolation to protect your workload?](https://wa.aws.amazon.com/wellarchitected/2020-07-02T19-33-23/wat.question.REL_10.en.html)
- [AWS Solutions Library — Guidance for cell-based architecture on AWS](https://github.com/aws-solutions-library-samples/guidance-for-cell-based-architecture-on-aws)
- [Rackspace — Cell-based architecture on AWS](https://www.rackspace.com/blog/cell-based-architecture-aws)
- [Ryan Harrison — Cell-based architecture engineering guidelines](https://guidelines.ryanharrison.co.uk/architecture/cell-architecture/)
- [Anudeep Balla — The guide to cell-based architecture](https://medium.com/@anudeepballa7/the-guide-to-cell-based-architecture-58cf607f9754)
- [AWS Well-Architected SaaS Lens (PDF)](https://docs.aws.amazon.com/pdfs/wellarchitected/latest/saas-lens/wellarchitected-saas-lens.pdf)
- [hidekazu-konishi — AWS SaaS multi-tenant architecture guide: tenant isolation, pool and silo models](https://hidekazu-konishi.com/entry/aws_saas_multi_tenant_architecture_guide.html)