the-custodian/research/2026-08-17-adr008-multi-tenancy-research-index.md

# ADR-008 multi-tenancy — external research index

> Research artefact — 2026-08-17. Four deep-research digests, one per plane of
> `canon/architecture/adr-008-multi-tenancy-model.md` (draft-2), surveying prior
> knowledge and best practice and holding our ladders against it.
>
> - `2026-08-17-adr008-plane-i-identity.md`
> - `2026-08-17-adr008-plane-a-authorization.md`
> - `2026-08-17-adr008-plane-e-enforcement.md`
> - `2026-08-17-adr008-plane-p-placement.md`
>
> Purpose: feed draft-3. Nothing here amends the ADR by itself.

## 1. Headline: the reframe is externally corroborated

The operator's draft-1 → draft-2 instruction — stop specifying one model, build
a framework of graduated levels — matches the most rigorous public guidance
almost verbatim:

> "Instead of viewing isolation as a discrete property, consider it a spectrum.
> You can deploy components of your architecture that are more isolated or less
> isolated than other components in the same architecture."
> — [Azure Architecture Center, Tenancy models](https://learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenancy-models)

Microsoft also derives our E↔P coupling independently: shared deployment ⇒
enforcement lives in application code; dedicated deployment ⇒ enforcement is
structural. Draft-2's shape is sound. What follows is about its details.

## 2. Corrections to draft-2 — things it currently states wrongly

Ranked by how badly they mislead.

| # | Correction | Plane | Where |
|---|---|---|---|
| C1 | **E3 does not prevent an application from routing around it.** Any session can re-`SET` the GUC; SQL injection or a compromised process defeats it entirely. E3 is strong against *accident*, weak against *compromise*. The ADR's phrase "the application cannot trivially route around" overclaims — the exact fault §6 prohibits. | E | §4.3 |
| C2 | **The pooling claim in §16 is backwards.** Transaction-scoped context (`SET LOCAL`) is what makes E3 *safe* under pooling; **statement**-level pooling is what breaks it — returning other tenants' rows, only under production concurrency. The ADR says E3 "forecloses aggressive transaction-level pooling". | E | §16 |
| C3 | **"Set at pool checkout" is the wrong scope.** Should be set transaction-locally inside an explicit transaction. | E | §4.3 |
| C4 | **E3 does not supersede E2 — it layers on it.** Practice treats them as coexisting; a service dropping E2 on reaching E3 is worse off, because E3 fails open under injection. Claiming E3 should require the E2 artifact too. | E | §4.3, §13 |
| C5 | **P cannot express per-tenant placement, which §11 requires.** A tier requiring `E3 P2` bought by only some tenants puts the service at two P levels at once. The posture vector cannot represent it, forcing an over- or under-claim. | P | §5, §11 |

## 3. Gaps — real concerns no plane covers

| # | Gap | Suggested home |
|---|---|---|
| G1 | **Tenant offboarding and deletion.** OWASP lists complete data deletion on offboarding as a MUST. `tenant-engine` has reversible retirement and explicitly no hard-delete; `platform-pg` retains backups 30 days. Falls between two repos' declared scopes. | New open question; a lifecycle stage across all four planes, not a fifth plane. |
| G2 | **Non-database stores.** Caches, search indices and background jobs are named leak surfaces. Our ladders are datastore-shaped throughout. | Scope note on the E ladder, or widened evidence artifacts. |
| G3 | **Identity-provider placement.** Realm-per-tenant vs Keycloak Organizations is the same silo/pool decision, live in our estate, undecided. Realm-per-tenant has a stated ceiling of 5–20 tenants. | Note in §3 that P scopes to data substrate; identity placement is parallel and unowned. |
| G4 | **Noisy-neighbour evidence.** Azure requires testing both that data does not leak *and* that noisy-neighbour outcomes are acceptable. §13 covers only leakage. | Add a P1/P2 evidence artifact. |
| G5 | **Tenant→deployment mapping record.** A first-class component in the reference guidance. Our placement facts in §1 were assembled by inspection. | Follows from G3/C5. |
| G6 | **Cell sizing.** `platform-pg` is an uncapped cell with a computed ~6-consumer ceiling and no overflow target. | Reframes §18.8: ask what its declared maximum size is. |
| G7 | **Tenant identifier encodes mutable headcount bands.** `tenant:<grouping>:<name>` embeds `small`/`medium`/`large`, which change. Contradicts identifier-design consensus. | Not ADR-008's; raise to `tenant-engine` / NetKingdom canon re ADR-0013. |
| G8 | **Logging.** OWASP MUST: include tenant context in all log entries. Unmodelled. | Minor; fold into evidence artifacts. |

## 4. Opportunities — cheaper or better than we assumed

| # | Finding | Effect |
|---|---|---|
| O1 | **pgTAP + pgrls** exist off the shelf; `rapp-postgres` already runs a 15-probe harness of the same shape. pgrls **verified directly** 2026-08-17: MIT, 67 rules (54 security, incl. BYPASSRLS / LEAKPROOF / `SECURITY DEFINER` / cross-tenant leaks), 19 auto-fixable, Z3 isolation proofs, SAFE/BREAKING policy-diff for CI gating, pytest plugin, PG 15–17. **But it is a 26-star beta project** — adopt the approach with confidence, treat the dependency as a supply-chain decision. pgTAP is long-established and carries no such caveat. | The E3 half of open question §18.3 is far cheaper than assumed. Splits the question into a tractable mechanical half and a human half. |
| O2 | **OWASP Multi-Tenant Security Cheat Sheet** is the closest external baseline to our framework and is referenced nowhere in the estate. | Adopt as the named external baseline in ADR §15. |
| O3 | **OpenID AuthZEN Authorization API 1.0** reached Final Specification January 2026; Keycloak shipped experimental support May 2026. Demonstrated PDP interoperability across 5+ engines. | `flex-auth`'s bespoke `CheckRequest` is pre-standard. Propose an `A4` rung for standard-interface PDP delegation. |
| O4 | **Token versioning** as a middle rung between cached claim and live re-query. | Cheap global invalidation without a hot-path lookup. |
| O5 | **BOLA / OWASP API1.** Our tenant-boundary gap is the industry's #1 API vulnerability, top of the list since it launched. | Cite by name. Converts an internal judgement into an external consensus for reviewers with a security background. |

## 5. Specific technical risks for the E3 implementation contract (§18.6)

Concrete requirements the contract must carry, each sourced in the enforcement
digest:

- `FORCE ROW LEVEL SECURITY` on every tenant-owned table. Without it the table
  owner bypasses policies **silently** — and ADR-0001 already established that
  objects created by `<consumer>_migrate` are owned by `_migrate`, so our
  migration role would bypass every policy on tables it created.
- No `BYPASSRLS` attribute on any leased role.
- `SECURITY INVOKER` for ordinary logic; `SECURITY DEFINER` reserved for
  deliberate, reviewed cross-tenant access.
- Context set with `SET LOCAL` inside an explicit transaction, never
  session-scoped.
- An `EXPLAIN` comparison before and after enabling RLS: functional indexes
  using non-leakproof functions (`lower`, `enum_eq`) stop being used under RLS.
  Relevant to `tenant-engine`, which sits on the synchronous authorization path
  and just requested a 5s statement timeout.

## 6. Recommended shape of draft-3

1. Apply corrections C1–C4 to §4.3, §13 and §16. These are the ones a reviewer
   will otherwise catch, and C1 is the ADR overclaiming in exactly the way it
   forbids.
2. Resolve C5 by making placement expressible per tenant class — a default plus
   exceptions in the posture vector — or by declaring per-tenant placement out
   of scope and accepting that isolation tiers become unmodellable.
3. Split open question §18.3 into a mechanical half (E1/E3, tooling exists) and
   a semantic half (E2, adversarial review on a recorded cadence). Cross-tenant
   findings "require a human tester who can set up separate tenant contexts and
   compare the responses" — a green CI run is not E2 evidence.
4. Add G1 (offboarding/deletion) and G3 (identity placement) as open questions;
   add G2 as a scope note.
5. Cite the external baselines — OWASP Multi-Tenant Security Cheat Sheet, Azure
   tenancy models, AWS SaaS Lens — in §15, and BOLA/API1 in §13.2.
6. Reframe §18.8 from "evaluate cells" to "declare `platform-pg`'s maximum
   size and its overflow target".
7. Route G7 to `tenant-engine` separately. It is a critique of ratified canon,
   not of this ADR, and folding it in would overreach.

## 7. Adoption stance — structure, not tooling (operator decision, 2026-08-17)

Research output is used as **design input**, not as a shopping list. The
operator's rule:

> Use the structure that supports, extends or challenges our design. Do not
> adopt the tooling unless it is an established industry standard with broad
> application. Otherwise build ground up, so we can optimize and refactor as we
> see fit.

Applied to this survey:

| Finding | Adopt? | How |
|---|---|---|
| OWASP Multi-Tenant Security Cheat Sheet | **Yes — baseline** | A published security baseline, not tooling. Cite as the external reference our ladders answer to. |
| OWASP API Security Top 10 (BOLA / API1) | **Yes — vocabulary** | Names our gap in terms a security reviewer already holds. |
| OpenID AuthZEN Authorization API 1.0 | **Yes — standard** | Final Specification from a standards body with demonstrated multi-vendor interop. Qualifies as broad and established. Propose as the `A4` rung. |
| Azure tenancy models / AWS SaaS Lens / cell architecture | **Yes — structure** | Taxonomies and patterns. Corroborate the spectrum reframe and the E↔P coupling. |
| PostgreSQL RLS mechanics (`FORCE`, `BYPASSRLS`, `SET LOCAL`, leakproof) | **Yes — facts** | Engine behaviour, not a vendor's product. |
| **pgrls** | **No — structure only** | Take its rule *taxonomy* (54 security rules over RLS enablement, BYPASSRLS, LEAKPROOF, `SECURITY DEFINER`, cross-tenant leak; the SAFE/BREAKING/REQUIRES_REVIEW/DANGEROUS policy-diff classification) as a checklist for probes we write ourselves. 26 stars and beta was already a supply-chain caveat; the stance settles it. |
| **pgTAP** | **No** | Established in the Postgres world but still tooling, and we already have the ground-up equivalent: `scripts/isolation-test.sh`, 15 probes, bash + psql, no dependency tree. Extend that harness to the tenant boundary. |

The practical consequence is small and good: `rapp-postgres` already owns a
probe harness that found four real defects in its own provisioning SQL. The E2
and E3 evidence artifacts become new probes in an existing tool, informed by
pgrls's taxonomy of what to look for, with no new dependency. The
policy-diff idea — classifying a policy change as safe or breaking before it
lands — is the genuinely novel structural contribution worth reimplementing.

## 8. Retention and erasure — designed, not just flagged

Gap G1 (tenant offboarding and deletion) has been taken up rather than left as
an open question. See `rapp-postgres/docs/adr/ADR-0002-data-retention-and-erasure.md`.

The shape, per operator direction: retention is declared per consumer as
`N` days defaulting to 30, `rapp-postgres` provides the abstract mechanism for
any dataset, and a consumer or its governance layer builds policy on top of
that interface.

The finding that emerged while designing it: **per-consumer physical retention
is not implementable.** CNPG has one WAL stream and one window, so the instance
retention is derived as the maximum across consumers and every consumer's
erasure horizon is that maximum. A consumer declaring 7 days beside one
declaring 90 gets 90. This is the retention analogue of ADR-0001 §5 — state the
coupling rather than imply an isolation that is not there — and it produces two
things the estate could not previously say:

- The **erasure horizon**: the interval between deleting data and it ceasing to
  be recoverable from anything the platform holds. Deleted rows survive in
  backups for the whole window; that is the difference between "deleted" and
  "erased", and it had never been written down.
- **Retention as a placement trigger.** A consumer needing a shorter horizon
  than the instance floor cannot have one at P1. It moves to a dedicated
  cluster for a reason unrelated to performance — which is exactly why it needs
  recording, since nobody looks for a retention argument when reviewing
  placement.

Deletion splits mechanism from policy: the platform sweeps whole datasets and
records an opaque `deletionPolicyRef` it never interprets, so every deletion
traces to the policy that authorised it. Rows are not a dataset.
`consumerDatabase` is deliberately not sweepable.

**Open for draft-3:** whether this is a fifth plane (`R` — retention and
erasure, with its own ladder from "no policy" through "verified erasure within
a published horizon") or a lifecycle stage cutting across the existing four.
The interface is ladder-shaped, which argues for a plane — at the cost of the
framework's name.

## 9. Method and limits

Four search-and-read passes, one per plane, against vendor architecture
guidance (AWS Well-Architected SaaS Lens, Azure Architecture Center), standards
bodies (OpenID Foundation), security baselines (OWASP), PostgreSQL practitioner
literature, and IdP vendor documentation. Full source lists are in each digest.

Limits worth stating:

- No primary academic sources beyond the Zanzibar lineage; this is an
  engineering-practice survey, not a literature review.
- Vendor guidance is not neutral — AWS and Azure both describe patterns their
  platforms sell. The Azure tenancy-models page is the most rigorous taxonomy
  found and is used heavily in the P digest; readers should discount its
  Azure-service-specific recommendations accordingly.
- Nothing here was tested against our own systems. Every claim about our estate
  is drawn from repo documents, not from probing running infrastructure.
- The E digest's corrections are the highest-confidence findings, being
  PostgreSQL mechanics rather than architectural opinion.