the-custodian/research/2026-08-17-adr008-gap-research-erasure-and-noisy-neighbour.md

Gap research — erasure mechanism (R4) and noisy-neighbour evidence

Research artefact — 2026-08-17. Fifth digest supporting canon/architecture/adr-008-multi-tenancy-model.md (draft-3). Closes the two gaps the four plane digests left open: how R4 (verified erasure) can be reached at all, and what a noisy-neighbour evidence artifact should actually assert. Both findings change a level definition in draft-3. Index: 2026-08-17-adr008-multi-tenancy-research-index.md.

---

Part 1 — Erasure: R4 needs a mechanism it does not currently have

1.1 The problem draft-3 left standing

ADR-008 §4.5 defines R4 as "deletion proven complete across live data, backups and derived copies within the published horizon", and rapp-postgres ADR-0002 establishes that on shared substrate the horizon is the instance maximum — every co-resident consumer's retention window, whichever is longest.

Read together, those say: erasure completes by waiting. A tenant exercising a deletion right is told the data is gone in up to N days, where N is set by somebody else's retention requirement. That is honest, and it is a weak answer. It also makes R4 unreachable in any useful sense, because "we waited" is not a proof.

1.2 Crypto-shredding is the established mechanism

The industry answer to erasure under immutable or long-retention storage is to make the data unreadable rather than absent.

"Crypto-shredding for backups encrypts each subject (or tenant) with a unique key derived from the subject identifier, and on erasure, destroys the key. The backups remain but are cryptographically inaccessible." — oneuptime — setting up crypto-shredding for GDPR right-to-erasure

It is aimed precisely at our situation: the named use cases are "append-only or immutable storage (event sourcing, WORM, immutable backups) where physical deletion is impossible" (Granit — crypto-shredding: GDPR erasure without deleting rows).

Regulatory standing, stated carefully because overclaiming here would be worse than in any other section of this framework. The EDPB has not formally endorsed crypto-shredding as Article 17 erasure. Several data protection authorities have accepted it where physical deletion would be manifestly disproportionate effort under Article 17(1), and EDPB Guidelines 5/2019, the UK ICO and the French CNIL are cited as recognising it provided three conditions hold: strong encryption (AES-256), irreversible key destruction, and auditable destruction (pg_hardstorage — GDPR Art. 17 crypto-shred, Opsio — right to erasure in cloud systems).

The third condition is the one this estate is well placed for and the one most implementations skip: destruction must be auditable. ADR-0002 already requires every sweep to emit an evidence record. Key destruction is the same shape.

1.3 The finding: shredding granularity is bounded by the key boundary

Crypto-shredding "requires per-entity key isolation: each entity instance gets its own encryption key" (oneuptime, ibid.). That single requirement determines what is available to us, and it is not uniform across the estate.

What we want to shred	Key boundary needed	Available today?
A consumer's logical exports	Per-consumer key	Close. The offsite lane already encrypts with age, and OpenBao holds key material. This is a key-management change, not an architecture change.
A consumer's physical backup	Per-consumer key on the Barman object store	No. One WAL stream, one credential, one encryption context for the whole instance. Same constraint that forced the derived maximum in ADR-0002 §2.
A single tenant's rows inside a consumer	Per-tenant key, applied by the application before the data reaches storage	No, and not the platform's to build. Requires application-level per-tenant encryption.

The third row is the architecturally interesting one. Per-tenant crypto-shred means the consumer encrypts tenant data under a tenant-specific key before writing it — which is an enforcement-plane property, not a retention one. It also happens to be a strong one: data encrypted per tenant is unreadable across the boundary even by a credential that can address the rows, which is E4-class protection achieved without per-tenant substrate.

This is a new coupling and it belongs in §3.2. Draft-3 records two couplings (E4 needs P3; R's horizon is bounded by P). There is a third: R4 at tenant granularity requires per-tenant keys, which is an E-plane capability. Reaching the top of the retention ladder is not a retention project.

A reference implementation exists for the consumer-granularity case: pg_hardstorage "implements crypto-shred by destroying the per-tenant KEK that wraps every encrypted backup's DEK" (ibid.). Per the §14 adoption stance we take the structure — envelope encryption with a per-consumer key-encryption key wrapping each backup's data-encryption key, and destruction of the KEK as the erasure act — and build it on our own OpenBao and age lane rather than adopting the extension.

1.4 The tension nobody should paper over: erasure versus audit

audit-core exists to hold a tamper-evident trail. Crypto-shredding a tenant's audit records destroys evidence, and ADR-0001 §2 deliberately built the role model so that "a compromised runtime credential could not rewrite history at the schema level". Erasure is a sanctioned way to do exactly that.

The literature treats this as a genuine design tension rather than a solved problem (Remote Reason — balancing auditability and privacy with crypto-shredding). The usual resolution is to separate the fact of an event (retained, not personal) from its personal payload (encrypted per subject, shreddable), so the trail survives erasure with its personal content removed.

This is a question for audit-core, not an answer this framework should supply. Flagging it because a naive "R4 everywhere" target would silently instruct the audit service to destroy its own evidence.

1.5 Proposed amendments to draft-3

1. R4 admits two routes, and must name which one a service uses:
   • Horizon-elapsed — deletion is complete once the published erasure horizon has passed. Available to everyone, proves little, slow.
   • Key-destroyed — data is rendered unreadable immediately by destroying the key that decrypts it. Requires per-entity keys, AES-256-class encryption, and an auditable destruction record.
2. Add the third coupling to §3.2: R4 at tenant granularity requires an E-plane capability (per-tenant encryption). R4 at consumer granularity requires per-consumer keys on the retained artifacts.
3. State the regulatory position honestly wherever R4 is described: DPAs have accepted key destruction as erasure under conditions; the EDPB has not formally endorsed it. A plan tier claiming "your data is deleted" on the strength of crypto-shredding is making a defensible but not settled claim, and §11.4's hard line should say so rather than implying R4 is a clean yes.
4. Raise the audit tension as an open question addressed to audit-core.

---

Part 2 — Noisy neighbour: the evidence artifact asserts the wrong thing

2.1 The problem draft-3 left standing

ADR-008 §13 proposes a P1/P2 evidence artifact: "one consumer saturating its connection or CPU allowance demonstrably does not breach another's."

That is not achievable, and the reference guidance says so in its first line of solution text:

"Sharing a single resource inherently carries the risk of noisy neighbor problems that you can't completely avoid." — Azure Architecture Center — Noisy Neighbor antipattern

An artifact that can only ever fail, or that passes by being run gently enough, is worse than no artifact — it is the overclaim §6 prohibits, dressed as evidence.

2.2 There are two failure shapes, and we only guard against one

Azure distinguishes them explicitly:

• One greedy tenant. "At peak times, tenant A uses all of the system's resources, which means that any requests that tenant B makes fail." (ibid.)
• Aggregate of many modest tenants. "The noisy neighbor problem also occurs when each individual tenant consumes only a small portion of the system's capacity. However, the combined resource usage of many tenants can result in a peak in overall usage." (ibid.)

Our per-consumer governance — connection_limit, statement_timeout, idle_in_transaction_session_timeout, applied at database scope per ADR-0001 §3 — addresses the first shape well. It does nothing about the second.

And the second shape is exactly what §17 computes. Seven consumers, each politely within its 14-connection allowance, request 98 of 100 connections. Every one of them is well-behaved; the instance still fails. The connection ceiling in §17 is not merely a capacity number — it is the aggregate noisy-neighbour bound, and framing it as a scaling statistic understates what it is.

2.3 The detection signature we can actually build

Azure's service-side diagnostic is specific and unintuitive:

"Failures when performing an operation for a tenant: Look for failures that occur when a tenant isn't consuming a large share of the system's resources. This pattern might indicate that the tenant is experiencing a noisy neighbor problem. Track resource consumption by tenant." (ibid.)

The signal is a failure for a quiet consumer. That requires per-consumer resource attribution, and we already have the raw material: ADR-0001 stamps application_name per role (<consumer>/runtime), PostgreSQL logs carry database, user, application name and client address, and make postgres-ops-status already reports sessions by application name.

What is missing is a baseline. "It's important to clearly understand your normal baseline resource usage and to configure monitoring and alerts to detect spikes" (ibid.). No baseline for platform-pg has been recorded. Without one, a spike is indistinguishable from a Tuesday.

2.4 A normative requirement we are not meeting

"Be transparent with clients about any throttling mechanisms or usage quotas that you enforce. It's important that they handle failed requests gracefully and aren't caught off guard by limitations." (ibid.)

tenant-engine was told its connection and timeout limits during provisioning, which is good practice by accident rather than by rule. Nothing requires it. A consumer discovering its statement_timeout by hitting it in production is the failure this warns about.

Proposed: fold quota transparency into §10 (blast radius must be published) — the same disclosure obligation, extended from what a leaked credential can reach to what the platform will refuse to do for you.

2.5 Mitigations we have, and the ones we do not

Azure-recommended control	Our status
Query time limits / max returnable records	Have. statement_timeout, per-role since the tenant-engine change.
Connection quotas per tenant	Have. Database-scoped, per ADR-0001 §3.
Monitor overall and per-tenant resource usage	Partial. Attribution exists; no baseline, no alerting.
Throttling / rate limiting	None. No request-rate control at the database boundary.
Quality-of-service prioritisation	None. All consumers are equal, including on the synchronous authorization path where tenant-engine now sits.
Run heavy non-time-sensitive work off-peak	Implicit. Backups at 02:15; nothing else scheduled.
Rebalance tenants across stamps	Not possible. One instance, no second cell (§19.8).

The QoS absence is worth noting against a specific fact: tenant-engine chose a 5s statement timeout because flex-auth calls it synchronously on the authorization path. That is a latency-critical consumer sharing an instance with audit-core, which is not. Nothing prioritises one over the other, and the framework has no vocabulary for saying one consumer's latency matters more.

2.6 Proposed amendment to draft-3

Replace the §13 P1/P2 artifact with one that asserts something true:

Noisy-neighbour characterisation (P1/P2, adversarial, load-generated). A recorded baseline of normal per-consumer resource usage; a load run in which one consumer saturates its declared allowance; evidence that (a) the governance controls bind — the greedy consumer is throttled at its declared limits rather than exceeding them, and (b) the measured degradation experienced by co-resident consumers is recorded and judged acceptable. Carries a review date and the aggregate headroom at time of measurement.

That is achievable, it is falsifiable, and it produces a number — degradation under contention — that a plan tier promising performance would need anyway.

---

3. Summary of changes these two gaps force

#	Change	Where
A	R4 splits into horizon-elapsed and key-destroyed routes; a service must name which	§4.5
B	Third coupling: tenant-granularity R4 requires an E-plane capability (per-tenant encryption)	§3.2
C	R4's regulatory standing stated honestly — accepted by DPAs under conditions, not formally endorsed by the EDPB	§4.5, §11.4
D	Erasure-versus-audit tension raised as an open question to audit-core	§19
E	Noisy-neighbour artifact reworded from "does not breach" to characterisation with a recorded baseline and measured degradation	§13
F	§17's connection ceiling reframed as the aggregate noisy-neighbour bound, not only a capacity figure	§17
G	Quota transparency added as a disclosure obligation	§10
H	No QoS vocabulary despite a latency-critical consumer sharing with a batch one	§19, new

Sources

• oneuptime — How to set up crypto-shredding for GDPR right-to-erasure compliance
• cybertec-postgresql/pg_hardstorage — GDPR Art. 17 crypto-shred
• Granit — Crypto-shredding: GDPR erasure without deleting a single row
• Granit — Crypto-shredding (dotnet/compliance)
• Opsio — Right to erasure, Article 17 GDPR, deletion in cloud systems
• Remote Reason — Balancing auditability and privacy with crypto-shredding
• Azure Architecture Center — Noisy Neighbor antipattern
• Neon — The noisy neighbor problem in multitenant architectures
• PingCAP — Playbook: stop noisy neighbors, multi-tenant MySQL at scale
• Causal inference for quantifying noisy neighbor effects in multi-tenant cloud environments (arXiv 2604.03145)
• Azure Architecture Center — Throttling pattern
• Azure Architecture Center — Rate Limiting pattern