coulomb/doc-store-pg

Fork 0

tegwick 893753488e Seeded intent research program and initial specs

2026-08-19 21:22:38 +02:00

44 KiB

Executable file

Raw Permalink Blame History

ArchitectureBlueprint.md

Repository

doc-store-pg

Status

Architecture blueprint / research baseline.

This document defines the initial technical architecture for doc-store-pg. It is intentionally a blueprint rather than a frozen implementation specification. The architecture should remain stable enough to support reproducible experimentation while leaving individual components replaceable.

The central design question is:

Can document storage become a composable capability of a CloudNativePG/PostgreSQL platform while preserving MongoDB-compatible application surfaces, strong multitenancy choices, PostgreSQL-native access, and an evidence-driven path to long-term optimization?

1. Architectural Intent

doc-store-pg should provide a document-oriented data surface on top of PostgreSQL without turning PostgreSQL into an opaque implementation detail.

The architecture therefore preserves two first-class access paths:

Document access through MongoDB-compatible protocols and semantics.
PostgreSQL access through SQL and PostgreSQL-native tooling.

The project should prefer composition of upstream technologies over reimplementation.

The initial architecture is built around:

Kubernetes as the runtime substrate
CloudNativePG as the PostgreSQL lifecycle and HA layer
PostgreSQL 18+ as the initial database baseline
DocumentDB PostgreSQL extensions as the primary document engine candidate
DocumentDB Gateway and FerretDB as interchangeable MongoDB-compatible gateway candidates
PostgreSQL-native JSONB as a baseline and fallback document representation
Kubernetes-native configuration and observability
explicit multitenancy and tenant-placement policies
reproducible benchmark and conformance workloads

The architecture is designed so that these choices can be challenged by research rather than becoming permanent assumptions.

2. Architecture Principles

2.1 PostgreSQL Remains Visible

Document functionality is added to PostgreSQL; it does not replace the PostgreSQL model.

Applications or services may intentionally use:

MongoDB-compatible document APIs
SQL
JSON/JSONB queries
BSON/document functions
relational joins
PostgreSQL transactions
PostgreSQL extensions

This enables hybrid relational/document workloads to be treated as a core capability rather than an accidental escape hatch.

2.2 Composition Before Reimplementation

Existing open-source implementations should be evaluated before new equivalents are written.

In particular:

BSON storage and document semantics should initially come from DocumentDB.
MongoDB wire-protocol translation should initially come from DocumentDB Gateway or FerretDB.
PostgreSQL HA, failover, replication, backup integration, and cluster lifecycle should remain CloudNativePG responsibilities.

doc-store-pg should only implement functionality itself where integration, compatibility, policy, observability, or research requirements cannot reasonably be satisfied upstream.

2.3 Gateways Are Replaceable

MongoDB compatibility is an interface, not a hard dependency on one gateway.

MongoDB client
    |
    +--> DocumentDB Gateway --> PostgreSQL + DocumentDB
    |
    +--> FerretDB -----------> PostgreSQL + DocumentDB

Additional gateway implementations may later be evaluated using the same conformance and benchmark suites.

2.4 Tenant Placement Is Explicit

Multitenancy is not represented by a boolean flag.

A tenant should have an explicit placement/isolation model:

POOLED
NAMESPACE
DATABASE
DEDICATED

Future versions may add distributed or federated placement modes.

2.5 Compatibility Is Measured

"MongoDB compatible" is not considered a binary architectural property.

Compatibility should be measured separately for:

wire protocol
CRUD operations
query semantics
aggregation semantics
indexing
drivers
tools
transactions
sessions
change streams
errors and edge cases
application-level behavior

2.6 Evidence Before Optimization

Architectural decisions that materially affect compatibility, tenancy, performance, or operations should be benchmarkable.

The system should make it possible to ask:

Which configuration is better for this workload and why?

rather than:

Which database do we generally prefer?

2.7 Upstream Escape Hatches Remain Available

DocStorePG should not create unnecessary lock-in to its own abstractions.

A deployment should remain understandable through normal:

Kubernetes resources
CloudNativePG resources
PostgreSQL tools
DocumentDB tooling
MongoDB-compatible clients

The project should add useful abstraction, not hide the underlying systems.

3. System Context

flowchart TB
    subgraph Clients
        A[MongoDB-compatible applications]
        B[PostgreSQL / SQL applications]
        C[Operations and automation]
        D[Benchmark and conformance clients]
    end

    subgraph DocStorePG["doc-store-pg capability"]
        G[MongoDB-compatible Gateway]
        P[PostgreSQL Access]
        T[Tenant / Placement Policy]
        O[Observability]
        X[Benchmark & Conformance Hooks]
    end

    subgraph DataPlane["CloudNativePG Data Plane"]
        CNPG[CloudNativePG Cluster]
        PG[(PostgreSQL)]
        DDB[DocumentDB Extensions]
        JSONB[Native JSONB]
    end

    A --> G
    B --> P
    C --> T
    D --> X

    G --> PG
    P --> PG
    T --> CNPG
    X --> G
    X --> P

    CNPG --> PG
    PG --- DDB
    PG --- JSONB
    PG --> O
    G --> O

The important architectural point is that DocStorePG is not initially a database engine.

It is a capability profile formed from:

PostgreSQL
+ document engine
+ compatible gateway
+ CloudNativePG operations
+ tenancy model
+ policy
+ observability
+ conformance
+ benchmarks

4. Layer Model

The initial architecture is divided into seven layers.

L7  Research & Evidence
    benchmarks, conformance, experiments, results

L6  DocStorePG Control
    profiles, tenancy policy, placement, configuration

L5  Application Interfaces
    MongoDB wire protocol, PostgreSQL protocol, admin APIs

L4  Document Semantics
    BSON, document CRUD, query, aggregation, indexing

L3  PostgreSQL Data Platform
    SQL, JSONB, transactions, RLS, extensions

L2  CloudNativePG Operations
    cluster lifecycle, HA, replication, backup hooks, services

L1  Kubernetes Infrastructure
    scheduling, storage, networking, secrets, resources

A change at one layer should be measurable without requiring unrelated layers to change.

For example, replacing FerretDB with DocumentDB Gateway should not require changing the PostgreSQL cluster, tenant workload, or benchmark definitions.

5. Reference Runtime Architecture

5.1 Default Reference Deployment

flowchart LR
    MC[Mongo Client]
    SC[SQL Client]

    subgraph GW["Gateway Tier"]
        G1[DocumentDB Gateway]
    end

    subgraph K8S["Kubernetes"]
        subgraph CNPG["CloudNativePG Cluster"]
            RW[(Primary)]
            R1[(Replica)]
            R2[(Replica)]
        end
    end

    MC -->|MongoDB wire protocol| G1
    G1 -->|PostgreSQL operations| RW
    SC -->|PostgreSQL protocol| RW
    RW -->|streaming replication| R1
    RW -->|streaming replication| R2

The default research deployment should use three PostgreSQL instances when testing HA behavior and may use a single PostgreSQL instance for local functional development.

The gateway tier should remain separately deployable from PostgreSQL so that:

gateway resource consumption can be measured independently
multiple gateway implementations can be compared
gateway replicas can scale independently
gateway failures can be distinguished from database failures
protocol translation overhead can be isolated

5.2 PostgreSQL Baseline

The initial baseline should target PostgreSQL 18 or later.

This is particularly useful because CloudNativePG's ImageVolume extension mechanism uses PostgreSQL 18's extension_control_path support and permits compatible extension images to be mounted dynamically into cluster pods.

The implementation should pin exact versions in deployment profiles rather than using floating image tags.

Example conceptual lock information:

platform:
  postgresql: "18.x"
  cloudnativepg: "<pinned>"
  kubernetes: "<supported-version>"

extensions:
  documentdb:
    version: "<pinned>"
    image: "<immutable-image-reference>"

gateway:
  implementation: documentdb
  version: "<pinned>"

The exact versions belong in machine-readable deployment profiles, not permanently in this blueprint.

6. Extension Architecture

6.1 Primary Document Engine

DocumentDB is the initial primary candidate because its architecture already separates document functionality into PostgreSQL-oriented components:

pg_documentdb_core
    |
    +-- BSON storage and primitives

pg_documentdb
    |
    +-- document API
    +-- CRUD
    +-- queries
    +-- indexes
    +-- aggregation functionality

pg_documentdb_gw
    |
    +-- MongoDB wire protocol
    +-- authentication/session gateway behavior

DocStorePG should treat these as upstream capabilities.

6.2 Extension Packaging

Preferred path:

OCI extension image
       |
       v
CloudNativePG ImageVolume
       |
       v
PostgreSQL pod
       |
       v
CREATE/UPDATE EXTENSION

This should be preferred over building permanently customized PostgreSQL images where the required extension can be safely managed as an image-volume extension.

A custom PostgreSQL image remains an allowed fallback for:

unsupported extension packaging
experimental builds
debugging
compatibility tests
extension combinations that cannot yet use ImageVolumes

6.3 Extension Lifecycle

Extension lifecycle must be treated separately from application schema lifecycle.

Required states:

available
installed
configured
upgradable
upgraded
rollback-tested

The benchmark program should eventually include:

extension install time
upgrade behavior
PostgreSQL major-upgrade interaction
replica behavior
recovery behavior
incompatible-extension failure behavior

7. Gateway Architecture

7.1 Gateway Contract

A gateway accepts MongoDB-compatible client traffic and maps it to the PostgreSQL-backed document engine.

The gateway layer must be independently observable.

Required gateway metrics should eventually include:

request count
command type
response latency
upstream PostgreSQL latency
translation latency where measurable
active connections
connection establishment rate
rejected connections
authentication failures
protocol errors
unsupported commands
PostgreSQL errors
memory
CPU
network throughput

7.2 Candidate A — DocumentDB Gateway

DocumentDB Gateway is the reference path closest to the DocumentDB implementation itself.

Use it as the initial compatibility baseline.

7.3 Candidate B — FerretDB

FerretDB 2.x uses PostgreSQL with the DocumentDB extension as its backend and translates MongoDB wire-protocol operations for clients.

It should be preserved as an alternative gateway because it permits a controlled experiment:

same client
same workload
same PostgreSQL
same DocumentDB engine
different gateway

This isolates gateway behavior from storage-engine behavior.

7.4 Gateway Selection Profile

Conceptual configuration:

gateway:
  enabled: true
  implementation: documentdb
  replicas: 2

  resources:
    profile: standard

  service:
    exposure: cluster-internal

  observability:
    enabled: true

Alternative:

gateway:
  implementation: ferretdb

The actual configuration mechanism may initially be Helm/Kustomize values rather than a custom CRD.

8. Native PostgreSQL Surface

MongoDB compatibility should never be the only supported path.

DocStorePG should deliberately preserve a native SQL surface.

flowchart TD
    DOC[(Document data)]
    M[MongoDB-compatible API]
    S[SQL]
    H[Hybrid application]

    M --> DOC
    S --> DOC
    H --> M
    H --> S

This enables research into workloads such as:

SELECT
    c.customer_id,
    d.document
FROM customers c
JOIN device_documents d
  ON d.customer_id = c.customer_id
WHERE c.contract_status = 'active'
  AND document_matches_capability(d.document, 'cooling');

The precise document functions may differ, but the architectural requirement is that document data remain usable from SQL where the upstream engine supports it.

This hybrid access mode is one of the central differentiators to test.

9. Baseline Storage Profiles

DocStorePG should initially define stable comparable storage profiles.

P0 — Native PostgreSQL JSONB

PostgreSQL
+ JSONB
+ native PostgreSQL indexes
+ no Mongo gateway

Purpose:

establish the minimum PostgreSQL document baseline
measure what PostgreSQL already provides without a compatibility layer

P1 — DocumentDB / Native PostgreSQL Interface

PostgreSQL
+ DocumentDB extensions
+ direct PostgreSQL access

Purpose:

isolate DocumentDB storage and document semantics
measure document-engine overhead without Mongo wire translation

P2 — DocumentDB Gateway

Mongo client
    |
DocumentDB Gateway
    |
PostgreSQL + DocumentDB

Purpose:

primary MongoDB compatibility candidate

P3 — FerretDB Gateway

Mongo client
    |
FerretDB
    |
PostgreSQL + DocumentDB

Purpose:

alternative compatibility path
gateway comparison

M0 — Native MongoDB Reference

Mongo client
    |
MongoDB

Purpose:

external reference implementation
semantic compatibility oracle where appropriate
performance and operational comparator

These profile identifiers should remain stable so benchmark results can be compared over time.

10. Multitenancy Architecture

Multitenancy is a first-class architecture dimension.

The initial tenant-placement ladder is:

T0  POOLED
T1  NAMESPACE
T2  DATABASE
T3  DEDICATED

These identifiers should remain stable in benchmark datasets.

10.1 T0 — Pooled

Multiple tenants share document structures.

PostgreSQL database
    |
    +-- shared document collection/table
          |
          +-- tenant_id = A
          +-- tenant_id = B
          +-- tenant_id = C

Required design goal:

Tenant identity should be enforced as deeply as the selected interface permits.

For PostgreSQL-native access, this should include evaluating Row-Level Security.

Illustrative pattern:

ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation
ON documents
USING (
    tenant_id = current_setting('docstorepg.tenant_id')::uuid
);

Research questions

Can tenant identity propagate safely through a MongoDB-compatible gateway?
Does the document engine expose sufficient hooks for RLS-based enforcement?
Can a gateway accidentally bypass RLS through privileged database roles?
How can connection pooling safely preserve tenant context?
What performance cost does RLS introduce?
Can tenant-aware compound indexes prevent cross-tenant hot spots?
How do backup and restore work for a single pooled tenant?

10.2 T1 — Namespace

Tenants receive PostgreSQL schemas or another logically separate namespace.

cluster
  |
  +-- database
       |
       +-- tenant_a schema
       +-- tenant_b schema
       +-- tenant_c schema

Advantages to evaluate:

stronger logical separation than pooled rows
separate objects and indexes
tenant-specific schema customization
relatively low compute overhead compared with dedicated clusters

Risks to evaluate:

object proliferation
PostgreSQL catalog growth
migration complexity
gateway mapping complexity
uncertain fit with document-engine assumptions
backup/restore granularity

T1 must be treated as an experiment until compatibility with the selected document engine and gateway is proven.

10.3 T2 — Database

Each tenant receives a separate PostgreSQL database inside one CloudNativePG cluster.

CloudNativePG cluster
  |
  +-- tenant_a database
  +-- tenant_b database
  +-- tenant_c database

CloudNativePG can declaratively manage additional databases and their extensions/schemas.

This mode should evaluate:

per-tenant ownership
per-database extensions
migration lifecycle
database count scaling
connection overhead
catalog/resource overhead
backup/restore behavior
gateway routing
tenant-specific configuration
noisy-neighbour effects

The compute and storage failure domain remains shared.

10.4 T3 — Dedicated

Each tenant receives a dedicated CloudNativePG-backed DocStorePG deployment.

flowchart LR
    A[Tenant A] --> CA[DocStorePG Cluster A]
    B[Tenant B] --> CB[DocStorePG Cluster B]
    C[Tenant C] --> CC[DocStorePG Cluster C]

This mode provides the strongest initial isolation boundary.

Potential isolation includes:

PostgreSQL process
PostgreSQL database
gateway
CPU request/limit
memory request/limit
persistent storage
Kubernetes scheduling
secrets
service identity
network policy
backup policy
failure domain

The primary downside is fleet overhead.

T3 is expected to be most relevant to:

large enterprise tenants
regulated workloads
tenants requiring special extensions
tenants requiring independent maintenance windows
high-volume "whale" tenants
contractual isolation requirements

11. Tenant Placement as a Lifecycle

The important abstraction is not merely selecting one of T0–T3.

The longer-term architecture should explore tenant mobility:

stateDiagram-v2
    [*] --> Pooled
    Pooled --> Namespace
    Pooled --> Database
    Namespace --> Database
    Database --> Dedicated
    Pooled --> Dedicated

    Dedicated --> Database
    Database --> Pooled

Not all transitions need to be supported initially.

The strategic goal is:

Keep the application-facing document contract as stable as possible while tenant placement changes underneath it.

This is DocStorePG's initial conceptual answer to some of the problems that MongoDB solves with shard placement.

It is not initially intended to reproduce MongoDB sharding.

12. Explicit Sharding Boundary

MongoDB provides native horizontal sharding in which data can be distributed across shards and routed through a unified cluster interface.

CloudNativePG's normal architecture is a PostgreSQL primary/standby cluster. It is not by itself a transparent horizontally sharded write architecture.

Therefore:

DocStorePG V1 must not claim MongoDB-equivalent horizontal sharding.

The initial architecture deliberately stops here:

single DocStorePG deployment
    |
    +-- primary
    +-- replicas

and permits multiple independent placements:

DocStorePG A
DocStorePG B
DocStorePG C

A future research track may investigate a placement/router layer:

flowchart TD
    APP[Application]
    R[Tenant Router]

    A[DocStorePG Placement A]
    B[DocStorePG Placement B]
    C[DocStorePG Placement C]

    APP --> R
    R --> A
    R --> B
    R --> C

That work should only begin once there is evidence that it solves a problem not already better solved by:

MongoDB sharding
Citus or another PostgreSQL distribution technology
application-level partitioning
independent tenant clusters
another existing distributed database

13. Proposed DocStorePG Control Surface

DocStorePG should not immediately create a Kubernetes operator.

The first implementation should prefer:

Helm / Kustomize
+ CloudNativePG CRDs
+ standard Kubernetes resources
+ benchmark profiles
+ explicit configuration files

Only after recurring orchestration requirements emerge should a DocStorePG CRD/operator be justified.

A possible future API is documented here to establish architectural vocabulary.

13.1 Future `DocStore` Resource

apiVersion: docstorepg.io/v1alpha1
kind: DocStore
metadata:
  name: application-documents

spec:
  postgres:
    profile: ha-standard

  documentEngine:
    implementation: documentdb

  gateway:
    implementation: documentdb
    replicas: 2

  interfaces:
    mongodb: true
    postgresql: true

  tenancy:
    defaultPlacement: pooled

  observability:
    enabled: true

13.2 Future `DocStoreTenant` Resource

apiVersion: docstorepg.io/v1alpha1
kind: DocStoreTenant
metadata:
  name: acme

spec:
  storeRef: application-documents

  placement:
    mode: database

  resources:
    class: standard

  compatibility:
    profile: mongodb-default

These resources are design sketches, not implementation commitments.

14. Configuration Profiles

Configuration should be grouped into composable profiles.

profiles/
├── platform/
│   ├── local-single.yaml
│   ├── ha-small.yaml
│   └── ha-standard.yaml
├── engine/
│   ├── jsonb.yaml
│   └── documentdb.yaml
├── gateway/
│   ├── none.yaml
│   ├── documentdb.yaml
│   └── ferretdb.yaml
├── tenancy/
│   ├── pooled.yaml
│   ├── namespace.yaml
│   ├── database.yaml
│   └── dedicated.yaml
└── workload/
    ├── document-crud.yaml
    ├── aggregation.yaml
    └── hybrid.yaml

A benchmark environment can then be expressed as composition:

platform/ha-small
+ engine/documentdb
+ gateway/ferretdb
+ tenancy/pooled
+ workload/document-crud

This makes experiments easier to reproduce and diff.

15. Security Architecture

The security model should be layered.

Kubernetes identity and RBAC
        |
NetworkPolicy / service exposure
        |
gateway authentication
        |
PostgreSQL authentication
        |
PostgreSQL role / database / schema privileges
        |
Row-Level Security where applicable
        |
document/application authorization

15.1 Security Rules

Gateway database credentials should not use PostgreSQL superuser privileges in production profiles.
BYPASSRLS roles must not be used for pooled tenant traffic where RLS is part of the tenant boundary.
Tenant context must be explicit and auditable.
Connection pooling must not leak tenant context between sessions.
PostgreSQL direct access must not silently bypass the intended Mongo-facing security model.
Mongo-compatible and SQL-facing identities must be mapped deliberately.
Secrets should be provided through Kubernetes-native secret mechanisms or external secret-management integration.
External service exposure should be opt-in.
NetworkPolicy should be part of production reference profiles.
Security tests belong in the conformance suite.

16. Identity Propagation

Tenant-aware pooled operation creates an important architecture problem:

MongoDB client identity
       |
       v
Mongo gateway
       |
       v
PostgreSQL session identity
       |
       v
RLS / database policy

DocStorePG should explicitly test multiple strategies.

Strategy A — Database Role Mapping

One PostgreSQL role per tenant or tenant class.

Potential strength:

database-native identity

Potential weakness:

large role counts
connection-pool complexity

Strategy B — Session Tenant Context

A trusted gateway sets a PostgreSQL session variable such as:

SET docstorepg.tenant_id = '...';

RLS reads that context.

Potential strength:

scalable logical identity

Potential weakness:

gateway becomes security-critical
session reset must be flawless

Strategy C — Database-per-Tenant Routing

Gateway selects the tenant database.

Potential strength:

stronger logical boundary

Potential weakness:

routing and connection-pool fragmentation

No strategy should be declared canonical until tested.

17. Network Architecture

Default production stance:

Mongo endpoint: cluster-internal
PostgreSQL endpoint: cluster-internal
admin endpoints: cluster-internal
metrics endpoints: monitoring namespace / authorized collectors
external exposure: explicit profile only

Conceptual Kubernetes services:

docstore-mongo-rw
docstore-postgres-rw
docstore-postgres-ro
docstore-metrics

The exact service structure should follow the selected upstream gateway and CloudNativePG conventions.

18. High Availability

CloudNativePG should own PostgreSQL HA.

flowchart LR
    GW[Gateway replicas]
    SVC[CNPG RW Service]
    P[(Primary)]
    R1[(Replica)]
    R2[(Replica)]

    GW --> SVC
    SVC --> P
    P --> R1
    P --> R2

DocStorePG should test gateway behavior during PostgreSQL failover.

Important questions:

Does the gateway reconnect automatically?
What do clients observe?
Which operations fail?
Are retries safe?
How do transactions behave?
How quickly does service recover?
Does MongoDB client retry behavior interact correctly with PostgreSQL failover?
Does the gateway retain stale connections?

Failover testing belongs in the benchmark program.

19. Read Scaling

CloudNativePG provides read-only service patterns through replicas.

DocStorePG must not assume MongoDB-style read preferences map automatically to PostgreSQL read replicas.

Possible future mapping:

Mongo read preference
       |
       v
Gateway policy
       |
       +--> primary
       |
       +--> CNPG read-only service

This requires explicit semantic research around:

consistency
replication lag
transaction semantics
session guarantees
MongoDB driver expectations
failover

Until proven, primary routing should be the correctness baseline.

20. Backup and Recovery

Backup architecture must distinguish between:

cluster recovery
database recovery
tenant recovery
collection recovery
document recovery

CloudNativePG provides cluster-oriented PostgreSQL backup/recovery mechanisms; the DocumentDB Kubernetes Operator also has its own evolving backup abstractions.

DocStorePG should not assume that cluster-level backup automatically satisfies tenant-level recovery requirements.

Research must measure:

backup duration
backup size
restore duration
point-in-time recovery
tenant-specific restore
restoring one tenant from a pooled dataset
restoring database-per-tenant
dedicated-cluster restore
Mongo-compatible application recovery behavior

Tenant recoverability is an important factor in choosing T0–T3.

21. Observability Architecture

Observability should expose each layer independently.

flowchart LR
    C[Client]
    G[Gateway]
    P[PostgreSQL]
    K[Kubernetes]
    B[Benchmark Harness]

    C --> G --> P
    G --> M[Metrics]
    P --> M
    K --> M
    B --> M

    G --> L[Logs]
    P --> L
    K --> L

    M --> O[Observability Backend]
    L --> O

Minimum metric categories:

PostgreSQL

transactions
query latency
query count
locks
connections
cache hit behavior
WAL generation
checkpoint behavior
replication lag
table/index size
I/O
CPU
memory

Gateway

operations
command types
latency
failures
unsupported requests
active connections
authentication
CPU
memory

Kubernetes

pod restarts
scheduling
CPU requests/usage
memory requests/usage
storage
network
failovers
PVC behavior

Tenant

Where possible:

requests per tenant
storage per tenant
latency per tenant
error rate per tenant
resource share
noisy-neighbour indicators

Benchmark

Every benchmark run must capture enough environment metadata to explain the result.

22. Benchmark and Conformance Plane

The benchmark plane is part of the architecture, not an external one-off tool.

flowchart TB
    W[Workload Catalog]
    R[Runner]
    M0[MongoDB Reference]
    P0[PostgreSQL JSONB]
    P1[DocumentDB Native]
    P2[DocumentDB Gateway]
    P3[FerretDB]
    C[Comparator]
    E[(Evidence Store)]

    W --> R

    R --> M0
    R --> P0
    R --> P1
    R --> P2
    R --> P3

    M0 --> C
    P0 --> C
    P1 --> C
    P2 --> C
    P3 --> C

    C --> E

Benchmark code should support deterministic seeds where possible.

Every result should include:

run:
  id: ...
  timestamp: ...
  gitCommit: ...
  workload: ...
  seed: ...

environment:
  kubernetes: ...
  nodes: ...
  cpu: ...
  memory: ...
  storageClass: ...

software:
  mongodb: ...
  postgresql: ...
  cloudnativepg: ...
  documentdb: ...
  gateway: ...

profile:
  storage: P2
  tenancy: T0

dataset:
  tenants: ...
  documents: ...
  averageDocumentSize: ...

result:
  throughput: ...
  latencyP50: ...
  latencyP95: ...
  latencyP99: ...
  errors: ...

This structure should evolve into a formal benchmark schema.

23. Differential Conformance Testing

For compatible operations, DocStorePG should support differential testing.

request
  |
  +------> MongoDB ----------> result A
  |
  +------> DocStorePG -------> result B
                                |
                                v
                         semantic comparator

Comparison dimensions:

returned data
BSON types
null/missing behavior
ordering
update results
aggregation results
errors
error categories/codes
transaction outcome
session behavior
index behavior

Differences should be classified as:

IDENTICAL
EQUIVALENT
DOCUMENTED_DIFFERENCE
UNSUPPORTED
INCORRECT
UNKNOWN

A compatibility score must not hide important individual differences.

24. Workload Classes

The architecture should support at least these benchmark workload classes.

W0 — Basic CRUD

insert
find by ID
update
delete
bulk operations

W1 — Flexible Schema

optional fields
heterogeneous structures
schema evolution
large/small documents

W2 — Nested Documents

nested objects
arrays
array element updates
nested predicates

W3 — Indexing

single-field
compound
nested
wildcard/dynamic-field equivalents
low/high cardinality
many-index write cost

W4 — Aggregation

filtering
projection
grouping
sorting
unwind
lookup/join-like behavior
facets

W5 — Transactions

single-document
multi-document
concurrent updates
abort/retry behavior

W6 — Change/Event Workloads

change streams or equivalents
consumer lag
reconnect behavior
failover behavior

W7 — Hybrid Relational/Document

relational filters over document sets
joins between relational and document data
cross-model transactions
reporting/analytics

W8 — Multitenancy

many small tenants
mixed tenant sizes
tenant provisioning
whale tenant
noisy neighbours
placement changes
isolation tests

W9 — Operations

failover
backup
restore
upgrade
extension upgrade
scaling
disaster simulation

25. Whale-Tenant Experiment

One canonical multitenancy experiment should be designed into the architecture from the start.

Example distribution:

1000 tenants total

950 small tenants
49 medium tenants
1 whale tenant consuming roughly half of traffic

The exact distribution should be parameterized.

Questions:

What happens to small-tenant p99 latency?
Can the whale be identified operationally?
Can the whale move from T0 to T2/T3?
How long does migration take?
How much downtime occurs?
Does the application connection contract change?
What happens to backup cost?
At what point does dedicated placement become economically sensible?

This should eventually become a signature DocStorePG benchmark.

26. Tenant Migration Architecture

Tenant migration is initially an experimental subsystem.

flowchart LR
    S[Source Placement]
    E[Export / Replication]
    V[Validate]
    C[Cutover]
    T[Target Placement]
    R[Rollback Window]

    S --> E --> V --> C --> T
    C --> R

Desired migration properties:

tenant-scoped
auditable
resumable
consistency-aware
verifiable
rollback-aware
application-transparent where possible

Potential mechanisms to investigate:

logical export/import
PostgreSQL logical replication
document API replication
change streams
dual writes
maintenance-window cutover
backup/restore extraction

No mechanism is canonical yet.

27. DocumentDB Kubernetes Operator Relationship

The DocumentDB Kubernetes Operator is a relevant upstream project and should be actively compared with DocStorePG.

It already uses CloudNativePG underneath and provides Kubernetes-native DocumentDB lifecycle capabilities.

DocStorePG should therefore avoid duplicating its implementation without evidence.

Initial relationship:

DocumentDB Operator
    |
    +-- upstream reference implementation
    +-- optional deployment profile
    +-- research comparator
    +-- source of operational lessons

DocStorePG
    |
    +-- broader PostgreSQL document capability research
    +-- interchangeable gateways
    +-- native JSONB baseline
    +-- multitenancy comparison
    +-- MongoDB comparison
    +-- conformance framework
    +-- benchmark framework
    +-- hybrid relational/document research

If the DocumentDB Operator eventually provides the best runtime deployment path, DocStorePG should be able to adopt it rather than compete with it.

28. Failure Domains

DocStorePG should model failure domains explicitly.

Failure	Likely scope	Expected owner
Gateway pod	Mongo API traffic through pod	Kubernetes / gateway
Gateway deployment	Mongo-compatible surface	DocStorePG deployment
PostgreSQL instance	instance	CloudNativePG
PostgreSQL primary	writes until failover	CloudNativePG
PostgreSQL cluster	tenant set on cluster	operations
PVC/storage	instance or cluster	Kubernetes/storage
Namespace	contained deployments	Kubernetes
Kubernetes node	scheduled workloads	Kubernetes
Kubernetes cluster	all contained placements	infrastructure
Tenant credential leak	tenant/security boundary	security model
RLS policy error	pooled tenants	DocStorePG policy
Extension defect	databases using extension	upstream + operations

Dedicated tenant placement should reduce blast radius at the cost of operational density.

29. Scaling Model

DocStorePG should distinguish four fundamentally different scaling mechanisms.

Vertical

Increase:

CPU
memory
storage performance

Read Scale

Add PostgreSQL replicas where semantics permit.

Gateway Scale

Increase stateless gateway replicas.

Placement Scale

Move tenants or workloads onto additional independent DocStorePG deployments.

                       scale
                         |
       +-----------------+----------------+
       |                 |                |
    vertical          gateway          placement
                                          |
                                 +--------+--------+
                                 |                 |
                              shared           dedicated

Transparent horizontal write sharding is explicitly outside the initial model.

30. Repository Architecture

Proposed repository structure:

doc-store-pg/
├── INTENT.md
├── ArchitectureBlueprint.md
├── ResearchProgram.md
├── BenchmarkSpecification.md
├── CompatibilityModel.md
├── MultitenancyModel.md
├── WorkloadCatalog.md
├── README.md
│
├── docs/
│   ├── architecture/
│   ├── decisions/
│   ├── findings/
│   └── references/
│
├── deploy/
│   ├── cnpg/
│   ├── documentdb/
│   ├── gateways/
│   │   ├── documentdb/
│   │   └── ferretdb/
│   └── mongodb-reference/
│
├── profiles/
│   ├── platform/
│   ├── engine/
│   ├── gateway/
│   ├── tenancy/
│   └── workload/
│
├── experiments/
│   ├── compatibility/
│   ├── performance/
│   ├── multitenancy/
│   ├── operations/
│   └── migration/
│
├── benchmarks/
│   ├── runner/
│   ├── workloads/
│   ├── schemas/
│   └── analysis/
│
├── tests/
│   ├── conformance/
│   ├── security/
│   ├── failover/
│   └── integration/
│
├── results/
│   └── README.md
│
└── scripts/

Generated benchmark results should be governed carefully so the repository does not become filled with large ephemeral datasets.

Machine-readable summaries belong in Git where useful; raw large datasets may require external object storage with manifests/checksums committed to the repository.

31. Initial Implementation Stages

Stage A — Functional Baseline

Establish:

Kubernetes
+ CloudNativePG
+ PostgreSQL

Verify normal SQL operation.

Stage B — Document Engine

Add:

DocumentDB extensions

Verify document operations through PostgreSQL-native interfaces.

Stage C — Mongo-Compatible Surface

Add:

DocumentDB Gateway

Run basic official-driver CRUD tests.

Stage D — Alternative Gateway

Add:

FerretDB

Run the same tests against the same backend profile.

Stage E — MongoDB Reference

Deploy native MongoDB as M0.

Run differential CRUD/query tests.

Stage F — Tenant Baseline

Implement T0 pooled tenancy and evaluate PostgreSQL-enforced isolation.

Stage G — Isolation Ladder

Add T2 database-per-tenant and T3 dedicated placement.

Treat T1 schema-per-tenant as experimental and implement only if document-engine behavior supports it cleanly.

Stage H — Benchmark Plane

Add machine-readable benchmark runs and environment manifests.

Stage I — Operational Experiments

Test:

failover
restart
backup
restore
upgrade
noisy neighbour
tenant movement

32. Initial Architecture Decisions

The following decisions are provisional but form the starting baseline.

AD-001 — CloudNativePG is the PostgreSQL operator

Decision: Use CloudNativePG rather than introducing a custom PostgreSQL lifecycle implementation.

Reason: PostgreSQL lifecycle and HA are not DocStorePG's differentiated problem.

AD-002 — DocumentDB is the initial document engine

Decision: Use DocumentDB extensions as the first BSON/document compatibility implementation.

Reason: It already implements document semantics on PostgreSQL and exposes a MongoDB-compatible gateway architecture.

AD-003 — Gateway implementation remains swappable

Decision: Support DocumentDB Gateway first; preserve FerretDB as an alternative profile.

Reason: This allows protocol-layer effects to be isolated experimentally.

AD-004 — PostgreSQL access remains first-class

Decision: Do not hide SQL/PostgreSQL interfaces behind the document API.

Reason: Hybrid relational/document capability is a core research hypothesis.

AD-005 — Multitenancy has explicit placement levels

Decision: Use T0–T3 as stable architectural vocabulary.

Reason: Isolation, cost, and scaling cannot be meaningfully represented by a single "multi-tenant" mode.

AD-006 — No custom DocStorePG operator initially

Decision: Start with manifests/profiles and existing operators.

Reason: Operator behavior should only be built once recurring control-plane requirements are empirically understood.

AD-007 — No claim of transparent write sharding

Decision: Treat MongoDB sharding as a distinct comparison point rather than something DocStorePG already provides.

Reason: CloudNativePG HA and MongoDB sharding solve different problems.

AD-008 — Benchmarks are architecture

Decision: Reproducible benchmark and conformance hooks are part of the system design from the beginning.

Reason: Long-term architecture choices must remain evidence-driven.

33. Architecture Questions to Resolve Through Research

The blueprint intentionally leaves these open:

How complete is DocumentDB's real MongoDB semantic compatibility?
Does DocumentDB Gateway or FerretDB provide the better compatibility/performance trade-off?
How should Mongo identities map to PostgreSQL identities?
Can PostgreSQL RLS reliably enforce pooled tenant isolation through the Mongo-compatible surface?
What tenant count is practical for database-per-tenant within one CNPG cluster?
Is schema-per-tenant useful with DocumentDB or merely theoretical?
How much gateway overhead exists relative to direct document access?
How much document-engine overhead exists relative to native JSONB?
Where does native MongoDB outperform the PostgreSQL-backed architecture?
How do both architectures behave with a whale tenant?
What is the operational cost per tenant under T0, T2, and T3?
How should tenant migration work?
Can a stable logical endpoint survive tenant placement changes?
Which MongoDB read/write/session semantics can safely map to CNPG primary/replica topology?
What backup granularity is achievable for each tenant model?
How should extension versions be upgraded independently from PostgreSQL?
When does an existing distributed PostgreSQL technology become preferable to DocStorePG placement routing?
When should a workload simply remain on native MongoDB?

These questions should seed ResearchProgram.md.

34. Target Architecture

The long-term target, if supported by evidence, is:

flowchart TB
    subgraph Applications
        M[MongoDB clients]
        S[SQL clients]
    end

    EP[Stable DocStorePG Endpoint]
    TC[Tenant / Placement Control]

    subgraph Shared["Shared Placement"]
        SP[CNPG + DocumentDB]
    end

    subgraph DB["Database-Isolated Placement"]
        DP[CNPG + Tenant Databases]
    end

    subgraph Dedicated["Dedicated Placements"]
        D1[CNPG Tenant A]
        D2[CNPG Tenant B]
    end

    OBS[Observability]
    BENCH[Conformance & Benchmark Plane]

    M --> EP
    S --> EP

    EP --> TC
    TC --> SP
    TC --> DP
    TC --> D1
    TC --> D2

    SP --> OBS
    DP --> OBS
    D1 --> OBS
    D2 --> OBS

    BENCH --> EP
    BENCH --> OBS

The important feature is not that all boxes necessarily become custom DocStorePG software.

The important feature is that the platform can express and test:

interface
+ document semantics
+ tenant identity
+ placement
+ isolation
+ PostgreSQL capability
+ operations
+ evidence

as separable concerns.

35. Architectural North Star

doc-store-pg should aim to make this statement testable:

A document store can be an operationally native PostgreSQL capability whose API, compatibility, tenant placement, and isolation level can evolve independently from the applications that use it.

The architecture succeeds if it allows the project to discover where that statement is true, where it is false, and where MongoDB or another architecture remains the better engineering choice.

The purpose is not to prove PostgreSQL wins.

The purpose is to create a system in which the trade-offs become measurable.

36. References and Upstream Baseline

The following upstream sources establish the initial technical baseline for this blueprint. Versions and capabilities should be revalidated when experiments are run.

CloudNativePG — Image Volume Extensions
https://cloudnative-pg.io/documentation/current/imagevolume_extensions/
CloudNativePG — Operator Capability Levels
https://cloudnative-pg.io/docs/current/operator_capability_levels/
CloudNativePG — Declarative Database Management
https://cloudnative-pg.io/docs/devel/declarative_database_management/
CloudNativePG 1.30 Release
https://cloudnative-pg.io/releases/cloudnative-pg-1-30.0-released/
DocumentDB — Getting Started / Architecture Components
https://documentdb.io/docs/getting-started
DocumentDB — PostgreSQL API / Gateway
https://documentdb.io/docs/postgres-api
DocumentDB Kubernetes Operator
https://documentdb.io/kubernetes-operator
DocumentDB Kubernetes Operator — API Reference
https://documentdb.io/documentdb-kubernetes-operator/latest/preview/api-reference/
FerretDB — Introduction
https://docs.ferretdb.io/
FerretDB — Migration to v2 / DocumentDB Backend
https://docs.ferretdb.io/migration/migrating-from-v1/
PostgreSQL — Row Security Policies
https://www.postgresql.org/docs/current/ddl-rowsecurity.html
MongoDB — Multi-Tenant Architecture
https://www.mongodb.com/docs/atlas/build-multi-tenant-arch/
MongoDB — Sharding
https://www.mongodb.com/docs/manual/sharding/
MongoDB — Scaling Strategies / Collection Placement
https://www.mongodb.com/docs/manual/core/sharding-scaling-strategies/
MongoDB — Manage Unsharded Collections
https://www.mongodb.com/docs/manual/core/sharding-manage-unsharded-collections/

44 KiB Executable file Raw Permalink Blame History Unescape Escape

ArchitectureBlueprint.md

Repository

Status

1. Architectural Intent

2. Architecture Principles

2.1 PostgreSQL Remains Visible

2.2 Composition Before Reimplementation

2.3 Gateways Are Replaceable

2.4 Tenant Placement Is Explicit

2.5 Compatibility Is Measured

2.6 Evidence Before Optimization

2.7 Upstream Escape Hatches Remain Available

3. System Context

4. Layer Model

5. Reference Runtime Architecture

5.1 Default Reference Deployment

5.2 PostgreSQL Baseline

6. Extension Architecture

6.1 Primary Document Engine

6.2 Extension Packaging

6.3 Extension Lifecycle

7. Gateway Architecture

7.1 Gateway Contract

7.2 Candidate A — DocumentDB Gateway

7.3 Candidate B — FerretDB

7.4 Gateway Selection Profile

8. Native PostgreSQL Surface

9. Baseline Storage Profiles

P0 — Native PostgreSQL JSONB

P1 — DocumentDB / Native PostgreSQL Interface

P2 — DocumentDB Gateway

P3 — FerretDB Gateway

M0 — Native MongoDB Reference

10. Multitenancy Architecture

10.1 T0 — Pooled

Research questions

10.2 T1 — Namespace

10.3 T2 — Database

10.4 T3 — Dedicated

11. Tenant Placement as a Lifecycle

12. Explicit Sharding Boundary

13. Proposed DocStorePG Control Surface

13.1 Future DocStore Resource

13.2 Future DocStoreTenant Resource

14. Configuration Profiles

15. Security Architecture

15.1 Security Rules

16. Identity Propagation

Strategy A — Database Role Mapping

Strategy B — Session Tenant Context

Strategy C — Database-per-Tenant Routing

17. Network Architecture

18. High Availability

19. Read Scaling

20. Backup and Recovery

21. Observability Architecture

PostgreSQL

Gateway

Kubernetes

Tenant

Benchmark

22. Benchmark and Conformance Plane

23. Differential Conformance Testing

24. Workload Classes

W0 — Basic CRUD

W1 — Flexible Schema

W2 — Nested Documents

W3 — Indexing

W4 — Aggregation

W5 — Transactions

W6 — Change/Event Workloads

W7 — Hybrid Relational/Document

W8 — Multitenancy

W9 — Operations

25. Whale-Tenant Experiment

26. Tenant Migration Architecture

27. DocumentDB Kubernetes Operator Relationship

28. Failure Domains

29. Scaling Model

44 KiB

Executable file

Raw Permalink Blame History

13.1 Future `DocStore` Resource

13.2 Future `DocStoreTenant` Resource