doc-store-pg/INTENT.md

# INTENT.md

## Repository

`doc-store-pg`

## Intent

`doc-store-pg` explores, integrates, and validates a CloudNativePG-native document-store capability built on PostgreSQL.

The project investigates whether the capabilities that make MongoDB attractive as a document database can be provided as a composable PostgreSQL capability without giving up PostgreSQL's relational strengths, operational model, extensibility, and flexible multitenancy boundaries.

The repository is intended to move this question from architectural theory into an inspectable, reproducible implementation and research program.

## Core Hypothesis

Document-oriented storage does not necessarily require a separate database architecture.

A PostgreSQL platform, extended with document-oriented data types, query semantics, indexing, MongoDB-compatible interfaces, and suitable operational tooling, may provide a competitive document-store surface while retaining the advantages of PostgreSQL and CloudNativePG.

`doc-store-pg` exists to test that hypothesis rather than assume it.

## Objectives

The project aims to:

1. Establish a reproducible CloudNativePG-based document-store reference architecture.
2. Integrate suitable PostgreSQL document-store technologies such as DocumentDB and compatible gateway implementations.
3. Expose MongoDB-compatible APIs where useful while retaining direct PostgreSQL access.
4. Measure semantic compatibility with MongoDB rather than treating wire compatibility as sufficient.
5. Compare document-oriented workloads across native MongoDB and PostgreSQL-based implementations.
6. Investigate hybrid relational/document workloads that make use of PostgreSQL-specific strengths.
7. Explore multitenancy models ranging from highly pooled tenants to strongly isolated tenant deployments.
8. Measure operational characteristics including deployment, failover, backup, restore, upgrade, scaling, observability, and resource efficiency.
9. Identify which capabilities belong in reusable PostgreSQL extensions, gateway layers, CloudNativePG configuration, Kubernetes control-plane resources, or higher-level orchestration.
10. Produce evidence that can guide long-term architectural and product decisions.

## Scope

The repository may contain or coordinate work relating to:

- CloudNativePG deployment profiles
- PostgreSQL extension packaging and lifecycle management
- DocumentDB integration
- MongoDB-compatible gateways
- FerretDB integration where useful
- BSON and JSONB storage models
- document query and aggregation semantics
- indexing strategies
- MongoDB driver and application compatibility
- transaction and change-stream behavior
- PostgreSQL-native access to document data
- relational/document hybrid queries
- schema validation
- observability and query diagnostics
- benchmark harnesses
- conformance tests
- workload generators
- operational tests
- multitenancy models
- tenant placement and isolation policies
- tenant migration experiments
- performance and cost analysis
- architecture research and decision records

## Multitenancy Research

Multitenancy is a first-class concern of the project.

`doc-store-pg` should investigate a continuum of tenant isolation models, including:

### Pooled

Multiple tenants share document structures and storage, with tenant boundaries enforced through mechanisms such as tenant identifiers, PostgreSQL roles, Row-Level Security, gateway policy, and application identity propagation.

### Namespace

Tenants receive distinct PostgreSQL schemas or comparable logical namespaces while sharing a PostgreSQL cluster.

### Database

Tenants receive distinct PostgreSQL databases within a shared CloudNativePG cluster.

### Dedicated

Tenants receive dedicated CloudNativePG / document-store clusters and corresponding compute, storage, security, and failure boundaries.

The project should evaluate whether tenants can move between these models as their requirements evolve without requiring applications to adopt different document APIs.

This creates a central research question:

> Can tenant placement and isolation become configurable properties of the data platform rather than permanent application architecture decisions?

## Compatibility Model

MongoDB compatibility must be treated as a spectrum rather than a binary property.

The project should distinguish at least:

- connection compatibility
- CRUD compatibility
- query compatibility
- index compatibility
- aggregation compatibility
- driver compatibility
- tooling compatibility
- transaction compatibility
- change-stream compatibility
- behavioral and error-semantics compatibility
- real-application compatibility

Compatibility claims should be backed by reproducible tests.

## Benchmarking Intent

Benchmarking is a core product of the repository rather than an afterthought.

The benchmark program should compare meaningful architectural configurations, potentially including:

- native MongoDB
- PostgreSQL with JSONB
- PostgreSQL with document-store extensions
- PostgreSQL document APIs used directly
- MongoDB-compatible DocumentDB gateways
- FerretDB or other compatible gateways
- different CloudNativePG deployment profiles
- different multitenancy and isolation models

Benchmarks should measure more than throughput.

Relevant dimensions include:

- latency
- throughput
- concurrency
- storage efficiency
- index size
- write amplification
- CPU efficiency
- memory efficiency
- connection overhead
- noisy-neighbour behavior
- tenant isolation
- failover behavior
- backup and restore
- recovery objectives
- upgrade complexity
- provisioning time
- tenant migration
- observability
- operational complexity
- cost per workload
- cost per tenant

Benchmark results should remain reproducible, inspectable, and comparable over time.

## Architectural Principle

The project should prefer composition over reimplementation.

Where mature open-source components already provide BSON handling, MongoDB protocol translation, query semantics, or other document-store functionality, `doc-store-pg` should integrate and evaluate them before implementing replacements.

The repository's durable value should primarily come from:

- architecture
- integration
- configuration
- compatibility profiles
- operational patterns
- multitenancy models
- conformance testing
- benchmark methodology
- comparative evidence
- automation
- reusable deployment abstractions

rather than unnecessary duplication of upstream database functionality.

## Non-Goals

`doc-store-pg` does not initially aim to:

- implement a MongoDB-compatible database engine from scratch
- reproduce every MongoDB feature
- claim drop-in compatibility without evidence
- replace CloudNativePG
- create a new PostgreSQL distribution
- build a distributed sharding system before simpler architectures have been evaluated
- optimize exclusively for synthetic benchmark scores
- assume PostgreSQL is superior to MongoDB
- assume MongoDB compatibility is the only valuable document-store interface

The repository should remain evidence-driven.

## Success Criteria

The project is successful when it can provide clear, reproducible answers to questions such as:

1. Which MongoDB workloads can run unchanged or with minimal adaptation?
2. What performance overhead is introduced by document abstractions and MongoDB protocol gateways?
3. Where does native MongoDB retain architectural or operational advantages?
4. Where does PostgreSQL provide equivalent or better behavior?
5. Which workloads benefit from combining relational and document models?
6. How do different multitenancy strategies affect density, isolation, performance, security, and cost?
7. How effectively can tenants move between pooled and dedicated deployment models?
8. Which configuration produces the best trade-off for a given workload?
9. Can a CloudNativePG platform offer document storage without introducing a second independent database operational stack?
10. Under which conditions should an operator deliberately choose MongoDB instead?

## Expected Repository Outputs

The repository is expected to evolve through artifacts such as:

- `ArchitectureBlueprint.md`
- `ResearchProgram.md`
- `BenchmarkSpecification.md`
- `CompatibilityModel.md`
- `MultitenancyModel.md`
- `WorkloadCatalog.md`
- `BenchmarkResults/`
- `experiments/`
- `deploy/`
- `profiles/`
- `tests/`
- `docs/`
- architecture decision records
- reproducible benchmark environments
- machine-readable benchmark results

## Guiding Principle

> Treat document storage as a capability to be composed, measured, and governed — not as a database category to be assumed.

`doc-store-pg` should make the trade-offs between PostgreSQL-based document storage and native document databases visible through working systems and reproducible evidence.