resource-control/workplans/RESOURCE-WP-0002-procure-postgres-backup-storage.md

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
RESOURCE-WP-0002	workplan	Procure and operationalize PostgreSQL backup object storage	financials	resource-control	active	codex	railiance	2026-08-10	2026-08-10	921496a3-280b-4dc8-a3c0-b4ec314142f5

RESOURCE-WP-0002 — procure PostgreSQL backup storage

Goal

Provide an off-host object-storage resource for rapp-postgres continuous WAL archiving and physical base backups, and prove that it supports both full recovery and point-in-time recovery (PITR). Record the resource, its real cost, utilization, failure domain, contract, and exit path so fin-hub can incorporate it into burn rate and resource-control can continuously test whether another provider or procurement model is more attractive.

The candidate provider baseline is:

• A — Host Europe
• B — Scaleway
• C — Hetzner

This workplan chooses on evidence. It does not presume that keeping compute and backup at one provider is cheaper or safer.

Context and current evidence

rapp-postgres has deployed platform-pg on reef-railiance at Host Europe. Its workplan requires continuous WAL archiving, scheduled physical backups, a 30-day recovery window, and timed full/PITR restore drills.

Host Europe's Backup Storage is reachable by FTP, SCP, SFTP, and rsync. It is suitable for an encrypted secondary logical-backup copy but is not directly usable by CloudNativePG's Barman object-store integration. Host Europe also publishes an older Cloud Storage specification describing an S3-compatible service; current orderability, endpoint, features, support, and price must be confirmed rather than inferred from that document.

Scaleway offers S3-compatible Object Storage in European regions, including a Multi-AZ tier with usage-based storage pricing. Hetzner offers S3-compatible Object Storage in Nuremberg, Falkenstein, and Helsinki with a monthly base price including 1 TB storage and egress. Exact effective prices are recorded at decision time, not copied permanently from changing marketing pages.

Primary references:

• Host Europe Backup Storage: https://www.hosteurope.de/faq/server/virtual-server/backup-storage
• Host Europe published Cloud Storage specification: https://www.hosteurope.de/download/PVL/Preis_und_Leistungsverzeichnis_CloudStorage_1-0-1.pdf
• Scaleway storage pricing: https://www.scaleway.com/en/pricing/storage/
• Hetzner Object Storage: https://www.hetzner.com/de/storage/object-storage/
• CNPG 1.25 object-store requirements: https://cloudnative-pg.io/docs/1.25/appendixes/object_stores/

Boundaries

This workplan owns requirements, comparison, procurement evidence, resource registration, cost/utilization monitoring, and proof that the purchased resource works.

• Human financial authority approves purchase and accepts contractual terms.
• railiance-platform owns provider credential custody and the approved Kubernetes Secret/credential-broker delivery.
• rapp-postgres owns CNPG manifests, schedules, retention configuration, and restore procedure.
• fin-hub owns budget, burn-rate, runway, and viability signals.

No access key, secret, payment instrument, or credential-bearing endpoint may be committed to this repository or sent through State Hub.

Acceptance requirements

The selected resource must provide:

• S3-compatible HTTPS endpoint accepted by the live CNPG/Barman version
• AWS Signature V4 and scoped access keys
• private buckets, multipart upload, list/get/put/delete operations
• versioning and lifecycle/retention controls, or a documented equivalent
• encryption in transit and at rest
• EU data location with provider and region recorded
• failure-domain analysis relative to reef-railiance
• at least 100 GiB initial usable capacity without a long commitment
• daily physical base backups and continuous WAL archive support
• target RPO no worse than five minutes and initially measured RTO
• export/delete path and credential revocation at exit
• monthly price and usage evidence attributable to platform:audit-storage

Object lock is desirable, but compatibility with CNPG retention deletion must be tested before it becomes mandatory. Same-provider placement is allowed only with an explicit correlated-failure decision and an independent second copy.

T01 — Establish demand and cost model

id: RESOURCE-WP-0002-T01
status: progress
priority: high
state_hub_task_id: "f578a9ec-dbdb-4b26-93bc-e53f7bc87ad1"

Measure the current platform-pg database size, daily WAL generation, expected growth, restore-test traffic, and retention requirement. Build low/base/high 12-month forecasts. Normalize each provider into monthly storage, ingress, egress, operations, minimum charge, tax, support, commitment, and estimated operator-labor cost.

Include the cost of one monthly restore drill and one complete provider exit. Do not compare only advertised price per GB.

Done when the demand forecast and cost model can calculate effective monthly cost for A, B, and C under the same scenarios.

Progress 2026-08-10: added a timestamped live database/WAL observation, low/base/high 12-month inputs, and a tested fail-closed calculator in data/, tools/cost_model.py, and docs/evidence/RESOURCE-WP-0002-demand-and-cost-model-2026-08-10.md. Scaleway and in-quota Hetzner costs calculate. Host Europe remains deliberately null until a current account quote exists; Hetzner excess pricing is also required before its high scenario and full exit can calculate.

Expanded 2026-08-10: added 1/2/3-node self-managed Garage estimates for Host Europe and Hetzner plus managed AWS S3, Azure Blob, Google Cloud Storage, and STACKIT Object Storage price points. Fixed-capacity VM options fail closed when forecast demand exceeds usable replicated capacity. Currency conversion, request charges, setup labor, recurring operations, and topology limitations are explicit in docs/evidence/RESOURCE-WP-0002-expanded-storage-comparison-2026-08-10.md.

Refined 2026-08-10: every quote now separates monthly infrastructure from internal operations labor and internal setup from external services. Added a normalized 320 GB running-cost view so bundled, fixed-capacity, and elastic products compare on the same stored volume and request/restore pattern.

Control refinement 2026-08-10: added an immutable 12-month forecast record, monthly actual-observation schema, and variance calculator covering stored bytes, database size, WAL, requests, restore egress, invoice cost, and internal labor. docs/forecast-actual-control.md defines error thresholds, evidence, revision discipline, and the three-month recalibration loop.

T02 — Complete provider due diligence

id: RESOURCE-WP-0002-T02
status: progress
priority: high
state_hub_task_id: "6dedb8db-c08f-4b58-90f2-32b1ea832d98"

For Host Europe, obtain current written confirmation of whether its S3-compatible Cloud Storage can be ordered for this account, plus endpoint, region, Signature V4, versioning, lifecycle, durability, availability, support, current price, and cancellation terms. Treat SFTP Backup Storage as a separate secondary-copy product.

For Scaleway and Hetzner, collect the same facts from current primary sources. Record data location, durability/availability statements, minimum billing, traffic rules, API limitations, credential granularity, versioning, lifecycle, object-lock behavior, and provider status visibility.

Done when every acceptance requirement has evidence, unknown, or a blocking answer for each provider—no blank cells and no marketing inference.

Progress 2026-08-10: completed the public-primary-source evidence matrix in docs/evidence/RESOURCE-WP-0002-provider-due-diligence-2026-08-10.md and added a provisional Scaleway inventory record. Host Europe current S3 orderability and commercial terms require written account-support evidence. Scaleway is the provisional primary because it documents Multi-AZ service, durability, S3 features, and managed at-rest encryption; selection still requires a live Barman preflight, contract review, and human approval.

T03 — Select and procure the primary resource

id: RESOURCE-WP-0002-T03
status: wait
priority: high
state_hub_task_id: "e4184350-dab2-4a0b-bee5-1a641e8a2df3"

Produce a decision record ranking the three providers on total cost, compatibility, resilience, sovereignty, operational effort, and exit cost. Recommend one primary and one independent secondary-copy path.

Human financial authority approves the purchase and contractual terms. Create the account/project/bucket with public access disabled, the narrowest practical service identity, versioning/lifecycle settings, cost alerts, and an owner.

Done when the decision is approved and the purchased resource has a non-secret inventory record with provider resource ID, region, service class, contract, renewal/cancellation dates, capacity model, owner, and cost-attribution key.

T04 — Establish credential custody and hand off to rapp-postgres

id: RESOURCE-WP-0002-T04
status: wait
priority: high
state_hub_task_id: "a2dc370a-b5e7-44b1-b46a-f3b84815b14a"

Route the scoped bucket credential to railiance-platform through the approved OpenBao/credential-broker lane. Permit only the bucket/prefix operations CNPG requires. Keep credential values out of Git, State Hub, shell history, logs, and chat.

Hand the non-secret endpoint, bucket/prefix, region, CA/TLS requirements, and retention parameters to rapp-postgres. Verify Barman's empty-WAL-archive check before enabling continuous archiving.

Done when credential delivery is renewable/revocable, a negative credential cannot access the bucket, and rapp-postgres can perform the storage preflight without disclosing a value.

T05 — Prove backup, full restore, and PITR

id: RESOURCE-WP-0002-T05
status: wait
priority: high
state_hub_task_id: "693953bf-f223-484b-b835-41369f5d2a02"

Enable continuous WAL archiving and daily base backups. Confirm archive health, backup age, object growth, retention behavior, and alerts. Then:

1. Restore the latest physical backup to a scratch CNPG cluster and compare per-database counts/checksums.
2. Create a restore point, write a marker, record a target time, perform a destructive change, and restore to immediately before that change.
3. Record measured RPO, full-restore RTO, PITR RTO, bytes transferred, and provider charges.
4. Delete scratch resources only after evidence is durable.

Done when both restores are verified and timed. A successful upload alone is not backup acceptance.

T06 — Establish the independent logical-copy lane

id: RESOURCE-WP-0002-T06
status: todo
priority: medium
state_hub_task_id: "76745e5b-a0eb-43b5-a6b7-9d19c06f6110"

Retain encrypted per-database logical dumps outside the primary object-storage failure domain, using Host Europe Backup Storage or the existing governed Nextcloud lane as justified by the final provider choice. Test a single-consumer logical restore.

Done when one primary-object-store failure or credential loss does not remove both recovery copies, and audit-core can be restored independently through the slower logical path.

T07 — Feed fin-hub and operate the optimization loop

id: RESOURCE-WP-0002-T07
status: todo
priority: medium
state_hub_task_id: "e6c848ce-ea4b-4ac6-99da-5890de242b50"

Publish monthly actual cost, forecast, stored bytes, WAL/base-backup growth, egress, backup success, and restore-test cost under platform:audit-storage. Set thresholds for budget variance, abnormal growth, stale backups, and unused commitment.

Re-evaluate A/B/C at least quarterly and before renewal. A switch recommendation must include migration traffic, overlap period, operator labor, restore proof, contract timing, and credential revocation—not only a cheaper unit price.

Done when fin-hub receives normalized evidence and resource-control can identify rightsizing or provider-switch opportunities from observed utilization and total cost.