railiance-platform/docs/backup-storage-tiers.md

Backup storage tiers and capacity — 2026-09-06

Design from the user's request to keep Nextcloud essentials-only. This records capacity and implementation acceptance; it does not change live exclusions, retention or delete existing good backups.

Measured capacity

The verified encrypted Forgejo archive is 5,353,024,293 bytes (5.35 GB / 4.99 GiB). The documented 14 daily + 4 weekly policy requires 96.35 GB / 89.74 GiB for these 18 archives alone, excluding separate SQL dumps, new-upload overlap, object versions, native PostgreSQL base backups/WAL and all other platform services. Allow roughly 125–150 GB for this Forgejo full-archive policy with upload overlap and growth. This is a planning allowance, not measured whole-platform demand or a provider allocation request. Measure native backup inventory and WAL growth before setting the platform-wide budget.

Live filesystem allocation (not compressed archive contribution):

Content	Size	Secondary treatment
Git repositories and history	480 MiB	Keep all repositories, including private repositories and wikis
Package blobs	4.45 GiB	Full copy on primary; independently protect selected essential, non-rebuildable/bootstrap artifacts
Last SQL dump	93 MiB before compression	Keep database metadata; one consistent logical export per generation
Generated repository archives	339 MiB	Regenerate; exclude
Actions logs	45 MiB	Exclude unless a separate audit retention requirement applies
Attachments, avatars and configuration	Below 1 MiB currently	Keep; encrypt configuration and preserve custody boundaries
LFS / Actions artifacts	No LFS directory observed; Actions artifacts currently negligible	Keep unique LFS and irreplaceable artifacts when present; absence is not permission to discard future data

Evidence: evidence/backup-capacity-assessment-2026-09-06.json. Package rebuildability has NOT been audited. Storage category alone does not make an artifact disposable. The live installed Forgejo dump CLI supports selective exclusions; see the official command documentation.

Intended division

Scaleway: authoritative full application archives plus native database backups and WAL. Preserve repository/package/attachment/LFS coverage and prove retrieval and combined application recovery under WP-0038-T04 before reducing the secondary archive scope. Different archive and Barman prefixes remain required.

Nextcloud: an independently recoverable essentials copy. Retain source/history, issues/PRs/users/settings in the database, unique attachments/LFS/custom files, and an encrypted recovery manifest. Retain necessary deployment definitions, small essential platform logical exports and protected OpenBao snapshot material under their owning contracts. Keep the decryption/unseal custody independently available; do not put the only recovery key inside its own encrypted backup.

Exclude regenerable indexes, archive caches, temporary files and unneeded build logs. Bulk package/OCI content remains on the full primary. Selectively retain or independently mirror exact bootstrap images and irreplaceable release assets; prove their availability without the production Forgejo registry or Scaleway. The manifest must state omissions, artifact digests and rebuild/restore steps. A secondary-only restore may recover source and collaboration while package service remains unavailable; test and document that degraded behavior explicitly.

Candidate 10 GiB Nextcloud budget

Target at most 600 MiB per compressed encrypted Forgejo essentials generation. Repositories plus SQL currently total about 573 MiB before compression, but this is an estimate: a real essentials capture and restore must establish actual size.

• Seven daily + two weekly distinct recovery points: at most 5.27 GiB.
• One incoming generation while the previous good copy remains: 0.59 GiB.
• Other service essentials and selective bootstrap material: 2 GiB provisional.
• Headroom for growth and provider accounting: 2 GiB.

Total: approximately 9.86 GiB. The other-service/bootstrap allocation is unmeasured; if it does not fit, shorten retention or adjust the allocation explicitly. Do not silently exclude unique data to meet the target. A weekly label may reference an existing retained generation instead of uploading Sunday's bytes twice.

The current uploader is create-only. Retention requires a separate bounded owner executor; do not give workload jobs the Backup account password or general delete access. Verify the new object and recovery manifest before retiring old points. Use exact managed names, concurrency control and conditional deletion; account for trash/version retention and recheck actual free bytes. Stop before upload when overlap/headroom cannot be met. Never delete the last verified recovery point.

Implementation acceptance in WP-0038-T04

1. Complete primary full-archive delivery and recovery with explicit storage and credential-delivery ownership.
2. Audit package/bootstrap dependencies and measure other service essentials.
3. Capture an explicit essentials profile and inventory, preserving unique data; retain the full backup path and record included/excluded categories.
4. Restore with primary storage unavailable; verify Git, issues/users, unique attachments and declared package limitations/bootstrap availability.
5. Implement the separate bounded retention executor and space preflight. Demonstrate upload overlap, failed-upload preservation and safe expiration within 10 GiB.
6. Switch scheduled secondary delivery only after these gates pass. Preserve the current good full Nextcloud copy until a verified replacement permits retirement.

Verified implementation — 2026-09-06

The actual encrypted essentials artifact is 286,298,520 bytes (273 MiB), versus 5,353,024,293 bytes for the tested full archive: about 94.7% smaller. Nine distinct copies at this measured size need about 2.40 GiB, before other services, incoming overlap, growth and provider accounting. Keep the 600 MiB admission ceiling; current size is not a perpetual promise.

The full archive was multipart-uploaded to the separate existing-policy prefix platform-pg/application-archives/forgejo/, retrieved by its returned object version, hash-checked, decrypted and restored successfully. Forgejo startup, two Git clones/fsck and all 2,040 package blob hashes passed. This is a verified primary archive and recovery; recurring full-archive scheduling is still pending. The existing bucket contract is versioned with 30-day current/noncurrent lifecycle; a separate calendar archive-expiration policy has not been activated.

The smaller candidate was uploaded/downloaded through Backup, decrypted and restored without primary access. Forgejo started with 142 repositories, six users and 2,050 package metadata records; the package service was explicitly disabled. Two Git clones/fsck passed. The newer essentials capture and older full archive are different recovery points; they are not claimed to be an atomic matching pair. Both disposable recovery environments were removed. The public pinned Forgejo and PostgreSQL recovery images were successfully pulled independently of both Scaleway and the production forge. Other five database clusters total about 209 MiB of physical data; compressed export budgets remain to be measured.

Implementation:

• scripts/capture_forgejo_archive.py --profile essentials uses native exclusions, seals the explicit manifest and enforces the size ceiling. Default remains full.
• scripts/forgejo_essentials_profile.py also removes generated Actions logs while preserving unique attachments/LFS/Actions artifacts and rejects unsafe paths.
• scripts/scaleway_forgejo_archive.py performs bounded multipart PUT, aborts failed in-flight multipart uploads, then downloads and verifies bytes. Run with uv run --no-project --with boto3==1.40.18 python ...; credential values stay in captured memory from the existing governed databases Secret.
• scripts/backup_retention_plan.py selects seven daily/two weekly points without duplicate uploads; protects the explicitly pinned recovered archive and newest point, ignores unknown/unverified objects and requires 2 GiB upload headroom.
• scripts/execute_nextcloud_retention.py previews by default. With --apply, the separate attended owner executor revalidates protected ciphertext, retained object presence and candidate sizes/ETags before conditional deletion. It uses a per-user local lock; deploy only one authoritative owner executor. It never changes workload credentials or deletes full archives. Unknown drill filenames are intentionally ineligible. No live expiration was executed in this review.

Before scheduled cutover, bind the durable primary caller and dependencies, produce canonical forgejo-essentials-YYYYMMDDTHHMMSSZ.zip.age names and verified inventory, and bind the separate owner retention execution/fresh quota check. The current local forgejo-backup cron remains unchanged. The activity-core worker has vendored age/kubectl, which its CNPG resolver adds to PATH; absence from its default PATH did not establish missing tools. Neither that mount nor an attended workstation proof establishes a scheduled primary archive job.