From 69187c2f4503fb648dd72be0cbd6b0ada237410b Mon Sep 17 00:00:00 2001 From: codex Date: Sun, 6 Sep 2026 01:41:43 +0200 Subject: [PATCH] Size backup retention and design Nextcloud essentials tier Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a06ecb-456a-71c2-b41e-0755d336e883 --- docs/backup-storage-tiers.md | 92 +++++++++++++++++++ ...backup-capacity-assessment-2026-09-06.json | 24 +++++ ...ckup-capacity-and-essentials-assessment.md | 22 +++++ ...-0038-forgejo-scaleway-primary-coverage.md | 7 ++ 4 files changed, 145 insertions(+) create mode 100644 docs/backup-storage-tiers.md create mode 100644 docs/evidence/backup-capacity-assessment-2026-09-06.json create mode 100644 history/2026-09-06-backup-capacity-and-essentials-assessment.md diff --git a/docs/backup-storage-tiers.md b/docs/backup-storage-tiers.md new file mode 100644 index 0000000..0432882 --- /dev/null +++ b/docs/backup-storage-tiers.md @@ -0,0 +1,92 @@ +# Backup storage tiers and capacity — 2026-09-06 + +Design from the user's request to keep Nextcloud essentials-only. This records +capacity and implementation acceptance; it does not change live exclusions, +retention or delete existing good backups. + +## Measured capacity + +The verified encrypted Forgejo archive is 5,353,024,293 bytes (5.35 GB / 4.99 GiB). +The documented 14 daily + 4 weekly policy requires 96.35 GB / 89.74 GiB for these +18 archives alone, excluding separate SQL dumps, new-upload overlap, object +versions, native PostgreSQL base backups/WAL and all other platform services. +Allow roughly 125–150 GB for this Forgejo full-archive policy with upload overlap +and growth. This is a planning allowance, not measured whole-platform demand or +a provider allocation request. Measure native backup inventory and WAL growth +before setting the platform-wide budget. + +Live filesystem allocation (not compressed archive contribution): + +| Content | Size | Secondary treatment | +| --- | ---: | --- | +| Git repositories and history | 480 MiB | Keep all repositories, including private repositories and wikis | +| Package blobs | 4.45 GiB | Full copy on primary; independently protect selected essential, non-rebuildable/bootstrap artifacts | +| Last SQL dump | 93 MiB before compression | Keep database metadata; one consistent logical export per generation | +| Generated repository archives | 339 MiB | Regenerate; exclude | +| Actions logs | 45 MiB | Exclude unless a separate audit retention requirement applies | +| Attachments, avatars and configuration | Below 1 MiB currently | Keep; encrypt configuration and preserve custody boundaries | +| LFS / Actions artifacts | No LFS directory observed; Actions artifacts currently negligible | Keep unique LFS and irreplaceable artifacts when present; absence is not permission to discard future data | + +Evidence: `evidence/backup-capacity-assessment-2026-09-06.json`. +Package rebuildability has NOT been audited. Storage category alone does not +make an artifact disposable. The live installed Forgejo dump CLI supports +selective exclusions; see the [official command documentation](https://forgejo.org/docs/latest/admin/command-line/). + +## Intended division + +Scaleway: authoritative full application archives plus native database backups +and WAL. Preserve repository/package/attachment/LFS coverage and prove retrieval +and combined application recovery under WP-0038-T04 before reducing the secondary +archive scope. Different archive and Barman prefixes remain required. + +Nextcloud: an independently recoverable essentials copy. Retain source/history, +issues/PRs/users/settings in the database, unique attachments/LFS/custom files, +and an encrypted recovery manifest. Retain necessary deployment definitions, +small essential platform logical exports and protected OpenBao snapshot material +under their owning contracts. Keep the decryption/unseal custody independently +available; do not put the only recovery key inside its own encrypted backup. + +Exclude regenerable indexes, archive caches, temporary files and unneeded build +logs. Bulk package/OCI content remains on the full primary. Selectively retain +or independently mirror exact bootstrap images and irreplaceable release assets; +prove their availability without the production Forgejo registry or Scaleway. +The manifest must state omissions, artifact digests and rebuild/restore steps. +A secondary-only restore may recover source and collaboration while package +service remains unavailable; test and document that degraded behavior explicitly. + +## Candidate 10 GiB Nextcloud budget + +Target at most 600 MiB per compressed encrypted Forgejo essentials generation. +Repositories plus SQL currently total about 573 MiB before compression, but this +is an estimate: a real essentials capture and restore must establish actual size. + +- Seven daily + two weekly distinct recovery points: at most 5.27 GiB. +- One incoming generation while the previous good copy remains: 0.59 GiB. +- Other service essentials and selective bootstrap material: 2 GiB provisional. +- Headroom for growth and provider accounting: 2 GiB. + +Total: approximately 9.86 GiB. The other-service/bootstrap allocation is unmeasured; +if it does not fit, shorten retention or adjust the allocation explicitly. Do not +silently exclude unique data to meet the target. A weekly label may reference +an existing retained generation instead of uploading Sunday's bytes twice. + +The current uploader is create-only. Retention requires a separate bounded owner +executor; do not give workload jobs the Backup account password or general delete +access. Verify the new object and recovery manifest before retiring old points. +Use exact managed names, concurrency control and conditional deletion; account +for trash/version retention and recheck actual free bytes. Stop before upload +when overlap/headroom cannot be met. Never delete the last verified recovery point. + +## Implementation acceptance in WP-0038-T04 + +1. Complete primary full-archive delivery and recovery with explicit storage and + credential-delivery ownership. +2. Audit package/bootstrap dependencies and measure other service essentials. +3. Capture an explicit essentials profile and inventory, preserving unique data; + retain the full backup path and record included/excluded categories. +4. Restore with primary storage unavailable; verify Git, issues/users, unique + attachments and declared package limitations/bootstrap availability. +5. Implement the separate bounded retention executor and space preflight. Demonstrate + upload overlap, failed-upload preservation and safe expiration within 10 GiB. +6. Switch scheduled secondary delivery only after these gates pass. Preserve the + current good full Nextcloud copy until a verified replacement permits retirement. diff --git a/docs/evidence/backup-capacity-assessment-2026-09-06.json b/docs/evidence/backup-capacity-assessment-2026-09-06.json new file mode 100644 index 0000000..eca6da4 --- /dev/null +++ b/docs/evidence/backup-capacity-assessment-2026-09-06.json @@ -0,0 +1,24 @@ +{ + "schema": "platform.backup-capacity-assessment.v1", + "date": "2026-09-06", + "measurement": "live du -sk; allocated KiB, not compressed archive sizes", + "forgejo_allocated_kib": { + "git_repositories": 491832, + "package_blobs": 4668080, + "attachments": 4, + "avatars": 40, + "repository_avatars": 4, + "actions_logs": 45596, + "actions_artifacts": 4, + "generated_repository_archives": 347200, + "configuration": 8 + }, + "last_verified_archive_bytes": 5353024293, + "last_sql_dump_bytes": 97282276, + "current_retention_archive_count": 18, + "full_archives_retained_bytes": 96354437274, + "nextcloud_quota_bytes": 10737418240, + "essentials_target_bytes": 629145600, + "essentials_size_measured": false, + "scope": "Forgejo sizing; total platform base-backup/WAL and other service volumes not measured" +} diff --git a/history/2026-09-06-backup-capacity-and-essentials-assessment.md b/history/2026-09-06-backup-capacity-and-essentials-assessment.md new file mode 100644 index 0000000..fc78ca3 --- /dev/null +++ b/history/2026-09-06-backup-capacity-and-essentials-assessment.md @@ -0,0 +1,22 @@ +# Backup capacity and essentials assessment + +The user asked how much space is needed and whether Nextcloud can hold only +essentials. Live aggregate sizing shows package blobs dominate at 4.45 GiB; +Git repositories are 480 MiB and the last SQL export 93 MiB before compression. +The existing 18-full-copy retention target needs 96.35 GB of Forgejo archive +storage alone, so it cannot fit the 10 GiB account. Its Sunday upload duplicates +that day's bytes under a weekly name. + +Prepared `docs/backup-storage-tiers.md`: plan 125–150 GB for the current Forgejo +full-archive retention on Scaleway, separately budget native database/WAL and +other services, and target a 600 MiB essentials generation on Nextcloud. A +provisional 7-daily/2-weekly policy plus incoming overlap, other-service reserves +and headroom fits 10 GiB only after measured acceptance. No compressed essentials +artifact or total platform capacity is claimed as measured. + +The reduced profile retains source/history, collaboration database, unique +attachments/LFS and recovery configuration; bulk packages require primary +coverage and an explicit rebuildability/bootstrap audit first. A primary-independent +essentials restore and separate owner retention executor are required before +cutover. This is folded into WP-0038-T04 instead of another overlapping workplan. +Live backup scope and retained good copies are unchanged. diff --git a/workplans/RPF-WP-0038-forgejo-scaleway-primary-coverage.md b/workplans/RPF-WP-0038-forgejo-scaleway-primary-coverage.md index b72ab81..3cd9e72 100644 --- a/workplans/RPF-WP-0038-forgejo-scaleway-primary-coverage.md +++ b/workplans/RPF-WP-0038-forgejo-scaleway-primary-coverage.md @@ -89,3 +89,10 @@ the successful full recovery and rejected-fixture DELETE, immediate Nextcloud metadata still reported 5,234,716,518 free bytes, below one current archive's 5,353,024,293 bytes. Recheck quota and settle bounded archive retention/capacity with the owning package; do not automatically delete retained good backups. + +User-directed storage split assessment: `docs/backup-storage-tiers.md` and +`history/2026-09-06-backup-capacity-and-essentials-assessment.md`. Keep full +archives on Scaleway and design a measured essentials-only Nextcloud profile. +T04 acceptance includes package/bootstrap audit, primary-independent essentials +recovery, quota preflight and separate bounded retention execution before +changing scheduled secondary scope. No blanket package exclusion is activated.