coulomb-social/docs/release/backup-operating-preflight-2026-10-30.md
tegwick 23ffe2cf4b
Some checks failed
Use-case regression gate / usecase-regressions (push) Failing after 7m19s
Reconcile backup snapshot pin and seal operating sources
Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a18e2d-f513-7742-b8e7-12af0b744ff3
2026-10-31 04:21:17 +01:00

5.2 KiB

Paired backup operating preflight

The first protected application recovery is proved. Recurring operation is not ready to activate. This review preserves the already approved daily 03:15 UTC schedule with five-minute jitter, 24-hour RPO, 26-hour stale threshold and 60-minute attended RTO target; it requests no repeat schedule/custody approval.

The old hourly check inspected only last-success age and local ciphertext. It could pass while a failed newer capture remained pending, and did not establish matching mirror evidence. scripts/check-paired-backup-health.py now checks:

  • Exact Coulomb primary identity, immutable version/readback flags, original capture time, image/recipient and both paired-store hashes.
  • Local ciphertext size and hash, plus an exact corresponding mirror receipt bound to the primary key/version/hash and dedicated recovery project/bucket.
  • Pending captures even while the previous point is fresh. An active lock grants at most 35 minutes for one in-progress capture; an abandoned pending directory, multiple captures or a stalled lock fails.
  • Private regular files/directories without symlinks, declared local state budget and minimum free space. It reads and acquires an existing lock without creating files. It prints only counts, times and fixed reason codes.

The candidate freshness unit invokes this checker with an explicit last-mirror receipt and proposed 8 GiB state budget / 8 GiB free-space reserve. These limits are review parameters, not owner-admitted storage or spending authority. A health check does not itself stop future uploads or enforce provider retention.

Native observation

The exact checker ran through the existing admitted SSH path on railiance01, without installing code, changing state, reading keys or contacting a provider. Receipt: backup-operating-preflight-2026-10-30.json.

At 2026-10-29 23:07 UTC the primary point was about 1.74 hours old; no pending capture or held backup lock existed. State occupied 2,123,252 bytes and free space was 119,716,765,696 bytes. The new check correctly failed because the worker's last-mirror.json was absent. The independently verified first mirror and its protected kit remain valid; their external receipts are not a recurring worker pipeline. Both backup/freshness timers were not found and inactive. The old scheduled checker and the proposed new one have not been installed by this work.

Required activation return

  1. Prepare and verify recurring mirror publication using only the worker's own admitted publisher. Retry must preserve the exact primary key/version/hash; retain each resulting public receipt independently. No private recovery key or independent reader belongs on the worker. Promote last-mirror only after the matching version/hash readback, and block another unattended capture while the previous point lacks its required mirror/receipt. The source worker, pinned dependency manifest and pre/post service hooks are now prepared in mirror-worker-operating-review-2026-10-30.md. Its synthetic integration and October 31 native publication/independent retrieval are verified; the scheduled node service run remains outstanding.
  2. Complete the rolling-retention/headroom contract, provider prefix expiry compatibility and pre-capture budget/reserve guard. Preserve protected kit and restored point outside rolling expiry. Historical private copies still await explicit owner disposition; this change deletes none.
  3. Bind the established evidence-owner receiver, then prove failed/pending/stale signal reception and response. The checker makes no sends. Journald output, a locally failed unit, or a queued notification is not reception proof.
  4. Review exact installed source/unit hashes and first native service run before enabling the already approved timers. Verify subsequent runs, readbacks, Ready app, timer state and next elapse; keep production restore attended.

Source unit verification and 35 targeted backup/monitor tests passed. Synthetic checks cover old/foreign mirror refusal, corrupted ciphertext, absent receipts, pending failures hidden by a fresh point, active/stalled locks, capacity and symlink refusal. They do not prove provider availability, received alerts or recurring operation. The existing storage release packet remains unchanged. The full make check gate passed 273 tests, including mandatory browser journeys.

Follow-up: the updated checker additionally requires retained public-receipt metadata. Its exact source hash and read-only native observation, plus exact candidate-unit verification on the target node, are appended to the JSON proof; the initial observation and source hash above remain historical evidence.

October 31 delivery follow-through: the source snapshot image pin now matches the already installed and verified storage-release helper byte for byte. Native default server dry-run passes. All operating sources and candidate units are sealed for review in backup-operating-delivery-2026-10-31.{md,json}; current node metadata confirms all four units remain absent/inactive. Capacity admission, retention/disposition, monitored receiver/reception and the exact installation and native service-run review still gate activation. No helper or unit was installed, and no timer was enabled.