coulomb-social/docs/release/source-date-journal-review-2026-10-30.md
tegwick ee67b6717b
Some checks failed
Use-case regression gate / usecase-regressions (push) Failing after 7m19s
Rehearse date-journal process crashes against the recovered application pair
Assistant: codex
Assistant-Model: gpt-6.1-sol
Assistant-Session: 01a18e2d-f513-7742-b8e7-12af0b744ff3
2026-10-30 20:09:43 +01:00

4.5 KiB

Creation-date cross-store recovery journal

Prepared source only. No productive date correction, deployment or acceptance. WP-0014-T04 still requires a supervised, backup-bound, quiesced executor and productive postflight. The sealed storage release proposal is unchanged.

date_backfill_journal.py adds a correction primitive to the existing byte-preserving patch compiler. Its caller must first fence all SQL and content writers and verify a fresh immutable paired backup/readback. A backup receipt SHA256 is required and retained in the journal; the primitive does not verify the receipt, its freshness, its content binding or the writer fence. It is not exposed through a management command or product route.

The operation locks the selected SQL row and checks its tenant, slug and observed creation date, plus the exact selected-space page inventory and hashes. It writes an exclusive, private, fsynced intent containing the original index bytes, plan and backup-receipt digest before either store changes. Then it updates only SQL created_at, atomically replaces the index with its compiled patch, fsyncs the file and parent, commits SQL, and marks the journal completed. Replacements preserve file ownership and mode. Symlink destinations, ambiguous page inventories, drift and journal replay refuse. Journal parents must be private; journals must stay outside served content because they contain original private page bytes.

On interruption, keep writers fenced. reconcile_date_patch locks the row and uses its committed date: the original date restores original index bytes, while the corrected date finishes the patch. Both the index and every other observed page must still match their permitted hashes. Unknown SQL state or later edits refuse without overwriting those edits. A pending intent survives an uncertain commit result. Keep the terminal journal as evidence; do not retry the original plan automatically. A malformed/partially created journal requires manual inspection before writers resume. PostgreSQL/file durability depends on the underlying storage honoring fsync.

Validation: tests/test_date_backfill_journal.py terminates actual child processes at durable intent, uncommitted SQL, replaced file and committed SQL. After death, the tests reconcile against persisted SQL and verify untouched page hashes. Other cases check successful correction, unrelated SQL fields, replay refusal, later-edit refusal, unknown SQL date, private permissions and symlinks. Together with the existing date planner/compiler tests, 30 focused cases pass on both SQLite and disposable PostgreSQL 16.14. The PostgreSQL instance used synthetic data, a temporary Docker container and a loopback-only port; it did not connect to any production/recovered database or provider. The full make check gate is recorded in the companion evidence JSON.

The retained independently recovered pair now also passes the journal rehearsal. scripts/rehearse-source-date-journal.py verifies the actual Docker network, database image, unique database alias and absence of published ports, then clones the recovered PostgreSQL database and copies its content. All commits and crash tests run against that disposable clone. Real child-process exits at intent, SQL, file and commit boundaries reconcile correctly. All 13 tables and 82 files match the expected state after every case; the successful correction changes only the selected SQL date and index. Replay refuses. Before/after SQL digests and file hashes prove the retained recovery pair is unchanged. The disposable SQL clone is removed. Evidence: source-date-journal-real-pair-2026-10-30.json. This uses explicit source overlays on the retained candidate image, not a newly released application image.

To repeat against a retained isolated restore, provide private paths locally:

uv run python scripts/rehearse-source-date-journal.py \
  --restore-receipt /private/restore-receipt.json \
  --plan /private/date-backfill-plan.json \
  --candidate-image '<locally retained image digest>' \
  --output /tmp/date-journal-real-pair.json

Remaining wrapper requirements: bind exact reviewed source/plan/target and fresh backup/readback hashes; verify the external writer fence and full live SQL/content baseline; supervise the privileged action; refuse outstanding journal state; retain the journal on durable private storage; perform full unrelated-row/file postflight and resolve every pending state before resuming writers. Rehearse that wrapper against the independently recovered real pair before productive use.