coulomb-social/docs/migration/corpus-import.md

Local corpus import and recovery prerequisites

import_content_corpus imports prepared ADR-0004 trees. It does not contact Bubble, provision accounts or verify the completeness of a Bubble export. Use private local trees and manifests outside the checkout. Never commit member mappings, corpus content, DB dumps or backup archives.

Manifest

Version 1 JSON. Tree paths are relative to the manifest directory or absolute. Every space requires an existing active member mapped as owner. Identity is matched by tenant + issuer + subject, never email or a password.

{
  "version": 1,
  "spaces": [
    {
      "tree": "trees/example",
      "slug": "example",
      "tenant": "tenant:example",
      "memberships": [
        {"issuer": "https://issuer.example", "subject": "existing-subject", "role": "owner"}
      ]
    }
  ]
}

Roles: owner, member, viewer. Product authorization denies writes for viewer grants and permits owner/member writes; reads require an explicit membership or creator link. Tenant-wide bootstrap visibility has been removed. The manifest supplies owner and explicit grants, but does not certify that they match original Bubble privacy. Review the effect on existing spaces before executing a real cohort migration.

Run

First provision accounts through NetKingdom and have users sign in to create local Members. Reconcile the corpus and approved grants. Pause all application writers; run with the deployment tenant's private CONTENT_ROOT and DB settings.

uv run manage.py import_content_corpus /private/path/manifest.json --dry-run
uv run manage.py import_content_corpus /private/path/manifest.json
# Explicit replacement only after reviewing the existing content and backup:
uv run manage.py import_content_corpus /private/path/manifest.json --replace

Preflight validates all entries before any writes: local pages, normalized filenames, metadata bounds, source symlinks, source/destination overlap, destination collisions, active identity/tenant mapping and at least one owner. A missing owner or identity in the last tree prevents the first tree being written. Dry-run performs that same preflight and creates no directories or rows.

Execution stages each tree beside its destination, then applies metadata and memberships and publishes by rename inside a DB transaction. Ordinary errors restore the prior tree and roll back that space's DB changes. The whole batch is not atomic: previously committed spaces remain if a later space fails. Receipts give the completed count. Reconcile local state and either import a manifest containing only remaining entries, or deliberately use --replace. Replacement removes the previous content tree, including assets not in source; it adds/updates mapped memberships and does not revoke other existing grants or overwrite a pre-existing creator. Review those extra accesses explicitly.

Writers must remain paused. A process crash, rename/DB commit race or failed recovery filesystem operation requires manual recovery; this is not a crash-safe journal or concurrent import protocol. Keep content-root and instance-per-client tenant boundaries intact: filesystem paths remain slug-keyed.

The single-tree command shares validation and staging. --member-subject must resolve to exactly one tenant-local Member; use the corpus manifest when issuer ambiguity is possible. Single-tree imports may still omit membership, in which case the space is hidden unless it has an existing membership/creator grant. They are not a replacement for approved cohort mapping.

Paired backup/restore protocol (acceptance still pending)

1. Quiesce app and import writers, including Forgejo refresh consumers where relevant. Record recovery point, exact image SHA, migration level and tenant.
2. Through the DB owner's documented credential lane, capture metadata DB; separately snapshot/archive the entire CONTENT_ROOT (pages and assets, including archived trees). Keep private data in the approved backup store.
3. Record SHA-256 file inventory, row counts and backup identities privately. Do not log credentials, content bodies or identity lists into this repo/hub.
4. Restore DB and CONTENT_ROOT together into an isolated instance. Disable outgoing integrations and use restore-specific settings and credentials.
5. Compare Space/Member/membership counts, file hashes, page list/render, visual assets, copied/moved content and denied reads by an ungranted member. Measure achieved recovery time and recovery-point age; record the non-secret receipts.
6. Resume writers only after accepted recovery checks. If import fails, restore both stores from that same recovery point rather than only rolling back DB.

This is an operational protocol, not evidence that scheduled backup or a restore drill exists. CSOC-WP-0008-T02 owns automation and measured acceptance; T01 owns storage admission, T03 the actual private cohort rehearsal.