diff --git a/SCOPE.md b/SCOPE.md index 68cc6a9..9794eda 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -1,109 +1,86 @@ -# SCOPE +# SCOPE — implemented capability and boundaries -> This file helps you quickly understand what this repository is about, -> when it is relevant, and when it is not. -> It is intentionally lightweight and may be incomplete. +Reviewed 2026-10-06 against application source, tests and dated repository +records. This describes what the checkout can do; historical live evidence is +identified separately. It is not a current production acceptance certificate. ---- +## What this repository is -## One-liner +A standalone Django co-creation application rebuilding coulomb.social on +Railiance / NetKingdom / HelixForge. Today it implements an authenticated spaces +and markdown-pages product plus local content migration tools. It is not yet +the complete social network or marketplace described by INTENT.md. -Reimplementation of the coulomb.social co-creation platform as a standalone Railiance / NetKingdom / HelixForge application, exiting the stalled bubble.io stack. +The accepted [stage-1 cut](docs/decisions/2026-08-12-feature-cut-stage-1.md) +prioritizes onboarding and productive spaces/pages transfer; marketplace, +favorites, payments and real-time collaboration are later capabilities. ---- +## Implemented in this checkout -## Core Idea +| Capability | Actual behavior and evidence | +|---|---| +| Identity | OIDC/PKCE login, logout, JIT local Member linked to user-engine; issuer+subject identity, session diagnostics and assurance handling. Development claims login is restricted to DEBUG with OIDC disabled. `apps/identity/`, identity tests. | +| Registration entry | Configured NetKingdom registration redirect; this app does not issue passwords or send verification mail. CSOC-IN-0001 is closed with 2026-08-14 Case B evidence. | +| Spaces | Tenant-filtered list/detail, member-facing create/update/archive, Title, Abstractor (`description`) and Visual metadata. `apps/spaces/services.py`, `tests/test_space_crud.py`. | +| Pages | Markdown/frontmatter list, render, create/update/delete, independent copy and transfer with source/destination access checks. Index cannot be deleted or transferred. `pageops.py`, PageOps tests. | +| Content sources | Local `CONTENT_ROOT/spaces//pages` and assets; optional Forgejo read/edit fallback, cache refresh and push webhook. Markdown is sanitized on render. Local writes do not automatically commit/push to Forgejo. | +| Import | `import_content_tree` for one ADR-0004 tree; `import_content_corpus` for a preflighted JSON batch with explicit existing tenant+issuer+subject membership mappings. Dry-run, replacement opt-in, per-space staging/rollback and aggregate counts. Synthetic multi-space regression coverage; one historical private-space rehearsal. | +| Delivery preparation | Docker/Gunicorn packaging, SQLite development / PostgreSQL URL support, migrations, health endpoint and deployment/runbook docs. Actual Helm values and cluster hosting belong to railiance-apps. | +| Evidence and coordination | Bubble surface/capability inventory, authenticated export scripts, design extraction, schema sketch, accepted cut decision, resource-demand/SLO/labor evidence and file-backed State Hub work records. | -coulomb.social connects people around shared projects and complementary capabilities. Development on bubble.io has stalled. This repo owns the full product rebuild — UI, content, data model, and migration — so the platform can evolve with the rest of the Coulomb agentic fleet under the business-app service contract. +Source paths above are under `coulomb_social/`. Member and Space metadata are +tenant-keyed, but filesystem paths are **slug-keyed**. Deployment is +instance-per-client: do not reuse a content root across tenants. Normal creation +and imports reject existing cross-tenant slug collisions; this is not a full +filesystem tenant namespace or a database-wide concurrency guarantee. ---- +## Practical limits -## In Scope +- Space visibility has a tenant-wide bootstrap fallback until any tenant + memberships exist. Membership roles are stored but viewer/editor enforcement + and Bubble privacy parity are incomplete. Imported mappings add/update grants; + replacement does not revoke old grants. Review access before a real cohort import. +- Copy/transfer handles relative Visual files, not arbitrary body-linked assets, + incoming page links, graph identity or Bubble parent/chunk semantics. Visual + metadata and copied files do not establish an authenticated asset-serving or + upload pipeline. +- Import requires local prepared trees and existing active identities. It does + not create platform accounts, import passwords, scrape missing Bubble content, + or certify corpus completeness. It runs with writers paused; the batch is + restartable per space, not globally atomic or crash-transactional. +- No app-local member export/delete product, paired DB/content restore drill, + automatic content git versioning, outline/include/projection authoring or + marketplace/social/payment product is implemented here. +- Historical health/auth evidence does not prove current image, persistence, + runtime SLOs, backup success or complete service-contract compliance. -- Product intent, scope, workplans, and ADRs for the coulomb.social rebuild -- Inventory of the existing bubble.io app (pages, workflows, data types, integrations, content) -- Data export, relational schema mapping, and migration rehearsal -- Feature-cut decisions (rebuild-as-is vs reduced v1) and acceptance criteria -- Application implementation: UI parity, domain model, app-local accounts, APIs -- Packaging for the business delivery lane (commit-SHA images, railiance-apps release values) -- Tenant-keyed data model from the first migration; member account export/delete -- Ops/runbook stubs required by the business-app service contract (§§3–6) -- State Hub workplans, progress events, and classification for this repo +## Recorded deployment state ---- +The host strategy is rebuild on `app.coulomb.social` with Bubble on apex until +an explicit cutover. Historical identity/app-shell evidence is in +[identity smoke](docs/identity/smoke.md). Public registration was evidenced on +2026-08-14. **CSOC-IN-0004 remains open** for PageOps deployment, migration, +persistent CONTENT_ROOT and live smoke. No fresh live deployment verification +was performed in this review. All six original numbered workplans are finished; +their residual intakes remain the source of unfinished work. -## Out of Scope +## Ownership boundaries -- Remaining on bubble.io as the primary development platform -- Owning Railiance cluster, platform, or app-chart infrastructure code (`railiance-*`) -- Owning NetKingdom identity platform implementation (Keycloak/OpenBao) — only the app-local auth boundary and future OIDC switch point live here -- Owning HelixForge standards or capability registry implementation -- Core Hub embedding or treating coulomb.social as a hub extension -- Fleet self-improvement loops (owner: `coulomb-loop`) -- Serving paying external multi-tenant SaaS before the service-contract preconditions are met +This repo owns product behavior, local identity integration, application models, +content/migration tooling, packaging and acceptance evidence. It does not own +Railiance infrastructure/charts, NetKingdom issuer/mail/secrets services, +HelixForge standards, or fleet improvement loops. Core Hub embedding and Bubble +as the long-term development surface remain out of scope. ---- +## Next work and orientation -## Relevant When +Read [INTENT.md](INTENT.md), [the timestamped assessment](history/20261005T223104Z-scope-intent-assessment.md), +[CSOC-WP-0007](workplans/CSOC-WP-0007-safe-transfer-readiness.md), and +[CSOC-WP-0008](workplans/CSOC-WP-0008-production-transfer-acceptance.md). +The local transfer tool closes a preparation gap; persistence/restore, private +cohort migration, access acceptance and founder cutover are live successor work. -- Planning or implementing the bubble.io exit and product rebuild -- Mapping coulomb.social features, content, or data into code-owned artifacts -- Packaging or deploying the coulomb.social app via the business delivery lane -- Checking tenancy, identity, backup, or cost-attribution obligations for this app - ---- - -## Not Relevant When - -- You need cluster or mesh infrastructure changes → use the relevant `railiance-*` repo -- You need platform SSO/secret custody work → use `net-kingdom` / OpenBao workplans -- You need fleet kaizen loop operation → use `coulomb-loop` -- You need a reference business-app implementation pattern → start with `vergabe-teilnahme` -- You only need the historical seed note → `the-custodian/docs/coulomb-social-rebuild-seed.md` (pointer; work originates here) - ---- - -## Current State - -- Status: **parallel hosts** — identity + app shell + Forgejo spaces live on app.* -- **`https://app.coulomb.social`**: Railiance rebuild (OIDC, app home, spaces markdown) -- **`https://coulomb.social`**: Bubble.io full product until cutover decision -- Finished product path: **`CSOC-WP-0004`** (spaces + Forgejo markdown + runbook) -- Finished identity path: **`CSOC-WP-0002`**, **`CSOC-WP-0003`** (Case A; Case B residual intakes) -- Content ADR: `docs/adr/ADR-0002-space-content-forgejo-markdown.md` -- Operator: `docs/deploy.md` runbook; residuals `docs/intakes/csoc-residuals.md` -- Finished resource evidence: **`CSOC-WP-0005`** → `docs/resource-evidence/` -- Bubble exit: **`CSOC-WP-0001`** — inventory done; stage-1 feature cut accepted (partial rebuild); export still needs Bubble editor -- Capability model: `docs/capability/` — stage 1 must = onboarding + spaces/pages (Title/Abstract/Visual) + page copy/transfer -- Open residuals: public registration (`CSOC-IN-0001`/`0002`, NK-WP-0025) - ---- - -## How It Fits - -- **Upstream decisions:** business-app service contract; DR-1 C / DR-2 B / DR-3 A -- **Delivery lane:** `vergabe-teilnahme` (pattern) → this repo (product) → `railiance-apps` (release values) → railiance01 -- **Platform peers:** `railiance-*`, `net-kingdom`, `helix-forge`, `ops-hub`, `fin-hub` -- **Coordination:** State Hub topic `coulomb-social` (domain `communication`); related engagement repo `coulomb-loop` - ---- - -## Getting Oriented - -- Start with: `INTENT.md`, `GOAL.md`, `.custodian-brief.md` -- Key paths: `workplans/`, `docs/` (as they appear), application source (once scaffolded) -- First workplan: `workplans/CSOC-WP-0001-bubble-io-exit-assessment.md` -- Contract: `the-custodian/canon/standards/business-app-service-contract_v0.1.md` - ---- - -## Related Repositories - -| Repo | Relationship | -|------|----------------| -| `vergabe-teilnahme` | Reference business-app delivery lane | -| `railiance-apps` | Helm/release values for deploy | -| `railiance-forge` / `helix-forge` | Forge and capability patterns | -| `net-kingdom` | Future OIDC / platform identity | -| `coulomb-loop` | Fleet improvement loops (same State Hub topic) | -| `the-custodian` / `state-hub` | Coordination, standards, seed history | +Development: `README.md`, `docs/dev.md`, `make test`, `make lint`. +Content architecture: ADR-0003/0004 and `docs/architecture/`. +Migration: `docs/migration/`, including `corpus-import.md`. +Delivery: `docs/deploy.md`; residual records: `docs/intakes/csoc-residuals.md`. diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index fd234ce..89027bd 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -14,6 +14,8 @@ | workplan | CSOC-WP-0004 | finished | — | workplans/CSOC-WP-0004-app-shell-and-space-content.md | | workplan | CSOC-WP-0005 | finished | — | workplans/CSOC-WP-0005-resource-demand-and-cost-evidence.md | | workplan | CSOC-WP-0006 | finished | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md | +| workplan | CSOC-WP-0007 | finished | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md | +| workplan | CSOC-WP-0008 | ready | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | | task | CSOC-WP-0001-T01 | done | — | workplans/CSOC-WP-0001-bubble-io-exit-assessment.md | | task | CSOC-WP-0001-T02 | done | — | workplans/CSOC-WP-0001-bubble-io-exit-assessment.md | | task | CSOC-WP-0001-T03 | done | — | workplans/CSOC-WP-0001-bubble-io-exit-assessment.md | @@ -46,7 +48,18 @@ | task | CSOC-WP-0006-T04 | done | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md | | task | CSOC-WP-0006-T05 | done | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md | | task | CSOC-WP-0006-T06 | done | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md | +| task | CSOC-WP-0007-T01 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md | +| task | CSOC-WP-0007-T02 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md | +| task | CSOC-WP-0007-T03 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md | +| task | CSOC-WP-0007-T04 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md | +| task | CSOC-WP-0008-T01 | wait | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | +| task | CSOC-WP-0008-T02 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | +| task | CSOC-WP-0008-T03 | wait | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | +| task | CSOC-WP-0008-T04 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | +| task | CSOC-WP-0008-T05 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | +| task | CSOC-WP-0008-T06 | wait | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | +| task | CSOC-WP-0008-T07 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md | | intake | CSOC-IN-0001 | closed | blue | docs/intakes/csoc-residuals.md | -| intake | CSOC-IN-0002 | open | green | docs/intakes/csoc-residuals.md | +| intake | CSOC-IN-0002 | closed | green | docs/intakes/csoc-residuals.md | | intake | CSOC-IN-0003 | open | blue | docs/intakes/csoc-residuals.md | | intake | CSOC-IN-0004 | open | green | docs/intakes/csoc-residuals.md | diff --git a/coulomb_social/apps/spaces/import_tree.py b/coulomb_social/apps/spaces/import_tree.py index 55d7c8e..023728d 100644 --- a/coulomb_social/apps/spaces/import_tree.py +++ b/coulomb_social/apps/spaces/import_tree.py @@ -1,32 +1,19 @@ -"""Import an ADR-0004 space tree into the content plane + Space index. - -Source layout (export or hand-authored): - - / - pages/*.md - assets/* (optional) - -Does not fetch Bubble; only copies local files. Bodies may be private — -callers should keep source trees out of git. -""" +"""Validated local ADR-0004 import; never fetches Bubble or creates identities.""" from __future__ import annotations -import re import shutil +import tempfile from dataclasses import dataclass, field from pathlib import Path from django.db import transaction -from django.utils.text import slugify from coulomb_social.apps.members.models import Member from . import pageops from .models import Space, SpaceMembership -_SLUG_SAFE = re.compile(r"[^a-z0-9]+") - @dataclass class ImportReport: @@ -41,133 +28,153 @@ class ImportReport: page_slugs: list[str] = field(default_factory=list) -def _read_index_meta(pages: Path) -> tuple[str, str, str]: - index = pages / "index.md" - if not index.is_file(): - return "", "", "" - text = index.read_text(encoding="utf-8") - meta, _ = pageops.parse_frontmatter(text) - title = str(meta.get("title") or "").strip() - abstractor = str(meta.get("abstractor") or meta.get("description") or "").strip() - visual = str(meta.get("visual") or "").strip() - return title, abstractor, visual - - def normalize_import_slug(value: str) -> str: - s = slugify(value or "")[:80] - if not s: - raise pageops.PageOpsError("Space slug cannot be empty") - return s + return pageops.normalize_slug(value) + + +def _metadata(tree: Path, slug: str, title: str | None = None) -> tuple[str, str, str]: + index = tree / "pages" / "index.md" + meta, _ = ( + pageops.parse_frontmatter(index.read_text(encoding="utf-8")) + if index.is_file() else ({}, "") + ) + display_title = (title or str(meta.get("title") or slug)).strip() + if not display_title or len(display_title) > 255: + raise pageops.PageOpsError("Title must be nonempty and at most 255 characters") + abstractor = str(meta.get("abstractor") or meta.get("description") or "") + visual = str(meta.get("visual") or "") + if len(visual) > 512: + raise pageops.PageOpsError("Visual path exceeds 512 characters") + return display_title, abstractor, visual + + +def validate_space_tree( + tree: Path, *, tenant_id: str, slug: str | None = None, + member: Member | None = None, replace: bool = False, title: str | None = None, +) -> tuple[Path, str]: + """Preflight used by dry-run and execution, before any content writes. + + CONTENT_ROOT is instance-local and keyed by slug, so importing a slug + already owned by another tenant is unsupported and must fail closed. + """ + tree = tree.expanduser() + if tree.is_symlink(): + raise pageops.PageOpsError("Source tree must not be a symlink") + tree = tree.resolve() + if not tree.is_dir() or not (tree / "pages").is_dir(): + raise pageops.PageOpsError("Source must contain a pages/ directory") + if not tenant_id.strip() or len(tenant_id) > 64: + raise pageops.PageOpsError("Tenant must be nonempty and at most 64 characters") + space_slug = normalize_import_slug(slug or tree.name) + if member is not None and member.tenant_id != tenant_id: + raise pageops.PageOpsError("Member belongs to another tenant") + if Space.objects.filter(slug=space_slug).exclude(tenant_id=tenant_id).exists(): + raise pageops.PageOpsError("Content slug belongs to another tenant") + for path in tree.rglob("*"): + if path.is_symlink(): + raise pageops.PageOpsError("Source tree contains a symlink") + pages = sorted((tree / "pages").glob("*.md")) + if not pages: + raise pageops.PageOpsError("Source contains no markdown pages") + for path in pages: + if not path.is_file() or pageops.normalize_slug(path.stem) != path.stem: + raise pageops.PageOpsError("Source page filenames must be normalized slugs") + path.read_text(encoding="utf-8") + _metadata(tree, space_slug, title) + root = pageops.content_root() + dest = pageops.space_dir(space_slug) + if tree == root or root.is_relative_to(tree) or tree.is_relative_to(root): + raise pageops.PageOpsError("Source tree and CONTENT_ROOT must not overlap") + for path in (root, root / "spaces", dest): + if path.is_symlink(): + raise pageops.PageOpsError("Destination must not be a symlink") + if dest.exists() and not replace: + raise pageops.PageOpsError("Destination already exists; pass replace=True to overwrite") + return tree, space_slug def import_space_tree( - tree: Path, - *, - tenant_id: str, - slug: str | None = None, - member: Member | None = None, - title: str | None = None, - replace: bool = False, + tree: Path, *, tenant_id: str, slug: str | None = None, + member: Member | None = None, title: str | None = None, + replace: bool = False, memberships: list[tuple[Member, str]] | None = None, ) -> ImportReport: - """Copy tree into CONTENT_ROOT and upsert Space row. + """Stage content, then publish with DB writes; restore on ordinary errors. - If ``replace`` is False and the destination pages dir already has files, - raises PageOpsError unless the tree is empty. + This is not a distributed transaction or crash recovery protocol. Run with + application writers paused and take a paired DB/content backup first. """ - tree = tree.expanduser().resolve() - if not tree.is_dir(): - raise pageops.PageOpsError(f"Tree not found: {tree}") - pages_src = tree / "pages" - if not pages_src.is_dir(): - raise pageops.PageOpsError(f"Missing pages/ under {tree}") - - idx_title, idx_abs, idx_vis = _read_index_meta(pages_src) - space_slug = normalize_import_slug(slug or tree.name) - display_title = (title or idx_title or space_slug).strip() - abstractor = idx_abs - visual = idx_vis - + tree, space_slug = validate_space_tree( + tree, tenant_id=tenant_id, slug=slug, member=member, replace=replace, title=title, + ) + grants = list(memberships or []) + if member is not None: + grants.append((member, SpaceMembership.Role.OWNER)) + for mapped, role in grants: + if mapped.tenant_id != tenant_id or role not in SpaceMembership.Role.values: + raise pageops.PageOpsError("Invalid membership tenant or role") + index = tree / "pages" / "index.md" + display_title, abstractor, visual = _metadata(tree, space_slug, title) dest = pageops.space_dir(space_slug) - dest_pages = pageops.pages_dir(space_slug) - dest_assets = pageops.assets_dir(space_slug) - - if dest_pages.is_dir() and any(dest_pages.glob("*.md")) and not replace: - raise pageops.PageOpsError( - f"Destination already has pages at {dest_pages}; pass replace=True to overwrite" - ) - - report = ImportReport(space_slug=space_slug, title=display_title) - - if replace and dest.is_dir(): - shutil.rmtree(dest) - - pageops.ensure_space_tree(space_slug) - - # Copy pages - for path in sorted(pages_src.glob("*.md")): - target = dest_pages / path.name - shutil.copy2(path, target) - report.pages_copied += 1 - report.page_slugs.append(path.stem) - - # Copy assets if present - assets_src = tree / "assets" - if assets_src.is_dir(): - for path in sorted(assets_src.rglob("*")): - if not path.is_file(): - continue - rel = path.relative_to(assets_src) - target = dest_assets / rel - target.parent.mkdir(parents=True, exist_ok=True) - shutil.copy2(path, target) - report.assets_copied += 1 - - if not (dest_pages / "index.md").is_file(): - pageops.init_space_content( - space_slug, - title=display_title, - abstractor=abstractor, - visual=visual, - ) - report.warnings.append("index.md missing in source; synthesized") - report.pages_copied += 1 - report.page_slugs.append("index") - - report.content_path = str(dest) - - with transaction.atomic(): - space, created = Space.objects.update_or_create( - tenant_id=tenant_id, - slug=space_slug, - defaults={ - "title": display_title, - "description": abstractor, - "visual": visual, - "is_active": True, - "content_root": "pages", - "default_branch": "main", - }, - ) - report.space_created = created - report.space_updated = not created - if member is not None: - SpaceMembership.objects.get_or_create( - space=space, - member=member, - defaults={"role": SpaceMembership.Role.OWNER}, - ) - if space.created_by_id is None: - space.created_by = member - space.save(update_fields=["created_by", "updated_at"]) - - # Sanity: count readable pages + dest.parent.mkdir(parents=True, exist_ok=True) + report = ImportReport(space_slug=space_slug, title=display_title, content_path=str(dest)) + temporary = Path(tempfile.mkdtemp(prefix=".import-", dir=dest.parent)) + recovery_needed = False try: - listed = pageops.list_pages(space_slug) - if len(listed) != report.pages_copied: - report.warnings.append( - f"list_pages returned {len(listed)} vs copied {report.pages_copied}" - ) - except pageops.PageOpsError as exc: - report.warnings.append(f"list_pages failed: {exc.message}") - + staging = temporary / "staged" + (staging / "pages").mkdir(parents=True) + (staging / "assets").mkdir() + for path in sorted((tree / "pages").glob("*.md")): + shutil.copy2(path, staging / "pages" / path.name) + report.page_slugs.append(path.stem) + assets = tree / "assets" + if assets.is_dir(): + shutil.copytree(assets, staging / "assets", dirs_exist_ok=True) + report.assets_copied = sum(p.is_file() for p in assets.rglob("*")) + if not index.is_file(): + (staging / "pages" / "index.md").write_text(pageops.dump_frontmatter( + {"title": display_title, "abstractor": abstractor, "visual": visual, + "space": space_slug}, f"# {display_title}\n", + ), encoding="utf-8") + report.page_slugs.append("index") + report.warnings.append("index.md missing in source; synthesized") + report.pages_copied = len(report.page_slugs) + backup = temporary / "previous" + published = False + try: + with transaction.atomic(): + space, created = Space.objects.update_or_create( + tenant_id=tenant_id, slug=space_slug, + defaults={"title": display_title, "description": abstractor, + "visual": visual, "is_active": True, + "content_root": "pages", "default_branch": "main"}, + ) + report.space_created = created + report.space_updated = not created + for mapped, role in grants: + SpaceMembership.objects.update_or_create( + space=space, member=mapped, defaults={"role": role}, + ) + owner = next((m for m, r in grants if r == SpaceMembership.Role.OWNER), None) + if owner and space.created_by_id is None: + space.created_by = owner + space.save(update_fields=["created_by", "updated_at"]) + if dest.exists(): + dest.rename(backup) + staging.rename(dest) + published = True + except Exception: + try: + if published: + shutil.rmtree(dest) + if backup.exists(): + backup.rename(dest) + except OSError as exc: + recovery_needed = True + raise pageops.PageOpsError( + f"Filesystem recovery failed; staging/previous tree retained at {temporary}" + ) from exc + raise + finally: + if not recovery_needed: + shutil.rmtree(temporary) return report diff --git a/coulomb_social/apps/spaces/management/commands/import_content_corpus.py b/coulomb_social/apps/spaces/management/commands/import_content_corpus.py new file mode 100644 index 0000000..2299dc0 --- /dev/null +++ b/coulomb_social/apps/spaces/management/commands/import_content_corpus.py @@ -0,0 +1,99 @@ +"""Preflight a JSON corpus manifest and import restartable space units.""" + +from __future__ import annotations + +import json +from pathlib import Path + +from django.core.management.base import BaseCommand, CommandError + +from coulomb_social.apps.members.models import Member +from coulomb_social.apps.spaces import pageops +from coulomb_social.apps.spaces.import_tree import import_space_tree, validate_space_tree +from coulomb_social.apps.spaces.models import SpaceMembership + + +class Command(BaseCommand): + help = "Import local space trees with explicit tenant/issuer/subject membership mappings" + + def add_arguments(self, parser): + parser.add_argument("manifest") + parser.add_argument("--dry-run", action="store_true") + parser.add_argument("--replace", action="store_true") + + def handle(self, *args, **options): + try: + plan = self.preflight(Path(options["manifest"]), replace=options["replace"]) + except (ValueError, KeyError, TypeError, OSError, pageops.PageOpsError) as exc: + raise CommandError(f"Manifest preflight failed: {exc}") from exc + pages = sum(len(list((tree / "pages").glob("*.md"))) for tree, _, _, _ in plan) + self.stdout.write(f"Validated {len(plan)} space(s), {pages} source page(s)") + if options["dry_run"]: + self.stdout.write("Dry run — no writes") + return + completed = 0 + for tree, tenant, slug, grants in plan: + try: + report = import_space_tree( + tree, tenant_id=tenant, slug=slug, memberships=grants, + replace=options["replace"], + ) + except Exception as exc: + # Report counts, not private source data or identity details. + raise CommandError( + f"Import stopped after {completed} committed space(s); " + "failed space rolled back. Inspect locally before retrying." + ) from exc + completed += 1 + self.stdout.write( + f"Imported space {completed}/{len(plan)}: " + f"pages={report.pages_copied} assets={report.assets_copied} " + f"memberships={len(grants)}" + ) + self.stdout.write(self.style.SUCCESS(f"Imported {completed} space(s)")) + + def preflight(self, manifest: Path, *, replace: bool): + manifest = manifest.expanduser().resolve() + data = json.loads(manifest.read_text(encoding="utf-8")) + if (not isinstance(data, dict) or data.get("version") != 1 + or not isinstance(data.get("spaces"), list)): + raise ValueError("Expected version=1 and spaces list") + if not data["spaces"]: + raise ValueError("Manifest must contain at least one space") + plan = [] + slugs = set() + for entry in data["spaces"]: + if not isinstance(entry, dict) or not isinstance(entry.get("memberships"), list): + raise ValueError("Each space needs a memberships list") + tenant = entry["tenant"] + tree = Path(entry["tree"]) + if not tree.is_absolute(): + tree = manifest.parent / tree + tree, slug = validate_space_tree( + tree, tenant_id=tenant, slug=entry.get("slug"), replace=replace, + ) + if slug in slugs: + raise ValueError("Duplicate destination content slug") + slugs.add(slug) + grants = [] + seen = set() + for identity in entry["memberships"]: + if not isinstance(identity, dict): + raise ValueError("Membership entries must be objects") + role = identity["role"] + if role not in SpaceMembership.Role.values: + raise ValueError("Unknown membership role") + member = Member.objects.filter( + tenant_id=tenant, issuer=identity["issuer"], subject=identity["subject"], + user__is_active=True, + ).first() + if member is None: + raise ValueError("Mapped identity is not an active member in the tenant") + if member.pk in seen: + raise ValueError("Duplicate mapped member") + seen.add(member.pk) + grants.append((member, role)) + if not any(role == SpaceMembership.Role.OWNER for _, role in grants): + raise ValueError("Each imported space requires a mapped owner") + plan.append((tree, tenant, slug, grants)) + return plan diff --git a/coulomb_social/apps/spaces/management/commands/import_content_tree.py b/coulomb_social/apps/spaces/management/commands/import_content_tree.py index 94d27b2..693a9e0 100644 --- a/coulomb_social/apps/spaces/management/commands/import_content_tree.py +++ b/coulomb_social/apps/spaces/management/commands/import_content_tree.py @@ -9,7 +9,7 @@ from django.core.management.base import BaseCommand, CommandError from coulomb_social.apps.members.models import Member from coulomb_social.apps.spaces import pageops -from coulomb_social.apps.spaces.import_tree import import_space_tree +from coulomb_social.apps.spaces.import_tree import import_space_tree, validate_space_tree class Command(BaseCommand): @@ -65,12 +65,25 @@ class Command(BaseCommand): tenant = options["tenant"] or settings.DEFAULT_TENANT_ID slug = options["slug"] or tree.name - md_files = sorted(pages.glob("*.md")) + member = None + subject = (options["member_subject"] or "").strip() + if subject: + candidates = Member.objects.filter(subject=subject, tenant_id=tenant) + if candidates.count() != 1: + raise CommandError("Subject must resolve to exactly one member in the tenant") + member = candidates.get() + try: + tree, slug = validate_space_tree( + tree, tenant_id=tenant, slug=slug, member=member, + replace=options["replace"], title=options["title"] or None, + ) + except (pageops.PageOpsError, OSError, UnicodeError) as exc: + raise CommandError(str(exc)) from exc + md_files = sorted((tree / "pages").glob("*.md")) assets_dir = tree / "assets" asset_count = ( sum(1 for p in assets_dir.rglob("*") if p.is_file()) - if assets_dir.is_dir() - else 0 + if assets_dir.is_dir() else 0 ) self.stdout.write( @@ -85,19 +98,6 @@ class Command(BaseCommand): self.stdout.write(self.style.WARNING("Dry run — no writes")) return - member = None - subject = (options["member_subject"] or "").strip() - if subject: - member = Member.objects.filter(subject=subject, tenant_id=tenant).first() - if member is None: - # try any tenant match for convenience in local smoke - member = Member.objects.filter(subject=subject).first() - if member is None: - raise CommandError( - f"No Member with subject={subject!r} " - f"(sign in once or pass a known subject)" - ) - try: report = import_space_tree( tree, diff --git a/coulomb_social/apps/spaces/services.py b/coulomb_social/apps/spaces/services.py index 4946019..321865a 100644 --- a/coulomb_social/apps/spaces/services.py +++ b/coulomb_social/apps/spaces/services.py @@ -54,8 +54,8 @@ def create_space( if not title: raise pageops.PageOpsError("Title is required") space_slug = pageops.normalize_slug(slug or Space.suggest_slug(title)) - if Space.objects.filter(tenant_id=member.tenant_id, slug=space_slug).exists(): - raise pageops.PageOpsError("A space with this slug already exists") + if Space.objects.filter(slug=space_slug).exists(): + raise pageops.PageOpsError("A space with this slug already exists in this content instance") space = Space.objects.create( tenant_id=member.tenant_id, diff --git a/docs/capability/model-v0.yaml b/docs/capability/model-v0.yaml index 7178910..0b4a82f 100644 --- a/docs/capability/model-v0.yaml +++ b/docs/capability/model-v0.yaml @@ -1,10 +1,13 @@ +# Reviewed 2026-10-06: implementation labels describe checkout capability. +# "shipped" is legacy vocabulary, not fresh production acceptance. +# PageOps live/persistence acceptance remains CSOC-IN-0004 / CSOC-WP-0008. # coulomb-social product capability model v0 # Vocabulary: docs/capability/vocabulary.md # Stage-1 narrative: docs/capability/stage-1-cutover.md schema_version: "0.1" product: coulomb-social resource_id: "resource:tenant:coulomb:coulomb-social" -updated: "2026-08-13" +updated: "2026-10-06" stance: | Partial rebuild is intentional. Stage 1 enables user transfer and retires Bubble as productive system; Bubble may remain reference-only. Later stages @@ -41,15 +44,16 @@ capabilities: status: shipped - id: feat.public-registration title: Public Create account + verified mail - status: blocked - blocked_on: "NK-WP-0025 / CSOC-IN-0001" + status: shipped + notes: "Historical Case B evidence 2026-08-14; CSOC-IN-0001 closed; recheck at transfer" toggle: NETKINGDOM_REGISTRATION_URL - id: feat.post-login-app-home title: Post-login app home (Spaces) status: shipped - id: feat.member-transfer-assist title: Assisted transfer of Bubble members (no password import) - status: todo + status: partial + notes: "Validated corpus manifest maps existing identities; real cohort pending WP-0008" - id: cap.space-content title: Spaces with markdown content diff --git a/docs/capability/stage-1-cutover.md b/docs/capability/stage-1-cutover.md index a4516a2..aba168c 100644 --- a/docs/capability/stage-1-cutover.md +++ b/docs/capability/stage-1-cutover.md @@ -4,7 +4,7 @@ |-------|--------| | Decision | [`../decisions/2026-08-12-feature-cut-stage-1.md`](../decisions/2026-08-12-feature-cut-stage-1.md) | | Model | [`model-v0.yaml`](model-v0.yaml) | -| Date | 2026-08-12 | +| Date | Decision: 2026-08-12; checkout status reviewed: 2026-10-06 | | Stance | **Partial rebuild is enough** for transfer; extend with later use cases after Bubble is reference-only | ## Goal of stage 1 @@ -17,6 +17,11 @@ Bubble on apex as a **productive** system. After transfer: - Additional capabilities (marketplace, social graph, monetization, …) ship **later** on the rebuild without blocking transfer. +Implementation statuses below describe this checkout, not fresh live acceptance. +CSOC-IN-0004 still owns PageOps persistence/deployment smoke; WP-0008 owns +transfer and recovery acceptance. Public registration has historical 2026-08-14 +Case B evidence (CSOC-IN-0001 closed). + ## Must capabilities (stage 1) ### 1. User onboarding — `cap.member-onboarding` @@ -29,9 +34,9 @@ Bubble for day-to-day login. | OIDC sign-in (existing LLDAP/KeyCape users) | shipped | Case A proven | | JIT Member + user-engine link | shipped | | | Session / account diagnostics | shipped | `/account/session/` | -| Public Create account + verified mail | partial / blocked | App entry ready; NK mailbox (`CSOC-IN-0001`) | +| Public Create account + verified mail | historically evidenced | CSOC-IN-0001 closed; NK owns mail/registration, recheck at transfer | | Guided first landing on app home | shipped | post-login → Spaces | -| Bulk / assisted member transfer from Bubble | todo | Process + optional import; no password migration | +| Bulk / assisted member transfer from Bubble | partial | Manifest maps existing tenant+issuer+subject members; actual cohort rehearsal pending | **Stage-1 acceptance:** known operators and community members can sign in; new members can register once NK path is live **or** operator-provisioned @@ -46,15 +51,15 @@ Members work in **spaces** whose durable content is **markdown** (ADR-0002). | Space list (tenant-scoped) | shipped | | | Space detail + render page from Forgejo | shipped | | | Edit in Forgejo + refresh / webhook | shipped | git SoR | -| **Space CRUD** (create/read/update/delete or archive) | partial | Admin/seed today; **member UI CRUD todo** | -| **Title** on space | partial | Model field exists; member edit UI todo | -| **Abstract** on space | partial | Use `description` as abstract until renamed | -| **Visual** on space (cover/image) | todo | Not in model yet | -| Page list within space | partial | path-based pages; richer list todo | -| **Page CRUD** (create/update/delete markdown pages) | partial | Via Forgejo only; in-app or first-class API todo | -| **Title** on page | partial | From H1 / filename; structured frontmatter todo | -| **Abstract** on page | todo | Frontmatter or metadata | -| **Visual** on page | todo | Frontmatter image / assets | +| **Space CRUD** (create/read/update/delete or archive) | implemented | Member create/update/archive (WP-0006) | +| **Title** on space | implemented | Model and member forms | +| **Abstract** on space | implemented | `description`, product label Abstractor | +| **Visual** on space (cover/image) | metadata implemented | Relative path; upload/authenticated asset delivery not accepted | +| Page list within space | implemented | Local pages sidebar | +| **Page CRUD** (create/update/delete markdown pages) | implemented | In-app metadata/body forms; index deletion prohibited | +| **Title** on page | implemented | Frontmatter and form | +| **Abstract** on page | implemented | Frontmatter and form | +| **Visual** on page | metadata implemented | Relative Visual copied; broader asset pipeline incomplete | | Fail-closed missing binding | shipped | | **Stage-1 acceptance:** members can create and maintain spaces and pages with @@ -66,8 +71,8 @@ structured form + Forgejo body) can vary; **utility** must not require Bubble. | Feature | Status | Notes | |---------|--------|--------| -| Transfer page to another space | todo | Move markdown (+ assets) + rewrite links; metadata follows | -| Authz (source + destination membership) | todo | | +| Transfer page to another space | implemented with limits | Markdown + Visual asset; arbitrary links/assets are not rewritten | +| Authz (source + destination membership) | implemented with limits | Tenant/membership visibility; viewer write roles still unenforced | **Stage-1 acceptance:** a page can leave space A and appear in space B without Bubble, with clear ownership after the move. @@ -76,8 +81,8 @@ Bubble, with clear ownership after the move. | Feature | Status | Notes | |---------|--------|--------| -| Generate copy of page (same or other space) | todo | New slug/title; independent thereafter | -| Copy assets referenced by page | todo | Or document “copy body only” if scoped | +| Generate copy of page (same or other space) | implemented | New slug/title; independent thereafter | +| Copy assets referenced by page | partial | Relative Visual copied; body-linked assets deferred | **Stage-1 acceptance:** member can produce an independent copy without manual git surgery as the only path (operator git escape hatch may exist but is not @@ -100,29 +105,26 @@ Examples — full list in `model-v0.yaml`: |------------|----------------| | NetKingdom OIDC + (eventually) registration mail | Onboarding | | apps-pg | Space/member metadata | -| Forgejo | Markdown SoR | +| CONTENT_ROOT + optional Forgejo | Local PageOps SoR; Forgejo fallback, no automatic local commit/push | | Railiance deploy | Host app.coulomb.social | -## Implementation backlog (derived) +## Current acceptance backlog -Priority order suggested for build work after this decision: +WP-0006 implemented the member CRUD/metadata/copy/transfer path. WP-0007 added +validated single/bulk import with explicit membership mapping. Remaining +acceptance is tracked in WP-0008: persistence/live smoke, paired recovery, +private cohort reconciliation, access roles/privacy, member portability and +explicit founder cutover. Later authoring features remain in the machine model. -1. Space member CRUD + Title / Abstract edit -2. Visual field (space, then page) + asset storage convention -3. Page metadata (title/abstract/visual) + create/update/delete path -4. Page copy -5. Page transfer -6. Onboarding: transfer cohort process + unstick public registration residual +## Resolved choices and remaining product questions -Bubble export/mapping (CSOC-WP-0001-T02) feeds **content migration** into these -capabilities but does not redefine the must set. - -## Open product questions - -1. **Visual** — cover image only, or gallery? Max size? assets next to markdown vs object store? -2. **Abstract** — rename `Space.description` → abstract in UI only, or schema rename? -3. **Page body editing** — stage 1: product forms for Title/Abstract/Visual + markdown body path; exact editor TBD. -4. **Delete** — hard delete vs archive (`is_active`)? -5. **Transfer cohort** — require public self-registration, or LLDAP bulk invite for stage 1? -6. **Content substrate** — see `docs/architecture/2026-08-12-content-substrate-exploration.md` (markitect-tool + markdown SoR; evaluate shard-wiki thin slice vs folder/git store). -7. **Authoring** — top-down and bottom-up both first-class; lazy title/abstractor/visual with regenerate + lock — `docs/architecture/authoring-modes-and-lazy-projections.md`. +- Visual: path metadata and relative-file copying today; upload, serving and + full body-linked asset fidelity require acceptance work. +- Abstract: UI uses Abstractor; persisted Space field remains `description`. +- Body: markdown form implemented; no real-time editor or conflict protocol. +- Delete: Space archive; page delete except index. +- Cohort: explicit existing NetKingdom identity mappings; no password import. + Actual cohort and lossiness acceptance are still required. +- Content: ADR-0004 thin local directory, optional Forgejo fallback. Git + automation, authoring projections, live includes and outline operations remain + future work, not prerequisites silently added to the accepted cut. diff --git a/docs/identity/smoke.md b/docs/identity/smoke.md index 746b4a3..3f49064 100644 --- a/docs/identity/smoke.md +++ b/docs/identity/smoke.md @@ -133,3 +133,18 @@ make test ``` See also: `docs/deploy.md`, `docs/spaces-content.md`, `docs/adr/ADR-0001-netkingdom-identity.md`. + + +## Independent negative regressions — 2026-10-06 + +CSOC-IN-0002 closed by CSOC-WP-0007 with synthetic application tests in +`tests/test_identity_regressions.py` (no live issuer or private accounts): + +- A successful callback consumes state and PKCE verifier. Replaying the same + callback returns 400; token exchange and session establishment each run once. +- Two issuer+subject identities sharing an email get distinct Member/User rows. + The second session authenticates the second user; the original member's id, + user link and display name stay intact. Email does not select an existing member. + +This independently covers the two narrowed intake checks. It does not constitute +a fresh live Case B or production rollout smoke. diff --git a/docs/intakes/csoc-residuals.md b/docs/intakes/csoc-residuals.md index 7342b23..53dff53 100644 --- a/docs/intakes/csoc-residuals.md +++ b/docs/intakes/csoc-residuals.md @@ -51,7 +51,7 @@ id: CSOC-IN-0002 kind: intake title: "Add callback replay and email-only takeover regressions" lane: green -status: open +status: closed priority: low owner: coulomb-social repo: coulomb-social @@ -67,6 +67,9 @@ description: | Capture the results in docs/identity/smoke.md and close this intake. notes: | + Closed 2026-10-06 by CSOC-WP-0007. Independent application regressions in + tests/test_identity_regressions.py prove consumed callback replay rejection + before a second token exchange and same-email/different-identity separation. Narrowed 2026-08-14 after NK-WP-0025 completed live Case B. No longer blocked on the registration or mailbox path. state_hub_intake_id: "019ff532-3ade-7d75-acb7-2bbfa264fcb8" @@ -101,6 +104,8 @@ description: | Prefer a dedicated workplan if bulk import + identity map spans multiple sessions; otherwise close this intake when (1)–(3) are evidenced. notes: | + Local tooling prepared by CSOC-WP-0007; actual corpus/cohort acceptance + continues in CSOC-WP-0008-T03. Intake remains open until real receipts. Pickup: docs/session-pickup.md · rehearsal: docs/migration/rehearsal-reichelag-2026-08-13.md Code: coulomb_social/apps/spaces/import_tree.py state_hub_intake_id: "019ffac3-f894-707a-9fdb-75c8ea14497d" @@ -132,6 +137,7 @@ description: | Local default CONTENT_ROOT is var/content (gitignored). Cluster must not rely on empty ephemeral disk without a volume decision. notes: | + Live persistence/deployment acceptance continues in CSOC-WP-0008-T01. Runbook docs/deploy.md · capability model docs/capability/model-v0.yaml Pickup: docs/session-pickup.md state_hub_intake_id: "019ffac3-fc7d-7850-93b6-b0d289881ec7" diff --git a/docs/migration/corpus-import.md b/docs/migration/corpus-import.md new file mode 100644 index 0000000..dcbff1c --- /dev/null +++ b/docs/migration/corpus-import.md @@ -0,0 +1,94 @@ +# Local corpus import and recovery prerequisites + +`import_content_corpus` imports prepared ADR-0004 trees. It does not contact +Bubble, provision accounts or verify the completeness of a Bubble export. +Use private local trees and manifests outside the checkout. Never commit member +mappings, corpus content, DB dumps or backup archives. + +## Manifest + +Version 1 JSON. Tree paths are relative to the manifest directory or absolute. +Every space requires an existing active member mapped as owner. Identity is +matched by **tenant + issuer + subject**, never email or a password. + +```json +{ + "version": 1, + "spaces": [ + { + "tree": "trees/example", + "slug": "example", + "tenant": "tenant:example", + "memberships": [ + {"issuer": "https://issuer.example", "subject": "existing-subject", "role": "owner"} + ] + } + ] +} +``` + +Roles: `owner`, `member`, `viewer`. Mapping preserves these labels; current +product authorization does not yet enforce viewer write denial. Do not rely on +role labels as a privacy guarantee. Each Space gets an owner and explicit grants, +which disables the tenant-wide bootstrap visibility fallback for that tenant. +Review the effect on existing spaces before executing a real cohort migration. + +## Run + +First provision accounts through NetKingdom and have users sign in to create +local Members. Reconcile the corpus and approved grants. Pause all application +writers; run with the deployment tenant's private CONTENT_ROOT and DB settings. + +```bash +uv run manage.py import_content_corpus /private/path/manifest.json --dry-run +uv run manage.py import_content_corpus /private/path/manifest.json +# Explicit replacement only after reviewing the existing content and backup: +uv run manage.py import_content_corpus /private/path/manifest.json --replace +``` + +Preflight validates **all** entries before any writes: local pages, normalized +filenames, metadata bounds, source symlinks, source/destination overlap, +destination collisions, active identity/tenant mapping and at least one owner. +A missing owner or identity in the last tree prevents the first tree being +written. Dry-run performs that same preflight and creates no directories or rows. + +Execution stages each tree beside its destination, then applies metadata and +memberships and publishes by rename inside a DB transaction. Ordinary errors +restore the prior tree and roll back that space's DB changes. **The whole batch +is not atomic**: previously committed spaces remain if a later space fails. +Receipts give the completed count. Reconcile local state and either import a +manifest containing only remaining entries, or deliberately use `--replace`. +Replacement removes the previous content tree, including assets not in source; +it adds/updates mapped memberships and **does not revoke** other existing grants +or overwrite a pre-existing creator. Review those extra accesses explicitly. + +Writers must remain paused. A process crash, rename/DB commit race or failed +recovery filesystem operation requires manual recovery; this is not a crash-safe +journal or concurrent import protocol. Keep content-root and instance-per-client +tenant boundaries intact: filesystem paths remain slug-keyed. + +The single-tree command shares validation and staging. `--member-subject` must +resolve to exactly one tenant-local Member; use the corpus manifest when issuer +ambiguity is possible. Single-tree operator imports may still omit membership, +so they are not a replacement for approved cohort mapping. + +## Paired backup/restore protocol (acceptance still pending) + +1. Quiesce app and import writers, including Forgejo refresh consumers where + relevant. Record recovery point, exact image SHA, migration level and tenant. +2. Through the DB owner's documented credential lane, capture metadata DB; + separately snapshot/archive the **entire** CONTENT_ROOT (pages and assets, + including archived trees). Keep private data in the approved backup store. +3. Record SHA-256 file inventory, row counts and backup identities privately. + Do not log credentials, content bodies or identity lists into this repo/hub. +4. Restore DB and CONTENT_ROOT together into an isolated instance. Disable + outgoing integrations and use restore-specific settings and credentials. +5. Compare Space/Member/membership counts, file hashes, page list/render, visual + assets, copied/moved content and denied reads by an ungranted member. Measure + achieved recovery time and recovery-point age; record the non-secret receipts. +6. Resume writers only after accepted recovery checks. If import fails, restore + both stores from that same recovery point rather than only rolling back DB. + +This is an operational protocol, **not** evidence that scheduled backup or a +restore drill exists. CSOC-WP-0008-T02 owns automation and measured acceptance; +T01 owns storage admission, T03 the actual private cohort rehearsal. diff --git a/docs/session-pickup.md b/docs/session-pickup.md index c38f433..40ae5fc 100644 --- a/docs/session-pickup.md +++ b/docs/session-pickup.md @@ -1,3 +1,18 @@ +# Current pickup — 2026-10-06 + +WP-0007 audits scope and implements validated corpus import with explicit +identity membership mapping. WP-0008 tracks live persistence/restore, actual +cohort migration and access/cutover acceptance. Public registration residual +CSOC-IN-0001 is closed (2026-08-14 evidence); CSOC-IN-0002 is closed with independent replay +and same-email identity separation regressions. CSOC-IN-0003/0004 remain open. + +Read SCOPE.md, history/20261005T223104Z-scope-intent-assessment.md and +docs/migration/corpus-import.md. Run the tests and import dry-run before any +private corpus work. The older pickup below is historical and does not override +current work records or authorize deployment. + +--- + # Session pickup — coulomb-social **Last closed:** 2026-08-13 diff --git a/history/20261005T223104Z-scope-intent-assessment.md b/history/20261005T223104Z-scope-intent-assessment.md new file mode 100644 index 0000000..e70fff3 --- /dev/null +++ b/history/20261005T223104Z-scope-intent-assessment.md @@ -0,0 +1,105 @@ +# Scope against intent — 2026-10-06 + +Timestamp prefix is UTC (2026-10-05 22:31:04); review date is Europe/Berlin. +Evidence: tracked application source, tests, workplans, ADRs and historical +migration/identity records. No live production or private-corpus execution. + +## Finding + +The repository is an implemented Django identity + spaces/pages application, +not a scaffold and not yet a complete co-creation marketplace. The previous +SCOPE under-reported member CRUD/PageOps and finished Bubble assessment work, +and incorrectly described public registration as still platform-blocked. + +INTENT's broad parity language must be read with the accepted 2026-08-12 +stage-1 cut: partial rebuild for productive transfer, broader social/marketplace +features later. This review does not change that human decision or authorize +retiring Bubble. + +## Intent success criteria against evidence + +| INTENT criterion | Assessment | Evidence / gap | +|---|---|---| +| Inventory all Bubble surfaces/workflows/data/integrations | Partial | `docs/bubble-inventory/`, client-package capability map and design crawl. Backend workflows/privacy and full editor schema remain lower-confidence; not a full formal export. | +| Tenant-keyed relational mapping and rehearsal import | Partial | Member/Space/membership tables, schema sketches, 2026-08-13 reichelag rehearsal (30 pages, 29 assets). Full 71-space corpus and member mapping not executed; filesystem content uses slug, not tenant keys. | +| Human-gated feature cut | Met as a recorded decision | `docs/decisions/2026-08-12-feature-cut-stage-1.md` accepts onboarding + spaces/pages/copy/transfer and defers broader parity. | +| Standalone v1 under service contract | Partial | Docker, migrations, health, OIDC/user-engine, deployment lane and demand evidence exist. CONTENT_ROOT deployment, paired restore, export/delete, ops/cost/provenance acceptance are not comprehensively evidenced here. | +| Members/content cut over without Bubble dependence | Not accepted yet | Core code works without Bubble; historical Case B registration closed CSOC-IN-0001. Live PageOps acceptance, full cohort/content transfer, recovery and explicit cutover evidence remain outstanding. | + +## Ranked gaps to address + +1. **Persistent, accepted content deployment and recoverability (high).** The + latest product capability is not proven on the app host. Markdown on an + ephemeral disk would undermine productive transfer; a DB-only backup loses + page bodies. Existing CSOC-IN-0004 is the deployment handoff. Require pinned + image/migration, volume decision, paired DB/content backup and isolated + restore with membership/render checks before acceptance. +2. **Safe multi-space and member migration (high).** One successful tree is not + a community transfer. Existing CSOC-IN-0003 covers 70 remaining spaces and + identity mapping; thin exports and parent/privacy lossiness need disposition. + The old importer wrote content before DB completion, dry-run skipped member + validation, and subject lookup could fall back to another tenant. Implement + local corpus preflight/mapping and rollback before touching private content. +3. **Access fidelity and portability (high).** Tenant-filtered rows are useful, + but slug-keyed filesystem paths, tenant-wide bootstrap visibility and roles + without viewer/editor enforcement do not establish privacy parity. Exact + Bubble privacy/member mapping, membership revocation and app-local member + export/delete remain acceptance gaps. Do not declare shared-root multi-tenant + SaaS supported. +4. **Capability evidence accuracy (medium).** Stage-1 cutover tables still + described member CRUD, Visual and copy/move as todo despite WP-0006. The + machine model used `shipped` without separating checkout and deployment; + session pickup also retained a closed registration blocker. Reconcile these + now so future work starts from actual capability and live evidence. +5. **Content integrity/versioning (medium).** Local PageOps lacks automatic git + history, arbitrary asset/link transfer and concurrent-edit conflict control. + Visual fields alone do not deliver an upload/asset-serving workflow. Outline, + live includes and lazy projections remain designed, not implemented. +6. **Full co-creation marketplace (later).** Discovery/matching, favorites, + social graph, payments and real-time editing are meaningful differences from + broad INTENT, but the accepted cut deliberately excludes them from stage-1 + acceptance. Implementing them now would miss the nearer transfer blockers. + +## Implemented response — CSOC-WP-0007 + +- Rewrite SCOPE with checkout capabilities, practical limits and dated live + evidence; reconcile stage-1 status and pickup pointers. +- Shared single-space import validation rejects cross-tenant member/slug use, + source symlinks, overlapping roots and invalid pages/metadata. Subject lookup + requires exactly one tenant-local member. Content is staged, published beside + the destination and restored on ordinary errors while DB writes roll back. +- New `import_content_corpus` preflights all entries before writing, maps active + members by tenant+issuer+subject, requires a mapped owner, rejects duplicate + destinations and supports read-only dry-run and explicit replacement. +- Documentation states additive membership semantics and per-space restart + behavior; private trees/manifests remain outside git. Regression checks use + synthetic data, not the real corpus. + +These changes close local preparation gaps. They do not certify corpus parity, +production rollout, crash atomicity, exact ACL replacement or restore success. + +## Live follow-up + +[CSOC-WP-0008](../workplans/CSOC-WP-0008-production-transfer-acceptance.md) +tracks persistence/live acceptance, paired recovery, real cohort mapping, +privacy/access portability and founder cutover. It takes over the execution of +CSOC-IN-0003/0004 without closing either on the strength of local tooling. +CSOC-IN-0002 is closed with independent callback replay and same-email identity +separation regressions in tests/test_identity_regressions.py. No residual is hidden +only in this report or SCOPE. + +## Validation + +Final test and lint results are recorded below at session close. Repository-wide +lint initially exposed pre-existing unused imports/variables, import ordering +and long lines; changed-code lint is checked separately from that baseline. + + +Final verification: **72 tests passed** (synthetic corpus, no private data); +Ruff passes all changed application/test files; Django migration check reports +no changes. Full repository Ruff retains **19 pre-existing findings** in +identity, members/admin/migrations, older PageOps/Forgejo code and tests. +WP-0008-T07 tracks that quality debt. Test warnings are Authlib jose deprecation +and missing collected staticfiles in the test environment. + +Production acceptance decision drafted in State Hub: `186be722-e2c7-4109-a1e6-d120fe8fca5b` (pending; no live change approval inferred). diff --git a/tests/test_identity_regressions.py b/tests/test_identity_regressions.py new file mode 100644 index 0000000..caa8150 --- /dev/null +++ b/tests/test_identity_regressions.py @@ -0,0 +1,57 @@ +"""Independent negative checks requested by CSOC-IN-0002.""" + +from unittest.mock import patch + +import pytest +from django.urls import reverse + +from coulomb_social.apps.identity import views +from coulomb_social.apps.members.models import Member + + +@pytest.mark.django_db +def test_consumed_callback_state_cannot_be_replayed(client, settings): + settings.OIDC_ENABLED = True + settings.OIDC_ISSUER = "https://issuer.test" + settings.OIDC_CLIENT_ID = "test-client" + settings.OIDC_REDIRECT_URI = "https://app.test/auth/callback/" + session = client.session + session[views.SESSION_OIDC_STATE] = "single-use-state" + session[views.SESSION_OIDC_VERIFIER] = "verifier" + session.save() + with ( + patch.object(views.oidc, "exchange_code", return_value={"access_token": "synthetic"}) + as exchange, + patch.object(views.oidc, "fetch_userinfo", return_value={}), + patch.object(views.oidc, "claims_from_token_response", return_value={"sub": "new"}), + patch.object(views, "establish_session") as establish, + ): + callback = reverse("identity:callback") + "?code=code&state=single-use-state" + assert client.get(callback).status_code == 302 + assert client.get(callback).status_code == 400 + assert exchange.call_count == 1 + assert establish.call_count == 1 + assert views.SESSION_OIDC_STATE not in client.session + assert views.SESSION_OIDC_VERIFIER not in client.session + + +@pytest.mark.django_db +def test_same_email_different_identity_cannot_take_over_member(client, settings): + settings.DEBUG = True + settings.OIDC_ENABLED = False + login = reverse("identity:dev_login") + client.post(login, {"issuer": "https://first.test", "subject": "original", + "email": "shared@example.test", "name": "Original"}) + original = Member.objects.get(subject="original") + original_user_id = original.user_id + client.post(reverse("identity:logout")) + client.post(login, {"issuer": "https://second.test", "subject": "different", + "email": "shared@example.test", "name": "Different"}) + original.refresh_from_db() + other = Member.objects.get(subject="different") + assert original.user_id == original_user_id + assert original.display_name == "Original" + assert other.pk != original.pk + assert other.user_id != original.user_id + assert str(client.session["_auth_user_id"]) == str(other.user_id) + assert Member.objects.count() == 2 diff --git a/tests/test_import_corpus.py b/tests/test_import_corpus.py new file mode 100644 index 0000000..e2190f4 --- /dev/null +++ b/tests/test_import_corpus.py @@ -0,0 +1,79 @@ +"""Corpus transfer acceptance: preflight, identity mapping and restart policy.""" + +import json + +import pytest +from django.core.management import call_command +from django.core.management.base import CommandError + +from coulomb_social.apps.members.models import Member, User +from coulomb_social.apps.spaces import pageops +from coulomb_social.apps.spaces.models import Space, SpaceMembership + + +@pytest.fixture +def corpus(tmp_path, settings): + settings.CONTENT_ROOT = str(tmp_path / "content") + member = Member.objects.create( + tenant_id="tenant:a", user=User.objects.create_user(username="owner"), + user_engine_user_id="owner", issuer="https://issuer.test", subject="owner", + ) + entries = [] + for slug in ["one", "two"]: + pages = tmp_path / "export" / slug / "pages" + pages.mkdir(parents=True) + (pages / "index.md").write_text(f"# {slug}\n") + entries.append({"tree": f"export/{slug}", "tenant": "tenant:a", + "memberships": [{"issuer": member.issuer, + "subject": member.subject, "role": "owner"}]}) + manifest = tmp_path / "manifest.json" + manifest.write_text(json.dumps({"version": 1, "spaces": entries})) + return manifest, entries, member + + +@pytest.mark.django_db +def test_corpus_dry_run_and_import(corpus): + manifest, _, member = corpus + call_command("import_content_corpus", str(manifest), dry_run=True) + assert not Space.objects.exists() + assert not pageops.content_root().exists() + call_command("import_content_corpus", str(manifest)) + assert Space.objects.count() == 2 + assert SpaceMembership.objects.filter(member=member, role="owner").count() == 2 + assert pageops.read_page("two", "index").body == "# two\n" + with pytest.raises(CommandError, match="already exists"): + call_command("import_content_corpus", str(manifest)) + call_command("import_content_corpus", str(manifest), replace=True) + assert Space.objects.count() == 2 + assert SpaceMembership.objects.count() == 2 + + +@pytest.mark.django_db +@pytest.mark.parametrize("failure", ["identity", "tenant", "duplicate", "owner", "missing"]) +def test_entire_corpus_preflight_before_writes(corpus, failure): + manifest, entries, _ = corpus + if failure == "identity": + entries[1]["memberships"][0]["subject"] = "missing" + elif failure == "tenant": + entries[1]["tenant"] = "tenant:b" + elif failure == "duplicate": + entries[1]["slug"] = "one" + elif failure == "owner": + entries[1]["memberships"][0]["role"] = "viewer" + else: + entries[1]["tree"] = "missing" + manifest.write_text(json.dumps({"version": 1, "spaces": entries})) + with pytest.raises(CommandError): + call_command("import_content_corpus", str(manifest)) + assert not Space.objects.exists() + assert not pageops.content_root().exists() + + +@pytest.mark.django_db +@pytest.mark.parametrize("data", [[], {"version": 1, "spaces": []}, + {"version": 1, "spaces": [42]}]) +def test_invalid_manifest_shape_is_command_error(tmp_path, data): + manifest = tmp_path / "manifest.json" + manifest.write_text(json.dumps(data)) + with pytest.raises(CommandError, match="preflight"): + call_command("import_content_corpus", str(manifest), dry_run=True) diff --git a/tests/test_import_tree.py b/tests/test_import_tree.py index f8e74b5..e831997 100644 --- a/tests/test_import_tree.py +++ b/tests/test_import_tree.py @@ -119,3 +119,139 @@ def test_import_content_tree_command(content_root, tmp_path, capsys): space = Space.objects.get(slug="cmd-space", tenant_id="tenant:cmd") assert space.title == "Sample Space" assert pageops.read_page("cmd-space", "index").exists + + +@pytest.mark.django_db +def test_import_rejects_other_tenant_before_writes(content_root, tmp_path): + tree = _write_sample_tree(tmp_path / "export") + member = Member.objects.create( + tenant_id="tenant:b", user=User.objects.create_user(username="other"), + user_engine_user_id="other", issuer="https://issuer.test", subject="other", + ) + with pytest.raises(pageops.PageOpsError, match="another tenant"): + import_space_tree(tree, tenant_id="tenant:a", member=member) + assert not content_root.exists() + assert not Space.objects.exists() + + +@pytest.mark.django_db +def test_import_rejects_cross_tenant_slug(content_root, tmp_path): + tree = _write_sample_tree(tmp_path / "export") + import_space_tree(tree, tenant_id="tenant:a") + before = pageops.page_file("sample-space", "index").read_bytes() + with pytest.raises(pageops.PageOpsError, match="another tenant"): + import_space_tree(tree, tenant_id="tenant:b", replace=True) + assert pageops.page_file("sample-space", "index").read_bytes() == before + assert not Space.objects.filter(tenant_id="tenant:b").exists() + + +@pytest.mark.django_db +def test_import_rejects_symlink_in_dry_run(content_root, tmp_path): + from django.core.management.base import CommandError + + tree = _write_sample_tree(tmp_path / "export") + (tree / "assets" / "outside").symlink_to(tmp_path) + with pytest.raises(CommandError, match="symlink"): + call_command("import_content_tree", str(tree), "--dry-run") + assert not content_root.exists() + + +@pytest.mark.django_db +def test_import_rolls_back_content_on_publish_error(content_root, tmp_path, monkeypatch): + tree = _write_sample_tree(tmp_path / "export") + import_space_tree(tree, tenant_id="tenant:a") + path = pageops.page_file("sample-space", "index") + original = path.read_bytes() + (tree / "pages" / "index.md").write_text('# Changed\n') + real_rename = Path.rename + + def fail_publish(self, target): + if self.name == "staged": + raise OSError("simulated publish failure") + return real_rename(self, target) + + monkeypatch.setattr(Path, "rename", fail_publish) + with pytest.raises(OSError, match="simulated"): + import_space_tree(tree, tenant_id="tenant:a", replace=True) + assert path.read_bytes() == original + assert Space.objects.get(slug="sample-space").title == "Sample Space" + + +@pytest.mark.django_db +def test_single_import_subject_cannot_fall_back_to_other_tenant(content_root, tmp_path): + from django.core.management.base import CommandError + + tree = _write_sample_tree(tmp_path / "export") + Member.objects.create( + tenant_id="tenant:b", user=User.objects.create_user(username="foreign"), + user_engine_user_id="foreign", issuer="https://issuer.test", subject="foreign", + ) + with pytest.raises(CommandError, match="exactly one"): + call_command("import_content_tree", str(tree), tenant="tenant:a", + member_subject="foreign", dry_run=True) + assert not content_root.exists() + + +@pytest.mark.django_db(transaction=True) +def test_import_restores_tree_when_database_commit_fails(content_root, tmp_path, monkeypatch): + from django.db import DatabaseError, connection + + tree = _write_sample_tree(tmp_path / "export") + import_space_tree(tree, tenant_id="tenant:a") + path = pageops.page_file("sample-space", "index") + original = path.read_bytes() + (tree / "pages" / "index.md").write_text('# Changed\n') + + def fail_commit(): + raise DatabaseError("simulated database commit failure") + + monkeypatch.setattr(connection, "commit", fail_commit) + with pytest.raises(DatabaseError): + import_space_tree(tree, tenant_id="tenant:a", replace=True) + assert path.read_bytes() == original + assert Space.objects.get(slug="sample-space").title == "Sample Space" + + +@pytest.mark.django_db +@pytest.mark.parametrize("failure", ["overlap", "page-name", "metadata"]) +def test_shared_validation_rejects_bad_trees(content_root, tmp_path, failure): + from django.core.management.base import CommandError + + tree = _write_sample_tree(tmp_path / "export") + if failure == "overlap": + import shutil + + content_root.mkdir() + shutil.copytree(tree, content_root / "source") + tree = content_root / "source" + elif failure == "page-name": + (tree / "pages" / "notes.md").rename(tree / "pages" / "Bad Name.md") + else: + (tree / "pages" / "index.md").write_text( + '---\ntitle: "' + ('x' * 256) + '"\n---\n\nbody\n' + ) + with pytest.raises(CommandError): + call_command("import_content_tree", str(tree), dry_run=True) + assert not Space.objects.exists() + assert not (content_root / "spaces").exists() + + +@pytest.mark.django_db +def test_failed_restore_preserves_previous_tree(content_root, tmp_path, monkeypatch): + tree = _write_sample_tree(tmp_path / "export") + import_space_tree(tree, tenant_id="tenant:a") + original = pageops.page_file("sample-space", "index").read_bytes() + real_rename = Path.rename + + def fail_publish_and_restore(self, target): + if self.name in {"staged", "previous"}: + raise OSError("simulated filesystem failure") + return real_rename(self, target) + + monkeypatch.setattr(Path, "rename", fail_publish_and_restore) + with pytest.raises(pageops.PageOpsError, match="retained"): + import_space_tree(tree, tenant_id="tenant:a", replace=True) + saved = list((content_root / "spaces").glob(".import-*/previous/pages/index.md")) + assert len(saved) == 1 + assert saved[0].read_bytes() == original + assert Space.objects.get(slug="sample-space").title == "Sample Space" diff --git a/workplans/CSOC-WP-0007-safe-transfer-readiness.md b/workplans/CSOC-WP-0007-safe-transfer-readiness.md new file mode 100644 index 0000000..b876c03 --- /dev/null +++ b/workplans/CSOC-WP-0007-safe-transfer-readiness.md @@ -0,0 +1,89 @@ +--- +id: CSOC-WP-0007 +type: workplan +title: "Evidence-based scope and safe corpus transfer tooling" +domain: communication +repo: coulomb-social +status: finished +owner: codex +topic_slug: coulomb-social +created: "2026-10-06" +updated: "2026-10-06" +state_hub_workstream_id: "d52a146e-5f5c-5107-9755-0eb2c9ad50a8" +--- + +# CSOC-WP-0007 — Safe transfer readiness + +Prioritize the accepted stage-1 cut rather than inventing marketplace features. +Implement local migration prerequisites; deployment, private corpus execution, +restore acceptance and founder cutover remain separately tracked work. + +## T01 — Audit scope against intent + +```task +id: CSOC-WP-0007-T01 +status: done +priority: high +state_hub_task_id: "0d6eb23c-a451-531a-b66b-e48c46ab64d4" +``` + +Update SCOPE from source and dated evidence. File timestamped assessment in +history; reconcile stale stage-1 status and identify outstanding contract gates. + +## T02 — Validate single-space import before writes + +```task +id: CSOC-WP-0007-T02 +status: done +priority: high +state_hub_task_id: "1e4a25a4-79c6-5a3d-abeb-1498de8f27cf" +``` + +Share real and dry-run validation; reject cross-tenant members, ambiguous subject +lookup, cross-tenant filesystem slug collisions, symlinks, malformed page names, +and overlapping source/destination. Stage content and restore the previous tree +if metadata or membership writes fail. No private corpus in git. + +## T03 — Bulk corpus import with explicit identity mapping + +```task +id: CSOC-WP-0007-T03 +status: done +priority: high +state_hub_task_id: "9973fbde-30cb-53c3-a899-396f23e97f24" +``` + +JSON manifest names local trees and existing tenant+issuer+subject memberships. +Validate the whole manifest before writes, require a mapped owner, refuse +unresolved identities and duplicate destination slugs. Dry-run is read-only. +Per-space atomic imports with explicit restart/replace semantics and counts. + +## T04 — Verify and hand off production gates + +```task +id: CSOC-WP-0007-T04 +status: done +priority: high +state_hub_task_id: "0a7c06f9-fe10-5b3b-ab08-2143ae1a875f" +``` + +Close CSOC-IN-0002 with independent callback replay/email identity regressions. +Regression-test migration rejection, filesystem rollback, multi-space import, +identity mapping and dry-run. Run full tests/lint/migration consistency. Document +migration use and paired DB/content recovery requirements. Create a successor +workplan for live persistence, restore, cohort migration and cutover acceptance +before finishing. Commit and sync all implementation and work records. + +## Completion evidence + +- SCOPE and history/20261005T223104Z-scope-intent-assessment.md reflect source, + historical deployment and accepted cut; capability/pickup docs reconciled. +- Single import and new corpus command implemented with tenant/member preflight, + per-space staging/DB rollback and retained recovery tree on restore failure. +- CSOC-IN-0002 closed by independent identity regressions. +- Full suite: **72 passed**. Changed Python/test files pass Ruff; Django reports + no missing migrations. Full Ruff retains 19 pre-existing findings, assigned + to CSOC-WP-0008-T07 rather than hidden as passing. +- Residuals are live successor **CSOC-WP-0008** (origin: residual, origin_ref: + CSOC-WP-0007), including existing CSOC-IN-0003/0004 acceptance. No private corpus + or live deployment was executed by this workplan.