Audit actual scope and implement safe corpus transfer tooling
Assistant: codex Assistant-Model: gpt-6.1-sol Assistant-Session: 01a10e2d-a5ef-7e92-b03d-b1694e915680
This commit is contained in:
parent
81c6663012
commit
57a329949f
17 changed files with 988 additions and 290 deletions
163
SCOPE.md
163
SCOPE.md
|
|
@ -1,109 +1,86 @@
|
|||
# SCOPE
|
||||
# SCOPE — implemented capability and boundaries
|
||||
|
||||
> This file helps you quickly understand what this repository is about,
|
||||
> when it is relevant, and when it is not.
|
||||
> It is intentionally lightweight and may be incomplete.
|
||||
Reviewed 2026-10-06 against application source, tests and dated repository
|
||||
records. This describes what the checkout can do; historical live evidence is
|
||||
identified separately. It is not a current production acceptance certificate.
|
||||
|
||||
---
|
||||
## What this repository is
|
||||
|
||||
## One-liner
|
||||
A standalone Django co-creation application rebuilding coulomb.social on
|
||||
Railiance / NetKingdom / HelixForge. Today it implements an authenticated spaces
|
||||
and markdown-pages product plus local content migration tools. It is not yet
|
||||
the complete social network or marketplace described by INTENT.md.
|
||||
|
||||
Reimplementation of the coulomb.social co-creation platform as a standalone Railiance / NetKingdom / HelixForge application, exiting the stalled bubble.io stack.
|
||||
The accepted [stage-1 cut](docs/decisions/2026-08-12-feature-cut-stage-1.md)
|
||||
prioritizes onboarding and productive spaces/pages transfer; marketplace,
|
||||
favorites, payments and real-time collaboration are later capabilities.
|
||||
|
||||
---
|
||||
## Implemented in this checkout
|
||||
|
||||
## Core Idea
|
||||
| Capability | Actual behavior and evidence |
|
||||
|---|---|
|
||||
| Identity | OIDC/PKCE login, logout, JIT local Member linked to user-engine; issuer+subject identity, session diagnostics and assurance handling. Development claims login is restricted to DEBUG with OIDC disabled. `apps/identity/`, identity tests. |
|
||||
| Registration entry | Configured NetKingdom registration redirect; this app does not issue passwords or send verification mail. CSOC-IN-0001 is closed with 2026-08-14 Case B evidence. |
|
||||
| Spaces | Tenant-filtered list/detail, member-facing create/update/archive, Title, Abstractor (`description`) and Visual metadata. `apps/spaces/services.py`, `tests/test_space_crud.py`. |
|
||||
| Pages | Markdown/frontmatter list, render, create/update/delete, independent copy and transfer with source/destination access checks. Index cannot be deleted or transferred. `pageops.py`, PageOps tests. |
|
||||
| Content sources | Local `CONTENT_ROOT/spaces/<slug>/pages` and assets; optional Forgejo read/edit fallback, cache refresh and push webhook. Markdown is sanitized on render. Local writes do not automatically commit/push to Forgejo. |
|
||||
| Import | `import_content_tree` for one ADR-0004 tree; `import_content_corpus` for a preflighted JSON batch with explicit existing tenant+issuer+subject membership mappings. Dry-run, replacement opt-in, per-space staging/rollback and aggregate counts. Synthetic multi-space regression coverage; one historical private-space rehearsal. |
|
||||
| Delivery preparation | Docker/Gunicorn packaging, SQLite development / PostgreSQL URL support, migrations, health endpoint and deployment/runbook docs. Actual Helm values and cluster hosting belong to railiance-apps. |
|
||||
| Evidence and coordination | Bubble surface/capability inventory, authenticated export scripts, design extraction, schema sketch, accepted cut decision, resource-demand/SLO/labor evidence and file-backed State Hub work records. |
|
||||
|
||||
coulomb.social connects people around shared projects and complementary capabilities. Development on bubble.io has stalled. This repo owns the full product rebuild — UI, content, data model, and migration — so the platform can evolve with the rest of the Coulomb agentic fleet under the business-app service contract.
|
||||
Source paths above are under `coulomb_social/`. Member and Space metadata are
|
||||
tenant-keyed, but filesystem paths are **slug-keyed**. Deployment is
|
||||
instance-per-client: do not reuse a content root across tenants. Normal creation
|
||||
and imports reject existing cross-tenant slug collisions; this is not a full
|
||||
filesystem tenant namespace or a database-wide concurrency guarantee.
|
||||
|
||||
---
|
||||
## Practical limits
|
||||
|
||||
## In Scope
|
||||
- Space visibility has a tenant-wide bootstrap fallback until any tenant
|
||||
memberships exist. Membership roles are stored but viewer/editor enforcement
|
||||
and Bubble privacy parity are incomplete. Imported mappings add/update grants;
|
||||
replacement does not revoke old grants. Review access before a real cohort import.
|
||||
- Copy/transfer handles relative Visual files, not arbitrary body-linked assets,
|
||||
incoming page links, graph identity or Bubble parent/chunk semantics. Visual
|
||||
metadata and copied files do not establish an authenticated asset-serving or
|
||||
upload pipeline.
|
||||
- Import requires local prepared trees and existing active identities. It does
|
||||
not create platform accounts, import passwords, scrape missing Bubble content,
|
||||
or certify corpus completeness. It runs with writers paused; the batch is
|
||||
restartable per space, not globally atomic or crash-transactional.
|
||||
- No app-local member export/delete product, paired DB/content restore drill,
|
||||
automatic content git versioning, outline/include/projection authoring or
|
||||
marketplace/social/payment product is implemented here.
|
||||
- Historical health/auth evidence does not prove current image, persistence,
|
||||
runtime SLOs, backup success or complete service-contract compliance.
|
||||
|
||||
- Product intent, scope, workplans, and ADRs for the coulomb.social rebuild
|
||||
- Inventory of the existing bubble.io app (pages, workflows, data types, integrations, content)
|
||||
- Data export, relational schema mapping, and migration rehearsal
|
||||
- Feature-cut decisions (rebuild-as-is vs reduced v1) and acceptance criteria
|
||||
- Application implementation: UI parity, domain model, app-local accounts, APIs
|
||||
- Packaging for the business delivery lane (commit-SHA images, railiance-apps release values)
|
||||
- Tenant-keyed data model from the first migration; member account export/delete
|
||||
- Ops/runbook stubs required by the business-app service contract (§§3–6)
|
||||
- State Hub workplans, progress events, and classification for this repo
|
||||
## Recorded deployment state
|
||||
|
||||
---
|
||||
The host strategy is rebuild on `app.coulomb.social` with Bubble on apex until
|
||||
an explicit cutover. Historical identity/app-shell evidence is in
|
||||
[identity smoke](docs/identity/smoke.md). Public registration was evidenced on
|
||||
2026-08-14. **CSOC-IN-0004 remains open** for PageOps deployment, migration,
|
||||
persistent CONTENT_ROOT and live smoke. No fresh live deployment verification
|
||||
was performed in this review. All six original numbered workplans are finished;
|
||||
their residual intakes remain the source of unfinished work.
|
||||
|
||||
## Out of Scope
|
||||
## Ownership boundaries
|
||||
|
||||
- Remaining on bubble.io as the primary development platform
|
||||
- Owning Railiance cluster, platform, or app-chart infrastructure code (`railiance-*`)
|
||||
- Owning NetKingdom identity platform implementation (Keycloak/OpenBao) — only the app-local auth boundary and future OIDC switch point live here
|
||||
- Owning HelixForge standards or capability registry implementation
|
||||
- Core Hub embedding or treating coulomb.social as a hub extension
|
||||
- Fleet self-improvement loops (owner: `coulomb-loop`)
|
||||
- Serving paying external multi-tenant SaaS before the service-contract preconditions are met
|
||||
This repo owns product behavior, local identity integration, application models,
|
||||
content/migration tooling, packaging and acceptance evidence. It does not own
|
||||
Railiance infrastructure/charts, NetKingdom issuer/mail/secrets services,
|
||||
HelixForge standards, or fleet improvement loops. Core Hub embedding and Bubble
|
||||
as the long-term development surface remain out of scope.
|
||||
|
||||
---
|
||||
## Next work and orientation
|
||||
|
||||
## Relevant When
|
||||
Read [INTENT.md](INTENT.md), [the timestamped assessment](history/20261005T223104Z-scope-intent-assessment.md),
|
||||
[CSOC-WP-0007](workplans/CSOC-WP-0007-safe-transfer-readiness.md), and
|
||||
[CSOC-WP-0008](workplans/CSOC-WP-0008-production-transfer-acceptance.md).
|
||||
The local transfer tool closes a preparation gap; persistence/restore, private
|
||||
cohort migration, access acceptance and founder cutover are live successor work.
|
||||
|
||||
- Planning or implementing the bubble.io exit and product rebuild
|
||||
- Mapping coulomb.social features, content, or data into code-owned artifacts
|
||||
- Packaging or deploying the coulomb.social app via the business delivery lane
|
||||
- Checking tenancy, identity, backup, or cost-attribution obligations for this app
|
||||
|
||||
---
|
||||
|
||||
## Not Relevant When
|
||||
|
||||
- You need cluster or mesh infrastructure changes → use the relevant `railiance-*` repo
|
||||
- You need platform SSO/secret custody work → use `net-kingdom` / OpenBao workplans
|
||||
- You need fleet kaizen loop operation → use `coulomb-loop`
|
||||
- You need a reference business-app implementation pattern → start with `vergabe-teilnahme`
|
||||
- You only need the historical seed note → `the-custodian/docs/coulomb-social-rebuild-seed.md` (pointer; work originates here)
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
- Status: **parallel hosts** — identity + app shell + Forgejo spaces live on app.*
|
||||
- **`https://app.coulomb.social`**: Railiance rebuild (OIDC, app home, spaces markdown)
|
||||
- **`https://coulomb.social`**: Bubble.io full product until cutover decision
|
||||
- Finished product path: **`CSOC-WP-0004`** (spaces + Forgejo markdown + runbook)
|
||||
- Finished identity path: **`CSOC-WP-0002`**, **`CSOC-WP-0003`** (Case A; Case B residual intakes)
|
||||
- Content ADR: `docs/adr/ADR-0002-space-content-forgejo-markdown.md`
|
||||
- Operator: `docs/deploy.md` runbook; residuals `docs/intakes/csoc-residuals.md`
|
||||
- Finished resource evidence: **`CSOC-WP-0005`** → `docs/resource-evidence/`
|
||||
- Bubble exit: **`CSOC-WP-0001`** — inventory done; stage-1 feature cut accepted (partial rebuild); export still needs Bubble editor
|
||||
- Capability model: `docs/capability/` — stage 1 must = onboarding + spaces/pages (Title/Abstract/Visual) + page copy/transfer
|
||||
- Open residuals: public registration (`CSOC-IN-0001`/`0002`, NK-WP-0025)
|
||||
|
||||
---
|
||||
|
||||
## How It Fits
|
||||
|
||||
- **Upstream decisions:** business-app service contract; DR-1 C / DR-2 B / DR-3 A
|
||||
- **Delivery lane:** `vergabe-teilnahme` (pattern) → this repo (product) → `railiance-apps` (release values) → railiance01
|
||||
- **Platform peers:** `railiance-*`, `net-kingdom`, `helix-forge`, `ops-hub`, `fin-hub`
|
||||
- **Coordination:** State Hub topic `coulomb-social` (domain `communication`); related engagement repo `coulomb-loop`
|
||||
|
||||
---
|
||||
|
||||
## Getting Oriented
|
||||
|
||||
- Start with: `INTENT.md`, `GOAL.md`, `.custodian-brief.md`
|
||||
- Key paths: `workplans/`, `docs/` (as they appear), application source (once scaffolded)
|
||||
- First workplan: `workplans/CSOC-WP-0001-bubble-io-exit-assessment.md`
|
||||
- Contract: `the-custodian/canon/standards/business-app-service-contract_v0.1.md`
|
||||
|
||||
---
|
||||
|
||||
## Related Repositories
|
||||
|
||||
| Repo | Relationship |
|
||||
|------|----------------|
|
||||
| `vergabe-teilnahme` | Reference business-app delivery lane |
|
||||
| `railiance-apps` | Helm/release values for deploy |
|
||||
| `railiance-forge` / `helix-forge` | Forge and capability patterns |
|
||||
| `net-kingdom` | Future OIDC / platform identity |
|
||||
| `coulomb-loop` | Fleet improvement loops (same State Hub topic) |
|
||||
| `the-custodian` / `state-hub` | Coordination, standards, seed history |
|
||||
Development: `README.md`, `docs/dev.md`, `make test`, `make lint`.
|
||||
Content architecture: ADR-0003/0004 and `docs/architecture/`.
|
||||
Migration: `docs/migration/`, including `corpus-import.md`.
|
||||
Delivery: `docs/deploy.md`; residual records: `docs/intakes/csoc-residuals.md`.
|
||||
|
|
|
|||
|
|
@ -14,6 +14,8 @@
|
|||
| workplan | CSOC-WP-0004 | finished | — | workplans/CSOC-WP-0004-app-shell-and-space-content.md |
|
||||
| workplan | CSOC-WP-0005 | finished | — | workplans/CSOC-WP-0005-resource-demand-and-cost-evidence.md |
|
||||
| workplan | CSOC-WP-0006 | finished | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md |
|
||||
| workplan | CSOC-WP-0007 | finished | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md |
|
||||
| workplan | CSOC-WP-0008 | ready | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| task | CSOC-WP-0001-T01 | done | — | workplans/CSOC-WP-0001-bubble-io-exit-assessment.md |
|
||||
| task | CSOC-WP-0001-T02 | done | — | workplans/CSOC-WP-0001-bubble-io-exit-assessment.md |
|
||||
| task | CSOC-WP-0001-T03 | done | — | workplans/CSOC-WP-0001-bubble-io-exit-assessment.md |
|
||||
|
|
@ -46,7 +48,18 @@
|
|||
| task | CSOC-WP-0006-T04 | done | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md |
|
||||
| task | CSOC-WP-0006-T05 | done | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md |
|
||||
| task | CSOC-WP-0006-T06 | done | — | workplans/CSOC-WP-0006-stage-1-space-page-capabilities.md |
|
||||
| task | CSOC-WP-0007-T01 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md |
|
||||
| task | CSOC-WP-0007-T02 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md |
|
||||
| task | CSOC-WP-0007-T03 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md |
|
||||
| task | CSOC-WP-0007-T04 | done | — | workplans/CSOC-WP-0007-safe-transfer-readiness.md |
|
||||
| task | CSOC-WP-0008-T01 | wait | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| task | CSOC-WP-0008-T02 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| task | CSOC-WP-0008-T03 | wait | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| task | CSOC-WP-0008-T04 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| task | CSOC-WP-0008-T05 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| task | CSOC-WP-0008-T06 | wait | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| task | CSOC-WP-0008-T07 | todo | — | workplans/CSOC-WP-0008-production-transfer-acceptance.md |
|
||||
| intake | CSOC-IN-0001 | closed | blue | docs/intakes/csoc-residuals.md |
|
||||
| intake | CSOC-IN-0002 | open | green | docs/intakes/csoc-residuals.md |
|
||||
| intake | CSOC-IN-0002 | closed | green | docs/intakes/csoc-residuals.md |
|
||||
| intake | CSOC-IN-0003 | open | blue | docs/intakes/csoc-residuals.md |
|
||||
| intake | CSOC-IN-0004 | open | green | docs/intakes/csoc-residuals.md |
|
||||
|
|
|
|||
|
|
@ -1,32 +1,19 @@
|
|||
"""Import an ADR-0004 space tree into the content plane + Space index.
|
||||
|
||||
Source layout (export or hand-authored):
|
||||
|
||||
<tree>/
|
||||
pages/*.md
|
||||
assets/* (optional)
|
||||
|
||||
Does not fetch Bubble; only copies local files. Bodies may be private —
|
||||
callers should keep source trees out of git.
|
||||
"""
|
||||
"""Validated local ADR-0004 import; never fetches Bubble or creates identities."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from django.db import transaction
|
||||
from django.utils.text import slugify
|
||||
|
||||
from coulomb_social.apps.members.models import Member
|
||||
|
||||
from . import pageops
|
||||
from .models import Space, SpaceMembership
|
||||
|
||||
_SLUG_SAFE = re.compile(r"[^a-z0-9]+")
|
||||
|
||||
|
||||
@dataclass
|
||||
class ImportReport:
|
||||
|
|
@ -41,133 +28,153 @@ class ImportReport:
|
|||
page_slugs: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _read_index_meta(pages: Path) -> tuple[str, str, str]:
|
||||
index = pages / "index.md"
|
||||
if not index.is_file():
|
||||
return "", "", ""
|
||||
text = index.read_text(encoding="utf-8")
|
||||
meta, _ = pageops.parse_frontmatter(text)
|
||||
title = str(meta.get("title") or "").strip()
|
||||
abstractor = str(meta.get("abstractor") or meta.get("description") or "").strip()
|
||||
visual = str(meta.get("visual") or "").strip()
|
||||
return title, abstractor, visual
|
||||
|
||||
|
||||
def normalize_import_slug(value: str) -> str:
|
||||
s = slugify(value or "")[:80]
|
||||
if not s:
|
||||
raise pageops.PageOpsError("Space slug cannot be empty")
|
||||
return s
|
||||
return pageops.normalize_slug(value)
|
||||
|
||||
|
||||
def _metadata(tree: Path, slug: str, title: str | None = None) -> tuple[str, str, str]:
|
||||
index = tree / "pages" / "index.md"
|
||||
meta, _ = (
|
||||
pageops.parse_frontmatter(index.read_text(encoding="utf-8"))
|
||||
if index.is_file() else ({}, "")
|
||||
)
|
||||
display_title = (title or str(meta.get("title") or slug)).strip()
|
||||
if not display_title or len(display_title) > 255:
|
||||
raise pageops.PageOpsError("Title must be nonempty and at most 255 characters")
|
||||
abstractor = str(meta.get("abstractor") or meta.get("description") or "")
|
||||
visual = str(meta.get("visual") or "")
|
||||
if len(visual) > 512:
|
||||
raise pageops.PageOpsError("Visual path exceeds 512 characters")
|
||||
return display_title, abstractor, visual
|
||||
|
||||
|
||||
def validate_space_tree(
|
||||
tree: Path, *, tenant_id: str, slug: str | None = None,
|
||||
member: Member | None = None, replace: bool = False, title: str | None = None,
|
||||
) -> tuple[Path, str]:
|
||||
"""Preflight used by dry-run and execution, before any content writes.
|
||||
|
||||
CONTENT_ROOT is instance-local and keyed by slug, so importing a slug
|
||||
already owned by another tenant is unsupported and must fail closed.
|
||||
"""
|
||||
tree = tree.expanduser()
|
||||
if tree.is_symlink():
|
||||
raise pageops.PageOpsError("Source tree must not be a symlink")
|
||||
tree = tree.resolve()
|
||||
if not tree.is_dir() or not (tree / "pages").is_dir():
|
||||
raise pageops.PageOpsError("Source must contain a pages/ directory")
|
||||
if not tenant_id.strip() or len(tenant_id) > 64:
|
||||
raise pageops.PageOpsError("Tenant must be nonempty and at most 64 characters")
|
||||
space_slug = normalize_import_slug(slug or tree.name)
|
||||
if member is not None and member.tenant_id != tenant_id:
|
||||
raise pageops.PageOpsError("Member belongs to another tenant")
|
||||
if Space.objects.filter(slug=space_slug).exclude(tenant_id=tenant_id).exists():
|
||||
raise pageops.PageOpsError("Content slug belongs to another tenant")
|
||||
for path in tree.rglob("*"):
|
||||
if path.is_symlink():
|
||||
raise pageops.PageOpsError("Source tree contains a symlink")
|
||||
pages = sorted((tree / "pages").glob("*.md"))
|
||||
if not pages:
|
||||
raise pageops.PageOpsError("Source contains no markdown pages")
|
||||
for path in pages:
|
||||
if not path.is_file() or pageops.normalize_slug(path.stem) != path.stem:
|
||||
raise pageops.PageOpsError("Source page filenames must be normalized slugs")
|
||||
path.read_text(encoding="utf-8")
|
||||
_metadata(tree, space_slug, title)
|
||||
root = pageops.content_root()
|
||||
dest = pageops.space_dir(space_slug)
|
||||
if tree == root or root.is_relative_to(tree) or tree.is_relative_to(root):
|
||||
raise pageops.PageOpsError("Source tree and CONTENT_ROOT must not overlap")
|
||||
for path in (root, root / "spaces", dest):
|
||||
if path.is_symlink():
|
||||
raise pageops.PageOpsError("Destination must not be a symlink")
|
||||
if dest.exists() and not replace:
|
||||
raise pageops.PageOpsError("Destination already exists; pass replace=True to overwrite")
|
||||
return tree, space_slug
|
||||
|
||||
|
||||
def import_space_tree(
|
||||
tree: Path,
|
||||
*,
|
||||
tenant_id: str,
|
||||
slug: str | None = None,
|
||||
member: Member | None = None,
|
||||
title: str | None = None,
|
||||
replace: bool = False,
|
||||
tree: Path, *, tenant_id: str, slug: str | None = None,
|
||||
member: Member | None = None, title: str | None = None,
|
||||
replace: bool = False, memberships: list[tuple[Member, str]] | None = None,
|
||||
) -> ImportReport:
|
||||
"""Copy tree into CONTENT_ROOT and upsert Space row.
|
||||
"""Stage content, then publish with DB writes; restore on ordinary errors.
|
||||
|
||||
If ``replace`` is False and the destination pages dir already has files,
|
||||
raises PageOpsError unless the tree is empty.
|
||||
This is not a distributed transaction or crash recovery protocol. Run with
|
||||
application writers paused and take a paired DB/content backup first.
|
||||
"""
|
||||
tree = tree.expanduser().resolve()
|
||||
if not tree.is_dir():
|
||||
raise pageops.PageOpsError(f"Tree not found: {tree}")
|
||||
pages_src = tree / "pages"
|
||||
if not pages_src.is_dir():
|
||||
raise pageops.PageOpsError(f"Missing pages/ under {tree}")
|
||||
|
||||
idx_title, idx_abs, idx_vis = _read_index_meta(pages_src)
|
||||
space_slug = normalize_import_slug(slug or tree.name)
|
||||
display_title = (title or idx_title or space_slug).strip()
|
||||
abstractor = idx_abs
|
||||
visual = idx_vis
|
||||
|
||||
tree, space_slug = validate_space_tree(
|
||||
tree, tenant_id=tenant_id, slug=slug, member=member, replace=replace, title=title,
|
||||
)
|
||||
grants = list(memberships or [])
|
||||
if member is not None:
|
||||
grants.append((member, SpaceMembership.Role.OWNER))
|
||||
for mapped, role in grants:
|
||||
if mapped.tenant_id != tenant_id or role not in SpaceMembership.Role.values:
|
||||
raise pageops.PageOpsError("Invalid membership tenant or role")
|
||||
index = tree / "pages" / "index.md"
|
||||
display_title, abstractor, visual = _metadata(tree, space_slug, title)
|
||||
dest = pageops.space_dir(space_slug)
|
||||
dest_pages = pageops.pages_dir(space_slug)
|
||||
dest_assets = pageops.assets_dir(space_slug)
|
||||
|
||||
if dest_pages.is_dir() and any(dest_pages.glob("*.md")) and not replace:
|
||||
raise pageops.PageOpsError(
|
||||
f"Destination already has pages at {dest_pages}; pass replace=True to overwrite"
|
||||
)
|
||||
|
||||
report = ImportReport(space_slug=space_slug, title=display_title)
|
||||
|
||||
if replace and dest.is_dir():
|
||||
shutil.rmtree(dest)
|
||||
|
||||
pageops.ensure_space_tree(space_slug)
|
||||
|
||||
# Copy pages
|
||||
for path in sorted(pages_src.glob("*.md")):
|
||||
target = dest_pages / path.name
|
||||
shutil.copy2(path, target)
|
||||
report.pages_copied += 1
|
||||
report.page_slugs.append(path.stem)
|
||||
|
||||
# Copy assets if present
|
||||
assets_src = tree / "assets"
|
||||
if assets_src.is_dir():
|
||||
for path in sorted(assets_src.rglob("*")):
|
||||
if not path.is_file():
|
||||
continue
|
||||
rel = path.relative_to(assets_src)
|
||||
target = dest_assets / rel
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(path, target)
|
||||
report.assets_copied += 1
|
||||
|
||||
if not (dest_pages / "index.md").is_file():
|
||||
pageops.init_space_content(
|
||||
space_slug,
|
||||
title=display_title,
|
||||
abstractor=abstractor,
|
||||
visual=visual,
|
||||
)
|
||||
report.warnings.append("index.md missing in source; synthesized")
|
||||
report.pages_copied += 1
|
||||
report.page_slugs.append("index")
|
||||
|
||||
report.content_path = str(dest)
|
||||
|
||||
with transaction.atomic():
|
||||
space, created = Space.objects.update_or_create(
|
||||
tenant_id=tenant_id,
|
||||
slug=space_slug,
|
||||
defaults={
|
||||
"title": display_title,
|
||||
"description": abstractor,
|
||||
"visual": visual,
|
||||
"is_active": True,
|
||||
"content_root": "pages",
|
||||
"default_branch": "main",
|
||||
},
|
||||
)
|
||||
report.space_created = created
|
||||
report.space_updated = not created
|
||||
if member is not None:
|
||||
SpaceMembership.objects.get_or_create(
|
||||
space=space,
|
||||
member=member,
|
||||
defaults={"role": SpaceMembership.Role.OWNER},
|
||||
)
|
||||
if space.created_by_id is None:
|
||||
space.created_by = member
|
||||
space.save(update_fields=["created_by", "updated_at"])
|
||||
|
||||
# Sanity: count readable pages
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
report = ImportReport(space_slug=space_slug, title=display_title, content_path=str(dest))
|
||||
temporary = Path(tempfile.mkdtemp(prefix=".import-", dir=dest.parent))
|
||||
recovery_needed = False
|
||||
try:
|
||||
listed = pageops.list_pages(space_slug)
|
||||
if len(listed) != report.pages_copied:
|
||||
report.warnings.append(
|
||||
f"list_pages returned {len(listed)} vs copied {report.pages_copied}"
|
||||
)
|
||||
except pageops.PageOpsError as exc:
|
||||
report.warnings.append(f"list_pages failed: {exc.message}")
|
||||
|
||||
staging = temporary / "staged"
|
||||
(staging / "pages").mkdir(parents=True)
|
||||
(staging / "assets").mkdir()
|
||||
for path in sorted((tree / "pages").glob("*.md")):
|
||||
shutil.copy2(path, staging / "pages" / path.name)
|
||||
report.page_slugs.append(path.stem)
|
||||
assets = tree / "assets"
|
||||
if assets.is_dir():
|
||||
shutil.copytree(assets, staging / "assets", dirs_exist_ok=True)
|
||||
report.assets_copied = sum(p.is_file() for p in assets.rglob("*"))
|
||||
if not index.is_file():
|
||||
(staging / "pages" / "index.md").write_text(pageops.dump_frontmatter(
|
||||
{"title": display_title, "abstractor": abstractor, "visual": visual,
|
||||
"space": space_slug}, f"# {display_title}\n",
|
||||
), encoding="utf-8")
|
||||
report.page_slugs.append("index")
|
||||
report.warnings.append("index.md missing in source; synthesized")
|
||||
report.pages_copied = len(report.page_slugs)
|
||||
backup = temporary / "previous"
|
||||
published = False
|
||||
try:
|
||||
with transaction.atomic():
|
||||
space, created = Space.objects.update_or_create(
|
||||
tenant_id=tenant_id, slug=space_slug,
|
||||
defaults={"title": display_title, "description": abstractor,
|
||||
"visual": visual, "is_active": True,
|
||||
"content_root": "pages", "default_branch": "main"},
|
||||
)
|
||||
report.space_created = created
|
||||
report.space_updated = not created
|
||||
for mapped, role in grants:
|
||||
SpaceMembership.objects.update_or_create(
|
||||
space=space, member=mapped, defaults={"role": role},
|
||||
)
|
||||
owner = next((m for m, r in grants if r == SpaceMembership.Role.OWNER), None)
|
||||
if owner and space.created_by_id is None:
|
||||
space.created_by = owner
|
||||
space.save(update_fields=["created_by", "updated_at"])
|
||||
if dest.exists():
|
||||
dest.rename(backup)
|
||||
staging.rename(dest)
|
||||
published = True
|
||||
except Exception:
|
||||
try:
|
||||
if published:
|
||||
shutil.rmtree(dest)
|
||||
if backup.exists():
|
||||
backup.rename(dest)
|
||||
except OSError as exc:
|
||||
recovery_needed = True
|
||||
raise pageops.PageOpsError(
|
||||
f"Filesystem recovery failed; staging/previous tree retained at {temporary}"
|
||||
) from exc
|
||||
raise
|
||||
finally:
|
||||
if not recovery_needed:
|
||||
shutil.rmtree(temporary)
|
||||
return report
|
||||
|
|
|
|||
|
|
@ -0,0 +1,99 @@
|
|||
"""Preflight a JSON corpus manifest and import restartable space units."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from django.core.management.base import BaseCommand, CommandError
|
||||
|
||||
from coulomb_social.apps.members.models import Member
|
||||
from coulomb_social.apps.spaces import pageops
|
||||
from coulomb_social.apps.spaces.import_tree import import_space_tree, validate_space_tree
|
||||
from coulomb_social.apps.spaces.models import SpaceMembership
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = "Import local space trees with explicit tenant/issuer/subject membership mappings"
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument("manifest")
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
parser.add_argument("--replace", action="store_true")
|
||||
|
||||
def handle(self, *args, **options):
|
||||
try:
|
||||
plan = self.preflight(Path(options["manifest"]), replace=options["replace"])
|
||||
except (ValueError, KeyError, TypeError, OSError, pageops.PageOpsError) as exc:
|
||||
raise CommandError(f"Manifest preflight failed: {exc}") from exc
|
||||
pages = sum(len(list((tree / "pages").glob("*.md"))) for tree, _, _, _ in plan)
|
||||
self.stdout.write(f"Validated {len(plan)} space(s), {pages} source page(s)")
|
||||
if options["dry_run"]:
|
||||
self.stdout.write("Dry run — no writes")
|
||||
return
|
||||
completed = 0
|
||||
for tree, tenant, slug, grants in plan:
|
||||
try:
|
||||
report = import_space_tree(
|
||||
tree, tenant_id=tenant, slug=slug, memberships=grants,
|
||||
replace=options["replace"],
|
||||
)
|
||||
except Exception as exc:
|
||||
# Report counts, not private source data or identity details.
|
||||
raise CommandError(
|
||||
f"Import stopped after {completed} committed space(s); "
|
||||
"failed space rolled back. Inspect locally before retrying."
|
||||
) from exc
|
||||
completed += 1
|
||||
self.stdout.write(
|
||||
f"Imported space {completed}/{len(plan)}: "
|
||||
f"pages={report.pages_copied} assets={report.assets_copied} "
|
||||
f"memberships={len(grants)}"
|
||||
)
|
||||
self.stdout.write(self.style.SUCCESS(f"Imported {completed} space(s)"))
|
||||
|
||||
def preflight(self, manifest: Path, *, replace: bool):
|
||||
manifest = manifest.expanduser().resolve()
|
||||
data = json.loads(manifest.read_text(encoding="utf-8"))
|
||||
if (not isinstance(data, dict) or data.get("version") != 1
|
||||
or not isinstance(data.get("spaces"), list)):
|
||||
raise ValueError("Expected version=1 and spaces list")
|
||||
if not data["spaces"]:
|
||||
raise ValueError("Manifest must contain at least one space")
|
||||
plan = []
|
||||
slugs = set()
|
||||
for entry in data["spaces"]:
|
||||
if not isinstance(entry, dict) or not isinstance(entry.get("memberships"), list):
|
||||
raise ValueError("Each space needs a memberships list")
|
||||
tenant = entry["tenant"]
|
||||
tree = Path(entry["tree"])
|
||||
if not tree.is_absolute():
|
||||
tree = manifest.parent / tree
|
||||
tree, slug = validate_space_tree(
|
||||
tree, tenant_id=tenant, slug=entry.get("slug"), replace=replace,
|
||||
)
|
||||
if slug in slugs:
|
||||
raise ValueError("Duplicate destination content slug")
|
||||
slugs.add(slug)
|
||||
grants = []
|
||||
seen = set()
|
||||
for identity in entry["memberships"]:
|
||||
if not isinstance(identity, dict):
|
||||
raise ValueError("Membership entries must be objects")
|
||||
role = identity["role"]
|
||||
if role not in SpaceMembership.Role.values:
|
||||
raise ValueError("Unknown membership role")
|
||||
member = Member.objects.filter(
|
||||
tenant_id=tenant, issuer=identity["issuer"], subject=identity["subject"],
|
||||
user__is_active=True,
|
||||
).first()
|
||||
if member is None:
|
||||
raise ValueError("Mapped identity is not an active member in the tenant")
|
||||
if member.pk in seen:
|
||||
raise ValueError("Duplicate mapped member")
|
||||
seen.add(member.pk)
|
||||
grants.append((member, role))
|
||||
if not any(role == SpaceMembership.Role.OWNER for _, role in grants):
|
||||
raise ValueError("Each imported space requires a mapped owner")
|
||||
plan.append((tree, tenant, slug, grants))
|
||||
return plan
|
||||
|
|
@ -9,7 +9,7 @@ from django.core.management.base import BaseCommand, CommandError
|
|||
|
||||
from coulomb_social.apps.members.models import Member
|
||||
from coulomb_social.apps.spaces import pageops
|
||||
from coulomb_social.apps.spaces.import_tree import import_space_tree
|
||||
from coulomb_social.apps.spaces.import_tree import import_space_tree, validate_space_tree
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
|
|
@ -65,12 +65,25 @@ class Command(BaseCommand):
|
|||
|
||||
tenant = options["tenant"] or settings.DEFAULT_TENANT_ID
|
||||
slug = options["slug"] or tree.name
|
||||
md_files = sorted(pages.glob("*.md"))
|
||||
member = None
|
||||
subject = (options["member_subject"] or "").strip()
|
||||
if subject:
|
||||
candidates = Member.objects.filter(subject=subject, tenant_id=tenant)
|
||||
if candidates.count() != 1:
|
||||
raise CommandError("Subject must resolve to exactly one member in the tenant")
|
||||
member = candidates.get()
|
||||
try:
|
||||
tree, slug = validate_space_tree(
|
||||
tree, tenant_id=tenant, slug=slug, member=member,
|
||||
replace=options["replace"], title=options["title"] or None,
|
||||
)
|
||||
except (pageops.PageOpsError, OSError, UnicodeError) as exc:
|
||||
raise CommandError(str(exc)) from exc
|
||||
md_files = sorted((tree / "pages").glob("*.md"))
|
||||
assets_dir = tree / "assets"
|
||||
asset_count = (
|
||||
sum(1 for p in assets_dir.rglob("*") if p.is_file())
|
||||
if assets_dir.is_dir()
|
||||
else 0
|
||||
if assets_dir.is_dir() else 0
|
||||
)
|
||||
|
||||
self.stdout.write(
|
||||
|
|
@ -85,19 +98,6 @@ class Command(BaseCommand):
|
|||
self.stdout.write(self.style.WARNING("Dry run — no writes"))
|
||||
return
|
||||
|
||||
member = None
|
||||
subject = (options["member_subject"] or "").strip()
|
||||
if subject:
|
||||
member = Member.objects.filter(subject=subject, tenant_id=tenant).first()
|
||||
if member is None:
|
||||
# try any tenant match for convenience in local smoke
|
||||
member = Member.objects.filter(subject=subject).first()
|
||||
if member is None:
|
||||
raise CommandError(
|
||||
f"No Member with subject={subject!r} "
|
||||
f"(sign in once or pass a known subject)"
|
||||
)
|
||||
|
||||
try:
|
||||
report = import_space_tree(
|
||||
tree,
|
||||
|
|
|
|||
|
|
@ -54,8 +54,8 @@ def create_space(
|
|||
if not title:
|
||||
raise pageops.PageOpsError("Title is required")
|
||||
space_slug = pageops.normalize_slug(slug or Space.suggest_slug(title))
|
||||
if Space.objects.filter(tenant_id=member.tenant_id, slug=space_slug).exists():
|
||||
raise pageops.PageOpsError("A space with this slug already exists")
|
||||
if Space.objects.filter(slug=space_slug).exists():
|
||||
raise pageops.PageOpsError("A space with this slug already exists in this content instance")
|
||||
|
||||
space = Space.objects.create(
|
||||
tenant_id=member.tenant_id,
|
||||
|
|
|
|||
|
|
@ -1,10 +1,13 @@
|
|||
# Reviewed 2026-10-06: implementation labels describe checkout capability.
|
||||
# "shipped" is legacy vocabulary, not fresh production acceptance.
|
||||
# PageOps live/persistence acceptance remains CSOC-IN-0004 / CSOC-WP-0008.
|
||||
# coulomb-social product capability model v0
|
||||
# Vocabulary: docs/capability/vocabulary.md
|
||||
# Stage-1 narrative: docs/capability/stage-1-cutover.md
|
||||
schema_version: "0.1"
|
||||
product: coulomb-social
|
||||
resource_id: "resource:tenant:coulomb:coulomb-social"
|
||||
updated: "2026-08-13"
|
||||
updated: "2026-10-06"
|
||||
stance: |
|
||||
Partial rebuild is intentional. Stage 1 enables user transfer and retires
|
||||
Bubble as productive system; Bubble may remain reference-only. Later stages
|
||||
|
|
@ -41,15 +44,16 @@ capabilities:
|
|||
status: shipped
|
||||
- id: feat.public-registration
|
||||
title: Public Create account + verified mail
|
||||
status: blocked
|
||||
blocked_on: "NK-WP-0025 / CSOC-IN-0001"
|
||||
status: shipped
|
||||
notes: "Historical Case B evidence 2026-08-14; CSOC-IN-0001 closed; recheck at transfer"
|
||||
toggle: NETKINGDOM_REGISTRATION_URL
|
||||
- id: feat.post-login-app-home
|
||||
title: Post-login app home (Spaces)
|
||||
status: shipped
|
||||
- id: feat.member-transfer-assist
|
||||
title: Assisted transfer of Bubble members (no password import)
|
||||
status: todo
|
||||
status: partial
|
||||
notes: "Validated corpus manifest maps existing identities; real cohort pending WP-0008"
|
||||
|
||||
- id: cap.space-content
|
||||
title: Spaces with markdown content
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
|-------|--------|
|
||||
| Decision | [`../decisions/2026-08-12-feature-cut-stage-1.md`](../decisions/2026-08-12-feature-cut-stage-1.md) |
|
||||
| Model | [`model-v0.yaml`](model-v0.yaml) |
|
||||
| Date | 2026-08-12 |
|
||||
| Date | Decision: 2026-08-12; checkout status reviewed: 2026-10-06 |
|
||||
| Stance | **Partial rebuild is enough** for transfer; extend with later use cases after Bubble is reference-only |
|
||||
|
||||
## Goal of stage 1
|
||||
|
|
@ -17,6 +17,11 @@ Bubble on apex as a **productive** system. After transfer:
|
|||
- Additional capabilities (marketplace, social graph, monetization, …) ship
|
||||
**later** on the rebuild without blocking transfer.
|
||||
|
||||
Implementation statuses below describe this checkout, not fresh live acceptance.
|
||||
CSOC-IN-0004 still owns PageOps persistence/deployment smoke; WP-0008 owns
|
||||
transfer and recovery acceptance. Public registration has historical 2026-08-14
|
||||
Case B evidence (CSOC-IN-0001 closed).
|
||||
|
||||
## Must capabilities (stage 1)
|
||||
|
||||
### 1. User onboarding — `cap.member-onboarding`
|
||||
|
|
@ -29,9 +34,9 @@ Bubble for day-to-day login.
|
|||
| OIDC sign-in (existing LLDAP/KeyCape users) | shipped | Case A proven |
|
||||
| JIT Member + user-engine link | shipped | |
|
||||
| Session / account diagnostics | shipped | `/account/session/` |
|
||||
| Public Create account + verified mail | partial / blocked | App entry ready; NK mailbox (`CSOC-IN-0001`) |
|
||||
| Public Create account + verified mail | historically evidenced | CSOC-IN-0001 closed; NK owns mail/registration, recheck at transfer |
|
||||
| Guided first landing on app home | shipped | post-login → Spaces |
|
||||
| Bulk / assisted member transfer from Bubble | todo | Process + optional import; no password migration |
|
||||
| Bulk / assisted member transfer from Bubble | partial | Manifest maps existing tenant+issuer+subject members; actual cohort rehearsal pending |
|
||||
|
||||
**Stage-1 acceptance:** known operators and community members can sign in;
|
||||
new members can register once NK path is live **or** operator-provisioned
|
||||
|
|
@ -46,15 +51,15 @@ Members work in **spaces** whose durable content is **markdown** (ADR-0002).
|
|||
| Space list (tenant-scoped) | shipped | |
|
||||
| Space detail + render page from Forgejo | shipped | |
|
||||
| Edit in Forgejo + refresh / webhook | shipped | git SoR |
|
||||
| **Space CRUD** (create/read/update/delete or archive) | partial | Admin/seed today; **member UI CRUD todo** |
|
||||
| **Title** on space | partial | Model field exists; member edit UI todo |
|
||||
| **Abstract** on space | partial | Use `description` as abstract until renamed |
|
||||
| **Visual** on space (cover/image) | todo | Not in model yet |
|
||||
| Page list within space | partial | path-based pages; richer list todo |
|
||||
| **Page CRUD** (create/update/delete markdown pages) | partial | Via Forgejo only; in-app or first-class API todo |
|
||||
| **Title** on page | partial | From H1 / filename; structured frontmatter todo |
|
||||
| **Abstract** on page | todo | Frontmatter or metadata |
|
||||
| **Visual** on page | todo | Frontmatter image / assets |
|
||||
| **Space CRUD** (create/read/update/delete or archive) | implemented | Member create/update/archive (WP-0006) |
|
||||
| **Title** on space | implemented | Model and member forms |
|
||||
| **Abstract** on space | implemented | `description`, product label Abstractor |
|
||||
| **Visual** on space (cover/image) | metadata implemented | Relative path; upload/authenticated asset delivery not accepted |
|
||||
| Page list within space | implemented | Local pages sidebar |
|
||||
| **Page CRUD** (create/update/delete markdown pages) | implemented | In-app metadata/body forms; index deletion prohibited |
|
||||
| **Title** on page | implemented | Frontmatter and form |
|
||||
| **Abstract** on page | implemented | Frontmatter and form |
|
||||
| **Visual** on page | metadata implemented | Relative Visual copied; broader asset pipeline incomplete |
|
||||
| Fail-closed missing binding | shipped | |
|
||||
|
||||
**Stage-1 acceptance:** members can create and maintain spaces and pages with
|
||||
|
|
@ -66,8 +71,8 @@ structured form + Forgejo body) can vary; **utility** must not require Bubble.
|
|||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|--------|
|
||||
| Transfer page to another space | todo | Move markdown (+ assets) + rewrite links; metadata follows |
|
||||
| Authz (source + destination membership) | todo | |
|
||||
| Transfer page to another space | implemented with limits | Markdown + Visual asset; arbitrary links/assets are not rewritten |
|
||||
| Authz (source + destination membership) | implemented with limits | Tenant/membership visibility; viewer write roles still unenforced |
|
||||
|
||||
**Stage-1 acceptance:** a page can leave space A and appear in space B without
|
||||
Bubble, with clear ownership after the move.
|
||||
|
|
@ -76,8 +81,8 @@ Bubble, with clear ownership after the move.
|
|||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|--------|
|
||||
| Generate copy of page (same or other space) | todo | New slug/title; independent thereafter |
|
||||
| Copy assets referenced by page | todo | Or document “copy body only” if scoped |
|
||||
| Generate copy of page (same or other space) | implemented | New slug/title; independent thereafter |
|
||||
| Copy assets referenced by page | partial | Relative Visual copied; body-linked assets deferred |
|
||||
|
||||
**Stage-1 acceptance:** member can produce an independent copy without manual
|
||||
git surgery as the only path (operator git escape hatch may exist but is not
|
||||
|
|
@ -100,29 +105,26 @@ Examples — full list in `model-v0.yaml`:
|
|||
|------------|----------------|
|
||||
| NetKingdom OIDC + (eventually) registration mail | Onboarding |
|
||||
| apps-pg | Space/member metadata |
|
||||
| Forgejo | Markdown SoR |
|
||||
| CONTENT_ROOT + optional Forgejo | Local PageOps SoR; Forgejo fallback, no automatic local commit/push |
|
||||
| Railiance deploy | Host app.coulomb.social |
|
||||
|
||||
## Implementation backlog (derived)
|
||||
## Current acceptance backlog
|
||||
|
||||
Priority order suggested for build work after this decision:
|
||||
WP-0006 implemented the member CRUD/metadata/copy/transfer path. WP-0007 added
|
||||
validated single/bulk import with explicit membership mapping. Remaining
|
||||
acceptance is tracked in WP-0008: persistence/live smoke, paired recovery,
|
||||
private cohort reconciliation, access roles/privacy, member portability and
|
||||
explicit founder cutover. Later authoring features remain in the machine model.
|
||||
|
||||
1. Space member CRUD + Title / Abstract edit
|
||||
2. Visual field (space, then page) + asset storage convention
|
||||
3. Page metadata (title/abstract/visual) + create/update/delete path
|
||||
4. Page copy
|
||||
5. Page transfer
|
||||
6. Onboarding: transfer cohort process + unstick public registration residual
|
||||
## Resolved choices and remaining product questions
|
||||
|
||||
Bubble export/mapping (CSOC-WP-0001-T02) feeds **content migration** into these
|
||||
capabilities but does not redefine the must set.
|
||||
|
||||
## Open product questions
|
||||
|
||||
1. **Visual** — cover image only, or gallery? Max size? assets next to markdown vs object store?
|
||||
2. **Abstract** — rename `Space.description` → abstract in UI only, or schema rename?
|
||||
3. **Page body editing** — stage 1: product forms for Title/Abstract/Visual + markdown body path; exact editor TBD.
|
||||
4. **Delete** — hard delete vs archive (`is_active`)?
|
||||
5. **Transfer cohort** — require public self-registration, or LLDAP bulk invite for stage 1?
|
||||
6. **Content substrate** — see `docs/architecture/2026-08-12-content-substrate-exploration.md` (markitect-tool + markdown SoR; evaluate shard-wiki thin slice vs folder/git store).
|
||||
7. **Authoring** — top-down and bottom-up both first-class; lazy title/abstractor/visual with regenerate + lock — `docs/architecture/authoring-modes-and-lazy-projections.md`.
|
||||
- Visual: path metadata and relative-file copying today; upload, serving and
|
||||
full body-linked asset fidelity require acceptance work.
|
||||
- Abstract: UI uses Abstractor; persisted Space field remains `description`.
|
||||
- Body: markdown form implemented; no real-time editor or conflict protocol.
|
||||
- Delete: Space archive; page delete except index.
|
||||
- Cohort: explicit existing NetKingdom identity mappings; no password import.
|
||||
Actual cohort and lossiness acceptance are still required.
|
||||
- Content: ADR-0004 thin local directory, optional Forgejo fallback. Git
|
||||
automation, authoring projections, live includes and outline operations remain
|
||||
future work, not prerequisites silently added to the accepted cut.
|
||||
|
|
|
|||
|
|
@ -133,3 +133,18 @@ make test
|
|||
```
|
||||
|
||||
See also: `docs/deploy.md`, `docs/spaces-content.md`, `docs/adr/ADR-0001-netkingdom-identity.md`.
|
||||
|
||||
|
||||
## Independent negative regressions — 2026-10-06
|
||||
|
||||
CSOC-IN-0002 closed by CSOC-WP-0007 with synthetic application tests in
|
||||
`tests/test_identity_regressions.py` (no live issuer or private accounts):
|
||||
|
||||
- A successful callback consumes state and PKCE verifier. Replaying the same
|
||||
callback returns 400; token exchange and session establishment each run once.
|
||||
- Two issuer+subject identities sharing an email get distinct Member/User rows.
|
||||
The second session authenticates the second user; the original member's id,
|
||||
user link and display name stay intact. Email does not select an existing member.
|
||||
|
||||
This independently covers the two narrowed intake checks. It does not constitute
|
||||
a fresh live Case B or production rollout smoke.
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ id: CSOC-IN-0002
|
|||
kind: intake
|
||||
title: "Add callback replay and email-only takeover regressions"
|
||||
lane: green
|
||||
status: open
|
||||
status: closed
|
||||
priority: low
|
||||
owner: coulomb-social
|
||||
repo: coulomb-social
|
||||
|
|
@ -67,6 +67,9 @@ description: |
|
|||
|
||||
Capture the results in docs/identity/smoke.md and close this intake.
|
||||
notes: |
|
||||
Closed 2026-10-06 by CSOC-WP-0007. Independent application regressions in
|
||||
tests/test_identity_regressions.py prove consumed callback replay rejection
|
||||
before a second token exchange and same-email/different-identity separation.
|
||||
Narrowed 2026-08-14 after NK-WP-0025 completed live Case B. No longer blocked
|
||||
on the registration or mailbox path.
|
||||
state_hub_intake_id: "019ff532-3ade-7d75-acb7-2bbfa264fcb8"
|
||||
|
|
@ -101,6 +104,8 @@ description: |
|
|||
Prefer a dedicated workplan if bulk import + identity map spans multiple
|
||||
sessions; otherwise close this intake when (1)–(3) are evidenced.
|
||||
notes: |
|
||||
Local tooling prepared by CSOC-WP-0007; actual corpus/cohort acceptance
|
||||
continues in CSOC-WP-0008-T03. Intake remains open until real receipts.
|
||||
Pickup: docs/session-pickup.md · rehearsal: docs/migration/rehearsal-reichelag-2026-08-13.md
|
||||
Code: coulomb_social/apps/spaces/import_tree.py
|
||||
state_hub_intake_id: "019ffac3-f894-707a-9fdb-75c8ea14497d"
|
||||
|
|
@ -132,6 +137,7 @@ description: |
|
|||
Local default CONTENT_ROOT is var/content (gitignored). Cluster must not
|
||||
rely on empty ephemeral disk without a volume decision.
|
||||
notes: |
|
||||
Live persistence/deployment acceptance continues in CSOC-WP-0008-T01.
|
||||
Runbook docs/deploy.md · capability model docs/capability/model-v0.yaml
|
||||
Pickup: docs/session-pickup.md
|
||||
state_hub_intake_id: "019ffac3-fc7d-7850-93b6-b0d289881ec7"
|
||||
|
|
|
|||
94
docs/migration/corpus-import.md
Normal file
94
docs/migration/corpus-import.md
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
# Local corpus import and recovery prerequisites
|
||||
|
||||
`import_content_corpus` imports prepared ADR-0004 trees. It does not contact
|
||||
Bubble, provision accounts or verify the completeness of a Bubble export.
|
||||
Use private local trees and manifests outside the checkout. Never commit member
|
||||
mappings, corpus content, DB dumps or backup archives.
|
||||
|
||||
## Manifest
|
||||
|
||||
Version 1 JSON. Tree paths are relative to the manifest directory or absolute.
|
||||
Every space requires an existing active member mapped as owner. Identity is
|
||||
matched by **tenant + issuer + subject**, never email or a password.
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"spaces": [
|
||||
{
|
||||
"tree": "trees/example",
|
||||
"slug": "example",
|
||||
"tenant": "tenant:example",
|
||||
"memberships": [
|
||||
{"issuer": "https://issuer.example", "subject": "existing-subject", "role": "owner"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Roles: `owner`, `member`, `viewer`. Mapping preserves these labels; current
|
||||
product authorization does not yet enforce viewer write denial. Do not rely on
|
||||
role labels as a privacy guarantee. Each Space gets an owner and explicit grants,
|
||||
which disables the tenant-wide bootstrap visibility fallback for that tenant.
|
||||
Review the effect on existing spaces before executing a real cohort migration.
|
||||
|
||||
## Run
|
||||
|
||||
First provision accounts through NetKingdom and have users sign in to create
|
||||
local Members. Reconcile the corpus and approved grants. Pause all application
|
||||
writers; run with the deployment tenant's private CONTENT_ROOT and DB settings.
|
||||
|
||||
```bash
|
||||
uv run manage.py import_content_corpus /private/path/manifest.json --dry-run
|
||||
uv run manage.py import_content_corpus /private/path/manifest.json
|
||||
# Explicit replacement only after reviewing the existing content and backup:
|
||||
uv run manage.py import_content_corpus /private/path/manifest.json --replace
|
||||
```
|
||||
|
||||
Preflight validates **all** entries before any writes: local pages, normalized
|
||||
filenames, metadata bounds, source symlinks, source/destination overlap,
|
||||
destination collisions, active identity/tenant mapping and at least one owner.
|
||||
A missing owner or identity in the last tree prevents the first tree being
|
||||
written. Dry-run performs that same preflight and creates no directories or rows.
|
||||
|
||||
Execution stages each tree beside its destination, then applies metadata and
|
||||
memberships and publishes by rename inside a DB transaction. Ordinary errors
|
||||
restore the prior tree and roll back that space's DB changes. **The whole batch
|
||||
is not atomic**: previously committed spaces remain if a later space fails.
|
||||
Receipts give the completed count. Reconcile local state and either import a
|
||||
manifest containing only remaining entries, or deliberately use `--replace`.
|
||||
Replacement removes the previous content tree, including assets not in source;
|
||||
it adds/updates mapped memberships and **does not revoke** other existing grants
|
||||
or overwrite a pre-existing creator. Review those extra accesses explicitly.
|
||||
|
||||
Writers must remain paused. A process crash, rename/DB commit race or failed
|
||||
recovery filesystem operation requires manual recovery; this is not a crash-safe
|
||||
journal or concurrent import protocol. Keep content-root and instance-per-client
|
||||
tenant boundaries intact: filesystem paths remain slug-keyed.
|
||||
|
||||
The single-tree command shares validation and staging. `--member-subject` must
|
||||
resolve to exactly one tenant-local Member; use the corpus manifest when issuer
|
||||
ambiguity is possible. Single-tree operator imports may still omit membership,
|
||||
so they are not a replacement for approved cohort mapping.
|
||||
|
||||
## Paired backup/restore protocol (acceptance still pending)
|
||||
|
||||
1. Quiesce app and import writers, including Forgejo refresh consumers where
|
||||
relevant. Record recovery point, exact image SHA, migration level and tenant.
|
||||
2. Through the DB owner's documented credential lane, capture metadata DB;
|
||||
separately snapshot/archive the **entire** CONTENT_ROOT (pages and assets,
|
||||
including archived trees). Keep private data in the approved backup store.
|
||||
3. Record SHA-256 file inventory, row counts and backup identities privately.
|
||||
Do not log credentials, content bodies or identity lists into this repo/hub.
|
||||
4. Restore DB and CONTENT_ROOT together into an isolated instance. Disable
|
||||
outgoing integrations and use restore-specific settings and credentials.
|
||||
5. Compare Space/Member/membership counts, file hashes, page list/render, visual
|
||||
assets, copied/moved content and denied reads by an ungranted member. Measure
|
||||
achieved recovery time and recovery-point age; record the non-secret receipts.
|
||||
6. Resume writers only after accepted recovery checks. If import fails, restore
|
||||
both stores from that same recovery point rather than only rolling back DB.
|
||||
|
||||
This is an operational protocol, **not** evidence that scheduled backup or a
|
||||
restore drill exists. CSOC-WP-0008-T02 owns automation and measured acceptance;
|
||||
T01 owns storage admission, T03 the actual private cohort rehearsal.
|
||||
|
|
@ -1,3 +1,18 @@
|
|||
# Current pickup — 2026-10-06
|
||||
|
||||
WP-0007 audits scope and implements validated corpus import with explicit
|
||||
identity membership mapping. WP-0008 tracks live persistence/restore, actual
|
||||
cohort migration and access/cutover acceptance. Public registration residual
|
||||
CSOC-IN-0001 is closed (2026-08-14 evidence); CSOC-IN-0002 is closed with independent replay
|
||||
and same-email identity separation regressions. CSOC-IN-0003/0004 remain open.
|
||||
|
||||
Read SCOPE.md, history/20261005T223104Z-scope-intent-assessment.md and
|
||||
docs/migration/corpus-import.md. Run the tests and import dry-run before any
|
||||
private corpus work. The older pickup below is historical and does not override
|
||||
current work records or authorize deployment.
|
||||
|
||||
---
|
||||
|
||||
# Session pickup — coulomb-social
|
||||
|
||||
**Last closed:** 2026-08-13
|
||||
|
|
|
|||
105
history/20261005T223104Z-scope-intent-assessment.md
Normal file
105
history/20261005T223104Z-scope-intent-assessment.md
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
# Scope against intent — 2026-10-06
|
||||
|
||||
Timestamp prefix is UTC (2026-10-05 22:31:04); review date is Europe/Berlin.
|
||||
Evidence: tracked application source, tests, workplans, ADRs and historical
|
||||
migration/identity records. No live production or private-corpus execution.
|
||||
|
||||
## Finding
|
||||
|
||||
The repository is an implemented Django identity + spaces/pages application,
|
||||
not a scaffold and not yet a complete co-creation marketplace. The previous
|
||||
SCOPE under-reported member CRUD/PageOps and finished Bubble assessment work,
|
||||
and incorrectly described public registration as still platform-blocked.
|
||||
|
||||
INTENT's broad parity language must be read with the accepted 2026-08-12
|
||||
stage-1 cut: partial rebuild for productive transfer, broader social/marketplace
|
||||
features later. This review does not change that human decision or authorize
|
||||
retiring Bubble.
|
||||
|
||||
## Intent success criteria against evidence
|
||||
|
||||
| INTENT criterion | Assessment | Evidence / gap |
|
||||
|---|---|---|
|
||||
| Inventory all Bubble surfaces/workflows/data/integrations | Partial | `docs/bubble-inventory/`, client-package capability map and design crawl. Backend workflows/privacy and full editor schema remain lower-confidence; not a full formal export. |
|
||||
| Tenant-keyed relational mapping and rehearsal import | Partial | Member/Space/membership tables, schema sketches, 2026-08-13 reichelag rehearsal (30 pages, 29 assets). Full 71-space corpus and member mapping not executed; filesystem content uses slug, not tenant keys. |
|
||||
| Human-gated feature cut | Met as a recorded decision | `docs/decisions/2026-08-12-feature-cut-stage-1.md` accepts onboarding + spaces/pages/copy/transfer and defers broader parity. |
|
||||
| Standalone v1 under service contract | Partial | Docker, migrations, health, OIDC/user-engine, deployment lane and demand evidence exist. CONTENT_ROOT deployment, paired restore, export/delete, ops/cost/provenance acceptance are not comprehensively evidenced here. |
|
||||
| Members/content cut over without Bubble dependence | Not accepted yet | Core code works without Bubble; historical Case B registration closed CSOC-IN-0001. Live PageOps acceptance, full cohort/content transfer, recovery and explicit cutover evidence remain outstanding. |
|
||||
|
||||
## Ranked gaps to address
|
||||
|
||||
1. **Persistent, accepted content deployment and recoverability (high).** The
|
||||
latest product capability is not proven on the app host. Markdown on an
|
||||
ephemeral disk would undermine productive transfer; a DB-only backup loses
|
||||
page bodies. Existing CSOC-IN-0004 is the deployment handoff. Require pinned
|
||||
image/migration, volume decision, paired DB/content backup and isolated
|
||||
restore with membership/render checks before acceptance.
|
||||
2. **Safe multi-space and member migration (high).** One successful tree is not
|
||||
a community transfer. Existing CSOC-IN-0003 covers 70 remaining spaces and
|
||||
identity mapping; thin exports and parent/privacy lossiness need disposition.
|
||||
The old importer wrote content before DB completion, dry-run skipped member
|
||||
validation, and subject lookup could fall back to another tenant. Implement
|
||||
local corpus preflight/mapping and rollback before touching private content.
|
||||
3. **Access fidelity and portability (high).** Tenant-filtered rows are useful,
|
||||
but slug-keyed filesystem paths, tenant-wide bootstrap visibility and roles
|
||||
without viewer/editor enforcement do not establish privacy parity. Exact
|
||||
Bubble privacy/member mapping, membership revocation and app-local member
|
||||
export/delete remain acceptance gaps. Do not declare shared-root multi-tenant
|
||||
SaaS supported.
|
||||
4. **Capability evidence accuracy (medium).** Stage-1 cutover tables still
|
||||
described member CRUD, Visual and copy/move as todo despite WP-0006. The
|
||||
machine model used `shipped` without separating checkout and deployment;
|
||||
session pickup also retained a closed registration blocker. Reconcile these
|
||||
now so future work starts from actual capability and live evidence.
|
||||
5. **Content integrity/versioning (medium).** Local PageOps lacks automatic git
|
||||
history, arbitrary asset/link transfer and concurrent-edit conflict control.
|
||||
Visual fields alone do not deliver an upload/asset-serving workflow. Outline,
|
||||
live includes and lazy projections remain designed, not implemented.
|
||||
6. **Full co-creation marketplace (later).** Discovery/matching, favorites,
|
||||
social graph, payments and real-time editing are meaningful differences from
|
||||
broad INTENT, but the accepted cut deliberately excludes them from stage-1
|
||||
acceptance. Implementing them now would miss the nearer transfer blockers.
|
||||
|
||||
## Implemented response — CSOC-WP-0007
|
||||
|
||||
- Rewrite SCOPE with checkout capabilities, practical limits and dated live
|
||||
evidence; reconcile stage-1 status and pickup pointers.
|
||||
- Shared single-space import validation rejects cross-tenant member/slug use,
|
||||
source symlinks, overlapping roots and invalid pages/metadata. Subject lookup
|
||||
requires exactly one tenant-local member. Content is staged, published beside
|
||||
the destination and restored on ordinary errors while DB writes roll back.
|
||||
- New `import_content_corpus` preflights all entries before writing, maps active
|
||||
members by tenant+issuer+subject, requires a mapped owner, rejects duplicate
|
||||
destinations and supports read-only dry-run and explicit replacement.
|
||||
- Documentation states additive membership semantics and per-space restart
|
||||
behavior; private trees/manifests remain outside git. Regression checks use
|
||||
synthetic data, not the real corpus.
|
||||
|
||||
These changes close local preparation gaps. They do not certify corpus parity,
|
||||
production rollout, crash atomicity, exact ACL replacement or restore success.
|
||||
|
||||
## Live follow-up
|
||||
|
||||
[CSOC-WP-0008](../workplans/CSOC-WP-0008-production-transfer-acceptance.md)
|
||||
tracks persistence/live acceptance, paired recovery, real cohort mapping,
|
||||
privacy/access portability and founder cutover. It takes over the execution of
|
||||
CSOC-IN-0003/0004 without closing either on the strength of local tooling.
|
||||
CSOC-IN-0002 is closed with independent callback replay and same-email identity
|
||||
separation regressions in tests/test_identity_regressions.py. No residual is hidden
|
||||
only in this report or SCOPE.
|
||||
|
||||
## Validation
|
||||
|
||||
Final test and lint results are recorded below at session close. Repository-wide
|
||||
lint initially exposed pre-existing unused imports/variables, import ordering
|
||||
and long lines; changed-code lint is checked separately from that baseline.
|
||||
|
||||
|
||||
Final verification: **72 tests passed** (synthetic corpus, no private data);
|
||||
Ruff passes all changed application/test files; Django migration check reports
|
||||
no changes. Full repository Ruff retains **19 pre-existing findings** in
|
||||
identity, members/admin/migrations, older PageOps/Forgejo code and tests.
|
||||
WP-0008-T07 tracks that quality debt. Test warnings are Authlib jose deprecation
|
||||
and missing collected staticfiles in the test environment.
|
||||
|
||||
Production acceptance decision drafted in State Hub: `186be722-e2c7-4109-a1e6-d120fe8fca5b` (pending; no live change approval inferred).
|
||||
57
tests/test_identity_regressions.py
Normal file
57
tests/test_identity_regressions.py
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
"""Independent negative checks requested by CSOC-IN-0002."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
from django.urls import reverse
|
||||
|
||||
from coulomb_social.apps.identity import views
|
||||
from coulomb_social.apps.members.models import Member
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_consumed_callback_state_cannot_be_replayed(client, settings):
|
||||
settings.OIDC_ENABLED = True
|
||||
settings.OIDC_ISSUER = "https://issuer.test"
|
||||
settings.OIDC_CLIENT_ID = "test-client"
|
||||
settings.OIDC_REDIRECT_URI = "https://app.test/auth/callback/"
|
||||
session = client.session
|
||||
session[views.SESSION_OIDC_STATE] = "single-use-state"
|
||||
session[views.SESSION_OIDC_VERIFIER] = "verifier"
|
||||
session.save()
|
||||
with (
|
||||
patch.object(views.oidc, "exchange_code", return_value={"access_token": "synthetic"})
|
||||
as exchange,
|
||||
patch.object(views.oidc, "fetch_userinfo", return_value={}),
|
||||
patch.object(views.oidc, "claims_from_token_response", return_value={"sub": "new"}),
|
||||
patch.object(views, "establish_session") as establish,
|
||||
):
|
||||
callback = reverse("identity:callback") + "?code=code&state=single-use-state"
|
||||
assert client.get(callback).status_code == 302
|
||||
assert client.get(callback).status_code == 400
|
||||
assert exchange.call_count == 1
|
||||
assert establish.call_count == 1
|
||||
assert views.SESSION_OIDC_STATE not in client.session
|
||||
assert views.SESSION_OIDC_VERIFIER not in client.session
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_same_email_different_identity_cannot_take_over_member(client, settings):
|
||||
settings.DEBUG = True
|
||||
settings.OIDC_ENABLED = False
|
||||
login = reverse("identity:dev_login")
|
||||
client.post(login, {"issuer": "https://first.test", "subject": "original",
|
||||
"email": "shared@example.test", "name": "Original"})
|
||||
original = Member.objects.get(subject="original")
|
||||
original_user_id = original.user_id
|
||||
client.post(reverse("identity:logout"))
|
||||
client.post(login, {"issuer": "https://second.test", "subject": "different",
|
||||
"email": "shared@example.test", "name": "Different"})
|
||||
original.refresh_from_db()
|
||||
other = Member.objects.get(subject="different")
|
||||
assert original.user_id == original_user_id
|
||||
assert original.display_name == "Original"
|
||||
assert other.pk != original.pk
|
||||
assert other.user_id != original.user_id
|
||||
assert str(client.session["_auth_user_id"]) == str(other.user_id)
|
||||
assert Member.objects.count() == 2
|
||||
79
tests/test_import_corpus.py
Normal file
79
tests/test_import_corpus.py
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
"""Corpus transfer acceptance: preflight, identity mapping and restart policy."""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from django.core.management import call_command
|
||||
from django.core.management.base import CommandError
|
||||
|
||||
from coulomb_social.apps.members.models import Member, User
|
||||
from coulomb_social.apps.spaces import pageops
|
||||
from coulomb_social.apps.spaces.models import Space, SpaceMembership
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def corpus(tmp_path, settings):
|
||||
settings.CONTENT_ROOT = str(tmp_path / "content")
|
||||
member = Member.objects.create(
|
||||
tenant_id="tenant:a", user=User.objects.create_user(username="owner"),
|
||||
user_engine_user_id="owner", issuer="https://issuer.test", subject="owner",
|
||||
)
|
||||
entries = []
|
||||
for slug in ["one", "two"]:
|
||||
pages = tmp_path / "export" / slug / "pages"
|
||||
pages.mkdir(parents=True)
|
||||
(pages / "index.md").write_text(f"# {slug}\n")
|
||||
entries.append({"tree": f"export/{slug}", "tenant": "tenant:a",
|
||||
"memberships": [{"issuer": member.issuer,
|
||||
"subject": member.subject, "role": "owner"}]})
|
||||
manifest = tmp_path / "manifest.json"
|
||||
manifest.write_text(json.dumps({"version": 1, "spaces": entries}))
|
||||
return manifest, entries, member
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_corpus_dry_run_and_import(corpus):
|
||||
manifest, _, member = corpus
|
||||
call_command("import_content_corpus", str(manifest), dry_run=True)
|
||||
assert not Space.objects.exists()
|
||||
assert not pageops.content_root().exists()
|
||||
call_command("import_content_corpus", str(manifest))
|
||||
assert Space.objects.count() == 2
|
||||
assert SpaceMembership.objects.filter(member=member, role="owner").count() == 2
|
||||
assert pageops.read_page("two", "index").body == "# two\n"
|
||||
with pytest.raises(CommandError, match="already exists"):
|
||||
call_command("import_content_corpus", str(manifest))
|
||||
call_command("import_content_corpus", str(manifest), replace=True)
|
||||
assert Space.objects.count() == 2
|
||||
assert SpaceMembership.objects.count() == 2
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize("failure", ["identity", "tenant", "duplicate", "owner", "missing"])
|
||||
def test_entire_corpus_preflight_before_writes(corpus, failure):
|
||||
manifest, entries, _ = corpus
|
||||
if failure == "identity":
|
||||
entries[1]["memberships"][0]["subject"] = "missing"
|
||||
elif failure == "tenant":
|
||||
entries[1]["tenant"] = "tenant:b"
|
||||
elif failure == "duplicate":
|
||||
entries[1]["slug"] = "one"
|
||||
elif failure == "owner":
|
||||
entries[1]["memberships"][0]["role"] = "viewer"
|
||||
else:
|
||||
entries[1]["tree"] = "missing"
|
||||
manifest.write_text(json.dumps({"version": 1, "spaces": entries}))
|
||||
with pytest.raises(CommandError):
|
||||
call_command("import_content_corpus", str(manifest))
|
||||
assert not Space.objects.exists()
|
||||
assert not pageops.content_root().exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize("data", [[], {"version": 1, "spaces": []},
|
||||
{"version": 1, "spaces": [42]}])
|
||||
def test_invalid_manifest_shape_is_command_error(tmp_path, data):
|
||||
manifest = tmp_path / "manifest.json"
|
||||
manifest.write_text(json.dumps(data))
|
||||
with pytest.raises(CommandError, match="preflight"):
|
||||
call_command("import_content_corpus", str(manifest), dry_run=True)
|
||||
|
|
@ -119,3 +119,139 @@ def test_import_content_tree_command(content_root, tmp_path, capsys):
|
|||
space = Space.objects.get(slug="cmd-space", tenant_id="tenant:cmd")
|
||||
assert space.title == "Sample Space"
|
||||
assert pageops.read_page("cmd-space", "index").exists
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_import_rejects_other_tenant_before_writes(content_root, tmp_path):
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
member = Member.objects.create(
|
||||
tenant_id="tenant:b", user=User.objects.create_user(username="other"),
|
||||
user_engine_user_id="other", issuer="https://issuer.test", subject="other",
|
||||
)
|
||||
with pytest.raises(pageops.PageOpsError, match="another tenant"):
|
||||
import_space_tree(tree, tenant_id="tenant:a", member=member)
|
||||
assert not content_root.exists()
|
||||
assert not Space.objects.exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_import_rejects_cross_tenant_slug(content_root, tmp_path):
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
import_space_tree(tree, tenant_id="tenant:a")
|
||||
before = pageops.page_file("sample-space", "index").read_bytes()
|
||||
with pytest.raises(pageops.PageOpsError, match="another tenant"):
|
||||
import_space_tree(tree, tenant_id="tenant:b", replace=True)
|
||||
assert pageops.page_file("sample-space", "index").read_bytes() == before
|
||||
assert not Space.objects.filter(tenant_id="tenant:b").exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_import_rejects_symlink_in_dry_run(content_root, tmp_path):
|
||||
from django.core.management.base import CommandError
|
||||
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
(tree / "assets" / "outside").symlink_to(tmp_path)
|
||||
with pytest.raises(CommandError, match="symlink"):
|
||||
call_command("import_content_tree", str(tree), "--dry-run")
|
||||
assert not content_root.exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_import_rolls_back_content_on_publish_error(content_root, tmp_path, monkeypatch):
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
import_space_tree(tree, tenant_id="tenant:a")
|
||||
path = pageops.page_file("sample-space", "index")
|
||||
original = path.read_bytes()
|
||||
(tree / "pages" / "index.md").write_text('# Changed\n')
|
||||
real_rename = Path.rename
|
||||
|
||||
def fail_publish(self, target):
|
||||
if self.name == "staged":
|
||||
raise OSError("simulated publish failure")
|
||||
return real_rename(self, target)
|
||||
|
||||
monkeypatch.setattr(Path, "rename", fail_publish)
|
||||
with pytest.raises(OSError, match="simulated"):
|
||||
import_space_tree(tree, tenant_id="tenant:a", replace=True)
|
||||
assert path.read_bytes() == original
|
||||
assert Space.objects.get(slug="sample-space").title == "Sample Space"
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_single_import_subject_cannot_fall_back_to_other_tenant(content_root, tmp_path):
|
||||
from django.core.management.base import CommandError
|
||||
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
Member.objects.create(
|
||||
tenant_id="tenant:b", user=User.objects.create_user(username="foreign"),
|
||||
user_engine_user_id="foreign", issuer="https://issuer.test", subject="foreign",
|
||||
)
|
||||
with pytest.raises(CommandError, match="exactly one"):
|
||||
call_command("import_content_tree", str(tree), tenant="tenant:a",
|
||||
member_subject="foreign", dry_run=True)
|
||||
assert not content_root.exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db(transaction=True)
|
||||
def test_import_restores_tree_when_database_commit_fails(content_root, tmp_path, monkeypatch):
|
||||
from django.db import DatabaseError, connection
|
||||
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
import_space_tree(tree, tenant_id="tenant:a")
|
||||
path = pageops.page_file("sample-space", "index")
|
||||
original = path.read_bytes()
|
||||
(tree / "pages" / "index.md").write_text('# Changed\n')
|
||||
|
||||
def fail_commit():
|
||||
raise DatabaseError("simulated database commit failure")
|
||||
|
||||
monkeypatch.setattr(connection, "commit", fail_commit)
|
||||
with pytest.raises(DatabaseError):
|
||||
import_space_tree(tree, tenant_id="tenant:a", replace=True)
|
||||
assert path.read_bytes() == original
|
||||
assert Space.objects.get(slug="sample-space").title == "Sample Space"
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize("failure", ["overlap", "page-name", "metadata"])
|
||||
def test_shared_validation_rejects_bad_trees(content_root, tmp_path, failure):
|
||||
from django.core.management.base import CommandError
|
||||
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
if failure == "overlap":
|
||||
import shutil
|
||||
|
||||
content_root.mkdir()
|
||||
shutil.copytree(tree, content_root / "source")
|
||||
tree = content_root / "source"
|
||||
elif failure == "page-name":
|
||||
(tree / "pages" / "notes.md").rename(tree / "pages" / "Bad Name.md")
|
||||
else:
|
||||
(tree / "pages" / "index.md").write_text(
|
||||
'---\ntitle: "' + ('x' * 256) + '"\n---\n\nbody\n'
|
||||
)
|
||||
with pytest.raises(CommandError):
|
||||
call_command("import_content_tree", str(tree), dry_run=True)
|
||||
assert not Space.objects.exists()
|
||||
assert not (content_root / "spaces").exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_failed_restore_preserves_previous_tree(content_root, tmp_path, monkeypatch):
|
||||
tree = _write_sample_tree(tmp_path / "export")
|
||||
import_space_tree(tree, tenant_id="tenant:a")
|
||||
original = pageops.page_file("sample-space", "index").read_bytes()
|
||||
real_rename = Path.rename
|
||||
|
||||
def fail_publish_and_restore(self, target):
|
||||
if self.name in {"staged", "previous"}:
|
||||
raise OSError("simulated filesystem failure")
|
||||
return real_rename(self, target)
|
||||
|
||||
monkeypatch.setattr(Path, "rename", fail_publish_and_restore)
|
||||
with pytest.raises(pageops.PageOpsError, match="retained"):
|
||||
import_space_tree(tree, tenant_id="tenant:a", replace=True)
|
||||
saved = list((content_root / "spaces").glob(".import-*/previous/pages/index.md"))
|
||||
assert len(saved) == 1
|
||||
assert saved[0].read_bytes() == original
|
||||
assert Space.objects.get(slug="sample-space").title == "Sample Space"
|
||||
|
|
|
|||
89
workplans/CSOC-WP-0007-safe-transfer-readiness.md
Normal file
89
workplans/CSOC-WP-0007-safe-transfer-readiness.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
---
|
||||
id: CSOC-WP-0007
|
||||
type: workplan
|
||||
title: "Evidence-based scope and safe corpus transfer tooling"
|
||||
domain: communication
|
||||
repo: coulomb-social
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: coulomb-social
|
||||
created: "2026-10-06"
|
||||
updated: "2026-10-06"
|
||||
state_hub_workstream_id: "d52a146e-5f5c-5107-9755-0eb2c9ad50a8"
|
||||
---
|
||||
|
||||
# CSOC-WP-0007 — Safe transfer readiness
|
||||
|
||||
Prioritize the accepted stage-1 cut rather than inventing marketplace features.
|
||||
Implement local migration prerequisites; deployment, private corpus execution,
|
||||
restore acceptance and founder cutover remain separately tracked work.
|
||||
|
||||
## T01 — Audit scope against intent
|
||||
|
||||
```task
|
||||
id: CSOC-WP-0007-T01
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "0d6eb23c-a451-531a-b66b-e48c46ab64d4"
|
||||
```
|
||||
|
||||
Update SCOPE from source and dated evidence. File timestamped assessment in
|
||||
history; reconcile stale stage-1 status and identify outstanding contract gates.
|
||||
|
||||
## T02 — Validate single-space import before writes
|
||||
|
||||
```task
|
||||
id: CSOC-WP-0007-T02
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "1e4a25a4-79c6-5a3d-abeb-1498de8f27cf"
|
||||
```
|
||||
|
||||
Share real and dry-run validation; reject cross-tenant members, ambiguous subject
|
||||
lookup, cross-tenant filesystem slug collisions, symlinks, malformed page names,
|
||||
and overlapping source/destination. Stage content and restore the previous tree
|
||||
if metadata or membership writes fail. No private corpus in git.
|
||||
|
||||
## T03 — Bulk corpus import with explicit identity mapping
|
||||
|
||||
```task
|
||||
id: CSOC-WP-0007-T03
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "9973fbde-30cb-53c3-a899-396f23e97f24"
|
||||
```
|
||||
|
||||
JSON manifest names local trees and existing tenant+issuer+subject memberships.
|
||||
Validate the whole manifest before writes, require a mapped owner, refuse
|
||||
unresolved identities and duplicate destination slugs. Dry-run is read-only.
|
||||
Per-space atomic imports with explicit restart/replace semantics and counts.
|
||||
|
||||
## T04 — Verify and hand off production gates
|
||||
|
||||
```task
|
||||
id: CSOC-WP-0007-T04
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "0a7c06f9-fe10-5b3b-ab08-2143ae1a875f"
|
||||
```
|
||||
|
||||
Close CSOC-IN-0002 with independent callback replay/email identity regressions.
|
||||
Regression-test migration rejection, filesystem rollback, multi-space import,
|
||||
identity mapping and dry-run. Run full tests/lint/migration consistency. Document
|
||||
migration use and paired DB/content recovery requirements. Create a successor
|
||||
workplan for live persistence, restore, cohort migration and cutover acceptance
|
||||
before finishing. Commit and sync all implementation and work records.
|
||||
|
||||
## Completion evidence
|
||||
|
||||
- SCOPE and history/20261005T223104Z-scope-intent-assessment.md reflect source,
|
||||
historical deployment and accepted cut; capability/pickup docs reconciled.
|
||||
- Single import and new corpus command implemented with tenant/member preflight,
|
||||
per-space staging/DB rollback and retained recovery tree on restore failure.
|
||||
- CSOC-IN-0002 closed by independent identity regressions.
|
||||
- Full suite: **72 passed**. Changed Python/test files pass Ruff; Django reports
|
||||
no missing migrations. Full Ruff retains 19 pre-existing findings, assigned
|
||||
to CSOC-WP-0008-T07 rather than hidden as passing.
|
||||
- Residuals are live successor **CSOC-WP-0008** (origin: residual, origin_ref:
|
||||
CSOC-WP-0007), including existing CSOC-IN-0003/0004 acceptance. No private corpus
|
||||
or live deployment was executed by this workplan.
|
||||
Loading…
Add table
Add a link
Reference in a new issue