feat: prepare isolated recovery and guarded fresh ReichelAg capture
Some checks failed
Use-case regression gate / usecase-regressions (push) Has been cancelled

Assistant: codex
Assistant-Model: gpt-6.1-sol
Assistant-Session: 01a10e2d-a5ef-7e92-b03d-b1694e915680
This commit is contained in:
tegwick 2026-10-06 22:11:42 +02:00
parent 1e7b94df1a
commit b897aa1ee6
12 changed files with 642 additions and 14 deletions

View file

@ -112,3 +112,11 @@ and platform mailbox custody exist outside this repo; Authelia reset SMTP
is now active from platform@coulomb.social; actual inbox receipt is confirmed.
Personal-account login acceptance is also confirmed. See
[platform mail assessment](docs/release/platform-email-2026-10-06.md).
Paired recovery preparation (2026-10-06): guarded app-quiesced PostgreSQL/content
snapshot and isolated restore helpers are implemented. Live pre-import pair
restored13tables/102rows with empty content; a synthetic populated pair restored
13tables/111rows and4files with rendering and permission checks. Retained backup
custody/retention and real corpus/rollback acceptance stay CSOC-WP-0008-T02.
Fresh real ReichelAg transfer is waiting for renewed Bubble authentication;
expired/public-only source captures are rejected by the targeted exporter.

View file

@ -125,3 +125,20 @@ Redacted notes for the repo go under `docs/migration/` after review.
- `scripts/capture-auth-state.sh`
- `scripts/export-bubble-session.sh`
- CSOC-WP-0001-T02
## Targeted fresh first-use-case capture
For the nominated ReichelAg transfer, use explicit target mode after renewing
the session. This excludes August seed data, selects the space in the Pages
filter then applies Search, rejects expired signature cookies, and refuses
a one-page or body-incomplete target. A captured tree still needs reconciliation
against the latest expected protocol and private membership map before import.
```bash
COULOMB_SOCIAL_TARGET_SPACE_LABEL=ReichelAG node scripts/export-bubble-corpus.mjs
```
Set COULOMB_SOCIAL_CORPUS_DIR to a private directory if needed. Do not put raw
exports in the checkout. Default target route is /vw_pages; override via
COULOMB_SOCIAL_CAPTURE_ROUTE only after confirming the actual authenticated
protocol surface. A successful public-route load is not authentication proof.

View file

@ -0,0 +1,83 @@
# ReichelAg first transfer: recovery prepared, fresh source needs login
## User authorization and source state
Founder answered “yes go on” to current Bubble capture and first ReichelAg
transfer following the empty-workspace assessment. New tegwick app sign-in and
reset inbox receipt remain accepted. Authority for bounded app snapshot/quiesce:
ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED; continue existing
personal-app/brief-interruption approval and the current first-transfer go.
Initial Bubble capture returned public records and looked usable, but no private
weekly protocols. Subsequent metadata inspection found expired Bubble signature
cookie. No fresh ReichelAg protocol capture or live import is claimed. User asked
to run scripts/capture-auth-state.sh privately and save a new authenticated
session after opening the real protocol list. No password/session/reset values
read into chat, Git or State Hub. Raw source dumps remain private temporary files.
## Capture fixes
Existing exporter guessed the largest captured space, mixed an old seed by
default, overwrote full records with partial responses and did not validate
expired signature cookies. Added explicit target mode using the Space filter
and Search action, disabling old seeds, retaining full fields across partial
fresh responses, normalizing typed Bubble lookup references, awaiting response
handlers and rejecting missing target protocols/body fields. Target mode defaults
to Pages following the user-confirmed Spaces -> ReichelAg -> Pages journey;
actual fresh authenticated protocol list still needs reconciliation.
Four synthetic Node tests cover typed references, expired signature/analytics
distinction, partial-record preservation, and explicit target selection rejecting
larger unrelated spaces or incomplete bodies. A pytest wrapper includes them
in the repo gate. Current expired-session run exits2 before creating any output
directory, so public inventory cannot silently become a real import tree.
## Paired snapshot and isolated restore evidence
Node snapshot helper scripts/snapshot-content-and-db.py default server dry-run
passed against the pinned cluster UID and revision11 image. Execute scopes
pg_dump only to coulomb_social_db; app writers are paused, retained PVC mounted
read-only by a separate pod with no credential projection or service selector.
Pod removed and app1replica restored. Snapshot started19:34:18Z; total operation
53.73seconds; completed/app resumed. Public app health200 afterward.
Database dump49843bytes SHA256
26a1630db0a0603dff33a4f2cb128e5ca3bd914d953c335a0f941162dbddf2b3;
content archive186bytes SHA256
7645781eca69efff473f67dcc736fde7e2a8a66a0172f59056ac57b651b19a25.
Snapshot files mode0600, enclosing directory0700, private node/workstation/tmp.
These are temporary rehearsal artifacts, not durable scheduled backup custody.
Restore helper scripts/restore-paired-snapshot.py validates artifact checksums,
restores native PostgreSQL in a Docker internal network without published ports,
compares every COPY row/table and sequence value, checks archive paths and file
hashes. Exact app image retained; offline instance disables identity integrations.
Initial production-pair restore:13tables/102rows, all data/sequences match, empty
content confirmed, measured database/content restore4.26seconds.
Populate that isolated instance with the synthetic established weekly-call
fixture: full copy, independently edited copy, an asset, owner/viewer/ungranted
actors. Snapshot both stores with no writers running; restore into a second
isolated PostgreSQL instance. Result:13tables/111rows,4content files, all logical
data/sequence/file hashes match, measured restore3.92seconds. Restored Django
checks: original and edited body matches; asset bytes and markdown render match;
owner can read; viewer write with validCSRF denied403; outsider read denied.
Both temporary database containers/internal networks removed afterward.
Four snapshot safety tests exercise default dry-run/no writes, corrupt dump and
cleanup failures resuming the app, and private paired artifact creation. Restore
unit checks exercise changed rows/sequences, incomplete COPY and archive escape.
Full make check:96passed, including established Chromium workflows. Focused
Ruff and Node syntax/capture tests pass.
## Remaining live work
CSOC-WP-0008-T02 remains progress: establish retained backup custody/retention,
measured real-content recovery and production rollback procedure; successful
empty/synthetic recovery does not close actual corpus/visual/privacy acceptance.
T03 waits fresh authenticated Bubble capture. After renewal, capture the correct
current ReichelAg protocols, reconcile latest expected call and assets/privacy,
rehearse import with tenant:platform and only the existing tegwick owner mapping,
refresh paired snapshot before a reviewed live import, then validate visible
space, latest protocol and explicit denied access. T09 is the user's copy/edit/
clipboard/mail-client acceptance; Bubble remains authoritative until accepted.

View file

@ -0,0 +1,83 @@
# ReichelAg first transfer: recovery prepared, fresh source needs login
## User authorization and source state
Founder answered “yes go on” to current Bubble capture and first ReichelAg
transfer following the empty-workspace assessment. New tegwick app sign-in and
reset inbox receipt remain accepted. Authority for bounded app snapshot/quiesce:
ADMINISTER @ realm:kubernetes/railiance01, activation=APPROVED; continue existing
personal-app/brief-interruption approval and the current first-transfer go.
Initial Bubble capture returned public records and looked usable, but no private
weekly protocols. Subsequent metadata inspection found expired Bubble signature
cookie. No fresh ReichelAg protocol capture or live import is claimed. User asked
to run scripts/capture-auth-state.sh privately and save a new authenticated
session after opening the real protocol list. No password/session/reset values
read into chat, Git or State Hub. Raw source dumps remain private temporary files.
## Capture fixes
Existing exporter guessed the largest captured space, mixed an old seed by
default, overwrote full records with partial responses and did not validate
expired signature cookies. Added explicit target mode using the Space filter
and Search action, disabling old seeds, retaining full fields across partial
fresh responses, normalizing typed Bubble lookup references, awaiting response
handlers and rejecting missing target protocols/body fields. Target mode defaults
to Pages following the user-confirmed Spaces -> ReichelAg -> Pages journey;
actual fresh authenticated protocol list still needs reconciliation.
Four synthetic Node tests cover typed references, expired signature/analytics
distinction, partial-record preservation, and explicit target selection rejecting
larger unrelated spaces or incomplete bodies. A pytest wrapper includes them
in the repo gate. Current expired-session run exits2 before creating any output
directory, so public inventory cannot silently become a real import tree.
## Paired snapshot and isolated restore evidence
Node snapshot helper scripts/snapshot-content-and-db.py default server dry-run
passed against the pinned cluster UID and revision11 image. Execute scopes
pg_dump only to coulomb_social_db; app writers are paused, retained PVC mounted
read-only by a separate pod with no credential projection or service selector.
Pod removed and app1replica restored. Snapshot started19:34:18Z; total operation
53.73seconds; completed/app resumed. Public app health200 afterward.
Database dump49843bytes SHA256
26a1630db0a0603dff33a4f2cb128e5ca3bd914d953c335a0f941162dbddf2b3;
content archive186bytes SHA256
7645781eca69efff473f67dcc736fde7e2a8a66a0172f59056ac57b651b19a25.
Snapshot files mode0600, enclosing directory0700, private node/workstation/tmp.
These are temporary rehearsal artifacts, not durable scheduled backup custody.
Restore helper scripts/restore-paired-snapshot.py validates artifact checksums,
restores native PostgreSQL in a Docker internal network without published ports,
compares every COPY row/table and sequence value, checks archive paths and file
hashes. Exact app image retained; offline instance disables identity integrations.
Initial production-pair restore:13tables/102rows, all data/sequences match, empty
content confirmed, measured database/content restore4.26seconds.
Populate that isolated instance with the synthetic established weekly-call
fixture: full copy, independently edited copy, an asset, owner/viewer/ungranted
actors. Snapshot both stores with no writers running; restore into a second
isolated PostgreSQL instance. Result:13tables/111rows,4content files, all logical
data/sequence/file hashes match, measured restore3.92seconds. Restored Django
checks: original and edited body matches; asset bytes and markdown render match;
owner can read; viewer write with validCSRF denied403; outsider read denied.
Both temporary database containers/internal networks removed afterward.
Four snapshot safety tests exercise default dry-run/no writes, corrupt dump and
cleanup failures resuming the app, and private paired artifact creation. Restore
unit checks exercise changed rows/sequences, incomplete COPY and archive escape.
Full make check:96passed, including established Chromium workflows. Focused
Ruff and Node syntax/capture tests pass.
## Remaining live work
CSOC-WP-0008-T02 remains progress: establish retained backup custody/retention,
measured real-content recovery and production rollback procedure; successful
empty/synthetic recovery does not close actual corpus/visual/privacy acceptance.
T03 waits fresh authenticated Bubble capture. After renewal, capture the correct
current ReichelAg protocols, reconcile latest expected call and assets/privacy,
rehearse import with tenant:platform and only the existing tegwick owner mapping,
refresh paired snapshot before a reviewed live import, then validate visible
space, latest protocol and explicit denied access. T09 is the user's copy/edit/
clipboard/mail-client acceptance; Bubble remains authoritative until accepted.

View file

@ -0,0 +1,30 @@
/** Pure capture guards; fixtures must never contain real cookies or notes. */
export function normalizeBubbleRef(value) {
if (typeof value !== 'string') return value;
if (!value.startsWith('LOOKUP__')) return value;
return value.match(/[0-9]+x[0-9]+/)?.[0] || value;
}
export function assertSessionFresh(state, now = Date.now() / 1000) {
if ((state.cookies || []).some(c => c.name.endsWith('.sig') &&
c.expires > 0 && c.expires <= now)) {
throw new Error('bubble_session_expired');
}
}
export function mergeCapturedRecord(existing, source, id) {
return { ...(existing || {}), ...source, _id: normalizeBubbleRef(id) };
}
export function selectTargetArticles(articles, rootId) {
const root = articles[normalizeBubbleRef(rootId)];
const spaceId = normalizeBubbleRef(root?.space_custom_space);
if (!root || !spaceId) throw new Error('target_space_identity_missing');
const selected = Object.values(articles).filter(a =>
normalizeBubbleRef(a.space_custom_space) === spaceId);
if (selected.length <= 1) throw new Error('target_protocols_not_captured');
if (selected.some(a => !Object.hasOwn(a, 'content_text'))) {
throw new Error('target_protocol_bodies_incomplete');
}
return [spaceId, selected];
}

View file

@ -7,6 +7,8 @@
* node scripts/export-bubble-corpus.mjs
*/
import { chromium } from "playwright";
import { assertSessionFresh, normalizeBubbleRef, mergeCapturedRecord,
selectTargetArticles } from "./bubble-corpus-utils.mjs";
import {
mkdirSync,
writeFileSync,
@ -25,6 +27,8 @@ const STATE =
process.env.COULOMB_SOCIAL_STORAGE_STATE ||
join(homedir(), ".config", "coulomb-social", "auth", "storage-state.json");
const BASE = process.env.COULOMB_SOCIAL_URL || "https://coulomb.social";
const TARGET_LABEL = process.env.COULOMB_SOCIAL_TARGET_SPACE_LABEL || "";
const TARGET_ROUTE = process.env.COULOMB_SOCIAL_CAPTURE_ROUTE || "/vw_pages";
const SEED_NET =
process.env.COULOMB_SOCIAL_SEED_NETWORK ||
join(
@ -73,11 +77,8 @@ function slugify(s) {
.slice(0, 80) || "untitled"
);
}
function normId(id) {
if (!id || typeof id !== "string") return id;
const m = id.match(/LOOKUP__([0-9]+x[0-9]+)/);
return m ? m[1] : id;
}
function normId(id) { return normalizeBubbleRef(id); }
function ingest(obj, spaces, articles) {
if (Array.isArray(obj)) {
for (const x of obj) ingest(x, spaces, articles);
@ -88,10 +89,10 @@ function ingest(obj, spaces, articles) {
const src = obj._source && typeof obj._source === "object" ? obj._source : null;
const id = obj._id || (src && src._id);
if (t === "custom.space" && src && id) {
spaces[normId(id)] = { ...src, _id: normId(id) };
spaces[normId(id)] = mergeCapturedRecord(spaces[normId(id)], src, id);
}
if (t === "custom.article" && src && id) {
const a = { ...src, _id: normId(id) };
const a = mergeCapturedRecord(articles[normId(id)], src, id);
if (a.space_custom_space) a.space_custom_space = normId(a.space_custom_space);
articles[normId(id)] = a;
}
@ -162,9 +163,16 @@ if (!existsSync(STATE)) {
process.exit(1);
}
try {
assertSessionFresh(JSON.parse(readFileSync(STATE, "utf8")));
} catch {
console.error("Bubble session unavailable or expired. Run ./scripts/capture-auth-state.sh.");
process.exit(2);
}
const spaces = {};
const articles = {};
if (existsSync(SEED_NET)) {
if (!TARGET_LABEL && process.env.COULOMB_SOCIAL_FRESH_ONLY !== "1" && existsSync(SEED_NET)) {
for (const name of readdirSync(SEED_NET)) {
if (!name.endsWith(".body.txt")) continue;
parseBody(readFileSync(join(SEED_NET, name), "utf8"), spaces, articles);
@ -176,7 +184,9 @@ ensure(OUT);
const browser = await chromium.launch({ headless: true });
const context = await browser.newContext({ storageState: STATE });
const page = await context.newPage();
page.on("response", async (response) => {
const pendingResponses = [];
page.on("response", (response) => {
pendingResponses.push((async () => {
try {
if (!response.url().includes("elasticsearch")) return;
const text = await response.text();
@ -184,6 +194,7 @@ page.on("response", async (response) => {
} catch {
/* ignore */
}
})());
});
async function loadSpacesList() {
@ -195,6 +206,46 @@ async function loadSpacesList() {
}
}
let targetRootId = "";
if (TARGET_LABEL) {
try {
await loadSpacesList();
const link = page.getByText(TARGET_LABEL, { exact: true });
if (await link.count() !== 1) throw new Error("target_space_link_not_unique");
const href = await link.getAttribute("href");
if (!href || !new URL(href, BASE).pathname.startsWith("/pg_chunk/")) {
throw new Error("target_space_root_not_found");
}
targetRootId = new URL(href, BASE).pathname.split("/").pop();
await page.goto(`${BASE}${TARGET_ROUTE}`, { waitUntil: "domcontentloaded", timeout: 60000 });
await page.waitForTimeout(4000);
const filter = page.locator("button").filter({
has: page.locator("svg text").filter({ hasText: /^\s*filter_list\s*$/ }),
});
await filter.first().click();
const selects = page.locator("select");
let selected = false;
for (let i = 0; i < await selects.count(); i++) {
const matches = await selects.nth(i).evaluate((el, label) =>
[...el.options].some(o => o.textContent.trim() === label), TARGET_LABEL);
if (!matches) continue;
await selects.nth(i).selectOption({ label: TARGET_LABEL });
selected = true;
break;
}
if (!selected) throw new Error("target_space_filter_not_found");
await page.getByRole("button", { name: "Search", exact: true }).click();
await page.waitForTimeout(5000);
for (let i = 0; i < 24; i++) {
await page.mouse.wheel(0, 1800);
await page.waitForTimeout(500);
}
} catch {
await browser.close();
console.error("Target capture failed; no import tree published.");
process.exit(3);
}
} else {
for (const name of SPACE_CLICKS) {
try {
await loadSpacesList();
@ -213,9 +264,11 @@ for (const name of SPACE_CLICKS) {
}
console.log("ok", name, "articles", Object.keys(articles).length);
} catch (e) {
console.log("fail", name, String(e.message || e).slice(0, 60));
console.log("fail", name, "capture_step_failed");
}
}
}
await Promise.allSettled(pendingResponses);
await browser.close();
const bySpace = {};
@ -239,11 +292,25 @@ function spaceLabel(sid) {
}
const ranked = Object.entries(bySpace).sort((a, b) => b[1].length - a[1].length);
const [sampleId, sampleArts] = ranked[0] || [null, []];
const lab = spaceLabel(sampleId);
let sampleId, sampleArts;
if (TARGET_LABEL) {
try {
[sampleId, sampleArts] = selectTargetArticles(articles, targetRootId);
} catch {
console.error("Target protocols unavailable; no import tree published. Renew Bubble login.");
process.exit(4);
}
} else {
[sampleId, sampleArts] = ranked[0] || [null, []];
}
const lab = TARGET_LABEL ? { ...spaceLabel(sampleId), slug: slugify(TARGET_LABEL),
title: TARGET_LABEL } : spaceLabel(sampleId);
const index = {
exported_at: new Date().toISOString(),
target_requested: TARGET_LABEL || null,
source_route: TARGET_LABEL ? TARGET_ROUTE : null,
old_seed_used: !TARGET_LABEL && process.env.COULOMB_SOCIAL_FRESH_ONLY !== "1" && existsSync(SEED_NET),
space_count: Object.keys(spaces).length,
article_count: Object.keys(articles).length,
sample_tree: sampleId ? `trees/${lab.slug}` : null,

View file

@ -0,0 +1,160 @@
#!/usr/bin/env python3
"""Restore a private snapshot into isolated Docker PostgreSQL; never production.
No published ports; an internal network blocks external integrations. Compare
all restored COPY rows and sequence state, and all archived file hashes. A
successful empty-content restore alone does not prove real corpus recovery.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import subprocess
import sys
import tarfile
import tempfile
import time
from pathlib import Path
class Refused(RuntimeError):
pass
def run(args: list[str], *, payload: bytes | None = None) -> bytes:
p = subprocess.run(args, input=payload, capture_output=True, timeout=180)
if p.returncode:
raise Refused('command_failed')
return p.stdout
def data_fingerprint(sql: bytes) -> dict:
"""Ignore SQL boilerplate, compare COPY data and explicit sequence state."""
tables = {}
sequences = []
current = None
rows = []
for line in sql.splitlines():
if current is not None:
if line == b'\\.':
tables[current] = {
'rows': len(rows),
'sha256': hashlib.sha256(b'\n'.join(sorted(rows))).hexdigest(),
}
current, rows = None, []
else:
rows.append(line)
elif line.startswith(b'COPY ') and line.endswith(b' FROM stdin;'):
current = line.decode('utf-8')
elif line.startswith(b'SELECT pg_catalog.setval('):
sequences.append(line.decode('utf-8'))
if current is not None:
raise Refused('unterminated_copy')
if not tables:
raise Refused('no_tables_in_dump')
return {'tables': tables, 'sequences': sorted(sequences)}
def content_fingerprint(archive: Path, destination: Path) -> dict[str, str]:
hashes = {}
with tarfile.open(archive, 'r:gz') as t:
for entry in t.getmembers():
parts = Path(entry.name).parts
if (not parts or parts[0] != 'content' or '..' in parts
or Path(entry.name).is_absolute()):
raise Refused('invalid_archive_path')
if not (entry.isfile() or entry.isdir()):
raise Refused('unsupported_archive_entry')
if entry.isfile():
hashes[entry.name] = hashlib.sha256(t.extractfile(entry).read()).hexdigest()
t.extractall(destination, filter='data')
actual = {str(p.relative_to(destination)): hashlib.sha256(p.read_bytes()).hexdigest()
for p in (destination / 'content').rglob('*') if p.is_file()}
if actual != hashes:
raise Refused('content_hash_mismatch')
return hashes
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('snapshot', type=Path)
parser.add_argument('--keep-instance', action='store_true',
help='Keep the isolated internal network/database for a content rehearsal')
args = parser.parse_args()
source = args.snapshot.resolve()
receipt = json.loads((source / 'receipt.json').read_text())
if receipt.get('status') != 'captured' or not receipt.get('writers_quiesced'):
raise Refused('snapshot_not_paired')
for name, key in [('database.dump', 'database_sha256'), ('content.tar.gz', 'content_sha256')]:
if hashlib.sha256((source / name).read_bytes()).hexdigest() != receipt[key]:
raise Refused('artifact_checksum_mismatch')
started = time.monotonic()
name = 'csoc-recovery-' + time.strftime('%Y%m%d%H%M%S', time.gmtime())
network = name + '-net'
work = Path(tempfile.mkdtemp(prefix=name + '-', dir='/tmp'))
image = run(['docker', 'image', 'inspect', 'postgres:16',
'--format', '{{.Id}}']).decode().strip()
if not re.fullmatch(r'sha256:[a-f0-9]{64}', image):
raise Refused('unresolved_postgres_image')
created = False
network_created = False
verified = False
try:
run(['docker', 'network', 'create', '--internal', network])
network_created = True
run(['docker', 'run', '-d', '--name', name, '--network', network,
'--network-alias', 'recovery-db', '-e', 'POSTGRES_HOST_AUTH_METHOD=trust',
'-e', 'POSTGRES_DB=coulomb_social_restore', image])
created = True
for _ in range(45):
p = subprocess.run(['docker', 'exec', name, 'pg_isready', '-U', 'postgres'],
capture_output=True)
if p.returncode == 0:
break
time.sleep(1)
else:
raise Refused('restore_db_not_ready')
archive = (source / 'database.dump').read_bytes()
original_sql = run(['docker', 'exec', '-i', name, 'pg_restore', '--data-only', '-f', '-'],
payload=archive)
baseline = data_fingerprint(original_sql)
run(['docker', 'exec', '-i', name, 'pg_restore', '-U', 'postgres',
'-d', 'coulomb_social_restore', '--no-owner', '--no-acl', '--exit-on-error'],
payload=archive)
restored_sql = run(['docker', 'exec', name, 'pg_dump', '-U', 'postgres',
'-d', 'coulomb_social_restore', '--data-only',
'--no-owner', '--no-acl'])
if data_fingerprint(restored_sql) != baseline:
raise Refused('database_fingerprint_mismatch')
files = content_fingerprint(source / 'content.tar.gz', work)
result = {'status': 'restored_verified', 'database_tables': len(baseline['tables']),
'database_rows': sum(t['rows'] for t in baseline['tables'].values()),
'database_rows_and_sequences_match': True, 'content_files': len(files),
'content_hashes_match': True, 'network_internal': True, 'published_ports': 0,
'postgres_image_id': image, 'elapsed_seconds': round(time.monotonic()-started, 2),
'content_was_empty': not files,
'database_container': name, 'network': network, 'restored_directory': str(work)}
path = work / 'restore-receipt.json'
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(fd, 'w') as f:
json.dump(result, f, indent=2)
verified = True
print(json.dumps(result))
finally:
if not (args.keep_instance and verified):
if created:
run(['docker', 'rm', '-f', '-v', name])
if network_created:
run(['docker', 'network', 'rm', network])
return 0
if __name__ == '__main__':
try:
sys.exit(main())
except Exception as exc:
print(json.dumps({'status': 'failed', 'failure_type': type(exc).__name__}))
sys.exit(1)

View file

@ -0,0 +1,38 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { normalizeBubbleRef, assertSessionFresh, mergeCapturedRecord,
selectTargetArticles } from '../bubble-corpus-utils.mjs';
test('typed lookup references normalize to the same space identifier', () => {
const id = '1700000000000x1234567890';
assert.equal(normalizeBubbleRef(`LOOKUP__custom.space__${id}`), id);
assert.equal(normalizeBubbleRef(`LOOKUP__${id}`), id);
assert.equal(normalizeBubbleRef(id), id);
});
test('expired signature blocks capture; an expired analytics cookie does not', () => {
assert.throws(() => assertSessionFresh({cookies:[{name:'app.sig',expires:9}]},10),
/bubble_session_expired/);
assert.doesNotThrow(() => assertSessionFresh({cookies:[
{name:'analytics',expires:9},{name:'app.sig',expires:11}]},10));
});
test('partial network responses retain full notes captured in this run', () => {
const first = mergeCapturedRecord(null,{content_text:'synthetic full notes'},'root');
assert.equal(mergeCapturedRecord(first,{nomer_text:'Synthetic call'},'root').content_text,
'synthetic full notes');
});
test('explicit target selection never substitutes the largest space', () => {
const records = {root:{space_custom_space:'target',content_text:'synthetic index'},
call:{space_custom_space:'target',content_text:'synthetic protocol'},
a:{space_custom_space:'other'},b:{space_custom_space:'other'},c:{space_custom_space:'other'}};
const [id, selected] = selectTargetArticles(records,'root');
assert.equal(id,'target');
assert.equal(selected.length,2);
delete records.call.content_text;
assert.throws(() => selectTargetArticles(records,'root'), /target_protocol_bodies_incomplete/);
records.call.content_text = 'synthetic protocol';
delete records.call;
assert.throws(() => selectTargetArticles(records,'root'), /target_protocols_not_captured/);
});

View file

@ -0,0 +1,12 @@
"""Run synthetic source-capture guards in the same Node runtime as the exporter."""
import subprocess
from pathlib import Path
def test_bubble_capture_guards():
root = Path(__file__).resolve().parents[1]
result = subprocess.run(
['node', '--test', 'scripts/tests/bubble-corpus-utils.test.mjs'],
cwd=root, capture_output=True, text=True, check=False,
)
assert result.returncode == 0, result.stdout + result.stderr

View file

@ -0,0 +1,41 @@
"""Logical backup comparisons must catch changed rows, sequences and paths."""
import importlib.util
import io
import tarfile
from pathlib import Path
import pytest
@pytest.fixture
def helper():
path = Path(__file__).resolve().parents[1] / 'scripts/restore-paired-snapshot.py'
spec = importlib.util.spec_from_file_location('csoc_restore', path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def test_data_comparison_ignores_order_but_detects_changes(helper):
a = b'COPY public.demo (id, value) FROM stdin;\n1\tone\n2\ttwo\n\\.\n'
b = b'-- SQL header\nCOPY public.demo (id, value) FROM stdin;\n2\ttwo\n1\tone\n\\.\n'
assert helper.data_fingerprint(a) == helper.data_fingerprint(b)
assert helper.data_fingerprint(a) != helper.data_fingerprint(b.replace(b'two', b'changed'))
assert helper.data_fingerprint(a) != helper.data_fingerprint(
a + b"SELECT pg_catalog.setval('public.demo_seq', 9, true);\n")
def test_partial_copy_is_rejected(helper):
with pytest.raises(helper.Refused):
helper.data_fingerprint(b'COPY public.demo (id) FROM stdin;\n1\n')
def test_content_archive_rejects_escape(helper, tmp_path):
archive = tmp_path / 'bad.tar.gz'
with tarfile.open(archive, 'w:gz') as t:
info = tarfile.TarInfo('content/../../outside')
info.size = 4
t.addfile(info, io.BytesIO(b'data'))
with pytest.raises(helper.Refused):
helper.content_fingerprint(archive, tmp_path / 'restored')
assert not (tmp_path / 'outside').exists()

View file

@ -102,7 +102,7 @@ snapshots; no private archive in git or State Hub.
```task
id: CSOC-WP-0008-T03
status: progress
status: wait
priority: high
state_hub_task_id: "0615eff2-bf2b-503b-8f15-0591d4666446"
```
@ -395,7 +395,7 @@ prerequisite. Diagnostic receipt: history/20261006T190725Z-empty-personal-spaces
## Fresh ReichelAg transfer execution — 2026-10-06
Founder answered “yes go on” to current Bubble capture and first ReichelAg
transfer. Existing private browser session valid. Old exporter selects largest
transfer. Saved browser session probed; expired signature cookie prevents private protocol access. Old exporter selects largest
captured space and assumes title click sets active space; neither proves target
selection. Fresh source capture must explicitly select ReichelAG in Pages filter
and apply Search, with old network seed disabled. Retain source-ID provenance,
@ -406,3 +406,24 @@ server dry-run passed. Quiesce app, scope pg_dump to coulomb_social_db, read-onl
PVC archive, restore app even on failure; isolated PostgreSQL restore must pass
before real import. Four focused safety tests pass (dry-run/no writes, corrupt
dump/app-resume, cleanup failure/app-resume, private paired artifacts).
Paired production snapshot completed; app resumed/healthy. Isolated PostgreSQL
restore matches13tables/102rows with empty content. A second synthetic populated
pair restores13tables/111rows and4files; full-protocol copy/independent edit,
asset/render fidelity, viewer validCSRF write denial and outsider read denial
pass after recovery.96tests pass. T02 remains progress for retained backup
custody/retention and real corpus/rollback review.
T03 wait: Bubble signature cookie expired; public-only captures are not accepted
source. User must privately renew via scripts/capture-auth-state.sh and open
actual ReichelAg protocols. Targeted exporter now fails before output on expired
session, uses explicit target/Space filter/Search, excludes old seeds, merges
partial fresh records and rejects missing protocol bodies. Source receipt:
docs/release/reichelag-transfer-2026-10-06.md. No productive import performed.
User confirmed the old journey: Spaces -> ReichelAg -> Pages, ordered by latest
changes. Capture defaults to /vw_pages. Prioritize current content migration;
UI framework architecture is a separate next phase (CSOC-WP-0011), before broad
UX rebuilding. Existing browser session renewal or a dedicated Bubble account
with read access can provide the source; never accept password/session contents
in chat. User renewal remains the next input needed for private protocol capture.

View file

@ -0,0 +1,68 @@
---
id: CSOC-WP-0011
type: workplan
title: "Establish UI framework architecture before broad Bubble UX rebuild"
domain: communication
repo: coulomb-social
status: backlog
owner: codex
topic_slug: coulomb-social
created: "2026-10-06"
updated: "2026-10-06"
---
# User milestone and priority
After the first content transfer, the interface has a consistent foundation for
finding spaces, opening pages ordered by latest changes, editing/copying protocols
and distributing notes by clipboard. New Bubble use cases can reuse the same
navigation, forms, interaction and permission patterns. The framework decision
is explicit and reviewed before broad UX investment; no new frontend stack is
assumed merely to make the current migration work.
User steering on 2026-10-06: prioritize content migration, then establish framework
architecture before rebuilding many UI elements. CSOC-WP-0008-T02/T03/T09 remain
first; this backlog workplan does not block source capture or the first transfer.
## Inventory existing UI and observed Bubble interaction patterns
```task
id: CSOC-WP-0011-T01
status: todo
priority: medium
```
Review current Django templates, routes, forms, permission checks and clipboard
JavaScript together with user-reviewed Bubble journeys. Describe navigation,
latest-change ordering, protocol copying and editing, reusable components and
accessibility needs. Use captured UI evidence separately from private content.
## Decide the UI framework boundaries and extension contract
```task
id: CSOC-WP-0011-T02
status: todo
priority: medium
```
Prepare an ADR comparing the existing server-rendered foundation with progressive
interaction or a distinct frontend only where user needs justify it. Specify
component ownership, routing/state, server validation, auth/permission boundary,
asset delivery and regression test contract. Review with the user before broad
UX implementation. Avoid coupling the content migration format to the UI stack.
## Demonstrate the foundation on the accepted ReichelAg journey
```task
id: CSOC-WP-0011-T03
status: todo
priority: medium
```
After content acceptance and framework review, implement one representative
space/page interaction on the decided foundation. Keep the full-protocol copy,
independent edit, latest-protocol discovery and native clipboard workflows under
established regression coverage. User reviews consistency and usability before
other Bubble UI surfaces are rebuilt. Register later UI gaps as live follow-ups.
Founder content-first decision: e67cfd2e-e55d-4b42-947a-b279b6287c55.