From ce65b359af162e9754e3af93b9c67dad895ae11f Mon Sep 17 00:00:00 2001 From: tegwick Date: Thu, 13 Aug 2026 10:04:46 +0200 Subject: [PATCH] Add ADR-0003 and ADR-0004 for page SoR and thin git content plane Accept page-centric markdown SoR with dual authoring, live includes, and lazy projections; accept thin dir+git plane with PageOps and supported upgrades to shard-wiki and kontextual. Demote ADR-0002 Forgejo-as-SoR. --- ...ADR-0002-space-content-forgejo-markdown.md | 3 +- .../adr/ADR-0003-page-centric-markdown-sor.md | 169 ++++++++++++++++ ...DR-0004-content-plane-thin-git-upgrades.md | 189 ++++++++++++++++++ docs/architecture/ArchitectureBlueprint.md | 18 +- 4 files changed, 369 insertions(+), 10 deletions(-) create mode 100644 docs/adr/ADR-0003-page-centric-markdown-sor.md create mode 100644 docs/adr/ADR-0004-content-plane-thin-git-upgrades.md diff --git a/docs/adr/ADR-0002-space-content-forgejo-markdown.md b/docs/adr/ADR-0002-space-content-forgejo-markdown.md index 94a35cd..83faad0 100644 --- a/docs/adr/ADR-0002-space-content-forgejo-markdown.md +++ b/docs/adr/ADR-0002-space-content-forgejo-markdown.md @@ -2,10 +2,11 @@ | Field | Value | |-------|--------| -| Status | **Provisional** (accepted for T04 vertical slice; revisit write path) | +| Status | **Superseded in part** by ADR-0003 / ADR-0004 (2026-08-13). Markdown form and “not Postgres for bodies” remain. **One Forgejo repo per space as product SoR** is demoted to optional remote/export. | | Date | 2026-08-12 | | Deciders | bernd | | Workplan | CSOC-WP-0004-T03 | +| Superseded by | ADR-0003, ADR-0004 | ## Context diff --git a/docs/adr/ADR-0003-page-centric-markdown-sor.md b/docs/adr/ADR-0003-page-centric-markdown-sor.md new file mode 100644 index 0000000..fb238c5 --- /dev/null +++ b/docs/adr/ADR-0003-page-centric-markdown-sor.md @@ -0,0 +1,169 @@ +--- +id: ADR-0003 +title: Page-centric markdown SoR, authoring, and live derived content +status: accepted +date: 2026-08-13 +deciders: bernd +related: + - ADR-0001 + - ADR-0002 + - ADR-0004 + - docs/architecture/ArchitectureBlueprint.md + - docs/decisions/2026-08-12-feature-cut-stage-1.md +workplans: + - CSOC-WP-0004 + - CSOC-WP-0006 + - CSOC-WP-0001 +--- + +# ADR-0003 — Page-centric markdown SoR, authoring, and live derived content + +## Status + +**Accepted** (2026-08-13). + +Supersedes the parts of **ADR-0002** that treated “one Forgejo repo per space +as the product SoR.” ADR-0002 remains valid as a **historical vertical slice** +and as **one optional git/Forgejo binding shape** under ADR-0004. + +## Context + +The rebuild must exit Bubble with **user-owned content** that is easy to +export and reuse. Live coulomb models spaces, pages, and chunks; stage-1 cut +requires Title, Abstractor, Visual, copy, and transfer without recreating +Bubble’s full chunk graph. + +Prior work delivered a Forgejo-bound read/write slice (ADR-0002). Architecture +discussion established: + +- Markdown (+ assets) as exportable system of record; database as cache. +- **markitect-tool** as structure library (not product host); markitect-main avoided. +- Page-first knowledge work with transclusion rather than a priori chunks. +- Dual authoring (top-down and bottom-up) and lazy display fields. +- Live includes by default (spreadsheet-like); pin as exception. + +## Decision + +### 1. System of record + +1. **Content SoR** is **markdown files** (frontmatter + body + backmatter) and + **assets** under a **directory tree** versioned in **git** (see ADR-0004). +2. **Postgres** (or any app DB) holds only: + - identity linkage (`Member` ↔ NetKingdom subject), + - product indexes / **projections** for cards and lists, + - optional pointers into the tree (paths, digests) — + **not** long-form page bodies as SoR. +3. Anything a user should take on export must live in the tree; DB content for + bodies is disposable and rebuildable. + +### 2. Information model + +| Concept | Meaning | +|---------|---------| +| **Page** | Primary knowledge object: one markdown document (or connected set after explode). | +| **Space** | Container / context for pages (directory + metadata; may itself be a page of `type: space`). | +| **Card** | UI projection only (title, abstractor, visual, actions) — not a storage type. | +| **Region / “chunk”** | Addressable unit *within* a page (heading, named region, selector). **Not** a required first-class storage row. | +| **Member / tenant** | NetKingdom + app session (ADR-0001); not stored as page content. | + +**Stage-1 fields** on pages and spaces (frontmatter or equivalent): + +- **title** +- **abstractor** (short description; Bubble “Abstractor”) +- **visual** (ref to an asset) +- **body** (markdown) + +### 3. Page-first structure (not a priori chunks) + +1. Authors write **pages** fluently; structure is headings and optional named regions. +2. Reuse uses **address → live transclude → promote** (markitect include/selector/region), not a mandatory `Chunk` entity graph. +3. Bubble `custom.chunk` migration **prefers** sections/regions or connected pages with includes — not permanent recreation of chunk tables. +4. Outline flexibility uses markitect **explode / implode** (and related compose/include): split one document into smaller **connected** documents; merge selected docs back into one. Connectivity after explode prefers **live** includes or content references. + +### 4. Dual authoring modes + +Both are first-class; no forced mode toggle: + +| Mode | Flow | +|------|------| +| **Top-down** | Title → abstractor → outline (headings) → elaborate body → optional visual | +| **Bottom-up** | Body first; title / abstractor / structure / visual established later or while writing | + +Creating or saving content **must not** require a complete metadata form. + +### 5. Lazy projections, lock, and staleness + +For display fields (title, abstractor, visual) and other **derived** values: + +1. If **absent** and display needs them → **generate lazily** → **persist** as normal field values (as if user-provided for export). +2. Per-field **regenerate** and **lock** (lock blocks auto-fill and regenerate until unlock). +3. If inputs change and the field is unlocked derived → **May be stale** badge → **Generate updated version** → side-by-side **Current | Alternative** → **Keep current** or **Use alternative** (replace). Optional ack updates fingerprint without changing text. +4. **Never** silently overwrite **locked** or last-**user**-edited values without consent. + +Generator policy: deterministic first (e.g. first H1, first paragraph); optional LLM/pipeline behind the same contract. + +### 6. Live by default; pin as exception + +| Mode | Behavior | +|------|----------| +| **Live (default)** | Includes and formula-like derived content resolve to **current** inputs on view/export (spreadsheet metaphor). | +| **Pinned (exception)** | Explicit freeze of a snapshot in the host document; refresh uses stale → alternative → replace \| keep. | + +Pure live includes do not require replace/keep; pinned/`mkt:derived` (markitect proposal) and projection fields do. + +### 7. markitect-tool role + +1. coulomb **depends on markitect-tool** for parse, frontmatter, query/extract, **live include**, explode/implode, and (when implemented) derived-slot stale/refresh APIs. +2. coulomb does **not** “run on” markitect as a product platform; markitect-main is not a runtime foundation. +3. Cross-repo: pin/derived contract tracked in markitect-tool + `docs/proposed-stale-derived-and-include-pinning.md`. + +### 8. Relationship to ADR-0002 + +| ADR-0002 item | Under ADR-0003/0004 | +|---------------|---------------------| +| Markdown as content form | **Confirmed** | +| Assets beside content | **Confirmed** (tree layout; asset-store later) | +| Postgres not body SoR | **Confirmed** | +| One Forgejo repo per space as SoR | **Demoted** — optional remote/export; SoR is the **content tree + git** (ADR-0004) | +| Edit-in-Forgejo as primary write | **Optional** path; product PageOps may write tree/git directly | + +## Consequences + +### Positive + +- User data remains files; agents and humans edit without the app DB. +- Stage-1 fields and card UI align with live Bubble vocabulary (Title, Abstractor, Visual). +- Fluency: top-down and bottom-up; live reuse; outline explode/implode without chunk ontology tax. +- Clear demotion of Forgejo-as-unique-SoR reduces lock-in to one forge layout. + +### Negative / costs + +- App must implement projection index invalidation when files change. +- Live includes need careful caching and security (path sandbox, cycles) via markitect. +- Lazy/AI generators need UX discipline so writers are not interrupted or overwritten. +- Explode/implode UX is non-trivial product work (connectivity, navigation). + +### Neutral + +- Identity remains ADR-0001; content authz consolidation is NetKingdom-side (ArchitectureBlueprint D3). +- Content plane storage mechanics and upgrades are **ADR-0004**. + +## Non-goals + +- Recreating Bubble marketplace, social graph, or monetization in this ADR. +- Choosing shard-wiki or kontextual as the *initial* plane (upgrade path only). +- Extracting a fleet asset-store repo (deferred). +- Full markitect derived-slot implementation timeline (owned by markitect-tool). + +## References + +- `docs/architecture/ArchitectureBlueprint.md` (D1–D4, R1–R12) +- `docs/architecture/transclusion-and-chunks.md` +- `docs/architecture/authoring-modes-and-lazy-projections.md` +- `docs/architecture/stale-derived-content.md` +- `docs/capability/stage-1-cutover.md` +- `docs/capability/existing-bubble-capability-map.md` +- ADR-0002 (provisional slice; partially superseded) +- markitect-tool: `docs/transform-compose-include.md`, `docs/explode-implode.md`, + `docs/proposed-stale-derived-and-include-pinning.md` diff --git a/docs/adr/ADR-0004-content-plane-thin-git-upgrades.md b/docs/adr/ADR-0004-content-plane-thin-git-upgrades.md new file mode 100644 index 0000000..4d62a09 --- /dev/null +++ b/docs/adr/ADR-0004-content-plane-thin-git-upgrades.md @@ -0,0 +1,189 @@ +--- +id: ADR-0004 +title: Thin git content plane, PageOps, and supported upgrades +status: accepted +date: 2026-08-13 +deciders: bernd +related: + - ADR-0001 + - ADR-0002 + - ADR-0003 + - docs/architecture/ArchitectureBlueprint.md +workplans: + - CSOC-WP-0004 + - CSOC-WP-0006 +--- + +# ADR-0004 — Thin git content plane, PageOps, and supported upgrades + +## Status + +**Accepted** (2026-08-13). + +Pairs with **ADR-0003** (what a page is and how authors work). This ADR decides +**where content lives**, how the app talks to it, and how the plane may grow. + +## Context + +Content needs a store that: + +- keeps **user data as files** in **git**, +- starts **simple** for stage-1 transfer, +- can later gain wiki federation or ECM-class power **without rewriting** the + product UI or stranding exports. + +Candidates considered: + +| Option | Role | +|--------|------| +| **A** Thin directory + markdown + git | Start here | +| **B** shard-wiki | Wiki orchestration / multi-shard later | +| **C** kontextual-engine | Scale, rich media, governed assets later | + +Also: shared **asset-store** extraction, and who owns content-plane PolicyGateway. + +## Decision + +### 1. Default content plane (stage-1 and near term) + +**Start thin (A):** + +- A **directory tree** of markdown pages and assets. +- Tree is **secured in git** (commits via product, operator, or linked remote). +- Illustrative layout: + +```text +spaces// + pages/.md + assets/… +``` + +Exact layout may evolve; the invariant is **files + git**, not “opaque DB rows +for prose.” + +**Forgejo / remote git** (including ADR-0002-style bindings) may host or mirror +the tree; they are **transports/remotes**, not a second product ontology. + +### 2. PageOps boundary + +All product code accesses content through a **narrow PageOps** façade (names +illustrative): + +| Op | Purpose | +|----|---------| +| list/get/put/archive page | CRUD | +| copy / transfer page | Stage-1 must features | +| explode / implode | Outline-editor flexibility (ADR-0003) | +| resolve_view | Live includes/derived for display | +| pin / unpin / refresh_pinned | Explicit freeze + stale workflow | +| project_field / regenerate / lock | Title/abstractor/visual lifecycle | +| put_visual / get_asset | Binaries | + +**Rule:** UI and product services **must not** call shard-wiki, kontextual, or +Forgejo APIs ad hoc from templates — only PageOps. Swapping the content-plane +**adapter** upgrades the backend without rewriting cards and flows. + +### 3. Supported upgrade path + +```text +A thin dir + git (+ markitect-tool) + │ + │ when wiki / federation / multi-shard matters + ▼ +B shard-wiki adapter (same markdown as shard material) + │ + │ when scale / rich multi-format media / heavy governance + ▼ +C kontextual-engine upgrade (migration tooling; markdown remains portable) +``` + +| Upgrade | When | Invariant | +|---------|------|-----------| +| **→ shard-wiki** | Wiki features, multi-shard, overlays become product-relevant | Exportable markdown tree remains valid; markitect still structures docs | +| **→ kontextual** | Ambitious structure, scale, rich media, audit/lineage | Supported **upgrade of the information base**; user export of markdown+assets still required where content is document-shaped | + +Upgrades are **planned migrations**, not emergency rewrites. Feasibility should +be re-checked with export/import sketches when approaching B or C. + +### 4. Assets + +1. **Now:** assets live **beside** markdown in the content tree (paths or simple + digests in frontmatter `visual` / refs). +2. **Later:** extract a shared **asset-store** (seed from kontextual BlobStorage + ideas) when a **second fleet consumer** needs the same API — **not** a + stage-1 gate. +3. Do **not** adopt markitect-main’s large assets package as runtime. + +### 5. Authorization ownership + +1. **Authn** remains NetKingdom OIDC (ADR-0001). +2. **Do not** create a long-lived coulomb-specific PolicyGateway ownership + stream. Prefer **NetKingdom consolidation** so authorization decisioning for + app resources is the obvious platform path (flex-auth / NK as one story). +3. **Until consolidation lands:** coulomb enforces coarse product authz + (authenticated session, `tenant_id`, SpaceMembership). Content plane fails + closed if a PDP is configured and denies or is required but unavailable for + sensitive ops. +4. Content-plane engines (shard-wiki, kontextual) must accept **external** + principal identity (e.g. issuer/subject + groups), not their own user + directories. + +### 6. markitect-tool on every plane + +Regardless of A/B/C: + +- parse, validate, query, **live include**, explode/implode, and future + derived-slot tooling live in **markitect-tool**; +- the plane stores and serves files (or equivalent); markitect does not host + multi-user product sessions. + +## Consequences + +### Positive + +- Fastest path to stage-1 with true user-owned files and git history. +- Clear upgrade story reduces fear of “too simple forever” without overbuilding. +- PageOps keeps UI stable across backend generations. +- Authz consolidation direction avoids duplicate IAM mini-platforms. + +### Negative / costs + +- Thin plane must still implement solid git/write, conflict, and cache-bust + behavior for a good product feel. +- Copy/transfer/explode semantics are coulomb+markitect work on the tree. +- Upgrade tooling to B/C is real future cost (accepted as supported path). +- NK authz consolidation is a **platform dependency** outside this repo’s sole + control. + +### Neutral + +- ADR-0002 Forgejo demo path can remain for public raw fixtures and optional + remotes. +- asset-store extraction is explicit **later** work (possible ADR-0005). + +## Non-goals + +- Implementing full shard-wiki or kontextual integration in stage-1. +- Bundling a wiki UI or ECM UI in coulomb (headless planes only). +- Replacing NetKingdom with app-local accounts. +- Making Postgres the content SoR. + +## Implementation guidance (non-normative) + +1. Define a `ContentPlane` / PageOps port in coulomb; ship **Filesystem+Git** + adapter first. +2. Index cards into Postgres on write and on git pull/webhook. +3. Wire markitect live `resolve_includes` on view/export. +4. Prototype explode/implode in product against the same tree. +5. Document export: zip/tar of tree without DB dump. +6. When approaching B/C: migration runbook + round-trip tests from a real + space tree. + +## References + +- `docs/architecture/ArchitectureBlueprint.md` (D1–D4) +- `docs/architecture/spike-checklist.md` (A chosen; B/C upgrade sketches) +- ADR-0003 (page model, live/pin, authoring) +- ADR-0002 (Forgejo vertical slice; demoted as exclusive SoR) +- shard-wiki INTENT (Coulomb as intended consumer) +- kontextual-engine (upgrade target; blob port for later asset-store) diff --git a/docs/architecture/ArchitectureBlueprint.md b/docs/architecture/ArchitectureBlueprint.md index c70c363..511a56e 100644 --- a/docs/architecture/ArchitectureBlueprint.md +++ b/docs/architecture/ArchitectureBlueprint.md @@ -247,16 +247,16 @@ Browser → NetKingdom OIDC → coulomb session **Later:** social, marketplace, monetization, shard-wiki/kontextual upgrades, asset-store extract. -## 10. ADR roadmap +## 10. ADRs -| ADR | Topic | -|-----|--------| -| ADR-0001 | NetKingdom identity (**done**) | -| ADR-0002 | Forgejo slice → **one optional git remote / export**, not unique SoR | -| **ADR-0003** | Page model + dir+git SoR + page-first + live default + authoring/stale | -| **ADR-0004** | PageOps + **thin plane** + **upgrade path** B/C | -| **ADR-0005** | asset-store extraction (**later**) | -| — | NetKingdom authz consolidation (NK work, not coulomb-owned forever) | +| ADR | Topic | Status | +|-----|--------|--------| +| ADR-0001 | NetKingdom identity | accepted | +| ADR-0002 | Forgejo vertical slice | partly superseded by 0003/0004 | +| **ADR-0003** | Page-centric markdown SoR, authoring, live/pin | **accepted** | +| **ADR-0004** | Thin git plane, PageOps, upgrades B/C | **accepted** | +| ADR-0005 | asset-store extraction | later | +| — | NetKingdom authz consolidation | platform (NK) | Markitect-tool: live include (done), explode/implode (done), pin/derived proposal (`proposed-stale-derived-and-include-pinning.md`).