diff --git a/docs/architecture/ArchitectureBlueprint.md b/docs/architecture/ArchitectureBlueprint.md index 1c03434..cd4cee7 100644 --- a/docs/architecture/ArchitectureBlueprint.md +++ b/docs/architecture/ArchitectureBlueprint.md @@ -271,9 +271,10 @@ See **[spike-checklist.md](./spike-checklist.md)** — executable evaluation tas ## References - `docs/architecture/2026-08-12-content-substrate-exploration.md` +- `docs/architecture/transclusion-and-chunks.md` — page-first model; chunks as extract/transclude - `docs/capability/stage-1-cutover.md` - `docs/decisions/2026-08-12-feature-cut-stage-1.md` - kontextual: `docs/architecture-blueprint.md`, `asset-registry-implementation.md`, `blob-storage-content-streaming-workplan.md`, `markitect-tool-reuse-boundary.md` - shard-wiki: `INTENT.md`, `SCOPE.md` -- markitect-tool: `docs/markitect-main-scope-assessment.md` +- markitect-tool: `docs/markitect-main-scope-assessment.md`, `docs/transform-compose-include.md`, `docs/content-references.md` - markitect-main succession: `docs/successor-gap-assessment.md` diff --git a/docs/architecture/transclusion-and-chunks.md b/docs/architecture/transclusion-and-chunks.md new file mode 100644 index 0000000..34dd750 --- /dev/null +++ b/docs/architecture/transclusion-and-chunks.md @@ -0,0 +1,189 @@ +# Transclusion vs a priori chunks — coulomb + markitect + +| Field | Value | +|-------|--------| +| Date | 2026-08-12 | +| Status | design note (aligns with ArchitectureBlueprint + existing Bubble map) | +| Layer | markitect-tool (syntax) · coulomb product model | + +## Thesis + +**Pages of markdown are the fluent unit of knowledge work.** +A “chunk” should usually mean **an addressable region or extracted view of a page** (or of another page), not a separate first-class storage object that documents are assembled from *a priori*. + +Bubble’s `custom.chunk` graph (stages, subchunks, backlog) is powerful but **tool-shaped**: it forces structure before fluency. Rebuild should invert that: + +```text +Write fluently in a page + → address regions when reuse is needed + → transclude / extract / promote + → only then materialize a derived page or asset if it earns independence +``` + +Tooling adapts; it does not demand an ontology of chunks up front. + +--- + +## What markitect-tool already provides + +### 1. Include / transclusion (FR-022) + +Documented in `markitect-tool/docs/transform-compose-include.md`. + +**Markers:** + +```markdown + + + + +{{include:sections/intro.md}} +``` + +| Attribute | Role | +|-----------|------| +| `path` | Target markdown file (relative; sandboxed under base dir) | +| `selector` | Optional query — **only matching content** is included | +| `heading_delta` | Shift heading levels on include | +| `include_frontmatter` | Optionally keep target frontmatter | + +**API:** `resolve_includes(...)` +**Safety:** cycle detection, max depth, no path escape, markers inside fences left literal +**Provenance:** include events record marker line, target path, selector, heading shift + +This is **transclusion without a database**: the including page remains editable markdown; resolution is a deterministic operation for render/export/agents. + +### 2. Selectors (query / extract) + +`docs/query-extraction.md` — extract sections/blocks without inventing chunk rows: + +```text +sections[heading=Context] +sections[heading~=risk] +sections[contains~=PROBLEM] +blocks[type=paragraph] +headings[level=2] +``` + +`mkt transform --extract SELECTOR` can replace a document with extracted content — **promote a region to a standalone body** when needed. + +### 3. Content references (addressable units) + +`docs/content-references.md` — name units without a priori chunk types: + +```text +path/to/file.md +path/to/file.md#section:introduction +path/to/file.md::sections[heading=Decision] +std:clauses/payment.md#region:boilerplate +``` + +**Named regions** (in-document, optional): + +```markdown + +Reusable text. + +``` + +Units carry `unit_id`, kind (`document` | `section` | `heading` | …), source span, content hash — enough for cache, ACLs later, and include targets. + +### 4. Explode / implode (optional materialization) + +`docs/explode-implode.md` — temporarily split a large page into section files + manifest, edit, reassemble. +**Not** “documents are made of chunk types”; it is a **reversible workflow** when humans want filesystem granularity. + +### 5. Compose + +`compose_files` — concatenate whole files (different from selective transclusion). + +--- + +## Mapping to coulomb / Bubble concepts + +| Bubble (live) | Page-first + transclusion model | +|---------------|----------------------------------| +| `custom.page` | **SoR page** — frontmatter + body + backmatter | +| `custom.chunk` as storage atoms | **Avoid a priori** — use sections/regions/selectors inside the page | +| Chunk list / SelectedChunk | **Derived view**: query `sections[...]` or region index on the page | +| Subchunks | Nested headings / nested regions, not separate types | +| Copy page | Duplicate page file (or deep-copy with rewritten includes) | +| Transfer page | Move/re-home page file; includes keep relative or get rewritten | +| Card UI | Projection of page (title, abstractor, visual) — may also project a **transcluded excerpt** as card body | +| Article + chunks_list | Page that **includes** other pages or regions, or a view over query results | +| Stage1…Stage5 on chunks | Optional **workflow metadata** on regions/frontmatter later — not required for S1 | + +### “Extract a chunk out of a page” + +Three elegant operations, increasing commitment: + +| Op | Meaning | Storage impact | +|----|---------|----------------| +| **Address** | Point at `page.md#section:foo` or `::sections[heading=Foo]` | None — still one page | +| **Transclude** | Another page includes that region via `mkt:include` + selector | Still one SoR for the region; host page has a marker | +| **Promote / extract** | Write region to `pages/foo-excerpt.md` (transform extract or explode entry) | New page; optional leave include behind in origin | + +Fluency prefers **address → transclude → promote**, not **create chunk type first**. + +--- + +## Product rules (proposed) + +1. **SoR is the page file** (markdown). No required `Chunk` table for stage 1. +2. **Cards** show page projections; optional card body = first section or selected region. +3. **Reuse** uses markitect include/reference, resolved at render/export/agent time (and optionally cached). +4. **Independent editing of a reused piece** = promote to its own page, then transclude from both places. +5. **DB/cache** may index section titles and region ids for search/cards — **disposable**, rebuilt from markdown. +6. Bubble chunk migration: prefer **serialize chunk trees into page sections/regions** or separate pages with includes, not recreate `custom.chunk` as a permanent model. + +--- + +## Where this sits in the architecture + +```text +coulomb UI (cards, edit, copy, transfer) + │ + ▼ +PageOps (get/put page markdown) + │ + ├─ markitect-tool: parse, query, resolve_includes, extract, references + └─ content plane: store page files (+ assets) +``` + +Transclusion is a **syntax-layer** concern (markitect-tool), not a reason to pick kontextual vs shard-wiki. Any content plane that stores markdown files can host includes. + +--- + +## Gaps / future (markitect-tool) + +From markitect docs themselves (not blockers for S1): + +- Variable / conditional transclusion not yet +- Includes currently path-first; content-reference-as-include path is designed for later +- Dependency graph invalidation for caches is planned (WP-0007 hooks) + +Coulomb can start with **path + selector includes** and **section/region addressing**. + +--- + +## Relation to stage-1 cut + +| Stage-1 need | Transclusion stance | +|--------------|---------------------| +| Title / Abstractor / Visual | Page frontmatter | +| Page CRUD | Whole page files | +| Page copy | File duplicate; resolve or rewrite includes | +| Page transfer | Move file between spaces; rewrite relative includes | +| Chunk UX (if users need “pieces”) | Address/transclude first; promote only if sticky | + +--- + +## One-liner + +**Don’t model knowledge as Lego chunks that assemble into pages. +Model pages as living markdown, and treat chunks as addressable, transcludable extractions that tooling materializes only when reuse demands it.** + +## Related + +- markitect-tool: `docs/transform-compose-include.md`, `docs/content-references.md`, `docs/query-extraction.md`, `docs/explode-implode.md` +- coulomb: `docs/architecture/ArchitectureBlueprint.md`, `docs/capability/existing-bubble-capability-map.md` diff --git a/docs/capability/existing-bubble-capability-map.md b/docs/capability/existing-bubble-capability-map.md index a3164bf..ce33ec2 100644 --- a/docs/capability/existing-bubble-capability-map.md +++ b/docs/capability/existing-bubble-capability-map.md @@ -154,7 +154,10 @@ Utility: **composable content block** inside pages/articles (finer than page). **Bubble type:** `custom.chunk` **Views:** `vw_chunks`, `pg_chunk` -**Rebuild note:** Stage-1 may treat **page markdown body** as the unit and **defer chunk graph**, unless migration shows chunks are the real user object. Confirm with editor export. +**Rebuild note:** Prefer **page-first + transclusion** (markitect include/selector/region) +over recreating `custom.chunk` as storage atoms. See +`docs/architecture/transclusion-and-chunks.md`. Confirm with product use whether +users need chunk lists as primary UX or only as derived views. ### 3.3 Article / scrap / content variants — `cap.bubble.article-family`