tegwick
d953aed4dd
D1 thin dir+git with shard-wiki then kontextual upgrade path; D2 defer asset-store; D3 NetKingdom authz consolidation; D4 outline explode/implode and live-by-default includes with pin as exception.
204 lines
8.1 KiB
Markdown
204 lines
8.1 KiB
Markdown
# Transclusion vs a priori chunks — coulomb + markitect
|
||
|
||
| Field | Value |
|
||
|-------|--------|
|
||
| Date | 2026-08-12 |
|
||
| Status | design note (aligns with ArchitectureBlueprint + existing Bubble map) |
|
||
| Layer | markitect-tool (syntax) · coulomb product model |
|
||
|
||
## Thesis
|
||
|
||
**Pages of markdown are the fluent unit of knowledge work.**
|
||
A “chunk” should usually mean **an addressable region or extracted view of a page** (or of another page), not a separate first-class storage object that documents are assembled from *a priori*.
|
||
|
||
Bubble’s `custom.chunk` graph (stages, subchunks, backlog) is powerful but **tool-shaped**: it forces structure before fluency. Rebuild should invert that:
|
||
|
||
```text
|
||
Write fluently in a page
|
||
→ address regions when reuse is needed
|
||
→ transclude / extract / promote
|
||
→ only then materialize a derived page or asset if it earns independence
|
||
```
|
||
|
||
Tooling adapts; it does not demand an ontology of chunks up front.
|
||
|
||
---
|
||
|
||
## What markitect-tool already provides
|
||
|
||
### 1. Include / transclusion (FR-022)
|
||
|
||
Documented in `markitect-tool/docs/transform-compose-include.md`.
|
||
|
||
**Markers:**
|
||
|
||
```markdown
|
||
<!-- mkt:include path="sections/intro.md" -->
|
||
|
||
<!-- mkt:include path="other.md" selector="sections[heading=Summary]" heading_delta="1" -->
|
||
|
||
{{include:sections/intro.md}}
|
||
```
|
||
|
||
| Attribute | Role |
|
||
|-----------|------|
|
||
| `path` | Target markdown file (relative; sandboxed under base dir) |
|
||
| `selector` | Optional query — **only matching content** is included |
|
||
| `heading_delta` | Shift heading levels on include |
|
||
| `include_frontmatter` | Optionally keep target frontmatter |
|
||
|
||
**API:** `resolve_includes(...)`
|
||
**Safety:** cycle detection, max depth, no path escape, markers inside fences left literal
|
||
**Provenance:** include events record marker line, target path, selector, heading shift
|
||
|
||
This is **transclusion without a database**: the including page remains editable markdown; resolution is a deterministic operation for render/export/agents.
|
||
|
||
### 2. Selectors (query / extract)
|
||
|
||
`docs/query-extraction.md` — extract sections/blocks without inventing chunk rows:
|
||
|
||
```text
|
||
sections[heading=Context]
|
||
sections[heading~=risk]
|
||
sections[contains~=PROBLEM]
|
||
blocks[type=paragraph]
|
||
headings[level=2]
|
||
```
|
||
|
||
`mkt transform --extract SELECTOR` can replace a document with extracted content — **promote a region to a standalone body** when needed.
|
||
|
||
### 3. Content references (addressable units)
|
||
|
||
`docs/content-references.md` — name units without a priori chunk types:
|
||
|
||
```text
|
||
path/to/file.md
|
||
path/to/file.md#section:introduction
|
||
path/to/file.md::sections[heading=Decision]
|
||
std:clauses/payment.md#region:boilerplate
|
||
```
|
||
|
||
**Named regions** (in-document, optional):
|
||
|
||
```markdown
|
||
<!-- mkt:region id="boilerplate" tags="legal reuse" -->
|
||
Reusable text.
|
||
<!-- /mkt:region -->
|
||
```
|
||
|
||
Units carry `unit_id`, kind (`document` | `section` | `heading` | …), source span, content hash — enough for cache, ACLs later, and include targets.
|
||
|
||
### 4. Explode / implode (optional materialization)
|
||
|
||
`docs/explode-implode.md` — temporarily split a large page into section files + manifest, edit, reassemble.
|
||
**Not** “documents are made of chunk types”; it is a **reversible workflow** when humans want filesystem granularity.
|
||
|
||
### 5. Compose
|
||
|
||
`compose_files` — concatenate whole files (different from selective transclusion).
|
||
|
||
---
|
||
|
||
## Mapping to coulomb / Bubble concepts
|
||
|
||
| Bubble (live) | Page-first + transclusion model |
|
||
|---------------|----------------------------------|
|
||
| `custom.page` | **SoR page** — frontmatter + body + backmatter |
|
||
| `custom.chunk` as storage atoms | **Avoid a priori** — use sections/regions/selectors inside the page |
|
||
| Chunk list / SelectedChunk | **Derived view**: query `sections[...]` or region index on the page |
|
||
| Subchunks | Nested headings / nested regions, not separate types |
|
||
| Copy page | Duplicate page file (or deep-copy with rewritten includes) |
|
||
| Transfer page | Move/re-home page file; includes keep relative or get rewritten |
|
||
| Card UI | Projection of page (title, abstractor, visual) — may also project a **transcluded excerpt** as card body |
|
||
| Article + chunks_list | Page that **includes** other pages or regions, or a view over query results |
|
||
| Stage1…Stage5 on chunks | Optional **workflow metadata** on regions/frontmatter later — not required for S1 |
|
||
|
||
### “Extract a chunk out of a page”
|
||
|
||
Three elegant operations, increasing commitment:
|
||
|
||
| Op | Meaning | Storage impact |
|
||
|----|---------|----------------|
|
||
| **Address** | Point at `page.md#section:foo` or `::sections[heading=Foo]` | None — still one page |
|
||
| **Transclude** | Another page includes that region via `mkt:include` + selector | Still one SoR for the region; host page has a marker |
|
||
| **Promote / extract** | Write region to `pages/foo-excerpt.md` (transform extract or explode entry) | New page; optional leave include behind in origin |
|
||
|
||
Fluency prefers **address → transclude → promote**, not **create chunk type first**.
|
||
|
||
---
|
||
|
||
## Product rules (proposed)
|
||
|
||
1. **SoR is the page file** (markdown). No required `Chunk` table for stage 1.
|
||
2. **Cards** show page projections; optional card body = first section or selected region.
|
||
3. **Reuse** uses markitect include/reference, resolved at render/export/agent time (and optionally cached).
|
||
4. **Independent editing of a reused piece** = promote to its own page, then transclude from both places.
|
||
5. **DB/cache** may index section titles and region ids for search/cards — **disposable**, rebuilt from markdown.
|
||
6. Bubble chunk migration: prefer **serialize chunk trees into page sections/regions** or separate pages with includes, not recreate `custom.chunk` as a permanent model.
|
||
|
||
---
|
||
|
||
## Where this sits in the architecture
|
||
|
||
```text
|
||
coulomb UI (cards, edit, copy, transfer)
|
||
│
|
||
▼
|
||
PageOps (get/put page markdown)
|
||
│
|
||
├─ markitect-tool: parse, query, resolve_includes, extract, references
|
||
└─ content plane: store page files (+ assets)
|
||
```
|
||
|
||
Transclusion is a **syntax-layer** concern (markitect-tool), not a reason to pick kontextual vs shard-wiki. Any content plane that stores markdown files can host includes.
|
||
|
||
---
|
||
|
||
## Gaps / future (markitect-tool)
|
||
|
||
From markitect docs themselves (not blockers for S1):
|
||
|
||
- Variable / conditional transclusion not yet
|
||
- Includes currently path-first; content-reference-as-include path is designed for later
|
||
- Dependency graph invalidation for caches is planned (WP-0007 hooks)
|
||
- **Pinned derived snapshots + stale → alternative → replace|keep** — proposed in
|
||
markitect-tool `docs/proposed-stale-derived-and-include-pinning.md`
|
||
(coulomb product note: `stale-derived-content.md`)
|
||
|
||
Coulomb can start with **path + selector includes** and **section/region addressing**,
|
||
then pin/freeze excerpts when “my version vs live source” matters.
|
||
|
||
### Live include vs pinned snapshot
|
||
|
||
| Mode | Behavior |
|
||
|------|----------|
|
||
| **Live** `mkt:include` (**default**) | Always resolve current source — spreadsheet-like |
|
||
| **Pinned** `mkt:derived` (proposed) | Explicit freeze; badge when inputs change; alternative → replace \| keep |
|
||
|
||
**Product default is live.** Pin only when the user freezes a value.
|
||
Outline flexibility uses **explode / implode** into connected docs with live
|
||
links, not a priori chunk tables.
|
||
|
||
---
|
||
|
||
## Relation to stage-1 cut
|
||
|
||
| Stage-1 need | Transclusion stance |
|
||
|--------------|---------------------|
|
||
| Title / Abstractor / Visual | Page frontmatter |
|
||
| Page CRUD | Whole page files |
|
||
| Page copy | File duplicate; resolve or rewrite includes |
|
||
| Page transfer | Move file between spaces; rewrite relative includes |
|
||
| Chunk UX (if users need “pieces”) | Address/transclude first; promote only if sticky |
|
||
|
||
---
|
||
|
||
## One-liner
|
||
|
||
**Don’t model knowledge as Lego chunks that assemble into pages.
|
||
Model pages as living markdown, and treat chunks as addressable, transcludable extractions that tooling materializes only when reuse demands it.**
|
||
|
||
## Related
|
||
|
||
- markitect-tool: `docs/transform-compose-include.md`, `docs/content-references.md`, `docs/query-extraction.md`, `docs/explode-implode.md`
|
||
- coulomb: `docs/architecture/ArchitectureBlueprint.md`, `docs/capability/existing-bubble-capability-map.md`
|