tegwick
c27a50e6d3
Fold page-first/transclusion, dual authoring, lazy projections, stale replace-or-keep, and open content-plane decisions into the main draft.
307 lines
15 KiB
Markdown
307 lines
15 KiB
Markdown
# Architecture Blueprint — coulomb.social content & platform
|
||
|
||
| Field | Value |
|
||
|-------|--------|
|
||
| Status | **draft for discussion** (revised 2026-08-13) |
|
||
| Product | coulomb.social rebuild (`coulomb-social`) |
|
||
| Related | stage-1 cut, capability maps, substrate exploration, authoring, stale/derived |
|
||
| Non-goals | Final store vendor lock-in; full UI chrome; implementing markitect features in this doc |
|
||
|
||
## 0. Review summary (2026-08-13)
|
||
|
||
This revision folds in topics settled after the first draft:
|
||
|
||
| Topic | Stance |
|
||
|-------|--------|
|
||
| Identity / tenants / groups | NetKingdom only |
|
||
| Deploy | Railiance |
|
||
| Content SoR | Markdown + assets; DB = cache |
|
||
| markitect | **tool** as structure lib; **not** “run on markitect”; **main** = legacy |
|
||
| Page vs chunk | **Page-first**; chunks = address / transclude / promote |
|
||
| Authoring | Top-down **and** bottom-up; no forced mode |
|
||
| Title / Abstractor / Visual | Real fields; may start **lazy**; then persist; **regenerate + lock** |
|
||
| Stale derived content | Badge → generate **alternative** → **replace or keep** |
|
||
| Includes | Live resolve **and** (proposed) pinned snapshots with same stale grammar |
|
||
| Content plane | Still open: thin store / shard-wiki / kontextual (spikes) |
|
||
| Assets | kontextual BlobStorage now; extract asset-store when 2nd consumer |
|
||
| Live Bubble | Mapped; S1 aligns with Title/Abstractor/Visual/spaces/pages/cards |
|
||
|
||
**Still open (decisions, not re-litigate principles):** content-plane pick (A/B/C), visual storage short-term, PolicyGateway owner, chunk migration strategy.
|
||
|
||
---
|
||
|
||
## 1. Architectural aim
|
||
|
||
Deliver a co-creation product where:
|
||
|
||
1. **Identity** (users, groups, tenants) is **NetKingdom** — not reimplemented.
|
||
2. **Runtime** is **Railiance** — not reimplemented.
|
||
3. **User-owned content** is **markdown (+ assets)** exportable without the app database.
|
||
4. The **database is a cache / projection** for cards, memberships, search speed.
|
||
5. **markitect-tool** makes markdown an efficient structured substrate (lib/CLI/API) — **not** the product host.
|
||
6. **coulomb** owns co-creation UI (cards, spaces, copy/transfer, authoring UX) and product verbs.
|
||
7. Knowledge work stays **fluent**: top-down and bottom-up authoring; structure and reuse appear when needed (transclusion, lazy projections), not as mandatory a priori chunk graphs.
|
||
|
||
## 2. Non-negotiable rules
|
||
|
||
| # | Rule |
|
||
|---|------|
|
||
| R1 | Markdown (frontmatter + body + backmatter) + assets are the **exportable SoR** for content. |
|
||
| R2 | App DB holds **projections and app linkage only** for content; disposable for bodies. |
|
||
| R3 | Authn/IdP = **NetKingdom**; coulomb maps OIDC → Member; no password store. |
|
||
| R4 | Authz composes NetKingdom groups, app membership, content-plane ACLs — **no second user directory**. |
|
||
| R5 | **markitect-main** is legacy reference, not a runtime foundation. |
|
||
| R6 | **markitect-tool** is the markdown structure dependency (parse, validate, query, include, later derived slots). |
|
||
| R7 | Content plane (shard-wiki / kontextual / thin store) sits behind a **narrow PageOps** interface. |
|
||
| R8 | Stage-1 must: onboarding; spaces/pages with **Title / Abstractor / Visual**; page copy; page transfer. |
|
||
| R9 | **Pages first** — do not model documents as a priori chunk assemblies; use address/transclude/promote. |
|
||
| R10 | Derived content (projections, pinned includes, pipelines, AI) uses **stale badge → alternative → replace\|keep** and **lock**; no silent overwrite of locked/user content. |
|
||
|
||
## 3. System shape
|
||
|
||
```text
|
||
┌──────────────────────────┐
|
||
│ NetKingdom │
|
||
│ OIDC, groups, tenants │
|
||
└────────────┬─────────────┘
|
||
│ principal
|
||
▼
|
||
┌─────────────────────────────────────────────────────────────┐
|
||
│ coulomb.social (product) │
|
||
│ • Card UI, spaces chrome, dual authoring flows │
|
||
│ • Session / Member / SpaceMembership │
|
||
│ • PageOps + projection UX (lazy / stale / lock) │
|
||
│ • Railiance deploy │
|
||
└───────────────┬─────────────────────────────┬───────────────┘
|
||
│ projections │ content ops
|
||
▼ ▼
|
||
┌───────────────┐ ┌───────────────────────────┐
|
||
│ App DB cache │ │ Content plane (adapter) │
|
||
│ (Postgres) │ │ A thin folder/git │
|
||
└───────────────┘ │ B shard-wiki (thin) │
|
||
│ C kontextual-engine │
|
||
└─────────────┬─────────────┘
|
||
│ structure
|
||
▼
|
||
┌───────────────────────────┐
|
||
│ markitect-tool │
|
||
│ parse, FM, query, include │
|
||
│ derived/stale (proposed) │
|
||
└─────────────┬─────────────┘
|
||
▼
|
||
┌───────────────────────────┐
|
||
│ Asset bytes (blobs) │
|
||
│ content-addressed │
|
||
└───────────────────────────┘
|
||
```
|
||
|
||
### PageOps (store-agnostic)
|
||
|
||
| Op | Meaning |
|
||
|----|---------|
|
||
| `list_pages(space)` | Card projections |
|
||
| `get_page(id)` | Full markdown + resolved visual; optional include resolve |
|
||
| `put_page(...)` | Create/update attributes + body |
|
||
| `archive_page` | Soft-delete / lifecycle |
|
||
| `copy_page` | Independent duplicate (rewrite includes as needed) |
|
||
| `transfer_page` | Re-home to another space |
|
||
| `put_visual` / `get_asset` | Binary visual/attachment |
|
||
| `project_field` / `regenerate` / `lock` | Lazy & stale projection actions |
|
||
| `derived_refresh` | Stale include/pipeline: alternative → replace\|keep (when markitect supports) |
|
||
|
||
Coulomb templates/services do **not** call Forgejo/shard/kontextual APIs ad hoc — only PageOps.
|
||
|
||
## 4. Information model (product)
|
||
|
||
| Concept | Representation |
|
||
|---------|----------------|
|
||
| **Page** | Markdown: frontmatter (title, abstractor, visual, space, projections lifecycle, …) + body + backmatter |
|
||
| **Space** | Container; may be `type: space` page or container id in frontmatter |
|
||
| **Visual** | Asset ref → blob store |
|
||
| **Card** | UI only: visual + title + abstractor + actions (+ stale badges) |
|
||
| **Region / “chunk”** | Addressable unit of a page (heading, `mkt:region`, selector) — **not** a required storage type |
|
||
| **Transclusion** | Live `mkt:include` and/or pinned `mkt:derived` snapshot (proposed) |
|
||
| **Member / tenant** | NetKingdom + app session |
|
||
|
||
### 4.1 Page-first vs Bubble chunks
|
||
|
||
Live Bubble has `custom.chunk` graphs. Rebuild stance:
|
||
|
||
```text
|
||
Write fluently in a page
|
||
→ address regions when needed
|
||
→ transclude / extract / promote
|
||
→ materialize a new page only when reuse earns independence
|
||
```
|
||
|
||
See `transclusion-and-chunks.md`. Migration may flatten Bubble chunks into sections/regions/pages+includes.
|
||
|
||
### 4.2 Dual authoring
|
||
|
||
| Mode | Flow |
|
||
|------|------|
|
||
| **Top-down** | Title → abstractor → outline (headings) → elaborate body → visual |
|
||
| **Bottom-up** | Body first; title/abstractor/visual/structure later |
|
||
|
||
No mandatory mode toggle. Empty body with metadata is valid; body-only is valid.
|
||
|
||
### 4.3 Lazy projections + lock + stale alternatives
|
||
|
||
For **title**, **abstractor**, **visual** (and similar display fields):
|
||
|
||
1. Missing on display need → **generate** (lazy) → **persist** as normal field values.
|
||
2. **Regenerate** and **lock** are per-field.
|
||
3. If inputs changed → **May be stale** → **Generate updated version** → compare Current \| Alternative → **Keep** or **Use alternative**.
|
||
|
||
Detail: `authoring-modes-and-lazy-projections.md`, `stale-derived-content.md`.
|
||
|
||
Same grammar for **pinned includes** and **pipeline/AI/rule** blocks (markitect proposal: `mkt:derived`).
|
||
|
||
## 5. Content-plane options (comparison)
|
||
|
||
### 5.1 Option A — Thin folder/git store + markitect-tool
|
||
|
||
| | |
|
||
|--|--|
|
||
| **Idea** | `.md` tree + assets; markitect validates; app indexes cards |
|
||
| **Pros** | Fastest S1; trivial export; low structure tax |
|
||
| **Cons** | Copy/transfer/versioning are app work; federation later costs more |
|
||
| **When** | Velocity first; plane maturity unclear |
|
||
|
||
### 5.2 Option B — shard-wiki (thin slice)
|
||
|
||
| | |
|
||
|--|--|
|
||
| **Idea** | Headless wiki orchestration; IdP external; coulomb = UI |
|
||
| **Pros** | Page union, git coordination, multi-shard later; INTENT names Coulomb |
|
||
| **Cons** | Structure tax if federation leaks into product; multi-tenant ops still maturing |
|
||
| **Thin-slice rule** | One InformationSpace / instance; one primary shard; hide federation from cards |
|
||
|
||
### 5.3 Option C — kontextual-engine
|
||
|
||
| | |
|
||
|--|--|
|
||
| **Idea** | Knowledge ops: asset identity, representations, blobs, audit, workflows; markdown via markitect adapter |
|
||
| **Pros** | Blobs ready; lineage; multi-format future; PolicyGateway port |
|
||
| **Cons** | Heavier; NetKingdom not wired; must enforce markdown export as happy path |
|
||
| **Auth gap** | Map OIDC → Actor; PolicyGateway → flex-auth |
|
||
|
||
### 5.4 Evolution (not exclusive forever)
|
||
|
||
```text
|
||
PageOps → A or thin B/C now
|
||
→ folder/git shard can become shard-wiki
|
||
→ assets can move to kontextual / asset-store
|
||
→ markitect-tool always structure layer
|
||
```
|
||
|
||
**Working shortlist (unchanged):** spike **A** as control + **B thin** and **C** in parallel; pick after scorecard (`spike-checklist.md`).
|
||
|
||
## 6. Asset management
|
||
|
||
| Source | Role |
|
||
|--------|------|
|
||
| kontextual BlobStorage | Prefer for Visual/bytes now (fs/S3/memory, content-addressed) |
|
||
| markitect-main assets | Ideas only — do not runtime-depend |
|
||
| Extract `asset-store` | When **second** consumer needs same API; seed from kontextual port |
|
||
|
||
Visual field stores a **ref** (path or `sha256:…`); bytes never live only in Postgres.
|
||
|
||
## 7. Identity & access control
|
||
|
||
```text
|
||
Browser → NetKingdom OIDC → coulomb session
|
||
├─ product authz: tenant_id, SpaceMembership
|
||
└─ content plane: Actor(external_ref=iss|sub, groups=…)
|
||
PolicyGateway → flex-auth (preferred)
|
||
```
|
||
|
||
Fail closed on ambiguous access. Spike: OIDC → Actor → cross-tenant deny.
|
||
|
||
## 8. Deploy sketch (Railiance)
|
||
|
||
| Component | Deploy |
|
||
|-----------|--------|
|
||
| coulomb-social | Existing chart; `app.coulomb.social` |
|
||
| Content plane | In-process, sidecar, or platform service |
|
||
| Asset bytes | PVC and/or S3; secrets via env-secret pattern |
|
||
| NetKingdom | Existing KeyCape / Authelia |
|
||
|
||
Apex DNS stays Bubble until stage-1 cut + transfer readiness.
|
||
|
||
## 9. Stage-1 product scope (capability cut)
|
||
|
||
**Must for productive transfer** (Bubble → reference-only):
|
||
|
||
- Member onboarding (NetKingdom; public reg may lag via provisioned users)
|
||
- Spaces + pages with **Title, Abstractor, Visual** + markdown body
|
||
- Page **copy** and **transfer**
|
||
- Card presentation
|
||
- Dual authoring + lazy projections (at least title/abstractor); stale replace|keep preferred early
|
||
|
||
**Later:** social, ventures/invest, challenges marketplace, monetization, full chunk-stage machine, multi-shard personal vaults.
|
||
|
||
Live Bubble map: `docs/capability/existing-bubble-capability-map.md`.
|
||
|
||
## 10. ADR roadmap
|
||
|
||
| ADR | Topic |
|
||
|-----|--------|
|
||
| ADR-0001 | NetKingdom identity (**done**) |
|
||
| ADR-0002 | Forgejo binding — **demote** to one adapter/export shape |
|
||
| **ADR-0003** | Page-centric model + SoR + DB cache + page-first/transclusion |
|
||
| **ADR-0004** | Content-plane choice + PageOps |
|
||
| **ADR-0005** | Asset-store extraction (if/when) |
|
||
| **ADR-0006** | Authoring modes + projections + stale/derived (or fold into 0003) |
|
||
|
||
Markitect-tool: implement `proposed-stale-derived-and-include-pinning.md` as its own work.
|
||
|
||
## 11. Spike checklist
|
||
|
||
See **[spike-checklist.md](./spike-checklist.md)**.
|
||
Add when scheduling: dual-authoring smoke, stale projection compare UX, include pin (after markitect).
|
||
|
||
## 12. Discussion prompts (remaining)
|
||
|
||
1. Content plane for next 3–6 months: **A velocity**, **B federation path**, or **C governance/blobs**?
|
||
2. Visual: kontextual/local blob first, or block on asset-store extract?
|
||
3. PolicyGateway production owner?
|
||
4. Bubble **chunk** migration: sections/regions only, or keep a chunk list UX as a *view*?
|
||
5. Default include mode in product: **live** vs **pinned** for shared excerpts?
|
||
|
||
## 13. Working recommendation (principles frozen; plane open)
|
||
|
||
| Layer | Recommendation |
|
||
|-------|----------------|
|
||
| Product UI | coulomb cards + spaces + dual authoring |
|
||
| Markdown structure | markitect-tool (includes + future derived slots) |
|
||
| Identity / deploy | NetKingdom / Railiance |
|
||
| Content plane | Spike A+B+C; choose with export fidelity as hard constraint |
|
||
| SoR | Markdown export always works; engines add IDs/lineage without trapping content |
|
||
| Assets | kontextual blobs now; extract later if shared |
|
||
| Chunks | Transclusion/extract, not a priori graph |
|
||
| Derived/stale | Badge → alternative → replace\|keep + lock |
|
||
| markitect-main | Avoid |
|
||
|
||
---
|
||
|
||
## References
|
||
|
||
### coulomb-social
|
||
|
||
- `docs/architecture/2026-08-12-content-substrate-exploration.md`
|
||
- `docs/architecture/transclusion-and-chunks.md`
|
||
- `docs/architecture/authoring-modes-and-lazy-projections.md`
|
||
- `docs/architecture/stale-derived-content.md`
|
||
- `docs/architecture/spike-checklist.md`
|
||
- `docs/capability/stage-1-cutover.md`
|
||
- `docs/capability/model-v0.yaml`
|
||
- `docs/capability/existing-bubble-capability-map.md`
|
||
- `docs/decisions/2026-08-12-feature-cut-stage-1.md`
|
||
|
||
### fleet
|
||
|
||
- kontextual: architecture-blueprint, asset-registry, blob-storage, markitect reuse boundary
|
||
- shard-wiki: INTENT, SCOPE
|
||
- markitect-tool: transform-compose-include, content-references, cache-incremental, **proposed-stale-derived-and-include-pinning.md**
|
||
- markitect-main: successor-gap-assessment (legacy)
|