coulomb-social/docs/architecture/2026-08-12-content-substrate-exploration.md

167 lines
9.3 KiB
Markdown
Raw Normal View History

# Content substrate exploration — coulomb.social
| Field | Value |
|-------|--------|
| Date | 2026-08-12 |
| Status | exploration (direction preferred; not full ADR yet) |
| Related | stage-1 cut, ADR-0002, markitect-tool, shard-wiki, NetKingdom |
## Constraints (agreed)
1. **Users, groups, tenants****NetKingdom** (OIDC, user-engine, later groups/roles). Coulomb does not reimplement identity directory.
2. **Deploy / operate****Railiance** (images, Helm, railiance01).
3. **User data is the users** — exportable in a **reuse-friendly** form. Prefer **markdown chunks** as **source of truth**.
4. **Database is a cache** for speed (lists, cards, search projections), not the SoR for page bodies/attributes that users care about owning.
5. **Not “running on markitect.”** markitect-tool is a **lib/service for markdown-as-efficient-datastore tooling**, not the product host.
6. **ADR-0002 Forgejo binding** is a useful vertical slice, not the definition of the product information model.
## Product information model (working)
| Concept | Role |
|---------|------|
| **Page** | Universal information object: frontmatter + body + backmatter (markdown) |
| **Space** | Grouping / context of pages (may itself be page-like later) |
| **Card UI** | Presentation of page projections (visual + title + abstract + actions) |
| **Member / group / tenant** | Identity plane — NetKingdom + app session, not page store |
## Layering (preferred)
```text
┌─────────────────────────────────────────────────────────────┐
│ coulomb.social product UI (cards, spaces chrome, flows) │
│ + session, membership UX, stage-1 product verbs │
├─────────────────────────────────────────────────────────────┤
│ Product application service (Django or equivalent) │
│ — authz against NetKingdom principal │
│ — page ops API: CRUD, copy, transfer │
│ — projection cache (Postgres): card lists, memberships │
├──────────────────────┬──────────────────────────────────────┤
│ markitect-tool │ content store / wiki orchestration │
│ parse, validate, │ markdown chunks on disk/git/… │
│ front/back matter, │ (adapter: folder, git, shard-wiki, │
│ transform, query │ Forgejo, …) │
├──────────────────────┴──────────────────────────────────────┤
│ NetKingdom identity │ Railiance runtime │
└─────────────────────────────────────────────────────────────┘
```
**SoR rule:** anything a user would take with them on export lives in **markdown (and assets)**. DB holds **derived** indexes and app-only linkage (e.g. Member UUID ↔ NetKingdom subject) that is not “content.”
---
## Decision: markitect-main vs markitect-tool
| | **markitect-tool** | **markitect-main** |
|--|--------------------|--------------------|
| Role | Syntax-layer **successor**: markdown as protocol | Older **monolithic** platform (parse + infospace + spaces + assets + UI experiments + ops glue) |
| Intent | Library + `mkt` CLI; no accounts/hosting | Full document platform; being **retired / split** |
| For coulomb | **Use** as dependency for structured markdown | **Avoid as runtime foundation** |
| Evidence | `markitect-tool/docs/markitect-main-scope-assessment.md`: tool is clean contract, not a trimmed copy of main; mains infospace/spaces/DB/graphql out of tool scope | `markitect-main/docs/successor-gap-assessment.md`: main → tool + kontextual-engine + infospace-bench + filter + quarkdown |
**Decision (this exploration):**
- Treat **markitect-main as legacy reference / mine for ideas**, not as coulombs foundation.
- Integrate **markitect-tool** (and its contracts) for document structure operations.
- Persistence/orchestration of multi-backend wiki shape → evaluate **shard-wiki** / engine, not markitect-main `spaces/`.
---
## Decision direction: shard-wiki as backend
### What shard-wiki is
From `shard-wiki` INTENT/SCOPE:
- **Git-based markdown wiki orchestrator and federation layer**
- Joins **shards** (folder, git, native engine, future adapters) into a **union of pages**
- **Headless, API-first**; UI is a consumer concern (fits coulomb UI)
- **Does not implement** authn / identity store — **delegates to external IdP** (NetKingdom / user-engine) — matches coulomb
- Explicitly lists **“Systems such as Coulomb spaces”** as intended consumers
- Early-stage but substantial foundation (adapters, union, overlays, git coordination log, native engine shard) — not vapor intent only
### Fit to coulomb constraints
| Need | shard-wiki fit |
|------|----------------|
| Markdown as SoR | Strong — pages are wiki-shaped markdown content |
| Export / user-owned data | Strong — shards can be folder/git; take the files |
| DB as cache | Compatible — coulomb projects cards into Postgres; shard-wiki not required to be the card DB |
| NetKingdom identity | Compatible — IdP external; shard-wiki owns authz model for pages/shards |
| Avoid reimplementing wiki federation | Strong |
| Risk: “too much structure” | Real — namespaces, overlays, federation, capability profiles, multi-shard resolution are more than a single-tenant space CRUD MVP |
### “Too much structure?”
shard-wikis structure is **mechanism** (resolution, overlays, provenance, multi-shard), not a fixed **product ontology** (it refuses a universal knowledge ontology). Risk for coulomb is **operational and API surface**:
- If coulomb calls the full federation graph on day one, stage-1 product work drowns in coordination concepts.
- If coulomb uses a **thin slice**: one InformationSpace per tenant (or per product instance), **one primary shard** (folder or git or native engine) for space pages, overlays only when needed — structure stays **under the hood**.
**Evaluation stance:**
| Approach | When |
|----------|------|
| **A. shard-wiki as content plane** | Prefer if we want federation (personal + space shards, export shards, git sync) and headless page ops soon |
| **B. coulomb thin page store + markitect-tool** | Prefer if stage-1 ships faster with a single folder/git layout; adopt shard-wiki when multi-shard or federation is real demand |
| **C. kontextual-engine** | Broader knowledge ops; evaluate if page-only wiki is too narrow long-term — heavier product fit analysis |
**Preferred exploration order:** prove stage-1 page ops against a **minimal shard-wiki deployment** (or adapter conformance test) **in parallel** with a **simple folder/git adapter design**. Choose A if integration cost is acceptable and API stays stable; else B with a migration path **to** shard-wiki (adapter contract is designed for that).
Do **not** pick markitect-main as the store.
---
## NetKingdom / Railiance (unchanged ownership)
| Concern | Owner |
|---------|--------|
| User accounts, login, MFA, groups/roles claims | NetKingdom |
| Application Member link (issuer, subject) | coulomb (JIT) |
| Tenant model | instance-per-client + `tenant_id` (DR-1 C); platform claims when ready |
| Images, deploy, probes, secrets mount | Railiance / railiance-apps |
| Content markdown + page authz | coulomb product + content plane (shard-wiki or thin store) |
---
## Data ownership sketch
```text
Export package (user or space):
pages/**/*.md # frontmatter + body + backmatter
assets/** # visuals, attachments
MANIFEST.json # optional: ids, space map, hashes
```
Rebuild / reimport should not require coulombs Postgres. Cache tables are disposable.
---
## Implications for ADR-0002 and stage 1
- **ADR-0002:** keep as “Forgejo may host a git shard / export”; demote “one repo per space is the only SoR” when ADR-0003 (page substrate) lands.
- **Stage-1 features** (CRUD, visual, title, abstract, copy, transfer) become **page operations** on the content plane + **card projections** in coulomb UI.
- **markitect-tool** validates/normalizes frontmatter fields; does not host multi-user sessions.
---
## Open evaluation tasks (if we continue)
Executable checklist + blueprint:
- [`ArchitectureBlueprint.md`](./ArchitectureBlueprint.md)
- [`spike-checklist.md`](./spike-checklist.md)
Includes kontextual-engine option, asset-store extraction thesis, and NetKingdom PolicyGateway gap.
## Summary decisions from this note
| Topic | Decision |
|-------|----------|
| markitect-main | **Avoid** as foundation; legacy reference only |
| markitect-tool | **Adopt** as markdown structure lib/service |
| “Run on markitect” | **Reject** framing |
| Markdown SoR + DB cache | **Adopt** |
| Identity / deploy | **NetKingdom / Railiance** (no reimplementation) |
| shard-wiki | **Serious candidate** for content plane; use **thin slice** to avoid over-structure; spike before hard commit |
| Next write-up | ADR-0003 after spikes or founder pick A vs B |