tegwick
e2aa4045f4
Document top-down and bottom-up writing, lazy title/abstractor/visual that persist after generation, and per-field regenerate plus lock. Wire into capability model and stage-1 notes.
281 lines
15 KiB
Markdown
281 lines
15 KiB
Markdown
# Architecture Blueprint — coulomb.social content & platform
|
||
|
||
| Field | Value |
|
||
|-------|--------|
|
||
| Status | **draft for discussion** (2026-08-12) |
|
||
| Product | coulomb.social rebuild (`coulomb-social`) |
|
||
| Related | stage-1 cut, capability model, content-substrate exploration |
|
||
| Non-goals | Final store vendor lock-in; implementing UI chrome in this doc |
|
||
|
||
## 1. Architectural aim
|
||
|
||
Deliver a co-creation product where:
|
||
|
||
1. **Identity** (users, groups, tenants) is **NetKingdom** — not reimplemented.
|
||
2. **Runtime** is **Railiance** — not reimplemented.
|
||
3. **User-owned content** is **markdown (+ assets)** that can be exported and reused without the app database.
|
||
4. The **database is a cache / projection** for cards, memberships, and speed.
|
||
5. **markitect-tool** enables efficient structured markdown (lib/service), not a product host.
|
||
6. **coulomb** owns co-creation UI (playing-card pages, spaces, transfer/copy) and product verbs.
|
||
|
||
## 2. Non-negotiable rules
|
||
|
||
| # | Rule |
|
||
|---|------|
|
||
| R1 | Markdown chunks (frontmatter + body + backmatter) + assets are the **exportable SoR** for content. |
|
||
| R2 | Postgres (or any app DB) holds **projections and app linkage only** for content; disposable for bodies. |
|
||
| R3 | Authn/IdP = **NetKingdom**; coulomb maps OIDC → Member; no password store. |
|
||
| R4 | Authz for product actions may compose NetKingdom groups, app membership, and content-plane ACLs — **no second user directory**. |
|
||
| R5 | **markitect-main** is legacy reference, not a runtime foundation. |
|
||
| R6 | **markitect-tool** is the markdown structure dependency (parse/validate/transform). |
|
||
| R7 | Content-plane choice (shard-wiki / kontextual / thin store) is behind a **narrow page-ops interface** so coulomb UI does not couple to federation or ECM graphs. |
|
||
| R8 | Stage-1 capability set remains: onboarding, spaces/pages with Title/Abstract/Visual, page copy, page transfer. |
|
||
|
||
## 3. System shape
|
||
|
||
```text
|
||
┌──────────────────────────┐
|
||
│ NetKingdom │
|
||
│ OIDC, groups, tenants │
|
||
└────────────┬─────────────┘
|
||
│ principal
|
||
▼
|
||
┌─────────────────────────────────────────────────────────────┐
|
||
│ coulomb.social (product) │
|
||
│ • Card UI, spaces chrome, stage-1 product flows │
|
||
│ • Session / Member / SpaceMembership projections │
|
||
│ • PageOps façade: list/get/create/update/archive/copy/move │
|
||
│ • Railiance deploy (Helm, env, probes) │
|
||
└───────────────┬─────────────────────────────┬───────────────┘
|
||
│ │
|
||
│ projections │ content ops
|
||
▼ ▼
|
||
┌───────────────┐ ┌───────────────────────────┐
|
||
│ App DB cache │ │ Content plane (choose one │
|
||
│ (Postgres) │ │ primary adapter) │
|
||
└───────────────┘ │ A thin folder/git store │
|
||
│ B shard-wiki │
|
||
│ C kontextual-engine │
|
||
└─────────────┬─────────────┘
|
||
│ structure
|
||
▼
|
||
┌───────────────────────────┐
|
||
│ markitect-tool │
|
||
│ front/body/back, schema │
|
||
└───────────────────────────┘
|
||
│
|
||
┌─────────────▼─────────────┐
|
||
│ Asset bytes (see §6) │
|
||
│ content-addressed blobs │
|
||
└───────────────────────────┘
|
||
```
|
||
|
||
**Product verbs (PageOps)** — store-agnostic:
|
||
|
||
| Op | Meaning |
|
||
|----|---------|
|
||
| `list_pages(space)` | Card projections |
|
||
| `get_page(id)` | Full markdown + resolved visual |
|
||
| `put_page(...)` | Create/update attributes + body |
|
||
| `archive_page` | Soft-delete / lifecycle |
|
||
| `copy_page` | Independent duplicate |
|
||
| `transfer_page` | Re-home to another space |
|
||
| `put_visual` / `get_asset` | Binary visual/attachment |
|
||
|
||
Coulomb never talks to Forgejo/shard/kontextual APIs ad hoc from templates — only through PageOps.
|
||
|
||
## 4. Information model (product)
|
||
|
||
| Concept | Representation |
|
||
|---------|----------------|
|
||
| **Page** | Markdown document: frontmatter (title, abstract, visual ref, space, type, …) + body + backmatter (relations, provenance) |
|
||
| **Space** | Container context; may be a page of `type: space` or a first-class container id in frontmatter |
|
||
| **Visual** | Asset ref in frontmatter → blob/asset service |
|
||
| **Card** | UI projection: visual + title + abstract + actions (not a storage type) |
|
||
| **Member** | NetKingdom subject ↔ app Member row |
|
||
| **Tenant** | Instance / `tenant_id` (DR-1 C) |
|
||
|
||
## 5. Content-plane options (comparison)
|
||
|
||
### 5.1 Option A — Thin folder/git store + markitect-tool
|
||
|
||
| | |
|
||
|--|--|
|
||
| **Idea** | Layout of `.md` files + assets; optional git; markitect validates; app indexes cards |
|
||
| **Pros** | Fastest stage-1; trivial user export; low structure surface |
|
||
| **Cons** | Reimplements copy/transfer/versioning carefully; federation later is a rewrite |
|
||
| **When** | Need product velocity; content plane maturity unclear |
|
||
|
||
### 5.2 Option B — shard-wiki (thin slice)
|
||
|
||
| | |
|
||
|--|--|
|
||
| **Idea** | Headless wiki orchestration; pages as union of shards; IdP external; coulomb = UI |
|
||
| **Pros** | Built for wiki pages, git coordination, overlays, multi-shard (personal + space); INTENT names Coulomb; no UI ownership |
|
||
| **Cons** | Early product maturity for multi-tenant ops; federation API can over-structure if not sliced thin |
|
||
| **When** | Want page federation / export shards / git-addressable coordination as first-class |
|
||
| **Thin-slice rule** | One InformationSpace per instance; one primary shard; hide federation from card UI |
|
||
|
||
### 5.3 Option C — kontextual-engine (knowledge ops / CMS-ECM backend)
|
||
|
||
| | |
|
||
|--|--|
|
||
| **Idea** | Headless knowledge operations: durable **KnowledgeAsset** identity, representations, metadata, relationships, workflows, audit, blob storage, optional CMIS; markdown via markitect-tool adapter |
|
||
| **Pros** | Strong identity for assets beyond path; lineage, versioning, export packages, multi-format future; **blob layer already implemented** (memory/fs/S3, content-addressed); PolicyGateway port ready for external authz; aligns with “ECM without being the UI” |
|
||
| **Cons** | Heavier than pure wiki; **not streamlined for NetKingdom** today (actors are engine-local; default `AllowAllPolicyGateway`); markdown is *a* format among many — SoR discipline must force “source representation is markdown file users can take” |
|
||
| **When** | Need governed asset registry, visuals/blobs, audit, multi-format, CMIS-ish interoperability; willing to invest PolicyGateway → flex-auth / NetKingdom |
|
||
| **Auth gap** | `Actor` + `external_ref` can map to OIDC subject; `PolicyGateway.authorize` must call **flex-auth** (or equivalent) with NetKingdom groups — not reimplement IAM inside kontextual |
|
||
|
||
**kontextual already has (relevant):**
|
||
|
||
- Asset registry (create, metadata, lifecycle, versions, relationships, audit)
|
||
- BlobStorage port + local/S3/memory adapters, dedupe by sha256
|
||
- Representation kinds (source / normalized / derived)
|
||
- markitect-tool reuse boundary docs + adapter package
|
||
- Service API surface for assets/retrieval/workflows
|
||
- Policy port (pluggable); default allow-all for tests
|
||
|
||
**kontextual does not yet have (for coulomb):**
|
||
|
||
- First-class “space + card product model”
|
||
- Wired NetKingdom / KeyCape session → Actor
|
||
- Production flex-auth policy adapter
|
||
- Guaranteed “markdown file tree export is the happy path” product packaging (export exists as concept; productize for coulomb)
|
||
|
||
### 5.4 How options relate (not exclusive forever)
|
||
|
||
```text
|
||
coulomb PageOps
|
||
│
|
||
├─ short term: A or thin B/C adapter
|
||
│
|
||
└─ adapters can evolve:
|
||
folder/git shard ──► shard-wiki InformationSpace
|
||
markdown assets ──► kontextual KnowledgeAsset + BlobStorage
|
||
markitect-tool ──► always structure layer
|
||
```
|
||
|
||
shard-wiki optimizes **wiki federation and page union**.
|
||
kontextual optimizes **governed multi-format knowledge assets + blobs + audit**.
|
||
coulomb optimizes **co-creation product**.
|
||
|
||
A hybrid is possible later: shard-wiki as page navigation plane, kontextual as asset/blob + governance — only if spikes show value without double-structure tax.
|
||
|
||
## 6. Asset management — extract to its own repo?
|
||
|
||
### 6.1 What exists today
|
||
|
||
| Source | Asset story |
|
||
|--------|-------------|
|
||
| **kontextual-engine** | Clean **BlobStorage** protocol + content-addressed store + representation linkage; small, intentional, portable adapters (~memory/fs/S3) |
|
||
| **markitect-main** `markitect/assets/` | Large (~0.5M+ tree) manager/registry/packager/optimizer/analytics — **orphan** in succession map; do not run coulomb on it |
|
||
| **coulomb** | No first-class visual/blob store yet (stage-1 Visual is todo) |
|
||
|
||
### 6.2 Extraction thesis
|
||
|
||
**Yes, worth evaluating a dedicated `asset-store` (name TBD) capability** if:
|
||
|
||
1. **coulomb**, **kontextual**, **shard-wiki**, and future apps all need content-addressed bytes + metadata without pulling full ECM or full wiki.
|
||
2. The core contract is small: `put/get/stat/delete_unreferenced` + digest + media type + optional ACLs.
|
||
3. We want Railiance-wide reuse (space visuals, wiki attachments, kontextual representations).
|
||
|
||
**Prefer extracting from kontextual’s BlobStorage** (clean ports), **not** from markitect-main’s asset mega-package (mine ideas only: dedupe, mdpkg packaging).
|
||
|
||
### 6.3 Suggested extraction shape (discussion)
|
||
|
||
```text
|
||
asset-store (new repo or helix capability)
|
||
ports: BlobStorage (digest-addressed)
|
||
adapters: fs, s3, memory
|
||
optional: reference accounting, GC, HTTP byte range
|
||
NOT: full asset registry, workflows, CMIS, markdown parsing
|
||
|
||
kontextual-engine
|
||
keeps KnowledgeAsset + Representation pointing at storage_ref
|
||
depends on asset-store library/service
|
||
|
||
coulomb / shard-wiki
|
||
may depend on asset-store for Visual without full kontextual
|
||
```
|
||
|
||
**Decision gate for extraction:** after coulomb stage-1 Visual needs a real store **and** a second consumer wants the same API. Until then, use kontextual blob adapters **in-process** or vendor the port as a small shared package without full service.
|
||
|
||
## 7. Identity & access control
|
||
|
||
```text
|
||
Browser → coulomb (session after NetKingdom OIDC)
|
||
│
|
||
├─ product authz: SpaceMembership, tenant_id
|
||
│
|
||
└─ content plane actor:
|
||
Actor(external_ref=issuer|subject, groups=OIDC groups)
|
||
PolicyGateway → flex-auth (preferred) or engine rules
|
||
```
|
||
|
||
| Plane | Responsibility |
|
||
|-------|----------------|
|
||
| NetKingdom | Who is the user; groups/roles claims |
|
||
| coulomb | Session; Member; space membership product rules |
|
||
| flex-auth | Optional PDP for resource actions |
|
||
| Content plane | Enforce policy on get/put content; fail closed |
|
||
|
||
**Spike required:** map one OIDC login → kontextual `Actor` → deny-by-default policy for foreign tenant assets.
|
||
|
||
## 8. Deploy sketch (Railiance)
|
||
|
||
| Component | Deploy |
|
||
|-----------|--------|
|
||
| coulomb-social | Existing chart / app.coulomb.social |
|
||
| Content plane | Sidecar or separate service in `coulomb-social` ns or shared platform ns |
|
||
| asset bytes | PVC and/or S3-compatible; secrets via existing env-secret pattern |
|
||
| NetKingdom | Existing KeyCape / Authelia |
|
||
|
||
No apex DNS change until stage-1 cut.
|
||
|
||
## 9. ADR roadmap (proposed)
|
||
|
||
| ADR | Topic |
|
||
|-----|--------|
|
||
| ADR-0001 | NetKingdom identity (done) |
|
||
| ADR-0002 | Forgejo markdown binding (provisional slice → demote to adapter) |
|
||
| **ADR-0003** | Page-centric model + markdown SoR + DB cache (to write after spikes) |
|
||
| **ADR-0004** | Content-plane choice (A/B/C) + PageOps boundary |
|
||
| **ADR-0005** | Asset-store extraction (if/when) |
|
||
|
||
## 10. Spike checklist
|
||
|
||
See **[spike-checklist.md](./spike-checklist.md)** — executable evaluation tasks.
|
||
|
||
## 11. Discussion prompts
|
||
|
||
1. Is **user export as markdown tree** more important than **multi-format ECM** in the next 6 months? (Biases A/B vs C.)
|
||
2. Do we need **multi-shard federation** (personal vault + space) at stage 1, or only later? (Biases B.)
|
||
3. Should **Visual** block on asset-store extraction, or ship with kontextual/local blob first?
|
||
4. Who owns **PolicyGateway** production adapter — coulomb, kontextual, or flex-auth team?
|
||
5. Accept **hybrid**: kontextual for assets+registry, thin git for human-facing markdown tree sync?
|
||
|
||
## 12. Working recommendation (to debate, not freeze)
|
||
|
||
| Layer | Recommendation |
|
||
|-------|----------------|
|
||
| Product UI | coulomb cards + spaces |
|
||
| Markdown structure | markitect-tool |
|
||
| Identity / deploy | NetKingdom / Railiance |
|
||
| Content plane shortlist | Spike **B thin** and **C** in parallel; keep **A** as fallback |
|
||
| SoR narrative | User-facing SoR = markdown export; engine may add durable IDs/lineage **without** trapping content in opaque DB-only forms |
|
||
| Assets | Prefer kontextual BlobStorage now; extract **asset-store** when second consumer appears |
|
||
| markitect-main | Avoid |
|
||
|
||
---
|
||
|
||
## References
|
||
|
||
- `docs/architecture/2026-08-12-content-substrate-exploration.md`
|
||
- `docs/architecture/transclusion-and-chunks.md` — page-first model; chunks as extract/transclude
|
||
- `docs/architecture/authoring-modes-and-lazy-projections.md` — top-down/bottom-up; lazy Title/Abstractor/Visual + lock
|
||
- `docs/capability/stage-1-cutover.md`
|
||
- `docs/decisions/2026-08-12-feature-cut-stage-1.md`
|
||
- kontextual: `docs/architecture-blueprint.md`, `asset-registry-implementation.md`, `blob-storage-content-streaming-workplan.md`, `markitect-tool-reuse-boundary.md`
|
||
- shard-wiki: `INTENT.md`, `SCOPE.md`
|
||
- markitect-tool: `docs/markitect-main-scope-assessment.md`, `docs/transform-compose-include.md`, `docs/content-references.md`
|
||
- markitect-main succession: `docs/successor-gap-assessment.md`
|