coulomb-social/docs/architecture/ArchitectureBlueprint.md
tegwick e2aa4045f4 Design dual authoring modes and lazy projection fields
Document top-down and bottom-up writing, lazy title/abstractor/visual that
persist after generation, and per-field regenerate plus lock. Wire into
capability model and stage-1 notes.
2026-08-13 00:07:32 +02:00

15 KiB
Raw Blame History

Architecture Blueprint — coulomb.social content & platform

Field Value
Status draft for discussion (2026-08-12)
Product coulomb.social rebuild (coulomb-social)
Related stage-1 cut, capability model, content-substrate exploration
Non-goals Final store vendor lock-in; implementing UI chrome in this doc

1. Architectural aim

Deliver a co-creation product where:

  1. Identity (users, groups, tenants) is NetKingdom — not reimplemented.
  2. Runtime is Railiance — not reimplemented.
  3. User-owned content is markdown (+ assets) that can be exported and reused without the app database.
  4. The database is a cache / projection for cards, memberships, and speed.
  5. markitect-tool enables efficient structured markdown (lib/service), not a product host.
  6. coulomb owns co-creation UI (playing-card pages, spaces, transfer/copy) and product verbs.

2. Non-negotiable rules

# Rule
R1 Markdown chunks (frontmatter + body + backmatter) + assets are the exportable SoR for content.
R2 Postgres (or any app DB) holds projections and app linkage only for content; disposable for bodies.
R3 Authn/IdP = NetKingdom; coulomb maps OIDC → Member; no password store.
R4 Authz for product actions may compose NetKingdom groups, app membership, and content-plane ACLs — no second user directory.
R5 markitect-main is legacy reference, not a runtime foundation.
R6 markitect-tool is the markdown structure dependency (parse/validate/transform).
R7 Content-plane choice (shard-wiki / kontextual / thin store) is behind a narrow page-ops interface so coulomb UI does not couple to federation or ECM graphs.
R8 Stage-1 capability set remains: onboarding, spaces/pages with Title/Abstract/Visual, page copy, page transfer.

3. System shape

                         ┌──────────────────────────┐
                         │   NetKingdom             │
                         │   OIDC, groups, tenants  │
                         └────────────┬─────────────┘
                                      │ principal
                                      ▼
┌─────────────────────────────────────────────────────────────┐
│  coulomb.social (product)                                   │
│  • Card UI, spaces chrome, stage-1 product flows            │
│  • Session / Member / SpaceMembership projections           │
│  • PageOps façade: list/get/create/update/archive/copy/move │
│  • Railiance deploy (Helm, env, probes)                     │
└───────────────┬─────────────────────────────┬───────────────┘
                │                             │
                │ projections                 │ content ops
                ▼                             ▼
        ┌───────────────┐         ┌───────────────────────────┐
        │ App DB cache  │         │ Content plane (choose one │
        │ (Postgres)    │         │  primary adapter)         │
        └───────────────┘         │  A thin folder/git store  │
                                  │  B shard-wiki             │
                                  │  C kontextual-engine      │
                                  └─────────────┬─────────────┘
                                                │ structure
                                                ▼
                                  ┌───────────────────────────┐
                                  │ markitect-tool            │
                                  │ front/body/back, schema   │
                                  └───────────────────────────┘
                                                │
                                  ┌─────────────▼─────────────┐
                                  │ Asset bytes (see §6)      │
                                  │ content-addressed blobs   │
                                  └───────────────────────────┘

Product verbs (PageOps) — store-agnostic:

Op Meaning
list_pages(space) Card projections
get_page(id) Full markdown + resolved visual
put_page(...) Create/update attributes + body
archive_page Soft-delete / lifecycle
copy_page Independent duplicate
transfer_page Re-home to another space
put_visual / get_asset Binary visual/attachment

Coulomb never talks to Forgejo/shard/kontextual APIs ad hoc from templates — only through PageOps.

4. Information model (product)

Concept Representation
Page Markdown document: frontmatter (title, abstract, visual ref, space, type, …) + body + backmatter (relations, provenance)
Space Container context; may be a page of type: space or a first-class container id in frontmatter
Visual Asset ref in frontmatter → blob/asset service
Card UI projection: visual + title + abstract + actions (not a storage type)
Member NetKingdom subject ↔ app Member row
Tenant Instance / tenant_id (DR-1 C)

5. Content-plane options (comparison)

5.1 Option A — Thin folder/git store + markitect-tool

Idea Layout of .md files + assets; optional git; markitect validates; app indexes cards
Pros Fastest stage-1; trivial user export; low structure surface
Cons Reimplements copy/transfer/versioning carefully; federation later is a rewrite
When Need product velocity; content plane maturity unclear

5.2 Option B — shard-wiki (thin slice)

Idea Headless wiki orchestration; pages as union of shards; IdP external; coulomb = UI
Pros Built for wiki pages, git coordination, overlays, multi-shard (personal + space); INTENT names Coulomb; no UI ownership
Cons Early product maturity for multi-tenant ops; federation API can over-structure if not sliced thin
When Want page federation / export shards / git-addressable coordination as first-class
Thin-slice rule One InformationSpace per instance; one primary shard; hide federation from card UI

5.3 Option C — kontextual-engine (knowledge ops / CMS-ECM backend)

Idea Headless knowledge operations: durable KnowledgeAsset identity, representations, metadata, relationships, workflows, audit, blob storage, optional CMIS; markdown via markitect-tool adapter
Pros Strong identity for assets beyond path; lineage, versioning, export packages, multi-format future; blob layer already implemented (memory/fs/S3, content-addressed); PolicyGateway port ready for external authz; aligns with “ECM without being the UI”
Cons Heavier than pure wiki; not streamlined for NetKingdom today (actors are engine-local; default AllowAllPolicyGateway); markdown is a format among many — SoR discipline must force “source representation is markdown file users can take”
When Need governed asset registry, visuals/blobs, audit, multi-format, CMIS-ish interoperability; willing to invest PolicyGateway → flex-auth / NetKingdom
Auth gap Actor + external_ref can map to OIDC subject; PolicyGateway.authorize must call flex-auth (or equivalent) with NetKingdom groups — not reimplement IAM inside kontextual

kontextual already has (relevant):

  • Asset registry (create, metadata, lifecycle, versions, relationships, audit)
  • BlobStorage port + local/S3/memory adapters, dedupe by sha256
  • Representation kinds (source / normalized / derived)
  • markitect-tool reuse boundary docs + adapter package
  • Service API surface for assets/retrieval/workflows
  • Policy port (pluggable); default allow-all for tests

kontextual does not yet have (for coulomb):

  • First-class “space + card product model”
  • Wired NetKingdom / KeyCape session → Actor
  • Production flex-auth policy adapter
  • Guaranteed “markdown file tree export is the happy path” product packaging (export exists as concept; productize for coulomb)

5.4 How options relate (not exclusive forever)

coulomb PageOps
    │
    ├─ short term: A or thin B/C adapter
    │
    └─ adapters can evolve:
         folder/git shard  ──► shard-wiki InformationSpace
         markdown assets   ──► kontextual KnowledgeAsset + BlobStorage
         markitect-tool    ──► always structure layer

shard-wiki optimizes wiki federation and page union.
kontextual optimizes governed multi-format knowledge assets + blobs + audit.
coulomb optimizes co-creation product.

A hybrid is possible later: shard-wiki as page navigation plane, kontextual as asset/blob + governance — only if spikes show value without double-structure tax.

6. Asset management — extract to its own repo?

6.1 What exists today

Source Asset story
kontextual-engine Clean BlobStorage protocol + content-addressed store + representation linkage; small, intentional, portable adapters (~memory/fs/S3)
markitect-main markitect/assets/ Large (~0.5M+ tree) manager/registry/packager/optimizer/analytics — orphan in succession map; do not run coulomb on it
coulomb No first-class visual/blob store yet (stage-1 Visual is todo)

6.2 Extraction thesis

Yes, worth evaluating a dedicated asset-store (name TBD) capability if:

  1. coulomb, kontextual, shard-wiki, and future apps all need content-addressed bytes + metadata without pulling full ECM or full wiki.
  2. The core contract is small: put/get/stat/delete_unreferenced + digest + media type + optional ACLs.
  3. We want Railiance-wide reuse (space visuals, wiki attachments, kontextual representations).

Prefer extracting from kontextuals BlobStorage (clean ports), not from markitect-mains asset mega-package (mine ideas only: dedupe, mdpkg packaging).

6.3 Suggested extraction shape (discussion)

asset-store (new repo or helix capability)
  ports: BlobStorage (digest-addressed)
  adapters: fs, s3, memory
  optional: reference accounting, GC, HTTP byte range
  NOT: full asset registry, workflows, CMIS, markdown parsing

kontextual-engine
  keeps KnowledgeAsset + Representation pointing at storage_ref
  depends on asset-store library/service

coulomb / shard-wiki
  may depend on asset-store for Visual without full kontextual

Decision gate for extraction: after coulomb stage-1 Visual needs a real store and a second consumer wants the same API. Until then, use kontextual blob adapters in-process or vendor the port as a small shared package without full service.

7. Identity & access control

Browser → coulomb (session after NetKingdom OIDC)
                │
                ├─ product authz: SpaceMembership, tenant_id
                │
                └─ content plane actor:
                     Actor(external_ref=issuer|subject, groups=OIDC groups)
                     PolicyGateway → flex-auth (preferred) or engine rules
Plane Responsibility
NetKingdom Who is the user; groups/roles claims
coulomb Session; Member; space membership product rules
flex-auth Optional PDP for resource actions
Content plane Enforce policy on get/put content; fail closed

Spike required: map one OIDC login → kontextual Actor → deny-by-default policy for foreign tenant assets.

8. Deploy sketch (Railiance)

Component Deploy
coulomb-social Existing chart / app.coulomb.social
Content plane Sidecar or separate service in coulomb-social ns or shared platform ns
asset bytes PVC and/or S3-compatible; secrets via existing env-secret pattern
NetKingdom Existing KeyCape / Authelia

No apex DNS change until stage-1 cut.

9. ADR roadmap (proposed)

ADR Topic
ADR-0001 NetKingdom identity (done)
ADR-0002 Forgejo markdown binding (provisional slice → demote to adapter)
ADR-0003 Page-centric model + markdown SoR + DB cache (to write after spikes)
ADR-0004 Content-plane choice (A/B/C) + PageOps boundary
ADR-0005 Asset-store extraction (if/when)

10. Spike checklist

See spike-checklist.md — executable evaluation tasks.

11. Discussion prompts

  1. Is user export as markdown tree more important than multi-format ECM in the next 6 months? (Biases A/B vs C.)
  2. Do we need multi-shard federation (personal vault + space) at stage 1, or only later? (Biases B.)
  3. Should Visual block on asset-store extraction, or ship with kontextual/local blob first?
  4. Who owns PolicyGateway production adapter — coulomb, kontextual, or flex-auth team?
  5. Accept hybrid: kontextual for assets+registry, thin git for human-facing markdown tree sync?

12. Working recommendation (to debate, not freeze)

Layer Recommendation
Product UI coulomb cards + spaces
Markdown structure markitect-tool
Identity / deploy NetKingdom / Railiance
Content plane shortlist Spike B thin and C in parallel; keep A as fallback
SoR narrative User-facing SoR = markdown export; engine may add durable IDs/lineage without trapping content in opaque DB-only forms
Assets Prefer kontextual BlobStorage now; extract asset-store when second consumer appears
markitect-main Avoid

References

  • docs/architecture/2026-08-12-content-substrate-exploration.md
  • docs/architecture/transclusion-and-chunks.md — page-first model; chunks as extract/transclude
  • docs/architecture/authoring-modes-and-lazy-projections.md — top-down/bottom-up; lazy Title/Abstractor/Visual + lock
  • docs/capability/stage-1-cutover.md
  • docs/decisions/2026-08-12-feature-cut-stage-1.md
  • kontextual: docs/architecture-blueprint.md, asset-registry-implementation.md, blob-storage-content-streaming-workplan.md, markitect-tool-reuse-boundary.md
  • shard-wiki: INTENT.md, SCOPE.md
  • markitect-tool: docs/markitect-main-scope-assessment.md, docs/transform-compose-include.md, docs/content-references.md
  • markitect-main succession: docs/successor-gap-assessment.md