coulomb/coulomb-social
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add ArchitectureBlueprint and content-plane spike checklist

Browse source 
Document coulomb layering (NetKingdom, Railiance, PageOps, markitect-tool),
compare thin store vs shard-wiki vs kontextual-engine, and outline asset-store
extraction from kontextual BlobStorage. Include executable spike checklist.
This commit is contained in:
tegwick 2026-08-12 18:38:27 +02:00
parent 27ba9206f7
commit a79a9e6e77
4 changed files with 492 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

2
INTENT.md
View file

@ -82,6 +82,8 @@ The rebuild is deliberately **product-faithful first**: UI and content parity ma
- `docs/resource-evidence/` — workload demand forecasts, observations, labor
- `docs/capability/` — product capability/feature model; stage-1 cutover must set
- `docs/decisions/2026-08-12-feature-cut-stage-1.md` — partial rebuild for productive transfer
- `docs/architecture/ArchitectureBlueprint.md` — content plane, NetKingdom, assets (discussion)
- `docs/architecture/spike-checklist.md` — evaluation spikes for store choice
- `docs/architecture/2026-08-12-content-substrate-exploration.md` — markdown SoR, markitect-tool, shard-wiki eval
- `the-custodian/docs/coulomb-social-rebuild-seed.md` — original workplan seed (CUST-WP-0058-T08)
- `the-custodian/canon/standards/business-app-service-contract_v0.1.md`

11
docs/architecture/2026-08-12-content-substrate-exploration.md
View file

@ -146,11 +146,12 @@ Rebuild / reimport should not require coulomb’s Postgres. Cache tables are dis
## Open evaluation tasks (if we continue)
1. Spike: coulomb → markitect-tool parse/serialize of a stage-1 page document.
2. Spike: shard-wiki single-space InformationSpace + FolderAdapter or native engine; implement copy/transfer as engine ops or app-level compose.
3. Compare authz model: shard-wiki page ACLs vs coulomb SpaceMembership vs NetKingdom groups.
4. Decide cache invalidation: write to markdown first, then reindex cards.
5. Promote winners into **ADR-0003** (information model + substrate) and revise capability model feature wording.
Executable checklist + blueprint:
- [`ArchitectureBlueprint.md`](./ArchitectureBlueprint.md)
- [`spike-checklist.md`](./spike-checklist.md)
Includes kontextual-engine option, asset-store extraction thesis, and NetKingdom PolicyGateway gap.
## Summary decisions from this note

279
docs/architecture/ArchitectureBlueprint.md Normal file
View file

@ -0,0 +1,279 @@
# Architecture Blueprint — coulomb.social content & platform
| Field | Value |
|-------|--------|
| Status | **draft for discussion** (2026-08-12) |
| Product | coulomb.social rebuild (`coulomb-social`) |
| Related | stage-1 cut, capability model, content-substrate exploration |
| Non-goals | Final store vendor lock-in; implementing UI chrome in this doc |
## 1. Architectural aim
Deliver a co-creation product where:
1. **Identity** (users, groups, tenants) is **NetKingdom** — not reimplemented.
2. **Runtime** is **Railiance** — not reimplemented.
3. **User-owned content** is **markdown (+ assets)** that can be exported and reused without the app database.
4. The **database is a cache / projection** for cards, memberships, and speed.
5. **markitect-tool** enables efficient structured markdown (lib/service), not a product host.
6. **coulomb** owns co-creation UI (playing-card pages, spaces, transfer/copy) and product verbs.
## 2. Non-negotiable rules
| # | Rule |
|---|------|
| R1 | Markdown chunks (frontmatter + body + backmatter) + assets are the **exportable SoR** for content. |
| R2 | Postgres (or any app DB) holds **projections and app linkage only** for content; disposable for bodies. |
| R3 | Authn/IdP = **NetKingdom**; coulomb maps OIDC → Member; no password store. |
| R4 | Authz for product actions may compose NetKingdom groups, app membership, and content-plane ACLs — **no second user directory**. |
| R5 | **markitect-main** is legacy reference, not a runtime foundation. |
| R6 | **markitect-tool** is the markdown structure dependency (parse/validate/transform). |
| R7 | Content-plane choice (shard-wiki / kontextual / thin store) is behind a **narrow page-ops interface** so coulomb UI does not couple to federation or ECM graphs. |
| R8 | Stage-1 capability set remains: onboarding, spaces/pages with Title/Abstract/Visual, page copy, page transfer. |
## 3. System shape
```text
┌──────────────────────────┐
│ NetKingdom │
│ OIDC, groups, tenants │
└────────────┬─────────────┘
│ principal
▼
┌─────────────────────────────────────────────────────────────┐
│ coulomb.social (product) │
│ • Card UI, spaces chrome, stage-1 product flows │
│ • Session / Member / SpaceMembership projections │
│ • PageOps façade: list/get/create/update/archive/copy/move │
│ • Railiance deploy (Helm, env, probes) │
└───────────────┬─────────────────────────────┬───────────────┘
│ │
│ projections │ content ops
▼ ▼
┌───────────────┐ ┌───────────────────────────┐
│ App DB cache │ │ Content plane (choose one │
│ (Postgres) │ │ primary adapter) │
└───────────────┘ │ A thin folder/git store │
│ B shard-wiki │
│ C kontextual-engine │
└─────────────┬─────────────┘
│ structure
▼
┌───────────────────────────┐
│ markitect-tool │
│ front/body/back, schema │
└───────────────────────────┘
│
┌─────────────▼─────────────┐
│ Asset bytes (see §6) │
│ content-addressed blobs │
└───────────────────────────┘
```
**Product verbs (PageOps)** — store-agnostic:
| Op | Meaning |
|----|---------|
| `list_pages(space)` | Card projections |
| `get_page(id)` | Full markdown + resolved visual |
| `put_page(...)` | Create/update attributes + body |
| `archive_page` | Soft-delete / lifecycle |
| `copy_page` | Independent duplicate |
| `transfer_page` | Re-home to another space |
| `put_visual` / `get_asset` | Binary visual/attachment |
Coulomb never talks to Forgejo/shard/kontextual APIs ad hoc from templates — only through PageOps.
## 4. Information model (product)
| Concept | Representation |
|---------|----------------|
| **Page** | Markdown document: frontmatter (title, abstract, visual ref, space, type, …) + body + backmatter (relations, provenance) |
| **Space** | Container context; may be a page of `type: space` or a first-class container id in frontmatter |
| **Visual** | Asset ref in frontmatter → blob/asset service |
| **Card** | UI projection: visual + title + abstract + actions (not a storage type) |
| **Member** | NetKingdom subject ↔ app Member row |
| **Tenant** | Instance / `tenant_id` (DR-1 C) |
## 5. Content-plane options (comparison)
### 5.1 Option A — Thin folder/git store + markitect-tool
| | |
|--|--|
| **Idea** | Layout of `.md` files + assets; optional git; markitect validates; app indexes cards |
| **Pros** | Fastest stage-1; trivial user export; low structure surface |
| **Cons** | Reimplements copy/transfer/versioning carefully; federation later is a rewrite |
| **When** | Need product velocity; content plane maturity unclear |
### 5.2 Option B — shard-wiki (thin slice)
| | |
|--|--|
| **Idea** | Headless wiki orchestration; pages as union of shards; IdP external; coulomb = UI |
| **Pros** | Built for wiki pages, git coordination, overlays, multi-shard (personal + space); INTENT names Coulomb; no UI ownership |
| **Cons** | Early product maturity for multi-tenant ops; federation API can over-structure if not sliced thin |
| **When** | Want page federation / export shards / git-addressable coordination as first-class |
| **Thin-slice rule** | One InformationSpace per instance; one primary shard; hide federation from card UI |
### 5.3 Option C — kontextual-engine (knowledge ops / CMS-ECM backend)
| | |
|--|--|
| **Idea** | Headless knowledge operations: durable **KnowledgeAsset** identity, representations, metadata, relationships, workflows, audit, blob storage, optional CMIS; markdown via markitect-tool adapter |
| **Pros** | Strong identity for assets beyond path; lineage, versioning, export packages, multi-format future; **blob layer already implemented** (memory/fs/S3, content-addressed); PolicyGateway port ready for external authz; aligns with “ECM without being the UI” |
| **Cons** | Heavier than pure wiki; **not streamlined for NetKingdom** today (actors are engine-local; default `AllowAllPolicyGateway`); markdown is *a* format among many — SoR discipline must force “source representation is markdown file users can take” |
| **When** | Need governed asset registry, visuals/blobs, audit, multi-format, CMIS-ish interoperability; willing to invest PolicyGateway → flex-auth / NetKingdom |
| **Auth gap** | `Actor` + `external_ref` can map to OIDC subject; `PolicyGateway.authorize` must call **flex-auth** (or equivalent) with NetKingdom groups — not reimplement IAM inside kontextual |
**kontextual already has (relevant):**
- Asset registry (create, metadata, lifecycle, versions, relationships, audit)
- BlobStorage port + local/S3/memory adapters, dedupe by sha256
- Representation kinds (source / normalized / derived)
- markitect-tool reuse boundary docs + adapter package
- Service API surface for assets/retrieval/workflows
- Policy port (pluggable); default allow-all for tests
**kontextual does not yet have (for coulomb):**
- First-class “space + card product model”
- Wired NetKingdom / KeyCape session → Actor
- Production flex-auth policy adapter
- Guaranteed “markdown file tree export is the happy path” product packaging (export exists as concept; productize for coulomb)
### 5.4 How options relate (not exclusive forever)
```text
coulomb PageOps
│
├─ short term: A or thin B/C adapter
│
└─ adapters can evolve:
folder/git shard ──► shard-wiki InformationSpace
markdown assets ──► kontextual KnowledgeAsset + BlobStorage
markitect-tool ──► always structure layer
```
shard-wiki optimizes **wiki federation and page union**.
kontextual optimizes **governed multi-format knowledge assets + blobs + audit**.
coulomb optimizes **co-creation product**.
A hybrid is possible later: shard-wiki as page navigation plane, kontextual as asset/blob + governance — only if spikes show value without double-structure tax.
## 6. Asset management — extract to its own repo?
### 6.1 What exists today
| Source | Asset story |
|--------|-------------|
| **kontextual-engine** | Clean **BlobStorage** protocol + content-addressed store + representation linkage; small, intentional, portable adapters (~memory/fs/S3) |
| **markitect-main** `markitect/assets/` | Large (~0.5M+ tree) manager/registry/packager/optimizer/analytics — **orphan** in succession map; do not run coulomb on it |
| **coulomb** | No first-class visual/blob store yet (stage-1 Visual is todo) |
### 6.2 Extraction thesis
**Yes, worth evaluating a dedicated `asset-store` (name TBD) capability** if:
1. **coulomb**, **kontextual**, **shard-wiki**, and future apps all need content-addressed bytes + metadata without pulling full ECM or full wiki.
2. The core contract is small: `put/get/stat/delete_unreferenced` + digest + media type + optional ACLs.
3. We want Railiance-wide reuse (space visuals, wiki attachments, kontextual representations).
**Prefer extracting from kontextual’s BlobStorage** (clean ports), **not** from markitect-main’s asset mega-package (mine ideas only: dedupe, mdpkg packaging).
### 6.3 Suggested extraction shape (discussion)
```text
asset-store (new repo or helix capability)
ports: BlobStorage (digest-addressed)
adapters: fs, s3, memory
optional: reference accounting, GC, HTTP byte range
NOT: full asset registry, workflows, CMIS, markdown parsing
kontextual-engine
keeps KnowledgeAsset + Representation pointing at storage_ref
depends on asset-store library/service
coulomb / shard-wiki
may depend on asset-store for Visual without full kontextual
```
**Decision gate for extraction:** after coulomb stage-1 Visual needs a real store **and** a second consumer wants the same API. Until then, use kontextual blob adapters **in-process** or vendor the port as a small shared package without full service.
## 7. Identity & access control
```text
Browser → coulomb (session after NetKingdom OIDC)
│
├─ product authz: SpaceMembership, tenant_id
│
└─ content plane actor:
Actor(external_ref=issuer|subject, groups=OIDC groups)
PolicyGateway → flex-auth (preferred) or engine rules
```
| Plane | Responsibility |
|-------|----------------|
| NetKingdom | Who is the user; groups/roles claims |
| coulomb | Session; Member; space membership product rules |
| flex-auth | Optional PDP for resource actions |
| Content plane | Enforce policy on get/put content; fail closed |
**Spike required:** map one OIDC login → kontextual `Actor` → deny-by-default policy for foreign tenant assets.
## 8. Deploy sketch (Railiance)
| Component | Deploy |
|-----------|--------|
| coulomb-social | Existing chart / app.coulomb.social |
| Content plane | Sidecar or separate service in `coulomb-social` ns or shared platform ns |
| asset bytes | PVC and/or S3-compatible; secrets via existing env-secret pattern |
| NetKingdom | Existing KeyCape / Authelia |
No apex DNS change until stage-1 cut.
## 9. ADR roadmap (proposed)
| ADR | Topic |
|-----|--------|
| ADR-0001 | NetKingdom identity (done) |
| ADR-0002 | Forgejo markdown binding (provisional slice → demote to adapter) |
| **ADR-0003** | Page-centric model + markdown SoR + DB cache (to write after spikes) |
| **ADR-0004** | Content-plane choice (A/B/C) + PageOps boundary |
| **ADR-0005** | Asset-store extraction (if/when) |
## 10. Spike checklist
See **[spike-checklist.md](./spike-checklist.md)** — executable evaluation tasks.
## 11. Discussion prompts
1. Is **user export as markdown tree** more important than **multi-format ECM** in the next 6 months? (Biases A/B vs C.)
2. Do we need **multi-shard federation** (personal vault + space) at stage 1, or only later? (Biases B.)
3. Should **Visual** block on asset-store extraction, or ship with kontextual/local blob first?
4. Who owns **PolicyGateway** production adapter — coulomb, kontextual, or flex-auth team?
5. Accept **hybrid**: kontextual for assets+registry, thin git for human-facing markdown tree sync?
## 12. Working recommendation (to debate, not freeze)
| Layer | Recommendation |
|-------|----------------|
| Product UI | coulomb cards + spaces |
| Markdown structure | markitect-tool |
| Identity / deploy | NetKingdom / Railiance |
| Content plane shortlist | Spike **B thin** and **C** in parallel; keep **A** as fallback |
| SoR narrative | User-facing SoR = markdown export; engine may add durable IDs/lineage **without** trapping content in opaque DB-only forms |
| Assets | Prefer kontextual BlobStorage now; extract **asset-store** when second consumer appears |
| markitect-main | Avoid |
---
## References
- `docs/architecture/2026-08-12-content-substrate-exploration.md`
- `docs/capability/stage-1-cutover.md`
- `docs/decisions/2026-08-12-feature-cut-stage-1.md`
- kontextual: `docs/architecture-blueprint.md`, `asset-registry-implementation.md`, `blob-storage-content-streaming-workplan.md`, `markitect-tool-reuse-boundary.md`
- shard-wiki: `INTENT.md`, `SCOPE.md`
- markitect-tool: `docs/markitect-main-scope-assessment.md`
- markitect-main succession: `docs/successor-gap-assessment.md`

205
docs/architecture/spike-checklist.md Normal file
View file

@ -0,0 +1,205 @@
# Spike checklist — content plane & assets
| Field | Value |
|-------|--------|
| Blueprint | [ArchitectureBlueprint.md](./ArchitectureBlueprint.md) |
| Goal | Falsify or confirm options A/B/C and asset extraction before ADR-0003/0004 |
| Time box | Prefer ≤2–3 working days per major spike; stop when decision quality is enough |
Scoring legend for each spike: **Pass** / **Partial** / **Fail** / **Skip** with one-line evidence.
---
## Spike 0 — Shared page document fixture
**Owner:** coulomb-social
**Depends on:** none
- [ ] Define one **stage-1 page** markdown fixture:
- frontmatter: `id`, `type`, `title`, `abstract`, `visual`, `space`
- body: short markdown
- backmatter: e.g. `relations` or provenance stub
- [ ] Define one **space** container convention (page-type vs directory).
- [ ] Define **export package** layout (`pages/**`, `assets/**`, `MANIFEST.json`).
- [ ] Commit fixtures under `docs/space-fixtures/` or `docs/architecture/fixtures/`.
**Pass when:** fixture is stable enough that all later spikes use the same bytes.
---
## Spike 1 — markitect-tool as structure lib
**Owner:** coulomb-social (+ markitect-tool if API gaps)
**Depends on:** Spike 0
- [ ] From Python in coulomb (or throwaway script): parse fixture with markitect-tool public API.
- [ ] Round-trip serialize without losing title/abstract/visual fields.
- [ ] Validate required frontmatter fields (even minimal schema).
- [ ] Note missing APIs (backmatter support, field maps) as markitect-tool gaps.
**Pass when:** coulomb can treat markitect-tool as dependency for read/write structure without shelling out to `mkt` for the happy path.
**Fail when:** only CLI works or public API cannot preserve frontmatter stably.
---
## Spike 2 — Thin folder/git store (Option A)
**Owner:** coulomb-social
**Depends on:** Spike 0–1
- [ ] Layout: `spaces/<slug>/pages/<page>.md` + `assets/`.
- [ ] Implement throwaway PageOps: list, get, put, copy, transfer (filesystem).
- [ ] Reindex card projection into SQLite/Postgres (title, abstract, visual path, space).
- [ ] Export = tar/zip of tree; reimport without DB.
- [ ] Measure: copy/transfer LOC and failure modes (broken links, name collision).
**Pass when:** stage-1 verbs work on files alone; export/import demonstrated.
**Decision input:** Is A enough for 3–6 months without federation?
---
## Spike 3 — shard-wiki thin slice (Option B)
**Owner:** coulomb-social + shard-wiki
**Depends on:** Spike 0
- [ ] Boot one InformationSpace with **single** FolderAdapter or native engine shard.
- [ ] Ingest/edit stage-1 page; read via headless API (no UI).
- [ ] Implement or map: list, get, put; prototype copy/transfer (engine op or app-level).
- [ ] Attach **fake Actor** from static OIDC-like subject; confirm authz hooks exist or gap list.
- [ ] Document API surface coulomb would depend on (count endpoints/concepts).
- [ ] Stress “structure tax”: can card UI ignore federation/overlays entirely?
**Pass when:** thin slice delivers page CRUD without coulomb importing federation UX concepts.
**Fail when:** product code must understand multi-shard resolution for basic CRUD.
**Decision input:** structure surface area vs future multi-shard value.
---
## Spike 4 — kontextual-engine page + markdown SoR (Option C)
**Owner:** coulomb-social + kontextual-engine
**Depends on:** Spike 0–1
- [ ] Create `KnowledgeAsset` for stage-1 page; store **source** representation as markdown bytes (BlobStorage).
- [ ] Put title/abstract/visual in metadata and/or inside markdown frontmatter (document which is authoritative for export).
- [ ] Get content stream; reconstruct `.md` file bit-identical (or document lossiness).
- [ ] Run markitect-tool adapter on stored markdown (existing boundary tests as reference).
- [ ] Prototype copy = new asset + blob refs; transfer = metadata/relationship change to space context entity.
- [ ] Export package: write assets + markdown to filesystem from engine API.
**Pass when:** user-visible export is clean markdown+assets; engine IDs are additive not mandatory for reuse.
**Fail when:** content is only queryable as opaque registry rows without faithful markdown export.
**Decision input:** governance/lineage value vs stage-1 complexity.
---
## Spike 5 — NetKingdom / PolicyGateway wiring
**Owner:** coulomb + kontextual (and/or flex-auth)
**Depends on:** Spike 3 or 4
- [ ] Map live or mocked OIDC claims → `Actor(external_ref=issuer|sub, groups=...)`.
- [ ] Implement **deny-by-default** PolicyGateway stub that allows only same-tenant resource metadata.
- [ ] Optional: call flex-auth `check` with coulomb vocabulary (if service available).
- [ ] Prove fail-closed: missing policy → no content bytes.
- [ ] Document gap list for production adapter ownership.
**Pass when:** written design + failing/passing test for cross-tenant deny.
**Fail when:** only AllowAll can run the stack.
---
## Spike 6 — Asset / Visual path
**Owner:** coulomb (+ kontextual or extracted store)
**Depends on:** Spike 2 or 4
- [ ] Store a cover image via content-addressed put; reference from page frontmatter `visual: sha256:…` or `asset:…`.
- [ ] Serve image for card UI (signed URL, app proxy, or static mount — pick one).
- [ ] GC/unreferenced dry-run once.
- [ ] Compare effort: in-process kontextual BlobStorage vs copy port to shared package.
**Pass when:** card shows visual without storing image bytes in Postgres.
**Decision input for extraction:** is a second consumer ready? If no → keep in kontextual for now.
---
## Spike 7 — Asset-store extraction feasibility
**Owner:** platform / kontextual
**Depends on:** Spike 6
- [ ] Inventory kontextual `BlobStorage` + adapters (fs/s3/memory) public surface.
- [ ] Inventory markitect-main `markitect/assets/` for **ideas only** (dedupe, packaging); do not port wholesale.
- [ ] Draft package name + repo boundary: bytes only vs registry.
- [ ] Estimate extraction PR: move port+adapters, leave KnowledgeAsset in kontextual.
- [ ] List consumers: coulomb, kontextual, shard-wiki (optional).
**Pass when:** one-page extraction proposal with “extract now / later / never” recommendation.
**Default bias:** **later** until two consumers need it.
---
## Spike 8 — Cross-option decision workshop
**Owner:** founder + implementers
**Depends on:** Spikes 1–6 results
- [ ] Fill scorecard (below).
- [ ] Choose primary content plane for stage 1 + migration path.
- [ ] Choose asset strategy (in-kontextual vs extract vs folder-only).
- [ ] File ADR-0003 (page model + SoR) and ADR-0004 (content plane).
- [ ] Update `docs/capability/model-v0.yaml` feature wording if PageOps replaces Forgejo-centric features.
### Scorecard (1–5, higher better)
| Criterion | A thin store | B shard-wiki | C kontextual |
|-----------|:------------:|:------------:|:------------:|
| Markdown export fidelity | | | |
| Stage-1 speed | | | |
| Structure tax (inverse) | | | |
| Copy/transfer naturalness | | | |
| Visual/blob readiness | | | |
| NetKingdom/authz fit | | | |
| Multi-shard / federation later | | | |
| Multi-format / ECM later | | | |
| Railiance operability | | | |
| Aligns with fleet layers | | | |
| **Total** | | | |
---
## Suggested order
```text
0 fixture → 1 markitect → 2 thin store (always, baseline)
├→ 3 shard-wiki thin
└→ 4 kontextual page+blob
→ 5 authz on winner(s)
→ 6 visual
→ 7 extraction feasibility
→ 8 decision / ADRs
```
Spike 2 is the **control experiment**: even if B or C wins, A defines the export contract users feel.
---
## Exit criteria for this checklist
Stop spiking when:
1. One primary content plane is chosen for stage 1 with explicit residuals, and
2. Asset strategy is chosen (where Visual bytes live), and
3. NetKingdom policy gap is owned (who implements PolicyGateway adapter), and
4. ADR-0003/0004 drafts exist or are scheduled.