coulomb-social/docs/architecture/stale-derived-content.md

Stale badge, alternatives, and derived includes

Field	Value
Date	2026-08-13
Status	design proposal (coulomb product + markitect-tool contract)
Related	authoring modes, transclusion, markitect include/provenance

Intent

Any derived content — projection fields (title, abstractor, visual),
transclusions/includes, or outputs of rules / external data / research /
pipelines / classic compute / AI — can become stale relative to its inputs.

The product (and markitect-tool) should:

1. Detect staleness via input fingerprints (content hashes, dependency set).
2. Surface a clear “may be stale” badge (never silent rewrite by default).
3. Let the user generate an alternative (more current candidate).
4. Let the user replace the stored version or keep it (and optionally lock).

Same interaction grammar everywhere: projections, frozen include snapshots,
and pipeline-produced blocks.

---

What markitect-tool has today (gap analysis)

Building block	Status
Include / transclusion markers + resolve_includes	Yes
Selectors / partial include	Yes
Operation provenance on include/transform (path, selector, deps metadata)	Yes (partial)
Content hashes on units / explode entries / processing provenance	Yes (scattered)
Cache fingerprint + mkt cache status (file changed?)	Yes
Backend refresh-plan (snapshot vs files)	Yes (backend fabric)
Context package refresh recipes	Yes (agent memory)
Stale badge semantics for a stored derived snapshot vs live resolve	No productized UX contract
Generate alternative → replace or keep for includes/projections	No
Pinned / locked include snapshot in the document	No first-class marker
Unified “derived block” model (rule, AI, external, include)	No — needs design

Conclusion: markitect has plumbing for change detection and provenance,
not yet a unified stale → alternative → replace|keep workflow for
in-document derived content. That belongs in markitect-tool as a reusable
contract; coulomb UI implements the badge and choices.

Cross-repo proposal:
markitect-tool/docs/proposed-stale-derived-and-include-pinning.md

---

Unified model: derived slot

A derived slot is any value that:

• has a current stored representation (frontmatter field, pinned body region,
  or last accepted snapshot), and
• has a recipe that can recompute a candidate from inputs.

DerivedSlot
  id
  kind: projection | include | pipeline | external | ai | rule
  stored_value          # what readers see by default
  recipe                # how to recompute
  input_fingerprint     # hashes of inputs when stored_value was accepted
  state: derived | user | locked
  locked: bool
  generated_at
  generator / recipe_version

Stale ⇔ recompute current_fingerprint(inputs) ≠ input_fingerprint
(or a dependency file is missing / recipe version bumped if policy says so).

User actions when stale (or anytime if unlocked)

Action	Result
Keep	Leave stored_value; optionally refresh fingerprint (“accept as current”) or leave stale badge
Generate alternative	Run recipe → candidate (side-by-side; does not overwrite yet)
Replace with alternative	stored_value ← candidate; update fingerprint; state → derived
Edit stored	Manual edit; state → user
Lock / Unlock	Block regenerate and auto-refresh

Default policy: never auto-replace locked or user slots;
never auto-replace stale derived without consent (badge + actions only).

---

Coulomb: projection fields

Extends authoring-modes-and-lazy-projections.md.

When projections.title.source_hash (etc.) ≠ current body/deps hash:

1. Show May be stale on the field/card.
2. Actions: Generate updated version · Keep current · Edit · Lock.
3. Generate opens compare UI:

┌ Current (stored)     ┌ Alternative (fresh)
│ …                    │ …
└──────────────────────┴──────────────────
   [ Keep current ]  [ Use alternative ]  [ Cancel ]

4. Use alternative writes frontmatter and updates source_hash / generated_at.
5. Keep current dismisses compare; optional “Don’t warn until body changes again”
   = update fingerprint only (ack_stale) without changing text.

Same for abstractor and visual (visual compare: two images or image vs placeholder).

---

Includes / transclusions

Two legitimate modes (both needed):

A. Live resolve (dynamic) — product default

<!-- mkt:include path="src.md" selector="sections[heading=API]" -->

At render/export, markitect resolves from current src.md
(spreadsheet-like: formula results stay up to date).

“Stale” for pure live includes is mostly viewer/cache (optional badge if
a cached card excerpt lags live resolve). No replace/keep needed unless the
host cached a display snapshot.

B. Pinned / materialised include (snapshot in host page) — exception

For “freeze this excerpt / paste values” (explicit pin):

<!-- mkt:derived id="api-excerpt"
     kind="include"
     path="src.md"
     selector="sections[heading=API]"
     input_hash="sha256:…"
     locked="false"
     generator="include@v1" -->
…frozen markdown of the included region…
<!-- /mkt:derived -->

Or shorter pin attributes on include + adjacent snapshot region.

Event	Behavior
Source file changes	Slot becomes stale (hash mismatch)
User generates alternative	Resolve include live → candidate body
Replace	Write new snapshot into the derived region; update input_hash
Keep	Leave snapshot; optional ack
Lock	No refresh until unlock

Live vs pinned: product default for co-creation cards may be pinned
(exportable, offline, “my version of this excerpt”); docs/wiki sites may prefer
live. Coulomb should support both; stage-1 can start with live resolve for
view and add pin when “freeze excerpt” is needed.

Pipeline / AI / external derived blocks

Same mkt:derived envelope:

<!-- mkt:derived id="summary-1"
     kind="ai"
     recipe="summarize@v2"
     inputs="body"
     input_hash="sha256:…"
     locked="false" -->
…generated summary…
<!-- /mkt:derived -->

<!-- mkt:derived id="metrics"
     kind="pipeline"
     recipe="research.pull@v1"
     inputs="config/query.yaml,data/…"
     input_hash="sha256:…" -->
| metric | value |
…table…
<!-- /mkt:derived -->

Kinds (open set): include | rule | pipeline | external | ai | compute | projection.

---

markitect-tool contract (proposed)

Expose library APIs (names illustrative):

fingerprint_inputs(recipe, context) -> str
is_stale(slot, context) -> bool
evaluate_recipe(recipe, context) -> Candidate  # markdown + provenance
diff_or_pair(stored, candidate) -> Comparison
apply_replace(document, slot_id, candidate) -> document
apply_keep(document, slot_id, *, ack_fingerprint: bool) -> document

CLI sketches:

mkt derived status page.md          # list slots + stale?
mkt derived refresh page.md --id api-excerpt --dry-run   # print candidate
mkt derived refresh page.md --id api-excerpt --replace
mkt derived refresh page.md --id api-excerpt --keep --ack
mkt derived lock page.md --id summary-1

Include path should record target content hash in provenance today;
stale detection reuses cache fingerprints + per-slot input_hash.

Implementation can land incrementally in markitect-tool (see proposal doc);
coulomb consumes the API for badges and compare UI.

---

UX summary

[May be stale]
   → Generate updated version
        → side-by-side Current | Alternative
             → Keep current  (optional: clear badge / ack)
             → Use alternative (replace stored)
             → Cancel
   → Lock field/block

Applies to: title, abstractor, visual, outline stub, pinned includes,
AI/pipeline regions — one mental model.

---

Relation to lock

	Unlocked derived	Locked	User-edited
Stale badge	yes if fingerprint mismatch	yes if mismatch (informational)	optional / policy
Generate alternative	yes	no (must unlock)	yes (warns overwrite of user text)
Auto-replace	never (default)	never	never

---

Stage-1 vs later

Slice	When
Stale badge + regenerate/replace/keep for title/abstractor	Stage-1 authoring
Visual same	Stage-1 if visual exists
Live include resolve only	With markitect integration
Pinned mkt:derived include snapshots	Soon after includes land
Full pipeline/AI recipes	As generators are added

---

Related

• docs/architecture/authoring-modes-and-lazy-projections.md
• docs/architecture/transclusion-and-chunks.md
• markitect-tool: docs/transform-compose-include.md, docs/cache-incremental.md, docs/backend-fabric.md
• markitect-tool: docs/proposed-stale-derived-and-include-pinning.md (proposal for implementation there)