coulomb-social/docs/architecture/stale-derived-content.md

# Stale badge, alternatives, and derived includes

| Field | Value |
|-------|--------|
| Date | 2026-08-13 |
| Status | design proposal (coulomb product + markitect-tool contract) |
| Related | authoring modes, transclusion, markitect include/provenance |

## Intent

Any **derived** content — projection fields (title, abstractor, visual),  
**transclusions/includes**, or outputs of **rules / external data / research /  
pipelines / classic compute / AI** — can become **stale** relative to its inputs.

The product (and markitect-tool) should:

1. **Detect** staleness via input fingerprints (content hashes, dependency set).  
2. **Surface** a clear **“may be stale”** badge (never silent rewrite by default).  
3. Let the user **generate an alternative** (more current candidate).  
4. Let the user **replace** the stored version **or keep** it (and optionally lock).  

Same interaction grammar everywhere: projections, frozen include snapshots,  
and pipeline-produced blocks.

---

## What markitect-tool has today (gap analysis)

| Building block | Status |
|----------------|--------|
| Include / transclusion markers + `resolve_includes` | **Yes** |
| Selectors / partial include | **Yes** |
| Operation provenance on include/transform (path, selector, deps metadata) | **Yes** (partial) |
| Content hashes on units / explode entries / processing provenance | **Yes** (scattered) |
| Cache fingerprint + `mkt cache status` (file changed?) | **Yes** |
| Backend `refresh-plan` (snapshot vs files) | **Yes** (backend fabric) |
| Context package **refresh** recipes | **Yes** (agent memory) |
| **Stale badge semantics** for a *stored* derived snapshot vs live resolve | **No** productized UX contract |
| **Generate alternative → replace or keep** for includes/projections | **No** |
| **Pinned / locked** include snapshot in the document | **No** first-class marker |
| Unified “derived block” model (rule, AI, external, include) | **No** — needs design |

**Conclusion:** markitect has **plumbing for change detection and provenance**,  
not yet a **unified stale → alternative → replace|keep** workflow for  
in-document derived content. That belongs in markitect-tool as a reusable  
contract; coulomb UI implements the badge and choices.

Cross-repo proposal:  
`markitect-tool/docs/proposed-stale-derived-and-include-pinning.md`

---

## Unified model: derived slot

A **derived slot** is any value that:

- has a **current stored representation** (frontmatter field, pinned body region,  
  or last accepted snapshot), and  
- has a **recipe** that can recompute a candidate from inputs.

```text
DerivedSlot
  id
  kind: projection | include | pipeline | external | ai | rule
  stored_value          # what readers see by default
  recipe                # how to recompute
  input_fingerprint     # hashes of inputs when stored_value was accepted
  state: derived | user | locked
  locked: bool
  generated_at
  generator / recipe_version
```

**Stale** ⇔ recompute `current_fingerprint(inputs)` ≠ `input_fingerprint`  
(or a dependency file is missing / recipe version bumped if policy says so).

### User actions when stale (or anytime if unlocked)

| Action | Result |
|--------|--------|
| **Keep** | Leave `stored_value`; optionally refresh fingerprint (“accept as current”) or leave stale badge |
| **Generate alternative** | Run recipe → **candidate** (side-by-side; does not overwrite yet) |
| **Replace with alternative** | `stored_value ← candidate`; update fingerprint; state → `derived` |
| **Edit stored** | Manual edit; state → `user` |
| **Lock / Unlock** | Block regenerate and auto-refresh |

Default policy: **never auto-replace** locked or `user` slots;  
**never auto-replace** stale `derived` without consent (badge + actions only).

---

## Coulomb: projection fields

Extends `authoring-modes-and-lazy-projections.md`.

When `projections.title.source_hash` (etc.) ≠ current body/deps hash:

1. Show **May be stale** on the field/card.  
2. Actions: **Generate updated version** · **Keep current** · **Edit** · **Lock**.  
3. **Generate** opens compare UI:

```text
┌ Current (stored)     ┌ Alternative (fresh)
│ …                    │ …
└──────────────────────┴──────────────────
   [ Keep current ]  [ Use alternative ]  [ Cancel ]
```

4. **Use alternative** writes frontmatter and updates `source_hash` / `generated_at`.  
5. **Keep current** dismisses compare; optional “Don’t warn until body changes again”  
   = update fingerprint only (`ack_stale`) without changing text.

Same for abstractor and visual (visual compare: two images or image vs placeholder).

---

## Includes / transclusions

Two legitimate modes (both needed):

### A. Live resolve (dynamic) — **product default**

```markdown
<!-- mkt:include path="src.md" selector="sections[heading=API]" -->
```

At render/export, markitect resolves from **current** `src.md`  
(**spreadsheet-like**: formula results stay up to date).  

“Stale” for pure live includes is mostly **viewer/cache** (optional badge if  
a cached card excerpt lags live resolve). No replace/keep needed unless the  
host cached a display snapshot.

### B. Pinned / materialised include (snapshot in host page) — **exception**

For “freeze this excerpt / paste values” (explicit pin):

```markdown
<!-- mkt:derived id="api-excerpt"
     kind="include"
     path="src.md"
     selector="sections[heading=API]"
     input_hash="sha256:…"
     locked="false"
     generator="include@v1" -->
…frozen markdown of the included region…
<!-- /mkt:derived -->
```

Or shorter pin attributes on include + adjacent snapshot region.

| Event | Behavior |
|-------|----------|
| Source file changes | Slot becomes **stale** (hash mismatch) |
| User generates alternative | Resolve include live → candidate body |
| Replace | Write new snapshot into the derived region; update `input_hash` |
| Keep | Leave snapshot; optional ack |
| Lock | No refresh until unlock |

**Live vs pinned:** product default for co-creation cards may be **pinned**  
(exportable, offline, “my version of this excerpt”); docs/wiki sites may prefer  
**live**. Coulomb should support both; stage-1 can start with live resolve for  
view and add pin when “freeze excerpt” is needed.

### Pipeline / AI / external derived blocks

Same `mkt:derived` envelope:

```markdown
<!-- mkt:derived id="summary-1"
     kind="ai"
     recipe="summarize@v2"
     inputs="body"
     input_hash="sha256:…"
     locked="false" -->
…generated summary…
<!-- /mkt:derived -->
```

```markdown
<!-- mkt:derived id="metrics"
     kind="pipeline"
     recipe="research.pull@v1"
     inputs="config/query.yaml,data/…"
     input_hash="sha256:…" -->
| metric | value |
…table…
<!-- /mkt:derived -->
```

Kinds (open set): `include` | `rule` | `pipeline` | `external` | `ai` | `compute` | `projection`.

---

## markitect-tool contract (proposed)

Expose library APIs (names illustrative):

```text
fingerprint_inputs(recipe, context) -> str
is_stale(slot, context) -> bool
evaluate_recipe(recipe, context) -> Candidate  # markdown + provenance
diff_or_pair(stored, candidate) -> Comparison
apply_replace(document, slot_id, candidate) -> document
apply_keep(document, slot_id, *, ack_fingerprint: bool) -> document
```

CLI sketches:

```bash
mkt derived status page.md          # list slots + stale?
mkt derived refresh page.md --id api-excerpt --dry-run   # print candidate
mkt derived refresh page.md --id api-excerpt --replace
mkt derived refresh page.md --id api-excerpt --keep --ack
mkt derived lock page.md --id summary-1
```

Include path should record **target content hash** in provenance today;  
stale detection reuses cache fingerprints + per-slot `input_hash`.

Implementation can land incrementally in markitect-tool (see proposal doc);  
coulomb consumes the API for badges and compare UI.

---

## UX summary

```text
[May be stale]
   → Generate updated version
        → side-by-side Current | Alternative
             → Keep current  (optional: clear badge / ack)
             → Use alternative (replace stored)
             → Cancel
   → Lock field/block
```

Applies to: title, abstractor, visual, outline stub, pinned includes,  
AI/pipeline regions — **one mental model**.

---

## Relation to lock

| | Unlocked derived | Locked | User-edited |
|--|------------------|--------|-------------|
| Stale badge | yes if fingerprint mismatch | yes if mismatch (informational) | optional / policy |
| Generate alternative | yes | no (must unlock) | yes (warns overwrite of user text) |
| Auto-replace | never (default) | never | never |

---

## Stage-1 vs later

| Slice | When |
|-------|------|
| Stale badge + regenerate/replace/keep for **title/abstractor** | Stage-1 authoring |
| Visual same | Stage-1 if visual exists |
| Live include resolve only | With markitect integration |
| Pinned `mkt:derived` include snapshots | Soon after includes land |
| Full pipeline/AI recipes | As generators are added |

---

## Related

- `docs/architecture/authoring-modes-and-lazy-projections.md`
- `docs/architecture/transclusion-and-chunks.md`
- markitect-tool: `docs/transform-compose-include.md`, `docs/cache-incremental.md`, `docs/backend-fabric.md`
- markitect-tool: `docs/proposed-stale-derived-and-include-pinning.md` (proposal for implementation there)