docs/proposed-stale-derived-and-include-pinning.md

@@ -1,183 +0,0 @@
-# Proposal: stale detection, pinned derived blocks, replace-or-keep
-
-| Field | Value |
-|-------|--------|
-| Date | 2026-08-13 |
-| Status | **proposal** (not implemented) |
-| Origin | coulomb.social product needs + gap analysis of include/cache/provenance |
-| Consumer | coulomb-social, agents, any markitect host UI |
-
-## Problem
-
-Markitect already has:
-
-- live **includes** (`resolve_includes`, selectors, provenance events),
-- **content hashes** / cache fingerprints / backend refresh plans,
-- **processing provenance** envelopes for extensions and workflows.
-
-It does **not** yet define a single, document-native way to:
-
-1. keep a **stored snapshot** of derived content (include, pipeline, AI, rule),
-2. detect that snapshot is **stale** vs current inputs,
-3. **generate an alternative**,
-4. let the user **replace** or **keep** (and **lock**).
-
-Live resolve alone is not enough for co-creation products: users need  
-exportable, offline, reviewable snapshots that can still be refreshed  
-explicitly.
-
-## Goals
-
-- One **derived-slot** model for: includes, projections (host app),  
-  pipelines, external data, classic compute, AI.
-- Deterministic **staleness** from input fingerprints.
-- **No silent overwrite** of locked or user-accepted content by default.
-- CLI + library APIs hosts can use for “may be stale” UX.
-- Stay Markdown-native and reversible; no required database.
-
-## Non-goals (v1)
-
-- Full workflow UI inside markitect-tool.
-- Automatic background regeneration of **pinned** snapshots without user consent.
-- Replacing live `mkt:include` as the default (live resolve remains the **default** product mode — spreadsheet-like).
-
-**Product policy (coulomb):** live includes by default; pin is the exception  
-(“paste values”). Stale → alternative → replace|keep applies to **pinned**  
-derived slots and to projection fields, not to pure live resolve.
-
-## Proposed document syntax
-
-### Live include (existing)
-
-```markdown
-<!-- mkt:include path="src.md" selector="sections[heading=API]" -->
-```
-
-Always resolves from current target at evaluation time.
-
-### Pinned derived block (new)
-
-```markdown
-<!-- mkt:derived id="api-excerpt"
-     kind="include"
-     path="src.md"
-     selector="sections[heading=API]"
-     heading_delta="0"
-     input_hash="sha256:…"
-     locked="false"
-     generator="include@v1"
-     generated_at="2026-08-13T12:00:00Z" -->
-…snapshot body (markdown)…
-<!-- /mkt:derived -->
-```
-
-Other kinds:
-
-```markdown
-<!-- mkt:derived id="sum-1" kind="ai" recipe="summarize@v2"
-     inputs="body" input_hash="sha256:…" locked="false" -->
-…
-<!-- /mkt:derived -->
-
-<!-- mkt:derived id="tbl" kind="pipeline" recipe="metrics@v1"
-     inputs="data/metrics.json" input_hash="sha256:…" -->
-…
-<!-- /mkt:derived -->
-```
-
-| Attribute | Meaning |
-|-----------|---------|
-| `id` | Stable slot id within the document |
-| `kind` | `include` \| `pipeline` \| `external` \| `ai` \| `rule` \| `compute` \| … |
-| `input_hash` | Fingerprint of inputs when snapshot was accepted |
-| `locked` | If true, refuse refresh/replace until unlocked |
-| `recipe` / `generator` | How to recompute |
-| `path` / `selector` / … | Kind-specific params (include reuses include attrs) |
-
-**Readers** that do not understand `mkt:derived` still see the snapshot body  
-as normal markdown between comments (comments ignored by most renderers).
-
-## APIs (proposed)
-
-```text
-list_derived_slots(markdown) -> list[DerivedSlot]
-fingerprint_slot(slot, context) -> str
-is_stale(slot, context) -> bool
-evaluate_slot(slot, context) -> Candidate   # body + provenance + new hash
-apply_replace(markdown, slot_id, candidate) -> str
-apply_keep(markdown, slot_id, *, ack: bool) -> str
-set_lock(markdown, slot_id, locked: bool) -> str
-```
-
-`context` supplies base_dir, body text, variable bindings, optional  
-extension registry for `ai` / `pipeline` recipes.
-
-### CLI (proposed)
-
-```bash
-mkt derived status DOC.md
-mkt derived refresh DOC.md --id ID --dry-run    # print candidate + stale?
-mkt derived refresh DOC.md --id ID --replace
-mkt derived refresh DOC.md --id ID --keep --ack # keep text, update hash
-mkt derived lock DOC.md --id ID
-mkt derived unlock DOC.md --id ID
-```
-
-### Include integration
-
-- `resolve_includes` stays **live**.
-- New optional mode: `materialize_includes(markdown) -> markdown`  
-  converts live includes to pinned `mkt:derived` snapshots (initial pin).
-- `is_stale` for `kind=include` hashes target file(s) after selector extract  
-  (reuse query/extract + cache fingerprint).
-
-## Stale → alternative → replace | keep
-
-```text
-status: stale
-  evaluate_slot → candidate
-  host shows Current | Alternative
-    replace → apply_replace
-    keep    → apply_keep (optional ack fingerprint)
-    cancel  → no write
-```
-
-Library returns both bodies; **host owns UI**. Markitect does not open GUIs.
-
-## Relation to existing pieces
-
-| Existing | Reuse |
-|----------|--------|
-| `OperationProvenance` / `ProcessingProvenance` | Emit on evaluate/replace |
-| `mkt cache fingerprint` | Input hashing for files |
-| `resolve_includes` + selectors | `kind=include` evaluate path |
-| Extension framework | Register pipeline/ai recipes |
-| Backend refresh-plan | Filesystem/index stale; derived slots are document-local |
-
-## Implementation sketch (workplan later)
-
-1. Parse/serialize `mkt:derived` regions (ignore inside fences).  
-2. `list` / `fingerprint` / `is_stale` for `kind=include` only.  
-3. `evaluate` + `apply_replace` / `apply_keep` / lock.  
-4. CLI `mkt derived *`.  
-5. `materialize_includes` helper.  
-6. Extension hook for non-include kinds.  
-7. Tests: cycle-free, path sandbox, lock enforcement, fence safety.
-
-## Acceptance (when implemented)
-
-- [ ] Pinned include snapshot detects source change as stale.  
-- [ ] Dry-run alternative differs when source changed.  
-- [ ] Replace updates body + `input_hash`.  
-- [ ] Keep + ack clears stale without changing body.  
-- [ ] Locked slot rejects replace.  
-- [ ] Live `mkt:include` unchanged for existing callers.
-
-## References
-
-- coulomb: `docs/architecture/stale-derived-content.md`
-- coulomb: `docs/architecture/authoring-modes-and-lazy-projections.md`
-- `docs/transform-compose-include.md`
-- `docs/cache-incremental.md`
-- `docs/content-references.md`
-- `docs/backend-fabric.md`