diff --git a/docs/proposed-stale-derived-and-include-pinning.md b/docs/proposed-stale-derived-and-include-pinning.md deleted file mode 100644 index 165f472..0000000 --- a/docs/proposed-stale-derived-and-include-pinning.md +++ /dev/null @@ -1,183 +0,0 @@ -# Proposal: stale detection, pinned derived blocks, replace-or-keep - -| Field | Value | -|-------|--------| -| Date | 2026-08-13 | -| Status | **proposal** (not implemented) | -| Origin | coulomb.social product needs + gap analysis of include/cache/provenance | -| Consumer | coulomb-social, agents, any markitect host UI | - -## Problem - -Markitect already has: - -- live **includes** (`resolve_includes`, selectors, provenance events), -- **content hashes** / cache fingerprints / backend refresh plans, -- **processing provenance** envelopes for extensions and workflows. - -It does **not** yet define a single, document-native way to: - -1. keep a **stored snapshot** of derived content (include, pipeline, AI, rule), -2. detect that snapshot is **stale** vs current inputs, -3. **generate an alternative**, -4. let the user **replace** or **keep** (and **lock**). - -Live resolve alone is not enough for co-creation products: users need -exportable, offline, reviewable snapshots that can still be refreshed -explicitly. - -## Goals - -- One **derived-slot** model for: includes, projections (host app), - pipelines, external data, classic compute, AI. -- Deterministic **staleness** from input fingerprints. -- **No silent overwrite** of locked or user-accepted content by default. -- CLI + library APIs hosts can use for “may be stale” UX. -- Stay Markdown-native and reversible; no required database. - -## Non-goals (v1) - -- Full workflow UI inside markitect-tool. -- Automatic background regeneration of **pinned** snapshots without user consent. -- Replacing live `mkt:include` as the default (live resolve remains the **default** product mode — spreadsheet-like). - -**Product policy (coulomb):** live includes by default; pin is the exception -(“paste values”). Stale → alternative → replace|keep applies to **pinned** -derived slots and to projection fields, not to pure live resolve. - -## Proposed document syntax - -### Live include (existing) - -```markdown - -``` - -Always resolves from current target at evaluation time. - -### Pinned derived block (new) - -```markdown - -…snapshot body (markdown)… - -``` - -Other kinds: - -```markdown - -… - - - -… - -``` - -| Attribute | Meaning | -|-----------|---------| -| `id` | Stable slot id within the document | -| `kind` | `include` \| `pipeline` \| `external` \| `ai` \| `rule` \| `compute` \| … | -| `input_hash` | Fingerprint of inputs when snapshot was accepted | -| `locked` | If true, refuse refresh/replace until unlocked | -| `recipe` / `generator` | How to recompute | -| `path` / `selector` / … | Kind-specific params (include reuses include attrs) | - -**Readers** that do not understand `mkt:derived` still see the snapshot body -as normal markdown between comments (comments ignored by most renderers). - -## APIs (proposed) - -```text -list_derived_slots(markdown) -> list[DerivedSlot] -fingerprint_slot(slot, context) -> str -is_stale(slot, context) -> bool -evaluate_slot(slot, context) -> Candidate # body + provenance + new hash -apply_replace(markdown, slot_id, candidate) -> str -apply_keep(markdown, slot_id, *, ack: bool) -> str -set_lock(markdown, slot_id, locked: bool) -> str -``` - -`context` supplies base_dir, body text, variable bindings, optional -extension registry for `ai` / `pipeline` recipes. - -### CLI (proposed) - -```bash -mkt derived status DOC.md -mkt derived refresh DOC.md --id ID --dry-run # print candidate + stale? -mkt derived refresh DOC.md --id ID --replace -mkt derived refresh DOC.md --id ID --keep --ack # keep text, update hash -mkt derived lock DOC.md --id ID -mkt derived unlock DOC.md --id ID -``` - -### Include integration - -- `resolve_includes` stays **live**. -- New optional mode: `materialize_includes(markdown) -> markdown` - converts live includes to pinned `mkt:derived` snapshots (initial pin). -- `is_stale` for `kind=include` hashes target file(s) after selector extract - (reuse query/extract + cache fingerprint). - -## Stale → alternative → replace | keep - -```text -status: stale - evaluate_slot → candidate - host shows Current | Alternative - replace → apply_replace - keep → apply_keep (optional ack fingerprint) - cancel → no write -``` - -Library returns both bodies; **host owns UI**. Markitect does not open GUIs. - -## Relation to existing pieces - -| Existing | Reuse | -|----------|--------| -| `OperationProvenance` / `ProcessingProvenance` | Emit on evaluate/replace | -| `mkt cache fingerprint` | Input hashing for files | -| `resolve_includes` + selectors | `kind=include` evaluate path | -| Extension framework | Register pipeline/ai recipes | -| Backend refresh-plan | Filesystem/index stale; derived slots are document-local | - -## Implementation sketch (workplan later) - -1. Parse/serialize `mkt:derived` regions (ignore inside fences). -2. `list` / `fingerprint` / `is_stale` for `kind=include` only. -3. `evaluate` + `apply_replace` / `apply_keep` / lock. -4. CLI `mkt derived *`. -5. `materialize_includes` helper. -6. Extension hook for non-include kinds. -7. Tests: cycle-free, path sandbox, lock enforcement, fence safety. - -## Acceptance (when implemented) - -- [ ] Pinned include snapshot detects source change as stale. -- [ ] Dry-run alternative differs when source changed. -- [ ] Replace updates body + `input_hash`. -- [ ] Keep + ack clears stale without changing body. -- [ ] Locked slot rejects replace. -- [ ] Live `mkt:include` unchanged for existing callers. - -## References - -- coulomb: `docs/architecture/stale-derived-content.md` -- coulomb: `docs/architecture/authoring-modes-and-lazy-projections.md` -- `docs/transform-compose-include.md` -- `docs/cache-incremental.md` -- `docs/content-references.md` -- `docs/backend-fabric.md`