Propose pinned derived blocks with stale detect and replace-or-keep
Document mkt:derived snapshot syntax, fingerprint/stale APIs, and CLI for include/pipeline/AI slots. Driven by coulomb co-creation needs; not implemented.
This commit is contained in:
parent
5489d2213e
commit
cb3bd78b98
1 changed files with 179 additions and 0 deletions
179
docs/proposed-stale-derived-and-include-pinning.md
Normal file
179
docs/proposed-stale-derived-and-include-pinning.md
Normal file
|
|
@ -0,0 +1,179 @@
|
|||
# Proposal: stale detection, pinned derived blocks, replace-or-keep
|
||||
|
||||
| Field | Value |
|
||||
|-------|--------|
|
||||
| Date | 2026-08-13 |
|
||||
| Status | **proposal** (not implemented) |
|
||||
| Origin | coulomb.social product needs + gap analysis of include/cache/provenance |
|
||||
| Consumer | coulomb-social, agents, any markitect host UI |
|
||||
|
||||
## Problem
|
||||
|
||||
Markitect already has:
|
||||
|
||||
- live **includes** (`resolve_includes`, selectors, provenance events),
|
||||
- **content hashes** / cache fingerprints / backend refresh plans,
|
||||
- **processing provenance** envelopes for extensions and workflows.
|
||||
|
||||
It does **not** yet define a single, document-native way to:
|
||||
|
||||
1. keep a **stored snapshot** of derived content (include, pipeline, AI, rule),
|
||||
2. detect that snapshot is **stale** vs current inputs,
|
||||
3. **generate an alternative**,
|
||||
4. let the user **replace** or **keep** (and **lock**).
|
||||
|
||||
Live resolve alone is not enough for co-creation products: users need
|
||||
exportable, offline, reviewable snapshots that can still be refreshed
|
||||
explicitly.
|
||||
|
||||
## Goals
|
||||
|
||||
- One **derived-slot** model for: includes, projections (host app),
|
||||
pipelines, external data, classic compute, AI.
|
||||
- Deterministic **staleness** from input fingerprints.
|
||||
- **No silent overwrite** of locked or user-accepted content by default.
|
||||
- CLI + library APIs hosts can use for “may be stale” UX.
|
||||
- Stay Markdown-native and reversible; no required database.
|
||||
|
||||
## Non-goals (v1)
|
||||
|
||||
- Full workflow UI inside markitect-tool.
|
||||
- Automatic background regeneration as default policy.
|
||||
- Replacing live `mkt:include` (live resolve remains valid).
|
||||
|
||||
## Proposed document syntax
|
||||
|
||||
### Live include (existing)
|
||||
|
||||
```markdown
|
||||
<!-- mkt:include path="src.md" selector="sections[heading=API]" -->
|
||||
```
|
||||
|
||||
Always resolves from current target at evaluation time.
|
||||
|
||||
### Pinned derived block (new)
|
||||
|
||||
```markdown
|
||||
<!-- mkt:derived id="api-excerpt"
|
||||
kind="include"
|
||||
path="src.md"
|
||||
selector="sections[heading=API]"
|
||||
heading_delta="0"
|
||||
input_hash="sha256:…"
|
||||
locked="false"
|
||||
generator="include@v1"
|
||||
generated_at="2026-08-13T12:00:00Z" -->
|
||||
…snapshot body (markdown)…
|
||||
<!-- /mkt:derived -->
|
||||
```
|
||||
|
||||
Other kinds:
|
||||
|
||||
```markdown
|
||||
<!-- mkt:derived id="sum-1" kind="ai" recipe="summarize@v2"
|
||||
inputs="body" input_hash="sha256:…" locked="false" -->
|
||||
…
|
||||
<!-- /mkt:derived -->
|
||||
|
||||
<!-- mkt:derived id="tbl" kind="pipeline" recipe="metrics@v1"
|
||||
inputs="data/metrics.json" input_hash="sha256:…" -->
|
||||
…
|
||||
<!-- /mkt:derived -->
|
||||
```
|
||||
|
||||
| Attribute | Meaning |
|
||||
|-----------|---------|
|
||||
| `id` | Stable slot id within the document |
|
||||
| `kind` | `include` \| `pipeline` \| `external` \| `ai` \| `rule` \| `compute` \| … |
|
||||
| `input_hash` | Fingerprint of inputs when snapshot was accepted |
|
||||
| `locked` | If true, refuse refresh/replace until unlocked |
|
||||
| `recipe` / `generator` | How to recompute |
|
||||
| `path` / `selector` / … | Kind-specific params (include reuses include attrs) |
|
||||
|
||||
**Readers** that do not understand `mkt:derived` still see the snapshot body
|
||||
as normal markdown between comments (comments ignored by most renderers).
|
||||
|
||||
## APIs (proposed)
|
||||
|
||||
```text
|
||||
list_derived_slots(markdown) -> list[DerivedSlot]
|
||||
fingerprint_slot(slot, context) -> str
|
||||
is_stale(slot, context) -> bool
|
||||
evaluate_slot(slot, context) -> Candidate # body + provenance + new hash
|
||||
apply_replace(markdown, slot_id, candidate) -> str
|
||||
apply_keep(markdown, slot_id, *, ack: bool) -> str
|
||||
set_lock(markdown, slot_id, locked: bool) -> str
|
||||
```
|
||||
|
||||
`context` supplies base_dir, body text, variable bindings, optional
|
||||
extension registry for `ai` / `pipeline` recipes.
|
||||
|
||||
### CLI (proposed)
|
||||
|
||||
```bash
|
||||
mkt derived status DOC.md
|
||||
mkt derived refresh DOC.md --id ID --dry-run # print candidate + stale?
|
||||
mkt derived refresh DOC.md --id ID --replace
|
||||
mkt derived refresh DOC.md --id ID --keep --ack # keep text, update hash
|
||||
mkt derived lock DOC.md --id ID
|
||||
mkt derived unlock DOC.md --id ID
|
||||
```
|
||||
|
||||
### Include integration
|
||||
|
||||
- `resolve_includes` stays **live**.
|
||||
- New optional mode: `materialize_includes(markdown) -> markdown`
|
||||
converts live includes to pinned `mkt:derived` snapshots (initial pin).
|
||||
- `is_stale` for `kind=include` hashes target file(s) after selector extract
|
||||
(reuse query/extract + cache fingerprint).
|
||||
|
||||
## Stale → alternative → replace | keep
|
||||
|
||||
```text
|
||||
status: stale
|
||||
evaluate_slot → candidate
|
||||
host shows Current | Alternative
|
||||
replace → apply_replace
|
||||
keep → apply_keep (optional ack fingerprint)
|
||||
cancel → no write
|
||||
```
|
||||
|
||||
Library returns both bodies; **host owns UI**. Markitect does not open GUIs.
|
||||
|
||||
## Relation to existing pieces
|
||||
|
||||
| Existing | Reuse |
|
||||
|----------|--------|
|
||||
| `OperationProvenance` / `ProcessingProvenance` | Emit on evaluate/replace |
|
||||
| `mkt cache fingerprint` | Input hashing for files |
|
||||
| `resolve_includes` + selectors | `kind=include` evaluate path |
|
||||
| Extension framework | Register pipeline/ai recipes |
|
||||
| Backend refresh-plan | Filesystem/index stale; derived slots are document-local |
|
||||
|
||||
## Implementation sketch (workplan later)
|
||||
|
||||
1. Parse/serialize `mkt:derived` regions (ignore inside fences).
|
||||
2. `list` / `fingerprint` / `is_stale` for `kind=include` only.
|
||||
3. `evaluate` + `apply_replace` / `apply_keep` / lock.
|
||||
4. CLI `mkt derived *`.
|
||||
5. `materialize_includes` helper.
|
||||
6. Extension hook for non-include kinds.
|
||||
7. Tests: cycle-free, path sandbox, lock enforcement, fence safety.
|
||||
|
||||
## Acceptance (when implemented)
|
||||
|
||||
- [ ] Pinned include snapshot detects source change as stale.
|
||||
- [ ] Dry-run alternative differs when source changed.
|
||||
- [ ] Replace updates body + `input_hash`.
|
||||
- [ ] Keep + ack clears stale without changing body.
|
||||
- [ ] Locked slot rejects replace.
|
||||
- [ ] Live `mkt:include` unchanged for existing callers.
|
||||
|
||||
## References
|
||||
|
||||
- coulomb: `docs/architecture/stale-derived-content.md`
|
||||
- coulomb: `docs/architecture/authoring-modes-and-lazy-projections.md`
|
||||
- `docs/transform-compose-include.md`
|
||||
- `docs/cache-incremental.md`
|
||||
- `docs/content-references.md`
|
||||
- `docs/backend-fabric.md`
|
||||
Loading…
Add table
Add a link
Reference in a new issue