coulomb/canned-prompts
4
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add the catalog index, and package a real prompt collection

Browse source 
CANP-WP-0004. The operator asked canned-prompts to build a database of
versioned prompts recording where each came from and when — the first step
toward a platform for collaborative prompting.

The format had nowhere to put that. `provenance` records who wrote a prompt and
where the idea came from; nothing recorded how a copy arrived in a particular
store. Section 20.3 now specifies an `index.yaml` as store metadata rather than
package data: how a copy arrived differs for every consumer, and recording an
arrival must never rewrite the package that arrived.

`add`, `install` and `publish` record registry, id, version, name, source,
method, first-inclusion date, and the package's declared author, source and
licence — the last three copied so a listing is readable without opening every
package. A new `index` verb lists it. `included_at` is never overwritten; a
re-run updates `last_seen_at`, because when a package first entered a
collection is a fact about history rather than about the last command run.

Tests 84 -> 90.

Also records two findings from actually using the format:

CANP-WP-0004-T03 — inclusion has no deduplication, so a diamond dependency
renders shared content once per path. Found by composing a real collection.
Not fixed here: deduplicating means choosing which occurrence survives and
deciding what happens when two paths resolve different versions, which is
resolver behaviour that section 10.4 deliberately avoids. Handed to
CANP-WP-0005 with a leaning: document it, warn at validation time, do not
deduplicate.

The add_ons workaround in practice/pqrst-estimate is evidence about section
23's deferred "richer template syntax" — an optional appendix has to be an
input with an empty default, because CPF has no conditionals.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bjefh8NUiEiahN4JLwoSKM

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 388925@bnt-lap001
Assistant-Session: 3507023f-e0fd-4a1e-9d90-a0d4217d1502
This commit is contained in:
tegwick 2026-09-06 17:14:21 +02:00
parent f55ef13c75
commit b3280df742
7 changed files with 386 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

45
CannedPromptFormat.md
View file

@ -1105,6 +1105,50 @@ registry/
A registry's own layout is not namespaced by registry name: within one
registry, an id is unambiguous by definition.
### 20.3 The index
A store MAY keep an `index.yaml` at its root recording which package versions
entered it, from where, and when:
```yaml
format: canned-prompt-index/v0.1
entries:
- registry: local
id: helix/repo-orient
version: 0.1.0
name: Repo Orientation
source: /home/worsch/helix-forge/prompts/repo-orient
method: add
included_at: "2026-09-06T13:31:02Z"
declared_author: Bernd Worsch
declared_source: personal prompt collection, contributed 2026-09-06
license: MIT
```
This is deliberately **not** package data. `provenance` (§ 13) records who wrote
a prompt and where the idea came from; the index records how a copy arrived in
*this* store — a fact about the store, not about the artifact, and one that
would differ for every consumer. Keeping it outside preserves immutability
(§ 17): recording an arrival never rewrites the package that arrived.
| Field | Meaning |
|---|---|
| `registry`, `id`, `version` | Which package version this entry is about |
| `source` | Where the copy came from: a path, a registry, a URL |
| `method` | How it arrived: `add`, `install`, `publish` |
| `included_at` | When it **first** entered this store |
| `last_seen_at` | When it was most recently re-recorded, if ever |
| `declared_author`, `declared_source` | Copied from the package's own `provenance`, so the index is readable without opening every package |
| `license` | Copied from the manifest, so licensing is visible in a listing |
`included_at` records first arrival and MUST NOT be overwritten when the same
`registry`/`id`/`version` is recorded again; a re-run updates `last_seen_at`
instead. When a package first entered a collection is a fact about history, not
about the last time someone ran a command.
Because identity is registry-scoped (§ 3.2), the same `id@version` from two
registries is two entries, not a conflict.
## 21. Reference CLI semantics
The reference tool uses two stores:
@ -1138,6 +1182,7 @@ satisfy.
```text
eval ID run an installed package's render checks
index [ID] show what entered this catalog, from where and when
```
`eval` runs the deterministic render checks of every recognized eval file and

20
README.md
View file

@ -181,6 +181,26 @@ signing and trust scoring are explicit non-goals. Ownership lives with the
registry rather than in the package, so no package carries an unverifiable
assertion of authority.
## The index
Every store keeps an `index.yaml` recording which package versions entered it,
from where, and when:
```bash
python canned_prompts.py index
```
```text
local:helix/repo-orient@0.1.0 2026-09-06T13:31:02Z add
from /home/worsch/helix-forge/prompts/repo-orient
declares source personal prompt collection, contributed 2026-09-06
```
This is store metadata, not package data. `provenance` records who wrote a
prompt; the index records how a copy arrived *here* — which differs for every
consumer, and which must never rewrite the package it describes. `included_at`
is first arrival and is never overwritten; a re-run updates `last_seen_at`.
## Dependencies are reported, not fetched
Installing a package that composes another warns when the dependency is

7
reference/README.md
View file

@ -2,7 +2,7 @@
This is intentionally a **small reference implementation**, not the intended final architecture.
It demonstrates eight verbs:
It demonstrates nine verbs:
```text
add PATH
@ -11,6 +11,7 @@ show ID
resolve ID --set key=value
render ID --set key=value
eval ID
index [ID]
install ID [--version VERSION]
publish PATH
```
@ -87,6 +88,10 @@ rather than resolved by guessing.
- `add` and `install` name declared prompt dependencies the catalog cannot
satisfy. They do not fetch them — resolution is the consumer's job (§ 10) —
but a package that looks installed and cannot render should say so.
- `add`, `install` and `publish` record an entry in the store's `index.yaml`:
source, method, first-inclusion date, and the package's declared author,
source and licence. `index` lists it. Re-adding keeps the original
`included_at` and updates `last_seen_at`.
- An optional `registry.yaml` names a registry and records namespace claims.
`publish` warns when a namespace is declared `closed` — it cannot
authenticate a publisher, and says so rather than implying it checked.

104
reference/canned_prompts.py
View file

@ -11,6 +11,7 @@ import argparse
import json
import os
import re
import datetime as dt
import shutil
import sys
from dataclasses import dataclass, field
@ -30,6 +31,8 @@ RENDER_CHECKS = ("contains", "not_contains", "resolves_all")
CAPABILITY_RE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$")
RESERVED_FILES = ("prompt.yaml", "README.md", "LICENSE")
RESERVED_DIRS = ("examples", "evals", "assets")
INDEX_FILE = "index.yaml"
INDEX_FORMAT = "canned-prompt-index/v0.1"
PLACEHOLDER_RE = re.compile(r"{{\s*([A-Za-z_][A-Za-z0-9_.-]*)\s*}}")
SEMVER_RE = re.compile(
r"^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)"
@ -678,6 +681,76 @@ def copy_package(src: Path, dst: Path, manifest: dict[str, Any], what: str) -> l
return skipped
def read_index(store: Path) -> dict[str, Any]:
"""The catalog index: which packages entered this store, from where, when.
Registry-side metadata, deliberately outside the immutable package (§ 20).
A package records who wrote it; the index records how it got here.
"""
path = store / INDEX_FILE
if not path.is_file():
return {"format": INDEX_FORMAT, "entries": []}
try:
data = yaml.safe_load(path.read_text(encoding="utf-8"))
except yaml.YAMLError as exc:
raise CannedPromptError(f"invalid YAML in {path}: {exc}") from exc
if not isinstance(data, dict) or not isinstance(data.get("entries"), list):
raise CannedPromptError(f"{path}: index must be a mapping with an entries list")
if data.get("format") != INDEX_FORMAT:
raise CannedPromptError(f"unsupported index format: {data.get('format')!r}")
return data
def index_entry_key(entry: dict[str, Any]) -> tuple[str, str, str]:
return (entry.get("registry", ""), entry.get("id", ""), entry.get("version", ""))
def record_inclusion(
store: Path,
registry: str,
manifest: dict[str, Any],
source: str,
method: str,
) -> dict[str, Any]:
"""Record that a package version entered this store, and from where.
Re-recording the same registry/id/version keeps the original `included_at`:
the date a package first entered is a fact about history, not about the last
time someone re-ran a command.
"""
index = read_index(store)
entry = {
"registry": registry,
"id": manifest["id"],
"version": manifest["version"],
"name": manifest.get("name"),
"source": source,
"method": method,
"included_at": dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
}
provenance = manifest.get("provenance") or {}
if isinstance(provenance, dict):
for field in ("author", "source"):
if provenance.get(field):
entry[f"declared_{field}"] = str(provenance[field])
if manifest.get("license"):
entry["license"] = manifest["license"]
existing = {index_entry_key(e): e for e in index["entries"]}
key = index_entry_key(entry)
if key in existing:
entry["included_at"] = existing[key].get("included_at", entry["included_at"])
entry["last_seen_at"] = dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
existing[key] = entry
index["entries"] = [existing[k] for k in sorted(existing)]
store.mkdir(parents=True, exist_ok=True)
(store / INDEX_FILE).write_text(
yaml.safe_dump(index, sort_keys=False, allow_unicode=True), encoding="utf-8"
)
return entry
def missing_dependencies(catalog: Path, manifest: dict[str, Any]) -> list[str]:
"""Declared prompt dependencies that this catalog cannot satisfy.
@ -737,6 +810,7 @@ def cmd_add(args: argparse.Namespace) -> None:
registry = check_registry_name(args.as_registry)
dst = catalog_package_path(catalog, registry, manifest["id"], manifest["version"])
report_skipped(copy_package(src.resolve(), dst, manifest, "catalog package"))
record_inclusion(catalog, registry, manifest, str(src.resolve()), "add")
report_missing_dependencies(catalog, manifest)
print(f"added {registry}:{manifest['id']}@{manifest['version']} -> {dst}")
@ -759,6 +833,7 @@ def cmd_publish(args: argparse.Namespace) -> None:
dst = registry_package_path(registry, manifest["id"], manifest["version"])
report_skipped(copy_package(src.resolve(), dst, manifest, "published package"))
record_inclusion(registry, name or "", manifest, str(src.resolve()), "publish")
label = f"{name}:{manifest['id']}" if name else manifest["id"]
print(f"published {label}@{manifest['version']} -> {dst}")
@ -778,6 +853,7 @@ def cmd_install(args: argparse.Namespace) -> None:
manifest = validate_package(src)
dst = catalog_package_path(catalog, name, manifest["id"], manifest["version"])
copy_package(src, dst, manifest, "catalog package")
record_inclusion(catalog, name, manifest, str(registry.resolve()), "install")
report_missing_dependencies(catalog, manifest)
print(f"installed {name}:{manifest['id']}@{manifest['version']} -> {dst}")
@ -1238,6 +1314,28 @@ def cmd_eval(args: argparse.Namespace) -> None:
raise CannedPromptError(f"{failures} render check(s) failed")
def cmd_index(args: argparse.Namespace) -> None:
store = Path(args.catalog).expanduser()
entries = read_index(store)["entries"]
if args.id:
_, package_id = parse_reference(args.id)
entries = [e for e in entries if e.get("id") == package_id]
if args.json:
print(json.dumps(entries, indent=2, ensure_ascii=False))
return
if not entries:
print("no packages recorded in this catalog")
return
width = max(len(f"{e.get('registry')}:{e.get('id')}@{e.get('version')}") for e in entries)
for entry in entries:
label = f"{entry.get('registry')}:{entry.get('id')}@{entry.get('version')}"
print(f"{label:<{width}} {entry.get('included_at')} {entry.get('method')}")
print(f"{'':<{width}} from {entry.get('source')}")
declared = entry.get("declared_source")
if declared:
print(f"{'':<{width}} declares source {declared}")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="canned-prompts", description=__doc__)
sub = parser.add_subparsers(dest="command", required=True)
@ -1274,6 +1372,12 @@ def build_parser() -> argparse.ArgumentParser:
render.add_argument("--set", dest="set_values", action="append", default=[], metavar="NAME=VALUE")
render.set_defaults(func=cmd_render)
index = sub.add_parser("index", help="show what entered this catalog, from where and when")
index.add_argument("id", nargs="?")
index.add_argument("--catalog", default=str(default_catalog()))
index.add_argument("--json", action="store_true")
index.set_defaults(func=cmd_index)
evaluate = sub.add_parser("eval", help="run an installed package's render checks")
evaluate.add_argument("id")
evaluate.add_argument("--version")

59
reference/tests/test_canned_prompts.py
View file

@ -808,3 +808,62 @@ def test_dependency_present_but_wrong_version_is_reported(composed: Path) -> Non
manifest = cp.validate_package(package_dir)
manifest["dependencies"]["prompts"][0]["version"] = "9.9.9"
assert cp.missing_dependencies(composed, manifest) == ["style/house@9.9.9"]
# --- catalog index (§ 20.3) ---
def indexed(tmp_path: Path) -> tuple[Path, dict]:
store = tmp_path / "catalog"
manifest = {"id": "helix/thing", "version": "1.0.0", "name": "Thing",
"license": "MIT", "provenance": {"author": "Ada", "source": "~/somewhere"}}
return store, manifest
def test_index_starts_empty(tmp_path: Path) -> None:
store, _ = indexed(tmp_path)
assert cp.read_index(store)["entries"] == []
def test_record_inclusion_captures_source_and_date(tmp_path: Path) -> None:
store, manifest = indexed(tmp_path)
entry = cp.record_inclusion(store, "local", manifest, "/src/thing", "add")
assert entry["id"] == "helix/thing"
assert entry["source"] == "/src/thing"
assert entry["method"] == "add"
assert entry["declared_author"] == "Ada"
assert entry["declared_source"] == "~/somewhere"
assert entry["license"] == "MIT"
assert entry["included_at"].endswith("Z")
assert cp.read_index(store)["entries"] == [entry]
def test_reincluding_keeps_the_original_date(tmp_path: Path) -> None:
"""When a package first entered is a fact about history, not about reruns."""
store, manifest = indexed(tmp_path)
first = cp.record_inclusion(store, "local", manifest, "/src/thing", "add")
again = cp.record_inclusion(store, "local", manifest, "/elsewhere", "install")
assert again["included_at"] == first["included_at"]
assert "last_seen_at" in again
assert len(cp.read_index(store)["entries"]) == 1
def test_distinct_versions_are_separate_entries(tmp_path: Path) -> None:
store, manifest = indexed(tmp_path)
cp.record_inclusion(store, "local", manifest, "/src", "add")
cp.record_inclusion(store, "local", {**manifest, "version": "1.1.0"}, "/src", "add")
assert len(cp.read_index(store)["entries"]) == 2
def test_same_id_from_two_registries_is_two_entries(tmp_path: Path) -> None:
store, manifest = indexed(tmp_path)
cp.record_inclusion(store, "local", manifest, "/a", "add")
cp.record_inclusion(store, "house", manifest, "/b", "install")
assert len(cp.read_index(store)["entries"]) == 2
def test_bad_index_format_is_rejected(tmp_path: Path) -> None:
store, _ = indexed(tmp_path)
store.mkdir(parents=True)
(store / "index.yaml").write_text("format: something/else\nentries: []\n", encoding="utf-8")
with pytest.raises(cp.CannedPromptError, match="unsupported index format"):
cp.read_index(store)

105
workplans/CANP-WP-0004-prompt-index.md Normal file
View file

@ -0,0 +1,105 @@
---
id: CANP-WP-0004
type: workplan
title: "Catalog index: where a package came from and when"
domain: agents
repo: canned-prompts
status: finished
owner: codex
topic_slug: practice
created: "2026-09-06"
updated: "2026-09-06"
---
# Catalog index: where a package came from and when
Driven by packaging a real collection (`helix-forge` `HF-WP-0005`). The operator
asked for canned-prompts to "build a database of versioned prompts", recording
the source a prompt came from and the date it was included — the first step
toward a platform for collaborative prompting.
The format had no place for this. `provenance` (§ 13) records who wrote a prompt
and where the idea came from, but nothing recorded how a *copy* arrived in a
*particular* store.
## Fix practice/pqrst-estimate to be the canonical prompt
```task
id: CANP-WP-0004-T01
status: done
priority: high
```
**Found while researching helix-forge.** `examples/pqrst-estimate` was not the
canonical PQRST prompt. The canonical one is `~/pqrst-practice/PqrstPrompt.md`,
normatively specified in `spec/PqrstEstimationPractice.md`, and
`hall-of-helix/CLOSING.md` requires pasting it unmodified. Ours was a paraphrase
with a different output shape — no `Confidence`, no `Signature`, no `Dominant
factors` — and `CANP-WP-0002-T03` had made it worse by prepending a
`house_style` inclusion to a prompt whose governing document says do not modify
it.
A copied prompt that silently forked its source, sitting in the examples
directory of the repo whose `INTENT.md` opens by naming that exact failure.
Fixed: the template is the canonical block extracted programmatically rather
than retyped, and the default render is byte-identical at 2886 bytes.
`evals/canonical-fidelity.yaml` guards it with sixteen render checks, including
`not_contains` checks naming the paraphrase it used to be. Version 0.2.1 →
1.0.0, the § 17 MAJOR case.
The source documents two optional add-ons appended after the block. CPF has no
conditionals, so `add_ons` is an input defaulting to the empty string and each
add-on is an example fixture. Adequate, but a workaround — recorded as evidence
about § 23's deferred "richer template syntax".
## Add the index
```task
id: CANP-WP-0004-T02
status: done
priority: high
```
A store may keep an `index.yaml` recording which package versions entered it,
from where, and when. Specified in § 20.3, deliberately as **store** metadata
rather than package data: how a copy arrived differs for every consumer, and
recording an arrival must never rewrite the package that arrived (§ 17).
`add`, `install` and `publish` record an entry carrying registry, id, version,
name, source, method, `included_at`, the package's declared author and source,
and its licence — the last two copied so a listing is readable without opening
every package. A new `index` verb lists it.
`included_at` is first arrival and is never overwritten; a re-run updates
`last_seen_at` instead. When a package first entered a collection is a fact
about history, not about the last time someone ran a command.
`reference/canned_prompts.py`: `read_index`, `record_inclusion`,
`index_entry_key`, `cmd_index`. Tests 84 → 90.
## Record the inclusion-deduplication gap
```task
id: CANP-WP-0004-T03
status: done
priority: medium
```
**Found by using the format.** `helix/repo-advance` composed
`helix/commit-sync`, which composes `helix/custodian-conventions`, while also
composing the conventions itself. The rendered prompt contained the conventions
block twice: CPF inclusion has no deduplication, so a diamond dependency renders
shared content once per path.
Not fixed in the format. Deduplicating would mean deciding *which* occurrence
survives and what happens when the two paths resolve different versions — that
is resolver behaviour, and § 10.4 keeps composition declarative on purpose.
Nothing in § 10.4 currently warns an author, which is the actual defect.
Downstream this was fixed by factoring (`HF-WP-0005-T03`), which is the right
answer for a collection and may be the right general advice.
**Handed to `CANP-WP-0005`:** document the diamond behaviour in § 10.4 and
decide whether a validator should warn when one package reaches the same
dependency by two paths.

47
workplans/CANP-WP-0005-inclusion-diamond.md Normal file
View file

@ -0,0 +1,47 @@
---
id: CANP-WP-0005
type: workplan
title: "Document and detect inclusion diamonds"
domain: agents
repo: canned-prompts
status: proposed
owner: codex
topic_slug: practice
created: "2026-09-06"
updated: "2026-09-06"
---
# Document and detect inclusion diamonds
Residual from `CANP-WP-0004` (`origin: residual`, `origin_ref: CANP-WP-0004`).
## Warn about repeated inclusion
```task
id: CANP-WP-0005-T01
status: todo
priority: medium
```
CPF inclusion (§ 10.4) does not deduplicate: when one package reaches the same
dependency by two paths, the included text renders once per path. This was found
by building a real collection — `helix/repo-advance` rendered its conventions
block twice — and worked around downstream by factoring the shared routine into
its own fragment.
§ 10.4 says nothing about it, so an author meets the behaviour only by reading
the output carefully. That silence is the defect.
Two questions, and the second depends on the first:
1. **Document it.** State in § 10.4 that inclusion is textual and repeated,
not deduplicated, and give the factoring pattern that avoids a diamond.
2. **Detect it.** Decide whether validation should warn when a package reaches
the same dependency by more than one path. A warning is cheap and catches the
mistake at `add` time. Silently deduplicating is a different proposition
entirely — it would mean choosing which occurrence survives, and deciding what
happens when two paths resolve *different versions* of the same dependency.
That is resolver behaviour, and § 10.4 keeps composition declarative on
purpose.
Leaning: document it now, warn at validation time, and do not deduplicate.