CANP-WP-0006 T05: HTTP registries, and one finding

--registry now accepts an http(s):// URL as well as a path, implemented with
urllib so reference/ keeps PyYAML as its only dependency.

This was the first real test of INTENT principle 10's claim that a hosted
registry layers on without changing package semantics. Verified against the
running service, almost everything survived the transport unaltered: identity
and its ambiguity rules (a bare id in two registries returns 409 over HTTP just
as it does locally), immutability of a published id@version (identical content
accepted, changed content refused, version bump accepted), strict packaging,
validation, and the index. A package published and then installed over HTTP was
byte-identical to its source — diff -r clean — and its canonical-fidelity eval
still passed after the round trip.

One thing did not survive: a URL is not a registry. A filesystem registry IS
one registry and section 20.1 names it from its directory; an HTTP service
HOSTS SEVERAL behind one base URL. The address therefore cannot name the
registry, so it must be named separately — --as when publishing, a qualified
reference when installing.

Recorded as section 20.4 rather than worked around silently in the client,
because the gap is in the specification's list of registry kinds, not in the
CLI. Publishing to an HTTP registry without --as fails with that explanation
rather than guessing a registry name.

Registry responses are treated as untrusted input (section 19): decode_files
refuses path traversal, with a test. The wire shape round-trips binary content
through base64, also tested.

Reference tests 99 -> 105.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bjefh8NUiEiahN4JLwoSKM

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 388925@bnt-lap001
Assistant-Session: 3507023f-e0fd-4a1e-9d90-a0d4217d1502

reference/README.md

@@ -100,6 +100,12 @@ rather than resolved by guessing.
   source, method, first-inclusion date, and the package's declared author,
   source and licence. `index` lists it. Re-adding keeps the original
   `included_at` and updates `last_seen_at`.
+- `--registry` accepts an `http(s)://` URL as well as a path. HTTP uses the
+  stdlib, so this stays dependency-light. `CANNED_PROMPTS_PUBLISH_TOKEN`
+  supplies the bearer token when publishing.
+- Publishing to an HTTP registry needs `--as NAME`: a URL addresses a service
+  that **hosts several registries**, so unlike a directory it does not name one
+  (§ 20.4). Installing names it in a qualified reference.
 - An optional `registry.yaml` names a registry and records namespace claims.
   `publish` warns when a namespace is declared `closed` — it cannot
   authenticate a publisher, and says so rather than implying it checked.

reference/canned_prompts.py

@@ -11,9 +11,13 @@ import argparse
 import json
 import os
 import re
+import base64
 import datetime as dt
 import shutil
 import sys
+import tempfile
+import urllib.error
+import urllib.request
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Any, Iterable
@@ -772,6 +776,71 @@ def record_inclusion(
     return entry
 
 
+# --- HTTP registries (§ 20) ----------------------------------------------
+#
+# § 20 lists "an HTTP service" as a valid registry. Implementing one surfaced a
+# distinction the specification does not draw: a filesystem registry *is* one
+# registry, but an HTTP service *hosts several* — the reference service keeps
+# `local`, `helix` and others behind a single base URL. So a URL does not name a
+# registry the way a directory does, and the registry must be named separately:
+# by `--as` when publishing, and by a qualified reference when installing.
+# Recorded as a finding rather than worked around silently.
+
+
+def is_http_registry(value: str) -> bool:
+    return value.startswith("http://") or value.startswith("https://")
+
+
+def http_request(
+    url: str, method: str = "GET", payload: dict[str, Any] | None = None
+) -> dict[str, Any]:
+    """A registry call. stdlib only, so `reference/` stays dependency-light."""
+    data = json.dumps(payload).encode("utf-8") if payload is not None else None
+    request = urllib.request.Request(url, data=data, method=method)
+    request.add_header("Accept", "application/json")
+    if data is not None:
+        request.add_header("Content-Type", "application/json")
+    token = os.environ.get("CANNED_PROMPTS_PUBLISH_TOKEN")
+    if token:
+        request.add_header("Authorization", f"Bearer {token}")
+    try:
+        with urllib.request.urlopen(request) as response:
+            return json.loads(response.read().decode("utf-8"))
+    except urllib.error.HTTPError as exc:
+        body = exc.read().decode("utf-8", "replace")
+        try:
+            detail = json.loads(body).get("detail", body)
+        except json.JSONDecodeError:
+            detail = body
+        raise CannedPromptError(f"registry refused ({exc.code}): {detail}") from exc
+    except urllib.error.URLError as exc:
+        raise CannedPromptError(f"registry unreachable: {exc.reason}") from exc
+
+
+def encode_files(package_dir: Path, manifest: dict[str, Any]) -> dict[str, Any]:
+    """The package's files, in the shape the service's archive returns."""
+    files: dict[str, Any] = {}
+    for relative in sorted(package_members(package_dir, manifest)):
+        raw = (package_dir / relative).read_bytes()
+        try:
+            files[relative] = {"text": raw.decode("utf-8")}
+        except UnicodeDecodeError:
+            files[relative] = {"base64": base64.b64encode(raw).decode()}
+    return files
+
+
+def decode_files(files: dict[str, Any], destination: Path) -> None:
+    for relative, item in files.items():
+        if relative.startswith("/") or ".." in Path(relative).parts:
+            raise CannedPromptError(f"unsafe path from registry: {relative}")
+        target = destination / relative
+        target.parent.mkdir(parents=True, exist_ok=True)
+        if "text" in item:
+            target.write_text(item["text"], encoding="utf-8")
+        else:
+            target.write_bytes(base64.b64decode(item["base64"]))
+
+
 def missing_dependencies(catalog: Path, manifest: dict[str, Any]) -> list[str]:
     """Declared prompt dependencies that this catalog cannot satisfy.
 
@@ -878,6 +947,27 @@ def cmd_add(args: argparse.Namespace) -> None:
 def cmd_publish(args: argparse.Namespace) -> None:
     src = Path(args.path)
     manifest = validate_package(src)
+
+    if is_http_registry(args.registry):
+        # A URL hosts several registries, so it cannot name one on its own.
+        if not args.as_registry:
+            raise CannedPromptError(
+                "publishing to an HTTP registry needs --as NAME: a URL addresses a "
+                "service that hosts several registries, so unlike a directory it "
+                "does not name one (§ 20)"
+            )
+        result = http_request(
+            args.registry.rstrip("/") + "/packages",
+            method="POST",
+            payload={
+                "registry": check_registry_name(args.as_registry),
+                "source": str(src.resolve()),
+                "files": encode_files(src.resolve(), manifest),
+            },
+        )
+        print(f"published {result['reference']} -> {args.registry}")
+        return
+
     registry = Path(args.registry).expanduser()
     check_legacy_registry(registry)
     name = registry_name(registry) if registry.is_dir() else None
@@ -900,9 +990,30 @@ def cmd_publish(args: argparse.Namespace) -> None:
 
 
 def cmd_install(args: argparse.Namespace) -> None:
+    catalog = Path(args.catalog).expanduser()
+
+    if is_http_registry(args.registry):
+        reference = args.id + (f"@{args.version}" if args.version else "")
+        archive = http_request(
+            args.registry.rstrip("/") + "/archives/" + reference
+        )
+        source_registry, _, rest = archive["reference"].partition(":")
+        with tempfile.TemporaryDirectory() as tmp:
+            staged = Path(tmp) / "package"
+            staged.mkdir()
+            decode_files(archive["files"], staged)
+            manifest = validate_package(staged)
+            dst = catalog_package_path(
+                catalog, source_registry, manifest["id"], manifest["version"]
+            )
+            copy_package(staged, dst, manifest, "catalog package")
+        record_inclusion(catalog, source_registry, manifest, args.registry, "install")
+        report_missing_dependencies(catalog, manifest)
+        print(f"installed {archive['reference']} -> {dst}")
+        return
+
     registry = Path(args.registry).expanduser()
     check_legacy_registry(registry)
-    catalog = Path(args.catalog).expanduser()
     name = registry_name(registry)
 
     qualifier, package_id = parse_reference(args.id)
@@ -1465,9 +1576,16 @@ def build_parser() -> argparse.ArgumentParser:
     install.add_argument("--registry", default=str(default_registry()))
     install.set_defaults(func=cmd_install)
 
-    publish = sub.add_parser("publish", help="publish a package to a filesystem registry")
+    publish = sub.add_parser("publish", help="publish a package to a registry")
     publish.add_argument("path")
     publish.add_argument("--registry", default=str(default_registry()))
+    publish.add_argument(
+        "--as",
+        dest="as_registry",
+        default=None,
+        metavar="REGISTRY",
+        help="registry name to publish into; required for an HTTP registry",
+    )
     publish.set_defaults(func=cmd_publish)
 
     return parser

reference/tests/test_canned_prompts.py

@@ -1,3 +1,4 @@
+import argparse
 from pathlib import Path
 
 import pytest
@@ -942,3 +943,68 @@ def test_explicit_registry_path_is_never_second_guessed(tmp_path: Path, monkeypa
     monkeypatch.setenv("CANNED_PROMPTS_HOME", str(tmp_path))
     (tmp_path / "registry").mkdir()
     cp.check_legacy_registry(tmp_path / "somewhere-else")
+
+
+# --- HTTP registries (§ 20.4) ---
+
+def test_http_registry_detection() -> None:
+    assert cp.is_http_registry("https://registry.example")
+    assert cp.is_http_registry("http://localhost:8878")
+    assert not cp.is_http_registry("/home/user/.canned-prompts/default")
+
+
+def test_encode_decode_round_trips_a_package(tmp_path: Path) -> None:
+    """The wire shape must reproduce the package byte for byte."""
+    src = write_evaluated(tmp_path / "src", RUBRIC)
+    (src / "assets").mkdir()
+    (src / "assets" / "binary.dat").write_bytes(bytes(range(256)))
+    manifest = cp.validate_package(src)
+
+    files = cp.encode_files(src, manifest)
+    assert "base64" in files["assets/binary.dat"], "non-UTF-8 must not go as text"
+
+    out = tmp_path / "out"
+    out.mkdir()
+    cp.decode_files(files, out)
+    for relative in cp.package_members(src, manifest):
+        assert (out / relative).read_bytes() == (src / relative).read_bytes()
+
+
+def test_decode_refuses_path_traversal(tmp_path: Path) -> None:
+    """A registry is untrusted input (§ 19); it must not write outside the package."""
+    with pytest.raises(cp.CannedPromptError, match="unsafe path"):
+        cp.decode_files({"../escape.txt": {"text": "no"}}, tmp_path)
+
+
+def test_publish_to_http_requires_a_registry_name(tmp_path: Path, monkeypatch) -> None:
+    """§ 20.4: a URL hosts several registries, so it cannot name one."""
+    src = write_pkg(tmp_path / "p", BASE + "inputs:\n  - name: greeting\n    required: false\n    default: hi\n")
+    args = argparse.Namespace(
+        path=str(src), registry="https://registry.example", as_registry=None
+    )
+    with pytest.raises(cp.CannedPromptError, match="needs --as NAME"):
+        cp.cmd_publish(args)
+
+
+def test_http_errors_surface_the_registry_detail(monkeypatch) -> None:
+    import urllib.error, io
+
+    def raise_conflict(*a, **k):
+        raise urllib.error.HTTPError(
+            "u", 409, "Conflict", {}, io.BytesIO(b'{"detail":"already exists"}')
+        )
+
+    monkeypatch.setattr(cp.urllib.request, "urlopen", raise_conflict)
+    with pytest.raises(cp.CannedPromptError, match="registry refused \\(409\\): already exists"):
+        cp.http_request("https://registry.example/packages")
+
+
+def test_unreachable_registry_is_reported_clearly(monkeypatch) -> None:
+    import urllib.error
+
+    def raise_urlerror(*a, **k):
+        raise urllib.error.URLError("connection refused")
+
+    monkeypatch.setattr(cp.urllib.request, "urlopen", raise_urlerror)
+    with pytest.raises(cp.CannedPromptError, match="registry unreachable"):
+        cp.http_request("https://registry.example/packages")