canned-prompts/workplans/CANP-WP-0006-hosted-registry-service.md
tegwick c076d8395e CANP-WP-0006 T05: HTTP registries, and one finding
--registry now accepts an http(s):// URL as well as a path, implemented with
urllib so reference/ keeps PyYAML as its only dependency.

This was the first real test of INTENT principle 10's claim that a hosted
registry layers on without changing package semantics. Verified against the
running service, almost everything survived the transport unaltered: identity
and its ambiguity rules (a bare id in two registries returns 409 over HTTP just
as it does locally), immutability of a published id@version (identical content
accepted, changed content refused, version bump accepted), strict packaging,
validation, and the index. A package published and then installed over HTTP was
byte-identical to its source — diff -r clean — and its canonical-fidelity eval
still passed after the round trip.

One thing did not survive: a URL is not a registry. A filesystem registry IS
one registry and section 20.1 names it from its directory; an HTTP service
HOSTS SEVERAL behind one base URL. The address therefore cannot name the
registry, so it must be named separately — --as when publishing, a qualified
reference when installing.

Recorded as section 20.4 rather than worked around silently in the client,
because the gap is in the specification's list of registry kinds, not in the
CLI. Publishing to an HTTP registry without --as fails with that explanation
rather than guessing a registry name.

Registry responses are treated as untrusted input (section 19): decode_files
refuses path traversal, with a test. The wire shape round-trips binary content
through base64, also tested.

Reference tests 99 -> 105.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bjefh8NUiEiahN4JLwoSKM

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 388925@bnt-lap001
Assistant-Session: 3507023f-e0fd-4a1e-9d90-a0d4217d1502
2026-09-06 20:39:04 +02:00

257 lines
11 KiB
Markdown

---
id: CANP-WP-0006
type: workplan
title: "Hosted registry and index service"
domain: agents
repo: canned-prompts
status: active
owner: codex
topic_slug: practice
created: "2026-09-06"
updated: "2026-09-06"
state_hub_workstream_id: "70c069e9-7abe-569a-aee2-ffcd15e8970d"
---
# Hosted registry and index service
Toward a production deployment on railiance01 as `rapp-canned-prompts`.
**Sequencing finding.** A `rapp-*` repo packages and operates a deployable its
`ownership_repo` already produces — `declarations/rapp.yaml` pins
`upstream_components.version` to a container digest. `canned-prompts` has no
deployable: a CLI module with no HTTP surface, no image, no database. A rapp
whose `upstream_components` points at nothing is a stub that tells the fleet's
tooling a deployment exists when none does. The service comes first.
**Decisions (operator, 2026-09-06):**
- *Hosted registry plus index* is the v1 surface. § 20 already permits an HTTP
registry and § 20.3's index is the query surface, so this validates INTENT
principle 10's "registry-ready" claim rather than inventing a new concept.
- *The service lives in `canned-prompts`*, so `rapp.yaml` gets
`ownership_repo: canned-prompts` — the `sbom-nexus` shape, where product
ownership stays out of the operations repo.
- *Platform service, tenant-keyed regardless.* Deployed as a platform service
like `sbom-nexus`, but all owned data is keyed by tenant from the first
migration. `business-app-service-contract_v0.1` § 1.3 makes that mandatory so
a later consolidation is a data-copy rather than a rewrite; a collaborative
prompting platform will need it, and retrofitting is the expensive path.
- *Service first, rapp when there is a digest to pin.*
**Stack**, matching `state-hub` and `sbom-nexus`: FastAPI, uvicorn, SQLAlchemy,
Alembic, Pydantic, PostgreSQL. `RailianceAppDeploymentGuide.md` additionally
requires unauthenticated `/healthz` and `/readyz`.
**Boundary.** The service hosts packages; it does not become an agent runtime.
`INTENT.md`'s deliberate boundary still holds — no model execution, no
orchestration. Rendering stays deterministic, and `derive` stays a declaration
the service does not satisfy.
## Service skeleton and health surface
```task
id: CANP-WP-0006-T01
status: done
priority: high
state_hub_task_id: "6e4178f0-a431-59de-8bd1-fb5b2de95302"
```
A `service/` package inside this repo: FastAPI application factory, settings
via pydantic-settings, and the health surface the deployment guide requires —
unauthenticated `/healthz` and `/readyz`, plus `/state/health` for consistency
with `state-hub` and `sbom-nexus`.
`/readyz` must actually check the database, not return 200 unconditionally. A
readiness probe that cannot fail is a liveness probe with a misleading name.
Keep `reference/` untouched. It is the format's conformance witness and must
stay dependency-light; the service is a separate consumer of the same package
semantics.
**Done.** `service/` with a FastAPI factory, pydantic-settings config, and the
three endpoints. `CANNED_PROMPTS_DATABASE_URL` has no default, so a
misconfigured service refuses to start rather than quietly using a local
database.
**Bug found by its own test.** `check_readiness` first caught every failure in
one `except` and reported "database unreachable". An unmigrated but perfectly
reachable database therefore reported a connection problem — sending an
operator to credentials and networking when the fix was `alembic upgrade`.
Connectivity and schema are now checked separately.
## Tenant-keyed schema and first migration
```task
id: CANP-WP-0006-T02
status: done
priority: high
state_hub_task_id: "bc0e4a49-63b1-5fa1-9ec9-5ed0f505baa1"
```
Alembic migration `0001` creating the store. Every table holding owned data
carries a tenant key from this migration onward — packages, versions, and index
entries — per the service contract § 1.3. No business logic may assume it is
the only tenant (§ 1.4).
Model the three things the format already distinguishes, and do not conflate
them:
- **package version** — immutable content addressed by `<id>@<version>`
(§ 17), scoped to a registry because identity is registry-scoped (§ 3.2);
- **index entry** — how a version arrived in this store (§ 20.3): source,
method, `included_at` which is never overwritten, `last_seen_at`;
- **manifest fields** worth indexing for discovery — name, summary, tags,
license, type.
Store package files as content, not as rows per file.
**Done.** Migration `0001` creates `package_versions`, `package_files` and
`index_entries`; a test asserts every table in the metadata carries `tenant`, so
adding an unkeyed table fails the suite rather than being noticed at
consolidation time. Uniqueness is `(tenant, registry, package_id, version)`.
## Read API
```task
id: CANP-WP-0006-T03
status: done
priority: high
state_hub_task_id: "0ae11e65-68b0-543c-b8fa-f3188a4d239a"
```
`GET /packages` (search over id, name, summary, tags), `GET /packages/{id}`
(versions), `GET /packages/{id}/{version}` (manifest),
`GET /packages/{id}/{version}/archive` (the package files), and
`GET /index` (§ 20.3 entries).
Ambiguity is reported, never guessed (§ 3.2): a bare id matching packages from
more than one registry returns the candidates, not a choice.
**Done**, with the route shape worth recording. Package ids contain `/`, so the
obvious `/packages/{id}/{version}` is ambiguous under a greedy path parameter.
Rather than invent an HTTP-specific identifier, the routes speak the format's
own `<registry>:<id>@<version>` syntax and parse it — `:` and `@` are both legal
in a path segment, each route keeps a distinct prefix, and the API therefore
tests § 3.2's reference notation rather than working around it.
Ambiguity returns **409 with the candidates**, not 300: the request is
answerable once the caller says which registry they meant. Omitting a version
applies § 17.1's selectors, so a prerelease is never chosen implicitly.
Validation is delegated to `reference/`, installed into the service
environment rather than reimplemented. One validator means the service and the
CLI cannot disagree about what a valid package is — a service accepting
something the CLI rejects would be exactly the divergence this project exists to
prevent. Importing it is not changing it; `reference/` stays the
dependency-light conformance witness.
**Test-vs-production difference found and handled.** SQLite autoincrements
`INTEGER PRIMARY KEY` only, never `BIGINT`, so the SQLite-backed tests could not
insert a row. `BigInteger().with_variant(Integer, "sqlite")` keeps `BIGINT` on
PostgreSQL, where this actually runs, while letting the tests exercise the same
models and the same migration.
## Publish API
```task
id: CANP-WP-0006-T04
status: done
priority: high
state_hub_task_id: "929932b6-931b-5409-a648-b6814f07f0b4"
```
`POST /packages` accepting a package, validating it with the same rules as the
CLI, and refusing a re-publish of an existing `<id>@<version>` with different
content (§ 17). Strict packaging applies: reserved paths and manifest-referenced
files only (§ 2).
Namespace policy (§ 20.1) is enforced here rather than advised, because unlike a
filesystem registry a service *can* authenticate a publisher. Note what identity
mechanism it uses; if there is none yet, say so and refuse writes rather than
accepting anonymous publishes into a closed namespace.
**Done.** `POST /packages` takes the same `files` shape `GET /archives`
returns, so an archive round-trips into a publish without translation and a
mirror is a GET followed by a POST — verified by a test, not just asserted.
**The identity mechanism, stated plainly.** A single shared bearer token
proving the caller is *the operator of this service*. Not per-publisher
identity: every token holder is indistinguishable. With no token configured the
service is read-only, which is the right default rather than an inconvenience —
§ 20.1 asks a registry to refuse publication into a closed namespace it does not
consider the publisher to own, and an unauthenticated service considers nobody
to own anything.
Namespace claims live in `namespace_claims` (migration `0002`) and are enforced,
which a filesystem registry cannot do at all — but only as precisely as the
identity allows. A closed namespace is protected from anonymous callers; it
cannot be attributed among several publishers. Until per-publisher identity
exists, a claim's `owner` is documentation rather than an access decision, and
`auth.py` says so rather than letting the code imply more than it delivers.
**Handed forward:** per-publisher identity is the main thing between this and a
registry several people can publish to.
**Migration hygiene found while adding `0002`.** Autogenerate proposed an
`ALTER COLUMN TYPE` on `package_files.package_version_id`, because the foreign
key's type was left to inference and compared as a variant against a reflected
plain type. SQLite cannot alter a column type, so `0002` failed halfway,
leaving the table created and the revision unstamped — the partially-applied
state that is worst to debug later. Fixed at the cause: the column is typed
explicitly, and `0001` was corrected rather than patched over, which is
legitimate only because it has never run outside this repo's tests. `alembic
check` now reports no drift.
## HTTP registry in the reference CLI
```task
id: CANP-WP-0006-T05
status: done
priority: medium
state_hub_task_id: "8dacbb68-aaa2-5f91-942d-a0667f1f1fc7"
```
Teach `--registry` to accept an `https://` URL, so `publish` and `install` work
against the service. § 20 already allows an HTTP registry; this is the first
implementation of one, and the first real test of whether package semantics
survive a transport change unmodified — which is what INTENT principle 10
claims.
If they do not survive it, that is a finding about the format, not a bug to
paper over in the client.
**Done, and it produced exactly one finding.**
*What survived unaltered*, verified against the running service: identity and
its ambiguity rules (a bare id in two registries came back 409 over HTTP just as
it does locally), immutability of a published `<id>@<version>` (identical
content accepted, changed content refused, a version bump accepted), strict
packaging, validation, and the index. A package published and installed over
HTTP was **byte-identical** to its source — `diff -r` clean — and its
`canonical-fidelity` eval still passed after the round trip.
*What did not*: **a URL is not a registry.** A filesystem registry *is* one
registry and § 20.1 names it from its directory; an HTTP service *hosts
several* behind one base URL. So the address cannot name the registry, and it
has to be named separately — `--as` when publishing, a qualified reference when
installing. Recorded as § 20.4 rather than papered over in the client, because
the gap is in the specification's list of registry kinds, not in the CLI.
Implemented with `urllib` rather than a library, so `reference/` keeps PyYAML as
its only dependency. Registry responses are treated as untrusted input (§ 19):
`decode_files` refuses path traversal, with a test.
## Image and smoke contract
```task
id: CANP-WP-0006-T06
status: todo
priority: medium
state_hub_task_id: "3c008ebe-bf07-5cee-8678-7b1ed27283ef"
```
Dockerfile and a published image, plus the checks a `rapp` smoke contract will
assert: health ok, migration at head, private service only, image digest match.
Completing this produces the digest that `rapp-canned-prompts` needs to pin, at
which point that repo can be created against something real.