CANP-WP-0002 T04: eval schema, render checks and output criteria

`evals/` was a reserved path holding unvalidated blobs: section 12 named the
directory and gave an illustrative snippet, but nothing was specified, so no
tool could act on an eval file.

Every eval file must now declare a `schema`, and CPF defines exactly one —
`canned-prompts/eval-rubric/v0.1`. Unrecognized schemas stay legal and are
skipped rather than rejected, so the format gains something actionable without
becoming an evaluation language, which remains a non-goal.

The schema splits along the same seam as T01 and T03. Render checks
(`contains`, `not_contains`, `resolves_all`) assert properties of the rendered
prompt text, need no model, and are therefore run by the reference CLI. Output
criteria describe a good result and are declared but not run, because judging
them requires a model. That division is now the format's consistent answer to
"deterministic locally, or not".

An eval references a fixture already declared in the manifest's `examples`
rather than carrying its own copy, so an example that is also an eval fixture
stays honest — both break together.

An eval declares assessment and must not record outcomes. Results are run
evidence and live outside the immutable package, per INTENT.md and section 17.

Spec: 12 rewritten with 12.1, 18 (rules 17-18), 21 (`eval` verb).

Reference CLI: read_eval, validate_eval, load_example_values,
run_render_checks, cmd_eval; a failed render check exits non-zero.
Tests 42 -> 51.

examples/pqrst-estimate/evals/quality.yaml is a real eval with four render
checks and four output criteria, and it passes.

Fixes a latent bug reaching a fixture exposed: coerce_value assumed every
value was a command-line string, so a YAML fixture carrying a real type
(include_rationale: true) crashed on .lower(). Typed values are now validated
but not re-parsed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bjefh8NUiEiahN4JLwoSKM

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 388925@bnt-lap001
Assistant-Session: 3507023f-e0fd-4a1e-9d90-a0d4217d1502

workplans/CANP-WP-0002-format-open-questions.md

@@ -250,7 +250,7 @@ resolve first; the report still lists inputs before parameters.
 
 ```task
 id: CANP-WP-0002-T04
-status: todo
+status: done
 priority: medium
 state_hub_task_id: "0daf1097-6599-563f-8965-739cbc874ddf"
 ```
@@ -259,12 +259,47 @@ Cheapest item to pin down. `evals/` is a reserved path and `evals:` is a
 manifest list, but § 12 defines no schema, so an eval file is an unvalidated
 blob that no tool can act on.
 
-Define a minimal eval-spec schema: identity, what is being asserted, the
-fixture it runs against, and how a result is reported. Keep it declarative and
-engine-neutral — "universal prompt evaluation" is an explicit INTENT non-goal,
-so this specifies the *file*, not an evaluation engine. Extend § 18 to validate
-eval files that declare the schema, and add one eval to
-`examples/pqrst-estimate` as a worked case.
+**Decisions (operator, 2026-09-06):**
+
+- *Envelope plus one canonical schema.* Every eval file MUST declare a
+  `schema`; CPF defines exactly one, `canned-prompts/eval-rubric/v0.1`. Other
+  schemas stay legal and are skipped rather than rejected, so `evals/` holds
+  something tools can act on without CPF becoming an evaluation language.
+- *Two kinds of check.* The same seam as T01 and T03: a **render check** is a
+  deterministic assertion about the rendered prompt text (`contains`,
+  `not_contains`, `resolves_all`) that any implementation able to render can
+  run, and **output criteria** are statements about a good result, declared but
+  not run because judging them needs a model.
+- *Fixtures are referenced, not duplicated.* An eval names a path already
+  declared in the manifest's `examples`, tying two reserved paths together and
+  keeping one copy of each fixture.
+
+Delivered:
+
+1. § 12 rewritten: the envelope rule, the unknown-schema ignore rule, and
+   § 12.1 specifying the rubric schema field by field.
+2. Render checks and output criteria are specified separately, each with the
+   reason it does or does not run locally.
+3. "Results are not part of the package" states that an eval declares
+   assessment and MUST NOT record outcomes; results are run evidence living
+   outside the immutable package, per `INTENT.md` and § 17.
+4. § 18 gains validation rules 17–18; § 21 documents the `eval` verb.
+5. `reference/canned_prompts.py`: `read_eval`, `validate_eval`,
+   `load_example_values`, `run_render_checks`, `cmd_eval`. A failed render
+   check exits non-zero. Tests 42 → 51.
+6. `examples/pqrst-estimate/evals/quality.yaml` (new) is a real eval with four
+   render checks and four output criteria, and it passes.
+
+**Fixed while implementing.** `coerce_value` assumed every incoming value was
+a command-line string, so a YAML fixture carrying a real type
+(`include_rationale: true`) crashed on `.lower()`. Typed values are now
+validated but not re-parsed — reaching a fixture through `eval` was the first
+code path that supplied them.
+
+**Deferred deliberately.** No regex render check. It would add a matching
+language and a backtracking hazard for little gain over `contains` at this
+stage; `contains`, `not_contains` and `resolves_all` cover the cases the seed
+actually has.
 
 ## Typed context and dependency contracts