diff --git a/CannedPromptFormat-v0.1.md b/CannedPromptFormat-v0.1.md index e8b3930..bcc00a3 100644 --- a/CannedPromptFormat-v0.1.md +++ b/CannedPromptFormat-v0.1.md @@ -678,73 +678,20 @@ Tools MAY render examples directly. `evals` is a list of relative paths to evaluation specifications. -CPF does not standardize a universal evaluation language, which remains a -non-goal. Every eval file MUST declare a `schema`, and a consumer MUST ignore a -schema it does not recognize rather than guessing at its meaning. +CPF v0.1 deliberately does not standardize a universal evaluation language. Eval files SHOULD therefore declare their own evaluator or schema. -CPF defines exactly one schema, so that `evals/` holds something tools can act -on rather than opaque blobs. Other schemas remain legal and are simply not -interpreted here. - -### 12.1 `canned-prompts/eval-rubric/v0.1` +Example: ```yaml schema: canned-prompts/eval-rubric/v0.1 -name: pqrst-estimate-quality -description: The estimate reads as a post-session audit, not a plan. -example: examples/basic.yaml - -render: - - contains: "must sum to exactly 100%" - - not_contains: "{{" - - resolves_all: true - -output: - criteria: - - Percentages sum to exactly 100%. - - Categories are not silently renamed or merged. - - Rationale cites evidence from the session summary. +name: code-review-quality +criteria: + - identifies correctness risks + - distinguishes blocking from advisory findings + - avoids inventing repository facts ``` -| Field | Required | Meaning | -|---|---:|---| -| `schema` | yes | MUST be `canned-prompts/eval-rubric/v0.1` | -| `name` | yes | Identifies this eval within the package | -| `description` | no | What the eval is for | -| `example` | no | Fixture to run against; MUST be a path declared in the manifest's `examples` | -| `render` | no | Deterministic checks on the **rendered prompt** | -| `output` | no | Model-judged criteria for the **result** | - -An eval declaring neither `render` nor `output` asserts nothing and SHOULD be -rejected. - -#### Render checks - -Render checks assert properties of the rendered prompt text. They need no -model, so **any implementation that can render can run them.** - -| Check | Passes when | -|---|---| -| `contains: ` | the rendered prompt contains that text | -| `not_contains: ` | it does not | -| `resolves_all: true` | every placeholder resolved to a value | - -Each entry is a single-key mapping, so a check may appear more than once. - -#### Output criteria - -`output.criteria` is a list of statements about a good result. Judging them -requires running the prompt and assessing what comes back, which CPF does not -specify and most consumers cannot do. They are **declared, not run** — the -same division as `include` and `derive` in § 10.2, and for the same reason. - -#### Results are not part of the package - -An eval file declares what to assess; it MUST NOT record outcomes. Results are -run evidence, which lives outside the immutable package (§ 17) and may be -associated with `@` by a registry or evaluation system without -mutating it. A tool reporting results SHOULD identify the package version, the -eval `name`, and each check's outcome. +A registry may associate externally collected run/eval evidence with `@` without mutating the package. ## 13. Provenance and lineage @@ -839,12 +786,7 @@ A v0.1 validator SHOULD verify at least: 15. every dependency referenced by a composition or derived default declares a `version`, and that version is a semver literal, `any`, `newest`, or a `>=` lower bound (§ 10.1); -16. inclusion does not form a cycle; -17. every eval file parses and declares a `schema`; -18. an eval declaring `canned-prompts/eval-rubric/v0.1` has a `name`, asserts - something via `render` or `output`, uses only known render checks, and — - when it declares an `example` — names a path listed in the manifest's - `examples`. +16. inclusion does not form a cycle. A validator SHOULD additionally warn when a package intended for publication declares an inline derivation prompt (§ 6.1). @@ -1018,13 +960,6 @@ The reference tool satisfies `include` defaults, because inclusion is deterministic and needs no model, and reports `derive` defaults it cannot satisfy. -```text -eval ID run an installed package's render checks -``` - -`eval` runs the deterministic render checks of every recognized eval file and -reports output criteria as declared but not run. - The reference tool never calls a model, so its `resolve` handles supplied values and static defaults only and reports any input whose derived default it cannot satisfy. `render` performs the same resolution and then substitutes; diff --git a/README.md b/README.md index c57e343..ab40b6c 100644 --- a/README.md +++ b/README.md @@ -114,32 +114,6 @@ There is no template inheritance. A package never extends another or overrides its parts; composition is by reference only, so a package's content stays readable without chasing ancestors. -## Evals - -An eval file declares what to assess. `evals/` used to hold whatever an author -put there; it now has one schema the tooling understands, split along the same -line as composition: - -- **render checks** — deterministic assertions about the *rendered prompt* - (`contains`, `not_contains`, `resolves_all`). No model needed, so the - reference CLI runs them. -- **output criteria** — statements about a good *result*. Declared, not run. - -```bash -python canned_prompts.py eval practice/pqrst-estimate -``` - -```text -local:practice/pqrst-estimate@0.2.0 - evals/quality.yaml (pqrst-estimate-quality) - render PASS contains "must sum to exactly 100%" - render PASS resolves_all - output -- 4 criteria declared (not run: judging output needs a model) -``` - -An eval declares assessment, never results. Results are run evidence and live -outside the immutable package. - ## Registries and identity An id names a package *within a registry* (`CannedPromptFormat-v0.1.md` diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index 65cfa06..5bc7886 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -15,7 +15,7 @@ | task | CANP-WP-0001-T03 | done | — | workplans/CANP-WP-0001-statehub-bootstrap.md | | task | CANP-WP-0002-T01 | done | — | workplans/CANP-WP-0002-format-open-questions.md | | task | CANP-WP-0002-T02 | done | — | workplans/CANP-WP-0002-format-open-questions.md | -| task | CANP-WP-0002-T03 | done | — | workplans/CANP-WP-0002-format-open-questions.md | +| task | CANP-WP-0002-T03 | todo | — | workplans/CANP-WP-0002-format-open-questions.md | | task | CANP-WP-0002-T04 | todo | — | workplans/CANP-WP-0002-format-open-questions.md | | task | CANP-WP-0002-T05 | todo | — | workplans/CANP-WP-0002-format-open-questions.md | | task | CANP-WP-0002-T06 | todo | — | workplans/CANP-WP-0002-format-open-questions.md | diff --git a/examples/pqrst-estimate/evals/quality.yaml b/examples/pqrst-estimate/evals/quality.yaml deleted file mode 100644 index 108bf9f..0000000 --- a/examples/pqrst-estimate/evals/quality.yaml +++ /dev/null @@ -1,19 +0,0 @@ -schema: canned-prompts/eval-rubric/v0.1 -name: pqrst-estimate-quality -description: > - The prompt asks for a post-session audit with a hard 100% constraint, and the - resulting estimate respects both. -example: examples/basic.yaml - -render: - - contains: "must sum to exactly 100%" - - contains: "post-session audit, not a planning estimate" - - not_contains: "{{" - - resolves_all: true - -output: - criteria: - - The five percentages sum to exactly 100%. - - Categories are not silently renamed, merged, or dropped. - - Effort is attributed from evidence in the session summary, not invented. - - The estimate reads as an audit of work done, not a plan for work ahead. diff --git a/examples/pqrst-estimate/prompt.yaml b/examples/pqrst-estimate/prompt.yaml index 8444a02..6e65221 100644 --- a/examples/pqrst-estimate/prompt.yaml +++ b/examples/pqrst-estimate/prompt.yaml @@ -49,5 +49,3 @@ provenance: author: canned-prompts seed examples: - examples/basic.yaml -evals: -- evals/quality.yaml diff --git a/reference/README.md b/reference/README.md index 5ed0c61..b1c1798 100644 --- a/reference/README.md +++ b/reference/README.md @@ -2,7 +2,7 @@ This is intentionally a **small reference implementation**, not the intended final architecture. -It demonstrates eight verbs: +It demonstrates seven verbs: ```text add PATH @@ -10,7 +10,6 @@ search QUERY show ID resolve ID --set key=value render ID --set key=value -eval ID install ID [--version VERSION] publish PATH ``` @@ -72,10 +71,6 @@ rather than resolved by guessing. print qualified `:` references. - `include` defaults are satisfied (inclusion is deterministic); `derive` defaults are reported, not run. Inclusion cycles are detected and named. -- `eval` runs the deterministic render checks of any eval declaring the - `canned-prompts/eval-rubric/v0.1` schema, and reports output criteria as - declared but not run. Unrecognized schemas are skipped, not rejected. A - failed render check exits non-zero. - An optional `registry.yaml` names a registry and records namespace claims. `publish` warns when a namespace is declared `closed` — it cannot authenticate a publisher, and says so rather than implying it checked. diff --git a/reference/canned_prompts.py b/reference/canned_prompts.py index 9bd6bf6..f015100 100755 --- a/reference/canned_prompts.py +++ b/reference/canned_prompts.py @@ -23,8 +23,6 @@ FORMAT = "canned-prompt/v0.1" REGISTRY_FORMAT = "canned-prompt-registry/v0.1" LOCAL_REGISTRY = "local" REGISTRY_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$") -EVAL_RUBRIC_SCHEMA = "canned-prompts/eval-rubric/v0.1" -RENDER_CHECKS = ("contains", "not_contains", "resolves_all") PLACEHOLDER_RE = re.compile(r"{{\s*([A-Za-z_][A-Za-z0-9_.-]*)\s*}}") SEMVER_RE = re.compile(r"^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:[-+].*)?$") REQUIRED_FIELDS = ("format", "id", "name", "version", "summary", "template") @@ -137,64 +135,6 @@ def prompt_dependencies(manifest: dict[str, Any]) -> dict[str, dict[str, Any]]: return declared -def read_eval(package_dir: Path, relative: str) -> dict[str, Any]: - path = safe_relative_file(package_dir, relative, "evals") - try: - data = yaml.safe_load(path.read_text(encoding="utf-8")) - except yaml.YAMLError as exc: - raise CannedPromptError(f"invalid YAML in {relative}: {exc}") from exc - if not isinstance(data, dict): - raise CannedPromptError(f"{relative}: an eval file must contain a mapping") - if not isinstance(data.get("schema"), str) or not data["schema"].strip(): - raise CannedPromptError(f"{relative}: an eval file must declare a schema") - return data - - -def validate_eval(relative: str, spec: dict[str, Any], declared_examples: list[str]) -> None: - """Validation rules 17-18 of § 18. Unknown schemas are ignored, not rejected.""" - if spec["schema"] != EVAL_RUBRIC_SCHEMA: - return - if not isinstance(spec.get("name"), str) or not spec["name"].strip(): - raise CannedPromptError(f"{relative}: eval must declare a name") - - render = spec.get("render") or [] - output = spec.get("output") or {} - if not render and not output: - raise CannedPromptError( - f"{relative}: eval asserts nothing; declare render checks or output criteria" - ) - - if not isinstance(render, list): - raise CannedPromptError(f"{relative}: render must be a list of checks") - for check in render: - if not isinstance(check, dict) or len(check) != 1: - raise CannedPromptError( - f"{relative}: each render check must be a single-key mapping" - ) - (kind,) = check - if kind not in RENDER_CHECKS: - raise CannedPromptError( - f"{relative}: unknown render check {kind!r}; expected one of " - + ", ".join(RENDER_CHECKS) - ) - - if output: - if not isinstance(output, dict): - raise CannedPromptError(f"{relative}: output must be a mapping") - criteria = output.get("criteria") or [] - if not isinstance(criteria, list) or not all(isinstance(c, str) for c in criteria): - raise CannedPromptError(f"{relative}: output.criteria must be a list of strings") - - example = spec.get("example") - if example is not None: - if not isinstance(example, str): - raise CannedPromptError(f"{relative}: example must be a path string") - if example not in declared_examples: - raise CannedPromptError( - f"{relative}: example {example!r} is not declared in the manifest's examples" - ) - - def check_composition_reference( name: str, reference: str, field: str, declared: dict[str, dict[str, Any]] ) -> None: @@ -303,10 +243,6 @@ def validate_package(package_dir: Path) -> dict[str, Any]: for item in manifest.get("inputs") or []: validate_input_default(item, declared) - declared_examples = [str(path) for path in (manifest.get("examples") or [])] - for relative in manifest.get("evals") or []: - validate_eval(relative, read_eval(package_dir, relative), declared_examples) - template = template_path.read_text(encoding="utf-8") placeholders = set(PLACEHOLDER_RE.findall(template)) undeclared = sorted(placeholders - names) @@ -636,16 +572,8 @@ def cmd_show(args: argparse.Namespace) -> None: print(yaml.safe_dump(manifest, sort_keys=False, allow_unicode=True).rstrip()) -def coerce_value(raw: Any, spec: dict[str, Any]) -> Any: +def coerce_value(raw: str, spec: dict[str, Any]) -> Any: kind = spec.get("type", "string") - if not isinstance(raw, str): - # Already typed — from a YAML fixture rather than the command line. - # Validate membership, but do not try to parse it. - if kind == "enum" and raw not in (spec.get("values") or []): - raise CannedPromptError( - f"invalid enum value {raw!r}; expected one of {spec.get('values') or []}" - ) - return raw if kind == "boolean": lowered = raw.lower() if lowered in {"true", "1", "yes", "on"}: @@ -947,91 +875,6 @@ def cmd_resolve(args: argparse.Namespace) -> None: ) -def load_example_values(package_dir: Path, relative: str) -> dict[str, Any]: - path = safe_relative_file(package_dir, relative, "examples") - try: - data = yaml.safe_load(path.read_text(encoding="utf-8")) - except yaml.YAMLError as exc: - raise CannedPromptError(f"invalid YAML in {relative}: {exc}") from exc - if not isinstance(data, dict): - raise CannedPromptError(f"{relative}: an example must contain a mapping") - values = data.get("values") or {} - if not isinstance(values, dict): - raise CannedPromptError(f"{relative}: example values must be a mapping") - return values - - -def run_render_checks( - rendered: str, resolution: Resolution, checks: list[dict[str, Any]] -) -> list[tuple[bool, str]]: - """Evaluate § 12.1 render checks. Deterministic: no model is involved.""" - results: list[tuple[bool, str]] = [] - for check in checks: - (kind,) = check - expected = check[kind] - if kind == "contains": - results.append((str(expected) in rendered, f'contains "{expected}"')) - elif kind == "not_contains": - results.append((str(expected) not in rendered, f'not_contains "{expected}"')) - elif kind == "resolves_all": - unresolved = sorted(set(resolution.origins) - set(resolution.values)) - ok = (not unresolved) if expected else bool(unresolved) - detail = f" ({', '.join(unresolved)})" if unresolved else "" - results.append((ok, f"resolves_all{detail}")) - return results - - -def cmd_eval(args: argparse.Namespace) -> None: - catalog = Path(args.catalog).expanduser() - package_dir, registry = resolve_installed(catalog, args.id, args.version) - manifest = validate_package(package_dir) - template_path = safe_relative_file(package_dir, manifest["template"], "template") - template = template_path.read_text(encoding="utf-8") - - relatives = manifest.get("evals") or [] - if not relatives: - print(f"{registry}:{manifest['id']}@{manifest['version']} no evals declared") - return - - print(f"{registry}:{manifest['id']}@{manifest['version']}") - failures = 0 - for relative in relatives: - spec = read_eval(package_dir, relative) - if spec["schema"] != EVAL_RUBRIC_SCHEMA: - print(f" {relative} schema {spec['schema']} not recognized — skipped") - continue - - print(f" {relative} ({spec['name']})") - checks = spec.get("render") or [] - if checks: - values = ( - load_example_values(package_dir, spec["example"]) - if spec.get("example") - else {} - ) - resolution = resolve_inputs( - manifest, values, composer=CatalogComposer(catalog) - ) - missing = sorted(set(PLACEHOLDER_RE.findall(template)) - set(resolution.values)) - rendered = "" if missing else render_template(template, resolution.values) - if missing: - print(f" render FAIL cannot render: unresolved {', '.join(missing)}") - failures += 1 - for ok, label in run_render_checks(rendered, resolution, checks): - print(f" render {'PASS' if ok else 'FAIL'} {label}") - failures += 0 if ok else 1 - - criteria = (spec.get("output") or {}).get("criteria") or [] - if criteria: - print( - f" output -- {len(criteria)} criteria declared " - "(not run: judging output needs a model)" - ) - - if failures: - raise CannedPromptError(f"{failures} render check(s) failed") - - def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(prog="canned-prompts", description=__doc__) sub = parser.add_subparsers(dest="command", required=True) @@ -1068,12 +911,6 @@ def build_parser() -> argparse.ArgumentParser: render.add_argument("--set", dest="set_values", action="append", default=[], metavar="NAME=VALUE") render.set_defaults(func=cmd_render) - evaluate = sub.add_parser("eval", help="run an installed package's render checks") - evaluate.add_argument("id") - evaluate.add_argument("--version") - evaluate.add_argument("--catalog", default=str(default_catalog())) - evaluate.set_defaults(func=cmd_eval) - resolve = sub.add_parser("resolve", help="report how each input and parameter resolves") resolve.add_argument("id") resolve.add_argument("--version") diff --git a/reference/tests/test_canned_prompts.py b/reference/tests/test_canned_prompts.py index 98a0041..2c489cd 100644 --- a/reference/tests/test_canned_prompts.py +++ b/reference/tests/test_canned_prompts.py @@ -504,114 +504,3 @@ inputs: ) with pytest.raises(cp.CannedPromptError, match="must declare a version"): cp.validate_package(pkg) - - -# --- evals (§ 12) --- - -EVAL_BASE = """\ -format: canned-prompt/v0.1 -id: demo/evaluated -name: Evaluated -version: 1.0.0 -summary: Exercise eval files. -template: prompt.md -examples: -- examples/basic.yaml -evals: -- evals/quality.yaml -""" - - -def write_evaluated(pkg: Path, eval_body: str, template: str = "Sum to 100%. {{ topic }}\n") -> Path: - pkg.mkdir(exist_ok=True) - (pkg / "prompt.yaml").write_text( - EVAL_BASE + "inputs:\n - name: topic\n required: false\n default: cats\n", - encoding="utf-8", - ) - (pkg / "prompt.md").write_text(template, encoding="utf-8") - (pkg / "examples").mkdir(exist_ok=True) - (pkg / "examples" / "basic.yaml").write_text( - "name: basic\nvalues:\n topic: dogs\n", encoding="utf-8" - ) - (pkg / "evals").mkdir(exist_ok=True) - (pkg / "evals" / "quality.yaml").write_text(eval_body, encoding="utf-8") - return pkg - - -RUBRIC = """\ -schema: canned-prompts/eval-rubric/v0.1 -name: quality -example: examples/basic.yaml -render: - - contains: "Sum to 100%" - - not_contains: "{{" - - resolves_all: true -output: - criteria: - - Answers the question. -""" - - -def test_valid_eval_passes_validation(tmp_path: Path) -> None: - pkg = write_evaluated(tmp_path / "p", RUBRIC) - assert cp.validate_package(pkg)["id"] == "demo/evaluated" - - -def test_eval_must_declare_a_schema(tmp_path: Path) -> None: - pkg = write_evaluated(tmp_path / "p", "name: quality\nrender: []\n") - with pytest.raises(cp.CannedPromptError, match="must declare a schema"): - cp.validate_package(pkg) - - -def test_unknown_eval_schema_is_ignored(tmp_path: Path) -> None: - pkg = write_evaluated(tmp_path / "p", "schema: someone/else/v1\nwhatever: true\n") - assert cp.validate_package(pkg)["id"] == "demo/evaluated" - - -def test_eval_asserting_nothing_is_rejected(tmp_path: Path) -> None: - pkg = write_evaluated( - tmp_path / "p", "schema: canned-prompts/eval-rubric/v0.1\nname: empty\n" - ) - with pytest.raises(cp.CannedPromptError, match="asserts nothing"): - cp.validate_package(pkg) - - -def test_unknown_render_check_is_rejected(tmp_path: Path) -> None: - pkg = write_evaluated( - tmp_path / "p", - "schema: canned-prompts/eval-rubric/v0.1\nname: q\nrender:\n - matches: 'x.*'\n", - ) - with pytest.raises(cp.CannedPromptError, match="unknown render check"): - cp.validate_package(pkg) - - -def test_eval_example_must_be_declared(tmp_path: Path) -> None: - pkg = write_evaluated( - tmp_path / "p", - "schema: canned-prompts/eval-rubric/v0.1\nname: q\n" - "example: examples/missing.yaml\nrender:\n - contains: x\n", - ) - with pytest.raises(cp.CannedPromptError, match="not declared in the manifest"): - cp.validate_package(pkg) - - -def test_render_checks_evaluate(tmp_path: Path) -> None: - resolution = cp.Resolution(values={"topic": "dogs"}, origins={"topic": "supplied"}) - checks = [{"contains": "dogs"}, {"contains": "cats"}, {"not_contains": "cats"}] - outcomes = cp.run_render_checks("about dogs", resolution, checks) - assert [ok for ok, _ in outcomes] == [True, False, True] - - -def test_resolves_all_reports_unresolved_names() -> None: - resolution = cp.Resolution( - values={"a": 1}, origins={"a": "supplied", "b": "unresolved (no default)"} - ) - outcomes = cp.run_render_checks("text", resolution, [{"resolves_all": True}]) - assert outcomes[0][0] is False - assert "b" in outcomes[0][1] - - -def test_typed_fixture_values_are_not_reparsed(tmp_path: Path) -> None: - """A YAML fixture carries real types; only CLI strings need parsing.""" - manifest = {"parameters": {"flag": {"type": "boolean", "default": False}}} - assert cp.resolve_inputs(manifest, {"flag": True}).values["flag"] is True diff --git a/workplans/CANP-WP-0002-format-open-questions.md b/workplans/CANP-WP-0002-format-open-questions.md index bf67c51..6a47ff4 100644 --- a/workplans/CANP-WP-0002-format-open-questions.md +++ b/workplans/CANP-WP-0002-format-open-questions.md @@ -250,7 +250,7 @@ resolve first; the report still lists inputs before parameters. ```task id: CANP-WP-0002-T04 -status: done +status: todo priority: medium state_hub_task_id: "0daf1097-6599-563f-8965-739cbc874ddf" ``` @@ -259,47 +259,12 @@ Cheapest item to pin down. `evals/` is a reserved path and `evals:` is a manifest list, but § 12 defines no schema, so an eval file is an unvalidated blob that no tool can act on. -**Decisions (operator, 2026-09-06):** - -- *Envelope plus one canonical schema.* Every eval file MUST declare a - `schema`; CPF defines exactly one, `canned-prompts/eval-rubric/v0.1`. Other - schemas stay legal and are skipped rather than rejected, so `evals/` holds - something tools can act on without CPF becoming an evaluation language. -- *Two kinds of check.* The same seam as T01 and T03: a **render check** is a - deterministic assertion about the rendered prompt text (`contains`, - `not_contains`, `resolves_all`) that any implementation able to render can - run, and **output criteria** are statements about a good result, declared but - not run because judging them needs a model. -- *Fixtures are referenced, not duplicated.* An eval names a path already - declared in the manifest's `examples`, tying two reserved paths together and - keeping one copy of each fixture. - -Delivered: - -1. § 12 rewritten: the envelope rule, the unknown-schema ignore rule, and - § 12.1 specifying the rubric schema field by field. -2. Render checks and output criteria are specified separately, each with the - reason it does or does not run locally. -3. "Results are not part of the package" states that an eval declares - assessment and MUST NOT record outcomes; results are run evidence living - outside the immutable package, per `INTENT.md` and § 17. -4. § 18 gains validation rules 17–18; § 21 documents the `eval` verb. -5. `reference/canned_prompts.py`: `read_eval`, `validate_eval`, - `load_example_values`, `run_render_checks`, `cmd_eval`. A failed render - check exits non-zero. Tests 42 → 51. -6. `examples/pqrst-estimate/evals/quality.yaml` (new) is a real eval with four - render checks and four output criteria, and it passes. - -**Fixed while implementing.** `coerce_value` assumed every incoming value was -a command-line string, so a YAML fixture carrying a real type -(`include_rationale: true`) crashed on `.lower()`. Typed values are now -validated but not re-parsed — reaching a fixture through `eval` was the first -code path that supplied them. - -**Deferred deliberately.** No regex render check. It would add a matching -language and a backtracking hazard for little gain over `contains` at this -stage; `contains`, `not_contains` and `resolves_all` cover the cases the seed -actually has. +Define a minimal eval-spec schema: identity, what is being asserted, the +fixture it runs against, and how a result is reported. Keep it declarative and +engine-neutral — "universal prompt evaluation" is an explicit INTENT non-goal, +so this specifies the *file*, not an evaluation engine. Extend § 18 to validate +eval files that declare the schema, and add one eval to +`examples/pqrst-estimate` as a worked case. ## Typed context and dependency contracts