tegwick
f55ef13c75
examples/pqrst-estimate was not the canonical PQRST prompt. The canonical one is ~/pqrst-practice/PqrstPrompt.md, normatively specified in spec/PqrstEstimationPractice.md, and hall-of-helix CLOSING.md requires pasting that block unmodified. Ours was a paraphrase with a different output shape — no Confidence, no Signature, no Dominant factors — and CANP-WP-0002-T03 made it worse by prepending a house_style inclusion to a prompt whose governing document says do not modify it. A copied prompt that silently forked its source is the exact failure INTENT.md opens with, sitting in this repo's own examples directory. The template is now the canonical block extracted programmatically rather than retyped, and the default render is byte-identical to it: 2886 bytes both ways. The source documents two optional add-ons appended after the block. CPF has no conditionals, so they are not a flag: `add_ons` is an input defaulting to the empty string, and each sanctioned add-on is an example fixture. This is worth noting as evidence about section 23's deferred "richer template syntax" — the workaround is adequate here, but it is a workaround. evals/canonical-fidelity.yaml guards the property with sixteen render checks, including not_contains checks naming the paraphrase this package used to be, so the drift cannot silently recur. Version 0.2.1 -> 1.0.0: changed inputs and a materially different intended output is the MAJOR case in section 17. Section 22's worked example and the house-style README both claimed pqrst-estimate composes the style fragment. It no longer does, by design, so both are corrected; composition is illustrated in section 10.4 instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bjefh8NUiEiahN4JLwoSKM Assistant: claude-code Assistant-Model: opus Assistant-Process: 388925@bnt-lap001 Assistant-Session: 3507023f-e0fd-4a1e-9d90-a0d4217d1502
38 lines
1.6 KiB
YAML
38 lines
1.6 KiB
YAML
schema: canned-prompts/eval-rubric/v0.1
|
|
name: canonical-fidelity
|
|
description: >
|
|
The default render must be the canonical prompt from
|
|
~/pqrst-practice/PqrstPrompt.md, verbatim. hall-of-helix CLOSING.md requires
|
|
pasting that block unmodified, so any drift in this package is a defect —
|
|
these checks are what stop it recurring.
|
|
example: examples/basic.yaml
|
|
|
|
render:
|
|
- resolves_all: true
|
|
- not_contains: "{{"
|
|
# The five dimensions, by their canonical names.
|
|
- contains: "P — Main Problem:"
|
|
- contains: "Q — Quality and Tests:"
|
|
- contains: "R — Research and Context Clarification:"
|
|
- contains: "S — Security and Credentials:"
|
|
- contains: "T — Task Organization:"
|
|
# The rules that make the record auditable rather than decorative.
|
|
- contains: "must sum to exactly 100"
|
|
- contains: "retrospective audit, not a planning target"
|
|
- contains: "S may legitimately be 0%"
|
|
- contains: '"Dominant factors" must name concrete session facts'
|
|
# The exact output shape the hall files.
|
|
- contains: "PQRST-Estimate"
|
|
- contains: "Signature: P<int> Q<int> R<int> S<int> T<int>"
|
|
- contains: "Confidence: <low|medium|high>"
|
|
# Guard against the paraphrase this package used to be.
|
|
- not_contains: "Include rationale:"
|
|
- not_contains: "Percentages must sum to 100%."
|
|
|
|
output:
|
|
criteria:
|
|
- The five values are integers in 0..100 summing to exactly 100.
|
|
- Signature agrees with the individual values.
|
|
- Dominant factors names concrete session facts rather than restating percentages.
|
|
- S is 0 when no security-specific work occurred, and is not inflated.
|
|
- No sixth dimension is introduced inside the 5-tuple.
|