Package the canonical PQRST prompt faithfully
examples/pqrst-estimate was not the canonical PQRST prompt. The canonical one is ~/pqrst-practice/PqrstPrompt.md, normatively specified in spec/PqrstEstimationPractice.md, and hall-of-helix CLOSING.md requires pasting that block unmodified. Ours was a paraphrase with a different output shape — no Confidence, no Signature, no Dominant factors — and CANP-WP-0002-T03 made it worse by prepending a house_style inclusion to a prompt whose governing document says do not modify it. A copied prompt that silently forked its source is the exact failure INTENT.md opens with, sitting in this repo's own examples directory. The template is now the canonical block extracted programmatically rather than retyped, and the default render is byte-identical to it: 2886 bytes both ways. The source documents two optional add-ons appended after the block. CPF has no conditionals, so they are not a flag: `add_ons` is an input defaulting to the empty string, and each sanctioned add-on is an example fixture. This is worth noting as evidence about section 23's deferred "richer template syntax" — the workaround is adequate here, but it is a workaround. evals/canonical-fidelity.yaml guards the property with sixteen render checks, including not_contains checks naming the paraphrase this package used to be, so the drift cannot silently recur. Version 0.2.1 -> 1.0.0: changed inputs and a materially different intended output is the MAJOR case in section 17. Section 22's worked example and the house-style README both claimed pqrst-estimate composes the style fragment. It no longer does, by design, so both are corrected; composition is illustrated in section 10.4 instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bjefh8NUiEiahN4JLwoSKM Assistant: claude-code Assistant-Model: opus Assistant-Process: 388925@bnt-lap001 Assistant-Session: 3507023f-e0fd-4a1e-9d90-a0d4217d1502
This commit is contained in:
parent
55b8600946
commit
f55ef13c75
8 changed files with 144 additions and 151 deletions
|
|
@ -1153,7 +1153,9 @@ These semantics are illustrative, not mandatory for other implementations.
|
|||
## 22. Worked example
|
||||
|
||||
This is `examples/pqrst-estimate` in the `canned-prompts` repository, abridged.
|
||||
It composes a shared style fragment, so it also demonstrates § 10.4.
|
||||
It packages a prompt whose governing document requires it be used *unmodified*,
|
||||
so the package's job is fidelity: the default render must equal the source
|
||||
byte for byte, and an eval enforces that.
|
||||
|
||||
`prompt.yaml`:
|
||||
|
||||
|
|
@ -1161,66 +1163,43 @@ It composes a shared style fragment, so it also demonstrates § 10.4.
|
|||
format: canned-prompt/v0.2
|
||||
id: practice/pqrst-estimate
|
||||
name: PQRST Estimate
|
||||
version: 0.2.1
|
||||
summary: Estimate how session effort was distributed across PQRST categories.
|
||||
version: 1.0.0
|
||||
summary: The canonical end-of-session PQRST effort audit.
|
||||
template: prompt.md
|
||||
|
||||
dependencies:
|
||||
prompts:
|
||||
- id: practice/house-style
|
||||
version: ">= 0.1.0"
|
||||
requirement: required
|
||||
|
||||
inputs:
|
||||
- name: house_style
|
||||
- name: add_ons
|
||||
type: content
|
||||
required: false
|
||||
default:
|
||||
include: practice/house-style
|
||||
|
||||
- name: session_summary
|
||||
type: content
|
||||
required: true
|
||||
|
||||
parameters:
|
||||
include_rationale:
|
||||
type: boolean
|
||||
default: true
|
||||
description: Optional text appended after the canonical block.
|
||||
default: ''
|
||||
|
||||
output:
|
||||
format: markdown
|
||||
|
||||
compatibility:
|
||||
capabilities: [long-context]
|
||||
format: text
|
||||
description: A PQRST-Estimate record.
|
||||
|
||||
license: MIT
|
||||
|
||||
examples: [examples/basic.yaml]
|
||||
evals: [evals/quality.yaml]
|
||||
examples: [examples/basic.yaml, examples/with-rationale.yaml]
|
||||
evals: [evals/canonical-fidelity.yaml]
|
||||
|
||||
tags: [retrospective, agentic-coding, pqrst]
|
||||
tags: [pqrst, retrospective, agentic-coding]
|
||||
|
||||
provenance:
|
||||
author: pqrst-practice
|
||||
source: ~/pqrst-practice/PqrstPrompt.md
|
||||
```
|
||||
|
||||
`prompt.md`:
|
||||
The source prompt documents two optional add-ons appended after the main block.
|
||||
CPF has no conditionals (§ 5), so they are not expressed as a flag: `add_ons` is
|
||||
an input whose default is the empty string, and each sanctioned add-on is an
|
||||
example fixture. The default render is therefore the source prompt exactly, and
|
||||
`examples/with-rationale.yaml` supplies one add-on when a reader wants it.
|
||||
|
||||
```markdown
|
||||
{{ house_style }}
|
||||
|
||||
Review the following coding-session summary and estimate the distribution of
|
||||
session effort across PQRST. Percentages must sum to 100%.
|
||||
|
||||
P = main problem
|
||||
Q = quality and tests
|
||||
R = research and context clarification
|
||||
S = security and credentials
|
||||
T = task organization
|
||||
|
||||
Session:
|
||||
|
||||
{{ session_summary }}
|
||||
|
||||
Include rationale: {{ include_rationale }}
|
||||
```
|
||||
`evals/canonical-fidelity.yaml` guards that property — `resolves_all`, the
|
||||
canonical dimension names and rules, and `not_contains` checks naming the
|
||||
paraphrase this package once was (§ 12.1). Composition is illustrated separately
|
||||
in § 10.4.
|
||||
|
||||
## 23. Open questions
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +1,7 @@
|
|||
# practice/house-style
|
||||
|
||||
A `type: fragment` package: a reusable block meant to be **included** by other
|
||||
packages rather than used on its own.
|
||||
|
||||
`examples/pqrst-estimate` composes it:
|
||||
A `type: fragment` package (§ 3.2): a reusable block meant to be **included** by
|
||||
other packages rather than used on its own.
|
||||
|
||||
```yaml
|
||||
dependencies:
|
||||
|
|
@ -25,3 +23,8 @@ text, with no model involved — so the reference CLI performs it during
|
|||
|
||||
Its `tone` parameter is inherited from the including package when that package
|
||||
declares one, and otherwise falls back to the default here.
|
||||
|
||||
**Not composed by `examples/pqrst-estimate`.** That package reproduces a prompt
|
||||
whose governing document requires it be used unmodified, so it composes nothing
|
||||
by design. Real compositions of this fragment live in prompt collections that
|
||||
consume the format; see `helix-forge/prompts/`.
|
||||
|
|
|
|||
38
examples/pqrst-estimate/evals/canonical-fidelity.yaml
Normal file
38
examples/pqrst-estimate/evals/canonical-fidelity.yaml
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
schema: canned-prompts/eval-rubric/v0.1
|
||||
name: canonical-fidelity
|
||||
description: >
|
||||
The default render must be the canonical prompt from
|
||||
~/pqrst-practice/PqrstPrompt.md, verbatim. hall-of-helix CLOSING.md requires
|
||||
pasting that block unmodified, so any drift in this package is a defect —
|
||||
these checks are what stop it recurring.
|
||||
example: examples/basic.yaml
|
||||
|
||||
render:
|
||||
- resolves_all: true
|
||||
- not_contains: "{{"
|
||||
# The five dimensions, by their canonical names.
|
||||
- contains: "P — Main Problem:"
|
||||
- contains: "Q — Quality and Tests:"
|
||||
- contains: "R — Research and Context Clarification:"
|
||||
- contains: "S — Security and Credentials:"
|
||||
- contains: "T — Task Organization:"
|
||||
# The rules that make the record auditable rather than decorative.
|
||||
- contains: "must sum to exactly 100"
|
||||
- contains: "retrospective audit, not a planning target"
|
||||
- contains: "S may legitimately be 0%"
|
||||
- contains: '"Dominant factors" must name concrete session facts'
|
||||
# The exact output shape the hall files.
|
||||
- contains: "PQRST-Estimate"
|
||||
- contains: "Signature: P<int> Q<int> R<int> S<int> T<int>"
|
||||
- contains: "Confidence: <low|medium|high>"
|
||||
# Guard against the paraphrase this package used to be.
|
||||
- not_contains: "Include rationale:"
|
||||
- not_contains: "Percentages must sum to 100%."
|
||||
|
||||
output:
|
||||
criteria:
|
||||
- The five values are integers in 0..100 summing to exactly 100.
|
||||
- Signature agrees with the individual values.
|
||||
- Dominant factors names concrete session facts rather than restating percentages.
|
||||
- S is 0 when no security-specific work occurred, and is not inflated.
|
||||
- No sixth dimension is introduced inside the 5-tuple.
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
schema: canned-prompts/eval-rubric/v0.1
|
||||
name: pqrst-estimate-quality
|
||||
description: >
|
||||
The prompt asks for a post-session audit with a hard 100% constraint, and the
|
||||
resulting estimate respects both.
|
||||
example: examples/basic.yaml
|
||||
|
||||
render:
|
||||
- contains: "must sum to exactly 100%"
|
||||
- contains: "post-session audit, not a planning estimate"
|
||||
- not_contains: "{{"
|
||||
- resolves_all: true
|
||||
|
||||
output:
|
||||
criteria:
|
||||
- The five percentages sum to exactly 100%.
|
||||
- Categories are not silently renamed, merged, or dropped.
|
||||
- Effort is attributed from evidence in the session summary, not invented.
|
||||
- The estimate reads as an audit of work done, not a plan for work ahead.
|
||||
|
|
@ -1,8 +1,6 @@
|
|||
name: feature implementation with unfamiliar codebase
|
||||
name: canonical, no add-ons
|
||||
description: >
|
||||
The default render. Produces the canonical prompt verbatim, which is what
|
||||
hall-of-helix CLOSING.md requires when it says to paste the block unmodified.
|
||||
values:
|
||||
session_summary: |
|
||||
The session traced an unfamiliar request path through the repository,
|
||||
implemented a new validation rule, added unit and integration tests, fixed
|
||||
two edge cases discovered during testing, and updated the implementation
|
||||
after finding a conflicting assumption in an internal helper.
|
||||
include_rationale: true
|
||||
add_ons: ''
|
||||
|
|
|
|||
7
examples/pqrst-estimate/examples/with-rationale.yaml
Normal file
7
examples/pqrst-estimate/examples/with-rationale.yaml
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
name: with the per-dimension rationale add-on
|
||||
description: >
|
||||
The sanctioned add-on from PqrstPrompt.md, used when the estimate is read
|
||||
by a person rather than filed.
|
||||
values:
|
||||
add_ons: |
|
||||
After the block, also give a one-line evidence-based rationale for each of the five dimensions, plus one sentence characterizing the overall session profile (implementation-heavy, quality-heavy, exploration-heavy, security-heavy, coordination-heavy, or mixed).
|
||||
|
|
@ -1,49 +1,41 @@
|
|||
{{ house_style }}
|
||||
Please produce a retrospective PQRST-Estimate for this completed coding session.
|
||||
|
||||
Review the coding session described below and produce a **PQRST Estimate** of
|
||||
where effort was spent.
|
||||
Allocate exactly 100% of the session's effective engineering effort across these five dimensions and no others:
|
||||
|
||||
Use these categories:
|
||||
P — Main Problem: direct work producing the primary requested outcome — implementing the feature, fix, algorithm, integration, or other core deliverable.
|
||||
|
||||
- **P — Main problem:** implementing or directly solving the requested deliverable.
|
||||
- **Q — Quality and tests:** tests, verification, edge cases, maintainability,
|
||||
error handling, cleanup, and production-quality hardening.
|
||||
- **R — Research and context clarification:** reading the codebase or docs,
|
||||
tracing behavior, investigating unknowns, reconciling requirements, and
|
||||
establishing missing context.
|
||||
- **S — Security and credentials:** authentication, authorization, secrets,
|
||||
credentials, trust boundaries, security validation, and security-specific
|
||||
handling.
|
||||
- **T — Task organization:** planning, decomposition, todo management,
|
||||
sequencing, coordination, and overhead required to keep the work organized.
|
||||
Q — Quality and Tests: work establishing correctness, robustness, maintainability, and confidence — tests, verification, edge cases, debugging for correctness, validation, error handling, and quality-oriented refactoring.
|
||||
|
||||
Treat this as a **post-session audit, not a planning estimate**. Estimate
|
||||
relative cognitive/work effort rather than tokens or wall-clock time. The five
|
||||
percentages **must sum to exactly 100%**.
|
||||
R — Research and Context Clarification: work required to understand the task, repository, architecture, dependencies, existing behavior, documentation, specifications, or ambiguous requirements.
|
||||
|
||||
Where activities overlap, assign effort according to the primary purpose of the
|
||||
activity. Do not inflate a category merely because it was important; estimate
|
||||
how much effort it actually consumed.
|
||||
S — Security and Credentials: security-specific work involving authentication, authorization, permissions, credentials, secrets, tenant isolation, trust boundaries, sensitive data, threat considerations, or security controls.
|
||||
|
||||
Session material:
|
||||
T — Task Organization: planning, decomposition, sequencing, todo management, coordination, scope management, and other effort spent keeping the work organized rather than directly implementing or validating it.
|
||||
|
||||
{{ session_summary }}
|
||||
Rules:
|
||||
|
||||
Return:
|
||||
1. Treat this strictly as a retrospective audit, not a planning target or productivity score.
|
||||
2. The five values must be integers in 0..100 and must sum to exactly 100.
|
||||
3. Estimate relative effective engineering attention — not tokens, tool calls, wall-clock time, compute, or hidden model activity.
|
||||
4. Where categories overlap, classify effort by the primary purpose of the activity at the time, and do not double-count it.
|
||||
5. Ground each non-zero category in concrete evidence from what actually happened in this session: files or modules inspected or changed, tests written or run, documentation consulted, security or credential work handled, planning or coordination performed.
|
||||
6. Do not force every category to be non-zero. S may legitimately be 0% and must not be inflated merely because security is generally important.
|
||||
7. Prefer honest coarse estimates, normally in roughly 5-percentage-point increments, over false precision. If you cannot defend a split, round to tens and lower the confidence.
|
||||
8. Report uncertainty only in the Confidence field. Never fold it into P, Q, R, S, or T.
|
||||
9. "Dominant factors" must name concrete session facts, not restate the percentages.
|
||||
10. If the session record is insufficient to estimate a category confidently, say so rather than inventing evidence.
|
||||
|
||||
```text
|
||||
P: NN%
|
||||
Q: NN%
|
||||
R: NN%
|
||||
S: NN%
|
||||
T: NN%
|
||||
Total: 100%
|
||||
```
|
||||
Return exactly this format, and nothing before or after it:
|
||||
|
||||
Then provide:
|
||||
|
||||
1. **Primary effort driver** — one sentence naming what dominated the session.
|
||||
2. **Interpretation** — what the distribution says about the session's shape.
|
||||
3. **Signal** — one notable imbalance, if any, that may be worth learning from.
|
||||
|
||||
Include rationale: {{ include_rationale }}
|
||||
PQRST-Estimate
|
||||
P: <int>%
|
||||
Q: <int>%
|
||||
R: <int>%
|
||||
S: <int>%
|
||||
T: <int>%
|
||||
Sum: 100%
|
||||
Confidence: <low|medium|high>
|
||||
Signature: P<int> Q<int> R<int> S<int> T<int>
|
||||
Dominant factors: <one or two sentences naming the concrete drivers of the largest slices>
|
||||
Notes: <optional; omit this line entirely if there is nothing to add>
|
||||
{{ add_ons }}
|
||||
|
|
@ -1,54 +1,49 @@
|
|||
format: canned-prompt/v0.2
|
||||
id: practice/pqrst-estimate
|
||||
name: PQRST Estimate
|
||||
version: 0.2.1
|
||||
summary: 'Produce a post-session estimate of effort distributed across the PQRST categories
|
||||
for an agentic coding session.
|
||||
|
||||
'
|
||||
version: 1.0.0
|
||||
summary: >
|
||||
The canonical end-of-session PQRST effort audit: allocate 100% of a completed
|
||||
session's engineering effort across Problem, Quality, Research, Security and
|
||||
Task organization.
|
||||
type: template
|
||||
template: prompt.md
|
||||
dependencies:
|
||||
prompts:
|
||||
- id: practice/house-style
|
||||
version: '>= 0.1.0'
|
||||
requirement: required
|
||||
|
||||
inputs:
|
||||
- name: house_style
|
||||
- name: add_ons
|
||||
type: content
|
||||
required: false
|
||||
description: 'Shared house-style block, composed from practice/house-style.
|
||||
description: >
|
||||
Optional text appended after the canonical block. Empty by default, so the
|
||||
default render is the canonical prompt verbatim. The two sanctioned add-ons
|
||||
are in examples/; see README.md.
|
||||
default: ''
|
||||
|
||||
'
|
||||
default:
|
||||
include: practice/house-style
|
||||
- name: session_summary
|
||||
type: content
|
||||
required: true
|
||||
description: 'Session transcript, summary, or sufficiently detailed account of the
|
||||
work performed during the coding session.
|
||||
|
||||
'
|
||||
parameters:
|
||||
include_rationale:
|
||||
type: boolean
|
||||
default: true
|
||||
description: Explain the evidence behind the estimate.
|
||||
output:
|
||||
format: markdown
|
||||
description: A 100% PQRST effort allocation with concise interpretation.
|
||||
format: text
|
||||
description: >
|
||||
A PQRST-Estimate record — five integer percentages summing to 100, plus
|
||||
Sum, Confidence, Signature, Dominant factors and an optional Notes line.
|
||||
|
||||
compatibility:
|
||||
capabilities:
|
||||
- long-context
|
||||
|
||||
license: MIT
|
||||
|
||||
examples:
|
||||
- examples/basic.yaml
|
||||
- examples/with-rationale.yaml
|
||||
evals:
|
||||
- evals/canonical-fidelity.yaml
|
||||
|
||||
tags:
|
||||
- pqrst
|
||||
- retrospective
|
||||
- agentic-coding
|
||||
- effort-estimation
|
||||
license: MIT
|
||||
|
||||
provenance:
|
||||
author: canned-prompts seed
|
||||
examples:
|
||||
- examples/basic.yaml
|
||||
evals:
|
||||
- evals/quality.yaml
|
||||
author: pqrst-practice
|
||||
source: ~/pqrst-practice/PqrstPrompt.md
|
||||
normative_spec: ~/pqrst-practice/spec/PqrstEstimationPractice.md
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue