Review hall PQRST corpus and close published-record pilot
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0e759-301a-78b1-bbc1-040ef094b12d
This commit is contained in:
parent
37b49a9517
commit
65fc8c77e3
7 changed files with 413 additions and 18 deletions
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Validate spec v0.1 in real session closes and land PQRST in hall-of-helix"
|
||||
domain: agents
|
||||
repo: pqrst-practice
|
||||
status: blocked
|
||||
status: finished
|
||||
flavor: planning
|
||||
owner: claude-code
|
||||
topic_slug: practice
|
||||
|
|
@ -24,8 +24,20 @@ repos:
|
|||
state_hub_workstream_id: "6a875b19-5a76-55c1-bd1f-2f5005cd416b"
|
||||
---
|
||||
|
||||
State Hub review decision: `2f67b409-c5df-4f8e-bce8-98e9c7be92ab`.
|
||||
|
||||
# Validate spec v0.1 in real session closes and land PQRST in hall-of-helix
|
||||
|
||||
**Completed 2026-09-28:** the operator selected hall entries as the main data
|
||||
source. [The pinned corpus review](../docs/reviews/2026-09-28-hall-pqrst-review.md)
|
||||
closes T01 under the revised retrospective-review scope; T03 retains v0.1 with
|
||||
editorial corrections grounded in those findings. All five tasks are done.
|
||||
No new tasks or workplans were opened, and no implementation residuals remain.
|
||||
Uncoached success rates, calibrated accuracy and self-estimation bias are
|
||||
explicitly unestablished; studying them is outside this repository's scope.
|
||||
|
||||
The original proposal and initial blocked review are retained below as history.
|
||||
|
||||
Spec v0.1 was consolidated from two independent drafts, neither of which had
|
||||
been applied to an actual session. It is internally consistent and entirely
|
||||
unvalidated.
|
||||
|
|
@ -52,23 +64,22 @@ It deliberately does **not** build tooling or a record store in *this*
|
|||
repository. Both are out of scope (`SCOPE.md`); the hall is the store, and the
|
||||
hall owns its own validation.
|
||||
|
||||
**Done when:** the closing routine is specified in hall-of-helix and reachable
|
||||
**Original done condition (superseded by the operator's corpus-review direction):** the closing routine is specified in hall-of-helix and reachable
|
||||
from the prompt above, entries carry a validated PQRST record, the prompt has
|
||||
been run unassisted at the end of at least eight real sessions, and v0.2 either
|
||||
incorporates the findings or records why v0.1 stands.
|
||||
|
||||
**Execution order:** T04/T05 are complete; T02 closed independently on
|
||||
2026-09-28 using the existing hall integration. T01 must supply the pilot
|
||||
evidence before T03 can settle the version decision. Task ids are identity,
|
||||
not sequence.
|
||||
**Completion order:** T04/T05 established the hall integration; T02 closed
|
||||
the storage boundary; T01 reviewed the published corpus; T03 applied the
|
||||
editorial findings and retained v0.1. Task ids are identity, not sequence.
|
||||
|
||||
**Cross-repo:** T04 and T05 changed `hall-of-helix`, not this repo. They were
|
||||
executed under `HOH-WP-0001` there on 2026-09-05 and are `done`; the routine
|
||||
lives in `hall-of-helix/CLOSING.md` and `make check` enforces the record on
|
||||
finished agent seats from 2026-09-06. What remains here is first-attempt pilot
|
||||
evidence (T01) and the resulting version decision (T03).
|
||||
finished agent seats from 2026-09-06. The read-only export script added for T01
|
||||
also lives in the hall. Entry content was not edited by this review.
|
||||
|
||||
**2026-09-28 review:** T02 is done. T01 and T03 are `wait`, so this workplan is
|
||||
**2026-09-28 initial review (superseded):** T02 is done. T01 and T03 are `wait`, so this workplan is
|
||||
`blocked`. Published hall records demonstrate adoption but do not establish
|
||||
eight unmodified, uncoached, unrepaired first attempts. Resume T01 when original
|
||||
closing outputs with that provenance are available across at least three repos
|
||||
|
|
@ -79,11 +90,24 @@ T03 waits for those findings. No new tasks or workplans were opened.
|
|||
|
||||
```task
|
||||
id: PQRST-WP-0002-T01
|
||||
status: wait
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "dd259b3f-e9ae-5bdd-8e02-07ae0083930a"
|
||||
```
|
||||
|
||||
**Completed under revised scope:** inspect the committed hall corpus, retain
|
||||
attempt history, structurally check records, and qualitatively review at least
|
||||
eight examples spanning three repositories and two task classes. The review
|
||||
contains eight verbatim examples (including a rejected first attempt), reviewer
|
||||
task classes, provenance limits, and answers to all six questions below. The
|
||||
exporter emits 196 entries, 95 with PQRST, containing 96 attempt blocks; 94 of
|
||||
95 frontmatter-selected blocks pass structural checks. Published validity is
|
||||
not unassisted first-attempt validity. This scope follows the operator's explicit
|
||||
instruction to use the hall as our main data source, not a claim that the
|
||||
original controlled protocol was fulfilled.
|
||||
|
||||
**Original protocol, retained for provenance:**
|
||||
|
||||
Paste `PqrstPrompt.md` unmodified at the end of at least **eight** real agentic
|
||||
coding sessions, spread across at least three repositories and at least two task
|
||||
classes (e.g. `feature`, `bugfix`, `explore`, `harden`). Once T04 has landed,
|
||||
|
|
@ -164,14 +188,18 @@ work here.
|
|||
|
||||
```task
|
||||
id: PQRST-WP-0002-T03
|
||||
status: wait
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "0a0bce15-ac0c-582a-842c-ae1bf0307afb"
|
||||
```
|
||||
|
||||
**2026-09-28:** blocked on T01's first-attempt evidence. The storage cross-reference
|
||||
is already landed through T02. Leave v0.1 unchanged pending the pilot; this is
|
||||
not a finding that no substantive revision is needed.
|
||||
**Completed 2026-09-28:** retain v0.1 after the published-corpus review. Fix the
|
||||
prompt checklist's demonstrated retry contradiction, clarify substantive-work
|
||||
scope and optional uncertainty/provenance Notes, and align the spec's ranking
|
||||
prohibition with INTENT. No dimension, format or validation rule changes; the
|
||||
fenced canonical prompt is unchanged. Appendix B records the editorial update.
|
||||
T02 already supplied the storage cross-reference. This disposition is supported
|
||||
by published-record evidence, not a controlled accuracy or first-attempt study.
|
||||
|
||||
From the pilot findings and the hall integration, revise the specification and
|
||||
prompt together:
|
||||
|
|
@ -306,8 +334,10 @@ exception in `CLOSING.md`.
|
|||
|
||||
## Pilot findings
|
||||
|
||||
_Populated by T01. One subsection per session: repository, task class, the
|
||||
verbatim record, and whether it validated unassisted._
|
||||
The completed [corpus review](../docs/reviews/2026-09-28-hall-pqrst-review.md)
|
||||
contains the method, findings, and eight verbatim source examples with validation
|
||||
and provenance dispositions. The initial evidence scan below is historical;
|
||||
the later corpus review supersedes its blocked-task conclusion.
|
||||
|
||||
### Evidence availability review — 2026-09-28
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue