Complete PQRST-WP-0001: bootstrap State Hub integration
T01 — Reviewed the generated integration files. Resolved the
{CREDENTIAL_ROUTING} placeholder in AGENTS.md from the fleet canon, refreshed
the SCOPE.md repository layout (stale after registration), and replaced the
README stub with orientation and entry points.
T02 — Documented the developer workflow in AGENTS.md. This is a Markdown-only
spec repository: nothing to install, build, or run, and no test suite. In place
of a toolchain, added a six-point verification checklist covering prompt/spec
coupling, the closed 5-tuple, example-record arithmetic, cross-references,
version discipline, and the append-only rule for history/. The signature
validation command is verified working — all nine example signatures sum to 100.
T03 — Seeded PQRST-WP-0002 "Validate spec v0.1 against real sessions and cut
v0.2": pilot the closing prompt unassisted on at least eight real sessions,
decide the record-storage boundary SCOPE.md leaves open, then revise. v0.1 was
consolidated from two drafts neither of which had been applied to a session, so
the open questions are empirical rather than editorial.
WP-0001 marked finished with DoR/DoD assessments (clears C-34). No open
residuals — the follow-on work is carried by PQRST-WP-0002.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ6oF1DtVDKcD1FCpvRVLx
Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 272883@bnt-lap001
Assistant-Session: f40c8f53-fb65-4980-9d29-bcdb3dd946f7
This commit is contained in:
parent
14d53264fa
commit
2b36fc187d
5 changed files with 333 additions and 7 deletions
125
workplans/PQRST-WP-0002-validate-v01-against-real-sessions.md
Normal file
125
workplans/PQRST-WP-0002-validate-v01-against-real-sessions.md
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
---
|
||||
id: PQRST-WP-0002
|
||||
type: workplan
|
||||
title: "Validate spec v0.1 against real sessions and cut v0.2"
|
||||
domain: agents
|
||||
repo: pqrst-practice
|
||||
status: proposed
|
||||
owner: claude-code
|
||||
topic_slug: practice
|
||||
priority: high
|
||||
created: "2026-09-05"
|
||||
updated: "2026-09-05"
|
||||
quality_dor: DoR-Ok
|
||||
quality_dor_at: "2026-09-05"
|
||||
quality_dor_by: claude-code
|
||||
quality_dor_note: "v0.1 consolidates two drafts that were written without either being applied to a session. Every open question is empirical — whether the prompt returns a valid record unassisted, whether attribution by primary purpose is decidable in practice, and where records are stored — so the next change is a bounded pilot, not more specification."
|
||||
related:
|
||||
- PQRST-WP-0001
|
||||
---
|
||||
|
||||
# Validate spec v0.1 against real sessions and cut v0.2
|
||||
|
||||
Spec v0.1 was consolidated from two independent drafts, neither of which had
|
||||
been applied to an actual session. It is internally consistent and entirely
|
||||
unvalidated.
|
||||
|
||||
The purpose of this workplan is to find out where the practice is wrong by using
|
||||
it, then cut v0.2 from the findings. It deliberately does **not** build tooling
|
||||
or a record store — both are out of scope for this repository (`SCOPE.md`).
|
||||
|
||||
**Done when:** the prompt has been run unassisted at the end of at least eight
|
||||
real sessions, the friction is written down, the storage boundary is decided,
|
||||
and v0.2 either incorporates the findings or records why v0.1 stands.
|
||||
|
||||
## Pilot the closing prompt on real sessions
|
||||
|
||||
```task
|
||||
id: PQRST-WP-0002-T01
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Paste `PqrstPrompt.md` unmodified at the end of at least **eight** real agentic
|
||||
coding sessions, spread across at least three repositories and at least two task
|
||||
classes (e.g. `feature`, `bugfix`, `explore`, `harden`).
|
||||
|
||||
Do not coach the model, do not repair a malformed answer, and do not re-run the
|
||||
prompt for a better result. A rejected record is the finding.
|
||||
|
||||
For each session capture, in this file under "Pilot findings":
|
||||
|
||||
- the returned record verbatim;
|
||||
- whether it passed spec §5.5 validation on the first attempt;
|
||||
- whether `Dominant factors` cited concrete session facts or restated the
|
||||
percentages;
|
||||
- any dimension the model hedged on, refused, or visibly guessed at;
|
||||
- the task class and the repository.
|
||||
|
||||
The specific questions to answer:
|
||||
|
||||
1. Does an uncoached agent return a **valid** record from the prompt alone?
|
||||
2. Is "attribute by primary purpose" (spec R5) actually decidable mid-estimate,
|
||||
or does it collapse under overlap?
|
||||
3. Does S stay at 0 when security work did not occur, or does it drift upward?
|
||||
4. Do estimates cluster at coarse increments (R7), or does false precision
|
||||
reappear?
|
||||
5. Is `Confidence` used meaningfully, or is it always `medium`?
|
||||
|
||||
## Decide the record-storage boundary
|
||||
|
||||
```task
|
||||
id: PQRST-WP-0002-T02
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
`SCOPE.md` excludes record storage from this repository but does not say where
|
||||
records go instead. Until that is answered the practice produces records with
|
||||
nowhere to live, and the trend analysis that justifies the whole exercise
|
||||
(spec §8, §9) cannot happen.
|
||||
|
||||
Decide and record the answer as a decision (`POST /decisions/`, then an ADR in
|
||||
this repo if the decision has repo-local consequences). Options to weigh:
|
||||
|
||||
- a State Hub surface (progress events already carry free text; a dedicated
|
||||
record type would need a hub change and is a bigger commitment);
|
||||
- an append-only file in each consuming repository;
|
||||
- a separate collection repository outside this one.
|
||||
|
||||
Whatever is chosen, this repository still ships no collector — the decision
|
||||
names the owner, it does not move the work here.
|
||||
|
||||
## Cut spec v0.2
|
||||
|
||||
```task
|
||||
id: PQRST-WP-0002-T03
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
From the pilot findings, revise the specification and prompt together:
|
||||
|
||||
- fix rules the pilot showed to be undecidable or routinely ignored;
|
||||
- tighten or relax the validation rules based on what actually failed;
|
||||
- add the storage-boundary decision from T02 as a `SCOPE.md` cross-reference;
|
||||
- bump to **v0.2** and add a row to the spec's Appendix B.
|
||||
|
||||
If a change alters a dimension's meaning, the validation rules, or the stored
|
||||
record format, say plainly in Appendix B that records made under v0.1 are less
|
||||
comparable.
|
||||
|
||||
If the pilot surfaces nothing substantive, record that outcome and leave v0.1
|
||||
standing. "No change needed, and here is the evidence" is a valid result and a
|
||||
better one than churn.
|
||||
|
||||
Run the verification checklist in `AGENTS.md` before committing, then sync:
|
||||
|
||||
```bash
|
||||
uv run --project ~/repo-manager rmgr sync --path . --push
|
||||
```
|
||||
|
||||
## Pilot findings
|
||||
|
||||
_Populated by T01. One subsection per session: repository, task class, the
|
||||
verbatim record, and whether it validated unassisted._
|
||||
Loading…
Add table
Add a link
Reference in a new issue