Enable documented monthly activity-core PQRST data review
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0e759-301a-78b1-bbc1-040ef094b12d
This commit is contained in:
parent
65fc8c77e3
commit
25d2ce38c4
4 changed files with 187 additions and 0 deletions
|
|
@ -54,3 +54,6 @@ see [the storage decision](docs/adr/0002-record-storage-boundary.md).
|
||||||
|
|
||||||
The [2026-09-28 hall review](docs/reviews/2026-09-28-hall-pqrst-review.md)
|
The [2026-09-28 hall review](docs/reviews/2026-09-28-hall-pqrst-review.md)
|
||||||
documents the extraction command, corpus findings, and why v0.1 stands.
|
documents the extraction command, corpus findings, and why v0.1 stands.
|
||||||
|
|
||||||
|
The [monthly activity-core review](docs/monthly-pqrst-automation.md) reports counts,
|
||||||
|
monthly averages, and trends at 09:30 Europe/Berlin on the first of each month.
|
||||||
|
|
|
||||||
6
SCOPE.md
6
SCOPE.md
|
|
@ -57,6 +57,12 @@ store or an empirical accuracy study.
|
||||||
|
|
||||||
PQRST is deliberately agent-agnostic and model-agnostic. Wiring it into a specific harness, plugin, skill, or CI pipeline belongs to that harness's own repository.
|
PQRST is deliberately agent-agnostic and model-agnostic. Wiring it into a specific harness, plugin, skill, or CI pipeline belongs to that harness's own repository.
|
||||||
|
|
||||||
|
The explicitly requested monthly review has a domain-owned scheduling declaration
|
||||||
|
in `activity-definitions/monthly-pqrst-review.md` and an
|
||||||
|
[operation guide](docs/monthly-pqrst-automation.md). Its collection, calculation,
|
||||||
|
Temporal scheduling and report sinks are implemented and operated in activity-core.
|
||||||
|
Reports remain in the configured external sinks; no runtime or toolchain is added here.
|
||||||
|
|
||||||
### Scoring, benchmarking, and evaluation
|
### Scoring, benchmarking, and evaluation
|
||||||
|
|
||||||
No thresholds, no healthy-profile templates, no grading, no leaderboards, no comparison of agents or models on PQRST values. This is excluded on principle, not for lack of time — see [`INTENT.md`](INTENT.md).
|
No thresholds, no healthy-profile templates, no grading, no leaderboards, no comparison of agents or models on PQRST values. This is excluded on principle, not for lack of time — see [`INTENT.md`](INTENT.md).
|
||||||
|
|
|
||||||
53
activity-definitions/monthly-pqrst-review.md
Normal file
53
activity-definitions/monthly-pqrst-review.md
Normal file
|
|
@ -0,0 +1,53 @@
|
||||||
|
---
|
||||||
|
id: monthly-pqrst-review
|
||||||
|
name: Monthly PQRST Data Review
|
||||||
|
enabled: true
|
||||||
|
owner: pqrst-practice
|
||||||
|
governance: custodian
|
||||||
|
status: active
|
||||||
|
trigger:
|
||||||
|
type: cron
|
||||||
|
cron_expression: "30 9 1 * *"
|
||||||
|
timezone: Europe/Berlin
|
||||||
|
misfire_policy: catchup_latest
|
||||||
|
context_sources:
|
||||||
|
- type: pqrst
|
||||||
|
query: monthly
|
||||||
|
params: {required: true}
|
||||||
|
bind_to: context.daily_triage_digest
|
||||||
|
---
|
||||||
|
|
||||||
|
# Monthly PQRST Data Review
|
||||||
|
|
||||||
|
At 09:30 Europe/Berlin on the first of each month, review the six completed
|
||||||
|
calendar months ending with the previous month. Read hall-of-helix's committed
|
||||||
|
main branch, pinned to one revision per run. Report one datapoint per structurally
|
||||||
|
valid agent entry, including drafts; select the unique block matching frontmatter
|
||||||
|
and retain attempt counts. Missing, invalid and ambiguous records do not enter
|
||||||
|
means. All five dimensions have equal-entry means and month-over-month changes
|
||||||
|
in percentage points; empty months and comparisons across a gap use null.
|
||||||
|
|
||||||
|
The report preserves source revision, counts, confidence frequencies and data
|
||||||
|
limitations. No worker/model rankings, target allocations, empirical accuracy or
|
||||||
|
causal claims. It creates reports, not tasks, and uses no LLM. See
|
||||||
|
`docs/monthly-pqrst-automation.md` for operation and rollback.
|
||||||
|
|
||||||
|
```instruction
|
||||||
|
id: monthly-pqrst-review-report
|
||||||
|
trusted_fields: []
|
||||||
|
model: deterministic
|
||||||
|
temperature: 0
|
||||||
|
max_tokens: 1
|
||||||
|
prompt: Deterministic monthly PQRST counts, averages and trends from published hall entries.
|
||||||
|
output_schema: ""
|
||||||
|
review_advisory: false
|
||||||
|
report_sinks:
|
||||||
|
- type: working-memory
|
||||||
|
path: /var/custodian/memory/working
|
||||||
|
filename_template: "pqrst-monthly-{date}-{run_id_short}.md"
|
||||||
|
timezone: Europe/Berlin
|
||||||
|
- type: state-hub-progress
|
||||||
|
event_type: pqrst_monthly_review
|
||||||
|
author: activity-core
|
||||||
|
topic_id: c1d199b6-55ee-4db6-b49e-257a9f0f15ac
|
||||||
|
```
|
||||||
125
docs/monthly-pqrst-automation.md
Normal file
125
docs/monthly-pqrst-automation.md
Normal file
|
|
@ -0,0 +1,125 @@
|
||||||
|
# Monthly PQRST review automation
|
||||||
|
|
||||||
|
## Schedule and ownership
|
||||||
|
|
||||||
|
`monthly-pqrst-review` runs through activity-core / Temporal at **09:30
|
||||||
|
Europe/Berlin on the first day of each month** (`30 9 1 * *`). It reviews the
|
||||||
|
previous calendar month and five preceding months. The first regular report
|
||||||
|
is due **2026-10-01 09:30 CEST (07:30 UTC)**. `catchup_latest` uses activity-core's
|
||||||
|
24-hour default recovery window and buffers at most one missed fire.
|
||||||
|
|
||||||
|
The [definition](../activity-definitions/monthly-pqrst-review.md) is owned here.
|
||||||
|
Activity-core owns `context_resolvers/pqrst.py`, tests, the worker runtime,
|
||||||
|
and the GitOps projection in `k8s/railiance/20-runtime.yaml`. Hall entries remain
|
||||||
|
the primary data store. This domain scheduling declaration is the explicitly
|
||||||
|
requested integration; no collector or validator implementation lives here.
|
||||||
|
|
||||||
|
Authority: the user's 2026-09-28 setup request; `ADMINISTER @
|
||||||
|
realm:kubernetes/railiance01`, `activation=APPROVED`, through the existing ArgoCD
|
||||||
|
application. Decision `e2c54d2f-074e-495c-841a-4b398006d85c`. This authorizes this
|
||||||
|
setup, not standing unattended release authority or automatic specification edits.
|
||||||
|
|
||||||
|
## Report contract
|
||||||
|
|
||||||
|
- Pin public `hall-of-helix/main` to a commit, enumerate its complete Git tree,
|
||||||
|
and read Markdown entries with Git blob-hash verification. Do not execute
|
||||||
|
entry text or load arbitrary source URLs. Collection has time, byte and count
|
||||||
|
bounds; an incomplete tree or failed file fetch produces an unavailable report,
|
||||||
|
never partial averages presented as a complete sample.
|
||||||
|
- Use `recorded_at` in Europe/Berlin to group entries by month. This is a
|
||||||
|
**recording-month** view, not reconstruction of when engineering effort occurred.
|
||||||
|
Exclude the open current month. Missing/invalid dates are reported as unclassifiable.
|
||||||
|
- One datapoint is one agent entry, including drafts, with a unique entry id and
|
||||||
|
a unique structurally valid block matching its frontmatter signature. Multiple
|
||||||
|
attempts remain an attempt count, not multiple datapoints. Duplicate entry ids,
|
||||||
|
ambiguous selections and invalid blocks are excluded. Distinct entries are not
|
||||||
|
claimed to be independently verified unique sessions.
|
||||||
|
- Structural checks cover the five integer values, 0..100 bounds, total 100,
|
||||||
|
confidence, signature agreement and nonempty Dominant factors. They cannot
|
||||||
|
establish whether narrative evidence is truthful, concrete or sufficient.
|
||||||
|
- Each month reports included count `n`, agent-entry count, missing/invalid
|
||||||
|
count, attempts, included draft count and confidence frequencies. Means weight
|
||||||
|
entries equally; no time/token weighting or averaging of already-rounded means.
|
||||||
|
- `mean_pp` and `change_pp` use the explicit `order: [P,Q,R,S,T]`. Changes are
|
||||||
|
differences from the immediately preceding calendar month's means, in
|
||||||
|
**percentage points**. Empty months have null means, and comparisons across an
|
||||||
|
empty month have null changes. Values round only for presentation to two decimals.
|
||||||
|
- Counts and task mix matter when interpreting movement. This voluntary sample
|
||||||
|
is not a census, first-attempt success measure, causal study or worker ranking.
|
||||||
|
No LLM calls, tasks, alerts to people, or automatic changes are produced.
|
||||||
|
|
||||||
|
## Finding a report
|
||||||
|
|
||||||
|
ActivityDefinition UUID: `18c2f495-74b8-59e6-b0c9-f6df68cd1cf5`.
|
||||||
|
Temporal schedule: `activity-schedule-18c2f495-74b8-59e6-b0c9-f6df68cd1cf5`.
|
||||||
|
|
||||||
|
Both configured sinks are durable and use activity-core's run idempotency:
|
||||||
|
|
||||||
|
1. State Hub progress event `pqrst_monthly_review`, topic
|
||||||
|
`c1d199b6-55ee-4db6-b49e-257a9f0f15ac`, author `activity-core`.
|
||||||
|
2. Railiance working memory:
|
||||||
|
`/home/tegwick/the-custodian/memory/working/pqrst-monthly-<date>-<run-id-prefix>.md`.
|
||||||
|
|
||||||
|
Inspect progress via the existing State Hub API:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS 'http://127.0.0.1:8000/progress/?event_type=pqrst_monthly_review&limit=12'
|
||||||
|
```
|
||||||
|
|
||||||
|
The current generic deterministic sink nests the bounded JSON document at
|
||||||
|
`detail.report.digest_preview`; parse that string as JSON. Its `months` array
|
||||||
|
contains the series and `status` distinguishes complete collection, entries
|
||||||
|
needing review, and unavailable source. The outer `output_validated` flag does
|
||||||
|
not certify the underlying narratives or source availability. Working-memory
|
||||||
|
notes use the same payload; their generic sink heading is inherited from WSJF.
|
||||||
|
|
||||||
|
## Verification and operation
|
||||||
|
|
||||||
|
The resolver has focused tests for monthly boundaries, year rollover, means and
|
||||||
|
deltas, no-data months, retries, duplicate ids/fields, invalid records, missing
|
||||||
|
metadata and source failure. The projected definition has a contract test and
|
||||||
|
is included in activity-core's image CI test stage. Related frontend, instruction
|
||||||
|
and GitOps checks passed. Runtime resolver bytes were matched to the tested source.
|
||||||
|
|
||||||
|
The September 28 manual smoke delivered run
|
||||||
|
`c7d826af-83e9-528e-9ccd-8b4f5b5e24f1`, progress
|
||||||
|
`622b7cda-ba7b-465b-aa63-90d60e1a5b63`, and working-memory note
|
||||||
|
`pqrst-monthly-2026-09-28-c7d826af.md`. It covers August (0 usable records), not
|
||||||
|
September; it proves collection and sink delivery, not a natural monthly fire.
|
||||||
|
|
||||||
|
Changes to the definition must be projected into activity-core, rendered with
|
||||||
|
`scripts/render_gitops.py`, reviewed with server dry-run and client diff, committed,
|
||||||
|
and synced via ArgoCD. Then call the existing activity-core
|
||||||
|
`POST /admin/sync?definitions=true&schedules=true` endpoint from the admitted
|
||||||
|
runtime and inspect definition/schedule state. Do not add a second cron.
|
||||||
|
|
||||||
|
To pause, set the domain definition and projection `enabled: false`, GitOps sync,
|
||||||
|
then admin-sync schedules. To roll back this installation, restore the previous
|
||||||
|
worker digest `sha256:afd8a7af73132becba432e7b315d70f88d66676020169d5d8811641304136437`
|
||||||
|
and remove/disable only this definition through GitOps and admin-sync. The previous
|
||||||
|
Argo application revision is `4bd07a4f44f38bb4f27cfb381cfe53d555c2c6fe`;
|
||||||
|
prefer a scoped revert if later unrelated deployments have landed.
|
||||||
|
|
||||||
|
## Activation receipt — 2026-09-28
|
||||||
|
|
||||||
|
- Definition enabled; Temporal `paused=false`, 24-hour catchup window.
|
||||||
|
- Next fires: October 1 07:30 UTC, November 1 08:30 UTC, December 1 08:30 UTC
|
||||||
|
(09:30 Europe/Berlin throughout).
|
||||||
|
- ArgoCD revision `ce9a8f8edf4ec162bae3aa087803bd57a1b33a66`: Synced and Healthy.
|
||||||
|
- Worker image `sha256:75e240ac23c1c6d7daae14691c2b4137d2d639a9af694aaf0d4c774a7e4d3886`;
|
||||||
|
resolver SHA-256 `41f91a68c75c758204e16319199d0a28060154630c45d74544d4470fe2a7a8ce`
|
||||||
|
matches source. Existing frontend resolver and repository-grant support remain present.
|
||||||
|
- Final admin sync: 34 definitions, 16 schedules upserted, 12 paused, zero errors
|
||||||
|
and zero orphan deletions. The monthly PQRST definition is enabled.
|
||||||
|
- Activity-core commits `eda44d9` / `ce9a8f8` and platform application pins
|
||||||
|
`a13fa57` / `7a12760` were pushed. Activity-core Repo Manager reconciliation
|
||||||
|
refused pre-existing `custodian-WP-*` legacy identifiers before projection;
|
||||||
|
this does not affect deployment or scheduling. No historic IDs were changed.
|
||||||
|
- No new tasks or workplans, credential changes, or unrelated entry edits.
|
||||||
|
|
||||||
|
The enabled definition also passed a **one-shot Temporal Schedule** smoke at
|
||||||
|
2026-09-28 10:21:38 UTC: run `27c73e81-342e-5a84-b582-cf7d874fb17d`, progress
|
||||||
|
`e50d5e66-4abd-4ddd-87b3-d6556550884e`, working-memory `memory/working/pqrst-monthly-2026-09-28-27c73e81.md`.
|
||||||
|
Workflow completed with zero tasks; collection status was `ok` and both sinks
|
||||||
|
were verified. The temporary smoke schedule was removed; the recurring schedule
|
||||||
|
remains enabled. This is scheduled-path proof, not the October natural run.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue