From 25d2ce38c4ccd12d9cacaaa69769bdfe4bcc4639 Mon Sep 17 00:00:00 2001 From: tegwick Date: Mon, 28 Sep 2026 12:23:27 +0200 Subject: [PATCH] Enable documented monthly activity-core PQRST data review Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0e759-301a-78b1-bbc1-040ef094b12d --- README.md | 3 + SCOPE.md | 6 + activity-definitions/monthly-pqrst-review.md | 53 ++++++++ docs/monthly-pqrst-automation.md | 125 +++++++++++++++++++ 4 files changed, 187 insertions(+) create mode 100644 activity-definitions/monthly-pqrst-review.md create mode 100644 docs/monthly-pqrst-automation.md diff --git a/README.md b/README.md index 259f81e..f9f37f5 100644 --- a/README.md +++ b/README.md @@ -54,3 +54,6 @@ see [the storage decision](docs/adr/0002-record-storage-boundary.md). The [2026-09-28 hall review](docs/reviews/2026-09-28-hall-pqrst-review.md) documents the extraction command, corpus findings, and why v0.1 stands. + +The [monthly activity-core review](docs/monthly-pqrst-automation.md) reports counts, +monthly averages, and trends at 09:30 Europe/Berlin on the first of each month. diff --git a/SCOPE.md b/SCOPE.md index d87f9ec..458c581 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -57,6 +57,12 @@ store or an empirical accuracy study. PQRST is deliberately agent-agnostic and model-agnostic. Wiring it into a specific harness, plugin, skill, or CI pipeline belongs to that harness's own repository. +The explicitly requested monthly review has a domain-owned scheduling declaration +in `activity-definitions/monthly-pqrst-review.md` and an +[operation guide](docs/monthly-pqrst-automation.md). Its collection, calculation, +Temporal scheduling and report sinks are implemented and operated in activity-core. +Reports remain in the configured external sinks; no runtime or toolchain is added here. + ### Scoring, benchmarking, and evaluation No thresholds, no healthy-profile templates, no grading, no leaderboards, no comparison of agents or models on PQRST values. This is excluded on principle, not for lack of time — see [`INTENT.md`](INTENT.md). diff --git a/activity-definitions/monthly-pqrst-review.md b/activity-definitions/monthly-pqrst-review.md new file mode 100644 index 0000000..6e10255 --- /dev/null +++ b/activity-definitions/monthly-pqrst-review.md @@ -0,0 +1,53 @@ +--- +id: monthly-pqrst-review +name: Monthly PQRST Data Review +enabled: true +owner: pqrst-practice +governance: custodian +status: active +trigger: + type: cron + cron_expression: "30 9 1 * *" + timezone: Europe/Berlin + misfire_policy: catchup_latest +context_sources: + - type: pqrst + query: monthly + params: {required: true} + bind_to: context.daily_triage_digest +--- + +# Monthly PQRST Data Review + +At 09:30 Europe/Berlin on the first of each month, review the six completed +calendar months ending with the previous month. Read hall-of-helix's committed +main branch, pinned to one revision per run. Report one datapoint per structurally +valid agent entry, including drafts; select the unique block matching frontmatter +and retain attempt counts. Missing, invalid and ambiguous records do not enter +means. All five dimensions have equal-entry means and month-over-month changes +in percentage points; empty months and comparisons across a gap use null. + +The report preserves source revision, counts, confidence frequencies and data +limitations. No worker/model rankings, target allocations, empirical accuracy or +causal claims. It creates reports, not tasks, and uses no LLM. See +`docs/monthly-pqrst-automation.md` for operation and rollback. + +```instruction +id: monthly-pqrst-review-report +trusted_fields: [] +model: deterministic +temperature: 0 +max_tokens: 1 +prompt: Deterministic monthly PQRST counts, averages and trends from published hall entries. +output_schema: "" +review_advisory: false +report_sinks: + - type: working-memory + path: /var/custodian/memory/working + filename_template: "pqrst-monthly-{date}-{run_id_short}.md" + timezone: Europe/Berlin + - type: state-hub-progress + event_type: pqrst_monthly_review + author: activity-core + topic_id: c1d199b6-55ee-4db6-b49e-257a9f0f15ac +``` diff --git a/docs/monthly-pqrst-automation.md b/docs/monthly-pqrst-automation.md new file mode 100644 index 0000000..2c17a2d --- /dev/null +++ b/docs/monthly-pqrst-automation.md @@ -0,0 +1,125 @@ +# Monthly PQRST review automation + +## Schedule and ownership + +`monthly-pqrst-review` runs through activity-core / Temporal at **09:30 +Europe/Berlin on the first day of each month** (`30 9 1 * *`). It reviews the +previous calendar month and five preceding months. The first regular report +is due **2026-10-01 09:30 CEST (07:30 UTC)**. `catchup_latest` uses activity-core's +24-hour default recovery window and buffers at most one missed fire. + +The [definition](../activity-definitions/monthly-pqrst-review.md) is owned here. +Activity-core owns `context_resolvers/pqrst.py`, tests, the worker runtime, +and the GitOps projection in `k8s/railiance/20-runtime.yaml`. Hall entries remain +the primary data store. This domain scheduling declaration is the explicitly +requested integration; no collector or validator implementation lives here. + +Authority: the user's 2026-09-28 setup request; `ADMINISTER @ +realm:kubernetes/railiance01`, `activation=APPROVED`, through the existing ArgoCD +application. Decision `e2c54d2f-074e-495c-841a-4b398006d85c`. This authorizes this +setup, not standing unattended release authority or automatic specification edits. + +## Report contract + +- Pin public `hall-of-helix/main` to a commit, enumerate its complete Git tree, + and read Markdown entries with Git blob-hash verification. Do not execute + entry text or load arbitrary source URLs. Collection has time, byte and count + bounds; an incomplete tree or failed file fetch produces an unavailable report, + never partial averages presented as a complete sample. +- Use `recorded_at` in Europe/Berlin to group entries by month. This is a + **recording-month** view, not reconstruction of when engineering effort occurred. + Exclude the open current month. Missing/invalid dates are reported as unclassifiable. +- One datapoint is one agent entry, including drafts, with a unique entry id and + a unique structurally valid block matching its frontmatter signature. Multiple + attempts remain an attempt count, not multiple datapoints. Duplicate entry ids, + ambiguous selections and invalid blocks are excluded. Distinct entries are not + claimed to be independently verified unique sessions. +- Structural checks cover the five integer values, 0..100 bounds, total 100, + confidence, signature agreement and nonempty Dominant factors. They cannot + establish whether narrative evidence is truthful, concrete or sufficient. +- Each month reports included count `n`, agent-entry count, missing/invalid + count, attempts, included draft count and confidence frequencies. Means weight + entries equally; no time/token weighting or averaging of already-rounded means. +- `mean_pp` and `change_pp` use the explicit `order: [P,Q,R,S,T]`. Changes are + differences from the immediately preceding calendar month's means, in + **percentage points**. Empty months have null means, and comparisons across an + empty month have null changes. Values round only for presentation to two decimals. +- Counts and task mix matter when interpreting movement. This voluntary sample + is not a census, first-attempt success measure, causal study or worker ranking. + No LLM calls, tasks, alerts to people, or automatic changes are produced. + +## Finding a report + +ActivityDefinition UUID: `18c2f495-74b8-59e6-b0c9-f6df68cd1cf5`. +Temporal schedule: `activity-schedule-18c2f495-74b8-59e6-b0c9-f6df68cd1cf5`. + +Both configured sinks are durable and use activity-core's run idempotency: + +1. State Hub progress event `pqrst_monthly_review`, topic + `c1d199b6-55ee-4db6-b49e-257a9f0f15ac`, author `activity-core`. +2. Railiance working memory: + `/home/tegwick/the-custodian/memory/working/pqrst-monthly--.md`. + +Inspect progress via the existing State Hub API: + +```bash +curl -fsS 'http://127.0.0.1:8000/progress/?event_type=pqrst_monthly_review&limit=12' +``` + +The current generic deterministic sink nests the bounded JSON document at +`detail.report.digest_preview`; parse that string as JSON. Its `months` array +contains the series and `status` distinguishes complete collection, entries +needing review, and unavailable source. The outer `output_validated` flag does +not certify the underlying narratives or source availability. Working-memory +notes use the same payload; their generic sink heading is inherited from WSJF. + +## Verification and operation + +The resolver has focused tests for monthly boundaries, year rollover, means and +deltas, no-data months, retries, duplicate ids/fields, invalid records, missing +metadata and source failure. The projected definition has a contract test and +is included in activity-core's image CI test stage. Related frontend, instruction +and GitOps checks passed. Runtime resolver bytes were matched to the tested source. + +The September 28 manual smoke delivered run +`c7d826af-83e9-528e-9ccd-8b4f5b5e24f1`, progress +`622b7cda-ba7b-465b-aa63-90d60e1a5b63`, and working-memory note +`pqrst-monthly-2026-09-28-c7d826af.md`. It covers August (0 usable records), not +September; it proves collection and sink delivery, not a natural monthly fire. + +Changes to the definition must be projected into activity-core, rendered with +`scripts/render_gitops.py`, reviewed with server dry-run and client diff, committed, +and synced via ArgoCD. Then call the existing activity-core +`POST /admin/sync?definitions=true&schedules=true` endpoint from the admitted +runtime and inspect definition/schedule state. Do not add a second cron. + +To pause, set the domain definition and projection `enabled: false`, GitOps sync, +then admin-sync schedules. To roll back this installation, restore the previous +worker digest `sha256:afd8a7af73132becba432e7b315d70f88d66676020169d5d8811641304136437` +and remove/disable only this definition through GitOps and admin-sync. The previous +Argo application revision is `4bd07a4f44f38bb4f27cfb381cfe53d555c2c6fe`; +prefer a scoped revert if later unrelated deployments have landed. + +## Activation receipt — 2026-09-28 + +- Definition enabled; Temporal `paused=false`, 24-hour catchup window. +- Next fires: October 1 07:30 UTC, November 1 08:30 UTC, December 1 08:30 UTC + (09:30 Europe/Berlin throughout). +- ArgoCD revision `ce9a8f8edf4ec162bae3aa087803bd57a1b33a66`: Synced and Healthy. +- Worker image `sha256:75e240ac23c1c6d7daae14691c2b4137d2d639a9af694aaf0d4c774a7e4d3886`; + resolver SHA-256 `41f91a68c75c758204e16319199d0a28060154630c45d74544d4470fe2a7a8ce` + matches source. Existing frontend resolver and repository-grant support remain present. +- Final admin sync: 34 definitions, 16 schedules upserted, 12 paused, zero errors + and zero orphan deletions. The monthly PQRST definition is enabled. +- Activity-core commits `eda44d9` / `ce9a8f8` and platform application pins + `a13fa57` / `7a12760` were pushed. Activity-core Repo Manager reconciliation + refused pre-existing `custodian-WP-*` legacy identifiers before projection; + this does not affect deployment or scheduling. No historic IDs were changed. +- No new tasks or workplans, credential changes, or unrelated entry edits. + +The enabled definition also passed a **one-shot Temporal Schedule** smoke at +2026-09-28 10:21:38 UTC: run `27c73e81-342e-5a84-b582-cf7d874fb17d`, progress +`e50d5e66-4abd-4ddd-87b3-d6556550884e`, working-memory `memory/working/pqrst-monthly-2026-09-28-27c73e81.md`. +Workflow completed with zero tasks; collection status was `ok` and both sinks +were verified. The temporary smoke schedule was removed; the recurring schedule +remains enabled. This is scheduled-path proof, not the October natural run.