From d1e24348c30ea57be3bb901034b388387fe81081 Mon Sep 17 00:00:00 2001 From: tegwick Date: Mon, 3 Aug 2026 18:51:37 +0200 Subject: [PATCH] Accept ACT-ADR-005 and open ACTIVITY-WP-0026 for ops_run claim queue. Split ops runs (claimable DB) from development work records (repo + State Hub), ban default Forgejo queueing for automation, and stack WP-0026 with rein-aharness and issue-core follow-ups. Forgejo-only forge language; no Gitea product path. --- SCOPE.md | 2 +- .../adr-005-ops-runs-vs-dev-work-records.md | 191 ++++++++++++++++ docs/recurring-automations-playbook.md | 28 ++- .../ACTIVITY-WP-0026-ops-run-claim-queue.md | 211 ++++++++++++++++++ 4 files changed, 422 insertions(+), 10 deletions(-) create mode 100644 docs/adr/adr-005-ops-runs-vs-dev-work-records.md create mode 100644 workplans/ACTIVITY-WP-0026-ops-run-claim-queue.md diff --git a/SCOPE.md b/SCOPE.md index 985f364..ea1883b 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -205,7 +205,7 @@ Open product/policy workplan: **ACTIVITY-WP-0022** (IssueSink no-default-Forgejo | Gap | Severity | Notes | | --- | --- | --- | | **G1. Sink default vs INTENT “emit to issue-core”** | High (policy) | INTENT names issue-core as the task landing zone. Practice and work-record canon push **State Hub / repo work records** for internal findings. WP-0022 owns policy; code already supports `state-hub` / `null` / `rest`. | -| **G2. Executor gap** | High (value) | Tasks are emitted (progress refs or issue-core), but **no per-repo executor** reliably consumes them (Binky briefs still need harness/agent pickup). INTENT assigns execution to per-repo Temporal workers — that substrate is thin. | +| **G2. Executor gap** | High (value) → **in progress** | Disposition: **ACT-ADR-005** + **ACTIVITY-WP-0026** (ops_run claim queue) + **REIN-A-0002** (claim loop). Do not use issue-core/Forgejo as default ops queue (WP-0022). Host timers interim only. | | **G3. Side-effect resolvers** | Medium | Forgejo prune `apply: true` runs platform deletes inside a context resolver. Allowed only as declared, gated side-effect; needs clearer SCOPE language (above) and operator refresh of live-images protection. | | **G4. Review queue** | Medium | `review_required` remains metadata-only; no downstream review queue. | | **G5. TaskExecutorWorkflow stub** | Low | Disabled unless `ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB` (WP-0023-T08). | diff --git a/docs/adr/adr-005-ops-runs-vs-dev-work-records.md b/docs/adr/adr-005-ops-runs-vs-dev-work-records.md new file mode 100644 index 0000000..623240c --- /dev/null +++ b/docs/adr/adr-005-ops-runs-vs-dev-work-records.md @@ -0,0 +1,191 @@ +--- +id: ACT-ADR-005 +type: architecture-decision-record +title: "Ops runs vs development work records — claim queue and plane split" +status: accepted +decided_by: Bernd Worsch +date: "2026-08-03" +scope: cross-repo +affects: + - activity-core + - rein-aharness + - state-hub + - issue-core + - domain repos (Binky, freedom-intelligence, …) +tags: + - architecture + - activity-core + - ops-run + - work-records + - rein-aharness + - issue-core + - forgejo +supersedes_notes: > + Clarifies ACTIVITY-WP-0022 sink practice and G2 executor gap from + ACTIVITY-WP-0023. Does not reverse WP-0022 (no default external tracker issues). +--- + +# ACT-ADR-005: Ops runs vs development work records + +## Status + +**Accepted** (2026-08-03). + +## Context + +The fleet has two healthy patterns that were forced into one vague “task” idea: + +1. **Development / coordination work** — structured workplans and tasks as + **repo files**, indexed by **State Hub** on **coulombcore** (`fix-consistency`, + UUIDv7 write-back). This matches ADR-001 and + `work-record-types_v0.1.md` and is how humans and coding agents ship product. + +2. **Operations / scheduled automation** — activity-core Temporal schedules + (when / what / where), rein-aharness execution (how), completion events + (idempotence). Instances of “today’s FI brief” or “this prune fire” are + **ephemeral runs**, not multi-day workplan bodies. + +Practice collapsed (2) into broken paths: + +| Path | Failure | +| ---- | ------- | +| Emit `activity_task_spawn` progress only | Append-only; **not claimable**; schedule can “succeed” while nothing runs | +| `ISSUE_SINK_TYPE=rest` → issue-core → Forgejo | Spam + 503s; issue-core INTENT forbids origin of fleet work | +| Host systemd timers dual-clocked with Temporal | Shadow scheduler; weak self-healing | + +issue-core’s **correct** role is a **connector facade** over external trackers +(Forgejo, GitHub, Jira, …) so agents need not know each backend. It is **not** +the origin of work records and **not** the default internal ops queue. +**Gitea is out of scope** for this fleet; the self-hosted forge is **Forgejo**. + +Canon already allows a DB-only exception for “runtime operations data (logs, +metrics, run histories, token events)” (`work-record-types_v0.1.md`). This ADR +names that exception for **ops runs**. + +## Decision + +### 1. Two planes for “work,” one vocabulary for people + +| Plane | Artefact | Home | Lifecycle | +| ----- | -------- | ---- | --------- | +| **A — Development / coordination** | Work records (`workplan`, `task`, `intake`, …) | Repo files + State Hub (coulombcore) | File-backed, reviewable, multi-day | +| **B — Recurrence control** | Activity definitions + Temporal schedules | activity-core (railiance) | Git definitions; DB schedule state | +| **C — Ops execution** | **`ops_run`** (runtime claim object) | **activity-core DB + claim API** | open → claimed → succeeded \| failed \| expired | +| **D — External collab** | Tracker issues | **issue-core** optional projection to **Forgejo** (and others) | Never default for internal automation | + +Coding agents orient on **Plane A** via State Hub. +Automation orients on **Planes B+C**. +Humans outside the fleet may see **Plane D** only when deliberately projected. + +### 2. activity-core owns the ops_run claim queue + +When a definition emits a TaskSpec for internal fleet work: + +1. Write **`ops_run`** row (durable, claimable) with idempotency key + (e.g. `fi-daily:2026-08-04` for once-per-day briefs). +2. Optionally dual-write `activity_task_spawn` progress for existing observers. +3. Do **not** open a Forgejo issue. +4. Do **not** create a workplan task file for that day’s fire. + +Claim API (sketch; implement in ACTIVITY-WP-0026): + +- `POST /ops-runs/claim` — lease next open run matching labels / worker id +- `POST /ops-runs/{id}/complete` — succeeded + completion metadata +- `POST /ops-runs/{id}/fail` — failed + retry policy +- `GET /ops-runs?state=open` — operator visibility + +activity-core remains **when / what / where** only: it does **not** run +domain LLM sessions or hold tenant git credentials. + +### 3. rein-aharness owns how (approach selection + execute) + +- Continuous **claim loop** (service / Deployment), not dual wall-clock timers + as source of truth. +- Host systemd timers become **break-glass** after cutover. +- **Approach selection** at claim time (minimal table v1): + + | Labels / definition family | Approach | + | -------------------------- | -------- | + | mail / mail-intake | deterministic adapter (+ optional triage) | + | research-brief / fi-daily | structured llm-connect (`fi-research-brief`) | + | binky rhythm | structured llm-connect (`brief-daily`) | + | agent-session | persona + tool profile session | + | unknown | refuse; do not invent | + +- Success posts **domain completion event** (e.g. `fi_daily_brief`) so + resolvers set `due=false`, and closes the `ops_run`. + +### 4. State Hub remains the work-record read model + +- Stays on **coulombcore**; railiance peers via edge relay / ops-bridge. +- May **project** open/failed ops runs for fleet ops UI (read-only), but is + **not** the claim authority. +- Development workplans/tasks continue file + `fix-consistency` only. + +### 5. issue-core is Forgejo (and multi-tracker) facade only + +- **Forgejo** is the self-hosted forge; do not plan for **Gitea** as a product. +- Optional **project / link** of an existing work-record UUID to an external + issue when collaboration needs it. +- Authenticated `POST /issues/` means “create/link **external** tracker work,” + never “spawn fleet automation.” +- Internal automation **must not** use issue-core as the ops claim queue unless + a future design adds an **internal-only** backend with projection **hard-off** + for automation labels — out of scope unless a later ADR says otherwise. + +### 6. Promotion path ops → dev + +When an ops run discovers multi-day product work (e.g. “collect Kimi K3 under +new quota”), the executor or human **promotes** an `intake` / workplan in the +domain repo (Plane A). Ops runs never become permanent fake workplans. + +## Consequences + +### Positive + +- Self-healing recurrence: schedule → claimable row → claim → complete/fail. +- Aligns issue-core INTENT with practice; stops Forgejo spam path. +- Keeps activity-core thin; keeps rein-aharness as sole session/credential shell. +- Preserves the good dev loop (files + State Hub). + +### Negative / cost + +- New schema + API + migrator in activity-core. +- rein-aharness must leave issue-core-only poll as primary for ops. +- Temporary dual-write and dual timers until cutover proven. + +### Non-goals + +- Replacing workplans for development. +- Running domain briefs inside activity-core workers. +- Global `ISSUE_SINK_TYPE=rest` to Forgejo. +- Gitea support or migration paths. + +## Topology + +```text +coulombcore: State Hub + work-record registry +railiance: activity-core (Temporal + ops_run queue) + rein-aharness (claim loop + approach + llm-connect) + domain checkouts +forgejo: optional external collab via issue-core projection only +``` + +## Implementation stack + +| Workplan | Owner | Role | +| -------- | ----- | ---- | +| ACTIVITY-WP-0026 | activity-core | ops_run schema, claim API, emit path, dual-write | +| REIN-A-0002 | rein-aharness | claim loop, approach table, FI/Binky cutover | +| ISSUE-WP-0006 | issue-core | Forgejo-only language; projection boundary; no ops queue | +| STATE-WP-0078 | state-hub | Read projection of ops_run for ops UI (optional consume) | + +## References + +- `docs/recurring-automations-playbook.md` +- `docs/task-emission-consumer-contract.md` +- `docs/issue-core-emission-boundary.md` +- ACTIVITY-WP-0022, ACTIVITY-WP-0023 (G2) +- `the-custodian/canon/standards/work-record-types_v0.1.md` +- `issue-core/INTENT.md` (work-record boundary) +- `state-hub/docs/cluster-operating-model.md` (coulombcore primary) diff --git a/docs/recurring-automations-playbook.md b/docs/recurring-automations-playbook.md index bc90e28..5aae481 100644 --- a/docs/recurring-automations-playbook.md +++ b/docs/recurring-automations-playbook.md @@ -1,11 +1,17 @@ # Recurring automations playbook — activity-core as the organizer **Audience:** operators and coding agents adding or fixing recurring work -**Principle:** activity-core answers **when / what / where** only. It does **not** execute. +**Principle:** activity-core answers **when / what / where** only. It does **not** execute. +**Architecture decision:** [ACT-ADR-005](adr/adr-005-ops-runs-vs-dev-work-records.md) — +ops runs (claimable DB) vs development work records (repo files + State Hub). +**Implementation stack:** ACTIVITY-WP-0026 · REIN-A-0002 · ISSUE-WP-0006 · STATE-WP-0078. This is the org standard. Prefer this path over workstation cron, ad-hoc shell timers, or “I’ll remember to run it.” +**Self-hosted forge:** **Forgejo** only (not Gitea). External tickets via +issue-core projection — never as the default ops claim queue. + --- ## The three layers (do not collapse them) @@ -86,15 +92,19 @@ kubectl -n activity-core exec deploy/actcore-worker -- python3 -c '...' # descr Ops UI: `activity.coulomb.social` → automations status / trigger (SSO). -### C. Executor (rein-aharness) +### C. Executor (rein-aharness) — target vs interim -1. Prefer a **named rein-aharness command** for the domain work - (e.g. `brief-daily`, `fi-research-brief`) using **llm-connect**, not Claude CLI on railiance01 -2. Install **railiance user-systemd timer** in the domain repo - (`scripts/railiance-rhythm/`) that calls rein-aharness -3. Timer should sit **just after** the activity-core cron (due check already - enforced by completion events on re-run) -4. On success, post the **domain completion event** so the resolver sets `due=false` +**Target (ACT-ADR-005 / REIN-A-0002):** continuous **claim loop** on +`ops_run` rows from activity-core; approach table selects adapter +(`fi-research-brief`, `brief-daily`, mail-scan, …). + +**Interim (until WP-0026 + REIN-A-0002 cutover):** railiance user-systemd +timer may call rein-aharness wall-clock — **break-glass / dual-clock**, not +cadence authority. + +1. Prefer a **named rein-aharness command** + **llm-connect**, not Claude CLI on railiance01 +2. On success, post the **domain completion event** so the resolver sets `due=false` +3. Close the **ops_run** (when claim queue is live) ### D. Prove the loop diff --git a/workplans/ACTIVITY-WP-0026-ops-run-claim-queue.md b/workplans/ACTIVITY-WP-0026-ops-run-claim-queue.md new file mode 100644 index 0000000..aa482a3 --- /dev/null +++ b/workplans/ACTIVITY-WP-0026-ops-run-claim-queue.md @@ -0,0 +1,211 @@ +--- +id: ACTIVITY-WP-0026 +type: workplan +title: "Ops run claim queue — durable claimable work for scheduled automation" +domain: infotech +repo: activity-core +status: ready +owner: grok +topic_slug: activity-core +priority: high +created: "2026-08-03" +updated: "2026-08-03" +depends_on: [] +related: + - ACT-ADR-005 + - ACTIVITY-WP-0022 + - ACTIVITY-WP-0023 + - REIN-A-0002 + - ISSUE-WP-0006 + - STATE-WP-0078 +--- + +# ACTIVITY-WP-0026 — Ops run claim queue + +## Origin + +ACT-ADR-005 (ops runs vs development work records). Closes **G2 executor gap** +without reopening default **Forgejo** issue spam (ACTIVITY-WP-0022). + +## Goal + +Make internal scheduled automation emit a **claimable `ops_run`** in activity-core +so rein-aharness (and peers) can claim, execute, complete, or fail with lease +semantics — while activity-core stays when/what/where only. + +**Non-goals:** execute domain work in Temporal activities; use issue-core or +Forgejo as the claim queue; invent Gitea support. + +## Architecture (target) + +```text +Temporal schedule / trigger + → resolve context (e.g. fi_brief_status) + → rules produce TaskSpec(s) + → INSERT ops_run (open) + optional activity_task_spawn progress + → rein-aharness claim loop + → complete/fail + domain completion event (fi_daily_brief, …) +``` + +## Tasks + +### T01 — Spec ops_run model and API + +```task +id: ACTIVITY-WP-0026-T01 +status: todo +priority: high +``` + +Document in `docs/ops-run-queue.md`: + +| Field | Notes | +| ----- | ----- | +| `id` | UUIDv7 | +| `activity_definition_id` | FK | +| `idempotency_key` | unique; e.g. `{def}:{date}` or hash of spawn key | +| `target_repo` | string | +| `title`, `description` | from TaskSpec | +| `labels` | JSON array | +| `priority` | | +| `state` | `open \| claimed \| succeeded \| failed \| expired` | +| `claim_owner` | worker identity | +| `lease_until` | timestamptz | +| `attempt` | int | +| `source_type`, `source_id`, `triggering_event_id` | audit | +| `approach_hint` | optional from definition/rule | +| `result` | JSON completion metadata | +| `created_at`, `updated_at` | | + +API (REST on actcore-api): + +- `POST /ops-runs/claim` body: `{ worker_id, labels?, limit? }` → runs +- `POST /ops-runs/{id}/heartbeat` extend lease +- `POST /ops-runs/{id}/complete` / `fail` +- `GET /ops-runs` filter by state, definition, since + +Auth: same operator/worker posture as existing admin APIs (document; prefer +service credential for harness, SSO for humans later). + +**Done when:** spec reviewed against ACT-ADR-005; OpenAPI-ready field list. + +--- + +### T02 — Schema migration + ORM + +```task +id: ACTIVITY-WP-0026-T02 +status: todo +priority: high +``` + +Alembic migration for `ops_runs` (+ indexes on `state`, `lease_until`, +`idempotency_key` unique). SQLAlchemy model. Unit tests for constraints. + +**Done when:** migrate up/down clean on dev; model tests green. + +--- + +### T03 — Emit path: create ops_run on TaskSpec emit + +```task +id: ACTIVITY-WP-0026-T03 +status: todo +priority: high +``` + +On successful rule emit (internal fleet path): + +1. Upsert/insert `ops_run` with idempotency (duplicate key → no second open row). +2. Keep **dual-write** `activity_task_spawn` progress for transition period. +3. Still write `task_spawn_log` audit. +4. **Never** require Forgejo / issue-core rest for this path. + +Default sink remains `state-hub` for progress; ops_run is **additional** +primary for claim. + +**Done when:** schedule fire or `/trigger` produces open `ops_run` for FI and +Binky-style definitions; tests with mocked sink. + +--- + +### T04 — Claim / lease / expire worker helpers + +```task +id: ACTIVITY-WP-0026-T04 +status: todo +priority: high +``` + +Implement claim with `SELECT … FOR UPDATE SKIP LOCKED` (or equivalent) and +lease TTL (config: e.g. 15m). Expire job or on-claim sweep: `claimed` past +`lease_until` → `open` (retry) or `expired` after max attempts. + +**Done when:** concurrent claim tests; lease expiry documented. + +--- + +### T05 — Ops visibility: list + SLA signals + +```task +id: ACTIVITY-WP-0026-T05 +status: todo +priority: medium +``` + +- Extend `/ops/automations/status` (or sibling) with open/claimed/failed counts. +- Signal: “spawned/open without complete by SLA” (config, default 1h for daily). +- Document in runbook. + +**Done when:** operator can see stuck ops_runs without SQL. + +--- + +### T06 — Contract + consumer guide + +```task +id: ACTIVITY-WP-0026-T06 +status: todo +priority: medium +``` + +Update: + +- `docs/task-emission-consumer-contract.md` — claim ops_run, not issue-core +- `docs/recurring-automations-playbook.md` — reference ACT-ADR-005 +- `docs/issue-core-emission-boundary.md` — ops_run is not issue-core +- Use **Forgejo** naming only (no Gitea product references) + +**Done when:** REIN-A-0002 can implement against the contract without guesswork. + +--- + +### T07 — Railiance rollout + dual-path flag + +```task +id: ACTIVITY-WP-0026-T07 +status: todo +priority: medium +``` + +- Feature flag `OPS_RUN_QUEUE_ENABLED=true` on railiance. +- Deploy migration + API. +- Smoke: trigger FI definition → open ops_run visible. +- Coordinate REIN-A-0002 cutover; keep host timers until harness claim proven. + +**Done when:** prod smoke checklist green; flag documented. + +## Acceptance + +- [ ] ACT-ADR-005 referenced in SCOPE gaps (G2 disposition → this WP) +- [ ] ops_run durable + claimable without Forgejo +- [ ] Emit dual-write progress for transition +- [ ] Docs use Forgejo-only language for self-hosted forge +- [ ] REIN-A-0002 unblocked + +## Out of scope + +- rein-aharness claim loop (REIN-A-0002) +- State Hub full write path for claim (read projection = STATE-WP-0078) +- issue-core internal SQLite “fake queue” +- Gitea compatibility