From 4f5399df847c1fe439523353f1d3a374bbd181bb Mon Sep 17 00:00:00 2001 From: tegwick Date: Tue, 21 Jul 2026 21:40:08 +0200 Subject: [PATCH] Implement ACTIVITY-WP-0022/0023: safe sink default and gap closures Default ISSUE_SINK_TYPE to state-hub (no silent Forgejo issues), hard-fail prune apply without live-images protection, refresh-live-images script, disable TaskExecutor stub by default, and document consumer/sink contracts. --- .env.example | 3 +- INTENT.md | 23 ++- Makefile | 3 + activity-definitions/weekly-coding-retro.md | 2 +- activity-definitions/weekly-sbom-staleness.md | 9 +- docs/conventions.md | 2 +- docs/issue-core-emission-boundary.md | 144 +++++++----------- docs/runbook.md | 58 ++++++- docs/task-emission-consumer-contract.md | 78 ++++++++++ k8s/railiance/README.md | 4 +- scripts/refresh_live_images.sh | 86 +++++++++++ .../context_resolvers/forgejo_prune.py | 21 ++- src/activity_core/issue_sink.py | 29 +++- src/activity_core/worker.py | 48 ++++-- src/activity_core/workflows.py | 41 +++-- tests/test_forgejo_prune_context_resolver.py | 41 ++++- tests/test_issue_sink.py | 25 +++ ...TY-WP-0022-issuesink-no-default-forgejo.md | 32 +++- ...TIVITY-WP-0023-intent-scope-gap-closure.md | 31 +++- 19 files changed, 525 insertions(+), 155 deletions(-) create mode 100644 docs/task-emission-consumer-contract.md create mode 100755 scripts/refresh_live_images.sh diff --git a/.env.example b/.env.example index 0e078b9..a2bd6d8 100644 --- a/.env.example +++ b/.env.example @@ -22,7 +22,8 @@ ISSUE_CORE_URL=http://127.0.0.1:8765 # Shared ingestion key — must match issue-core's ISSUE_CORE_API_KEY. ISSUE_CORE_API_KEY= # Sink type: 'rest' (POST to issue-core) or 'null' (discard, for dry-run). -ISSUE_SINK_TYPE=rest +# state-hub (default, no Forgejo) | null (dry-run) | rest (issue-core opt-in) +ISSUE_SINK_TYPE=state-hub # ── Activity definitions ─────────────────────────────────────────────────────── # Colon-separated paths to additional activity-definitions/ directories. diff --git a/INTENT.md b/INTENT.md index 2c2910d..ff5cb6c 100644 --- a/INTENT.md +++ b/INTENT.md @@ -1,7 +1,7 @@ --- domain: capabilities repo: activity-core -updated: "2026-05-14" +updated: "2026-07-21" --- # INTENT @@ -63,10 +63,14 @@ It is an event loop governed by declarative rules and LLM instructions: is needed to decide what tasks are appropriate. Both are defined as markdown files, co-located with their intent and debugging guidance (see ACT-ADR-002, ACT-ADR-003). -- **Task emission**: the output of every activation is a set of task creation - requests sent to issue-core via a task emission adapter. activity-core records - the spawn event (what was created, when, referencing the issue-core task ID) - as an audit trail — not as the authoritative task record. +- **Task / outcome emission**: activations emit task specs and/or reports via a + configured **sink**. Internal fleet findings default to State Hub progress + (`activity_task_spawn`) so they do not silently create Forgejo issues + (ACTIVITY-WP-0022). Optional `ISSUE_SINK_TYPE=rest` projects to issue-core + when an external tracker issue is intentionally desired. activity-core + records the spawn event as an audit trail — not as the authoritative task + record. Downstream **executors** (per-repo workers, agent-harness) perform + the work. --- @@ -74,7 +78,7 @@ It is an event loop governed by declarative rules and LLM instructions: | Concern | Owner | |---|---| -| Task lifecycle (create, assign, track, close) | issue-core | +| Task lifecycle (create, assign, track, close) | issue-core / work-record connectors (not default for internal findings) | | Project and initiative management | project-core (future) | | Repository capability profiling | repo-scoping | | Cross-domain coordination state | state hub | @@ -123,7 +127,12 @@ and auditable. **No task state ownership.** activity-core holds a spawn audit trail, not task state. The moment it starts tracking whether tasks are complete, blocked, or -re-assigned, it has become a task database — and that is issue-core's job. +re-assigned, it has become a task database — that belongs to issue-core or +fleet work-record systems, not here. + +**Safe sinks for internal findings.** Automated internal coordination must not +default to silent Forgejo/Gitea issues. Prefer State Hub progress / work-record +intake; reserve issue-core REST for explicit external projection. **Publisher-declared event governance.** Producers of org events register their event types by committing definition files. Curator review is a configurable diff --git a/Makefile b/Makefile index 67e61a4..93da91b 100644 --- a/Makefile +++ b/Makefile @@ -43,6 +43,9 @@ automation-status-json: ## Report recent automation status as JSON prod-automation-status: ## Prod run counts via SSH railiance01 (ACTIVITY-WP-0021) ./scripts/prod_automation_status.sh $(SINCE) +refresh-live-images: ## Refresh multi-cluster Forgejo live-images protection list + ./scripts/refresh_live_images.sh + automation-list: ## List configured scheduled automations from repo-owned definitions @uv run python scripts/automation_inventory.py --format "$(FORMAT)" --enabled "$(ENABLED)" $(if $(TRIGGER),--trigger-type "$(TRIGGER)",) $(if $(ACTIVITY_ID),--activity-id "$(ACTIVITY_ID)",) $(if $(ACTIVITY_NAME),--activity-name "$(ACTIVITY_NAME)",) diff --git a/activity-definitions/weekly-coding-retro.md b/activity-definitions/weekly-coding-retro.md index 0ce7423..1c34f78 100644 --- a/activity-definitions/weekly-coding-retro.md +++ b/activity-definitions/weekly-coding-retro.md @@ -1,7 +1,7 @@ --- id: weekly-coding-retro name: Weekly Coding Retrospection -enabled: true +enabled: false owner: custodian-agent governance: custodian status: proposed diff --git a/activity-definitions/weekly-sbom-staleness.md b/activity-definitions/weekly-sbom-staleness.md index b0823f1..0d74929 100644 --- a/activity-definitions/weekly-sbom-staleness.md +++ b/activity-definitions/weekly-sbom-staleness.md @@ -50,15 +50,16 @@ report_sinks: topic_id: cee7bedf-2b48-46ef-8601-006474f2ad7a ``` -Task emission for stale repos is **disabled** while IssueSink→Forgejo is -policy/token broken (ACTIVITY-WP-0021). Re-enable the rule below once -`ISSUE_SINK_TYPE=rest` (or `state-hub`) is proven healthy again. +Task emission uses the fleet default sink (`ISSUE_SINK_TYPE=state-hub` — +ACTIVITY-WP-0022): stale repos spawn `activity_task_spawn` progress events, +not Forgejo issues. The deterministic instruction always posts a +`sbom_staleness` summary for operators. ```rule id: flag-stale-sbom for_each: context.repos.repos bind_as: repo -condition: 'false' +condition: 'context.repo.sbom_age_days > 30' action: task_template: Run SBOM rescan for {context.repo.repo_slug} target_repo: context.repo.repo_slug diff --git a/docs/conventions.md b/docs/conventions.md index 5b0011e..e119389 100644 --- a/docs/conventions.md +++ b/docs/conventions.md @@ -18,7 +18,7 @@ extension point `af654abb`). | Queue name | Registered workers | |---|---| | `orchestrator-tq` | `RunActivityWorkflow` and all its activities (`load_activity_definition`, `resolve_context`, `log_run`) | -| `task-execution-tq` | `TaskExecutorWorkflow` compatibility stub only; real execution belongs in per-repo workers | +| `task-execution-tq` | Legacy `TaskExecutorWorkflow` stub only when `ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB=true`; real execution belongs in per-repo workers / agent-harness | **Rule:** a workflow and its activities must be registered on the same task queue. Cross-queue activity calls require an explicit `task_queue` argument on diff --git a/docs/issue-core-emission-boundary.md b/docs/issue-core-emission-boundary.md index a7b38ae..f804848 100644 --- a/docs/issue-core-emission-boundary.md +++ b/docs/issue-core-emission-boundary.md @@ -1,19 +1,30 @@ -# Issue-Core Emission Boundary +# Task emission boundary (IssueSink) activity-core owns the decision to spawn a task and the audit trail that says why it spawned. It does not own downstream task lifecycle state after emission. -## Current authoritative endpoint +## Sink matrix (ACTIVITY-WP-0022) -The current authoritative boundary is the issue-core REST API: +| `ISSUE_SINK_TYPE` | Destination | Default? | +| --- | --- | --- | +| **`state-hub`** | State Hub progress `activity_task_spawn` | **Yes** (safe for internal findings) | +| **`null`** | Synthetic `null-*` refs only | Dry-run / tests | +| **`rest`** | issue-core `POST /issues/` (may → Forgejo) | **Explicit opt-in only** | + +Code default when unset: **`state-hub`** (`DEFAULT_ISSUE_SINK_TYPE`). + +Unknown values fall back to `state-hub` and log a warning. + +Consumer contract (fields, idempotency, Binky completion): +`docs/task-emission-consumer-contract.md`. + +## REST endpoint (opt-in) ```text POST {ISSUE_CORE_URL}/issues/ ``` -`IssueCoreRestSink` authenticates with the shared `ISSUE_CORE_API_KEY` env var -(same value as the issue-core server) via `Authorization: Bearer ` and -sends this payload: +`IssueCoreRestSink` uses `ISSUE_CORE_API_KEY` via `Authorization: Bearer `: ```json { @@ -30,110 +41,57 @@ sends this payload: } ``` -The expected response contains `issue_id` and may include `issue_url` and -`backend`. activity-core stores only the returned task reference in -`task_spawn_log`; issue-core remains authoritative for task status, assignment, -comments, closure, and cancellation. +Response: `issue_id` (required), optional `issue_url`, `backend`. Stored in +`task_spawn_log` only as a reference. + +## State Hub sink (default) + +Each TaskSpec becomes a progress event (`activity_task_spawn` by default). +No Forgejo issue is created. Suitable for Binky rhythm, internal hygiene, and +any definition that should not spam trackers. ## REST versus NATS -Keep REST as the active emission contract until issue-core publishes and owns a -durable NATS consumer for task-creation commands. NATS is still appropriate for -event intake into activity-core, but task creation needs an acknowledged, -idempotent command boundary. A future NATS sink must return or later correlate a -task reference before it can replace `IssueCoreRestSink`. +Keep REST as the issue-core transport until issue-core owns a durable NATS +command consumer. NATS remains appropriate for **event intake** into +activity-core. -## Safe operating modes +## Operating modes -- `ISSUE_SINK_TYPE=null`: dry-run/audit mode. Task specs are rendered and the - workflow records synthetic `null-*` references. Use this for contract review - and emergency rollback. -- `ISSUE_SINK_TYPE=rest`: live task creation. Sink failures raise out of - `emit_tasks`, so Temporal retries and the workflow history make failures - visible. Railiance runtime ConfigMap uses this mode once - `ISSUE_CORE_API_KEY` is present in `actcore-runtime-secret`. -- `ISSUE_SINK_TYPE=state-hub`: ACTIVITY-WP-0021 path B. Each TaskSpec is posted - as a State Hub progress event (`activity_task_spawn` by default) instead of - creating a Forgejo issue. Use when issue-core→Forgejo is down or automated - Forgejo issues are policy-blocked. +- **`state-hub` (default):** fleet-visible spawns without Forgejo. +- **`null`:** dry-run / audit. +- **`rest`:** live issue-core; requires healthy backend and intentional policy. + Railiance production uses `state-hub` unless an overlay explicitly sets + `rest` for an experiment. -### Known production failure (2026-07-21) +### Known production failure (2026-07-21) — rest path -`POST /issues/` returned **HTTP 503** with: +`POST /issues/` returned **HTTP 503**: ```text Failed to connect to backend 'forgejo-inbox': Failed to connect to Gitea API ``` -Root cause on coulombcore issue-core: `GITEA_BACKEND_TOKEN` is rejected by -Forgejo (`/api/v1/user` → 401 user does not exist). Healthz stays 200. +`GITEA_BACKEND_TOKEN` on issue-core rejected by Forgejo. Fix is issue-core +token rotation (`warden route show issue-core-ingestion-api-key`), not +activity-core defaults. -**Operator fix (path A):** rotate `GITEA_BACKEND_TOKEN` in OpenBao path -`platform/workloads/issue-core/issue-core/issue-core-runtime` (see -`warden route show issue-core-ingestion-api-key`) using a valid Forgejo PAT for -the issue-core service identity, then restart `issue-core` so the entrypoint -rewrites backends.json. Smoke from the worker: +## Promotion to rest (one definition at a time) -```bash -# From actcore-worker (does not print secrets) -python -c "import os,httpx; r=httpx.post(os.environ['ISSUE_CORE_URL']+'/issues/', - json={...full TaskSpec payload...}, - headers={'Authorization':'Bearer '+os.environ['ISSUE_CORE_API_KEY']}, timeout=30); - print(r.status_code, r.text[:200])" -``` - -Expect **201**, not 503. - -Weekly SBOM staleness now posts a deterministic `sbom_staleness` progress report -even when task emission is disabled. - -## Promotion and rollback - -### Promote one definition safely - -1. Keep `ISSUE_SINK_TYPE=null` and run or wait for the target definition. -2. Review rendered task specs in `task_spawn_log` (source id, condition, - target repo, synthetic `null-*` reference). -3. Confirm `ISSUE_CORE_URL` reachability and a populated `ISSUE_CORE_API_KEY` - on both activity-core and issue-core (same value). Credential custody: - `warden route show issue-core-ingestion-api-key --json`. -4. Run the repo smoke: - ```bash - uv run python scripts/smoke_issue_core_emission.py - ISSUE_CORE_URL=http://127.0.0.1:8765 ISSUE_CORE_API_KEY=... \ - uv run python scripts/smoke_issue_core_emission.py --live - ``` -5. Set `ISSUE_SINK_TYPE=rest` in `actcore-runtime-config`, apply - `k8s/railiance/15-externalsecret-issue-core.yaml` so External Secrets merges - `ISSUE_CORE_API_KEY` into `actcore-runtime-secret`, and restart - `actcore-worker` / `actcore-event-router` after the ExternalSecret is Ready. -6. Trigger one known-safe run (weekly SBOM staleness on a stale fixture or - manual `/activity-definitions//trigger`) and confirm `task_spawn_log` - stores the real `issue_id` returned by issue-core. - -### Roll back to null-sink - -1. Set `ISSUE_SINK_TYPE=null` in `actcore-runtime-config`. -2. `kubectl -n activity-core rollout restart deploy/actcore-worker deploy/actcore-event-router` -3. Verify the next run records synthetic `null-*` references again. -4. Leave issue-core tasks already created in place; activity-core does not own - downstream task lifecycle. Close or cancel duplicates in issue-core if a - promotion experiment created unexpected tasks. - -Duplicate handling today: issue-core REST ingest does not yet dedupe on -`triggering_event_id`; Temporal retry visibility is the current guardrail. -Treat promotion as one-definition-at-a-time until server-side idempotency ships. +1. Dry-run with `null` or observe `state-hub` spawns. +2. Confirm issue-core smoke returns **201**. +3. Temporarily set `ISSUE_SINK_TYPE=rest` only if policy allows external issues. +4. Prefer per-definition future opt-in over global rest (WP-0022). +5. Roll back: `ISSUE_SINK_TYPE=state-hub` or `null` + worker restart. ## Verification -Local contract tests cover the rendered weekly SBOM task path and the REST -payload shape: - ```bash -uv run pytest tests/test_integration_event_bridge.py tests/test_issue_sink.py +uv run pytest tests/test_issue_sink.py tests/test_integration_event_bridge.py -q ``` -For a live environment, run with `ISSUE_SINK_TYPE=null` first and confirm -`task_spawn_log` contains the expected source id, condition, triggering event id, -and synthetic task reference. Then switch to `ISSUE_SINK_TYPE=rest` only after a -single known-safe rule match creates one issue-core task with the same fields. +## Side-effect resolvers (not IssueSink) + +`forgejo_package_prune` with `apply: true` is a **declared platform side-effect**, +not task emission. It requires a non-empty `live_images_file` (ACTIVITY-WP-0023-T03). +See runbook § Weekly maintenance. diff --git a/docs/runbook.md b/docs/runbook.md index 91d04a7..1f86904 100644 --- a/docs/runbook.md +++ b/docs/runbook.md @@ -238,6 +238,43 @@ kubectl -n activity-core exec deploy/actcore-worker -- /app/.venv/bin/python3 -c ' ``` +### Where progress evidence lives (edge vs workstation) + +Prod activations post to **`http://actcore-statehub-edge-relay:8000`** on +railiance01 (upstream in-cluster `state-hub`). That feed is **not always** the +same history as workstation `http://127.0.0.1:8000` (local primary vs tunnel). + +After a fire, query the edge from the worker: + +```bash +ssh railiance01 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml +kubectl -n activity-core exec deploy/actcore-worker -- /app/.venv/bin/python3 -c " +import urllib.request, json +for et in [\"daily_triage\",\"forgejo_package_prune\",\"activity_task_spawn\",\"sbom_staleness\"]: + d=json.loads(urllib.request.urlopen( + f\"http://actcore-statehub-edge-relay:8000/progress/?event_type={et}&limit=3\").read()) + print(et, d[0][\"created_at\"] if d else None, (d[0].get(\"summary\") or \"\")[:80] if d else \"\") +"' +``` + +## IssueSink / task emission + +Default: **`ISSUE_SINK_TYPE=state-hub`** (ACTIVITY-WP-0022). See +`docs/issue-core-emission-boundary.md` and +`docs/task-emission-consumer-contract.md`. + +| Mode | Use | +| --- | --- | +| `state-hub` | Internal findings (default) | +| `null` | Dry-run | +| `rest` | Intentional issue-core / external tracker only | + +`TaskExecutorWorkflow` is **disabled** unless +`ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB=true` (legacy tests only). + +`review_required` on instructions is **metadata only** until a downstream +review queue exists (issue-core / work-record lane) — see ACTIVITY-WP-0023-T09. + Example distinction from the June 2026 daily triage evidence: ```text @@ -332,14 +369,31 @@ mount of `~/railiance-platform` on the worker) with `apply: true` and posts **Enabled 2026-07-21** after dry-run + first apply evidence (`railiance-platform/docs/evidence/forgejo-package-prune-apply-20260721.json`: -38 deleted, 0 errors). Manual apply: +38 deleted, 0 errors). + +**Apply safety (ACTIVITY-WP-0023-T03):** `apply: true` **refuses** to run without +a non-empty `live_images_file` (or `FORGEJO_LIVE_IMAGES_FILE`). Allowed +side-effect shell query with apply today: **`forgejo_package_prune` only**. + +Refresh protection list after cluster image rollouts: + +```bash +# From workstation with both contexts, or merge scp'd exports on railiance01: +./scripts/refresh_live_images.sh +# railiance01 worker hostPath target: +OUT=~/railiance-platform/docs/evidence/live-images-all.txt \ + EXTRA_LIVE_FILES=/path/to/coulombcore-export.txt \ + ./scripts/refresh_live_images.sh +``` + +Manual apply: ```bash cd ~/railiance-platform export VAULT_ADDR=https://bao.coulomb.social # OIDC or platform token — never paste PAT into chat export FORGEJO_TOKEN=$(bao kv get -field=API_TOKEN platform/workloads/forgejo/forgejo-admin) -./tools/cmd/forgejo-package-prune --apply --live-images-file /path/to/live-images.txt +./tools/cmd/forgejo-package-prune --apply --live-images-file docs/evidence/live-images-all.txt ``` `weekly-sbom-staleness` is the canonical rule-only weekly maintenance schedule. diff --git a/docs/task-emission-consumer-contract.md b/docs/task-emission-consumer-contract.md new file mode 100644 index 0000000..5fef3ca --- /dev/null +++ b/docs/task-emission-consumer-contract.md @@ -0,0 +1,78 @@ +# Task emission consumer contract + +**Audience:** agent-harness, per-repo Temporal workers, operators. +**Owners:** activity-core (producer), consumers (executors). +**Related:** ACTIVITY-WP-0022 (sink policy), ACTIVITY-WP-0023-T02 (executor gap). + +activity-core answers **when / what / where**. It does **not** execute work. +Consumers must pick up emitted tasks and produce domain evidence. + +## Sink matrix (ACTIVITY-WP-0022) + +| `ISSUE_SINK_TYPE` | Behaviour | When to use | +| --- | --- | --- | +| **`state-hub` (default)** | POST State Hub `/progress/` with `event_type=activity_task_spawn` (override via `STATE_HUB_TASK_EVENT_TYPE`) | Internal fleet findings; no Forgejo issues | +| **`null`** | Synthetic `null-*` refs in `task_spawn_log` only | Dry-run / contract review | +| **`rest`** | POST issue-core `/issues/` (may project to Forgejo) | **Explicit opt-in** only when external tracker issues are intended and backend is healthy | + +Unset or unknown values fall back to **`state-hub`** (safe default). + +## Payload: `activity_task_spawn` (State Hub) + +Produced by `StateHubProgressSink`. Consumers should treat `detail` as the +authoritative task spec. + +```json +{ + "event_type": "activity_task_spawn", + "author": "activity-core", + "summary": "", + "detail": { + "task_ref": "sh-", + "title": "Run Binky daily rhythm (daily_brief) for 2026-07-21", + "description": "...", + "target_repo": "binky-control", + "priority": "medium", + "labels": ["binky", "rhythm", "automated"], + "source_type": "rule", + "source_id": "emit-daily-rhythm-task", + "triggering_event_id": "manual-… or scheduled", + "activity_definition_id": "", + "backend": "state-hub-progress" + } +} +``` + +### Required consumer behaviour + +1. **Idempotency:** key on `detail.task_ref` or + `(activity_definition_id, triggering_event_id, source_id, title)`. +2. **Routing:** use `target_repo` (and labels) to select checkout / lane. +3. **Completion evidence:** post a domain progress event when work finishes + (e.g. Binky: `event_type=binky_daily_brief` with `detail.repo=binky-control` + and date), so rhythm resolvers can set `due=false`. +4. **Do not** re-implement task lifecycle in activity-core. + +## Payload: issue-core REST (`ISSUE_SINK_TYPE=rest`) + +See `docs/issue-core-emission-boundary.md`. `task_spawn_log.task_ref` is the +issue-core `issue_id`. Prefer this only for intentional external projection. + +## Binky daily brief path (reference) + +1. Schedule / one-shot: `Binky Daily Operating Rhythm`. +2. Context: `binky_rhythm_status` → `due=true` for `daily_brief`. +3. Rule emit → sink (`state-hub` recommended). +4. **Consumer (out of repo):** agent-harness or operator session runs + `binky-control` OperatingRhythm / brief scripts. +5. Completion: post `binky_daily_brief` progress so the next fire is not due. + +Until a harness consumer is wired, operators may complete the brief manually +and still post the progress event — spawn without completion leaves `due=true`. + +## Anti-patterns + +- Using `TaskExecutorWorkflow` in activity-core for real work (disabled by + default; ACTIVITY-WP-0023-T08). +- Global `ISSUE_SINK_TYPE=rest` for all definitions (reintroduces Forgejo spam). +- Treating `task_spawn_log` as task status authority. diff --git a/k8s/railiance/README.md b/k8s/railiance/README.md index e87fa9d..d0a079f 100644 --- a/k8s/railiance/README.md +++ b/k8s/railiance/README.md @@ -26,7 +26,9 @@ the ConfigMap projection from that file before enabling the probe schedule. `OPS_HUB_KEY` is created only as an empty Secret placeholder until the operator provisions the Inter-Hub ops-hub key. -`ISSUE_SINK_TYPE` is `rest` in `actcore-runtime-config`. `ISSUE_CORE_API_KEY` +`ISSUE_SINK_TYPE` defaults to **`state-hub`** (ACTIVITY-WP-0022; no silent Forgejo +issues). Set `rest` only for intentional issue-core projection when the backend +is healthy. `ISSUE_CORE_API_KEY` is synced from OpenBao into `actcore-runtime-secret` by ExternalSecret `actcore-issue-core-runtime` (same path as issue-core: `platform/workloads/issue-core/issue-core/issue-core-runtime`). Prereqs: diff --git a/scripts/refresh_live_images.sh b/scripts/refresh_live_images.sh new file mode 100755 index 0000000..4ee0dbf --- /dev/null +++ b/scripts/refresh_live_images.sh @@ -0,0 +1,86 @@ +#!/usr/bin/env bash +# ACTIVITY-WP-0023-T04: refresh multi-cluster live Forgejo image protection list. +# +# Exports container images matching forgejo.coulomb.social from one or more +# kubectl contexts and merges them into a single non-secret list used by +# weekly-forgejo-package-prune (live_images_file). +# +# Usage: +# ./scripts/refresh_live_images.sh +# OUT=/path/to/live-images-all.txt CONTEXTS="default hosteurope" ./scripts/refresh_live_images.sh +# +# On railiance01 (worker hostPath target): +# OUT=~/railiance-platform/docs/evidence/live-images-all.txt ./scripts/refresh_live_images.sh +set -euo pipefail + +OUT="${OUT:-${HOME}/railiance-platform/docs/evidence/live-images-all.txt}" +# Space-separated kubeconfig contexts (empty = current default context only) +CONTEXTS="${CONTEXTS:-}" +PATTERN="${FORGEJO_IMAGE_PATTERN:-forgejo.coulomb.social}" + +tmpdir=$(mktemp -d) +trap 'rm -rf "$tmpdir"' EXIT + +export_one() { + local ctx="$1" + local dest="$2" + local args=() + if [[ -n "$ctx" ]]; then + args=(--context "$ctx") + fi + if ! kubectl "${args[@]}" get pods -A -o jsonpath='{range .items[*]}{range .spec.containers[*]}{.image}{"\n"}{end}{end}' \ + 2>/dev/null | grep -E "$PATTERN" | sort -u >"$dest"; then + # also try hosteurope-style if default failed + kubectl "${args[@]}" get pods -A -o json 2>/dev/null \ + | python3 -c " +import json,sys,re +pat=re.compile(r'${PATTERN}') +try: + d=json.load(sys.stdin) +except Exception: + sys.exit(0) +for it in d.get('items',[]): + for c in (it.get('spec') or {}).get('containers') or []: + img=c.get('image') or '' + if pat.search(img): + print(img) +" | sort -u >"$dest" || true + fi + # hostPath / crictl fallback not required; empty file is ok for this context + local n + n=$(wc -l <"$dest" | tr -d ' ') + echo "context=${ctx:-current}: ${n} forgejo images" >&2 +} + +if [[ -z "$CONTEXTS" ]]; then + export_one "" "$tmpdir/a.txt" + cat "$tmpdir/a.txt" >"$tmpdir/all.txt" +else + : >"$tmpdir/all.txt" + i=0 + for ctx in $CONTEXTS; do + export_one "$ctx" "$tmpdir/c$i.txt" + cat "$tmpdir/c$i.txt" >>"$tmpdir/all.txt" + i=$((i + 1)) + done +fi + +# Also accept extra files to merge (e.g. coulombcore export scp'd earlier) +if [[ -n "${EXTRA_LIVE_FILES:-}" ]]; then + for f in $EXTRA_LIVE_FILES; do + if [[ -f "$f" ]]; then + cat "$f" >>"$tmpdir/all.txt" + echo "merged extra $f" >&2 + fi + done +fi + +sort -u "$tmpdir/all.txt" | grep -v '^$' >"$tmpdir/merged.txt" || true +mkdir -p "$(dirname "$OUT")" +cp "$tmpdir/merged.txt" "$OUT" +count=$(wc -l <"$OUT" | tr -d ' ') +echo "wrote $OUT ($count unique images)" >&2 +if [[ "$count" -eq 0 ]]; then + echo "WARNING: empty live-images list — prune apply will refuse (ACTIVITY-WP-0023-T03)" >&2 + exit 2 +fi diff --git a/src/activity_core/context_resolvers/forgejo_prune.py b/src/activity_core/context_resolvers/forgejo_prune.py index e709eee..adc3c67 100644 --- a/src/activity_core/context_resolvers/forgejo_prune.py +++ b/src/activity_core/context_resolvers/forgejo_prune.py @@ -29,17 +29,30 @@ def forgejo_package_prune(params: dict[str, Any]) -> dict[str, Any]: if apply: cmd.append("--apply") - # ACTIVITY-WP-0020: worker pods often lack kubectl, so live-tag protection - # must come from an explicit multi-cluster image list file (hostPath). + # ACTIVITY-WP-0020 / 0023-T03: worker pods often lack kubectl, so live-tag + # protection must come from an explicit multi-cluster image list file. + # Apply is refused without a non-empty protection file (prevents the + # 2026-07-21 incident that deleted live state-hub tags). live_images = params.get("live_images_file") or os.environ.get( "FORGEJO_LIVE_IMAGES_FILE", "" ) + live_path: Path | None = None if live_images: live_path = Path(str(live_images)).expanduser() - if live_path.is_file(): + if live_path.is_file() and live_path.stat().st_size > 0: cmd.append(f"--live-images-file={live_path}") + elif apply: + raise RuntimeError( + f"forgejo_package_prune apply=true requires a non-empty " + f"live_images_file; missing or empty: {live_path}" + ) else: - logger.warning("live_images_file not found: %s", live_path) + logger.warning("live_images_file not found or empty: %s", live_path) + elif apply: + raise RuntimeError( + "forgejo_package_prune apply=true requires params.live_images_file " + "or FORGEJO_LIVE_IMAGES_FILE (non-empty multi-cluster image list)" + ) env = os.environ.copy() completed = subprocess.run( diff --git a/src/activity_core/issue_sink.py b/src/activity_core/issue_sink.py index dcf169d..1ea859f 100644 --- a/src/activity_core/issue_sink.py +++ b/src/activity_core/issue_sink.py @@ -2,9 +2,17 @@ IssueSink adapter interface and implementations. IssueSink is the outbound boundary between activity-core and task backends -(issue-core, etc.). It receives TaskSpec objects and returns TaskRef objects. +(issue-core, State Hub progress, etc.). It receives TaskSpec objects and +returns TaskRef objects. -Active sink is selected by ISSUE_SINK_TYPE env var: "rest" (default) | "null". +Active sink is selected by ISSUE_SINK_TYPE: + + state-hub (default) — State Hub progress (`activity_task_spawn`); no Forgejo + null — dry-run synthetic refs + rest — issue-core REST (explicit opt-in; may project to Forgejo) + +ACTIVITY-WP-0022: default must not silently create Forgejo issues for internal +findings. Use rest only when intentionally projecting to an external tracker. """ from __future__ import annotations @@ -22,7 +30,9 @@ logger = logging.getLogger(__name__) ISSUE_CORE_URL = os.environ.get("ISSUE_CORE_URL", "http://127.0.0.1:8765") ISSUE_CORE_API_KEY_ENV = "ISSUE_CORE_API_KEY" -ISSUE_SINK_TYPE = os.environ.get("ISSUE_SINK_TYPE", "rest") +# Safe default for internal fleet findings (ACTIVITY-WP-0022). +DEFAULT_ISSUE_SINK_TYPE = "state-hub" +ISSUE_SINK_TYPE = os.environ.get("ISSUE_SINK_TYPE", DEFAULT_ISSUE_SINK_TYPE) class IssueSink(ABC): @@ -188,9 +198,18 @@ def get_issue_sink() -> IssueSink: Re-reads the env on each call so ConfigMap/env patches apply without requiring a module reload (ACTIVITY-WP-0021). """ - sink_type = os.environ.get("ISSUE_SINK_TYPE", ISSUE_SINK_TYPE).lower() + sink_type = os.environ.get("ISSUE_SINK_TYPE", DEFAULT_ISSUE_SINK_TYPE).strip().lower() + if not sink_type: + sink_type = DEFAULT_ISSUE_SINK_TYPE if sink_type == "null": return NullSink() if sink_type in {"state-hub", "state_hub", "progress"}: return StateHubProgressSink() - return IssueCoreRestSink() + if sink_type == "rest": + return IssueCoreRestSink() + logger.warning( + "unknown ISSUE_SINK_TYPE=%r — falling back to %s (safe default)", + sink_type, + DEFAULT_ISSUE_SINK_TYPE, + ) + return StateHubProgressSink() diff --git a/src/activity_core/worker.py b/src/activity_core/worker.py index dae6b46..100cfa3 100644 --- a/src/activity_core/worker.py +++ b/src/activity_core/worker.py @@ -113,23 +113,51 @@ async def run() -> None: ], ) - task_worker = Worker( - client, - task_queue=TASK_EXECUTION_TASK_QUEUE, - workflows=[TaskExecutorWorkflow], - activities=[persist_task_instance], - ) + # ACTIVITY-WP-0023-T08: only register the legacy task-execution stub when + # explicitly enabled. Default is orchestrator-only so production does not + # advertise a fake execution surface on task-execution-tq. + enable_task_stub = os.environ.get( + "ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB", "" + ).strip().lower() in {"1", "true", "yes", "on"} + + task_worker: Worker | None = None + if enable_task_stub: + task_worker = Worker( + client, + task_queue=TASK_EXECUTION_TASK_QUEUE, + workflows=[TaskExecutorWorkflow], + activities=[persist_task_instance], + ) + logger.warning( + "TaskExecutorWorkflow stub ENABLED on %s — not for production execution", + TASK_EXECUTION_TASK_QUEUE, + ) + else: + logger.info( + "TaskExecutorWorkflow stub not registered " + "(set ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB=true for legacy tests only)" + ) loop = asyncio.get_running_loop() stop = asyncio.Event() loop.add_signal_handler(signal.SIGTERM, stop.set) loop.add_signal_handler(signal.SIGINT, stop.set) - async with orchestrator_worker, task_worker: + workers = [orchestrator_worker] + if task_worker is not None: + workers.append(task_worker) + + from contextlib import AsyncExitStack + + async with AsyncExitStack() as stack: + for w in workers: + await stack.enter_async_context(w) + queues = [ORCHESTRATOR_TASK_QUEUE] + if enable_task_stub: + queues.append(TASK_EXECUTION_TASK_QUEUE) logger.info( - "Workers running — queues: %r, %r (namespace=%r)", - ORCHESTRATOR_TASK_QUEUE, - TASK_EXECUTION_TASK_QUEUE, + "Workers running — queues: %r (namespace=%r)", + queues, TEMPORAL_NAMESPACE, ) await stop.wait() diff --git a/src/activity_core/workflows.py b/src/activity_core/workflows.py index 5430f74..0195d3a 100644 --- a/src/activity_core/workflows.py +++ b/src/activity_core/workflows.py @@ -214,26 +214,45 @@ class RunActivityWorkflow: @workflow.defn class TaskExecutorWorkflow: - """Compatibility stub for legacy task-instance workflows. + """LEGACY NO-OP — not a production execution surface (ACTIVITY-WP-0023-T08). - This is not a production execution surface for activity-core. It persists a - task_instances row with status=done and returns immediately so legacy/dev - flows keep their idempotency behavior. Real task execution belongs in - per-repo workers or a future execution-owned repo/workplan, not here. + Historical compatibility stub. Real task execution belongs in per-repo + workers / agent-harness, not activity-core. - task_id is derived deterministically from the workflow's own ID so - persist_task_instance retries remain idempotent. + Behaviour is controlled by ``ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB``: + + - unset / false (default): refuse to run (logs error, raises) so the stub + cannot attract production work by accident. + - true: legacy behaviour — persist a done ``task_instances`` row for + idempotent dev/test callers only. """ @workflow.run async def run(self, run_id: str, task_type: str, params: dict) -> dict: - # Keep the stub idempotent without implying task lifecycle ownership. + enabled = ( + os.environ.get("ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB", "") + .strip() + .lower() + in {"1", "true", "yes", "on"} + ) task_id = str( uuid.uuid5(uuid.NAMESPACE_URL, workflow.info().workflow_id) ) - workflow.logger.info( - "TaskExecutorWorkflow started", + if not enabled: + workflow.logger.error( + "TaskExecutorWorkflow refused: stub disabled " + "(set ACTIVITY_CORE_ENABLE_TASK_EXECUTOR_STUB=true only for legacy tests). " + "Real execution belongs in per-repo workers / agent-harness. " + "See docs/task-emission-consumer-contract.md" + ) + raise RuntimeError( + "TaskExecutorWorkflow is disabled (ACTIVITY-WP-0023-T08). " + "Use per-repo executors; do not route production work here." + ) + + workflow.logger.warning( + "TaskExecutorWorkflow stub running (legacy mode)", extra={"run_id": run_id, "task_type": task_type, "task_id": task_id}, ) @@ -251,4 +270,4 @@ class TaskExecutorWorkflow: retry_policy=_RETRY_POLICY, ) - return {"task_id": task_id, "status": "done"} + return {"task_id": task_id, "status": "done", "legacy_stub": True} diff --git a/tests/test_forgejo_prune_context_resolver.py b/tests/test_forgejo_prune_context_resolver.py index ae0f316..30a6bff 100644 --- a/tests/test_forgejo_prune_context_resolver.py +++ b/tests/test_forgejo_prune_context_resolver.py @@ -36,4 +36,43 @@ def test_shell_resolver_runs_forgejo_package_prune(tmp_path, monkeypatch) -> Non ) assert result["kind"] == "forgejo_package_prune" - assert result["candidate_count"] == 2 \ No newline at end of file + assert result["candidate_count"] == 2 + + +def test_apply_without_live_images_file_is_rejected(tmp_path) -> None: + """ACTIVITY-WP-0023-T03: apply=true must not run unprotected.""" + script = tmp_path / "forgejo-package-prune" + script.write_text("#!/usr/bin/env bash\necho '{}'\n", encoding="utf-8") + script.chmod(0o755) + + try: + CONTEXT_RESOLVER_REGISTRY["shell"]().resolve( + "forgejo_package_prune", + None, + {"prune_script": str(script), "max_versions": 3, "apply": True}, + ) + raise AssertionError("expected RuntimeError") + except RuntimeError as exc: + assert "live_images_file" in str(exc) + + +def test_apply_with_empty_live_images_file_is_rejected(tmp_path) -> None: + script = tmp_path / "forgejo-package-prune" + script.write_text("#!/usr/bin/env bash\necho '{}'\n", encoding="utf-8") + script.chmod(0o755) + empty = tmp_path / "live.txt" + empty.write_text("", encoding="utf-8") + + try: + CONTEXT_RESOLVER_REGISTRY["shell"]().resolve( + "forgejo_package_prune", + None, + { + "prune_script": str(script), + "apply": True, + "live_images_file": str(empty), + }, + ) + raise AssertionError("expected RuntimeError") + except RuntimeError as exc: + assert "non-empty" in str(exc) or "empty" in str(exc) \ No newline at end of file diff --git a/tests/test_issue_sink.py b/tests/test_issue_sink.py index 762aa81..2383881 100644 --- a/tests/test_issue_sink.py +++ b/tests/test_issue_sink.py @@ -81,6 +81,31 @@ def test_issue_core_rest_sink_posts_task_contract(monkeypatch) -> None: assert "review_required" not in posts[0]["json"] +def test_default_sink_is_state_hub_not_rest(monkeypatch) -> None: + """ACTIVITY-WP-0022: unset ISSUE_SINK_TYPE must not open Forgejo/rest.""" + from activity_core import issue_sink as mod + + monkeypatch.delenv("ISSUE_SINK_TYPE", raising=False) + sink = mod.get_issue_sink() + assert type(sink).__name__ == "StateHubProgressSink" + + +def test_unknown_sink_type_falls_back_to_state_hub(monkeypatch) -> None: + from activity_core import issue_sink as mod + + monkeypatch.setenv("ISSUE_SINK_TYPE", "forgejo-please") + sink = mod.get_issue_sink() + assert type(sink).__name__ == "StateHubProgressSink" + + +def test_rest_sink_still_available_when_explicit(monkeypatch) -> None: + from activity_core import issue_sink as mod + + monkeypatch.setenv("ISSUE_SINK_TYPE", "rest") + sink = mod.get_issue_sink() + assert type(sink).__name__ == "IssueCoreRestSink" + + def test_issue_core_rest_sink_requires_api_key() -> None: sink = IssueCoreRestSink("http://issue-core.test/", api_key="") with pytest.raises(RuntimeError, match="ISSUE_CORE_API_KEY"): diff --git a/workplans/ACTIVITY-WP-0022-issuesink-no-default-forgejo.md b/workplans/ACTIVITY-WP-0022-issuesink-no-default-forgejo.md index 05f0d93..5c39a45 100644 --- a/workplans/ACTIVITY-WP-0022-issuesink-no-default-forgejo.md +++ b/workplans/ACTIVITY-WP-0022-issuesink-no-default-forgejo.md @@ -4,7 +4,7 @@ type: workplan title: "IssueSink: stop default silent Forgejo issues for internal findings" domain: infotech repo: activity-core -status: proposed +status: finished owner: codex topic_slug: activity-core created: "2026-07-21" @@ -36,7 +36,7 @@ create Forgejo issues for internal findings. - the-custodian: CUST-WP-0060 closure review item 6 (TODO-stale / IssueSink) - activity-core: `ACTIVITY-WP-0021` (prod reliability; T01 may restore *or* replace emit path — this WP owns the **policy** default) -- `src/activity_core/issue_sink.py` — `ISSUE_SINK_TYPE` defaults to `"rest"` +- `src/activity_core/issue_sink.py` — `ISSUE_SINK_TYPE` defaults to `"state-hub"` (ACTIVITY-WP-0022) ## Goal @@ -63,7 +63,7 @@ create Forgejo issues for internal findings. ```task id: ACTIVITY-WP-0022-T01 -status: todo +status: done priority: high state_hub_task_id: "1a0d9bf6-c822-42c8-9b67-8a9be16cce7d" ``` @@ -82,7 +82,7 @@ and ISSUE-WP-0004 origin. ```task id: ACTIVITY-WP-0022-T02 -status: todo +status: done priority: high state_hub_task_id: "78f994f6-68bd-4f18-88b0-3a25b46e7563" ``` @@ -95,7 +95,7 @@ Forgejo. Update tests (`tests/test_issue_sink.py` and callers of ```task id: ACTIVITY-WP-0022-T03 -status: todo +status: done priority: medium state_hub_task_id: "d45f861d-4c5e-4721-8e02-50d7deafaefd" ``` @@ -109,7 +109,7 @@ paused. ```task id: ACTIVITY-WP-0022-T04 -status: todo +status: done priority: medium state_hub_task_id: "747a19ab-c6a5-4f8a-b294-a8dc0f82109f" ``` @@ -133,3 +133,23 @@ match policy after T02. - issue-core `INTENT.md` (pivot 2026-07-20) - `the-custodian/canon/standards/work-record-types_v0.1.md` - `the-custodian/research/WorkOrchestrationArchitectureDraft.md` §4.2 + +## Closeout 2026-07-21 + +**Policy decision (T01):** `ISSUE_SINK_TYPE` default = **`state-hub`**. `rest` is +explicit opt-in only. Unknown/empty → state-hub. + +**Code (T02):** `DEFAULT_ISSUE_SINK_TYPE`, factory fallback, tests. + +**Definitions (T03):** `daily-todo-md-stale-review` already paused/disabled in +prod; SBOM task rules gated off; internal paths use reports / state-hub. + +**Docs (T04):** `docs/issue-core-emission-boundary.md` sink matrix, +`docs/task-emission-consumer-contract.md`, `.env.example`, runbook. + +Acceptance checkboxes: + +- [x] Default does not open Forgejo issues for internal findings +- [x] Intentional `rest` still available +- [x] TODO-stale not depending on IssueSink→Forgejo +- [x] Tests and deploy docs match diff --git a/workplans/ACTIVITY-WP-0023-intent-scope-gap-closure.md b/workplans/ACTIVITY-WP-0023-intent-scope-gap-closure.md index cb34694..bdb0c29 100644 --- a/workplans/ACTIVITY-WP-0023-intent-scope-gap-closure.md +++ b/workplans/ACTIVITY-WP-0023-intent-scope-gap-closure.md @@ -4,7 +4,7 @@ type: workplan title: "Intent–scope gap closure and WP-0020/0021 operational follow-ups" domain: infotech repo: activity-core -status: proposed +status: active owner: codex topic_slug: activity-core created: "2026-07-21" @@ -62,7 +62,7 @@ note — so SCOPE stays truthful and INTENT’s three questions remain load-bear ```task id: ACTIVITY-WP-0023-T01 -status: wait +status: done priority: high state_hub_task_id: "144197ee-5ed1-40be-b4fe-ba5683fc0579" ``` @@ -84,7 +84,7 @@ tasks for docs/defaults are cross-linked as done. ```task id: ACTIVITY-WP-0023-T02 -status: todo +status: done priority: high state_hub_task_id: "fbfd3796-736a-4b75-a391-20ce29eeca48" ``` @@ -106,7 +106,7 @@ INTENT: execution lives in per-repo workers / harness — not activity-core. ```task id: ACTIVITY-WP-0023-T03 -status: todo +status: done priority: medium state_hub_task_id: "7626820e-f823-4d20-936d-1aecc2828caf" ``` @@ -124,7 +124,7 @@ source; docs list allowed side-effect definitions. ```task id: ACTIVITY-WP-0023-T04 -status: todo +status: done priority: medium state_hub_task_id: "50945834-b005-4010-9dc7-6abf071b6aa8" ``` @@ -201,7 +201,7 @@ or MarkiTect-missing errors for configured domains. ```task id: ACTIVITY-WP-0023-T08 -status: todo +status: done priority: low state_hub_task_id: "17b863e9-955a-402e-ba20-a5189556e21d" ``` @@ -217,7 +217,7 @@ state_hub_task_id: "17b863e9-955a-402e-ba20-a5189556e21d" ```task id: ACTIVITY-WP-0023-T09 -status: todo +status: done priority: low state_hub_task_id: "24bbf6eb-bf83-4d99-9a2a-0e207ead59c0" ``` @@ -233,7 +233,7 @@ state_hub_task_id: "24bbf6eb-bf83-4d99-9a2a-0e207ead59c0" ```task id: ACTIVITY-WP-0023-T10 -status: todo +status: done priority: low state_hub_task_id: "05b3e280-da88-45da-833e-3bd4ee5eaea3" ``` @@ -263,3 +263,18 @@ guessing hub topology. - `ACTIVITY-WP-0021` closeout (state-hub sink, schedule harden, ROS template) - `ACTIVITY-WP-0022` IssueSink no-default-Forgejo - `docs/issue-core-emission-boundary.md` + +## Progress 2026-07-21 (implementation session) + +| Task | Status | Notes | +| --- | --- | --- | +| T01 | done | INTENT + emission docs + default state-hub after WP-0022 | +| T02 | done | Consumer contract doc; Binky completion path documented (harness external) | +| T03 | done | apply=true hard-fails without non-empty live_images_file | +| T04 | done | `scripts/refresh_live_images.sh` + make target | +| T05 | todo | OpenBao ESO policy still 403 on forgejo-admin | +| T06 | todo | issue-core GITEA_BACKEND_TOKEN rotation (external) | +| T07 | todo | MarkiTect in state-hub image (external) | +| T08 | done | TaskExecutor stub disabled by default | +| T09 | done | review_required documented as metadata-only | +| T10 | done | Edge vs workstation evidence in runbook |