activity-core/history/2026-08-23-scope-against-intent-assessment.md
tegwick c384f60530
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 1s
Plan intent boundary guardrails
Assistant: codex
Assistant-Model: gpt-5.6-sol
Assistant-Session: 01a028de-e2c8-7732-8521-46a7fc5db82f
2026-08-23 11:24:58 +02:00

12 KiB
Raw Blame History

type title date author repo
history activity-core scope assessment against INTENT 2026-08-23 codex activity-core

activity-core Scope Assessment Against INTENT — 2026-08-23

Purpose

This assessment records what activity-core can actually do at source head and in its current railiance01 deployment, then compares that surface with INTENT.md. It supersedes the capability conclusions in history/2026-06-16-intent-gap-analysis.md; that earlier note remains useful historical evidence.

Reviewed surfaces:

  • INTENT.md and SCOPE.md
  • src/activity_core/, migrations 0001 through 0008, and the test suite
  • activity-definitions/ and event-types/
  • docs/ops-run-queue.md, docs/execution-queue-boundary.md, and ADRs 001006
  • active workplans ACTIVITY-WP-0031, ACTIVITY-WP-0032, and ACTIVITY-WP-0034
  • live evidence already captured from railiance01 during 2026-08-20 through 2026-08-23

Open-work decision

The repo has open work, but no task that can be advanced independently at the time of this review:

Task State Gate
ACTIVITY-WP-0031-T01 wait OpenRouter account owner must replace a key that the provider reports as User not found
ACTIVITY-WP-0031-T05 wait Named external consumer/routing handoffs
ACTIVITY-WP-0032-T05 wait GLAS-IN-0002 managed-consumer namespace and rein/model runtime/egress contract
ACTIVITY-WP-0034-T02 progress First unassisted weekday 09:15 controlled-source SBOM fire

Another manual Glas pilot or SBOM trigger would weaken rather than improve the acceptance evidence, so this review did not perform either.

Implemented capability

Trigger and definition plane

  • Markdown ActivityDefinitions support five-field cron schedules with timezone and misfire policy, one-off scheduled datetimes, and named events.
  • Temporal Schedule reconciliation creates, updates, pauses, deletes, and smoke-tests schedules with row-level error isolation.
  • NATS JetStream routes registered events. Gitea and GitHub webhook receivers normalize external events into the same envelope and verify HMAC when the corresponding shared secret is configured.
  • Definitions and publisher-declared event types sync from repository files. The FastAPI row CRUD surface is useful for basic definitions but cannot author the complete rules/instructions file contract.

Context and evaluation plane

  • Context adapters cover event payloads, repo-scoping, State Hub, hub-core, reuse-surface, Kaizen metadata, ops inventory, SBOM Nexus, CNPG backup, and controlled shell integrations.
  • Deterministic rules execute through an allowlisted AST evaluator with for_each; they do not use Python exec.
  • Instructions support trusted-field rendering, llm-connect, schema validation, item-granular recovery, deterministic report mode, and multiple report sinks.
  • The workflow preflights a complete governed emission batch so a malformed or missing required profile cannot partially emit earlier tasks.

Output, delivery, and evidence plane

  • IssueSink can target issue-core REST, State Hub progress, or null/dry-run. Production defaults internal findings to State Hub progress.
  • Report/evidence sinks write bounded working-memory, State Hub, and hub-core interaction records with idempotency and queued-write awareness.
  • ops_runs provides idempotent machine-run delivery with concurrent claims, leases, heartbeats, bounded retry/reopen, terminal outcomes, and status/UI visibility without depending on Forgejo or issue-core.
  • Governed ops runs carry a structurally validated, version-pinned Glas profile plus allowlisted organizational refs. Completion/failure retains a normalized evidence envelope and drops prompts, tool output, provider blobs, and secret fields.
  • The activity review CLI merges repository commits, definition fires, ops-run outcomes, and artifacts across named activity-core services.

Declared bounded operations

The workflow currently performs three classes of in-process side effect when a definition explicitly declares them: SBOM Nexus ingest for a frozen oldest-N selection, Forgejo package pruning with protection/dry-run controls, and CNPG backup invocation. These are real execution capabilities, even though they are not general agent or repository-work execution.

Alignment with INTENT

INTENT commitment Assessment Evidence
Answer when Met Temporal cron/one-off schedules, NATS events, webhooks, manual trigger
Answer what Met Sandboxed rules, validated instructions, deterministic reports
Answer where Met Explicit target repos, configurable issue/report sinks, ops-run queue
Markdown as definition Met activity-definitions/, event-types/, sync path
Rules before instructions Met Rules and deterministic reports do not require an LLM
Durable, auditable automation Met in architecture Temporal, PostgreSQL run/spawn/ops-run records, deterministic status surface
Do not own project/campaign state Met Retired /execution/workplan* routes return 410 with owner pointers
Do not own work-item lifecycle Met with qualification Workplan/tracker state stays downstream; machine delivery state is local in ops_runs
Do not execute domain work Partly met General work is external, but three bounded operation classes execute here
Complete instruction audit Partly met Model, prompt hash, validation metadata, definition version, and context are stored; literal rendered prompt is not

Gaps and required clarifications

G1 — The ops-run state machine qualifies the “no task lifecycle” rule

Severity: high documentation/architecture clarity.

ops_runs stores open, claimed, succeeded, failed, and expired, along with lease owner, attempt, result, and timestamps. ADR-005 correctly distinguishes this machine delivery instance from a workplan task or tracker issue, but INTENT.md currently says activity-core does not track task lifecycle without making that distinction.

Recommended closure: amend INTENT to permit bounded operational delivery state while explicitly prohibiting ownership of human/workplan/tracker lifecycle. If that distinction is rejected, the queue must move to an execution-plane owner.

G2 — Bounded side effects qualify the “does not execute work” rule

Severity: medium architecture boundary.

RunActivityWorkflow can invoke SBOM ingest, and context adapters can invoke Forgejo prune and CNPG backup. These operations are bounded, declared, idempotent/evidence-producing, and narrower than general task execution, but they are still execution.

Recommended closure: codify a bounded-operation contract in INTENT/ADR with an explicit allowlist, fixed target selection, dry-run/apply gate where relevant, idempotency, credential routing, timeout/retry ceiling, and mandatory evidence. New operation classes should require an ADR rather than appearing as another generic shell resolver.

G3 — Provider-backed instruction execution is unavailable in production

Severity: high operational; external owner.

The instruction implementation and safe error surface are present, but the live llm-connect path returns a sanitized upstream OpenRouter 401. Deterministic rules/reports still operate; FI, Binky, and daily-triage judgement paths cannot complete until the account owner issues and provisions a valid key.

Owner/evidence: ACTIVITY-WP-0031-T01 and the railiance-platform OpenBao lane.

G4 — Profile-driven execution lacks a successful end-to-end proof

Severity: medium operational; external owner.

Activity-core has proven profile/refs carriage, execution-side resolution, normalized failure evidence, sandbox creation, and teardown. The post-fix pilot fails closed at session_start before provider dispatch because the managed consumer cannot enter the namespace and the rein/model runtime/egress contract is absent. No profiled commit has been produced.

Owner/evidence: GLAS-IN-0002, ACTIVITY-WP-0032-T05, and docs/evidence/ACTIVITY-WP-0032-glas-profile-pilot-2026-08-23.md.

G5 — The compatibility State Hub path remains the production default

Severity: medium integration.

INTENT describes State Hub as a compatibility read model being retired. hub-core context/evidence adapters and explicit retired execution semantics now exist, but production internal task/report progress still defaults to State Hub. The boundary is documented; the actual sink migration is incomplete.

Recommended closure: finish the hub-core/repo-manager consumer contracts, switch definitions deliberately, and retain State Hub only as an explicit compatibility adapter until retirement.

G6 — review_required has no routing destination

Severity: medium product.

Instruction evaluation records review_required in reports and spawn audit metadata, but there is no pending-review queue or named downstream owner that turns it into an operator action.

Recommended closure: either define an explicit downstream review sink/port or rename the field to make its evidence-only semantics unambiguous.

G7 — Audit wording promises more than the safe persistence contract

Severity: medium assurance/documentation.

INTENT says operators can see “what prompt and model produced the output.” The implementation intentionally stores the prompt hash, model, definition version, context snapshot, validation metadata, and bounded response metadata—not the literal rendered prompt or provider response.

Recommended closure: change INTENT to promise reconstructable inputs plus a prompt hash, or define a separately governed encrypted prompt-evidence store. Do not add raw prompts to ops_runs or progress records.

G8 — Legacy and partial administrative surfaces remain

Severity: low drift/documentation.

  • TaskExecutorWorkflow and task_instances remain as disabled compatibility residue and are attractive names for future boundary drift.
  • FastAPI ActivityDefinition CRUD does not expose rules_json or instructions_json; markdown sync is the authoritative complete authoring path.
  • README.md still identifies the repository as repo-seed.

Recommended closure: remove the disabled workflow/table in a migration when compatibility permits, label REST CRUD as basic/legacy or make it round-trip the full contract, and replace the seed README in a small documentation change.

G9 — Controlled-source SBOM promotion awaits natural-run evidence

Severity: low verification.

The implementation, deployment, attended canary, next-batch selection, and read-only production report probe are complete. Acceptance deliberately remains open until the first unassisted weekday schedule fire proves controlled-source provenance for the naturally selected batch.

Owner/evidence: ACTIVITY-WP-0034-T02.

Overall assessment

activity-core materially fulfills the Event Bridge intent: it durably decides when coordination fires, derives what to emit from declared context, and sends the result to an explicit destination. Its main drift is not project-management scope; it is the growth of a narrow execution-delivery plane and a few bounded operations around that bridge. Those additions are useful and controlled, but INTENT must name them precisely so future contributors can distinguish accepted operational state from prohibited task lifecycle and accepted bounded actions from a general executor.

Follow-up workplan

ACTIVITY-WP-0035 addresses the locally actionable and boundary-defining gaps:

  • G1 operational delivery state vs work-item lifecycle;
  • G2 bounded-operation admission and workflow separation;
  • G6 truthful review-worthy output semantics;
  • G7 bounded instruction audit promises; and
  • G8 misleading compatibility surfaces.

G3 remains with ACTIVITY-WP-0031, G4 with ACTIVITY-WP-0032, G5 with the State Hub retirement program after ACTIVITY-WP-0029's completed port slice, and G9 with ACTIVITY-WP-0034's natural-run acceptance gate.