Assistant: codex Assistant-Model: gpt-5.6-sol Assistant-Session: 01a028de-e2c8-7732-8521-46a7fc5db82f
12 KiB
| type | title | date | author | repo |
|---|---|---|---|---|
| history | activity-core scope assessment against INTENT | 2026-08-23 | codex | activity-core |
activity-core Scope Assessment Against INTENT — 2026-08-23
Purpose
This assessment records what activity-core can actually do at source head and
in its current railiance01 deployment, then compares that surface with
INTENT.md. It supersedes the capability conclusions in
history/2026-06-16-intent-gap-analysis.md; that earlier note remains useful
historical evidence.
Reviewed surfaces:
INTENT.mdandSCOPE.mdsrc/activity_core/, migrations0001through0008, and the test suiteactivity-definitions/andevent-types/docs/ops-run-queue.md,docs/execution-queue-boundary.md, and ADRs 001–006- active workplans ACTIVITY-WP-0031, ACTIVITY-WP-0032, and ACTIVITY-WP-0034
- live evidence already captured from railiance01 during 2026-08-20 through 2026-08-23
Open-work decision
The repo has open work, but no task that can be advanced independently at the time of this review:
| Task | State | Gate |
|---|---|---|
| ACTIVITY-WP-0031-T01 | wait | OpenRouter account owner must replace a key that the provider reports as User not found |
| ACTIVITY-WP-0031-T05 | wait | Named external consumer/routing handoffs |
| ACTIVITY-WP-0032-T05 | wait | GLAS-IN-0002 managed-consumer namespace and rein/model runtime/egress contract |
| ACTIVITY-WP-0034-T02 | progress | First unassisted weekday 09:15 controlled-source SBOM fire |
Another manual Glas pilot or SBOM trigger would weaken rather than improve the acceptance evidence, so this review did not perform either.
Implemented capability
Trigger and definition plane
- Markdown ActivityDefinitions support five-field cron schedules with timezone and misfire policy, one-off scheduled datetimes, and named events.
- Temporal Schedule reconciliation creates, updates, pauses, deletes, and smoke-tests schedules with row-level error isolation.
- NATS JetStream routes registered events. Gitea and GitHub webhook receivers normalize external events into the same envelope and verify HMAC when the corresponding shared secret is configured.
- Definitions and publisher-declared event types sync from repository files. The FastAPI row CRUD surface is useful for basic definitions but cannot author the complete rules/instructions file contract.
Context and evaluation plane
- Context adapters cover event payloads, repo-scoping, State Hub, hub-core, reuse-surface, Kaizen metadata, ops inventory, SBOM Nexus, CNPG backup, and controlled shell integrations.
- Deterministic rules execute through an allowlisted AST evaluator with
for_each; they do not use Pythonexec. - Instructions support trusted-field rendering, llm-connect, schema validation, item-granular recovery, deterministic report mode, and multiple report sinks.
- The workflow preflights a complete governed emission batch so a malformed or missing required profile cannot partially emit earlier tasks.
Output, delivery, and evidence plane
IssueSinkcan target issue-core REST, State Hub progress, or null/dry-run. Production defaults internal findings to State Hub progress.- Report/evidence sinks write bounded working-memory, State Hub, and hub-core interaction records with idempotency and queued-write awareness.
ops_runsprovides idempotent machine-run delivery with concurrent claims, leases, heartbeats, bounded retry/reopen, terminal outcomes, and status/UI visibility without depending on Forgejo or issue-core.- Governed ops runs carry a structurally validated, version-pinned Glas profile plus allowlisted organizational refs. Completion/failure retains a normalized evidence envelope and drops prompts, tool output, provider blobs, and secret fields.
- The
activityreview CLI merges repository commits, definition fires, ops-run outcomes, and artifacts across named activity-core services.
Declared bounded operations
The workflow currently performs three classes of in-process side effect when a definition explicitly declares them: SBOM Nexus ingest for a frozen oldest-N selection, Forgejo package pruning with protection/dry-run controls, and CNPG backup invocation. These are real execution capabilities, even though they are not general agent or repository-work execution.
Alignment with INTENT
| INTENT commitment | Assessment | Evidence |
|---|---|---|
| Answer when | Met | Temporal cron/one-off schedules, NATS events, webhooks, manual trigger |
| Answer what | Met | Sandboxed rules, validated instructions, deterministic reports |
| Answer where | Met | Explicit target repos, configurable issue/report sinks, ops-run queue |
| Markdown as definition | Met | activity-definitions/, event-types/, sync path |
| Rules before instructions | Met | Rules and deterministic reports do not require an LLM |
| Durable, auditable automation | Met in architecture | Temporal, PostgreSQL run/spawn/ops-run records, deterministic status surface |
| Do not own project/campaign state | Met | Retired /execution/workplan* routes return 410 with owner pointers |
| Do not own work-item lifecycle | Met with qualification | Workplan/tracker state stays downstream; machine delivery state is local in ops_runs |
| Do not execute domain work | Partly met | General work is external, but three bounded operation classes execute here |
| Complete instruction audit | Partly met | Model, prompt hash, validation metadata, definition version, and context are stored; literal rendered prompt is not |
Gaps and required clarifications
G1 — The ops-run state machine qualifies the “no task lifecycle” rule
Severity: high documentation/architecture clarity.
ops_runs stores open, claimed, succeeded, failed, and expired, along
with lease owner, attempt, result, and timestamps. ADR-005 correctly distinguishes
this machine delivery instance from a workplan task or tracker issue, but
INTENT.md currently says activity-core does not track task lifecycle without
making that distinction.
Recommended closure: amend INTENT to permit bounded operational delivery state while explicitly prohibiting ownership of human/workplan/tracker lifecycle. If that distinction is rejected, the queue must move to an execution-plane owner.
G2 — Bounded side effects qualify the “does not execute work” rule
Severity: medium architecture boundary.
RunActivityWorkflow can invoke SBOM ingest, and context adapters can invoke
Forgejo prune and CNPG backup. These operations are bounded, declared,
idempotent/evidence-producing, and narrower than general task execution, but
they are still execution.
Recommended closure: codify a bounded-operation contract in INTENT/ADR with an explicit allowlist, fixed target selection, dry-run/apply gate where relevant, idempotency, credential routing, timeout/retry ceiling, and mandatory evidence. New operation classes should require an ADR rather than appearing as another generic shell resolver.
G3 — Provider-backed instruction execution is unavailable in production
Severity: high operational; external owner.
The instruction implementation and safe error surface are present, but the live llm-connect path returns a sanitized upstream OpenRouter 401. Deterministic rules/reports still operate; FI, Binky, and daily-triage judgement paths cannot complete until the account owner issues and provisions a valid key.
Owner/evidence: ACTIVITY-WP-0031-T01 and the railiance-platform OpenBao lane.
G4 — Profile-driven execution lacks a successful end-to-end proof
Severity: medium operational; external owner.
Activity-core has proven profile/refs carriage, execution-side resolution,
normalized failure evidence, sandbox creation, and teardown. The post-fix pilot
fails closed at session_start before provider dispatch because the managed
consumer cannot enter the namespace and the rein/model runtime/egress contract
is absent. No profiled commit has been produced.
Owner/evidence: GLAS-IN-0002, ACTIVITY-WP-0032-T05, and
docs/evidence/ACTIVITY-WP-0032-glas-profile-pilot-2026-08-23.md.
G5 — The compatibility State Hub path remains the production default
Severity: medium integration.
INTENT describes State Hub as a compatibility read model being retired. hub-core context/evidence adapters and explicit retired execution semantics now exist, but production internal task/report progress still defaults to State Hub. The boundary is documented; the actual sink migration is incomplete.
Recommended closure: finish the hub-core/repo-manager consumer contracts, switch definitions deliberately, and retain State Hub only as an explicit compatibility adapter until retirement.
G6 — review_required has no routing destination
Severity: medium product.
Instruction evaluation records review_required in reports and spawn audit
metadata, but there is no pending-review queue or named downstream owner that
turns it into an operator action.
Recommended closure: either define an explicit downstream review sink/port or rename the field to make its evidence-only semantics unambiguous.
G7 — Audit wording promises more than the safe persistence contract
Severity: medium assurance/documentation.
INTENT says operators can see “what prompt and model produced the output.” The implementation intentionally stores the prompt hash, model, definition version, context snapshot, validation metadata, and bounded response metadata—not the literal rendered prompt or provider response.
Recommended closure: change INTENT to promise reconstructable inputs plus a
prompt hash, or define a separately governed encrypted prompt-evidence store.
Do not add raw prompts to ops_runs or progress records.
G8 — Legacy and partial administrative surfaces remain
Severity: low drift/documentation.
TaskExecutorWorkflowandtask_instancesremain as disabled compatibility residue and are attractive names for future boundary drift.- FastAPI ActivityDefinition CRUD does not expose
rules_jsonorinstructions_json; markdown sync is the authoritative complete authoring path. README.mdstill identifies the repository asrepo-seed.
Recommended closure: remove the disabled workflow/table in a migration when compatibility permits, label REST CRUD as basic/legacy or make it round-trip the full contract, and replace the seed README in a small documentation change.
G9 — Controlled-source SBOM promotion awaits natural-run evidence
Severity: low verification.
The implementation, deployment, attended canary, next-batch selection, and read-only production report probe are complete. Acceptance deliberately remains open until the first unassisted weekday schedule fire proves controlled-source provenance for the naturally selected batch.
Owner/evidence: ACTIVITY-WP-0034-T02.
Overall assessment
activity-core materially fulfills the Event Bridge intent: it durably decides when coordination fires, derives what to emit from declared context, and sends the result to an explicit destination. Its main drift is not project-management scope; it is the growth of a narrow execution-delivery plane and a few bounded operations around that bridge. Those additions are useful and controlled, but INTENT must name them precisely so future contributors can distinguish accepted operational state from prohibited task lifecycle and accepted bounded actions from a general executor.
Follow-up workplan
ACTIVITY-WP-0035 addresses the locally actionable and boundary-defining gaps:
- G1 operational delivery state vs work-item lifecycle;
- G2 bounded-operation admission and workflow separation;
- G6 truthful review-worthy output semantics;
- G7 bounded instruction audit promises; and
- G8 misleading compatibility surfaces.
G3 remains with ACTIVITY-WP-0031, G4 with ACTIVITY-WP-0032, G5 with the State Hub retirement program after ACTIVITY-WP-0029's completed port slice, and G9 with ACTIVITY-WP-0034's natural-run acceptance gate.