From bbbc5819c58c8894c4e69fde35c558595950a387 Mon Sep 17 00:00:00 2001 From: tegwick Date: Tue, 21 Jul 2026 23:52:08 +0200 Subject: [PATCH] docs: restructure SCOPE.md as shipped inventory vs product boundary Rewrite SCOPE to separate what issue-core provides today (models, backends, CLI, REST, sync, deploy) from in-scope growth and out-of-scope fleet work. Align package __init__ description with the connector role. --- SCOPE.md | 413 +++++++++++++++++++++++++---------------- issue_core/__init__.py | 26 +-- 2 files changed, 257 insertions(+), 182 deletions(-) diff --git a/SCOPE.md b/SCOPE.md index 6f4938d..d963a4f 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -1,185 +1,235 @@ # SCOPE — issue-core -Concrete in-scope / out-of-scope decisions for issue-core. Paired with `INTENT.md`, -which explains *why*; this file states *what* and *what not*. +Concrete **what this repo provides**, **what it owns**, and **what it does not**. +Paired with `INTENT.md` (*why*) and `docs/uuid-external-id-mapping.md` (*mapping design*). -**Fleet framing (2026-07-20):** issue-core is a **connector** to external -issue trackers and the home of the **UUID ↔ external-id mapping** concept. -Work records originate as repo files (ADR-001 / work-record canon). See -`INTENT.md` and `docs/uuid-external-id-mapping.md`. +**Fleet role (2026-07-20):** issue-core is a **connector** to external issue +trackers — not the origin of fleet work records (ADR-001 / work-record canon). +Work records live in repo files and state-hub; this package projects to and +operates on tracker backends when that is intentional. -## In scope +**Package:** `issue-core` **v0.2.1** · Python ≥ 3.8 · entry points `issue`, +`issue-core` (alias), `issue-tracker` (legacy deprecation hint). -### External-issue CRUD across backends +--- -- **Create** issues with title, description, labels, priority, type, milestone, - assignee, due date — when a tracker projection or direct tracker use is - intentional. -- **Read** individual issues and lists with filters (state, labels, priority, - assignee, text search). -- **Update** any mutable field on existing issues. -- **Close / reopen** with state transitions enforced at the domain layer. -- **Delete** where the backend allows (local SQLite; soft-archive elsewhere). -- **Comment** threads on issues. +## 1. What this repo provides today (shipped) -### Backend abstraction +Honest inventory of implemented surface area. Prefer this section over +aspirational lists when deciding “does issue-core already do X?” -- A single `IssueBackend` ABC contract that every backend implements. -- `BackendCapabilities` declares which optional features a backend supports - (bulk update, search, milestones, etc.). -- A `BackendFactory` registry that maps config to backend instances. +### 1.1 Domain model (`issue_core.core.models`) -### Backends shipped today - -- **Local SQLite** — offline cache / source of truth for disconnected work. -- **Gitea** — REST API integration (Forgejo-compatible in deployment). - -### Backends planned - -- **GitHub** — Issues + PRs. -- **GitLab** — Issues. -- **JIRA** — issues with story-points. - -### UUID ↔ external-id mapping (target shape) - -Primary *fleet* responsibility of issue-core going forward — design only in -this repo until stage-3 work-record architecture lands; see -`docs/uuid-external-id-mapping.md`. - -| Concept | Meaning | +| Concept | Notes | | --- | --- | -| Internal id | Work-record UUID (UUIDv7, hub-assigned / written back to source file as `state_hub_*_id`-style fields) | -| External id | Backend issue id / number + backend identity (`gitea`, `github`, …) | -| Mapping row | Durable association: which work record projects to which external issue | -| Direction | Create projection outward; optional two-way sync of status/comments at the boundary | +| `Issue` | id, number, title, description, state, timestamps, labels, assignees, milestone, comments, backend_id/type, `sync_metadata` | +| `IssueState` | `open`, `closed`, `in_progress`, `blocked` (Gitea maps non-closed → open on the wire) | +| `Label` / priority / type | Priority and type are primarily **label conventions** (`priority:high`, `bug`, …), not separate backend columns everywhere | +| `User`, `Milestone`, `Comment` | First-class models on the backend interface | +| `Priority`, `IssueType` enums | Helpers for label-derived classification | -Named here so scope and design agree before implementation. +### 1.2 Backend plugin architecture (`issue_core.core.interfaces`) -### Operator / automation surfaces +| Piece | Shipped | +| --- | --- | +| `IssueBackend` ABC | Full CRUD-ish contract: issues, labels, users, milestones, comments; bulk update hook | +| `BackendCapabilities` | Declares milestones, assignees, comments, labels, search, bulk, webhooks, limits | +| `IssueFilter` | state, labels, assignee, milestone, search text, limit, etc. | +| `SyncableBackend` | prepare/finalize, modified-since, conflict resolve hooks | +| `BackendFactory` | Register + create by type string | +| Python import use | e.g. `from issue_core.backends.gitea import GiteaBackend` | -- **CLI** (`issue` / `issue-core`) for humans and agents on a shell — direct - backend use. -- **REST** (`POST /issues/`) for **intentional** external-issue creation by - authenticated clients. Remains useful; is **not** the primary fleet path for - originating work. -- **REST worker surface** (`GET/PATCH /issues/`) for executors that operate on - external issues when a tracker is in the loop: list, claim - (`in_progress` + assignee), close. -- **NATS subscriber** (design stub only — see `docs/nats-task-ingestion.md`). - Any NATS path must respect the same rule: external projection only when - explicitly configured, never silent origin of fleet work. +### 1.3 Backends implemented -### Synchronization +| Backend | Module | Role | +| --- | --- | --- | +| **Local SQLite** | `issue_core.backends.local` | Offline store; hard delete supported; bulk ops; sync-capable | +| **Gitea** | `issue_core.backends.gitea` | Remote REST; **Forgejo-compatible** in railiance01 deploy; no true delete (close-as-delete); rate-limit aware; sync-capable | -- Local SQLite ↔ remote backends, bidirectional. -- `get_issues_modified_since()` on `SyncableBackend` for incremental sync. -- Conflict resolution via `SyncableBackend.resolve_sync_conflict()`. -- Sync metadata (last-synced timestamps, remote IDs) stored on `Issue.sync_metadata`. +**Not implemented as code:** GitHub, GitLab, Jira backends (optional deps may be +declared in `pyproject.toml`; no backend package under `issue_core/backends/`). -### State Hub integration +### 1.4 CLI (`issue` / `issue-core`) -- Registered as a custodian-domain / infotech repo. -- May emit `add_progress_event()` on significant *connector* lifecycle moments - (e.g. projection created, mapping updated) when wired. -- Does **not** replace work-record indexing; state-hub remains the fleet read - model for work records. +| Group | Commands (shipped) | +| --- | --- | +| Issues | `list`, `show`, `create`, `edit`, `close`, `reopen`, `comment` | +| Backend | `backend list`, `add`, `remove`, `test`, `set-default` | +| Sync | `sync status`, `pull`, `push`, `bidirectional` | +| Server | `serve` — FastAPI process (requires `[api]` extra) | -## Out of scope +- JSON-friendly output for agents (`--format=json` on list/show paths). +- Backend configs: `~/.config/issue-core/` (default backend + named configs). +- Gitea token typically from env (`GITEA_API_TOKEN` / config); never commit secrets. -### Origin of fleet work +### 1.5 REST API (optional install: `pip install 'issue-core[api]'`) + +Auth: shared secret `ISSUE_CORE_API_KEY` via `Authorization: Bearer …` or +`X-API-Key`. + +| Method | Path | Purpose | +| --- | --- | --- | +| `GET` | `/healthz` | Liveness | +| `POST` | `/issues/` | Create issue from **TaskSpec** (intentional external emit) | +| `GET` | `/issues/` | List (filter: state, label, limit) | +| `GET` | `/issues/{issue_id}` | Get one (id or numeric fallback) | +| `PATCH` | `/issues/{issue_id}` | Claim / update state and assignee | + +**Routing today:** single **default backend** from CLI config. `target_repo` is +stored in `sync_metadata.ingestion` and labels (`repo:…`); it does **not** yet +select a per-repo backend. + +**Ingestion metadata stored on create:** `target_repo`, `source_type`, +`source_id`, `triggering_event_id`, `activity_definition_id`, `ingested_at`, +optional `due_at` derived from `due_in_days`. + +### 1.6 Synchronization (CLI) + +- Bidirectional pull/push between configured backends (e.g. Gitea ↔ local). +- Uses `get_issues_modified_since` / conflict hooks where backends implement them. +- Conflict handling is basic (operator / force flags) — not a full merge engine. + +### 1.7 Packaging, registry, deploy + +| Artifact | Provides | +| --- | --- | +| PyPI package | Built wheel/sdist; Makefile targets publish to Coulomb Gitea/Forgejo registry | +| Docker + `k8s/railiance/` | GitOps-style deploy on railiance01 (deployment, service, configmap, ExternalSecret) | +| `docs/argocd-gitops.md` | Operator runbook for that deploy | +| `docs/package-release.md` | Release notes for packaging | +| `registry/` + `CAPABILITY-issue-tracking.yaml` | Reuse-surface / capability federation metadata | +| Tests | `tests/` unit coverage for models, CLI, backends, API ingest/query | +| Examples | `examples/agents/` programmatic patterns | + +### 1.8 What is *not* shipped in the runtime (despite docs/plans) + +| Item | Status | +| --- | --- | +| Work-record UUID ↔ external-id **mapping store/API** | Design only — `docs/uuid-external-id-mapping.md` | +| NATS subscriber | Design stub — `docs/nats-task-ingestion.md` | +| Runtime calls to state-hub (`add_progress_event`, etc.) | **Not in `issue_core` package**; repo participates as a normal ADR-001 workplan host only | +| Auto-detect backend from git remote | Roadmap / Makefile hints; not a reliable product path yet | +| MCP server | Not shipped | +| GitHub / GitLab / Jira backends | Planned only | +| Per-`target_repo` backend routing on REST | Planned only | +| Issue claiming locks beyond assignee + state | Convention only | +| First-class due-date field on `Issue` | `due_in_days` → ingestion metadata only | + +--- + +## 2. Product boundary — in scope + +Things this repo **owns** and may grow, consistent with the connector role. + +### 2.1 External-issue operations + +- Create / read / update / close / reopen / comment on issues in configured + backends when tracker use is intentional. +- List and filter (state, labels, assignee, milestone, text search as backends allow). +- Label, assignee, and milestone management through the backend interface. +- Delete where the backend allows (SQLite hard delete; Gitea effectively close). + +### 2.2 Connector / mapping (owned direction) + +- Durable **work-record UUID ↔ (backend, external issue id)** mapping + (design agreed; implementation stage-3 work-record architecture). +- Optional two-way **boundary sync** when an integration is switched on; + **zero load** when not configured. +- Keep `triggering_event_id` as emitter lineage; **extend** with + `work_record_uuid` rather than overloading it (see mapping design). + +### 2.3 Surfaces + +- CLI for humans and agents on a shell. +- REST for intentional automation and worker claim/list/close. +- Python library for embedding. +- Optional future NATS consumer **only** for intentional external projection + (never silent fleet work origin). + +### 2.4 Additional backends (when needed) + +- GitHub, GitLab, Jira (or other trackers) as further `IssueBackend` plugins — + in scope as product growth, not as current deliverables. + +### 2.5 Ops for this service + +- Config, packaging, container image, k8s/GitOps for running the connector + where the org needs it (e.g. railiance01). +- Credential **routing** via ops-warden / OpenBao conventions — this repo does + not store secrets in git. + +--- + +## 3. Out of scope + +### 3.1 Origin of fleet work Creating workplans, tasks, intake items, decisions, or engagements. Those -originate as repo files (or schema-valid blocks in context) and are indexed by +originate as repo files (or schema-valid blocks) and are indexed by `fix-consistency` / state-hub. issue-core may **project** an existing work record to a tracker; it does not birth the work record. -### Default sink for internal automation findings +### 3.2 Fleet coordination substrate -activity-core (and similar) must not treat `POST /issues/` as the always-on -default for every matched rule. That product decision lives in the emitter; -issue-core will keep accepting authenticated POSTs but will not redefine itself -as the landing zone. Follow-up: activity-core IssueSink policy (filed from -ISSUE-WP-0004-T05). +Claiming and executing **fleet** work happens on work records + harness / +state-hub views — not on Forgejo issue lists by default. issue-core is not the +org’s task board. -### Project management +### 3.3 Default sink for internal automation findings -Phases, campaigns, milestones-as-plans, dependency graphs between tasks, -gantt-style scheduling, OKR linkage. **That is workplan / project tooling.** -issue-core operates on individual external issues. A milestone field exists for -flat grouping, not for multi-stage plans. +activity-core (and peers) must not treat `POST /issues/` as the always-on +default for every matched rule. Emitters own that policy; issue-core keeps +accepting authenticated POSTs without advertising itself as a landing zone. +Follow-up: `activity-core` **ACTIVITY-WP-0022** (from ISSUE-WP-0004-T05). -### Spawn audit trail +### 3.4 Project management -When an emitter files an issue here, the *spawn event* (who fired, which rule, -which triggering event) is recorded by the **emitter** (e.g. activity-core's -`task_spawn_log`). issue-core stores the resulting issue and traceability -fields (`triggering_event_id` today; work-record UUID in the mapping design) — -nothing more. +Phases, campaigns, dependency graphs across work, Gantt scheduling, OKRs. +Workplan / project tooling owns that. Milestone on an issue is flat grouping +only. -Symmetrically: issue-core does not push fleet work-record status updates back -to activity-core. Lifecycle of external issues stays here; lifecycle of work -records stays in repo files + hub. +### 3.5 Spawn audit trail -### Event bus / message broker +Emitters record who/what spawned an external issue (e.g. activity-core +`task_spawn_log`). issue-core stores the issue plus traceability fields +(`triggering_event_id` today; work-record UUID when mapping lands). -Inter-service communication runs on NATS managed elsewhere. issue-core consumes -specific subjects (future) and exposes a REST surface; it does not relay events -between other services. +### 3.6 Event bus, notifications, workflow engine, IdP, UI -### Notifications +| Concern | Owner elsewhere | +| --- | --- | +| NATS / inter-service relay | Platform NATS ops — issue-core only consumes/serves its boundary | +| “Your task changed” notifications | UI / digest / chatbot services | +| Approval chains, SLA timers, multi-step workflows | Workflow tooling — states stay small | +| User directory / OIDC / MFA | key-cape / Keycloak / backend IdPs | +| First-party web UI for issue-core | Not planned; use backend native UIs | -Telling a human "your task changed" is the job of the relevant UI, digest, -chatbot, or notification service — not issue-core. +--- -### Workflow / approval engine +## 4. Integration boundaries -State machine is intentionally small (OPEN, CLOSED, IN_PROGRESS, BLOCKED). -Conditional routing, approval chains, multi-step workflows, SLA timers — all -out of scope. +### 4.1 Fleet primary path (work records) -### UI +| Step | Owner | +| --- | --- | +| Author / promote work record | Repo files + tools (ADR-001 / work-record canon) | +| Index + UUID write-back | state-hub `fix-consistency` | +| Claim / execute | Humans, agents, harness on work records | +| Optional external projection | **issue-core** (CRUD today; mapping when implemented) | -issue-core is CLI + REST first. Each backend brings its own native UI -(Gitea web, GitHub web, etc.) and that is enough. A web UI for issue-core -itself is not on the roadmap. +### 4.2 Clients of this repo -### Identity / access management - -Authentication relies on backend credentials (Gitea tokens, GitHub tokens) and -on a service-level API key for the REST endpoint. issue-core is not a -user directory. - -## Integration boundaries - -### Fleet primary path (work records) - -| Step | Owner | Notes | +| Client | Transport | Role today | | --- | --- | --- | -| Author / promote work record | Repo files + tools | ADR-001 / work-record canon | -| Index + UUID write-back | state-hub `fix-consistency` | Hub is read model | -| Claim / execute | Humans, agents, harness | On work records, not on Forgejo by default | -| Optional external projection | **issue-core** | Mapping table + backend CRUD | +| Human / agent shell | CLI | Direct tracker admin and ops | +| Library consumers | Python API | Same backends without shell | +| Workers (e.g. harness) | REST GET/PATCH | List / claim / close external issues when in the loop | +| activity-core IssueSink | REST POST | **Optional, intentional** external issues only — not default for internal findings | +| Future mapping clients | REST/CLI TBD | Project/link work-record UUID ↔ external id | -### Surfaces into issue-core +### 4.3 TaskSpec contract (`POST /issues/`) -| Client | Transport | Payload | Role | -| --- | --- | --- | --- | -| Human CLI | local process | CLI args | Direct tracker use / admin | -| Agents | REST or CLI | issue fields or CLI args | Tracker ops when in the loop | -| activity-core IssueSink | REST `POST /issues/` | `TaskSpec` | **Optional, intentional** external issues only — not default for internal findings | -| Future mapping API | REST/CLI (TBD) | work_record_uuid + project request | Primary fleet connector path once designed | - -### Downstream consumers - -| Consumer | Mechanism | Notes | -| --- | --- | --- | -| Humans / external collab | backend UI, `issue list` | Third-party tracker collaboration | -| state-hub | work-record index; optional progress events | Fleet orientation | -| Backend remote (e.g. Gitea) | direct backend write | Storage / projection layer | - -### `TaskSpec` payload (activity-core IssueSink — retained contract) - -Still accepted for backward compatibility and intentional emits: +Retained for intentional emits and backward compatibility: ```json { @@ -196,19 +246,9 @@ Still accepted for backward compatibility and intentional emits: } ``` -`triggering_event_id` is accepted as a non-empty string. Event-driven -emissions should send the upstream activity event UUID. Scheduled or cron -emissions that do not have a concrete event row may send a stable source key -such as `scheduled`; issue-core stores the value verbatim in ingestion -metadata for traceability. - -**Relation to mapping design:** `triggering_event_id` is emitter-side -traceability, not a work-record UUID. The mapping surface will add an explicit -work-record identity field (and durable mapping rows). See -`docs/uuid-external-id-mapping.md` — extend rather than overload -`triggering_event_id`. - -### `POST /issues/` response +- `triggering_event_id`: non-empty string; activity event UUID or stable key + such as `scheduled`. Stored in ingestion metadata — **not** a work-record UUID. +- Response: ```json { @@ -218,17 +258,60 @@ work-record identity field (and durable mapping rows). See } ``` -The `issue_id` is the external-issue back-reference an intentional emitter may -store (e.g. activity-core `task_spawn_log`). It is owned by issue-core's -backend; it is not a work-record UUID. +`issue_id` is the **backend** issue identity for the emitter’s log; not a +work-record UUID. (`github` appears in the response enum for forward +compatibility; no GitHub backend is shipped yet.) -## See also +### 4.4 Credential routing (operators / agents) -- `INTENT.md` — why issue-core exists and the connector pivot. -- `ROADMAP.md` — feature trajectory. -- `docs/uuid-external-id-mapping.md` — mapping design (not yet implemented). -- `workplans/ISSUE-WP-0004-align-with-work-record-canon.md` — alignment work. -- `the-custodian/canon/standards/work-record-types_v0.1.md` — work-record kinds; - task kind notes that issue-core issues are external projections only. -- activity-core `docs/adr/adr-001-event-bridge-architecture.md` — IssueSink - contract (emitter side; policy follow-up separate). +| Need | Owner | +| --- | --- | +| Gitea/Forgejo API token for backend | OpenBao / operator path (`warden route`) | +| `ISSUE_CORE_API_KEY` for REST | Shared secret via deploy secrets (e.g. ExternalSecret) | +| SSH certs | ops-warden only | + +Do not message ops-warden for API keys; do not commit secrets. + +--- + +## 5. Stack and layout (quick reference) + +``` +issue_core/ + core/ # models, IssueBackend ABC, factory + backends/ + local/ # SQLite + gitea/ # Gitea/Forgejo REST + cli/ # Click: issue, backend, sync, serve + api/ # FastAPI: ingest + query (+ auth, schemas) +tests/ +docs/ # gitops, nats design, mapping design, release +k8s/railiance/ # deploy manifests +workplans/ # ADR-001 work items for this repo +``` + +| Concern | Choice | +| --- | --- | +| Language | Python 3.8+ | +| CLI | Click | +| HTTP | FastAPI + Pydantic v2 + uvicorn (`[api]` extra) | +| HTTP client (Gitea) | requests | +| Local store | SQLite | +| Tests | pytest | + +--- + +## 6. See also + +- `INTENT.md` — why the connector role exists and the 2026-07-20 pivot +- `README.md` — operator quick start +- `ROADMAP.md` — phased growth (auto-config, mapping implementation, backends) +- `docs/uuid-external-id-mapping.md` — mapping design (not implemented) +- `docs/nats-task-ingestion.md` — NATS design stub +- `docs/argocd-gitops.md` — railiance01 deploy +- `AGENT_INTEGRATION.md` — library patterns for tracker ops +- `workplans/ISSUE-WP-0004-align-with-work-record-canon.md` — framing alignment +- `the-custodian/canon/standards/work-record-types_v0.1.md` — work-record kinds + (issue-core issues = external projections only) +- `the-custodian/research/WorkOrchestrationArchitectureDraft.md` §4.2 +- activity-core IssueSink / **ACTIVITY-WP-0022** — emitter-side default policy diff --git a/issue_core/__init__.py b/issue_core/__init__.py index 9778f34..911fd17 100644 --- a/issue_core/__init__.py +++ b/issue_core/__init__.py @@ -1,24 +1,16 @@ """ -issue-core — Authoritative Task Lifecycle Manager +issue-core — external issue-tracker connector -The single observable place in the Coulomb org where tasks land — -regardless of whether they were created by a human, by activity-core, -or by an agent. Backend-agnostic via a plugin architecture. +Backend-agnostic CLI, library, and optional REST surface for operating on +third-party issue trackers (Gitea/Forgejo, local SQLite). Not the origin of +fleet work records (see INTENT.md / SCOPE.md). -Features: -- Unified issue model across all backends -- Plugin-based backend architecture -- Local SQLite backend for offline work -- Bidirectional synchronization -- CLI-first interface -- REST ingestion endpoint for activity-core's IssueSink - -Supported Backends: -- Local SQLite (offline/standalone) -- Gitea (GitHub-compatible API) -- Future: GitHub, GitLab, JIRA, Redmine +Shipped: +- Unified issue model and IssueBackend plugin architecture +- Local SQLite + Gitea backends; bidirectional sync CLI +- CLI (`issue` / `issue-core`) and optional FastAPI REST (`issue serve`) """ __version__ = "0.2.1" __author__ = "Coulomb / MarkiTect Project" -__description__ = "Authoritative task lifecycle manager with plugin architecture" +__description__ = "External issue-tracker connector with plugin backends"