issue-core/SCOPE.md
tegwick f88e9a28fe
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 2s
Build and Publish Container Image / build-and-push (push) Successful in 1m22s
feat: implement ISSUE-WP-0005 connector mapping and scope alignment
Add MappingService (work_record_uuid ↔ external id), project/map CLI with
outward push-status, optional TaskSpec work_record fields, boundary-sync
policy, and packaging/capability framing cleanup. Mark WP-0005 finished.
2026-07-22 00:25:06 +02:00

340 lines
14 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# SCOPE — issue-core
Concrete **what this repo provides**, **what it owns**, and **what it does not**.
Paired with `INTENT.md` (*why*) and `docs/uuid-external-id-mapping.md` (*mapping design*).
**Fleet role (2026-07-20):** issue-core is a **connector** to external issue
trackers — not the origin of fleet work records (ADR-001 / work-record canon).
Work records live in repo files and state-hub; this package projects to and
operates on tracker backends when that is intentional.
**Package:** `issue-core` **v0.2.1** · Python ≥ 3.8 · entry points `issue`,
`issue-core` (alias), `issue-tracker` (legacy deprecation hint).
---
## 1. What this repo provides today (shipped)
Honest inventory of implemented surface area. Prefer this section over
aspirational lists when deciding “does issue-core already do X?”
### 1.1 Domain model (`issue_core.core.models`)
| Concept | Notes |
| --- | --- |
| `Issue` | id, number, title, description, state, timestamps, labels, assignees, milestone, comments, backend_id/type, `sync_metadata` |
| `IssueState` | `open`, `closed`, `in_progress`, `blocked` (Gitea maps non-closed → open on the wire) |
| `Label` / priority / type | Priority and type are primarily **label conventions** (`priority:high`, `bug`, …), not separate backend columns everywhere |
| `User`, `Milestone`, `Comment` | First-class models on the backend interface |
| `Priority`, `IssueType` enums | Helpers for label-derived classification |
### 1.2 Backend plugin architecture (`issue_core.core.interfaces`)
| Piece | Shipped |
| --- | --- |
| `IssueBackend` ABC | Full CRUD-ish contract: issues, labels, users, milestones, comments; bulk update hook |
| `BackendCapabilities` | Declares milestones, assignees, comments, labels, search, bulk, webhooks, limits |
| `IssueFilter` | state, labels, assignee, milestone, search text, limit, etc. |
| `SyncableBackend` | prepare/finalize, modified-since, conflict resolve hooks |
| `BackendFactory` | Register + create by type string |
| Python import use | e.g. `from issue_core.backends.gitea import GiteaBackend` |
### 1.3 Backends implemented
| Backend | Module | Role |
| --- | --- | --- |
| **Local SQLite** | `issue_core.backends.local` | Offline store; hard delete supported; bulk ops; sync-capable |
| **Gitea** | `issue_core.backends.gitea` | Remote REST; **Forgejo-compatible** in railiance01 deploy; no true delete (close-as-delete); rate-limit aware; sync-capable |
**Not implemented as code:** GitHub, GitLab, Jira backends (optional deps may be
declared in `pyproject.toml`; no backend package under `issue_core/backends/`).
### 1.4 CLI (`issue` / `issue-core`)
| Group | Commands (shipped) |
| --- | --- |
| Issues | `list`, `show`, `create`, `edit`, `close`, `reopen`, `comment` |
| Project / map | `project`, `map link`, `map show`, `map detach`, `map list`, `map push-status` |
| Backend | `backend list`, `add`, `remove`, `test`, `set-default` |
| Sync | `sync status`, `pull`, `push`, `bidirectional` |
| Server | `serve` — FastAPI process (requires `[api]` extra) |
- JSON-friendly output for agents (`--format=json` on list/show paths).
- Backend configs: `~/.config/issue-tracker/` (default backend + named configs).
- Mapping store: `~/.config/issue-tracker/mappings.db` (independent of CRUD backend).
- Gitea token typically from env (`GITEA_API_TOKEN` / config); never commit secrets.
### 1.4b Work-record mapping (shipped v0.2.x)
| Piece | Notes |
| --- | --- |
| `MappingService` | `issue_core/core/mapping.py` — SQLite `work_record_issue_map` |
| Keys | Work-record **UUIDv7**`(backend, external_id)`; optional canonical id denorm |
| Uniqueness | One active mapping per `(uuid, backend)` and per `(backend, external_id)` |
| Outward status | `map push-status` uses `docs/boundary-sync-and-status-mapping.md` |
| TaskSpec | Optional `work_record_uuid` / `work_record_id` / `work_record_kind` on `POST /issues/` upserts mapping; idempotent reuse |
### 1.5 REST API (optional install: `pip install 'issue-core[api]'`)
Auth: shared secret `ISSUE_CORE_API_KEY` via `Authorization: Bearer …` or
`X-API-Key`.
| Method | Path | Purpose |
| --- | --- | --- |
| `GET` | `/healthz` | Liveness |
| `POST` | `/issues/` | Create issue from **TaskSpec** (intentional external emit) |
| `GET` | `/issues/` | List (filter: state, label, limit) |
| `GET` | `/issues/{issue_id}` | Get one (id or numeric fallback) |
| `PATCH` | `/issues/{issue_id}` | Claim / update state and assignee |
**Routing today:** single **default backend** from CLI config. `target_repo` is
stored in `sync_metadata.ingestion` and labels (`repo:…`); it does **not** yet
select a per-repo backend.
**Ingestion metadata stored on create:** `target_repo`, `source_type`,
`source_id`, `triggering_event_id`, `activity_definition_id`, `ingested_at`,
optional `due_at` derived from `due_in_days`.
### 1.6 Synchronization (CLI)
- Bidirectional pull/push between configured backends (e.g. Gitea ↔ local).
- Uses `get_issues_modified_since` / conflict hooks where backends implement them.
- Conflict handling is basic (operator / force flags) — not a full merge engine.
### 1.7 Packaging, registry, deploy
| Artifact | Provides |
| --- | --- |
| PyPI package | Built wheel/sdist; Makefile targets publish to Coulomb Gitea/Forgejo registry |
| Docker + `k8s/railiance/` | GitOps-style deploy on railiance01 (deployment, service, configmap, ExternalSecret) |
| `docs/argocd-gitops.md` | Operator runbook for that deploy |
| `docs/package-release.md` | Release notes for packaging |
| `registry/` + `CAPABILITY-issue-tracking.yaml` | Reuse-surface / capability federation metadata |
| Tests | `tests/` unit coverage for models, CLI, backends, API ingest/query |
| Examples | `examples/agents/` programmatic patterns |
### 1.8 What is *not* shipped in the runtime (despite docs/plans)
| Item | Status |
| --- | --- |
| Inward boundary sync (tracker → work-record file write) | Not shipped — v1 is outward-only |
| REST `/mappings/` resource | CLI primary; REST mapping is via optional fields on `POST /issues/` |
| NATS subscriber | Design stub — `docs/nats-task-ingestion.md` |
| Runtime calls to state-hub (`add_progress_event`, etc.) | **Not in `issue_core` package**; repo participates as a normal ADR-001 workplan host only |
| Auto-detect backend from git remote | Roadmap / Makefile hints; not a reliable product path yet |
| MCP server | Not shipped |
| GitHub / GitLab / Jira backends | Planned only |
| Per-`target_repo` backend routing on REST | Planned only |
| Issue claiming locks beyond assignee + state | Convention only |
| First-class due-date field on `Issue` | `due_in_days` → ingestion metadata only |
| Canon work-record YAML back-reference field | Field name TBD in the-custodian; mapping authority is issue-core |
---
## 2. Product boundary — in scope
Things this repo **owns** and may grow, consistent with the connector role.
### 2.1 External-issue operations
- Create / read / update / close / reopen / comment on issues in configured
backends when tracker use is intentional.
- List and filter (state, labels, assignee, milestone, text search as backends allow).
- Label, assignee, and milestone management through the backend interface.
- Delete where the backend allows (SQLite hard delete; Gitea effectively close).
### 2.2 Connector / mapping (owned direction)
- Durable **work-record UUID ↔ (backend, external issue id)** mapping —
**store + CLI shipped** (ISSUE-WP-0005); REST `/mappings/` and file
back-references remain optional growth.
- **Outward** status projection shipped; **inward** boundary sync later,
never silent ADR-001 mutation; **zero load** when no mapping is used.
- Keep `triggering_event_id` as emitter lineage; optional
**`work_record_uuid` on TaskSpec** is shipped (do not overload
`triggering_event_id`).
### 2.3 Surfaces
- CLI for humans and agents on a shell.
- REST for intentional automation and worker claim/list/close.
- Python library for embedding.
- Optional future NATS consumer **only** for intentional external projection
(never silent fleet work origin).
### 2.4 Additional backends (when needed)
- GitHub, GitLab, Jira (or other trackers) as further `IssueBackend` plugins —
in scope as product growth, not as current deliverables.
### 2.5 Ops for this service
- Config, packaging, container image, k8s/GitOps for running the connector
where the org needs it (e.g. railiance01).
- Credential **routing** via ops-warden / OpenBao conventions — this repo does
not store secrets in git.
---
## 3. Out of scope
### 3.1 Origin of fleet work
Creating workplans, tasks, intake items, decisions, or engagements. Those
originate as repo files (or schema-valid blocks) and are indexed by
`fix-consistency` / state-hub. issue-core may **project** an existing work
record to a tracker; it does not birth the work record.
### 3.2 Fleet coordination substrate
Claiming and executing **fleet** work happens on work records + harness /
state-hub views — not on Forgejo issue lists by default. issue-core is not the
orgs task board.
### 3.3 Default sink for internal automation findings
activity-core (and peers) must not treat `POST /issues/` as the always-on
default for every matched rule. Emitters own that policy; issue-core keeps
accepting authenticated POSTs without advertising itself as a landing zone.
Follow-up: `activity-core` **ACTIVITY-WP-0022** (from ISSUE-WP-0004-T05).
### 3.4 Project management
Phases, campaigns, dependency graphs across work, Gantt scheduling, OKRs.
Workplan / project tooling owns that. Milestone on an issue is flat grouping
only.
### 3.5 Spawn audit trail
Emitters record who/what spawned an external issue (e.g. activity-core
`task_spawn_log`). issue-core stores the issue plus traceability fields
(`triggering_event_id` today; work-record UUID when mapping lands).
### 3.6 Event bus, notifications, workflow engine, IdP, UI
| Concern | Owner elsewhere |
| --- | --- |
| NATS / inter-service relay | Platform NATS ops — issue-core only consumes/serves its boundary |
| “Your task changed” notifications | UI / digest / chatbot services |
| Approval chains, SLA timers, multi-step workflows | Workflow tooling — states stay small |
| User directory / OIDC / MFA | key-cape / Keycloak / backend IdPs |
| First-party web UI for issue-core | Not planned; use backend native UIs |
---
## 4. Integration boundaries
### 4.1 Fleet primary path (work records)
| Step | Owner |
| --- | --- |
| Author / promote work record | Repo files + tools (ADR-001 / work-record canon) |
| Index + UUID write-back | state-hub `fix-consistency` |
| Claim / execute | Humans, agents, harness on work records |
| Optional external projection | **issue-core** (CRUD today; mapping when implemented) |
### 4.2 Clients of this repo
| Client | Transport | Role today |
| --- | --- | --- |
| Human / agent shell | CLI | Direct tracker admin and ops |
| Library consumers | Python API | Same backends without shell |
| Workers (e.g. harness) | REST GET/PATCH | List / claim / close external issues when in the loop |
| activity-core IssueSink | REST POST | **Optional, intentional** external issues only — not default for internal findings |
| Future mapping clients | REST/CLI TBD | Project/link work-record UUID ↔ external id |
### 4.3 TaskSpec contract (`POST /issues/`)
Retained for intentional emits and backward compatibility:
```json
{
"title": "string",
"description": "string",
"target_repo": "string",
"priority": "high | medium | low",
"labels": ["string"],
"due_in_days": 7,
"source_type": "rule | instruction",
"source_id": "string",
"triggering_event_id": "event uuid or stable source key",
"activity_definition_id": "string",
"work_record_uuid": "optional UUIDv7",
"work_record_id": "optional canonical id",
"work_record_kind": "optional kind"
}
```
- `triggering_event_id`: non-empty string; activity event UUID or stable key
such as `scheduled`. Stored in ingestion metadata — **not** a work-record UUID.
- `work_record_uuid` (optional): when set, upserts mapping and makes re-POST
idempotent for that UUID + backend.
- Response:
```json
{
"issue_id": "string",
"issue_url": "string or null",
"backend": "gitea | sqlite | github"
}
```
`issue_id` is the **backend** issue identity for the emitters log; not a
work-record UUID. (`github` appears in the response enum for forward
compatibility; no GitHub backend is shipped yet.)
### 4.4 Credential routing (operators / agents)
| Need | Owner |
| --- | --- |
| Gitea/Forgejo API token for backend | OpenBao / operator path (`warden route`) |
| `ISSUE_CORE_API_KEY` for REST | Shared secret via deploy secrets (e.g. ExternalSecret) |
| SSH certs | ops-warden only |
Do not message ops-warden for API keys; do not commit secrets.
---
## 5. Stack and layout (quick reference)
```
issue_core/
core/ # models, IssueBackend ABC, factory
backends/
local/ # SQLite
gitea/ # Gitea/Forgejo REST
cli/ # Click: issue, backend, sync, serve
api/ # FastAPI: ingest + query (+ auth, schemas)
tests/
docs/ # gitops, nats design, mapping design, release
k8s/railiance/ # deploy manifests
workplans/ # ADR-001 work items for this repo
```
| Concern | Choice |
| --- | --- |
| Language | Python 3.8+ |
| CLI | Click |
| HTTP | FastAPI + Pydantic v2 + uvicorn (`[api]` extra) |
| HTTP client (Gitea) | requests |
| Local store | SQLite |
| Tests | pytest |
---
## 6. See also
- `INTENT.md` — why the connector role exists (work-record-aligned rewrite 2026-07-22)
- `README.md` — operator quick start
- `ROADMAP.md` — phased growth (auto-config, mapping implementation, backends)
- `docs/uuid-external-id-mapping.md` — mapping design (not implemented)
- `docs/intent-work-record-alignment-review.md` — INTENT vs work-record audit
- `docs/nats-task-ingestion.md` — NATS design stub
- `docs/argocd-gitops.md` — railiance01 deploy
- `AGENT_INTEGRATION.md` — library patterns for tracker ops
- `workplans/ISSUE-WP-0004-align-with-work-record-canon.md` — initial framing pivot
- `workplans/ISSUE-WP-0005-connector-alignment-implementation.md` — implement mapping + close scope gaps
- `the-custodian/canon/standards/work-record-types_v0.1.md` — work-record kinds
(issue-core issues = external projections only)
- `the-custodian/research/WorkOrchestrationArchitectureDraft.md` §4.2
- activity-core IssueSink / **ACTIVITY-WP-0022** — emitter-side default policy