issue-core/ROADMAP.md
tegwick d60175c354
All checks were successful
CI Smoke / host-smoke (push) Successful in 0s
CI Smoke / container-smoke (push) Successful in 2s
docs: align issue-core with work-record connector role (ISSUE-WP-0004)
Retarget INTENT, SCOPE, README, ROADMAP, and agent docs from "task landing
zone" to external-tracker connector per work-record canon and architecture
draft §4.2. Add UUID↔external-id mapping design; mark ISSUE-WP-0004 finished
and cross-file activity-core ACTIVITY-WP-0022 for IssueSink policy.
2026-07-21 04:22:12 +02:00

24 KiB

Issue Core Roadmap

Long-term vision for issue-core as an external issue-tracker connector — not as the fleet's work-origin or internal coordination substrate.

Framing (updated 2026-07-20)

Fleet coordination runs on work records (repo files + state-hub), per:

  • the-custodian/canon/standards/work-record-types_v0.1.md
  • the-custodian/research/WorkOrchestrationArchitectureDraft.md §4.2
  • INTENT.md / SCOPE.md / ISSUE-WP-0004

issue-core's trajectory is therefore:

  1. Keep multi-backend CRUD, CLI, REST, and sync as a solid connector.
  2. Add durable work-record UUID ↔ external issue id mapping and boundary sync (design: docs/uuid-external-id-mapping.md).
  3. Stop treating "everything lands in issue-core" as the product north star. Agent claim/execute for fleet work belongs on work records; tracker claim/locking remains useful for external issues.

Phases below that talk about multi-agent coordination should be read as coordination through an external tracker when one is in use, not as replacing workplans/tasks.

Current Status: v1.0 (Production-Ready Connector Core)

Complete:

  • Core CRUD operations (100%)
  • Gitea backend (production-ready)
  • Local SQLite backend (fully functional)
  • CLI with JSON output
  • Python programmatic API
  • Basic synchronization
  • REST POST/GET/PATCH /issues/ (intentional external issues)
  • Comprehensive test suite

⚠️ Limitations:

  • Manual backend configuration
  • No auto-detection
  • Basic conflict resolution
  • Hardcoded user context
  • No work-record UUID mapping yet (design only)
  • Docs historically framed issue-core as a landing zone (corrected in ISSUE-WP-0004)

Phase 0: Work-record canon alignment (docs) — ISSUE-WP-0004

Goal: Align product framing with the connector decision; design mapping.

Item Status
Rewrite INTENT.md this workplan
Rewrite SCOPE.md (mapping table concept) this workplan
README / ROADMAP framing this workplan
Design note: UUID ↔ external-id mapping docs/uuid-external-id-mapping.md
File activity-core IssueSink policy follow-up cross-repo task

Implementation of the mapping store is stage-3 in the-custodian work-record architecture — not blocked on inventing a parallel coordination system here.


Phase 1: Auto-Configuration (v1.1) - Next engineering priority

Goal: Enable agents and operators to attach issue-core to any repo's external tracker without painful manual setup.

1.1.1 Git Remote Detection

Implementation:

# issue_core/core/detection.py

def detect_git_remote() -> Optional[Dict[str, str]]:
    """
    Parse git remote URL to extract platform, owner, repo.

    Returns:
        {
            'platform': 'gitea' | 'github' | 'gitlab',
            'base_url': 'https://gitea.example.com',
            'owner': 'myorg',
            'repo': 'myproject'
        }
    """

def parse_remote_url(url: str) -> Optional[Dict[str, str]]:
    """
    Parse various git remote URL formats:
    - https://gitea.example.com/owner/repo.git
    - git@gitea.example.com:owner/repo.git
    - https://github.com/owner/repo
    """

Tests: tests/test_detection.py

  • Test various URL formats (HTTPS, SSH, with/without .git)
  • Test platform detection (Gitea, GitHub, GitLab)
  • Test edge cases (subgroups, custom domains)

Effort: 2-3 days

1.1.2 Environment-Based Configuration

Implementation:

# issue_core/core/env_config.py

def load_backend_from_env() -> Optional[Dict[str, Any]]:
    """
    Load backend config from environment variables:
    - GITEA_URL, GITEA_TOKEN, GITEA_OWNER, GITEA_REPO
    - GITHUB_TOKEN (with auto-detection)
    - GITLAB_URL, GITLAB_TOKEN
    """

def create_backend_from_env(platform: str) -> IssueBackend:
    """Create and connect backend from environment."""

New CLI command:

issue config auto
# Tries: git remote → environment → user prompt

Tests:

  • Test environment loading with various combinations
  • Test fallback priority (git → env → prompt)
  • Test error handling for missing credentials

Effort: 2-3 days

1.1.3 Per-Repository Configuration Files

Implementation:

.issue-core/
├── config.json          # Repository-specific settings
├── issues.db           # Local cache/backup
└── credentials.json    # Optional encrypted credentials

Config format:

{
  "backend": {
    "type": "gitea",
    "url": "https://gitea.example.com",
    "owner": "myorg",
    "repo": "myproject",
    "token_source": "env:GITEA_TOKEN"  // or "file:/path/to/token"
  },
  "sync": {
    "enabled": true,
    "interval": "1h",
    "auto_pull": false
  },
  "agent": {
    "identity": "agent-coder",
    "claim_timeout": 1800
  }
}

Functions:

def load_repo_config(path: Path = Path.cwd()) -> Optional[Dict]:
    """Load .issue-core/config.json from repo root."""

def save_repo_config(config: Dict, path: Path = Path.cwd()):
    """Save config to .issue-core/config.json."""

def find_repo_root() -> Optional[Path]:
    """Walk up directory tree to find git root."""

Effort: 3-4 days

1.1.4 Unified Auto-Configuration

Implementation:

# issue_core/core/auto_config.py

def auto_configure_backend() -> IssueBackend:
    """
    Auto-configure backend with fallback priority:
    1. Check .issue-core/config.json
    2. Detect from git remote + environment token
    3. Check global config (~/.config/issue-core/)
    4. Prompt user for manual configuration
    """

CLI integration:

# New commands
issue config detect          # Detect and show config (no save)
issue config init           # Detect, confirm, and save
issue config show           # Show current config
issue config edit           # Open config in $EDITOR

Tests:

  • Integration test: git repo → auto-detection → working backend
  • Test fallback priority
  • Test config precedence

Effort: 3-4 days

1.1.5 Integration Script UX Improvements

Problem: Integration script has poor UX - hardcoded defaults, no backup, doesn't reuse existing config.

Implementation:

# .capability/integrate.sh improvements

# 1. Smart default for backend name (derive from project)
PROJECT_NAME=$(basename "$PROJECT_ROOT")
read -p "Backend name [$PROJECT_NAME]: " backend_name
backend_name="${backend_name:-$PROJECT_NAME}"

# 2. Pre-populate existing settings when replacing
if issue backend show "$backend_name" &>/dev/null; then
    echo "⚠️  Backend '$backend_name' already exists"

    # Load current values
    CURRENT_URL=$(issue backend show "$backend_name" --field=url 2>/dev/null || echo "")
    CURRENT_OWNER=$(issue backend show "$backend_name" --field=owner 2>/dev/null || echo "")
    CURRENT_REPO=$(issue backend show "$backend_name" --field=repo 2>/dev/null || echo "")

    read -p "Replace existing backend? [y/N]: " replace
    if [ "$replace" = "y" ] || [ "$replace" = "Y" ]; then
        # Create timestamped backup
        TIMESTAMP=$(date +%Y%m%d_%H%M%S)
        CONFIG_FILE="$HOME/.config/issue-core/backends.json"
        if [ -f "$CONFIG_FILE" ]; then
            BACKUP_FILE="$CONFIG_FILE.backup.$TIMESTAMP"
            cp "$CONFIG_FILE" "$BACKUP_FILE"
            echo "✓ Backed up config to: $BACKUP_FILE"
        fi

        # Offer existing values as defaults
        read -p "Gitea URL [$CURRENT_URL]: " url
        url="${url:-$CURRENT_URL}"

        read -p "Repository owner [$CURRENT_OWNER]: " owner
        owner="${owner:-$CURRENT_OWNER}"

        read -p "Repository name [$CURRENT_REPO]: " repo
        repo="${repo:-$CURRENT_REPO}"
    else
        echo "Keeping existing configuration"
        exit 0
    fi
fi

Benefits:

  • Reduces typing (project name auto-detected)
  • Prevents accidental config loss (automatic backups)
  • Speeds up reconfiguration (existing values as defaults)
  • Better error recovery (timestamped backups)

Implementation tasks:

  1. Add detect_project_name() function to integration script
  2. Add load_existing_backend_config() function
  3. Add backup_config_file() function with timestamp
  4. Update interactive prompts to use defaults from existing config
  5. Add rollback instructions to output

Tests:

  • Manual testing of integration script with various scenarios:
    • Fresh installation (no existing backend)
    • Replacing existing backend (should show defaults)
    • Backup creation and verification
    • Directory name extraction accuracy

Effort: 2-3 days

Total Phase 1: ~15-20 days (3-4 weeks)


Phase 1.5: UUID ↔ external-id mapping (stage-3 work-record arch)

Goal: Persist and query the association between internal work-record UUIDs and backend issue ids so projections are first-class.

See docs/uuid-external-id-mapping.md for the design (no code in ISSUE-WP-0004). Expected deliverables when scheduled:

  • Mapping table / store (local SQLite + optional remote annotation)
  • API/CLI: project work record → create or link external issue; resolve either way
  • Optional fields on TaskSpec / project request for work_record_uuid
  • Two-way status/comment sync at the boundary only when enabled
  • Idle cost zero when no mapping/integration is configured

Phase 2: Agent Features for external trackers (v1.2)

Goal: Native helpers when agents operate on external issues (not a replacement for work-record execution).

1.2.1 Agent Identity Management

Problem: Currently uses hardcoded "cli-user" for all operations.

Implementation:

# issue_core/core/agent.py

@dataclass
class AgentContext:
    """Context for agent operations."""
    agent_id: str
    agent_type: str  # 'coder', 'reviewer', 'tester', etc.
    capabilities: List[str]  # ['python', 'javascript', 'testing']
    metadata: Dict[str, Any]

def get_agent_context() -> AgentContext:
    """
    Get agent context from:
    1. Environment (ISSUE_AGENT_ID, ISSUE_AGENT_TYPE)
    2. Config file (.issue-core/config.json)
    3. Default to system username
    """

def set_agent_context(context: AgentContext):
    """Set global agent context for operations."""

Backend integration:

# Update all comment/assignee operations
def add_comment(self, issue_id: str, body: str):
    """Add comment with current agent context."""
    ctx = get_agent_context()
    comment = Comment(
        author=User(id=ctx.agent_id, username=ctx.agent_id),
        body=f"[{ctx.agent_type}] {body}",
        ...
    )

CLI:

issue config agent set-id "agent-coder-v1"
issue config agent set-type "coder"
issue config agent show

Tests:

  • Test context loading from various sources
  • Test context inheritance in backend operations
  • Test agent metadata propagation

Effort: 4-5 days

1.2.2 Issue Claiming/Locking

Problem: No native support for claiming issues (prevents race conditions).

Implementation:

# issue_core/core/locking.py

class IssueClaim:
    issue_id: str
    agent_id: str
    claimed_at: datetime
    expires_at: datetime
    metadata: Dict[str, Any]

class LockManager:
    """Manages issue claims/locks."""

    def claim_issue(self, issue_id: str, timeout_seconds: int = 1800) -> IssueClaim:
        """
        Claim an issue for exclusive work.
        Raises ClaimError if already claimed.
        """

    def release_issue(self, issue_id: str):
        """Release claim on issue."""

    def check_claim(self, issue_id: str) -> Optional[IssueClaim]:
        """Check if issue is claimed and by whom."""

    def extend_claim(self, issue_id: str, additional_seconds: int):
        """Extend claim timeout."""

    def cleanup_expired(self):
        """Clean up expired claims."""

Storage: Store claims in issue metadata or separate tracking table.

For Gitea backend:

// In issue.sync_metadata
{
  "claim": {
    "agent_id": "agent-coder",
    "claimed_at": "2024-01-15T10:00:00Z",
    "expires_at": "2024-01-15T10:30:00Z"
  }
}

CLI:

issue claim 42 --timeout=30m          # Claim for 30 minutes
issue claim release 42                # Release claim
issue claim check 42                  # Check claim status
issue claim extend 42 --add-time=15m  # Extend by 15 minutes

Tests:

  • Test claim acquisition and release
  • Test claim expiration
  • Test concurrent claim attempts (should fail)
  • Test claim cleanup

Effort: 5-6 days

1.2.3 Structured Agent Metadata

Problem: Agent state tracked in comments (unstructured).

Implementation:

# Extend Issue model
@dataclass
class Issue:
    ...
    agent_metadata: Dict[str, Any] = field(default_factory=dict)
    """
    Structured metadata for agent operations:
    {
        'assigned_agent': {
            'agent_id': 'agent-coder',
            'assigned_at': '2024-01-15T10:00:00Z',
            'progress': 0.75
        },
        'work_state': {
            'stage': 'implementation',
            'checkpoints': ['analysis', 'design'],
            'next_checkpoint': 'testing'
        },
        'dependencies': {
            'blocks': [43, 44],
            'blocked_by': [41]
        }
    }
    """

API:

def update_agent_progress(issue: Issue, progress: float, status: str):
    """Update agent progress metadata."""

def mark_checkpoint(issue: Issue, checkpoint: str):
    """Mark a workflow checkpoint complete."""

def get_agent_state(issue: Issue) -> Dict[str, Any]:
    """Get current agent state for issue."""

Effort: 3-4 days

1.2.4 Webhook Support

Problem: Agents poll for changes (inefficient for real-time reactions).

Implementation:

# issue_core/core/webhooks.py

class WebhookManager:
    """Manage webhooks for real-time notifications."""

    def register_webhook(
        self,
        url: str,
        events: List[str],  # ['issue.created', 'issue.updated', 'issue.closed']
        secret: Optional[str] = None
    ):
        """Register webhook with backend."""

    def validate_webhook_signature(payload: bytes, signature: str, secret: str) -> bool:
        """Validate webhook payload signature."""

# Simple webhook receiver
class WebhookReceiver:
    def start(self, port: int = 8080):
        """Start HTTP server to receive webhooks."""

    def on_event(self, event_type: str, callback: Callable):
        """Register callback for event type."""

Example usage:

receiver = WebhookReceiver()

@receiver.on_event('issue.created')
def handle_new_issue(issue: Issue):
    if 'priority:critical' in [l.name for l in issue.labels]:
        agent.claim_and_process(issue)

receiver.start(port=8080)

CLI:

issue webhook register http://agent.example.com/hook --events=issue.created,issue.updated
issue webhook list
issue webhook remove <id>

Effort: 5-6 days

Total Phase 2: ~20-25 days (4-5 weeks)


Phase 3: Advanced tracker features (v2.0)

Goal: Richer connector behavior for large external-tracker deployments (dependencies between external issues, query DSL, webhooks already partly in Phase 2). Fleet dependency graphs for work records remain out of scope here (workplan tooling / hub).

2.0.1 Issue Dependency Tracking

Implementation:

@dataclass
class Issue:
    ...
    depends_on: List[int] = field(default_factory=list)
    blocks: List[int] = field(default_factory=list)

class DependencyGraph:
    """Manage issue dependencies."""

    def add_dependency(self, issue_id: int, depends_on: int):
        """Mark issue as depending on another."""

    def remove_dependency(self, issue_id: int, depends_on: int):
        """Remove dependency."""

    def get_blocking_issues(self, issue_id: int) -> List[Issue]:
        """Get issues blocking this one."""

    def get_blocked_issues(self, issue_id: int) -> List[Issue]:
        """Get issues blocked by this one."""

    def can_start(self, issue_id: int) -> bool:
        """Check if all dependencies are resolved."""

    def get_ready_issues(self) -> List[Issue]:
        """Get all issues with no blocking dependencies."""

CLI:

issue depends add 42 --blocks=43,44     # 42 blocks 43 and 44
issue depends remove 42 --blocks=43
issue depends show 42                   # Show dependency graph
issue depends ready                     # List issues ready to start

Effort: 6-7 days

2.0.2 Query DSL

Problem: Filtering is verbose and limited.

Implementation:

# issue_core/core/query_dsl.py

class QueryParser:
    """
    Parse query DSL:
    - is:open is:closed is:in-progress
    - assignee:me assignee:agent-coder
    - label:bug,priority:high
    - created:>7d updated:<24h
    - milestone:"Q1 Release"
    - sort:created-desc sort:priority
    """

    def parse(self, query: str) -> IssueFilter:
        """Parse query string into IssueFilter."""

# Examples:
# "is:open assignee:me label:bug,critical"
# "is:closed created:>30d sort:closed-desc"
# "label:needs-review -label:blocked"  # exclude label

CLI:

issue search "is:open assignee:me label:bug,priority:high"
issue list --query="is:in-progress created:>7d"

Effort: 5-6 days

2.0.3 Activity Streams & Event Logs

Implementation:

# issue_core/core/activity.py

@dataclass
class ActivityEvent:
    event_type: str  # 'issue.created', 'issue.updated', 'comment.added'
    issue_id: str
    actor_id: str
    timestamp: datetime
    changes: Dict[str, Any]
    metadata: Dict[str, Any]

class ActivityStream:
    """Track all issue activity."""

    def log_event(self, event: ActivityEvent):
        """Log an activity event."""

    def get_issue_activity(self, issue_id: str) -> List[ActivityEvent]:
        """Get all activity for an issue."""

    def get_agent_activity(self, agent_id: str) -> List[ActivityEvent]:
        """Get all activity by an agent."""

    def stream_events(self, since: datetime) -> Iterator[ActivityEvent]:
        """Stream events since timestamp."""

Storage: Add activity_log table to schema.

Effort: 6-7 days

2.0.4 Distributed Locking

Problem: Current locking is local only (single instance).

Implementation:

# issue_core/core/distributed_lock.py

class DistributedLockManager:
    """Distributed locking using Redis/database."""

    def __init__(self, redis_url: Optional[str] = None):
        """Use Redis if available, fallback to database."""

    def acquire_lock(
        self,
        resource: str,
        owner: str,
        ttl: int = 30
    ) -> bool:
        """Acquire distributed lock."""

    def release_lock(self, resource: str, owner: str):
        """Release distributed lock."""

    def extend_lock(self, resource: str, owner: str, additional_ttl: int):
        """Extend lock TTL."""

    def is_locked(self, resource: str) -> bool:
        """Check if resource is locked."""

Integration:

# Use for claim operations
with distributed_lock(f"issue:{issue_id}", agent_id):
    # Work on issue
    pass

Effort: 7-8 days

2.0.5 Conflict Resolution Strategies

Problem: Sync has basic conflict detection but no resolution.

Implementation:

# issue_core/core/sync_strategies.py

class ConflictResolutionStrategy(ABC):
    def resolve(self, local: Issue, remote: Issue) -> Issue:
        """Resolve conflict between local and remote."""

class TimestampStrategy(ConflictResolutionStrategy):
    """Take newer version based on updated_at."""

class ThreeWayMergeStrategy(ConflictResolutionStrategy):
    """Three-way merge using common ancestor."""

class InteractiveMergeStrategy(ConflictResolutionStrategy):
    """Prompt user to resolve conflicts."""

class AgentMergeStrategy(ConflictResolutionStrategy):
    """Use agent to intelligently merge changes."""

def sync_with_strategy(
    source: IssueBackend,
    target: IssueBackend,
    strategy: ConflictResolutionStrategy
):
    """Sync with specified conflict resolution strategy."""

Effort: 8-10 days

Total Phase 3: ~35-40 days (7-8 weeks)


Phase 4: Platform Expansion (Future)

GitHub Backend

  • Full API integration
  • GitHub-specific features (projects, discussions)
  • Effort: 10-12 days

GitLab Backend

  • Full API integration
  • GitLab-specific features (epics, boards)
  • Effort: 10-12 days

JIRA Backend

  • REST API integration
  • JIRA workflow mapping
  • Effort: 12-15 days

Linear Backend

  • GraphQL API integration
  • Linear-specific features
  • Effort: 8-10 days

Implementation Priority

Critical Path (for agent use)

  1. Auto-configuration (Phase 1.1) - 2-3 weeks
  2. Agent identity (Phase 2.1) - 1 week
  3. Issue claiming (Phase 2.2) - 1 week

Total: ~5-6 weeks to full agent-ready state

Nice to Have

  1. Agent metadata (Phase 2.3)
  2. Webhooks (Phase 2.4)
  3. Dependency tracking (Phase 3.1)
  4. Query DSL (Phase 3.2)

Future Enhancements

  1. Activity streams (Phase 3.3)
  2. Distributed locking (Phase 3.4)
  3. Advanced merge (Phase 3.5)

Success Metrics

Phase 0 Success (canon alignment)

  • INTENT/SCOPE/README describe connector, not landing zone
  • Mapping surface designed (docs/uuid-external-id-mapping.md)
  • activity-core IssueSink default no longer dumps internal findings to Forgejo

Phase 1 Success

  • Agent can attach to any repo's tracker with zero manual config
  • Environment-only setup works: GITEA_TOKEN=xxx issue list
  • Auto-detection accuracy: >95% for common platforms

Phase 1.5 Success (mapping)

  • Project work-record UUID → external issue id is durable and queryable
  • Reverse lookup (external issue → UUID) works for linked issues
  • No mapping configured ⇒ no extra load / no accidental Forgejo spam

Phase 2 Success

  • Multiple agents can operate on external issues without race conditions
  • Agent identity propagates to tracker operations
  • Claim/lock prevents concurrent work on the same external issue

Phase 3 Success

  • External-issue dependency helpers work where backends support them
  • Query DSL covers 90% of common tracker queries
  • Real-time tracker event processing with <1s latency when webhooks enabled

Migration Path

Each phase maintains backward compatibility:

Phase 1:

  • Old: Manual issue backend add still works
  • New: Auto-detection as optional enhancement

Phase 2:

  • Old: Hardcoded user still works for CLI
  • New: Agent context for programmatic use

Phase 3:

  • Old: Basic filtering still works
  • New: Query DSL as superset of old filters

Contributing

Want to help implement these features?

  1. Pick a feature from Phase 1 or 2
  2. Create issue: issue create "Implement <feature>" --label=enhancement
  3. See CLAUDE.md for development guide
  4. Follow architecture in core/interfaces.py
  5. Add tests (maintain >60% coverage)
  6. Submit PR

Quick wins for new contributors:

  • Environment config loading (Phase 1.1.2)
  • Git remote parsing (Phase 1.1.1)
  • Agent context API (Phase 2.1)

Version Timeline (Estimate)

  • v1.0 (Current) - Production-ready core
  • v1.1 (Q1 2025) - Auto-configuration
  • v1.2 (Q2 2025) - Agent features
  • v2.0 (Q3 2025) - Advanced coordination
  • v2.1 (Q4 2025) - Additional platforms

Timeline assumes single developer part-time. Can accelerate with contributors.


Questions or Feedback?

  • Architecture questions: See CLAUDE.md
  • Agent integration: See AGENT_INTEGRATION.md
  • Examples: See examples/agents/
  • Issues: Create issue in main repository