feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.
- Migration o2j3k4l5m6n7: token_events table with FK indexes on
task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
|
|
|
import uuid
|
|
|
|
|
from datetime import datetime
|
2026-05-23 13:59:05 +02:00
|
|
|
from typing import Any
|
|
|
|
|
|
2026-08-22 00:27:48 +02:00
|
|
|
from sqlalchemy import (
|
|
|
|
|
DateTime,
|
|
|
|
|
Float,
|
|
|
|
|
ForeignKey,
|
|
|
|
|
Integer,
|
|
|
|
|
Text,
|
|
|
|
|
UniqueConstraint,
|
|
|
|
|
func,
|
|
|
|
|
)
|
2026-05-23 13:59:05 +02:00
|
|
|
from sqlalchemy.dialects.postgresql import JSONB, UUID
|
feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.
- Migration o2j3k4l5m6n7: token_events table with FK indexes on
task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
|
|
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
|
|
|
|
|
|
|
|
|
from api.models.base import Base, new_uuid
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class TokenEvent(Base):
|
|
|
|
|
__tablename__ = "token_events"
|
2026-05-23 13:59:05 +02:00
|
|
|
__table_args__ = (
|
|
|
|
|
UniqueConstraint(
|
|
|
|
|
"measurement_kind",
|
|
|
|
|
"source_provider",
|
|
|
|
|
"source_id",
|
|
|
|
|
name="uq_token_events_source_identity",
|
|
|
|
|
),
|
|
|
|
|
)
|
feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.
- Migration o2j3k4l5m6n7: token_events table with FK indexes on
task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
|
|
|
|
|
|
|
|
id: Mapped[uuid.UUID] = mapped_column(
|
|
|
|
|
UUID(as_uuid=True), primary_key=True, default=new_uuid
|
|
|
|
|
)
|
|
|
|
|
task_id: Mapped[uuid.UUID | None] = mapped_column(
|
2026-08-22 00:27:48 +02:00
|
|
|
UUID(as_uuid=True),
|
|
|
|
|
ForeignKey("tasks.id", ondelete="SET NULL", onupdate="CASCADE"),
|
|
|
|
|
nullable=True,
|
|
|
|
|
index=True,
|
feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.
- Migration o2j3k4l5m6n7: token_events table with FK indexes on
task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
|
|
|
)
|
2026-06-22 13:52:13 +02:00
|
|
|
workplan_id: Mapped[uuid.UUID | None] = mapped_column(
|
2026-08-22 00:27:48 +02:00
|
|
|
UUID(as_uuid=True),
|
|
|
|
|
ForeignKey("workplans.id", ondelete="SET NULL", onupdate="CASCADE"),
|
|
|
|
|
nullable=True,
|
|
|
|
|
index=True,
|
feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.
- Migration o2j3k4l5m6n7: token_events table with FK indexes on
task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
|
|
|
)
|
|
|
|
|
repo_id: Mapped[uuid.UUID | None] = mapped_column(
|
|
|
|
|
UUID(as_uuid=True), ForeignKey("managed_repos.id", ondelete="SET NULL"), nullable=True, index=True
|
|
|
|
|
)
|
|
|
|
|
session_id: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
|
model: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
|
tokens_in: Mapped[int] = mapped_column(Integer, nullable=False)
|
|
|
|
|
tokens_out: Mapped[int] = mapped_column(Integer, nullable=False)
|
|
|
|
|
agent: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
|
ref_type: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
|
ref_id: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
|
note: Mapped[str | None] = mapped_column(Text, nullable=True)
|
2026-05-23 13:59:05 +02:00
|
|
|
measurement_kind: Mapped[str] = mapped_column(
|
|
|
|
|
Text, nullable=False, default="estimated", server_default="estimated", index=True
|
|
|
|
|
)
|
|
|
|
|
source_provider: Mapped[str] = mapped_column(
|
|
|
|
|
Text, nullable=False, default="manual", server_default="manual", index=True
|
|
|
|
|
)
|
|
|
|
|
source_id: Mapped[str | None] = mapped_column(Text, nullable=True, index=True)
|
|
|
|
|
source_path: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
|
source_created_at: Mapped[datetime | None] = mapped_column(
|
|
|
|
|
DateTime(timezone=True), nullable=True, index=True
|
|
|
|
|
)
|
|
|
|
|
ingested_at: Mapped[datetime] = mapped_column(
|
|
|
|
|
DateTime(timezone=True), server_default=func.now(), nullable=False, index=True
|
|
|
|
|
)
|
|
|
|
|
parser_version: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
|
|
|
confidence: Mapped[float] = mapped_column(
|
|
|
|
|
Float, nullable=False, default=0.35, server_default="0.35"
|
|
|
|
|
)
|
|
|
|
|
cached_input_tokens: Mapped[int] = mapped_column(
|
|
|
|
|
Integer, nullable=False, default=0, server_default="0"
|
|
|
|
|
)
|
|
|
|
|
reasoning_output_tokens: Mapped[int] = mapped_column(
|
|
|
|
|
Integer, nullable=False, default=0, server_default="0"
|
|
|
|
|
)
|
|
|
|
|
raw_total_tokens: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
|
|
|
|
cost_estimated_usd: Mapped[float | None] = mapped_column(Float, nullable=True)
|
|
|
|
|
raw_metadata: Mapped[dict[str, Any]] = mapped_column(
|
|
|
|
|
JSONB, nullable=False, default=dict, server_default="{}"
|
|
|
|
|
)
|
feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.
- Migration o2j3k4l5m6n7: token_events table with FK indexes on
task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
|
|
|
created_at: Mapped[datetime] = mapped_column(
|
|
|
|
|
DateTime(timezone=True), server_default=func.now(), nullable=False, index=True
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
task: Mapped["Task | None"] = relationship("Task", lazy="selectin") # noqa: F821
|
2026-06-22 13:52:13 +02:00
|
|
|
workplan: Mapped["Workplan | None"] = relationship("Workplan", lazy="selectin") # noqa: F821
|
feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.
- Migration o2j3k4l5m6n7: token_events table with FK indexes on
task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
|
|
|
repo: Mapped["ManagedRepo | None"] = relationship("ManagedRepo", lazy="selectin") # noqa: F821
|