Telemetry and Agent Effectiveness Tracking

WP-0001 T04 design — aligned with ADR-004 and WP-0004 ecosystem integration.

Two layers (do not merge)

Layer	Question	Mechanism
Project	How is agent X performing in this repo?	`kaizen-agentic metrics record` → `.kaizen/metrics/`
Fleet	How are coding sessions performing across repos?	agentic-resources Helix Forge

kaizen-agentic does not ship a parallel session transcript ingestion pipeline.

Project telemetry (implemented)

Memory-enabled agents record per-session outcomes at close:

kaizen-agentic metrics record <agent> --success --time <s> --quality <0-1>
kaizen-agentic metrics record <agent> --success --time <s> --quality <0-1> --emit-event
kaizen-agentic metrics optimize [agent]
kaizen-agentic memory brief <agent>    # includes Performance Summary

Optional fleet correlation via HELIX_SESSION_UID (see integrations/helix-forge-correlation.md).

Fleet telemetry (agentic-resources)

Helix Forge owns session capture, digest storage, baselines, and weekly retro. kaizen-agentic consumes correlation fields only.

CLI install / usage analytics (future)

Potential v1.1 additions (not yet implemented):

Opt-in anonymous counters on install / memory init (no PII, no project paths)
Aggregate effectiveness reports via metrics list across a monorepo checkout

tele-mcp evaluation (deferred)

tele-mcp is a candidate MCP adapter for IDE-level telemetry (WP-0001 note). Assess before depending on it. Project and fleet layers above satisfy INTENT's "measurable agents" requirement without tele-mcp.

Feedback loop

User experience feedback uses FEEDBACK.md and Forgejo issue templates — separate from execution metrics.

1.9 KiB Raw Permalink Blame History