dev-ai-coding-metrics

# AI Coding Metrics Measures coding assistants and coding agents without collapsing results into vanity metrics or one blended score. The critical distinction is **mode**: assistants help inline or in chat; agents execute multi-step work and need task-level measurement. Do not measure them as if they were the same thing. ## When to Use This Skill | Trigger | Example | |---------|---------| | Designing a pilot or rollout scorecard | "We're rolling out Copilot to 200 engineers — what do we measure?" | | Diagnosing usage-up / outcomes-flat | "Seat utilization is 80% but PR throughput is unchanged" | | Comparing assistant vs. agent workflows | "Should we instrument these separately?" | | Building an ROI model or leadership report | "Finance wants a renewal decision by Q3" | | Designing an experiment better than vendor benchmarks | "We can't trust the vendor's numbers — how do we run our own study?" | ## Defaults | Rule | Rationale | |------|-----------| | Start from the decision, not the telemetry available | Prevents instrument-what-is-easy bias | | Separate assistant and agent funnels | Mixing hides which workflow drives results | | Pair every speed metric with quality + experience | Speed alone is misleading | | Aggregate at team level | Individual dashboards become surveillance | | Treat benchmarks as capability signals, not business KPIs | Benchmark gaps do not equal production gaps | ## Workflow 1. Define the decision. 2. Pick the program mode: assistant, agent, or ...