dt-obs-tracing

# Application Tracing Skill ## Overview Distributed traces in Dynatrace consist of spans - building blocks representing units of work. With Traces in Grail, every span is accessible via DQL with full-text searchability on all attributes. This skill covers trace fundamentals, common analysis patterns, and span-type specific queries. --- ## Use Cases ### 1. Investigate Slow Requests - **Goal:** Find and diagnose requests exceeding a latency threshold - **Trigger:** "slow requests", "high latency", "p99 response time", "find traces over 5 seconds" - **Done:** List of slow traces with duration, endpoint, service, and trace IDs for drilldown ### 2. Analyze Request Failures - **Goal:** Identify failed requests, failure reasons, and exception patterns - **Trigger:** "failed spans", "HTTP 500 errors", "exception analysis", "failure rate by service" - **Done:** Failure breakdown by reason (HTTP code, exception, gRPC status) with exemplar traces ### 3. Map Service Dependencies - **Goal:** Understand service-to-service communication patterns and external API calls - **Trigger:** "service dependencies", "what services does X call", "outgoing HTTP calls" - **Done:** Dependency map showing call counts, latency, and error rates between services --- ## Core Concepts ### Understanding Traces and Spans **Spans** represent logical units of work in distributed traces: - HTTP requests, RPC calls, database operations - Messaging system interactions - Internal function invocations - Cust...