fabric-spark-perf-remediatelisted

# Microsoft Fabric Apache Spark Performance remediate Systematic workflows for diagnosing, analyzing, and resolving Apache Spark performance problems in Microsoft Fabric Data Engineering and Data Science workloads. ## When to Use This Skill Activate when encountering any of the following scenarios: - Spark notebooks or jobs running slower than expected - Capacity throttling errors (HTTP 430 / TooManyRequestsForCapacity) - Data skew detected by Spark Advisor in notebook cells - Excessive shuffle read/write in Spark UI stages - Small files accumulation in Delta Lake tables - Streaming ingestion throughput degradation - Need to select or tune a Fabric Spark resource profile - VOrder vs. Optimized Write decision-making - Autotune configuration and validation - Right-sizing Spark pools, node counts, or Fabric capacity SKUs ## Prerequisites - Access to a Microsoft Fabric workspace with Data Engineering enabled - Contributor or higher role on the workspace - Familiarity with PySpark or Spark SQL - PowerShell 7+ (for diagnostic scripts) - Fabric REST API access token (for API-based diagnostics) ## Quick Diagnosis Decision Tree Start here when a Spark job is slow: 1. **Is the job queued or throttled?** Check Monitoring Hub for HTTP 430. - Yes → See [Capacity and Concurrency Tuning](./references/spark-configuration-tuning.md#capacity-and-concurrency) - No → Continue 2. **Did the Spark Advisor flag warnings?** Check notebook cell indicators. - Data Skew detected → See