Technologies/Apache Spark/spark_stage_count_active_tasks
Apache SparkApache SparkMetric

spark_stage_count_active_tasks

Number of active tasks in the application's stages
Dimensions:None
Available on:DatadogDatadog (1)
Interface Metrics (1)
DatadogDatadog
Number of active tasks in the application's stages
Dimensions:None
Sources
spark.stage.num_active_tasksgithub.com
Related Insights (1)
Autoscaling Lag During Load Spikeswarning

Databricks autoscaling takes 3-5 minutes to provision new VMs during demand spikes, causing queued tasks and degraded performance while pending tasks exceed available executor cores.

▸