JVM Garbage Collection Thrashing
criticalExcessive JVM garbage collection time relative to task execution indicates memory leaks, inefficient data structures, or insufficient executor heap allocation, severely degrading Spark performance.
Calculate the ratio of spark_executor_peak_mem_major_gc_time to spark_executor_count_time. High ratios (>10%) indicate GC pressure. Cross-reference with spark_executor_memory_used approaching spark_executor_max_memory and increasing spark_executor_peak_mem_jvm_heap_memory.
Reduce cores per executor to allocate more memory per task. For memory-intensive workloads, configure 8-16GB executors with 2-4 cores instead of maximizing parallelism. Unpersist cached dataframes when no longer needed. Review code for large temporary objects in UDFs and replace with built-in Spark functions.