Apache DataFusion

Pipeline duration exceeds baseline indicating performance degradation

warning
performanceUpdated Feb 17, 2026(via Exa)
How to detect:

Pipeline run duration exceeds expected threshold (e.g., 3600 seconds), indicating data skew, network bottlenecks, or insufficient resources

Recommended action:

Create alert policy on 'pipeline/run_duration' metric with threshold based on your baseline. When triggered, check Spark UI for data skew across executors, verify Data Fusion instance is in same region as data sources, consider upgrading from Developer to Enterprise edition or increasing Dataproc cluster configuration