Apache DataFusion

Error record count exceeds threshold indicating data quality issues

warning
performanceUpdated Feb 17, 2026(via Exa)
How to detect:

Pipeline error_count metric exceeds threshold (e.g., 100 records), indicating data quality issues, schema mismatches, or transformation failures

Recommended action:

Create alert policy on 'pipeline/error_count' metric with threshold of 100 (adjust based on your tolerance). When triggered, check error handler output tables for failed records, analyze patterns in error data, update pipeline schema or add transforms to handle edge cases