Dynamic filter pushdown causes type comparison error when joining VALUES against Dictionary-encoded Parquet columns
criticalperformanceUpdated Mar 13, 2026(via Exa)
Technologies:
How to detect:
Query fails with 'Can't compare arrays of different types' error when pushdown_filters is enabled and a VALUES clause is joined against Parquet tables with Dictionary(Int32, Utf8) encoded columns. The dynamic filter InList unwraps Dictionary arrays from Parquet batches to plain Utf8 values, but doesn't unwrap the stored in_array values, causing make_comparator(Utf8, Dictionary) to fail.
Recommended action:
Workaround: SET datafusion.execution.parquet.pushdown_filters = false to disable row-level filter pushdown. Permanent fix: Normalize in_array in ArrayStaticFilter::try_new() by unwrapping Dictionary arrays to their values before storing, ensuring both sides are the same plain type during comparison.