Column removal or type changes break dependent software

critical

availabilityUpdated Apr 17, 2024(via Exa)

Sources

Ensure Reliable Data with Dagster+dagster.io

Technologies:

dbtdbt metrics correlate with this issue and help confirm diagnosis

How to detect:

Column removal or data type changes in tables cause dependent software and dashboards to fail. Changes may be intentional migrations or accidental results of SQL query modifications.

Recommended action:

Enable built-in asset checks for column schema changes. Use Dagster's dbt integration to automatically capture table column schemas. Configure alerts when schema checks fail. Optionally halt pipelines on schema changes to prevent breaking downstream consumers.