Task Metrics Lack Granular Labels for Analysis

info

configurationUpdated Dec 16, 2025

StatsD metrics emitted by Airflow use flat naming (dag.task.duration) without structured labels, making it difficult to aggregate, filter, or correlate metrics across DAGs, tasks, and runs in Prometheus/Grafana dashboards.

Sources

Monitoring Airflow with Prometheus, StatsD, and Grafana | Tracerwww.tracer.cloud

Technologies:

Apache AirflowSymptoms of this issue are visible in Apache Airflow metrics and logs

How to detect:

Monitor if exported Airflow metrics in Prometheus lack label dimensions (dag_id, task_id, run_id). Check if metric cardinality is high due to each task creating a unique metric name rather than labeled variants.

Recommended action:

Configure Prometheus StatsD Exporter with mapping rules to extract DAG ID, task ID, and other identifiers as Prometheus labels; use community-provided mapping configs (e.g., Databand's open-source config); structure metrics as airflow_task_duration_seconds{dag_id='X', task_id='Y'} instead of flat names; regularly review and update mappings as new metric patterns emerge.