Aggressive Dashboard Auto-Refresh Overloading Data Sources
warningLow dashboard refresh intervals (e.g., every 30 seconds or less) create unnecessary query load on data sources, degrading performance and potentially causing timeouts or errors, especially when querying long time ranges or high-cardinality data.
Track grafana_api_dashboard_search_milliseconds_datadog and data source query error rates (e.g., Prometheus query failures, timeouts) alongside dashboard refresh rate configuration. If error rates or query latency climb in lockstep with dashboard refresh activity, and dashboards use long time ranges (e.g., one week), auto-refresh is likely overloading the backend.
Disable auto-refresh on dashboards with long time ranges (week or more). For dashboards requiring auto-refresh, increase the refresh interval to match data source update frequency (e.g., 1 minute, 10 minutes, or hourly). Enable refresh only on essential dashboards and encourage manual browser refresh for others.