Disk Queue Depth Signaling I/O Saturation

critical

Resource ContentionUpdated Oct 1, 2017

High DiskQueueDepth indicates I/O requests are queuing up faster than the storage subsystem can serve them. This directly increases read and write latency and signals that the instance is hitting storage performance limits, often due to insufficient provisioned IOPS or burst credit depletion.

Sources

Monitoring RDS MySQL performance metrics | Datadogwww.datadoghq.com

Best practices for Amazon RDS - Amazon Relational Database Servicedocs.amazonaws.cn

Technologies:

MariaDBSymptoms of this issue are visible in MariaDB metrics and logs

Amazon RDSThe root cause of this issue originates in Amazon RDS

aws.rds.disk_queue_depth

aws.rds.read_latency

aws.rds.write_latency

aws.rds.read_iops

aws.rds.write_iops

aws.rds.burst_balance

How to detect:

Monitor DiskQueueDepth continuously. Sustained values above 10–20 (depending on workload) indicate I/O saturation. Correlate with rising ReadLatency/WriteLatency and check if ReadIOPS/WriteIOPS are plateauing at provisioned limits. For gp2/gp3 storage, verify burst balance isn't depleted.

Recommended action:

Upgrade to Provisioned IOPS storage (io1/io2) with higher IOPS allocation if using gp2/gp3. If already on Provisioned IOPS, increase the provisioned IOPS limit. Scale up instance class to improve EBS-optimized bandwidth. Optimize queries to reduce I/O—add indexes, increase buffer pool size, or cache hot data in memory. Consider read replicas to distribute read I/O.