OpenSearch insights

When ThreadPool_QueueSize reaches ThreadPool_QueueCapacity, OpenSearch begins rejecting requests, causing search failures and degraded cluster performance. This often cascades as rejected requests retry.

Elevated Net_TCP_Lost packets and high Net_TCP_RxQ/Net_TCP_TxQ indicate network congestion or partial partitions between nodes, leading to cluster instability and potential split-brain scenarios if combined with cluster manager election failures.

Search request slow logs capture queries exceeding configured thresholds per phase (query, fetch), enabling identification of inefficient query patterns that degrade cluster performance over time.

ClusterManager_PendingQueueSize growth indicates the cluster manager node is overwhelmed with cluster state update tasks (index creation, mapping updates, shard allocation), potentially leading to slow cluster state propagation and operational delays.

Prolonged garbage collection events pause all cluster operations, manifesting as sudden latency spikes in search and indexing. Heap pressure from poorly tuned queries or cache evictions triggers these pauses.

Indexing operations consuming excessive memory trigger rejection when Indexing_Pressure_Current_Bytes exceeds Indexing_Pressure_Current_Limits, protecting cluster stability but causing write failures.

Segment merges saturate disk I/O causing Disk_Utilization spikes and elevated Disk_WaitTime, leading to query slowdowns and potential timeout failures. Merge_Time and Indexing_ThrottleTime increase as OpenSearch throttles indexing to protect disk resources.

Circuit breakers trip (CB_TrippedEvents increases) when CB_EstimatedSize approaches CB_ConfiguredSize, preventing out-of-memory errors but causing request failures. Multiple breaker types protect different memory pools.

High Cache_Query_Miss and Cache_Request_Miss rates force expensive query re-execution and document retrieval, amplifying latency and CPU usage. Often follows cache evictions or cluster restarts when caches are cold.

The Live Queries API enables real-time identification of currently executing queries consuming excessive CPU, memory, or running for unexpectedly long durations, allowing immediate intervention before they impact cluster stability.