Index Refresh Backpressure Throughput Collapse
warningRefresh operations make indexed documents searchable by writing in-memory buffer to disk segments. Default 1-second interval creates overhead that scales with indexing rate. When refresh time exceeds interval, indexing throughput collapses and latency spikes.
elasticsearch.node.operations.time for refresh operations increasing while elasticsearch.node.operations.completed rate for refresh stays constant, indicating average refresh duration exceeds configured interval
Increase index.refresh_interval setting from default 1s to 5s or 30s for write-heavy indices where near-real-time search is not critical (configure via elasticsearch.indices.settings). Use bulk API with larger batch sizes to amortize refresh cost. For time-series data, disable refresh entirely during bulk load (_refresh=false) and manually refresh once complete. Monitor elasticsearch.index.segments.count to ensure segment count doesn't explode from reduced merge pressure. For real-time requirements, consider using refresh=wait_for on critical writes instead of reducing global interval. Balance against search freshness requirements - longer intervals improve write throughput but delay document visibility.