Commit Frequency vs. Indexing Throughput Trade-off
warningconfigurationUpdated Aug 14, 2025
Frequent hard commits ensure data durability but open new searchers that spike memory usage and can degrade performance. Infrequent commits risk data loss but improve throughput. Commit timing directly impacts query latency during indexing.
Sources
How to detect:
Monitor commit latency and frequency. Track searcher warming time via solr_searcher_warmup. Correlate query latency spikes with commit operations. Watch for memory spikes when new searchers open.
Recommended action:
For large collections (70M+ docs), commit in smaller batches (3-4M documents) rather than bulk commits. Adjust autoCommit settings in solrconfig.xml based on data freshness requirements vs. performance impact. Consider manual commits during controlled maintenance windows for very large loads.