Frequent Upsert Segment Fragmentation
criticalHigh-frequency upsert operations generate many small, unindexed segments that force query nodes to scan raw data instead of using optimized indexes, dramatically increasing vector search latency and CPU usage until compaction completes.
Monitor the segment count and segment state distribution, watching for growth in 'Growing' or 'Flushing' segment counts relative to 'Sealed' segments. Track correlation between segment count increases and rising vector search latency. Alert when unindexed segment ratio exceeds 20% or when search latency increases >50% after upsert bursts.
Batch upsert operations to reduce segment creation frequency. Schedule compaction during off-peak hours and allocate sufficient CPU/memory resources for background compaction. Monitor compaction lag and adjust datacoord.segment.maxSize and datacoord.compaction.interval parameters. Consider reducing upsert frequency or implementing application-side deduplication.