Vector index indexing slows significantly with tens of millions of documents

When vector indexes grow to tens of millions of documents (e.g., 17M), the indexing time for batches of 200-250k documents increases dramatically. At 17M vectors, indexing 250k documents took ~1200 seconds (20 minutes), with 50k disk write IOPS sustained. The 'writing the items' phase in the trace 'processing tasks > indexing > writing embeddings to database > writing the items' dominates indexing time and grows with index size.

Monitor the trace 'processing tasks > indexing > writing embeddings to database > writing the items' duration. For large-scale deployments (10M+ documents), consider horizontal sharding across multiple Meilisearch instances. Use SSD NVMe storage for lower disk write latency. Upgrade to Meilisearch v1.38+ which includes major indexing optimizations that remove full DB scans and improve parallel read access.