DjangoCelery

Synchronous bulk exports cause memory exhaustion and timeout

critical
performanceUpdated Feb 24, 2026(via Exa)
How to detect:

Serving complete dataset exports as synchronous API responses loads millions of rows into memory at once. Django loads entire queryset into memory before iteration without iterator(). Causes OOM and 90+ second response times that exceed client timeouts.

Recommended action:

Implement async export pattern: return HTTP 202 Accepted immediately, queue Celery background task using iterator(chunk_size=1000) to stream rows in 1000-row batches keeping memory flat, write to CSV/file, upload to storage, send email with download link when complete. Never serve full dataset as synchronous response.