LangfuseRedis

Worker container high memory usage with low CPU under ingestion load

critical
Resource ContentionUpdated Sep 28, 2025(via Exa)
How to detect:

Langfuse worker containers consume >90% memory (6.8 GiB out of 8 GiB) while CPU usage remains around 10%, leading to container restarts. Occurs under high ingestion loads (441K traces, 2.7M observations, 47K scores per day) with ~150 delayed jobs per BullMQ queue and ~2,000 Redis keys.

Recommended action:

Scale out worker containers horizontally. Tune LANGFUSE_INGESTION_QUEUE_PROCESSING_CONCURRENCY and LANGFUSE_INGESTION_CLICKHOUSE_WRITE_BATCH_SIZE to reduce memory spikes (lower values reduce memory at cost of throughput). Set NODE_OPTIONS=--max-old-space-size=6144 to cap Node.js heap. Monitor langfuse.queue.ingestion.length metric to guide scaling. Lower LANGFUSE_TOKEN_COUNT_WORKER_POOL_SIZE from default 2 if tokenization jobs accumulate. Check for stalled BullMQ jobs. Profile with Node.js heap snapshots if persistent.