LangfuseAmazon S3

S3 rate limits cause worker job failures and dead letter queue accumulation

warning
Connection ManagementUpdated Sep 28, 2025(via Exa)
How to detect:

S3 request rate limits cause worker job failures. BullMQ automatically retries failing requests, and persistent failures move jobs to dead letter queue. This can contribute to memory growth in worker containers during retry cycles.

Recommended action:

Monitor S3 request rates and adjust to stay below limits. Use BullMQ admin API to inspect dead letter queue for accumulated failed jobs. Replay failed events from dead letter queue after resolving rate limit issues. Consider implementing backoff strategies or rate limiting at application layer. Upgrade SDK from outdated versions (e.g., 2.59.1) as newer versions may reduce S3 request volume.