DramatiqRabbitMQ

Non-thread-safe broker causes infinite wait with multiple threads

critical
concurrencyUpdated Sep 13, 2023(via Exa)
How to detect:

When using --processes 1 --threads 2 or higher with non-thread-safe message brokers (e.g., kombu), workers hang indefinitely. Both threads call _read simultaneously and read different parts of incoming data, preventing either from constructing a complete frame. This causes both threads to wait infinitely for the next data that never arrives.

Recommended action:

Use --processes 1 --threads 1 to eliminate thread-safety issues. Alternatively, switch to a thread-safe broker implementation such as dramatiq-kombu-broker (https://github.com/spumer/dramatiq-kombu-broker/). If using kombu directly, ensure kombu.Connection is not shared across threads as it is not thread-safe.