Single task resource exhaustion requires container-based limits
warningResource ContentionUpdated Apr 6, 2016(via Exa)
Technologies:
How to detect:
Individual Luigi tasks can consume excessive CPU or memory resources without built-in per-task limits. Luigi does not provide native resource restriction mechanisms for individual task execution.
Recommended action:
Run resource-intensive tasks in containers using ulimit or Docker to enforce CPU and memory constraints. This prevents individual tasks from exhausting system resources.