API server overload at high scale causes HTTP 500 errors

warning

performanceUpdated Feb 5, 2025(via Exa)

Sources

Prefect Workers crash when server returns 500s · Issue #16977 · PrefectHQ/prefectgithub.com

Technologies:

Prefectsubject

KubernetesKubernetes metrics correlate with this issue and help confirm diagnosis

How to detect:

Self-hosted Prefect API server becomes overloaded and returns HTTP 500s at high scale (20k flow runs/day, 200k tasks/day), impacting worker stability and flow execution.

Recommended action:

Increase server deployment size (CPU, memory, replicas). Monitor server resource utilization and API response times. Consider horizontal scaling of API servers and database optimization for high-volume deployments.