nginx_upstream_peers_response_time

When NGINX proxies to async application servers (FastAPI, Node.js) but those backends make blocking I/O calls, the event loop stalls, causing serial-like request processing despite async infrastructure. Symptoms include flat throughput curves and rising tail latency even when CPU is moderate.

When NGINX reaches MaxRequestWorkers (or event MPM's equivalent capacity), new requests queue at the load balancer level, causing gateway timeouts even when CPU and dependencies appear healthy. This often manifests as moderate CPU (~50-60%) but rising tail latency and 502/504 errors.

Increases in nginx_server_zone_responses_4xx or nginx_server_zone_responses_5xx require differentiation between client errors (4xx), NGINX configuration issues (502/503), and upstream failures (504, backend 5xx). The same metric can indicate completely different root causes depending on code distribution.