NGINX insights

KeepAlive Misconfiguration Exhausts Connectionswarning

KeepAliveTimeout set too high causes idle connections to consume worker threads/processes, reducing effective concurrency. KeepAliveTimeout set too low forces excessive TCP handshakes, increasing latency. Optimal values depend on traffic profile: APIs need 2-5s, static content 5-15s.

When nginx_upstream_peers_response_time p95/p99 percentiles increase but nginx_upstream_peers_active stays low and backend resources (CPU, memory, database) appear healthy, the issue often lies in configuration inefficiencies: suboptimal keepalives, excessive DNS lookups, or poorly tuned caching.

Increases in nginx_server_zone_responses_4xx or nginx_server_zone_responses_5xx require differentiation between client errors (4xx), NGINX configuration issues (502/503), and upstream failures (504, backend 5xx). The same metric can indicate completely different root causes depending on code distribution.

Load Balancer Queue Buildup from Backend Concurrency Limitscritical

When backend FastAPI workers experience event loop blocking, NGINX ingress accumulates requests in upstream queues as backends appear busy despite having available CPU capacity. This results in increased queue wait times and eventually gateway timeouts for clients, creating a cascading latency problem.

Latency Distribution Drift Under Sustained Loadwarning

Services with hidden event loop blocking show subtle latency drift over time, where median response times remain acceptable but p95/p99 percentiles creep upward over weeks. This gradual degradation becomes acute during traffic spikes, revealing that the service is operating with marginal concurrency headroom.

Event Loop Blocking Causes Serial Request Processingcritical

When NGINX proxies to async application servers (FastAPI, Node.js) but those backends make blocking I/O calls, the event loop stalls, causing serial-like request processing despite async infrastructure. Symptoms include flat throughput curves and rising tail latency even when CPU is moderate.

When NGINX proxies to backends (PHP-FPM, FastCGI, uwsgi) without proper connection limits, a single site can exhaust the proxy connection pool, blocking other sites on shared infrastructure. Default mod_proxy behavior allows each child process to open max connections equal to ThreadsPerChild × ServerLimit.

When NGINX reaches MaxRequestWorkers (or event MPM's equivalent capacity), new requests queue at the load balancer level, causing gateway timeouts even when CPU and dependencies appear healthy. This often manifests as moderate CPU (~50-60%) but rising tail latency and 502/504 errors.

Upstream backend health check failures (nginx_stream_upstream_peers_health_checks_fails, nginx_stream_upstream_peers_health_checks_unhealthy) provide early warning of backend degradation before user-facing errors occur. These often precede increases in nginx_upstream_peers_fails and nginx_upstream_peers_downtime.