Optimizing Nginx API Gateway Performance: Upstream Timeouts and Buffering

Scenario

Your team uses Nginx as an API gateway to route traffic to multiple microservices. Lately, users report intermittent slow responses and timeouts. This guide walks through diagnosing and resolving common performance issues caused by misconfigured upstream timeouts and proxy buffers.

Symptoms

• Increased latency (p95 > 5s)
• Upstream connection timeouts (504 Gateway Timeout)
• High nginx worker CPU usage with many connections in CLOSE_WAIT state.

Diagnosis

1. Check nginx error logs: tail -f /var/log/nginx/error.log | grep upstream
2. Analyze upstream response times using curl -w: curl -w "Connect time: %{time_connect}\nTTFB: %{time_starttransfer}\nTotal: %{time_total}\n" -o /dev/null -s http://localhost/your-endpoint
3. Review current nginx config for proxy settings: grep -E "proxy_(read|connect|send)_timeout|proxy_buffer" /etc/nginx/conf.d/*.conf

Commands

• To increase upstream timeout from default 60s to 120s for a specific location: location /api/ { proxy_pass http://backend; proxy_connect_timeout 30; # upstream connect timeout proxy_read_timeout 120; # wait for response from upstream proxy_send_timeout 30; # sending request to upstream }
• To adjust proxy buffers (reduce disk I/O and improve streaming): proxy_buffering on; proxy_buffer_size 4k; proxy_buffers 8 4k; proxy_busy_buffers_size 8k;
• To enable keepalive connections to upstream for reuse: upstream backend { server 10.0.1.1:8080; keepalive 32; # max idle connections } # in location: proxy_http_version 1.1; proxy_set_header Connection "";

Risk Controls

• Before reloading nginx, test config: nginx -t
• Apply changes gradually using canary or blue-green deployment if possible.
• Monitor error rates and latency after changes. Consider setting up alerts.

Rollback

• Revert config files from backup or version control.
• Run nginx -t and systemctl reload nginx (or nginx -s reload).
• Verify with same testing commands.

Verification

• Repeat curl -w and check that TTFB decreased.
• Check nginx error logs for fewer timeouts.
• Use ab or wrk for load testing: ab -n 1000 -c 100 http://localhost/your-endpoint
• Review system resource usage: top, netstat -tan | grep :80 | wc -l

When to Submit an OpsGlobal Ticket

• If after adjustments, latency remains high or errors persist.
• If upstream server is underperforming and needs hardware/cloud resource tuning.
• For architectural advice on scaling Nginx ingress in Kubernetes (e.g., optimizing ingress controller configurations).