Practical Performance Tuning for Nginx as an API Gateway Middleware

Scenario

A microservices architecture uses Nginx as the API gateway. Users report increasing latency and frequent timeout errors (504) and upstream unavailable errors (502). The SRE team initiates investigation.

Symptoms

• Average response time increased from 50ms to 800ms
• Error rate rose from 0.1% to 5%
• Some clients experience persistent timeouts

Diagnosis

1. Check error logs: tail -100 /var/log/nginx/error.log shows many "upstream timed out" and "no live upstreams" messages.
2. Analyze access logs: tail -100 /var/log/nginx/access.log | awk '{print $NF}' (assumes last field is response time) reveals requests taking over 10 seconds.
3. System resources: top shows CPU idle but high I/O wait (wa). Upstream services are actually healthy and memory-bound.
4. Strace: strace -p $(pidof nginx) -e trace=network -c shows many epoll_wait timeouts and slow TCP connection establishments.
5. Configuration review: proxy_pass points to an upstream name without keepalive or proper timeouts, causing a new connection per request.

Commands & Optimization

Current Configuration Inspection

nginx -T | grep -E 'proxy_connect_timeout|proxy_send_timeout|proxy_read_timeout|keepalive'

If keepalive is missing, each request opens a new connection.

Optimize Upstream Connection Pool

Add to http or location block:

upstream backend {
    server backend-svc:8080;
    keepalive 32;
}
server {
    location /api/ {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_connect_timeout 5s;
        proxy_send_timeout 10s;
        proxy_read_timeout 30s;
    }
}

Adjust Worker Processes

worker_processes auto;
events {
    worker_connections 2048;
    multi_accept on;
}

Enable Caching (for idempotent requests)

proxy_cache_path /tmp/cache levels=1:2 keys_zone=my_cache:10m max_size=1g inactive=60m use_temp_path=off;

Risk Controls

• Back up config: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
• Validate with nginx -t
• Apply changes with nginx -s reload (zero-downtime)
• Test in staging first if possible

Rollback

If issues arise after changes:

cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf
nginx -t && nginx -s reload

Verification

• Run load test: ab -n 1000 -c 100 http://api.example.com/endpoint
• Monitor response time distribution: awk '{print $NF}' access.log | sort -n | uniq -c | sort -k1 -n
• Confirm absence of 502/504 errors

When to Submit an OpsGlobal Ticket

• If performance does not improve after basic tuning
• If deeper issues are suspected (kernel tuning, SSL offload, architecture)
• When production is affected and you lack time for thorough investigation
• When professional performance benchmarking is needed

OpsGlobal's SRE experts provide 24/7 remote support to quickly resolve Nginx gateway performance issues.