Nginx API Gateway Middleware Performance Tuning in Production

Scenario

An e-commerce platform uses Nginx as an API gateway. During a flash sale, traffic spikes cause response latency and timeouts.

Symptoms

• Clients observe high latency (>5s)
• Nginx error logs show 'upstream timed out'
• System load high but CPU not maxed

Diagnosis

1. Check Nginx status: curl http://localhost/nginx_status - look for high Active connections or Waiting queue.
2. Analyze access logs: tail -f /var/log/nginx/access.log to identify slow requests.
3. Trace system calls: strace -p <nginx_pid> reveals excessive epoll_wait.
4. Verify upstream: time curl -XGET http://upstream/health to ensure backend is responsive.

Commands

Adjust Worker Processes

worker_processes auto;

Optimize Event Model

events {
    worker_connections 10240;
    use epoll;
    multi_accept on;
}

Increase Proxy Timeouts

location /api {
    proxy_connect_timeout 60s;
    proxy_send_timeout 60s;
    proxy_read_timeout 60s;
}

Enable Cache (optional)

proxy_cache_path /tmp/cache levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=60m;

Risk Controls

• Backup config: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
• Validate with nginx -t before reload
• Monitor connections during reload: watch nginx_status
• Use canary testing: apply changes to a subset of upstream servers first

Rollback

If performance degrades or errors appear:

cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf
nginx -s reload

Verification

• Check latency: tail -f /var/log/nginx/access.log | awk '{print $NF}' - should decrease
• Load test: ab -n 10000 -c 100 http://localhost/api/test
• Confirm no timeouts in error.log

When to Submit an OpsGlobal Ticket

• Performance doesn't improve after tuning
• Upstream is healthy but Nginx still times out
• Need deeper analysis (e.g., kernel tuning, custom modules)
• Capacity planning or architecture review required