Nginx API Gateway Performance Tuning in Production

Scenario

An e-commerce platform using Nginx as an API gateway experiences response timeouts and connection failures during traffic spikes.

Symptoms

• Clients receive 502/504 errors.
• Backend CPU is normal, but Nginx worker processes spike.
• Connection count hits system limit.

Diagnosis

1. Check Nginx error log: tail -f /var/log/nginx/error.log
2. Count active connections: netstat -anp | grep nginx | wc -l
3. Inspect worker_connections: grep worker_connections /etc/nginx/nginx.conf
4. Enable status module:

location /nginx_status {
    stub_status on;
    allow 127.0.0.1;
    deny all;
}

Access http://localhost/nginx_status for real-time metrics.

Optimization Commands

1. Tune worker processes and connections:

worker_processes auto;
events {
    worker_connections 65536;
    use epoll;
}

1. Enable HTTP/2 multiplexing:

listen 443 ssl http2;

1. Configure upstream connection pool with keepalive:

upstream backend {
    server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;
    keepalive 256;
}

1. Rate limit to protect backend:

limit_req_zone $binary_remote_addr zone=api_limit:10m rate=1000r/s;
limit_req zone=api_limit burst=200 nodelay;

1. Enable shared dictionary caching:

proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=my_cache:10m max_size=1g inactive=60m use_temp_path=off;
server {
    location /api/ {
        proxy_cache my_cache;
        proxy_cache_key "$scheme$request_method$host$request_uri";
        proxy_cache_valid 200 302 10m;
        add_header X-Cache-Status $upstream_cache_status;
        proxy_pass http://backend;
    }
}

Risk Controls

• Apply changes incrementally, not all at once.
• Use nginx -s reload to avoid service disruption.
• Set rate limits conservatively at first, tighten after observing.

Rollback

• Keep backup of original config: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
• Restore: cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf && nginx -s reload

Verification

• Load test: ab -n 10000 -c 200 http://yourdomain.com/api/test
• Monitor connections: watch -n 1 'curl -s http://localhost/nginx_status | grep Active'
• Check cache hits via X-Cache-Status header.

When to Submit an OpsGlobal Ticket

• Issue persists after tuning and affects production.
• Need advanced kernel tuning or custom load balancing.
• Suspect underlying infrastructure bottlenecks (network, disk I/O).