Nginx and API Gateway Middleware Performance Tuning: A Practical Guide

Scenario

An e-commerce platform uses Nginx as an API gateway middleware for routing, rate limiting, caching, etc. During a flash sale, users report slow API responses and timeouts.

Symptoms

• API latency spikes from 50ms to over 500ms
• Error rate jumps from 0.5% to 5%
• Nginx worker CPU usage consistently above 80%
• Memory usage abnormally high

Diagnosis

1. Check access logs: tail -f /var/log/nginx/access.log to observe response time distribution.
2. Check error logs: tail -f /var/log/nginx/error.log for connection timeouts, worker crashes.
3. Enable status module: Add to server block: location /nginx_status { stub_status; } Access /nginx_status for active connections, requests, etc.
4. Load testing: Use ab -n 10000 -c 100 http://localhost/nginx_status to simulate concurrency.
5. Analyze configuration: Review worker_processes, worker_connections, keepalive, proxy_buffers etc.

Tuning Commands

# Test configuration syntax
nginx -t

# Graceful reload
nginx -s reload

# Adjust worker processes (match CPU cores)
worker_processes auto;
# Or manual: number of CPU cores * 1-2

# Increase worker connections
events {
    worker_connections 4096;
    use epoll;
    multi_accept on;
}

# Optimize buffers
proxy_buffers 8 16k;
proxy_buffer_size 16k;
proxy_busy_buffers_size 24k;

# Enable keepalive
gzip on;
gzip_min_length 1000;
upstream backend {
    keepalive 32;
}

# Rate limiting
limit_conn_zone $binary_remote_addr zone=addr:10m;
limit_conn addr 10;

Risk Controls

• Backup config before changes: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
• Always validate with nginx -t before reload
• Avoid changes during peak traffic
• Monitor at least 10 minutes after each adjustment

Rollback

cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf
nginx -s reload

Verify by checking error logs and response times.

Verification

• Measure response time: curl -o /dev/null -s -w %{time_total}\n http://api.example.com/status
• Observe active connections on Nginx status page
• Compare load test results before and after
• Confirm error rate returns to normal

When to Submit an OpsGlobal Ticket

• Performance still unsatisfactory after tuning
• Frequent worker crashes
• Upstream servers healthy but Nginx proxies abnormally
• Complex configuration with unclear bottlenecks