Mastering Nginx and API Gateway Performance: A Practical Guide for SREs

Scenario

You are responsible for a production Nginx or API gateway (e.g., Kong, NGINX Plus, or custom OpenResty) and observe increased API response latency, high CPU usage, and occasional timeouts. This often results from misconfigured workers, missing caching, or inefficient middleware processing.

Symptoms

• Average response time increases by 2x or more
• Frequent 504 gateway timeouts from upstream
• "upstream timed out" errors in logs
• Nginx worker processes near 100% CPU
• Clients report connection resets or slow loads

Diagnosis

First, collect baseline metrics:

# Check active connections
$ curl http://localhost/nginx_status | grep -E 'Active|Waiting'
# Measure upstream response time
$ curl -w "@curl-format.txt" -o /dev/null -s https://api.example.com/endpoint
# Analyze system load
$ htop -p $(pgrep -f 'nginx: worker' | tr '\n' ',')
# Packet capture
$ tcpdump -i eth0 port 80 -w capture.pcap

Examine key Nginx configuration parameters: worker_processes, worker_connections, proxy_pass, proxy_cache, proxy_buffers. Use nginx -T to view full config.

Commands

Apply the following optimizations in a staging environment first.

1. Optimize Worker Processes

worker_processes auto;          # Match CPU cores
worker_rlimit_nofile 65536;    # Increase file descriptor limit

2. Enable Proxy Caching

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m max_size=1g inactive=60m use_temp_path=off;
server {
    location /api/ {
        proxy_cache my_cache;
        proxy_cache_valid 200 302 60m;
        proxy_cache_use_stale error timeout updating http_500;
        add_header X-Cache-Status $upstream_cache_status;
    }
}

3. Tune Buffers and Timeouts

proxy_buffer_size 8k;
proxy_buffers 8 8k;
proxy_busy_buffers_size 16k;
proxy_connect_timeout 10s;
proxy_send_timeout 10s;
proxy_read_timeout 30s;

4. Keep Alive Upstream Connections

upstream backend {
    keepalive 32;
    keepalive_requests 100;
}
server {
    location / {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_pass http://backend;
    }
}

5. Enable Gzip Compression

gzip on;
gzip_types application/json text/plain text/css;
gzip_min_length 1000;

6. Middleware Specific Tuning (e.g., Kong)

• Rate limiting: use redis strategy to avoid shared memory contention
• Authentication: precompute and cache JWT public keys to reduce upstream calls
• Plugin chain: disable unnecessary plugins to avoid per-request overhead

Risk Controls

• Backup current config: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
• Validate syntax: nginx -t
• Gradual rollout: reload on one node first, observe 5 minutes
• Set monitoring alerts: response time, error rate, CPU usage
• Use configuration management tools (e.g., Ansible) for consistency

Rollback

If issues arise, immediately:

# Restore backup
$ cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf
# Graceful reload
$ systemctl reload nginx
# Verify connections and error log
$ tail -f /var/log/nginx/error.log

If system parameters were changed (e.g., sysctl), revert with sysctl -p.

Verification

Use these tools for performance validation:

# Load test with wrk
$ wrk -t12 -c400 -d30s https://api.example.com/endpoint
# Compare latency percentiles (p50, p95, p99) before/after
# Simple test with ab
$ ab -n 10000 -c 100 https://api.example.com/endpoint
# Check cache hit ratio
$ curl -I https://api.example.com/endpoint | grep X-Cache-Status

Ideal outcome: response time reduced by 50%+, CPU usage lower, no timeouts.

When to Open an OpsGlobal Ticket

Escalate to OpsGlobal support when: - Performance does not improve after above tuning - Deep kernel-level network or TCP parameter analysis needed - Third-party gateway (e.g., Kong) shows internal errors or plugin bugs - Custom Nginx module (e.g., lua-resty) or compilation optimization required - Environment exceeds single-node capacity; load balancing architecture advice needed

OpsGlobal's SRE team brings extensive production tuning experience and can remotely assist with diagnosis and safe implementation of changes.