Nginx as API Gateway: A Field Guide to Middleware Performance Troubleshooting

Scenario

An e-commerce platform uses Nginx as its API gateway, funneling all microservice requests through this layer. Recently, with increased traffic, the gateway began exhibiting intermittent high latency and 5xx errors, especially when upstream services responded slowly. We were tasked to troubleshoot and tune the Nginx gateway middleware performance.

Symptoms

• Customers report API response times rising from a 200ms average to 1 second or more.
• Nginx error logs show many upstream timed out and connection refused entries.
• System load shows high Nginx worker CPU usage, but memory is normal.
• Monitoring charts indicate active connections nearing the worker_connections limit.

Diagnosis

Start by inspecting the Nginx configuration and runtime status.

1. Validate Nginx process and config bash nginx -t # Validate config syntax nginx -V # Show compile options and modules ps aux | grep nginx

2. Analyze logs bash tail -f /var/log/nginx/access.log | awk '{print $7, $9, $10, $11}' | sort | uniq -c | sort -nr | head -20 Check status code distribution to see if 5xx is concentrated. Error logs are typically in /var/log/nginx/error.log.

3. Check upstream response performance bash curl -w 'time_total: %{time_total}s time_connect: %{time_connect}s time_ttfb: %{time_starttransfer}s\n' -o /dev/null http://upstream-service/health The difference between time_connect and time_ttfb indicates upstream processing time. If upstream is slow, the issue might not be Nginx.

4. Check connection count and worker utilization bash # View Nginx status page (requires stub_status configuration) curl http://localhost/nginx_status # Or count connections with netstat netstat -an | grep :80 | wc -l

5. Packet capture or slow log If you suspect middleware logic (e.g., rewrite, proxy_pass, rate limiting) is causing delays, temporarily enable Nginx debug logging or use strace to trace worker processes.

Commands

After diagnosis, we execute the following targeted actions:

• Tune worker processes and connection limits: Edit nginx.conf, set worker_processes to match CPU cores, and increase worker_connections. nginx worker_processes auto; events { worker_connections 4096; }

• Configure upstream timeouts and retries: In the location block, adjust proxy_connect_timeout, proxy_send_timeout, and proxy_read_timeout. Example: nginx proxy_connect_timeout 5s; proxy_read_timeout 10s; proxy_send_timeout 10s;

• Enable caching or compression: If API responses are cacheable, add proxy_cache; enable gzip for textual responses to reduce transfer size.

• Review current configuration: bash nginx -T # Dump full configuration Save and test config: bash nginx -t && nginx -s reload

Risk Controls

• Backup config before reload: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak.
• Avoid nginx -s stop: A graceful reload (reload) does not drop connections.
• Change one parameter at a time: Use ab or wrk to load-test after each change to catch regressions early.
• If concurrency is extremely high, consider adjusting kernel parameters like net.ipv4.ip_local_port_range and net.core.somaxconn, but this affects the host and should be done carefully.

Rollback

If a change worsens the situation, roll back:

1. Restore the backup config: bash cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf
2. Validate config: bash nginx -t
3. Gracefully reload: bash nginx -s reload

Verification

• Functional check: Call key APIs and confirm response codes and latencies return to normal.
• Load test: Use ab -n 10000 -c 500 http://your-gateway/api to observe throughput, latency, and error rates.
• Monitor review: Confirm active connections have decreased, 5xx rate is low, and timeout entries are gone from logs.

When to Submit an OpsGlobal Ticket

Consider submitting a ticket to OpsGlobal when:

• The bottleneck cannot be identified at the application layer and may require kernel tuning or infrastructure upgrades.
• Nginx optimization alone cannot meet SLAs, and a more advanced gateway solution (e.g., Kong, Envoy) is needed.
• Multi-team coordination is required for global traffic policy or security audits.
• You need 24/7 monitoring and incident response. OpsGlobal's professional SRE team can assist.