Nginx API Gateway Middleware Operations: A Deep Performance Guide

Scenario

Imagine you manage a production environment with a microservices architecture, and Nginx serves as the unified entry point for all API requests. During peak business hours, users start complaining that some endpoints have response times jumping from 200ms to over 5 seconds, and some are timing out. Initial investigation shows the Nginx node's CPU usage remains above 90%, and connection queues are piling up.

Symptoms

• High latency: API average response times increase significantly, and p99 latency exceeds business tolerance thresholds.
• Increased error rate: A large number of 504 Gateway Timeout and 502 Bad Gateway errors appear.
• Connection anomalies: Nginx's active connections approach the max_connections limit, and many connections are in waiting state.
• Resource pressure: Nginx worker processes consume high CPU, and system load average exceeds the number of cores.
• Backend load imbalance: Some upstream services have extremely long response times or even timeouts.

Diagnosis

1. Confirm the Source of Symptoms

First, inspect Nginx error logs and access logs to identify the specific request paths and upstream response statuses that cause errors.

# Error log (typically /var/log/nginx/error.log)
tail -n 100 /var/log/nginx/error.log

# Access log, filter 5xx errors
grep ' 5[0-9][0-9] ' /var/log/nginx/access.log | tail -n 100

If the log contains upstream timed out (110: Connection timed out), it indicates a network timeout between Nginx and the backend service.

2. Inspect Live Connection Status

Nginx provides the stub_status module to get current connection state. Ensure it is enabled in your configuration:

location /nginx_status {
    stub_status;
    allow 127.0.0.1;  # Restrict access to avoid exposing internal info
    deny all;
}

Then execute:

curl http://127.0.0.1/nginx_status
# Example output:
# Active connections: 1234
# server accepts handled requests
#  567890 567890 567890
# Reading: 5 Writing: 30 Waiting: 1199

If the Waiting count remains high, it suggests many connections are idle, possibly due to improper keepalive settings.

3. Check Backend Upstream Health

Use curl to test response times of each upstream:

for url in http://service1:8080/health http://service2:8080/health; do
  echo "$url: $(curl -o /dev/null -s -w '%{http_code} %{time_total}\n' $url)"
done

Also check backend service logs for slow queries or resource bottlenecks.

4. Analyze System Performance

# CPU and memory usage
top -bn1 | head -n 20

# Network connection status
ss -s

# Observe worker process states
ps aux | grep nginx

If nginx: worker process consumes high CPU and each worker is near 100%, it may be CPU-intensive TLS encryption or routing logic.

5. Inspect Kernel and Nginx Configuration

# File descriptor limit
ulimit -n

# TCP connection queue length
cat /proc/sys/net/core/somaxconn

# Current Nginx settings
ginx -T | grep -E 'worker_processes|worker_connections|keepalive|proxy_read_timeout'

Commands and Actions

1. Adjust Nginx Configuration Safely

Before making changes, back up the existing configuration:

cp -r /etc/nginx /etc/nginx.bak.$(date +%Y%m%d%H%M%S)

Modify key parameters, for example:

events {
    worker_connections 10240;
    use epoll;
}

http {
    keepalive_timeout 65;
    keepalive_requests 1000;
    proxy_connect_timeout 5s;
    proxy_read_timeout 30s;
    proxy_send_timeout 30s;
    upstream backend {
        server service1:8080 max_fails=3 fail_timeout=30s;
        server service2:8080 max_fails=3 fail_timeout=30s;
        keepalive 32;  # Number of idle keepalive connections to upstream
    }
}

Key focus: - worker_processes should typically be set to the number of CPU cores to avoid excessive context switching. - worker_connections should be adjusted based on system file descriptor limits and actual concurrency. - Using epoll event model improves performance. - Enable keepalive connection pool for upstream to reduce TCP handshake overhead.

After modification, test the configuration:

nginx -t

If the test passes, perform a graceful reload:

nginx -s reload

Safety Note: nginx -s reload does not interrupt existing connections and is safe. However, if the configuration syntax is invalid, do not force a reload; fix it first.

2. Implement Rate Limiting and Buffering

If the API gateway faces traffic spikes, you can temporarily enable rate limiting:

limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

location /api/ {
    limit_req zone=api_limit burst=20 nodelay;
    proxy_pass http://backend;
}

Set reasonable proxy buffering to avoid memory exhaustion:

proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 8 4k;
proxy_busy_buffers_size 8k;

Risk Controls

• Canary release: Validate configuration changes on a subset of nodes or in a staging environment first, then roll out to production after confirming no anomalies.
• Rollback readiness: Keep the previous configuration version. If error rates spike after reload, you can immediately execute nginx -s reload to revert (requires having old config content saved).
• Monitoring and alerting: During operations, closely monitor key metrics like active connections, error rate, and backend response time. Set up threshold alerts—for example, trigger an alert if error rate exceeds 1% or p99 latency exceeds 500ms.
• Avoid over-tuning: Change only one parameter at a time, and base decisions on measurable data to prevent unpredictable behavior.

Rollback

If the problem worsens after reload, or new errors appear (e.g., connection refused), use the backup configuration to roll back:

# Restore backup
cp /etc/nginx.bak/nginx.conf /etc/nginx/nginx.conf
# Test and reload
nginx -t && nginx -s reload

If the configuration itself is fine but the Nginx process crashes, you can execute:

ginx -s quit   # Graceful exit, waiting for all requests to finish
systemctl restart nginx

Safety Note: Do not use nginx -s stop to force termination, as it may drop ongoing requests.

Verification

• Functional verification: Use curl to test a few critical API endpoints, confirming responses are normal and free of 5xx errors.
• Performance verification: Run curl -w again to measure response times, or use Apache Bench (ab) for simple load testing: bash ab -n 1000 -c 100 https://your-gateway/api/health
• Log verification: Check whether error logs have new entries, and inspect response time distribution in access logs.
• Monitoring metrics: Confirm active connections have dropped, and CPU usage has returned to normal levels.

When to Submit an OpsGlobal Ticket

If the performance issue remains unresolved after following the above steps, or any of the following situations arise, we recommend submitting an OpsGlobal ticket immediately:

• Kernel parameter tuning: Need to modify system-level parameters such as net.ipv4.tcp_tw_reuse or net.core.somaxconn, but are unsure of the impact on higher-level business.
• Architectural issues: For example, upstream services face connection pool exhaustion, requiring load testing to evaluate capacity or designing more complex load balancing strategies.
• Security and compliance requirements: Need to enhance WAF rules without affecting performance, or configure mTLS mutual authentication.
• Persistent performance bottlenecks: After multiple configuration adjustments, SLA requirements still cannot be met; professional performance analysis and tuning recommendations are needed.

OpsGlobal's SRE expert team provides 24×7 remote support and can assist you with deep performance profiling, configuration optimization, and disaster recovery, ensuring your API gateway runs stably and efficiently.

This article was written by the OpsGlobal technical team. We specialize in commercial DevOps/SRE operations support services to help you tackle infrastructure challenges.