Linux SRE Runbook: Troubleshooting Out-of-Memory (OOM) Kills in Production

Scenario

A critical application on a production Linux server is frequently terminated by the Out-Of-Memory (OOM) Killer, causing service disruptions.

Symptoms

• Application crashes; client requests fail
• System logs show OOM Killer messages: dmesg | grep -i oom outputs Killed process
• Memory usage gradually increases over time; free -m shows declining available memory
• top or htop indicates one or more processes consuming large amounts of memory

Diagnosis

1. Confirm OOM killer events: dmesg -T | grep -i 'oom-killer' or journalctl -k | grep -i oom
2. View current memory usage: free -h, cat /proc/meminfo, vmstat -s
3. Identify memory hogs: ps aux --sort=-%mem | head or top -o %MEM
4. Deep dive into process memory mapping: pmap -x <PID> | sort -n -k2 | tail or smem -t -p -k | sort -n
5. Check cgroup memory limits (if using containers): cat /sys/fs/cgroup/memory/memory.usage_in_bytes
6. Inspect system memory configuration: sysctl vm.overcommit_memory, sysctl vm.panic_on_oom

Commands

# View recent OOM events
dmesg -T | grep -i 'oom-killer'

# Check total and used memory
free -h

# List processes sorted by memory usage
ps aux --sort=-%mem | head -20

# Detailed memory map of a process (example PID 12345)
pmap -x 12345 | sort -n -k2 | tail -10

# Using smem to see USS/PSS
smem -t -p -k -c "pid username command pss uss"

# Temporary OOM behavior adjustment
echo 2 > /proc/sys/vm/overcommit_memory  # disable overcommit
echo 0 > /proc/sys/vm/panic_on_oom      # usually 0, 1 causes kernel panic

# Restart the problematic service (e.g., nginx)
systemctl restart nginx

Risk Controls

• Before killing a process, confirm it is safe to terminate (e.g., systemctl status to check if it's a core service).
• Use renice +10 -p <PID> to lower the priority of the troublemaker, potentially avoiding immediate OOM.
• Setting vm.overcommit_memory=2 reduces overcommit, but may cause memory allocation failures; apply during low traffic.
• If the process is legitimate business logic, consider adding physical memory or setting cgroup limits.

Rollback

• If the wrong process is killed, restart the service using its command: systemctl start <service> or run the executable manually.
• If sysctl changes cause issues, revert to defaults: bash sysctl -w vm.overcommit_memory=0 sysctl -w vm.panic_on_oom=0 or restore from /etc/sysctl.conf backup.

Verification

• Monitor MemAvailable in /proc/meminfo for stability.
• Check dmesg for no new OOM killer messages.
• Application health check endpoint returns HTTP 200.
• Load test with normal traffic confirms no memory leak.

When to Submit an OpsGlobal Ticket

• Root cause remains unclear after following the above steps.
• Kernel parameter tuning requires global production changes.
• Suspected kernel bug or need for professional monitoring setup (e.g., Prometheus + Grafana).
• Need OpsGlobal engineers to help create automated recovery scripts or custom runbooks.