Nginx Load Balancing Complete Guide

Nginx has powerful reverse proxy and load balancing capabilities built in. With just the upstream block, you can implement Round Robin, Least Connections, IP Hash, and Weighted distribution — all without additional software — building a production-grade load balancing environment.

---

upstream Block Basic Structure

The upstream block is the core of Nginx load balancing. Define your backend server list and distribution method here, then connect with proxy_pass.

# /etc/nginx/conf.d/load-balancer.conf

upstream backend_servers {
    # Default: Round Robin
    server app1.example.com:8080;
    server app2.example.com:8080;
    server app3.example.com:8080;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://backend_servers;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

---

Algorithm Configuration

Round Robin (Default)

Simply list servers without any directive and Round Robin is applied.

upstream backend {
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080;
}

Requests are distributed in order: 10.0.0.1 → 10.0.0.2 → 10.0.0.3 → 10.0.0.1.

Weighted Round Robin

Set weights when servers have different performance. Higher weight means more requests.

upstream backend {
    server 10.0.0.1:8080 weight=5;   # handles 5/8 of traffic
    server 10.0.0.2:8080 weight=2;   # handles 2/8 of traffic
    server 10.0.0.3:8080 weight=1;   # handles 1/8 of traffic
}

Real-world example: When mixing new high-spec (32-core) and old (8-core) servers, set weights proportional to performance.

Least Connections

Sends requests to the server with the fewest active connections. Best for API servers.

upstream backend {
    least_conn;
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080;
}

Can be combined with weights:

upstream backend {
    least_conn;
    server 10.0.0.1:8080 weight=3;
    server 10.0.0.2:8080 weight=1;
}

IP Hash

Determines the server from a hash of the client IP address. The same IP always connects to the same server.

upstream backend {
    ip_hash;
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080;
}

Note: In mobile or NAT environments, many users may share the same IP, concentrating load on a specific server.

Hash (Custom Key)

Uses an arbitrary variable as the hash key. Useful for distributing cache servers by URI.

upstream backend {
    hash $request_uri consistent;   # consistent: use consistent hashing ring
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080;
}

The consistent option enables Consistent Hashing, minimizing the keys affected when servers are added or removed.

---

Complete server Parameter Reference

upstream backend {
    server 10.0.0.1:8080
        weight=3           # Weight (default: 1)
        max_conns=100      # Max concurrent connections (exceeding queues or returns 503)
        max_fails=3        # Max failures within fail_timeout (default: 1)
        fail_timeout=30s   # Failure count window & server exclusion period (default: 10s)
        backup;            # Standby server used only when all others are down

    server 10.0.0.2:8080;

    server 10.0.0.3:8080 down;  # Manually disabled (under maintenance)
}

max_fails and fail_timeout Behavior

3 failures within 30 seconds → exclude that server for 30 seconds
After 30 seconds → automatically try to recover (send 1 test request)
                   success → return to rotation, failure → exclude for another 30s

Production configuration example:

upstream backend {
    server app1.example.com:8080 max_fails=3 fail_timeout=60s;
    server app2.example.com:8080 max_fails=3 fail_timeout=60s;
    server backup.example.com:8080 backup;  # Temporary maintenance page on full failure
}

---

Keepalive Connection Pool

By default, Nginx establishes a new TCP connection with backend servers for each request. Setting keepalive allows connection reuse, significantly improving performance.

upstream backend {
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;

    keepalive 32;            # Idle connections to maintain per worker
    keepalive_requests 100;  # Max requests per connection (Nginx 1.15.3+)
    keepalive_time 1h;       # Maximum connection lifetime
    keepalive_timeout 60s;   # Idle connection timeout
}

server {
    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;        # HTTP/1.1 required for keepalive
        proxy_set_header Connection ""; # Remove Connection: close header
    }
}

Performance benefit: Eliminates TCP 3-way handshake per request, saving several milliseconds. Very effective in high-TPS environments.

---

Production Configuration: Complete Load Balancer Setup

A complete configuration example ready for production use.

# /etc/nginx/conf.d/app-lb.conf

# Log format with upstream server information
log_format upstream_info '$remote_addr - $remote_user [$time_local] '
                          '"$request" $status $body_bytes_sent '
                          '"$http_referer" "$http_user_agent" '
                          'upstream: $upstream_addr '
                          'upstream_time: $upstream_response_time '
                          'request_time: $request_time';

# Upstream group definitions
upstream app_backend {
    least_conn;

    server 10.0.0.1:8080 weight=2 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:8080 weight=2 max_fails=3 fail_timeout=30s;
    server 10.0.0.3:8080 weight=1 max_fails=3 fail_timeout=30s;

    keepalive 32;
}

upstream static_backend {
    server 10.0.0.10:80;
    server 10.0.0.11:80;
    keepalive 16;
}

server {
    listen 80;
    server_name example.com www.example.com;

    access_log /var/log/nginx/access.log upstream_info;
    error_log  /var/log/nginx/error.log warn;

    # Static files go to dedicated server group
    location ~* \.(jpg|jpeg|png|gif|ico|css|js|woff2)$ {
        proxy_pass http://static_backend;
        proxy_set_header Host $host;
        proxy_cache_valid 200 1d;
        expires 7d;
    }

    # API uses least connections
    location /api/ {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_connect_timeout 5s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;

        # Retry configuration
        proxy_next_upstream error timeout http_502 http_503 http_504;
        proxy_next_upstream_tries 3;
        proxy_next_upstream_timeout 10s;
    }

    # All other requests
    location / {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Error page
    error_page 502 503 504 /50x.html;
    location = /50x.html {
        root /usr/share/nginx/html;
        internal;
    }
}

---

proxy_next_upstream: Automatic Retry

Automatically retries with the next server when a server responds with an error.

proxy_next_upstream error timeout http_502 http_503 http_504;
# error    : server connection error
# timeout  : response timeout
# http_502 : Bad Gateway
# http_503 : Service Unavailable
# http_504 : Gateway Timeout

proxy_next_upstream_tries 3;     # Up to 3 retries
proxy_next_upstream_timeout 10s; # Total retry time limit

Warning: Applying retries to non-idempotent requests like POST and PUT risks duplicate processing.

---

Nginx Status Page for Monitoring

Enable the stub_status module to view basic statistics.

server {
    listen 8080;
    server_name localhost;

    location /nginx_status {
        stub_status;
        allow 127.0.0.1;
        allow 10.0.0.0/8;  # Allow internal monitoring server
        deny all;
    }
}

The output looks like:

Active connections: 291
server accepts handled requests
 16630948 16630948 31070465
Reading: 6 Writing: 179 Waiting: 106

---

Algorithm Summary

Algorithm	Directive	Best Situation
Round Robin	(default)	Same-spec servers, short requests
Weighted RR	weight=N	Servers with different specs
Least Connections	least_conn	Variable processing times
IP Hash	ip_hash	Fixed routing without session sharing
Custom Hash	hash $var	URI-based cache distribution

The next page covers Apache mod_proxy_balancer load balancing.