Nginx Load Balancing Setup: Multi-Server Guide

This guide walks you through setting up Nginx as a load balancer to distribute incoming traffic across multiple backend servers. You will configure Nginx to proxy requests to a group of servers, improving reliability and performance. By the end, you will have a working load-balanced setup with health checks and basic tuning.

What You’ll Need

Two or more backend servers (physical or virtual) running your application (e.g., web server, API).
One dedicated server (or the same server) running Nginx to act as the load balancer. This can be a separate machine or the same as one of the backends, but a separate one is recommended.
Root or sudo access on all servers.
Nginx installed on the load balancer. Most Linux distributions have packages; on Ubuntu/Debian: sudo apt install nginx; on RHEL/CentOS: sudo yum install nginx.
Basic familiarity with the command line and editing config files.

Step-by-Step

Identify your backend servers. Note the IP addresses or hostnames of the servers that will serve the actual content. For example, 192.168.1.10 and 192.168.1.11.
Create a backup of the default Nginx config. On the load balancer, copy the default site config: sudo cp /etc/nginx/sites-available/default /etc/nginx/sites-available/default.bak (or /etc/nginx/conf.d/default.conf depending on your setup).
Define an upstream block. Open the Nginx config file (e.g., /etc/nginx/sites-available/default) and add an upstream block inside the http context (usually at the top of the file, outside the server block). Example:
```
upstream backend {
    server 192.168.1.10;
    server 192.168.1.11;
}
```
This creates a group named backend with two servers. Nginx will use round-robin by default.

Configure the server block to proxy requests. Inside your server block, set the location to forward all requests to the upstream group. For example:

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

The proxy_set_header lines preserve the original client IP and host, which many applications need.

Test the configuration. Run sudo nginx -t to check for syntax errors. If all is well, you’ll see “syntax is ok” and “test is successful”.
Reload Nginx. Apply the changes with sudo systemctl reload nginx (or sudo service nginx reload).
Verify load balancing. Access your load balancer’s IP or domain in a browser. Check the logs on each backend server to see that requests are being distributed. Alternatively, you can temporarily add a unique identifier (like a hostname) to each backend’s response to verify distribution.
Add health checks (optional but recommended). To avoid sending traffic to a down server, add a max_fails and fail_timeout to each server in the upstream block:
```
upstream backend {
    server 192.168.1.10 max_fails=3 fail_timeout=30s;
    server 192.168.1.11 max_fails=3 fail_timeout=30s;
}
```
Nginx will stop sending requests to a server that fails 3 times within 30 seconds, then retry after the timeout.
Choose a load balancing method (optional). By default, Nginx uses round-robin. You can change it by adding a directive like least_conn; (sends to the server with fewest active connections) or ip_hash; (ensures the same client IP always goes to the same server, useful for sessions). Add it inside the upstream block.

Common Mistakes

Forgetting to set proxy headers. Without proxy_set_header Host $host, the backend may see the load balancer’s hostname instead of the original domain, breaking virtual hosting.
Not testing with nginx -t before reloading. A syntax error will cause Nginx to fail to reload, potentially taking down your site.
Using the same port for upstream and listen. If your backend servers listen on port 80, and your load balancer also listens on 80, that’s fine, but ensure the upstream block uses the correct port (e.g., server 192.168.1.10:8080; if your app runs on 8080).
Not adjusting firewall rules. The load balancer must be able to reach the backend servers on the specified port. Check firewalls (iptables, security groups) on both sides.
Ignoring sticky sessions. If your application stores session data locally, round-robin will cause session loss. Use ip_hash or a shared session store.

Tips

Use a dedicated upstream config file. Place your upstream block in a separate file like /etc/nginx/conf.d/upstreams.conf and include it in the main config. This keeps things organized.
Enable gzip compression. Add gzip on; in the http block to reduce bandwidth.
Set appropriate timeouts. Add proxy_connect_timeout, proxy_send_timeout, and proxy_read_timeout to avoid hanging connections.
Monitor with access logs. Check the load balancer’s access log to see the distribution of requests.
Consider SSL termination. If you use HTTPS, you can terminate SSL at the load balancer and forward plain HTTP to backends, reducing their CPU load. Use listen 443 ssl; and configure certificates.

Troubleshooting

502 Bad Gateway. This usually means Nginx cannot reach the backend servers. Check that the backend servers are running, the IP/port in the upstream block is correct, and firewalls allow traffic.
504 Gateway Timeout. The backend is taking too long to respond. Increase proxy_read_timeout or investigate why the backend is slow.
Requests all go to one server. If you’re using ip_hash, that’s expected for the same client IP. Otherwise, check that all servers are marked as up in the upstream block and that health checks aren’t failing.
Nginx fails to reload. Run sudo nginx -t to see the error. Common issues: missing semicolons, duplicate server names, or invalid directives.
Backend sees the load balancer’s IP. Ensure you set proxy_set_header X-Real-IP $remote_addr; and X-Forwarded-For so the backend can log the real client IP.

Frequently Asked Questions

What load balancing algorithms does Nginx support?

Nginx supports round-robin (default), least_conn, ip_hash, and generic hash. You can also use third-party modules for more advanced methods.

Can I load balance HTTPS traffic?

Yes. Configure Nginx to listen on port 443 with SSL, and then proxy to your backends over HTTP or HTTPS. You’ll need to set proxy_ssl_server_name on; if proxying to HTTPS backends.

How do I add a new backend server later?

Simply add a new server line in the upstream block and reload Nginx. No downtime is required.

What happens if one backend goes down?

With health checks (max_fails/fail_timeout), Nginx will stop sending traffic to that server and continue with the others. Without them, requests may fail intermittently.

Can I use Nginx as a load balancer for non-HTTP protocols?

Nginx supports TCP and UDP load balancing with the stream module. You’ll need to configure it separately from HTTP.

Is it better to use a dedicated load balancer or the same server as a backend?

A dedicated load balancer is better for performance and isolation. If you use the same server, ensure it has enough resources to handle both roles.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →