Welcome back to the Load Balancing and Performance Tuning course! You've successfully completed the first lesson and established a solid foundation in basic load balancing with round-robin distribution. Now, as we move into our second lesson, we'll explore more sophisticated load balancing strategies that address specific real-world challenges. Different applications have different needs: some require session persistence to maintain user state, others need to route requests based on content, and many must handle backend servers with varying capacities. In this lesson, we'll implement four advanced load balancing algorithms in NGINX and learn when to apply each one effectively.

While round-robin distribution works well for many scenarios, it has limitations that become apparent in production environments. Consider a situation where one backend server is processing a long-running request while others sit mostly idle. Round-robin would still send the next request to that busy server, even though other servers could handle it more efficiently. Or imagine a web application where users need to maintain session state; round-robin might send consecutive requests from the same user to different servers, breaking their session.

NGINX provides several alternative algorithms to address these challenges:

• Least connections: Routes requests to the server currently handling the fewest active connections.
• IP hash: Ensures requests from the same client always go to the same server.
• Generic hash: Routes based on any custom key, enabling content-based distribution.
• Weighted distribution: Accounts for servers with different processing capacities.

Each algorithm serves distinct use cases, and understanding when to apply each one is key to building robust, efficient systems.

The least_conn algorithm monitors how many active connections each backend server is currently handling and routes new requests to the server with the fewest connections. This approach naturally balances the load more intelligently than round-robin, especially when requests have varying processing times.

Here's how we configure an upstream group with least connections:

The least_conn directive tells NGINX to use connection-based balancing for this upstream group. When a new request arrives, NGINX examines the connection count for each server and selects the one handling the fewest active connections. This is particularly effective for applications where some requests take significantly longer to process than others, as it prevents busy servers from being overwhelmed while others remain underutilized.

Many web applications rely on server-side session state, such as shopping carts, authentication tokens, or user preferences stored in memory. When using standard round-robin, a user's subsequent requests might be routed to different servers that don't have their session data, causing the application to malfunction. The ip_hash algorithm solves this by creating session persistence, also known as sticky sessions.

With ip_hash, NGINX computes a hash value from the client's IP address and uses this to consistently select the same backend server for all requests from that client. As long as the client's IP address doesn't change and the server remains available, all their requests will be routed to the same backend. This maintains session continuity without requiring shared session storage across servers.

The hash directive provides even more flexibility by allowing us to route requests based on any variable or combination of variables. This enables content-based routing, where specific content consistently goes to specific servers. This is valuable for cache optimization, as repeatedly sending requests for the same content to the same server increases cache hit rates.

In this configuration, we hash based on $request_uri, which is the path and query string of the request. The consistent parameter is critical here. Without it, adding or removing a server would cause almost every URI to remap to a different backend—roughly 75% of requests would shift if you added a fourth server to this three-server pool. This invalidates your entire distributed cache simultaneously, forcing all servers to rebuild their caches at once. This cache stampede can overwhelm your origin servers or databases, creating a thundering herd problem.

The consistent parameter uses an algorithm that minimizes this redistribution. When you add or remove a server, only about 1/N of requests (where N is the total number of servers) get remapped—the rest continue routing to their existing servers, preserving their cached content. This makes consistent essential for production environments where you need to scale without triggering catastrophic cache invalidation.

With this setup, requests for /api/users will always go to the same server, requests for /api/products to another, and so on. This creates natural content affinity that improves caching efficiency while allowing safe cluster scaling operations.

Not all servers in a cluster are created equal. Some might have more CPU cores, more memory, or faster disks than others. Round-robin treats all servers as equals, which means powerful servers might be underutilized while weaker ones struggle. Weighted distribution addresses this by assigning different weights to servers based on their capacity.

The weight parameter determines how many requests each server should receive relative to others. In this example, for every six requests, server 5000 receives three, server 5001 receives two, and server 5002 receives one. This allows us to match traffic distribution to actual server capabilities, ensuring optimal resource utilization across heterogeneous infrastructure.

To demonstrate and compare these different algorithms, we need multiple location blocks within our server configuration. Each location will use a different upstream group, allowing us to test various strategies through different URL paths.

This structure creates four distinct endpoints, each demonstrating a different load balancing algorithm. Requests to /least-conn/ will use connection-based balancing, /sticky/ will use IP hash, and so forth.

The location block for least connections proxies to our least_conn_backend upstream and adds headers to help us observe the algorithm's behavior:

Notice the trailing slash in proxy_pass http://least_conn_backend/. This tells NGINX to replace the /least-conn/ portion of the request path when forwarding to the backend. The X-LB-Method header identifies which algorithm was used, while X-Upstream-Addr shows which backend server handled the request, just as we used in the previous lesson.

The sticky session endpoint follows the same pattern but references the sticky_backend upstream:

When testing this endpoint, you'll notice that repeated requests from the same client always route to the same backend server, demonstrating the session affinity provided by IP hash. The X-Upstream-Addr header will show the same server address across multiple requests.

For content-based routing, we use the hash_backend upstream that distributes based on the request URI:

With this configuration, requests to /hash/resource1 will consistently route to one server, while /hash/resource2 might route to a different server. The URI becomes the determining factor, creating natural content affinity.

Finally, the weighted endpoint demonstrates how to handle servers with different capacities:

When sending multiple requests to this endpoint, you'll observe that server 5000 receives three times as many requests as server 5002, and server 5001 receives twice as many, reflecting the weights we configured earlier.

The custom headers we've added serve an important purpose beyond just verification. In production environments, tracking which algorithm and backend server handled each request helps with debugging, performance analysis, and capacity planning:

• X-LB-Method: Identifies the load balancing algorithm used, which is especially useful when running multiple strategies in parallel.
• X-Upstream-Addr: Shows the actual backend server that processed the request, including its IP address and port.

These headers appear in the HTTP response and can be inspected using browser developer tools, command-line utilities like curl, or monitoring systems. They provide visibility into the load balancer's decision-making process.

Each algorithm produces distinct patterns when tested. With least_conn, if you send multiple concurrent requests, you'll see NGINX distributing them to minimize active connections. With ip_hash, consecutive requests from your IP address will consistently show the same X-Upstream-Addr. Using hash with different URIs will produce different backend selections, but the same URI will always route to the same server. And with weighted distribution, you'll observe server 5000 appearing three times more frequently than server 5002 in the response headers.

Understanding these patterns helps you choose the right algorithm for your specific use case and verify that your configuration works as intended.

Excellent work! You've now learned how to implement four distinct load balancing strategies in NGINX. We explored connection-based balancing with least_conn for handling varying request processing times, session persistence with ip_hash for maintaining user state, content-based routing with generic hash for cache optimization, and weighted distribution for heterogeneous server capacities. Each algorithm addresses specific challenges you'll encounter when building production systems. With this knowledge, you can now select the most appropriate strategy for your application's needs. It's time to put these concepts into action through practical exercises that will help you master these advanced load balancing techniques!