In-depth understanding of classic load balancing cases

Preface

I have nothing to do so I write an article to summarize load balancing.

Load Balancing

Load balancing is a key component of high-availability architecture, which is mainly used to improve performance and availability. Load balancing distributes traffic to multiple servers, and multiple servers can eliminate single points of failure .

Of course, the load balancer itself is a single point of failure. You can consider the load balancing dual-machine hot standby or other solutions mentioned later in the article to eliminate single points of failure and improve availability.

A web architecture without load balancing generally looks like this:

No load balancing architecture

In this example, users connect to the Web Server directly through yourdomain.com. If this Web Server fails, the entire system will be unusable, which is what we often call a single point of failure in the system. Similarly, if a large number of users access this server at the same time, then these users are likely to experience slow loading times or be unable to connect at all.

This single point of failure can be mitigated by introducing a load balancer and at least one other Web Server. Generally speaking, all backend servers will provide the same content so that users will receive consistent content no matter which server they visit. At the same time, since there are multiple servers providing services at the same time, the system's load capacity is increased and performance is improved.

What types of traffic can load balancing handle?

Since most of the load balancing that general programmers come into contact with may handle HTTP and HTTPS traffic, but in fact load balancing can also handle TCP and UDP traffic (such as access to database clusters, DNS, etc.).

Load Balancing Algorithm

The load balancing algorithm is used to determine which healthy server the traffic should be distributed to. Several common algorithms are as follows:

Round Robin — Round Robin means that servers are selected in order, for example, the load balancer will assign the first request to the first server, and then the next request to the second server, and so on, after a round of allocation, it will go back to the beginning and assign it to the first server (review the operating system scheduling algorithm). This method is more suitable when the processing power of each server is the same and the processing volume of each business is similar.

Least Connections — The Least Connections algorithm means that the load balancer will choose the server with the least current connections.

IP hash — In this algorithm, the load balancer determines which server to distribute the request to based on the IP address of the request source. This method ensures that a specific user always accesses the same server.

There are some other less common algorithms, such as Url hash, Random , etc.

Health checks

In the load balancing algorithm section, we have a premise that traffic will only be distributed to healthy servers. So how does the load balancer determine whether the server is healthy now?

To monitor healthy servers, health checks typically attempt to connect to the server using the configured protocol and port to ensure that the server is listening. If a server fails a health check, meaning that the server is no longer able to respond to requests, it will be automatically removed from the pool and traffic will not be directed to the broken server until it passes the health check.

For more information, please refer to Alibaba Cloud's document on load balancing.

How does load balancing handle state?

We all know that session-based user authentication will store some session information on the server, but this will cause some problems when load balancing is introduced into the system.

Take an e-commerce website as an example. When the login request sent by user U is distributed to server S1 and the session information is recorded in the server, when the user wants to submit a shopping request, the request is distributed to server S2, but server S2 does not save the session information of user U.

To solve this problem, one way is to use the IP hash algorithm mentioned earlier. This algorithm assigns the corresponding server to the traffic based on the IP, so that the traffic of the same user will access the same server. Another application layer method is sticky session , which should be called sticky session in Chinese. The load balancer will set a cookie and then all sessions with this cookie will be assigned to the same server.

Load balancing hot standby

As mentioned at the beginning, the load balancer itself is a single point of failure. One solution is hot standby (a basic way to improve availability is redundancy).

In order to solve the single point failure problem of the load balancer, the dual-machine hot standby solution introduces a second load balancer. When the primary node fails, it switches to the backup node. I found a more vivid gif on the Internet:

The architecture of my previous graduation project used dual-machine hot standby, and the implementation was mainly through keepalived to achieve high availability of nginx

postscript

This article is a preliminary summary of load balancing and some of my own understanding. It is more suitable for people who want to have a preliminary and comprehensive understanding of load balancing. However, since I am just a newbie, many advanced things such as LVS and practical analysis of some large companies have not been added, and I am not able to add them for the time being. I will make up for it if I have contact with it in the future (I have dug a pit first).