How do Internet giants achieve load balancing and high availability? You will understand after reading this article

[[284447]]

1. What is load balancing?

What is load balancing?

I remember the first time I came into contact with Nginx was in the lab. At that time, Nginx was needed to deploy websites on the server. Nginx is a service component used for reverse proxy, load balancing, HTTP caching, etc. So what is load balancing here?

Load balancing (LB) is a technical solution used to distribute load among multiple resources (usually servers) to optimize resource usage and avoid overload.

Resources are equivalent to the execution operation unit of each service instance. Load balancing is to distribute a large number of data processing operations to multiple operation units for execution, which is used to solve the problems of large traffic, high concurrency and high availability of Internet distributed systems. So what is high availability?

2. What is high availability?

First, what is high availability?

This is the CAP theorem, which is the basis of distributed systems and also the three indicators of distributed systems:

1. Consistency
2. Availability
3. Partition tolerance

What is High Availability? High availability, or HA for short, is a characteristic or indicator of a system, usually referring to providing a service running time with certain performance that is higher than the average normal time period. Conversely, eliminating the time when the system service is unavailable.

The measure of whether the system meets high availability is that when one or more servers go down, the overall system and services are still available normally.

For example, some well-known websites guarantee an availability of more than 4 9s, which means an availability of more than 99.99%. Then 0.01% is the percentage of so-called failure time. For example, the e-commerce website Youzan, service unavailability will cause merchants to lose money and users. So on the basis of improving availability, there will be compensation for system downtime and service unavailability.

For example, for order services, you can use multiple order service instances with load balancing instead of a single order service instance, that is, use redundancy to improve reliability.

In short, load balancing is one of the factors that must be considered in the design of distributed system architecture. Generally, load balancing and redundancy of the same service instance are used to solve the problems of large traffic, high concurrency and high availability of distributed systems. The key to load balancing is whether the distribution is even.

3. Common load balancing cases

Scenario 1: In a microservice architecture, the gateway routes to a specific service instance hello:

Two identical service instances hello service, one on port 8000 and the other on port 8082

Through Kong's load balancing LB function, the requests are evenly distributed to the two hello service instances.

Kong has many load balancing strategy algorithms: the default weighted-round-robin algorithm, consumer: consumer id as the hash algorithm input value, etc.

Scenario 2: In a microservice architecture, service A calls a cluster of service B. This is done through the Ribbon client load balancing component:

• The load balancing strategy algorithm is not advanced. The simplest ones are random selection and polling.

4. Internet Distributed System Solutions

The common Internet distributed system architecture is divided into several layers, generally as follows:

• Client layer: such as user browser, APP end
• Reverse proxy layer: Technology selection Nignx or F5, etc.
• Web layer: In the scenario of front-end and back-end separation, the Web side can use NodeJS, RN, and Vue
• Business service layer: Java, Go, general Internet companies, the technical solution selection is SC or Spring Boot + Dubbo service
• Data storage layer: DB selection type MySQL, Cache selection type Redis, Search selection type ES, etc.

A request from layer 1 to layer 4 requires load balancing at each layer. That is, when each upstream calls multiple downstream business parties, the calls need to be evenly distributed. In this way, the overall system is more load balanced.

Layer 1: Client layer -> reverse proxy layer load balancing

How to implement load balancing from client layer to reverse proxy layer?

The answer is: DNS polling. DNS can set multiple IP addresses through A (Address, returning the IP address pointed to by the domain name). For example, the DNS for accessing bysocket.com here is configured with ip1 and ip2. For high availability of the reverse proxy layer, there will be at least two A records. In this way, the two redundant ips correspond to the nginx service instances to prevent single point failure.

Each time the bysocket.com domain name is requested, the corresponding IP address is returned through DNS polling. Each IP corresponds to the service instance of the reverse proxy layer, which is the external IP of nginx. In this way, the request distribution to each reverse proxy layer instance is balanced.

Layer 2: Reverse proxy layer -> Web layer load balancing

How to implement load balancing from reverse proxy layer to web layer?

It is handled by the load balancing module of the reverse proxy layer. For example, nginx has multiple balancing methods:

1. Request polling. Requests are assigned to web layer services one by one in chronological order, and then repeated over and over again. If the web layer service is down, it is automatically removed.

 upstream web-server {
 server ip3;
 server ip4;
 }

IP hash. According to the IP hash value, determine the route to the corresponding web layer. As long as the user's IP is uniform, the request to the web layer is also uniform.

1. Another benefit is that requests from the same IP address will be distributed to the same web layer service. This way, each user can access a fixed web layer service, which can solve the session problem.

 upstream web-server {
 ip_hash;
 server ip3;
 server ip4;
 }

weight weight, fair, url_hash, etc.

Layer 3: Load balancing of the web layer -> business service layer

How to implement load balancing from web layer to business service layer?

For example, Dubbo is a service governance solution that includes service registration, service degradation, access control, dynamic configuration of routing rules, weight adjustment, and load balancing. One of its features is intelligent load balancing: it has multiple built-in load balancing strategies, intelligently perceives the health status of downstream nodes, significantly reduces call delays, and improves system throughput.

In order to avoid single point failure and support horizontal expansion of services, a service is usually deployed with multiple instances, that is, Dubbo cluster deployment. Multiple service instances will be combined into one service provider, and then according to the configured random load balancing strategy, one of the 20 providers will be randomly selected for calling. Suppose the 7th provider is randomly selected. The LoadBalance component uses the balancing strategy to select a provider from the provider address list to call. If the call fails, another provider will be selected for calling.

Dubbo has four built-in load balancing strategies:

• RandomLoadBalance: Random load balancing. Randomly select one. It is the default load balancing strategy of Dubbo.
• RoundRobinLoadBalance: Round Robin Load Balancing. Select one in round robin.
• LeastActiveLoadBalance: minimum number of active calls, random with the same number of active calls. The active number refers to the difference between the counts before and after the call. Make the slow Provider receive fewer requests, because the slower the Provider, the greater the difference between the counts before and after the call.
• ConsistentHashLoadBalance: Consistent hash load balancing. Requests with the same parameters always fall on the same machine.

Similarly, due to business needs, you can also implement your own load balancing strategy

Layer 4: Business service layer -> data storage layer load balancing

The load balancing of the data storage layer is usually achieved through DBProxy, such as MySQL sharding.

When a single database or table has too much access and too much data, vertical and horizontal splitting are required. For example, the horizontal splitting rule is:

• Range, Time
• Hash modulus, orders based on store ID, etc.

However, the following problems will arise with this load and need to be solved:

• Distributed Transactions
• Cross-database join, etc.

There are many product solutions for sharding: Dangdang Sharding-JDBC, Alibaba Cobar, etc.

V. Summary

From an external point of view, load balancing is a whole system or software. From an internal point of view, there are calls from upstream to downstream. As long as there are calls, the factor of load balancing needs to be considered. Therefore, load balancing is one of the factors that must be considered in the design of distributed system architecture. The main consideration is how to make the requests received by the downstream evenly distributed:

Layer 1: Client layer -> reverse proxy layer load balancing. Through DNS polling

Layer 2: Reverse proxy layer -> Web layer load balancing. Through Nginx's load balancing module

Layer 3: Load balancing of the Web layer -> business service layer. Through the load balancing module of the service governance framework

Layer 4: Business service layer -> data storage layer load balancing. Through the horizontal distribution of data, the data is evenly distributed, and theoretically the requests will also be evenly distributed. For example, sharding by buyer ID is similar.