Nginx's four-layer load balancing (Stream module) configuration based on TCP/UDP ports

HTTP load balancing, which is what we usually call "seven-layer load balancing", works at the seventh layer "application layer". TCP load balancing, which is what we usually call "four-layer load balancing", works at the "network layer" and "transport layer". For example, LVS (Linux Virtual Server) and F5 (a hardware load balancing device) also belong to "four-layer load balancing".

[[276718]]

 nginx-1.9.0 has been released. This version adds the stream module for general TCP proxy and load balancing. The ngx_stream_core_module module will be enabled after version 1.90. However, it will not be installed by default.
 This module needs to be activated by specifying the --with-stream parameter at compile time.   
  
 1) Configure Nginx compilation file parameters
 ./configure --with-http_stub_status_module --with-stream  
 -------------------------------------------------------------------------------   
  
 2) Compile and install, make && make install
 -------------------------------------------------------------------------------   
  
 3) Configure nginx.conf file 
  
 stream {
 upstream kevin {
 server 192.168.10.10:8080; #This is configured as the address to be accessed
 server 192.168.10.20:8081;
 server 192.168.10.30:8081; #The port that needs to be proxied. Here I proxy a kevin module interface 8081
 }
 server {
 listen 8081; #Port to be listened
 proxy_timeout 20s;
 proxy_pass kevin;
 }
 } 
  
 Create the highest level stream (the same level as http), define an upstream group named kevin, which consists of multiple services to achieve load balancing, define a service to listen for TCP connections (such as port 8081),
 And proxy them to Kevin in an upstream group, configure the load balancing method and parameters for each server; configure some things such as: number of connections, weight, etc. 
  
 First, create a server group to be used as a TCP load balancing group. Define an upstream block in the stream context, add the server defined by the server command to this block, specify its IP address and
 The host name (a host name that can be resolved into multiple addresses) and the port number. The following example creates a group called kevin, with two servers listening on port 1395 and one server listening on port 8080. 
  
 upstream kevin {
 server 192.168.10.10:8080; #This is configured as the address to be accessed
 server 192.168.10.20:8081;
 server 192.168.10.30:8081; #The port that needs to be proxied. Here I proxy a kevin module interface 8081
 } 
  
  
 It is important to note that:
 You cannot define the protocol for each server because the stream command establishes TCP as the protocol for the entire server. 
  
 Configure reverse proxy to enable Nginx to forward TCP requests from a client to a load balancing group (e.g., kevin group). In each server configuration block, pass the server configuration information of each virtual server and
 The configuration information of the listening port defined in each server (the proxy port number required by the client, such as the Kevin protocol, the port number is: 8081) and the proxy_passs command send TCP communication to
 Next, we will send TCP communication to the kevin group. 
  
 server {
 listen 8081; #Port to be listened
 proxy_timeout 20s;
 proxy_pass kevin;
 } 
  
 Of course, we can also use a single proxy method: 
  
 server {
 listen 8081; #Port to be listened
 proxy_timeout 20s;
 proxy_pass 192.168.10.30:8081; #The port that needs to be proxied. Here I proxy the interface 8081 of a kevin module
 }
 -------------------------------------------------------------------------------   
  
 4) Change the load balancing method:
 By default, nginx uses a round-robin algorithm to load balance communications. It directs the request cycle to the server port configured in the upstream group. Because it is the default method, there is no polling command here.
 Simply create an upstream configuration group in this stream context and add the server to it. 
  
 a) least-connected: For each request, nginx plus selects the server with the least number of current connections to handle: 
  
 upstream kevin {
 least_conn;
 server 192.168.10.10:8080; #This is configured as the address to be accessed
 server 192.168.10.20:8081;
 server 192.168.10.30:8081; #The port that needs to be proxied. Here I proxy a kevin module interface 8081
 } 
  
 b) Least time : For each connection, nginx pluns selects the server based on several points: The lowest average delay: calculated by the parameters specified in the least_time command:
 connect : The time it takes to connect to a server
 first_byte: The time when the first byte is received
 last_byte: the time when all data is received. Minimum number of active connections: 
  
 upstream kevin {
 least_time first_byte;
 server 192.168.10.10:8080; #This is configured as the address to be accessed
 server 192.168.10.20:8081;
 server 192.168.10.30:8081; #The port that needs to be proxied. Here I proxy a kevin module interface 8081
 } 
  
 c) Ordinary hash algorithm: nginx plus selects this server through the user_defined keyword, which is the IP address: $remote_addr; 
  
 upstream kevin {
    hash $remote_addr consistent;
 server 192.168.10.10:8080 weight=5; #This is configured as the address to be accessed
 server 192.168.10.20:8081 max_fails=2 fail_timeout=30s;
 server 192.168.10.30:8081 max_conns=3; #The port that needs to be proxied. Here I proxy a kevin module interface 8081
 }

The execution principle of Nginx TCP load balancing

When Nginx receives a new client connection from the listening port, it immediately executes the routing scheduling algorithm, obtains the specified service IP that needs to be connected, and then creates a new upstream connection to connect to the specified server.

TCP load balancing supports Nginx's original scheduling algorithms, including Round Robin (default, polling scheduling), Hash (select consistent), etc. At the same time, the scheduling information data will also work with the robustness detection module to select the appropriate target upstream server for each connection. If you use the Hash load balancing scheduling method, you can use $remote_addr (client IP) to achieve a simple persistent session (the connection with the same client IP always falls on the same service server).

Like other upstream modules, the TCP stream module also supports custom forwarding weights for load balancing (configuration "weight=2"), as well as backup and down parameters for kicking out failed upstream servers. The max_conns parameter can limit the number of TCP connections to a server. Setting an appropriate configuration value based on the server's capacity can achieve overload protection, especially in high-concurrency scenarios.

Nginx monitors client connections and upstream connections. Once data is received, Nginx will immediately read and push it to the upstream connection without doing data detection within the TCP connection. Nginx maintains a memory buffer for writing client and upstream data. If the client or server transmits a large amount of data, the buffer will increase the memory size appropriately.

When Nginx receives a connection closing notification from either party, or the TCP connection is idle for more than the proxy_timeout configuration time, the connection will be closed. For TCP long connections, we should choose an appropriate proxy_timeout time, and at the same time, pay attention to the so_keepalive parameter of the listening socke to prevent premature disconnection.

Nginx TCP load balancing service robustness monitoring

The TCP load balancing module supports built-in robustness detection. If an upstream server refuses TCP connections for more than the time configured by proxy_connect_timeout, it will be considered invalid. In this case, Nginx immediately tries to connect to another normal server in the upstream group. The connection failure information will be recorded in the Nginx error log.

If a server fails repeatedly (exceeding the max_fails or fail_timeout configuration parameters), Nginx will also kick the server. After the server is kicked off for 60 seconds, Nginx will occasionally try to reconnect to it to detect whether it has recovered. If the server is back to normal, Nginx will add it back to the upstream group and slowly increase the proportion of connection requests.

The reason why it "slowly increases" is that usually a service has "hot data", that is, more than 80% or even more requests are actually blocked in the "hot data cache", and only a small number of requests are actually executed. When the machine is just started, the "hot data cache" is not actually established. At this time, a large number of requests are forwarded explosively, which may cause the machine to be unable to "bear" and hang again. Take MySQL as an example. Usually, more than 95% of our MySQL queries fall into the memory cache, and not many queries are actually executed.

In fact, whether it is a single machine or a cluster, in a high-concurrency request scenario, restart or switch, there is this risk. There are mainly two ways to solve it:

1) Requests gradually increase from a small number to a large number, gradually accumulating hot data and eventually reaching a normal service status.

2) Prepare "commonly used" data in advance and proactively "preheat" the service. After the preheat is complete, open access to the server.

TCP load balancing is consistent with LVS in principle, and works at a lower level. Its performance is much higher than the original HTTP load balancing. However, it is not better than LVS. LVS is placed in the kernel module, while Nginx works in user mode. In addition, Nginx is relatively heavy. Another point is that it is a pity that this module is a paid feature.