7 bond modes of Linux multi-NIC

Linux multi-NIC bonding

There are seven network card bonding modes (0~6): bond0, bond1, bond2, bond3, bond4, bond5, bond6.

There are three commonly used

• mode=0: Load balancing mode with automatic backup, but requires "Switch" support and settings.
• mode=1: Automatic backup mode. If one line is disconnected, other lines will automatically backup.
• mode=6: Load balancing mode with automatic backup, no need for "Switch" support and settings.

It should be noted that if you want to achieve mode 0 load balancing, it is not enough to just set options bond0 miimon=100 mode=0. The switch connected to the network card must be specially configured (the two ports should be aggregated) because the two network cards in bonding use the same MAC address. Let's analyze it from the principle (bond runs in mode 0):

In mode 0, the IP addresses of the network cards bound to the bond are all modified to the same MAC address. If these network cards are connected to the same switch, there will be multiple ports corresponding to this MAC address in the switch's ARP table. So which port should the switch forward the packet sent to this MAC address? Normally, the MAC address is unique in the world. One MAC address corresponding to multiple ports will definitely confuse the switch. Therefore, if the bond in mode 0 is connected to the switch, these ports of the switch should be aggregated (called Ethernet Channel by Cisco and Port Group by Foundry), because after the switch is aggregated, the aggregated ports are also bundled into one MAC address. Our solution is to connect the two network cards to different switches.

[[241871]]

There is no need to configure the switch in mode 6 because the two bonding network cards use different MAC addresses.

Description of the seven bond modes:

The first mode: mod=0, that is: (balance-rr) Round-robin policy

Features: The order of data packets is sequential (i.e. the first packet goes through eth0, the next packet goes through eth1, and so on, until the last packet is transmitted). This mode provides load balancing and fault tolerance. However, if the data packets of a connection or session are sent from different interfaces and pass through different links in the middle, there is a high possibility that the data packets will arrive out of order on the client side. The out-of-order data packets need to be resent, which will reduce the network throughput.

The second mode: mod=1, that is: (active-backup) Active-backup policy (primary-backup policy)

Features: Only one device is active. When one fails, the other immediately switches from backup to master. The MAC address is visible from the outside. From the outside, the MAC address of the bond is unique to avoid confusion in the switch. This mode only provides fault tolerance; it can be seen that the advantage of this algorithm is that it can provide high network connection availability, but its resource utilization is low. Only one interface is in working state. In the case of N network interfaces, the resource utilization is 1/N.

The third mode: mod=2, that is: (balance-xor) XOR policy (balance strategy)

Features: Transmit packets based on the specified transmission HASH strategy. The default strategy is: (source MAC address XOR destination MAC address) % number of slaves. Other transmission strategies can be specified through the xmit_hash_policy option. This mode provides load balancing and fault tolerance.

The fourth mode: mod=3, that is, broadcast (broadcast strategy)

Features: Transmit each data packet on each slave interface. This mode provides fault tolerance.

The fifth mode: mod=4, that is: (802.3ad) IEEE 802.3ad Dynamic link aggregation

Features: Create an aggregation group that shares the same rate and duplex settings. Multiple slaves work in the same activated aggregation according to the 802.3ad specification.

Slave election for outbound traffic is based on a transmit hash policy, which can be changed from the default XOR policy to another policy via the xmit_hash_policy option. Note that not all transmit policies are 802.3ad compliant, especially considering the packet out-of-order issues mentioned in section 43.2.4 of the 802.3ad standard. Different implementations may have different adaptability.

Necessary conditions:

Condition 1: ethtool supports obtaining the speed and duplex settings of each slave

Condition 2: The switch supports IEEE 802.3ad Dynamic link aggregation

Condition 3: Most switches require specific configuration to support 802.3ad mode

The sixth mode: mod=5, that is: (balance-tlb) Adaptive transmit load balancing (adapter transmission load balancing)

Features: Channel bonding does not require any special switch support. Outgoing traffic is distributed on each slave based on the current load (calculated based on speed). If the slave receiving data fails, another slave takes over the MAC address of the failed slave.

Necessary conditions for this mode: ethtool supports obtaining the rate of each slave

The seventh mode: mod=6, that is: (balance-alb) Adaptive load balancing (adaptive load balancing)

Features: This mode includes the balance-tlb mode, plus receive load balancing (rlb) for IPv4 traffic, and does not require any switch support. Receive load balancing is achieved through ARP negotiation. The bonding driver intercepts the ARP reply sent by the local machine and rewrites the source hardware address to the unique hardware address of a slave in the bond, so that different peers use different hardware addresses for communication.

Receive traffic from the server is also balanced. When the local machine sends an ARP request, the bonding driver copies the peer's IP information from the ARP packet and saves it. When the ARP reply arrives from the peer, the bonding driver extracts its hardware address and initiates an ARP reply to one of the slaves in the bond. One problem with using ARP negotiation for load balancing is that the bond's hardware address is used every time an ARP request is broadcast, so once the peer learns this hardware address, all receive traffic will flow to the current slave. This problem can be solved by sending updates (ARP replies) to all peers, which contain their unique hardware addresses, resulting in a redistribution of traffic. When a new slave is added to the bond, or an inactivated slave is reactivated, receive traffic is also redistributed. The receive load is distributed sequentially (round robin) to the fastest slaves in the bond.

When a link is reconnected, or a new slave is added to the bond, the incoming traffic is redistributed among all currently active slaves, and an ARP reply is sent to each client using the specified MAC address. The updelay parameter described below must be set to a value greater than or equal to the switch forwarding delay to ensure that the ARP reply sent to the other end is not blocked by the switch.

Necessary conditions:

Condition 1: ethtool supports obtaining the rate of each slave;

Condition 2: The underlying driver supports setting the hardware address of a device so that there is always a slave (curr_active_slave) using the bond's hardware address, and each slave in the bond has a unique hardware address. If the curr_active_slave fails, its hardware address will be taken over by the newly elected curr_active_slave

In fact, the difference between mod=6 and mod=0 is: mod=6, first occupies eth0 traffic, then eth1, ....ethX; while if mod=0, you will find that the traffic of the two ports is very stable, with basically the same bandwidth. And if mod=6, you will find that the traffic of the first port is very high, and the second port only occupies a small part of the traffic.

Linux network port binding

Through the network port bonding technology, it is easy to achieve network port redundancy and load balancing, thereby achieving the purpose of high availability and high reliability. Prerequisites:

The two physical network ports are: eth0, eth1

The virtual port after binding is: bond0

The server IP is: 192.168.0.100

The first step is to configure the settings file:

 /etc/sysconfig/network-scripts/ifcfg-bond0  
 DEVICE=bond0  
 BOOTPROTO=none  
 ONBOOT=yes  
 IPADDR=192.168.0.100  
 NETMASK=255.255.255.0  
 NETWORK=192.168.0.0  
 BROADCAST=192.168.0.255  
 BROADCAST broadcast address  
 /etc/sysconfig/network-scripts/ifcfg-eth0  
 DEVICE=eth0  
 BOOTPROTO=none  
 MASTER=bond0  
 SLAVE=yes  
 /etc/sysconfig/network-scripts/ifcfg-eth1  
 DEVICE=eth1  
 BOOTPROTO=none  
 MASTER=bond0  
 SLAVE=yes

The second step is to modify the modprobe related setting files and load the bonding module:

1. Here, we directly create a dedicated configuration file /etc/modprobe.d/bonding.conf to load bonding

 [root@test ~]# vi /etc/modprobe.d/bonding.conf

Append

 alias bond0 bonding
 options bonding mode=0 miimon=200

2. Load the module (you don’t need to load it manually after restarting the system)

 [root@test ~]# modprobe bonding

3. Confirm whether the module is loaded successfully:

 [root@test ~]# lsmod | grep bonding
 bonding 100065 0

The third step is to restart the network and then confirm the status:

 [root@test ~]# /etc/init.d/network restart  
 [root@test ~]# cat /proc/net/bonding/bond0  
 Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)  
 Bonding Mode: fault-tolerance (active-backup)  
 Primary Slave: None  
 Currently Active Slave: eth0  
 … 
 
 [root@test ~]# ifconfig | grep HWaddr  
 bond0 Link encap:Ethernet HWaddr 00:16:36:1B:BB:74  
 eth0 Link encap:Ethernet HWaddr 00:16:36:1B:BB:74  
 eth1 Link encap:Ethernet HWaddr 00:16:36:1B:BB:74

From the confirmation information above, we can see 3 important information:

1. The current bonding mode is active-backup

2. The current Active network port is eth0

3. The physical address of bond0,eth1 is the same as the physical address of eth0 in active state, so as to avoid confusion in the upper switch.

Unplug any network cable and then access your server to see if the network is still accessible.

Step 4: The system starts automatic binding and adds the default gateway:

 [root@test ~]# vi /etc/rc.d/rc. local  

Append

 ifenslave bond0 eth0 eth1
 route add   default gw 192.168.0.1

If you can access the Internet, you don't need to add a route, and the 0.1 address can be modified according to the environment.

Note: The above example only shows two network ports bound into bond0. If we want to set up multiple bond ports, for example, physical network ports eth0 and eth1 form bond0, and eth2 and eth3 form bond1,

Then the method of setting the network port setting file is the same as the method described in step 1 above, except that the settings of /etc/modprobe.d/bonding.conf cannot be simply superimposed as follows:

 alias bond0 bonding  
 options bonding mode=1 miimon=200  
 alias bond1 bonding  
 options bonding mode=1 miimon=200

There are two correct settings:

The first one, you can see, in this way, the modes of multiple bond ports can only be set to the same:

 alias bond0 bonding  
 alias bond1 bonding  
 options bonding max_bonds=2 miimon=200 mode=1

The second method is to set the modes of different bond ports to different values:

 alias bond0 bonding  
 options bond0 miimon=100 mode=1  
 install bond1 /sbin/modprobe bonding -o bond1 miimon=200 mode=0

Take a close look at the two setup methods above. Now if you want to set up 3, 4, or even more bond ports, you should be able to do it!

Postscript: Here is a brief introduction to the meaning of some parameters in options when loading the bonding module:

miimon monitors the frequency of network links in milliseconds. We set it to 200 milliseconds.

max_bonds: The number of bond ports configured

There are mainly the following mode bond modes. In general practical applications, 0 and 1 are more commonly used.

If you want to understand the characteristics of each of these models in depth, you will need to look up information and practice it yourself.