"One arrow is easy to break, but ten arrows are hard to break" - From single link to MC-LAG

"It is easy to break one arrow, but difficult to break ten arrows."

This is a proverb story that I often heard when I was a child. The literal meaning is that a single arrow is easily broken, but multiple arrows tied together are difficult to destroy. This tells us the importance of "unity and cooperation".

In the field of bearing, this story also has a wonderful use. Let's take a look at it together~

Single link, one arrow can easily break

​In the early days of IP networking, two network devices (A and B) generally communicated by establishing a single link. ​

Obviously, using a single link connection has the following drawbacks:

• There is a bandwidth bottleneck. The total bandwidth between devices A and B is equal to the bandwidth of this single link.
• The link has no redundant backup. Once a problem occurs on this link, communication between devices A and B may be interrupted.

Even though a single link has obvious flaws, it can still meet the needs of the public in an era when network requirements were not high.

As the scale of networks grows, a single link can no longer meet the bandwidth and reliability requirements of the network, so people came up with the idea of ​​bundling multiple links like arrows. As a result, link aggregation technology came into being.

Link Aggregation: Ten Arrows Can’t Be Failed

Link aggregation "bundles" multiple links between two devices to form an aggregation group. The multiple links in the aggregation group can now be regarded as one logical link.

At this point, the traffic between the two devices can be shared on each link in the aggregation group.

So, what benefits can link aggregation bring to the network?

Increased network bandwidth

Multiple links between A and B are bundled into one logical link. The bandwidth of the bundled link is the sum of the bandwidths of all links.

For example, there are three links between A and B for link aggregation, and the bandwidth of each link is 10 Gbps. Then the maximum bandwidth of this aggregation group can reach 30 Gpbs.

Improved network connection reliability​

If one link between A and B fails and is interrupted, the traffic will be automatically redistributed among the remaining links without causing traffic interruption between A and B.

Implementing traffic load balancing

Link aggregation can evenly distribute the traffic between A and B to all member links, minimizing the risk of traffic blocking the link generated by each member link.

Avoid Layer 2 loops

When link aggregation is used between A and B, these links are no longer single links working independently, but become a logical link to the outside world. Therefore, even if STP (Spanning Tree Protocol) is not used, loops will not occur, effectively avoiding the risk of Layer 2 loops between A and B. Based on the above advantages, link aggregation has been widely used in IP networks.

MC-LAG: A more reliable “bundle”

With the advent of the mobile Internet era, the Internet has become increasingly closely integrated with our daily lives. While we enjoy the convenience of the Internet, it also brings about the interactive processing of massive amounts of data, which puts higher demands on the network's bandwidth and reliability.

However, traditional link aggregation technology is limited to one-to-one link aggregation between two devices and cannot achieve link aggregation between one-to-many devices.

Therefore, in order to provide a more reliable network, MC-LAG (Multi-Chassis Link Aggregation Group) was created.

When an access device (which can be a server or switch) is connected to two upper-layer network devices A and B, the MC-LAG technology can be used to form a cross-device link aggregation group.

​

The basic idea of ​​MC-LAG is to allow two network devices A and B to perform link aggregation with the access device in the same state. From the access device's point of view, it is as if a link aggregation relationship has been established with the same network device. In this way, link aggregation technology is expanded from one-to-one device docking to being able to access two devices at the same time and form a dual-active system.

Let’s take a look at how this active-active system works.

MC-LAG working process

Before understanding the working process of MC-LAG, first learn some basic concepts involved in MC-LAG technology.

DFS Group, or Dynamic Fabric Service Group, is mainly used to pair the two network devices (A and B in the figure) that make up MC-LAG and synchronize the interface status, table entries and other information of the two devices.

In a DFS Group, the roles of devices A and B are differentiated as master and backup. Under normal circumstances, the master and backup devices forward service traffic at the same time.

Peer-link is a direct Layer 2 link between two MC-LAG devices A and B, used for negotiation message exchange and partial traffic transmission.

Keepalive is the heartbeat link between two MC-LAG devices. It carries heartbeat data packets and is mainly used to send dual-active detection messages between the active and standby devices to prevent dual-active devices A and B from being used as dual-active devices.

MC-LAG member interfaces are interfaces on two network devices A and B that connect to access devices.

After understanding the basic concepts of MC-LAG, we will further understand the process of establishing MC-LAG, which includes the following five steps.

​

• After the configuration of MC-LAG is completed, the two devices at both ends will periodically send Hello messages through Peer-link. The Hello messages carry information such as their respective DFS Group ID, protocol version number, and system MAC.
• After receiving the Hello message from the other end, it determines whether the DFS Group ID of the other end is the same as its own. If they are the same, the pairing is successful.
• After pairing is successful, the master/backup device is elected. The election is based on the MC-LAG priority, and the one with the higher priority is the master; if the MC-LAG priorities are the same, the system MAC addresses of the two devices are compared, and the one with the smaller MAC address is the master.
• The master and backup devices send synchronization messages to synchronize information.
• The master/slave devices send heartbeat detection messages through the Keepalive link, mainly for dual-master detection when the peer-link fails.

After completing the above establishment process, MC-LAG can work normally.

MC-LAG Traffic Forwarding

​MC-LAG is mainly used in dual-homing access scenarios, that is, the access-side device C uses the MC-LAG technology to access the network-side devices A and B. During normal operation, the upstream and downstream traffic is forwarded through devices A and B in a load-balancing manner.

If the above network fails, how does MC-LAG protect the network from working normally?

Member interface link failure

​If a member interface of MC-LAG fails, for example, a member interface of device B fails. Access side device C senses the failure of the member interface of device B and sends all upstream traffic to device A, which forwards it.

When device B receives traffic from the network side to device C on the access side, it will hand over the traffic to the normally functioning device A through the peer-link and forward it to device C on the access side.

​

MC-LAG Device Failure

If an MC-LAG device fails, for example, device B fails, device B cannot forward traffic, and all traffic is forwarded by device A.

​

Peer-link failure

If a peer-link failure occurs, devices A and B cannot forward traffic at the same time, otherwise it will cause a series of problems such as broadcast storms and MAC drift, so only one device is allowed to forward traffic.

At this time, the backup device of MC-LAG (here is device B) will process all its physical interfaces except the peer-link interface and the management network port in Error-down mode. At this time, all traffic will only be forwarded by the MC-LAG master device.

​

Conclusion

From the previous introduction, we can see that MC-LAG technology has more advantages than traditional link aggregation technology. While enhancing network reliability, it simplifies networking and realizes device-level high-availability redundant protection and multi-path forwarding.

In addition, the two network devices of MC-LAG run independently and can be upgraded separately. During the upgrade process, as long as one device is kept working normally, there will be almost no impact on the running business.

Currently, MC-LAG technology is being widely used in new IP metropolitan area networks and cloud data centers. While adopting the Spine-Leaf network architecture, MC-LAG can be deployed to ensure network reliability.

In the process of 5G evolution, MC-LAG technology will surely provide more reliable protection for IP bearer networks. ​