How many Layer 2 networking technologies do you know?

1. Analysis of traditional STP technology application

STP is a standard defined in IEEE 802.1D and applied to Ethernet switches. This standard defines a set of rules for switches to detect link layer topology and control the link layer forwarding behavior of switches. If STP finds a loop in the network, it will select an appropriate position on the loop to block the port on the link - preventing the port from forwarding or receiving Ethernet frames, thereby eliminating broadcast storms that may occur in the Layer 2 network.

[[248531]]

However, in actual deployment, to ensure high availability of the network, both data center networks and campus networks usually use a physical topology with loops and use STP to block the forwarding of some ports. For blocked ports, they can only be added to the forwarding tree of the Layer 2 data frame by STP when the port or link in the forwarding state fails.

This mechanism of STP leads to insufficient utilization of Layer 2 links, especially when network devices have a fully connected topology. As shown in Figure 1, when the full-network STP Layer 2 design is adopted, STP will block most links, reducing the bandwidth from access to aggregation to 1/4 and the bandwidth from aggregation to core to 1/8. This defect causes the port congestion to be more serious for switches closer to the root of the tree, resulting in more serious waste of bandwidth resources.

It can be seen that STP can well support traditional small-scale Layer 2 networks, but in some data centers (or between data centers) where virtualized applications are deployed on a large scale, large-scale Layer 2 networks will appear, and STP has serious deficiencies in such networks. The main problems are as follows (as shown in Figure 2).

(1) Inefficient Path

• Traffic bypasses N-1 hops
• The routing network only needs N/2 hops or even shorter

(2) Low bandwidth utilization

• Block the loop and interrupt the link
• A lot of idle bandwidth
• Traffic is prone to congestion

(3) Low reliability

• Failover in seconds
• High consumption of equipment

(4) Difficulty in maintenance

• Links cause complex topology changes
• Prone to causing broadcast storms
• Configuration and management difficulty increases dramatically as scale increases

Due to the above shortcomings of STP, it is difficult to manage and control large-scale Layer 2 networks.

2. Analysis of IRF technology application

H3C IRF (Intelligent Resilient Framework) is an N:1 network virtualization technology. IRF can virtualize multiple network devices (member devices) into one network device (virtual device) and manage and use these devices as a single device.

IRF virtualization technology not only simplifies multiple physical devices into one logical device, but also turns multiple link connections between network layers into direct connections between two logical devices. Therefore, multiple physical links can be aggregated across devices to become one logical link, increasing bandwidth while avoiding loop problems caused by multiple physical links. As shown in Figure 3, the access, aggregation and core switches are virtualized in pairs, and the layers are interconnected by cross-device link bundling. The physical topology of the entire network remains unchanged, but the logical topology becomes a tree structure. Ethernet frames are forwarded along the topology tree, there is no Layer 2 loop, and the bandwidth utilization rate is maximal.

In short, the benefits of using IRF to build a Layer 2 network include:

• Simplify network topology and management
• Reduce the number of devices and management workload
• Merging multiple devices can effectively improve performance
• Seamless switching can be achieved between multiple devices, effectively improving network HA performance

At present, the maximum number of frame switches that can be stacked using IRF technology is four. That is to say, when using IRF to build a Layer 2 network, a maximum of 4 aggregation switches can be used. For example, a frame switch with 16 service slots (4 upstream and 12 downstream) is deployed at the aggregation layer, and is equipped with the industry's most advanced 48-port line-speed 10G single board. Considering the 1:4 convergence ratio of upstream and downstream, the number of downstream 10G ports of the aggregation switch is 48*12=576. The access switch is deployed with a box switch with 40G upstream and 48G downstream. The aggregation switches behind 4 IRFs can access 13,824 dual-NIC 1G servers without blocking at Layer 2, which can meet the Layer 2 networking needs of most domestic customers.

A small number of customers expect their server resource pool to be effectively expanded to 20,000 or even larger, which requires other technologies to provide greater network capacity.

3. Analysis of TRILL Technology Application

The large Layer 2 network of a data center built using TRILL technology is shown in Figure 4. The network is divided into a core layer (equivalent to the aggregation layer of a traditional data center) and an access layer. The access layer is the boundary between the TRILL network and the traditional Ethernet. The core layer RBridge does not provide host access, but is only responsible for high-speed forwarding of TRILL frames. Each access layer RBridge is connected to multiple core layer RBridges through multiple high-speed ports.

To be precise, TRILL can support up to 16 core layer RBridges. This puts higher requirements on the access layer switches: support 16 ports of 10G upstream and 160G downstream. The current mainstream Gigabit switches are 40G upstream and 48G downstream. The maximum density can support up to 100G upstream and 96G downstream. If the same aggregation (TRILL core) equipment and convergence ratio are used as the previous IRF network, TRILL can currently support up to 10 core networks, and its maximum capacity can access 27,648 dual-NIC Gigabit servers without blocking. It can be seen intuitively that with the increase in the number of aggregation switches, the access scale of the second-layer network servers has risen sharply.

This is the most obvious advantage of TRILL over IRF. Although TRILL has successfully expanded the scale of the virtual machine resource pool, the current large-scale Layer 2 network lacks operation and maintenance experience, which means that the operation and maintenance costs will increase significantly and bring huge risks to the business system. At the same time, TRILL technology currently has objective defects in chip implementation: the core layer cannot support Layer 3 termination, which means that the core layer of TRILL cannot be used as a gateway device. It is necessary to add another layer of equipment on the core layer to serve as a gateway (as shown in Figure 5). This makes the network structure complicated, the management difficulty increases, and the network construction and operation and maintenance costs increase.

4. Analysis of SPB technology application

The networking scheme of SPB is basically the same as TRILL (as shown in Figure 4). Similarly, SPB can expand 16 aggregation switches to increase the access scale of the Layer 2 network; it also places higher requirements on the access density of the access switch; it also has the chip defect that the gateway and the SPB core must be separated (as shown in Figure 5), which leads to an increase in network layers and an increase in management and operation and maintenance costs. Compared with TRILL, the advantage of SPB is that it can easily support VLAN expansion functions, which has attracted the attention of many operators who need to support multi-tenant services and enterprises with large-scale operation needs.

5. Analysis of EVI technology applications

Due to the lack of successful operation and maintenance experience in large-scale Layer 2 networks, the most reasonable virtualized network should be the L3+L2 network model. As mentioned above, since the EVI feature can achieve Layer 2 intercommunication through the IP network between the aggregation layer and the core layer, when expanding multiple Layer 2 domains through EVI, there is no need to change the wiring or equipment, and only the EVI feature needs to be enabled on the aggregation device. This can smoothly expand the scale of the Layer 2 network.

Currently, L3 routing + L2 IRF + EVI is the most suitable model for cloud computing virtualized data center networks. Its main advantages include:

• Mature technology and stable architecture
• Rich operation and maintenance experience, easy to maintain
• Smooth expansion capability, capable of supporting large-scale Layer 2 networks

6. Comparison of Technology Applications

7. Conclusion

In virtualized data centers, server Layer 2 access solutions are usually used to achieve flexible expansion of resource pool capabilities. As enterprises continue to increase their requirements for flexible scheduling of computing resources, they will inevitably face large-scale Layer 2 network problems. This article lists five different implementation technologies, each with its own characteristics. There is no best technology, only the most suitable one. I hope that through the explanation and analysis of this article, I can give readers some help and inspiration so that they can choose the most suitable technical solution when implementing large-scale Layer 2 networks in data centers in the future.

Brief introduction to other major Layer 2 networking technologies:

(1) Fabricpath

Fabricpath was proposed by Cisco and is very similar to TRILL. Compared with TRILL, Fabricpath is more compact in encapsulation, supports multi-topology capabilities, and is more mature in terms of control and management.

(2) Qfabric

It is a technology proposed by Juniper. The control software of the switch is pulled to an external server to run, and the entire network is centrally controlled and managed. The disadvantages are: poor scalability, few deployment cases, and maturity needs to be tested.

(3) VXLAN/NVGRE

Recently, a technology that supports L2oIP on vSwitch has emerged in the industry, including VXlAN (Virtual eXtensible LAN) and NVGRE (Network Virtual GRE). The former is a standard proposed by VMware and Cisco, using L2oUDP encapsulation and supporting 16M tenant IDs; the latter is a standard proposed by HP and Microsoft, using L2oGRE encapsulation and also supporting 16M tenant IDs. The main features of these two technologies are that the starting and ending points of the tunnel are mainly on the vswitch, not on the physical switch. The encapsulation of the tunnel is already set up on the vswitch inside the server, and then the physical network is penetrated as a large IP backplane, and the large Layer 2 range can cross DCs. In order to achieve the purpose of rapid deployment and flexible creation of virtualized networks.