VXLAN and MPLS: From Data Center to Metro Ethernet

In recent years, the evolution of cloud computing, virtualization, and containerization technologies has promoted the application of network virtualization technology. Both MPLS and VXLAN apply the concept of virtualization to express logical network architectures to achieve more complex and flexible domain management, but they meet different needs. This article will compare the differences between VXLAN and MPLS, and explain why VXLAN is more popular than MPLS in metropolitan area networks and wide area networks.

VXLAN or MPLS?

First, let’s quickly review what VXLAN and MPLS are.

Multi-protocol Label Switching (MPLS) is a technology that uses labels to guide high-speed and efficient data transmission on an open communication network. Multi-protocol means that MPLS can not only support multiple protocols at the network layer, but also be compatible with multiple data link layer technologies at the second layer. This technology is specifically used to simplify data transmission between two nodes, replacing long network addresses with short path labels.

MPLS allows adding more sites without extensive configuration. MPLS is also IP agnostic, it just simplifies the implementation of those IPs. MPLS over VPN adds an extra layer of security, as MPLS has no built-in security features.

Virtual Extensible Local Area Network (VXLAN) encapsulates Layer 2 Ethernet frames in Layer 3 UDP packets, so devices and applications can communicate over a large physical network as if they were on the same Ethernet Layer 2 network. VXLAN technology uses the existing Layer 3 network as the Underlay network and builds a virtual Layer 2 network, the Overlay network, on top of it.

VXLAN is a relatively new technology compared to MPLS. As a network virtualization technology, VXLAN solves the scalability issues associated with large-scale cloud computing setups and deployments.

Why is VXLAN more popular than MPLS in data center networks? Three points can be summarized:

• Routers that support MPLS tend to cost more than data center Layer 3 switches that support VXLAN.
• MPLS-based VPN solutions require tight coupling between edge devices and core devices, so every node in the data center network must support MPLS.
• There are relatively few data center network engineers who are proficient in MPLS expertise.

(1) MPLS router cost

Some service providers have long been interested in the idea of ​​building low-cost metropolitan area networks using data center-class switches. More than 20 years ago, the first generation of competitive metropolitan Ethernet service providers, such as Yipes and Telseon, built their networks using Gigabit Ethernet switches, which were the most advanced in enterprise networks at the time. However, such networks had difficulty providing the scalability and resiliency required by large SPs, as shown in Figure 1.

Figure 1: Traditional Layer 2 Network

Therefore, most large SPs turned to MPLS (as shown in the figure below). However, MPLS routers are more expensive than ordinary Ethernet switches, and this cost difference problem has not been effectively solved in the following decades.

Figure 2: IP/MPLS Network

Today's data center-level switches combined with VXLAN Overlay architecture can largely eliminate the shortcomings of pure L2 networks without the high cost of MPLS routing, which has attracted the attention of a new round of SPs.

(2) Tight coupling between core and edge

MPLS-based VPN solutions require tight coupling between edge devices and core devices, so every node in the data center network must support MPLS. In contrast, VXLAN only requires VTEP in edge nodes (such as leaf switches), and can use any IP-enabled device or IP transport network to implement data center spine and data center interconnect (DCI).

(3) MPLS expertise

Outside of large service providers, MPLS technology is difficult to learn, and relatively few network engineers can easily build and operate MPLS-based networks. VXLAN is relatively simple and is becoming a basic technology widely mastered by data center network engineers.

Advances in data center switching technology make VXLAN-based metropolitan area networks and wide area networks possible

Today’s data center switch silicon, such as Broadcom’s Trident 3 and Trident 4, integrates many features that make VXLAN-based metropolitan area networks possible. Here are two key examples:

• Hardware-based VTEP supports line-speed VXLAN encapsulation;
• The extended tables provide the routing and forwarding scale required to create resilient, scalable Layer 3 underlay networks and multi-tenant overlay services.

Additionally, newer data center-class switches have powerful CPUs that can support the advanced control planes that are critical to scaling Ethernet services, whether it is BGP EVPN (a protocol-based approach) or a protocol-free control plane based on SDN.

As a result, in many metropolitan area network applications, specialized (i.e., high-cost) routing hardware is no longer required.

VXLAN Overlay Architecture for Metropolitan Area Networks and Wide Area Networks

Overlay networks have been widely used in various applications, such as data center networks and enterprise SD-WAN. A key commonality of these overlay networks is that they are loosely coupled with the underlay network. In principle, the underlay network can be built with any network technology and use any control plane as long as the network provides sufficient capacity and resilience. Overlay is only defined at the service endpoint, and there is no service provisioning in the underlay network nodes.

One of the main advantages of SD-WAN is that it can use a variety of networks, including broadband or wireless Internet services, which are widely available and cost-effective and provide sufficient performance for many users and applications. Similar benefits are also brought when VXLAN Overlay is applied to metropolitan area networks and wide area networks, as shown in Figure 3.

Figure 3: VXLAN Overlay Architecture

When building a metropolitan area network to provide services such as Ethernet private line (E-Line), multi-point Ethernet local area network (E-LAN), or Layer 3 VPN (L3VPN), care must be taken to ensure that the underlay can meet the SLA of such services.

VXLAN-based Metropolitan Area Network Overlay Control Plane Options

So far, we have focused on the advantages of VXLAN over MPLS in terms of network architecture and capital cost, that is, the advantages of the data plane. However, VXLAN has no specified control plane, so we also need to look at the Overlay control plane option.

The most famous control plane option for creating VXLAN overlays and providing overlay services should be BGP EVPN, which is a protocol-based approach that must configure services in each edge node. The biggest disadvantage of BGP EVPN is the complexity of operation.

Another protocol-free approach is to use SDN and services defined in the SDN controller to program the data plane of each edge node. This eliminates most of the operational complexity of the protocol-based BGP EVPN. However, the centralized SDN controller architecture (acceptable for data center architecture within a single site) poses serious scalability and resiliency challenges when applied to metro and wide area networks. Therefore, it is not clear whether it is a better choice than MPLS for metro networks.

There is a third option - decentralized or distributed SDN, where the SDN controller functionality is fully replicated and distributed throughout the network. This can also be called "controllerless" SDN, as it does not require a separate controller server/appliance, and it completely eliminates the scalability and resiliency issues of centralized SDN control while retaining the benefits of simplified and accelerated service provisioning.

Table 1: Comparing MPLS and VXLAN options for metro networks

Deployment options

Because VXLAN allows overlay service delivery to be decoupled from the underlay network, it creates deployment options that MPLS cannot match, such as virtual service overlay on the existing IP bottom layer, as shown in Figure 4. VXLAN-based switches are deployed at the edge of the existing network and expanded according to business needs, which can add new Ethernet and VPN services without changing the existing network and increase new revenue.

Figure 4: VXLAN overlay deployment on an existing metropolitan area network

The metro network infrastructure shown in Figure 5 can support all the services that can be provided by MPLS-based networks, including business Internet, Ethernet and VPN services and consumer triple play services, while completely eliminating the cost and complexity of MPLS.

Figure 5: Converged Metro Core with VXLAN Service Overlay