There is a network engineer who doesn’t understand: What is Overlay network?

An overlay network is one or more virtual logical networks built on the same underlay network through network virtualization technology. Although different overlay networks share the equipment and lines in the underlay network, the services in the overlay network are decoupled from the physical networking and interconnection technologies in the underlay network.

The multi-instance of the overlay network can serve different businesses of the same tenant (such as multiple departments) as well as different tenants. It is the core networking technology used in solutions such as SD-WAN and data centers.

1. Why do we need an Overlay network?

Overlay network and Underlay network are relative concepts. Overlay network is a logical network built on Underlay network. Why we need to build Overlay network starts with the concept and limitation of the underlying Underlay network.

Underlay Network

The Underlay network, as its name suggests, is the underlying physical foundation of the Overlay network.

As shown in the figure below, the Underlay network can be a physical network formed by interconnecting multiple types of devices and is responsible for transmitting data packets between networks.

In an underlay network, interconnected devices can be various types of switches, routers, load balancing devices, firewalls, etc., but the various devices in the network must use routing protocols to ensure IP connectivity between them.

Underlay networks can be either Layer 2 or Layer 3 networks. Layer 2 networks are usually used in Ethernet networks and are divided by VLANs. A typical application of Layer 3 networks is the Internet, which uses OSPF, IS-IS and other protocols for routing control in the same autonomous domain, and BGP and other protocols for routing transmission and interconnection between autonomous domains. With the advancement of technology, underlay networks built using MPLS, a WAN technology between Layer 2 and Layer 3, have also emerged.

However, traditional network devices forward data packets based on hardware, and the Underlay network constructed by them also has the following problems:

• Since the hardware forwards data packets based on the destination IP address, the transmission path dependence is very serious.
• Adding or changing services requires modifying the existing underlying network connections, and reconfiguration is very time-consuming.
• The Internet cannot guarantee the security requirements for private communications.
• Network slicing and network segmentation are complex to implement and cannot achieve on-demand allocation of network resources.
• Multipath forwarding is cumbersome and it is impossible to integrate multiple underlying networks to achieve load balancing.

Overlay Network

In order to get rid of the various limitations of the Underlay network, network virtualization technology is now often used to create a virtual Overlay network on top of the Underlay network.

In an overlay network, devices can be interconnected through logical links as needed to form an overlay topology.

A tunnel is established between the interconnected Overlay devices. When a data packet is ready to be transmitted, the device adds a new IP header and tunnel header to the data packet, and the inner IP header is shielded. The data packet is forwarded according to the new IP header. When the data packet is passed to another device, the outer IP header and tunnel header will be discarded, and the original data packet will be obtained. In this process, the Overlay network does not perceive the Underlay network.

Overlay networks have various network protocols and standards, including VXLAN, NVGRE, SST, GRE, NVO3, EVPN, etc.

With the introduction of SDN technology, the addition of the controller's Overlay network has the following advantages:

• Traffic transmission does not rely on specific lines. The Overlay network uses tunneling technology, which can flexibly select different underlying links and use a variety of methods to ensure stable traffic transmission.
• Overlay networks can establish different virtual topology networks as needed without making changes to the underlying network.
• The problem of protecting private traffic on the Internet can be solved by encryption.
• Support network slicing and segmentation. Separating different services can achieve optimal allocation of network resources.
• Supports multi-path forwarding. In an Overlay network, traffic can be transmitted from the source to the destination through multiple paths, thereby achieving load sharing and maximizing the use of line bandwidth.

2. What are some examples of overlay networks?

Overlay networks are widely used in SD-WAN and data center solutions. Due to the different architectures of the underlying Underlay networks, the topology of Overlay networks exists in different forms.

Overlay network in data center

With the evolution of data center architecture, most data centers now use the Spine-Leaf architecture to build underlay networks and use VXLAN technology to build interconnected overlay networks. Business packets run on the VXLAN overlay network and are decoupled from the physical bearer network.

The Leaf and Spine are fully connected, and equal-cost multi-paths improve network availability.

As a network function access node, a leaf node provides various network devices in the underlay network with VXLAN network access functions. It also serves as an edge device of the overlay network and assumes the role of VTEP (VXLAN Tunnel EndPoint).

Spine nodes are backbone nodes and are the core nodes of the data center network. They provide high-speed IP forwarding functions and connect various functional Leaf nodes through high-speed interfaces.

Overlay Network in SD-WAN

The Underlay network of SD-WAN is based on the wide area network, and achieves interconnection between headquarters sites, branch sites, and cloud sites through hybrid links. By building the logical topology of the Overlay network, the interconnection requirements in different scenarios can be met.

Figure 1-5 SD-WAN Overlay network (using Hub-Spoke as an example)

The SD-WAN network is mainly composed of CPE devices, which are divided into two types: Edge and GW.

• Edge: It is the egress device of the SD-WAN site.
• GW: A gateway device that connects SD-WAN sites and other networks (such as traditional VPNs).

Depending on the enterprise network scale, the number of central sites, and the need for inter-site access, multiple different types of overlay networks can be built:

• Hub-spoke: Applicable to enterprises with 1~2 data centers, where services are mainly located in the headquarters and data centers, and branches centrally access services deployed in the headquarters or data centers through WAN. There is no or little need for mutual access between branches, and branches bypass the headquarters or data centers.
• Full-mesh: Suitable for small enterprises with a small number of sites, or for deployment in large enterprises where branches need to collaborate. Collaborative services of large enterprises, such as high-value applications such as VoIP and video conferencing, have high requirements for network performance such as packet loss, latency, and jitter, so such services are more suitable for direct visits between branch sites.
• Hierarchical networking: Suitable for large multinational companies and enterprises with large network sites or sites scattered in multiple countries or regions. It has a clear network structure and good network scalability.
• Multi-Hub Networking: Applicable to enterprises with multiple data centers, each of which deploys business servers to provide business services to branches.
• POP networking: When operators/MSPs provide SD-WAN network access services to enterprises, enterprises cannot transform all sites into SD-WAN sites at once. There are both traditional branch sites and SD-WAN sites in the network, and there is a demand for traffic interconnection between these sites. An IWG (Interworking Gateway) networking can provide site connectivity services for SD-WAN sites and existing traditional MPLS VPN networks for multiple enterprise tenants at the same time.

3. Overlay network VS Underlay network

The differences between Overlay network and Underlay network are as follows:

Table Underlay Network VS Overlay Network