Veteran Network Engineer: Talking about Several Typical Deployments and Practices of SD-WAN

A few days ago, I talked with my colleagues about the technical experts in the network industry. We accidentally talked about Tony Li, the earliest proposer of the BGP4 draft (a Cisco senior). The BGP4 protocol proposed more than 20 years ago is still in use and inherited in today's Internet, VXLAN EVPN in cloud data centers, and backbone networks based on MPLS, SegemntRouting, etc. Another amazing person is Professor Nick McKeown, one of the earliest proposers of SDN architecture and OpenFlow. Now he has turned to the dominant programmable language P4 and founded Barefoot Networks. Professor Nick has gained both fame and fortune in technological innovation and commercialization. Today, whether it is the latest Internet technology, the hottest cloud-network integration and SDN network, they still inherit the thoughts and ideas of these two legendary figures.

[[226728]]

With emotion, a series of SD-WAN deployment cases were released in the market at the beginning of 2018, which brought SD-WAN to a new climax. Speaking of SD-WAN, there are many good articles on the Internet recently, especially many articles about SD-WAN technical principles, development history, special functions, etc., but there are not many articles from the perspective of specific practical deployment of SD-WAN. As an old network engineer, I have experienced several major transformations of wide area network technology and recently participated in several large-scale WAN reconstruction and construction projects based on SDN. On the occasion of the tenth anniversary of the development of SDN technology, I would like to share some experiences from the perspective of practical deployment of SD-WAN, and encourage everyone.

This article will share SD-WAN deployment cases in three different scenarios, including: SD-WAN access-based on Internet Edge solution, SD-WAN backbone network-based on SRTE traffic scheduling Core solution, and SDN collaborative controller or business collaborative orchestrator based on multi-vendor WAN. These three scenarios are quite representative and are also the typical SD-WAN requirements currently encountered. The following figure uses the architecture diagram of Dadi Cloud Network as a reference for your understanding!

The first deployment scenario:

SD-WAN access service is also the most typical and popular scenario, sometimes also called SD-WAN Edge solution. Operators can use this technology as a complement to MPLS or a replacement for the next-generation MPLS. Enterprises can use this technology to achieve on-demand networking of branch offices.

Representative customer cases: In March 2018, Nanling Technology announced that it had become "the first domestic WAN solution service provider capable of providing customers with a nationwide range of MPLS VPN, IPSEC VPN, SD-WAN and other applications."

Market demand: With the increasing demand for enterprises to go to the cloud and access the WAN on demand, traditional private line services such as MSTP and MPLS have been unable to meet the needs of the cloud and Internet era due to problems such as cost and long deployment cycle. SD-WAN technology based on Internet and POP path selection has been born. This is an SDN-WAN deployment scenario for flexible branch access. Since domestic operators' MPLS VPN networks have been deployed on a large scale, operators will not replace MPLS or other private line services with SD-WAN in the short term, but will use SD-WAN technology to enrich MPLSVPN services or as the last mile access technology.

Main technical implementation: In fact, SD-WAN has no essential innovation in technology, but SD-WAN has made new breakthroughs in concept, integrating SDN control ideas and POP line SLA detection technology, and can realize cloud-network integrated collaborative deployment. The technical implementation is shown in the figure: SD-WAN general technology and general functions are not described here. There was an article written in great detail two days ago. Here are a few considerations from the perspective of actual deployment:

• When designing and deploying SD-WAN, the Internet bottleneck problem of north-south operators in China needs to be considered. When designing and deploying, customers will choose to deploy multi-line POP nodes in multiple computer rooms, and deploy vPE devices at each POP node. The vPE of each POP node forms a backbone network through a dedicated line to ensure the SLA of the traffic aggregated by SD-WAN. In this case, the vPE of the POP node is formed by the customer's MPLS PE node (in other projects, we use SRTE as the backbone network traffic scheduling). In a large SD-WAN deployment, each CPE will detect and select the best POP node based on the list sent by the controller, and connect to the vPE of the best operator line. When designing SD-WAN, both Edge and Core need to consider the SLA guarantee of the line, and then use the SDN controller to deploy unified routing, security and QOS policy deployment and control for the entire network, thereby solving the Internet access service quality problem and the problem of unified policy deployment for the entire network;
• Another problem to be solved is how to connect SD-WAN tenants with MPLS tenants in the existing network (the tenant branch may be an SD-WAN line, or a hybrid MPLS dedicated line network). As mentioned earlier, the major operators basically have their own MPLS networks. The connection between SD-WAN and MPLS VPN is an issue that must be considered. We have multiple solutions to achieve automatic connection between the vPE at the PoP point and the MPLS PE node, including managed PE, Option B, Overlay, etc.
• The automatic deployment of CPE and traditional routing equipment, including backup and load sharing between lines, is also a key factor in the success of SD-WAN. Since most manufacturers' SD-WAN controllers are tightly coupled with CPE boxes, they are locked by manufacturers in the later stage. Therefore, there are several suggestions for consideration: For example, does the CPE box support OpenWRT? How is the automatic deployment mechanism of the small box ZT-PnP (firewall traversal capability)? How does CPE automatically select the access POP point with the smallest delay or the best bandwidth? Can it be plug-and-play, automatically connect, and automatically switch?
• The core of the entire system is the SDN controller. The reliability, clustering, tenant self-service management and ease of use of the controller cannot be ignored. As an operator, the northbound integration of the SD-WAN controller with the existing BSS/OSS is also considered. Although SD-WAN does not have major innovative technologies, a complete and mature SD-WAN solution is not simple in terms of technology. The deployment and operation of SD-WAN is simple and easy to use, and enterprises no longer need professional CCIE personnel to design and operate. For the expansion and deployment of branches, traditional MPLS deployment may take several months, but now it can be done in minutes in theory.

The second deployment scenario:

SD-WAN core backbone network scheduling (including DCI) for service providers and large enterprises. Typical scenarios: SD-WAN backbone scheduling (including DCI) for large operators and OTT customers, SD-WAN core backbone network for large enterprises. Representative customer cases: Google B4's commercial deployment SD-WAN classic case project (released in 2012) and the SDN deployment based on MPLS backbone network released by Industrial and Commercial Bank of China in February 2018.

Main market demand: The core idea of ​​SD-WAN backbone scheduling is traffic scheduling and multi-tenant-based services and management. Sometimes we also call it the SD-WAN DCI/Core solution. This solution is completely different from the Internet-based SD-WAN Edge mentioned above in function and positioning, but the two solutions are complementary.

There are currently three main technical implementation methods:

• The first category is based on white-label machines + Openflow SDN controller - Google B4 is based on this solution (2010-2012). The core of Google B4 is its TE scheduling and algorithm, and it cleverly avoids many defects of Openflow, including forwarding strategies based on source and destination addresses with DSCP as flow tables, but the key technical details of the project (such as SDN controller platform algorithms) are neither announced nor sold to the public. Although latecomers have imitated it, they rarely surpass it. In addition to Google, I learned that some customers still have some problems with this combination solution, including the white-label machine's support for SRTE, BFD/Tunnel support performance and quantity, routing strategy and VPN capabilities, switch flow table and port cache size, etc. After all, it is a switch solution and it cannot be forced, not to mention that even Nick McKeown has now turned to the dominant programmable language P4 and founded Barefoot Networks; Of course, there is a new idea in the industry based on the overlay solution of white-label machines, using vPE with white-label machines to solve the above-mentioned problems. VPE and switches complement each other to realize functions such as traffic scheduling and routing strategies. Due to space constraints, I will not discuss it here;
• The second category is based on MPLS + SDN controller to achieve network-wide traffic scheduling and VPN tenant management - similar to ICBC SD-WAN backbone network (released in 2017). MPLS TE has been deployed many years ago. Judging from the current deployment of operator customers, most complain that the deployment of MPLS TE is too complicated, so there are not many TE tunnels in the actual production network. Since MPLS is mature but not advanced enough, I don’t want to go into too much detail here;
• The third type of solution is also what I would like to see more. It is based on SR (Segment Routing) to implement traffic scheduling and management SR+SDN controller. It is similar to the MPLS network, but SR can build a complete LSP Path based on the source and is compatible with the existing MPLS network. Because SR is also based on label switching, it only needs to simply expand the existing IGP protocol to implement TE, FRR, MPLS VPN and other functions, including automatic delivery, automatic calculation, automatic adjustment, automatic diversion and automatic scheduling of traffic engineering TE.

[[226729]]

The SDN controller based on SRTE is currently a very leading technology in the industry. The basic forwarding table of SR is even simpler than LDP, and it perfectly combines source routing technology with SDN concepts. In terms of traffic TE management, SRTE has much fewer states than RSVP-TE, and does not require LDP/RSVP signaling as complex. However, there is still a certain gap in the specific implementation of SRTE among various hardware manufacturers (including third-party controllers + SR routers). However, there are a few points that need to be considered during deployment:

• How to dynamically adjust TE paths in real time based on link quality (load/loss/delay) to achieve global load balancing
• Tunnel fast switching strategy and escape algorithm (such as Cisco PBTS technology),
• Configuration rollback consistency, offline traffic planning,
• Symmetrical fault detection of TE paths, etc.

The functions of various solutions vary greatly, and there are only a handful of domestic SDN controllers that can truly fully implement SRTE. As a domestic company focusing on the overall SDN architecture and software technology platform, Dadi Cloud Network has successfully overcome this arduous technical challenge after more than two years of technical research and development testing. It has realized the first complete SR-TE commercial controller platform and started commercial deployment.

The third deployment scenario:

Based on multi-vendor SDN collaborative controllers or business collaborative orchestrators, large operators, OTT customers and super-large enterprises have begun to consider SD-WAN multi-vendor heterogeneous environments.

Representative customer cases: In March 2018, China Unicom announced that "the first large-scale operator cloud-network integrated commercial SDN in China was successfully launched (based on Unicom A network's SD-WAN DCI system)" Main market demand: When deploying the first two types of SD-WAN, customers often need multiple vendors' equipment to achieve a balance. Customers do not want to be locked in by vendors, but most SD-WAN networks are still closed management systems. It is a difficult problem to use SDN collaborative controllers or business collaborative orchestrators based on multiple vendors. Interoperability and unified resource management issues require upper-level SDN collaborative controllers to solve. This type of solution is particularly important for large-scale SDN network operations. At present, several major operators and the OTT industry are aware of or have begun to consider this issue. I believe that enterprise customers will have similar needs in the future as SDN is deployed.

Main technical implementation: Take a certain operator as an example. Two years ago, they started to pre-research the coordination of controllers of multiple vendors in the MPLS backbone network under the cloud environment. After nearly two years of hard work in design, development, testing and joint debugging, the customer became the first operator in China to realize cloud-network integration services on the national backbone wide area network. At the same time, it also pioneered the selection of independent core SDN software companies to work together with multiple large network equipment manufacturers in large-scale domestic SDN projects to ensure that operators can fully dominate and control SDN operation requirements and technical architecture direction and decision-making voice, providing a successful case for the final cloud-network integration product to be commercially launched on time.

However, the collaborative controller of multiple vendors needs to be customized and developed according to the actual business situation of customers, which requires SDN software vendors to have very strong R&D capabilities and industry experience, including the northbound interface specifications of vendor hardware, in-depth understanding of cloud network technology (such as VXLAN EVPN, L2/L3 MPLS, SR TE, Neutron, Docker CNI, ODL, etc.), docking with mainstream public cloud systems, and integration with customer OSS/BSS business systems. In practice, it is very complicated and not something that ordinary SDN vendors and companies can afford. As shown in the figure, with the increasing number of SD-WAN multi-scenario and multi-vendor deployments, the collaborative management and unified orchestration of multiple vendors will become an important topic of future SD-WAN.

Finally, let’s look at the future of SD-WAN

SD-WAN is a new idea and architectural innovation under the background of rapid development of the Internet and integration of cloud and network. The breakthrough of SD-WAN in concept is far greater than the innovation in technology. Due to time and space constraints, the three types of SD-WAN related cases and deployment scenarios shared with you this time seem to be three independent SD-WAN solutions on the surface, but in terms of architecture, it is a complete three-dimensional architecture.

Today, the SD-WAN journey has begun. As a new business model, the value and significance of SD-WAN are immeasurable. We believe that future SDN will be able to better understand applications and serve them (Intent-Based SDN and intelligent traffic analysis), provide precise intelligent scheduling capabilities (especially in SRTE, POP detection and selection algorithms), and provide more powerful intelligent operation and maintenance tools to protect Underlay and Overlay. In addition, the openness, universal standards and interoperability of future SDN (not locked by manufacturers) are also the goals we have been pursuing. Let us stay true to our original aspiration and wait and see.