Veteran Network Engineer: A brief discussion on the thinking and practice of SDN design under cloud-network collaboration

The Internet has been quietly changing over the years

I remember when I first entered the IT industry, the network was just emerging. The network was a mysterious and high-end technology that made countless IT people envious. Now with the advent of the hybrid cloud and multi-cloud interconnection era, the network has once again become the focus of the industry. However, today, the network is not just the underlay network of routers and switches in the traditional sense, but more of an overlay SDN network that provides services on demand based on tenants.

In today's cloud environment, computing resources provide service capabilities on demand and are available in seconds. However, there are still certain challenges compared to on-demand services and automated deployment of networks, especially in heterogeneous network environments and hybrid overlay environments. How to achieve cloud-network collaboration in multi-domain environments and multi-cloud interconnection, and how to design and deploy SDN networks have become a hot topic of concern for Chinese operators, line operators, cloud operators and large enterprises. (Some people also call it "cloud-network integration or cloud-network fusion", and the author prefers to use the term "cloud-network collaboration").

As an old network engineer who is still working on the front line, I have recently participated in the design and construction of large-scale cloud network SDN collaborative management platform projects for some operators and cloud vendors. Today I would like to share with you about this topic.

There have been many discussions about SDN, especially the emergence of SD-WAN, which has set off a huge wave among cloud vendors, operators and line vendors, and everyone has begun to capture many new business opportunities. Due to business needs, cloud vendors first adopted SDN technology in cloud centers and between DCI backbone nodes to meet the needs of cloud network collaboration and traffic scheduling. In recent years, with the emergence of SD-WAN, rapid cloud migration, extension of cloud services to customer branches, and networking for customers have brought new opportunities for cloud vendors and service providers.

Against this background, traditional large-scale operators are facing increasing business pressure. Fortunately, operators have unparalleled national operating line resources and customer resources. In recent years, they have begun to drastically adjust their cloud network strategies and increase investment in cloud network collaborative technologies and reconstruct existing network resources to provide customers with one-stop service capabilities for cloud access and cloud connection. Although operators started a little later, they have many online and offline resources, strong strength, and a high starting point, providing strong end-to-end cloud network collaboration capabilities from the last mile to the cloud.

The imbalance between the two major technology engines of cloud and network

The essence of cloud computing is network computing. Cloud computing uses the horizontal capabilities of the network platform to connect the logical computing resource pool and realize the service capabilities of on-demand resource scheduling and load balancing. The superposition and synergy of computing and network as IT dual engines have achieved the rapid development of information and communication today. However, objectively speaking, the two major technical engines are unbalanced. Today, the cloudification of computing resources and on-demand scheduling and deployment have forced the traditional network model to change.

In order to support the automatic activation and second-level deployment of cloud computing, network engineers began to think about the evolution of data center cloud networks 10 years ago. From the early FabricPath/Trill/OTV large Layer 2 to today's popular lightweight Overlay technology based on VXLAN, it can be said that the development of data center cloud networks is relatively fast. Especially with the emergence of SDN technology, on-demand service capabilities and automated management of cloud center internal networks have been basically realized. However, heterogeneous network environments, hybrid Overlay environments, and multi-domain network environments still have great challenges. Objectively speaking, the large number of network protocols and complex technologies lead to the difficulty of end-to-end network automated deployment and management:

• On the one hand, the network will involve the coordination of multiple domains: in addition to the data center network, the backbone network and the edge access network must also be considered. The coordination of multiple domains and network service capabilities are a focus of SDN today;
• On the other hand, network-related protocols or technologies are very complex. There are countless L2/L3/GRE/VXLAN/EVPN/MPLS/SR/OF/ Netconf/Yang/Neutron/ODL/ONOS protocols or technologies. In addition to QOS/encryption/FW/NAT/traffic scheduling/TE tunnel technology, it is not easy to be a good network engineer these days, and it is even more difficult to design a large-scale cloud network collaborative SDN network.

As shown in the following figure:

Cloud-network architecture's demands for SDN

The SDN network architecture design for cloud-network collaboration needs to meet the needs of intra-cloud networks, inter-cloud interconnections, and cloud networks. It needs to manage complex multi-domain and heterogeneous network resource systems to achieve collaborative work and one-stop management services for cloud-network services. Since the cloud-network collaboration architecture involves a lot of content and the needs of various industries are different, the current planning and deployment of large domestic operators, cloud vendors, and OTT companies in this field are relatively advanced. Taking operators, cloud vendors, and line vendors as examples, we briefly analyze the main demands of cloud-network collaboration for SDN design:

(1) Unified management requirements for cloud resource pools

How to manage multiple virtualized resource pools (Openstack, VMware, K8S, etc.) and physical resource pools in a unified and collaborative manner, how to manage heterogeneous network devices in a unified manner, how to choose or mix HostOverlay and HW Overlay networks? And the resource management and supervision of the cloud and network, etc.

(2) Coordinated scheduling of multi-domain network systems

This is a problem that large service providers may encounter. Cloud networks involve multiple network domains, including: the backbone network has cloud interconnection, the edge access network SD-WAN, the network system within the cloud center, and the multi-cloud Internet network system (internal cloud resource pool network, partner cloud system). Multiple domains use completely different network technologies. How to use SDN cloud network collaborative technology to achieve cloud resource layout suitable for cross-domain deployment, so that users can have one-point access, multi-point deployment, and full-network services.

(3) Network Collaboration Service Capabilities and Standards

Large service providers hope to formulate their own API specifications and Yang models to reconstruct the cloud-network collaborative architecture, standardize the management of multi-domain network resources and the interface specifications and standards for cloud pool docking. This idea is very good, but it puts higher requirements on network vendors. However, in reality, the network functions of each domain are different and the interface specifications of vendors are different. How to create a unified cloud-network collaborative network service capability for unified calling of business platforms is also an interesting topic.

Design of SDN under cloud-network collaborative architecture

The design of SDN under the cloud-network collaborative architecture involves many aspects, including the automatic connection of end-to-end tenant VPNs between multiple domains from access to the cloud, the management and collaborative deployment of network resources in different domains, and end-to-end line SLA detection and traffic scheduling, as well as decentralized and domain-based management capabilities. At the 2019 Second China SDNLAB Summit, the author shared "How can cloud vendors, line vendors and operators become winners in the SD-WAN era?", in which the topic of SDN design was discussed. First of all, it is a network technology attribute: in traditional networks, traditional vendors such as Cisco have done a very good job in network basic technologies such as routing strategies, service quality, and security; however, as a technological innovation and management model innovation, SDN/SD-WAN design must be built on a robust network basic functional capability, and more SDN attributes and abstract service functions are required. The author believes that the design of a good SDN under the cloud-network collaborative architecture must be well designed in three aspects. Today, I will focus on these three design points:

(1) Design of SDN southbound orchestrator adaptation layer

The SDN orchestrator module is responsible for the unified collection, abstraction and organization of the entire network resources, and realizes the modeling of network resources and capabilities. It is the basic component of the SDN network orchestration platform capabilities and service support. The SDN orchestrator southbound connection: Openstack platform, backbone network controller, SD-WAN controller, data center network equipment controller and third-party cloud platform, etc.

Regarding the SDN collaborative adaptation layer, our suggestions are as follows: The SDN coordinator realizes the transformation of the physical network into the logical network, provides a unified network service capability abstraction for physical network resources, integrates heterogeneous vendor equipment and realizes decoupling, which is the foundation of the SDN network design. Multi-domain involves multiple network devices and technologies, controllers from multiple vendors, YANG models, command contents, and configuration methods defined by different vendors. SDN designers must master the system integration design of the latest technologies in multiple fields and have the ability and experience of innovative practice.

(2) SDN network service orchestrator module

The SDN network service orchestrator module relies on the unified network abstraction, logical network, and network resource model provided by the SDN coordinator to provide the basic mechanisms and management of all network services required by the business platform. The basic mechanisms of network services include tenant-based VPC network routing, tenant-based VRF virtual networks and routing, Layer 2 VPN and Layer 3 VPN in cloud data centers, BGP/IGP routing, SD-WAN access and SD-Core backbone integration, traffic engineering and SLA service policies and security, etc. The SDN service orchestrator abstracts network resources and capabilities into a unified business model and provides it to the cloud business platform. The business layer does not need to be aware of any underlying vendor equipment details and is completely decoupled. We have made two suggestions for the unified service orchestration module:

• The entire system architecture of the business orchestrator is split according to business and function. It is recommended to implement the smallest granular service module based on the existing mature microservice architecture. Microservices can be combined on demand to complete the opening of various business scenarios and realize a fully decoupled modular system architecture design.
• The design of the unified business orchestrator needs to be completed by professional architects who are familiar with the underlying network architecture, technologies and protocols, and have a full-stack understanding of new cloud network technologies and software architectures. At the same time, they also need to have an in-depth understanding of the relationship between the customer's business and the network.

(3) SDN network service northbound API specification

The cloud-network collaborative SDN architecture provides end-to-end network access services for data centers, private clouds, public clouds, and branch offices. API specifications and Yang models must not only meet the management and connection of network resources, but also need to be connected and connected with multiple cloud platforms, adapt to multi-domain network systems, multi-cloud platforms and other programmable platforms, and be uniformly called by northbound business systems, providing users with a one-stop cloud-network connection. Since the API business processes and calling processes of cloud vendors and operators are very different, the cross-domain deployed network architecture and multi-cloud resource layout are very different. The technical difficulty of API specifications for SDN network orchestration is also very complex. Regarding API specifications, our suggestions are as follows:

• Cloud vendor integration must be carried out as early as possible: Most cloud vendors have their own non-standard API specifications, and the access method, routing design and API of each cloud vendor are still very different. Experienced developers must participate in the discussion and design as early as possible.
• The API call process must be coordinated well: for example, in resource application, whether to log in at one point or allow cloud vendors and service providers to log in at multiple points in a multi-domain environment must be coordinated well. Typical customer needs: users can access at one point, deploy at multiple points, and provide services across the entire network, which requires multi-service capabilities involving cross-domain deployment and multi-site resource layout and unified management.

Practical sharing of SDN under cloud-network collaborative architecture

As analyzed above, the cloud-network collaborative architecture of large service providers covers at least several levels, including access, backbone network and cloud center. This involves not only the unified planning of the overall architecture but also the integration of multi-domain technologies.

The following is a sharing of a customer's SDN practice deployment: the customer is a large service provider, and hopes to coordinate and unify cloud and network management and work together. Based on SDN technology, the network infrastructure is reconstructed, public cloud and private cloud resources are connected, and end-to-end network service automatic deployment and scheduling (SDN business orchestration) are provided. The customer provides enterprise customers with cloud services, cross-cloud connections (including data centers, public clouds, private clouds) and branch networking, and other scenarios, including solving the problem of the "last mile" of cloud computing.

This client is a large operator project, and the architecture design pays special attention to the following points:

• Unified business orchestration from access to backbone to cloud, one-stop service
• SD-WAN Multi-line Data Center POP Node + SLA Detection
• SD-WAN POP and MPLS PE nodes seamlessly connect
• SRTE traffic scheduling based on tenant SLA services
• DCI L2/L3VPN services are automatically enabled, and the internal VR in the cloud DC is automatically connected to the GW.
• Networking scenarios: enterprise front-line cloud, enterprise cloud networking, enterprise cloud broadband, Internet + dedicated line hybrid networking, etc.

Paying special attention to the above points in the architecture design, we adopt the Dadi Cloud Network SDN solution (also a small advertisement) as shown in the following figure:

The following is a detailed introduction to the customer deployment:

(1) SDN orchestration and service platform

In the project, we provide customers with multi-domain service orchestration capabilities, business orchestration for connecting to public cloud and hybrid cloud interconnection, L2/L3VPN network business orchestration, and SD-WAN access cloud and backbone network integration business orchestration based on customer needs and service processes. Through our orchestration and service platform, we provide end-to-end unified scheduling resources, SLA service quality assurance, path planning and VPN tenant security management, and provide unified northbound API specifications and YANG models for multi-domain services from edge to backbone and connecting to various cloud vendors, so as to achieve the connection and decoupling of controllers from different manufacturers. This greatly simplifies the trouble of traditional customer business systems needing to call multiple network domains (shielding the differences in various complex network resources and interface specifications in the south), and lays the foundation for providing cloud-network collaboration capabilities for customer business middle platforms. Due to the complex business needs, environment and processes of customers, we have done a lot of customized development based on the Terra business orchestrator. At the same time, we have managed the SDN data center controller, SD-Core backbone network controller and SD-WAN edge access controller in the Terra business orchestration collaboration layer.

(2) Cloud Data Center Network

We focus on the following points: In the cloud center, the VPC network of the cloud resource pool and the BM network of the physical resource pool are deployed with a hybrid overlay network, including Openstack Neutron linkage. The second is that due to the customer's bidding requirements, network equipment from multiple vendors was purchased in different POD areas. How to manage the VXLAN/EVPN networks of multiple vendors and provide a unified logical network service capability for the upper layer; the third key design consideration is the collaborative management of VPC and external networks, especially the backbone network, to achieve interoperability and unified management of VPC and border routers (GW/VBR). Traditional network vendors are difficult or unwilling to solve the heterogeneous environment of customers. This is hard work and technical work. We implement the above functions based on the TerraDC controller of Dadi Cloud Network and add L4-L7 NFV service management.

(3) Backbone network

We focus on the automatic activation of L2VPN and L3VPN services and the SLA and traffic engineering design of important services. The backbone network of this service provider uses multi-protocol label switching (MPLS)/segment routing tunnel (SRTE) as the main backbone network router network, and connects with the cloud center and tenant branches through a virtual private network (L2VPN/L3VPN). At the same time, customers use SRTE to achieve SLA service quality assurance for different gold, silver and copper services. Customers use SR+SDN to reconstruct a new generation of backbone networks to achieve traffic scheduling and management. Since PE nodes need to extend to data centers and public clouds, the interconnection between PE and border routers (GW/VBR) and the automatic connection with SD-WAN access are also the focus of this issue. We implement the above functions based on TerraCore controller of Dadi Cloud Network.

(4) Edge access network

During the design, we focused on the SD-WAN PoP point networking design and detection, multi-line POP node design and networking, and how the POP point is integrated with the backbone network MPLS. The access side CPE equipment uses automatic detection technology to select the best POP node. Another point is that the integration with MPLS networking should consider how to prevent routing loops and backflows, as well as the perfect unification of SD-WAN tenants and MPLS tenants, to solve the MPLS last mile problem. With the epidemic this year, customers are integrating mobile office into the SD-WAN system. In this part, we implement the above functions based on TerraEdge.

Conclusion

New cloud service providers pursue logical resource management and second-level response. Traditional operator networks or traditional network models can no longer support the elasticity and rapid growth of business. The emergence of cloud-network collaboration and SDN technology abstracts and simplifies the entire physical network into a "single" logical network resource pool, and through software-defined user business automation processes, realizes the linkage of multiple systems and multiple domain networks, and completes end-to-end rapid automatic deployment of business. Over the years, SDN network technology has been favored and recognized by new cloud service providers and operators as an open software architecture, but there is still a long way to go in terms of universality, standard specifications and interoperability. However, no matter what, SDN and cloud computing are the two most basic innovation engines for the future development of IT and are unstoppable. I believe that in the near future, the cloud-network collaboration combined with SDN and cloud computing will bring a new world to operators, cloud vendors and customers.