Let ChatGPT tell you how to build a lossless network that supports ChatGPT computing power

With the acceleration of digital transformation of global enterprises, applications represented by Chat GPT are becoming increasingly in-depth in the fields of production and life. Behind the surge in popularity of ChatGPT, the demand for infrastructure required for automatic content generation technologies such as artificial intelligence is also rising.

In the next five years, my country's intelligent computing power will grow at a compound annual growth rate of 52.3%.

According to the "2022-2023 China Artificial Intelligence Computing Power Development Assessment Report", China's intelligent computing power will reach 155.2 EFLOPS (FP16) in 2021, and it is expected that China's intelligent computing power will reach 1271.4 EFLOPS by 2026. During the period of 2021-2026, the annual compound growth rate of China's intelligent computing power is expected to reach 52.3%. With the introduction of national policy plans such as the "East Data West Computing" project and new infrastructure, my country's intelligent computing center has set off a construction boom. At present, more than 30 cities in my country are building or proposing to build intelligent computing centers. The overall layout is mainly in the eastern region and gradually expanding to the central and western regions. From the perspective of development foundation, around the development ideas of AI industrialization and industrial AI, the artificial intelligence industry has initially formed an architecture system with heterogeneous chips, computing facilities, algorithm models, and industrial applications as the core, and the intelligent computing center has the foundation for construction.

Building a large-scale intelligent computing power base

Currently, ChatGPT's training model is mainly based on the general basic large model base GPT-3. Training a super-large basic model requires the support of many key technologies, and algorithms, computing power and data are indispensable. The algorithm relies on the improvement of large model parameters and the optimization of the model itself, while computing power and data need to rely on traditional GPU servers, storage and networks to achieve mutual promotion. Data shows that the total computing power consumption of ChatGPT is about 3640PF-days (that is, if one trillion calculations are performed per second, it will take 3640 days to calculate), and 7 to 8 data centers with an investment scale of 3 billion and a computing power of 500P are required to support operation. According to the 13 million visits per day, ChatGPT is estimated to require 30,000+ GPUs. GPUs will communicate frequently during the training process, including P2P communication and Collective communication. Within the node, the communication interconnection bandwidth between GPUs can reach 400GB/s. Between nodes, GPU communication uses the RDMA network. With the support of GDR (GDR, GPU Direct RDMA) technology, the RDMA network card can bypass the CPU and memory and directly read data from the remote node to the GPU memory.

At the network level of the computing center, it is necessary to achieve network and application system integration optimization through technologies such as intelligent lossless storage network, and to improve the overall network throughput and reduce network latency through traffic control technology and congestion control technology. For H3C intelligent lossless network, having ultra-large-scale networking is the only way to build intelligent computing power. At present, AIGC represented by ChatGPT, including the significance of the large model behind it, is not only in the implementation itself, but its scientific research value may be greater. It is generally believed that the first few industries to be implemented may include scientific research, education, and Internet-related industries. Taking the Internet industry with large-scale deployment as an example, an Internet company closely followed the AI ​​training such as ChatGPT to build a cluster computing network with 4,000 200G ports supported by a single PoD . In the intelligent computing center based on scientific research and education, the number of ports deployed in the current PoD is usually between 1,000 and 4,000. Therefore, H3C provides a variety of optional high-performance network solutions to fully meet the scale of different business scenarios of users.

Box-box networking: Taking the current main GPU server 100G /200G/400G network card rate as an example, H3C uses the latest S9825 /S9855 series three-layer ToR /Leaf/Spine networking architecture as an example . Spine uses dual planes and ensures that the ToR uplink and downlink convergence ratio meets the 1:1 requirement. At a server access rate of 400G , a single PoD can support 1024 servers, and a cluster can provide access to 2048 400G servers; if a 200G rate is used, a single PoD can support 2048 servers, and a cluster can support up to 32 PoDs , which can theoretically meet the access of 65,000 servers; if a 100G rate is used, the cluster can meet the access of more than 100,000 servers.

Figure 1: Three-level box architecture 200G access networking

As for the lossless network with deterministic scale, H3C provides a lightweight intelligent lossless network deployment solution of "one frame means lossless", which can also meet the intelligent computing networking needs of most scenarios. Taking the S12516CR fully equipped with 576 400G ports as an example, a single frame can be directly connected to the server network card as a ToR to achieve 1 :1 convergence, and can support up to 576 400G QSFP DD ports per PoD; 200G QSFP56 can meet the maximum access of 1152 ports; and 100G QSFP56 can meet the maximum access of 1536 ports. It should be noted that the direct splitting of 400G DR4 can obtain more than 2000 100G ports in DR1 package , while the current mainstream network cards do not support DR1 . The advantages of using a single-frame lossless network are obvious. Using a networking architecture that abandons the traditional Leaf /Spine architecture can effectively reduce the number of devices, reduce the number of data forwarding hops, and effectively reduce data forwarding latency. At the same time, there is no need to calculate the convergence ratio and device scale under multiple levels, which greatly simplifies the deployment and selection difficulty and effectively improves the networking efficiency. It is a new attempt for a deterministic scale intelligent lossless network.

Figure 2 : “ One - frame, lossless ” 200G access networking

Frame-box networking: For larger-scale networking needs, H3C data center networks provide frame-box lossless architecture. Taking the 100G/200G/400G network card rate of the GPU server as an example, if the H3C flagship data center frame product S12500CR series is used to build a ToR /Leaf/Spine three-layer networking architecture , a single S12516CR is used as the Spine and the ToR uplink and downlink convergence ratio is guaranteed to meet the 1:1 requirement. At a server access rate of 400G , a single PoD can support thousands of servers, and the cluster can theoretically provide nearly 59 400G servers at a scale of access; if a 200G rate is used , a single PoD can support 2,000 servers, and the cluster can provide nearly 1.18 million servers at a scale of access; if a 100G rate is used, the cluster can provide more than 2 million servers at a scale of access.

Figure 3 : Three-layer frame architecture 200G access network

Both large-scale networking and cell switching are required

For data center switches, whether they are traditional frame-type or box-type switches, as the port rate increases from 100G to 400G, they not only face power consumption issues, but also need to solve the hash accuracy and elephant and mouse flow of box-type networking. Therefore, H3C data center switches give priority to using DDC (Distributed Disaggregated Chassis) technology to cope with the growing computing power network solution when building intelligent lossless computing power data center networks. DDC technology distributes and decouples large frame devices, uses box-type switches as forwarding line cards and switching network boards, and flexibly distributes them in multiple cabinets, optimizing the networking scale and power consumption distribution issues. At the same time, DDC box switches still use cell switching.

D DC system role names:

NCP: Network Cloud Packet (Line card in Chassis)

NCF：Network Cloud Fabric (Fabric card in Chassis)

NCM: Network Cloud Management (Main Management card in Chassis)

Figure 4 : D DC architecture

Figure 5 : D DC architecture decoupling, 400 GF ull me sh fully interconnected

Taking S12516CR as an example, a single device can support 2304 100G servers and support 1:1 convergence. The DDC solution decouples the control end elements independently, adopts 400G full interconnection between NCP and NCF , supports cell forwarding, supports non-blocking Leaf and Spine in the data center, and effectively improves the efficiency of data packet forwarding . After testing, DDC has certain advantages in the All-to-All scenario, and the completion time is improved by 20-30%. At the same time, compared with traditional box-type networking, DDC hardware convergence performance has obvious advantages, from the port up down Test comparison shows that the convergence time of using D DC is less than 1% of the box-type networking time.

Network intelligence and traffic visualization are needed

The service model of the intelligent computing center has shifted from providing computing power to providing "algorithms + computing power". The AI ​​lossless algorithm is also needed in the intelligent lossless network. The data traffic characteristics forwarded by each queue in the lossless network will change dynamically over time. When the network administrator statically sets the ECN threshold, it cannot meet the real-time dynamic changes in network traffic characteristics. H3C lossless network switches support the AI ​​ECN function, which uses the AI ​​business components on the local device or analyzer to dynamically optimize the ECN threshold according to certain rules. Among them, the AI ​​business component is the key to realizing ECN dynamic tuning. It is a system process built into the network device or analyzer. It mainly includes three levels of functional framework:

• Data collection and analysis layer: provides a data collection interface for acquiring massive feature data to be analyzed, and preprocesses and analyzes the acquired data.
• Model management layer: manages model files and infers AI ECN thresholds based on AI function models loaded by users.
• Algorithm layer: Call the interface of the data collection and analysis layer to obtain real-time feature data, and calculate the AI ​​ECN threshold according to the fixed-step search algorithm.

Figure 6 : AI ECN functional implementation diagram

In addition, H3C data center network provides AI ECN operation and maintenance visualization. According to the different implementation locations of AI service components in the network, AI ECN functions can be divided into two modes: centralized AI ECN and distributed AI ECN:

• Distributed AI ECN: AI business components are integrated locally on the device, and the computing power requirements of AI business components are met by adding specialized neural network (GPU) chips to the device.
• Centralized AI ECN: AI service components are implemented by the analyzer. Suitable for future SDN network architecture, it facilitates centralized management and visual operation and maintenance of all AI services including AI ECN.

In both scenarios above, the advantages of SeerAnalyzer can be leveraged to present users with a visual representation of AI ECN parameter tuning effects.

Figure 7 : PFC back pressure frame rate comparison before and after AI ECN tuning

Looking back, H3C has reached in-depth cooperation with many leading companies in the field of intelligent lossless networks. In the future, H3C data center networks will continue to focus on ultra-wide, intelligent, integrated, and green evolution, and provide smarter, greener, and more powerful data center network products and solutions.