5G is here. How to build the emerging edge DCs?

[[285118]]

What does the data center and computer room architecture look like in the 5G era?

The user plane function UPF of the 5G core network is sunk to form an edge data center together with ICT infrastructure such as edge computing and CDN, and is distributedly deployed in the access, aggregation and core computer rooms of the 5G network, thereby achieving integration with cloud computing through an end-to-end network pipeline, forming a "cloud-edge-end" integrated architecture.

Distributed edge data centers bring cloud computing power, content, and applications that used to be thousands of miles away down to tens of kilometers or even several kilometers from users, making them closer to the user side, shortening the delay in data transmission back and forth, and reducing the network backhaul load. At the same time, edge data centers store and process data locally, and also ensure data security and privacy in various industries such as parks, hospitals, and factories. This can not only greatly improve the business experience, such as making the VR experience more immersive, but will also accelerate the maturity and implementation of massive 5G industry applications such as networked autonomous driving, smart factories, and smart grids.

As a key component of 5G, edge computing has unlimited value potential and has been a hot topic in the past two years. The industry has conducted a lot of research on 5G wireless, application scenarios, and edge computing, but little is known about its "base" edge data center infrastructure.

How many edge DCs will be needed in the future? Where will tens of thousands of edge DCs be built? How much room space will edge DCs need? How to obtain site locations? How to solve basic supporting facilities such as power supply and distribution, refrigeration, etc.? How to design? How to plan? How to build? How to operate and maintain in the later stage?…

To deploy edge computing, infrastructure must come first. Looking to the future, the industry should plan ahead and lay a solid foundation for edge computing as early as possible. Recently, Huawei released the industry's first "White Paper on Edge Data Center Infrastructure for 5G". Now let's learn about edge DC infrastructure.

Edge DCs will emerge in large numbers. How to obtain sites?

How many edge DCs will be needed in the future? Where will they be built? These are the questions that must be considered first when deploying edge DCs. The white paper points out that the extent of the sinking of edge DCs depends on the experience requirements of 5G services, and the size of each edge DC depends on the computing power requirements of 5G services.

The further the edge DC sinks, the closer it is to the user side, the lower the network latency, the more 5G service types it can support, and the more edge DC nodes are required. Taking South Korea's LG U+ as an example, in the early days, only two edge DCs were deployed in Seoul to meet the current VR/AR service needs. However, with the continuous development of 5G services, the scale of edge DC deployment will continue to increase in the later period. For example, to meet the ultimate VR experience with a latency of less than 20ms in the future, more edge DCs need to be built; and for future Internet of Vehicles services, in order to help cars interact with real-time road conditions and formulate execution strategies, edge nodes may be deployed along the way in smart lamp poles or bus stops in units of kilometers.

From the perspective of business types, in addition to general 5G services such as video, games, and Internet of Vehicles, parks such as medical and industrial manufacturing are another major application scenario for edge computing. As the digital transformation of various industries advances, 5G services continue to expand into vertical fields, and there will be more and more edge DCs in parks.

It is foreseeable that a large number of edge DCs will emerge in the future. Taking Zhejiang Province as an example, each operator plans to have more than 1,000 edge data center sites. Where do these edge DC sites come from? The white paper points out that before the early edge computing business broke out on a large scale, the construction of edge DCs mainly focused on reusing existing access and aggregation communication rooms to achieve low-cost and rapid deployment. However, with the continuous development of 5G business, a large number of edge DCs will need to be newly built in the later stage. Therefore, the construction of edge DCs is mainly divided into reuse scenarios and new scenarios.

But the question is, how should these edge DCs be built next?

How to build it? Let’s talk about the challenges first

For the reuse and renovation scenarios, the existing communication room will integrate edge data center functions, wireless access CRAN functions, as well as aggregation, transmission, and telecommunications billing functions. That is, in the same communication room, access equipment, transmission and switching equipment, core network user-side network elements, and edge cloud computing power will be deployed. Various ICT equipment coexist, and the cabinet size standards, power supply and distribution requirements, and refrigeration forms of different equipment are not the same. This puts forward new requirements for the space, power supply and distribution, refrigeration and other infrastructure capabilities of the room.

In terms of computer room space requirements, if different devices cannot be deployed in an integrated manner or are deployed in a high-density manner, IT equipment, BBU, access and transmission equipment and other equipment will each have a cabinet, and UPS, -48V DC power supply, HVDC, batteries and other power supply and backup equipment will also each have a cabinet. This will take up more computer room area and lead to space constraints in the computer room.

In terms of power supply and distribution requirements, most existing communication rooms only have one mains supply, which cannot accommodate oil generators and has a short backup time, which cannot meet the data center's two-mains supply and higher backup time requirements. At the same time, multiple devices and multiple services coexist in edge DCs, and different devices have different power supply requirements, and different services have different backup time requirements, which requires a multi-convergent and integrated power supply and distribution architecture.

In terms of cooling requirements, existing communication rooms generally use room-level air conditioners, which have long air supply distances, low cooling efficiency, and cannot provide precise cooling. They cannot meet the cooling requirements of high-power density cabinets such as IT cabinets and BBU cabinets, and cannot support the partitioned deployment of cabinets with different power densities. At the same time, different ICT equipment has a variety of air inlet and outlet methods, including front-to-back outlet, left-in-right outlet, right-in-left outlet, and central air inlet and outlet in both the upper and lower directions. This will lead to disordered airflow organization and low cooling efficiency.

If the old communication room cannot be reused, a new one will need to be built at a new site, which will face challenges such as difficulty in introducing fiber optic transmission, high site rental and investment costs.

In response to these situations, the white paper points out that edge DC deployment faces challenges throughout the entire life cycle, from planning, design, construction to operation and maintenance.

During the planning stage, we faced two challenges: large workload of site survey and difficulty in accurate planning. In the scenario of reusing old equipment, the infrastructure of most existing communication rooms cannot meet the requirements of edge DC deployment, and the sites are scattered and the environment of the equipment rooms varies greatly. This requires professionals to visit the site many times to balance multiple factors such as space, load bearing, power supply and backup, cooling, safety, and environmental protection, which is a lot of work.

Considering the unpredictable demand for edge computing, it is also difficult to plan and invest accurately on demand. If the traditional method is used to plan for the final demand at one time, it will lead to high vacancy rates in the early stage and long payback period. If it is planned and built according to the new business demand, there may be problems such as the original initial architecture cannot meet the requirements of the new business launch, transformation is difficult, and the reset cost is high.

During the design phase, on the one hand, there are a large number of site locations with varying types, and each computer room design needs to be customized according to the available space, especially for renovated computer rooms and a large number of rented computer rooms, which cannot be designed in a standardized manner; on the other hand, there are also problems such as difficulty in multi-professional collaboration and more design changes.

During the construction phase, on the one hand, edge DC deployment faces the challenge of difficult room renovation due to the different sizes of cabinets for various ICT equipment and electromechanical equipment, the different power supply requirements in various countries, the inability of cooling methods to meet the needs of high-power density cabinets, complex airflow organization, and complex on-site environment. On the other hand, new edge DC sites also face challenges such as long acquisition cycles and high costs.

During the operation and maintenance phase, the increase in the number of cabinets, power density, and low cooling efficiency will lead to an increase in electricity consumption, which will increase OPEX expenditures. If the massive distributed edge DCs cannot be remotely managed, visualized, and controlled, and digital and intelligent operation and maintenance cannot be achieved, the operation and maintenance costs will increase.

To solve the problem, full stack simplicity and full stack efficiency are the direction

In response to the above challenges, the white paper points out that full-stack simplicity and full-stack efficiency are the inevitable trends in edge DC construction.

Full stack minimalism

Full-stack simplicity: through "full-stack modularization", a series of modular solutions from site to system solution, to architecture, to components are realized to meet the business's demand for large or small scale edge DC; through "full-stack integration", the integration of electromechanical equipment, wireless, transmission, computing power and other equipment is realized, and ultimately the minimalist deployment of edge DC infrastructure is achieved.

Full-stack modularization should first be planned from a global perspective. Typical configurations should be used to plan sites for rapid batch site evaluation and design. Prefabricated solutions such as pre-installation and pre-commissioning should be used to achieve simple and rapid on-site installation, thereby eliminating the need for site surveys, design, and engineering, and meeting the needs of rapid deployment of massive edge DCs and rapid service launch. Modularization should also be used to achieve rapid, agile, and flexible deployment from system solutions and architectures to components. For example, in the power supply and distribution architecture, each power distribution module can flexibly support rapid service launch through hot-swap.

Full-stack integration means integrating IT equipment, BBU, access, transmission and other ICT equipment into one cabinet or one module, and integrating -48V DC, AC mains power supply, HVDC, UPS, refrigeration, batteries and other electromechanical equipment into one cabinet or one module, thus solving the challenge of occupying a large area of ​​the computer room.

To meet the cooling needs of the computer room, the same integration concept is adopted, using row-level air conditioners and rack-mounted air conditioners with smaller granularity, and deploying them in the same cabinet as ICT equipment as much as possible to achieve close-range and precise cooling and save floor space.

Full stack efficiency

In response to challenges such as large workload in engineering surveys and difficult operation and maintenance management, the white paper points out that it is necessary to use digital technologies such as AI and big data to promote the development of edge DC towards full life cycle efficiency and full-stack collaboration.

Efficiency throughout the entire life cycle means creating digital twin models of infrastructure through technologies such as BIM, 3D modeling, and AI, building digital sites that are identical to physical sites in the digital world, and realizing digital engineering surveys, design, delivery, and acceptance, thereby making edge DC deployment more efficient and standardized. At the same time, considering that a large number of edge DCs are unattended, in order to improve operation and maintenance efficiency, digital and intelligent operation and maintenance should also be achieved by relying on the Internet of Things, AI, and big data technologies.

Full-stack collaboration refers to the linkage between edge DC infrastructure, equipment hardware, and business to achieve system-level intelligent energy management such as automatic shutdown or startup of power according to load and precise cooling, so as to reduce PUE and save electricity bills.

Edge computing is a key component of 5G. To deploy edge computing, infrastructure must come first. As the industry's first white paper on edge data center infrastructure in the 5G era, the "White Paper on Edge Data Center Infrastructure for 5G" brings together the insights of 29 industry experts, and for the first time deeply expounds on the trends, challenges and deployment ideas of edge DC in the 5G era, pointing the way for the development of edge data centers. This is a white paper that is well worth reading.