5G becomes a strong driving force for edge computing

Edge computing is one of the most exciting new concepts in the digital world. Using a network of micro data centers that take up very little space, edge computing enables systems to collect and analyze important data in real time without burdening existing infrastructure.

In IoT systems, a large amount of data is usually obtained in a specific highly sensor-intensive environment in an end-to-end manner, and the data is generated and processed at the edge to reduce latency and reduce the load on data centers. Previously, edge computing focused on the technology of devices connected to the IoT, such as industrial robots.

However, as technology continues to develop, the combination of big data, IoT and AI has brought unlimited potential, and the demand for edge computing has evolved from solving the bandwidth cost problem of long-distance data transmission caused by the growth of data generated by IoT to processing real-time applications. In this type of combination, edge computing needs to meet low latency and accelerate real-time creation and support applications.

What is edge computing?

There are many terms for edge computing, including “edge cloud computing” and “fog computing.” “Edge computing” itself is usually described as applications running on local servers, designed to bring cloud processes closer to end devices.

"Enterprise computing" is similar to edge computing, but tends to accurately describe network functions rather than the location of computing. The concept of "fog computing" was coined by Cisco, and many people define it as computing that is located above or below the edge computing space, or even as a subset of edge computing.

For reference, endpoint devices and endpoints are often referred to as "edge devices" to avoid confusion with edge computing. Edge computing can take many forms, including small aggregators, local servers, or micro data centers. Micro data centers can be distributed regionally in permanent or transportable storage containers.

The value of edge computing

Typically, sensors, cameras, microphones, and a host of different IoT and mobile devices collect data from their location and send it to a centralized data center or cloud.

Data shows that by 2020, there will be more than 50 billion smart devices connected worldwide. These devices will generate zettabytes (ZB) of data each year, which will grow to more than 150 ZB by 2025. Sending this data to the cloud will bring some major problems.

First, 150ZB of data will cause capacity problems. Second, it is expensive to transfer large amounts of data from their original locations to centralized data centers. It is estimated that only 12% of data is currently analyzed and processed, and only 3% of data helps to produce meaningful results. The remaining 97% of data is wasted after being collected and transmitted. Third, storing, transmitting and analyzing data consumes a lot of energy. Therefore, we need to find an effective way to reduce costs and waste.

Introducing edge computing and storing data locally can reduce transmission costs. At the same time, using AI capabilities can also eliminate data waste. For example, new low-power edge computing server CPUs are now in use, which are connected to AI acceleration SoCs in the form of GPUs and ASICs or a series of chips.

In addition to addressing capacity, energy, and cost issues, edge computing can also improve network reliability because applications can continue to operate during widespread network outages and improve security by eliminating certain threat profiles, such as global data center denial of service (DoS) attacks.

Most importantly, edge computing can reduce latency for real-time scenarios (such as virtual reality malls, video caching on mobile devices), while creating many new application opportunities in environments such as self-driving cars, gaming platforms or fast-paced manufacturing.

5G becomes the strongest driving force for edge computing

5G infrastructure is one of the most compelling drivers for edge computing. 5G telecom providers are finding that they can build an ecosystem to host unique local applications in addition to traditional data and voice connections. By placing servers next to base stations, cellular traffic providers can open their networks to third-party host applications, thereby improving bandwidth and latency.

Credence Research believes that by 2026, the entire edge computing market will be worth around $9.6 billion. In contrast, Research and Markets analysis believes that the mobile edge computing market will grow from a few hundred million dollars today to more than $2.77 billion in 2026. Although the telecommunications industry may be the fastest growing growth driver, it is estimated that they will only account for one-third of the total edge computing market. This is because web scale, industrial and enterprise groups will also provide edge computing hardware, software and services to their traditional markets, and expect edge computing to open up new application opportunities as well.

For example, the kitchens of popular fast food restaurants are moving towards more automation to ensure food quality, reduce employee training, improve operational efficiency, and ensure that the customer experience meets expectations. Chick-fil-A is a fast food chain that announced in 2018: "By making kitchen equipment smarter, we are able to collect more data. With this data, we can build more intelligent systems and expand our business." They also pointed out that with the help of edge computing, many restaurants can now handle up to three times the business volume before.

Overall, a successful edge computing infrastructure requires a combination of local server computing capabilities, AI computing capabilities, and connectivity to mobile/automotive/IoT computing systems.

Understanding edge computing with examples

To understand the latency benefits of using edge computing, Rutgers University and Inria used Microsoft HoloLens to analyze the scalability and performance of edge computing, or “edge cloud.”

In the case, HoloLens reads a barcode scanner and then uses scene segmentation in a building to navigate the user to a designated room and display an arrow on the Hololens. The process uses both small packets of mapping coordinates and larger packets of continuous video to verify the improvement in latency of edge computing compared to traditional cloud computing. HoloLens reads the QR code first and then sends the mapping coordinate data to the edge server, which uses 4 bytes plus the header and takes 1.2 milliseconds (ms). The server finds the coordinates and notifies the user of the location, which takes a total of 16.22 ms. If the same packet is sent to the cloud, it takes about 80ms.

Similarly, they also tested the latency when using OpenCV for scene segmentation to navigate the Hololens user to the appropriate location. HoloLens streams video at 30 fps and processes the images in the edge computing server on an Intel i7 CPU with 15GB RAM at 3.33 GHz. It took 4.9 ms to stream the data to the edge computing server, and an additional 37 ms to process the OpenCV image, for a total of 47.7 ms. The same process on the cloud server took nearly 115 ms, clearly showing the obvious advantage of edge computing in reducing latency.

This case study shows the significant advantages of edge computing in reducing latency, but there will be more new technologies in the future that can better achieve low latency.

5G outlines cases with less than 1ms latency today, and 6G is already discussing reducing it to 10 microseconds (µs). 5G and Wi-Fi 6 will increase connection bandwidth, with 5G expected to increase bandwidth to 10Gbps and Wi-Fi 6 already supporting 2Gbps bandwidth. AI accelerators claim scene segmentation in less than 20µs, which is a significant improvement over the Intel i7 CPU cited in the example technical paper above, which processes each frame in about 20ms.

Obviously, if edge computing is so advantageous over cloud computing, wouldn’t it be best to move all the computing to the edge device? Unfortunately, this is not the case for all applications at the moment. In the HoloLens case study, the SQL database used for the data was too large to store in the headset. Today’s edge devices, especially those that are physically worn, do not have enough computing power to handle large data sets. In addition to computing power, software on the cloud or edge server is cheaper to develop than software on edge devices because cloud/edge software does not need to be compressed into smaller memory resources and computing resources.

Since some applications can run reasonably based on the computing power, storage capacity, memory availability, and latency capabilities of different locations in the infrastructure, the future trend is hybrid computing capabilities, whether in the cloud, in edge servers, or in edge devices. Edge computing is the first step in building a global hybrid computing infrastructure.

Edge computing and AI

Many new services using edge computing have low latency requirements, so many new systems are using the latest industry interface standards, including PCIe 5.0, LPDDR5, DDR5, HBM2e, USB 3.2, CXL, NVMe over PCIe, and other next-generation standard-based technologies that reduce latency by improving bandwidth compared to previous generations.

These edge computing systems are also adding AI acceleration capabilities. For example, some server chips provide AI acceleration through new instructions such as the x86 extension AVX-512 Vector Neural Network Instructions (AVX512 VNNI).

In addition, custom AI accelerators are added to most new systems. The connectivity required for these chips often uses the highest bandwidth host for accelerator connectivity. For example, in a certain switched configuration with multiple AI accelerators, many systems use the PCIe 5.0 interface because the bandwidth requirements affect latency.

Outside of local gateways and aggregation server systems, a single AI accelerator typically does not provide enough performance, so very high-bandwidth chip-to-chip SerDes PHYs are needed to scale these accelerators. The recently released PHYs support 56G and 112G connections.

AI algorithms are pushing the limits of memory bandwidth requirements. For example, the latest BERT and GPT-2 models require 345M and 1.5B parameters, respectively. To meet these requirements, not only high-capacity memory capabilities are required, but also many complex applications need to be executed in the edge cloud. To achieve this capability, designers are adopting DDR5 in new chipsets. In addition to capacity challenges, the coefficients of AI algorithms need to be accessed for large numbers of multiple cumulative calculations performed in parallel in non-linear sequences. As a result, HBM2e has also become a new technology that has been rapidly adopted, with some chips implementing several HBM2e instantiations in a single chip.

In the future, the demand for edge computing will focus on reducing latency and power, ensuring that there is enough processing power to handle specific tasks. The new generation of server SoC solutions will not only have lower latency and lower power consumption, but will also incorporate AI functions, namely AI accelerators.

But it’s clear that the demands of AI and edge computing are also changing rapidly, and many of the solutions we see today have advanced multiple times over the past two years and will continue to improve upon them.

Conclusion

Futuriom once wrote in "5G, IoT and Edge Computing Trends" that 5G will become a catalyst for edge computing technology, and applications using 5G technology will change traffic demand patterns, providing the greatest impetus for edge computing in mobile cellular networks.

In general, edge computing is an important technology for achieving fast data connection. It brings cloud services closer to edge devices, reduces latency, and provides consumers with new applications and services. It will also derive more AI functions and extend them beyond the cloud. In addition, it will become the basic technology to support future hybrid computing.