How to comprehensively and objectively evaluate the quality of 5G networks? This is the correct approach

As 5G commercialization continues to advance, the quality of 5G networks has become a hot topic of concern. We have seen a lot of news about "5G peak rate breaking records", various network speed evaluation reports, and even personally tested it with a mobile phone. Can these test results reflect the quality of 5G networks?

In the early days of 5G deployment, when the network was not yet mature in scale, these test results could be used to reflect network capabilities. But now, as the scale of 5G networks continues to expand, the number of 5G users continues to increase, and network performance continues to be optimized, these test results cannot represent the real experience of users and are no longer suitable for the new stage of network evaluation. After all, a large-scale 5G network consists of several adjacent base stations/cells. The wireless environment, signal coverage, capacity, software and hardware parameters, number of users, and user behavior of each cell are different, and users in the entire network are constantly moving. In this super complex and changeable system, it is definitely unfair to measure the quality of the entire network by relying only on single-point testing or single indicators.

How can we evaluate the quality of a 5G commercial network as comprehensively and objectively as possible when entering a new stage of 5G commercialization? Not long ago, I saw a piece of news: Beijing Unicom organized 5G tests on 6 lines from the perspective of 5G terminal users. 5G users conducted a comprehensive evaluation of the 5G network based on 12 indicators in four aspects: occupancy, stability, excellent experience, and good signal. I think this test is the most comprehensive and objective evaluation method so far. Let's explain why.

Occupy

Whether the 5G logo is displayed on the phone and whether it can occupy the 5G network directly affects the user experience and network brand influence. In the early stage of 5G construction, you should have encountered the phenomenon of displaying the 5G logo for a while and the 4G logo for a while, and you may have complained about the poor 5G network.

Why does this happen? One main reason is that the 5G network did not form continuous coverage in the early stage. When users move to the edge of the 5G cell, they will reselect or switch to the 4G network in order to maintain service continuity. In addition, even after the 5G network has achieved continuous coverage, because the mobile phone is in the overlapping coverage of multiple adjacent 4G and 5G cells, this situation will also occur if the network planning is unreasonable or the interoperability settings between 4G and 5G are unreasonable. For example, when the 4G and 5G signal strengths at the user's location are almost the same, the 5G network cannot be given priority.

Therefore, a 5G network that can "occupy" not only needs to achieve continuous coverage of 5G signals, but also needs to be optimized to an excellent level in terms of network planning, software and hardware parameters, hardware performance, etc., to ensure that users can always display the 5G logo at any location on the network and can always occupy the 5G network.

So how to comprehensively evaluate the 5G network's ability to "occupy" this capability? It is necessary to combine two professional test indicators, "duration residence ratio" and "mileage residence ratio", to conduct a comprehensive evaluation from the dimensions of time and space. The duration residence ratio refers to the ratio of the time a 5G user stays on the 5G network to the total time, and the mileage residence ratio refers to the ratio of the mileage that a 5G user stays on the 5G network during the mobile test to the total mileage.

Why do we need to combine these two indicators for comprehensive evaluation? For example, the 5G signal at my home has always been very good, and I can always occupy the 5G network. If I test it at a fixed point at home, the 5G duration residence ratio must be 100%. However, this cannot reflect the actual situation of the entire network. Only after driving the entire network to test, the network's duration residence ratio and mileage residence ratio have reached 100%, which means that users can always occupy 5G at any location on the network, and can truly occupy 100% of 5G.

Good signal

In addition to the 5G logo, whether the signal is full is also often used to measure the quality of a network. So the question is, does "full signal" mean "good signal"? From 2G to 4G, have you ever encountered such a situation: the phone shows full signal, but you can't access the Internet, can't make calls, or the Internet speed is very slow?

That’s right, “strong signal” does not mean “good signal”. It only means that the network coverage is good, which is the basis of “good signal”, but not the only condition. Even when the mobile phone is in the state of “full signal”, if there is interference in the network, hardware failure, unreasonable parameter configuration and other reasons, there will be problems with poor wireless channel quality, which will affect the user experience. For example, according to the working principle of 5G network, the base station will select the appropriate modulation and coding method according to the quality of the wireless channel. If the channel quality is poor, the system will select a lower-order modulation and coding method, and the lower the modulation and coding, the slower the network speed. This will result in the network speed being slow even when the mobile phone signal is full due to the poor quality of the wireless channel. On the contrary, even if the signal is not full, but the wireless channel quality is very good, the network speed is still very fast.

Therefore, "good signal" needs to be reflected by two aspects: "signal strength" and "signal quality". Professionally speaking, it is mainly measured by two indicators: RSRP (reference signal received power) and SINR (signal to interference plus noise ratio). The larger the RSRP value, the stronger the signal coverage and the more signal grids; the larger the SINR value, the better the signal quality. If the RSRP and SINR values ​​are very good during the network speed test, or the proportion of 5G weak coverage and 5G poor quality are very low, it means that the network signal is really good.

Keep steady

You have used mobile phones for so many years, and you probably have encountered unexpected interruptions during mobile phone communications, right? Although the probability of this happening is very low, once it happens, it will seriously affect the user's perceived experience and easily leave a bad reputation for the operator. Therefore, the 5G network must be "available", "have a good signal", and "remain stable", which reflects the network's ability to maintain business communications, as well as the stability and reliability of the network.

To achieve "maintain stability", first of all, the mobile phone can successfully access the network when surfing the Internet or making calls. The corresponding measurement indicator is the "5G access success rate", which is the ratio of the number of successful service establishment attempts to the number of service establishment attempts. Secondly, after accessing the network, the mobile phone can still maintain stable communication without unexpected interruptions. The corresponding measurement indicator is the "5G drop rate", which is the ratio of the number of drop times to the number of successful service establishment attempts. In addition, since 5G has made a leap forward in network latency compared to 2/3/4G, ultra-low latency is also a key capability to enable 5G innovative applications. To measure the performance of 5G SA networks, we must also focus on the indicator of network latency. Especially for some latency-sensitive 5G applications, such as cloud games, VR/AR, etc., if the network latency is too large or unstable, the user experience will be greatly reduced. Whether it is the access success rate or the drop rate, it is related to factors such as network interference, neighboring area planning, parameter optimization, capacity allocation, hardware performance, etc. Therefore, from the perspective of the network, "maintaining stability" also reflects the overall operation quality of a 5G network.

Excellent experience

Whether the network speed is fast or not is the most intuitive experience of the majority of users. But how can we objectively evaluate the speed of a 5G network? Look at the peak rate? The average rate? Or the minimum rate?

The network peak rate is the highest rate tested under ideal conditions, and is often used to measure the product performance and functions of wireless devices. Due to the influence of factors such as the wireless environment and the number of users, it is difficult for users to reach the peak rate in actual commercial networks. For example, since wireless resources are limited in the same cell, the more users access the Internet at the same time, the fewer resources each user gets on average, and the slower the average rate of users. Therefore, it is not rigorous to use the peak rate to evaluate the quality of a commercial network, and the average rate must be used to measure it.

But the average rate alone is still not comprehensive enough, because it ignores a key indicator that affects the user experience - the minimum rate. Services such as video and games have minimum rate requirements. Once they are lower than a certain value, the user experience will be seriously affected. As mobile video services continue to develop towards high definition and immersion, the minimum rate requirements for the network are higher. "Only the media pays attention to the peak rate, and we are more concerned about how to improve the minimum rate." I remember that as early as the 4G era, a senior executive in charge of network planning and optimization of an operator made such a statement. Therefore, to judge whether a network experience is good or not, it is necessary to combine the average rate and the minimum rate. If a network has a high average rate and the minimum rate can also meet a certain threshold, then the experience of this network is excellent.

In short, the 5G network is a very complex system. To comprehensively and objectively judge the quality of the network, a comprehensive evaluation from multiple dimensions is required. Beijing Unicom's 5G test was the first to be conducted for the public and from the user's perspective. It comprehensively evaluated the network from the four aspects of good signal, occupancy, stability, and excellent experience, using 12 indicators such as 5G duration residence ratio, 5G mileage residence ratio, 5G access success rate, 5G drop rate, latency, uplink and downlink average rate, uplink and downlink minimum rate, 5G weak coverage ratio, and 5G poor quality ratio. This set a precedent in the industry, not only allowing users to fully understand the true quality of the 5G network for the first time, but also played a positive demonstration role in the industry, which is conducive to promoting the high-quality and healthy development of the 5G network.