What is active monitoring without embedding code? Technology exploration behind improving user experience

In the Internet age, whoever controls the users controls the capital. This has almost become a truth.

Improving user experience is becoming the focus of many Internet businesses

According to the 47th Statistical Report on the Development of China's Internet released by the China Internet Network Information Center (CNNIC), as of December 2020, the total number of Internet users in my country reached 989 million, an increase of 85.4 million from March 2020, and the Internet penetration rate reached 70.4%. Among them, the number of online news users reached 731 million, accounting for 80.9% of the total number of Internet users; the number of mobile online news users reached 726 million, accounting for 81.0% of mobile Internet users.

It is not difficult to see from the data that the penetration rate of the Internet and the number of Internet users in my country are constantly increasing. At the same time, with the rapid development of smart phones and mobile Internet, people's work, life, consumption, entertainment and other needs are more closely aligned with web applications and mobile applications. On the one hand, it has promoted the prosperity of online transactions. On the other hand, it has also posed severe challenges to how Internet service providers can quickly and accurately grasp user experience.

As the number of users grows, demands increase, business scale expands, and the degree of openness increases, the architecture of application systems becomes more and more complex. IT systems have undergone a comprehensive transformation from language to architecture, from organization to system, from Web to mobile, from open source to independent research and development, from general experience to personalized services, etc. Any problem in any business chain will affect the final user experience.

In addition, SmartBear research has shown that for every 1 second that Amazon's loading time increases, it will lose $1.6 billion in revenue per year. "1 second" is actually a problem that every company will face. The factors that affect the user service experience of this "1 second" may involve a complex full chain of regions, terminals, network facilities, middleware, databases, etc. Traditional APM, NPM and other IT operation and maintenance service products of different dimensions may give different answers to this question.

As one of the earliest companies involved in APM in China, Borui Data has accumulated many years of experience in IT operation and maintenance management. In the process of APM being gradually used by IT operation and maintenance management, APM has formed two major development trends, one is active simulation monitoring based on non-embedded code, and the other is passive real user monitoring based on embedded code.

What we are going to talk about today is Borei Data's non-embedded active analog monitoring, also called non-embedded perceived user experience.

How does Borei Data's active products monitor?

Active simulation monitoring without embedding code can be understood literally as a monitoring method that does not require embedding corresponding code to modify the target application. It only needs to use real devices and environments to simulate the end-user behavior of using the target application, and capture the performance and experience data during the use process, and then use big data technology, statistical methods, etc. to conduct user experience analysis.

Bonree Data's active products include Bonree Net, Bonree APP, Bonree Stock, Bonree Box, etc. Among them, Bonree Net, as Bonree Data's star product, is mainly aimed at active monitoring scenarios such as web/wap applications, streaming media playback, network transmission, and SMS quality.

Bonree APP is the first active monitoring product for mobile Android/iOS applications launched by Borei Data in China. It is mainly aimed at two monitoring scenarios: interactive use of mobile applications and streaming media playback.

The operating structure of Borei Data's active products can be roughly divided into three levels:

The first layer: By continuously recruiting Internet terminal users and purchasing backbone network room services, we ensure that the active monitoring network covers the terminal user side (Lastmile & Mobile monitoring points) and the Internet backbone node side (IDC monitoring points), forming a systematic multi-level monitoring system throughout the country and at home and abroad. At the same time, in order to ensure that the data returned to users is accurate, true, and effective, Borei Data periodically updates domestic and foreign IP libraries, and also provides a complete set of monitoring point scoring management system, which comprehensively scores the monitoring point hardware, software, bandwidth, error rate of historical monitoring tasks, etc. When the monitoring point score does not meet the standard, the platform will not select the monitoring point for monitoring.

The second layer: Powerful task scheduling mechanism and data collection technology can ensure that a large number of tasks with different monitoring cycles, different monitoring frequencies, and different monitoring types are efficiently and orderly distributed to real PCs and mobile devices that meet the task requirements, thereby completing a series of monitoring requirements such as visiting the target website, running the target App, transmitting the target file, playing the target streaming media, testing the target network, receiving the target SMS, and collecting user experience data in real time.

The third layer: After the collected user experience data is transmitted back to the platform through the network, it is cleaned, filtered, extracted, aggregated and calculated by the big data engine to form a rich chart and indicator system, thereby helping customers analyze monitoring results in different dimensions and evaluate the user experience level.

Borei Data also has its own unique insights into user experience evaluation.

Borei Data believes that in terms of performance experience indicators, using average values ​​to reflect user experience is not accurate enough. The performance fluctuations of individual samples affect the calculation results of the average values ​​and deviate from the actual situation. Unlike its competitors, Borei Data introduces median and projection values ​​as the basis for judging performance experience in addition to the average value. It is also the only APM manufacturer in China that can calculate the median and projection values ​​of daily TB-level monitoring data.

In terms of analysis models, different from the single calculation model of competitors, Bonree Net splits the monitoring result data into abnormal data through a health assessment system. Based on a more advanced problem analysis model, it combines basic dimensions such as time, region, operator, host, domain name, and performance, slow ratio, error type and other data items one-to-one or one-to-many, and analyzes the distribution and regularity of problems from multiple data levels of one dimension.

At the same time, the data is deeply mined. That is, while capturing data above the TCP layer, it also captures more low-level and richer real test data (involving network layer and transport layer data) to assist operation and maintenance personnel in quickly and accurately locating problems; based on the captured packet information, the application scenarios can be extended, such as content hijacking verification and other applications.

Why are they all using Borei Data’s proactive products?

It is based on this monitoring concept that Borei Data has won the recognition and support of customers.

For example, in cooperation with a financial institution, the average daily usage of the website and App of the financial institution exceeds 10 million, and users cover all provinces and cities across the country. The annual expenditure on the operation and maintenance of the website and App is tens of millions of yuan. The industry is highly competitive, and users often turn to other manufacturers' services due to user experience issues such as "waiting too long to open the website or App" and "the usage process is too complicated".

Bonree Data used Bonree Net and Bonree App to record interaction scripts for the financial institution's own applications and competitor applications, selected monitoring points of operators in different cities across the country, set periodic monitoring tasks, and conducted comparative analysis of its own applications and competitors from multiple dimensions such as region, operator, and time trend. It kept abreast of the performance and experience differences of competitors, helped financial customers improve user experience, further deepened user stickiness, and successfully retained users for the financial institution.

Another example is the cooperation with a large Internet company, whose App has powerful functions, more than 50 million daily active users, and users all over the country. The annual expenditure on App operation and maintenance exceeds 90 million yuan. However, the App function is frequently updated, and users from all over the country often complain that "the page cannot be loaded" and "the user experience is poor".

Bonree Data uses Bonree App to record scripts and select monitoring points of operators in different cities across the country before the official release of each new version of the App of the Internet giant. It sets periodic monitoring tasks and obtains real network performance, video performance and user experience data in advance according to the interaction process of real users when using the App before the App is officially fully upgraded, providing strong data support for the Internet giant to formulate App operation and maintenance strategies.

In the future, there is more I want to do

As enterprises accelerate their digital transformation, Generation Z's demand for and experience with Internet products are increasing, and improving user experience is clearly becoming the business focus of various Internet companies.

Therefore, in order to adapt to the ever-changing Internet, Borei Data's active products must also be continuously updated and iterated to adapt to changes in the ability to actively monitor mobile applications and provide enterprises with more efficient and high-quality monitoring services.